Rust Programming Language Rust Linz, 23 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rust Linz, March 2021 - Florian Gilcher - Ownership and Borrowing from a Systems Construction PoV

Description

Rust's Ownership and Borrowing System has repercussions beyond the mere topic of memory safety. This talk approaches the topic from a different angle: interaction between components. I found this view a useful one to take when teaching Rust to programmers that enter Rust with a background of memory-safe programming languages such as Ruby, Java, or Python.

Florian on Twitter: https://twitter.com/Argorak
Rust Linz on Twitter: https://twitter.com/rustlinz
Rust Linz on the web: https://rust-linz.at
Submit your talk: https://sessionize.com/rust-linz

A

So this is actually the first time I give a talk about that subject um and I'm not sure how long the talk is going to be, um but first of all uh a little bit about myself. So hey my name is florian. um You can find me on github as skate and on twitter's agruc, which was basically uh the other nickname was not available. I'm a rust team member part of the core team and part of the rust foundation board since a couple of weeks.

A

um I am a rust trainer and business owner, I'm one of the co-founders of ferris systems and it's subsidiary critical section. um The second one is focusing on rust and safety, critical environments and the things that you need to provide for that- and this is still a couple of years out.

A

um But one thing that I want to talk about is ownership and borrowing is always a big problem um for people to get their head around, especially if you're starting out with rust and there's not a lot of concepts out there that you can map to it. So you need to learn it from a new they're, completely new concepts, and you need to start. They start cracking them pretty much immediately, and that is a little bit of a rampant curve to rust.

A

But one thing that I always um find interesting is that um they are often just seen under the aspect of rust as a memory safe language.

A

um But it's not only that um the and the goal of this talk is to to take a little bit of a different perspective on both of these concepts um from the perspective of systems construction, because that is something that, especially for newcomers, is much more relatable and gives a little bit of a of a of a view into um the hardships that they may find with it, because both of the systems are actually not that hard.

A

If you're, holding them right and the holding right problem is, um is yeah what I see. People run into um a lot, and that is, I think, the more inaccessible part, and I would actually just like to talk about three functions in this talk and they're all called the same. They're all called right to file and they write a string to a file, but in three different ways, can you see my mouse pointer yeah? uh Currently? No, yes, yes, we can see your mouse pointer. Yes, it works fine.

A

I should have made it larger, and the difference here is just in the way that we're passing all data. It's not it's not even like the result type here. It's just these two parameters differ in each of the function calls the first one is taking a five pointer in an old fashion and the string that is going to write you in a known fashion.

A

Second, one is going to take the file in a known fashion, but we'll get a loan out on the on the buffer, so it will take a reference, so the borrowing system gets activated and the third one actually completely borrows here and the perspective that I want to uh to discuss these three functions under is not in the sense of okay. These are references in memory, or this is like these are own memory locations, all the things that we usually do, but rather from the perspective on uh component boundaries.

A

um I come to why component binaries is useful, uh expect here, but if we talk about components interacting so any mid to large system, that is something that we need to deal with. We also often talking about the concept called coupling, so how strong are two components interrelated?

A

You can see that here in the first picture, we see two components with a low coupling, in that they don't have a lot of touch points, and the nature of the touch point is also very interesting, so the one does not rely on a lot of internals of the other and the second one being quite the other way around.

A

A lot of internals of the one here are also relying on internals of number two, and vice versa, so they're kind of cross-related and do know a lot about each other, and that is what is usually expressed by uh coupling there's the reverse concept. To that that you might have heard, which is cohesion.

A

um That works precisely the other way around um how um but expresses uh same thing so for this talk I will I will settle on coupling, because that's the one that I like more in that low coupling is good high coupling is bad, or at least harder to work with. There are reasons why you want to have high coupling um or why it might be useful to pay the cost of that particularly performance.

A

um But um so this is the. This is the thing where you can select on uh on how much you, uh where you want to land in this equation and rust may have a v complex surface, but in the end, boils down to a language that has two core components, which is data and functions and everything everything where an action happens or data is getting passes.

A

It's usually a call a call to a function in some way, so reading function, signatures and seeing what we can read just out of the function signatures, you won't see a function body anywhere in this talk um is actually pretty useful and being able to make to have some conclusions of that also um particularly on what they say about the outside world and the introvert inside world, um and that particularly means that, at all points where information crosses between components, you can see that at some point through some function, call to give an example.

A

If you're passing data between two pieces of your system through a channel the left side at some point, we'll call send.

A

And the other one will uh we'll call receive and those two functions, those two function. Signatures are interesting for you.

A

And if we talk about components, we also need to quickly talk about component life cycles.

A

There are components with a life cycle relationship so as an example, a component a calls into a rendering component b to render an image yielding control fully to the renderer. So you have the maybe your application and at some point it wants to render so calls into a rendering component, but does not do anything until the rendering is done and then gets control back.

A

um This is probably an example where, where tie coupling actually makes sense, the application prepares data in a format that the renderer can render. So it knows a lot about the data format that the renderer needs um and maybe even allows to write the renderer to play around in some pre-allocated memory spot.

A

There are components without a lifecycle relationship where, for example, you have a database connection driver which constantly works in the background reading data from the network, um putting it into buffers for you to then take and continue working in it, but it kind of has a life on its own.

A

So it's spinning there and um you may submit, for example, query to that database driver and tell it to notify you when it actually, when it's actually done through, for example, an async runtime or some other mechanism, but the database driver may, for example, require resources in it by itself, for example, threads and because that has a life on its own decoding. It makes sense, because not knowing all the details of what this thing does allows you to change. It use a different one use mysql instead of postgres um and all these kind of things.

A

So um you it there's a conscious approach um to not wanting to know too much about what that thing does internally, that both gives that uh driver the ability to change its own data structures all the time for optimizing it like as long as they are not. They do not become part of the public interface. It's no problem which brings me to. Why did I introduce these three functions? So imagine you have a component that is just there for writing to disk, so it accepts a file pointer. This is where you want to write it.

A

It takes a um and it takes a buffer, and we have this function called right to file and with one component that calls right to file from a another component and if we pass ownership here, that particularly means the calling component is at after that function call immediately freed of all duties to manage both the file and the buffer.

A

The call component will take over this kind of this. Ownership can destroy these things at any time, can turn the string into a bike buffer or whatever, um whatever is needed. So this means the two components. Only have this one touch point and the first component gives over all management duties to the other one. The moment it is called so we have a clear handover, and after that touch point, they don't need to know anything about the internals they just negotiate at this point file and string.

A

The section function that I had there or the second function, signature for the same process differs a little bit in that it gives a string buffer to the component that is responsible to writing, but in a borrowed fashion, so the calling component keeps responsibility of the buffer.

A

That particularly means it needs to keep it around until the call component is done with that, so we still remain retained some responsibility, and that means both of these components need to negotiate, and that is expressed to the borrowing system in rust.

A

The call component takes over ownership of the file to write you, so the uh first component does not care what happens to that, and the call component can break that coupling by copying the buffer that is completely feasible, can say. Okay, I need to keep this around for longer.

A

For example, I want to spin off a thread and immediately return from the right to file function and may end up in a situation where returning a result may not be feasible anymore, but that's a that's another thing. I want to put that, um for example, I want to put that on the background thread and to be able to do that, I copy the string buffer in some way. That is all possible.

A

And the third one is coupling even a little tighter in that all responsibilities remain in the calling component and especially because we're passing in a mutable reference here, the amount of stuff that the this component needs to fulfill becomes a larger and larger immutable pointer in rust or immutable reference and rust. Sorry is unique, so the first component needs to make sure that no one has access to that file until the second component is done with it.

A

That means more and more negotiation needs to be ahead, needs to happen between those and particularly clones files aren't cloned and copy, so the second component is not even able to break uh this.

A

To break this coupling, for example, at the cost of copying, this one still works the second problematic, and that brings you to a couple of rules of thumb.

A

um If you are dealing with a system that is larger than just essentially a module that does some stuff in memory. Ownership is the preferable handover mechanism between components, and you see that in a lot of apis and rust standards that spawn takes ownership, a threat gets spawned. It takes ownership of all the data that it operates on. It's not working through borrowing. It particularly bans borrowing, especially when we're talking about components that might run in parallel with life cycles on their own.

A

They should never borrow between each other, because the borrowing rules are static, something that has a life on its own at runtime is really hard to be described with a static with a static rule set and borrowing across components introduces a lot of pain, especially if we're not talking about passive components- and this is where I see most of the problems arising is not even like. I build a data structure and I do some.

A

I do some small lightweight referencing into it, but rather people referencing between components just naively at the beginning, but running into precisely precisely those problems that the larger the project gets the harder those rules get to fulfill while they should move away from using the borrowing system and rather focus on how how they can use ownership to model things, because ownership is a great method to ensure proper handover between components. I give up ownership, and I give it to another component- is a very, very common way to construct systems.

A

For example, we have an http server. The first part of the http server accepts an http request.

A

Does all the basic checking whether the connection is all right and whether it's actually well-formed or whether the headers are there and then hands it over to a router that starts without selecting what a function should actually be called and what action should be taking for that request?

A

If you look at those apis, they always work through ownership connection is accepted. We're done. We found that it's an http request next component and we pass over ownership um rather than any kind of ordering system. Modeling, that with borrowing would be extremely hard, if not impossible.

A

Quick warning coupling is a good mental concept to figure out whether you should select borrowing or ownership coupling may arise from other behaviors, though.

A

There's a bonus slide here that if we talk about a a structure like this, where we actually use share data through an arc, um we end up in a situation where it is not easy to read from the signature. What is what actually, the interaction between those two components are so also reading function? Signation request only gets you that far at some point, you need to read what the actual components are doing.

A

Especially when it comes to responsibility, if you're talking about borrowing the risk, a lot of the rest responsibility is still with the owner, so the one that lends out the reference.

A

But in these cases, when we're talking about shared smart pointers, then this becomes less clear and might need some more work to establish.

A

As a final slide and pointers for especially beginners, if you want to observe this kind of behavior and practice with some easy to use types, I can highly recommend you um to you to have a look at the channel api with how senders and receivers work and particularly why sender and receiver are different types precisely because of that um and channels facilitate ownership exchange between components.

A

So, if we uh imagine the picture from the beginning with those two low coupled components with the green thing in between the green thing could be your channel where, on the left side, you put things in and on the right side, you get things out arc and rc break data coupling over the lifetime that it remains in memory because arc and rc make sure that data remains in memory as long as there's parts of the system using it at the cost of mutation. So that is a trade-off here.

A

If I want to, uh because immutable data components reading from a section of immutable data in memory does not they don't need to care about each other once they introduce mutation and writing again we're back to not square one. But maybe square three where they do need to find some way to negotiate.

A

um There are, for example, great types for a memory, saving optimization pass. If you want to to try this out, these are rather simple constraint types that will help you help you a lot in the beginning and there's types that are consciously coupled.

A

So, for example, if you look at the mutex interface, which is also rather accessible, there's a uh there's, a type called mutex, which can produce a mutex card which represents that you lock the mutex and that you currently have access to its inner pieces and the mutex guard is consciously consciously coupled with the mutex and cannot get out of range of it. So to say um so. You cannot stop referencing the mutex and but still keep the meeting spot around.

A

It's the same goes factor the iterators of vector all the iterator types either and either move particularly they cannot live without the vector being around, and so this is a a very useful method um for for ensuring uh iterator vitility, because iterators do not make sense if the base, uh if the base structure is not around- and um this is what the borrowing system is very well enforces. So if you're writing these modules and data types where these kinds of relationships forcibly exist, then it's the system to read and up into and buy into.

A

So. The conclusion here is um ownership and borrowing have an interpretation from the perspective of systems, construction and, if you are used to that, it might sometimes help to take that one if you run into problems with boring and you fight with a lifetime checkup all the time.

A

It might be the case- and this is the case I observe a lot- is that what you're trying to do is you're trying to borrow and reference the memory of other components that have an active uh that have an active life on their own. And if you take that stance of how can I make sure that um or.

A

With a component that is arbitrarily working around in this memory, all the time and the second one working in parallel can I find a static rule that makes sure references to the other components. Memory always stay valid, it's pretty hard, and in this case well, it's not you it's the system. um Borrowing in this case is just not the right solution and the references are not the right solution and that isn't even subject to rust. That is the same in a lot of other languages as well as well.

A

um Just that just actually denies you to writing those references, but the fix there is not to understand the borrowing system better, but um to actually um move off of that and try to go uh towards uh owned types and uh yeah and ownership is the most useful tool to decouple systems and the really really useful uh for one to have, and, in my opinion, actually the the more important uh of the two uh two concepts in rust, and I hope I give you a little bit of a different perspective on on why that is and why you should reach for it by default.

A

Essentially, okay, thank you. That was my awesome.

A