Rust Programming Language FOSDEM Rust Devroom 2018, 6 Feb 2018

Previous Meeting

⏯

youtube image

►

From YouTube: Zeeshan Ali - Rust Memory Management

Description

A short intro

A quick introduction to the unique memory management concepts of Rust.

Rust is a systems programming language that focuses on safety and performance at the same time. Most people new to Rust, often struggle with memory management. The goal of this talk is to give a very quick overview of Rust's memory management.

Developer, helicopter pilot and cat lover.
I'm a software developer with many years of C and Vala experience but more recently I've been in love with Rust language. I work for Kinvolk GmbH in Berlin. I've a private pilot license for helicopters and a very loving and lovely cat.

Rust devroom @ FOSDEM 2018
https://fosdem.org/2018/schedule/event/rust_memory_management_intro/

These talks have been recorded at FOSDEM (https://fosdem.org/)
This work is licensed under the Creative Commons Attribution 2.0 Belgium Licence.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/2.0/be/deed.en

A

B

Everyone I wasn't expecting this many people say I care for people, because it's 10:00 in the morning, but yeah thanks for coming I. My name is Ishaan I work at a company small company in Berlin called kinfolk. It's a far higher Linux expertise, company, mostly I work around kubernetes and stuff container technologies and I've been fond of rust for a long time.

B

Ever since it was first introduced, it was pretty unstable, so nobody could use it, but lately I have been using it, especially now also at work a bit so yeah I wanted to introduce memory management because a lot of people when they come across this new language they stumble with at least I, did I was really having a hard time with the memory aspect of it. So it's good to have an introduction to that.

B

So how many of you know essential rust, like some, some of it? Okay, quite many so I- do I, won't get into a lot of details anyway, but I'll just quickly introduce the aspects that I'll use later in the examples.

B

Well, mainly, it's like a system, programming language. So it's there, some programming languages. They ensure safety, some programming languages, ensure efficiency, but with rust you have both of them. At the same time there there is the focus on both of these things in the design of the language, so it makes it really really awesome language, in my opinion, and you have a concept of zero cost abstractions, so everything you code in in rust, you it will be, have a equivalent performance as it were.

B

If you write it in C or C++, actually, Sebastian he's in the queue outside I. Think he's not here right, he's working on GStreamer, he's, writing gstreamer, plugins and elements in in rust, and he actually saved a lot of CPU. He made it made the code perform much better because in rust you can, you can achieve it more easily because of the zero cost abstractions that we have in rust.

B

It has a non mutable state by default. So if you want to mark something as mutable, you have to use a keyword. So when you have a problem in your code base, so you can, you know which variables and which parameters to to look for, because if it's a non mutable state you, it won't give you any problem because it doesn't change. So why would Pro create a problem?

B

So it's a mutant, mutable state that you need to look for and that's why it's by default, you have I'll be taking on mutable and it has strict ownership semantics in C and C++. You usually just give a pointer to someone, and then you have to establish some sort of way to make sure that nobody double freeze it or especially in C we have like we have been using in genome.

B

G live libraries for reference, counting all objects, so you have to handle that manually in C, of course, but in rust you don't have to do that manually, but you have to think about how ownership works and at one time there can be only one owner to a resource. I'll show it examples later how that is so.

B

We will start with a simple example in an other in other programming languages, something like this you're passing an array vector to to a function, and it's just adding two first, two elements of that and returning it and in many languages this would just work fine. But now you add that and then you show it it's fine, but in rust this won't work, because when you as it says, when you past the vector there, you pass the ownership to that function and you never get got that ownership back so later on.

B

You can't use the the resource that you already pass to it because you gave it the ownership and the main thing is. There are two kinds of types and in rust there is copy types and then there's move types by copy types. There is the simple types in in rust, for example, integers and volumes, and those kind that can be passed around like as copies without any performance problems easily copied those always get copied.

B

So when you pass it to a function, if, in the in the example, if I was passing integers directly, that example would have worked because it's a copy type but move types which are most of the types by default. You create your own data type, it will be by default, a move type and by move I, don't mean the whole thing moving, but moving off ownership. So you give the ownership to someone else when you pass it to them.

B

If you pass by by value- and you can't use it anymore and that's as I said in rust by default, everything has only an only one. Every juicer resource has only one owner at a time in the code, so it's very clear and it's been rushed so knows where it went to free the resource and not you.

B

But the thing is you can't you can't work like that? You have to you, know: I have multiple ownership owners and stuff, and one of the first ways to do it and one of the most obvious and most used is borrowing which is like in C++ and C, both pass by reference more like C++.

B

So the same example, we just turned the parameter into a reference.

B

So when you pass it by reference, you are borrowing the ownership of that resource to that function, you're calling temporarily and once that school when which ships borrowed that's over, then the boring is finished and you have the your resource back, but if I had kept ownership, if I, for example, if I had said ampersand we and then assign it to a local variable, it has the same scope as V, then and then I could not have used it while it's still borrowed.

B

So while it's borrowed, you cannot use it, but once the borrowing is finished which is based on scopes and the scope in this case is the from that function that you called so when it returns, you got the ownership back. I went following so far or I have 20 minutes to explain this very complicated subject so I'm trying my best, but the problem with borrows is, as I said, it's temporary you, you get it back and the resource that borrowed it.

B

If you can't keep it forever and you don't have the ownership at the same time exactly so we will. We will start with an example. I fly helicopters, so I have. My examples are linked to that. So the average patient every aircraft has a registration. So it's a simple structure and I have an implementation for it. It's just a constructor get a registration as a string.

B

This string type is the owned string type. So you it's instant. There is two string types in rust. One is the own. One is not owned. There are none owned. Is there is a borrow? It's a piece of the string which could be the whole string itself as well, but it's a borrow, but this string with the capital S is a is the own type.

B

So we from main we pass it a string and on string and then we want to just use it there and then after we passed it to the Hallie, the struct constructor we again use it ourselves and that won't work, because when we pass by value we gave it the ownership. So as you like last time, so it's very similar or actually it's the same, just different context. You move the value and you can't use it anymore. So what would we do in this case?

B

The easiest solution is to use a data type called RC, which it's abbreviation for reference counting and what it does it's a container type which you you put something in it and it adds reference counting to that resource. So the resource itself. You don't need to copy it around if you want to pass it around and have multiple owners in the same code of the same resource and instead you just use RC and what it does is like each time you create a new user for it.

B

You increase the reference count on on that resource, contain resource and then once you're. All these scopes are finished that have borrowed their that have increased the reference count, the rest automatically the destroys the resource, the underlying resource, because now there is no owners for it. So it's just like any reference counting in any programming languages. You must have seen so in this case we just we do it exactly the same, but instead of passing the string directly, we now put it in a RC and pass the RC clone method.

B

So by clone I created a new reference count, I increase the reference count on it, so the Bandit goes to that one. Then, when the new function, ap keeps the ownership. So now we have two owners of the same resource. So when I said in the beginning that you can only have one owner at a time, I kind of lied, but not exactly because you really can't have two owners, but just provides you ways of working around it in a safe way in other programming languages.

B

You just you can just really nearly pass things around and ownership they can. There can be multiple owners and language have no way of testing if you will destroy the resource, while another part of the code is still using that resource, but in rust, since you put it in RC breast ensures that the resource is always there. As long as there is a user of that resource.

C

B

It's not it's a it's a yeah, it's in this case. It's just increases there. Let's count.

B

Good question, but the problem is our city's, not for multiple threats. If I run this, for example, now I introduce it's the same example as previously almost except this time. I have added a method and I'm calling that method from another thread, so I launched another threat to call that method, and now multiple threads are using the same RC, which we have RC in the in the structure. So this one will end up in an error. Something like this. Arrests.

B

Have these traits concept of traits and different structs implement different traits and that's how it ensures that different guarantees are met, and in this case there is a trait called sin that needs to be implemented by structures if they want to be accessible by different threads.

B

So we need a different struct which implements that trait that rust needs. If you want to access it from different threads and that structure we have in the STD in the standard library is arc, it's just atomic reference counting, so the reference count is atomic in this case, it's very similar to the RC. The main thing is that it's thread safe, so instead of RC I use arc and the same example, it's same code will just work.

B

B

But the problem is then R sees that it's not a sorry arc is not. Is that it's not mutable, so you can't like get a resource out of it and then modify it. That's not possible. So let's look at example. So this time it's the same exact code again and I accept that in the hover method. Now we are modifying the resource, and these are the methods I'm calling is just the normal methods, the string type, clearing the string and then pushing another spring on on the existing string.

B

But this this won't work because modifying it and the main thing is I can't borrow, is as mutable, because it's not a mutable resource and then market as money.

B

So for those things you need yet another data structure called mutex, which is just like a mutex in any programming languages and libraries. You get locks on the resource, so we put our mutex in an arc because we want to access it from multiple threads atomic with the atomic reference counting. So the arc is adding atomic reference counting and then the mutex is ensuring that you can have mutable locks on the resource. So you have multiple container types you.

B

You will see that in some times and rust you, you need to have multiple containers to the same resource. But the thing is, you have type aliasing, so you can use that to create a new type for your complex type very, very easily and that's just a elías, so it makes things pretty easy to follow anyway.

B

So this time you just use mutex you you create like when you created the arc you in there you just to a new mutex and put the resource in the mutex, but in the hover you need a you, take a lock of the off the from the mutex and when you access the mutex is since it's contained in arc you you get an atomic reference.

B

So you it's the thread safe and once you get the lock you, then you can do whatever you want, because you have gotten yourself a new tech, mutable reference to the to the resource, and then the thing is that rust is very much. The memory management is very much based on scopes. So once you are out of that scope, which is the hover method, the lock that you acquired will be freed automatically for you, so you don't have to Freed's.

B

If you want you at as far as I know, there is an unlock method, but you usually you don't need to you. Just use scopes to signal like that. Okay, I'm done with the with a lock and similar to the mutex, there is a type called read/write lock, which is, if you have multiple threads, that some of them needs only read access, not write access, so it it could be very inefficient if you have, if you're.

B

If you use new Tex, because every thread will have to wait for the other tribe to finish access and then be able to access with the readwrite lock, you can have multiple readers of the same resource. So multiple this, the threads can get lock on the same resource at the same time as a read, read lock, but not write lock. So, but if there is a thread X having a write lock, then others have to wait for for that to finish yeah. It's very similar.

B

I'm I'm, really sorry I'm going really fast, but I have very little time as I mentioned. So box is the type when you when you usually act, create a resource. You put it usually on the stack and but if you use those data structures that I mentioned RCE arc and those you put it on the heap, but it's it's more. Heavy RCS has a reference counting and it needs to ensure that reference count increases decreases thathe.

B

So you don't want to use it just for allocating on the heap. So if you want to just allocate something on the heap to pass it around well and keep the the resource around for longer than one function or something you can use something called box, and it's very simple: you just create a new box and the v in there is not the size. People think fits the size, but that's just an integer that I want to keep in the box, and then you just use it like, as if it's not contained in a box.

B

It's like usable, like as if it was just the integer itself, just like in our case of our CN arc. Another thing with the box is very important is that it gives the resource it keeps a size.

B

So if, for example, this case in rust you, your enums, can be complex, you can have data inside your inner particular enums, and so in this case we want to create a list that is have other nodes in it.

B

So we want to keep a list in a list and you we can't do that, because rust won't be able to tell how much memory to allocate to this, because you know it's infinite recursion.

B

We can break that infinite recursion by using box because box the list that, if it's it's going to be a list, it's on the on the heap and you only get a pointer in the box and pointer, is always the same size.

B

So you solve the problem and without using any pointers itself in rust, you can use pointers, especially an unsafe code, and you mark it as unsafe, but typically you can't avoid it and you should avoid it because that's where the problems are usually yeah I wanted to discuss briefly the life times, and this was another reason. I was hurrying up. I have done the same talk before in other conferences, but I always avoid lifetimes. So this is my first time addressing it.

B

I hope I can do some justice to it, because it's people are afraid of it.

B

So, as I mentioned you can, you can pass things by reference and references are temporary and the the scopes define the lifetimes of things. So in this case you get two strings in the longest function. It just returns the longest string. You have sorry I, have my pointer and.

B

So this should work right because you just pass the reference you get the reference to the longest string and yeah, but it wouldn't work because rust doesn't know when and you get helpful very helpful messages and nowadays from from Russ compilers. So that's that's really cool, so you borrowed a value and then you return a reference to one of the board, values and Russ doesn't know.

B

Well, you can easily can t to tell which which one it is, and you have multiple n parameters and one out parameter so in in this case you have to tell it. How long would the reference that you're returning have to be kept alive and what other resources it is associated with? So it keeps them both alive long enough.

B

So we can I know there, the syntax of lifetime's it's a bit takes a while to get used to, and even when you get used to its you still hate it, but that's how it is I, don't I have no better suggestion. So I, don't criticize much, but I know that it seems really really weird.

B

So you you declare and the function definition a lifetime.

B

It's an abstract concept of the lifetime so and you declared that all the all the resources that you're using like the in input parameters and the return value, have the same exact lifetime, which means to tell two compiler res compiler, that if you, if a s1 s2 and the return value, needs to be kept alive together, so you can't it can't free, for example, the s-1 that is being passed before the usage of the return value is finished or the same with s2 so in in here.

B

In the example, we sorry we passed the two values and then we get the reference back and at this time it's it. The usage is exactly the same as we we had in the previous example. It's just that we now told rust what are the association of the different lifetimes involved and the lifetimes are more like declarative than instructive. So it's like you, give hint to the compiler and it's up to the compiler, how it handles those you just tell it like. These resources are related, don't free them before you free others, yeah!

B

Actually, that's all I had so I think I have a lot of time for questions and that's good thing.

B

B

Do you want me to move to the previous slide.

B

Not exactly sorry I'm not used to it so and I was told to repeat the question. The question is that there is case where s1 is is to be returned and not s2, and in that case we don't care about s2 anymore and it's the s-1 that that can't be freed while they return well. You still well around that's true, but it's very hard for compiler to to find these out.

B

One thing I did not mention is that in rust you have something called lifetime elysian, like the one of the code that you saw earlier, where I was using references, but you noticed that there was no lifetime involved there and the reason was that lifetimes are always there, except that in many cases at least the simple cases. Rust is smart enough to detect okay, the compiler, and it declares them for you in the code, so you don't have to care for them. But it's like a bit complicated temples like these.

B

Then you will start to when you have multiple references being passed and then you have returned as a reference as well, then rust will get, would not know how to handle that because it can handle it on its own, but not through their lifetime. Syntax. It's like because it's more dynamic, like we don't know which one the witch string that will be passed is is going to be longest. So it's more like a dynamic decision and Russ does all all thing statically. So it needs to repeat to do it. This way.

A

B

You can have multiple lifetimes, that's true, but in this case you need both of them, because you don't know before you know which parameters will be passed because it's more dynamic thing like you, don't know what, when you're compiling just that function, let's say in a library and it will be used by people, you don't know, and you don't know what will be passed so it will. You will only know dynamically, which, which case, would it be? So that's why it's it's not possible.

B

No other questions.

B

You put like in there in this example you you have like a lifetime, a you just put a comma lifetime B, so you can declare multiple and then you can use them like the same as C I'm using a left.

B

Sorry, the question was: if I can have multiple lifetimes in there and that's what answered sorry.

B

The question is: if the lifetime is part of the reference type, not exactly, but it's always there like when you pass things by reference, the life there's a lifetime involved. It's just the difference is whether you declare you have to declare it or not, yeah. Actually, it's implemented through something called generics in rust. If you look into generics, you will see that syntax of declaring them exactly exactly the same and that's how it's implemented, and you might want to read up on that. It's it's pretty interesting stuff. How lifetimes are implemented.

C

B

That the question is, if I understood correctly, is that if the in multi-threaded application is it possible for one tattoo or write the other resource of others.

C

B

No, like you can't really mess up with lifetime. No I sure you know it.

B