Rust Programming Language Other conferences 2016, 14 Jul 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: PolyConf 16: Owning Ownership / Sean Griffin

Description

Every function or method we write in any language has some hidden semantics. Who owns the values passed in? What am I allowed to do with the values returned? In 2015, Mozilla released the Rust programming language, which brought to the table ways to represent these "ownership" semantics in programs. In this talk, we'll take a deep dive into how thinking about ownership can help keep defects out of our programs, regardless of language.

A

A

Normally wear my hat, but it doesn't really work with this microphone, so I feel kind of naked hello, so my name is Sean. Griffin I maintain a web framework called Ruby on Rails I'm, a tess Grif on Twitter. If you want to tweet at me, while I'm giving this presentation, that's how I'm going to judge my self esteem afterwards. So you know, if you want to there, it is if not that's!

A

Okay, I'll just go cry in the bathroom anyway, so I'd like to start this talk with a very controversial statement: Ruby JavaScript and Python are dynamically typed languages and I'm.

A

Sorry, if I made anybody do a spit-take there, but the important point I'd like to make about this is that just because these languages are dynamically typed it it doesn't give you what I mean my save is dynamically typed is that it does not give you a way to express types or verify that your code is well typed, but thing is your program is going to have types either way it doesn't matter if you're working in a statically, typed language or dynamically typed one, you still have types.

A

You still expect a variable to adhere to a certain interface. You still reason about your program based on the types of your objects and by type I don't mean the specific class of an object. Thing that responds to to string is still a Titan and when you think about your program this way. Well, of course, your program has types: if something didn't have a type, then you wouldn't be able to do anything with it.

A

In fact, I mentor at a programmer I used to mentor at a program in Denver called Turing, and when they, when the students would start to get lost, they would be learning ruby. It's their first programming language. One of the questions I would ask to get them back on track. Is what are the types here? What do you expect? What do you need? How do you get from A to B types? Are lovely thing thinking about them in the back of your mind when you're writing your code helps to prevent a lot of bugs.

A

So the point to all of this is that semantics still exist, regardless of whether or not you're working in a language that gives you a way to express or to enforce them.

A

So types are one type of semantics that some languages don't have. Let's talk about another type of semantics that these languages quote don't have so ownership is something that was explored a bit in C++ starting in 2011, but was truly introduced to the world by a new programming language called rust. Rust was introduced last year by Mozilla I'm, so all values have an owner. You either own an object or you're borrowing it. So what does it mean to own an object in the simplest sense, owning something means that you have the right to destroy it.

A

So now you might be asking well what does it mean to destroy something? It actually doesn't specifically matter what destruction means other than that after something has been destroyed? You aren't allowed to use it anymore. So now you might be asking what's an example of something that can no longer be used after it's been destroyed. You might also be asking when I'm going to stop answering these questions with more questions. Okay, so, let's, let's talk about?

A

Let's take a look at some code, I'm going to show some examples in rust, because it's the only language that has syntactic ways to express ownership and that's what we're trying to demonstrate we're gonna work with the file type, because that is something that has very clear destruction: semantics in almost every language you open up a file handle and at some point you're going to call closed on it and you aren't.

A

If you're in object-oriented language, you likely have a single object there and you should not be using a file handle after after it's been closed or bad things will happen. So we've got two functions here. The first one is going to do some stuff with a file and the body is completely empty there, because what it's doing doesn't matter, but it takes in a file and specifically because it takes a file that that in rust is referred to as taking ownership of it.

A

So when you call this function, the value is going to be moved in there, and so when the function returns, that file will be destroyed and so in our main function here we're opening up a file, we're gonna, try and do stuff with it, and if we try do more stuff with the same file, we would get a compiler error, because we have moved that in somewhere else. We no longer own that file, so we are no longer allowed to use it. Now.

A

It wouldn't be terribly useful if you could ever call only ever pass a value to a function exactly one time. So this is why we have borrowing.

A

So this is the same example, but you'll notice that in that first function we take a an ampersand file instead of just a file. So that means that this function wants to borrow a file, and then we ampersands in front of it when we're calling it so the first time we're borrowing it we there are limits to what we can do with it when something is being borrowed which are not super relevant to the general concept of ownership. But then we can pass ownership to it later on because we have not given up ownership.

A

So a reference for those of you. Who've worked in c-like languages. A reference in rust is there are mutable and immutable references or shared and unique references depending on how you want to talk about it, but it's just a pointer, that's guaranteed to be non null for those of you who haven't worked in c-like languages or what I just said made no sense, I'm very jealous of you and I hope that you never have to over can see like languages.

A

One other thing that that comes out this, at least in rust, is so mutable vs. immutable references. So you can, you can pass around a mutable reference to something, but you can only ever have one mutable reference and no- and you can only mean take something if you are the only person or the only code that has access to that to that data. You cannot share and mutate at the same time.

A

So objects are structs can own things. It's not just functions. This would. This goes into the definition of a type. You say whether it's fields, whether it owns those fields or whether it borrows those things if you store something in an instance variable in a language like Ruby or Python, you want you almost always own it. It's very rare to have actual sharing between that in most programs.

A

One thing I've noticed as I started to think about ownership. While writing Ruby is that I've started to notice that God objects in my system also tend to own a lot of stuff so and God object is a term that we refer to for something that a lot of things need to know about, or has a lot of behavior on it. If we focus less on it has too much behavior and more on it owns too many things, then I would like propose that we rename God object to I.

A

Don't you've heard he claims he's very rich. Also, it's it's Canada day. I didn't I was thinking about cutting this since I'm, not in America, but it since I think I'm gonna have some new neighbors soon, potentially I thought so there's one other caveat and there are certain types which just like these semantics completely do not apply at all. We call it.

A

We call these types copy and rust, and it's basically anything where you can copy this as many times and it doesn't matter the these are things like boolean's and numbers things where the immutable value types. Basically, nobody owns the number three, although Trump might try to I, don't know alright. So let's go back to the original point, so owning an object means that you have the right to destroy it. Values have exactly one owner. If you need to temporarily do something with an object, you borrow it and things to be borrowed either immutably or immutably.

A

So my original, the the first slide I had here right, Ruby, JavaScript and Python are dynamically typed, but your code sauce to reason about types, and these are languages that also don't have an explicit concept of ownership. But I would like to propose that your code still has to reason about ownership, because semantics still exists, regardless of whether languages give you an explicit way to express them or to enforce them so I'm, showing a language that there's probably like two people in here who've ever seen before, and nobody cares about.

A

So, let's look at some examples that are in other languages, so this is. This is an example in Ruby. This is some code that it that is a real example of where ownership semantics matter in the rails framework, so active record is R as R RM, and most of the specifics here of, like all of the code, are aren't important other than the general idea. That user has a setter and getter for a field called name.

A

It's going to course that to a string and what we're doing here is we are assigning a variable called name to a value passing that into the constructor for this object and the Emme assumption the active record is going to make here is that it takes ownership of anything that you assign to one of its attributes. It has certain things it does internally to maintain state with regards to the database and it needs to know it needs to be able to know when things are mutating.

A

It needs to know that you're not going to change something out from under it. So if you were to then hold a strong reference to that value and try to mutate it, that would cause bucks, and this was possible in older versions of rails and it did cause bugs and it this now actually breaks in newer versions of rails. But this is this is the slightly the same code, but slightly different you'll notice.

A

Here we are no longer mutating at the local variable, we're doing user dot, name and mutating, and so you can actually just eliminate the local variable at that point, and it's doing this- and this is now fine, because when you're calling that name method, active record is loaning the name back to you, but it the point being. It has this explicit ownership contract that the language doesn't Express.

A

But what can what will occur if you have, if you are holding references, something outside or trying to assume what is or isn't shared versus going through the API that it wants you to go through? You get very different results, and so I want to look at a real bug that occurred in rails that wasn't just like user code bugs but an actual bug. So this is a person that people who do Ruby will recognize who introduced this bug and I, don't normally like to like blame people, but I put fun ahem.

A

um So we had this method and some code, and or this is essentially what the code did. So it accesses the instance variable called n, which is a hash here, and it's say it's trying to say if something the value, if there is no value at this key, assign it to an empty hash and return whatever the value is after so, the change that was introduced was this rather than accessing the instance variable directly.

A

It's calling this get header method and instead of doing or equals it's just doing or so get the value at that header or return and empty hash, and the commit message was use Get, Set header, to avoid, depending on the NVR, so the thought process here was just very, very specific. It's not.

A

It doesn't give a ton of context about the code, but you can tell exactly what they were trying to do and what they were and weren't thinking about so somewhere else in in unrelated code, we had something that looked like this, so controller dot, request, dot path, framers that path parameters bit at the end. That's the method that we were just looking at and it's you know it's doing square bracket equals and then trying to assign the result right.

A

So that's code that is getting the the value at that header or returning an empty hash, and in this case it's mutating it so.

A

So, looking at looking at this case before we were always returning, a a hash that has the same owner is always owned by that request, object or by that at end of Ivar, whereas afterwards this empty hash, it disappears into the nether. It's not it's not kept anywhere mutating it effectively is a no op, and so, if you were to look at the signature of this function in rust, this is what it would look like. Path.

A

Parameters takes a reference to self and returns a reference to a hash map, and one thing: that's implicit here is that the reference is owned by self just because it is a reference and you can't pull a reference out of nowhere. It has to be referencing, something that is owned somewhere else unless you want to leak memory.

A

So when you think about this and your think about ownership, not explicitly, but it's in the back of your mind, one of the things that came to my mind when I was trying to figure out what was going on with this bug and I saw this code was who owns the empty hash and lob. This is just about making API contracts very explicit, so you have to think about when you're returning values, especially values that can be that exist in mutable languages.

A

You have to wonder what do we expect the caller to do with the result of this method? Are they allowed to mutate it? Are they allowed to keep a strong reference to it, or should they be calling the same method every time they want to access it or they'll? Are we allowed to mutate it without telling them?

A

Are we allowed to just replace this with a completely different hash, or it's expected to always be associated with with what what was first given to us, and so looking at this I? Looking at the code that introduced this bug, I'm traffic I tried to figure out what was the intention of this change, and this was the commit message called the path parameters equals setter, rather than rely on mutations.

A

So ultimately, a lot of this code is is trying to get away from allowing the result of this from being mutable in the first place, so the signature that we have is we want to take a reference, a mutable reference itself in return, a mutable reference hashmap, but we actually want is an immutable reference. So how can we make this ownership contract more explicit in a language that does not give us a way to reason about ownership?

A

So this is a completely a different place that had a very, very similar, not actual bug that occurred, but a similar solution. So we used to have this method changed, attributes which would just memorize and empty hash and would get mutated elsewhere and eventually I introduced code. That did this, which, if you're not familiar with Ruby these methods, don't matter the important thing is this always returns a new hash? It is not returning a hash that is stored somewhere. So if you mutate this, it would have no effect.

A

So the return value isn't owned by an active record. Your mutations wouldn't be seen by active record. They could be seen by you if you stored it somewhere, but but we don't own it, we don't know what you're doing with it. So this was the solution.

A

Freeze in Ruby, for those who aren't familiar just says, make this thing immutable blow up. If anybody ever tries to mutate this thing and just like that, the contract suddenly becomes explicit. Nobody can think that the ownership Smita contract with the return value of that function is something other than what it is.

A

So the bug was eventually resolved. This was the change. We're actually still just doing the same thing that that hash does need to get muted, mutated in places so now we're just making sure that we memorized that empty hash.

A

This was the commit message. Remember the parameter hash we returned callers expect to be able to manipulate it I, but to put that another way, I would describe that as we expect our callers expect to borrow a value that we own.

A

So if this whole process, if this whole commit had been gone, this whole bug resolution had been gone through ownership in mind the entire time. I. Don't think this bug would have been introduced in the first place. I think it's important to think about all aspects of API contracts. An ownership is just one more that we should have in the back of our head. So let start thing about ownership in our code and let's have fewer bugs I have stickers. They look like this, therefore n ORM that I've recently written in rust, called diesel.

A

It's the fastest way to interact with databases in any language. You can check the sound the internet at this URL. Thank you very much. I will now take questions.

B

I just wanted to thank you and one question about Pawnee language. Have you heard about it? It also introduces ownership. Classifiers I have.

A

Not heard that language now, but you should, we should talk afterwards. I would love to hear more about it and will not leave this stage until somebody else asks the question just for the record, also water, it's bubbles in it here and that weirds me out. It's not like that in America.

C

A

B

C

Quite ignorant and superficial I'm gonna ask it anyway, um so I'm just wondering about so. You started with talking about types and then I notion of ownership right. What I'm wondering is because I can meet, maybe because, because the ownership concept is kind of new to me, I'm thinking how how we can talk about the ownership is an object, objective entity. Does it really exist, or is it just a sort of reference framework that helps us to think about the protocols and contracts?

C

Because a type system is something very real to me that the thing needs to have a type well.

A

So a type system is very real right, but the idea of a type is very much not and can be, and you can have very different interpretations of what a type system actually means and if you look at say the Java type system, a compare that to say the Haskell type system. What it means for something to be of a given type or actually go is another good example of a language that has a come. That is statically typed, but completely changes. What a type means so I think it's.

A

It's like what I'm trying to sell rust part of what I was describing was rusts specific ownership system, but that is independent of the idea of ownership, and so the the the and so I think it's a very similar relationship where there is the abstract idea of ownership, which is more of that, as you put it the framework to help reason about about those contracts. But then, if languages have their have an actual semantics towards ownership right that that there aren't necessarily going to be concrete rules for exactly what ownership means in the abstract.

A

But languages may or may not introduce more specifics. Does that make sense.

D

Hello thinking.

E

About ownership is like some sort of like a horizontal property of the types. Would you consider any other properties like that.

A

Yeah thread safety for sure like whether or not something is able to be safely sent to another thread and whether or not something is able to be safely shared across threads. Even if it's immutable, for example, a database connection in almost every language is implemented in a way that you cannot share that between threads reasonably.

A

It's an interesting question: there are probably other concepts that that would have that those sort of same properties- those are the ones that come to mind off the top. My head, though,.

A

We got one down in front.

E

So isn't ownership something you have to think about only if you deal with mutable states, because I can, like think of situation in which you would have immutable data, and you would have to deal with ownership right.

A

So I knew I was gonna, get that when I saw the Haskell stickers, haha so no but immutable languages do tend to push you towards things that help you express the ownership contract more correctly in the first place so I'm, just your data structure is immutable, but you're still going to be making changes to that structure and those and those changes need to be reflected where who everybody else who needs to care?

A

Haskell, for example, just tends to push you towards the state monad and structuring your functions in such a way that having correct ownership semantics there falls out of that naturally, but there is still sort of the top level like I'm responsible for this and and and anything else that is going to potentially update this I need to make sure that I'm those and assign things correctly, our signs, not the right word but I- think in due notation, so I see it I think it is assignment but say it, but Same Same Same in in cloture right, like all of your days, all of your data structures are persistent, but you do still need to represent where those changes are somewhere and there is a place that is responsible for managing that yeah, hello,.

F

A

F

It was question was about the first example where you were showing borrowing so before you were borrowing something.

F

A

F

So in the second line, where uses lino let file blah blah blah, you know do something so I. You know you owned that file at that point. Right is there's there's ownership. Stopping me call that method at all, or is it being able to make any changes to that method, because the second slide I understand like I, can borrow actors borrowing, let me call the method or does it borrowing? Let me kind of change the font change file so.

A

Borrowing, just simply me, borrowing simply means whether or not you're gonna destroy something when you're done with it. So it's in rust, it's specifically based around lexical scope. So when do stuff with file returns, file is destroyed, and this is a compile time guarantee. Not I mean it. It's also a runtime guarantee, but like this is a compile time check, but so borrowing just means that that by passing it in here you're not destroying it methods in rust, like our kind of a thing, they just happen to take self as the first argument.

A

But it's the same thing you can take our reference to self or you can take ownership of self, but all it means is that after the second, the second line in main, you cannot like, if you do anything with the variable file other than like rebinding it to a new, a new thing. If you try to do anything with that, it will fail to compile, because at that point file is known to be destroyed. So there is nothing that you can do with it. Okay,.

F

So in the second example when I'm borrowing it I could do something see, like reverse the contents of the file, but I can't destroy it. If.

A

Yeah, so so, and you never explicitly destroy things in rust, they they get implemented astray just when an owned value, exits exits scope, so you would never but uh another way to put it, though there that right, you could never call a method that takes ownership of that file, which would effectively be destroying it. Yeah I.

D

Have a question about ownership and shirt pointers, so I worked in two big companies in large C++ code bases and obviously in C++ you got a a concept of smart pointers, right, unique pointers which guarantee that something is unique and that's a strong guarantee that you hold ownership of that. And then you have a short pointer right. So in one company like if you introduce the shared point or you get a punch in the face during the code review right, it's great and.

G

The other company.

D

Everyone just assumed that, because ownership is difficult, let's just use short pointers because it solves the problem right.

A

D

Would you, how would you recommend to I suppose evangelize the other community, that shirt pointers are not great because they seem to work right.

A

Well, they do right and we and we have shared pointers in rust as well. So we have our C and a our C which stands for reference, counted and atomically reference counted, but they only allow you to get an immutable reference to the to the interior to the interior contents, like references and rust actually map directly a mutable reference is a basically a unique pointer and a shared references is. It is a shared pointer, a shared pointer, but basically like I guess the way to evangelize it with being the actual question right.

A

Shared mutable state is sort of the thing that everybody is trying to solve shared point. C++ gives you sharing, but doesn't give you ways to manage that sharing. Well, once you have it or more ways to manage the mutability. Well, so shared pointers, kind of give you give you this gateway to having shared mutable state, which tends to be the source of a lot of bugs, especially if you're in a multi-threaded context.

G

Hi so there's a there's, a trend that goes on right like that over decades that concepts that people thought about some time ago start to make their way out into the larger world migrate like what started happening with with functional programming approaches you know, and and and starting to bring those into other languages where they weren't at first used, and somebody put sicp up on the screen yesterday and it was kind of like okay well but I, wonder many of the languages that are around are kind of suitable for bringing in functional concepts.

G

But it's interesting that you're showing languages where ownership is not at all explicit and I wonder. If do you tend more to think that we want to go in the direction of finding ways to make ownership explicit in those languages or just using these languages that are doing a better job of making it a concept? In the first place? No.

A

I'm, definitely thinking I'm definitely trying to find ways in the same way that we find ways to to reason about types in dynamically typed languages. That is that that is definitely what went up. What I'm pushing towards just understanding the concepts I found is very helpful, so that.

A

This slide here, like I, actually would get for quite some time, but reports about about like the issues here that would occur from this and basically once I actually started start to understand ownership, semantics and and reason about them like that actually became a very succinct way for me to describe exactly why.

A

No, you should you're not allowed to do this. Here's the explicit contract that we hold, so that was the first step of it and then the second step for me was realizing. Okay. Now that now, let's see, if there's ways, I can express this at runtime, which in many cases just turns out to be freezing things, but freezing things remem, oh I, think things tends to be. It tends to be a solution that a lot of time, but finding ways to make explicit runtime is definitely the second part of it, which isn't always possible.

A