Rust Programming Language Rust Belt Rust 2017, 6 Jan 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building on an unsafe foundation — Jason Orendorff

Description

Beneath every safe programming language, library, or virtual machine is a whole lot of unsafe code. Rust is no exception. This talk will explain why “unsafe” code is allowed at all in a safe language, show the unsafe code at work inside safe features like Vec, and teach how to write a safe library that uses unsafe code.

A

It's a great speaker great to be in Columbus, so quick show hands like how many of you are from are from out of town, okay, almost everybody that is great. So so am I and but I, but I love Columbus, because- and this is like I made these slides weeks ago- I like it, because this is where Ginny's ice creams got started, they're here and they're in my hometown of Nashville, and if you haven't so, did anybody go there for lunch? So what do you think yeah? So try the bramble berry crisp.

A

It's the real thing! The Columbus has a lot going on. It's like there's a giant University here. So it's like got the arts and and nightlife. You expect it has a killer, indie bookstore, but like what I like is the weird stuff. It's got topiary park. You should check it out. It's like right around the corner, walking distance from here 10 minutes and it's like free Instagram likes just like sitting there. So why would you not.

A

The drainage Hall of Fame is here, that's a big tourist attraction. If you have time for a little bit of a drive, you can a little bit outside the town. There's a wolf sanctuary where you can meet actual like live. They call an ambassador wolves because wolves are nature's diplomats.

A

Corne hinges here: I feel like Jimmy Fallon. We have a great show for you tonight. Folks, Korn hinge is here and I play a little football. I guess I, don't really know, don't try and drive. Tomorrow is the advice that I got so I bring up football because football is my excuse for not knowing anything about Columbus who knows at the impressionable age where I could like actually learn the names of cities.

A

I only learned the ones that had pro sports teams, because I grew up in the suburbia in the 1980s was a weird time anyway. So I knew that Cleveland was a great American city like right up there with New York and Chicago I'm like Green, Bay and I, knew about Cincinnati and I could even like I knew what these cities look like on a Monday night from a blimp, but but not Columbus, even though they're like the same size, city and I, just like this week found out what the deal is.

A

It's it's that Columbus has Ohio State, and so they never had any use for the NFL and in fact, to this day most people here, don't know what that is.

A

The NFL has like an even more serious problem than that. So did you see this? This is a report that came out in July of this year. Researchers study the brains of like 111, former NFL players and found evidence of brain damage and 110, and, if I had to guess based on the league's response to this stuff, I would guess that they don't think that better equipment or schedule changes or rules changes are gonna. Make this go away.

A

I think they're acting like nothing's gonna help and they could be right, and so there's a question like: where does that leave us like? As a you know, as a fan, I mean I always knew that, like a career in the NFL is not a walk in the park, it's risky right, it's hard on the body, even if you say healthy and injuries happen, serious injuries all the time, but this is different. It's beginning to look like a career in the NFL.

A

Is just a bad idea if you value your brain and as a fan like, on the one hand football, but on the other hand like I, don't want people like burning up their brains for my entertainment.

A

So you probably won't hear about rust, but I like to start every talk with a lengthy non sequitur that gets really heavy at the end.

A

Thanks, no, it really it's like I, don't know how to give it talk longer than five minutes so enough about enough about serious, dangerous, being ignored. Let's talk about C++.

A

Actually I'm gonna take a moment here. A nun mirror my display, so I can see the words I'm supposed to be saying.

A

Yeah did not do it. Give me a second.

A

A

Thank you for laughing I got the impression that that there might be some people in this room who have you C++. So if you've used C or C++ professional, you could you just raise your hand, all right cool, so the rest of you just look around like the people that raised their hands. They're like these people have seen some stuff.

A

They know how the sausage gets made. They know that all of software is built on this giant pile of sausages and that knowledge haunts them so for the rest of you. I just want to give like a quick impression of like you know what C++ has done to these poor people thread.

A

So like say, you're writing some C++. You have a very well nice integer here and you want to print the value of that integer which, for some reason and C++, looks like this well, this is terrible. This is a calamity.

A

You forgot to initialize the variable, so so you're, probably thinking like.

A

You're, probably thinking like well C++, is going to like throw an exception, that's very sad or maybe it'll print 0 or something or maybe we'll print like Eve. You really thinking you're thinking like okay, so it's putting the value of a variable. We never assigned anything to that variable, so maybe we'll just print whatever random bits it finds in memory. Well,.

A

It might C++ might do any of those things or all of them, because this is what's called undefined behavior using an initialized value.

A

Undefined behavior is something that is talked about a lot in the C++ standard and what it means is. Basically, your program might do whatever all right, but this isn't the only way to get undefined behavior. It's actually the several things that can happen. There's a couple more I'm just going to talk about like one or two so say: you've got a nice variable, an integer here, I initialize it this time right but I'm about to do something very stupid about to add up some numbers right.

A

This is what a loop looks like and in C++ and I'm like adding up numbers, and this will work fine until the total gets too big to fit in like 32 or 64 bits, whatever an integer is and then guess what I'm defined behave. So don't do that.

A

One more real, quick since I've got 30 minutes to fill so math is tough. Let's do something really simple this time, let's just like pass an argument to a function. So this code right here this is fine.

A

This is this is this is safe like because this is clearly crazy right, but this is totally fine and will always work unless, unless you're passing this, unless this function takes its argument by reference, because then what could happen is it could be like storing the reference somewhere to use it later right now, in a language with garbage collection.

A

That would be fine, because you know you have reference to something and the system basically just keeps it alive as long as you need it and C++ once this code is done running this and the the string leaf scope, that string is dismantled and the parts are taken away. So it's like it's for real gone. So what happens? The next time your program tries to print a prompt undefined behavior. So don't do this there's a couple more things: haha I'm, just gonna go through them quickly, because life is short.

A

I had to make the font size smaller accessing off the end of an array. Nobody ever does that right, undefined behavior, don't assign assigning things just some of these things, there's like yeah, don't don't modify a container while you're iterating over it you'll get undefined behavior, oh, and if you try and use threads in C++ that comes with his own whole set of rules and there it's actually like really kind of the hardest thing to get right in C++. So so don't do undefined behavior, kids.

A

But how bad is it really well? I had two pins there they're like several things that could help me. Well anything could happen right, but in practice usually one of three things: either your program just crashes and dies- and this is the good case, because if you like, it's kind of like an exception, you notice it right and there's actually tools a good tools to help.

A

You like debug, that the slightly worse thing is like your program, read some uninitialized memory and treats it as data right or it like, writes some information, and then we in club or something and then your programs just gonna, be flaky from then on or it you know. Maybe right, maybe maybe nothing bad will happen if a book, if it doesn't bite, is it really a bug.

A

So then there's the other case, like the other bad thing that can happen is that your program could turn against you. This really happens because the thing is like the reason. There's such a thing as undefined behaviors what's going on, is the compiler takes your program? You know it looks at this like integer addition or whatever, and it's job is to think real hard and like work on that code and then spit out a sequence of machine instructions that carries out what you want it to do right so with undefined behavior.

A

The deal is the compiler sees your code I think it works really hard on that code, and it just assumes that you would never do something so stupid as to leave a variable initialized right, because the compiler thinks very highly of you.

A

But then, when you run the program, what's gonna happen is like obviously the behavior of compiled code. That assumes something false is going to be unpredictable, not predict not useful in practice, right, not good for us as programmers, but it's still a program right and unscrupulous persons could like look at that sequence of machine instructions and figure out what your program is going to do after it goes off the rails, and this happens in practice.

A

A

A

This concludes our crash course in C++ see what I did there.

A

So we know rust. This is a safe language because it says so on the label right here, I.

A

Thank you for the slide. You know it's funny like in other contexts. The word safe, like the meaning, is pretty obvious, like if you're a young hero and there's an old guy, and he says to you it's dangerous to go alone. I want you to take this. You don't ask like what dangerous. What do you mean interest, because the fact that he's giving you a sword kind of answers that question right like it's, not the roads, the monsters.

A

But if someone says like it's dangerous to go low and take this and gives you like moves and ownership and references and a borrow checker and really long error messages, you might be a little perplexed. So so the first thing I want to tell you here, is to like resolve that in history, for you, Rus notion of safety is best understood, I think as a response to C++ and I define behavior right. So what it's protecting you from is undefined behavior and some other stuff, but mainly it's the monster. Right.

A

We've always known that writing a bunch of C++ is risky right. It's the consequences are bad. You screw up and programmers make mistakes, but it's starting to look like now that we're a couple decades into this internet thing. Now that we're seeing the consequences it's starting to look like we're writing a bunch of C++ code is just a bad idea and we need a better way right.

A

We need a language, that's as fast as C++, without the monsters would that be great without undefined behavior well funny story, so has anybody used unsafe code and Russ I know at least one person who gave a lightning talk right: okay, okay, good cool, so for the rest of you, what is unsafe code and rust? Well, unsafe code is code where you screw up and then mmm thank you.

A

You get undefined behavior and, like the saving, grace is well, we put a label on it and as it's unsafe, but this you know this brings up a question which I'm sure a lot of you have which is like. Why do we even have that lever hike? You know language designed from scratch for reliability like why. Why would you ever include undefined behavior for that matter? Why does C++ allow it like what what? What is the purpose?

A

So then I say I want to talk about is like is why we do that and it's gonna, it's gonna, take a while and I get around to it.

A

First, I want to show you some code, so this is actual real live code from the standard library, except with the comments removed, so it found a slide, and this is from the VEX I. This is the generic type, it's a vector that holds values. You call this push method to push a value of type T, my to the end of the array right. So what does this actually do? How does this work well, the way the way of vector works is you've got some memory.

A

Set-Aside and you've got some existing elements in there and then you've got some extra room right and push is going to use that extra room to store the next element. But the first thing this function does is check to see. If actually, your memory is full because then it has to allocate some more memory right and it needs to. Basically, it works by doubling the size of the buffer, okay and then.

A

Every time, every time you put you push a value into array, this code runs and it gets a raw pointer into the like to the vector, and then it does. Some pointer arithmetic, like C++ style, uses an unsafe method right to like write data to an arbitrary address in memory and then just for good measure. It bumps this private field, like the correctness of which is absolutely critical to Beck, not crashing so is this way. Is this sort of thing going on anywhere else in the standard library.

A

What do you think.

A

That's not going oh.

A

Well, wrist is in good company. Think about it like what's a safe programming language Java, what language do you think the JVM is written in quarter million lines of C++ Python, half a million lines of C right? Here's, here's the Ruby documentation for the their their push method was something I love that Ruby is you? Can you can view the source of anything and their documentation is pretty neat? Oh wait. Rust! Has that too? So, let's look! Oh, it's see ya. So what about like?

A

What about a language designed by smart people right like what about Haskell right has yeah yeah, so Haskell yeah, it's a little special! It has like a the Haskell. Runtime is actually written in Haskell all psychic. It's fifty thousand eighty thousand lines of C. It's a cartridge collector! You can write that in Haskell and that's not even all of it right. Just like all these languages, all these programs right they run on top of an operating system. It's Linux, Mac OS right.

A

It's millions of lines we'll see when we think of like a safe language as being like this cozy secure experience, we're like ignoring all of the supporting code that has to go underneath that right, it's a little bit of a fairy tale. The truth is more like, like Saruman tower and Lord of the Rings, where you like. Okay, that's a nice tower like that's clearly designed to be robust, but then like underneath it. You got like this maze of goblin Warren's and like there's building code violations everywhere.

A

A

So so what like, why is it like? This is unsafe code under every rock like and if there is like how safe are we really? Is there any point in using a so-called safe language? Should we all give up and go be goatherds? That's the real question here and if not, then how do we engineer safety in unsafe environments and unsafe code? That's what it talks about you. You made it to the title. Slide! Congratulations!

A

Okay! So that's a lot of questions that you just asked: I'll, try and answer them. First, I'll start with like why unsafe good sipping, so there's like two basic reasons. One of them is, you know, speed. It's like just kind of the fact that that sometimes the safe thing to do sometimes the safest thing.

A

Is you check again right, like you check the array access you make sure it's in balance, you redo the lookup to make sure the table hasn't changed right and in cases where you the programmer, know that that's extra work, that's not necessary. It's just gonna make things slow. You use a little bit unsafe code and it goes faster.

A

So that's one reason, but then there's another reason is that there's a lot of code that, like there's a whole kind of code, that you may need to write that isn't a very good fit for rust safety system, because that safety system, the like the moves in ownership and the borrows, and all that it makes some assumptions right and the assumptions are things like? Well, all your code is rust and then we're okay right, because obviously rust can't verify anything.

A

That's going on once you call in to see and there's a whole bunch of C code, that's loaded into every rust program as soon as it starts to run right. So so, so that's why the interface between rust and other languages is always going to be on safe functions.

A

The other big assumption is that there's no such thing as uninitialized memory like this is actually a really great thing about the rust programming language. It protects you from having that right, but remember the vector remember how the vectors work is by having a little bit of extra memory, a little extra room to use to put in more elements. That's uninitialized memory: how do we implement vectors in a language where you're forbidden from seeing that see? There's a like, there's a comfort zone of like normal code? That's the bulk of what we do.

A

We're Russ safety system is a perfect fit right, and then there are these. There are these parts. Sometimes you just have to say like what I need to write code. That's gonna cross some lines here, right and I just need you compiler to trust me I'm, going to use some unsafe code to do that when you run the Russ compiler on your program. What that, what all that, what all the safety checking is doing? Well, the type system is doing it with the with the borrow checker is doing.

A

The reason you have arguments with the borrow checker is that rust is building a proof of the soundness of your code right and all those all those elements, the language, they're parts of that proof and you're convincing rust that your code really is safe. That's what it is unsafe code makes a gap in the proof as a as a part where, like the russ compiler, just can't be sure it's right, it doesn't know right. You were responsible for the correctness of that.

A

This unsafe block and rust it really should be called trust me right because, like that's what you're saying you're saying like here, you know: I've I have looked at this code and I've looked at this pointer right and I've thought about it, and it's correct trust me now. I know what you're thinking I, don't trust that guy, which is fair, a little hurtful.

A

But I mean it's it's! It's telling that it's in rust that we have this conversation right, I've been writing C++ for years; I never had to type trust me: okay, I'm, just gonna XOR. These pointers together. The compiler trusts me it's crazy and.

A

Rust is not quite so trusting right, rusts checks, even in an unsafe block, the type system is still there right, like types are still checked. Lifetimes are still checked. The only thing that's unlocked for you in this mode is, you can call unsafe methods and you can dereference raw pointers. So now you know how to shoot yourself in the foot and rust.

A

My last few slides here are about how not to shoot yourself in the foot with unsafe, good.

A

These techniques, by the way, they're all classics so evil, there's nothing new from here on out in in, like in good rust tradition. This is like well-established stuff.

A

So the first thing to keep in mind is don't do it.

A

But you know we already talked about reasons you might have to and the things you need are you like? You need to know what contracts are, and you know what like what the contract is just the requirements on a function that you have to fulfill as the caller to use it correctly.

A

You need to know how to use invariants in the types that you create to make sure that what needs to be true for that unsafe code to work stays true, and you need right. You need this thing that comes from smoke. Functional programming, languages called vo like make invalid states unrepresentable.

A

These are like the common thread here is simply like. You've got to be able to reason about your code because you know you're playing in territory, but the compiler is not doing that for you now these four techniques I think contracts could could use a little more explanation. So, let's go into detail about that when I was getting started in rust, I like one of the things I couldn't I didn't, couldn't figure out immediately was whether or not a function.

A

I'm writing ought to be unsafe because, like here's, the here's, the push method, here's two versions of it. One of the right is the one I showed you earlier, the real one, but another way to write. That is the one on the Left, where you just like put the unsafe keyword at the top there and then the whole body of the function is covered, and this like this, the language is basically just letting me choose whether my functions safe or unsafe, right, which is kind of surprising for us.

A

It lets you just pick like I'm gonna be safe today, like and and the rest. The language then trusts you right. So this is. This is how this is. Why that matters right, because the function signatures are different right, the one on the Left, everybody who uses it would have to know. Whoa, that's unsafe, now, I need to decide like how you know am I. Do I really want to call this, and if so, I've got a stick of trust me block in my code.

A

So what's the right choice here and like? How much does it really matter so.

A

Again, when I started out, my thinking was well. If this code on the left is dangerous, we're calling this a dangerous pointer right, then, obviously calling this function is going to be dangerous. So the answer is you use the one on the left right, but of course the one on the right is the one that's actually in a standard library. So what gives now?

A

What I think is that, like the one on the right is kind of like a key thing and rust like rust, is a toolkit for making stuff safe right, rapping, carefully written unsafe code and a safe API? Is the underappreciated cornerstone of rust.

A

The contract of a function is simply the set of requirements you have to use in order to in order to use that function correctly and every function has this in every way. It's not just for us to write.

A

You call a function you're supposed to pass a certain number of arguments of particular types right and sometimes there's extra things that you have to keep in mind in order to use it correctly right, and if you don't do that, if you misuse the function, then what well and on a good day like you're, using push and you pass the wrong number of arguments right, the compiler stops you and in fact, that's enough for push the the set of rules that the compiler actually enforces, based on the types and the borrow checker and this the type signature to the function signatures.

A

That's enough, like that's. Those are the only requirements to use this function correctly, but then there are other functions like pointer right that are dangerous and they have a contract. That's pretty complicated. Actually, if you look at the at the documentation for this function, right sure enough, it's got five paragraphs of stuff that you need to read in order to operate this thing successfully right. The fine print of the contract and talks about things like the last paragraph is the pointer must be aligned which is like, like a thesis like you have to.

A

You, have to go and look up what that word means. Pointers are tough, but then, if you look at, if you look at vector push no safety rules in the contract right, there's this rule about panicking it like this is some bizarre corner case that happens when you have zero size types, but panic is safe. Panic is actually kind of a bad name for it. It should be called like orderly emergency procedure.

A

A

Yeah but panic is safe and it leaves the elisha program in a good state afterwards, even if one thread panics, others keep going and you can even catch it. If you really need to, but there's nothing you can do with this function. That's gonna cause undefined behavior. Therefore it should be safe.

A

What that means, though, is dangerous, safe.

A

It's on the implementer of this method, then, to make sure that they're using that dangerous piece in a safe way right and you can do it- what you need to do as the implementer of this method is you need to make sure that, like, when you get to the pointer right on this line, that the data you're passing to it is correct and that your color, no matter what they do, cannot cause you to violate the contract on that function right so this is. This is the basic idea right.

A

So how do you do it well, you're passing this pointer end. It comes from here, and that comes from this offset and that's computation right where you'd like to take the pointer to the beginning of the array, and then you add some number of elements and get to the end. That in turn is valid.

A

If the length argument you pass, it is actually less than the capacity of the buffer and there's an invariant in the vector class that the length is always less than the capacity or less than or equal to the capacity of the buffer less than or equal to equal. To would actually be a problem here, and therefore we have this other code to make sure that, if they're equal, then we double the capacity to make it to make room.

A

All the parts of this function are working together to make sure that one line of unsafe code goes smoothly.

A

That's basically what we, what I wanted to say there that so what have we learned? We've learned that, while you're in Columbus, you need to make a stop at Jennie's ice-creams get the Brambleberry crisps. We learned what can cause undefined behavior in C++. We learn what that means. We all know what safe means in rust and why we have unsafe code and we learned a little bit about how to use it wisely.

A

A

Well notice that in certain times and in certain places, engineers get this idea in their heads that they can like build a safer world. It's funny, it's happened in lots of different fields at different times. It's happened in you, know: medical equipment, airplanes, space, travel, industrial machines used to be incredibly dangerous highways, and when, when that happens, we do safety engineering, we study failures right like we get inside, then we figure out why we have these failures and we design systems to prevent them.

A

So where are we right now with software? Are we safe enough like, as is the code we write? Is that safe not as a good enough, and if not do we think we can fix it? But what do we think nothing will help.

A

I think we can do better right and if you go away with one thing out of this talk once you understand kind of at a technical level like how rust fits into that effort right sure, rust prevents some bugs out right, and that is great right, but is there's actually more to it than that rust will help. You understand the rules that good C++ programmers have in their heads that they got the hard way right.

A

Russ gives you the ability to put a safe API on top of unsafe code and have reason to believe that it's actually correct. That's a great like that's a software engineering tool right, a safety engineering tool. Russ has expressive types so like the types can cover more of the contract so that human brains don't have to, and maybe best of all like rust is economically viable. So you can actually choose it and use it and it can be your competitive advantage.

A

Unsafe code isn't going away all right if you were using unsafe code. If you have to write unsafe code for your job, you should be using rust.

A

It's ready today. It's the right tool for the job. Let's do it. Thank you.

A

I got one more thing, so when I finish in bladdy and I, we wrote a book, it's you can it's in early access right now you can buy it and it will be in print. Probably in January you can you can you can get. You can absolutely get a PDF with like almost the entire book in it, ah it's out it really handsome. So sorry, okay, oh you're, wrong.

A

A

I'm, sorry to hear it: okay, well, you can pre-order it okay! Well! Thank you anyway,.