Rust Programming Language RustConf 2019, 16 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustConf 2019 - Class Fixes; Or, You Become the Rust Compiler by Isobel Redelmeier

Description

RustConf 2019 - Class Fixes; Or, You Become the Rust Compiler by Isobel Redelmeier

What's a "class fix"? Well, a class break is a is a security term for a type of vulnerability that breaks not just a single instance (e.g., a single database), but instead a whole category of systems (e.g., all MySQL databases). Let's think of class fixes, then, as the happy opposite: one solution that addresses a whole slew of problems - much like Rust does by enforcing code correctness so strictly.

We'll go on a tour of the Rust features that provide us with class fixes, such as our dear, persistent friend, the borrow checker. But let's not stop there! Learning Rust can help train you to think more like the Rust compiler. We'll turn that into your new superpower, so that you can start applying the same rigor to even your non-Rust projects.

Expect to walk a way with a better toolkit for holistic debugging and system design!

A

I'm Isabella's mentioned really really excited to be here and talk to you all about class fixes or how you can become the rest, compiler yourself, so quick mini agenda, basically just intro and then I'm going to talk about class breaks and how we can invert that concept and use it to think about things in terms of class fixes and then how that can can help you to become the compiler either when you are writing rust or also when you are not writing rust and then maybe maybe a quick Q&A at the end.

A

You can definitely come find me afterwards. It's all stick around and I'll be here all day and so on. More than happy to talk about this, so Who am I um I work on open source, mostly open telemetry. At light step, these days, I used to work at cloud print on Cloud Foundry at pivotal, so I mostly do go by day. Also some Python Ruby JavaScript done a bunch of whatever is kind of necessary. A lot of bash turns out often is what's necessary, but by night rust is where I'm happiest also get to.

A

Maybe do it at work and hopefully more soon, but mostly go these days, I work or text because specs. So what are the goals of this talk? What do I hope that you can take away from it?

A

Basically, the hope is that you'll be able to use this as a framework for approaching error mitigation, so, in other words, how to not necessarily solve individual bugs, but instead how to prevent whole categories of bugs, and ideally so I thought I had another slide in there, but I forgot to put it into this version, ideally how to actually use rust and therefore leverage the compiler directly, but also how, when you're, not using rust, how to essentially think the same way that the compiler and therefore kind of conventionalized, some of the things that rust does for you, so class breaks start there.

A

Basically, this is a security term, um don't think it was originally coined by Bruce Schneier, but he's certainly how I first heard about it. So a class break is a particular security vulnerability that breaks not just one system but entire entire class of systems. He has a really good essay about this out there.

A

So an example would be locks. You know traditional locks, you have one lock. You need one key for that lock. Therefore, in order to break in she, let's say ten houses, you would need to break into ten walks separately, so that's essentially safe in that regard. However, there are your locks, however, if you instead have the same lock. For example, maybe you have a skeleton key or you know the TSA they had. They had this issue a few years ago, where they had a skeleton key for all luggage locks and someone.

A

Someone printed like the Washington, Post I think printed a copy of it online. Basically, so now, suddenly everyone had access to every luggage piece out there or similarly, smart locks also have the same potential vulnerability and was exploited a couple years or so ago, with a hotel where basically, someone went and like held all the hotel guests hostage. I think this was in Austria. So essentially, there are some nice genes that you get from having from essentially drying up your locks drying in the software sense, not in the meteorological sense.

A

However, there are also potential problems, and this isn't just true for things like locks, it's true for basically every piece of every dependency we.

A

Then every every machine using that same version of OS or another another version affected by the same vulnerability is affected. Those are thermometers for anyone who can't tell turns out it's really hard to draw a thermometers. I am NOT an artist, so essentially um you know hopefully you're using a common operating system. I mean I were a trust.

A

Conf I suspect that some of you here today are also working on your own and that you have very awesome reasons to do so, but in general, if you are running running most companies, you probably don't want to be running. I want to be writing your own OS from scratch.

A

For this just in order to skirt around class breaks, if you're you know, Google or whomever you might have your own Fork or you might go and write fuchsia or something um I suspect that there are some fuchsia people um here, especially since I think it's largely rust, but in general, if you are, for example, someone like me working at like 100 or so person startup that is not focused on delivering on OS.

A

That probably isn't something you should be focusing on, because you will instead have other problems if you try to write your own, so essentially a class break break once break everywhere. So what if we invert that? What if we instead have something that we fix once and there by fixed everywhere, can think of this as a class fix it's what I'm calling it?

A

So what would it mean to fix our code everywhere.

A

If we think about bugs and if we think about them kind of in just like a basic cs101 sense, at least then you essentially have kind of three main categories of bugs. You have logic errors, so things that those things where you as the programmer got something wrong.

A

So maybe you, for example, the other day couple co-workers were working on on an interesting algorithm and they were miss implementing it. The algorithm was theoretically correct, just the implementation, it was wrong.

A

Then you have runtime errors, which we most of us have probably encountered, hopefully not too much in production, but almost definitely in productions, and because these happen, most often in production, they're quite hard to tackle near compile time errors, and while these might be annoying when you're, actually writing code. They're also kind of awesome.

A

In another sense, in that, if you know at compile time that something doesn't work, then you can just kind of fix it before you deploy it or send it out to your customers or whatever, depending on your particular deployment, stress your delivery mechanism so compile time errors essentially are the safest of these three categories. In that regard, they're kind of better than the other two of types of errors, and that, as an engineer, it's a lot easier to to fix them.

A

So the more that we can move logic and runtime errors towards becoming compile time errors the safer our systems are going to be.

A

Essentially if we can downgrade a bugs category, ideally all the way to compile time, then it's I would argue a form of fixing it everywhere. So the rust compiler does this a lot for us and sorry for the large wall of text up there, basically Gary Bernhardt, he have watt fame, also the future of JavaScript or life and death of JavaScript. It's called had this quote the other day. Programming is rolling, boulders uphill forever. A type system tells you when boulders roll back down. If you look at that and say, I will not expend effort.

A

No, when boulders roll back down, you should at least know how much effort you're talking about versus how many boulders it stops. So he was talking mostly about type systems. In this context, I think it was a tweet thread about about typescript versus regular untyped JavaScript. However, you can also extrapolate and apply it to other aspects of a language um even just kind of from the ground up. You can think of how how basically any modern language that we're using improves a lot on for, on like assembly, for example.

A

Again some of us are probably using assembly. I personally am NOT or haven't since had to you for CS reasons, but day to day, I suspect some people are which.

A

Rest that is supposed to be a cape. It's okay, if you think of it as a chef's hat. Instead, I I won't I won't blame you, um but the point is Harris is awesome, so Russ type system gives us so so so many niceties it's awesome, I'm not going to go into all of them here, because I think most of us are probably pretty familiar with them and all of them also aren't the the improvements that Russ type system gives us compared to most other languages.

A

Type systems are awesome, but in some ways less relevant than other things. Also more languages are kind of moving in moving towards rest, like type systems, I would say so. For example, typescript has has pattern matching similar to Russ I, don't know exactly how similar I have really written much typescript, but it has some go, is getting generics.

A

It's still a bit unclear what they're generics are going to look like, but they'll have generics I, really like breast choice type system, but the general, the general factor of kind of a type system versus note type system. I think most of us are familiar with how how having your compiler be aware of your types and thus stop you from trying to like add an int to a string. For example, it's pretty nice, we we when we write on type languages, we can do that to some extent. We can.

A

You know, anticipate that we probably shouldn't be adding an int to a string. However, it just is nice to have to have the compiler do that for us, which gets back to what Gary Burton Hertz that here about how much effort you're talking about versus how many boulders have stops like you as a programmer, can stop some by just kind of thinking, knowing that, like you shouldn't, add an in to a string.

A

However, when your compiler does it you, you get more or less 100 percent again depends on whether you have like any types or things like that, but it removes you towards perfection.

A

So another big thing rust doesn't have reflection, which in some ways might make it harder than other languages getting back to the the analogy of how many boulders it stops versus how much effort it takes to UM to lever it to harness it in order to stop those boulders. However, what it gives us instead is pattern. Matching I know that they're not kind of a that. It's not a perfect flip.

A

However, in many ways it gets you what you might it gets you what you would use reflection for traditionally, so essentially, if you wrap a tricky enum types with their own types, you can capture a lot of what a lot of the use cases for reflection in the first place, an example: here's some go, it's very messy go, but basically there you have, you want to open it.

A

A database connection varies depending on what what database type you're using see how this open DB function that just takes a DD type, which is the post it's own type, but this code compiles despite the fact that it huh it lets you throw in random string. So here we have to to find to tie to defined DB types Postgres in my sequel, we're using my sequel correctly.

A

However, Postgres there's a mismatch between the strings and then this switch statement also lets us handle foo, which is not a database, and it lets us call a function with something that again is invalid. Literally the string invalid in this case so well, we have the type there too kind of hint at at ourselves and other engineers I personally like to think of kind of myself from yesterday as another engineer, because they have really bad short-term memory. Apparently so it lets.

A

You suggest that you should be using a database type, but it isn't really that good at enforcing that and a lot of other languages that have enums have kind of similar shortcomings in their types. Go also does have enums, but they're about as unsafe I would say as doing a type like this not unsafe in the rest sense just unsafe in the sense of like the compiler. It doesn't really let you do anything interesting with them.

A

If you compare this, though, to what you would write in crust here here, you just have a simple enum and then you have an open method which is aware of what type it should be handling and the rust compiler is able to tell us whether, if we are missing a type or not handling something, it also is able to ensure that we are handling everything that we should be and that we're not handling anything. That is invalid. So now we can just say Postgres open and carry on.

A

If we went back to go, we could kind of hack, it have something similar ish together. This is not the neatest go codes, alright, but basically for those unfamiliar with go capitalized things get our public uncapitalized. Things are private, so here this DB type interface is something that can only be implemented within the package, because it has this private method that it requires is DP type.

A

So now we have two type, two types that implement the interface post for us in my sequel, and they both are forms of this DB type, which again we can use anywhere. We can only, but we can only implement within the one package and then they also have their own open methods, and so now I can just say Postgres open kind of similar to what you would do in rust.

A

It just doesn't have quite as much of what the compiler gives us and, in my opinion at least this is a lot messier than what you have in rust, certainly rust fewer lines of code, and if you were missing it type, the rust compiler would complain unless you were handling it in some other way.

A

But again the compiler would ensure that another thing option instead of null so who here has ever encountered some sort of null pointer exceptions, yeah I think that's everyone's, so option pretty awesome. Some languages, like Java, for example, are moving in the direction of gaining it. um Personally, the last time that I wrote a like application in Java we attempted to use option.

A

However, like half the team wasn't really on board with it, so we still kind of, and then we were using a bunch of libraries that didn't that didn't support it and we ended up having nil checks all over the place, despite having options. I. Imagine that others have more success with it. Also that that another group have even less success than my team did.

A

So if we again go back to go here, we have this kind of plain bit of a and example here, where we have an interface and then another interface that contains that first, one so interface and interface container, really creatively named I'm I know and in the container one we call into the first ones method. So we say we have this wrapper dot internal dot foo.

A

Essentially, however, as you see at the bottom, basically, when we use when we use a valid interface for this things, work properly, because our struct implements few correctly, however, in go and in many other languages on nil, is a valid empty inter neil satisfies any interface, and so nil in this case is actually of type interface. It just doesn't actually implement foo and therefore, when we attempt to call to call foo on nil it compiles, but we get a panic.

A

So essentially, there are two other mechanism, two main mechanisms that we can use in languages that that don't support option. So one is when one is on get and that is to return basically a bool. This is assuming that your language has has like tuple types or multi return values, otherwise you're kind of stuck with the second option, and then the second option is to use the null object.

A

Pattern which basically is well I'll, show you in a second, and so these partly are dependent on what language you're using since again, not all languages support like tuples or multi return values, also partly dependent on on how you want your your client to consume it. So we look at examples of both here this one. This one is using the first one where Mensah don't get by by representing whether or not it existed with a pool. So essentially in here we check on the getter.

A

If the thing is nil and if it is your turn, basically a nil value as well as false and then when we actually use it, we say hi like that. We say if the thing actually exists then use it otherwise just carry on, and so in this case the program will print hello response and then just nothing else.

A

It doesn't even print like an empty line or something it just doesn't hit that line in more realistic scenarios, sometimes knowing that the thing is that the thing wasn't ever set, it's useful, that's why in rest, we up, we often like having having option rather than using the null the null object pattern, and then the other option is this null object pattern.

A

Where now we have a no abstract and on the sutter or in this case and the constructor we check to see if the thing would be nil and in that case fill basically back fill it in with a no op thing, so no op in the sense that, like it'll, comply with all the interfaces and so on without blowing up, but it doesn't do anything interesting.

A

So, in this case, for example, it prints an empty line on or it returns an empty string on foo go doesn't have a go, doesn't support like nil for for Strings, so empty string is all you guys, and so in some ways using this one is nicer since it's shorter. On the other hand, you don't know how to distinguish between empty string and unset.

A

So it really just depends on your use case as well again as your language, and essentially there again is where the nil, where the no op gets provided.

A

Then the Bor Oh tracker, so I know that there's a heart here and we all have probably struggled with the Baro checker at some point. But you know it's kind of our frenemy. It's there to help us at the end of the day, and even if it means that sometimes writing the code is is trickier, means that running. It is a lot easier, so here again don't mean to be picking you on go. It just was convenient since its language I personally write a fair amount, and it also is tight.

A

Here's a little program where basically half of the time it attempts to append something to an array in a what's called a go routine. So, basically, you know not a thread but like a green thread. Type thing, however, goes slices or rays are not tight, are not thread safe, and so this part, this part, will give you some funkiness know the go hat. The go.

A

Compiler has a runtime race detector, but it's only runtime so here like if you build, if you build the program with the race detector on it thinks that everything is hunky-dory, we got a nice exit zero. It thought the program was great when we actually attempt to run it with the race detector. Half the time we get this where it finds the data race the other time we don't have any complaints.

A

We just have our nice little empty array, also in here there's kind of an extra data race where, since it doesn't actually wait for the go routine to complete, even when it does I love, even when it does detect the data race, it also doesn't successfully append it time. Essentially, the main doesn't wait for that girl routine to complete.

A

Now compare this to rust, where the naive version would be. You have again a vac of events and you attempt to push that you attempt to push something in there into there in a thread and I know that that that's, maybe not the prettiest way of checking to see if the time is even but it was way, um could have also probably just thrown in a ranch too or something. But anyway this doesn't compile. We get nice set of errors. Things we've all probably seen before.

A

Basically, where the compiler helpfully tells us hey, there's a problem: here's how like here's, where it is, and here's how you might be able to fix it, and so essentially, if you only use type, safe, threads or methods, things become happier so now back to rust. If we use an arc and mutex and just was trying to omit the import statements so that things should be bigger font in here, then we then now this program compiles and is safe, there's no longer a data. Race goes sorry.

A

If that's hard to read up back basically I made a thread safe, inch slice again. They don't have generics right now, so bear with me, um but basically in here, whenever you're attempting to append it grabs a lock and when you are attempting to to read from it or just grab the string in this case the string of fight version, it again sets a read lock and now this is safe, its Cruiser, but it's safe.

A

So here we can see that, with the race detector on, we were able to successfully print both the empty and non-empty versions.

A

So in short, thinking of things in terms of class fixes can really help you, when you are actually writing rust, because the compiler does it for you, but even when you're writing other things other languages, you can still basically treat some of the rust compiler compiler niceties as conventions and attempt to kind of backfill your own patterns for runtime, and then you can go to your team and say: hey we're using all these patterns they're better than having runtime errors, but there's still conventions.

A

What if we had the compiler do it for us and then your team can become that much more powerful. Thank you.