Rust Programming Language RustFest Zürich 2017, 5 Nov 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustFest Zürich 2017 - A RustFest Carol by Felix Klock

Description

Is Rust doomed to stagnate under the yoke of stability guarantees? Is change an inevitable force that will invalidate all your existing code? Is there a reasonable medium between these extremes? We explore these questions, under the guise of a modern day fable.

Felix “Scrooge” Klock remembers the early days when Rust development was fast and loose, an untamed beast. As a result, today he fears changes to the language definition.

Gather round as this “Scrooge” is visited by a series of spirits (manifesting themselves as Rust crates), including:

- The Ghost of Rust Epochs Past (code circa the 1.0 release),
- The Ghost of Rust Epochs Present (code today), and
- The Ghost of Rust Epochs Yet To Come (code as envisaged by the host of RFC’s as we head into the 2017 impl Period).

About Felix Klock:
Felix Klock is a research engineer at Mozilla, where he works on the Rust compiler, runtime libraries, and language design. He previously worked on the ActionScript Virtual Machine for the Adobe Flash runtime. Felix obtained his PhD in Computer Science from Northeastern University, and is also one of the developers of the Larceny Scheme language runtime.

A

Okay, can you hear me yeah, okay, I, wanted to clear I did say: Couture's Julie, I first I didn't say that steel bestie day first, but uh he definitely did not know what got towards Julie a meant so yeah, it's great to be here. um I had a quick question, I kind of missed the earlier poll about how many people were at rust, fest, but it's okay. How many people saw my the rust fest Berlin talk? I gave okay. This is gonna, be a very different talk today.

A

Just so, if you should feel better I guess to you, if you sat through that one okay, um so I have a book. This is the rust fest Carol, it's a story by myself, Felix o'clock, otherwise notice, punk, pink, Felix, of course, I'm, making an allusion to Charles, Dickens and also you might see a little David, Foster Wallace or mark the loose Danielewski mark disease and Danielewski.

A

If you look closely, maybe- and of course this is a work of fiction, there can't possibly be anything related to anything, that's actually going on in the real world here. Okay, so our story opens on an IRC conversation where Manish says reimplemented entrusts, to which pink scrooge says bah humbug, and why is it? Why does he have this attitude about about this simple phrase we implement and rust? He says: well Russ this thing: it's just a language, we're writing types. You don't have to ever get actually get any code out of it.

A

You sit there, you wait, you become a compiler cycle older. You don't have any object file to show for it. If I could work, my will every idiot who goes around saying re-emit it and rust on their lips. They should be nailed up on their own crates and buried with a bicycle gear through their hearts, for which he was. He was promptly ejected from the chat room for such for such talk and Scrooge's.

A

Behavior wasn't just you know constrained to that. He was also kind of mean to his colleagues as well. He yells crickets cargo is broken to which Alex said wait. Are you talking to me I'm crichton? um So why is Scrooge acting this way? Well, the basic problem, as you can see from this IRC snippet, is that his problem is that he's really depressed because he says why are we implementing feature after feature forcing people to learn whole new ways to program?

A

They'll just leave the language, because we're always changing and they'll go to some stable, like Java or C++. They never change. Yes and, as we all know, change is death and once again, once again kicked by Manish.

A

Okay. So you know, Alex eventually went home and Scrooge pondered over the error he saw on his screen from the borrowed checker. He sees this error message Scrolls off the screen, so he has to you know, get a larger window to get some perspective on it, but lo and behold he sees the ghost of Nico. ah Oh it's, the ghost of Nico and Nico, says oh the chains of stability and backwards-compatibility. They are heavy. You must change your way of thinking tonight.

A

You Scrooge will be visited by three spirits: Nico's, not dead, at least he's not in the story or well he's not and definitely not dead in real life and he's not dead in the story or maybe he is, or maybe he's just overwhelmed just a notes. So at this point you might be wondering what is going on in this talk. I thought I came to a technical conference and the answer is well tonight. You are all going to be visited by three ghosts. We're gonna, you know, do it through the Scrooge Scrooge is being visited.

A

There's me three ghosts the ghosts of epochs past ethics past, which is actually going to be a summary of some of the changes going up to rust, 1.0 back two years ago and may 2015. The ghosts of epochs presidents are going to be talking about the changes. We're gonna, see the changes from 1.0 up until now and then will be visited by a third ghost, and that will be talking about the changes for future releases.

A

So the first two, the three spirits came upon Scrooge said: I am the ghost the deep locks passed and Scrooge said long pass. He said, no, no russets passed said. Oh because Russ is so young, it doesn't have a long past, hey and what business brought you here: spirits, Russ welfare. So in case you're not aware, we had the right up to 1.0 itself and this actually was marked by a number of intermediate releases, each one separated by six weeks. We were trying out this six week six week, rapid release cycle.

A

So we had the one point on one alpha. Then, six weeks later we had the 1.0 alpha two and then six weeks later we had the 1.0 beta yeah. So we sort of cheated there and didn't go from alpha to beta. You can, you know, talk to the various team members to found the story there, but we had these releases and the interesting question is at least for the purpose of this talk.

A

The interesting question is: what were the big changes that were happening in that time period because Scrooge's problem is, he doesn't believe change is possible and that you can keep your community together throughout change. So we had the 1.0 alpha release. We saw the in pocalypse. Indeed a scary thing. Scrooge says: oh I, remember those days. We've once had this int and you went types and it was a big problem because people didn't know which one to use they didn't know not that they don't know which to use between you in.

A

If anything that might've been the simplest question but know is: do I use I-32 or do I use int. They didn't realize that it might be target specific or is target specific. So you, you know, program using int and you end up with these portability hazards.

A

So there are many many proposals to rename these things starting I think with rust issue, nine, nine four zero and then there were some RFC's after that, but we resisted change the core team resisted change and we almost launched 1.0 using int and you ins and in some ways it was because of community backlash to this decision on part and the manner in which the core team went to a decision that we learned a lot about community inputs and about how to interface with the community, in particular the core team learned.

A

It can't just make it can't decide something based on an argument that they made in private and then deliver. That is if they were gods coming down from on high. They learned that no that's not a good way to engage with their community and in fact the right way to do it is to still interact with them. You come up with a new argument presented, don't present a decision present the new argument and then let the community and you figure out okay. Well, where does that leave us?

A

And by the end of all of this, we ended up with I sighs and you sighs, oh, but first we went through a bunch of different potential. Other options like I'm, mu, mem off set size in pu NP, but we said on the I sighs you sighs. Okay, six weeks later, six weeks later, we have rust 1.0 alpha 2, and for me the big change that we had it is added in. There was the drop check rules.

A

So before that point, we had we used to not allow you to write a destructor that looked like this. We didn't let you write a destructor that was parameterised over lifetimes or type parameters. At least we didn't, let you do it unless you added a little annotation saying. Basically, this might be unsafe, or rather this is an unsafe thing, because we don't know how to check it, and that was the status quo and rust for quite a long time.

A

But we managed to figure out something that we thought was sound for 1.1 1.0, alpha 2, so we added the drop check rules and that in fact, it changed things. In particular, it started it made code start getting rejected, sometimes was code that was worth rejecting. It actually had potential problems, but a lot of time was code that we just weren't smart enough to check properly, and so you know the Spirit says the screws. Look, you were willing.

A

Your one swing to accept breakage, you were responsible for this and Scrooge said: I was young, I was a young man then, and it's all different. Now now the true history of Job draw check is probably worth a 45-minute talk on its own, not gonna, make you suffer through that, but suffice it to say there were its evolved, a lot since then it's not like we had the right answer at that time. We didn't, we definitely didn't. It wasn't sound the way we did at 10 and we fixed a lot.

A

Ariel is here, and he can tell you a lot because he was responsible for a number of the fixes that we had to drop check.

A

Then we have 1.0 beta 6 weeks later Shh there catch panic was introduced, and this was an interesting thing because it wasn't really for this points to recover from the panic on the same thread that issued the panic, you had to have some sort of monitor thread to be the one to catch the panic or monitor the panic and say: oh this thread and somehow reports, the user that it died.

A

um So we ant, we landed catch panic and then there was a lots of controversy over the manner in which cash panic was added and about the details about what catch panic means and what it means for rust and what the name it, etc. There were lots of RFC's and in fact, we only stabilized it in the 1.9 release under a different name and with some other rules, so we're talking about my year before this thing was actually stabilized.

A

So this represents how change, in a way, that's usable because can sometimes also take time to finally evolve to reach the final thing and then from 1.0 that the change that I noticed there we made them forget a safe function. Up until that point, we had this attitude of forgetting to run your disruptors. That sounds scary and therefore we're gonna make this unsafe. You have to know what you're going to assert. You know what you're doing to do that to run runs to function.

A

The problem with that reasoning was that you didn't need to call them forget to avoid running instructor. We had there's a. There is a little trick you can do, involving involving basically reference counted cycles where you can put an object and forget it in entirely safe code, even without men forget so that line of argument was basically to say, look what's the point of making this unsafe? What's the meaning behind it, if I can already get the same effect via safe code, and so we shifted our way of thinking about what unsafe meant.

A

That was another change, so the spirit said to Scrooge, you see, Russ can change your thinking. Can change Scrooge says no? No, that was before 1.0 that wasn't that mad time when we were all rushing to bring the 1.0 release, this means nothing for now. So we're now visited by the second of three, the three spirits, the at box, the ghost of F Box presence and now Scrooge is sort of like saying. Okay, I've had some thinking about this whole. You know change thing and I'm, not so big on this being visited by spirits.

A

So all right tell me what you have to say. Maybe maybe you have a point teach me. So this spirit is all about. What's happened in Russia since 1.0 and in particular, I wanted for this part of the talk to give you a view that wasn't based on my experience, you may have noticed the ghost the outbox past that was sort of bias towards things. I really knew about deeply like drop check for this, though I decided I have to ask Lear people I have to get the input of other people.

A

You know, because we're gonna be like that spirit in the Christmas Carol, who visited the other families that knew Scrooge and what their lives were like at that time. So I did a poll and I asked what are the changes that matter to you since 1.0? It's a very small poll and there were some people. It said what mattered to me. The most was the IDE support. We had we've added the trust up tool and a lot of people said our bet are improved. Air diagnostics was a huge, huge win.

A

How many people are familiar with the change, their Diagnostics that have happened since 1.0? Okay, now of you, how many of you think it was a you know, they made things worse in your life. If you, okay, yeah, no hands yeah, everyone agrees. Everyone agrees. This is so much better. There are some features that have been added since pompano that are only available in the nightly channel of rust. You can't use them stable rust, for example.

A

There's this never type feature where, if you match on something and if you match on something in one of the variants, it can be proven via the type system to actually never have an instance, a so-called empty type, or you know like an enum with no variants in it is an example of a void type and the never type feature lets you do things where you say. Look if I have a result where the OK type is T and the error type is void where it cannot, it cannot be instantiated.

A

You can actually write a match arm that doesn't include the error case, because it's impossible that it could ever occur. There are some features that are only available in or s beta channel. We have our C of unsigned in the beta channel. In particular, we have our C of stirrer, which is kind of interesting and there's some features. Most people point out, features that are available in stable rusts.

A

So the most important features in terms of the rotor people responded about them are things that actually have made it to stable, like that was by far the longest list of things that I saw was things that are available for the first stable use. Things like if, let's the question mark syntax, that's been the replacement for trough, who remembers, try, bag and try exclamation point now who still uses, try exclamation points. Ok, a lot of people put your hands down, but I. You know, I respect the people who still have their hands off.

A

I am one of you. I still use tried because I have my finger's haven't quite remember: I, don't the muscle memory and my fingers yet things like pub crates for billing to expose visibility, just here crate and not the outside world things like infiltrates, but by far by far the thing that the most people respond saying. This is the change that mattered was macros 1.1 the ability to have stable, Dirac customized derive and in particular its use answered. So this part of the talk I decide there's so many people responded that way.

A

I have to show you some code I'm obligated to actually work through an example, so here's a hypothetical trait called weights. This is nothing so far to do with custom derive not exactly the weight. Trait is meant to be something that all it does is counts. The number of heap allocations that are owned by this type. This is not something that exists in rust today.

A

We're gonna try to implement ourselves in a very stupid way, not stupid, like inefficient, just stupid, like it's not going to be able to handle certain cases that one might care about. So by default, most values aren't heap allocated as and they're allocated in line and someone else's thing it might exist on the heap, but it itself doesn't own anything. That's heap allocated, for example, an integer doesn't own anything. It's heap allocated whether it's I 30 through are I 64.

A

Both those things have a weight of zero and likewise a reference has weight zero, because even though it might point to something on the heap, it doesn't own anything that's on the heap, it's just a reference to something that somebody else owns. So all these cases of integers and of references are all things that should return zero for their weights. The first interesting case are things like string and box. A string is heap allocated. So since some people allocated object, we're gonna have its has a weight of 1.

A

That's very simple: it's a single heap allocated a character buffer array. A box is more interesting. It has Eve allocation, but it also have owns contents that may have other heap allocations. So the way you implement weight for box is you. You have the heap allocation for the box itself. That's the one and you have to recursively, ask how much did my contents way and add them together and you can test this.

A

I have a this is all running code by the way, the way I write, my talks in case you're, not familiar, is I use. This tool called tango to like make Russ code. That is also a markdown file and then I can run all my code. So here we have this construct, that's a box of a box of a box and then the string hi, and this has a weight of three because there's three boxes in it.

A

So here's another demo of this weight function, so I can have it. I might have a struct called French toast it's way, it'll be zero because it has no it's just an empty struct. So what if I had a struct called pancakes that had two fields in it? What's the weight of that you just yell it out? If you know zero, yeah p1 got way to zero because it has no boxes inside of it. Okay, how about this thing here this this period, P 3, it's a pancakes with butter and berries and syrup.

A

So three things I mean it's got a weight of three: what's the weight of a P 3 here, that's right just count the boxes. Very simple, ok, great, except I, just lied to you, because this code doesn't actually run because why any any takers for why this code won't actually do anything. Yet that's right, weights of traits and I haven't implemented. Yet you have to actually implement the trait for every type that you want to use the time. So that also means, for example, that running weight on an option of box won't work.

A

You really want it to. You want to be able to do things like ask what the weight of a bot of some box of one is and get back one, and what the weight of none is getting zero. You want to do that and have it just be just count the boxes, but we don't have that yet we haven't implemented the shrinks. So, okay, let's start inputting the traits. Oh yeah, for this you for an option. You you match. You'd have a some case. You recur for the none case.

A

You have zero, ok, oh, we have to do it for result, I guess so. We've matched! You have the ok case. You recur error case recur! Look! So the problem here is that this is ridiculous. Those implementations there mechanically drivable, you can really you don't have to think to make them. In fact, you might say to yourself: why don't we make a program to make these for us, and in fact you can do that through the magic of procedural macros.

A

You can write a macro that takes in a type definition like this enum of option with the sum case in the none case and it'll generate this code, it will expand to this code down here, which is basically the same thing as what I just showed you modulo some small details in terms of you know why it has certain kinds of patterns being in there and certain kinds of additions, but it's basically the same thing: okay, so this was less readable, but you know the point is it's a macro, and this is a lot easier to write down and get right than all that code.

A

I showed you in the previous slide. Now you might ask. What does the macro implementation look like? How hard is that it's this I'm, a schemer, so I was a little horrified by this. This page, like I'm, used to macros. That are, you know, pretty compact, but in reality this is actually pretty tight code.

A

It's like a hundred lines of code news and it's pretty complex because it has to actually you know, D structure, the input and do handle the type parameters of the struct declaration and whether it's a structure- and you know- there's there's interesting stuff in here, but I'm- not gonna, show it to you.

A

Here's the sad news, procedural macros are not yet available in the rust, stable Channel. You can't actually use this in stable, rust. You have to use nightly, but there's one special case. The special case is derive, derive, is available in stable and it uses the same technology and the same isin I was usually, as that same crate that had that macro definition and reuse, basically the same functionality to make the derive so that now, when I make french toast I can say, derive weights and it'll create this implementation automatically, where it just says it's 0.

A

It says in a long winded way it matches stupidly and then produces 0, but you know we're trusting the compiler optimize all that away and likewise for pancakes it'll do the right thing. There it'll expand into something that matches in the pancakes binds the two parts and then recursively calls weight on the two parts.

A

So the lesson here, if there is a lesson, is that of these cases derive is actually the most important one to support. So that's why it was so important that it become part of stable rust, because the reason why it's important is because, while it is painful as a crate authoress, if I'm the crate author of the weight trait yeah, it's painful for me to go and write the implementations for option result and stuff. But at least I can do that.

A

At least as the author of the weight trait and the weight crate I have the option of going to the standard library and adding implementations for all the types of center library. I. Can't as the author of the weight crates do that for all of my clients, I can't write all their implementations. So the reason they derive is so important is because derive is what gives me the ability, as the weight crate author, a zero effort way for my clients to use my trait.

A

That's the crucial thing about it, and a particular instance of this is surd. So surd is this serialization deserialization library, and the crucial thing is that it's available in stable rust and it uses this drive mechanism to provide automatic, serialization and deserialization for a whole host of formats, including JSON, and pickle and lots of others, and the crucial thing is that it does this automatically it's very easy, as a user to get these automatic implementations and the code it creates is super efficient in that it's not using reflection or runtime type introspection.

A

The way that other languages might to accomplish this. Instead, all the code is generate a compile time and specialize to the type okay and there's other crates too. That use derive diesel is another one. Another example of note, so the insight, if there's any insights being taken from all this, is that a really relatively small language, ition procedural macros. It's not that small but derive it's like this sort of nice use case. You don't expect.

A

We don't expect many people to implement a macro, but we do expect a ton to use derive, and so these small changes to the language in terms of you know how many people expect to use them, can have huge changes in your library in your language, ecosystem of your community.

A

Okay, so Scrooge has been pretty convinced at this point. That change is possible maybe, but he still has the last the spirits so he's going to visit him and the ghost of f Fox. You have two commenters the IRC channel, but he says nothing and then he points any points again, Oh spirit. What is this you're showing me o spirits o.

A

Spirits why? Why is what is what could this be, and he points again o.

A

Spirit, no no I can change, I can change spirit and rust can change too. Okay, I personally, am NOT actually concerned about rust tying. This is all just you know, narrative cetera, and why am I not concern is because we have an awesome community of people that are super enthused to work on it and we have our friends the actual companies, like our sponsors that are using rust in practice and are going to keep it alive, regardless of what happens in the future, so want to make that clear.

A

So end of the story, Scrooge has come to accept change and he's a fundamentally different person than he was perhaps before this visits you might call him a Scrooge for 2018 and Aaron Turin enters the room, the IRC Channel and says what what is scrooge been going off about? We aren't gonna, do any breaking changes anyway. So what Aaron's talking about is this? We already have a number of things that enable change within rust.

A

We have the nightly stable splits, we have a rapid release cycle and we have support for deprecation, so we can evolve rust over time. This we've known this this that's why this second spirit was able to show us changes that have happened since 1.0, but but the thing that the mechanisms we have in place are insufficient.

A

They because in particular they don't allow us to make certain changes and also the mechanisms we have in place. Russ kind of evolves in this weird way where you know a change comes at a sort of random time and another one comes at a random time and the docs aren't necessarily like updated to. You know, incorporate these new idioms and there aren't blog posts announcing. Oh look all these awesome new things that are available. You see a blog post here, a blog post here about one bit clear change or another.

A

If you don't see, you know a really enthusiastic, here's why you should look at Russ now instead of before, and we want to change that, so the EPOC system has been accepted as RFC 205, and the idea of the epoch system is that we're gonna declare an epoch every two or three years and it's gonna eat up. Epoque is gonna.

A

I've heard this number different ways: I've learned that some people pronounce it epic, okay, okay, there's some of you raising your hands for that I always thought was like I, always thought it was epoch or F Hawk at Pakistan, almost always save, and now because of UNIX I say epoch anyway.

A

Each epoch provides a set of features that have been stabilized since the last epoch, and the crucial thing is that we want everything to sort of move in tandem. All the tooling the docs and our standard library should be updated to make use of the new features, and the way it works is that you have to a crate, has to opt into the given app Hawk. By saying upfront here is the epoch that I expect to be expect to work with the features that I want to use.

A

And if you don't say it, then you're assumed you're in the 2015 epoch and the other crucial pieces of this puzzle are that one. The russ compiler is going to support all of the epics going back to the original one in 2015.

A

We're gonna support linking crates that come from different epochs. So you can take a crate that has not been updated since 2015 and had it interoperate with your crate from 2018, and then your crate that incorporates the one from 2015 and your crate for 2018 will be able to work with somebody else's crate from 2024, and they should all work together.

A

We ought to be able to link code together and preserve the semantics of the code that was old and finally, in terms of evolution. An epoch can only introduce a heart error. If we actually warned you about it, we can only issue and there's a hard error for a certain form. If the previous epoch was telling you via deprecation, warning, hey hey, this is gonna break, so this adds up to an assertion that the epochs do not split our ecosystem, nor do they break existing code. That is the claim that we are making.

A

So we get to consider new changes. Changes like the catch construct to go with the question. Mark changes like changing our module hierarchy system to be inferred instead of be explicit. These are things were considering nothing set in stone, deine trait, instead of saying. Instead of saying bear traits, so you might say, ampersand dine trait instead of ampersand traits and changes to our visibility system, we might change things so that it's a you say, crate, F, n, instead of saying pub, crate FN will see.

A

The crucial thing to take away from these bullets is that, for the most part, the epic system will enable syntactic changes. Those are the things we mostly be focusing on. She just saw like you know the way things look now. Syntax changes can enable semantic changes, but keep in mind because you have to link old code and new code. You I don't expect to see massive revisions to semantics, because in the end we have to be able to things together in a sane way.

A

So none of these changes, though they're talking about it, can happen without our community, and so it's very important that everyone, you know get involved and and hack on rusts use rust, but also give back and work on, helped contribute to the standard library and contribute new crates to the ecosystem and, in particular, I want to advertise the info period. That's going on for the remainder of 2017, we're encouraging everyone at every level of knowledge to contribute to rust itself, to making rush better during the Imperial I getting mentorship.

A

So you can read the blog post, the info future for rust, blog post on the rust blog that talks about this. Basically there's something: there is something for you. If you want to come help, there is something for you and we will help you find it so final message on change, so don't be like the Scrooge who's that buck was 2015, be like the Scrooge from 2018, because, yes, change is hard, but it can be hard fun, let's get out and get in playing all right. Thank you.