Rust Programming Language RustConf 2017, 1 Sep 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustConf 2017 - Building Rocket by Sergio Benitez

Description

Building Rocket by Sergio Benitez

Rocket is a web framework for Rust that delivers simplicity, reliability, and performance -- a rare combination. Rocket's ability to deliver in all three areas is largely enabled by Rust's code generation facilities. Based on developer input, Rocket generates code that eliminates boilerplate and leads to more robust, secure, and predictable web applications.

This talk dissects Rocket and its code generation by describing how Rocket is built. Special attention is placed on elucidating Rocket's use of present and future code generation facilities in Rust.

A

Awesome well, this is Rockets logo. This is my name. That's my email address. You'll, see it again at the end of the talk- and this is the second rocket logo. So today we're talking about well rockin a bit, but mostly I want to talk about code generation, which is obviously low. If you know about a rocket, it's obviously what sort of makes rocket rocket I think cogeneration is a somewhat underappreciated and sometimes way over appreciated feature of rust, but I.

A

Think if you use it well, you get really nice api's that actually help you write better code or help. You write better software and I like to think rocket is an instantiation of that. If we can use code generation well, then we can get good applications, good software, so the way I want to structure. This talk is sort of in three three stages.

A

First, I want to introduce you to to rock it. If you don't know about rocket it's a what I think is a simple, fast and tight site type safe web framework for rust and I'll, say a bit about what I mean by simple and type safe. Fast. Is self descriptive and it's powered by rust cogeneration without Russ cogeneration we wouldn't have rocket, and the idea is that rocket lets. You write, secure and robust web applications in a way that nothing else really lets you do the closest that other web primers get is.

A

Are those in languages like Haskell using something like type Haskell or some point yeah.

A

Whatever templates, yes, template, Haskell! Thank you. That's the right word template Haskell great, then don't want to talk about cogeneration. So after you get to know a little about a little bit about how rocket looks how rocket code looks and what rocket does I want to talk about cogeneration in rocket and in rust, so when rocket was launched, there was this pervasive comment about how rocket was so magical? How rocket was like rails and it's magic in like avoid it like the plague, because it's too much magic and I want to demystify that I.

A

Think cogeneration isn't magic, I, think monkey hatching and Ruby mix-ins are magic, but not cogeneration and doing so I want to show you very briefly what the cogeneration api's look like and when I say that I mean the procedure of macro api's, which are very similar to what you can do now is derive if you've used that and then I want to talk about a bit a bit about the feature so what's coming in future versions of rocket and how rocket will continue to use cogeneration, to give you a nice interface and in really strong promises.

A

So that's that's sort of the the the path we're taking for the talk today, quick introduction on rocket spend about 10 15 minutes on that. If you already know rocket, feel free to doze off for 15 minutes, then we'll talk about cogeneration and rocket. Let you know how rocky needs cogeneration, what rocket actually does what Kodi generates and how that works and we'll talk a bit about the future of rocket? Ok. So, let's start with a finished introduction to rocket so again rocket is a web framework for rust. That ideally makes it simple.

A

Ie, like it's very easy, to read very easy to write to write fast web applications. What you would expect from a program written in we're in rust, without sacrificing flexibility, so you can do what you want to do, not what the framework wants you to do, or type safety and I've underlined all of these things, but I really want to underline type safety.

A

It's it's interesting that if you write a program in rust, it doesn't mean your program actually takes advantage of type safety ie you can dynamically cast and rust you're not aware- and that's like a really nice escape hatch if you're coming from the web, because the web is the string you're trying to take this string and make sense of it in rust.

A

But if that's what we do, if we really do just dynamically cast things, then we're not really using the power of rust the power of the static, strict, strong type system that russ gives us and rocket aims to exploit that type system. So that's what I mean by type safe there. So how did we get to rocket? So this is the timeline and every time I do a talk and rocket.

A

This timeline gets, you know wider, which is nice so rocket, launched about seven eight months ago, eight months ago or so, and shortly thereafter there was a virgin 0.2 about a brought a bunch of nice stuff to rocket and about five months after that there was a release of version 0.3 so just last month and now we're at the end there, it's gamed awesome traction. There are at least thousands of users and a couple dozen companies using it, which is pretty awesome.

A

Something that's not on this timeline. That totally surprised me was was this blip here was how quickly someone told me: hey, I, just launched something in production and my company is now using rocket. It was like two weeks after rocket launched. I got an IRC message, saying I rewrote this thing in rocket thanks, so much I can throw away note. I was like alright cool. That's great thanks for using version. 0.1 I appreciate that.

A

So that was quick. You know now, when I get those kinds of messages, I'm more excited than scared or nervous. I think, but it was surprising how quickly that happened just as a bit of fanfare here when rocket launched it, you know, I got really nice coverage. Hacker news loves rust, sometimes, but they generally liked rocket so much so that rocket was the top trending project when we launched, which I thought was awesome, so I got over a thousand stars in the first day, I'm github, so that was fantastic.

A

Ok, so this is the timeline now obviously I'm missing a big component of this timeline, which is what happened before you know the launch and well obviously, I worked on it. That's what happened before the launch, so I've been working. I've been working on rocket for quite some time about you know eight nine months before rocket actually launched, and mostly a full-time effort kind of have to step back from rust for a few months in the middle, mostly full-time effort and I just want to give a very tiny blur above about why rocket exists.

A

It wasn't just to create a new web framework because there are so many of those in so many different languages. You know it's like stop! No more web frameworks, you know, I believe that, so why did I make rocket well yeah I'm working the Rebbe for a very long time. Writing web applications for a very long time and I've been writing rust for less time. Obviously, he says existed that long, but almost five years now, like four years so way before at one point out and at some point I wanted to marry the two ideas.

A

I was like yeah. Let me write a web application and rust. Let's see how this works and I tried doing it, and it was not fun when I programmer I like to have fun otherwise I just thought programming and it was not fun and it was not taking advantage of the type system and for those two reasons rocket came into existence. It was really an idea of how do we actually make this super easy.

A

Even though Russ type system is not always easiest thing to use and at the same time, how do we take advantage of that type system, even though we're gonna make it easy to do this? Okay, so rocket web framing for us makes it simple to write fast web applications without sacrificing flexibility or type safety great. So, let's take a look at what a rocket application looks like this is hello world. It's four lines. This is really all you write well caveat. You need a main function and yeah.

A

So this is what you write and, first of all, I just I just think. It's so cool that you can write this, and this is rust. You know like this is like a sea successor and you're. Writing this I just think it's awesome. So what what? What is this? What are you looking at? Well, first thing up top here is an attribute, if, obviously, you guys are familiar with rust, so it's an attribute and then rush in rocket. We call it the route attribute. It's a description of matching conditions. It tells rocket.

A

These things have to be true for a web request to be routed to this route for rocket to give a web request to this route. These conditions have to be true, so in this case you need a HTTP request with the get method to the route path.

A

The thing at the bottom is the route handler: it's just the function, it's not a special function, it's just the function and it will be called if the route matches and raw and rocket will call it and it'll produce a response which is an arbitrary type choose the type as long as it implements a certain type class, you can return it and that's it. That's really.

A

What makes up rocket applications are these routes and if you were to actually run this thing- and you were to go to the root path- well, you'd see hello, as you would expect. That's that's really all it takes to write a HelloWorld application in in rocket now, like I said, you do need a main function. So this is not the 100% complete application and the main function looks like this. So this is the main function and there are a couple of things that are happening here. The first thing you see is this rocket ignite.

A

So obviously, I need to turn on the rocket if you're going to launch this thing, so you ignite the rocket and then you need to put some stuff on the rocket in this case when you put routes on the rocket because those belong in space. Apparently so you mount some stuff on the rocket and what mounting means is you just namespace the route so, for instance, here we're not really Inc spacing anything so we're just saying mounted to the route path.

A

But if I were to change that to /hello, for instance, then every route in sort of the list of routes would have to be prepended with slash hello in order for rocket to route things to it and then, finally, you launch the rocket, and that starts up the server and it also prints a bunch of emojis, which is a contentious point.

A

It was like ii github issue after a rocket launched. Emojis stop, please they don't want that. But that looks like this. You get three emojis, you know not so bad when you launch it, you get this thing rocky tells you rocket tries to be very, very helpful. While you develop the application, it tells you everything it knows about the environment. You know where you're, where it's actually serving this thing, the configuration of your application.

A

It tells you what routes it knows about, and finally it tells you where your application is actually being served, and you know emojis no more emojis after this just three emojis, but still a lot of backlash.

A

So be careful if you choose to put emojis in your applications, you also can't get rid of them, which is great. You can try. They'll come back alright mounting and launching great. So that's that's what our full application looks like pretty simple this. This is really all you're. Right, obviously need to like extra and crate rocket and things of that nature, but this is really the entire application.

A

A

This is fine right. This is like cool. We can write a hello world, but we can't really do anything particularly interesting. Yet from what I've shown you, we can change the path. So we can change, get /, for instance, get slash world and now we'd have to go to localhost. Localhost 8080 get to you, know, slash Sergio and then we'd have to go to localhost 8080.

A

What immediately we can do more and of course you can do more, and so I want to talk to you about dynamic pads. So don't worry pads, as the name implies, allows you to have path segments that are dynamic, ie, the user tells you what the value for that pad segment is, and so here is our. We worked hello world example, and you can see that we have parameters in brackets in the route attribute, so we have name in brackets and that makes something dynamic.

A

So whatever the user types in to the path at that segment will be the value that the name parameter gets, and so again these are in brackets. That's how you tell rocket hey. This is dynamic. I, don't mean the literal, slash name, I mean the dynamic name, and you can do more than these. You can do more than that.

A

You can do slash things, that's age, for instance, and now you have two dynamic parameters and the the names the parameters have to match the names of the function parameters, if not you'll get a very nice compile time error that tells you hey these don't match. You know, look here! Look here not the same. Please figure it out.

A

What's going on and any type that implements any type that implements this trait called from Purim is allowed, so string implements from pram that rocket influence from parameter for uu8 also points from per am, and so you can use these things, and the semantics of this are that rocket will call from per M for every parameter you list and if the conversion succeeds, I affirm program says yep. This is a valid parameter of this type. Then it'll call your handler, otherwise it can't call your handler, and so it does it.

A

So the implication of this is this thing is rather simple, but the implication of this is actually deeper. For instance, if we want to get a path from a user ie, we're gonna have a static file server, so we want a full path. You know one very common attack when people try to write static file servers is to allow any file to be read, as opposed to just the files in one particular directory, and there was like there's been like a bunch of people on reddit.

A

Slash trusts are like look, I wrote a static file, server and hyper, and you know the first comment is like: oh I can read your password, it's great, and so, and so even here, even something as simple was for parameters we can rock it protects you here. So, for instance, this is how you write a file server in rocket and it's totally safe, and so what happens here what's happening here is well we've added these dot.

A

Dots I'm not gonna, explain them, but all it says is that your matching, not only this segment but every segment they're after then we're capturing that in a path parameter, which is a path buff and the front prime notation for path. Buff will actually verify the path and ensure that you are not la. You cannot have a directory, traversal attack, and so this is exactly what this is. In fact, what we put in the guide about how you write a static file, server in rocket and it's safe and you're protected alright.

A

So the last thing you want to talk about before we talk about how this actually working is what I believe the most exciting part about rocket and that's this concept of request guards so for what we've seen so far rocket helps you with input validation, but there's there's more to it than just the path. There's arbitrary data in the request- and you want to make sure that this request contains some information. For instance, I, want to check that it identifies an admin user. Well, there's nothing to match in the path. For that, there's really nothing.

A

It's just arbitrary. How what an admin user is? Maybe it means looking at the cookie? Maybe it means looking at a header who knows so Rockets mechanism to be able to to describe these kinds of validations is or our request guards so any route. You can list any number of parameters in the handler that implement the trait from request. So here we have one and it's admin user.

A

So we have some type admin user and what the admin user type does it implements this from request straight and the trait will verify that the request actually contains an admin, user and it'll convert whatever needs to to give you a type that says: yep, there's an admin user here and if they can't do that.

A

Well, it has one of two options: it can either say it can either fail, which says: no do not call any handler that needs an admin user or it can forward which says: okay, look, this route, don't call it I, it's not an end user. Maybe someone else can validate the request, maybe try a different kind of user or whatever just try something else rocket and when you forward something rocket, will try the next route that the nomenclature, or whatever we say, is to collide.

A

So, for example, here we have two routes and they are both requests there, but they're both match the admin route, the admin path for get requests. So these these collide and if you were to actually run this rocket would say, hey these collide, but we can fix that collision by ranking these routes.

A

So we can add, this rank equals two to our routes, and that means hey when there's a collision, try the thing with the lowest rank and then try the thing with the next highest rank and so on and so forth and, like I said, if you do this wrong rocket, will give you an ideally a very helpful error here. It is that says: rocket failed to launch because there are collisions these two things collide. You know, please fix them. Oh by the way, did you know about ranking that sometimes fixes this okay?

A

So you have this. So what's gonna happen here? Well, if you were to go to get slash, admin then rocket will try the lowest ranked route first. So that's the one with the admin user, which means you need an admin user to be validated to be authenticated by the request. And then, if that fails rocket, will try the next ranked route, which is the one below that which says you just need a regular user, and so what we've done here is actually implemented, both authentication and authorization.

A

So we've actually encoded our policy with routes, requests cards and forwarding mechanism, and we can go further and keep ranking things so, for instance, if you want to have a router that has no requirements, maybe to tell the user hey login, so I can figure out if you're, actually a nabbed and user or regular user. Then we can do that. So I think this is the most exciting part about rocket. Is the ability to really encode arbitrary octet, occation authorization or whatever request cards and forwarding? Allow you to do that?

A

Okay, so that's really all I want to say about rocket. There is a lot more and so I mean you can do pretty much whatever you want. So there's a lot that I'm not going over I will link you to rocket website which has a guide. It's pretty extensive. The documentation is very it's fairly extensive. It talks about much much more that I just don't have time to cover okay, so next up is code generation in rocket and rust. So let's go back to our hello world example. Here it is.

A

We have our route or hi routes for the route path, and then we, however, have our main function now I've reached out it a bit because I need to fit code in the slide. So what does rocket what code does rocket generate when you write this program? Well generates this code gray yay? What is this? Well, let's watch, let's walk through it. So the first thing that happens is rocket, takes your route attribute and the signature of the function and generates this route info structure, and really you can sort of just map it directly.

A

You can see exactly how the mapping works it takes. The name makes it a string. Takes the method makes it an actual method type takes the path, puts it in a string.

A

This handler we'll talk about in a second and form at a rank, or are there aspects of rocket that we don't have time to talk about the nice thing it does? Is it generates this this function at the body here? So that's this high route? That's what that handler is, and what this is. It's a it's a montemor physician of of every route, so it is taking any possible signature that you can write and converting it into one signature, because at the end of the day we are statically typed.

A

We can't take a bunch of different signatures without doing dynamic dispatch, so we create one signature that effectively does dynamic dispatch at at compile time and all it does is it will call that function. So what you actually wrote, it'll get that response and it will generate an outcome from that response and the outcome is just what happens when you actually try to write that out on the network and, finally, to tie things back together.

A

We have that routes, bang macro in the mount in the mount, and all we do here is we construct a route from the static information that we generated so that routes, bang becomes a Beck and the vector contains all the routes. In that you that you wrote in the routes and rocket will do all the name matching necessary to actually make that happen. So that is the code that rocket generates actually pretty straightforward and the most interesting or the key here is this monomorphic function here?

A

It's that we can take any signature and convert it into a function that has a static type that has one type okay. So this is a pretty simple example, but what about when we have guards and parameters what code gets generated here? Well, the code for the route info is pretty much the same. We just kind of read everything: parse everything and birth, that into a route info structure and then rocket has all this information to be able to convert it into something that we can use at runtime for routing.

A

But the code for the for the guard and the parameters is, is more involved and I could not fit the code that rocket generates so I, you know took the code that rocket generates and like rewrote it into what you might actually write by hand, but even that won't type check, so I've actually had to fuzz the types to fit this into one slide, but note that it is, it is pure to what I actually was actually rocket is actually generating. So what code is rocket generate?

A

We're gonna move this function up here, so we have room for the code, and this is what rocket generates again. It's a it's a function that goes from request to outcome just as before. It's always gonna be that and what it does inside is sort of what you'd expect so for every parameter. It generates a little bit of code, so, for instance, for this user parameter.

A

It generates the code that calls from request for a user and it checks the outcome of that of that call, and it does what you might expect if, if it's successful it gives you that value. If it's a forward, it just forwards the entire request, and if it's a failure it it fails with that status code. Well, what does it do for ID, something very, very similar?

A

It calls the from per am implementing, and if that succeeds, then you get a value, and if that does not succeed, then you forward the request to says: hey, try something else, and this really is the code that rocket will generate module and making it very pretty to fit in the slide alright. So in summary, this is all the code generation that rocket. Does it takes that route attribute and generates a route info structure, a static route, information structure?

A

It takes your route handler and it montemor faces it into something that goes from request to outcome, and then it takes your routes, bang and converts it into a vector that containing routes that match your static route information. That's that's it! If you tell me this is magic, then I don't know what magic is.

A

This is super simple if you, if you just sat and thought about it, this is probably what you do this is. This is not something super complicated. You can cargo expand. You know before you compile, you can do rocket. Poage ended bug equals one it'll show you exactly the code that rocket is emitting and there's no magic. It's just doing things that you would do otherwise in your handler for you, so you don't have to so you don't get it wrong, so you do the right thing, every single time. Okay, how does this work?

A

How does rocket make this cogeneration happen? So let's talk a bit about the internals, so the procedural macro API, it's actually pretty simple rust will call you and it will give you the syntax that say the attribute was applied to or the syntax inside, of a macro of a macro call, and then your job is to take that syntax and emit other syntax that it should replace whatever is in whatever syntax was called with wit. So you take some syntax return.

A

Some syntax, that's it pretty really rather simple the types of this syntax thing and I've simplified it, but this is effectively what's going on syntax. It's really just a vector of these token tree things and a token tree contains a node so like, for instance, a semicolon or a plus or a string. Literal is up like that, and a span which says here is where, in the code this token occurred and the span is useful for error, reporting and things of that nature.

A

The span is not something that you can actually get with derive, for instance. So if you were to derive the span information, it's just gone, you panic and you can't tell the user. Why you're panicking, just you know, try to explain what's going on, but spans are extremely important and they will be in the proc macros 2.0 API. So this is sort of what it looks like now and what it will look like in the future except much much much much simpler. So this is the feature the present is money.

A

The feature is much nicer all right, so that is code generation and rocket and rust. So, what's next, what's coming up well, we're gonna keep using cogeneration. Is you might imagine and we're gonna do type you our eyes. The next thing, the next for the next release of rocket I, really want to have typed you our eyes, and the idea is, if you have a route like this, that takes in a a parameter of ID that you can then create a URL to that route, using the actual type that you expect.

A

So, for instance, I can call URL bang for the retrieved route and specify that the ID is a pasted idea of 100 or will actually call you know into for you automatically. So you can just give anything that implements into for your type. In this case, seeing you size or whatever you 32 implemented into for paste ID, then we could just write the thing at the bottom, and the nice thing, of course, is that if you get this wrong, you'll get an error that says nope.

A

Those are not the right types, so you might imagine changing URLs and then everywhere you try to generate a URL for that new route. It'll fail at compile time, which is a really nice profit to have in this case. We'll just get you know, slash 100, so it's just replacing that ID with you know whatever you pass in and getting slash, 100 pretty straight board.

A

The next thing we're gonna do is database support so to use databases with rocket. Now it's it's a bit cumbersome. This is what you write if you were to do it. This is what you would have to write. You can do it, but it's not super elegant. It's gonna get much much simpler. So what I want this to look like in version? 0.4 is like this.

A

So in your rocket gamma file, you can specify I have database name to my DB and the adapter is Diesel's s, sequel, Lite and then in your your actual rocket file in your code, you would just derive some traits and tell rocket oh the database that I want to access what this type is my DB, and then you can just use that type as a request guard in your code, and so it really is, you know, taking something very big to something very small, to make things really nice and easy to use again.

A

We really want to get rid of this boilerplate, but we still want to keep type taking. So that's all I have to say what's coming next at this point, you've heard all about rocket. You've have some idea about how cogeneration works in rocket and I could end the talk here, but I thought I'd have some fun at the end of this and and trying to dispel some myths about rocket.

A

It's very interesting. What happens when you release something, and then you let other people tell you about the thing you just released. I.

A

Feel like the web is like anything that has to do with the web is particularly sensitive to this topic. People have very strong opinions about the web, and then we get JavaScript so I, don't know how this works so.

A

The most or yeah this is an opinion. It's not really a myth, but I think it's a myth and that's that there's way too much magic in rocket. You know, I, don't know. What's going on, hopefully now you're convinced that that's just not true what rocket is doing with cogeneration is simple. It's what you would write except it's writing it for you, so why? Why do something the computer can do for you when you gonna do the same thing over and over and over again.

A

Hopefully those of you who thought there was too much magic you're, not convinced that there is in fact not too much magic, there's just cogeneration.

A

The next sort of myth is that rocket is somehow unstable and if you've used rocket, you might think. Well, of course, rocket is unstable. It's using nightly, and if you look at the list of features that rocket uses, there's a lot of rocket uses. A lot of nightly features want me to tell you that that's true, but rocket is not unstable. Rocket is stable in December sense.

A

If you use rocket 0.3 Rocky 0.3 0.1 will not break your application, not sure where this why people started saying this, but just because rocket uses lightly doesn't mean that you're, a rocket application itself is unstable. It means that we have to keep track of nightly. We have to keep track of nightly and we do keep track of nightly there. At most in the history of rock, there has been, there have been 24 hours between a new nightly breaking something in rocket and a new release of rocket fixing itself for that nightly.

A

So at worst, if you'd, like update your rot year, rust insulation every single day, you wait 24 hours and there's a brand new release of rocket works on on the latest nightly, or you can just stick on the latest nightly and it'll. Just keep working so rocket. A rocket applications are not unstable and, finally, on the topic of nightly is that rocket is forever on nightly. I should not try using rocket because my company yeah we use stable software every day. That's all we use in our company stable software.

A

We don't use npm 5.0, we use. You know MP m 4.0 because that's that's stable, I think.

A

No I know, but well we can talk about that one later alright, so you know if I want to use rocket that I'm committing to to nightly- and you know nightly- does make things like deploying more difficult and things of that nature so is rocket for ever on nightly well rocket ever work on stable. Well, yes, of course, rocket will eventually work on stable, every single feature that rocket uses is being worked on right now for stabilization, every single one of them. Some of them are actually not needed.

A

It's just because we're using unstable stuff, we actually need to use more unstable stuff, but everything that rocket is using is being worked on right now. The procedure of macros are being worked on right now. You know everything the rocket uses a specialization everything the rocket use is being worked on right now. When is it gonna happen? I, don't know that depends on these guys over here.

A

Some things are hard like specialization is hard. It's it's not an easy problem, so it will happen. Maybe you know this 2019 Russ thing, maybe that'll get it I'm hopeful it'll happen much much much much sooner than that like within a year. It seems like there's enough progress to get there, but we'll see we'll see where that goes, but it is not forever and nightly. It will work on stable and I am committed to having it work on stable as soon as soon as possible, including writing. You know, code and rust to make rocket happen.

A

I had to write a bunch of code in the Russ compiler and I'm committed to do that again to make it work on stable as soon as possible. So that's rocket. That's all I have to say about rocketing cogeneration. Take a look at cogeneration. If you're trying to do something the computer can do for you in a really nice way. You know take a look at cogeneration.

A

The api's would get much better, they're, already better and nightly, and I think cogeneration can really improve the simplicity of api's and their robustness and applications using libraries that use cogeneration. Here's the website there is a guide, there's a really long guide. There are tutorials, there's a bunch of Russ stocks, there's news and you can there's a link to the github repository I'm Sergio, here's, my email and thank you for your time.