Rust Programming Language RustFest Barcelona 2019, 28 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: RustFest Barcelona - Tyler Mandry: Life of an async fn

Description

async/await is one of the headline features to ship in Rust this year. But how does it actually work, and what does that mean for performance?

We’ll walk through what happens to async/await code inside the compiler. We’ll see some of the performance implications that result from this, and how to avoid common pitfalls. You’ll walk away with a deeper understanding of how your async/await code behaves at runtime and what the compiler is doing under the hood.

Finally, we’ll explore some of the cutting-edge use cases for async/await and coroutines beyond web servers.

https://barcelona.rustfest.eu/sessions/life-of-an-async-fn

A

Tyler: well, in case you haven't been paying any attention,. Async/Await is now stable in.

A

A

Language and I think, systems development in.

A

General, so very exciting.

A

again,, my name is tyler, or tmandry..

A

I work on fuchsia,, which is an experimental operating system at google.

A

We use rust for a number of things in the operating system.

A

I work on the toolchain team supporting rust,, which means that sometimes I get to spend.

A

Time fiddling around in the compiler and I'm also a rust release, team member and compiler.

A

Team contributor.

A

So I just want to briefly give kind of a primer on futures before I dive into some of the.

A

Async stuff. florian covered some of this,, so just to recap, a future is any computation that progresses.

A

Asynchronously,, so that could be like a request to some external database server,. It could.

A

Be like reading and writing on disk, or doing some expensive data processing on another.

A

Thread, something like that.: all of these things can be represented with a future..

A

It eventually produces a result which is called its output and we represent futures in rust.

A

With the future trait,, so you can see there is an associated type called output, which.

A

Is the output of your computation, and then there's this method called poll which has.

A

Kind of the scary signature, but the job of the poll method is to make progress on the.

A

Future, as much progress as possible.

A

Poll returns, an enum called poll, and the result of the poll is either pending, which means.

A

That I made as much progress as I can and now I'm waiting for some external event, or.

A

Ready,, which means that I'm completely done- and here is the result of my computation.

A

An important thing to remember is that in rust we have a lazy futures, model,, so futures.

A

Don't do anything until they are told and it's the job of the executor to do. That.

A

Okay,, so I want to talk about async functions in this talk and I will start with a really.

A

Simple example of an async function,, so here is an async function named, handle request.

A

We are going to take some arguments.

A

The first thing we are going to do is to issue a request to an external database server,.

A

To get a row from our database corresponding to the id, we were handed.

A

We are going to await the result of that,, so basically suspend execution of this function.

A

Until we get the row back.

A

once we get it back, we are going to encode it into json and then finally, we are going.

A

To write the encoded json out onto a tcp stream.

A

Now,, if you are coding this synchronously, your function would block your thread at two.

A

Potential points. one is basically when you are awaiting the database server and the other is when you.

A

Are writing out to a tcp stream, because tcp sockets many times have a buffer that can.

A

Fill up so you will block your entire thread and at minimum you will pay a contact, switch.

A

Or your thread could be locked for a really long, time.

A

The way this connects to future,, as was mentioned earlier,, is we re-write the signature in the.

A

Compiler of your async function., it could just be a normal function that returns impl future, and that means that we are returning.

A

Some type,, we are not going to tell you what type it is, that happens to implement the.

A

Future crate., in this case, our async function, did not have a recurrent type, so the output is just the.

A

Empty [inaudible] and then inside of the function body,, we just wrap it with an async block.

A

So let's talk about what the compiler actually does when you compile this async function.

A

To do that, I want to walk through how you would implement this by hand in the pre-async/await.

A

World., so the way you would do that is you want to create a data structure that represents the.

A

State of your request,, and so we are going to do that, we are going to call it request.

A

Handler and start with what we call the argument star function into that struct and then we.

A

Are going to implement future on a request? Handler, again, our async function doesn't.

A

Return anything so the output associated type is just tuple, and then we are going to implement.

A

The poll method, so the way you implement this by hand is by writing a state machine.

A

Which is what we are going to do.. We are going to add a state field to our struct, which keeps track of what state our request.

A

Is in and then, inside the poll method, every time I get polled we are going to match on.

A

That state and basically behave accordingly.

A

So the very first state that we can be in is we've never been polled before,. This corresponds.

A

To the very top of the async function;, the next one is that we are getting the row so.

A

Awaiting the result of the database;, the third is that we are writing to our tcp stream.

A

Notice, I didn't create a state for encoding the json, because that's a synchronous, operation.

A

There's no await., then. Finally, we are completely done and ready to return. Our result.

A

A

In the case, where we've never been polled before, we are going to just basically execute.

A

The very first line in our function,, so we are going to call get row and it's going to.

A

Return a future which we are going to store in a field and then we will update to getting.

A

Row. get row returns a future.

A

We are going to store that in a field called row fut and we have to initialise all the.

A

Fields, need to be able to say:, I don't have it yet, none, and then, when we do have it,.

A

Initialise it, some,, with the value of the future.

A

Then once we are polled we are polled when we are in the getting row. State.

A

We are awaiting,, so we need to propagate this to pull down the future, that we are awaiting,.

A

So we will do that., we know there's a future inside of row_fut.

A

if that returns pending. That means we can't make any more progress. So we simply put that.

A

Back up the stack., if we get poll ready,, that means the row is ready and we can make progress. So the very.

A

First thing we can do is deallocate the row_fut,. We don't need that anymore and then we are.

A

Going to do the next thing in the function, which is to encode our row into json.

A

Then we do the next thing, which is to create the write all.

A

So we call write all.

A

again,. We are going to store the value in our state machine and then we are going to.

A

Update our state to be writing.-.

A

All right,, so the next state is. Writing.-.

A

This sounds really simple,, basically, the same thing that we saw before,, where we are now.

A

Await the write_fut, so we will propagate the poll down to the write_fut.

A

Again, if it's pending,, we will return poll pending.

A

if it's ready, there's no output. So we immediately update our state to be ready and that's it.

A

The ready state is really simple., you simply [inaudible].

A

So there's one last thing that we need to do, which is add a loop round. The inside of.

A

Our body- and that means that every time poll gets called, we make as much progress as we.

A

Possibly can until we either hit a poll pending on subfuture or we are completely done,. So.

A

Basically, we will constantly loop until we hit one of these three return statements in.

A

The body., you notice, there's quite a bit more code on the right side than the left.

A

It's not as readable.

A

that's one major benefit of async/await, of course.

A

And in fact, we actually cheated and actually made the code a little bit more readable than.

A

It would have to be in real life because we are actually borrowing one of our struct members.

A

With [inaudible] encoded inside of the value of the write, so when we call write all we.

A

Are handing it a reference to basically put it to buffer, and you can't actually write.

A

This in normal rust., you would have to introduce a layer of indirection with a smart pointer or something, and it ends.

A

Up being really not fun.,.

A

This is an issue that comes up a lot in async.

A

Thankfully, with async/await all that is taken care of for you and it's more efficient.

A

Just says:, I know that you are [inaudible] yourself and thanks to some extractions, we.

A

Can guarantee that.

A

Another problem with our implementation that you might notice is, we are actually wasting.

A

Some space here.

A

any time we are in the middle of getting the row, we don't have any need for the write_fut,.

A

Encoder bytes, and then in the next stage, when we are writing to the tcp stream, we don't.

A

Need the row_fut anymore, so it would be really nice if we could not allocate all of that.

A

Space and just re-use the bytes.

A

If we look at the way that it's laid out in memory right, now,, we've got offset zero of.

A

Our structs and you have all of the fields,, so this is the way it is now.

A

What we would really like to be able to do is basically again re-use those same bytes,.

A

Row_Futs in the next stage when we are awaiting the write_fut.

A

So, as recently as a few months ago, it didn't do this, and this problem compounds really.

A

Quickly., if we assume, for simplicity's sake, that the size of our future is basically dominated.

A

By the size of all the subfutures,, then we are roughly twice as big as we need,, not re-using.

A

Though bytes. and that's a problem, but it actually gets worse because row_fut and write_fut, those.

A

Are written as async functions and those each have their own problems where they are awaiting.

A

Different subfutures, and so really those are twice as big as they need to be.

A

So really the future we are, writing, isn't twice as big as it needs to be,. It's really.

A

Four times as big as it needs to be, and then if that gets awaited by some other future.

A

Maybe that's eight times a big as it needs to be.

A

This really did happen in practice., so in fuchsia we would have futures that took up half a megabyte on the stack,, which is.

A

Really big and we had a lot of futures that actually blew the stack.

A

You can [inaudible] in rust., it's not undefined behaviour,, but you can.

A

What the compiler wants to do is it wants to ask the question: okay,, which of these.

A

Fields can I overlap?.

A

The way it does that is, it looks at your programme. I will zoom in here, so you can read some.

A

Things so this is a graphical representation of what's called mirror or the mid-level ir.

A

Inside the rust compiler.

A

We represent the entire program as this control flow graph,. We are inside of every block.

A

These are called basic blocks,, so, on the left side in the middle, you will see basically.

A

Block number 14 and inside of that block, there's a bunch of statements.

A

These are very simple statements where you can only do things like:. I need to allocate.

A

Storage on the stack for a local variable, or I need to access one. Member of this variable,.

A

Reference something or assign something.

A

there's no control flow inside of the basic blocks..

A

All the control flow happens between the basic blocks along these edges. Here.

A

So at the end of block 14 we drop. Something and drop is an operation which can panic,.

A

So there are two options.: either we return and go down the happy path,, which is normally what you think about, down.

A

To basic block 23, or we panic and we unwind and go into basic block 7 to do all the clean-up.

A

So what we are going to do is crawl. This entire info graph and work out which variables are.

A

Storage, live at the same time.

A

storage live is how we represent that. We actually need storage for a variable.

A

We are not actually going to do that because we don't have three hours for this talk so.

A

We are just going to squint at the source and see if we can figure anything out.

A

You can do this at home. that works pretty well..

A

So the first question we are going to ask is:: can I overlap row_fut with encoded?.

A

The rule of thumb here is,, if the last time you ever use a future, is sorry,. If the last.

A

Time you ever use field a happens before the first time you ever use field b, then you can.

A

Definitely overlap.

A

so in this case, you will notice the last time we used row_fut is we set it equal to.

A

None and then the following line is the first time we ever use encoded and you can kind.

A

Of reason about this by looking at the source and saying: okay,, definitely these are never.

A

Needed at the same time., so yes, we can overlap these in memory..

A

When I say overlap, I mean just re-use the bytes, right?.

A

Okay,, so what about row_fut with write_fut?.

A

Again,, in this case row_fut, the last time we ever use it is setting it equal to none.

A

And then later on, we use write_fut,, so we can definitely overlap these two.

A

Then the last two fields that we care about are write_fut and encoded..

A

So in this case, we assign to encoded and then we assign to write_fut and we are actually.

A

Handing out a reference to the encoded bytes,, which means that we definitely need these.

A

Write_Fut contains a reference to encoded, so we can't overlap these in memory.

A

So this allows us to overlap these bytes.

A

What this ends up looking like is not so much a struct, but more like an enum where you have.

A

One variant per every state your future could be in and inside of each enum variant. You.

A

Have all the fields that you need for that particular state,, so in general, in this table.

A

We have all of the enum variants, so we could be in unpolled, state,, suspend 0,, 1,, 2, etc.

A

And then, all of our local fields on the left here.

A

the only difference between a regular rust enum, and this is that you can have one field.

A

And multiple variants,, so in this example, we need local, a starting in the suspend zero.

A

Variant, but also in the suspend 1 variant, so you can have one local that persists across.

A

Multiple await points., that comes up quite a lot.

A

So I want to talk about. This is all very cool,. But what are the performance applications?.

A

So, let's look at some patterns that you might not realise have subtle performance, issues.

A

So in this example, we have a function called do-stuff,. We take a context which is just.

A

A smart pointer to some context somewhere and what we are going to do is. We are going.

A

To log some development fill with some contexts.

A

There is a subtle performance issue with this code.

A

Can anyone spot it?.

A

Nod, if you think you see it.

A

Not seeing any nodding, okay., so what can happen is, at the end of every scope is when we drop our local variables,.

A

Right?, so the compiler is going to insert an implicit drop at the very end of our function, but we.

A

Don't actually need context until the end of the function; we just log in at the very.

A

Beginning., so we are actually hanging on to this context, all the way through awaiting that future and.

A

Then finishing the task,, which is kind of inefficient.

A

If we have the only reference to context, then we will be heaving that memory around and.

A

We don't need to be. what we are going to do. Is, one way to mitigate this is to move that context into basically.

A

An interscope in the function, and the effect of this is that we are actually moving the.

A

Drop to the end of that interscope before the await.

A

This is quite a bit better., not hanging on to context or awaiting some other future.

A

That future could take a year to complete.. We have no idea., but it's still not perfect.. The reason is: remember, futures do nothing until they are polled and in the case of an.

A

Async function that includes all of the lines at the top of the function,. So none of these.

A

Lines are run until the first time the poll is called,, which means we are still hanging.

A

On to contexts or in that unresumed state.

A

So the "perfect" way of dealing with this is actually to desugar our async function.

A

Into a regular function, which returns impl future, like we saw earlier, compiler does.

A

This for you, but you can do it yourself and at the very top of that function we will do.

A

Our log and then we will write an async block which drops this into an async context. Where.

A

We can await anything we want and actually evaluate your future return.

A

Because we don't reference context inside of that async block at all,, we never have.

A

To save it inside of our state machine,, so we aren't allocating any bytes to save the.

A

Smart pointer and also never hanging on to that reference inside of the future.

A

So once due step is called, we immediately log., then we return a future that can make progress. Later.

A

Another pattern to look out for is when you are awaiting expressions that have really.

A

Big, complicated temporaries.

A

So in this example, I have a struct big, which has an array in it. That is just kilobyte and.

A

Link,, a pretty big type..

A

We are going to implement drop for big and do something silly like print a message when.

A

It is called., then I have a function through from usize to usize.

A

It doesn't really matter what it does.

A

Inside of my async function, bar,, we are going to construct a big simply to get the length.

A

Of the array out and then call foo with that, link.

A

So this is semantically equivalent to calling foo with 1024.

A

Let's say we don't want to repeat ourselves and we forgot the thing in rust, so we are.

A

Going to construct it on the stack and get the link up., normally the compiler could optimise this away and it would be fine.

A

But what happens when we try to print the size of the state machine that is returned?.

A

We are going to get something really big back.

A

Our state machine is over a kilobyte in size..

A

That's because it is embedding a copy of big inside of the state machine.

A

The reason for this is that rust has well-defined semantics for when the structs are run and.

A

Any time you create a temporary inside of a statement,, the destructor for that temporary.

A

Is called at the very end of the statement.

A

basically,, you have a temporary expression inside of a larger statement, fast forward.

A

To the next semicolon you see and that's when the destructor will be run for that temporary,.

A

Which, in this case, is not good, because that, after our await, right?.

A

So this is something to look out for and it's pretty easy to. Mitigate.

A

So in this case, we could simply get the link inside of one statement, there's a semicolon.

A

In between that statement and the next await, and that would be fine.

A

We can also write it like this.

A

actually,, constructing the entire foo future inside of one statement and then awaiting.

A

On the next line., it doesn't really matter. in either case, when we print the size of our state machine. It's going to be much, much.

A

Smaller., we are not embedding a copy of big., you might say, okay,. That seems like a problem, but if the temporary doesn't have a constructor,.

A

In other words, it doesn't implement drop,. We should be able to optimise that away.

A

We don't need to hang on to this copy of big in between, because we never need to call the.

A

Constructor on it and you would be right, except unfortunately we don't do that. Optimisation.

A

Today., so that's kind of a future thing that we would want to do in the future.

A

So for now, just always look out for big, complicated temporaries.

A

This only comes up occasionally in practice, but it's something to be aware: of.

A

So should I use async/await? yes,, definitely use it., it's now stable, but it's a "minimum viable product",, and that means we are taking advantage.

A

Of the rust release process, where we release a new compiler every six weeks to iterate.

A

On the language, compiler and implementation., all of your code, all of the state machines that get produced should be as good or better.

A

Than hand-rolled, in most cases with a few notable exceptions, and that's going to improve.

A

I got to talk to some of the people. Writing the c++ coroutines proposal for 2020, which.

A

Is a really interesting talk and that's some of their version, c++'s version of async/await.

A

So there are a couple of differences between theirs and ours.

A

One major difference between rust and the current proposal for c++ is that any time.

A

We allocate anything on the heap in rust,. That allocation is always made explicit, so.

A

If you await another future,, it always gets rolled up, all of the state for that future.

A

Gets rolled up into your state machine, your data structure,, and we don't allocate it on.

A

The heap, unless you create a box and put the future in there.

A

The c++ proposal, there's actually an implicit heap allocation every time. You await a future.

A

And in many cases the compiler can optimise that away,, but there are no guarantees.

A

There's a possibility to flip over a new an optimisation and have an optimisation fail.

A

And now you have a new heap allocation in your programme, in the c++ programme.

A

Rust has fewer language-level extension points, so one consequence of releasing a new standard.

A

Only once every three years is that you can shove everything in at once.

A

This works for c++,, but in the case of rust, we kind of have a minimal service area.

A

The benefit of that is that you have to do less work, keeping code, performant, and the rest of the stuff. You don't need.

A

We have less legacy.

A

Async/Await is not just for web servers.

A

There are a lot of things you could use it for, and I think there are really a lot of.

A

Interesting applications out there.

A

one is high-performance compute.

A

Another is actually filesystems, so any time you are writing to or reading from disk, that's.

A

A whole lot slower than your cpus..

A

I think there are some interesting applications there.

A

Device drivers might be an interesting thing., I'm working on operating systems. So that's what I'm thinking about.

A

Build systems: any time you are, writing a build system,. You are basically starting a.

A

Compile step and then that can take a very long time, so there might be some interesting.

A

Applications there for expressing build systems and bug rules and ui is another way,. So you.

A

Can imagine showing a pop-up user and then awaiting their response.

A

Things like that., it's really just a paradigm in a way of writing code that allows you to express what you want.

A

In a really high level, way.

A

If any of this interested you in terms of the compiler or just getting involved in the.

A

Rust project in general, I highly encourage you to contribute.

A

There are lots of ways you can contribute.

A

Probably the simplest way is, if you hit an issue,, whether it's a compiler bug or confusing.

A

Error message,: it's a great idea to file an issue for that.

A

There are people who do nothing except look for confusing messages in the compiler and.

A

Fix them, and they are awesome., so definitely report things..

A

If you feel like picking up an issue to tackle, there's a lot of easy or mentor tags in the.

A

Rust repo,, so you can look for that, and take advantage of the community.

A

I mean, one of the things I love about rust is that it has a really strong community and.

A

People that are always happy to support you.

A

So there's a few different places that you can go to kind of post, ask questions,. There's.

A

Me,, you can find me on twitter, github, gmail,, and there are also working groups that you.

A

Can join if you want more of an ongoing contribution.

A

Here is a shortlist of some working groups in the compiler team, but there's a lot of.

A

Other working groups and other teams.

A

You can find a list online.

A

And that's everything.

A

So, do we have time for questions?.

B

You mentioned that fuchsia's use of the.

B

Generators prior to those optimisations being implemented was about, what,, half a megabyte.

B

On the stack or more? after the optimisations,, can you share how much it shrank by?.

B

Tyler: yes,, that's a good question.

A

I do not remember that particular one.

A

I think we shrunk it by over 80%.

A

Some of them so some of the futures that we had shrunk by didn't shrink at all,, but.

A

Some shrunk by 80 or 90% in size, so it just depended how deep the stack of futures that.

A

Were being awaited, was.

C

The desugaring of the loop showed that it always there's a loop and a match,, so.

C

That, even if you go from one basic block to the next, it would jump to the top of the loop.

C

And rematch again. tyler: yes., is that how it actually compiles, or does it optimise it so that, if you don't suspend.

C

The execution,, it just falls through to the next basic block without jumping up to the.

C

Top and then matching it there? jumping back down.

A

Tyler: so, right, yes,, so it does actually have a loop in there,. So any time you have.

A

An await, that is actually where the loop is introduced.

A

So any time you await some subfuture,, we basically poll that future in a loop,. So what I'm doing.

A

Is semantically equivalent, but I'm putting the loop inside of the poll function.

A

Around the body of our entire poll, function,, essentially the same thing,, I'm just not embedding.

A

A loop at every single await point because then it wouldn't hit on the slide.

D

So async/await has been stabilising for four years, right, or something like that?.

D

Tyler:, it has been a while. now it's stable,. So are you seeing something that might show us that that was a mistake?.

D

We should have waited a little bit: more?, no pun, intended., tyler:, ooh.,.

A

uh Oh., so did we make any mistakes?.

A

I think time will tell.

A

I'm really happy that we picked the time,, even though a lot of people understand build.

A

Were upset by this.

A

yes,, I don't think that there's any red flags or things missing.

A

Some of the performance things that are covered, like the fact that we hang on to a temporary.

A

Until the end of a statement,, even if there's an await,, that was a change that was introduced.

A

A couple of months before stabilisation.

A

So I think that makes sense for the consistency of the language,. That's how the language works.

A

Outside of an async context,, but it's possible, we could have defined the language differently.

A

And just silently changed the semantics so that you don't hold on to temporary as across.

A

An await. time will tell. hi,. Thank you for the talk,. It was really interesting.

E

Tyler: thank you., I'm curious about the state machine..

E

Can you ask the compiler to omit not rust stores, but the mirror for what the state machine.

E

Generates? tyler: yes,, you can.

A

It's a nightly only flag,, so you have to download the nightly compiler and it's not super user.

A

Friendly., what am I doing?? I am going to show you.

A

So yes,, this diagram that I showed in the slides is actually generated by that.

A

It actually emits a dot graph file and you can render it.

A

So there's a couple of different modes but there's z, emit, mirror and then z. I can't.

A

Remember the flag, but message me and I will send you the flag.

A

There's a really nice documentation called the rustc guide, so people working on the compiler.

A

You can go and find it there,, even if you just want to see your state machine.

A

B

Give a big hand to tyler.