Rust Programming Language Rust Linz, 2 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Async Rust: Portability and Interoperability - Nick Cameron - Rust Linz, April 2022

Description

Async Rust: Portability and Interoperability

I'll talk about the portability and interoperability initiative of the async working group, an attempt to make async programming in Rust less tied to specific runtimes and improve interoperability in the async ecosystem.

I'll cover some of the problems with async programming today, what the initiative will try to achieve, and some detail about current work on async IO traits (Read, Write, etc).

No async programming experience required!

Nick's slides: https://t.co/v0UPeOQpqH

About Nick Cameron

Rust at Microsoft. Previously: Rust core team, distributed databases at PingCAP, Firefox graphics and layout, research in PL and type systems. He/him.

Nick on Twitter: https://twitter.com/nick_r_cameron
Rust Linz on Twitter https://twitter.com/rustlinz

A

The for the intro okay yeah so um uh well, let's start what is the async working group, so um async rust is in this kind of interesting place in the uh you can do async programming today. Very much so like the key kind of async and await um keywords are implemented. Stable, there's some great libraries, um but it's not exactly like a great programming experience.

A

It's extremely rough around the edges, there's a lot of kind of like really key functionality, that's missing, um and so the async working group was formed to fix that and it cuts across kind of language and libraries and tooling. So it's a real kind of like cross-team effort and um it's doing a lot of exciting work, and today I'm just going to talk about the portability and interoperability initiative.

A

But there are other initiatives uh looking at things like um async functions everywhere, so having letting you have async methods and traits or ace enclosures um or initiatives, looking at improving debugging, async code um and and so forth.

A

So it's uh it's quite a wide-ranging initiative with a with a whole bunch of working on it, but like I want to kind of like step back a little bit, because I'm um because I want to kind of like make this talk open for people who aren't like really heavily into async programming already, and I also want to if you are really heavily into asynchronous already.

A

I want you to kind of like uh share the mindset that presumably you once had when you discovered async, programming and rust, and what somebody new to to that today would would feel like so async programming and russ. uh um I I'm going to go through an example and the example is, is reading a file, um so I'm going to start like, let's just ignore all the async stuff. This is what we, if we were, writing regular synchronous rust. This is what um it would look like.

A

So so this is uh it's just a short function for for reading a file. It takes uh the path of the file as an argument and it returns the contents of that file or an error. So the first thing we we do on the first line. We open the file okay good. Next, we create a string that we're going to read the contents into, and then we call this function read to string which reads the context of the file into the uh into the string buffer.

A

Then, if there was no error, we return that string, easy peasy and then calling this function, which we've done the last line in the example, uh it's so trivial that it's not really even worth me explaining. You just call it um so there's a couple of things that I want to point out. The first is that read to string is not a function on file. It's a function on the read trait, which is an abstraction over any resource.

A

You can read from uh that's going to be important later, but I just wanted to point that out now. What's actually interesting to our kind of motivation here is that um when we're doing this io when we're opening the file or when we're reading from the file, we're actually asking the operating system to do that, and it's going to take quite a long time to do it.

A

I mean not like really a long time like a fraction of a second but like a long time in computer terms right um and whilst that's happening the the user process, or at least this thread is just sat there doing nothing, it's literally just waiting um for um for the for the operating system to complete and that's just a waste of time, um and so that's often the case with io and so asynchronous.

A

Rust is basically trying to answer the question like what, if we could actually use that time productively, um rather than just sitting waiting and uh do that without having to use kind of uh like full uh operating system threads um for for various reasons that it's not worth getting into right now. Okay, so if we wanted to do it, asynchronously first thing we need to do.

A

Is you need to use the async keyword to say this is an async function um and one effect that that has is that when we call the function, it's not quite as simple as before. Actually calling the function now won't actually do anything. We have to await the results of that function, and this is basically saying well. This is a function that potentially could take a long time to execute. So let's give the runtime um an opportunity to do something else, while we're waiting.

A

This is the the whole point that I just talked about um about asynchronous, giving you an opportunity to do other things, while you're waiting.

A

The other thing that um the async keyword lets you do. This is an async function, and that means that within this function we can await other async functions.

A

So when we had this file open to open the file, and when we have this read to string to read from the file these two I o functions now we want to await the results and again this is signaling to the runtime. These are asynchronous functions. Here's an opportunity um to get some other work done, while we're waiting for the for the for the operating system to to finish off doing what we asked of it.

A

um Wait did that sorry. um So there's a couple of problems, though um so so far, so good right, you think you're going to do asynchronous programming, you've, just sprinkled around a few more keywords, and hopefully things have got more performant right. We're going to be able to do stuff while um we're waiting, which is great but there's a couple of problems.

A

First of all, these two functions like file open in the standard library and read to string on the read trade from the standard library, they're, not actually asynchronous functions, so we can't wait for them and in fact there are no asynchronous functions like this in the standard library. So somewhere we've got to find like functionality. That does what we were doing before, but asynchronously.

A

The second problem with this is actually calling this function at all. um So I said we had to await um calling this function, but I also said that you can only call away inside an async function. Now you can call inside a nice block as well, but that's a detail. The fact is, you have this chicken and egg situation in that. If you actually try and call this function from main, the compiler is going to complain at you.

A

It's going to give you the error, that's that's on the slide um and we have to find some way um to get out of this kind of chicken and egg thing and actually have a way to call async functions.

A

So that's kind of uh these are two kind of big problems and hopefully uh you'll see in a few slides time why that's kind of what to do with the interoperability and portability stuff, but let's try and solve these problems. Let's imagine that we're just starting to do asynchronous rust. Well um kind of this is a sucky situation. Actually, the the compiler error here is correct, but it's not really doesn't actually help us solve the problem.

A

um But let's just imagine that somehow you know that you need a runtime like I've said the word runtime a few times, and um so maybe you know that you need to kind of somehow you know that you need to go looking for a runtime. Well, let's go to the docs um great place. To start when you're learning is to go to the documentation, here's kind of the official or as official as it gets documentation for racing programming. This is the the async book on the rustling website: okay, async runtime, so this looks promising.

A

um Async runtimes are libraries used for executing async applications. Well, that sounds like what we want to do here right uh and so it continues. Runtime is usually bundled together a reactor with one or more executors. No, I don't know what those words mean, but let's carry on reactors provide subscription mechanisms for external events like asynchio, well, async io is promising.

A

I did we do want to do some io into process, communication and timers, and I think runtime subscribers are typically and, and it carries on and frankly it's just kind of word salad and it's not really helping me. But let's, let's press on the future is great. Well um if you've just started. Writing some. Like async code, like you've, we've talked about async functions, we've talked about async io. I haven't actually mentioned futures at all.

A

Now, if yours kind of like deep in the kind of like rust ecosystem, you probably know that under the hood async functions are implemented as futures, but if you're new there's no reason, you should know that. So I like in when I'm role playing here, I have no idea why we're talking about features but anyway, let's read it and find out the futures crate contains traits and functions useful for writing async code. Well, actually that sounds like what we want. Okay, let's keep going. This includes a stream sync async read and async right traits.

A

Well, we had before the read tray and we wanted an async version of it. So maybe async reader is good. This okay, let's keep going futures, has its own executor, but not its own reactor still don't really know what this means, so it does not support execution of async, io or timer futures that doesn't sound great. For this reason, it's not considered a full run time.

A

Okay. Well, let's keep reading popular async runtimes. There is no asynchronous runtime in the standard library and none are officially remain recommended.

A

The following crates provide popular run times and then we've got a few crates here, one of which is a popular asic ecosystem with http, grpc and tracing. I actually want to read a file, but maybe that'll work. Who knows the next one's a crate that provides asynchronous counterparts to standard library components? Well that sounded like half the problem I had, but didn't it just say that futures would do that, but it wouldn't run my code. um The next runtime is a small, simplified, async runtime.

A

Well, I'm not sure whether I want a small and simplified one or a popular one. um Do I actually want an async trait? I definitely don't want a unix stream and the final one is an executor for use in the future os, and I I don't even know what that is. So this like this is a really silky situation at this point, if I were kind of learning, I would want to stop and I don't need to think that I'm just kind of picking on the documentation here.

A

The problem is the the ecosystem right, the the the documentation is an accurate reflection of reality. Unfortunately, um so at this point I wouldn't really blame you. If you just give up and decide to learn, go because if you wanted to do this in go, you just add the go keyword and it's all async now.

A

So uh that's kind of what we're up against and where we're at, um if you've, not if I've not convinced you to go and learn, go instead, if you're still with me, then I'm gonna, try and kind of like talk about how we might try and fix this in rust.

A

um But first of all, like you might be thinking well like this sounds horrible. Why do we have multiple runtimes anyway? And this isn't just a mistake or I I don't think it's a mistake at all to be clear um in rust, you're using rust, because you want precise control of the hardware you're using rust, because you want to get like absolutely optimal performance out of your code and we can write async code in lots of lots of places.

A

So if you're writing asynchronous code, that's executing on like a tiny embedded processor, that's a really different context from writing. Asynchronous code, that's executing on like a beefy rack server in a data server in a data center like powering some like big cloud system.

A

Likewise, if you're implementing, like the the file system layer of a database, that's a very different kind of um context from implementing like a network load balancer, and so these different contacts have wildly different constraints, and that means that you might want different runtimes and to to really optimize like your code, um which is why you're using rust in the first place right, then um uh you might need not just like tweets to the parameters of these runtimes, but actually like completely different architectures of these runtimes.

A

It's not uncommon for people um to write their own runtimes just to get the kind of like performance and other kind of characteristics they need for that code.

A

Kind of the problem with runtimes is that out of necessity, because there's been no support in the standard library for async um uh programming, they're kind of grown beyond just kind of scheduling executing your async functions.

A

They've um have to have their own like traits for io their own implementations of I o uh utilities like timers, timeouts, concurrency, like um channels, mutexes, fences barriers, whatever um all kinds of utilities, from like low level, things like pinning to higher level things like combining futures and and so on, and the problem with this is that when you go to choose a runtime, you actually have to commit to like a whole ecosystem, and you can't operate with code outside that ecosystem.

A

That's using a different runtime and that's a really like sucky position for um for the rust community for the rest ecosystem to be in and that's where the portability and interoperability initiative comes in.

A

So the my kind of like one sentence goal for for portability. Interoperability is that we want to support many runtimes, there's like really strong motivation for having different runtimes, but we want just one async ecosystem right. We want you to be like just because you choose such and such a runtime, or you decide to write your own runtime. You are still able to use all these libraries that are out there in the async ecosystem.

A

um You're not limited to to just those that have made that same choice that you have a quality to this is that it should be easier to switch runtime um if you start off using a general purpose one.

A

But it turns out, after a few months of coding on your project that you need something a little bit more specialized, then you should be able to make that switch relatively easily and finally, like, as I kind of like hinted in the motivation- well, probably more than hinted, we want to make it easy to to start asynchronous programming. We want to remove a whole bunch of that friction that comes from um having like multiple uh runtimes um in in the async ecosystem.

A

To get a bit more concrete, um the things that we want to do as part of the portability into our ability initiative is. We want to move a whole bunch of stuff into the standard library, so things like traits for I o um utility functionality that can be shared across run times. That kind of thing so there's, there's kind of like standard issue, stuff um we'd, like some kind of abstraction of this concept, of a of an executor to make it easier to change executors.

A

Although um there's a lot of work to be done, that I don't know exactly what that abstraction will look like. Maybe we'll want to have some executor in the standard library, and we definitely don't want to have any kind of you know, um best executor or default executor, or try and replace any kind of existing runtime.

A

What what we want to provide is the kind of minimum necessary executor for somebody to be able to start async programming, so they can learn how to do it, so they can kind of like start a project before they have to kind of have all this like deep knowledge that about their projects or whatever, to choose the best.

A

Runtime, there's also kind of like a bunch of kind of paper cuts that we want to um get rid of in order to kind of just make async programming easier so like what does the shiny future of async programming and rust, look like if we managed to achieve our goals.

A

So let's go back to the example that I started with and see what it would look like. Well, it actually looks pretty much like what I wanted. What we thought it ought to look like and that's kind of the good thing, but what's interesting is how that's possible. So, first of all, let's look at the open function.

A

Well, it would be nice if the standard library on the on file had not just the synchronous open but an asynchronous open, and they might even be the same name and we know which one to call based on the context. So here we're calling a weight. It's got to be. The async function um now read to string. That's a function, that's part of a trait, and so we should have like an async equivalent of the read trait that's in the standard library, and we can import that pretty easily.

A

Just like shown on the first line in this example.

A

Finally, to call this function, um we should just be able to stick async in front of um fun main in the same way that we can put results as the return type for main and the compiler will just figure everything else out for us and it'll just work.

A

Okay, so that's that's kind of the what we want it. um You know a simple case of async programming to look like in in our shiny future.

A

Okay, so for the remainder of this talk, um I want to kind of like change the pace, a little bit and dive kind of kind of deep into into the work. um I've been doing most recently as part of this initiative, and I want to talk about um the async I o traits and what they might look like.

A

um So the the il traits um are in the standard library at the moment and read, write, um there's, buff read and seek, um and we want kind of like asynchronous versions of these and I'm just going to talk about, read to kind of try and illustrate some of the issues with with async traits and what what the eventual design might look like. Some of the same things apply for kind of um the write and seek and so forth, but limited amount of time.

A

So this is a a vast simplification of the the current retreat in in the standard library, uh the the current synchronous version. So it's just a function read: um oh sorry, the trade abstract of a kind of a resource that can be read and so there's a function read you pass it a mutable slice of bytes and it will read into that slice and it will return you. The number of bytes returned so async retract.

A

Maybe it'll, look like this. It's the only thing that's changed is we've got the acing keyword and set in in front of the read method, and now um we can. We can await this and we can kind of get on with something else. Whilst the operating system or or whatever is doing the read um and to you can't do this right now we need async methods and traits, but, like I said earlier, this is something that the the async working group is looking at and hopefully will be be implemented soon.

A

And so then this will just work, and I think this is a really great starting point. um It matches very closely the synchronous, um trait, and so it should be very easy to transfer your your knowledge and experience and synchronous programming to asynchronous programming.

A

um But it's it's also only a starting point and to kind of like explain why I have to kind of get a bit more into the details of how asynchronous io can work.

A

So I just want to to start going back with sync. So here's how a a read call works in kind of the synchronous universe, so the user process, probably viral library, initiates a read to the operating system. The operating system actually reads from the resource.

A

This could take some time and when that read is done, the operating system returns that red data back to the to the user process um and the library presumably will pass it to the actual user code and, like I said before, the downside of this is that, while the operating system is doing all this reading, the the user process is just sat there twiddling his thumbs with nothing better to do.

A

So the the async equivalent of this there's actually like a couple ways that I think io can work well more than a couple there's a couple that we're really interested in there's the readiness model- and this is what epoll on linux uses and there's the completion model, and this is used by iocp on windows and, u ring on linux, um so I'm going to go over how both of these two um uh models of async io work and how that kind of affects the the design of a read an async read, trace and rust.

A

So, let's start with readiness. So again the user process initiates the read, but this time the operating system immediately returns and that gives the user some opportunity to the user process some opportunity to get on with something else right. So that's getting us! The icing benefit that we that we wanted.

A

Meanwhile, the operating system is doing the read from its resource in exactly the same way as before, and when that read is done once the operating system has read that data into its internal buffers. It's going to tell the user process somehow that it's that it's ready that we're done and at that point the user process can read the data out from the operator's operating system extremely quickly, all right. So this is the the async call.

A

The um the nice thing about this is that we don't the the user process doesn't need to provide a buffer for the to to read the data into until the buffer until that data was ready to read. um So that means, if it's allocating, can allocate that buffered like at the last minute. um uh You know just in time and then pass the um the buffer to the operating system and to fill up.

A

It doesn't need to pass the buffer when it's initiating the um the read- and this can be really important so consider if this is like a web server or something similar. um Then, when you're, when the operating system is reading from the resource, it's not necessarily just reading the bytes of a disk, it could be waiting for those bytes to arrive from a remote host. And if that remote host is waiting for a user somewhere to like type into a web browser or something you could be waiting a really long time.

A

For the for this io to complete and of course, it's a website you're not dealing with just one client you're, potentially dealing with many many clients, each of which is waiting on one of these reads and because you don't know how many bytes you're going to get back. This buffer has to be quite big.

A

So you've got many, many quite large buffers that have to be held um for potentially quite a long time, and that would be a huge waste of of um memory.

A

So um being able to kind of like have this memory kind of like on a kind of just-in-time basis, is a really big advantage for for the readiness model.

A

um Completion is uh the completion model for asynchro is a bit different again. We initialize the the I o with the read the operating system returns immediately. The user process can get on on some work, whilst the operating system actually does the read.

A

But this time the operating system tells the user process not when it's ready to read, but then, when the read is completed and the way this works is the user passes a buffer to the operating system or a reference to a buffer to the operating system?

A

When the when it initiates the I o and the operating system hangs on to that, um writes directly into that and then tells notifies the user when the buffer is full okay. Now you obviously lose out on the advantage of readiness that I was just talking about about having this just-in-time allocation of buffers.

A

But what you get instead is that you could. The operating system can read bags directly from the resource into the uh into the user's uh buffer, and so this can end to end. You only have like one read into a buffer, so it's a zero copy um process and that could be really efficient. So if you're doing disk io, for example, then completion is a has a whole bunch of advantages um in terms of efficiency over readiness all right. So that's a lightning, quick explanation of how async I o works.

A

um Let's see how that relates to the async um read tray. Well um before I said we could just have like an async read method, but unfortunately that doesn't really work very well for either model for readiness, um the because we only have this one function. We we have to start that right at the beginning, there's no opportunity to pass the buffer in later. So then do this just in time allocation of the buffer.

A

We have to pass it in at the beginning, and so we lose this like big advantage of readiness to get around this, um I'm proposing. We have an um another trait so as well as just having the asynchronous retreat. We have a trait that is called ready. So now you have two options.

A

If you, um if you don't care about this memory usage, so much, then you can just use the asynchronous read method very ergonomic. Does what you'd want very easy to use great? If you need to be writing memory optimal code, then? Instead, you can call the asynchronous ready method and note that we don't pass a buffer in here.

A

Okay, I've elided a few things from the example, but it's it's just flags basically um and so the uh so so your task would would wait for the ready method to complete asynchronously and when it returns it says, hey, I'm ready, and then you can call this non-blocking read function and notice that this isn't an asynchronous function.

A

This is going to return immediately and in fact it promises not even to block like a like the synchronous read function does um and that's when you pass the the buffer, and so you have this opportunity to do the the just in time um allocation or to reuse buffers or whatever, because you have this opportunity to get that memory between when ready completes and when you call the non-blocking read function.

A

um So, uh let's look again at completion, so in this case you do pass the buffer in the beginning, but unfortunately this still doesn't work with the the simple async read method and that's because of cancellation.

A

So if we were to do the simple async read um with and the um the runtime were to pass this buffer straight to the operating system, then uh one aspect of evasing programming in rust is that you can always cancel um a future like an async function. uh And if that was a happen, then um the the buffer might get d de-allocated right, because um as far as the the rust world is concerned, you don't need this buffer anymore.

A

The the future was cancelled, but unfortunately, just because the future was cancelled doesn't necessarily mean the read with the operating system was cancelled. We, the operating system, could have already started reading and if that buffer has been deallocated, the operating system is reading data into essentially just a dangling, pointer and kind of the whole point of rust is that we don't read data into a dangling pointer. So this is very suboptimal um and uh the so the problem here is really one of ownership.

A

So we're passing like a a slice of data um uh from uh from from the the caller of the read function, all the way into the operating system. But, logically, we actually need to transfer ownership to the operating system because the operating system has to own the buffer. It has to be the one: that's the the only one is able to write into that buffer while it's. um While it's doing the read.

A

So this actually fits kind of well with the buff read trade. So I I I said earlier: there's the retrait and there's also the buffery trait uh and the buffery trait is a version of reading where um we read into a buffer that is managed internally rather than having a buffer that the the caller manages.

A

So if you're happy for the run time to manage your buffers, then using buff read is a really nice fit um you. uh Rather than calling like the read from the read trade, you call read on buff read uh the runtime takes care of the buffers if it's cancelled, the buffer isn't deallocated. So you never have this dangling point situation um and you just get back a slice of data, so this is great. This fits the completion model really well, and it means that the the operating system can still be zero copy.

A

The problem is that we're returning a borrowed slice of data to the caller um and that's okay. If the caller is going to immediately consume those bytes, but if the caller wants to store those bytes, then it has to the caller has to copy the bytes, rather, um whereas um what we'd like to be able to do is the caller to pass in a reference to the buffer that it owns, um which the completion uh which the the operating system can write directly in.

A

So it's zero copy, both on the the user process side and in the operating system. So a possible solution to do that is yet another trait um and owned. Read is a terrible name for this trade. I haven't figured out a good one yet, um but the the idea is that it's very much like the retreat, except rather than passing in a borrowed slice. We pass in and owned uh an abstraction over an owned buffer, so owned, read.

A

Buff is just uh um a trait that you would implement for like a vac of u8 or for your own kind of custom uh data types so that you could temporarily transfer ownership into the um into the library and kind of hypothetically the operating system so that the ownership um works out, uh and this kind of model seems to work really well with the completion system.

A

um It's kind of the part of the design, that's still the most up in the air, but I'm hopeful that something like this will will work well um as part of the as kind of suite of async io traits.

A

Okay, so that's kind of all I've got for this talk um just to quickly summarize, I kind of gave you um some motivation on what async program is like today and why we need to improve that. I shared kind of the goals of the portability and interoperability initiative, primarily that we support many runtimes within a single async ecosystem, making it easier to switch runtimes and make it easier to start asynchronous programming, and I also got a little bit into the design work that is kind of currently ongoing with the um the async io traits.

A

So that's what I have today thanks for attending this talk and and for paying attention um if you're interested in this work, the the async working group does all its work in the open, mostly on zulip, so um log on to zulu the the rust student and there's uh a stream uh for the for the async working group, um and you can see all the discussions we have there and I'm nick cameron on zulip.

A

If you want to follow this uh work on on social media or whatever, um I'm nick r cameron on twitter and nrc on github or if you prefer email. That's on the slide and thank you again.

A