Cloud Native Computing Foundation Cloud Native Wasm Day EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Serverless WebAssembly for compute-intensive workloads - Robert Aboukhalil, Invitae

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Serverless WebAssembly for compute-intensive workloads - Robert Aboukhalil, Invitae

Serverless functions are a cost-effective way to build scalable APIs, but they can be impractical for certain applications due to cold starts and lack of support for your language of interest. However, now that several platforms offer FaaS solutions powered by WebAssembly (e.g. Cloudflare, Fastly), they can now provide near-0 function initialization times, and support for languages that compile to WebAssembly. This talk explores how to use serverless WebAssembly to build APIs for compute-intensive workloads. As a concrete example, Robert will showcase a serverless function that performs bioinformatics analysis; this is done by compiling an existing library from C to WebAssembly, and deploying it as a serverless function.

A

Hey everyone thanks so much for being here, I'm really excited to share with you a few thoughts on serverless web assembly and how that could be a really powerful tool for compute intensive workloads.

A

Now before we get into that just a bit about me. My background is a mix of computer science and bioinformatics, I'm currently a software engineer at nvitae, which is a genetic testing company and I'm the author of the book level up with webassembly.

A

Right so before we dive into the serverless webassembly parts, I did want to review why webassembly can be a powerful tool for analyzing data in the browser, so, whether you're talking about a desktop application, image, processing, machine learning or bioinformatics.

A

These four are really good examples of tools that leverage existing code written in languages like c sql, plus and rust, and are able to bring those tools into the browser and the reason why this is powerful first of all is because you are able to reuse existing code. So you don't have to rewrite these complex algorithms into javascript.

A

It's also nice, because in some cases you can get some performance improvements, for example in the tool ribbon that I showed in the previous slide. We saw a 3x speed up by replacing slow javascript computations with an off-the-shelf tool that we compile to webassembly.

A

Using webassembly, the browser is also useful because you can leverage the user's compute power, which means that, as a tool builder, this is much cheaper to host, because all the costly compute is distributed to the users.

A

It's also useful, if the data is local and you're analyzing that data and it doesn't have to go in the cloud. In that case, it makes a lot of sense to use webassembly in the browser.

A

But you know what, if these assumptions don't pan out what? If you can't leverage the user's compute power, what if the data is not local, and so if that data, for example, is in the cloud and it's pretty large, you don't want to start download loading large data files to your user's browser. Just so you can run the webassembly there.

A

So a few alternatives that one is, you know just run it in the cloud spin up some vms download the data locally run the analysis, save the results and then shut down the vm.

A

You can even use more specialized things like aws batch, where you can essentially develop a whole pipeline of tools and then each step can have its own set of cpu and ram needs.

A

An alternative to that would be serverless functions.

A

You know, analysis in the cloud is is very nice, but it's also can be pretty costly and adds up pretty quickly and so serverless functions can be an alternative, for example, lambda or google cloud functions where you can essentially have scaling done for you, so you don't have to worry about that, and the pricing is kind of on a per use basis.

A

But in this talk I do want to focus on one particular flavor of serverless functions, namely those that are specifically webassembly, focused, for example, cloudflare and fastly.

A

You know the benefits of those are basically the same as the serverless functions, but on top of that, because webassembly is so light compared to something like a container, it is easier for serverless providers to make that function have very little cold starts or none whatsoever, and so that can be really important if your functions are short-lived. In the first place, you don't want to have to wait for hundreds of milliseconds before your function, even initializes.

A

It's also neat, because you can now use webassembly both in the browser and in a serverless function, and you know, obviously you can do that today with javascript. But with this you don't need your tools to be already written in javascript. You could take things within other languages, and that's really. The beauty of webassembly is that you can then run these tools wherever you see fit.

A

So, let's, let's take a look at a concrete example of this, so the application here is to build an api that simulates data from a dna sequencing experiment, and so what that's going to look like? Is you call the api?

A

It does some analysis and then it outputs some simulated dna sequences, and so what that looks like is just strings of a's, c's, t's and g's that represent the genetic code essentially, and so what we want to do is have these simulations that simulate errors during the experiment, but also you know, genetic variations between individuals and so to do that simulation.

A

We're going to use cloudflare workers and I'll talk later on, why that is, but basically the very powerful local starts runs at the edge, but to do the actual simulation itself, we're going to rely on a tool called wg sim.

A

Now this is a tool that's commonly used in bioinformatics to simulate dna sequences, as it happens, it's written in c, and so we can compile it to webassembly and run it as part of that api call.

A

Now, one thing to note is that wg sim can't run on its own. It needs in some initial data to start with, and so it needs a reference sequence so that it can, on top of that, make small changes randomly and so that reference genome we're going to store on s3 and fetch a subset of it on each api call.

A

One thing to keep in mind, though, is that cloudflare workers are limited to 128 megabytes of ram for now um and to get around that limitation. What we have to do is not keep in memory all the data and then send it. What we have to do is essentially stream. The data in from s3 run the simulation and stream the data out, as we have it to the user.

A

So this would let us, for example, output a lot more than 128 megabytes of data.

A

What's neat about this architecture also, is that I can run the same thing in the browser or as a serverless function. In this case, it makes sense for the serverless function, but there are cases where you can imagine you want to run things both in the browser and the those functions depending on where the data comes from. Is it local or is it already in the cloud.

A

So as a demonstration, I've deployed this to wg sim.sandbox.bio. This is the api endpoint. You can tell it how many sequences you want, which chromosome you're interested in and the error rate that you want to simulate. So if you go to that endpoint, you will see this big mess of a result. uh This is actually a file format in bioinformatics called fastq.

A

It's not the prettiest, but you can ignore most of this and focus on the actual dna sequences, and so this is what we want to simulate, and so, if we were to refresh, we can get different simulated dna sequences from random regions in chromosome 1..

A

So this works it's fast, it's scalable, it's cost effective, um and so you may be wondering well why. Why did I use cloudflare workers for this? So I wanted to talk a bit more about that and just as a side note, I'm not affiliated with cloudflare I'm. This is just my personal opinion.

A

So one reason is, you know, like any other serverless provider, they give you the scalability of the paper use, but they also give you first class webassembly support and in part because of that they are able to reduce their cold start so that they're non-existent.

A

You know again because webassembly is so light compared to containers, um and so so actually, workers used to have a five millisecond coil start, and then they recently modified it so that when you make an api call during the https handshake, they actually warm up your function then, and so by the time the handshake is done. It starts executing it's very cool.

A

The other thing that's neat is that this is powered by v8, and so v8 is the chrome browser's engine that executes javascript and webassembly, and you know the reason why I mentioned this as an advantage and not just as a random trivia fact is because this means, when you're running webassembly binaries on cloudflare workers, you're still running them in browsers.

A

It just so happens that that browser is hosted in the cloud, and so what that means is you know, like I said earlier, we can be in a point where we can use webassembly in the front and end of the back end. Well, this takes it a step further, where we use the exact same wasm binaries in our browser and in cloudflare's browser and which is really powerful if you're building tools that need to run in both places.

A

The other thing, that's really good, is the debugging experience and just to demonstrate it. This is what it looks like when I'm looking at my wg sim script. So this is the javascript code part, and so what I can do is come over here and click send. This will execute the code on cloudflare servers and return. The result right here so far, so good.

A

No, no! This is really earth shattering. But what's really neat is this part over here, which is the developer tools.

A

This is not the developer tools in my browser in a way, it's a developer tools and in the browser that executes your code and you can actually look behind the scenes what's happening.

A

So if you remember, I mentioned that our algorithm needs to reach out to s3 first to download some reference data, and so you can actually see that by going to the network tab- and you can see that it actually click on it, it reaches out to amazon, aws and fetches some data and we can preview it.

A

This is the sequence that is being used as a starting point by the algorithm, and then it does mutations on top of that, and so what's what's beautiful is that this is not running on my browser, but it lets us get a view into the execution platform.

A

Another neat application is the javascript profiler. This lets you see which parts of your code are taking the most time. Another thing I wanted to demonstrate is: if you modify this script, you can actually rerun it without deploying the changes. So, for example, if we went over here and added a console.log statement, you could then come here and click. Send you see this refreshed this? It says the script was modified, and so we can see the console.log statement.

A

You know and again this this console log did not execute on my browser um executed on cloudflare servers, but we're able to to see the output of that which makes it a lot easier to debug these kinds of applications.

A

The other thing that's useful is a key value store. So if you want to store key value pairs, that's kale. This is really good for that, especially for read heavy applications. So one thing I've used this for recently is for storing permalinks. So there's this app. Where you give it your your local data, it analyzes it stores the results into a key value store and gives you a permalink that you can share with others.

A

So they could see the results without having your input data, and so this is a great place to store that kind of thing.

A

Other useful tools are cron jobs, so you can run your javascript and web assembly on demand via an api call, but you can also have them run at schedules. So, for example, I have a script that runs every half an hour looks at some recent logs calculates summary statistics and stores the results in the key value store. So that way, when I go to the statistics page, it just pulls the cached results from the key value store and doesn't have to recompute them. Every time.

A

In terms of the limitations, so, as I mentioned, 128 megabytes of ram and in terms of cpu we're talking about 30 seconds for workers unbound, you know, as I mentioned, you know asterisk. This is the current limits, it's possible that these limits get increased over time.

A

And yeah, that is all I had to share today. If you enjoyed this presentation and you're just getting started with webassembly, you can check out my book level up with webassembly and I do also have a whole bunch of free articles and videos that you can check out as well with that. Thank you very much for your time.