Rust Programming Language Rust Zürichsee, 12 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rust Zürisee, Dec 2022: Supercharging Zero-Copy Deserialization

Description

Talk by Manish Goregaokar

Internationalization involves a lot of data. A Lot. And loading data can be slow.

ICU4X has dealt with this by going all-in on zero-copy deserialization, a technique that ensures that data loading involves fast validation as opposed to slow allocation. To help with this, we have built a suite of crates (yoke, zerovec, zerofrom, and databake) that make zero-copy deserialization easier to work with and more broadly applicable. This talk will tour through these crates, talking about what they do, how they work, and how they can be useful to you.

Slides: https://tinyurl.com/manishearth-zerocopy-talk
Meetup link: https://www.meetup.com/rust-zurich/events/289518586/

-- Chapters --
00:00 Supercharging Zero-Copy Deserialization by Manish Goregaokar
35:03 Questions

A

Now we're gonna have to the second talk of the evening by manish about.

A

Zero copy deserialization and how to make it screaming fast.

B

So hi everyone, I'm manish. I work on the internationalization team here at google.

B

I'm going to be taking you all through a journey of zero copy, deserialization.

B

Join me and I'll show you just how deep the rabbit hole goes.

B

A bit about me: first, I've been a rustacean for four years. uh Eight years back from.

B

The days when rust was less stable than your average azido-tetrazole and it still.

B

Pretended to have a garbage collector (it never did it was always broken)..

B

I also created rust, clippy known for being the most annoying tool in the rust universe.

B

It's been the single greatest act of petard hoisting. I have.

B

Ever committed .., it hates my own code.

B

I currently head up the rust devtools team.. I also think.

B

That languages, both computer and human, are pretty neat ..

B

To give you an idea of this, the flow of this talk I'm going to start out by talking a bit.

B

About why we care about deserialization. shane already talked a bunch about this, but I kind of.

B

Want to go through an example;, then I'm going to explain the concept of zero copy. Deserialization.

B

And then, finally, I'm going to try to address three questions about zero copy, deserialization:.

B

Can we make it work for more things,? Can we make it pleasant to use,, and can we even avoid it all together.

B

So I work on icu4x earlier my colleague, shane introduced icu4x.

B

But for those who are maybe watching this online or got there in the middle of the talk.

B

Icu4X is a new modular internationalization library in rust.

B

As an internationalization library, we answer questions like "how. Do you format this date in.

B

French in canada, with the gregorian, calendar?" or "how, would this list be sorted in japanese?".

B

Or "how, do you format this number in marathi?".? This needs a ton of locale data.

B

Let me demonstrate:, as I just mentioned, one thing we do is formatting dates;.

B

For example, you can give ic4x a certain time and ask it to format it. A couple of different.

B

Ways for american english and for bangla .. So here are the two. You can see a bunch of.

B

Formats, some of them are really long. Some of them are short. You can see the american english one is.

B

Month, first and then date, but the bangla one .., if you can read those numbers, is the reverse.

B

Let's look at just one of these date formats across the two locals.

C

Yes, uh let's look at one of these two date formats.

B

Across the two locales., what data do we need to produce? Just these two?.

B

First, we need to know what, how to format the day of the week:. We need to have the string.

B

"Tuesday" from somewhere., so here's the data needed for formatting the day of the week;.

B

You can see that there's some like short ones, some really short ones, medium sized.

B

And long ones, .., depending on how big you want the day of the week to be .

B

What else?? You also need the month and here's the data for that .... I know the font's small,.

B

You don't actually need to be able to read it just to give an idea of the size of the data.

B

What else do you need to know? what the digits look like for the requested language .

B

... and that's not that complicated, just 10 digits, nothing, fancy.

B

What else you need to know? the order to display it everything in.? If you look.

B

Closely you'll notice that in american english, it's october 11th, but in bengali.

B

It's 11 october ..., they're swapped.. Here's the data needed for those this.

B

Contains things about ordering, but it also contains things like mm/dd/yy and dd/mm/yy.

B

Which is also just a kind of ordering, it also tells you kind of how to combine day and date and time when you need to put them together.

B

There's more, if you want to format things like eras like ad or ce, or whatever, for example, these.

B

Dates are formatted such that they also need error data which looks kind of like this .... Oh, and this.

B

Was just showing off some of the data needed for a *single*, icu4x component : date time formatting.

B

We have a whole host of useful operations, each with its own requirements for internationalization.

B

Data, so we care about data, quite a bit.. We have a lot of data;. We need to handle it.

B

One of our goals is that we're modular in both code and data:. We want to give you the.

B

Flexibility of how much data you load,, how you load it, and when you load it.

B

The main strategy we employ for this is deserialization:.

B

The data is some format like json or james munns', postcard.

B

And you can fetch it. However, you want ., you can get it on demand over the network,.

B

From the file system, whatever, and then you use a deserialization library to load it into memory. .

B

In rust, usually deserialization looks something like this:. Most people use the.

B

Serde library, (there are some others).. You slap this custom drive on your type.

B

And that makes them serializable and then you can just load data with serde plugins.

B

Unfortunately, that deserialization has a cost:. If the text at the right is a snippet of a json.

B

File in memory that's being deserialized, every time you deserialize this person type.

B

Rust needs to allocate space for that string and then copy it over. for a single string. This is.

B

Not that expensive,, but if you looked at the size of data icu4x needs,, this starts becoming a cost.

B

Serde does support zero copy deserialization and what happens in zero copy deserialization.

B

Is in rust, specifically, is you add a rust lifetime to indicate borrowing.

B

You also add this special attribute and with this setup, when deserializing serde.

B

Can simply validate that the data in the loaded file represents a valid string.

B

It'll, just look at that,, be like "that's, correct, utf-8,, great!" and then it'll.

B

Just take a reference to that data itself, that's already loaded in memory as part of the file .

B

This happened without an allocation or a string copy,. Instead, person just holds a reference to the.

B

Original blob of data, it was deserializing from.. This is significantly cheaper. If you have a lot of.

B

Strings or some large strings,, you can just load the file into memory and then deserialization is.

B

Just making a bunch of pointers... much faster. and, of course, it's much cheaper on memory.

B

Because you're never allocating, except for original loading of the file into memory.

B

You might have noticed that I switched over to a binary format, postcard, in the lines below.

B

This is because human readable formats, like json, often contain things like escape sequences and.

B

Zero copy deserialization doesn't work with that, because the escape sequence will now be a.

B

Backslash and an n or something in memory- and you want the actual new line character in your string.

B

If you want stuff to work with both a human, readable and binary data formats,.

B

You typically use something like this,, you use these owned or borrowed types like cow, so that.

B

You can kind of copy it into memory for the human readable form. copied into an allocation.

B

For the human readable format, but still lets you take a reference for binary formats.. It's.

B

Just good practice to use cow and cow like types everywhere when doing zero copy, deserialization.

B

Overall, the basic idea of zero copy deserialization is that deserialization.

B

Should be just a pure cheap validation, step;, it shouldn't do any allocations or major copies.

B

So zero copy deserialization is cool and all, but out of the box it's got limitations...

B

One is around collections.

B

We had some simple data in the earlier example, but what about the date time stuff? We talked about ?.

B

We'd like to store all of that data in a zero copy fashion and we'd need something like this: ... You.

B

Know like a bunch of month, names, a bunch of weekday names,, a bunch of digits, ...we'd, also.

B

Need some way to do this kind of nested cow in a way? That's still zero copy, because now.

B

This is a slice of strings .., which is we'd, need some kind of cow of cow, or something that works.

B

This doesn't work in serde by default; and there are a bunch of different reasons. Why.

B

They're separate problems for sized data like char and u8, and unsized data like string.

B

:Et's walk through a bit of why this is tricky.

B

Let's look at sized data first: suppose we have a buffer of u32s that we're trying to deserialize:.

B

Let's say this buffer., there are two ways this slice could get represented in memory.

B

As a series of bytes., the first way would be with the most significant bits. First, like this:.

B

Pretty straightforward, right? the other way would be with the least significant bits. First,.

B

Like this. feels kinda backwards., these two approaches are called big endian and little.

B

Endian and they're actually originally named after terminology from the novel, gulliver's.

B

Travels. they refer to the end of the hard-boiled egg that a person cracks when eating. personally.

B

I don't actually do either;, I crack them on their side, which .., which makes me like middle.

B

Endian or something.- and this seems to be normal amongst americans, because we don't use egg cups,.

B

So it doesn't matter how you just put it on your plate and eat it. .... and that's.

B

The kind of stuff you get to think about when you work in internationalization.

B

Anyway, back to endian-ness most systems, these days are a little endian ...

B

Which is the backwards one,, but big indian systems do exist.

B

This is a problem for zero copy deserialization,. The whole point of it is that you can directly.

B

Refer to a region of memory from the serialized data and have it work, but the same u32 integers.

B

Are represented differently on different platforms, so the same data file can't be used on both.

B

This is only a problem around slices, bear in mind, with plain u32s. They need to be copied.

B

Out anyway, so the deserialization can handle the change of endian-ness if needed.. So you.

B

Can still use u32s with zero copy deserialization, you just can't put them in slices and it's worth.

B

Pointing out that when, when we say zero copy, it's a bit of a misnomer ..., what it really is is it's.

B

"Zero allocations" or "zero *large* copies ". All your normal primitives and copy types are.

B

Still getting copied, but your vectors and strings and stuff are being just validated.

B

Another issue is alignment.: let's look at this array. Again:.

B

Most types in rust and other systems languages have a required alignment,.

B

For example, u32s can only exist at memory addresses that are multiples of four.

B

But for deserializing data from a larger serialized blob, there's going to be data before.

B

And after it- and we don't actually know where it's going to end up, so it may or may not be aligned.

B

We could maybe have our serialization format insert padding, especially and then pinky promise.

B

That the original data was loaded aligned., this isn't super safe and it doesn't really.

B

Work well with serde's model.. There are other crates that do this kind of thing.

B

But it doesn't work with serde and we kind of wanted to use serde, which.

B

Is somewhat of the standard amongst the rust community.

B

All right, so those were the problems with fixed size data. What about variable sized.

B

Data?, let's say we want to do a slice of strings like the slice of month, names.

B

In memory, it might look something like this:, with each element being some pointer and a length,.

B

That's going to expand to something like this when viewed as a list of integers,.

B

And then the two pointer values will then point to the actual text: data in the slice,, which will.

B

Be somewhere ... in memory. ..., the problem is these pointer values will change over time. They.

B

Only make sense for the current execution of the program. .., not even the same computer,, not.

B

Even the same across executions, it'll be just the same for one execution,, and this is really bad for.

B

Serialization, because you want serialization ..., you want things you serialize to work. Everywhere.

B

Unless we're going to snapshot the entire heap, these pointer values are completely useless for.

B

Serialization. for zero copy deserialization, we have to avoid having indirection in any of our.

B

Types which makes most rust collections by default a bad fit because they expect pointers in them.

B

So ,the solution here was: we developed the zerovec crate.

B

It provides the zerovec and varzerovec containers that step around these problems.

B

They're all internally copy on right, so there's no need to actually use cow with them. They kind of do.

B

That internally for you., because if you're doing zero copy deserialization, you probably want cow.

B

Varzerovec is for variable size, types like strings and zerovec is for fixed size types.

B

Like u32 or char., they both have designs that work around the problems mentioned here.

B

Zerovec works by only accepting types that implement this trait called asule, which means that.

B

Which, which means they have a corresponding ule type, that they can be converted to and from. and.

B

Ule stands for unaligned little endian, which describes the nature of the type.. Firstly, it must.

B

Have no alignment requirements? It can exist at any point in memory and, secondly, it's expected to.

B

Have the same representation in memory regardless of endianness, because most systems are a little.

B

Endian., we just recommend that you make maintain the little endian representation so that getting.

B

It out of memory on little endian platforms is basically free.. For example, if you're looking at.

B

A zerovec of u32s,, the ule type, will be something like that,. Just an array of four bytes,.

B

Which is aligned .., which is byte aligned and .., has no endianness requirements.

B

This type has the same representation on all platforms.

B

Under the hood, the zerovec would then be similar to kind of like a slice of these types,.

B

Looking something like that., if we take the u32 slice, we're looking at before,.

B

It's zerovec would kind of contain like these arrays, with this kind of memory, representation,.

B

But when you fetch elements from the zerovec they just get converted back.

B

So users don't actually see this internal type,.

B

They just see these nice ... that the normal type they wanted, which was u32.

B

On the other hand, varzerovec works by using kind of a special data format where all the data is.

B

Splatted into a single contiguous buffer and the metadata header tells you where to find the data.

B

Similar to zerovec, it requires varule types which are variable length; types that can.

B

Be encoded in the same kind of unaligned little endian format., for example. If you.

B

Want to make a varzerovec of strings and have it represent the same pair of strings from before,.

B

It would look something like this in memory., the first four bytes says that marks .., that it.

B

Contains two elements, and then the next two bytes kind of are the internal offsets of each element..

B

When indexing varzerovec will do the appropriate calculations to return the correct pointer.

B

Zerovec and varzerovec support some pretty complex types and they can be nested deeper.

B

For example, we've already seen that you can use zerovec with a fixed size, type and varzerovec.

B

With a simple variable, size type like str,, but you can represent types like a nested vector of u32s,.

B

A nested vector of strings, and even more nested vector strings.. We also support maps which are.

B

Internally, built out of sorted zerovecs., someone's, actually working on a hashmap, that's zero copy.

B

Using perfect hashing., that's a rather cool thing. If anyone wants to look at it, later.

B

You can also use custom types in these structs. manually. Implementing the ule traits is tricky and.

B

Also unsafe, so we have some convenient proc macros that do the hard work for you.. For example, if you.

B

Want a zerovec contains this kind of custom date type. You can slap this zerovec::make_ule macro on.

B

It which generates an internal ule type like this.. You don't really need to care much about it, but it.

B

Generates this, and the repr(packed) means that it has no alignment requirements. .. the repr(packed).

B

*Plus* the fact that all of the fields have no alignment requirements means that it has.

B

No alignment requirements., because all the fields don't care about endianness,. It also.

B

Doesn't care about endianness. if you substitute the actual ule types, it looks something like this,.

B

But again the user doesn't need to care about. What's going on around the hood, it's just now.

B

Hooked up and they can use this directly. if you want like a custom, monthinfo type,.

B

Then you can slap, make varule on it and then you, you have to use the corresponding.

B

Varule type in the zerovec., this is slightly different from before,.

B

Since, instead of using the normal type, you have to use the generated varule type in varzerovec.. There are a couple reasons behind this. You can ask me, after the talk., but.

B

Yeah, it's just slightly different.. This generates an unsized type like this,. The last field is an.

B

Unsized string, which means this is a dynamically sized type. in rust. If you do this, then this.

B

Type can only exist behind a pointer. and it also generates convenient conversion functions between.

B

The two so that you can kind of cheaply view the latter type as the former type when you need it.

B

Zerovec tends to deserialize around 10 times faster than regular vectors. varzerovec's exact.

B

Wins are far more variable, since they strongly depend on the size of the elements,. The complexity.

B

Of the vectors, and whether you're using ... kind of serde's support for zero copy, deserialization.

B

As far as you can take it, without using these crates., while not shown here, the performance of.

B

Reading from the zerovec types is slightly slower than regular vectors... for individual reads: it.

B

Rarely matter,s it tends to be 20% slower. To sum all of the elements in a zerovec., if you're trying.

B

To read everything: that's one of the that's one of the heaviest operations you can perform. So it's.

B

More of a worst case scenario., otherwise, basically we don't notice it. in icu4x. All of this turns out.

B

To be a net win, shane's talk had some nice graphs showing serialization costs, basically go away.

B

Using zerovec we've managed to make all of an icu4x's non-experimental data work in zero copy.

B

This covers more than 50 different types of data covering a lot of different data. Keys. icu4x can.

B

Guarantee that there will never be allocations when loading data ..., which is pretty good!.

B

Would be remiss of me to talk about zero copy deserialization without mentioning rkyv.

B

Rkyv's, a crate by david koloski;, it came out a couple years: ago. ot provides a lot.

B

Of the same benefits using a rather different kind of holistic approach., it's really cool,.

B

It's super robust and it's worth checking out if you're working in this space.

B

The main reason why we don't use rkyv is that rkyv is its own serialization format.

B

We want to use serde,, we want to have control over the serialization format,.

B

Because we want to be very flexible around data., we want to support json, postcard, bincode,.

B

Xml, if you want, anything. rkyv's, derives also end up creating their own parallel versions of.

B

The types ... for everything, so it's far trickier to write code that works both with rkyv and serde,.

B

Especially if you want to keep the code size low, which we care about quite a bit.

B

But for a lot of use, cases archive is great and you should check it. Out.

B

All right, we solved one limitation.. What's next?.

B

Another limitation is the pervasive lifetimes. It introduces to the program.

B

For example, the date symbol data might look something like this.

B

With a 'data lifetime, referring to where the data was loaded from .. context is just a type.

B

Kind of that covers whether it's long, short, medium, whatever., the type doing the formatting.

B

Might then look something like this, and it'll contain a loaded copy of the data and it's using.

B

The lifetime to mean the same thing ... might have methods that look like this.. I have more ... when.

B

You construct it, you need to like also know about the 'data lifetime, and then it might get used like.

B

This ..., where you have to hold on to the data provider without moving it for the duration of.

B

Time, you're formatting data., you end up with like a lot of 'data lifetimes everywhere., a lot.

B

Besides, being annoying, lifetimes, prevent us from using more complex data management.

B

Strategies., for example, we might want to have a data provider that, let's us say dynamically,.

B

Load data from the network or disk.: it only wants to hold on to loaded data as long as it's needed,.

B

Which can be a variable period of time, not known at compile? Time. individual data users may even.

B

Need it for different lifetimes,, for example, the user might load data for a date formatter, a.

B

Different part of the code base might also want a date formatter,. Then they might need to load data.

B

For a number formatter,, some other formatters,, all from different areas of the code base that.

B

Don't know too much about each other.tthey could then go out of scope at different times,. Perhaps.

B

With one of the date formatters going out first,, but then the other one still needing the data,.

B

Followed by some other stuff,, followed by the final date, formatter going away.

B

We want the data to stay as long as it's needed, but not longer, and we don't necessarily know.

B

These lifetimes at compile time. it's even more interesting if we want to add, like a cache to.

B

This., the .. you know .. hold on to the last 10 bits of data, even if no one's using it right, now.

B

Most of these are strategies that you typically implement in both rust and c++using. Some form of.

B

Reference counting .. but hold on, reference counting is typically an *alternative* to.

B

Lifetimes., you can use reference counting to *replace* code. That's got lifetimes,.

B

But then the .. lifetimes have to go away. zero copy deserialization, as we've seen so.

B

Far *needs* the lifetimes .. it mandates them because it needs to be able to mark.

B

That it's holding references to a subset of some other type.. We can't just switch over.

B

In rust, using dyn to go from compile time types to runtime types is called "type erasure".

B

What we needed here is a way to go from compile time, lifetimes, that is regular, lifetimes,.

B

To runtime lifetimes, that is,, basically things like rc and arc., we need "lifetime erasure". enter.

B

Yoke. with yoke, you can mark zero copy data like cow, or a more complex object, and say that it's.

B

Borrowing from some data source, which we usually call a "cart" by kind of "yoking" them together.

B

The 'static lifetime is not actually accurate,, it's just the cleanest way.

B

To represent an erased lifetime .., but the end result of this is this is a type that doesn't.

B

Impose any lifetime requirements outside of its use. there's no 'data or anything,.

B

You can use it and you don't need lifetimes. Everywhere.

B

The way you use it is that you first kind of load data from somewhere,, perhaps into an rc,, to enable.

B

The life cycle, flexibility- we were talking about earlier., you might do some fancy. Caching thing,.

B

Who knows., then you construct a yoke out of it by calling yoke::attach_to_cart() with a.

B

Closure that deserializes the data from the cart., this object can be moved. Around,, cloned,, etc,.

B

It will safely hold on to the cart [until]. It goes out of scope.. You can perform all your caching.

B

If you want .. and finally,, you can kind of get access to the inner type by calling .get() on it.

B

It doesn't deref because get() has an interesting signature, but now you have this type: ... you're, no.

B

Longer your yoke is no longer tied to the compile time lifetime of data,. You can return your yoke.

B

You can do whatever, um and it's all fine.. The magic here is behind the signature of get().

B

The full generic signature is rather gnarly,, but here's what the signature of get() for a yoke.

B

Of datesymbols looks like it basically shortens the inner lifetime to that of the local borrow;.

B

You can see that there are two 'a s: there,, there's one of the borrow, and then there's the one of the inner reference, and that keeps it safe, because ...

B

... that keeps it safe, because this pins the lifetime of the yoke to .. as much as you're.

B

Using the get() for, and as for that period of time, all those lifetimes are valid and.

B

Then, after that, you can move the yoke again and you can call get() again and get new lifetimes.

B

Yoke can only wrap types implementing a trait called yokeble.. It's tricky to manually, implement.

B

But there's an easy custom derive available.. Unlike most custom drives, it doesn't even.

B

Require the struct's fields to implement the trait,, so it tends to be less kind of infectious.

B

Than most custom derives in a code base where you typically just have to slap them on everywhere.

B

There are a couple edge cases around complex generics, but most of the time it just works.

B

Yoke supports some more advanced operations too.. You can kind of.

B

Project a yoke down to the borrowed data contained in it to get a different yoke.

B

That's pointing to a subset of the data,. If you want to kind of split things, up.

B

You can even copy and write mutate local data inside a yoke, which feels kind of ..

B

It feels like it shouldn't be allowed,, but it does work,. It is safe. the the lifetime.

B

Machinery.., the trait bound machinery required to make that safe is very scary, but it is safe.

B

Going back to the code we had earlier, all this put together means that the.

B

Core data type still has a lifetime,, but then the lifetime's gone everywhere, else.

B

And you can kind of pass around objects like dateformatter, like normal owned objects,.

B

Because they don't contain any lifetime, so they're not really bound to any scope.

B

So how does this work under the hood ...? Well, it's a bit complicated. users of the crate. Don't really.

B

Need to worry too much about this due to the custom derive .., it just kind of works,, but if you.

B

Peek under the hood you'll get to see some very .. gnarly things with some weird trait bounds and.

B

Lifetimes. yoke kind of ... bullies the compiler into letting it transform lifetimes. This way and.

B

Letting it know that the lifetimes are safe to transmute. the process of developing yoke involved.

B

Us discovering around 12, complicated compiler bugs. properly explaining yoke would require me.

B

To say confusing rust jargon on like "covariant lifetime" and "higher, ranked trait bound", and I'm.

B

Not going to inflict that upon you all today., I do have a more in-depth explanation up on my blog,.

B

Which I'll link at the end of the slides, and I'm happy to talk about this after the presentation.

B

All right. so far, we've made zero copy, deserialization work for more things,.

B

We've made it more pleasant to work with.. Ultimately, it still involves deserialization.

B

What if we could get rid of that entirely. wait ... what? that doesn't make any sense.

B

At all, how can you avoid deserialization during data loading.? Why would you even.

B

Want to? the whole point is to load the data,. That's what deserialization does.

B

So, stepping back a bit with libraries like icu4x, you need to load a lot of data. that.

B

Data can be loaded from memory in many different ways, but deserialization is the most flexible.

B

But sometimes you know the data beforehand when you're building your application., if you care.

B

About startup times, you might not want to pay the deserialization costs,, even if they're not.

B

As slow because of zero copy., you could construct that data manually in rust, .. and just stick. It.

B

In a static, but anytime that data changes, you have to change your rust code, manually.

B

One of icu4x's users is firefox, browsers, really care about startup times .. startup times,.

B

Page load times, all this like initial stuff people care about.. They may need to load.

B

Internationalization data dynamically, so they still need our pluggable data loading stuff,.

B

But there's also a bunch of stuff they just always need,. They just know that they.

B

Need this ..internationalization data, all the time., it's not great to have that have to pay.

B

That deserialization cost on startup, a time where slowness is so noticeable to the user.

B

The thing you kind of want to do is deserialize a bunch of data into a static.

B

It would be really cool if this worked .... It doesn't,, because no deserialization library in.

B

Rust is const and rust. Const support is not far ahead enough to be able to do this. .., though it.

B

Might be in like a year?, we could write a very unsafe serde, deserialize, ... kind of something.

B

That looks like serde sterilized, but is very unsafe,, and it operates on fully trusted data.

B

And skips all validation,, but you still kind of need to scan the data to pick up all the pointers.

B

And load them., it's generally very tricky to also rely on the data always being trusted.

B

What we need is a serialization format for rust.

B

That needs no validation, needs no scanning to find all the pointers, and can be loaded safely.

B

This sounds like an impossible set of goals ..., but is it?. There's such a format that already.

B

Exists and has existed for years, .. rust code in statics. loading compiled rust code in statics.

B

Is just a pointer read since the format on disk is exactly the format rust uses to store these.

B

Objects in memory., if we can serialize rust values to rust, code, that can work inside a static,, we're.

B

Good to go., we've basically made the rust compiler itself a part of the serialization process,.

B

Because you're just converting to rust code and the compiler is converting to data that.

B

Just can be loaded for you when you compile your binary. enter. Databake. databake is the.

B

Crate we designed for this purpose. it contains a trait called bake with the following signature.

B

Basically, values of types that are bake can be serialized to rust code that compiles to those.

B

Same values. databake doesn't require that code to work in rust, const .. in rust's, const contexts,.

B

But we do require that in icu4x, because we plan on using that in a static.

B

It comes with this simple custom derive that can be slapped onto most types, provided that most.

B

Fields implement the trait., you do have to tell it the path of the type though,.

B

It's a bit annoying, but it's generating rust code. So it needs to know where it lives.

B

Calling bake will generate code that looks ... something kind of like this.

B

With the individual fields filled in by the bake implementations of those types.

B

Here's a real world example of the output for the data we use to format numbers for bangla.

B

Our data generation code needs to do a bunch of work to wrap this and hook it up to the.

B

Data loading machinery,, but most of the data big output in icu4x looks kind of like this.. It's.

B

Like .. moderately readable most of the time .., but it's not supposed to really be read. icu4x's.

B

Ci builds test data in around 20 locales for everything we support .. just for test purposes.

B

This data turns out to be a 3.9 megabyte blob in postcard ... Well, it's 20 megabytes of json.

B

When using databake, it turns into 32 megabytes of rust, code. code!. This is large enough that when we.

B

First, did this: it crashed rust-analyzer .. and we had to kind of like opt it out or do something so.

B

That rust-analyzer wouldn't go looking for it.. This data is ultimately code, and the optimizer.

B

Can get rid of dead code,, so it's actually kind of hard to measure the impact on an actual binary,.

B

Because all of this depends on how exactly you're, using it ... the compiler will just get rid of.

B

Things, it doesn't think you're using., it's around .. half the size of the postcard files from some.

B

Like relatively unscientific, attempts at measuring this. take that number with a grain of salt,.

B

It's really hard to get an accurate measurement.. An additional win from using databake is that if.

B

You're only using databake,, your binary doesn't need to include any deserialization code,.

B

So you get another size win there. as shane mentioned earlier. We really care about code, size,.

B

We want to be able to like run on embedded platforms that don't have much space at all.

B

Here's similar data for just the stuff need to format, dates and times for the same 20.

B

Locales ... so just kind of more of a subset to the data that we've been talking about so far.

B

So, with databake we've given users the option to thoroughly.

B

Yeet deserialization overhead within icu4x.

B

All in all, we've achieved three goals with three different crates.

B

Firstly, we've made deserialization completely zero copy with the zerovec crate.

B

We've also made the zero copy deserialization pleasant to work with, with the yoke crate.

B

Finally, we've made deserialization unnecessary in certain situations with the databake crate.

B

To quote one of the greatest christmas movies: "and, when alexander saw the.

B

Breadth of his work, he wept for there were no copies left to zero" thank you.

A

Thank you manish. Do we have any questions about cows and baking and related topics.

B

So the question was how much how much of this work here was stuff, that rust is really good. At,.

B

And how much were the crates, like stuff, we invented in the crates. the concept of zero copy.

B

Deserialization is one that has predated rust, .. people do it in c++ .., it's trickier to be safe in.

B

It for in c++,, but people do do it.. So that's been around for a while,. It's been around in rust for a.

B

While too .. and ..., there are already crates that achieved some of the things we want to do. Here,.

B

Just in a very different way that might not work for us. and rust, specifically, is very.

B

Good around zero copy deserialization because lifetimes mean that it's very hard to mess it up.

B

Normally you either have to do lots of reference counting or whatever to be safe, and that too is.

B

Often tricky to get right, .., it's easy to still mess it up in c++ in stuff. I've seen,.

B

And with rust, like because of the lifetimes, you can be very, very ... loose about how.

B

You're dealing with data because you know the compiler will catch you. If you do anything wrong.

B

S, you don't have to be, you don't have to be too cautious, which is good.

B

So zerovec is a concept ... that kind of that kind of way of solving zero deserialization problems.

B

Is a very old established, way, zerovec is a way of fitting that into rust's model, and serde's model.

B

It is rust specific in that way,, but the ideas of "oh. You need to kind of have a metadata thing for.

B

Indirection", "you need to deal with alignment and endian-ness" ... These are old, concepts.

B

The yoke thing .. it is extremely rust specific because it is .. kind of ...

B

It's solving a problem. You can only really have in languages like rust,, where lifetimes are a thing.

B

Because in other languages, you would either probably have just designed it to be always.

B

Reference counted, or always cached,, like very pervasively .., which would make it less ... less.

B

Generally useful but safe, or you would just be using references and just be like "be, careful.

B

Don't mess this up",, .. or you'd have a gc., so yoke is very rust specific,, but this is kind of us.

B

Extending the thing of "zero copy deserialization in rust lets you be much .. much looser in what.

B

You're doing, much more flexible, because you're not worried about messing things, up" and yoke is.

B

Kind of pushing the boundary there. and databake is something ... You could do databake for c++,.

B

C++ has constexpr, as well. rust only recently got a very flexible constexpr,. It's been in the works.

B

For ages,, it's been on nightly for a while,. Only recently has it been to a point where you could.

B

Do a lot of the things here., so that is .... I haven't really seen anything like databake before,.

B

But that's not really rust, specific .., that's just languages with constexpr .... There aren't that.

B

Many. especially compiled languages that compile to things without a runtime like rust and c++.

B

That's already a small set and out of them rust and c++ both have constexpr., but I haven't seen.

B

Stuff like this before .., so that is a newer idea, not necessarily specific to rust.. It would be.

B

Cool for someone to do this in c++. custom derives and rust, make life so much easier,.

B

Because for c++ .. you could use like a clang plugin or something to do the same thing,.

B

But custom derives mean that we can kind of just do this .... It's just.

B

Part of the code .., you just slap these attributes around and you're kind of done.

A

Is it something or we which would be included in standard rust.

B

The question is, would could yolk be included in standard rust, ... yoke itself is kind of specific ...

B

This has to do with yoke's implementation, but ... being able to talk about lifetime variants.

B

In trait bounds, in a way that doesn't require me to do a custom derive and stuff like.

B

That ... would be kind of nice. in the long run, having magic, compiler traits that are.

B

Implemented on ... that, let you kind of sniff the lifetime variants of a type would be very cool.

B

Some of yoke's complexities are made simpler by .. higher ... generic associated types.

B

Yoke does not use generic associated types, because we wrote it before that was stable,, but.

B

Those things do help., not that much,, unless also the compiler came up with ways of talking about.

B

Lifetime variants- that's useful in those contexts., so ... yeah yoke itself- I don't see ever making it.

B

Into the standard library., none of the crates here, I think, would make it into the standard library,.

B

The standard library ... tries very very hard not to be too niche, um and mostly .. takes things that.

B

Really benefit from being common across ecosystems and yoke. .. doesn't ..., there are. There are other.

B

Crates that do self-referential things like yoke, ..., there's ouroboros, which is another good one.

B

It is not .., it doesn't have the same goals and it doesn't .... It would not be useful for us here,.

B

But if you're doing .., if you're doing things like that,.

B

That's another crate to look at.. There are a bunch of different strategies for.

B

Doing this, and they have different pros and cons. yoke works really well for what we.

B

Want to do,, which is zero copy, ... erase lifetimes in zero copy deserialization.

B

I think there's a question back there.

B

Oh! the question was how what's the compile time with the macro generating 32 megabytes of rust.

B

Code. we don't. The macro does not generate 32 megabytes of rust code.. The macro generates a.

B

Small amount of rust code, that can be called, and when you call that rust code with all of its kind.

B

Of functional dependencies,, with all the data ..., we already have a lot of data part of this.

B

Is like ..., we have a large amount of data. The number that was on the slide was some number of.

B

Megabytes of ... it was like a couple megabytes of I don't remember anymore., but it's a large.

B

Amount of data ... yeah- it was like 300 kilobytes of postcard. right. and that's all the data in a.

B

Very compact format ... and that's for 20 locales.. So some of that's just. We have a lot of data.

B

We don't generate that data at compile, time. .... The way you use icu4x is once you know what data.

B

You need you run this tool called datagen.. You say like — there's the static data slicing stuff.

B

Shane talked about earlier, so you can kind of give it a binary and give it like. "I want english and.

B

French., english, us and french.", or whatever you like, give it a bunch of locales and it'll, generate.

B

Data. and then you also tell it what kind of data you want ... and it can be a postcard blob. It can.

B

Be a json thing, like set of folders,. There are a bunch of different ways to ask it for data., one.

B

Of those options is databake., so basically, you run it once,. You get some rust code,.

B

You make that a part of your build tool chain ... you make that .... You put that in a crate.

B

Or put that in a module or put that somewhere, and then you only need to update that stuff.

B

When you're generating new data., which should not be that often .. like you, might need it if.

B

You're, adding a locale or you might need, if you want to update to the latest cldr data.

B

Or if you're kind of patching data with your own kind of changes. so yeah, if datagen .. the.

B

Data generation step is not super fast and we have not ever tried to really optimize it. Much.

B

Because it's not supposed to be run that often., otherwise, yes, it would take a little while.. It.

B

Wouldn't actually be that slow though .. like it's still something we could, we could run it in the.

B

Build process and like you would just notice the build being kind of slow,, but it would not be like.

B

"Oh the build is taking two hours",. It would just be like "oh. Why is the bill taking five minutes".

B

The question was the performance difference between zerovec and a normal allocated.

B

String .. for a slice of small strings.. My understanding is that the slice of small strings.

B

Is still like, faster,, it's not 10 times faster, like zerovec,, but it is more like ... I think two.

B

Times faster for that kind of case., it actually yeah .. for like a small one if it gets bigger.

B

Then uh some suddenly it becomes more expensive.. It also kind of depends on the allocator you're.

B

Using and some other stuff. yeah., I don't think I've seen ... zerovec has never like .. zerovec.

B

Is never slower than the original thing to deserialize. Of course it can be slower to read,.

B

So the actual net cost is harder to calculate., but it tends to be ... Like I said earlier, it.

B

Tends to be .., it's almost always .... As far as I can tell it's always been a win in icu4x.

B

The question is: does yoke work with multithreaded code. yeah there's nothing in yoke. That assumes.

B

Multi-Threading., I was using it with an rc here. You could use an arc in icu4x.

B

We kind of encapsulate all of that., like icu4x users, don't need to care about yoke. We have.

B

Some other types that kind of encapsulate all that., so icu4x, actually ... for that we.

B

Use a cargo feature so that you can .. turn on sync versus non-sync reference, counting,.

B

Because we ..., but we technically could make that more pluggable.. It just .. generically abstracting.

B

Over sync and non-sync is very annoying so and it would be pervasive, so we didn't do it that way.

A

Any more questions.

A

Serde be extended with some of this, like.

A

In the machinery from the zerovec and make it easier, so you just say, borrow the normal way.

B

It could .., I think some of this becomes .... Now you've made serde more opinionated., because serde.

B

Is very happy to have these kinds of things work;. It works very nicely with these ... It wasn't hard.

B

Making this work with serde .., you just have some serialize and deserialize impls, and we can make.

B

Up ... we've made a lot of opinionated choices. Here;, I suspect serde would not want to take this,.

B

And I think also someone could write something similar that has different trade-offs,.

B

Used for a different purpose., for example, one of the trade-offs we've made is,. We really do not.

B

Care about serialization, performance. zerovecs are cheap to make, but varzerovecs ... they're actually.

B

Not that slow to make,, but they are definitely much slower, because of the metadata,. So you.

B

Can't really manipulate them like normal vectors and expect the same kind of performance, behavior.

B

You can still construct an entire air varzerovec from an existing vector pretty cheaply. But if you.

B

Want to do stuff like ..., if you want to ..., if you want to like push and pull and insert and pop,.

B

That's gonna stack up really quickly. and that's the trade-off we've made and we're fine with it,.

B

Because we really don't care about how slow serialization is., but someone else might and.

B

Then then they can design a different crate.. We actually also ..., technically the the format.

B

Used is pluggable now,, so technically speaking, you could actually plug in a different format.

B

With different trade-offs into varzerovec but ..., you could also just write your own crate.

A

Any more questions.