.Net Foundation On .NET, 19 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Walking Down the Memory Lane with Bing – Part 2

Description

Get started with .net
https://dotnet.microsoft.com?WT.mc_id=ondotnet-c9-cxa

Memory Management and garbage collection in .NET
https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/memory-management-and-gc?WT.mc_id=ondotnet-c9-cxa

In this episode, Maoni (CLR) and Mukul (Bing) talk about frozen objects in .NET Core 3.0 and how you can try it out in your scenario.

Follow Maoni and Mukul on Twitter:
https://twitter.com/maoni0
https://twitter.com/mjsabby

A

Hi, we're back with mucku to talk about frozen objects in this episode of on.net.

A

Hi friends. welcome to the next episode of the on.net show.

A

I'm moni, the owner of the.net core garbage collector.

A

Hey,, I'm mucku,. I work on the bing performance scene.

B

So mucku, last time we.

A

Were going to talk about frozen, objects,, right?.

A

That's one of the features that you are using during your startup stage., so frozen objects had existed in clr for a long time.

A

I mean I'm talking like it started to appear in.

A

Clr 2.0 not.net core,, but.net 2.0.

B

.Net 2.0. yeah., so why did we have this feature?.

A

Well,, if you observe that some objects are immutable.

A

They don't change after you create them, and very likely that they would live throughout the process lifetime..

A

So it would be nice if we could save the work.

A

To have to reprocess them every time we say, do a gen, 2, gc, right?.

A

Because a large part of your gentoo consists of these immutable objects,.

A

We want to optimize that., so we introduced this feature called frozen. Objects.

A

In the gc's perspective, they actually live on segments that we call read-only segment.

B

A

So normal gc heave lives on read-write segments because you modify them., but read-only segments are for users to put their frozen objects. On.

A

We use this on frozen strings..

A

I think that's the only object type. We actually use this on.

A

So, can you tell me what, in your scenario that made you want to use this feature?.

B

Yeah., when I found out.

B

About frozen strings like in [inaudible], two days,.

B

Obviously much later, and then I tried them and I was like,.

B

Oh, it'd be great to try them in the next versions of dyna framework.. I think has been if deaf out or has been disabled in the actual.net frameworks after 2.0.

B

But in.net core,, we sort of were like, hey, we'd like to use it in the use case, for us is that we have these like any files,.

B

There are these large called them. In-Memory database read-only.

B

Right., it's essentially like configuration data,, other sorts of things that we never change or we'll change either after a process is torn down,, essentially read-only,, and I was thinking it be great if we could actually have all of these objects which have to be read from this any file as strings, parse them,. Do all this work at startup., essentially to make them never be able to change them.

B

So I was thinking, wouldn't it be cool if we could use this,, and this can get large like multi-gigabyte.

A

Right. yeah., it would be nice if gc doesn't need to reprocess them.

B

Reprocess them, yeah., but the cost of read-only objects or read-only.

A

Segments in gc is very little compared to read and write ones,.

A

Because all we do is if we observe that there are in range,.

A

In range means, if they're in the gc heap range, and we have a lowest and highest address for the gc heap and then if a segment of read-only objects are frozen, objects happens to land. In this range, we have to basically materialize the gc bookkeeping for.

A

It because you might be using the mark beds on its own right, nods.

A

When gc actually happens,, all we do is, we would naturally mark there and then you just clear the mark beds..

A

That's all we do, is it's very cheap compare..

A

So what you just described sounds like the perfect scenario for using the frozen objects.

A

of course,. If you want to use this feature, you have to somehow construct these objects.

A

They can't appear in thin air., so you have to construct to add and historically,.

A

We just constructed them in the runtime itself, right?

A

runtime would go, make frozen strings somehow construct them.

B

And [inaudible], and I put them in.

A

The image and then when the clrs load, that we'll load them up.

A

Right. basically, all they do is they register that memory region as a read-only segment to the gc, right?.

A

Correct., but you did those completely without the runtime changes and it's all in managed. Code.

A

B

So that's very cool., so let's talk about that.

B

yeah., so in clr2 or these, when you have this thing, you would basically be only be able to do strings. I had to use engine which is somewhat limiting.

B

So for our use case, neither of these came.

B

we're running on.net core 3 and we want to be able.

B

To actually use these for all these different kinds of objects.

B

They have lists in them. strings in them,. All sorts of complicated objects., so actually creating them is sort of interesting.

B

The code is actually a managed nuget package,.

B

And you can add this to your application.

A

That's right., anybody can go and download. anybody can go and add downloaded added.

B

To their application- and essentially you think about this as like a deployment or a built post, build set,.

B

You've got your application and you want to use..

B

You want to make these frozen. Objects. right., you run your application,, you point the frozen..

B

We call this a frozen objects. Utilizer. you say, hey,. I have this object and I want to just serialize this to disk as a frozen segment.

B

Another cycle, another step in your deployment.

B

another step in your deployment process.. You basically do this and essentially voila, you have these objects from non-contiguous memory on the gc heap.

B

Yeah., you get contiguous, essentially objects on disk, which is really cool because you get locality, benefits and all that stuff from these frozen objects..

B

So I can now show the sort of how we actually serialize this on disk.

B

Yeah., let's do that. so here basically, we're going to start with like this is an object and you can see that the object has three references and some long fields.

B

We want to start serializing this object.

B

A thing to note is: if you have this object and has other references, we have to take care of storing the information. What we call that are not live memory, references.

B

They're actually going to be cookies, that we store on disk.

B

So, for example, that method table pointer has to be stored in a persistent in some interesting way..

B

So what we start is we have this, and I'm going to show on the upper right corner is basically like a queue and where's the state and action being done, whereas we enqueue and dequeue things,- and this is sort of like the first step in the serializer.

B

We go,, we find these references and we say,.

B

Hey, we found object,, we dequeue object, one which was the root object that you basically started with, and we added object, four which, if you look is from you, have to go. Look it from the left side. Come that object; four,, object, two and object three.

A

Sorry to interrupt., so that means, if I have multiple objects, I want to serialize,.

A

I can just make like an array that points to all of them.

A

B

So you start from the root objects. right.. We always start from the root object, but if you wanted to, for example, have thousands of objects,, you could just make an object. Array. exactly, yeah. with all those and then give the object array as the root object.

A

B

Exactly. so here,.

B

We basically have a few things going on.

B

at the bottom,. You have basically the data that's been written to disk., so object. One has been written to disk, but you'll see that thing called reserved and there's a dictionary there.

B

So what we needed to make sure is,, when you have the same reference, that you've encountered,, we can't actually put this, those references have to be have referential identity., so we can actually enqueue and write multiple objects of the same instance onto disk., so that dictionary helps with that.

B

also with the ability to point back at.

B

References that you have encountered as you do the object graph.

B

So here,, basically we go and say this object, which.

B

Is its size is 64.? We put that.

B

Then we've discovered object, four,.

B

Object, two, and object three in that order, and we've enqueued that in our queue.

B

note that knowing its size, it's overall size that we've encountered is actually very helpful,, because then we can do all this in one pass. right. to talk about the one pass thing..

B

This is not actually how if the gc would do it, it would do it differently.. It would actually go do what you would do normally mark all the objects.

B

yeah. then go back again on them. yeah.. We decided not to do that because we want to do this. A 100 percent outside the runtime in managed code.

A

Exactly, yeah. we're not mutating your objects. of course. right.. I mean we want people's feedback on this feature. And-.

B

It's experimental. exactly., but if this is.

A

Very helpful to a lot of people and scenarios. Then we could,.

A

Of course, consider putting it as a long-term feature.

B

As a feature, in writing.- right., so we basically go find the next object, serialize, that to disk and again you'll notice that the reserved region hasn't changed,, because we have not actually encountered another object with object. References which is the interesting part here.

B

Then we get to the next object. here,. We have still not encountered any new object with object references..

B

So you see we haven't reserved more space for the next set of references. right.. Then we find an object again with more references and we go in and figure out, obviously,.

B

Write this object but then find the reserved stuff that is going to be there and coming in the future. As we serialize that.

B

right., then finally, we've arrived at the final object and now there's essentially nothing else to serialize..

B

So if you could imagine these objects could have been.

B

Anywhere in memory, but we put them together, a contiguous on disk.

B

Exactly., which is really helpful for [inaudible].

A

Right. yeah and all this is on github, right?.

B

All this is on github. github.com, microsoft-, slash, microsoft, slash frozen objects.

A

Right., it's a public repo.

B

A

It's all managed code. yeah. feel free to go check out the code. If you.

A

Want to the concrete code corresponds to the slide.

A

Yeah., let us know if there's things,.

B

Like there are limitations, for example,, we don't want to serialize things into ptr,, azure or things that may have file handles that are there.

B

So there's a few things that are there. but it'll be good to get us feedback to see what to improve.

A

Definitely require feedback from customers on this feature.

B

Yeah., so this is great.

A

So when we're talking about this speech- and we really want to see how can we make this faster?, so you talked about serialization, right?, so when you deserialize,, when the process starts out, we need to load frozen objects or just.

A

Whenever that registration time, you need to deserialize things into memory., that's where we really care about the [inaudible].

B

Exactly., so we did a bunch of work, as you can tell on tools.

B

We have decree and lookups. we're doing all this work. right., but serialization is much faster.

A

Right., so all we do is we have two kinds of fix: ops, right?.

A

There's the offsets that you put in for references.

A

B

Then there's the method table pointer, fix ops.

A

So of course, we're thinking like how can we make those to go faster??

A

So if you can imagine, if we had a preferred address,, we can load it there.

B

For frozen objects. right, yeah., then we don't have to write offsets,.

A

We just write the actual address.

B

That's one fixed op, gone. right and if we cannot manage.

A

To load or add the preferred address,, we just go to the fixed ops as normal anyway, right?.

A

Then, how do we avoid the fixed ops for methods able pointers?.

A

It turned out that in the runtime, there's no way to tell the type system, where to put the type info., you can give it a memory chunk and say, hey,, just put it here., which means if we can manage to again load off for the preferred address,.

A

We're already knew beforehand, where the method.

B

A

So we were so excited when we found that out, right?, but we're only happy for like a day.

B

Before the security people came and said,.

A

That, because these are live, pointers,.

B

The fact that you can call methods on objects, it can get, their security can become an issue.

A

Right. there is another concern which is the unpredictability,.

A

Which means, if you cannot manage to load up with a preferred address, then becomes slower.

A

correct.. But personally, I think that in the last part we talked about large objects or large pages.

A

And unpredictability there.

B

Right., but I think this is a different kind of.

A

Unpredictability because with large pages, you're going to have unpredictable results throughout the process.

A

Whereas with this,, it's like, at this time, you-.

B

This is quick, phase.

A

Right. exactly,, it's very limited in scope, right?.

A

Yeah., so we could tell you, entry event log and what not just to tell you that, hey,, we didn't manage load a preferred address, so you know why that part was slower.

A

Yeah., it's different from.

B

The large object case or large pages case., I'm sorry., where the idea of running your app at 10 percent, slower performance, maybe significantly worse, is because you're running it for a long time versus.

A

A discreet phase like startups. so no,, I completely agree. exactly, yeah., so this is actually one of the other reasons why we really wanted your feedback. On, right?, you can. Let us know what you think of this unpredictabilities.

A

Is that tolerable, to you or not?, tell us why.

A

We're going to put our on twitter links, below.

A

so feel free to connect with us on twitter and tell us what you think.. We really appreciate your help. yeah. we've been talking about onsite per features.

A

So clearly, you care about.

A

With that,, I think diagnostic ability is another thing.

A

That you have to care about a lot.

B

Because when things run slower, you want to know why, right?.

A

So we're going to wrap up for this episode.

A

next episode,. Let's talk about the diagnostics that we worked on together for core 3.0.

A

Cool. sounds good.

B

All right. bye.

A