Rust Programming Language RFC 2229 Working Group, 10 Jan 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: blitzerr closure upvar tys

Description

Discussing how to refactor the way closure upvar types are represented with @blitzerr.

A

B

Paying corporate otaku, where we look that we store the actual path to the valuable to the member of the composite data that is actually being used as an awkward for the.

B

Alright, yeah I can see your screen.

C

Okay, what do you see here? You see nice, you see her no I.

B

Don't see a prompt I see Athena like.

C

B

That I mean I, don't see the dollarfranc.

C

B

Nice, what does that see now mean it's.

C

The name of a computer- in this case, okay, let's see.

C

C

C

Call this closure a part time: okay,.

B

C

Wait so the idea was we're gonna, try to get started on justice and then trying to get started on refactoring the up bar types so that, instead of having one type parameter for up bar, we have a tool, and we hope that this is going to work.

B

C

The only hindrance.

B

We had was that the poeple that I checked later or.

A

B

A

B

That problem you have to overcome, but.

C

So, just to make sure we're on the same page the why we're doing this is that we're not? We want to change it so that, in order to figure out how many type parameters a closure type has, we don't have to know how many things it captures, because we want to figure out how many things it captures during type checking and not before here I see yeah.

C

The reason I mean that lets us construct closure types and wait. Now this is sort of a closure. Type is sort of conceptually a struct like this, and so we're basically talking about doing is changing this like so so now. You still have this, basically all the same information, but the number of type renders it's not dependent, and the number of our parties will always be the same, but the arity of this tuple is dependent, but you think it's not a problem.

C

You, of course, couldn't.

B

Socially write this.

C

Struct definition rust because, like these, this is really a type parameter like yeah, so one of the things I was saying: I think we can maybe just cheap. This is not really a struct and a compiler, so I think we can actually cheat. But if we didn't want to cheat I think we could also do this, that is to say, we could sort of change it.

C

So they've had a single field of type tuple and then everywhere there are parts in the compiler which refer to fields of the closure to mean the up bar, so might be, like Clojure dot, zero to get the zero or not one to get the zero apart and they would become closure dot, zero dot one you know so that we go through the tuple and go to the first field of the tuple um I think we can avoid doing that and if we can I would at least for the first round prefer to do so.

C

Okay, so it's sort of cheap as I said, it's not actually something you could make in last.

A

B

Ask a question so in this structure the.

C

First fields, our lifetimes easy.

B

Row, EJ archetypes and then seek a CSR.

B

C

C

So CK is the type that we use to represent the closure time. You use a kind of random.

C

Basically, we use it's always an integer type, so it's either i8i 16 or 32, and if it's I hate that means function, can you function? So this is really like an internal thing. It's not your Evernote or sneeze. And similarly CS is the closure signature and it's always a function. Type. Nice.

C

So we're kind of packing a lot of information into this struct in the form of types, because it's cannot do that, but.

B

And so that's why the parity of the table will be.

B

C

So see you represents the closure up bars, I, see, there's always a cool.

C

B

C

Seems you know I'm just thinking we're gonna be able to get a regular this well, if we're gonna have to change, maybe we should I'm trying to decide if we're gonna be able to get away with.

C

With the up, for is those fields that have reference for some parts of this to blow I'm, not sure I, guess we'll find out yeah, it might be.

B

An alternative.

C

Might be to first, we factor the up farce and introduced the tuple, which we could do right like this.

C

Well, basically, we could, um you could start by leaving this part as it is, and just we factor this to be up for people.

C

See that means and then we'd have to.

C

We have to refactor all the things that reference, those fields to go to the big reference like food, zero, dot whatever.

C

But then we would be very easy for us to do the next step.

C

Okay, I'm, not sure. The only reason I'm hesitant is that maybe it's not so bad I think most code sort of doesn't tear like it, wouldn't be so hard to introduce this field. It's mostly I think the mirror there's some special taste code in the borrow checker, especially for Diagnostics it tries to identify access, needs to up bars and bad code would have to be adjusted I'm, not so bad I.

C

Guess we shouldn't see okay.

C

C

C

Try to remember how this works.

A

C

Right, so it's a little more back now, so the closure sucks, a closure type first of all, is to find.

C

Right so these are all the variants types that we have compiled when you see closures here, yeah.

B

C

Def ID- and it has one of these structs- and these are just a new typed list of type rendered like subst- is just the usual list of values for type number groups that you use elsewhere any provided, but we.

A

C

This struck to kind of give it some extra interpretation that these are specifically a list for closers and then there's some helper methods is.

B

Like a mnemonic or something or a substitution, substitution.

C

Because you substitute like, if you have something like this, then when you have, who of you, 32t gets substituted.

C

So so one of the things you can see that you have is this function apart eyes which says given a closure substance, you can get out the types of every up, far yeah right now. All that does is call split, which.

C

Which gives which basically takes the so split, takes the list of type parameters and breaks it up into categories right. The closure kind. This is that seek a upload, your signature and on a verbal length list of up bars, and then we just iterate through that list to get the list, then of all the upper are types you iterate through that list and down castes. So have you seen this type kind of work.

B

C

With either a type or region.

C

And so here we're just saying normally in the list of substitutions, we have kinds because they could be lifetime cookie types book. What we're saying the up part should always be.

C

So if we wanted to change this, basically it's gonna happen is instead of returning a.

C

Slice here we're gonna return type right.

C

Let's check, let's give a little more reading, so up fart eyes: this is the function. It gets back. The types of each bar I want to see you kind of where it's called from. Okay.

C

So because the reason I want to see is if we change this to be a tuple type and then down here we're going to basically have to check that it is a tuple typing and then iterate over its contents right.

C

So it's actually sort of similar right. So this function, closure signature yeah, gives back the signature of the closure, like which argument types in its return type and it works in a somewhat similar way. So it gets that type CS from the substitution.

C

It checks. It says. Yes, this is basically saying see us must be a function type, so fun, okay and then it extracts the signature from there and returned it, and otherwise he points um and we're gonna do something we would do something similar like here. We say, match up our minds, not sty, and you would expect ID to pull.

C

But I this. This basically gives us the same thing as the park lines used to be, but there's a kind of catch.

C

Yes, there's a slight catch, which is it's actually in the case that when we first create the closure, but when we first create an instance at the closure type during type checking, we don't know yet, for example, what kind it is or when its signature is or.

C

You would like it if we don't have to know how many up parts it has right. That's the whole point of this, so that means that actually, this type isn't always a tuple. Please it's either an inference variable or.

B

C

So actually like this is true also for the closure sig. So you can see here there's some comments that says only usable outside of an inference context, and we enforce that by taking this, as it happened like by taking it using some lifetime trip sizing. But the reason for that is because of this problem of we only you may not if you're, if you haven't fully completed typing.

B

C

So inference is where we figure out types and they're not explicitly written. The way it basically works is that you make these things called variables represent some type, not yet known and in some point of another of those variables get bound, which means that basically, we assignment number valor Knights would say this is the type that the you figured out the type now, but in the meantime you can use these verbal, so you might make a variable.

C

Usually I have denote these variables using a prologue like notation question mark X, you might say: okay well, make it tight, we're gonna call it question, mark X and then I can use that type in all kinds of places. So I could say like if you wrote this, you might say what is the type of X?

C

Well, it's question mark X, you know it's something, I don't know yet, but then somewhere along the line that will get unified or bound with you 32 and now we can resolve it and find out what did it get down to and that's that right, back phase that you modified. That's exactly what he does. Oh those two and takes all the typer won't replaces them their ultimate value.

C

C

Right so basically, this method, apartheid, some reason I'm talking about all this is if it's getting called during type checking before we have inferred everything that could be a problem, so I wanted to look over these fall sites and see whether whether they fall into that like where they happen in the pipeline. So this one is mere construction that comes way after ten shipping. That's fine.

C

This one is a little bit different, so.

C

So this some of these uses- this is part of trait and type check. This actually occurs during type checking, although it occurs very lazy to type checking so I think all types have been resolved by this period, um but there's something interesting about this, which is that a lot of the uses of up far ties would actually be fine to operate on a tuple like what this is doing. This particular function is walking down, but basically examining all the data that's inside of something right, so if you're in it too.

C

So during a closure, it goes through and recursively processes the type of each out far. But if you look right here, if you, if it encounters a to bolt it does the same thing with the types in the tuple right. So actually we could have.

C

Another method like up for are tied to pull that just returns the tuple of all up for our types and doesn't try to sort of dereference it and iterate it through the internal types and that we can do no matter what, because it might be an infant's variable, but we oh it's just a single type.

C

No matter what right, so, the problem, the reason that the infant variable causes a problem for the current signature is that we would have to sort of open up the to bulletin sense and we have to know what it's bound to. But if, if we just have something that says, give me well what is now or will eventually become the tuple of all of our types we could use that here and in a lot of other places yeah.

C

So similarly, here.

C

I think that here yeah we could also this could be as well the tuple um this debug info. This is way after typing friends way after similarly, this all this all this code comes way after tightening friends, layout, also I, believe layout, already sort of so so layout is what computes like what fields of a struct occur which offset and so on, yeah.

B

C

And you can see here that layout already fails if it encounters the type inference, Mayer, ball and fog, so this must be after detecting prints as well about that now, trait selection. This is a little bit different, but this is another. So this this code is saying: when is a closure copy or clone and a closure? Basically, some closure type C implements clone. If all the up flower types are cloneable, but we could again use the tuple type right today we sort of do that, but we could instead do something like.

C

And that's true, because two bulls are cloneable: if all our constituents employed in fact, you see the coat right here, so that's good. Similarly, I think this is just fine I think all of these cases can also be a people, and it's.

A

C

Looked at them before.

C

Similarly, here so this is computing.

C

What you know like well forming this conditions, so if you have types they have criteria for them to be valid, but like maybe this something like this, this is not actually true, but it might need it for this to be valid. Okay, hash must be true, um I think the actual hash set does not require this, but this is the code that we computed if they do and.

A

I think again, we.

C

A

Use it to Boyer.

C

Similarly, this I'm not mistaken.

C

You use the tip, oh.

C

Okay, they are this whispered again use the two ball.

C

I think they also comes after 10 minutes. This is pretty printing I.

C

Don't know this one's a little tricky because pretty printing happens at any time, so we have to, like you, probably have to adjust this code to be to account for the possibility that it doesn't know how many hours it has. Yet. The point where we are pretty printing is PPA us, it's the pretty print auxilary horrible.

C

No one's ever figured out a way to make it nice. Basically it basically prints types in a user readable way.

C

It's not I mean it's kind of hard for us to print things like a closure type anyway.

C

B

You do that question.

C

No, no, this is used when you're printing, an error message, do.

B

C

To like print print.

C

B

C

Can we change it easily I didn't get.

B

C

I think what we want to do in that case is.

B

C

I think we just have to make it able to check whether.

C

Basically, yeah I think it has to degrade gracefully.

C

So I think what I mean is I think it means that code means to look at the type and if it's an inference variable it just won't print anything.

C

B

Don't know when it gets called.

C

But it might there call if nothing else, you can get calling debug output, anytime yeah.

C

This is still me. This is still after type checking after type checking well. This is a case where we can use the too long after after after so I think it should be didn't.

A

C

That's right- and there is called during type checking, but only very late when it should already be going alone or in all the cases where it's sort of ambiguous I think we can make them almost all operate over the cool they'll be fun. Ok,.

A

C

Let's try it I guess we'll find out. So the first thing that needs to change I mean we'll start by changing this closure sub structure so we'll see a bar.

C

So basically, the where we were giving a slice we're now going to expect there to be a single argument.

C

And then here the up bar ties. So let's make any the new helper.

C

So this helper will give back to type as a tuple about flowers right.

C

So here I'm going to change this signature, also so that it can't be called during tightening prints.

C

So what we're gonna do now is.

C

This is giving back into a Raider okay, we'll keep that we're basically just copying this.

C

Is set up that other parts of this code have alright, so we'll call the up bar to pull time. It should be a tuple some types, and we can call this.

C

Okay, you with me so far, yeah.

B

C

C

C

Let's do this: let's do this in somewhat smaller steps. Sorry, what I'm doing here is reviewing my kit, mystery I'm, gonna leave this comment alone, we'll hold off on editing that comment until we actually know what a birth you doing, but what I'm thinking is what, if we leave everything the same but we'll just introduce the four tuple type helper.

C

Or no well we'll do this so.

C

What well we'll keep them well, keep it so that we have a kind of unknown number of of our substitutions and what will make a tuple type of them, and that way we can adjust. But people like sort of the code that uses this without independently from the rest, yeah.

B

Okay, we can yeah.

C

So I think to make a tuple type.

C

A

C

I make a tuple type.

C

It used to be called me: okay, it's called mk2. So let me show you this function, so there's a function here to make it to ball. It takes something that can be interned as a slice of types. Yes,.

C

So what we'll do is well.

C

So I've got this list of kinds. Basically, I have to.

C

Kind of convert it to a list of types.

C

Since we are serving that there are actually an archetype to not wait times and now, I think this should be enough.

A

C

So now I can call make two times see what happens. Theory this. This will not probably build I'm. Just gonna call xpy check this.

C

If I can my preferred way to build with the compilers refactoring things that I only have to check if it builds I, don't need to actually run another.

B

Just positive and.

C

Then and then maybe go back and run the tests and if I have a bug, bisection.

B

C

B

So quickly, for me, this I can click five to ten minutes.

C

B

C

You running Chuck, this is Jack.

C

Probably could.

C

But that's kind of what I'm saying he said: I like to use check I like to set up my work so that I can use check and just do type checking in part cause it's much faster. Oh.

C

And, of course, it's also nice, like it leads you through a sort of series of small changes that are independently processable.

C

um Okay, so now I have some of our substance.

C

C

I see okay, so this code is just trying to sort of be generic over two kinds of up fires: I didn't realize that generator subs was their own struct. That's going to be a little annoying we'll deal with that.

C

So the reason is what wonder why it's an integrator? Well, it makes sense. So the problem is imple. Iterator means that you have one iterator type, but there's actually sort of two possible I'm. Just.

A

Gonna put this in a slightly different way.

C

So this is basically saying you're getting yourselves from one or two places right and in theory, you could write this you'll see that this doesn't actually build.

A

But we can tweak it to make your build in a second.

C

You'll know if this makes sense but saying, depending on whether you have a closure or generator you'll, have slightly different types. If you can they still each have an upfront eyes method, so we could call it like this right.

C

In practice, right, this is the problem problem. Is they each have a different iterator type and we need to return exactly one iterator type? Well, if you haven't worked with them infiltrate, this is the problem that occurs sometimes.

C

So one way to solve that is, you can use this either type which basically it's it's kind of exactly this. To be honest, it's like exactly the same as this up fire subtype, it's just an enum with left and right and it implements iterator if left and right, both implement iterator and it does whatever dispatch is appropriately.

C

Have you seen that before or not.

B

C

See I'm adding it to the cargo tom, oh here without telling you, but it's a.

C

It's a it's like crazy, okay, Oh, putting anyone.

B

So we are fine picking up a dependent.

C

Just not in this particular grace but other crates do.

C

So I'm just taking what's already there, if you see here, the Liberace mirror uses either.

A

This is a this is a.

C

B

C

Okay, so now that code is all building but we're getting errors because I made this I made it so that the up fart eyes helper method can only be called if you know they had that enduring influence which right it's getting us. Some compilation errors we're about to see, which I think is okay, because these are exactly the cases where you can try to use the tuple type instead,.

C

For at least some of them are you see how it goes.

A

Like here, I think we just need.

C

I'm just going through these and changing them to.

A

Use this apart to bow-tied appropriate.

C

Drops calculus yeah. So if you look at this code, what it's doing is it's going through each of the upper ties and for each man it's calling trivial drop check out, lives, okay, so what I'm doing instead is I'm calling trivial drop check out lives on the tuple and then that will cause it to be the rate that make sense. Yeah.

B

C

Similarly, here I.

C

Can, instead of iterating through the list, I can just.

A

You can just call it once yes, mistress, I.

C

Think the same here who work.

C

Okay, so this is a what's happening here is I. The TC X has all these lifetimes. Have you seen.

C

It so happened. The last one in the list is sort of the lifetime of the shortest, like. Basically, the lifetime of the innermost enclosing inference context and it's kind of the the lifetime of types that you're working with physically, and it so happens that those functions didn't need to constrain it before because they weren't making types. So when you make a new type that gets interns, it gets stored into out in the arena and it gets that lifetime the shortest one, but they weren't making new types. But now we are so I had indicate.

C

No, it's just that.

B

We weren't our killing.

C

Any memory in those functions, so they didn't need to be. We need to say how long you will live.

C

Now I'm getting some.

C

No, so what I'm gonna do in order to avoid I'm gonna take off this. This is the requirement that was forcing us to not call it where we might be where it might be in inference code, I'm.

B

Gonna take that out because.

C

I want to get it working without if I did, instead of all the ones I'm gonna just go through now. This is the rip crap out plan. I'm just gonna go through and find places. I think we can manually convert away from calling up fart eyes, even if they would compile, because we know.

A

C

So here again I'm just converting on what.

B

Was an iteration from.

C

Crack to a single call, confused.

A

That should work. Yes,.

C

This one I'm gonna leave alone, because this is a generator which I will have to change generators to, but I haven't done them. Yet this one.

C

Incidentally, if we were going to change the memory layout, you know I said we still model or to pull closure as having and feels instead of one field is a tuple.

C

This layout code is if we were going to change that we would probably be, among other things, changing the memory layout could but anyway, so that's a generator here.

C

I'm going to change this, so I think I'm gonna have just looking at the time. I'm gonna have to go soon, but what I was thinking is once we get. Hopefully we can get this building and I'll push it to a branch, and then maybe you can try to we'll have to discuss this. She Bella sort of come up with some next steps.

C

C

This is annoying so.

C

Strattice, this is an annoying lifetime. You are denied.

C

A

Side side step so.

C

Yeah so what's happening, I mean here is that in the pretty printing code we have, we actually normally we'd, get given tcx the type context as a perimeter, and so we can kind of constrain its lifetime. However, we like.

C

But in the case of the preprinting code, we can't do that because we get invoked from a debug imple. So what we do is we grab the type context from we use like thread-local storage, and then we have to do some horrible things because the because we weren't giving it as a type parameter. The compiler doesn't really know like what is lifetimes over and so forth. It doesn't know kind of what its full type is it so we're sort of doing some hacks and that's what I'm trying to solve here.

C

Unfortunately, it's not working and I, don't know why, but.

C

So what happens is this this tie TLS, which says, give me the tight context, it's um in scope and you wind up having to give it a closure which works sort of for any lifetimes. And now it's getting unhappy because.

C

Because it has to work for any lifetime. Oh.

C

It has to work for any lifetime because it doesn't really know how how long the data is. Gonna live after all, and yet we told it it. We sort of we're basically combining data with unknown left with one known lifetime confusing.

C

So what we have is we have the scheme.

C

If you see this so here where I call T CX dot lift what that does is actually uses a little bit of unsafe code, but it basically checks dynamically that the data that we're lifting was allocated in the right arena and it will return none if not, and so it kind of lets you bridge this, where a compiler doesn't know that these two things are the same. This kind of dynamically checks it and lets you assert to do the same by unwrapping.

C

So when I call T, CX dot lifts up start on rep I'm, saying basically check that this is in the right arena and if it's not, then abort, but it will be this there's only one arena. So it's not. We know that.

A

Don't worry, that's like a superfood.

C

Okay, hopefully this will build now yeah. It.

A

Is start to introduce.

C

Up 4-tuple time and use in some places, let's take the last five minutes. I'll push this.

C

Let's take so the next steps, so what we did today, we introduced this up for tuple type, um I, think I called it.

C

More or less everywhere that we want to be calling it I think all the rest of these locations, almost with one exception, this yep, so, okay, there's all the rest of these locations that call up for our ties are fine, except for two things, and by fine I mean they come after typing things, I see, except for two things. First, some of them are calling on a generator and I didn't convert I, converted the closure, but not the generating. um So that needs to be changed.

C

We probably just need to change the tenor ago, the same way, and that might be a good next step to like we played these steps that we did that I just did real fast, and you can try to do it yourself, but the other thing is this particular usage here is in. This is actually in the type checker in the very last phase, where it it's exactly the phase where it figures out the kinds of like they see where it infers all the upper part types in it.

C

So we're gonna have to update this eventually, but you don't have to yet because we're doing this intermediate step where we just have a tuple, so I think what I would say is the next steps probably are so. First I would change the generator subst here.

C

Okay, so that basically this.

C

What we did today was we changed. The closure subs introduced this code to make a coupon right right. We do the same thing here in the generator. Self split basically wants the same codes a little bit different, because the numbers are different. Things are a little different but same idea, and then once that's done, we might want to change the the pretty printing code- okay, because I, because right now, it's assuming that you know how many first there are, and we want it to basically want to rewrite this code to.

C

Fix me, you write this to call up our people time and check the results if it is an inference variable.

C

You might want to add, like maybe add, a helper like opt up Bart eyes or something there returns.

A

I don't know, there's still like no capsulate.

C

Things a little bit, but here that's one thing: the same: that's for generators! The same logic applies me record closes.

C

Maybe what I'm gonna write.

C

That's your name.

B

C

And then he said we should change the generators.

C

So once we do I think.

C

I don't know if this will work, but I would like it if we could change this signature to this to prevent this from the faltering infants. If we're going to do that, we would need to update the peepee the pretty printing code. No, this is really just bad luck.

C

So once we do that, then we're basically ready for the next step, which would be to actually make it a tubule from the beginning.

C

I'm, not oh, you should just talk again. Yes, sounds good all right! Well, so here's what I'll do I will push this branch I will post this video.

C

B

Right thanks a lot for your time.

C

It's fun to see this I hope it's that I look forward to this future.

C