Home
Contribute Contact Us Browse all meetings
Home Contribute Contact Us Browse all meetings
Agoric / SES / 22 Dec 2022
Agoric / SES / 22 Dec 2022
Previous Meeting Next Meeting
⏯
Add meeting Rate page Subscribe
youtube image
►
From YouTube: SES Meeting: Virtual Module Sources

Description

Caridy Patiño and Kris Kowal investigate a lower-level design for virtual module sources using what we might call a contour controller, an object that stands for a module record and controls its linkage and bindings, presumably imperatively.

A

Kris “cowbert” kowal: welcome to this s meeting. it's december 20. First.

A

Kris “cowbert” kowal: the cess meeting next week will not occur as it. It's bracketed by the holidays.

A

Kris “cowbert” kowal: and today kris “cowbert” kowal:. I have would like to have a conversation with kari about virtual module sources like carity is expressed in the past. That.

A

Kris “cowbert” kowal:, but dissatisfaction with the modules virtual virtual module source proposal that there's possible, possibly a simpler framing of that. Last time we had a conversation of this form.. It resulted in layer 0 of the.

A

Kris “cowbert” kowal:, the hot module harmonies epic.

A

Kris “cowbert” kowal:, which was a good outcome,, I'm hoping for a good outcome from this conversation, too.

A

Kris “cowbert” kowal:, so kari, do you have a concrete idea of which direction you wanted to go?.

A

Kris “cowbert” kowal: yes, what you found dissatisfying about the current direction.

B

Caridy: so so this is a combination of things., so I was reviewing the.

B

Caridy: proposal. the first proposal, the contour proposal from.

B

Caridy: ron dave, herman caridy:, 10 years ago,, or something like that, and some of the work that we did on the loader side of things as well.. And there was always this notion of.

B

Caridy:, a very,, very low level, api caridy: that allow you to caridy:, create an environment record and then.

B

Caridy: set up some values there in such a way that you can control those values.

B

Caridy:, in the same way that a module x board will control them.

B

Caridy: and caridy:, by doing so, caridy: you, you cool, articulate, any kind of source.

B

Caridy: or equivalent to a source caridy: by from us to creating the environment and setting out the bodies that you want to export.

B

Caridy:, if we do that,, then why, we will need to have any sort of obstruction.

B

Caridy: for the different type of sources.

B

Caridy: in the spec., I do that with that you can build whatever you want,, any kind of active back that produces a module.. You could do it in user lab, rather than having to provide any kind of obstruction or level control, there,. So on.

B

Caridy:, I don't know if that's very different from what you're proposing. but, in my mind, is basically.

B

Caridy:, the caridy:, the the module source is.

B

Caridy: intended to we use caridy: for a.

B

Caridy: and anything that is not an at my scores, should not be a all the source issues.

B

Caridy:, do you have a api to create an environment record on this environment? Record,, allow you to control the volume that will go into that.

B

Caridy: and caridy: and that's all you need in order to create a module who is bound to that environment record.

B

Caridy: for all internal purposes., that's what a module will be. just a giving you a name and space object that corresponds to.

B

Caridy: values that are in that environment record and the ability to link to values that are being import from another module to also be described. There. but.

B

Caridy:, I'm not sure.

B

caridy:. That is even a requirement in this harmon proposal.

B

Caridy:, I remember we.

B

Caridy: at some point we agree that.

B

Caridy: trying to to solve the problem of of re exporting for another module.

B

Caridy: was caridy:, something that wasn't super important,. I would say.

B

Caridy: so., that that was the conclusion that I don't remember the details of how to.

B

Caridy:, the the some some caridy: discussions and the comments around that., but that was the general idea in the past like it it it you to do a lot more, that what we will be able to do.

B

Caridy:, it would provide an instruction that requires a new type of source to be specified, either by the whole or by this specification.. It's just a more generic way of doing it.

A

Kris “cowbert” kowal: okay., kris “cowbert” kowal:, so first off.

A

kris “cowbert” kowal:. I agree with most of the substance of that.

A

Kris “cowbert” kowal:, the kris, “cowbert” kowal:, I, or but I would revise it, I would not.. I do not think that a.

A

Kris “cowbert” kowal: an api,, an api, that allows us to create an entangled pair of module, exports, namespace, and then the.

A

Kris “cowbert” kowal: and then a a namespace object that can be used to govern.

A

Kris “cowbert” kowal:, the assignment of variables into the internal namespace that would be reflected by the external namespace is necessary.. I wouldn't call it sufficient.

A

Kris “cowbert” kowal: you hint at one of them, as I I think that.

A

Kris “cowbert” kowal: for one the exports do need to be solved.

A

Kris “cowbert” kowal: for.

A

Kris “cowbert” kowal: the the greater version of this proposal., not the lesser version. We don't need re exports in order to model common js. or on the on that one,. Just to give you more more details of what we discuss is that.

B

Caridy: the api that allow you to control the environment record does not need to have the concept.

B

Caridy: we export, because that's not how it works in a in this back.

B

Caridy:, so if you have module a, that important.

B

Caridy: something from module b. but module b is exporting that name from what you'll see.

B

Caridy:, what happened is that in the module, a environment record,, it will be a binding.

B

Caridy: that will be pointing directly to the value.

B

Caridy: on the on the environment, record of the c. and b is never in the picture.

B

Caridy: so when creative record that api.

B

Caridy: is the one that does not need to know or have any sort of concept around the environment. They they re-exporting aspect of it.. The the namespace object does not need to know about it. The module exports, namespace.

A

Kris “cowbert” kowal: and kris “cowbert” kowal:, and because the module exports namespace needs to there. The api for constructing an entangled pair of the internal namespace and external namespace would necessarily need to accommodate.

A

Kris “cowbert” kowal: a way to refer to bindings and other in other exports and in other namespaces.

A

Kris “cowbert” kowal:, so kris, “cowbert”, kowal: so,, so to be clear,. I agree. the internal namespace object does not need it,, but the internal namespace object does need to be entangled with an external namespace.

A

Kris “cowbert” kowal:, so whatever api generates, that pair needs to account for the bindings to other namespaces.

B

Caridy: I didn't quite get that., then I still trying to separate the area to create a a.

B

Caridy: is,, it is a it's basically, a reflective api.

B

Caridy: or environment record. well,, let's, let's, let's get., let's get concrete.

A

Kris “cowbert” kowal: and let's borrow the notion of bindings from the current proposal.

A

Kris “cowbert” kowal:, that's a an array of objects that represent all of the names that are bound.

A

Kris “cowbert” kowal: between the the that are bound in the internal and external namespace of a particular module right like a a way to express.. I import foo from this, and or I export fu as bar from that,, but just like the a way that some means of reflecting.

A

Kris “cowbert” kowal: what the static dependency, graph and namespace, and then the names that are required to exist on this pair of external, of of exports and internal namespace objects.

A

Kris “cowbert” kowal: I'm., proposing that your idea is to create a function that accepts a bindings array and returns and exports, and a like,. A an external and internal namespace object for a particular module. Is that a fair.

A

Kris “cowbert” kowal: concrete way to express what you're suggesting.

A

Caridy: no, no,. I I'm.

B

Caridy: no, I'm talking about different things., so I'm sorry., very slow. today.. The.

B

Caridy:, the foundation of the api that I'm. talking about has nothing to do with the module itself.

B

Caridy:, it's an api whose only job is to create an environment, record.

B

Caridy: and caridy: in that environment record, allow the creation of by meetings.

B

Caridy:, whose value must be provide by whoever is creating or has access to the environment. record.

A

Kris “cowbert” kowal: okay,. So it's like a builder pattern for a namespace object. that is, that is,, that that one is not even the name. is.

B

Caridy: that thing. just create an environment record caridy: with bindings there caridy:, whose only job is.. Basically it's just a dictionary kind of thing, right?. It's it.. It just create an environmental record and allow you to create a certain value.

B

Caridy: in that environment. record binding,, so we call it, will will be equivalent to accounts. the declaration there,, which is what we have today in the in the spec, anyways for export values.. So you you create a binding there,, and that thing is.

B

Caridy: is going to be used by any name in space office.

B

Caridy:, who needs to get a value out of that environment.

B

Caridy: that's all. caridy:. Then we have to talk about how we create such name is based on you.

B

Caridy:, which I don't know., but the fact that when you create a name in space, there is an implicit information about where each of the export from that name is space. update.

B

Caridy: points to caridy:, which is to call on environment record.. One of these environment record can be that environment record that you create.

B

Caridy:, that's the our email idea from from. they, in a sense that you have a very low level mechanis to create.

B

Caridy:, something that can be participant of this mode, autograph.

B

Caridy: by virtue of the fact that a name is based on it, points to a environment record, to get a value out.

A

Kris “cowbert” kowal: I see. kris “cowbert” kowal: okay., kris “cowbert” kowal: that part kris “cowbert” kowal: is interesting., kris “cowbert” kowal: the thing that it does that the proposal, as rit does not is account for the possibility of namespace objects that grow over time.

A

Kris “cowbert” kowal:, like the the proposal, as written, kris, “cowbert” kowal: in the compartments proposal. Right now for layer.

A

Kris “cowbert”, kowal:, 2, kris “cowbert” kowal: for virtual module sources does not account for the possibilities of the namespace may evolve.

A

Caridy: and your proposal does, and it we haven't- talked about the semantics of that., but the way the way we were thinking about the in the past was that I have something that re ensembles it a a a source of a module somehow.. Whatever the the thing is.

B

Caridy: you want it to be participant on the monroe graph.. Therefore, there is a correlation about what that thing.

B

Caridy: that caridy: code that you have on in some language correspond to the actual module.

B

Caridy:, so what you can do when you use a line, you can do the parse, you know,, whatever language that thing is.

B

Caridy: you define what the sports are.

B

Caridy: and you will be able to create,- and I have specifically mentioned export- only not export from, but export, and then you will be able to create the environment, record.

B

Caridy: create via reflection and api, create optics or some sort of values, like must be values that are valid,, that you will place into those different findings that you will add.

B

Caridy: to the environment record that you just create., and now you have a fully qualifying environment record that can be used by any name in space., and now we go into your api,, which is okay.. You now have that environment record.. It has values on it., it's basically a record.. It's just a record that has.

B

Caridy: names of binding that you will be able to use from somewhere else.

B

Caridy: and then you go into an api that creates a name. space.

B

Caridy: was caridy:, only job is to link to environment, record.

B

Caridy: we are caridy:, one of which could be that one that you can just create out of the source that you are.. So that's kind of the mechanics of what what we're talking about, and I think that one is very,, very low level, and allow you to do whatever you want.

B

Caridy:, if you can produce.

B

caridy: from some sort of information,, you can produce an environment record that has binding.. That is all you need to make that participant of the actual.

B

Caridy: module graph with the corresponding liveness of the export,, because you can at any time change the value of the binding, and that will get reflected everywhere, as it should be, and as a question of whether it grows or not. Over time..

B

Caridy:, I believe caridy: that is, on the name is paid,, because when you create a namespace.

B

Caridy:, you must validate that the bindings that you're looking for should exist on the actual environment, record.

B

Caridy: that you caridy: are linking to such name and space object. yeah,. I can grow over time,, but not from that name of space.. The name is pace is locked down in terms of what it export?. Does it export from more than 1 one or more name, environment, records.

A

Kris “cowbert” kowal:, so that's a detail that isn't actually important, I think, for this conversation.. What I can say is that.

A

Kris “cowbert” kowal: I kris “cowbert” kowal: can see kris “cowbert” kowal: such an api being useful.

A

Kris “cowbert” kowal: for, for emulating any language within the system.. I can also see such a thing being useful for doing things like rebels.

A

Kris “cowbert” kowal:, where there's a where there's a gl, where there's effectively a global, where there's a contour, that each evaluation is able to augment by adding cons, declarations, or let declarations or whatnot.. I think that that would be generally useful for more than just modules and.

A

Kris “cowbert” kowal: and serves the same purpose as what I was describing for the purposes of building them, building out toward having something like virtual module. Source. I'd say that it fulfills a requirement,, but it isn't sufficient to implement the whole thing.

A

Kris “cowbert” kowal:, you need to have kris “cowbert” kowal: for a virtual module source to participate in the module graph.. It isn't sufficient to just have a namespace object.

A

Kris “cowbert” kowal:, you have to also be able to express to the module loader what its dependencies are.

A

Kris “cowbert”, kowal: and and where to link them in and when to link them. and then also when to trigger the actual evaluation. Behavior of the module.

A

Kris “cowbert” kowal:, which isn't,, which isn't implied by just having a namespace object you need to have.. There needs to be a hook.

A

Kris “cowbert” kowal: for virtualizing, the behavior of.

A

Kris “cowbert” kowal: examining the dependencies of of a virtual module source.. It is something just like an instance of module source,, except that it expresses its bindings by another general mechanism.

A

Kris “cowbert” kowal: and then also trigger the evaluation at the right time. right?. The import machinery needs to be able to say, you execute. now.,.

A

Kris “cowbert” kowal: here is your here's, your contour, governor., the thing you're describing., I'll, I'll, just call it a contour governor,. For now. Here's your contour, governor.

A

Kris “cowbert” kowal:: please do your evaluation behavior to populate the values or read from the values that you're linked against at this time.

A

Kris “cowbert” kowal:, which necessarily happens underneath the stack of the import,, the dynamic import kicker and not at some other user-defined time.

A

Kris “cowbert” kowal:, so so in summary,. I think you're right that we need to have a lower.. I I I think that it's great to have an even lower level api than what I've described.

A

Kris “cowbert” kowal: or governing the contour kris “cowbert” kowal: awesome.

B

Caridy: so a few, few few more comments.

B

Caridy:, because we don't have a reflective api to create a name in space of the. I don't think we'll have that.

B

Caridy:, the mechanics.

B

Caridy:, that are in place,, at least in layer, 0, or through a module in terms.

B

Caridy:, so you create a module inc. and then that module intends, specify certain things that allow you to create an image space out. Right?.

B

Caridy:, so I think that will remain the same.. The difference here is that the module.

B

Caridy: that you will create caridy: that will have these.

B

Caridy:, one or more environment record associated to it. it it!. It's not coming from a sort.

B

Caridy: and as a result, we might have a different api, or the the different kind of module. Constructors that allow you to create a a module who has a very specific details in terms of where it's coming,, where are they? export coming, and where they export from.

B

Caridy:, resolve, and so on, and the kicking process of the life cycle of the module itself in terms of when to do certain things.. So I I I figured that the.

B

Caridy:, the.

B

Caridy: such constructor will will not be.

B

Caridy: equivalent to the one that we already have caridy: for module from a source., that's why,, every time that we talk about the virtual source, I get,, I know we're feeling about it,, because I I keep thinking in terms of this is.

B

Caridy:, the module,, not the source,, the one that you want to control,, because when you imagine that you can create such modules.

B

Caridy: that control the life cycle.. So it knows when the import is.

B

Caridy: call on it. caridy:, meaning it needs to be.. They need to be ready.. You have a hope there that allow you to maybe evaluate some code that fulfills the environment record that you want.

B

Caridy:, ahead of time before anyone can grab a name, is.

B

Caridy:, you also describe what this name is based on.. It looks like.

B

Caridy:, it also describe., if that is the name of space of it, will be getting binding from order. environment record.. That is not the one that you control and can create as many as you want.

B

Caridy: but you you should link every export to an environment record, whether that environment record is a record that you create or a record that someone else create.

B

Caridy: through the normal pathway of creating a an environment record, which is another module that you're importing from someone else.. So I think that description of what the name is phase.

B

Caridy: of this should look like, and what the details of it are, should be part of the module constructor rather than considering this part of the source., and I I understand that when you look at the source, you might get.

B

Caridy:, I I'll not say it quick, bye, bye!

B

caridy:, but it's very.. It's very easy.

B

Caridy: to look at the source and saying, oh, the source is the thing. Is they want that? You me all the details that I need to build such a structure., but I'm looking from the perspective of well.

B

Caridy: the source might be.

B

Caridy: important for us to create these 2 things. one is.

B

Caridy: environment, record with the actual bodies that are relevant to that sort, and a.

B

Caridy: api that allow me to create an image space object.

B

Caridy: that contains the metadata about what the source supposed to be, which might include the usage of this environment record as a storage mechanis for values that the source to produce.

B

Caridy:, that's kind of the the way I I I'm looking at this.

A

Kris “cowbert” kowal: so.. Let me highlight some things that I think that we agree upon in order to verify that we agree upon them.

A

Caridy:, I think that we, both agree, and our proposals, are consistent with the fact that.

A

Kris “cowbert” kowal: that namespace objects are not., kris, “cowbert”, kowal: that there is no,. There is no api.

A

Kris “cowbert” kowal: that would just construct a namespace object.

A

Caridy:, that's correct,, because even even imagine that you construct.

B

Caridy: that the imagine that you want to construct this thing from a source.

B

Caridy: and you want it to have it as a dependency on a module that you create from source.

B

Caridy: api. that we have well, that monitor that you create from source can only resolve in the import hook to a module instance,. It's not even a name is space,, so it will not work.. If it is a reflective name is phase api.. It has to be a module,, so you can link it.

A

Kris “cowbert” kowal: so to be clear,. I think that both of our proposals are consistent with the fact.

A

Kris “cowbert” kowal: that there's no user a. that. the only entity in 262 that would ever construct a namespace. object. Is the machinery hiding behind dynamic import.

A

Kris “cowbert” kowal: that that's consistent.

A

Kris “cowbert” kowal:, the kris “cowbert” kowal:, the discrepancy, I think, between.

A

Kris “cowbert” kowal: okay,. So let me, as a straw man., let's propose that there is an alternate module. Constructor.

A

Kris “cowbert” kowal:. I think that that's equivalent., kris “cowbert” kowal:, that is equivalent to what we're proposing.. But I maintain that there is a distinction between the responsibilities of the module source and the module instance that that would not reflect well.

A

Kris “cowbert” kowal:, and that is that module sources are reusable.

A

Kris “cowbert” kowal: right? they and stateless., kris “cowbert” kowal:, the the virtual module source doesn't contain an environment record. it it has a hook,. It's more like it's more like our handler object.. It's more like a more elaborate handler object on the module cons for the module constructor, except.

A

Kris “cowbert” kowal: being placed in the position of the source, because it's because and only because it's reusable.

A

Caridy: is it? but is it?. I don't think it is so. no,. It's not.

B

Caridy: so. for for those reasons,, I think caridy:, the reusability of it. it is on the is, is on basically the the user, the one describing how this thing is, we use, and it could be a well,. The the user has a function that can be used to reconstruct one of these.

B

Caridy: environment, record and module module, intenses associated with that or the in on source, philosophically consistent with the existing positions right. That.

A

Kris “cowbert” kowal: module sources are not themselves serializable., they have a host-defined serialization behavior in the same way.

A

Kris “cowbert”, kowal:, mit ctl and virtual module sources have a virtual sort. virtual host-defined serialization. behavior.. They are not themselves serializable,, but a virtual host can contrive of a means for serializing them. 100.

A

Kris “cowbert” kowal: right?.

C

Saleh:, so I have one question here just to to ask about this,, because.

C

Saleh:, when, when you get the module source for a script versus a module from the same url,,.

C

Saleh:, so are we saying here that we so so let's say you get the the same.

C

Saleh: url, and it's being invoked.

C

Saleh: as a module. saleh: by being imported.

C

saleh: and it's being referenced as a script.

C

Saleh:, so I think this this creates 2 instances of the modulus which I'm not sure, I'm not., I'm just wondering if that's.

C

Saleh:, that's why we're saying here,, because to my understanding,, the the module source.

C

Saleh: differs in how you bind.

C

Saleh:, how the findings go. kris “cowbert” kowal:? The word comes in instance. they so to be clear.. That is not the extent of the differences. that modules module, sources. module text to the text of a module is a different grammar production, and has different semantics, and is implicitly strict mode.. It's very different than evaluating of the script.

C

Saleh: so so to clarify what you mean here by module source., you mean the the not not the fully parsed source.

C

Saleh: but the possible source.

A

Kris “cowbert” kowal: I'm going to refer to the text of a module as a module text.

A

Kris “cowbert” kowal: and module source, as the compiled.

A

Kris “cowbert” kowal:, not the compiled., an object for presenting the parse of a module text and a virtual module source is a protocol for emulating that behavior for non javascript. Modules.

C

Saleh: okay, that that that's clear. thank you.

B

Caridy: good. I to drop on our meeting coming up., but.

B

Caridy: right, yeah,, caridy:, I' I'll, be out for a couple of weeks,, so I'll try to maybe.

B

Something.

B

caridy: so see we can caridy: debate it more concretely on the api side of thing.

B

Caridy: and then caridy:, then we can read just on their seat, if that new direction is worth it to explore, or or which will stick to what we have right. Now.

A

Kris “cowbert” kowal: yeah., so I'm convinced that the new direction is worth exploring. and might even.

A

Kris “cowbert” kowal: further further dig into the fact that numbering the layers of the of the compartment proposal was a mistake. from the beginning. we're going to have to resort to and do a decimal system,. I think.

A

Kris “cowbert” kowal:, but it sounds to me, like modules. Layer 2 is effectively going to grow another layer. in order to accommodate this lower level. api. that that accounts for the very specific case of binding.

A

Kris “cowbert” kowal: by a of of governing a contour object, which I think is,, which has other uses, too., I'm excited.,.

A

Mark s. miller:, I look for mark s. miller:. Do we have distinct short names for the layers?.

A

Kris “cowbert” kowal: we're yeah,, we do. layer, 0 is.

A

Kris “cowbert” kowal: module and module source. constructor module. one is reflective, bindings, module, model, module,, binding reflection, layer, 2.

A

Kris “cowbert” kowal: is virtual module source, and it may decompose into at most it's like at as many as 3 layers. So far within it. there's the contour governor,, which is what kara is discussing.

A

Kris “cowbert”, kowal: and then, what I like to refer to as virtual module source, the lesser versus the greater.

A

Kris “cowbert” kowal:, the lesser being sufficient for non javascript modules, and the greater being sufficient for emulating javascript.

A

Kris “cowbert”, kowal: and, and then evaluators is the next layer.. Then compartments is the layer. after that.

D

Mark s. miller: okay,, I suggest that.

A

Kris “cowbert” kowal:, that's right., yeah,, good, good call.. Let's do that. it's a graph. anyway.. The numbers were useful for for, for for for communicating that there was a layering to them that they, but it was it's it's a it's a it's. A total order. burn and such exists.

D

Mark s. miller: right., it was rhetorically., it was a great thing to start with,, but I think we've gotten past that., it's no longer useful. all right.. It looks like.

A

Kris “cowbert” kowal:, it looks like that's a meeting for us., I'm gonna stop the recording. thanks.