Agoric SES, 6 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SES Meeting: Records, Tuples, and JSON

Description

Nicolò and Mathieu hash out interactions with JSON.

A

Good morning- and uh I am hosting in place of uh chris today or uh says, meeting um on the agenda, I believe we have interactions between box uh and json and two-string and the interactions with virtualization and compartments um nicolo. You put the item on the engine that so I'll. Let you go.

B

Okay, so, yes, we were discussing this on gita with matthew, um so the like, we found a problem that we didn't consider before.

B

So we moved uh the unbox method from the box prototype to the to the box, constructor the static method, so that compartments can replace the box constructor and the unbox static method per compartment and provide a different unbox function.

B

However, we found two other ways that would observe the contents of a box and one is passing a box to the string function, uh because that function like currently uh invokes to string on the box content, and so we might need to change that and the other one is json.stringify which currently uh like unwraps boxes and stringifies their contents.

B

However, this gives away first of all to observe the contents of the box in the resulting string and, most importantly, if you pass a replacer to the json.stringify function, you could like steal the the contents of the box without relying on the box.unbox function, and so we also need to to change that. Okay,.

C

So nicola, let me just um matthew, explain this to me and I feel very confused. um The rationale for introducing boxes is we didn't want to just have a transparent transition from records and tuples to objects. We wanted to put a barrier in there that was explicitly non-transparent um and then, furthermore, um we decided that it would not work across realms, uh so it was not just explicitly non-transparent.

C

It was opaque without permission and that that's that's a consistent perspective, but for but that perspective is incompatible with two string or jason stringify.

C

uh Just trying to pretend, like things are transparent again, uh the other side I mean the the the other. I mean extremes are simple things between extremes generally tend to get messy and confusing. The other extreme is. uh If we want things to be transparent, let's say to two string or json stringify, then why not?

C

Just have them be transparent and get rid of the box concept completely, and the only intermediate thing that I can think of that's coherent is that we've got box there as a um as a non-transparency, but not as a barrier, uh in which case it should it should work across realms. Not just within a realm which it should work unconditionally, uh and you know, and and for each of these things there are objections, but the point. So I'm not saying that I'm advocating any particular one of these.

C

What I'm saying is, I don't find any stance other than those three to be coherent, and I think that the the the thing that that I'm I'm currently hearing uh just doesn't make any sense to me.

B

Yeah, so I agree with your like with your point of view, I personally see boxes as something that should be fully like uh opaque, like non-transparent.

B

uh The reason that we currently on wrap boxes in jazz stringify is that we found is that it makes it way easier to like often, for example, you might stringify something to send that to another program, and that program does not really care about the difference between, like the immutable part and not mutable part or, for example,.

C

So I'm sorry I'm just I'm sorry to interrupt it. Just if the other program doesn't care about the transition, then probably it doesn't care about the difference between records and tuples on the one hand and frozen objects and arrays. On the other hand, in which case simply the imposition of a box at the boundary is already an impediment to that program.

B

uh I mean the program that has the record and travel with books inside uh might care about. This difference is just that the like, when sending this to somewhere that somewhere might not care about this.

C

So I'm saying the receiver the receipt: if the receiver doesn't care about it, then the receiver probably uh uh doesn't want to see a box in the way either.

A

So the way that jason's stringify unwrapping boxes uh works currently is that it is fully transparent. It's like the box was never there um same for the way records and tuples are stringified. It's like they are the equivalent uh object or array. I believe um the I think one question to ask is: usually you don't send data one way or uh just one way. uh You also receive it.

A

If, if a program is able is receiving data that doesn't have this information about records, box and tuple and then somehow constructs it, it should be able to also explicitly deconstruct it if you need to send it back. um So I I agree like it might be a little bit less ergonomic to have to explicitly unbox um boxes in a serializer or or something like that.

A

But I can't really imagine a program uh except something I just want to be able to accept anything regardless of what it is and somehow uh make a stringified version. uh Any program that handles boxes in the first place will want to be explicit in the serialization.

C

Yeah, the whole point of boxes is to make things non-transparent, is to provide an ex is to enforce that there's an explicit step needed to get from the from the box to the box content. So I think I mean my main concern here is simply that we have a clear conceptual notion of what a box is and what purpose it's supposed to serve, and then we stay true to whatever that abstract notion is throughout all the details of the design.

B

Okay, so I guess what the proposal like the what the proposal is currently like direction where country moving it is that boxes are just uh like, non-transparent things put in the structure like as if they were just symbols, and so they should not be unwrapped by jesus stringify and they should not be like stringified nicely by the string function. Constructor.

C

Correct um and then with and with json.stringify there's a way for somebody to get the effect if they want it, which is a replacer that, ha where the replacer the replacer can use the box constructor if it wants to unwrap a box when it encounters it, um but it should, but it should definitely not be the default.

B

Yeah, so it would be similarly to how begins our country handled that, like if you string, if I begin to throw unless you pass a replacer that transforms it to something else,.

D

E

I I think ara may, when we first talked about jason stringify. I think the the main use case was around just the common thing of developers doing json stringify if they're debugging in a place without like an interactive console, so it was like. I don't think we had great use cases for it, apart from it just being a really quick shorthand for debugging, something you know rather than making them also have to pass in to replace her just in terms of their like art box is supposed to be this explicit boundary.

E

It's like their explicit boundary, if someone's debugging. Maybe it would be nice that they can, just if they've got a little debug function that just dump something to the console that will still work. So I think that that was the reasoning behind it. Rather than thinking boxes of this explicit boundary, except then, in this case there for some reason not.

C

Okay for the node console there is this weird util thing that peers into the insides of objects in magic ways, uh and it's you know it's outside the javascript spec. It's actually something that that is a little offensive, because it's doing things that can't be done in javascript, it's not, but it, but it's there and having it peer into boxes uh is no is is no more of a problem than having you know. An interactive debugger appear into boxes.

F

Right I mean we actually talked about this recently like, for example, you can observe the set of things in a weak map in the chrome console right, like there's. No reason why debug tools cannot reach past that member, like through that barrier right um and in terms of um like making it easy, like. I think, the the argument against it might be that um people will end up writing lots of code that, like lots of revivers, that will like put things in earth.

F

uh Sorry replacers that take things out of boxes, but I could easily see popular user lane. Libraries that implement like a very simple uh replacer function that people use those don't have to be like that's easily implementable user land, and it could be a simple.

C

Package also, we should be not. We should not be under any illusions that imposing the box non-transparency step whether we made a special case for two string or j and jason or not. What we're saying is for everything other than the the special places we made an exception. We're going to cause developers to write a lot of code to get through a purposely, non-transparent boundary that that, uh where we're currently with with frozen objects and arrays containing things, there is no such boundary.

C

So the whole point of this is is to force developers to write new code to traverse the boundary, and, if we're not happy with that, we shouldn't be happy with box.

D

Plus one yeah, I was going to say um at the risk of opening a can of worms in perhaps making mark scream. um uh If, if the issue is that you need a special replacer, uh perhaps the box class itself could provide that replacer as a thing you would just toss in when you call jason that stream.

F

We certainly can. I would be interested to see how much demand there is for it, especially if there's like you know a popular npm package that comes out of this, that, like people like to use and then the community effect takes over, but that could also. I think that it would be certainly easy to add later, as well, just as a new property on the constructor.

E

The other place that function might be useful is the other thing we were talking about with. If the unbox isn't on the prototype, it means you can't just chain. Oh I'm accessing some deep record. Oh now, I've got to a box. I need. I just need to unbox. If it's a static thing I have to do, I even have to extract the box out first and now do box dot unbox. You know I can't just inline it in one expression as easily right.

C

E

I say if you you can like: if the pipeline proposal goes forwards, you can kind of still get a similar thing right. I can have record dot this dot, this dot, this dot, this right, the way deep down. Oh and now, I've got to a box pipe that into box on box, but then that can be annoying if box is sometimes not undefined, so it'd be another like a function that takes a value checks. If it's a box, if it is a box, it unboxes it.

E

You could use that both when you're like deeply accessing something and also pass it to uh string a fire replacer. It would work in both situations.

A

I I think, no matter what you're not never going to be able to have a prototype method uh that that works, because if you want uh for a replacer to do it properly, uh if you want to do, if you want to restore transparency with a replacer, you have to actually um uh deeply unbox because you might have a box of blocks of box of box. uh So if you return another box, that's just not gonna help. uh Your replacer has to remove every box uh on on the path to something useful.

A

um Yes, we can implement. We can provide a utility function for that, but no matter what that the prototype wouldn't be where it should live.

E

Yeah, that's what I was saying, these utility things I think they could have.

E

I can see them popping up and being useful in other places, whether it's like a utility that if you have a box of a box of a box of a box, it will just unbox all of them or if you pass this thing that isn't a box. It just goes. That is the equivalent.

E

You know it it unboxes n boxes. So if you give it something that isn't a box, then it's already unboxed. If you give it a box of a box for a box for box, you know it just unboxes all the way through. I can see that being something that, if there's high enough demand, you could add in the future, and it would be like a useful thing that people would probably want a lot.

A

E

A

Is only useful if people want to transparently unbox the box and and and not and remove them from uh serialization, uh for example. That is clearly something we personally wouldn't want to do at all. uh We want to want. We would want to attack them uh specifically, and in that case recursively, is it doesn't help? You don't need it, so it's really dependent on the use case, and- and that's why, like, I don't think, we should try to add that to the language uh right away.

F

Right, that's I. I think that that is a prime case for something we could add later. It certainly doesn't need to be part of the original proposal. Good.

B

um By the way like because of how compartments clone the box function, we will be able to add new static function later without introducing like communication channeling like compartments implementation, that will support blocks, but not disney functions right. Yes, okay, every.

A

Intrinsic including nested ones are specifically listed.

B

Okay, so guess this is all for this topic. uh The the conclusion is to not transparently support it, and we will figure out something similar to how to stringify a firebox like what is the exact string that will return it, but it will not depend on the box contents.

A

Excellent, uh should it be silently ignored or treated through a typo like uh begins. I prefer.

B

Like begins, but I I would personally be fine with either yeah.

C

I strongly prefer throwing I I'm fine with either. Let me just mention that most of the precedent in jason, stringify.

C

Is to silently ignore things: if you try to stringify a symbol, you just get undefined.

C

If you try to stringify something which some of the uh um you know in which there are uh symbol named properties or non-innumerable properties or inherited properties, all of the functions.

E

Functions are ignored as well.

C

And functions are ignored, that's right! So there's a tremendous number of things that you know, sort of the general style of json stringify is to just ignore anything that it's not happy with, and bigint is really an exception to the general rule. That said, I, like the exception better than the general rules, I'm happy either way.

B

I will try looking for the notes to why begin stroke.

D

um One final question: if you attach data onto the box value itself like a property, I guess those would be dropped if it unwraps automatically.

B

uh Blocks are primitives, so you cannot attach number.

D

B

A

And- and I believe that would be the same as attaching uh something to a box version of a primitive, so uh a number instance, uh I believe, that's dropped as well, because it's automatically uh unwrapped.

B

And by the way, boxed in his in my sentence was the the object wrapper, not the new box yeah, a wrapped.

F

Box, a box with a bow tie on it.

A

Yeah, should we bite shed the name box?

A

I it's it. I have to be honest. It was something that irked me since the beginning, and I didn't want to bring it up, but since it's been brought up by other people like I uh box is extremely confusing to me.

B

D

B

Sorry, okay, I think most javascript developers do not really think about object wrappers for primitives. So I think that this is a problem that we have, but that most people do not have.

E

C

B

Are open to changing the name like if we find something short and descriptive? I.

A

My problem was not actually with any uh concept in javascript, it came from c sharp and uh things like that, where other languages have the concept of boxed primitives.

A

So it's not specific to javascript. It's it's a common concept in programming.

C

Yeah and it's an and and because I'll go ahead and say I'm confused by it as well, because it's not just a common concept in programming, it's a concept that already exists in javascript right. If you have a um a box and you say: object open paren the box close paren, then what you've got is a boxed box.

C

um uh You know the the object wrap for all the primitives. We've got object, wrappers already in javascript the universal term, for that across programming languages uh is boxing. So uh I will say I don't want to bike shed it here, but I do want to encourage this proposal to bike shed it and hopefully choose a different name before this goes forward.

B

Yeah and I think the open mount records and double skull will be like a good place to start discussing about this, with people who might actually like with people outside of this group, who might have like ideas about the a possible name.

A

Okay, sorry for the interlude. um I also assume uh that two string would just return a box open, close.

B

Yeah, probably something like that.

A

A

Great um is there anything else that anyone wants to discuss.

E

I had a curiosity, uh as I think you might have talked about it last week, but I was struggling to keep up as usual um just that the use case around wanting if boxes can pass callable boundaries um and they can't be unboxed cross-realm, but they can if they and boxes can contain primitives.

E

So if all of those things are given why we would then still allow a box containing a primitive to be unwrapped, I can see that it doesn't carry the security implications yeah, but um I couldn't see like um in my mind. You could still implement a membrane if, um if you couldn't unwrap it, but I was also struggling to keep up.

C

I was not aware of this exception, actually.

A

Yeah I uh mark, I that's something I realize, and I I need to think about this again. um So actually here's here's the problem. So what I, what I said is if you have a box of primitive.

A

It doesn't carry the realm identity with it. So if you send the box of primitive from one realm to another, you can actually unbox it on uh in the other realm. Oh.

C

I'm sorry, I'm sorry, uh objects, don't carry it around either. There's no realm internal slot in an object.

A

D

A

C

We have to account for it as the box containing the realm.

A

Yeah exactly so the box remember: the box remembers the realm and uh checks that the realm is the same. uh If the content of the box is an object, only um the it's, it's it's a bit obscure, uh but I believe there is no other way uh to make to make it really work. uh If you check the realm, regardless of the content, um that means you cannot.

A

uh You have to remap a box regardless of the content uh when you make it go through the uh the membrane, the the that mapping is expensive.

A

uh However, the problem is that if you have a record uh of purely boxes of primitives that doesn't contain any objects anywhere at that point you were, you cannot put it in a weak map to remember that mapping.

A

So that means, if you have a uh a record tuple that contains boxes, but that those boxes, none of them, contain an object. You have to do an extensive mapping and- and you can't remember, you can memorize the result of that that operation. So every time you see that record again, you have to redo that mapping over and over and over again. um Similarly, uh in going both directions.

C

I see um so that I I I see that's a problem. um The uh the security problem with the current suggestion is that as soon as you get used to uh boxes hide their contents from those who don't have the box constructor, then you put a secret string in the box. Thinking that the box will hide it from those who don't have that box constructor and then you're surprised that the secret got revealed. So as soon as you introduce a um a hiding mechanism to have it only hide some things uh is quite hazardous.

E

There is also a few other things. One records are expensive anyway, in the depending on how engines implement these things, of course, but assuming naive implementations, because engines haven't had the time to optimize things or haven't, had the motivation to just doing like an equality check.

E

So if I'm passing a record to the other side of the realm as soon as someone does a quality check on that that quality check's, possibly going to have to like deeply traverse that whole thing, so the general expectation of if you're, dealing with deep records is that some operations are costly. So the fact that the membrane thing would potentially be costly if you you're having to traverse it. It's like that traversal is something you're probably going to have to do at some point anyway in your program, but.

A

It's a lot more expensive, it's traverse it take every box you encounter and uh and and send information about how to recreate another box. With the same with that content on the other side, and so you have to do that mapping over and over every time you cross.

E

The boundary I mean you only need to send across the boxes that.

A

You need for the boxes and then uh you need to send over the the record and then, when you get it on the other side, you need to reconstruct a new record with different boxes.

C

What is the reason why we want boxes to be able to contain uh primitives.

A

I I don't want to, but people do that. That's one of the reason I it's a it makes boxes a lot harder to uh to deal with uh again that that's a problem that would go away if, uh if we only allowed objects in in boxes.

C

What are the reasons that other people then want to put primitives in boxes.

E

The the argument is that, if box can contain any value, so it's just box of t potentially boxes of boxes of boxes, then it becomes a generic container type in the language that could have other use cases yet unimagined so like, rather than seeing them purely as something.

E

This is not me saying I'm just repeating what I've read online, the um rather than purely seeing them as the thing that gives you an escape hatch in records and tuples, where in that case it the only reason you would possibly really want it is because you're storing a non-primitive.

E

If you remove that restriction, they become a type that could have other use cases in other domains. That's my understanding of it.

C

Does anybody have an example of a use of it in another domain.

B

Yes, so, for example, jordan uh suggested a um array.find like function which can differentiate the result between like not found and found undefined by using box, as if it was like a maybe type and you're torn on the fine. If you didn't find anything and box of undefined, if you found undefined there and other use cases that have.

C

I'm sorry, okay, just I mean. Does anybody here find that suggestion not to be silly.

D

I mean I don't find it to be silly because dealing with when you have promises that return, undefined is kind of a pain, sometimes.

C

Yeah, but I mean there's, there's there, given the the non-transparency of boxes there's you know, we've got a zillion different ways to um to build a non-transparent container around something uh in order to differentiate it. The the fact that we can't have a promise for a promise is the the classic case of that promises are only promises for non-venables.

C

So if you want to effectively get a promise for t where t is anything, then you instead have a promise for a container of t with many different kinds of containers having been used, such as a singleton array. You just say: here's a promise for a singleton array of tea. Singleton arrays are never promises. uh So now you effectively have a um you know. You you've got an encoding.

E

I will admit that is my go-to monad in the language is an array with one value, because then I have map flat map. You know empty now it is a nice little pattern.

A

I think the argument against that was that at least it would be something standard in a language, so multiple libraries could agree on on on the shape of the container.

A

It could become a de facto standard generate container.

C

So can we just agree now on singleton arrays being that the container for that purpose.

A

I mean that's purpose for promise. Maybe.

C

What wasn't it a purpose for find.

A

Well, what if your your content is supposed to be an array in the first place, how did different do you differentiate? It.

C

Because if you're always going to take the result and put a singleton array around it, uh then if the actual result was a singleton array, you'd have a singleton array, wrapping a singleton array and if you're not happy with that, then you shouldn't be happy with the box containing a box.

D

C

It's the same problem.

D

I'd need to think on it. A little dynamic import is the nightmare case, but I don't think either of those would solve needing a maybe type on dynamic import that doesn't auto. Then.

B

D

But I do think it would be very useful if now that import does have a second argument in its dynamic form, if there was a way to invoke it, that auto wrapped the result, rather than performing recursive thins.

B

Another use case, which was brought up, uh which I found more like convincing, was that whenever you use records records and tuples to build a like a structure that potentially accepts parts from the outside, for example, if you have a virtual dom library based on records and tuples, where you have like props coming from the outside, you, you probably would want to just put the box to contain the those properties without really caring about what those properties are and so like. If box accept anything, it will just work.

A

Yeah semantic separation between uh system data and user data.

C

So I I still I'm so so what is the advantage of box for this over singleton array.

B

uh If that you cannot put the singleton array inside of records and tuples, and if you put a box containing a singleton array containing a primitive, then you kind of break the equality rules for records and tuples.

C

I I think, I'm not understanding what the use case is. Why what is it that we're putting inside a box.

A

C

User data, that is primitive data or user data.

A

That we don't want to care, have to care about, a library might get user data. It wants to record it inside an immutable structure, but it doesn't want to bother about what type of data that is. It just wants to put it in the box and save it in in its immutable structure.

C

So if it does it uh um so, if the data are objects, then they have unique identity. So we don't care that we're not doing a deep equality by structure of the user data if user data is objects, so in that case we're still not breaking anything. We put it inside a singleton array inside box.

A

But if you're getting that same identity uh now, if, uh if you create a new uh singleton array, every time, you've lost that identity, uh that equality.

C

The and there's a lot of assumptions here about um you're you're handling, user data, opaquely you're, putting it inside a record or tuple, and you care about the identities inside the user data.

A

Yeah that actually is fairly common for vdom type use cases.

E

I mean it is still doable right it just. It means the code has to do their own thing of some way where they represent. This is the user data they've got. You know a record that has a key metadata user data and then they know the thing there will either be a box.

E

If, if it's a box, then it's because the person gave them something that couldn't be stored in a box or if it's not a box, then it's a primitive because they gave us a primitive or you know that they can still get around it. It just means they're gonna have to write more code to make sure they're tracking. What's going there, the advantage of it box destroying anything means that code is, is one line. So it's not it's not like a capability. It's a an ergonomics thing.

F

This all comes, this all seems like it comes back immediately to wanting to use boxes, generic single item container, because if you want like in that case, like you just mentioned, it seems possible to just not use boxes. But if you care about using boxes as a generic delineation of like of a container then, but that it just seems it seems counter to the now more solidified principles behind lack of transparency through a box, because a container doesn't explicitly mean that it's like opaque right. It just means that it's a container right.

F

So it seems like it's not solving the same problems.

F

Ultimately, I I don't really care um whether they soar, primitives or not, but it seems kind of gross in my head, like a box, is a reference to an odd like it's more of like a in my head. A box is like a reification of the pointer to the object, uh so it doesn't make sense in my head to have a revocation to a pointer to a string that doesn't exist like it doesn't, but I get it. I guess.

B

I think no one in the champions team is like enthusiastic about primitives in boxes, but we are mostly trying to reflect the feedback that we get on github.

C

Okay, so so I'll just contribute that my feedback on the other side so uh now you've got feedback on both sides.

B

Okay, yes thanks.

A

The generic delineation between system data and user data, that example, did somewhat convince me as a making box a a container that you have to have the you have to unbox to look at what it is, but you you just don't care about. What's in there uh and- and I agree thinking about it now- it does completely go against um being able to pass that box and being able to unbox it in another realm.

A

A

F

An argument could be made that we could disallow well, it depends on how the patterns end up peeking out, but an argument could be made that you could disallow primitives now and come to conclusion later, but if people start deciding to throw things in boxes to find out if they're primitives and that becomes a problem also just don't do that. Don't stop.

C

There's enough current ways to find out, if something's a primitive, that I think that there's not going to be temptation to start using boxes just to test if something's a primitive.

D

You're, probably.

F

Right but we're still nervous anyway, but yeah I mean we. We could also take the route of well.

G

Later so there are many many ways to do that mark, but now many of them are very quick.

C

The uh the shortest code- wise is uh capital object of x, triple equal x and the quickest is uh type of x, is either object or function.

G

Right, but by the end, by doing a try, catch on a box will be a lot faster than both of those. I I believe.

E

Life, it froze right right now, writing a try. Catch is a little bit annoying because you've got a there's. No try catch, isn't like an expression.

F

It works in statement position yeah. This is yeah now you're talking about wrapping in another function, call now you've got an indirection, it doesn't seem very ergonomic in comparison. Yeah.

C

F

Throwing is slow in general, like throwing much lower than a function call or like a type of check. Yeah.

C

And on some engines I think, maybe today all engines uh a try catch, even if the tripod doesn't throw is not free.

D

A

I believe it's the optimized.

A

A

I can just hear the pushback we're going to get if we're saying no primitives in uh in boxes.

C

Okay, well I mean I just like I said: I'm registering pushback if we say primitives are in boxes now we have push back on both sides, and now we can try to figure out what makes sense.

B

Yeah, and by the way, I think that so like matthew mentioned multiple times that the problem is that boxes with primitives cannot be put in weak maps.

B

uh However, I think that boxes with primitives past across three arms would be like a rare like uh not common thing and so like, even if, in this edge case, it will be slower because it needs to do like all the work every time it would still be like rare to do it.

A

Why do you believe it would be rare.

G

Can you say that again nicole, I didn't quite get that yeah.

B

So like, I believe that primitives in boxes would still not be that common, uh mostly because, uh like the main reason for boxes, is to put objects there like to put objects in records and tuples and that's how they, I think that will be used the most, and so even if the primitive case is lower.

B

That might be okay,.

A

It's it's a lot more slower and, and I'm I'm really worried that uh there's just no way to uh like. If you allow it to happen, people will construct records with boxes and no objects in them.

G

Yeah, I I feel like the the.

G

The issue will not be that you intentionally put something in a box that is a primitive value, is unintentionally creating boxes around primitive value that you're receiving and you're throwing a box around, or something like that. um So accidentally, boxing primitive values. I would say.

E

I still don't fully get this the slow down so like if engines have so I'm constructing a record, as I construct that record, you know I'm putting my key values in every time. Every time I see a value, I can tell it creation time if I'm, that value is primitive or not and effectively. You know, like then have a flag on my resulting object, which is just the you know, the bitwise or flag of my contents.

E

You know if, if none of my contents have the flag, I'm an object, then my flag is also I'm not an object. So then, when you get a record or tuple, and then you say you know, there's an apr if there was an api. That just says: are you deeply primitive?

E

That would be o1 just checking that flag, because it's being pre-computed and then you can just pass the whole thing across to the other side, because we're.

A

Deeply immutable.

E

But you can just pass the whole thing.

A

Across that's the main issue, if you, if we don't allow um boxes of primitives to be unboxed on the other side, you have to look find all the boxes uh in the sending realm, send the content of the boxes to the other realm and then send the record. And then, when you get the record on the other side, you have to identify all those places where you have those uh boxes and replace them by other boxes. With the content that you received, yeah.

E

Well, why why I still can't.

A

Because you can't, if you, if you receive a record that has a box in it with a primitive, you can't unbox it, which is, which means you can't use it if you can't use it on the other side, it's well, it's useless! You have to provide a record to the program that the program can actually use, for. That means a record that uh that has a box of the original contents, but it's a different box, but because now that box is uh flagged by uh attacked by the realm okay.

C

Good matthew, let me let me make sure I'm understanding you're talking about if you build a membrane on top of the call of a boundary you're, not suggesting call it boundary itself needs to do this. Yes,.

A

Correct: okay, good! It's it's the membrane that lives on top of the collarbone boundary that has to do all that work, and he can't remember the work it does, because it's not allowed to put those in in weak maps.

G

And so so much just just so I uh maybe maybe I can help to clarify here so more so. The approach that you're describing is the same that you describe it, the in the in the in the gist, uh which is a.

G

The work of transferring the boxes to the other side needs to happen ahead of time, so it's ahead of time, transferring of values that then, once you receive the record, you can replace uh the the one that we're using right now or experimenting with is a lit more uh just in time. uh The decision making on the boxes. Basically one side of the membrane, give you a record.

G

uh The other side received the record and says: do you have boxes? Let me look through them. When I find one of those boxes, I ask the other side hey. I got this record that I cannot open. Can you help me to provide the proxy of the content of this box, because I cannot open it? You can open it. You can open it and send me back the the thing that I need in order for me to create a new box and replace it. So it's a kind of a single loop.

G

So if we, if we get to have some something like a replace like it, imagine that you have a method that you given a record. You have a a a replace um or map. It's basically a map, so you do a map. So it's a single it's a single uh traversal, because you just go one by one: every record that you find that it happens to be a box. You ask the other side. Give me the thing.

G

I create a new box on my side with the proxy of the thing that is in the box and uh replace the existing one, and the result of that is a new record. So so it seems a little bit more simple, but also a faster, because you only do one one pass.

A

But it's it's basically the same problem you have, I mean, that's why I posited you need a helper like a map, but regardless, whether you do it, whether you push the content of the box ahead of time or you query it when you get it, uh get it on the other side, no matter what you have to do, that deep mapping before you can give the record to the program because uh of the identity that that record at that point, carries um you it's not records.

A

Unlike objects, uh you can't you can't build a lazy proxy on top of it, you have to deeply uh fill the holes uh and even if the program is not going to actually use the content of the boxes, even if they're never going to reach for those boxes, you have to uh eagerly uh do that and if you can't remember that results the fact that you created a new box with another uh with a map content.

A

If you can't remember that uh you have to do it every time and every time you have to go through the callable boundary to ask for hey what is the content of this box? um So it's extremely expensive to do it. If.

E

Is the key thing that I'm missing that we're saying you want to build a membrane where I now have two realms that usually, if one passes the record of boxes across, you can't open it the other side. But you want to build a membrane that then.

B

E

To do that, yes, even though normal javascript, that would you can't.

B

E

And people would have to, they would have to pass the box constructor across to the other side and say if you do any unboxing, you have to use this unbox function.

A

Because this is the one that has.

E

The key the membrane hides that from you, but why? What I thought the idea of a membrane is that it's transparent, but this is now not transparent.

G

You cannot observe that. Is that sorry, from the other side, you don't you don't know, but I.

E

Mean it's it's not transparent. Maybe I don't send it with transparency. I thought transparently that it still feels like I'm using the same language or is it transparent that I don't even know things are cross-realm as in like it's as if I was working in one realm. That's what the transparency.

A

Is actually a very good point that you brought up? um The membrane could potentially provide its own box unbox that you have to use to reach inside the box so that you can just pass the records and tuples without mapping them, and you only map at the unboxing time yeah, because.

E

That's what that's, what non-membrane code will have to do? That's going to be the pattern. People need to learn.

G

I didn't quite get that part.

E

As in so I'm not using a membrane, I'm just using realms through either, you know iframes or node vm, like I'm, I'm using realms that aren't anything to do with membranes. I need to learn as a javascript developer. If that other side unboxes, I have to give them either they had to give me their box constructor, so I'm constructing boxes they can use. So we now share the same the same box, constructor and box unbox, or vice versa. I have to get yeah.

G

But that but but it wasn't but that's not possible in a callable membrane.

E

It could be hosts could allow it like.

A

E

Could allow it no.

A

You can build a box that goes to the membrane. At that point, uh the problem is actually figuring out which realm you need to go through. So if you, if you replace box and box uh or provide a new one, uh the membrane does that uh with an implementation. That would ask the original realm what the content is and at that point create the proxy uh lazily uh that that would actually would work nicely. The problem is that works great.

A

uh If you only have two realms, if you have multiple rounds, uh you the same with an object with something that you can put in weak map. You can remember where that box comes from uh with something that you can't put in a weak map. You don't know which realm uh that thing came from, so you somehow have to uh try all of them before you find the right.

E

But this is, I guess the point I'm trying to make is that's what everyone else is gonna have to do anyway, like that is, if I'm not using a membrane. So if I'm using node vm to create my realms, I have to even make sure everyone uses the same box and keep or keep a list of boxes. So my I fake the box constructor, it has an internal list of boxes that it tries.

E

You know so. I have like in a list of n boxes, I've been given and I try each unbox until I succeed. You know that. That's what that's, what code's going to have to do if they don't have a membrane, so just just to say that solving is for membrane. This problem still.

A

E

Performance problem is still there.

C

I want to introduce an important distinction in the conversation uh there's two forms of transparency uh that mem that different member you know, there's there's uh the normal membrane, the historical membrane. Well, the transparency that membranes were designed to provide is exactly what ashley was talking about, uh which is that uh realms, separated by a membrane are uh very similar to re, to realms, separate, not separated by a membrane that the the membrane is a very you know.

C

Membrane between realms is a very good emulation of a round boundary without the membrane there um uh and then there's the new invention, which is you know, very intriguing that that um I think this group has spent more time lately on which is coridi's near membrane, which, where you place it between realms and it's able to mask the round boundary and make it appear that the objects from the other realm are actually objects locally. Here within this realm, which is where the term near comes from.

C

These are both very interesting but they're, very different kinds of transparency, and I think for this discussion. We need to distinguish them.

E

Thanks mom, that's that definitely explains why I'm getting confused, because I hadn't I hadn't distinguished that there were two things. So I think, knowing that there are two things helps me understand why I was getting confused.

G

Yeah, although the the the technique related to boxes for both membranes will be pretty much the same.

C

uh The the the the the the internal membrane.

G

Because that that remember that the identity thing the identity aspect of it, which is, are you an object from this rom or from the other realm that happens when you're, interacting with the proxy, which is the content of the box, but not how you access the content of the box? So that's why I I feel that the the box conversation doesn't have much of is, is orthogonal to what kind of membrane you're using.

C

I don't understand if, in the, if we're just going between realms, not not shadow realm, but just you know the the same same origin, iframe realms, uh then a box I received from the other realm is just opaque. uh I don't see, I don't see a proxy for the oh.

G

I see I see what you mean, I see what you mean now: yeah yeah yeah. Yes,.

C

G

The regular membrane you're right in the regular membrane, I should not be able to open the box anyway exactly and to open the box. I need to go through the membrane to get the other sides unboxing capability and then call it, and then I will be able to do so.

C

Yeah and for the near membrane, uh the the there's you know with this conversation, we should now distinguish two degrees of near there's with making it appear like it's within the same realm, but but different compartments versus making.

C

It seem to be within the same realm within the same compartment and the reason why that's relevant here is because of the box constructor per compartment, even if I'm in the same realm, if I don't have access to the box constructor that box this a box that I get from other code in the same realm is still opaque to me.

C

So I think, even near membrane, we should consider it acceptable to preserve opacity over the membrane, to not to not try to unwrap and put proxy around it, because that's what would happen even within a single realm between compartments.

E

Yeah, that was the the reason we moved unboxed to be a static method right, because that's that gives you that compartment.

C

Yeah, so let's not undo all of that progress by having the membrane try to auto unwrap things.

A

Okay, fair, we can membrane, uh can provide a box and box uh that is that can unbox from using the unbox from the other uh realm or compartment.

C

Why should I, why should it even provide that.

A

I mean if it wants to oh.

C

If it wants to, but but but a transparent near membrane is under no obligation to provide that in order to claim transparency.

D

A

um If it wants to, however, it's still incredibly efficient to implement, because for primitives you have to go in and ask every other unbox that you know about and try them one by one. Until you find the right one.

A

You can't remember which realm a certain box comes comes from.

G

Yeah also so one thing for for and keep thinking, so the mechanics that you're using the two type of membranes is the same. You still have to replace the box.

G

In one case, you replace it with something that is a box that you can access. In the other case, you replace it with a box that you cannot access directly, but indirectly you will be able to access it. Why.

C

Do you need to replace it if, if the box is opaque on the other side, then opacity seems to be adequate.

G

So if, in any membrane you replace it because you wanted to replace the box with a box of of the proxy, so it looks like you have access to you.

C

Why do you if the box is opaque on the other side? Why do you need to replace it with the box around the proxy.

G

I feel that we're talking about different things. Let me let me see so I'm talking about the the the implementation of the membrane um and whether or not you will be able to have an implementation that serves well both purposes with some configurations, obviously some tweaking but the same implementation, the same mechanics that I need to get this object from this side to the other side. I need to get this record from this side to the other side.

G

If it happens to be a record and it has boxes on it, I need to do some work um before I hand it over to the other side or, and part of that work is to replace the boxes with boxes that I can control in some degree, and so, when you try to interact with those boxes, they will do one thing or the other for any membrane. When you interact with the boxes, they should be able to allow you to unbox them, because you don't see you don't know that there is another realm.

C

G

C

You know it might, but what I was suggesting is that, if the ambition we have for near near membrane is it's only trying to act like same realm different compartment, if it's not trying to act like same realm same compartment, and I think that's a reasonable level of ambition. In that case, we don't need to replace the box. There are plots the box would be opaque in a different compartment.

C

So the fact that the box is opaque across the membrane means that it's already fine.

G

Okay, I get that um I I need to think more about these, because we don't use multiple compartments in the membrane that we use.

G

So I need I need to think more about this.

A

Yeah I mean I can definitely imagine use cases where you want a membrane, a fully transparent membrane and fully transparent, including for uh for boxes.

G

Yeah and that's what we have, because I think about what component composition. I have a pattern and a child component both of them running in different compartments kind of thing, like basically different, sandboxes and and the pattern is giving the child a a record. That has a box and the child will receive that record and will be able to access the content of the box, because it's handed over to the child to use.

C

I I find this incoherent. The let me give it. Let me give an example of why it's incoherent. Let's say on one side of the near membrane: you have compartments a b and c and on the other side of the near membrane, you have compartments d, e and f.

C

Now um you send through a box that has been boxed with the box constructor of one of the three compartments on one side of the membrane, on the other side of the membrane which of the three compartments should be able to unbox, it.

G

Only the one that reflects the component on the other side and.

C

Which one is that.

G

C

That, within a realm, you already have multiple compartments and you can have multiple compartments on each side of the near membrane and there's no necessary correspondence between them. So if on, let's say the blue side of the membrane, everything from the red side of the membrane, uh you know acts like it is from as many different compartments on the red side as there actually were different compartments on the red side.

C

That's the proper emulation of the absence of the membrane is that if you had three three three compartments on one side and three compartments on the other side, um uh then the in the absence of the membrane, you still have six compartments.

C

Now, if you, if you are emulating six compartments, then the thing that you have on the blue side, that represents the box. Constructor from a red compartment is a proxy for the box constructor.

C

If you use the proxy of the box constructor as if it's a box, constructor everything just works, because now the unbox, the unbox request, the box constructor goes back through the membrane to the red side and the red box. Constructor succeeds in unboxing the box.

E

Right, yeah everything happens at box and unbox time.

C

Yeah and and the the fact that the identity, the distinct identities of the box constructors, are, are preserved across the membrane, because box constructors are normal objects uh and that they can be transparently. Proxy is exactly what you want.

G

Yeah, no, I think I understand that in our case, we only have one component uh or equivalent to components. We don't have compartments. We only have one constructor, the box contractor on the blue side and.

C

When you say currently, currently you don't have any box constructors.

G

I mean once once it lands, we don't plan to introduce the concept of compartments um just.

C

Let's, let's have it land I mean since it's something that hasn't landed yet and compartments haven't lended yet and they're both in progress. Why don't we co-design them so that they're coherent when they play together.

A

I I think my concern is that this is this is the use case of a one-to-one mapping uh between two realms, where I'm a little bit more concerned? Is that if you have more like a, uh if you have more like a star topology uh for your realms, uh which.

G

Is what we have we have? We have multiple sandboxes that reflects the blue side. So basically you have one blue, multiple reds and they in the inter interact with blue through the membrane, and if you want two of these reds to interact between them, that happens through the blue, so you interact with blue and then back to red, so the marshall is in place there. um So in that, in that particular case for blue, it only sees one box constructor one way of unboxing things.

C

But it should see one box constructor per compartment. Just like you should see one compartment per compartment.

G

No that doesn't know that there are compartments blue, doesn't even know that there are compartments right well,.

C

The the thing is on each realm might have multiple compartments. The membrane should not collapse the compartment distinctions because those compartment distinctions are semantically significant.

G

So what I think well, we have to dig in into this because uh this so what what we have.

G

What what we have right now- and maybe this will help to explain at least what we have right now if we were to have boxes today, the box constructor inside one of these sandboxes in the red side, the box contractor is a proxy of the blue.

G

So when you're creating a box, you are going to go through the membrane you're going to have to push the content of the box to the other side via proxy yeah.

C

Well, not well, you have to push it through according to how it passes through the membrane, yeah.

G

Yeah and then on the other side, you create the actual box with the proxy of the thing on the on the red side, and then you return back a record that has a replacement for that box to a thing on the other side. um So I'm so fussy about what the problem is but or what the incoherence that you mentioned is. But I had to put some some time on thinking about this.

A

Yeah, I um I think we're not gonna solve that now and we are uh quite a bit over time. um So I think our next step is just to think hard about uh how multiple uh realms and boxes uh interact and whether the cement takes there.

C

And multiple compartments.

A

And multiple compartments um in the meantime, I think the resolution here was uh we. The realm uh should be checked regardless of the contents, whether it is a uh an object or a primitive uh to keep these semantics of a box. That is completely opaque and you need permission to unbox.

A

So my apologies for pushing in in the wrong direction uh earlier this week, and I think that is it. I will now stop the recording.