.Net Foundation Design Reviews, 20 Mar 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: .NET Design Reviews GitHub Triage

Description

-11:-34:-38 - Approved: Path Span APIs that write into a specified buffer https://github.com/dotnet/corefx/issues/27418

A

Are we going to get to? We were like blow well.

B

I'm pretty sure.

C

A

Kel is not spending attention again, it's just putting in our.

C

A

Oh, it's all hot, very big stuff, I've had so many definitions, man, oh my.

D

Betters I think.

A

Your perfusion, but we should probably talk about efficient.

C

And it's done at this point: I guess.

A

Why do I feel like this is different.

E

You know David's, like a change to the method we added okay.

A

We have a delta with the old change. Look at what the current shape looks like I believe.

E

He's that the main thing is rather than accepting the actual buffer to use, he's replaced that last buffer argument with two knowable integers for receive buffer size and send buffer size.

F

So, are we going to break people already passed a stage?

F

E

F

This release I know what do I. Even if we have to make a screen up, we start making changes to consume this. That becomes a pretty change. They all support, Oh David did, but that doesn't necessarily mean David's gonna go fix. All is you know.

D

G

Understand how this change eliminates battery Steve? Do you understand.

E

G

You personals, it doesn't bother. Is that the idea I believe.

E

That is his intent. I asked in the.

H

E

And I haven't seen a response. Oh I.

G

E

E

E

I

E

Net to do a zero byte read on the connection to know when there's data available and then only when there's data available issue a Request, the WebSocket to get the next packet basically wants to eliminate first from the WebSocket itself other than the 14 by tougher, that's necessary. The matter also.

G

Only when the when he's doing 0 by treat then he'd hopes to eliminate the batterer. It's not when he's doing regularities when.

E

Were using regular reads, then, he wants to you'll receive a sync: he passes in the buffer like the user buffer, and he would then want it to write directly into that.

C

J

Reason I can't find the API so no idea because.

F

J

F

Package we added it in two places and we Steven we added it more. On WebSockets operator, it's.

E

On both WebSocket, which is in box and.

I

Physically for a speed-up man, oh man, yes,.

F

But with this API is now, it's gonna have.

E

To go on both edges change to both would be changing. Both WebSocket get from stream and WebSocket protocol create from stream I.

G

Still don't fully understand so you could you you can't create from stream once when you wrap the stream into a in a WebSocket. Why would you want to specify the buffer sizes at the creation time if you want to control the buffer sizes for those special to zero butter reads so.

E

Today, the implementation basically maintains its own receive buffer that allocates essentially once when the stream, when you create the WebSocket, it has a receive buffer. It issues all receives into that buffer and then it's out the relevant pieces into the users buffer when they call receive a sink for send a sink. We didn't want to make to networking work. You know to actual set is under the covers one for the header and one for the buffer that we take from. We rent from the array pool that we put the header into.

E

We copied the users data into, and then we issue a single send for that whole buffer and return the array to the pool. So with today. The WebSocket that gets created from this has a bumper that it allocates for the duration WebSocket receives and a buffer that events percent.

E

He wants to get rid of all of those so that he can do zero.

E

Byte receives that know when there's data available and then when he knows our stated available, haven't read, read the header into a small fourteen, byte buffer and then issue a separate read into the users buffer and for sense, not use the array pool at all and do multiple network sends or multiple writes to the underlying stream, rather one for the header and one for the body, and the thinking is, if he gives it right now, the WebSocket is written with the idea that the underlying stream that's passed to.

E

It doesn't do any buffering, but if the underlying stream does do buffering, then we're essentially double double buffering by having the WebSocket buffer itself, and so he wants to be able to say effectively know the underlying stream is doing all the buffering. It's cheap to do multiple, writes or reads on the underlying stream, because it's buffering for you.

F

If we do, this, I would suggest we editors it over at this point, not a replacement of the existing budget, why it's less people have to react to it. If there's already people consuming this guy up above they're, changing.

E

Exactly to people using this API client WebSocket, which is in the same code base and signal R, which David is doing this specifically because these change things they know are right, I'm not terribly concerned I, don't think we should ship API as we don't expect to be used.

K

G

Fair enough also, another thing is: if somebody's using this API today, it will just require recompilation, correct. It's not the source, breaking change, it is a signatures. Changed get.

K

Rid of a byte array, yeah.

F

And did the last two parameters change to that, what they are they're? They were used to be memory, a bite if.

E

You were so the previous signature has a an optional buffer is the last argument. If you were passing in a buffer, this would be a sort, breaking change. I.

G

See so we take away ability to pass in your buffer and you just pass in sizes, and then we borrow the buffer, says: okay,.

E

Because they be passing in there.

E

They wouldn't use that they much prefer this approach. Yes,.

G

I

The way can you check your microphone because, like weeds oftentimes, you, like you, know from far distance almost come hear you at all. We experience the Doppler effect.

J

Almost like you're swinging, you might not want you're.

A

E

C

E

On a headset, the Doppler effect was probably me rocking back and forth in my chair.

J

You must have an amazing range well.

E

When you're, when you're, only a foot from your microphone.

E

G

Be nitpicky Doppler effect is not volume going out. The other.

J

Great, no, your sisters very quite bringing up. Yes, let's do some everybody's fine. The API is just that we just talked about whether we should place it or not seems like in placing is fine right and yeah well,.

F

I think we had to get shipping room pool for that at this point, we're we're past lockdown and it's gonna break SP net they're gonna have to soon as we make this change. They're gonna have to react to it and our official is gonna, be uninformed for.

I

F

I

We had it an overloaded, we wouldn't have these problems, we couldn't do it in stage like you know, one day check it in the overall and then you know, wait for asp.net change and immediately remove that, but do a temporary you know, transition I would not like to ship API that we don't care about honestly.

L

F

We're not or pass SRS were in asking a preview. So what's.

I

The process, where do you send the mold we have to we have to send email or people, have followers to send email to ship room and ask them like you.

M

Know, hey I want to make.

I

This just does not crush it wrong. Yeah, it's not yeah I.

L

I

Know you can I'll tell him to work with me. Yeah.

K

We can I was gonna, also suggest that we could try doing that side by side for Pb and then get rid of the second one or get rid of the previous one for RC.

F

Removing api's and RC.

E

Sorry before we like you, know, approve this, it also be good or we can approve it, but it'd be good, really understand fully what the behavior is for when the receive buffer and send buffer sizes are null and when they're smaller than the implementation would want. If it was buffering. I.

J

Suppose you'll ask the question what the evidence why.

E

We just whoever follows up with David. If that's you Carol be great to just get that clarified. Yeah yep.

F

And we should also make sure it's clear that this is going on both places, not just the one. Yes.

I

I'm to work with me should I want to open more up as well.

F

And what must be the sounds, but another thing would.

F

Make the soundless button that the ping button likely like the silence I got in.

J

There was the last one we had in the two more months, and is that actually correct, or is anything else that we need to do otherwise? I will just go? Are the ones that have I wish our muscles attention.

N

Not only only future yeah.

I

J

Don't think we should do a whole pass over that I'm, giving that he cut it anyway from to 100 not spend too much time on that, but maybe we should pick the other ones here. They're like also the hot beer bucket like this one. This.

C

One this one, this.

O

One is a tough one that we have the current.

O

O

And it's confidence bad once the goods.

C

J

The victor is on the throne. You should be able to talk to that yeah.

P

This is about introducing spanner withouts for a couple of api's. There was some internal discussion if we should also introduce memory of charm api's, so I talked with yarn about it and he thinks there's some. If it's called on. You can see that there's some memory overload.

P

Yeah, we don't need memory, for example, and young things that we could probably introduce new return types like the match, value, capture value and the group value, but I'm not sure if this is working, because currently we have an inheritance between capture group and match and therefore memory.

P

E

You were talking about the match, value and capture value being for this for the span based overloads and those would be ref struck types. That's what you were exploring.

P

Because young thinks that those api's should could benefit from accepting this band and that's.

E

What I was asking the options? It was either have a exception, memory and return the existing match or accept a span and return. The ref struct match.

P

For example, twice Cape try an escape to replace and split those API. It's currently accept span, and they should be easy to with you.

J

So how much do we benefit from those, because I mean we basically have to from all of this stuff through the entire metro, to really take advantage of.

P

Their drama, the thing is, we are not interested in providing with us Spano. We look for patterns only for the inputs. That means that we only need to change the interpreter or the runner, but not the part at all. It's the only.

J

Point is we're not just converting the span back to a string and forwarded right. Let me actually do the data from this band.

P

There's a link to one of my peers and I already did the work for all that stuff. It's just changing the interpreter to work with his band.

L

H

We have a concept of compiled registers right yeah.

P

The thing is compared with expose a lot of fields, so I had to split the interpreter from from the compiler, because the compiler is exposed publicly publicly, so I can't change it to excel at the span. So this only benefits currently the compiler.

E

Because I mean that if you use the new span based API is with compiled regex that you end up allocating a string for the span.

F

How deep do this go? How much allocations do we eliminate in the actual interpreter, by going with these rapiers, it.

P

Depends, for example, for example,.

P

Explications before my changes now it's nearly zero, so it depends if you're just doing a match. It's not a lot less. But if you're doing things about the match like you would place.

P

J

Have some benchmarks of numbers that we see like you know? Okay, for this particular rejecting particular input. We would use their locations, but I don't know to 2%, or you know, we went from 80 megabytes to like 12 I'm.

E

J

B

A little concerned with the issue.

E

That Levi brought up, though hadn't realized that for compiled regex is this action. These new api's would actually be more expensive and are what you use when you want to, when you're going to be invoking something so many times that you want to significantly reduce the cost.

J

There's two notions right: there is the you know when you, when you constructive ideas, you pass in the you know, generate il anything and then there is the other part of it, which is the pre-compiled regex to an assembly which is on.

E

About us I'm talking about the former compiles so.

J

If we use il Victor, do we are you saying that, even in the ILO case, we're not like, we basically have to convert the span to a string, or is it just when you compile it for assembly? No.

P

It's for both these changes all only benefit through them.

G

And so the the limitation is a fundamental limitation or is just it's a lot of work to read to redo this.

P

P

You see that we have exposed.

P

F

Imagine we could probably do it for the end memory compile elation once that's complete in the same library, but when there's a separate library involved, the tools that it used to generate all have to change. You know there's closed from from about it.

P

Here you can see, for example, the runtex is exposed, which is the spring, so.

G

But this is could be lazy. Great, no, no.

J

It's a field, it's not okay, so the problem is I mean like there's two sides right I mean. If you have an existing compiled tuna, see on the regex order this they will continue to use the regex one-hour class that exists in the ferric and those wouldn't benefit right, because it would be a breaking change for that. But I would I would guess.

J

If we construct the new regex in-memory engine, specifying the regex options on how to IL, we don't have to go through the vertex one, our factory as it is today right we can, we can do whatever we want to do there and like have that you to whatever mechanism we see. If it that's.

P

That's the burger haven't done yet, but of course, um if we.

N

P

We would produce new IL or run new compiled tracking system. We could we, we watch turn engine yep.

J

So the thing to me is like agree with Stephan like there's a little bit of a bit of failure here that we they open up with that right because on the one inside we're adding new api's, specifically aim to make it more performant, but then use credits with the existing API, where you compile to IL. If you want to speed up you basically now I have two things that are designed to speed up that make it slower than what it wanted, but then what it used to be right.

J

That seems like it bad I guess. Oh yeah,.

P

With what I said, it would be like that, but if it would also change the way it's compiler, then it would benefit from it. Just rewriting that ILO generation no.

J

I get that I'm not saying like while you're not done yet. I am but I'm trying to get it that if we, if you ship the new API, as you probably should only do that once we have the compiler thing working through that as well. Otherwise, your I.

G

Think that you know sometimes intermediate state where it's like. You know it's not the a lot of work and we doing incremental change and longer time we will do you know that the more work makes sense in this case it may not make sense because of the strange situation Victor.

E

Is this concern with the compile? Is this for all the api's are just the span ones like your the memory api's you added. Does this have the same? Does this suffer from the same issue? You.

P

Know so yeah Victor.

K

I see regex rotor is public, but I don't see how anybody would ever get one.

Q

P

These two types only supposed to make precompiled way right, but.

K

All the real implementations of this class are internal sealed, so who would ever read these protected fields like if we wanted to change it and just not set them anymore? Who would care.

E

Doesn't throw the pre compiler? Doesn't it spit out a type that derives from this and uses those.

K

And it will already have its field.

P

K

Just wonder if we can make the base class support both the spam or memory and the string like if the precompiled external things are using one field ref and the new things are using another field. Raf everything will just work. Yeah.

P

So it's just about disgusting if we should create a new type way, though just to make the existing type I guess.

J

I mean once he already flying to the connectivity change the code gen for whatever user code gets generated. Advice from that you could just have a different part. I mean it doesn't have to be the same time that you're gonna have you know new fields on the same time, but you v it at I/o. You have to change your code generator. It.

G

Still doesn't change the fact that it's a lot of additional work, I can imagine yeah.

K

I'm just saying we have the ability test it for both cases by just doubling the number of fields in this guy is making it there's.

J

No fundamental significance: it's just.

P

This is not for too long, so this is for the next version of.

J

Just looking at it guys now for the future, it I think to me, like the question is more like: do you think that it would be worthwhile spending money on it magazine like? Do we see ourselves actually doing the work change in the case for compiler edge X, because if the answer is no, then I think not adding the overloads to begin with is probably better than adding it like falling out the cases I.

P

Think, because of our current state and we've got nuts, that we have pretty slow in comparison with our yeah, with, for example, rust or whatever we should stand some effort in those ApS right.

J

I think this is another larger program that you have on your side, right to like figure out what to do with the regex engine itself by there's this other I forgot the name for it, but there's open source C basis but yeah and then the question. What are we doing with that? I'll? Be changing our implementation or replicating a new API surface and I think that seems like the bigger bang for the buck right to figure that out first and then see how these span des over those would fit into them.

E

Maybe I make a suggestion. Let's assume we were just going to do the work to fix, compiled and review the api's. In that context, maybe that'll give Victor a sense of whether this is a direction worth pursuing and whether it's you know worth than looking at the compiled, but we, you know, was already there yeah I.

J

Need to write this down first, so we should.

O

Like try result or what.

J

All right so then that's Goldy, theorized and presume that we already have the best implementation everywhere.

J

So, what's the semantics now for capture like the idea is that if you constructed it from a span special memory and I access value, that will be bad because no we're allocating essentially right, but if I access menu span then I get better results. Is that fair yeah.

P

It's like the value is the caption of the input string, ham.

K

What's it wasn't, holding a memory.

G

No I meant to the original string. It.

P

Can't curl it holds either a memory or a pin span depending on the API. You use.

P

Yeah and that's one of the things which I mentioned before that yarn thinks we could probably introduce new types like a capture value and the Metro Ally and hold of stuff.

K

Class has to hold of memory, then shouldn't don't. We say that we should return the memory so that you don't need to re-up convert it.

P

G

Okay, so if we have a memory field inside capture, memory has three fields inside, so it stores the original string and.

G

Tweets, I still capture already stores the original strain.

H

G

So it's kind of like it's a bit more expensive to store memory as a field than just two int, so a solution that I've done it's sample. That I wrote is if I already have that the reference type, the string I will just start to additional ins and then on-the-fly produce either a memory or span because.

J

G

Don't have that right.

J

Because if you construct the whole regex from and memory, no sir we'd only memory of char, there is no string in the whole picture. Color.

G

In this case, whatever underlying they have.

R

G

The original thing.

K

Is not the strengthen yes, because either value the original total that's got to be a subset of the match, so the implementation would change. If this changed to a memory, I would I would do it as a memory and a string that is null and the first time you call the string property, be lazy in it, the string and then return, but the so even.

G

Is even if the input is memory memory, not a string, then you need to. If you store this substring this match as a memory, then you have two memory fields. They both store. You know pointer to some string which again makes it a bit more cheaper to store memory and then to ins for the subset of memory, but.

J

Like Vania made, you spend my like we're promoted into the same data. Alright, it's just the one that happens to be the representation of the string.

I

J

The other one is they see it. Sliced memory right, yeah,.

S

J

I'm not sure if I understand so you construct the regex. You say match this regex against this text. This text may be a 4 kilobyte text block and it's in the shape of a read-only memory on the jar right now when the regex finds a match somewhere in their buffer. The capture represents that particular location, and so there's no to the so. The Vania spammy expose here is a subset of the original memory and then same goes for the baby. So they said the Vania was just the substrate for the whole thing.

J

So maybe oh man, you spend a logically the same part of the buffer is just the problem is in this finger was interested in the span, representation and I think what Jeremy is saying is that you know be able. You have a read-only memory for the whole thing, so creating a slice we'd only memory for the sub part is relatively cheap, but if we only give you a spanning, you wouldn't need a memory. Well then, you never can have to copy the same to your memory, because you can oh.

G

Yeah I want the only thing that I was saying is whether it this type was good store memory inside or it would start to ends so.

H

Currently, it stores the overall string and two hands which sounds an awful long white memory.

R

G

Easy we just replace it to storing a memory, but that assumes that we always have this string and emo is saying that we not always have the string, because sometimes you start this whole regex operation from not a string. It doesn't matter. It's.

S

H

H

So if we change.

K

Changing the input to allow passing in a read-only memory instead of charge, then we would have the read-only memory of char would be what we help in the capture. The string property will continue, allocating every call, because that's what it does, which is easy. We just do the memory to extreme okay and every exit.

H

We probably also stir every staff and then, if the length is the original, if the link is the original string, it returns this, but I mean you are never able to use back by a string anyway.

H

G

This just is a memory. It's just exposed yeah, it can be optimizing, many different ways. I mean you know. First time we do the comparison to string. We could now store a memory of charge that owns to this string that we just created now next time you call to string, it doesn't actually allocate the string. It just gives you the string.

I

G

Don't we let you get to shrink back remember well only if the memory the index is 0 and the length is the length of the string. Well, we give you the skin idea which an exit.

D

G

Then you need to slice if this is not what you want to return understand.

K

Somebody could observe the effective. Oh, you called the trimming one.

K

J

That you were saying yeah.

P

Could you please open the link to PI at the top one.

D

P

Go to the changes and stand spiritually capture class.

J

P

Go to the changes and search for the capture class, because that will help a lot understanding what we discussing right now now you can search by just typing key. Ok,.

A

So what should I look for this.

P

One yeah this one here you see that sometimes you accept pin span, and sometimes you accept the memory. It's not always in memory. It's sometimes stand because, for example, when we call we place a split, we don't need to allocate the memory. We can just use the span because we don't return the match if we return the match we need to. We need to have a memory, but if we don't return the match, we can just operate on spans.

J

But what does it logically hold on to like a wall pointer, yeah.

P

You can stretch with just stretch from Emily.

H

L

J

J

Guess it does what the name suggests.

P

So there's a reason why we expose the value span, another value memory, because sometimes we have a span of not a memory.

J

You return to capture our instance. Now it holds on to a pin span, no.

P

No, we never so we only use pin spans if we don't return an object to the collar. Oh, it.

J

Shows we currently pass it around between different things, yeah.

P

Because rackets, replace and Vegas split uses also a match internally.

K

I think we really should get rid of this and just use memory. It's gonna be really easy to completely destroy ourselves with this. If we leave it around well,.

G

Use memory but help so if he just has a pointer to this drink, what does he do it.

K

Doesn't you never all all the public API takes same objects? Okay, so don't drop to a pointer. Okay,.

G

So you're not saying just use memory, you basically say don't use pointers.

J

Then, though, I get that but I mean I think to just echo is concerned, my the problem is partners to search for.

K

Countries, it just means that the the match that was gonna take a read. Only span of charm needs to take a read. Only memory chart.

P

It takes it, we don't need memory of a child. Okay,.

K

So as long as all the API takes memories in a knot span, then we don't need to talk about char star, yet you're ever yeah.

P

But we have some ApS, which takes pain and some a past six memory right.

K

So so the ones that currently take span will take memory, and then you give it a try. Did.

G

K

To call to the data you.

G

Can imagine that there are some spends in the ideas that are like just temporary? Like you know much them is this thing? Maybe a algorithm doesn't help to the to the value. Then it doesn't need to be memory and in fact, memories. Let's save this man in this case yeah.

P

It doesn't make sense to use to memory yeah I.

K

Mean the problem is I mean yes, I guess if the when the entirety of the public API is never going to give back any delayed access, then then that's one thing, but it's very very easy to get to lean into a pointer too early and then now you're, pointing it unmapped memory using using spanish just.

H

Kind of viral right, because if you take spam, that anything that you return has to be a restaurant, if that's referring back to you yeah.

G

That's why what I'm saying is like if the argument that you take in you store for later yeah, it should probably be memory, but if it's short-lived I would keep using span because span is safer. Well,.

H

In in this case, as you were discussing like time management stuff, this would be easy. Correct, like this, doesn't actually do any more work on the background thread. Once you call match, or once you call right so I mean span, would be appropriate, assume return, values, I'm, trying to figure out the in live scenario is you're going to have.

P

Yeah some some cases like, for example, if you call escape or on escape.

T

String why, with needs to return a capture like right.

J

So near being that you have a pattern or you have a string that you may use inside to construct a penance, you want to escape everything that is magically projects, yeah.

K

Well, I don't see that meaning to use the memory or in Japan this isn't returning anything. So this hope you find yeah yeah. This is his match, would be the weird one, because it would that's when it's presumably going to need to do a unified representation to walk through the rest of the engine.

H

K

Being that it doesn't return, I.

H

K

Like that's the reason you would need a bifurcated implementation or the Oh do a span. This is fast, then the inside says stop I need to go rent. An array is I need to be safe from here.

G

By the way to me, it's not about this, be spans are safer than memory I'm, just a.

T

Drive lifetime.

G

Issue I'm trying to so.

T

G

Makes the existing.

T

G

Makes sense with like it I just comment it on like because I don't want to get like I, don't think the right guidance is. If you don't need speed, take memory because it's more flexible, because memory is yes, it's more flexible, but it's also more dangerous and.

K

Yeah I'm just I'm trying to avoid the situation where we need to use pointers inside if we think it's sensible to avoid it and to have an API that says span. So you think it's fast and then the first thing it always does is: go rent, an array, the size of the span to copy into it. So it can use an inner.

I

For the unit itinerary, yeah.

K

Except that it can't print it, because if it's giving you back the capture, then it would have needed to have allocated so every call to the span. One is the meaty make a string and then call the string. So so then,.

J

Someone I think, is it? Is it basically when you call match so it is match? You know that you're never ever going to return the capture, so it's completely fine to just pin the span and then have the rest of the engine basically allocated stuff from the heat that we know can never leak but, like logically, once you've returned. These objects are all pointing to network. They should never access anymore, I, hope.

G

You don't need to.

J

Pin this map, when you kind of do.

H

Because you cannot really pass the span around, you turned into a pointer. Your internal structure all have to be restaurants. Yeah use my internal implementation.

R

Yes, you mean that their implementations.

J

R

They Nicole would you put the span inside of fields.

J

G

What what are those glasses you store so.

J

Imagine this so imagine think of is match effectively is doing the same thing as match, so it gets back a match, collection whatever and it just checks for success true, but they see that you call the interesting the construct, an object grab and you just you never leave that object graph to the user.

J

G

I understand but that so if we were doing the implementation redoing, the implementation I would like to avoid it. I mean the idea behind spans is that you know like it makes the system safer, because now we don't have to use pointers.

I

It's not the other way.

G

That suddenly now we have to have much of pointers and hope that I will produce what.

J

The problem that you have those like I mean if you look at the allocation cost of the object graph, there's some sizeof operator being created right, but the input buffer that you may running dividers against, might be ginormous right. So let's say you have.

I

J

Megabyte X file right, so you know in order for us to avoid copies from the capture stuff back to the original thing. By allocating strings, we kind of need a way to do that, so we can either say screw, be always using memory of T, remember of char and never use that or you have to store a pointer, but it's one or the other.

P

You can, I think,.

J

P

Think what Christopher is saying makes sense, because I was playing with the idea that to change Intel internally the structure tree to be UF struct. For for these cases, where I don't return dimension, but just introduce a new type and use that one sure.

J

Yeah I mean, if you kid replumb you implementation to basically never hold on to span inside of a class and I guess that is definitely beneficial. What I'm saying is that there's too many we can't do that without massive cost. Then I don't see an alternative to what we currently and which is I. Think.

K

J

K

The way that you would do it to make this band work- and everything be saying, is in once you get inside, the thing is match calls match with and don't allocate the return craft, and so it gets to go through all the same common codes. It just doesn't return into the objects, so then it can be playing with the like. You give it a span and a memory it like the memory, but.

J

But your proposal is like similar to the other one which is basically assuming you can change. Implementation to do is enter that, like you know, say yeah, so.

K

So I guess it is, is possible to avoid the pointer and to have is match and match share a code base. It just requires extra plumbing and, if willing to do that, costume then span is the right type of the shape. It's like we.

O

F

Mail, it would.

K

Have to write API and then the wrong implementation right.

U

Presumably, that cost also comes with additional perf benefits for the is match, yeah I'm sure that would help, and there you know the just want to make sure that I call out the specific scenario. That's interesting for the file matching is like we're doing very small strings so is match. Is super useful to be super performant, so the.

M

One thing I was on vacations.

U

J

Oregon and so let's say we can't change the implementation to make it restaurants or villages of this. Let's say we can't do that politically will be then either say we never take the span ever and we only take memories in the public API to avoid having storing a border all we're doing what Victor is currently doing, which is me story, basically pointers, Oh.

J

We prefer etiquette.

G

Why would we not be able to do the implementation? This.

H

G

A cost: no, what I do.

H

To do that, how he'll order the strains that you expect to be input to this, the the read-only Saanich are like what like do. You expect to be pasta for.

U

My scenario: it's 2 6 255 characters so.

H

You have a red, localstorage string and that's your buffer, that you just keep writing into every single time. You all natural, no.

K

I mean like well that's what Jeremy Kay needs for file system like the average color of this think exactly the anti pattern of dictionary contains key, followed by dictionary, give value someone's going to open a hundred and eighty-seven megabyte text file, they're going to call is match and then they're going at home at.

U

So yeah I actually had my toyed with that idea, actually like being evil with strings, and you know, while the tutor 55 character thing is the normal one, I also like with glob star stuff, you actually have to look at the full path, stuff and I.

N

M

About it, like cash I thought about well, maybe.

U

I could catch the string and then can that thing and fill data into it and like well. No there's just too many too many common combinations to throw in.

G

Question before we know that cannot do it the right way. I would not explore how we do it. The wrong way is that.

J

The wrong way it's more about I, guess what our guidance is for those cases like. Is it basically like don't do whatever so rather not having any span, and we don't like memory overloads or boobies in decades just accept memory. The word pointers like like I'm, just our current guidance is it's.

G

An async operation to take memory, if it's otherwise you'd span and maybe because, if you need to store it for later than its memory, otherwise it's fine. Here's.

U

A question can we like add the span. One later is an overload next to memory basically well yeah, exactly if match starts with memory, because we don't have the implementation and then we add a span one. So.

G

Tweak, here's again my my cathode movie, but it is I. Maybe I used to take memory are less say, I, don't agree, we can appoint them. We should avoid them so I'm, not in favor of let's add memory, because it's easier and then that span no I think it.

J

May extrapolate from what you said: it's not about a single zone, I think it's more about. If the thing is stored for later access right.

J

So it doesn't do it for it's meant, but it will surely do it for match right because from your basic, your back and measure magic collection. So the other alternative that we have is by procuring the API service, similar to what Yann suggests. We add capture, value and I.

G

Also thought why match needs to store it. There is the value that you, it just needs to a.

G

K

Will give you back the entire substring that matched the regular expression and captures each of the like parentheses groups inside right, so match tells you. This is the substring that match this pattern, so it is, it is complicated slice, so.

M

K

To get back a it needs to give back a span, but since it already returned to class, it needs to get back again and.

J

Our problem is, we have existing API suppose it happens to be classes right so like we either have an option to say the old talk in terms of memory, because it's the only thing you can stored in a class or we say no school, we add new types capture of a new match, made it whatever that are restaurants. Well,.

C

Helping and again.

J

Do that well, we would have to probably change the API shape in something sensible, they'll be flat there or something.

U

But that's that's where you do the innumerable.

J

We basically have to say you get back a basically this panel, you get back, I lose the bathroom every for that sale. Well, it would be you pass on this panel, everybody to it. Yeah it gets on safe, but I mean like ignoring that for a second. That will be the only way we can use spans everywhere in okay, so yeah.

P

And that's also what Yan was proposing. That's.

J

I agree with you that I think, once you see trying to replicate, but current mention capture, give you and do it as best tracks, it would not be a knowing the fact that we know that we have the API service for the results. It also is messy in terms of you know what the API ship it's, especially because when you think of the typical interaction point of spans, you give it a span that divided into it is not large enough.

J

Like no I, don't know how we would store this on the other hand and replicative about.

K

To be matching the whole thing, everything.

J

K

To be a win32 enumerate API, you call it and it always writes one more into the next thing. If you're doing, then you call it again, you yeah that case. You really want to persist in red, X and.

J

Actually, delegate yeah, yeah I, don't know, I, don't know how much of this will be viable without basic fighting in humanistic, yeah, I think that becomes so file at that point. It's like okay, there's a new name spaces with you, a peer I shared because that's so I know.

K

You read Rick's, so so it's a Christoph, the question I mean we've talked about how his match could be done to be a span right now, but.

I

We're gonna have to the.

K

Point coming forward of, as we start evolving things when we have existing patterns that we need to go into, even if the right shape of the API is span. If we have a heap limitation, do we take the memory sure really if we basically run.

G

Into a case like it's not just you know, cost.

G

Sure saying like well they're, you know the shape of the API that we want and we want to have consistency with existing API so that our classes and store things and whatnot sure and then I'm fine with memory. It's more tricky to use those api's, but it's fine. What I don't want to do is well unless the cost is like enormous month, but I would not want to say jeez. You know this is this is gonna be a week more work, let's have a sub optimal API.

K

Alright, so you know from the pre-code review I think both Levi and I will get very snarky if the final code contains char star in it. So it's need to make sure that the implementation correctly sports fan in memory, but otherwise.

J

Like you said, memories, let's say that's fine, but I think we agree that it's a little safer than water is apparently right. So, yes,.

R

Fine, it's safe memories, fine own memories, well,.

G

Except, if you have memory, you don't know whether there's no.

H

It's an abstraction that you don't know if it's backed by something.

R

It's a gun, it may or may not be loaded. It may or may not be.

R

G

But I wouldn't call it bad abstraction, just don't point the gun and don't put the trigger. Maybe there's no bullet, but you know it's not worth trying it snatcher action. You.

S

Don't know where the guns point you just have a trigger no you're sunk. Every time you also have a trigger lock. Yes, retaining.

K

What was either so better take goddess to oh, but now we still need to decide. Should capture return, the Spanish bear I think since it has the memory, it should return it following our best type available. It.

H

Sounds like for guidance, we should be saying if you have an API that needs to return like a substring of us and would always take membranes that expand, with the exception, being things like sand. Also, if you need to write out outputs apart, take your destination operand.

S

Yeah I guess it's a trade-off here. Yeah I mean it's usable. This.

K

S

K

Deep safety and performance- it's a function then span this like span slicing, it's fine. The problem is this: isn't function? There's an intermediate state object. It gets passed back which is on the heat so that moves it to memory to memory.

H

Due to the existing implementation, well, we already said that if you are, if you're returning the past commercial, but you can't do that with rough trucks, you have to take it as a match as voters. At that point like what are you returning aside from zero boss, it then the other thing is now. You have potentially issue of air somebody's calling match, like regex, does Oman work that process the data and then, after doing a lot of work, says I hope I had a trillion matches yeah, you should tumble and try again.

H

Okay, let you do more work. Oh, you should try.

H

For, like a squared.

G

Well, when we can say at Oriel, they can create a collection of spawns of child.

H

Right, Stano sanatory, you can create a span of I.

H

Haven't that I'm not yeah.

G

We can add: okay, yeah.

H

We can span can't be used in generic, can use a restaurant at all visit.

S

G

Well, I will say: I was not saying that we, it would be a genetical job, but even non-genetic one. You cannot because, and is you cannot see on.

K

You need to be a an enumerator API.

K

G

Could be a very large.

M

G

H

C

K

Okay, so capture should return a memory and then physically the value memory, yeah.

G

So I was looking at the name of this. Property would be nicely together. Genetic name convention for things like this and how you slice, yeah I, was thinking about buying slice, gonna, see that you're not, but it may not be a slice, but.

K

It might be, and if it's the entire string, that is still about it with poison.

E

But in this particular case, won't value being null like it's a little strange to say, you're, taking a slice of something. That's the.

K

Value property currently always calls substrate, so how your property would just call value slice Tuesday. Well then, it's not a slice, it's always the full thing. No, no. The the new property would potentially be a slice of the original implement.

E

What is the, what is the value property going to return? The.

H

Same thing, that always happens the value property and the value stamp property, or whatever we call it return the same data exactly so how.

K

Is it a slice, so value slice is a slice off of the original input? String value is a copy of that data. It's.

G

It's like this engineer that went to buy bread and milk. Do you know this joke.

K

So Steve, the implementation of value right now is essentially capture is capture is memory, it holds the original string and an offset in the links. So the implementation of value is same value, dot substring off suddenly, so that will return will change capture to hold a memory. It will return that memory dot go, allocate a string and stuff the value spans value slice whatever we want to call. It would just return that memory. No.

E

I, get that but value dot length will always equal value spam dot, length, correct.

K

But value already has an existing somebody could be annoyingly depending on the fact that it doesn't return the same reference on multiple costs. I think I think Stephen feel same slices, a terrible name exactly well.

S

G

I think Jeremy is saying is subset. Asset is its own subset yeah.

E

But that is suggest that it will mean calling something: a sly suggest that it could at some point be smaller, and this will never be alright.

K

I was just trying to come up with a property name of when we need to add a span or memory next to an existing string or array that may be slice is the right suffix on it, but.

G

E

K

Mean we could we can take that argument and say well then clearly it needs to be a memory representing the same reference eNOS of value, but we're not like we're intentionally not doing that because we don't want to copy value, has existing semantics. But if we only look at the semantic value that is returned then introduced a derived class and use. Remember writing.

H

K

A value, then you need to cast so I'm fine with value.

T

Memory or value slicer, can you guys read that like this is basically what I just wrote down as.

P

Summary of that make sure the last part is. The last part is wrong, because it's natural or they accept suspend all.

J

Right, that's what I'm saying earlier that we shouldn't based on the current state of the world.

K

So it's his match shouldn't take a span if we can't sensibly operate on spans, but X yeah, yeah.

P

K

We can, since, if we operate on spans than it should take a span, yeah.

P

And I would say it can. Okay, but the thing is it doesn't today right it just it's, it just tends to expand and works with the pointer exactly.

J

We said we don't know what the optimization should do that all right, so that that would be the Jeremy document like this is not a sensible way to operate on span. I would.

H

Just need to change.

H

J

Pay but I don't think you get to separate them like if you care about security like I think the Christian is like it's a function of like how. How fragile is our implementation and like how it didn't.

K

Have modes where both returns persisted reference against the memory you gave it and just a boot like if it was always the span based thing, that's fine, and if it was always the thing, that's fine. The problem is this: one bridges, the gap and I just want to make sure we don't end up with an implementation state where we start leaking pointers to deallocate yeah yeah I mean if we have to make the trade-off. We got to you. It's just I mean I.

J

Think this is the other thing that was then about like I I, really think we need to take it to home of X and not especially the engine discussion to see whether you know is it even worth trying to like massively fix reg X, or are we not ending up in a world where the basic create an entirely new API surface anyway?

J

In which point it might be cleaner to Disney projects as it is, and just have a new API on the side and design that API above the constraints? And is we don't end up having to do this crazy, X, because I think it might be first I mean even with yun suggestion like, even if we could pull it off I think you make regex substantially worse.

J

If we now I have two different return types, two different modes, but they are all in the same type, and now it's really hard to reason about what the heck is going on. There seems to be potentially a words trade-off on the.

P

Other hand I think we need a separate meeting for that exactly.

G

So but I think that you know well, but it would be usefully like forget about implementation. It would be useful to if somebody sketch like what would the API look like, because if it's very similar, maybe it's okay to just you know, bolt it on top of this. If it's very different than I I kind of agree yeah, it would look like a frank and design of two different ideas put together.

J

And that's the one I don't have a good handle on I like talking to Dan. There were a few things where you know we have this concept of about what it is. I think it's was it captures like. This is one thing that the regex engine has indoor netball that nobody else has which makes it really hard to replicate it over the other engine. So there's certain things we can do I forgot, but there was one thing where we yeah. We can't do that if you, if you allow that we.

G

Also have many features that make the regex a hundred times slower for the common scenario, so we can support some corner case scenarios. I know at some point. Somebody did an analysis, I don't know I, don't remember now the details, but there are some features that are just crazy, expensive, but.

H

Presumably is when we're looking at the regular expression string, not the string to be matched at the actual pattern is.

I

H

To say, oh we're off music.

C

C

It, let me just add the one I know what's going in with me.

C

Q

They make sense, Victor yeah.

P

It makes sense. Could we also briefly discuss the other points which I pointed out.

P

Let's go down because I yeah.

P

So we all already discussed the first one, but the start at overload, for example, is interesting to me.

J

Entirely sure what what does start at do like what does it mean.

P

It means that you, thank you, you yeah. It means that you only start at searching at a specific location.

J

Saying, like the regex pattern, will only start matching. Let's start at not anything before it, because.

G

You cannot slice drinks cheaply, it's probably. Basically you pass an int and says you know, search this string from this location for span. It doesn't make sense because you can just slice this path.

U

Backtrack from that starting point.

G

Yeah I agree. We, we kind of you, know many AP ice in the framework that take today array or array and means we now just have. You know same method that takes a span we don't add span to take. It takes an int, I mean span and I need.

O

This will help you acre.

P

Yeah, let me check quickly what I discussed with young. What was the exact issue here.

M

Bunch of huge words strung together, they don't know what they mean.

C

Alright, to let the excess almost think about time, they are.

P

Yeah the thing is um started marks the start of the location where to start searching.

P

Yeah it it has different semantics because you start searching, for example, at position six, but there, which no input string will still be the full string for cases like Vegas split and where you place. That makes sense because it returns. It always returns the full input string.

P

O

P

um You call away extra place, we've start at six. He will still get the zero to five slice as the first as the first capture are.

G

You saying it just doesn't replace in the in the slides, but it operates on the whole slice. Yeah.

P

G

P

J

It's probably also two for backtracking right. They match the thing of the text before is part of the backtracking right back, essentially yeah.

P

That supposed to be relevant, yeah.

J

That so that means we can't do this slicing, because you need to basically able to see past.

G

Yeah yeah sure, that's so that's it that's! Basically not this operating on slice is just saying operate on the whole thing. Just do something special from this location, yeah, yeah.

C

J

So what does right up left doing? Redheads, we literally like read the input from from the end to the beginning, yeah.

P

It's not searching from from the end at the end.

J

How does this work with like utf-16.

H

I'll say if you're reading from you just look for a low sir.

H

J

That's another interesting question of: like you know, let's say we want to support utf-8 by text. Matching like today would be school or parametric. Then maybe.

C

Which another reason I'm here you're going to look at the API, doesn't do that.

J

Yeah I think in this case it would not support my two left, probably for you today. They would just say you know.

O

M

J

If so, what's your question anacondas, you want to know whether.

P

Yeah, if you, if you call hwagok split, which returns a numerator, the thing is, if you part right to left as an option in you can only get to return, they returned in you, enumerable objects from right to left, not from left to right because you start matching from right to left. So in the current implementation, we store all the return values in the list and then call reverse on the list. Then returned return objects from left to right, but as we introduce an enumerator here, we can't do that.

C

U

You can still do it. It's just gonna copy at all. Yeah.

P

But the idea is with the numerator that you only do it cost, which you need in the sense of a loop, for example, sure.

U

But like having a specific area from right to left, just because the way where you left it is like it's a big, a documentation problem.

P

J

Though, in those cases I'm also cable throwing because you can just say you know you specified by the next options that given the data that you pass in, doesn't blend very well, so you just say: if you really want to do that, that's why I didn't call the other elbow or.

U

J

It seems pretty bad to just magically deteriorate performance if you passed in the wrong arguments, especially because you're using api's that are geared for not go ahead.

P

Yeah and the last point is from Stephen the thing that I exposed it.

E

I'm trying to page this back in oh right, so the split method that Victor's defined is returning in a numerator, but in order to be able to for each hit since for each doesn't work with a numerator. Only a new Bulls he's exposed to get a numerator method off of the enumerator that returns itself.

J

It does seem a little bit like.

H

J

I'm thinking about that, the cabinets I think, though they always trying to be resilient to when you call it twice so, and this one will presumably not work with you when the second gas or getting writer, because you just return the same answers and bad things are happening in.

U

The places where we're already using that's just pretty copy from the second call.

P

Yeah it's compared to copy, but this is not.

G

A numerator in.

J

Like positive for each I mean so pretty straightforward pattern like.

J

G

I'm not sure what you're saying, because you can do it now.

E

You can't before each doesn't work with the numerators only anymore goals, yeah.

G

But you can have a method, get a numerator and then return yourself. Well, that's what he's doing, but the problem is now. You are I, know.

E

The focus that's working or it's working around the lack of a language feature I think that's what.

H

Detroit integrating over when you call for each over, because I am rater Albany house state, it has position, presumably whoever thought it already has it inside of it using statement orange yeah. So now you have like now you're intentionally doing things like the whole disposed. Double integration.

G

Yeah, so this is the negative of you know. A single value have been involved. Well, so the next solution is, you need to have two different values. You have the collection and the numerator, maybe I'm, not sure how it's a language limitation. It's just physics limitation.

J

Ensuring our I mean like we always you know the motivation to add getting writer mastered because you you wanted to write for each lobes over that or.

P

Yeah I wanted it to be as clear as spam itself, which also has a good numerator, but works kind of different yeah.

J

The registries and winners that you typed in right, like that, doesn't exist today. Yep.

C

If their public was internal, I spell it right in the design which.

P

E

Fast, it has to be public because so it's a rough struct who.

J

Would return that from there so.

P

The split call do I get split color, which answer we're next door. Splitting numerator.

C

What is need another face if he never returned in something is logically new move on the heels of spats and it's a birthday another.

M

C

E

Only place we have is spans, get a numerator.

E

Well, you're, if you're for eating it you're for eating over the span, so it doesn't have the same issue. It's for each is going to end up calling get a numerator on the span which returns the span a numerator or span.

C

E

Numerator, oh, it.

C

Was a rough start.

J

And making it a split in new football and like handling a numerator meaningfully, it's not possible. There's.

P

This possible, but what's most original for that.

J

Consistency index doctors works.

J

Fast response to that, because I mean acoustic is Katun right, it has it been weird for each over a numerate or in general, because it does make the question of like what you want the semantics to be. Do it I can think your particular case. It was actually like for each. You know bar item in split its looks, reasonable and but I like in the other cases that.

U

We have this where they're joined, we called developed in innumerable yeah that they're. So do these split innumerable, not splitting numerator the.

E

Other option here is you say you don't use for each you get the enumerator back and you just call move next paradigm. Yeah.

J

True, but every like that is like it's unfortunate now, where you go from string to we'd only spend like your co-parent completely change, then.

E

The next option is we mean that the issue of whether for each supports numerator comes up like every version of c-sharp. We can revisit that as well. The language thing I.

J

Think, that's probably a better discussion to have and to heck at the road I guess we.

U

Change this into split enumerable.

P

E

The question constants that to this is a case where we're artificially injecting in a weight of a numeral, in order to make this thing for each of all, and if we believe that's going to be more common now, maybe we should revisit it in the language. If it really is going to be one-off things like this, then we can introduce the extra layer of the innumerable we've.

U

Combined them in other places, though, from a performance standpoint this decent, that's.

E

A class all right good point: sorry I mean.

J

I would say that I mean one thing. It is reasonable, like towards most of the problems. If you I was mentioning, is that to say you know for each works with a method called if the method returns it in your Mariah tour, but you cannot actually for each instance right. Then you don't have to dealt with this whole issue because you clearly acquired the thing. It's the ownership, so really an issue.

H

Per se, because, like.

H

G

Have to know that you cannot enumerate it twice unless it's like you finish the first one and then probably you can enumerate it again because then you know reset, you just implement reset and you can do it. Then you just you, have to be careful, it's it. Suddenly, it's a special time which.

J

Is my thing then, combining it with Mexico might be your the first right and you can only reach it if you never.

J

G

Just introduces new kinds of types in the language, there's a new type that you cannot just store in a variable. Well, that's a new type from a type system perspective. Do.

C

You can you just certain things, don't compose, but that's there.

G

Is a definition of new type of type things don't compose? Is it.

J

Is literally like the same as today like, if you you can totally so, they say say for each bar acts in split right, I can totally take the split thing and assign it to a variable, and it just means that you don't have this thing there. Don't you? Oh, you can't actually do this well now you cannot for each over the instance yeah only for each it in the context when you acquired it, yeah I.

G

Don't know that's a weird type, so you're putting new restrictions on that are not visible in the metadata of this type, because a method get a numerator and you cannot preach it. That's right! Well,.

J

No, it's like signature, other, don't work the same way more.

I

J

A text description with anything like you think it was like, if you say, new fool, I can add braces and initialize properties. I can't.

Q

Do this or instances I.

J

Can only do it by that compiler to start the call I guess it's kind of like you know just basically forcing a certain ownership. That's all it does, but anyway I think it's something we should be looking at. You all right, then I think we're done with this. One I may have poppin or less.

C

Whatever sauce, that was, it good exercise, yeah all.

J

Right talking about spans and apps.

U

So the first ones the ones we added, where we're actually writing into spans. I didn't cross the bridge with full path and the others down below, because it was just more work than I was able to bite off.

C

What even the end of June did we say Vigo of joy, is that what you said, yeah.

U

We've, unfortunately, so those are the approved ones up there, it so the second. The second block, is the ones that I want to add. There's one other weird piece to this and I guess it's also true for joining. Although you can kind of calculate it yourself, but is that I wasn't sure we already had a pattern where we could pass out like cars needed, because we know what the likely count is going to be and I call it out in the notes. So can you try to get full path? You know we'll get.

U

Some sort of feedback is don't like either from the OS or when we know in our own calculations. What it's intended to be like.

G

Kind of a hint of what your destination.

S

Size should be yeah, it's a very accurate hint, but it's not 100% accurate.

U

Because it's based off of the current working directory so.

G

We have, we have api's like this in some of our transformation IP eyes.

J

G

And here it would be, it will be.

J

And then a movie larger or smaller, because we said it needs to be an apple right. It's.

U

There is no upper bound with this one. It can change because of the current working directory thing process wide.

U

So I mean it it's just it's a subtlety, that's there and in a number of api's on either side, and they don't do anything to call out the fact that, like you, shouldn't like depend on the because it means people do this sort of pattern right or they call it with a zero length to get what the buffer size should be and then assume that it's going to fit, which happens correctly.

G

99.9 percent of the time so yeah. So this is a bit dangerous, because if it's marks it will never fail. If you don't have it, the color of this API is like geez I guess you know, I'm gonna try this destination and if it doesn't work, I'm gonna try something else. If we have a method that is not the upper bound, somebody writes called, it always works, they never hear they fall back and then it stops working.

H

Was over it well, maybe do.

U

We have an X bar right as I put it there yeah, so so they've America man, no.

H

But you, you know why you max out is ever like, is fixed to 259, for all time is because people allocate max power.

U

But like it's, it's actually it's more complicated than it is changing. Now, because the above the API is it's very hard to get into a situation where you, where you didn't mess up, but because you have to pass on the size of the output buffer.

H

U

So one of the things I was considering. Well, it's effectively static, so you can change the environment variable and blow everything to hell but like if you do, it I mean system-wide, but if you do it in the process and it's not um her dough that people do this sort of thing. It's like, oh well, I want.

U

My I want my taxi to have a different tempo right, that I'm gonna control and set the environment variable, and they do it some time after setup startup and it's kind of it's playing with fire, but I mean I. I was considering changing the the get temp path to actually cash. The value, yeah I didn't do I didn't do it because we're so late, it's something I would do in the early part of the well.

H

If you, if you have cash, the value that what's the purpose of well.

U

No that's. There is no real point outside of late that if we find in the case in that particular case, you know it'll be something well. Yeah yeah try that, because it seems like a rational thing to do, and then we find out good lord. It like falls over in all these cases where people are doing these, these sort of semi normal things it becomes a. We can abstract the behind the implementation detail right.

J

Especially when you quantitate between crosses season- and they realize.

J

U

They're, you know there are there be dragons, but it seems reasonable. So, first friends, but not something I, wouldn't want to have like bait for a long time before we ship it. So.

J

I would say like if you can give people a max, API and I think it's using business it into API interaction. Is you write the loop right and then it might take you three integrations. Do you get it yeah.

U

Well, the only the only tricky part is it's like you know on unix, it's typically a thousands of characters and mac, it's four thousand, but like on windows, it could be 32,000 characters. Long, it seems seems abusive to me no, for example, right off the bat and it's probably going to be a thousand characters and make the users jump through that hoop I. Just.

H

Start with 256, that seems like a good number. Well, how do we expect users to expand they're like are we talking about?

H

You know, typically, what I.

B

H

H

Why so don't stop calacatta Lou, certainly, and don't do it based on know.

G

What so, I think that common pattern? Is you stuck a log? Try a if it fails, then you read. That's that's that's! That's! That's.

U

What we're doing internally.

R

It actually bumps I thought they compute. This text and I statically put.

H

Like if we don't give people an easy.

H

J

Kit is actually I was not continue. Joking I think it's reasonable to start was to 55, who doesn't see where you, because I mean most pass, probably will be less than two.

U

So I mean that's, that's the typical scenario. What's 260 but like.

J

U

You know the one of the things that would be potentially useful going forward is, if we actually like develops like the value string, building or something that allows people to do the patterns around building off the stack and then I mean we're using internally. But if we had something public like that and took that as a roof.

J

In API you can use, it basically codifies. Oh, so you just basically call methods on them rather than having to like know the actual sizes into our mechanics, yeah.

U

So we have that the value stream builder does that I mean that's what we're doing internally right so yeah to be able to have something like that public would solve a lot of this.

U

Because then it's not your problem to figure out anymore. It's all internal.

E

With a new version of Skype, we can actually hear your thoughts so.

G

I would I wonder whether we shouldn't start with just this API without the hint think about the value builder solution or some other solution. If we get feedback that you know, we cannot do the other solution and we get feedback that people still need this property. We can add it later, but I would add something that kind of it's never happy or how the overload that's. My.

J

G

This one is, this idea is good, and then we either at the like a helper that lets. You grow this this buffer and maybe even do the loop, or we just add the property. Saying like this is there you know expected buffer size that you should pass first.

G

C

S

G

Like pass, that guess what.

H

What they do run our issues to yeah, which is reason we.

A

Should still call path of necklace, we should think.

J

Yeah I mean any Jeremy, but did you add the api's because you needed them or just for consistency or did somebody ask for them.

U

Well, we do need them, although you know getting back to that original point, we would have. We would have some journal ones because, like we don't want to jump through that hoop either we want something, I think value, stringbuilder.

U

Because, like it's pointless for us to jump through that hoop, but the similar thing is there and like we have this in the in a lot of places where we're needing to normalize. So.

J

We improve the API as a propulsive here, because it's so sad like V, okay, this is the minimum we have to provide and it's not convenient, but it does the job. And then we do a separate review over the Vedas chamber. Let's see whether we can prioritize that better yeah.

U

I'd like to push that one forward.

U

Stephen store yeah.

E

U

Are you interested in working on that too.

E

Second, work with you sure.

U

Q

Seen your thing I'm going to thanks, but I have questions with this revenues, be consistent instead of path destination, so justin.tv discuss that there.

U

They are, they are test they're consistent with the existing okay, occasionally with their consistent, would get full path and get relative path. Get random file name answer outside of the buffer, because there was no buffer on the other one, so I use destination, which is our.

G

So I have a question about some of the parliamentarian sure ordered a wise path. He plotted the base.

U

Path because of the way the overloads are, we didn't want to flip them. Oh you mean existing overload, yeah. Well, you see, get full path takes path, and that was the existing one and then the new ones. It's just weird to have it flip yeah.

U

G

Can revisit that again, but so get full path after destination? Oh, it's! A it's! A relative path. I see grateful path from a relative path. The second one baseball, the second one is not logically just an overload of the first one. The same.

U

Thing like what the one is, the the overload one just basically does not use the current working directory base path is the is your effective current working directory? The.

J

First, one calling the second one passing in current directory the base path.

M

J

U

M

U

Is essentially what it is? It's it's. It's filling in base path with director.

G

It's just the one where you pass their base path explicitly. It's just combine correct.

U

It's more complicated than that, but yeah even effectively code itself, because we we only combine them where you don't really have to the combined roles, are pretty complicated on Windows.

G

And try get relative path. Let me parse what it does relative to destination.

G

Path is the full path and the relative it to is basically a base yeah. Why so, how do you get a power from so? Why would we not call the parameter base path, so it's consistent with the it's basically reverse of the one that takes base path because we ship the two versions ago, Oh again.

M

All right well rest, my case.

U

Alright I'll follow up on the value stringbuilder.

C

Alright, so I approve DPS, fugly yeah. Oh.

U

There's a lot of discussion there, because some community members want to contribute why I pushed it.

C

Yep they're just trying to navigate little complexity with the quests and people's the other one awesome just decide if I can figure out do it.

J

C

High DPI chuckling, it was a different.

N

Dprs, so the changes of a automated update, the order post, but.

N

Retreating arbitrary strokes as bites lead to unexpected behavior, either crashes with that and a simple repro is a fuse decimal on whole framework and you try to write, read arbitrary bytes into the decimal in the crash, so fuse.

H

What does he I mean the visibility checks in their context, so, basically you wouldn't be able to you wouldn't be able to make a union struct. That is like a decimal of spite without going basically without saying I am no longer verifiable. Confident this is verifiability really.

N

Also, if you have an internal is like a private field and you decide to change that from a struct and it you're sending it as as bytes. You could cause breaking change, but just anything internal to the private field. Okay,.

J

Say you construct a different file that matches the existing layout and it just cast one to the other, and this way I can meet the contents. Is that very same.

H

It is it's basically, trucks like that's more marketable, but this is treating them as if all trucks are livable. So this way I can basically.

J

Extract a virtual because I can just create something else and then listen to this representation right, yeah, yeah. So.

G

I I I agree that we have a problem, but I would just change as bytes to read on this fan.

H

That doesn't solve this delicious sure and we don't care about verified verifiability anyway, going forward. So.

N

It doesn't solve some of the issues like.

G

N

G

Just lets you read memory an.

N

Example that uh yeah I just mentioned to me, is if you have a struct with a private int, yes, and you call a special that struggle, it.

M

N

Faster on the wire to observe mm-hmm that's worse and unless you change it from private aim to a private employer. And now your app could crash because of adverse.

H

Generalization other first off, if you're doing that, we're going to fail security review anyway, so I'm not worried about that. Okay I mean.

N

Estimate who had this job but.

G

Well, it's say: they're gonna. How do you fail? I said.

T

If you see like oh.

M

Okay, well sure.

D

M

J

Honestly, like you to me, it's a good way to say you know: I can I mean I can do this state right, I can I can stay, I can just take an arbitrary object. Grab go to binary for murder and I. Have a bunch of bias and I can't be done whose utilize private state, then you you have to skip verifiability.

L

Truck stuff, you know yeah, you do you.

H

Know the object itself right this day.

H

You can't do that outside of partial trust, which is by definition or skipping or if I, don't trust your cell.

J

Phone because I as I have followed an object and then object base. It can't utilize all its own private estate of nobody skip anything because of access to their own state. But I write this to memory supreme.

H

In the neck and just beat device out of the interface like the RUC realizable device, I think that's park, security, critical, yeah, okay,.

G

The object itself, concurrent method, foo that writes its internal state, yeah.

H

I think that's weird or 2 bytes with all parts. Yes, but.

G

H

Objects are saying: I am explicitly political and I'm, giving you an API that exposes my gotta go sure.

G

So, ok, so there's another one which is if we said: okay, we, let's not do it for decimal I would be I, am also ok with it. What I would not want to happen? Suddenly binary, primitives, try, read machine and Ian doesn't work. A I want to read an int out of this pan and you have to use some memory marshal API stuff.

N

But I think it's taking it to heart primitive types. We have primitive old, O's I, try read in the UN 16 ula, so those were work files it's only when he is struck. Oh like.

G

If we limit it to you know we remove the ability to do it for decimal or for protract. It worries me a bit more. We should discuss it, but I I was just saying initially when I read the proposal. I think some people propose that we just like move all these api's to memory Marshall and.

B

Even I think only the ones that allow you to get visibility, charts.

H

Would make sense, but none of the other binary.

N

Yes, I lifted out which, if you guys, would change there, it.

G

Would be this set? So ok, let's, let's really know so as bytes changes to read on is.

N

Fine, okay I know those ones are going and they're all moving down interrupt service as much memory. Marshall signature, yes,.

G

I see so what I would prefer is that yeah we moved, we move, we move as bytes that returns the span to memory partial, but the one that returns read on this fan. We just leave it I, don't.

H

Think that's right here, the reason being that, if you're trying to let us wrap this arbitrary bytes, you're- probably going to do the reverse operation at some point. So it doesn't make sense to have one every partial.

G

So let me give you a scenario where I don't played it back, so we have transformation interfaces and they all operate on bytes because they are typeless. They just transform bytes. So if I have a span of chars and I want to transcode it to span of bytes, I need to use this one cast it to bytes, and then it just do the transformation and.

H

Of course, by this kind of a nonsensical world.

G

H

Utf-16 with some arbitrary animals.

G

With with the same, it's a reinterpret castle, let yes yeah, but if there's a reinterpret castle.

G

H

It's a reinterpret.

S

Ass, so what reinterpret cast is, by definition, on safe ways.

G

It can say, oh.

S

G

From chars to bite, I don't think it sounds like I, don't know. What's your definition of unsafe I'm.

H

Saying in the sense that you're taking the data and you're attempting to read it in a format that it was not intended to be read, I think this point is like Jim.

J

As soon as you pimp lip anything right, we have to know what you're doing yeah, which is basically what memory partial is. It's literally a memory.

R

Marshal at serine.

J

G

Yeah I just think that we are taking it too far to a point where, like we can keep going like all IP.

N

Isis can be misused, but the solution be then have primitive, overloads for El Pais that are on memory sessions rather than of T, and then the Octavia just move Mary Marshall similar to let's say we have or read on read machine Indian. It.

G

Just complicates API: now we need to have s bytes, that you know from all spans of all ins like. Are we really buying anything I I think we are going too far, just complicating API and like you're too far a.

J

Minute because, basically all you have to change is the class name right so you're. Basically, you know the exact same thing. The exact number of types is just instead of doing memory, extensions.

G

I, you say yeah.

J

G

Made the API is more difficult to use, you cannot use, use it errors and, as an extension and I, don't know what we are getting in in return. Where.

H

We're basically saying anything that we think developers have the potential to shoot themselves with we're going to move off to another and.

G

This API you access in.

H

G

We have many API cinder framework where developers can show themselves in the foot.

H

And we're all you do memory much I'm trying to do.

J

But the argument goes either way, one which is so, let's say, let's say, I, buy your argument that this is a this is something you need to think about. The downside of pushing everybody to memory marshal is that all memory martially also eighty is that have a quite large continuum from potentially unsafe to holy. This is probably extremely unsafe rights if you point them to one time.

D

J

May make it work explosion to more on saying.

H

If I argue that the people who were doing reinterpret casts are already probably using memory Marshall anyway,.

J

Well, I mean his code today doesn't mind.

L

So that would be a contract. I mean.

S

I'm sorry sister, whether played as to whether your code is even correct: okay, okay, fine code, the code and then.

H

You you just can't arbitrarily go from char to fight, for instance, without with, without special case in your platform, for example like what I don't even know what that could be, but you can't send those bikes across a wire like I mean I. Suppose you can, but now you have to tell the person on the other side, this is utf-16, maybe big, endian, baby little endian blah blah blah like I hope you had a byte order mark on there somewhere sure.

G

I mean now you're talking like we're going from a very simple operation that is very clear what it does and it's super safe. Well, it's seeing the what.

H

What are you, what I'm trying to argue is you probably use it incorrectly because we made a to visible, we.

G

Have so many API, so you can use them so in the product. It's not the bad memory.

B

G

It's not the reason for memory marshal memory Morrow's meant to be for unsafe idea. It's not for 80s that you can misuse I think misses the difference. I think I think I think we should separate misuse.

J

From like the API is very context, sensitive right, so, for example, I can you know GC right is an API. You should use sparingly because if you don't, then bad things might happen right but like if you, if you use it incorrectly, what's the worst that can happen. Your just run slower right, those guys when you're do it incorrectly. Well, you know, depending on what things you reinterpreted it.

G

Might actually be pretty bad? No, it's you have to this. One always works and there's nothing wrong with it. What you do with this data sure, but we have already api's, we have encoders in you know in nothing. We have basically api's that produced by a race. Okay. Now, if you send the bytearray- and you know like you- put the the bytes in the wrong order and send it over the wire you're gonna have problems sure like API.

H

Name explicitly says, like I am converting from utf-8 to like utf-16 big India, but it's actually part of the API name like if you do that and then send the bikes to somebody who's, not expecting it like that's on years.

H

G

Here, I've done you, if you just take the who's.

J

Your what do you think these are because that's the only span off my fancy span. That's somebody know.

I

You can get essentially.

J

The raw bytes, yes and I can know whatever I want so and so that so the thing is you I think you'll end up a memory wall and just by the correct, which you know we have on Danaher, with disabled, private reflection, at least in store, because people misuse it all the time and it to get access to private state. Now, when you combine these, the bottom is each of those API is has one it's like the sequence of this one followed by week, machine and you may basically click once tracked to a difference tracked.

J

We just mark mark all fields, public and all sandy can read a betray to you. So.

H

Help help out those just looking for us and if we can say freedom, if we can say as bytes of T on every single architecture on every single fitness returns, the exact same potable representation.

H

But if you have a dotnet application on machine a that calls as bytes and adopt that application on machine. Be that, given the exact same T calls as fights and they return different things like that sounds dangerous. To do.

J

You don't even care about very well that the fight layer may be different in that cause. Bad things happen.

G

Well, the the compromise that you proposed is not doable, but.

H

What's the advice, if you're not only might transfer that somewhere or do something like what I call well, you might just do it yeah, but like presentism, presumably you're, not calling it to use.

S

Our transformations to use our transformation.

J

Well, in this case, for example,.

N

J

Want to do it on the same machine.

N

When are you in a transfer from one representation to the other? Well that it stays on the same right? Let's say yourself, you are in operation, you've got as bytes and you take operation, York Airport, the bytes in them cells. You just get what the result, which is the input index. You found the values that yeah but index of. We already have an implementation, but it doesn't example that some people.

B

Do that in terms of uses onset right.

H

Like that's already easy.

B

H

But I think the question is: if you.

J

Yeah I think your argument first party visitor, probably know you can apply for anything. You do it's the one thing we do with spans, basically to push the envelope on like how far you can go a lot of actually doing like.

H

You know and I'm totally fine with enable in this scenario. I just I'm gonna happen later, might don't want to expose it as something we need people to do unless, like they know, especially if it's an extension method so.

G

Another thing, okay, so another proposed fix I, would be fine if we move it to binary primitives right, because, basically, it's the same thing. It's like there are many API. Is there that you need to know what you're doing with and E&S and and in the name we could say as machine endianness bites I'm fine with that too. It makes it very clear that it is just give.

H

Your device initial status, also business like if I, have a 32-bit and 64-bit app on the exact same machine. They might still return different data when I call as byte city.

G

You mean first tracks that any.

I

Questions padding.

G

And stuff like this is that what you're saying yeah.

B

So that's that's! Okay, so.

G

We get another tweet before I mentioned that I would not be happy if we had, as as bytes for each primitive type to memory extensions, because I feel like suddenly like we are adding thousands and thousand overloads the binary primitive. It already has this done for the very same reason. So again, if we move these api's to binary primitives and make them work just on the primitives I think it's fine.

H

So, like I think this is only we should want to discuss more stuff like that on, because you need to be able to.