.Net Foundation Design Reviews, 18 Jul 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: .NET Core Design Reviews: Add initial Span/Buffer-based APIs across corefx (Round 2)

Description

Round two of walking through the proposed API set:

https://github.com/dotnet/corefx/issues/21281

A

Perfect all right so open that guy and then I think the other edit.

B

So the problem is that the second one we will fill all remaining time, which is why we should probably start with a shorter one. Yeah.

C

Okay, so the post, API changes to add a new method called, is path fully qualified. That adds that's a little bit more clear than what is popular history so right now, all.

D

Right I asked you to share your screen whenever regions.

A

B

You will Mike so if we actually the chance hearing what's going on.

A

Let's check here we go so.

C

Right now, this path routed returns. Whether or not the specified pack contains a route, and the assumption is that if it is passed through to returns true, then the path is not relative and her most cases. This is true, but there's one case where he have a directory with the rear of the directory like C, colon and then the path without a backslash.

C

Then his path will return true, but this path is actually relative, and so these function, that is populated, returned from being counted as false, and so the suggestion is to have a new method is possible right. That would tell you whether changing the current directory will change the path, and so for is baffle.

C

Aqualified started holy it's and backslash, and this would be a little bit more clear than what is currently doing, and the good thing is that this is already written as an internal API, so we would just have to make it public I see.

D

Can I ask, is there the fact that his path rooted returns? True in this case, that's desired behavior or that's an artifact of the past that we would fix if we would like if we could or.

E

Lieutenant doubt it.

C

Yeah it's desired because it is, it does contain, like the name of a root directory, but I think it's just that a common usage is to use is bathroom to determine whether or not the path is relative to.

E

Put a different way: would anyone wanna call is perforated after we add, is about fully qualified? Oh.

C

D

Wondering if we actually did a new method or whether we could just fix his path, routed.

C

Well, I think I would.

F

Expect it to break a ton of apps if we change it, because if people are actually looking at it, this routed, which doesn't mean that it doesn't include the current directory. Weirdo just means it's routed to a device, is what it's supposed to mean or what it does mean at the moment. Is that linger lying on that behavior? Then it would break them. Yeah.

B

I mean the question is: what do we do for like C, :, food or txt right Cinco.

G

And my der is explicitly in our things of the list of things it returns. True for on msdn turns true for that.

B

That's what I mean so, basically, if we, so the question is make.

F

Sure the more specific problem here is things that begin with a backslash, hey.

B

This is, this is basically a windows-only concept. Nobody else has that you want worth that then I think we need two different api's, because otherwise you're like okay, the past is not rooted or except it has a drive on it. So like that, that seems very weird I mean that's the behavior that we could add a new API like you know, as as drive route or something, but then the question is well I. Guess it's pass through. This is home with.

E

The so the statement, true if the current directory does not affect the evaluation of the part, the meaning of the path is when we find on all operating systems. Yes, what? What does his pathway to do on UNIX right now? So for that, because I don't have.

B

Great books on Linux is like space is starting with a slash right there. Anything else is relative, and then this is where you go from the device, so we're at tilde. Basically.

A

D

F

Recognized in our UNIX code tell.

D

There's a shell specific character so.

F

The can you hear me: yes, all right, so the specific the real issue we're looking at here is the one with the with us. That starts with a directory separator.

F

So if it starts with a directory separator, which we just looked at the UNIX one, but on the windows side, that's considered ruined, but it's also relative.

D

Weight you're saying that that path that is routed returns true for on Windows, for /foo yep.

F

And that's that's always been the case and that's and because of that right. It's really hard to change so and.

E

We could fix it and then make it quick, but what like working is app context right. So you, if a library relies on it, you can't change your applications on pursuing suite. Well.

B

I would also question whether this desired behavior I'm.

D

Wondering like what? What does anyone use path but is routed for if it returns true, when it begins with a slash on Windows.

E

B

Certainly written code like this into a store up, my own make relative path function, which you also know have a bit didn't used to have and I basically add cool. Like you know, his route like take, it is, is otherwise make relative and I'm. Sorry is, if yeah, if it's not make it relatively so that all of that cool was kind of like busted in the past.

E

What about did you consider an overload of is routed like we used? We ended spring split with spring split options. Remember that, like yeah, you could have something. Light is routed on I mean it's I think this is lame, but it's funny. If we could, we could have an overload of is routed right because I I'm thinking of you know, path, dot and now I, don't know which one to pick. Whereas if I had an overload and I know, I'm I'm gonna pick his path routed and then all.

H

E

Options be boolean, do you really mean it yeah.

E

F

Current definition of his path route, it is, does this pass train contain a route you know? It's not. Is this thing relative? Everybody takes it as that, but it's not the way it's defined and that's it's an unfortunate thing.

G

Yeah, the first real caller or searching on hub if people are using it like if it's it's routed, return, it otherwise do pass that combine so they're there they seem to all be using it for is they're, not his relatives.

E

If it's not fully qualified, is it always correct to do path combined with some group to make it fully qualified.

F

No because that's C : case that you're talking about so.

E

They're getting it wrong again.

G

If you combine current working directory with C : food text, do we not give you a thing on the C, Drive I believe.

F

We're fine filled with that one. So.

E

I still don't know why anyone would do that, because if you wanted a full part, you just you get full pass on it unless you think that's slow, which it is and then you might do is route kid. And then you get yeah.

F

That's that's the that's. The real usage for these things is to optimize out of like doing full normalization to.

D

Jeremy's question I: don't think we are out I! Think if I remember correctly path. Combine checks is routed on the hour on the second I like it, you give it foo and C : tags.

F

D

Think it checks is rooted on the second argument and if it's rooted it just gives you back. The second argument: that's.

E

Right so that code is not.

C

I mean history we wanted to. We would want this property to return true for sue :, a.

F

You're correct here's. Here's the other problems that, like his path route, it actually returns false 4c :. It only returns true for C colon backslash. It's.

E

Really broken well, this this is from 1.0 right and some of his letters and around since forever yeah. So I still haven't heard a single case where it's valid to call his path routed where that would remain. The sensible thing to do well, I mean the I.

B

Guess one of the niceties about this thing is that I think is Pat. Would it is yes, it's a neat job. You have for a long time, but I don't think it matches when people who look for anyways, so I think generally speaking, like people. Think of you know getting full path, and it's fully qualified, as kind of like things that are related is parador's only something that you find out the browser there and like not seeing anything else, and then you get a chance right.

B

So I think, generally speaking like it's, not that the best things already taken out argue that would help. Would we call it path? Is where is relative path, which is what most.

I

People think of an in terms.

B

Yeah, like I, think Jeremy considered that material in Asia, where that's not what you want it to be big relative and fully qualified are two different things. Just.

E

Stephanie, did you consider Corian path is full path? Does that make sense.

C

F

Because I think that would would be confusing with get full path, because we need to say that it's fully qualified does not mean that it doesn't have relative segments on it. So I think you.

E

Get full path, you don't care, I mean I've, never cared about relative segments to me. Get full path, just mean well, no you're, right, sometimes user to canonicalize, and sometimes you wanted to make it because there would be a normalized and I think what does I would know. Well, it's.

I

E

Right beside you go, get full path to normalize since I was you just coming.

G

Directory independent, they seem to want to know if they need to call path that combined, except since we called this method from path that combined. They should apparently just really always call paths that combine.

F

What man you it's unfortunate? What's.

G

The distinction like someone said that this isn't the same as not relative, so.

F

What's relevant, what is relative? If you look at the link? That's on the other thing. It goes to the MSDN documentation, but relative basically means that some part of the path is relative, not that it has current working directory, so the documentation. She has a link on documentation there. Okay, so there's fully qualified versus relative paths, so he is fully qualified, would return false if you had a whole path ahead, dot dot in it. No it's fully qualified would return. True, fully qualified means that it does not, and it will not change.

F

If you change the current working directory.

F

Is a special case: that's.

E

When a qualified partner so.

F

Changing the current working directory is not going to change dots that are embedded if it starts with dot dot. Yeah, that's doesn't have a root, and it's not that special case of dried relative, which they show there. So.

J

Yours to get full path will not evaluate dot. Dots in the middle get full path. Does ok, it.

F

Wipes them out so get full path, that's actually it it normalizes the path completely yeah.

F

Incidentally, for something that removes that the that normalizes, the directory, the relative directory segments, but doesn't like put the doesn't like fully resolve them right, doesn't have the current working directory in which we have that as well. We just don't have it exposed.

B

So basically, I think is path through dirt is something we should just add. Some provide guidance. What the people like you, don't you probably don't want to call this API and I see a really super narrow use case that I can't even come up with and then the other one is. Basically one really is for the client. This is fully qualified.

B

I. Think that seems reasonable to me. I need like concerns from that I I. That's.

G

A terrible example: C colon backslash checks hmm your screen, that is horrible, just.

B

Yeah, but it completely makes sense right. The to understand would see : world of exits. Yes, use.

G

The current load current context path off of C : right back up a directory in an open file.

E

See Carolyn through where I read.

G

The secret environment variable you're not supposed to know about no.

E

um We mentioned that if we just fix this path, we today would break people with it. Yeah I mean CDC, C would say well, it will pass yep. You work it and everything else, but but would it be impossible for someone to opt in to the clerk because they'd likely break well? The way clerking would usually worked is that you would say if you're compiling it.

E

Let's say for seven, which is the next work you get the new behavior, so I guess by quirk I mean whatever we did on my core, which is like app flags or something I mean using.

B

Those for a bowl of the Epcot, where the app is compiled for an older version of the old behavior, if you should be targeted later to.

E

The current or later or I, agree that it's probably on, except for breaking change and just having trouble thinking what someone will be doing so.

F

What somebody could inadvertently be looking at in I mean it's their input, data that not having fully under like a fully understand their input data, or maybe they don't understand their input data, but like all the cases that they have or coming in, is either starting with a directory separator or not right.

F

That API would behave in a particular way right. The.

G

Problem would be if somebody right now is, will return true for C colon backslash, Tim I, think because it has a colon in the.

F

Second position: it needs to have no it like the current implementation that needs to have a directory separator at the at the end of the colon. So we will return false for this one. Yes,.

F

G

The problem would be for something that we're returning true for right now. If somebody doesn't call at that, combine to resolve it and their code happens to work with a relative load, and then we change the behavior and then now they start calling back combined and then evaluates to a different directory than whatever was going to have happened, because the place that they do the check and the place they do. The evaluation could have changed. Yeah.

E

H

E

Case is my back: is this producing different results? It's gonna break something right, but I.

H

Think of a sensible.

E

Oh but I know we don't want to break nonsense located. Sorry well.

B

The bottom of Cynthia nonsensical, is not so much whether it's you know, since it is nonsensical, it's more about. Do you change the condition of the program, so let's say the previous one returned a different era than the new one does or it wasn't no I mean. You could imagine, for example, that the old Co path, the file didn't resolve to an existing file on disk or the program handled that case and continue now. You're, changing it to a father. X does exist and is another program, trust our supply, but the yeah.

B

These are all the cases where you know given enough customers.

E

In you are eyes file colon backslash, who are naughty naughty boy; that boy.

G

Don't posit you're out to this is all your own I, like the name of the new thing, because it I don't know if it's POSIX terminology, but at least it matches the terminology that is yeah, and what about / question.

E

Mark and all that the.

F

This handles it. Okay, the current implementation handles that all right it handles it, handles all all known path formats. It just code is actually relatively simple. There's a link to that as well and the thing once you know all the rules for the windows pass, the check is pretty much simply looking for the special drive, : thing and also looking for to double backslashes is is UNC fully qualified? Yes, because you cannot have a relative path for UNC, so.

G

Double front: slash is still fully qualified Killah.

F

Forward slashes in any valid path that are always all the way leading white.

E

Spaces, that is, poverty, doesn't trim it right and.

F

No so yeah and resolving it's one of the many many many quirks of the win32 path, resulting while it will normalize separators right to combine them and collapse them. It doesn't do it for the first two specifically because of UNC's right, sweet.

G

Fare if you took a thing that started with a space- and we say.

G

It will give you the trim value, but.

H

If you get you.

E

Guys, mass we've.

F

We've been recommending back those trimming stuff and leaving it to windows. To do it, which windows won't trim a preceding space but and.

E

And finally, the unions- this is always well define.

F

Unix, it's always just doesn't have a directory separator in the beginning, so.

J

Is it fair to say the drinks is path watered and his holy qualified are effectively the same on.

F

Unix they are because there is only one route. So that's that's kind of the fundamental problem here is that windows there are multiple routes. There are, you know and numbers, not even 26 right. You know.

B

But I mean the problem is effectively the way windows works. Is that there's an actual absolute which is effectively the you know, backside access question mark thingy right, which is the actual way and.

E

There's the bits.

B

E

You know, drive specific get Jeremy started because he'll tell you that, but it's far more complicated mad. It.

F

Is more complicated than that I won't start? That's.

E

Really interesting but we'll.

F

E

So I I have no diction's to okay, so then we leave that is, they can prove it suppose.

F

And we just need to get the commentation to be updated to clarify between the two. Wouldn't you want to use them.

G

And update the documentation to say: if we don't think, there's any real case that people would want the current one, then we should add it to emos tool functions. You should never call him and.

F

I I think I should do that as I mean. You know, I still think it's conceivable that people have some situations where they might be using it, but or want to use it, but we should encourage them not to I mean only use it. If you really know what you're doing we.

E

Have any curse? Nicole's is paw through that in the eye. Okay, other than pop combined hold on assuming we'd like to not combine to change its fully qualified, but we contents at either. Oh no.

F

Then actually needs rooted.

F

E

On it is really, but it's gonna get it wrong to see comeback time.

F

It's used all over the place, but in I.

D

Think I was specifically wrong. I was wrong about combine. It looks like combines literally just checking for directory separators.

G

Now it's like combine calls combined, at least in net FX combined calls combined, no checks which checks both. Oh, it does check yeah.

F

Yeah, it doesn't look what we call it very often. A lot of a lot of internal code calls, it sure, looks.

D

Like there are a bunch of asserts that uses path, routed, yeah.

G

And it so for net effects. If we would want to switch that fully qualified for correctness, that'll probably require work. That's the ECC nut, hey.

E

Except for maybe assets I don't see it changing any closer.

B

All right, I think we're done with this one.

G

A

D

Okay, so our favorite topic can we start with big converter for just one minute: okay,.

K

D

I just have one question, so I was making issues last night and I for both bit converter and gooood. I had a question so try right, bytes I'd, initially assumed that we didn't have to tell the call or how many bytes were being written because well, if they're writing along it's eight and if they're writing a float, it's four and whatnot, but do we think we need an out parameter? That says how many bytes were written same question would go for good.

I

Isn't it still simple doing your size of loose time just to keep that information yeah like, especially because these are all fixed size and.

G

So if we do precondition, checks will either have written the size that they should know nothing. So.

G

A

Big is this bull? It's one bite, I think whatever size it bull is exactly what size. Oh yeah,.

A

D

Is the right answer.

B

Excise obstructs, for example, don't mention the match how big they are, because one of them is the interrupt size either. One is the actual size, so we lay it out or we.

A

Correct in Auto a class with.

G

Auto layout, yes, you're on site, which is why size up requires on safe. The super prototypes in.

F

A pool is not little, no matter what you do.

D

A

D

So my that was my initial thought as well, which is why didn't include out here, so people can just do size of what about for gooood. If we want to his size of goo, it does produce the correct answer.

D

But you need to use unsafe code to call it size of on a good or.

G

Marshall that size or.

D

Marshall I haven't verified that it was the right thing, one sec size of good yeah. It does what.

F

It assigns a bull, give you the one mmm interesting.

L

And then make sense: oh.

D

Sorry wait, you said size of, did you mean Marshall dot, size of because Marshall doubt size that will gives you four.

D

But sizable gives you one.

E

Just in purpose of size of border because it matches the native signs, although.

G

It isn't that sizable would have. Maybe why.

E

Would I ever want to know that if I'm, not this like licking stuff into space,.

D

For try right, bytes.

F

What is what is, what is the behavior? There does try right, bytes right there item one.

D

Or a zero, huh it writes one or zero as a bite you if.

F

We document that that's.

D

Fine I'm believe it's already documented for particular, okay.

F

Just we do weird things with.

F

G

Speaking having them, because.

F

I'd say no I, just won't like this one case just needs documented.

D

Okay and same for good.

F

Gooood is always the same: yeah, okay, I.

L

Mean the question is whether we want I mean to be very I, think size of is annoying, because you have to be unsafe right well,.

D

Size' for all of these built-in types, no you no longer have to be, but for gooood you do have to be unsafe. All.

F

Interesting, oh I, see then work. There's a your pending! That's going to get rid of that the little bowl Struck's.

F

So you can attribute worse trucks to. We would do that for good and all the rest that would allow C sharp to because it really isn't an unsafe operation for for good. We just make you do it.

D

So maybe for now say no we'll leave it off and we reevaluate in the future.

I

D

Right that was my big convertor question. Moving on and.

B

Did you see that I added some more notes to the thing yesterday and Merson from last time, and there was one thing that we added last time it was a leonis. Oh.

D

D

Was that, but that was sort of a I thought we had decided that was sort of a largely separate issue like we can. We could add nd in this overloads today as well. We just haven't, and if we wanted to do it, it would just be over loads of these methods. All.

M

Right I mean the only notes that I have is. We should take this into account and provide all those it allows consumers to specify the ending is day one and.

B

The only thing left is that I mean like yeah. You review that this is a separate issue.

G

We should probably write down that issue so.

H

Then we'll just argue about what the Indian dysfunctional.

H

D

Alright, so I think we left off on in the stream section.

K

B

Time, beater.

K

Killed and XV depicts rider so.

D

This is very similar in nature to stream synchronous methods taking span asynchronous one's taking buffer.

D

Basically, the the core virtual methods that we have on text reader and text writer today that that worked with char arrays, I've added them with spans. My buffers.

G

D

So I think the logical that sort of the only thing that makes sense like in this stream case is. Oh sorry. So there are two deferral issues. One is what do we defer to and the other is: when can we do more optimal things and derived implementations so for the deferral thing, basically there's a corresponding method for all of these and I think we just delegate to that one so read span, you know, read span char span delegates to read char arrays, whatever it's called I. Think there's that's pretty straightforward.

D

The other issue is the one I. We had an email thread on where we want to then override these methods, like on stream memories stream, would override the new read, read async right right, async methods.

D

The trouble is, you know it wants to do the more efficient thing, but that more efficient thing involves not calling the corresponding method. But what? If someone derived from memory stream and overrode those other methods, they would still presumably expect their overridden method to be called, and so our overrides on our streams are gonna have to do some sort of currently. The best thing we can do really is just a type check. Are we a concrete memory stream? If yes do the more efficient thing?

D

If no calls to the base, which we'll end up doing the delegation and what I don't have in this issue, but which we should talk about when we actually should talk about now, it's probably a good time as any is. If we want to. We could then expose out from our concrete streams that are not sealed a protected version of the more efficient implementation which the public method would just call and derive types could call if they wanted to as well so memory streams, efficient implementation, just copies between two and memory buffers.

D

Basically, so we would have a protected method that just copied between the buffers. We would have the public method that called the protected method, and but only if it was a concrete memory stream. If not, it would call to the base and then a drive type if it was just over. If it was overriding the method entirely, it wouldn't have to do anything. If it wasn't, it could choose to override it and call the base protected method. Presumably.

G

We'd have to do the same thing in stream, reader and stream reader.

D

And all our other non seal exactly, unfortunately, we have a bunch of these unsee that, well fortunately, or unfortunately, depending on how you look at it. We have a bunch of these on seal types, yeah.

G

So I guess I'm wondering if in something like text reader, if we could do yes so find the place that we can make the best, you know shared implementation and then like in the constructor of that type, set a protected or set a private boolean based off of doing the type check. So we only do it at construction time and not. You know every call into read.

D

Let's see I mean it's an implementation detail we could we can do that.

H

Yeah I don't know.

D

That it's actually gonna make a difference, but we could we could check I, don't I've have not compared the perf of what I mean the JIT optimizes get type equals type of blah and I haven't compared the performance versus the extra space to store a bool which may or may not increase the size of the type and the field read yeah.

G

I guess I just don't know if we'd want to make it be a my this type or if it's gonna become a list of types or if it's gonna become, if I'm defined in any bcl assembly or I. Think.

D

The check is I, thought of it is gonna, be if I'm in tight foo if get type equals type of foo than X else. The. Why but I mean it's possible. We could do perf testing and find that it's better to do that check in the constructor and cash it, in which case we can totally do that. It's hidden from the API surface area and.

G

I'm just wondering if they're places that we have like a middle class, which is where the right place to like that, we have like eight different derived types and we're like well, we want them all to use the good thing now.

D

Yeah I mean it's pretty rare, actually that we have an intermediate type, at least in streams. There's like one or two cases, I can think of I. Think like SSL stream, derives from no, but even then it's it's pretty rare that maybe I think it's. The only one I saw was isolated file stream deriving from file stream, maybe but I think we should just deal with it on a case-by-case basis. All.

I

Right, do you wanna introduce an interface kind of query for this, these optimizations? What are the weekends I has buffer.

D

Can you elaborate I'm, not sure what that would look like well.

I

You're, basically going to be doing type check for specific streams like memory stream, but it seems like you, don't necessarily need to do that if you just need it well,.

D

No, the issue is in in the type itself. You need to check whether you are that type yeah.

G

I think if we, if we slap I support spam on memory stream, a direct memory stream says I support spam, but they didn't ability I.

I

Was I was thinking about it? Being that instance of the thing I was thinking, I was taking of wrapping.

G

Yeah so Stephen I know over years found lots of streams that you know didn't, do task-based a sink right etc.

G

Are we gonna have a confidence that we'll have a good way of finding what all the places we need to span if I are? Is that going to be another book? Well.

D

I think the the base implementations will just work, its it'll be a question of: are they as efficient as they could be? The base implementations will either like for the buffer taking ones like if the buffer wraps an array, they will just be functionally and almost performance-wise equivalent to the array ones we already have and four otherwise, you know they'll they'll, be the extra costs of array, pool interaction.

D

I, think the tail will be streams where there is something more efficient that can be done and having to go, and you know do the overrides on all of them. So I don't think we're gonna find much of a functional bug tale, but there could be a perfect tale. You know, there's there's something like tens, ten or twelve streams in core FX and core CLR, where it's gonna be important, that we go in over I these for performance reasons.

G

Yeah I was worried about yeah. It's.

D

Not gonna be nearly as bad as some of the like blocking deadlocking issues. We saw that, but those were mainly due to the issue where we had completely removed the intermediate begin end methods in some profiles and because of that, we introduced a whole host of potential problems. If you ran on a platform that actually had them.

D

But yeah we we are going to have I expect. There will be a tale of streaming implementations where oh right, we have to go, implement these there as well. So I will proactively open issues for all the ones that I can find. In course, you learn core effects, but we'll need to we'll probably need to track down and help others in their own repos. You know: do the more efficient thing the interesting part will be when people start consuming these.

D

You know, if someone's using, if someone's using a new word neck core, that has these methods and so they're calling they're consuming them and someone else has targeted net standard and thus hasn't overridden these you're gonna get the base implementation, which will at best be as efficient as the current him. You know stuff but may actually add a little bit of overhead.

I

Looking at the text, reader and writer, but we're talking about the streams, it's.

D

The same it was just the delegation thing was it's the same issue in both cases.

I

What the reader writers are you're gonna, be wrapping streams, which is why I was thinking about the marker interfaces in my head, but basically again taking a stream and you're going to potential different cause based on stream and type itself.

D

Potentially, although I my assumption and maybe incorrectly, my assumption is that if you're, if you're, given a span or you're, given a buffer you're, going to call the corresponding.

D

Stream, one on the stream, although we'd have to look at the implementation, it probably depends I mean we may not even change the implementation, or rather we may change the implementation, like I. Think if I remember correctly, stream reader has like one method that it always goes through to read to fill its buffer from the underlying stream to fill its.

D

You know, array and so no matter what which of these entry points we're using you're, basically going to end up there, we will probably actually go and change that one to always use the the new span and buffer based API is because, in the 99% case they will be more efficient.

D

But somebody yeah I, don't think it's gonna matter, yeah I, don't know, maybe I mean. Maybe if we had such a marker interface, it would switch, it would say if it you know if it implements those great use them otherwise don't, but that probably leads to its own complexities.

B

D

I

Maybe we talked about, why is the returning value task and the writer state returning tasks.

D

So tasks you you can always there. There is only one possible completed task. It is tasks. There are an infinite number of possible: well, not infinite. There are 2 billion 4 billion possible int return values. So there is a single cash task task that completed tasks, which you can always return. But there is we can't cash, 4 billion tasks, events, so we returned value tasks. You don't have to allocate to return value.

I

But you're, assuming that everything to be returned tests are completed on these things. Obviously, it's not going to be that way.

D

If it's async than value task wraps the task, yeah I,.

B

Mean in the journal assumptions when you do read a sync on a buffer that the underlying buffer is bigger than what you pass in so that here is that majority of cases you can just return the completed tasks and every once in a while. You actually have to do a tasting operation, so their camera twice once you locate that. But why is writing oh yeah? Because writing it seems that you can catch the task, so in that case you don't have to defeat it. Your swimming everything's good, most of the joy it takes. If.

D

No or not so if things complete asynchronously there's nothing to cache anyway, a task is going to be allocated whether it was returning tasks or returning value task.

I

Do we have any guidelines around when to use value tests versus tests now in our EP exercise? I believe you do, I mean we.

D

Have shared guidance and various forms on various posts and whatnot I, don't know if we actually have any in the doc we should. My guidance has basically been if the thing, if, if your, if your API is always going to return, asynchronously just used just use the task. If your API is going to return a a fixed number of possible values like if it's either non-generic or it returns a bool or something other with a fixed number of possible results, then the overturn task, return task, a bool, etc.

D

If there's a high likelihood that it's going to return synchronously and it's impossible to cache all possible results, then use value. Tusks.

D

The problem is value, task, isn't always a win like it's bigger than a task, for example, because it you know it. It contains a reference to a task n to the T, and so, for example, if you await a value task returning function, your state machine will have an extra field in it more than what it would have had in the task and every call to access this goes through an extra branch and etc etc.

D

So there is a small amount of overhead associated with something being a value task, so you it has a lot of upside if you expect things to return synchronously, but if you don't or if it's pure overhead there, it's not worth paying the extra cost for I.

I

Just find it's gonna be quite difficult for our customers to figure out when to use one or the other, but my dismissed me also not getting fully rented to it. So.

B

Those are the docks for major tasks in the remarks section going to the track up decision and provide guys early did.

D

We pull these out of the XML Commons, because I'm not believe we did.

H

D

Cool excellent.

B

But we need some cleanup like there's some Hey all right, so.

I

We have something that all right, sweet, I, don't really like the day. The the symmetry is not there, but I guess I can understand reason yeah.

B

D

Mean we've we've gone I've been asked us a bunch in the past like why not have a non-generic value task. We could add one, it would add zero value other than naming symmetry.

I

These are two different types of the symmetries, not as isolated, but when you look at stream, it's more like oh yeah, writing different tests, yeah.

D

I get it I, don't have a good answer. There's.

B

Not one necessarily get a time machine it'll be one. That's.

D

A great answer see.

B

I'm always open one, although.

D

I've seen those movies you but ends up having is you end up going a little bit to the future, and then you fall asleep, and then you end up like 4000 years in the future and yeah never works out and then the more like to you exactly.

B

I

Buffer did we review the is buffered part we did last. We talked.

B

I

Briefly, when we.

B

Talked about that.

I

B

Time, machine and mentally, it's a will behead. Last time it's the set of public times. These could be internal buffer stream versus the only person who.

I

Talked to us instead.

B

Of public actors could be internal and we can have a factory method that returns you. A stream will probably require the I ask for pattern to make sure it's about the telus, explicit types and I think that just requires.

I

Do you wanna tie it together because I thought we also said it might benefit for adjusting streams. Every stream another's I think.

D

It's potentially mine, so there are some places where code will check whether it can get the array from a memory stream, for example, and if it can't like it'll it'll type check. Is this a memory stream? If it's a memory stream can I get its array? If it can, then I can do various things based on that and so having the interface. If we implemented it on memory stream on it, you know memory stream, buffer, unmanaged memories, stream and I guess read-only buffer stream.

D

We could you know we could potentially expand some of those optimizations to be broader.

B

So I have a more gentle question I should we start filing individual issues for those things to be, don't lose track of them because I think having this alt is my.

I

I agree: I think this is probably.

D

I think we should I wanted this one issue as a way to kind of go through everything together, but I've already started opening separate issues. If you scroll up you'll, see places where I said separated out as XYZ. You know issue and my goal would be that once we're done with once we're done with this review, we can close this issue and just have the individual issues that people can tackle individually. So I will make sure that everything has individual issues and then we can close this.

B

So can you repeat which dreams you think this is basically at me. So buffer stream we'd only both know strain you're.

D

Asking about the high I has buffer yeah.

B

D

Off the top of my head, I think would be memory stream, unmanaged memory stream and then the new, whatever we call them buffer stream and read-only buffer stream us or it maybe they're, just internal types and it influenced it. On the internal types.

D

There may be some other streams as well. I mean it's possible, you could even imagine you could imagine implementing it on rapper streams as well, and it would just you know, check whether the stream that I was rapping, implemented the interface and then, if it did delegate to it, if it didn't return, false.

B

That's exactly why I was skeptical of the value of this thing is because, as your delegation, the interface check alone doesn't actually help you right, because you have to statically decide whether you dynamically rap on it does, and then you would always have to do. It itself would have a query method and before you might as well just add it to the base. Well,.

D

The that's true me: we could just add it to stream, it's already a try method, so it can already fail even on something like memories stream. It can fail, because when you construct a memory stream, you can say I don't want my buffer exposed and then try get buffer will fail or try get array whatever it's called.

D

That's true, we could just make it. We could just add virtuals to the base stream, which.

B

I think this is my general. They could think like a better development. Think it soon as you have the link dynamic delegation scenarios, you're better off, adding it to the abstraction itself rather than doing interfaces, because then you.

D

B

End up lying anyways.

K

What is it called tri-gate buffer that.

D

Was my suggested name yeah because I think memory streams is maybe try get array.

D

D

My memory streams is called, try get buffer.

G

Now that we possibly have a type called buffer, that's only slightly confusing yeah.

A

D

It'll be even more confusing, there was some talk about renaming buffer to memory, at which point memories stream is a total misnomer.

D

Just call it emo, oh.

B

Yeah, you should emo up.

B

But only do rename tasks to a tub of tea and will be also cool. Perfect, then concept symmetry on that one um all right. So all.

D

Right so file ID initially had a large list here, but I I, trimmed it back for now. Basically, alright, let's start with the right all and append all because I think those are easier.

F

D

You know you give it a pass, do we have? We already have byte array versions of these, so this is just the span versions. Basically you call it and you give it the data you want. It opens the file, it writes it out. It closes the file and write and append versions of those. The read ones are more complicated and I'm I. Don't I, don't know what the right answer here is, so you can read, but you don't necessarily know how big your span is.

D

So you could, like I, have this out long to tell you how much extra space might have been needed. I really wasn't sure what to do here. I.

B

Mean that seems like literally the super inefficient API, because you keep opening and closing the file. Essentially exactly and.

D

At that point you talked about. Well, maybe we have a some kind of handle that we can use. So you then have like you know you open the handle and then you past the opening, but at that point you're almost to the having filestream so.

H

D

Could potentially have you could potentially have like safe file handle and give safe file, handle and open method, and then have these read all bites. Excuse me methods taking a say file handle, so maybe that's something we want to consider for the future. You potentially have a similar thing with right all by the way, because you might you know if you're getting your your data in increments and spans, you may potentially you similarly be writing 4 Kbytes, writing, 4, Kbytes, writing, 4 Kbytes, so we're in sort of this gray area here.

G

Let's is there significant overhead to file stream that would make exposing file handle based write operations significantly better.

D

If you know what you're doing- and you like- set the buffer- the file streams buffer size to 1, which basically means don't buffer I- would expect that the overhead is basically the cost of the stream itself.

D

Like the you know, any virtual calls you might be making and the actual object that you allocate in memory, so I I would be surprised if it was substantial. So.

B

The only thing we could consider is like having this method in such a way that it matches the existing one, which means it will only ever succeed if you view buffer large enough, but.

G

B

G

Its give you back an array.

H

G

Makes it be big enough right.

B

But in this case you would be on the hook to it. Pre allocate the right size, but you see that, like you give me, you can ask for the size upfront right.

G

Sort of you can ask what the size is and then later you can read, but it's not necessarily the same file. True, in which case I mean you should just combine it into this one method, and you think we.

B

We blow up if that happens, so.

I

So reading this API, so I don't even know where meaning means there's.

N

A meaning, the remaining bits that need to be read in the file yeah.

D

Like how many, how many bytes.

N

Are left in the file that weren't.

D

Read that's what I'd intended at least but I like I, said these all get into this complex territory, and it could be that the value-added here is so little you might as well. Just do you know open this file. Dot open right, make a call on the stream, because at that point you can just call you know stream, dot right and pass in the buffer or the span, and you close it. So maybe these helpers are no longer particularly valuable. Everything.

G

I can the write and append, because there's no, you didn't give me a big enough thing. Thank those seem easy, but again, they're also.

F

G

Later people complain about them, but the read one definitely seems weird.

G

F

I agree seems weird to me too I, honestly.

B

Like those ones, the reason why I like those is it there, if they effectively the printf of binary data, you can just say I the buffer, just dump it to the file, and you just look at it. You don't have to really using or you I think they're just convenient I would still have those but making a highly performant app. It's also not something you would use a lot right, but.

B

These ones are just then they're like in the worst possible triangle, but they're not efficient that they're, not that.

G

Read all byte is I, don't know just give me a thing, that's big enough and I don't feel like bumping. The string I would.

D

Argue that append all bytes is also bit like write all bytes, because you're gonna wipe wipe out the stream like you have to be writing the full thing. So it kind of there's it's sort of the most efficient thing you could do append all bytes I would worry that you're gonna be using this in some sort of repeated fashion, where you're gonna get ten bytes and write them out. Ten bytes and write them out and then you're gonna be repeated.

D

It's gonna have the exact same, read issue, so maybe we just say we'll add the right ones and and not the appended, not the read, I think.

F

I'm, fine with that, it seems to be a good place to start what let people come back with scenarios that we might be missing. One.

I

May also want write on charge for Spanish are rattle characters with character, I carry, but we have my awesome text from a string. So oh that's.

D

G

Have to wrap that through it Cody no.

H

G

Jar's to a pot to disk they went picking a coating for you. Oh argue it seems right. Oh yeah, whatever read I'll text those but.

B

Those ones I would say they I, don't think there will be probably called I.

B

Don't wanna see I'm fine with just having one because yeah it is something you can imagine in real world and, as I said to me, it's the printable binary data I was like oh yeah, I mean the only thing why maybe append all bytes is useful is essentially is.

B

If you have a loop when you want to print out the whole thing that you loop to over a penile bytes can be very convenient, I mean it's not being to be it's not efficient, but at least when you run your program, they don't have to like you, crazy gymnastics. In the end of the whole thing, yeah.

D

But at that point, like you all it takes, is one extra line at the beginning of the file and two braces to say open the file for writing. And then you just call stream dot. And if you want a.

G

One-Shot, you can make a static class that holds the static file stream and.

I

Most of these are supposed to be one-shot, API yeah.

B

I'm, just keeping that for.

D

Now and see where leads this I'm mister? Where do we land done Wes a suggestion of right, alt text and right alt text, async yeah,.

B

It's me there. Those are kind of odd, because Jeremy said you can either pick an encoding, but you also have to so different than you just basically creating a string with the read-only char. It is but like.

I

Is it the common scenario I? Don't.

D

I

Right alt text also has a rule: detection, encoding, I.

B

Don't I mean to be honest, I can't charge it I'm, not sure you.

F

B

That's exactly my thing: I'm not I'm, not convinced that span of char will be as common as string movie, maybe I'm wrong, but I. Don't.

G

See that yeah but I mean read only span of char is string. We can add the string overlooked too, but.

I

Just question I think it's fine leave it out for now can edit, if it felt like it's necessary, I, music.

G

Yeah I mean like because really what we want to do is.

G

Yeah, so if somebody has a string, they can still call right, alt text and then we would just want to have changed the plumbing of that to use a non. Allocating thing: if that's our goal,.

G

The question would be when, with somebody have a read-only span of char I guess if they separate all right, if they it's heist ring.

H

G

So now now changes the scenario.

G

Feels marginal but not difficult, maybe it's worth doing.

B

And this is something where I think we can be something more reactive, yeah Clara. This.

I

Entire API, you know, file with general I mean it's a convenience.

B

Method, it's not about performance, I mean what what see just said like it's like you can write all bytes, that's right, so I mean I. Given them you already the remaining ones. Maybe we should just start with file, doesn't have any overloads and until proven otherwise, I think that's also visual, offering today at least it's somewhat consistent by was it's bit weird to have on.

A

The chunk of it, let's.

D

Do that we'll just scrap this whole thing, this whole section and we'll revisit it as we find real need for it. I.

B

B

Alright stream builder.

B

No one seems like.

I

A no-brainer to me ginger also be a format over a little worse up to you, pin think.

D

There is a for, like you, where you basically pass it, the string format and the arguments.

B

I

But I think that's one of the biggest safety as we need to help improve upon through an inventory.

G

So wait a non allocating or a minimally allocating string that format a minimum was it the only time that it had to do complicated allocations is, if you just like open, curly, zero colon and then wrote a custom format, because then I had because in order to call legacy it guys, I had to call it take that as a substring and pass it to somebody else.

G

That was the only time it a located of the exact need us to copy of every round. So.

D

I, don't have a great set like so you I, unless we added some sort of specific overloads for now. I, don't know how we would you wouldn't use this read-only span of char as the arguments that were like inserted into the format string and at least in my experience most times the format string isn't dynamically generated. It's some fixed bit of text with the holes to be filled in which wouldn't be a read-only span of char either. So where would a span fit into a format.

I

Maybe it's not even a public if you have, but it would be internally. I feel like we need to get a sort of a good code path through the append logic in and stringbuilder such that we can past abuse spans as the buffers that we're writing to. As we're building these things up, we can eliminate a lot of allocations. We.

D

Should obviously, internally, we should do whatever we can to fix implementation details, but I'm wondering like from a well what is the API signature with format that involve spam I? If we don't already have.

G

A plan of having read only span of char format, or you know, consult that out the same as string. We probably want to make it do so so that we don't have allegation just to print the substring. That's.

D

Interesting you're, saying a string dot form a string, tri format that takes all the arguments that string that format does but a span as well.

G

Yeah well like, if I pass.

D

One that into a span rather than formatting, into a string.

G

Well, that's one thing but I'm saying actually passing a span or off or whatever as because, if span can't be boxed, then that would be weird but having it as an argument so that if I have a string and I want to instead of substring it, which is allocating slice it to get non, allocating and then say, go write that to the console right.

D

Well, that's where I I don't know how we do that with a format unless, unless you Slyke, so if the format string is you know, foo, open, brace, 0, close brace bar, open, brace, 1, close brace and I want to use span as the first argument or the second argument we filled in, we would have to have overloads for every possible position span could be and for all possible arguments to be filled in the format string.

G

So span can't be.

D

Boxed to the object- and it can't be generic type parameter.

I

Static, a piano string, the string that format API. If we pass in a span, you basically do all the formatting into that buffer. Well, behind the scenes my post string builder, which is what need to be passed through or in half use buffer as well so I, don't know what a PS we need to get make that all yeah.

G

Like can we get a read-only buffer of a string yeah, any sort of slicing method, a read-only.

D

Buffer of chari mean yeah no.

G

It's like people, love substring and people, love, stringed, up format and right now we're saying that those two things don't play well together with fans or what I mean lower Emily yeah.

D

So you're saying I think that you're saying get a read-only buffer from string I, don't believe we have any way to do that and then also have string builder, dot format, methods that take one two, three, four five, six. Whatever generic type arguments such that you could pass a read-only buffer as one of those without boxing it.

G

Just care that, if it gets dropped in position three that we write the characters down instead of like an object, hash code.

B

Make it sound like string, formatting and span is a larger topic in and of itself yeah. Maybe.

L

We should just spend another meeting just talking about that. Yeah I mean.

B

L

The end of the day, string builder is going.

D

You know hopefully I'll end up using like today it's really inefficient, you call string builder, dot, append or string builder dot whatever and into 32. It first calls to string on the in 32, and then it writes that string to the underlining thing like this. We.

F

Should just we.

D

Should just fix that and we can fix that easily once we have methods like in 32, dot, try format with a span like it'll, be trivial yeah the form the whole format string thing I agree. You know if we had a a string format that, instead of generating a string, had wrote to a span of char, like that seems pretty nice, but maybe I agree. Maybe we should separate out that higher level format strings thing into a separate issue and someone tackles that directly and make sure end-to-end. It looks nice.

I

Last question for spring boat, for me, is a fringe. We also have a constructor that takes a buffer, the basic it can be used as the underlying for the memory from stream builder.

D

D

I

Don't know what, if, how much that we've got the stringbuilder implementation book.

D

So right now stringbuilder is a linked list of string builders. You know and I don't remember, I think it stores its data under the covers in each of those segments as a char array, if I'm remembering correctly, yes,.

D

So you're saying basically, instead of storing, is the char array stored as a buffer of char yeah.

I

I

D

You know if it if it didn't, hurt existing use existing perf on string builder to do. That seems perfectly reasonable. Any memories stream allows you to it to either dynamically generate the memory uses or you can pass it in seems reasonable for string builders to do the same thing you.

F

Have to deal with not chunking yeah.

D

You basically have a flag about whether you were allowed to expand the string builder or not.

G

Just replaced char array with.

H

D

Replacing char away with buffer no, but if you, if you were then able to pass in either I mean you could do the same thing today, you could give it you could give the string builds of the underlying char array you wanted to do. We just don't have that constructor.

I

We chucking that goes on in string builder too, so new char array shows up quite a lot. Yeah.

D

I think it's an interesting idea that should be left as an orthogonal unrelated issue to be addressed separately yeah, but it would alleviate, would potentially alleviate the need for something like stringbuilder cash. It's yeah, you could just be you could well, maybe not. You still need to allocate the stringbuilder yeah I, don't know.

D

Alright, so encoding, the encoding class already has a bazillion methods on it, but you know that work in terms of charge, stars and char arrays, but there's a small hand. If you just add a few that work with span you, you know you could basically use encoding without having to worry about allocating arrays and the like. You.

D

Guys still, there yeah.

J

I

On the springy FDLE of coding, API students, look there.

B

It's me that seems generally fairly straightforward.

D

The preamble thing, so that was one that just invent patent and Yan, had both talked about adding today. There's a get preamble method that returns a byte array containing the preamble for the encoding like the the bomb, and if we did it as property that just returned to read only span, we could simply have an encoding cache, an array of of its preamble and then always just return to read only span around that same array.

D

Vbs things allocate cuz today. It can't do that in case someone changes the return to Ray I.

G

Mean we still would be vulnerable to you know jerks. Yes,.

G

D

You can you can call the the dangerous get pinnable reference or whatever it is, and then change the data yeah.

B

There's only coupon will.

A

Lead to and say you can always do crazy stuff, I.

G

Mean I'm just pointing it out making sure that we're still okay with we don't care. You.

D

Know it's a good point. I raised a similar issue and I think John's point was valid. Is you know it's called dangerous if you hidden it now that we're in a world where you know Kaz is basically deprecated and you're single. You know you're you're you're in a world where no you're, you own your destiny, for your whole process, like you can do whatever you and if you mess with it. Well, that's your problem. I.

G

Believe we've said that if, if you host stupor, if you host stupid code, that was your vault yeah, somebody writes a jerk library library and they make all of your utf-8 their utf-8 strings. Have an invalid preamble? Well I! Guess.

B

They can't because utf-8.

G

Has none be able.

B

I'm 18 of 16 right the wrong ball and I. Give this even consistent with a volatile cast, so I don't mean. Thank you. We had private reflection. We just disabled that there's a lot of it cats right in the same way, but as you call those the API, you could load them in cast as well. I mean it's just about speed, bumps.

G

Right it does I'm I'm, pretty sure that we have decided that we don't care about unsafe or dangerous and that if you do something to screw up your own process, that's bad. But if you can do something to screw up someone's remote process, then that's a security, and this wouldn't be what.

I

We still for the general question of try methods. Do we have a common rule? That's gonna be returned here for every day or case. Is it gonna still be like your output? Buffer is not big.

D

Enough, well, these these cases actually have existing methods so, like encoding dot, mainly bringing up, try.

I

D

Actually they're not called try, so it did I call them trying the issue. Maybe I did. Maybe we shouldn't like there's already a you know: encoding dot, get bytes method that takes a charge star in a char count and a bite star in a byte count and returns an int we could. Maybe we shouldn't even call these try. We should just emulate the existing methods. Well,.

G

The one that thinks a char star doesn't I guess it takes the byte count. I was gonna, say it doesn't really have a bounce. So we just sort of like destroys memory right.

D

Has it has both a char count on the input and a byte count on the output? Is.

I

It looks like it's still doing both inputs I, don't see how pram well.

G

It returns how many bytes were written, but by count is the bounding. A me array, yeah.

D

The thing that Christophe had pushed back on a little here was it doesn't return how many chars were consumed, which this version I believe duh yeah. So if you wanted to, like you, know, you're you're reading along you run out of destination space. So now you want to like pick up where you left off the current methods. Don't tell you how to do that because they don't tell you where you left off in your chars or your input bytes. So these methods that I added do.

I

So the idea is that if you didn't assume all you would slice the parent charge your hours span, you have an int called again exactly.

B

So isn't that something we decided, you want uniformity and always return the transforming coding or the transform status if it.

A

Has obvious illegal data.

B

No, even for cases we don't have a legal data, that's what they do in corvex lab, or they just have a uniformity in the you know, you indicate a repetition of the color. This is the with your one out of source stuff itself. Right. These are to this distilled is to cases you can have.

B

Actually prefer that I'm saying.

I

So we're stuff you wouldn't worry about. If you ran out of sourcing, that's exactly what you want. You won't be able to write them all. Yes, yeah.

D

Well, I guess: the concern is, if you run out of source in the middle of a transfer like, if, if you only have half of your utf-16 character available or the opening of a surrogate pair, that's.

G

Invalidated yeah well.

D

It's it's not invalid data like invalid data would be the data you've provided to me like this. It will never be transformable into the output, whereas this is you need to give me a little bit more before. I can continue like that stream.

G

Well, it gave you all the bytes that had at the time recover.

F

G

G

So it looks like the the get bytes that takes pointers, at least for some of the encoding types that results at an exception if it didn't have enough space.

G

So if we didn't make this try get bytes, then we would have to inherit the same. Do I, just stop or do I be exceptional, so probably just moving to a try pattern or moving to the transform status is just cleaner, yeah honestly.

B

Like I think that to me considering, the complexity of Collin is having uniformity and the appearance, I think is very important, so I'd rather move to a give either we do booths across the board over the transformations that is across the board. I mean one of the two: it's either one of them right and considering that the pool doesn't work in all cases, probably going to wait.

D

So you mean you want to go back for all the places where we talked about adding dual try. Thus far, you want to change those two returning any new that.

B

Will be my my standard position because I think it's easier because then you can say yeah this is you know. Illegal data can never occur, but this is a sure to small needs more so later or done it always a girl, but.

D

Bit converter: try right: bytes returns a transformation status, yeah.

B

And what should they do? Try I think I think what they do in the corvex lab, call it encode or transform, or something to that extent, the thing we have to keep I don't know whether we just go with getting at that point. It may be weird.

I

For these encoding ones, I think we should just give it a try, make it be symmetric, particular capi I could see.

A

I

With get by summation status, no I haven't returned intent more than the work but White's written, which is with all the other, get bytes to well.

A

And always, is that good enough I think it is? We still have the UH parameter for the concern.

I

George watch what.

J

G

So you think that somebody would return. This would derive the success status by charge, consumed being equal to charge dot link I mean the.

B

Gentleman pardon that they have a Carfax lab is right. They have source of destination they've consumed they've witnessed all the return transformations, that's what they have across the board. I think that's actually quite nice.

B

We do need to take into consideration the existing your peers on no I, don't believe so because, because if you try, if you change your code from byte arrays to spans it's gonna drop in replacing you write the entire like the way you call the API is, will be different and so I think. If you always move to the same I, don't it's much easier as if you move to better name class, ain't, right, I'm being classy and.

A

Then we have that we.

B

Have the pointer versions are neither am I talkin to you versions so.

A

B

Know but the buffers.

A

As many as you want all right, yeah.

B

So you has to do something else anyway, like I mean currently, you you basically have to catch an exception than the I mean you're too small. You should recover from that. So this is the whole point. Is that whatever code you have currently, you have to change the way you said it. There won't be any exceptions. The only exception.

I

Should be argument, exceptions which is.

G

I

G

A property on encoder, which is throw on overflow, which throws if it hits a point where there's not enough space of the destination I, don't know who said? Oh all the gift, bytes I'm. Looking you here check the bounds before you start writing the ones that they're raised and then they go. They get bites that takes a char star and the bike start yeah.

I

Where's the overflow exception.

G

So get by it's calls get bites that I, followed by count to lying I, guess I'm, an ASCII encoding right now line six thirty throw bytes overflow, because if.

I

It doesn't, we would ignore that little property on encoding in the produce. Api sin is, it sometimes throws.

G

There's a conditional throw that I don't understand. I just found it while looking to see what our current behavior was.

I

Even in that case, if there's an argument exception.

I

Like input, oh.

G

There's input issues like you gave me Nolan and there's data issues like SSSs is not illegal.

I

Then we start doing this, there's a bunch of other exceptions. You look like it come out too, when you start factoring in including fallbacks and others English.

I

They're invalid data or well the four box. Essentially we call back into a delegate and they can do whatever.

B

The heck they want yeah, the only problem with like adding these to existing types, I mean I. Think your principal has been that we generally want to keep the semantics of the existing time, meaning the kind of issues that we are throwing for. Not throwing for should be all about the same I think they were like you want to deal up spans. We kind of have to make a decision with more important for us, I.

B

Think, generally speaking, when you move to spans I believe we should make sure that when we call this method, we generally don't throw for size issues. I think side. Users should be just communicated through the pattern, because otherwise you have a really hard time using span based overloads. You know across the VCL, because in some cases at those other kids, it doesn't I think that's where having a bit more I, think symmetry with the usage is I'm, not an entire sure we can do. We can be successful in that, because.

I

You don't think you can plug it through. In all cases, yeah like a lot of I mean I'm looking at custom, some of the custom encoders that we have. We basically have to go rewrite all these custom encoders in order to fool them properly. If you want to sing tonight, they'll be all implemented.

G

Saying that was all back we're sorry. If they have they take a delegate. We can't pass the spam to the delegate so either these would need to take a buffer or will need to do complicated well max.

I

Can change the size is of the outputs in inputs trying to put the others.

B

They start doing it like that. What you just said, I think, is a slightly orthogonal angle, which is like the king, Kimmy Kimmy do spends in principle all right period and then I think yours is more like for the error conditions that we have I mean the one thing you could do. You could just catch the exception and return the puppet error code, but it may still be better than propagating.

I

That point I'm not sure it's better.

B

But you know I mean like I think otherwise it will be this weird dance where and I have to read the docs now across the board, every single time you call the span based API or.

I

What you kind of use the same pattern already exists and there's API some people.

G

Are kind of used to it so the other option is we come up with a word that is synonymous with try, but does it try if.

B

Some before we go there like I, think the de cheveux value such as I said I, think jelly doesn't hold, because the problem is that the existing API is they actually produce arrays versus not not the.

I

Pointer ones, well, the point one's a slightly less is why I'm thinking these are pretty much exactly like the point.

G

Of intercept their shape, pointers the encoding when the way that the encoding classes work is when you call the safe API that returns an array, it calls it count which passes the data. It figures out how big the thing is going to be, and then it calls go and then internally they're, like one stupid, we do all the work twice, so they just hope that they have a big enough buffer and they call go so.

B

They separate these two issues like what you just said, I think I buy like if they have a pointer base overloading already. We should probably match that consume. That is it we space it just save pointers if you will and then the other cases where we have methods that produce an array, no move to spend. Like my point there is that I, don't think we can't have the same semantics logically, because you have to call this table tennis, trainer, right and so I. Think in that case, the question is: what should you do.

I

Yeah to be messy yeah.

F

I

The I'm looking at the in the patient condition of the and it's using the incoming buffer, that's allocated where the pipe start, because.

B

It let's ignore it for a second like the buffer thing. That's really the signature issue well, Jeremy, just for tapas that can be do span based overloads on encoding, a principle all right, because if we, if we have to call delegates and oh, how would we do that? We're.

I

Gonna have to pull it through those delegates to if we really want to push it through. The system which means it'll have to be converted to a buffer and then copy.

G

Or we, we also add a new delegates.

I

Herbs the fault, the.

G

Fall backs aren't actual delegates their attractions, so we can.

I

Potentially, on api's to them, okay,.

G

The because it's the delegate, time I think what does the ball been collected? A actually the it seems.

A

G

Of the form I can believe. Oh my, my comment about the word tries with try we're we're clearly going with whatever call it. The track parks pattern, which is common errors, return false and if we picked some other words, so we were introducing a new pattern, then we could still stick with in places. We would throw an exception right now. We throw an exception on invalid data and then we only return false for a links problem, and then this would be the new, like.

G

H

Consume get might sir I don't know I'm, not a.

B

Source yeah I think at some point, I think crew. So for me you have to write a few conditions that we need to handle I. Think if you literally will just write something and you pass innovated that is not from a buffer I. Think I ride is very same pattern. The other thing is, if you need, then try also seems to be sensible pattern, because you really only want to fill a buffer and you only ever condition you can have. Is it the buffer is too small right?

B

So if the only condition you have is buffer too small, either the destination side or on the source side, because you only have one buffer I think try is holding on the back pattern as soon as you do transformation, meaning you read from Bambaataa, you write to another buffer and you do processing in between. You multiply your error code, so I think that's where I think the transform pattern is is more sensible because you no longer have a single error case.

I

The encoding fallbacks actually look like they're going character, my character, so you what say that passed in the fluid the character to the other side, yeah you're, basically getting next character, there's basically a being phobic buffer. Let it go and then you can get busy passing an unknown character and then you get a set of character to do back.

I

So how would they handle a surrogate? That's exactly how they handle. Sorry, you get out of them, kicked in you potentially get one or more or no characters every turn use it and they actually have a the for surrogate. They basically have to use another load on the callback that sticks to the high and low. Oh is this: is this any bites to chars problem or the charge? The bites? I think this is probably going to end up being using both to some degree, there's a decoder in I. Guess.

B

To the existing encoder, like seems like they only ever pass you a char, there's no buffers everywhere. It was a typo decoder fall back popper, it really is an abstraction it you override the phobic, then you handle basically visual charts right, so that doesn't seem bad. So.

I

B

You mean, besides the.

I

Variant suck, but there is a lot of potential air condition can come out of it. That's right so.

B

Thing that seems far.

B

So then, for this case, so so we have two options: either match the existing pointer based ones, which doesn't seem to be a bad principle or we could match the gentleman transform better either. One I think: okay.

I

First thing I think we should mention just think pointer based one, given that we're not going to really be a fully wrapped and handle all the air cases and you'll just make it match the order.

I

We can make it my supporter, based one with still the addition of returning the ounce of the tourism suit, but still a magician.

A

K

What would the signature be then for the dimension of wonder 1, so we would really call this guy and bytes.

B

We would return an int, we would accept what source would be read-only Spanish our destination would be span up by its I would have out in chars consumed and out in flight with which.

H

Is gonna be redundant.

A

ha That's right: we would not have that yeah all right. All.

B

A

B

The API type yeah and the throat of the screen so.

A

Yeah actually closed left.

H

H

D

In this case, UV, you would throw if you didn't, have enough destination space or you just returned. How many you wrote. We.

G

Would do whatever get bytes does just return.

D

G

Which is sometimes they throw or there's they throw in there somewhere I, don't know how you did it so.

D

You can so you can determine so it would still throw for invalid input like a bad encoding right and it would return if you didn't have enough source characters. You could determine that by checking whether chars consumed equals source dot length and if you didn't have enough destination space, you could determine that by checking whether well, basically the opposite of that. Basically, if it didn't, if get bytes returns, destination, dot, length and chars consumed as less than source dot length, you know you have to call again the.

I

Pointer, based ones are very inefficient. They actually end up allocating a raisin cone near you. That's.

D

G

D

Those pointer ones ride later they had the exact same problem that the stream, the new methods on stream and text writer, have.

B

One thing we know he was using efficient nature. We.

D

Are so good at it.

G

I think I, like the transformation status better than bearing the right behavior right.

I

You we're not going to be a or you when you expand it to have more air cases than that, because it's gonna have warriors from the Cho. Or would you just first up those well.

G

I mean we still have success destination to small it valid input, and if we have some, we mean some sort of like I, don't think I've done yet like a 64, we don't have a whole lot for a.

H

Any where's the rest of my.

G

Data and I think all of those apply to it coding. Well, alright, okay, that makes no sense you haven't given or I. Think we got cut off. I ran out of space to write and we seem to be done. Those four seem to.

G

Alternately the people who are comfortable calling the Pointer without the encoding pointer base to get bytes. They obviously understand or think they understand what they're doing so, adding the span versions to get the better link balance of needing to manage them in parallel is a little better I.

B

Think that's something to review here. At some point, I mean.

F

I've just rolled it out based.

B

On what we said earlier, but it may not hold up, but I think we need to basically have that written up somewhere and like agreed upon cuz. Otherwise you will have this discussion on every single time, so my gut feel generally is and I think that's, maybe something that is similar to how West thinks about this.

B

So we could make the stance and say: ok, we generally try to match existing types, because the idea is, if you're, really going after high-performance stuff, you probably would use you know the new API is that our inquiry effects lab anyway. So the value that adding spam is overloads have two existing types is less about. You know, you know highly getting it's more about she'd already had it done in part of my app so now I have a span in my hand, there's not a part of my app that it hasn't been satisfied.

B

Yet so after going through existing API and so I, don't have an overload now. I have to do a whole bunch of gymnastics to create a byte array or something and then copy the data manually and then whatever the other type can do, is pull it better than what I could do. Plusses. Let's work on my side, they're still probably doesn't mean that I get the best, and it also goes back to things like here and we say: oh you know the existing one throws so this one still keeps drawing.

B

Well, then you know exceptions are not great to performance. If that's a major concern. Okay, so.

G

But what about the other gate lights? So the because the the problem with like convert from base64 is you. There was never a scenario where you gave it too small of a destination because it produced the destination array, so it always gave you one that was the correct size. But now because there's the overload here we're not talking about which is the gate charge which returns by the right, it asks how big it needs to be. It feels it using the pointer method and then it returns it to you.

G

So now, we've introduced an error condition, but by changing the method to try to get bites, that sounds to a person who's already used to call and get bites that we are removing the removing the normal error case, which would be invalid data. But what.

F

We're really doing is.

G

We've introduced a new error case and they were removing that error case. I think the problem is, if we moved to this, for overloading try in a way that I think will confuse colors so, which is where, if we're just introducing a new prefix for I used to return you an array that was the right size and now you have to give me a buffer. Well.

B

That's why I grieve you, but that's why I think the overloaded as it's written here is, in my opinion, correct, so we either go with one that is aligned with the transform pattern, in which case it wouldn't be try. It would return, transformational, setters and just be called get by it, or we mentioned the existing over. Those two also happens to be called get bytes, and then you know returning in the same way. It's there I think this try. When you have multiple different error cases, it's generally a bad idea.

B

I think try is only viable if you have a single span and a single error condition which is this is the guy we're trying to fix.

G

The one that returns, the byte array.

B

Yes, and that is basically the cool that is this one here right right, yes and so I'm saying the signature of this thing is I think wrong. Either we match the existing pointer based one, which is also called get bytes, or we go with the transform, based approach, in which case you would call it some either either you could call it public, get bytes or note.

D

Or something so for now, why don't we just go with it, basically exactly they're as close as possible? What's there, which would basically be int, get bytes read-only span, chars span, bytes close close paren, and if we want to subsequently add the transform thing that we can add an entirely new API, that's exactly what we want it to be. If we miss miss.

B

Miss you're proposing to you, leave out the charts consume. So, yes,.

H

G

I find that you so then to remind to where we previously use transform status. Does that mean we just don't do from base64 string or from base64 charms, because it didn't have an overload that you got to pass it up, reallocate.

B

Yeah I think for basics before even that it was so close. Christopher possibly could almost merge, base64 elected to rfx type, with a fifth convert in which was the you know, the point would become good because the new one is effectively the low-level high performance versus this. One is more about you know. This is not the high performance encoding API.

L

Right I'm just.

G

Like because, if we do, if we bring down transformation status and we already had transformation status, then I don't know would be, we would probably here say level coalition to be a transform.

B

Yeah, that's why I think it would be helpful for us if you would first agree on that principle and then see whether we can actually apply it in all cases, I mean, as I said. My gut feel is that the I think, if you have static upper methods like basic support or the convert class I, think it's almost possible to say the span.

B

Based versions are the high performance ones, but when you talk about things like encoding very effectively, has whenever class hierarchy with virtual methods or text reader text writer that sort of thing the problem is you never get to the high performance thing anyway? So then the question is okay.

B

If we, if we, if you don't get there, what do we think is more important like match to high perform an API, shame or just have it so that if you happen to have a span in your hand, if you can still call these api's meaningfully and then it deal with, you know, say narrow conditions, because I think the problem is also. If you go with transformations. That is it's possible that we can never return to you transformations that it's invalid data, because the way the API is are structured.

B

They always end up throwing anyway and then we would have to enemy, probably caption. It was remedy, but they may not be the right approach either, but yeah I mean I'm somewhat undecided, but I think that's what we should. This all just wrote this down and then let's just look over their remaining api's in the last seven minutes. If you have hopefully we can you actually finish this on all other thing is way more stuff, let's at least try to finish the like.

D

B

Optimism though I know, but the remaining stuff is just network, so I think.

B

Topic on its own.

L

D

So this is basically the same as the other, the other primitive types we talked about. It's just you know not a primitive, so it takes it. The big integer currently has a byte array. Constructor. We would have one that takes a read-only span.

D

It provides a method for getting out the bytes. You know representation of what it contains, so we would want to provide a span based. One I think we would based on our previous discussion. This would be try, write, bytes and it would probably need to take an out.

D

It would need to take an out into bytes written as well unlike gooood and the various primitives. There is no size of big integer to determine how many bytes it contains and then parse and try parse and I. Don't remember if there's a big integer about Parsons act that we would need to go add again, but I guess based on our previous discussion. So we need that to we.

I

Need to have the full size of the big energy earth- that's already there! So you we have this right. That's a B iterable.

B

Like we can iterate well, my first question is like just to construct it. What happens is the read only span? A positive is too small. It just blows up, as in the indeed nice input should I believe.

H

B

But it's caused to be too small and I. Don't know how you should be able to refer.

H

To a number, in fact, if you give it away.

G

H

G

On the way out, if it's, if we have no bits, then we've returned the new byte array of zero I. Guess in that case never thought, but.

A

I'm working to battery not.

H

The type constructor so.

K

We said try copy tool would be trial right right, right, based.

D

On we've previously talked about I think would be tribal, try right, bytes, read-only or span by destination, comma out, int bytes written.

K

By right, bytes and by the destination out in writes, really I believe so yeah.

N

That's not gonna be enough I. Can it doesn't if you didn't give it a bad enough, never been bytes, you roached, and how do you continue writing you do this. You start over.

D

And the theory is in a case like these print, like what we previously talked about with the primitives. Is you just it's such a short operation in general that you just start from the beginning? Now, maybe big integer is you know, maybe it's more expensive so that doesn't hold but I I, don't even know what big is your, whether you can so go ahead? Job.

B

L

B

0 or it's exactly long enough to return true I think.

G

That we could take the calculation it's doing right now in into byte array and factor that out to a method, because, given the current value, it will always have a big size and we can just expose the method of get by get by count, because even if we do the part where we didn't come back and say man, it really sucks. The big integer is little endian. Instead of like lets, you specify your encoding that the byte count.

H

Wouldn't change between Indian Asst you're.

D

Saying how to byte count method, so you know how to size your span in advance, yeah.

G

Otherwise, because somebody else could call it by going by doing a log base 2 and add one, but that's ugly, so we could just add to get by count. Yeah.

I

It would be kind of annoying the base. You have to call this method twice and it would take the bite. Try to compute a valuation size of the span you passed in and then recompute.

G

But yeah Christophe seems to believe that the general approach people is just like. Oh, it said, I wasn't big enough, go hey buffer pool and he buffer twice as big as the one I have right now and I. Don't necessarily agree with that, but that's what that's? What you're.

B

Handsome, what is it so I think for things like this Ryu, but you could know the size. It makes sense to give me an API to do that. Right. I mean like there's no point in growing things. If you're in other day I mean he's younger growing, is it growing is something that happens very real because I mean you if you write data or like say in a given four K buffer, so how many cases you can actually run into when the buffer is too small? So when it actually finally happens?

B

Well, then the cost of that is not too bad. If you have to do a massive dance, because it takes you three or four days, we have the correctly sized buffer, then you're only wasting memory also wasted, but I think in cases like this, but you could actually ask for the correct size, yeah, so I think every.

G

Big integer value has one, and only one how big am I going to be when I get written down, so we should just expose the counter.

A

G

We did it I, don't see. We definitely don't call that helper function inside to byte array. It calculates it and does a bunch of plus ones, I think it's quite a rate on things. Yeah I think that's how you determine.

A

How long it'll be okay.

B

And then I think parts is five parts per client correct. Well, you know the typical yeah.

D

We'll just double-check that they actually match the signatures that big integer currently exposes and I also.

I

Have too many comments, yeah that one comment to snack on Kapow, whatever things that don't compile are not.

A

B

K

B

Alrighty this was a long meeting, so it seems like they've another two hours in front of us to go over the networking base. Yes, yeah.

G

Ip address we'll probably also need the count, because we don't know for unless we already exposed before v6 no.

D

But- and that's do we really? Okay I, don't know I kind of buy it for big integer because you could have arbitrarily long things, but for IP address I mean we're talking about like what 16 bytes or 32 bytes or well you're, either four bytes or you're. Sixteen I'm, sorry, I'm, sorry bits I meant yeah, but yeah. Those.

G

Are the two valid of the of the current values of IP address it's either for sixteen bytes yeah. This is support both. Do you know? Yes, yes, it supports four and 16, which is ipv4 and ipv6.

G

Yeah I mean I'm. Fine I mean like somebody who wants to write it down. They can either guess or just allocate the bigger one and make sure that they're that they could have held up v6, who we exposed, there's I think before versus I. Think this I don't know cuz. If you do, then you could also just ask that yeah you shares what family it is. Oh then you can know ahead of time. Then it's fine.

G

If people complain that they need too much domain knowledge, we can add API later yeah.

B

Again, no like this is a case improve it crystallises, I think holds up, which is like how many times to run out of space for that. Well,.

G

It's not running out of space because you're encoding, an IP address. It's you're running out of space because you're encoding an IP address here and then oh no copy the whole, but like no.

B

Truly I mean how many cases do you, you allocate expense. This is it you just hold this thing like it and it's super rare. Thank you. My dad was first, you would probably encode it in to make this gate buffer. It holds much more data anyway, right, educate after the fact that you, you know many bytes, you can advance the pointer correctly and then the questions of the. How many places inside me.

G

Didn't you call it again with like I need a buffer from the pool. That's like three mics bigger! It's like you know what here's one! That's a K bigger, shut up! Exactly that's what you do.

A

In this case, but it decides, doesn't matter, okay suck it. That's: okay, HTTP, crypto, yeah,.

G

L

Names faces I think it's probably going to be assaulted, another either at least an hour and Steve. Is there anything as bad as I crypto transform nope.

D

Okay, that's it's pretty much, the worst all right! Okay, I, don't think. We've talked about any cases where interfaces are not what we want so that'll be the first okay.

B

All right, then, let me suspend this one and then there's another until always, waiting for us I will have another meeting. Thank you all. Thank.

D