.Net Foundation Design Reviews, 12 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitHub Quick Reviews

Description

00:00:00 - Approved: HttpContent.ReadAsStream sync API https://github.com/dotnet/runtime/issues/37495
00:07:15 - Approved: Update UnmanagedCallersOnlyAttribute to align with C# function pointers https://github.com/dotnet/runtime/issues/37612
00:22:21 - Approved: TPL Dataflow TransformManyBlock support for IAsyncEnumerable https://github.com/dotnet/runtime/issues/30863
00:30:46 - Approved: Consider introducing BitOperations.IsPow2 or Math.IsPow2 https://github.com/dotnet/runtime/issues/31297#issuecomment-643407800
00:35:57 - Approved: Add API to get bit length of BigInteger https://github.com/dotnet/runtime/issues/31308#issuecomment-643406480
00:46:49 - Needs Work: Add [CallerMustBeUnsafe] attribute to denote APIs which should be called in an unsafe block https://github.com/dotnet/runtime/issues/31354#issuecomment-643417759
01:11:27 - Rejected: SequenceReader.TryReadTo(out ReadOnlySpan(T) sequence, ReadOnlySpan(T) delimiter) https://github.com/dotnet/runtime/issues/31355#issuecomment-643421064
01:19:33 - Needs Work: Utf8Parser overloads for ReadOnlySequence(byte) https://github.com/dotnet/runtime/issues/31357#issuecomment-643427770
01:35:55 - Approved: ChannelReader(T).TryPeek https://github.com/dotnet/runtime/issues/31362#issuecomment-643438552

A

Alright, first issues, so let's talk about I should be content. Read a stream. Do you walk us through it.

B

Yes, this is kind of follow, ups forgotten little things from sink HTTP, client API. As of now, if we implement the HTTP client sink API. The second method will work synchronously, but the return response and its content will have no way how to retrieve the underlying string synchronously. So in the end, the user will end up using async method anyway.

B

So this one introduces I, think methods to get to the content stream and we tend to implement it for all of our own HTTP contents, and we have ability to do it reasonably and without any blocking over async on anything, because we already have the stream available for all of our contents and in the little proposed API. You can see the methods and they follow up the same pattern as the async methods, with a little exception, open and little exception that the protected virtual methods, great content, rich stream.

B

We decided to go with only one method, with a default value for optional parameter instead of two overloads, because the two overloads would force the implementation to overload both of them and this way since it's only protected and not visible. On the external API of the class, we think it's like more easier way to implement this way.

B

Any questions, or do you want to know any details? I.

A

Mean if this would have been public, we would have just a public non-virtual create content via stream. Without a cancellation to open. They would just call the other one right. So dick you wouldn't have to implement both of them. You would only have to implement one of them, but given it as protected I, think I, don't care I. Think an optional cancellation. Token works for me. Yeah.

B

A

Don't know how Jeremy feels about this.

C

Yeah, like I, think so we'll we said virtual method shouldn't have defaults. I think we said virtual method shouldn't have default, except for cancellation tokens.

C

A

Seems fine, okay,.

B

A

Then read a stream might make sense to me. What's the other idea.

C

What yeah I was, why would read a stream we do overloads instead of a default? Did.

B

Too, followed because in the HTTP clients and method, we had one method with default values, and he wanted me to do the overloads instead of it, because the async method had the overloads to be consistent with the async, but an async method on the same class. Every.

A

Base is having money model, what what the existing api's are doing and they were largely overloads, no default. Okay, that's a reason.

A

Alright, then this was easy. Unless there's an you push me I'll approve this now, I don't hear any so that means.

A

And blocking I can remove now.

D

A

Right you're done for the day good job. Thank.

D

E

Do easy, I wish everything is that easy, I didn't.

F

Even have time to get distracted.

C

I mean if people would just accept when I say no right off the bat and the healthy baby come.

E

C

Came they lead on that six.

E

At the exact same is talking five done that sevens the same. You start at six. Every.

C

Now and then there will be a crypto future.

A

Featuring quotes, of course, um alright. So who wants to talk us through it Aaron or if Fred I.

F

Can take it unless Aaron wants to so. Basically, the idea here is that, as unmanaged collar as unmanaged collars only stands today, it has its the top thing here, which is: has a public calling convention type now what this means for c-sharp is that we need to actually understand what calling conventions map to what set of mod ops.

F

If the runtime wanted to later add something like a vector call, for example, it would need to add vector call to the calling convention enum and then the c-sharp compiler would need to be updated to know how to translate from that to a set of mod ops and the whole point of the the API, the extensible calling convention API that we went around and designed was that the c-sharp compiler shouldn't actually need to understand what what calling conventions are valid or how to translate them to a representation it should just be able to take whatever the representation is and just say.

F

If it's equivalent, then it's the same. If it's not equivalent, then it's not the same.

G

A

On like I'm still somewhat lost so basically the this is for function, pointers in C sharp right so who puts unmanaged callers only on that the developer or so.

F

That's the user would put unmanaged colors only on a method that they want to be only callable via a function pointer. So, basically, when you put on manage colors, only you're saying that I want I want the runtime to actually create a a stub for this. That is calling convention, C, deckle or calling convention fast call or whatever I. Some I think.

G

I'm Han to call.

F

It directly for managed code, and it is convertible when you take an address of it, that that pointer is see, tackle or fast call, or what have you, and so in order to determine what the what the correct type of that in c-sharp should be. We have two options: one. We can say that we don't do any help at all and we'll just trust you when you take the address of a function pointer and we won't do any validation whatsoever. That doesn't seem like a great experience.

F

So if we want to actually like give an error, if you attempted to take the address of a function pointer, that was a C deckle and then stored in a in a function pointer that was fast call. For example, then we need to be able to have some way of converting between the attributes specified in unmanaged callers only and the actual calling convention of the function pointer, and so basically, what this proposal is is the current the current c-sharp proposal for how the how the function pointer will actually look like.

F

Is you specify a set of types you say unmanaged and then you would say, call consider coal and then maybe call con suppressed GCI, just press g c transition or some number of types, and so those same types that go in the in the function. Pointer, syntax would go and the unmanaged callers only in this array, and so this the compiler doesn't have to actually understand anything. We just look to see whether or not the types are so.

A

I'm certain I should, like so the so the array that the user passes in like where whether these kocoum standard call mode of school, they come around.

F

System well so they their MA direct on the return type of the function born a signature or mod opt I mean on the on the return type of the function, pointer, signature so.

A

The function pointer itself is a some some type representation that is effectively comprised of mod Rex, I guess.

F

Basically, so there's an existing call kind in the function, pointer representation that has default, which is managed C deckle. This call stood, call fast, call varargs, then some reserved fields, but there's potentially even more calling conventions that the runtime would want to add.

F

So the proposal for the runtime support is that we're going to add a new call kind, one more new call kind, something, like extension, that's the word I've been using for it, but doesn't some some new flag to indicate that the calling convention should be determined by looking at the mod mod ops on the return type and.

A

So normally like mod R, X and mod ups are part of a signature of something, so they can go anywhere. They can go on parameter times. We can go in the return type. They can, you know, go below.

B

It's just good.

A

Ways, so are we sure that the flat array on the on the attribute is sufficient for all scenarios? Yes,.

F

Because there's only one place so the the extension that we're proposing only goes in one place on the function pointer, it's a mod opt on the return type before the ref 51 exists, I see, that's it.

A

I mean it is a bit verbose. Let's say that but I guess: if that's what we do, that's what we do.

F

And then we were talking offline emo that there might be a small change to the c-sharp syntax I hope it doesn't happen, but there there is a small proposal that maybe, instead of just having the types, we also have basically there's there's two pieces to this metadata right. There's the call kind and there's the mod ops. Currently, the c-sharp syntax only lets you represent the mod ops and then we control the call kind based on the MA tops.

F

There is a small proposal that maybe we have C sharp syntax to represent both the call kind and the mod ops at which point unmanaged. Only unmanaged callers only would need to update to also represent both the call kind and the MA tops in order, so that that notice, that we have that same ability to not care about what the representation is and just use the same logic for both.

A

If yet, the only thing is like, so we basically still call it calling conventions. But like is this the right name for the field now.

A

B

You're, basically,.

A

Supposed to pass an arbitrary modifier.

F

What so that maybe.

A

That should be different.

F

The types aren't arbitrary, they do have to come from system run time, compiler services, and they have to start with call kampf and we're going to error. If you attempt to set anything there that.

H

Is oh I see okay, nice there's.

F

H

At the towards the bottom that mentions the Roslyn, compiler will be validating things, I mean.

F

Obviously, in run time, you can construct that with what I like like, if you write some il you can you can do whatever you want there, but right.

A

But okay, so that seems like any any more rec that we would add would still be called conf, vector, call or something and then and then then, what's still in contention based okay, that makes sense then, and it.

F

Has to come from system private coilette, it can't come from some other, the new library users, users can't define it. Yeah.

A

I mean that makes.

F

A

Me I mean then also the name makes sense. I was about to say, Calkins is weird, but given that all the things start of Corinth today, it probably makes sense to keep the prefix.

A

Yeah, it's been to appear at the other day, like I'm kind of that, unless Jeremy takes offense to that.

E

I actually had a quick question from so there's. This is in order to allow multiple different models on to the return. Correct is if, if the common case, I assume, the common case is going to be that you really only care about one, and in that case we would still just refer people to the old syntax. Just because it's easier.

F

Well, so the common case is that you possibly only care about one. One of the things, for example, that we've been talking about is the ability to suppress GC transition for a management attribute, and so that would go as a calling Convention on the return. That way would be call comm, suppress, GC transition so, and the other thing is that we're trying to it is.

F

It is a bit of a wart in that it's not super pretty syntax, but we one of one of the things here is that it's it's the same syntax between unmanaged colors, only and the function, pointer, syntax and the comp and the compiler doesn't have to actually like care about what the set of valid calling conventions are. We just have to make sure everything matches, and so then the runtime is free to add new things without having to get to compile a comparative compiler update and everything kind of just works. Sure.

E

I guess I guess what my question was coming down to is: does the original unmanaged callers only calling convention blah blah blah API still makes sense in the new world and it sounds like the answers? Yes, we wouldn't.

F

Back that API, oh no, we would back that 80. I am because that requires the compiler to understand this. The translation between this calling convention enum and the set of mod ops and call kind- and the whole point is that we don't want to understand that. Okay and.

A

So do I always.

E

A

I always have to specify the calling conventions or.

G

Is this optional hide I believe so I.

F

G

G

H

The function pointer, spec, if I recall, there's the unyu, can just specify unmanaged right and at that point the runtime basically says I'll determine this based on whatever platform I'm running on, like it's, the implied on Windows I'm, pretty sure that's a standard call in x86 and and on unix at sea deco, and so we could say, yeah, there's gonna be a qualcomm unmanaged and that that is equivalent to the unmanaged function. Pointers semantic keyword, but I I think that that's I think just not setting it is yeah use whatever the platform default. Is it.

F

Doesn't require anything that actually does make sense, because it's the same as the C sharp syntax, where you don't specify anything right, yeah.

H

So not specifying anything is completely legit and to be quite frank, that is actually one of the reasons why we kind of looked RiRi examined the c-sharp function. Pointers because originally we were gonna. Have you know, C deckle is a that's semantics keyword and then we were like that's very backwards. Looking because it's it's limited to x86 32-bit only, and so the idea is in a world where 64-bit is the de facto or the most common case.

H

You know why should users even think about this, so the only for right now, the only real, interesting scenario where users would probably specify a call comm is if they're on a 32-bit x86 machine. That would be the only real reason.

I

We we are planning on introducing another calling convention into the run time. That's that exists across different platforms, particularly on Windows, because of the way instance methods are handled. So that's the that's the only one that would make sense off of x86 at the moment for the unmanned colonization.

I

Whereas the default would work in most cases, otherwise I believe.

F

A

I mean: do we make sense, I mean like if that's where we are that's where we are I think my only question was: if this is not optional, then we should not make it. A property. Probably probably should be constructor argument, but if it's optional, then the shape makes sense to me.

H

Yeah I would be very surprised how often this is used in other cases. Like I mean there are cases where it might be interesting, but that's definitely going to be the lower percentage, not even considering the fact that this ideally will be used for a very small, very narrow slice of the.net ecosystem.

A

Yeah, that's famous laughs right, okay,.

H

So yes, I can imagine people abusing this and having fun doing something but yeah. That's fair. The.

A

Honor you've already said they love function. Pointers are scary,.

F

All live on this court and they all know who I am.

A

All right, then, all right thanks everyone. Thank you.

G

See y'all later.

F

H

I said one one, quick question: I did Fred jump off. He did that. One question that I had is I that there was a he alluded to potentially adding something back to this API. If we need to, depending on their conversation, that for function, pointers and I thought we were going to actually ask if we could figure that out like if there if we could get, we could do any small modifications to this API offline, as opposed to spinning up another meeting. Yeah.

A

I mean that yeah I have no idea what this looks like I mean I, assume it's another property of an array, probably or something like that, which could totally.

B

A

Over email, like I, mean, and it depends how long it takes us if it takes us four months. You probably don't remember the details. No.

H

All right, III think that this we're gonna we're gonna, get to consensus here next week, sometime as far as if we do need to add something small, all right all right, so we'll do that online on get Ober offline. If we need to yep.

A

Sounds good, okay, awesome thanks awful thank you and then I think we can just continue with regular backlog, so there's a few blocking ones that are still marked. Let me just pull them up.

A

You say that okay, it is actually opening.

A

Funny I'm accident I, don't know what is it whatever? It is.

A

I believe we had a brief conversation with Arthur on this. Basically, the question was whether they should change existing behavior, whether they should add a new attribute and I. Think you just said. Probably new attribute is fine, but I. Don't actually remember the details of this so probably need to make sure that poor novice in the room, so you can talk about it unless somebody else has contacts on the call which I assume was not the case all right, then, let's just ignore this one I posed for this in the same vicinity.

J

A

Like we want to have enough in the room for that, one.

K

Tim Tim Tim Tim.

E

It did we lose our fearless leader, no.

A

One was just silent: mm-hmm, just typing is hard.

A

Assume you can still hear me.

E

Yeah your videos just frozen yeah.

A

My video is like my cpu was struggling hard.

A

So I'm just going all the seniors now and see how far we get.

A

Yeah, who marked this one here.

C

A

I guess I need to send anybody for this thought. I did that already.

E

Yeah, we're actually uh not ready for that. Yet I've been on I've, been working offline with the EF team on this. So what's missing here, it there's just some some other like documentation. That needs to be done. First, regarding all of us, it's it's just politicking and stuff, nothing! uh Nothing that should is she's really concerned this crew too much that's good yeah.

A

But just curious, like hey, you know, do you know roughly know when it will be ready for review I.

A

E

A

What's his name again chase.

L

Let's, on market ap ready for review, yeah.

E

A

It it's not, but yeah I mean I'll, sorry method of shy, and then we see where we land. We can. What we set up something for two weeks from now and probably will be enough. All right.

A

Steve another call.

A

I think he was really I know yeah.

C

Sorry I mean I was on mute.

A

Do you know what this is.

C

Yeah, so in the data flow library there's this transform many block, which is effectively a select. Many you give it a delegate, and you know know as input data comes in it. It runs that delegate over the input data and produces something that about container some of you overload and because it's producing potentially multiple things, the delegate returns and I numerable and so there's the corollary, which should have a delegate that returns and IA sync a new report. So that's on the processing side and then in terms of consuming output from various blocks.

C

Just like we have with channels where you can kind of asynchronously enumerate the output, there's a desire to do that with dataflow blocks as well, so basically adding an extension method to receive all async from these various blocks, with an extension method that allows you to wait for each the contents.

J

Yeah I would have used both of these in the past, so I like them.

E

Yeah agree: this seems pretty straightforward for the scenarios.

A

So I don't know enough about sous-chefs rules here but like if I do a sync here, it will automatically create Anaya singing new mobile.

E

It should because of the yields in the block.

A

Yeah I thought maybe flee like the way that would have the ruled stirs that you, basically just I mean for backwards coming through. The reason I would assume that they sing here just means you're referring to tasks if I knew mobile, but I guess.

M

They don't allow you.

A

To do that before right, they didn't have healed and.

N

Yeah, well, you couldn't have yield an iterator.

A

Okay, that makes sense in.

A

Yeah this anybody has any concerns that seems ease no.

N

C

I met y'all did.

A

You do what I did yeah that's what I heard I think.

N

We ready for this Levi.

E

I'm still waiting for feedback from the internal document that I sent around so no okay.

A

Then skip design.

A

Anybody opinions on this tanner, maybe.

O

Yeah this one is basically just requesting. We add a new method to quickly check is pao2. You can do this using pop count, but in the case that pop count is not supported. There's a faster implementation that should be provided just a fallback.

A

Then makes sense to me I.

C

Think this is fine. Dad I'm, just curious, enter other places will use this internally. I seem to remember us doing checks for power of two minute riot e at places, but I can't recall any of the top. My head I believe.

O

That I believe there's at least one or two places where we could use it internally. At the very least, we've got similar checks already in the floating-point parsing in formatting code.

E

Array pool uses this one return as well or could use it.

A

So pop count is not.

E

A

A hot van trinsic way we just happen to be basically always provided implementation for a pop count right.

O

Correct we've got both the actual hardware intrinsic and then we've got the bit operations pop count, which provides a software fallback in the case that pop counts not supported.

A

A

And I think we do operations is a type we headed recently right to hold all of the stuff.

O

Right but correct, it also holds like a leading zero count, trailing zero count, rotate left and rotate right.

A

Yeah, it seems like the right place. It seems like the right semantics. I mean, makes total sense to me. Yeah.

C

It's like we have like the power function is on mass. Does this not make sense on maths right.

O

That was a topic that came up here for one of the things that keeps coming up with math is it may not necessarily be the best place to continue exposing a lot of functions because they're they're quickly get you quickly get into overload resolution problems as you want to expand it in the future, given that it trying to support mainly double, but also potentially, all the other primitive types as well, depending on which operation it is. This is realistically.

C

O

C

The argument for putting this on bit operations is not that its power of two. It's that we're checking to see if it is set like what and only one none right.

O

And while it might be beneficial to have it on math and more discoverable, there I think the the the downsides that we continue coming into with math. Just push it over that edge into bit operations as being the better place.

A

Yeah I already did math and math F right and then it it's a bit weird to have. Math dot is power to with not even any doubles right.

E

Yeah the the reason I like this on bit operations as well as because it is fundamentally a bitwise manipulation, organ why's inspection, except for the zero case, which is special case here.

A

So we were very cool.

O

A

O

No particular reason: why know why we didn't expose short and bite other than we generally don't expose explicit operations for short and right outside of a few specialized scenarios.

D

C

Shortened bite would both up convert to end and that's fine on a 32-bit system and I. Don't think we support any 16-bit systems. So there's no like on a 32. You don't want the up convert to long, but since we don't have 16s, we don't need those smaller value types right.

O

And there there is no explicit hardware acceleration for pop count over a small.

A

O

16 or eight you.

K

O

You are very quiet, though Eric sorry.

A

All right that was easy at API to get bid lengths of big integer.

E

Actually of related, should we add, is how to to big integer as well, because it is a numeric type or is that kind of out of scope, I.

A

Mean we generally don't do bit fiddling with the integer? Do we it.

E

Will certainly such an api probably wouldn't be on bit operations? It would be on bigger teacher itself if we added up right.

O

I think that's out of scope. For today it's.

C

Already a public property yeah again, a Gernot is power of t wo.

E

Works for me, okay, somebody.

A

O

So so this is basically just proposing a new method to get the the number of bits that are in the big integer. This allows you to determine exactly how many bits which allows you to do other interesting tricks. Mathematically speaking with the big integer.

O

It's it's currently expensive to compute yourself, and it would be much cheaper for us to do it internally. Given we have access to members and data that that isn't exposed publicly.

C

This is the bit number of the most significant bit +1 right. It's not the it's not pop count. Correct.

O

This is basically length, but in bits, rather than in bytes.

A

So what would lengthen bits be better than or more interesting than bits in bytes I'm? Sorry, then, number of bytes isn't the basically just number of ice times. 8 10.

O

Not necessarily because we may be using more bytes to represent the data, so, for example, if you have the value 1 you're using 1 bit out of the 8 bits in the byte, that's used to represent the underlying data. So this basically tells you. How much is this going to need to grow by if I'm doing specific operations, essentially how much more space do I have? In the current? In the current backing array, I see.

C

Because multiplying two numbers, essentially, the size, is bit length of one-plus bit length of the second yeah.

O

And then, if you're, adding the maximum value returned is bit length times the larger of the two bit lengths plus one I.

K

Mean seems reasonable to me.

K

Yeah, so what's the actual proposal next proposal is this just get bid length yep.

O

They originally proposed a property, but given that this isn't necessarily always cheap to compute, it's fairly cheap, though I proposed to be a method. Yeah.

C

I was gonna, encourage it to be a method instead of a property, so I'm good. With that proposal,.

A

Do we have any profit like we don't have any part for this stuff right.

A

The only thing.

B

They could probably.

A

Do is maybe we should have get bit count rather than get bit length, because we haven't get bite count already.

K

It's on big integer, yeah.

C

It looks like we also public get length.

A

That's great, so we already very consistent in our use of terminology. I don't see.

A

Sure I don't see a length at all on beginner.

A

Are you looking at the right big integer? Are you looking at the mono version, I.

K

C

Looking at the right but I'm, looking at source net, so maybe I'm finding something I shouldn't be able to see. Maybe it's not in the ref yeah, because they get byte count. Yeah that wouldn't take a parameter is clearly I added that one, because it was the day that I said is unsigned. Instead of unsigned the.

C

Because that one needed to know if it needed padding or not so it was it's not about the number it's about the encoding.

O

Yeah, it looks like there is no length property on publicly exposed.

G

A

And because it would almost say get bid count.

O

Your Oh Jeremy you're, looking at xml XSL XPath, convert big integer.

C

Emma I guess as a system that number that big integer I think this is the thing driving like X. Pretty printing, though not.

C

Yeah, this is a it's a public method on an internal type, which is why I hate when we use the public key word on the internal types.

D

A

So I wasn't wrong with it. I'll tell you looking at the right time, because big integer I've consistently looked at the wrong time.

A

C

I was looking at system, got number that big in internet systems at numerix that big integer.

A

Number, who does that that's just out wide evil?

A

Alright, then yeah any objection to what I said or is it? Does it make sense of people I.

O

Don't have any objection to be get count. Yeah.

C

I mean, like certainly I, feel that bit length is a more normal term, but that maybe crypto is talking so.

J

I mean I feel like the by account is more related to maybe serialization and bit count is more related to arithmetic, so I, don't think that they need a symmetry here. I would.

C

Also worry that it bit count as opposed to bit length suggests it's pop count. Instead of most significant bit, I was just about to say that as well.

A

So what was the last part that don't that.

C

That, when hearing somebody else or when hearing Cory say the word just now when he said get bit count, I heard pop count. So how many bits are set to one as opposed to what is the index of the most significant bit so I feel that bit length is the length of the vector whether they're ones or zeros, and that count feels more like which of the how many of the bits are set to one.

C

That's what my brain heard.

K

Works for me so.

C

So I'm back to get bit link.

J

Is there a reason this wouldn't just be a property bit length? Is it expensive.

G

O

It's not exactly it's not necessarily cheap and someone could implement it very poorly. What.

A

Doesn't get parts for manipulation? Do they know so.

O

So big integer is a little bit magic and how it works it. It has an array as a backing type, but that array might not be used. It might instead use the use a special I think it's a long value which contains the sign if the array is used and otherwise contains the low 64 bit value of the big integer.

O

So you have to do, but you have to do several checks to see like which bit of data is actually in use by this big integer based on the size, and then you have to do the appropriate call in computation. Based on that, like you, wouldn't really want to do.

C

The you wouldn't want to have, you know valid bit length as a property in a in a while loop or a or a for loop, because it it does more work than you would want it to do in the loop scenario. So it's better as a method. It says, call it once and remember the answer.

A

Yeah I mean make sense to me: I mean like it seems, generally speaking, intuitively it. It needs to be an Owen operation. Right I mean the end, might be small, but like fundamentally you it's it's a whole other. Why this guy is right. So yeah.

E

Well, the thing is, fundamentally: it's a log 2 operation that returns an int instead of a double.

A

E

Which implies it could be called log 2 and.

C

Yeah well, I mean, but it's not log 2 drunk it.

E

Has to handle ie 0 C.

C

Love log 2, basically.

A

All right, so let me just post my comment, then.

D

Sorry, like so the big integer, having is power of two and then this being is POW to kind of sort of rubs me the wrong way that they're, inconsistent I.

A

C

Down different bigoted spelled out the T if.

D

We spell out power, Oh, f, TW, oh yeah,.

A

We say it's power off and I ever say is power tool, I.

O

Mean is power to is more consistent with what some other systems call it and with how the terminology on math is regarding POW in I would say that big integers is the odd one out here. Actually systemwide I agree.

E

With Tanner uh bit, operations, in fact, already has an API called log, two with a numeral two, instead of the word two written out, even though it's a log base, two API.

A

Yeah I mean I, guess maybe I'm just weird, but from a operations I. Always it's probably the way we started because we happen to mirror the C names but like I personally, have no objection to that. I mean I, hear what you're saying, but to me, consistency between types and their address to the different areas is not ideal as well, so not terrible like on the same type. Yes, but on different types. I think that.

D

A

Yeah I give you that okay.

D

I just wanted to call it out that it like it's doing the same things and we're calling it different things right.

A

I mean it's also just a shear shape. Right, like you call an instance method rather than a static method. I mean it will never look like super clean right. If you actually happen to process ins, lungs and big integer and the same method right but I guess, yeah I think to me is more important than within bid operations. Then they make sense and I guess its power to would work, but its power of two where we split earliest fell out too. That would seem weird.

C

K

C

Yeah I think that, based on the fact that we already have log and then that the decimal digit to means is POW to is matches the type better.

D

It's funny how the math stuff gets a pass for design guidelines, yeah.

A

I mean it's not just math I mean it's. Obviously you know the the the context in which you try to make an API work.

A

Why it's a crypto is another area where we have sort of given up because it's the prior art is overwhelmingly weirdly named, and so, if you add one more thing, it's just like well, the naming conventions in crypto are this, and so we just stick to those right and that's the same thing with math I mean if we were to design system math today we will probably use different names, but even on the hardware.

B

A

Side, we basically said like look. You can't give everything super nice names, because we kind of still have to somehow use the terminology that is used by the you.

B

Know manufacturers.

A

Of the ISAs and sometimes they're just weird, oh and.

O

I'm also not convinced that your statement, emo on we use different names, is correct. When you look at calculators, latex and basically any mathematical use of the terms. No one uses power, they use pal. No one uses log T wo, they use log to log ten etc in the mathematical world and in the context of other programming languages, work with the math libraries they under they overwhelmingly use the shortened names here.

O

A

We know in cyberspace.

O

A

Out the numbers, but I mean you know: people I mean power of two seems reasonable power. To seems reason to was the name right, but yeah I mean I, don't agree with you like I think we we we have sometimes given abbreviations of pass. If it's effectively an industry term right and I guess for power, can it can be convinced that that's the case.

D

I'm satisfied I just wanted to make sure.

A

E

A

It's bike time, I think it's I, think it's fair I mean it'd, be like the framework. I mean we try to be consistent, but we also have to recognize that total consistency is not achievable, and so we need to do the second best, which is you know, local consistency to a certain extent, all right. So then this one is done.

A

Is this ready Leroy.

E

Speaking of Pyke, shutting, yes, it actually is ready.

E

The the general idea behind this is that there is a desire, at least by me and a few other people, to introduce a Roslin analyzer that says, like hey, you're, calling an API that you would normally need to use unsafe code for like you're, basically calling an unsafe, equivalent API, but we didn't put unsafe in the name and people don't necessarily know that it's unsafe, so the analyzer would say, like you, have to call those in an unsafe block, even though you're not technically using pointers in order to identify such api's, they would be attributed with caller, must be unsafe, and that's this proposal to introduce this attribute.

E

This was a little contentious just from the point of view that well, what does unsafe mean, like is private reflection, unsafe stuff, like that, and the answer is yes, but I. Think for this we would want to limit it to api's, which are doing pointer like things so. A good example of that would be like memory, Marshall dot create span ref of T, comma and like there is quite literally no way to represent that in typesafe, c-sharp you're, quite literally, going into an unsafe world. When you call that API.

A

So I, it makes sense to me and I mean since you bought up analyzers I, think it's that would address my other concern, because, basically, almost all the things that you could do like this, we already have kind of API as forum. It.

B

A

Force that if it's a compiler feature, but if it's an analyzer feature, we could totally like start warning when people do that, but I didn't require unsafe. Would we do that, though, or would you only use it for a new attribute for new API.

E

If, if we were deliberate about where we added it, we could consider attribute in existing api's, create span and create read-only span were two examples that are listed here. I wouldn't attribute types like unsafe, unsafe, not as stuff like that, because the word unsafe is already in the name like you, you literally wrote the word unsafe. You know in your code. That's fine yeah,.

M

I guess I, don't know I I kind of feel differently, I feel like ever gonna do it. We should do it on anything that if.

C

If there was language support for what is required unsafe, but at the same time, I don't see any way that this you know this analyzer can be warnings or errors by default like at best. We would want to make it an info by default, and then we go. You know and annotate everything that which should really be in an unsafe yeah.

O

I would say the same. My thinking is that even on something like unsafe, you can remove that unsafe keyword and hide it by using using static system run time. Compiler service is unsafe, and then you can just call like skipping it, in which case having it even on those types is beneficial. Given that it's going to be an analyzer, the analyzer will likely not be on by default. Should this be callers should be unsafe, I.

E

Don't have a strong preference honestly I mean.

C

We could decide, we.

E

C

Turn the analyzer on by default in six I I, don't know, I feel like well, so two things, first I think you should describe what the method is doing, not the requirement to places on the caller and then the analyzer is choosing to basically say as a result of that, you should really be in an unsafe lock. So for me it's not caller must be unsafe. It's this lets. You do unsafe things whatever. That name is Kali.

O

Is unsafe, yeah.

C

The equivalent of that and then for me I, have trouble imagining a world where we ship the analyzer to customers with it anything more than info aside the bulk. We would probably turn this on and buy this on time, as you know, warning which basically makes it error by default, but in general I I feel uncomfortable about.

C

Being able to put this on things, and it really affects the semantics of the language, because anyone could add this attribute to anything and all of a sudden you're getting warnings or errors. For you know the compiler saying this is unsafe. When it's not, it least, that's my mental model. What.

E

About for new code, what new code be a warning instead of info by default, I'm, sorry for new API guys if we attributed this on new api's with that, would that analyzer ship is on by default.

E

C

Don't know it seems like a lot of noise like for someone who doesn't care it's. It seems like a lot of noise. I mean.

A

That's kind of a concern right. It's like the question is: what would that mean? I mean so to elaborate on Stephens point, like my other concern is if we, so if you say unsafe, what does it really mean? I mean? Usually it will mean that the caller has to be in an unsafe block, because it doesn't make much sense to annotate the calling member itself, because you don't probably don't have pointers in your API to begin with, and then.

B

A

Question is: will the compiler ever say something like I'm using unsafe, but you know using any pointers. That's actually somewhat like here. You know it's not necessary kind of analysis and then you get conflicting warnings. That says one one part. It says you don't need either the party says you need to use it and then really the question is what would I mean the fix that would be relatively simple. You would just surround the code with a lock that says unsafe, maybe update the project setting to say is unsafe or some or whatever right.

A

But that's and Stephen said right. I mean it's probably an easy fix in the sense that you controlled all your way through that, and hopefully it's not that many places in your corpus unless your systems on run time or system, the private or call it, and so it's probably fine for most of our customers. But it does seem like one of those things where, if you care you care, but if you don't care, you really don't care right. It's yeah! You just make me yeah.

M

A

M

To do the pay for work here, we're enforcing our own opinions on.

C

A large number of people, I.

M

Don't think we.

C

Have right well.

O

It's not necessarily an opinion, given that, like memory marshal dot create span, is unsafe, it bypasses language semantics and if you're using it, you are very it's very possible to screw your code up easily.

M

In that, you need to be using the unsafe keyword in C sharp.

O

In which case the analyzer doesn't say that the actual code block needs to be unsafe, but rather the compilation should have been compiled with allow unsafe code.

N

E

Was going to say, like the the definition of this, basically would be that the code that you're about to write is not verifiable. That's really what this was tracking. It doesn't have to be a pointer per se. It's just anything you were about to do. That's not verifiable right.

C

And is there anything that you would already be wanting to put this on? That is something other than unverifiable memory, because Steve's earlier point is you're you're painting the action that someone should have for the call context, as opposed to describing the problem. So if the, if we made it the positive assertion that this is non verifiable memory operation or something, then then now we have a an analyzer of hey. Tell me if I'm, making non verifiable memory assertions and it's like okay well here you go.

J

O

Marshall dot get span from list is one example where I think this would be applicable which isn't unverifiable, but it is unsafe in that you may be pointing to an array that no longer backs the list anymore right.

M

Right yeah, but the same thing can be said about a rape wall.

E

Like we had us for a pool about Red Hook, so according to yon, yes, but right so.

B

I'm sorry I shouldn't that worked.

E

In his mouth, but he he did say that he sees the array pool as a male equivalent right.

N

C

But your weight now we're going down a path where hey there pretty much. You can't ride code without putting on st. Brendan every method. I mean like I.

E

Mixed I'm, exaggerating, but.

M

E

Yeah I, wouldn't I, wouldn't put this around a rate whole honestly, like that that ship has sailed as much as I want to I.

A

Mean to Stevens Point, oh I, think one thing that the analyzer does do is: is it basically forces an opinion on how the caller has to annotate the fact that they do something unsafe right? Because I mean it really doesn't change the semantics of the code right? It just says so you can already do it today. It's unsafe! It has the behavior that it does, but we want you to record that fact in your code base we're surrounding it with an unsafe keyboard or something along those lines. Right and that's that's.

B

All mostly a coding.

A

It's almost like a coding guy that you force from people right, so in that sense, I would be.

E

The reason for that by the way is because there it turns out there are a lot of enterprises that require strict scrutiny of code that uses unsafe blocks. Those enterprises allow the code, it's just. It requires, for instance, several people to sign off on it.

E

What I've seen in practice is that, because people don't have to apply that they are more relaxed about calling some of the api's that we've provided in dotnet or three, not realizing that hey like you're, you really should have gone through your proper review process here, because because you didn't have to write the unsafe keyword, you might not know that what you're doing is unsafe.

N

So these the groups you describe, though, are the minority of developers. That's not to say they don't matter. They absolutely great by enforcing an opinion on everyone else by default is a problem. I mean.

A

I will say, though, like if, let's turn this around, let's say we would not apply this attribute or anything that already exists right. Let's say we would only use it for new API. It's I mean we have said in meetings that we all said we wish we could force the unsafe keyword on certain api's right. Then that would be a way to do it.

A

So I guess I tried to separate the you know. What do you do for existing code versus what do we do? Moving forward right, yeah.

N

But I have trouble that distinction I mean some of the worst things that have been around for 20 years. So.

A

I agree will not read right, but I guess maybe maybe part of it is also because people know know that anything in system interweb services, especially anything called Marshall, probably is something unsaved where people.

E

Don't realize that, like people really do not really.

C

Are we gonna? We gonna require this on every single diello import. Sorry.

N

E

Like if, like, if you're calling an API that takes an int pointer or a void star as a parameter like you, are clearly doing something unsafe like clearly, the the thing is that the api's will be provided on on types like memory. Marshall, don't clearly delineate that, which is why people don't realize that they're dangerous, and this was attempting to solve that. My.

C

Point is I can write at yellow import with no complainers, no pointers in the signature. It's just basic things and screw-up.

M

Memory way worse, then, you know any of the things that we're activating is unsafe. Yeah.

E

But people don't do that in practice.

O

We also don't publicly export DLL imports in the framework itself. We we'd only be putting this on members which we publicly expose which would qualify under that unsafe Ness. If.

C

It's not going to be a public attribute. Well,.

O

It would but oh I guess you're asking: what would our public recommendations for applying it be then right.

E

Right so the the recommendation that I would give for applying it would be. This API allows you to do something that is not expressible in type safe, verifiable, c-sharp, I.

A

Guess one thing to drive this list would be: can you give us a number like a list of API, as you think, qualifies, and and also I guess it is a short description of what a new API is. We would apply this to and also maybe this some examples of II would not apply to right. I mean I to me. It's like the the attribute name probably needs to change.

A

Where we talked about this I can see the analyzer being valuable for the, for the reasons you stated, I can even see us annotating existing API s so long the analyzer is off by default and the people that really want to get this can opt into that. That's also fine, in my opinion, but I do see the problem that Steven trains, which is it's not a clear-cut thing right.

A

So we would have to be somewhat consistent about how we use this, because if you live in a world where a third of our API is end up having this attribute that we failed, but it needs to be somewhat of a very rare case similar to unsafe today. Otherwise, I really feel that its purpose, because then you do the reverse, then you train everybody to ignore and say flag, which is not the way you want to do this.

E

Ok yeah the the list right now of API so would qualify, would be basically everything on the unsafe class. Almost everything on memory marshal and, unfortunately, a rape hold on Trent and the reason that a rape hold on Trent qualifies is because, as an implementation detail, it calls GC decreed uninitialized array. If it didn't do that, it wouldn't qualify as having this attribute.

E

So then, DC create average length array would also qualify. Yes, it would and because.

B

It's not something you can.

E

Express in type safe, verifiable, c-sharp and anything that we ate arrays with that. We then.

N

End out in one way, shape or.

C

Form to use your code, yes, which is probably an obscene number of places, I.

E

Would hope that we don't pass an initialized array out to user code, all that often well.

C

Any anyone we're creating one and callings stream read streamed up right, but the create uninitialized array, if I recall correctly, if it blah blah blah or contains blah blah blah reference types still or that still clears it. The only time that you get garbage in the array is when it's all little types, so it's.

E

C

It's just it's unexpected. It.

E

Bypasses the constructor and when you bypass a constructor, that's no verifiable.

O

Reference types are still forced zeroed, always yes,.

E

But there are structs such as those a trap and pointers that are non-zero right.

E

Yes, but the fact that they're not zeroed is something that could not have been done in c-sharp without using unverified Lyle.

C

Right, but if you have a, if you have a if you use an array pool of any kind, whether it's a custom, object, pool or use array pool that create or whatever, if you have a struct that has an int fitter in it and you're reading it. A second like in int pitter is the safe version of unsafe code right. It's the moment that it's there, unless you yourself know where the value came from, is semi untrustworthy. So.

E

C

E

So, for, for example, if I have, if I have an array of runtime method handle as an example, runtime method handle is backed by an int pointer, at least it used to be I, don't know if it still is, and then I try and voguing a method on it. That method is guaranteed never to Av.

E

Unless the original array was created via create an initialized array, that's the one, and only time you can ever shove a bad object in there, because you never would have been able to instantiate an invalid handle in the first place, which means, if you can never instantiate it. You can never assign an array element to that value. So.

C

Private and Pitters, as opposed to public with sediment Pitters. Yes,.

E

Okay, so, but it sounds like there, it sounds like there are still resistance to the Sun, so I probably don't want to by showing this too much. So we can get to other stuff.

E

C

I mean I think that what would help is the the what's yeah, the the what's and the whys cuz. That would help us decide how many potentially different attributes we want for different analyzer levels and and what state we want them to be in and then some of them would be like.

C

Is there a way that we could figure out how to make a rule that rent where it would only trigger or where it wouldn't trigger for the the primitive types, because the most common array pools are bite and char? Those.

E

Yeah I open I open that issue and it was closed as one facts. What did.

C

E

Open an issue saying we should only clear for things like bite and charge, and that issue was closed as one folks.

C

But but we could make the analyzer say that array pool I mean we could explicitly exempt it if it's a repo by a pool char. If that's the concern, so we could slap the attribution and tell the analyzer that these two specific generic instantiation are safe. Okay, so.

E

So an analyzer, rather than changing the array pool implementation.

C

Right well, because I think that largely what we were talking about was assume that all the code stays the same. What attributes and analyzers can we add to to help people write better code or at least be aware of the codes of writing and debris pool bite rent is dangerous because you may not have a unique reference to the array when you think you do, but it's not dangerous in the sense that you're gonna go into that you're going to do a V's right.

E

Right, you're not going to AV it's just. You could end up reading memory that you should not have had access to verifiable.

C

Something like the array of runtime handle, we're now you're getting uninitialized in Pitters that are just getting dereference ly. Wouldn't you and that's or you know anything else that has a hidden in pitter that just gets read by the native.

M

C

Yeah so I think with what's in wise it. It probably makes sense. There are definitely things that I call being like man I want to just add a void star ignored to this just so it requires the unsafe keyword, but but I don't remember. This left up ahead sounds.

E

A

Is this a fair summary what I just pasted it.

E

E

Yeah and I agree, the the naming people did have problems with a naming and I understand that it's. If we want to say that it's you know unverifiable, then the name should reflect that. If, in fact that is what we want to say, it is right.

C

And then, depending on what things were applying it to and what like say, we end up with three different attributes describing three different levels of unsafety and we're like okay. Maybe we do feel that not a whole lot of people would be doing creating a span from a ref and account. Maybe we would be comfortable go ahead and turning on that, analyzer is warning by default, but that would be a discussion that we'd have when we have the list.

C

Steve made just say no anyway,.

E

E

C

Clear I, don't say no I just discuss.

C

A

C

A

So it seems like this was files were ready for review so.

A

Anybody know what this is.

E

Yeah, it's uh you have an incoming sequence and you're reading everything until a delimiter, so say that um say that you're looking for a comma.

E

Yeah, oh I'm, sorry you're, looking for a like a comma space, for instance, this would read until it finds that and then it returns back to you what it read.

A

So somebody in shed just said we rejected to my api's before because I locates across bands.

A

Y will be allocate across bands because.

C

The span it needs to return like the false cases become weird: does it mean it didn't, find a delimiter or it didn't find it in a single? Can spam that can do a slice of or does I have to allocate I think the problem is the implementation, as proposed would allocate a temporary array copy the stuff into it and then emit a span to that array, and then there was the big like wait. This looks like a non allocating operation that may be allocating every time. Oh.

A

Because the sequence is disjoined right, yes, I know I get it and why would you ever have to allocate if you just returned we'd only spam but yeah if it's disjoint and of course,.

J

I mean we have we already have a try, read to API on sequence reader. Does that allocate like I think if it's consistent as long as this one is consistent with that one, it would be no different. I.

A

Think we talked about this before and I think we said like yeah, that ship has sailed on this API, but it doesn't even need to it. You should add more api's like that. I.

A

Mean that was the discussion we had before, where I think I'm I'm more in the cable follow camp where I would be like yeah, okay I mean if you're allocating really enough. It's still probably fine.

E

Okay, there is an allocation if, if the span exists across multiple segments that allocates a brand-new array and that returns, India.

C

Because there's there's the delimiter e1, it looks like there, there is one and it Out's this sequence. Sorry then, like so there's try read two out sequence: comma delimiter, comma, optional boolean, which is not the way that it should have been structured. But it's already public.

C

So I think it's yeah, it's outing the sequence and then you can ask the sequence: are you a single span and get the span and if not, then do the complicated things? Well, I, don't see the value in oh yeah,.

F

In making more.

C

We create things for you that are much more expensive than they look.

E

Yeah, if, if we did expose this, we would have to take a parameter that says: I allow you to make a copy, if necessary,.

C

Yeah but then either that's either bool throw or it's bull make false, ambiguous yeah.

E

It's it, and now you start returning to Nino yeah it's it gets complicated.

D

A

Think the rationale why father wanted this was because they they will have to allocate anyway, and it's like whoa, don't make me write the code.

C

K

C

Like if they already have a buffer- and they call the try read which produces the subsequence, then they can take each of the individual segments of the subsequence copying and into their buffer, and that was never allocating. Yeah.

A

I think yeah, that's right everything. We also had a discussion before whether you can pass in a scratch buffer or something right. But then the question is: what does the sketch refer is large enough and blah blah blah I. Think that's right.

A

Fair enough.

A

Okay, so then that's good I.

E

Want to say part of it too, as a performance optimization, because Aldi and a Spanish cheaper than out in a sequence prefer the implementation that.

A

Is the correct order.

C

It's not much bigger if it's a single span, I think and then, if it is across, then it's still probably cheaper to do the sequence than the intermediate copy.

E

Yeah, if it's across you definitely, then it's definitely cheaper to do the sequence.

A

So we okay with that as a justification for closing there.

E

Yeah all right well, I would probably say an API that might allocate unexpectedly, especially on a type that whose entire purpose is to avoid allocations.

K

Title split', hey, hey.

A

Good enough all right, then.

A

A

Suddenly, husband even has opinions in this eye.

E

I had commented on it that I think the operations are actually going to be more expensive than most people think trying to see where it was marked as ready for review.

A

It was filed as that, oh.

E

You know what I don't want. It got moved when the issue don't move between repos I get I, think it got lost. At that point, that's.

A

O

So this is, this would only be expensive in the case where you cross two sequences or two spans right BK and that's likely going to be uncommon. Considering that everything, except for float and double, is going to be no more than 20 characters. There.

E

Is a weird boundary condition where you call something like parse int and then you consume the entire first span. Now you have to inspect the suck of the first byte of the second span to see if the integer would have overflowed.

J

E

J

Tanners comment: I think that would be just just rare.

O

Like like, outside of floating double you're, not going to get more than 20 digits that you're really going to have to process and for floating double that's a no more than a hundred and twelve and 768. Those are extreme edge cases that realistically no one describes values that precise well.

C

What's the alternative, if I have a read-only sequence and I want to parse out an INT, what do I do differently.

E

What I would say at that point is if you should really read to the to the token delimiter and then get the entire span up to that point and parse that spam as an INT in one shot.

C

But that's essentially what the sample code here is right. It's if it's a single segment defer to the other one and if it's not then copy out more than you would need and or copy out a number that is guaranteed to be too big and then move on.

J

So I think my observation with read-only sequence has been that if you are parsing it directly in small chunks like this you're gonna have bad performance. So I guess I wonder what what the point of these are like. It would almost be better on sequence reader unless these are purely for convenience but I'm, trying to think of where I would use these and I I'm kind of coming up. Blank.

A

Yeah, isn't that smiley question? So why would you not use sequence reader because it seems like this most straightforward way of doing that right, I.

J

Guess you would use these if you know that there's just an and int a single int to parse to pull off a header or some sort of envelope from your sequence and you're not doing a full parse of the entire sequence. It could be a little more convenient in that case, yeah.

A

I guess that's my question like? Is it like? What would I mean you could still have a sequence? Then you have a sequence reader right, you don't have to, but do it if you use a secret video, you can along I use the sequence, because it's basically owning that. At that point,.

A

Like I mean I, my guess is that's the scenario that they're in right, because what they do is they parse HTML, but then their parse in the last chunks, and then they posted large chunks to the individual worker methods that then do whatever sequence, reader, stuff, they're doing right and so in that sense, I can see that. But it also seems you know somewhat rare ish right, I.

J

Feel like I would push back on this and say show me where you're actually, using this show me the scenarios cuz I, just don't I feel like anywhere that you would use this. You would probably want to optimize later on and stop using it yeah.

E

Because I'm, looking at the sample code, that David wrote earlier in the in the issue description and that sample code should actually probably not be using utf-8 parser, because utf-8 parser will actually stop when it sees invalid data. The behavior that he wants is actually take this entire buffer, try to pull an integer from the entire buffer and if there's anything whatsoever left over failed.

E

Basically, he wants in Dom parse because he has already segmented the buffer. Oh, I.

C

Mean his his else case looks like he's, expecting the stop, when you hit a terminator the the fact that he copied into an overly big buffer.

E

I'm, what I'm looking at is the the comment on October 30th 2019. There are lots of them, it's the one, that's two or three under what's on the most screen right now: okay,.

C

Because this top one looks like but that he is looking for what I mean he has a problem that he's always reading 128, even if he didn't know I kid into it, but and we have skipped locals in it, he's getting garbage. But yes, honestly,.

A

Looking at the temperature, that kind of is what Corey was saying: I know you're, basically parsing a lot more than just you know, basically, a minimum right and really use.

E

What you know, what what I would probably want is if they intend for this to be used in Kestrel. I would probably want for them to create the methods themselves on a helper class, see how it plumbs through their own code and then, if it makes sense for their scenarios, then hand it over the wall to us.

E

Especially the implementations literally just going to be try pulling out a span. If it succeeds, then you know do some other stuff.

A

They do CAD Levi, oh.

E

I'm not muted, sorry, it's.

A

Awesome I like it is this like is this like I, was thinking the whole time. Somebody has a speaker. That sounds is that you're like a cat, and then it occurred to me that we actually probably have a kid on the call.

E

There they're a bit afraid because there's construction going on on the house right now, so they're they're curled up in front of me, trying to be safe, I.

A

Love it all right.

A

Yeah I was still entirely sure how this is different from sequence. Meter, though,.

A

On behalf sequel to the API is to do that right, I.

E

Don't know if we have sequence, Reader API, so two things like read in to read long yeah.

J

E

The same way to please do rather I.

N

A

K

Have you reviewed this.

A

So then we are supposed to do this, then busy get a span and then write all the boundary conditions yourself.

E

The suggestion that was brought up earlier was, if you know that an int, for instance, can't be more than 10 or 11 bytes, then pull out 10 or 11 bytes, run it over utf-8 parser and see what it gives you doc.

D

A

Wide looking at sequence, video, it doesn't seem to have any of this. Yes,.

A

Sequence, video seems to be more like you know, seek to and then give me the buffer.

E

But that's all so I was wondering what the scenario is, because, if you're, if you're going until you hit a space or a new line or whatever token anyway, at that point, you have a complete buffer that you expect to be in its entirety, an integer for instance, and you want to parse the entire thing as an integer and that's not really what utf-8 partial is meant for like it can be used that way, it's just kind of weird to use it that way.

K

A

It seems like that's the discussion. We should probably have a father at this one, because none of us feel we actually sponsoring this API, but it also seems like there was a little discussion on it already so.

J

A

J

Alternate ways to handle parsing and it's not super relevant to this issue.

E

Yeah some discussion here was like shutting how we can most in a high-performance fashion, implemented I.

A

Mean I guess my question is like: if we don't have an API like this, what do we expect users to do right and it seems like the the way we expect people to do. It is to say: well, your sequence is better be large, as in the buffers a bit a bit but it'll be large, and then you may see a process for K or 8k buffers and there are just single spans and we have parsing api's for those and at the boundary you have to decide what to do.

A

You basically have to allocate some time stack, a lock, thingy and then handle those things there and, yes, you have to do this for primitive types or things you care about right. Well,.

E

That's why I was thinking that, um if, if they want to use this inside of kestrel, then they could make they can do these methods themselves see how it flows through their code base and then, if it ends up being useful for those scenarios and bring it back so.

A

That so we can ask them, but my gut feel if that's what they have already done,.

E

But I see we see no evidence of that. That's the problem like we, we don't know what their implementations are, how it's actually used through their code base, like the the code that David gave, is a hypothetical rather than a. We tried this.

J

And I, pretty sure, I've put in an example here of how you could accomplish something very easy, with just a single extension method.

A

So, what's the primary concern with this API, then proof.

J

It will encourage bad parser design.

E

Part of the same reason that we don't want people calling like memory dot span in a loop, it's you. You might think that you're doing something pretty efficient, but in reality your performance would be better. If you didn't leave thing.

A

So is this fair, then, as a statement was, I should really be over spans, not sequences.

J

A

To me, the question really is I mean probe. Results are very relative right. We I think my concern is I'll, be optimizing for the right set of people here, because I mean just because you could be faster in principle, doesn't mean that we need to make everybody write a lot of boilerplate code right. We could still say.

A

Yes, you still avoid allocations, which still gives you a lot of perf gains in the general case, and yes, your parser might not be as fast as it could be, but that's still better than people doing the super naive thing of just basically, you know to arraying their stuff and then working over there over the data structures right.

E

I think that naive things for many scenarios might actually be faster well.

A

Me said, but there's two different things like this throughput for one operation and there's throughput: once you have a million of those operations right and so if locations is one of your biggest problems, then to arraying clearly is not better than avoiding them right.

E

It has to be measured, but if you, if you have a temporary, if you allocate a temporary array and then do a bunch of very quick operations on that and let the garbage collector eventually reclaim it did depending on your scenario that might actually be faster than doing an expensive cracking open of the sequence. Reader, fair.

A

E

A

It's the old discussion right like gee, see how fast is the right back and I think that's very honest. They've done opposite ends of the spectrum, but yeah and.

E

Honestly, if we get something like escape analysis to, then a lot of the problem goes away anywhere, but what, if we're, if we're marketing sequence reader toward a type of developer, who wants really really high performance, really low allocations? It's the same. It's the same reason that we said, like maybe try, Ritu, isn't the right idea like it's going to encourage you to use patterns that we would. Rather, you not use.

A

Yeah to me, it seems like that seems to be the the primary thing that we need to do is bouncing back to API needs work.

E

Post a comment: so here's another way that I see this like we would. We would never have a utf-8 parser that takes a read-only memory of T, because at that point we're now forcing you to crack open the memory, get this band and then operate on an inner loop, because we would never do that for memory of T. That gives me reservations that we would want to do it for sequence for eternity, because sequence really is backed by memory effectively.

C

Yeah I was thinking of a similar argument.

A

All right, then, I'll I'll, go see, cover and maybe see what the response of this is yeah.

E

I I'm not I'm, not opposed to to. Actually you know doing it if the evidence shows that we should do it, it's just I've. Without that evidence, we're kind of flying blind here, yeah.

A

I mean, like my gut feel, is that it's not that you were wrong. I think that it's just that there is a continuum in the middle all right. If your primary concern is you, you would use allocations and you have a large enough system, you want it to be convenient for people to do so. Then I can see that argument. Fly I mean he basically saying I'm willing to trade some CPU cycles in order to reduce that locations, because I know that in the long run, allocations is more important than absolute throughput right.

E

Sure, but if we, if we start, if we start saying it's a continuum, then we shouldn't have a single type, such as sequence, reader that exists along that continuum. That type should exist at one end or the other of the continuum. Otherwise, it's hard for developers to reason about when they should use it.

A

Sure but I mean I guess all I would say is that if span we have kind of are in this half pregnant set already, because we give we already know, we have basically 12 different types effectively to do the same thing right we mean we have two different kinds of spans. We have three different kinds of memory.

A

We have arrays and they're all more or less do the same thing, they're all a representation of a buffer in in various shapes right, and then we added sequences on top of that, and so in that sense we already have that continuum. Whether we want it or not, and people have to choose the right side for the right job and I.

A

Think the the question here is just: can we make it so that one of those types you can use you know for much longer on the continuum than just on one particular spot on the continuum which makes it easier to write that code? Why didn't I buy that, but yeah I I, don't have strong opinions one way or the other I'm just trying to like quick all right. So then, channel reader see you guys are still above us.

A

Doesn't sound like it down.

C

Here, Kato reader try peak.

A

Here me: yep-yep-yep refer to you, fine you're, walking your dog, probably.

J

C

Was I'm no longer but I am outside so channel reader? Has various methods for reading read a sink? Try read, read all a sink etcetera.

D

C

M

Been a request to peek to see what's there, so it's basically, this is try.

C

C

Say that so does it? Is it going to have a reasonable default, or is it going to throw derive? Types must deal with this return. False, oh right,.

E

Implement the default implementation up there on the screen.

O

It looks like there's no non, try equivalent correct.

E

So the the downside of having the default implementation return know is that if I call try peek and it returns, false is if. Because there was no data or because the reader didn't support it right.

C

So we can support on all the built-in ones, but if someone else had a custom reader, you know hadn't yet overridden this or just never got around to it. Someone calling this doesn't know the difference, as you suddenly, like that's the fundamental downside. Unfortunately, there's nothing on there's no existing API that we can build it in terms of. Can we or should we add a can peek for optional feature pattern we could.

C

That would let someone understand, if can peek, is true and try. Peek returned false. That means it's empty. If can peak returns false then don't call try peak. We could also make the default virtual of try peak throw a can peak was false. Why are you here, but I'll leave that up to you the thing that I struggle with in some of these capability?

C

You know and whatever it guys and is I, don't know like if I'm writing, an algorithm that relies on peak I, don't I, don't know what I do with the can other than like. My component doesn't work when it's not available right, but if you were writing an algorithm that depends on peak, then you would probably, rather that our default implementation was to throw so that you know this is never gonna work, sure no sure, but then yeah.

M

The same problem, if throw was so.

C

Then the call side checks well yeah. Then you can't avoid the exception. So then you just have a more expensive try because you've got a try-catch right.

J

Seems like it's about behaviors, fine! No, because this is an inherently concurrent class, so you would always have to write code. Assuming that peak could return false and just call read: async I guess you.

E

Kind of look, if you, if you were calling try peak too optimistically, see if there was any work in the reader for you to read and it kept returning false, and that means that you might never DQ it.

C

Yeah but I mean it. It also seems like I guess: I have a little bit of trouble understanding what the scenario would be because channel readers in in it's expected to have multiple readers right like it's a concurrent model. Well, the the type allows it assuming you've set it up to allow it, but that doesn't mean you do like you. Can absolutely by construction, only have a single reader right, but then it's yeah, like tribe, try peak is a little I.

C

Guess it's for visitors like it would let you do a visitor pattern with a reader of oh. There was a thing, but it wasn't my kind of thing so I didn't consume it. It's.

E

The same reason you, you peek at a collection, yeah.

C

A

Mean the question really is.

C

That were digging a concurrent collection is again a little like weird.

N

About we do have a Duncan Kirk, you sad I mean others here. Would.

J

Be basically calling wait to read a sink, try, peek and then try read if it's the thing that you want to decay. The thing you want. Yes,.

A

But this still means in between somebody else might have already gotten it right.

J

Yeah yeah, but in the case of a visitor pattern, where you might have you know something like TPL data flow that allows filtering readers to, you might have two readers. Each one expects their own specific type of message. I, don't know why you wouldn't have multiple readers in that case, but I.

C

Can see scenarios where it might happen, yeah or wrap the one reader with a filtering reader? That only gives the right kinds of things to the right, cues yeah you.

E

Know the thing about this API is it's it's the moral equivalent of introducing an is empty property, making it virtual and having it return true by default, which we would clearly never do, but it seems to be the equivalent of that and I'm having trouble figuring out how to make this clearly very useful API work without contorting the API stamps yeah.

C

I mean like cuz in order to handle it the right way for third-party types, any caller to peek would need to current as proposed. We need to back it up with a call to read in case they didn't support it. If we made this instead throw and had can peek, then that at least gives you the. If can peek I can use peek and otherwise I either. Don't support your type or I'm, gonna, use, read and hope. I didn't need to put it back.

E

Would we be willing to make an abstract instead of virtual and just say all derive types no need to implement it like? If do people derive these things in practice? I.

C

Don't know I mean it's relatively new: it's been around for a couple years, I'm sure someone has derived from it. Besides us, but I, don't know, I mean.

A

It seems to be a bit extreme I would because that's kind of an API breaking change, I would probably I can probably convince them. He would do a virtual that always to us, though, because they would only penalize people that call the new API, which is not as bad of a break as just making a new member abstract.

E

A

E

C

Making the member abstract, assuming that the Sardi ship does net standard 200 means if you implemented one is if you implemented one in a library, that's not standard! Oh! You now can't load it on that five, because you could not have overridden the new abstract member.

E

Yeah, so so a virtual method that throws and then a like, you said earlier Jeremy. Why did you call me because can peak returned false yeah.

C

E

Like the only way that we.

C

Can add something lower than because if we had peak and take, we could have written read, but since this is inserting something at the bottom it it feels like. That's the right way to do it so that we're not stuck with double there's everyone in order to be correct needing to do a double call.

C

And you can certainly say that you just don't bother reading the can peak and that you're just gonna assume they support it, and maybe it maybe that'll never bite. You I mean.

A

This this is the we already have analyzer in the queue for like soft abstract, where it's like. You really want to override this thing.

C

You've mentioned it multiple times and I think as soon as you write up, a draft will greenlight it, but that.

J

Nobody's pursued it I mean I, guess we could make it through if I there probably aren't people who implemented this yet so it would. It probably be safe and not cause issues. Imagine right now.

A

Yeah I mean to me the can one I would only do the can one if you truly believe this is an optional feature right on streams and makes sense right, because you basically say like you know: a stream can technically be only writable and unreadable right so that that pattern makes sense. You will only do a can API here, because you have a throw implementation, but realistically we don't want anybody to throw right like we want everybody to support, try peek, so this would only be a yeah.

A

Unfortunately, you loaded something that is like from three versions old, and so the person didn't override it. But the expectation really is that people should support, try, peek and so.

B

E

You know, and so.

A

We were just supporter than the built-in types, and that's probably already handing 99% of the of the usage of this type and then for the more advanced people is like yeah. You just have an analyzer or something else that also you would want to override this. So should we go back and remove can count can't count.

A

Where's, Kent County would.

A

C

We had a yeah count and can't count is.

A

This, but is this something where we said like you know, maybe you can't count because there's infinite, or did we say all because it's not overridden.

C

Count is how many are I mean: I, guess, I, don't know if we even talked about the concept and a reader that you can always read from dev.

A

Random, no because I think that to me that's the difference right because you would say that it's a concept that may not be present, even if I'm compiling against the latest version right versus the other one, is really more of a versioning problem where the earlier version just couldn't give you the API to overwrite to begin with, but like because.

E

I mean because if it is sorry.

A

Yeah, because if we add these can things, he basically asked everybody to call this API. It's now is that and then you have to have the elds case and I'm, not even sure what you would put in the else case here. Besides throwing right, it's like well, my my Twitter or my consumer. The reader doesn't work on a second peak. So tough luck right.

C

I mean either yet it's the can. Essentially what I look at here is four four can peak is if you, if you needed, peaking because you're doing something like the visitor pattern, its when you got the original reader. If it said it can't peak you're, gonna wrap it in a I. Remember one item reader and now can peak is true, and you never read it. You never call it anywhere else in your code. It was just an uptake problem.

C

The but I can imagine, especially because of versioning that you are a reader or you have a reader that is reading from some other reader and it doesn't have peak.

C

So you can't have peak unless you do the the hold back of one yourself and, and maybe there are algorithms that you can only do destructive reads, and so maybe there would be readers since we didn't declare it as something that was part of the original definition it that might make sense to them of not being P Keable, but I agree it's less likely than I'm I. Don't want to agree that you can count me yeah.

E

I mean if we have uncountable readers, if we have infinite readers and I can think of a scenario where you can't peek, but you can read dev random is one that I mentioned earlier. Didn't.

C

E

Didn't know didn't one of you earlier have the brilliant idea of making the ref assembly have an abstract member, even though the real implementation is virtual, not abstract, yeah.

C

That was my suggestion of how we can introduce new ab new abstract methods. I thinking about cameras off so you didn't see my air quotes going forward, so the ref would say it's virtual and then you know an attribute must override that. Then we make the ref generator turn must override into abstract and now, if you're, building a type against net five, it would tell you hey, you have to override this and then, if you're building it against that standard. Oh it wouldn't see it. So you it wouldn't tell you.

J

But I like that idea. That sounds like a larger conversation. Yeah.

C

That's a pretty that's a math breaking change, a source, breaking change.

C

E

To donate five and then attempt to recompile yeah.

M

E

C

Yeah I hope people do right, yeah right, but whether it's that or an analyzer that you know produces warnings and you're like most build environments that I've ever seen and worn as errors turned on much to jared chagrin that well three warnings fair enough. But.

F

A

Mean I'm I think I'm. Okay with that, because I mean even if you had a soft or you know, abstract analyzed I, we will probably have it on by default and probably give you a warning, at least by default right, yeah.

C

But developer has the option of turning it off. They don't ask you to turn off, but.

A

Hey I guess I'm sort of okay with it, because once breaking changes to me are nowhere near as bad as binary breaking changes.

M

Battery your your code base cannot.

C

Work unless you do more work to add additional methods to your types like.

M

Imagine we added a new soft abstract earth because we're calling it the stream how many code bases could simply not be done it by without a crap ton of work, but.

E

I I think we would never do this on and on a base class as widely uses stream like we don't we don't think a lot of people.

C

Would love to because then it would let them actually have the array of things implemented in terms of the span things instead of the other way around right. We want to be able to encourage people to add their overrides on stream, so I'm not sure why we would use a different solution there. Yeah, yes,.

A

C

Sense, yeah there's something to me.

A

It's kind of less concerning than the binary breaking changes, so just making abstract to me would be a no-go, but the you know, reference assembly, abstracted, implementation being virtual, seems doable, but yeah I would agree you that it will be bad for widely used types. So maybe we just have only the soft obsolete with an analyzer that gives you a warning and then call it a day, because that we can use everywhere and then good and.

C

Then you know if we were actually defining this feature there, there would be possibly a difference between hey I'm, a default, throw implementation. You really need to override this versus like what we did on stream, which is I'm a horribly inefficient implementation. Please override me and make me better and because one of them is I won't work and the other one is I will work, but you may not like it, but.

C

Yeah, so the fact that I came up with a scenario of the like that you could wrap it with a I hold one, or do one value hold back on read to implement. Try peek whether wrapper means I think that there's value in exposing the can peek and then at that point it should be virtual, throw but otherwise it's fine, then I will say in general.

B

C

Discouraged public virtual but since you have no I assume, there's no state or argument validation here, so it's probably fine, you know, can you click the link at the top I? Don't remember what the scenario was that was driving this.

C

Yeah all right, this is the basically the multiple visitors thing, I.

E

Think everyone's in agreement that the API is definitely valuable. I I haven't heard any pushback against the API in principle.

D

Trying to read and write to a message queue using channel.

C

D

Like message queues, you kind of wanted the same functionality where you don't pull the message from the queue until you actually are completed. Processing it that way. If you're pure service dies, it's still in the queue, and somebody else can pick it out.

D

B

D

Try peek in that scenario, it's kind of what blocked me there.

C

Are variations on that too, like you want to, you want a peek at the and what's there to get some information about it, do some pre-processing and then pull it out. I guess.

C

Yeah because I.

E

C

Guess, on Channel Reader I mean just making sure hear, read is or the the things that do the reading are our public virtual write, abstract, sorry, they're, but they're they're the place. The thing that you write in overwrite for the word that goes in front of it is public and not protected, correct, okay, because if, if it, if all, if the virtuals were protected, we could make the base class do the I will hold one value for you. But since we've exposed the virtual and it's directly, we can't we could.

C

It would make those base implementations way more expensive, but you wouldn't be able to get in front of the the base class couldn't get in front of try read to say that it needs to return the held back value first, if tribe.

B

Peak was called.

C

And did a hold back there did, but even if it was structured, the way that you're saying it would be a lot more expensive, because every call to read would also have to consult that special hole bath before delegating to the derived types, implementation right, basically they're having two layers of all the functionality we could. If, if we could have a different discussion, if all the virtual / abstract members were protected, sure so.

A

What we called a Ken Pico can try, Peggy I would probably propose, can peek, because we might want to have a pin method at some point right. Yeah.

C

Let's I can't peek I mean in theory we could add a peek a sink, and it's basic that we once we have tri peak a peak, a sink I believe I have to think through it more, but I believe it could be virtual implemented in terms of wait to read a sink and try peek.

E

They think that's correct. A philosophical question on this type. Is this type generally meant for four sources to send data to this thing and then to basically push a notification to any listener. I know have data for you or our listeners, intended to just effectively pull it as they have availability or both both.

C

It's really just a general handoff of data from one party to another or from multiple parties, multiple parties or any variations on those and the consumption pattern might be a single reader who just wants to consume things as fast as they come in, or multiple readers that are paralyzing over input produced by any number of producers, and so they're all just sort of pulling that up. The next thing that's available. It could be someone who's just coming along and polling and grabbing everything.

C

That's there when it, you know when they have an opportunity, so it it's all variations on that.

C

Why do you ask that I.

E

I was trying to figure out if, if this really made sense in a in a greater context of like a has data event or something where the event gets triggered, and you just get the tea and that doesn't DQ it, it doesn't read it. It just says: here's some piece of data and that event would also be the equivalent of the peak.

E

Basically just designing on the fly. I might not I'm, not saying it's a good idea to do that. I'm just wondering if such a system can be built on top of all of us. Oh I will.

C

Say I I believe I have to double check. I believe I verified that this can be implemented. We have, we have basically three implementations of channel reader internally right now and I'm, pretty sure I verified that functionally convinced in terms for all of them. If there is one for which adds non-trivial excuse me, expense. It's trivial to implement, but concurrent cute right peak has a fair bit of overhead and the main channel reader is influences in terms of concurrent Q. So this stripe, equal inherits at right needs expense.

J

Okay, implementing try, peek I, believe it's the first API we introduced that sort of enforces ordering, whereas if you call try PQ then require try, read to pull out that same result, yeah.

E

That's why it would be we.

J

Have a random generator with try peek yeah is there a I could see implementations of queuing logic that could be made slightly more efficient if, if they didn't have to respect ordering so we're kind of boxing ourselves in here a little bit well,.

C

They would just say can't because false.

J

Yeah I suppose soon well,.

C

Once we have the broadcast, if I don't support peeking its I didn't support peeking, don't do it.

J

D

Just because, if you've got something back from try peek, somebody else might have read it right afterwards and pulled it out of the channel yeah.

J

But expectation be if you have a single reader and it calls tri peak if it calls tri read it better, get the same value that it peaked.

D

For a single reader, okay.

D

C

Think we should think we should hold off on shipping this and done it by, even though we could probably whip it out pretty quickly. I I think we probably want more feedback on it that it won't get in this cycle.

A

That's fair I mean so that means, if you gently, are okay with the API. We just don't want to put it in 5/4 I.

C

Think so I mean something could convince me otherwise, but it feels like we believe, there's value, but we also have some insurance about it and it's probably not. If we were to screw it up, it could have negative impact, so we should probably hold off until we have more previews, more people trying it out, etc, yeah, basically getting in like as soon as we switch it over two dozen six.

A

N

A

Right then, let's even like this, it's already 1207 and it's calling today.

A

All righty, then everybody have a great weekend, stay safe and enjoy your time.

G

Okay, bye, bye.

G