.Net Foundation Design Reviews, 14 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: .NET Design Review: char8

Description

https://github.com/dotnet/corefx/issues/34094

A

All righty, so so before we go into the Jason thing, I'm probably going to the utf-8 stuff, but I should say we should talk about the Jason Becker's thing. So this is a Josh Hansen and me were like spitballing like different naming conventions. So there's two things I want to get out of this. One of them is I, don't want to give a super attractive name to it, because I think we will eventually we might eventually shape an actual route for this thing and I really don't think they should be in the same package.

A

That seems very weird so effectively. We want to have a name that conveys that it's a source package, but you know what you're getting yourself into when you're consuming it, and the other thing is that it should be collide. The movie name that we wallet actually ship are there. So why? Why don't? We think that or.

B

Packages that don't have exchanged eyes, shipping sources is not the right thing to do, because if there are exchange types, it's a different ballgame. If there are no exchange types, if it's a library, I think many people would actually prefer sources, especially with this nice trick, where you gonna make all the types in there not so easy P, then I don't have dependencies, I compile it with my sent, the customers can choose whatever other version they want. If they have an app I mean it sounds many, it's actually quite nice. If.

C

I exchange say something that would show you publicly PR X. So currently there are net types like G object. Yes,.

D

C

We are talking about shipping, the reader and writer, but currently we're talking about shipping. The entire.

A

Document could.

C

Be considered an exchange type, but.

A

Let's, let's just assume for the sake of argument, the only thing we would ship is just the reader writer right. The problem that you still have is, you still have blowed and you still see sharp only.

A

Which basically means I think it doesn't work for everybody, I.

B

Agree I'm not saying that we should intercept other.

A

Decision I will.

B

Never switch to.

A

The other device it would not remove the source package, maybe we're just not promoted I mean it will always be there, but I think the problem is given that source and binary and different axes. We cannot put the same package because you have no way of choosing, so they have to be different packages. Yes, so that's why I'm saying from my point of view, whether we do it long term or not, is almost your mother when you just need to make it a different name.

B

I agree, but so that one I agree: okay, what I kind of I'm not sure about is trying to find it and make it temporary and whatnot I. Think it's a very nice long-term solution. If you should just append that sources and basically when you search or for the component, you can either get the sources or the deal yeah.

A

I mean so the one polymer sauce packages and amulets, if you haven't really done it a lot right so like I, don't know what about what all the shortcomings are.

A

My concern is that we will find more issues and then it's like well. How do we take this ever back way? I mean so the only two things that I know of is just loading like language specific, that's why they are potentially other issues that get mean effectively. It only works for cases where you literally make the type internally as soon as somebody makes something called X it's falling apart fast.

A

So following will you suggest crystal, you will basically have like that in our system, but X dot, Jason, taught resources.

B

Having said that, I don't think that you know it's a very valid point: I, don't only do it for things that are basically libraries and no strange types, I mean you know technique you can exchange anything but like in practice. Yeah. You know JSON the reader and the writer. If data they shouldn't be or won't.

A

Be exchanges, yeah, I, think just for one of you, like I, think the video writer is not enough. We have to get me I mean for 300. In order to make this actually fly for partners. You will be sucked into this your liner. Why? Wouldn't you can probably get away with most of it not being exchanged, but I mean that you look at Jason I'm in an ace net, for example, this thing that quick because you'd see Eliezer well, that's gonna be probably get the I.

B

Mean we could do something like say if you want to use it as an exchange, type use the DLL, but you feel you're not planning to do it just use the source package, what I meant by it? There are some things that are like fundamentally exchange that there's no reason to you know, span or memory, there's no reason to make it internal and not have it as an exchange type like it's completely non scenario.

B

E

On something like the reader, like you know, we talked before about I think that's more back cut yeah, so decimal.

E

So a big integer like we didn't, put get big integer on it and we're like it's an easy extension I think for someone to write well with the extension that that means it's down there change time. So every time that we've ever written is an exchange type to an extension method. Kennedy.

B

Said like every public type could be an approach time. The question is I mean it's it. The primaries. Are you already heard.

A

Of it, for example, like in Nixon range, there's absolutely no point in different sources: yeah.

B

A

This time makes absolutely no sense unless anything, it's an exchange versus this one is like yeah. You can put anything in your signatures, but are you likely I didn't I? Think we look at Jason I? Think the number one thing that is very likely is the JSON setting staying away because it's places and I'm one thing you have to agree on with everybody.

A

You can probably get away with putting this in a dictionary of string, object or something, but like I mean at that point. You really you really butchering the feature in order to shape sources rather than doing the right thing like looking.

C

At what you have to date, she rather settings J object, I'm, J token and things even just a reader writer text. Writer we are, and so wrapping publicly again service is our G game.

A

So if I had to guess so here's what I think will happen, we have to ship something for first party and early adopters to allow them to do to use JSON. We will eventually be forced to ship this thing as new I think that we can. We can try to delay this because I mean if you look at how we can ship moves, there's only two two ways to do it. In my opinion, in the same way, it's the biggest one hoops is the evolution right.

A

If we, if you don't journey, if you actually believe it's frozen, which is basically what span was it's not horrible? The only complication with that is like well, you need to know pretty much up front where the times we live, because you also don't wanna with the time for words right, so you need to basically know what the exact shape of the feature is and never touch it again, or the thing is purely for Nixon range.

A

We know all always have a try forwarding issue because it has to be in column, but we can probably lock down the service area, but it quickly this one I'm not sure how soon we can lock down the surface area, so I would save to me. The source package is more like this bridging thing. Until we have a proper loop once you have a proper. Why would you want use? You do not cut dependencies on the package that can cause problems? It's always I.

B

Think, though, that the.

A

Yeah, the the fundamental follow who that is, is that you basically force you position to make a choice and I think the only other downside with the dependency is that it really only is a problem for you photo in camera, but I end up back from published ships in inbox at some point, so you dependency effect with disappears but effective. The platform already has it, which is the other reason why I don't think it's horrible for a bit of the package. So that being said, I think something like system collections unit will work.

F

A

But it's it's a good example of this effectively improve because I mean even even on the neck or there's a one-to-one mapping between the implementation assembly and the fabric assembly right for Nixon range. It's. It will never be like that, because these types have to live and call it with chemical quality. So you have to basically ship something else, I just type forward. At that point you have, this lived way became. You can never add it. Yes to that and the problem of JSON is that we might get away with that.

A

But the problem is always when you've all the I suppose you really have to be sure what everything like you need to know. The lender like if you ever split this assembly into two, for example your school yeah. Again, that's the kind of a thing that has broken us in the past left and right, and that's why it would be kind of voids your shipment move until we know what it what the API services.

A

So, for example, we know Rider and civilization might end up with different assemblies, might not be right, I don't know so in an effort to dime box back to the name, yeah, so I think system text RJ single sources I. It.

C

Seems I know as a first source packet. It seems weaker too much risk how much to put in system in space and focus on that and actually up. Next with my news right and.

F

I agree with that. So until we have some experience, publishing source packages me like that may be a good.

E

Long-Term plan yeah, if we start doing it, but for the first one that we're really just using it as a handoff to to first party and select third party Microsoft compatibility, seems like a good prefix to get it out of the way and the Christoph's point of he would want to take the source package and never reference the dll. Once we're happy with things, and we understand our flow of publishing sources and we understand the consequences of publishing sources. We.

F

Can always relist it as system detective, Jason sources there so.

A

My proposal, I mean I, think a vision who pulls Microsoft of comfortability, but we don't really have that thing yet. The other thing we could be doing because we've done this in the past, is Microsoft or PCL. Don't JSON that sources like it's a horrible name but ok, I compatibility, I actually so.

B

First, my care level about the name is super low. Was it kind of does AmeriCorps acting? But the thing is that compatibility is weird like it's.

A

B

A

Is to convey down level, but it's similar to the AC targeting packet. We basically gave you something for for all its dislike. Here's some.

B

Yeah, meaning so you mean terrible, it should be Microsoft portable. That sources.

A

So how do you feel about Microsoft OBC other place in the sources? That's the I mean anything outside of Mexico. Well, we have ship packages before was that they were basically like that. So, like all the all the dollar level groups we have shipped before like we did this for the get a waiter stuff for tasks or may have done it before, but does it actually mean anything to anybody know that no in it, but that's, but that's the thing. That's even it offer sauce, because it's a little rainy so.

B

Why not just basically change their personal to Microsoft, but then keep the rest? What does the thing is the same.

A

And source sources, so the two things we just service, Microsoft or credit compatibility dog chasing those sources, the other one I, just Windows Microsoft the PCL projects in the sources.

B

Why not Microsoft text JSON sources.

C

This repair system, it doesn't add sources to.

A

My website, but you don't want to do nation to be system like yourself at home right. You don't want what we've done with the microcell your name's mr. mica said I would hold it in the actual package.

B

Yeah, probably.

A

That it won't be easy to transition like the rapid system. I can live with that, so the names based differs from the packaging. It does that just ignore it yeah, because you're a minute so.

G

Lose connection somewhere.

A

That this name search will be different form from the beginning yeah. It will just be in the package name. This is what services, the text updates minutes or so I thought we wanted to.

F

Change the name space in the sources so that it wouldn't conflict if we wouldn't when we have it as a public in that crap. What would you do there? Because if you include the sources and you reference that core app, what would you do then?.

A

Like that seems like at that point you, what do? What you really want to do is any multi targeting things say: if target from Annette core I have dog reference to package, that's for the main space remain the same as where it was with my name in it. If it wasn't a by the back way, because.

C

I mean that to get goes.

A

Back to McChrystal said I, don't if you, if you let's say your multi target between for firmer connect web three, the easiest thing for you is to have the same source code and just say when you're targeting, when you can call in for full framework, add the package dependency. Then you don't have to touch any of this was false because of the same with yes, okay. That seems like a cleaner approach. Rather than forcing everybody to yeah like they said it's just easier.

B

To transition between with you, another thing we could do is basically just don't like you know: Microsoft sources that I don't know that then use this answer.

A

We have the other reason why I don't like that, it's like it seems very richest sources at the top level team, because it's not like technology. At that point, it seems like an orthogonal decision point. So if you have Microsoft sources talk JSON is like what is the? What does that mean?

A

B

Microsoft sources, which system and you get to the DLL version of psychology.

A

Yeah but there's like my system, don't source this you get.

E

Any other point: if the long-term plan of once we figured out what it means to be publishing source packages- and you know, maintenance on them, etc is to go to the name of the package that sources then having sources be at the end and in our interim solution. That seems better.

E

Alright, so Microsoft PCL, Jason sources, Microsoft PCL, is not an organized.

F

Prefix so don't mess up, don't you.

A

Yeah I mean it's not even Arabic. That's the ugliest name. Waking him up over it kind of said it's fax. What they do like I mean the thing with Microsoft. Obviously, other people know what it is. It's basically don't let a crap like it's effectively what it has meant in the past, where we can quickly reserve the prefix that level.

C

You don't have that term. Today, I mean you.

A

A

Over the top, because we used it to mean that portable is it's like, it sounds like TCL's, which is really not about I.

C

Mean meter for to system memory as a portable containing portable span.

C

And that's English words: people can read rather than this year.

A

Yeah I think I think you have enough names and all listed up there that you can't call the closest over. You know like. Let me just summarize the answers that we have and then you can just make a quick decision. Don't have to do this right now. Wait, I, think Magnus of the texture Jason. Those sources was another bad idea all right. So then, let's move.

E

On to trade, what can you actually refresh this real fast I made some very brief changes, while you were talking so I want to start off with motivating the the chari type. I I won't bore you by make you read that whole or about the beginning. We did to us yes, I will I will read it to you and Lowell you all asleep. In this document, I use the name you tfh are just or just to be more explicit, but you know charlie whatever the name doesn't really matter to me.

E

um So the the purpose of the type is considerate as equivalent to how char represents a piece of you T of 16 code unit char is basically similar to UN to 16, but a text turbulent char 8 is similar to you and 8, but a text code unit for utf-8. Specifically, the reason for this is just like you don't want to treat arbitrary binary data as utf-16. You don't necessarily want to treat arbitrary binary data as utf-8, and, let me let me put an asterisk on that statement.

E

When I say you don't want to I mean the data that's coming in from the wire you're, you have a web server or something like that that that data is binary right. The client has told you you should interpret this data as utf-8.

E

However, until you actually make the determination that the data is utf-8, it's just binary data that you can't really do much with the purpose of the char. A beauty of a char data type is to say, if you have a sequence of these things, the sequence represents text data like data that I actually know to be like real honest-to-god utf-8. The reason that this matters is and I speak at length about this in this document and I won't I'll. Just briefly summarize here, you don't necessarily want to encourage developers to create applications.

E

Web servers where they read in some read-only stand a byte from the network. They perform some manipulation on it and then they throw that to a back-end server, a back-end database, something like that, which now looks at that exact, same read-only santa bite and then takes different action. Based on that, an example of that is, uh if I tell you, if I tell you here's, my username and I have submitted this in utf-8, and you call things like to lower odds great, like your.

E

Your application believes that the lowercase version of that of that blob is something, but if I have crafted that blob in such a way that it's not actually valid utf-8 at the time that you pass it back to the database layer, the database layer might normalize it in a way that your application is not expected right and at that point, your database and your application they'll disagree on what the actual text sets, which leads to your application having undefined behavior the the motivation for read-only span of char aids, as distinct from read-only satellite, is to get developers to to not treat arbitrary binary data as utf-8.

E

That's sound. If we did, if we did want to say like screw and read-only span, a byte is or a representation of utf-8 data like I I can go over at the end of this. How that might change the entire API surface? But that's the general motivation.

E

If you read through the document in your spare time, you'll also see how this compares to what Swift does what go. Does I won't really go into that right now. Can we can you just look for the word struct in the document I mean to me. It's also similar to in C one more charge. Another charge char star is both have some amount of binary data and I have some amount of text yes, and that any questions I answer on Stack Overflow that are in C or invariably explaining.

D

E

Someone that char, star and char star are actually different types, yes, and in in see the way that the way the developers generally work around, that NC is it's very common to use, Hungarian notation, not just in parameter names but also in locals, and that's a lemma C developers keep track of whether this char store happens to be binary versus types data, but that's not a very dotnet way of doing things. We tend not to require our developers to use Hungarian. You make this beam with it.

E

It can so anyway, what you see on the screen right here is well very large where you get cold sex. Yes! Well you! What you see on the screen right here is the actual definition of the UT f HR type, this. This type is 8 bits in size, just like a byte. In fact, it's backed by a Biden, it wraps it might effectively, and there are it.

E

It's basically think of it like a new integral type. There are conversion operators between this and all the other integer types, including by char and other things. The conversion operators are checked in this proposal. I'm I'm open to alternative ideas here. That means that if you have a int which is negative one and you try casting it to beauty of HR, you actually get an overflow exception. If you have an intimate snake, ative one, you cast it to a bite and then you cast it to UT, f HR.

E

You get I found where's the dummy field and that's.

A

A typo that should be like well, the way this works is generated by reference in San, Joaquin yeah. They can type doesn't matter, it's race, it just the primitive know. The time doesn't it occur.

E

D

Device, it doesn't work.

E

A

The reference assembly.

E

It doesn't matter I believe the sea-shore compiler requires a reference to someone to be correct for size up to work. You know no okay, little little I'd say.

A

No sometimes thing it's, we wouldn't know that globe Anderson over in the same is, is: do you have any field yeah and then do you use generic? They give you a virgin. If you have a generic, do you use them and fuse to make cyclic layout checks or blah blah? They did as a bunch of rules we just applied to so right on those things and that's whether its private dummies about.

D

A

Assignment yeah not about yeah, there's thickness, and it was basically everything field.

E

But if a little size of can read this, then it may be yeah as long as long as we make little size of work like it doesn't and really matter here. So, as you can see from the api's like most of the api's are actually just conversion operators.

E

There are also things like equality operators like the typical stuff that you would expect the strong to have right. If you scroll down a little bit just pass, the type definition you'll actually see there examples of what that would look like in this case. You can see that things, like addition, subtraction just work, and the reason for that is just like with bite, char and other times when you add, there's an implicit widening to int.

E

So if you have UT, f, HR plus 10, well, you get an int as a result that exact that's the exact that matches the behavior that you get today. If you were to say like some bite, plus 10 or some char cost, an equality works. As expected, you can compare a UTF, a charm against event, just like how you can compare a bite or a chard incident, and again that's due to the fact that there's.

G

An implicit whitening operation, so what's the purpose of doing math on a child like Annie, I'm kind of think when I've seen it done on chow, and it's generally like when you're like checking for another casing and do the 8 is going well, the other other rita. I yeah, I so you're parsing an integer, you read the character and you subtract. I come the separation between upper and lower case yeah, you guys interacting yeah, I'm subtracting the value of charge, 0 either that or or yeah I mean.

G

What are the reasons you have for doing math, but.

E

That's it. That's mainly that's.

G

Mainly the reason currently.

E

You'll to go often find that arithmetic operations aren't that common, but comparison operations are right, so I want to say like if my char is less is less than or equal to like Kaposi right, that's actually pretty lonely, but this this shows what some of those would look like in practice. You'll see the only one that actually fails is the one that's second to the bottom, which is what I mentioned earlier, where you have a negative number that we try casting to aut fhr, because that's checked it leads to no before in central London.

E

Everything else will work as expected: oh okay! So now the question is like, so how? How does this actually relate to bite like this? It's all well and good that I might have a sequence of UT fhr inside my application, but ultimately, like I, want to do something like write this out to a file write this to the wire or something what I had that I know we're going to be jumping around the document.

G

Of it, sorry so can I try think you said the stuff. Is it not the same interview for there? Just advise you aware the Detroit very few before we plant in the US and.

E

Even yeah, so the the difference is on a plumbing store with systems our system, doc char, is a utf-16 code unit. What that means is it is a 16-bit integer where a sequence of such integers represents, hopefully well-formed utf-16 data. The code unit by itself may or may not represent like a single letter, so I can have a system. Char represent the letter, the character E or, if I want to represent an emoji. For instance, I require two system: charminar UT, f HR is similar.

E

It's a it's an 8-bit code unit, where a sequence of them represents utf-8 text. Any arbitrary 8-bit weight is.

G

E

Will sequence not necessarily it's? It is any. It is any 8-bit utf-8 code unit. However, you see if they turn by itself is not checked for validity right, for example, like the byte is never ever ever valid anywhere in utf-8 ever I'm. Sorry sweetie. It's an 8-bit white chunk of a byte array that.

G

You are claiming, is you can't fake, but.

E

Anyway, not for you, so you can.

G

E

Can absolutely count FM to you thr, but if you ever try throwing them through an API that expects utf-8 it's gonna blow up, whereas rune is what the whole room is. A room is the entire scalar value that is represented by one or more code units in sequence. So, for example, an emotion character.

E

Generally speaking, is one room. You know mate, which is probably going to be two utf-16 code units yeah, poor or usually.

D

Have no power equivalent.

E

In the South sixteen whoa correct a room. A room is generally thought of as a sequence of code units where the sequence length might be one write on a displayed character like a good, is his influence of one or more rooms right, since it's basically collections on HTML, but again we have no equivalent in the utf-8 utf-16 world.

E

Well, I wouldn't say that right, like they utf-16 and utf-32 a sequence of rooms. How do I encode them in binary format? Right? Okay, so you can. You can always I can almost take a utf-16 sequence and turn it into a rune sequence right it always taking utf-8 sequence.

D

E

Similarly, 32, you know that was in current a the.net definition of a room is a Unicode scalar, which is a subset of a utf-32 code. Unit like it is, it is impossible to make a system dot text on room that wraps a value negative one, the constructor for Vincent, whereas a utf-32 code unit is basically just an int. That's by the way. That's the go definition of the wind, the go definition of arenas, a literally a 32-bit integer I am type def desn't.

E

So can you look for the word next and then we'll go to the second match because it gets them weaker? No, so this is. This is showing what I meant earlier about uh about, like I, have a suppose at utf-8 data and I want to do something like right into the stream right a to to disks or something like that. The idea is, we would have a bunch of useful extension methods, hanging off of read-only span, WTF HR, just like how today we have a bunch of useful extensions hanging off of read-only span of char.

E

The extension method I'm showing here is as bytes, which basically says if I have a read-only span of UTF HR, but immediately projected as a read on expanded bite. It doesn't perform validation, it doesn't. It doesn't need to because of as soon as the infant's well-formed, it's basically the equivalent of memory marshal cast, but without requiring you to call into memory marshal, because memory marshal is what we consider to be a danger. As they've done. We actually have that as bytes expansion method right, because.

D

It operates on that name, arbitrary G,.

E

No, we ended up not making them an extension method. We ended up making that a regular method, a regular static method in memory Marshall and the reason the reason that we never made it an extension method is because it allows you to inspect private members of of T, but this this would be an extension method that is absolutely equivalent to memory Marshall as lights.

E

We would just expose it as an extension method because it's convenient and safe what this, what this code sample is showing is that if you were to call as bytes like you're, getting back a stand that literally just points back to the original data, if you, if you call fixed on each of them and compare the pointers they're the same, the other thing that this are showing is because we would actually have a type UTF HR we can enlighten certain AP is just like. We do for char.

E

Today, for example, read only span of utf-8 char to string. We can actually make it return, a string that represents that data as utf-16 just like how today read-only stand up charge ought to string. Actually string applies. The data compare that, with read-only span of light dot to string, which literally just returns the text read-only span of bytes of length and the same for Costas mean there is no cast to string here. There's there's no cast between read-only span of charm to string there isn't implicit counts.

E

The other way if we had a utf-8 string type that a parallel cast to exist. Okay, but I'm, not I'm, considering in these API proposals that such a type does exist, but I'm not really going over. What that I could look like as part of these proposals all right by the way, because.

B

Of this issue that if I just mentioned I'm literally removing read on this panel bite right now as a removing support for utf-8, because it's impossible to debug the code and developers on Monday they're, actually developers starting at menus kind of a pipeline framework that then I create it. It still has I basically wanted to make it utf-8 enabled so I used read on his panel bite but since impossible to debug I, already kind of I.

B

Was I D bugging one thing with a developer on our team and she was so confused that like without this feature? Basically the only way to debug is when you have a problem, you go and like change your code and create this. You know debug string, yeah,.

D

B

You can bunch of methods and they take three dummies but of bite. Yeah and I know that it's utf-8 and now inside the method. Something doesn't work and I don't want to know. What's inside and basically two strings and yeah.

C

Exactly you can.

B

D

Just to make sure like I mean you thr, but there are other ways to solve that. You have a two string for read-only span and by assuming it's utf-8, no.

B

No, but then debugger doesn't show it so I have to call this man, you don't have to call yeah I, basically have to call it manually, so I have to now change temporarily. Why I'm debugging this method, I have to change the source code in it, and then I have to remember to remove this to utf-8 is 16. Well, okay,.

A

B

Tricks we could.

A

Be doing is they'd be just a hand span. The two string method gives you calculate back suddenly. It looks like a hex view. We have to buy some left inside the text in the right hand, side, and you can at least look at the data.

E

But historically, that's been very difficult to debug, because I've been trying to debug the utf-8 logic while having a hex key. What's up and it's very painful exactly the same correctly, why is the execution to do things right? Well, it's if I have.

A

To use just going to be HAP's right now, I need to go, consult and ask you know: I can actually execute, would show you the Vice and the left-hand side of the text in the right-hand side and that's what you would return the string that formative effect is also.

E

So a little waste and avoid well, you would base it.

A

Soon as you, if you have a string yeah but yogi of the text, viewer, like maybe take on a user's I, need format, it nicely.

D

It's not like this.

A

It's not as convenient as having a tie but I think what it so I could make that fundamentally better than what it is today.

A

So what your serger position, my god well I, mean it hecqs use a hex view right so likely it will work. The only problem is that it's not as compact, so if you have like five locals you, it doesn't doesn't render nicely and then the other problem that you still have is that I think montiel, bigger problem is just the time so I'd like if you look at the method signatures, if you say I, take we'd only spend off char I can kind of have that right on what it is we take.

A

We'd only spend off by that kinda have to know that you might get a baby.

E

I know I: do I I didn't want to go over later what it would look like if we were to use read-only satified everywhere, because it is a viable proposal and languages like ago are considering that, but they aren't running into problems with it and I would I want to share at the end what those problems might look like if we were to do that as well. So.

A

My question is like all whites, but is it already like I, maybe I'm wrong, but like even that we look at a lot of these api's I. Have this impression that we usually only spend on fight already a lot for you to find out that.

E

New, you use it right now in the sense that I have received data directly from the wire or I'm writing directly to the warrior. Okay and I want to do parsing on that data which cheaply as possible without any type of intermediary.

B

Well and one of the sort of things all these methods are try methods or which kind of basically, you can look at it. Well, if it happens to be utf-8 text, I'm gonna pass it if not and I won't and it. Therefore it's a tri methyl and to me it's one about the.

A

Let's say we ship this well yeah. Now you have a gazillion options on what the api's can expose right. I did one can take UK right string, you can take we'd only span of which are aid and like you can we think you only back if we don't.

E

Spam hi, so all those dollars are all those apos have utf-8 in the name, but they're, actually all landscape and will always be asking for all time. The reason that they will always be asking for all time is because fundamentally they're working with not text, fundamentally they're working with things like integral types and they're just saying what is what is the ASCII representation of this integral type like as its traveling across the network yeah? And they starting about imitations and we're talking about what.

B

Like can you use it make sense. Is that effectively? So we already have this situation we'd be in utf-16 world. We basically have the system string and we use the span of charge in two slides, yeah. Let's there is nothing more to it here, but we don't use my own five.

D

I will say we.

B

Don't really use span of bite? Okay, we use it for those. We few methods that, like tripod, basically is the only one. Do we have another well, we have.

H

Each a sparse beauty of a parser and if you have big formatter and.

E

Then the only other place that I can think of that we have. It is the Jason stuff right now, yeah and and so I like. If we add this type, what we do going forward is we say if you're reading utf-8 text you take Chari and if you're reading from the lyre you can take bite and then you should in the in the parameter name saying you can't fail sort. That's how your namesake and I do.

E

I did talk about that briefly at the beginning of the document, even though we didn't go over it here, I basically laid out like if exactly what you said. If, if you operate on bytes assume it's coming from the wire assume that it might not actually be utf-8 and you're required to do validation under the covers. If you take UTF HR, you can assume it's utf-8, because you can assume someone in front of you actually did not chop now.

E

You can't you can't make such an assumption that you can AV if the data is bad, but you can at least be a little fast and loose with how you do things. So when you say validation.

F

If we ship this tight, then.

C

Does that mean obviously only family? Might it yes? Now you're gliding fertilization, all.

E

Gradle was founded by the api's, are already required to do validation. Now, that's the behavior that currently exists today alleviation equal one to make sure that the data is valid. You see a table that every every API we have then uses read on this data byte and treats it as utf-8 already.

G

Today, does validation top I matter long tail of what validation means in this can Shin the tolerate, so you can treat it as I love utf-8, but not completely rely on anything out so me doing any better than Hungarian it's it's not it's a lot of Hungarian type.

E

Thing it's because, if you're, if you're working with discontinuous buffers of data, the super sequence, that's generated from all of these buffers being accountant, ated together, may be valid, even if the individual buffer itself isn't, for instance, if data is coming from the wire- and it does happen to be valid utf-8 but just happened to get split. Yet in the middle of a multi byte sequence, the individual sequence. That you're reading like logically represents part of a valid utf-8 string right, even though that span itself is not valid. To what I say. Valid I mean.

G

Nil forms and what value am I getting from saying or charring is valid like if it's not necessarily actually more valid that it bites. But you.

E

Wouldn't mean expects it to I.

G

Have three bytes remaining markers.

E

In a row yeah, okay, it means you, it means you should never see that, but more importantly, it means that you can pass it to something like your database API, and you should have an expectation of how that API will behave as in every every every component that you pass, that sequence to should behave in a well-defined manner and not cause bugs in your application. So then.

B

You will shown as this method as binds a because I could equivalent as char paints that no validation, basically I, would imagine a method that takes a span of by the runs validation through it and basically interprets it as try right so I. There.

E

Are actually can, can you look for the word text? Writer in the stuff in this document, I mean my my initial problem. That would be encoding, that utf-8 exposes that it is type utf-8 encoding and we compare on that validate which takes in a mountain. There outs a span of takes in a span of banking-house.

F

The span of char, if it's valid yeah, so what sort of.

B

It you think we will be basically gonna, have an idea like that, what I said yeah, but it wouldn't be an extension. Oh okay. Yes,.

E

B

Are we gonna have an idea like yes,.

C

So what would send.

B

C

Actually come back to that in a few minutes, but you were stressed about expectations. If you go use the API to validate the spanner by 200 views that, if you give a chart, then we could guarantee that you give it give a charge value.

H

Yes, it'll be something they won't be so nice.

C

C

Person accepting the UDHR students to validate no.

B

Sir also it's well, the truth is somebody can create an array of Charite put the random values that don't exceed have measured utf-8 X, and we cannot do it.

E

You can do that with an array of char right now and, to be perfectly honest, I'm not really worried about that, because no one does that with char arrays in practice, so I'm not terribly concerned so people doing inventory to rinse larmes. Don't do it yet.

C

Except you give it John, you have this kind of you keep a job. Do they need to manage it, but no, but if they accept it even spend a bite, they don't need about it. Yes, honestly, though,.

A

I think that's the nothing you have to validate. After the time code. Instantiate I mean it's exact same thing you have today. Somebody can call a constructor about a valid data by with reflectional with the fields and drop the object, but you're not protecting yourself against that by such everything.

G

A

Doesn't let you stay.

G

One way in my mind, it adds more value than hard. Aryan. Is that if you are careful in your code or your library's code that, when you translate from readers front of a green inside of how you do validate at that point, if you do that, the compiler will make sure that that nobody is using a con. Validated well I mean that that's the types of impatient. Yet right.

E

Ones guys I'm Hungarian I did yeah yes. So due to your question else in like that, that is I, guess one could make the argument that it is a little bit of an academic distinction, because, if you're doing something like attempts, manipulation of a process, an API, you might already have to interpret the text just as a matter of your API doing its work at that points, you're kind of already doing validation anyway. One example of that was the like.

E

The the JavaScript escaping routines that we were talking about the other day like if we have to determine in some particular room, is in a whitelist. Well, the only way that we can determine that is if we actually decode the sequence, which necessarily involves an elevation anyway. So some AP is some api's do have an academic distinction, but what I?

E

What I was getting up earlier is on say that again, like you're you're, talking to your database through an API like that, any frame of something like that like if that takes a read when we stand a beauty FK chart as a parameter. That API is very clearly saying. I only expect you to give me valid utf-8, maybe I'm not going to do validation before I, actually split it out to the Y or maybe I'm just going to project an end copy and go.

E

But if you give me an L, a utf-8 and it makes its way to the database, you now have undefined behavior and again, if you read through the document, I have links to some interesting CDs resulting from that exact thing. So let me do not even put on that with the option. I.

C

Have a pieasure yeah, except instead of like today, yeah it's dual escape anymore: it just copies the bite that you give me exactly to the line yeah now you had an API, for that is, it is going to give each other that forces a bite one way to base your guideline 2008. That is not adjudicated. The.

E

Read-Only spam of bytes should already be done. What any nothing that I'm proposing here should change. What exists in a peer I should be doing so now. Okay,.

C

That means it all bringing in.

E

Yes, the utf-8, the Jason reader needs to know about it. It's taking grade only span a byte. It needs to make sure it received legal. Yes, so look at those temper there and say.

A

E

If you share this time, you shouldn't have the API there. Might there might be bail initiation already doing so? I, don't I, don't want to say never right like there might still be valid use cases for something like Jason reader, Jason writer.

E

It does make sense to expose to the developer, like I, actually am expected to be writing to a network or reading from a network, and at that point when it might be more convenient to work with bytes and with UCF a charm, so I, don't necessarily want to say that you should always use one versus the other, but it's there.

E

We we want to make sure that developers have an expectation of what kind of data they're getting back. In your particular case, your your utf-8 Jason reader I believe you have a property which is raw value, which is exposed as a read-only span of bite. Correct, like here's, the you boss, stuff, like all from the wire I, haven't done any type of inspection on Miguel. I can't tell you if it's valid, that's totally fine, because you're exposing it as real monsters like.

C

18 is it done because.

E

C

A

Think anything that involves like you know, you know it's a string like the opportune, a yeah I would yeah member. Why would you ever go with only span applied at that point? You would I would hope you want yeah, it's I'm, saying I. Think the API you just talking about you just wouldn't have held like you. You would say right: property ball, except the real news event of the day, video.

C

Marketing is it if you, if you had a bike you next week, whatever Joey give Italian, you don't know, because he was euro.

A

C

Was already validated yeah.

A

This water comes, but there's no guarantee well, no, but I think you're, throwing out the baby with the bathwater right. So you could. We can make the same argument for any type that as a constructor that takes logically in on now element right. Can you can you reflection, inspect the fields in certain from now? Yes, right is now every method on this time required check. The field is not valid for in doing operation. No, yes say if you, if you Sanchez described in the same way, you know it's valid.

A

So if you, if you got new thr by using memory Marshall and go to back, cast, were then shipped even.

B

Are very handled it, so if that was the case, I would agree with you. You can just create an array trade, and this time you know you can you can totally do that, but.

C

B

Can assign 2500 7e not valid utf-8 and.

E

So I can actually answer your question. Awesome! Yes to the question. A the only time where a readable span of you thr may not be valid, just like the only time wearing, real and of char might not be valid, is if it happens to represent chumped data within a larger piece of text. Where that larger piece of text is well-formed, so say that I'm reading, like the entire works of Shakespeare and I'm just reading it in 4k chunks like one of those chumps, might go across a multi per unit sequence and that's okay.

E

But if I'm talking about something like the JSON writer, you're Jason writer expects to take a full string as a parameter. Basically right, like the read only span of char or you, thr that you take is not expected to be a chunk contestants expected to be D text. At that point, you should assemble Animus.

D

To charge yeah, there are valid, I mean I. Could I could cast them into a char and for each of those and come up with something that wasn't a valid current? That's right.

E

Yeah, but in in practice, though again the reason they go to worry. Does people just don't do that in practice.

G

In my mind, like it does mean that as long as you're, careful about the at the boundary, where you're creating these things and then on, the compiler will make sure that you're in a world of safety. But that's but yes, thing is I. Would.

C

Say the dilation of that report span of bite you allocate expanded to give it you have it. You don't know nothing, tears, I, don't nobody can guarantee that about it. Now the.

E

Reason the reason that I don't want to force validation. There is because the idea is, if you're, if you're working with them with safety rails, that the language and the types are giving you like, we should be able to assume that the developer has done the right thing and assuming the developer has done, the right thing allows us to mate, allows us to take shortcuts that maybe we wouldn't have been comfortable taking otherwise, so what I don't understand is that go back to Nix? What's the interest of just talking about.

A

What so, are you saying the only case where you're really in value to a great secret is effective would be when you just break characters apart? Is it the only case if.

E

You have ill if you have a read-only stem of UTF, HR and use the developer, have not done something like manually fudge, witty the underlying data or like sliced it weirdly like that's, then that statement is correct. The only time you should ever have that is, if it represents a chunk, were these super sequences Mike? That's not actually do it. I go to might even like not to kind of target. You use memory, Marshall block house to that point, I see if you run it through the whatever API of dozens of lights.

E

You give me your tournament explanatory. There is. That is the mat so.

A

Let's actually go over this grant to you example, this of it to constructing the Ray yeah. A few of you. Th are young. You fill it with that bath and I fill it with random data. Yeah then I cannot construct like even valid values of you. Th are just get it resulting sequence. A little rough on. You know that with Chara racetrack, exactly so like an X and I think that's.

B

So I have to tell you this whole validation thing to me. It's yes, today, it's very inconvenient for a developer, because I have to use Hungarian notation. Every parameter that takes read-only span of bite, I have to say it's utf-8 and same for locals and fields, and so on and so on. I, don't have the debacle support its elements? There are many benefits, I, don't think. Validation is one of the strongest benefits and the reason for it is that if my life depended on it, I have to validate anyway.

B

My life does not depend on it because, as a side effect, I'm all validating, because I'm, parsing and I expect you know digits and if I find something that it's not a digit I'm, you know like I'm failing the main benefit is it makes programming using this type saying, as opposed to you know very clearly: it's not really about safety at validation. Nobody.

A

Going down the path right so like, if you say, the primary value of the child is to just have a different type so that people can have overloads another thing, then what that also mean that if you do validation, that you also get the benefit by dueling for the.

E

Type conveys the exact same thing that we've created the the notion of the strongly type strain for it tells you.

A

E

Is what kind of data you should expect, and this is what it means so it it removes the utf8 from the parameter names. Well, it says as long as you've gotten one of these. In the same way, the data is probably saying the same way that if you have a read, always ban of charts, they're obably saying that could be insane exactly it's.

B

I don't have to help this company winning I, don't have to call this defensively if my life depended on it, I would be very defensive and I guess what I'm trying to say is it like so.

A

There's some overhead involved, not because we do some validation somewhere.

B

So or someone dancing, the interesting thing is so: yes, we don't have two ways to cast bytes to this thing. One is through memory marshal like has no cost and I don't get anything, and then there's gonna be another one where I don't have to use unsafe, Marshall api's and yes, there's a cost, but because it's not very long, if you don't get that we do some validation and yeah.

A

B

If I notice, the back of a performance probably might just switch to my shadow cast yeah and then I went in the routine when I use, Marshall I'm thinking, is it really utf-8 text, or maybe it could be something else? Yes,.

E

And I: don't I don't want to foreclose the ability for the developer to do that right, like I I, don't want to say you know, validation and conformance is the end-all be-all of everything, because maybe you're reading something from a local file on disk, and maybe you know damn it I know those data stores where they write like I.

E

You shouldn't have to pay a penalty just for just to satisfy our paranoia, but regarding things like overloads and what um what what the actual API conveys to you like take a look at stuff like this is on the screen right now, text writer, so right now, text writer is primarily written in terms of string in char. There are over loads of write that take things like int long and so on. We had already previously approved for meeting a text writer dot, write of room. This is text writer, dot, write of utf-8 a no.

E

You can imagine that if we had a utf-8 strain type, there would also be overloads for that. But let's just consider expanded memory for now.

D

Which is abstract and different text writers will use different things, so this.

B

Actually takes the parameter capacity, encoding and there's.

E

B

Does transcoding on the client depending what you yes Todd, what.

E

What I wish what I was trying to say and what Steve corrected me on is that if you don't pass anything stream, wider assumes encoding, not utf-8, yeah, okay, I'm, writing that down myself, so that I can collect intensity.

D

E

Okay thanks, so the idea here is, if you have a, if you have an abstract text writer, which is probably a screenwriter or similar- that's that's using utf-8 encoding under the covers. Writing utf-8 data to that writer should be free as far as transcoding is concerned. That's why I believe it might make sense to add these overloads onto the text writer base class. Maybe maybe we don't add them on text right or maybe we only put them on screen writer.

E

Like that's, that's a conversation you know we could have, but the idea is if your data is already massaged in such a manner that it's appropriate for just it blasting on to the network. Why would we stop you from doing what.

D

Areas where you landed around this area, you know your asp net and you've got a read-only memory, a byte from the wire. Yes, and now you- and you know- if we have a method like this, presumably because you're gonna write out some data that you got from somewhere else. It's read-only memory, five persons, we're doing that! Writing you thr. What do you think's thoughts are on that so.

E

Reap step so spam, you can bounce back and forth because you can use memory, metal, diecast memory, you're kind of in a hard place. Unfortunately, my thinking on that was, if you have memory, if you have memory of bytes for asp.net they're, actually using a custom text. Writer under the covers so I think they're, actually in an okay position, because they can just cast to their derived. I can go for the average developer election say the average of elver for D for the power developer, who doesn't have their own derive type?

E

That power developer would need to fabricate a memory of bytes which wraps the memory of utf-8 chart, which is possible but ugly to do and.

D

D

E

Believe that can be done in such a manner as not to require allocation that I'd have to get back to you on that. You can do with a matter manager the dispatch through the to ask.

D

Allocation I was thinking yeah.

E

That would be the the easiest way to do it. I think you can get away without it, but I would have to jargon. I mean anyone. One object to hold the reference: it's not it's not order in its its own, and you can't you could also cool those who can you to, but but yeah. So looking at this one, the first thing that I see of if text writer, because text writer does because text writers have multiple one codings yep.

E

The obvious thing on this is the encoding also should get where it takes chars and strings and goes to bytes. It should take utf-8 chars to go to bytes and encoding utf-8 says done, and then that lets the type the default implementation of text writer on this just say, sickness to your encodings magic else.

E

So I I didn't want to add it on to the encoding class, because the encoding class has a lot of overhead and legacy that I, don't think, is appropriate for high-performance applications like had proposed a separate parallel set of api's which operate on UT, fhr, bite and char right. But the problem is, you need to make it fit for the places where they already took it encoding and yes, and you can't ask it: is this encoding being utf-8, you can only ask, is it the shared utf-8 encoding decide.

D

Attorney just understand that you're saying the existing encoding methods work to and from utf-16 and you're saying we would need methods for working to and from you to page at.

E

Least from utf-8- yes, that is, that was a good. So so you get bytes of utf-8 or get bytes that accepts UT fhr. Sorry, we don't fan of you. Th are just like. We have the the one that takes, we don't we span of utf-16 chart and then the the obvious answer is utf-8 encoding says return as yes like I, don't I don't have the api's for that.

E

In this proposal and I know that you've mentioned that I agree, we will have to do something there, but in its all the text writer it gives us the default, but what's the return type of the good bytes method? It's probably spam spam. I guess well, the one that we have that takes a span in is the thick span of chart in is it takes a span of charter or span of byte destination, I mean we can we.

B

Can think about this later, but there's an issue with you know today: good bytes, it returns a new array, its retry. Then it's yours. Yes, if we want to reinterpret cast, then you know, depending on the encoding, you can do it or you cannot do it right.

E

We can do smarter things in other in other cases, but I'd like we would need to solve it for the for the stream writer that took it encoding that wasn't any of our shared encode. Yes, I get it we'll need to make the the new virtual corner, and so that's my the cost of this is. We have to put something on encoding, to be able to directly and and by the way, we're doing I'm.

E

Okay, with that particular scenario, not being highly optimized, because I think it's sufficiently real right, because we would do what we always do in these games that we would override the stuff on most of our times for and and along those lines like all of the API. So by proposing here are very, we optimized for when the sequence is well-formed, but these api's might under the covers, allocate if the sequences ill-formed and then they have to do fix. Ups under the covers again.

E

I haven't optimized for that, because it should never happen in practice, but just in case it does I didn't want these things to blow up anyway, so so yeah, binary writer and text writer are basically like two sides of the same coin: room isn't existing to prove type. It's already in that cork Rio preview. There is an existing API on rune, you scroll down a tiny bit. You know there is an existing API on ring, which is trying code which takes a span of char as destination and Alps.

E

The number of chars Riddler, here's a utf-8 equivalent to encode utf-16. The other method shows call triangle. We could just make this an overload with the same name: yeah I, don't think we'd have any type that would implicitly go to both of them. So we wouldn't add an implicit short hairy as I.

A

Know it goes to here and TV must unhak there, because you would, you would think people can fix anything.

E

It's the making sure that nothing that we have right now would ever make sense. They have gone to both and then we can add this overload. Now we have a rule that I can't ever because we would well.

B

We could so this API is super low level and kind of ugly yeah. Do we need to have it on the room like? Why not put it on the utf-8.

E

That it already matches what's on room, but room hasn't shipped yet right. It's on the cranium. We can rip the existing method out of room and put it on YouTube, 16 time right and I just want cemetery. What okay, what that looks like in practice? I might.

B

Open two because I kind of cannot be like this- is an ID I put somebody who writes these. You know routines that actually people use yes yeah. Then we shall have those on the road, but I get that you know we already added the one forgot.

E

Again, this, it hasn't shifted an ordinal release. Yeah right, like we can't a little char char, moon and you th are all have a relationship when we have to just decide yeah, because oh so memory, extensions of the the api's you see here are basically just clones of the API. Is that already exist as specializations for enormous biochar right I? Don't think it makes sense to go over all of them, but there are some very specific ones. I wanted to call out so I want.

B

To ask Kai a little question yeah, so whatever these methods are, will I be able to get them in the portable system memory market I, don't see! Why not.

E

Though the implementation might be less efficient, oh no, even article table yeah, that's why that I mean part of a medley is we'll see if HR you could I could give you I could give you implementations of these, where the API names themselves might be different. Maybe it's not hanging on memory extensions, but there's nothing prohibiting us from making a portable version for making portable equivalents of these APRs. You might have.

F

B

But just to make sure this is super important. That's what I said: I I spend now a month prototyping something I used redundant span of a byte and because of not having this not having that debugging story and some other games, I'm literally removing it I will hand it back if I have it otherwise who cannot use it. It's unusable read-only span of my it, as that is literally such pain in the part that, like I, don't want to subject any developer so now on.

E

My cell phone in funny, when you mention that, because I mentioned that I wanted to go over later, like what would it mean if we were to use read-only span of bite instead of read-only span of beauty of a char? There are two things that immediately fall out of that with respect to the api's we're looking at right now. The first is that these api's are no longer extension methods.

E

They would just be regular, static methods on, let's just say, the types name, utf-8 right you say: utf-8 dot upper and then you pass in the read-only santa bite that you want to uppercase right. The the second point of that is that's actually kind of hell. Girl works today they have, they don't have a concept of extension methods, or instance, methods hanging off of their slice types. Instead, they just have a class, a namespace.

E

What you will a package named utf-8 and this package exports a bunch of methods to upper to lower level law that take and buy spans engineering. So this is cinematic.

B

To go, it's not very dramatic to correct, correct, oh I would keep that I would keep like some of the ones that make sense as extensions.

G

Right so there.

B

G

The easiest answer is if, if.

E

You know if we're making, if you thr, is a ghost portable right, then all of these things move to a different extension class, which would then ship in the same assembly is a chart which area like utf-8 memory, extensions, something well.

B

I will do what kind of hybrid what anybody was saying. Should they go ta fête died, they would be static method and just keep them as extension and then utf-8 type in special static methods. You can either call them as extensions or edit or the static methods. Later. Okay,.

E

I got I, don't really married to naming or were they same flow I'm spacing the weird one, because if it's an, if we wanted as an extension method on span, then it needs to live in system was his utf-8 utility class. Here he has insistent text. Unicode man, yeah I,.

A

Would not be affected, you think about don't know, but I would say it is anything independent.

E

Of them Noah for the time go so one other thing I wanted to call out here is so I mentioned that these are already an equivalence of. What's on read-only Spanish are two points. The first is look at the contains method, you'll notice that I have extra overloads. That might not make sense at first being extra. You split up he's typing.

E

The scroll up to air contains, as it is so two points contains their. There is an overload that says, like does the span a contained, the span B?

E

Well, there's a tarpaulin or whatever, there's also an overload that takes a char in overloaded, takes a room, and the reason for those overloads is because I think it's going to be more convenient for people to write something like if my utf-8 span dot contains single quote: X single quote: close currents like that's already an idiom the developers are used to I want them to just continue to work here.

E

I, don't necessarily want to extend that to being able to pass in an entire string, because at that point you have stuff like localization to deal with. You have transcoding blah blah blah, but for individual charms and individual rooms. I think it's it's helpful to have those overloads. Why would we have them on contains and it will not have them on index M? It would be before contains index of starts with M's with another thing, I just didn't.

B

Want to pollute us so I would say till we have littered utf-8 string, literals, yeah I think we need to have an overload. The big system stream, the performance of that would probably not be what people would expect. Unfortunately,.

E

But we found it it's kind. It's super country.

B

Okay, so maybe you, if we can, we have the calculator that you mention, like the literal dot utf-8 it's in the.

E

Feature branch, but it hasn't been approved for you as far as I can tell, but.

B

What does it do? It basically will do it would do the slow path, basically just like taking the strain if I am NOT on the newest runtime and- and we were sad that will do what's basically doing, which is. It is a helper method for the compiler that just points to metadata. So you can section.

E

B

E

In in the future branch right now, you can do utf-8 string down literal open closed hello. What works? The jet will recognize a method call and will emit the equivalent of low stored on, and it will pretend that you had written the equivalent of load stored on utf-8 instead of just a regular load. Let's the return to work is the return type duty of a true. It's part of the utf-8.

A

Strains the legal it's not like, so it doesn't matter what the method except it's.

E

Just with these shortcuts to call the victim it's, it doesn't just shortcut. The call like the jet actually turns it into a know of one operation. Yeah, that's the key part, because otherwise you would have to transfer every single call cycle which.

A

We won't yet I mentioned in said: it's Riva is the last time I talked to Matt's about this. That is idea of target typing in the literals, by which I find not very compelling, because the basic prevents us from adding overloads to existing method. The text string yeah because you never call them, which is why I think in practice. Let me design overloads like I, think Jerry. What is up? There's a lot head string overloads because I could inflict the magaman would be well. We have to solve this problem regardless.

A

We will probably want to have overloads for things that have taken string today, so I think we need either either two different effective to call always has to cast- or we just say this special syntax, but otherwise you write a little in the source code and in the code view you have no idea of knowing is going to transport or not because it all depends on what the reason for itself and.

E

It gets I think they are gonna need, some sort of which will soon taxi Allison I I do have separately a proposal for that would look like, but it's a conversation for a different day. It's at our internal team of me so.

B

But the question is why we are waiting for the compiler feature but I think like I, think the goal is, can we end utf-8? You know slice support and make it practical for people to use and I think we're on you get.

C

A string is not gonna, be seen in time. Well,.

B

I, don't need for this method contains I, don't you do using.

E

We could we could give you a method very well. Let's talk offline I can do that. I think is which is very similar to what they ask anymore. But let's talk about them here: okay, no I'm just make it money. That's the.

B

Idea right that each of these.

F

B

A

This is speaking of a particular cut span on utf-8 Curtis. So how married is this time to YouTube and drink like this? It wouldn't be possible to ship this without beauty messaging, whether miss yes,.

E

That is, that is correct. It would be possible to ship them separately. However, when I was talking with Josh the other day, he thinks that the strongest argument is going to be shipping them together and I agree with him, and the reason for that is what's the easiest way to get a real nice man, the beauty of HR, to start with a you, th attorney yeah, so I would I would disagree.

E

Sorry I mean I would say that, like if the UTS stream you tape, each string without utf-8 chart is bad, but I think I would not hold this on utf-8 string, because then we can take things like JSON document and say that it takes utf-8 charm and not like their matter. Our compiler features of I would like to see come in before this hits, but I don't think that's going to happen.

E

For example, there's a work item on the Rosalyn team to say, like I I want to be able to have both a checked and an unchecked cast operator for a given type. That would be great, but timing shows not gonna work out for us. Yeah.

A

So I mean in general, like I, think that so what you said about YouTube 8 string, it's the easiest producer for charm, yeah nature is I. Think it's true of you have little support. If you don't have the books of rituals I think you're much more likely to have a binary or they would only spend on, but you want to just reinterpret the car. It's one you wouldn't wait to get to the radius what you would do in practice. Okay, so your literals right well, I'm saying they could mean. Thank you. How cool?

A

How do people get weed all this fair enough? You th are right, I mean I. Would I would think. So, even if we have you, th string is the time with all the tools support. You basically have to start with something else right right. You.

E

Would education yourselves, you would need a helper method which is basically like your own intern dictionary or something which is what we had spoken about the other day right or if something's coming in from the wire. You would use one of the one of the colors nobody's my point.

A

Is that when you help them with it, you control what? What is the type you converting from yeah, so you can say bypass utf-8 string effectively entirely. You know, utf-8 charm,.

E

Try validate takes in read-only Spano by Canal Street. Only spam, you thr right. Okay, all we need to do is add one thing of these bytes in the you thr. Okay, that's accessories.

B

From the minds, the more I need to have a little support for string, yeah well,.

A

I mean I would say that depends on volume of you or you think about this head deck I. Think.

D

Was the firm the.

F

Wires, king would leave I.

A

Said it's like no, we are not the most common scenario to getting a char today. It's some ritual I buy that, but I think that that only flies if the compiler has support for the to us yeah, if we don't get compiler support for the tools, shipping utf-8 string doesn't help us at all about video. For my thing is of why.

E

We would ship you tfh are without music if, if utf-8 strings, not ready, this release, but utf-8 chart was why you thr, would makes sense to be included. Is we can add the you have bytes from the network turn this into chars now call new API, the 10.

B

Th are what we are it's just. We started to talk about the the scenario of like literals and calling contain.

D

B

Where I'm saying and I agree, more utf-8 string doesn't help you here unless there's basically literal support is the thing that I need for it. Yeah, that's it I, don't need anything else from yeah. Let me maybe face.

A

Well said we made the decision of the compiler will not have support for you to write in the tools in the C sharp edge release, meaningful three or which means we could decide to ship utf-8 string in for your I. Don't think it buyers with anything I. Think I think that so basically I would say that if you look at three over base, you have to except utf-8 string was not going to ship, which no means, while shipping char.

A

It only makes sense if we believe char a door to david schardt is meaningful enough without utf-8 string. If we think there try to do it, then that by the by extension, always you have to wait for like it stops us.

E

From minus--.

A

Release polluting things with read-only span.

E

A bite when we would say in the next release, they should have been, you know and.

A

Obviously, a paycheck, that's it. It.

E

Makes us not have the ugly wart of dual inputs for for the interpretation condition from the wire? So that way and.

B

E

You win yes, each our points.

B

Better coming out it's better than nothing, it's better than nothing yet I cut still without little and support. The teacher is not it's not a scam, but.

E

There's always a race condition with the runs.

F

It is the run time the framework.

E

And the command the compiler and the language of windy things and some usually they align together, but sometimes pieces go in different waves. So.

B

By the way we had the discussion, I don't know what your thoughts are on. It I claim that we don't need to run tech support or what, if, if the we just need, compiler and I agree with amor, rather instead, I think it should be explicit, but imagine that we had compiler syntax. Where you can, you know prefix you, a internal suffix, you ate a string and it actually returns read only span of of char a that's basically what it is, because it's read on this panel of tolerate it just points to the metadata.

B

If you want to create he people type from it, well, you're gonna be making copy anyway, because it's gonna be on the heap now in some other place, so we you know like there can be a constructor for future utf-8 string. That takes this this thing and just comes to array and copies. You know or not worry, but like copies, that the data so to me the kind of MVP for this feature is this type and literal support in the languages that that produces it. Yeah.

E

And I'm saying us dotnet core 3 having utf-8 charter without Roslyn having support is fine, because the next version of Rosalind will work against that core 3, because, like the other way around is the.

B

One that says it's.

E

Weak and very unfortunate, but yes, technically I, do if you have a date of release, sometimes one things ready in the other ones, not and I'm. Just saying that we shouldn't artificially hold this. That's my call I'm trying to say right.

B

That it's just I'm! The point is that we we talked about the little support for you, know strings in season three years ago, yeah so.

E

There are, there are two other method classes on member extensions. I want to call attention to. The first is trim yeah yeah, it's uh you'll notice of the trim overloads are relatively pale compared to the arena lifespan of chart, but once the reason for that is just the the difficulty of implementing all of those other equivalents. So this is he. This is what I consider to be the Minimum Viable Product. Can you tell me what we're actually missing we're missing? It overloads the trim.

E

We then there are other overloads to take read-only span of Chora or you tfh or whatever that says, for any element in this sequence trim. All of those elements off of off of a stand that you have I'm gonna clip.

B

Equivalent read on the memory of tyranny: yes, I.

E

Think so I mean, if you, if you start with a utf-8 string that can be in this all works in the future, where that can be converted to a read-only memory of UTF HR or a read-only memory of bytes. Each of those can be converted to the appropriate spans. Additionally, read-only span of UTF HR can be converted to read understand about anything else requires on sale, so.

B

You will need read on the memory of terminate overloads for trendiest, not only for trim.

D

B

Don't have memory overload or anything yeah the only for the ones that have everything yeah, which right now is.

E

Only open the other, the other methods that are weird here at least compared the utf-16 equivalents- are to upper into lower and I. Have a giant comment explaining why they're different the key difference is that the two upper and two lower methods, the tape read-only stanitch, are and write to Spanish are changing case where utf-16 does not change the number of utf-16 code units. That means that you can. You can perform that case inversion in place. The reason that it doesn't change the length is because we use what's called simple case folding under the covers.

E

I won't get into the full details, but it guarantees the link, will never change. Utf-8 is different utf-8. When you change case, you can expand the length of the string shrink, the look of the string. That means that this method necessarily behaves differently than the utf-16 equivalent. So concretely, that means grown to this manner. So concretely, that means that a take a look at just to our two levers. Just before.

E

Just after the comment you know, including the comment in the comment down concretely, that means of a you can't pass the same buffer for both source and destination. If you do, you might corrupt your data, UT of the utf-16 equivalence, do allow you to pass the exact same buffer. That also means that if we run out of room in the destination, while we're reading the source and doing the conversion will report false bull, which will Oh will return negative one to you and then leave the destination corrupted.

E

Basically I shouldn't say corrupted: they leave the destination of partially converted. Do we have any other places that we've added right to spend them, because most of the places that I've seen we we can check the length upfront and then we don't write anything so either we didn't. We give you a valid thing or were in operation status, and we tell you that we feel you know sort of stuff. So the upper there are operation status. Ap is later in this document.

E

I, don't think we're gonna get to them today, but they exist in their written here. This was simply an analogue of what already existed on memory, extensions, those those methods today detect that this nation's too small. They were term negative one and they presented what I present amendment the existing methods to upper to lower front.

D

E

Span of char extension methods, but these don't fit excited serangoon. They don't resize, but because the length can never change when you're doing case conversion for utf-16. They know what the destination length requirement is up front, which means they can just return negative one. If the destination is too small, I can't make that determination without actually walk in the entire data, which means I can't return negative one eagerly I have to do all of the work to do the case and then return negative one.

E

But if I've already done all the work for case conversion, why didn't I just start popular than the destination.

D

The existing methods.

E

And the extension methods- yes, because yeah I would expect do it to in no two hundred to instill in. You can always remember the first place that you actually had to do work and you get a copy of that. So we could do that, but that kind of works against the value talk or having all of this utf-8 work in the first place, which is I want to operate on data coming from the network with as minimal amount of overhead as possible. But once of.

B

The one thing we could do is say we have operation status API. So therefore the you know I really want to. At week we deal with like non-contiguous. You know rooms and stuff like this. Okay, and then we have kind of CalPERS. They are not completely insane like allocate every single time, but at least you know they either work or don't they are the triac waiver? Yes, walk it walking it twice is fine.

E

It's not like yeah, like all you have to do, is count the number of bytes you have to increase in the first walk and I've agreed with Christophe. The operation status would be the one of you wanted to do, one in and you're willing to pay the loop yourself and if you just want it to to work, we shouldn't go around corrupting, but we shouldn't go around. Writing the buffers and saying oh I'm. Sorry I forgot what I'm supposed to do that and then that's fair.

E

That's why I wanted to bring it up for discussion, and maybe maybe the answer is just we don't even have this look? That's the third alternative right I mean it's. If we shift we'd immediately get somebody saying hey, you seem to hit Plus this in your future purely for things like utf-8 string. This isn't an issue because they allocate any way, maybe in utf-8 string bell to upper to lower right I'm.

E

Just like that, someone would found the issue and then we have, to you know, keep shutting it down or eventually do or and welcome direct them to the operation status based APRs, I.

B

See operation status API is something that we build on top, so you know we have a prototyping cone effect, slab of text writer that takes transformers and then transformers used operation status ad is to do all this magic for you, I think. That's the only say what the same way to use the idea is again. I'm kind of you know, thinking.

B

It's not even kind of the line of business developer developer and the answer is decaying and we cannot tell them to do this. I mean they gonna, be basically like okay, I'm, not using utf-8 string anymore, because in ut of x16 I can just basically you know to upper to lower and it all these words can be chained. Only.

E

Few names, usually a fake string as an exchange like everything just works, as expected, I know it's just methods like this, where you're operating on raw sequences. You have to be aware of the fact that you have to be aware of the underlying economy right that point you're like it.

E

My answer is: this should do exactly what encoding does if we said that encoding needs to count and see how many bytes it will, if the coding utf-8 counselor number of bytes before writing anything yeah, and this needs to count the number of output charts before it writes anything, and if we say that coding is allowed to be messy and this is allowed to be messy. Okay, like that's, that's very exactly my like. No, we need to have consistency with what we do with.

E

If we return a negative memory. That means we didn't do anything or if we return a negative number, it means I, probably destroyed all of the data that you imagined how. How do we keep from getting into a situation where developers might expect an operation to be O of something, but it's actually o of something else. I mean documentation of the answer. Only.

D

Mana zone in whether.

E

It's 2 min or 3 N or 16 in yeah, but the constant factors do show up in that form. So I.

B

Think you know we have operation status, AP is and all complexity, but is very well-defined. And then we have many api's in.net framework, where nobody can tell you what the Lexie's and that's okay, okay,.

E

That's that's our let's scroll down a little bit more than to the is well form sequence. Api is it's the last thing in this large block of code.

A

Sorry, sneezing Boston. What does Stephen? Because they that took died this one here we should I, saw them.

E

Battery notice and then okay, so is well-formed sequence, helper methods for both utf-8 and utf-16. That simply tell you give me a scam, I'll tell you: if it's well-formed or not, why would we not make.

B

It read Aaron return.

E

And diversity, though these panels to write yeah I would I would make I mean you can have the balloon one very well, but that I would make it. You know, try read that, like you come up with something to start off, try you know, and it returns false if it's not well formed.

E

That was the reason why I didn't propose out here is because the caller has to do something in the else case and I didn't really have a good feeling for a common pattern that everybody would be able to write in order to handle that correctly, like what I, what I had actually done in my own sample code when I was using, this was I had an API which ones here's a read-only span of bytes.

E

If this represents well form utf-8 projected as a read-only span of UTF HR and then just return that exact same thing back to me. Otherwise, new up a UTF, a char array that contains this data, but where everything's been replaced to be well formed and then returned east and pointing to that array, but.

B

I thought that very common scenario would be I call this meant out or something like this, and if it's utf-8 well-formed I do whatever I was doing otherwise I'd throw an exception and I say: I, don't know what you want me to do, because you wanted to pass utf-8 JSON and it's not even utf-8. You forget about Jason.

H

B

Therefore, well, this API is really bad for it, because it would require me now going through the same code, twice first I check whether it's valid and then I want to actually have read on the you know, span of Charlie because I'm about to execute the medical we.

E

Want to the the success path from this should hide from the user, the unsaved as okay. So so it sounds like what we actually wanted are three different methods. The first is as when we hear like given a spam like. Is it just welcome yes or no? The second is given a spam. Try projecting it as read-only Stan, WTF, HR and return false. If you can't do it. Yeah I, I, I'm,.

B

Wondering what the difference one is useful, because, when I check whether it's valid the next step is either I fail or I want to have already validated see yet I never will so.

E

Just serve and I was little it just.

B

The trying we look up and they can out underscore if.

E

They don't the.

B

Result unless you, what is this scenario.

E

For just checking, because I'm a database layer and I have some kind of validation turned on before I actually convinced stuff to desk. It's all those fights in the end, but I want to make sure I want to make sure the caller didn't get me bad data, because I'm running in a debug mode or something then I just work on disk, or is there any way that I registered a third? The third overload would basically be always get me back a really stay on UTF HR.

E

If you have to fix it up, allocate a new array and give me a standard out all right, I would.

C

Want to put one which is give me an easy time and keep it down with them. So I'm not write the kind. If not my sentence for myself well I mean that matches our.

E

Guideline, if you have a twenty heaven on trying yeah, we couldn't because it up I, don't like to think what I'm gonna do. Okay,.

E

Yeah I mean yeah, so I fix up what may be useful. The the assert this is valid and give it to me as useful. Try but I, don't know that the the one that just hides, because, obviously, if you have this I guess either one could be written in terms of the other of which is this doing the out underscore for you or is the that one complication and maybe.

D

E

Actual underlying method which isn't listed here because it's private is return the index of the first invalid code units and also out the number of utf-16 code units that would be equivalent and out the number of UTF thirteen code unit. Some of the equivalent. That's.

D

The underlying.

E

D

So it's already destroying registers this time.

E

It's it's fantastic by the way that that method, once we clean it up, it does eventually need to be made public because everything is written in terms of that, if we were to have an overload that says, fix up whatever you need to do and then give the Baptist an acute EHR. Would we be okay with that? Just allocating considering his relationship like this should never happen in the first place. The only reason I'm doing this is to fix up that day.

E

I think it would be even more sensible than that one would be very clear of what it's doing and it takes in a span and and out some memory.

B

So I needs to be clear, because I can imagine a scenario where I could have either I have an array yeah I, pass it to this method. I want to know when you suddenly swapped there, a or I still have to hold to the array that are you. Okay, yeah does.

E

B

Pulling there a story.

E

B

To figure that out, yeah.

E

And I no need to feel suffocated, don't.

B

Give up safety I said, do I, think about except we'll just always allocate.

E

They were taking a span and out of memory, so at that point it will return thing. I got two over and of at that point it would literally just return you to your fixture. There's no reason for it to return anything else. If it's allocated Louis, that's true, so then it would have. um Essentially it would be the utf-8 string of read-only span of mic, so you mean the utf-8 strength, instructor, yeah, yeah, that's yeah constructor was implicit in my head with you out there yeah, so.

H

That's the fix-up.

B

Assuming that you're doing at that yeah.

E

So basically, the.

B

Common scenario is I call try if it fails, then I just create a new utf-8 yeah I, wouldn't do it and constructive because I it would be nice if the method said I'm gonna be doing big steps for you, yeah, okay. So it's.

E

So it sounds like we see, we see that we all agree on what the internal shape and don't let them should be. We just need to like massage what the parameters and stuff are to make it more useful. Okay, the only other thing that's in this document, our operation status returned the api's I, don't know. If that's interesting for this region. For me it is a planet. Very good string, escape string is not actually forbidden, but I.

E

Think things that you want to like to upper to lower or in this document again is not the other than system. Thanks yeah awesome, we don't care if it's public, so we could. We could word mean so. The operation status API is that I have at the end of this document, are to offer to lower and to upper impairing to lower invariance, and also translating ones are interesting. Yeah. The reason that I have separate transcoding api's is to work around limitations of system text Mikoto getting school I can as soon as if you.

E

Scroll down until you see a large block of code with comments there, we go so there's two upper to lower its a typical operation status, API, the two upper and two lower. If you're given Mel, if they're given ill-formed inputs may return invalid data, unless the underlying localization library says I'm just going to try to do so at which point we'll just say: okay, we don't we defer to live ICU or we defer to the class event. I see it's.

C

My job again IJ on at the end ID part I. Do you think it's in the middle, but I'd have to get back to your that it.

E

It will match whatever pattern at does: do we have a government influence API or to upper to lower no for the reason that I mentioned earlier, but he declared expand or shrink and I would corrupt the buffer at that point, what what I would have to do under the covers is I would have to I would have to rent a temporary buffer to write everything into and then then copy everything back into the original source buffer, but if I'm doing that, I may as well just make that evident. Why would you not.

B

Keep going when you encounter the resize undo the changes and.

E

Then return I can't do anything at all, because now I have to keep now I have to copy everything that I've overwritten just in case I need to back it out.

D

Where you wouldn't have to to resize to be smaller, sorry.

E

What was that serious, so.

D

I'm I'm trying to understand is the concern that upper casing or lower casing may change the size of the the the character. Yes,.

E

It could it could result in a longer or shorter utf-8 sequence. It.

D

Can actually result in a shorter that's what I was question. It can result in tiger I.

B

Still don't know why we couldn't undo the changes. You know how to happen, and we know how to to lowered when we encounter a resize. We just work backwards and you know to happen to or to lower, depending what the methods work.

E

Well, if they, if the original data was to lower, was already lowercase and you call to or and then I realize, I have to back out a might now. Uppercase I know I'm stupid I'm. Just writing the original data. At that point, that's the only way for me to recover is to make a copy, but it's by making a copy when it wasn't in place anyway right.

E

So you have so that's to upper to lower any coding redundant in code. What do you mean info Crisco, yeah, no transcoding can't we've done a place for the same reason that can shrink one die channel and he goes right. No, a lot of them the code. You would count going from a to T of 16 to utf-8, but there will never drink. However, utf-16 is 16-bit bytes.

E

So when you're talking in terms of actual amount of memory consumed not code unit count yeah, it can shrink or expand, but in the canonical example is asking us write. The word hello in utf-16 is 10 might sing, but I need to give an something like that. Would.

B

It make sense to have the clearing method, to ask whether it's gonna recess or not and then have in place, because, basically, the travel that I have is now. We create an API that basically forces data copy, but almost never is it. The data compute.

E

Be acquired so there there already. Our API is on system text encoding to say things like how many, how many bytes, or how many chars build this thing result in. However, you still can't use that to perform an in-place update, because maybe this part of the string shrink we're just gonna bittering expanded. So maybe the total number of code units is the same in the end, but the string did a like a weird accordion thing and, as a result, basically to make a duplicate on the other.

B

So the ID I would not be is that it is the sense the same is, can I do after and to lower in place? That's basic, but if I'm already doing, if I'm.

E

Already doing the case, conversion work in order to give you that answer. Why didn't I just do it into a temporary buffer and then say here's here's the bottle containing all the way, because.

B

The the tag can be done on basically rules, you don't need any buffers, let us keep working and seeing whether there's anything that will be quite.

E

Anything, but this this comes back to what Jeremy and I were talking about earlier in regarding this API is now over. 2N mean it's like: why don't we just? Why? Don't we just not have an in-place API ingesting what they're not in place API is over 1m. They will.

C

Be doing but you didn't go to the check-in, god didn't get anything true and then what would they do in the.

F

B

They would call this a peon okay, now that's o2 and as well, but.

F

B

Department is it's so rare or, for example, I know my data is, is you know ASCII and now I have to have copies between part just to do a portal away, and it's so rare that.

E

This thing will be nice to meet you if you know for that, our potatoes asking it go ahead and pass on the same buffer for both source and destination.

B

F

Don't believe is that, okay, right so using it with a I, can pass the same battle. I mean you. If the data's not asking you like you're.

B

In for a world of pain, well, it's not it's not, and he doesn't have to be a ski icon, the right! So should we provide a queen that basically just.

G

E

What is an API, which is now.

B

Because it can be utf-8 so.

E

You, when you really want to get either else, can this data change the face in place.

E

That would be a very, very cold API. For me to have to be honest, because I would be.

B

If there is even a single ruining this thing, that requires changes in the you know number of code units, then you return for.

C

Very concerned about it, because we got intense.

E

The only so what the waiting.

F

With the way that.

E

That would have to be implemented is because it's using live ICU or windows only the covers what I would have to do is under the covers. I would have to allocate or rent a temporary buffer to get live isotope anyway, based even for in very I'll, make.

B

It super contracted, meaning, don't you tell me it's a musical.

E

B

E

That's the only thing I can guarantee.

E

But if you know the, if you know is asking them, you could just pass the same source and destination I'm.

B

Going okay, it's a ski! By the way that takes you read on this panel trying we heard I didn't have it here, but okay, then maybe that's basic. You know yeah frankly those two because I'm thinking about you know common operation, some like payloads of HTTP, yes, I kind of would prefer to just too many of them in places as much as possible. We could have a.

E

Little we could have ASCII equivalents instead of utf-8 equivalents that would operate on, read-only stand a bite and those would just fail if they, if they saw anything that wasn't asking, especially since HTTP, is a very specific scenario of yours and HTTP. As far as the payload and headers go are in volume, utf-8 they're, generally nasty.

E

So maybe these ap guys wouldn't be appropriate for your needs, because.

B

You want to enjoy some things like in the patient's forms as much as possible. I.

E

Mean sometimes I know, then it's not positive. We've got some things did so. Maybe that's something we should just consider separately. Then ASCII specific transforms, because, honestly, those would even make faster than the utf-8 ones, because what he would to do what they would do is they would simply say you know what, if this data is not asking I'm not even going to bother trying to inspect or fix things up, I'm just immediately going to return there, I couldn't do the operational aspect.

E

The only other things are here: transcoding API, it's immediately after in this comment. So it's from chars in 2000.

E

Yeah, the transcoding API is there's replaced invalid sequence, which basically says if I see an invalid UTF subsequence, while I'm doing the transcoding. If that is set to true I'll, replace it with the replacement character, a 50, I'm gonna says next to return. This was an alligator. Yes, however, it makes it very difficult to consume, because okay, great I return to you invalidated. What do you do like? How do you know what needs to be fixed up? If you just, let me fix it up for you and makes.

C

It more either sending.

E

My chicken Lucine yes.

C

But helping them.

E

In that yeah, but how do you know what to do with that information once I give it to you like, say, say, say that I tell you by the bite into the next number. 52 represents the beginning of an invalid subsequence of utf-8. How many bytes do you strip off what order to meditate about to be back people in this one, the user look, so the user will see invalidated coming back and we'll probably just well.

E

But if, if the user didn't want to fail, if the user wanted to recover by doing a normal substitution, the user has to have some ungodly complicated logic, to figure out where to insert that character, which we should just do more than or not.

E

The reason for that is again come back to the scenario I mentioned earlier say that I returned to you: I found a bad bite at index, 52 and then you're now looking at these bites, and you need to think you're out how many of those to show about replaced with a single ffft character in the output and then continue. The only way that you can do. That correctly is to basically read the universal.

C

E

C

You watch our data yeah.

E

And that's why you always, but that's what this on this moment, if you pass me, partial data and I see an imbalance sequence at the end, I'm not going to assume the data's invalid I'm going to assume it's incomplete if I see it in the middle and going to submit it is really right, and this can differentiate between enveloping and prove anyone in all cases like if I see an FF byte at the very end, I'm not going to return, incomplete I'm going to return invalid, because I know that there's nothing you've, never give me a encounter about that, will ever make an analogy, but it is really it's for convenience, because I, don't think the average developer can look at invalid data and know how to fix it up.

E

Let's sit over there.

C

H

Letters come to feel weird about it, but.

C

I, don't they can do it, but you're assuming.

H

The user scenario is I want to fix it. Maybe their mother.

E

And if it were done, and in that case house false handle invalid data and then do something do your own thing, then I think the common case is going to be. The developer wants to be a character, substitution and because I think that's the common case. We should just handle it on their behalf. Do we let people pass the substitution character? No, it's FF updater.

E

It's the like diamond with a question mark in the middle. The Unicorn is the official Unicode replacement character. Unicode also lets you I mean if they say that it's really important even defined in what you should replace it with. If you seen in balancing once they really really really want you to use that very specific character. But maybe you use a question mark or something else. You know it. If you want that behavior in system text encoding, is your friend well.

B

Know you can call this method and then maybe place you Captain Planet.

E

Because similar to what I asked us and earlier, how do you know how many bytes to.

B

Strip out and how many question on the.

E

B

This method does the replacement to Diamond question marker, then I call again replace and replace all diamonds question mark with whenever I want. Twelve out of diamond question mark was actually in the original paper. It's.

E

B

Okay, fine, it's not the same, but we are now talking about some very theoretical by.

E

The time in question part is totally valid. We know halo, really yeah, it's it's very rarely use it.

E

Yeah you, you have now mutated the data in such a way that it's no longer a faithful representation of what came in over the wire. If you send people diamond question mark here at the end, no.

D

If I some people.

E

Dive in question marks like a right now right. This is gonna might use our name from now on a.

B

Dollar right had a blackout of Bachelor, dynamic customer I think whenever I start a new account somewhere and when I use a question like so.

A

We have almost no time left yeah. What are the next steps here? So next steps are you marked, as as API probe, yeah I? Think I right now in the market is needs more work to.

E

Full yeah I expected that, but I was hoping to get like actionable feedback out of this and I think I have gotten that for the most part, yes I, think yeah I think on our end, we still have to.

A

Go through the motions through the define, what it is you're shipping in three or yeah. Nobody have to do that with other. We need to see the scope of Jason right now and we haven't really done much for the seal that I get either I didn't like you have to make some homework for some. What did it mean to expose so Christopher knew and like how soon do you see yourself using I'll.

B

Be blocking you we're not making forward forward so as well. What I said I? Do they usually don't Hispanic like.

B

Start starting and I can read but better view and I just realized, because I was actually yesterday debugging, something with the developer, who doesn't understand this whole thing hoodie and she was so confused and if deviations discussed.

D

B

The heck is, is this good stuff like well.

A

If only you were there.

E

And it sounds like there's another discussion that he had regarding very specific API, so to make Christoph scenarios easier, which are separate from the API, so you think and save in here and those a two hours for your particular scenarios. Maybe shouldn't just be based APL, but.

B

This one this one makes it like. Let's say now: it's available right, I will try again. I will basically now go back change, some places or other overloads that you know on Monday, we'll be taking strings Ingrid on this panel charge. Eight and I will attempt again. If I, could this and literal support for four strings, but you need own level right, yes, yeah, that's I! Think.

A

The thing that move our hard time doing, I.

B

Know as I said, I don't understand why but well.

A

It's obviously designed magnetic sure sure. So it's what you can.

E

Be a resource based.

A

E

This one, it would be bit more tricky right so for usually.

A

E

Itself, we probably wouldn't be able to use the source base packet, but for the individual, hopefully is writing. On top of that, we might be able to you.

A

Know it's shipping adjust the time signal to we're ready, Emily, that's better! So the question is you have any opportunity pendency besides Chris of tracing a.

E

Asp.Net is everything is already on fire there. They, they are more interested in the actual exchange type things like utf-8 string as opposed to those because they're they're, considering us from the perspective of how do we actually exposes to our overall development I think they would find this interesting, but I don't see them using these often.

B

I suspected that they actually tried to start doing work in this space. They would like to be yeah cuz. It's I understand that they are thinking. Ultimately, what do we expose the developers, but they have any internet in the internal code that could give you other things that right there, everybody who will try to move their system to the effect will be back after a while, because you literally cannot debug code. Like you know we be talking like hey.

B

You know you can do these things well, they are not done, and today, using read on this panel by its little, not working in.

E

A speedo that's priority.

E

I think right now is on making improvements and changes in the capsule I, don't think Kestrel is, will only use anything beyond byte as their own internal representation, which is totally fine right because they are dealing with networking up, but that so I think they will find useful API it's similar to what Christophe needs which is here is here, is HT data I got that represents like an HTTP header or request line, or something let me to opera or to lower this thing in place right, like I I'm, going to a super Cornell.

E

This is asking because it's one of the HP's got sound soon. Those API is I think are going to be more universally understand.

E

Unfortunately, they're not presented them and run doesen't. Thank you for your fun yeah. Thank you.

E

If you want it or context on any of this, like feel free to read the wall of text in here, yeah, so I, just one horrible thing.

A

I've captured most of the stuff like into what the engine was not quite sure it was like other things. At the same time, so I was somebody PR and you can just add the other thing. Did you and then I think we we.

E

Can have a good first couple of notes: yeah, like my they mean that I'm, you know generally a security person like that I guess a lot of the way that I approach. This is, if we do use freedom most and invite for everything like what are the potential pitfalls of that so I, you spend some amount of time in the document discussing those, including historically, how its cost problems right.

E

That doesn't mean it doesn't do that. We won't end up doing that. In the end, it just means that we need to be aware of the fact that it causes us. How do we ignore them right? If we want to go down there.

A

Well, I think it's tomato, my ugly I.

E

Also tried looking at a debt, go to see how they handle all of that, especially with regard to utf-8 severely, and a lot of that seems to be it's an application-level concern a lot of framework on their concern.

E

Like the the application developer, if they care about you TM, if they care that the utf-8 that's coming in from the wire is well-formed, they should do that.

E

Chuckling sounds rented the framework the gold language to go SDK like does provide the facilities for them to perform that track, but it is still a track that the application developer explicitly out to go generally looks like they just expect all of those uh they just expect two strings to be slices of bytes, not good, and the banks are doing to giving them right well, they are to an extent like they do differentiate between string like side, so that might change in the future. But at the moment there is that resentment.

E

When this change might be probably be bigoted. Well, it's. The real reason is that they don't have a concept of read all those slices, so you can't projector string as a bite slice because now, all of a sudden suitable, if you had a read-only slice, they may actually have an interesting discussion. They've been having over the past few years. If you have a read-only slice type, could you project a string as a really nice slice and there I don't think they quite settled on an answer?

E

But it is something they're very interested in personally and the and as far as I understand their most recent thoughts around that the implicit production from strange read-only might slice that projection will be implicit. There would be no projection in opposite direction, the reason being that they want the string to be not just an immutable representation of data, but they wanting to be an immutable representation, immutable data, if that makes sense, and because there's no way to guarantee that the read-only slice is itself immutable because someone else could have a reference to it.

E

Then, at that point, a string that becomes meetable, which means to hold some right all right.

A

That's good ice cream. At this point.

A