.Net Foundation Design Reviews, 2 Jul 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitHub Quick Reviews

Description

00:00:00 - Approved: HTTP2: Create additional connections when maximum active streams is reached https://github.com/dotnet/runtime/issues/35088#issuecomment-653248184

A

Hello friends, we are live again. This was not a joke. We're actually doing second empire view today too, this time, it's pretty late in the day for us, but it's probably still very fairly early for James, because he's in New Zealand, so finding some common time so take it away. James.

B

Okay, so this is adding a property to sockets, HTTP handler and I believe the proposal has been expanded to also add a property to when HTTP handler. So the issue we're solving as HTTP to has a setting called Max and current streams. This is configured on the server and it is the upper limit of active streams on a single TCP connection.

B

The reason the setting exists is to prevent a client from doing malicious things like opening a thousand or a million streams active on a single TCP socket so, but the server sort of puts a puts, a limit on it and usually will.

A

B

And usually it's about 100 but can be a bit higher. So the problem with the setting is today with HP client. It will always open a single connection to a host so for a given host name, there'll be a single connection and as soon as you hit that limit, so you call a send a sync or one of the methods which cause them to send a sent like get async.

B

That method will await and it will form a queue so first in first out queue and it will wait for a current active call to complete before it continues. So you you have to do this on the client. If you didn't do this on the client and you still were tempted to send the request of a server, the server would error. It would send you a message to tell you that you're making too many requests, and for that reason the client tracks the number of active streams and it will form that queue.

B

So there's a couple of problems with us. One HTTP client is quite common to just have a single turn instance within an app now, if you're in a like client app, probably heaven they get like some thicker client having 100 concurrent requests is probably quite a few, but if you're doing something like one micro service talking to another micro service and all the requests are going through a single HTTP connection, a single yeah, it should be client.

B

You could easily hit their 100 limit and it's a little worse for ERP C, because G RPC introduces the concept of long live streams. So you could create 100 streams that you expect to last for a long time and then any subsequent subsequent calls would just be queued and would never never go anywhere. Never progress until one of those long live streams were closed and an additional problem is like there's very little feedback about.

B

What's going on, like most people won't understand this max concurrent stream, setting and they'll probably not understand why their request through it should be client have have paused. There are any comments. What I've said so far.

A

No, that make sense I mean like I'm still, understanding like. Why do you think you said, like a hundred connections for normal apps would be a lot, but why do you think it's more common to have it to reach the limit here? That's the thing, I, don't quite understand.

B

Sorry with reached a limit where so.

A

You basically said normally a hundred connections would be quite a bit for an app right, but, like you said in the context of micro-services, you can hit this limit much much quicker. Well,.

B

It's it's more common for a market service to have make I have many parallel callers talking to one particular server I, you, you think about all.

A

You mean on the server, not the client I, see, okay. That makes a lot more sense. Now.

B

It's the client, because the client is the one who's creating the connection. I just I think this Mexican current streams makes more sense in the context of a browser like you have hundreds of browsers talking to one server and for each of those browsers. 100 connections, 100 concurrent streams, so download images- that's that's, probably fine, but when you have one server talking to another server like all the activity of an application of a server application is going through that one TCP connection, then a 100 concurrent requests, and quite so many right.

C

Where all of your clients are sharing that one connection, because the server is the one creating that connection yeah.

D

Right and like I, think you said it's. The GRP streaming scenario makes this way more common. Each of those uses an HTTP request like each stream yeah.

C

So, to provide a little devil's advocate balance here, the HTTP 2 spec actually does say, don't do this, it says, don't go over the stream limit and don't open more than one connection, but we've seen other clients do not respect that limit and- and it really seems it seems like a lot of the industry- is ok with kind of going over the limit in certain scenarios such as the microservice one.

C

Although we have seen reports of some servers kind of detecting this as bad behavior and start returning like error, five hundreds. If you go over the stream limit across multiple connections, yeah.

B

I think I think this sitting was probably when I was thought of. It was thought of mainly in the context of browsers. It was there's no reason a browser should start opening multiple connections to download one webpage but and the context of a server talking to another server and all the all the HTTP activity is just between one client on one server. Then multiple connections do do sap making sense. So some examples of where we've seen other clients get around this issue is go will actually do the spider fold.

B

When you exceed that limit, it will just create a new connection for you. Other clients tend to require opt-in, like when HTTP yeah, when HTTP I believe in Windows has a setting to like a flag which you can unable to create additional connections and I. Think I've seen Java clients have a setting to create additional connections. So what we're? What we're proposing as a solution is a setting which will allow it should be client to create additional connections when the limit is reached so rather than pausing and forming a first and verse out queue.

B

Http client will instead create a new connection which would reset that concurrent streams limit back to zero and new requests would happen on that new connection. Up until the find where the original connection saw less activity and new requests would be, then we made on the original condition so.

A

I'm still somewhat had lost at lack because, because of the you know, party that initiates it and the party that deceives it. So who would said that setting this the the server that initiates the contact to another server? What actually the browser that talks to the server the code.

E

Said the Mex.

A

Http to connection per server, setting the client or the server, hopefully the server right just.

B

Whatever does like the, the client could have its own session, which, if it decided it had more than a certain number of concurrent strings.

E

B

Decide to create a new connection, but the server won't have any knowledge of that, but the the sitting we're talking about at the moment as as the the server setting that's what HTTP, client and sockets they should be handler, receives when a connection is first established and it is what respects Cori.

D

You mentioned earlier that some servers are looking for multiple connections from the same client and rejecting them if they're for hv2 connections, what what server was that I.

C

Don't recall it I'm actually don't know if I was given the server name, but I was talking to the win HTTP guys about this I think or something related to this, and they said that they found that problem out. There.

E

Just to reiterate, though, if you have a scenario where say a front-end web server is talking to a back-end web server. Presumably you control both servers, so you could suppress that behavior on the back end, if you really needed to I, would.

D

Think the and I wonder how many I was surprised that a server would actually block multiple HTTP connections from like what the same source IP, because I feel like NAT would like break that as well. But maybe I'm missing something there.

D

C

This setting is essentially a workaround for servers that maybe aren't configured correctly or aren't configurable. So maybe you have a micro service, API server that is limited to a hundred maximum streams, and you either don't know that it's a problem or you can't change it. So this is kind of working around it by having HTTP client open up multiple connections. Oh.

B

Oh adds sitting max concurrent streams to a really high value there is there is that that does so have some downsides like if you did ever have packet loss, then it would affect every single request because there is still head-of-line blocking when packet loss occurs on a single TCP connection, and there is probably finite limits of how many requests you can send via a single TCP connection.

B

I think, if you add tens of thousands of concurrent requests on a single TCP connection, just concurrency issues would eventually lead to issues with throughput, so I think I think having the ability to open up multiple TCP connections as a is a better option than just trying to send everything. Why one want is to be connection.

D

Especially like in petrol, we just run every request on its own stream, like it does her performance with that design or sorry. We run like each request on its own thread. I, don't know if I said the screamer thread, but yeah that does cause concurrency issues. There are too many streams in one HMT connection.

C

Yeah, so to kind of play it a little safer and comply with the spec by default. This will still be one, but the user can change it all the way up to like int max value if they want to just go bananas. Opening connections.

B

So the the properly we're talking about max http/2 connections per server when that's one there would be the current behavior. So we're not talking about changing the behavior of ATP to with a cheaper client by default. The current behavior would still be the default. When you increase this, you could have create up to that number of connections.

B

And then, when you had their number of connections, you would fall back to a current behavior, which is that first in first out queue and if you serve us so n max 32, then that's pretty much just yeah create create unlimited, which was my original proposal. But having a limit like this, as is fine.

A

So I'm not quite sure like what does this mean with a hundred the default value for the Mexican current streams? If I said this, to two, does it mean I have maximum 200 concurrent connections? Yes,.

C

Right maximum 200 concurrent requests, but it would be over to TCP connections, great right, I.

A

See and then basically, everything else would be cute after.

B

After, like a two hundred and first would be queued,.

A

And you said the default media where she will be shipping is one because the currents in here yes.

B

So that's to comply with the the word of the hb2 spec, which is to say you probably you shouldn't, create an additional connection. If you run out of maximum current streams, you should you should be patient and wait so.

A

Corey you mention, you talked to the Windows guys like do you know what they plan on doing so.

C

They they've already done it. They just have a pool essentially setting where you can enable an unlimited number of connections that will just open up if it needs another one. So.

A

They don't have queuing at all, let's just say like either lock after 100 or you have Sinnott effectively yeah.

C

We chose to go a little more controllable, we're out of allowing the user to cap. The number of connections is that.

A

Because they only care about Klein's or is it like I'm just curious like? Why did they go to down a different route than we went.

C

That I'm not sure of we, they do have a connection limit for HTTP one I'm, not sure, if why they didn't do one for HTTP two. It's probably because you get so many requests out of one connection that you would really have to be doing something crazy to open up a problematic number of connections. Mm-Hmm.

F

C

We are currently kind of investigating scenarios where the number of connections really matters, because firewalls and things like that just start dropping packets. If you go over that so making it configurable here kind of helps us helps our users stay within their own limits.

C

F

Thank you is there a scenario where you would manually want to control affinity between request and connection for life, doing drought, robbing or whatever algorithm we implement in HP client is acceptable default that nobody would want to mess with.

C

So the the implementation here will actually not be round-robin.

C

It's essentially going to try to fill up one connection as far as possible and then spill over into another and then continue trying to fill up one connection as much as possible in the hopes that the other connection will eventually drop when your traffic goes down.

C

Load. Balancing is a scenario that we considered it's it's something that we chose to not solve for now, because we think load-balancing is kind of a larger issue that we can tackle in other ways through some more sort of low-level api's than what httpclient does cool.

F

A

So that makes sense to me this there's one section where you talk about some other property.

A

Is this just yeah this one here? Is this relevant or is this some alternative.

B

Was there was the initial proposal, so that was a flag.

A

B

Like what is on when hgp, which is basically do do unlimited, so really that's just changed to now be an integer Ahmet and true is the max number of connections.

A

Makes sense you know, there's anybody has any concerns and I think that seems reasonable to me. So.

B

There was discussion about, should the property name have http/2 in it, because hb3 has it's not similar and implementation, but it's somewhere in concept of limiting the number of concurrent requests having fire a single socket so that their discussions right at the bottom of this issue.

B

I'm, not an expert, and that should be three I- haven't looked closely at it. So I I, don't know how much I can comment on whether there is a as appropriate for them and whether whether we should like what the correct name should be.

A

C

So HTTP 3 goes over UDP, which causes a much different load on things like network hardware and firewall things like that to me, it makes sense to kind of allow the user to configure both of them separately, but I could see if we were going for. You know maximum ease-of-use, simplest API. Combining them is an option. I.

D

Also vote for separation in part, because quick doesn't have the head of line blocking. They should be two streams out.

C

Yeah in HDTV three and quick, it makes much more sense for the server to actually set a limit of like ten thousand or something if they actually can service that amount of a server.

A

Sorry like it, when you say HTTP or quick like it, isn't that the same I should be to me include the.

C

A

C

Yeah they're they're the same they're two different protocols that are the same I, see.

A

Okay, so I'm as smart as I was before this meeting.

D

More than hp3 does because quick gives HP three streams. Basically, I swear as TCP doesn't give you that I mean.

A

From all of us hearing from HTTP so far that it should be three seems very different from HTTP what we currently have, including two, but it seems hard to think about what it does to our API surface. Like I mean I, don't know, I should be client was a good API is also you know, not super young anymore right and I wouldn't be shocked if you find ourselves writing a brand new API for that altogether, then so, I would kind of say like assume that this is HTTP to for now and if three comes along.

A

If we have hb2 in the name, we can split it or we can say yeah it's four to an upright.

A

G

A

C

Handlers are also not experts in the room right now. We actually have an implementation in HTTP client in Kestrel today. So the current API is are fine. It's really just about resource usage. For me, it going over UDP and not having head of line blocking, makes me think it should just be a separate sitting. Yeah.

A

I mean I'm officially.

C

D

That, like the same property, I think would be like leaky like yeah.

A

We might the question.

D

Is yeah dinner noise.

A

So we put it right now on the specific handlers like. Why would we not put it on the base handler and why would we not put it or I should be client directly.

C

Http client doesn't know anything about the implementation that it's using, so it could be using sockets and where it could be using the fetch API in in webassembly with blazer.

C

So it doesn't know, it really has to be something on the handler, and this is we've put all of the other configurability options on the handler to date. There's tons in there, okay.

A

I'm, you ought to really believe it doesn't belong in the base handler. It belongs on the actual specific derivatives yeah.

C

Then it would be weird if you could set it somewhere and it just had no effect.

B

So I've got a question about NIT standard and net five. Oh so currently the the Geo PC client target standard, two one and it uses HTTP client Handler, and you can see there and the GRP see usage currently on the screen. What we would probably do in the Chi RPC is, we will create a handle or ourselves and we will configure it. So there is no limit, so people are using GRDC and they're doing these long live streams.

B

New new straight you connect shion's will be created as needed, but they're like brings up the question about how how do we know that sockets, a should be handler, is going to be available and as the right and we use compared to HTTP client handler like? Will there be some kind of boolean test to check whether sockets a should be handler? Is the right thing to use or I'm just trying to figure out like will the future version of the GI PC client that uses this?

B

Will it be targeting the knit four knit five TFM, or will there be a new standard Phoenix and like what's the what's the way to determine whether to use sockets, they should be Handler and set this flag versus use this using a attribute line handler I.

A

Mean it kind of depends on like how do you envision the usage in the context of somebody who consumes your PC? Is that something they would pass in to you, or? Is that something that you do in the implementation of them? Calling API is in your end, I would.

B

We would do it for them, so when you create a channel by default, we would, when you create ABC channel, which is kind of like an abstraction over a connection internally, we create the handler for you and ideally we'd like to create a soccer's heb handler, which has the concurrent stream limit disabled and you.

A

Wind, but you would not take it like this. Basically, no, it gets what I'm trying to get it is like. Would you ever have a public API that would have to accept the sockets HTTP handler, probably not right, because you would probably.

B

Always be concurrently, let you customize the handler that you want to use but like if you're customizing it, then it would be your responsibility to say this line.

A

B

Thinking about the default experience right.

A

So that means you and I would like. Maybe just take an HTTP message center, that the client that the consumer can customize. But if they don't give you one, then you would manufacture one. So basically that means you have two options. You can either multi target. You can either say you know if a binary four net, five, when you directly new up Socrates DP handler setting this property to false and you would still have a net standard reference assembly. So that means anybody who consumes you from that standard? You know they.

A

You know the API service is the same and then the deploy for net five to just get into better behavior. The other option you have is you still have a single binary. The target is net standard. You know 2 1 in your case, and you would use reflection, that's also possible, but I mean you know what it's you know the you know. You know the names of the types you know the names of the properties you would just effectively. You know.

A

Do the typical light up that we do where you probe for the type probe for the property and then maybe use a delegate to to cache the invocations? We don't have to go through a reflection every single time, but that would basically the two patterns you would be using multi targeting might be easier because you know it's not crazy reflection and it's all static and you consumers don't need to know this. It just works magically for them. Yeah.

B

I wonder blazer is the main scenario I'm thinking of because it would be using handler, which would internally be using fetch rather than sockets, but I'm wondering like a very gonna a vague. You know, target nip five yeah.

A

So the thing with.

B

A

You see a blazer like I just played over the last week, a little bit of it, so their service, a blazer and client-side blazer. So my mental model was probably not the best one, but the way I understand it is that we don't have a TFM, that's specific for blazer, but there will be a rid that is specific for them, so their own implementation can totally have a binary that is different for what the client would get in blazer right.

A

The actual thing that you sent to them to the browser to execute and that one would basically do the right thing and go through the fetch, API and but basically logically, they would both be targeting that v, both the client and the server. And so basically you would just you do the differentiation by a writ specific winery's. Okay,.

C

It's my understanding that the Blazer clients fed blazer will just instantiate HTTP client using a different handler, so it won't even be using either HTTP client and ler or sockets HTTP handler yeah.

A

It's the fetch, Handler I, know whatever the thing is that we call it right.

C

Yeah it'll be like webassembly handler something like that.

A

Yeah something like that and the downside is like you: don't want to generally do that for other people's projects, because targeting wits is a pain in the ass. Given that we basically have no tooling so I. Think in your case, you would get away with just doing the normal multi targeting variety FM, because you don't really care whether you in the browser or not I, suppose right.

B

No, as long as as long as things were as long as when you make the call, the call wire client happens right and.

A

Then Cory we might be like how does it? How does it connect if I do new hgp client in a blazer client, app I should say in a webassembly blazer app like I'm, not passing in a handler like to the default constructor? Will the manufacturer the one that takes the fetch, thingy yeah.

C

And as far as I know, webassembly has no support for sockets. So I'm not sure what will happen if you actually instantiate a sockets HTTP handler so.

B

A

So me I, guess worst case. What you would do is you would just say you know. If OS platform is browser, then you say instead of saying new, so Cassandra, you would say you know, new I should be client or whatever, and then you just use the default handler. That comes from that.

E

It's actually a related question based on the fact that you don't necessarily have control over what handler you're using. Is there ever a scenario where a caller of HTTP client would want to know whether the request has been dispatched immediately or whether it's in a queue waiting to be dispatched? Is this important for to your variables there.

C

Are important scenarios for that and we're working on other API is to solve it. Okay,.

E

Great because it seems like a dovetails with us pretty nicely honestly, yes, absolutely is the idea that that would come in perhaps in the 5o timeframe, along with us, or is that future and.

C

That is future we're going to be launching an experimental package with some new api's for that and depending on how they go, merge them into net six. Okay,.

E

Awesome I'm glad I'm glad that you're thinking about that okay, cool thanks.

D

Sweet James with gr PC on the you know, client side blazer. Will that be true, gr PC or does that have to be like translated like to be like gr PC over, like I, forget what it's.

B

Called right, but you.

D

Can't be still, and so.

B

It's not as true chappies a PC, but all the VAP eyes are the same. The user experience is pretty much the same. The only slight difference as it could be over HTTP one one or two and the content of the messages slightly different to include the hitters. Okay.

D

And with G our PC web, if it is over HP, it's still like a request for stream or whatever. So it's similar in that regard, yeah, okay,.

A

So unless it any more concerns, I would just to prove the API as is, and then we can go into the weekend.

E

C

We need to give.

E

You a gavel so that you can, you know smack the desk. I can just.

A

Fist bump the desk.

B

Api review to a close.

A

Will the gentleman yield it's hopefully a phrase I never have to use alright, so I think this. Is it for today anything else that we need to look at because we have 20 minutes left. Is there anything else James you want us to look at now. Is your time.

B

I've got one night where I have like.

B

A

B

Okay: three one one: nine eight three.

A

One one: nine eight.

C

I'm doing good I.

A

Guess there's no proposal for an API but like this hat.

B

There is there is where, and my post and my my comment this one here.

E

The one that's on the screen, yep.

B

So I haven't looked at this issue a little while. Let me just go back for discussion so.

C

Working team gone through the approval process with this hit. I can't recall if we added a comment or something saying like. Is it marked API ready for review.

A

No, it's marked API suggestion. Okay,.

C

Yeah so then we need to talk 'ln with the networking team about this before it comes to this meeting. Okay.

E

Did anything else James you wanted to to sneak him.

B

No I think that was there was the only one and I still want to talk about connection abstractions. Is that still going on I.

A

Think that is another meeting that we have doesn't well, there's one more right.

G

It's not controversial at all.

B

To us, no I don't have anything else. All.

A

Right then, the meeting shall be adjourned or whatever. The phrase is alright, then enjoy your time down there like you're, the only ones who are out of couvade I, think.

B

A

Don't hurt yourself to bed in the gym.

E

Night for the rest of us enjoy your long weekend.

A

Exactly Independence Day, alright see you guys bye, bye,.