Internet Engineering Task Force 100, 14 Nov 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IETF100-TAPS-20171114-0930

Description

TAPS meeting session at IETF100
2017/11/14 0930

https://datatracker.ietf.org/meeting/100/proceedings/

A

Good morning everybody we're gonna get started. Just we do have a few remote participants, so please be sure to use the microphone when you're making a comment, so they can know what's going on.

B

Hello welcome to the tough session for idea of 100, if you're not here for taps. Maybe you can think what you want to do. You can leave your stage, so this is not well not very well.

B

If you are not know what it is it is, uh you can read about it on now online yeah, as I said, PC stops working of I took hundred meeting, and my name is Jo Jo McArthur and co-chair to our own fault. Erin.

B

So this is our agenda. huh It looks like so we'll have chairs update. Then we have a couple of jobs that will be presented and then at the end we have like ten minutes for discussing on Saturday and three anybody wants to bash this as endure.

B

C

I'm willing to have my section be shorter than 25 minutes, because it probably doesn't need to be that long. So we can have more time on the Charter at the end, I think we should have that yeah.

D

B

See, okay, so this status we have two drops that we have sent in artist. Editor queue it's waiting to so so congratulations and I would like to especially like to especially thank the authors and the reviewers of the draft job. Well done, thank you on the Charter random we have between the item 9900. We have adopted this draft, so I think we are going to have some updates on that. One. On charter item, 3 we're pretty good in a good situation. There are a bunch of drops addressing these socket soccer.

B

Intents happy eyeballs, and there is a guideline and trapped on. How do you do resting and there is a new draft from neat project right, so we're pretty much in very good shape there. We have multiple document addressing Chapter three, so in this in this meeting, actually we're going to talk a lot about about those traps and how we're gonna proceed with that one. So that looks pretty good on the security topic. I think we had an action point so updating the Charter.

B

We have some such detects that will very soon we'll be doing discussion discussing and there will have a solid draft. Chris told me, you guys have been updating that one and you wanted to have some review from the on the working group. So you want to say something more about that. Yeah.

E

Chris would Apple we didn't make up its in time for the tracker, but as discussed in the last medium we're trying to go through and just add, as many more protocol descriptions that we can to get it pretty expensive and cover all our ground so yeah. When we do update it will send an email to lists, and hopefully people can read it and give us feedback and see how it goes from there. I don't know what we have to do with regards to the Charter update. We talked about it during the last meeting.

E

I, don't know official what the official process is for updating things, but okay.

B

Yeah, so so this is how it looks like so yeah so I think we're in a pretty good shape. So let's talk about these security functions, so we.

A

Had a discussion of the last meeting about whether the Charter permitted us to include transport security protocols within the scope of the abstract, API I went back and look at it and it doesn't specifically exclude it, but it seems like it's a good thing to have charters talk about what you're going to do, and that was the direction that we got from from our area director.

A

So I made a small hack to one of the paragraphs in the Charter and basically basically the first thing you do is you set up your new keyboard, and so we had a bunch of discussion about TCP, because TCP Inc was just kicking off when we were starting off here and that's not really relevant anymore, and so that's a little bit of distraction, but that a sentence here that says tab support for security services provided by existing ITF transport protocols and that's a phrase that we use for the rest of the scope of what taps is doing is in scope.

A

However, taps is not chartered perform detail of thousands of security aspects of transport protocols, so the so we kept that original statement, which is to reassure people we're not trying to do new security stuff here, but we do want to make it clear that security related functions are in the scope of what a taps API could do based on a quick interaction with Spencer. He I got an early thumbs up for this.

A

If the group folks in the room are comfortable with it, we'll send it out on the list and then assuming that there's a, however, that converges then we'll go in submit that as a charter update and now our area director has something to say.

F

Spencer Dawkins, responsible area, director, Aaron and so head and I had had the conversation mostly about the beginning of this. You know what that would say and not so much about the part. That's going away about TCP Inc are you thinking. Are you question is? Are you all thinking that that has been overtaken by events or they.

A

F

That there's no work to do I.

A

Think that it's basically a subset of the work that's described in the new sentence, that's added to the first, and if we left it in there, then it just makes it better kind of confusing and charters are hard enough to read as it is dead.

F

Makes more that that makes even more sense to me than what I was hoping. Thank you so much.

A

So you're, alright, with that Chris Kyle, you got a comment. I.

G

Do Kyle Rose, so the the discussion that we had offline about about there are properties of security protocols that we may need to abstract away and provide as knobs for taps users. So, for instance, for token binding needing to have a channel binding is something that the security layer needs to provide. Does this language preclude that.

A

My intention of framing it that way is to allow us to have that conversation. I didn't want to try to do it before writing the Charter I did want to allow it to happen in the room. Is that you know, ideally, if we can come up with, like some general statements of how an application might ask for or interact with security functions that are integrated in the transport layer and I? Think that what you're time out we took combining is part of that. I think that this language permits that okay.

G

I mean I just say it's like sort of the fine line of we're not chartered to perform detailed analysis of security aspects of transfer protocols. Is we need it? We need the security, you know any security layer that taps is going to make use of, to provide a channel binding. Is that detailed analysis I see.

A

You're pining I didn't really want to touch the earlier text because that was sort of the let's not alarm the folks in the security area statement, but but I interpreted that as a we're not going to try to do anything related to cryptography and or new security mechanisms.

G

That's that's the intent right.

A

That's really sort of security, design and engineering, where's I think you're, right that what we are talk about doing is some analysis. I mean.

G

It seems like, like the the draft Tommy and Chris's draft on on the survey of transport of the transport.

H

Security protocols.

A

G

A

How about why don't we replace the word analysis with design.

G

Perform well, let's maybe think about that. A little bit. Okay,.

A

Well, yeah think about that I mean we don't need to make a decision here, but your points, a good one and and I- would rather have the charter, be correct, then not touch things just because it could be okay. Thank you, Amy. It's.

D

E

Yeah I was just gonna, say: I interpret the security services as the knobs that we want to expose and yeah analysis maybe could be construed as potentially creating things that, and we don't want to touch that or open that kind of worm. So maybe just simply say that we're not doing any. Like you said, security, engineering or creation of transport security, things yeah.

A

um So, are you comfortable if I replace something along lines, we're placing the word analysis with design and then we'll see whether Spencer gets any pushback from the isg and worst case? We leave the existing wording in there and do what we're going to do. Anyways, yes, I, would like design better I. Think, okay, Kyle you're you're, taking notes right tails. Not here yet. Can you write that down? Thank you.

A

It's not sure you got something: okay, okay,.

I

A

In it, you want you, some security engineering. You have to leave the room.

J

Hey all right, this should be smooth and easy I hope it's a quick update about what I did to the min set document.

J

In a nutshell, oh yeah, it begins by saying also because I changed the sequence on this slide. Sorry, first of all, one thing that I added to this document was to have early configuration so that we can avoid issues so Francis. If you read the main set right now and you would say well, okay, this is how we'll implement, isn't associate and the protocol and let's say, let's not build the tab system. Let's follow some of one of the API proposals. You may end up.

J

You know making a choice for UDP based on what the application wishes and then the application says. Let me have reliability and that wasn't a good thing. So you know a tap system needs to have some information early to make some decisions, and there is a proposal in there on what, for instance, such a decision tree I called it. The decision tree could look like, but this is really only written as Nix.

J

So this is not that not as systemically arrived as the rest, like they're back I want I, don't want to say: I can prove this. The ideal decision tree based on all these things. It's an example of how you could avoid cornering yourself, and then people asked at the last ITF to consider. Also you know, supporting UDP and like discussing you know what would happen or like how much which of the things would work with UDP or not and I've done that and make it clear that this is not an API proposal.

J

That was a request from a couple of people here. So I wrote, very explicit text. I considered, writing don't be afraid of the main set, but it isn't an API proposal. It's just you know a more clearer, simpler, coherent way of writing the services up. So this is why the test is kind of form. I. Think it's clear enough. If you object to any of the text, there is not clear enough. I've I can repeat the sentences three three times or something yeah I think you will.

J

Okay, so the example decision tree I'll just present it because it's in there I'm not saying this is perfect. Anything you know very happy to get feedback on. It's just an example, but you could early on you could I mean this would actually, at least if it seems to me this would cover at least the things that are in the main set, and you wouldn't even call yourself on them. You could ask for some form of reliability. Do you need this if node and you could I'll use all the protocols that are available?

J

You know, based on the upcoming next requests. You could then ask.

K

Is any of the following useful.

J

To the application, this is just looking for the set in the min set on specific checks on coverage, specified checks and coverage required by the receiver, and you can choose between you to be light or UDP, and if you do need some form of reliability, you really should be using SDPO TCP out of the set of existing protocols there.

J

So there is a disclaimers that we will call TCP. You know with a TCP replacing vampy, TCP or T or TCP.

J

hmm Said you have two following useful handing over a message to reliably transfer before connection set up. That's something that for instance, SHP doesn't do, but at the TCP does suggesting a timeout would appear notification of excessive retransmits notification of ICMP message: arrival, that's the stuff that TCP does and SCTP doesn't. So. In this case, you will be having TCP instead of SCTP. Otherwise the recorder preferable.

J

You know full set up and then you can make your decisions. Possibly some stuff is missing. I, don't know I, don't think so. I mean I, went, went through the list and tried to basically cover all the stuff that is in the remaining list of the min set at the end and sure make sure that there wouldn't be anything that you were pick honoring yourself against.

J

This is just an update of the interface description. The description is pretty much the same. That I presented the last time, except that I marked some things with being impossible to support UDP fallback that you know doing this over UDP. This would be the configure time mod it because obviously TCP doesn't have timeouts. It doesn't have closed in that form. It has some sort of an abort.

J

In that you know you can tell that you don't want to use this connection state anymore locally, but close comes with the semantics of a message to the other end. To tell the other end about it.

J

Tui, repeaters and UDP doesn't do that and then the parameters of send frame like choosing reliability, choosing all that or not order, choosing idempotent that stuff that UDP doesn't do and if you, okay, yeah so I guess add important, is actually a mistake, because that was about falling back and that doesn't actually you know just because they specified that it's unimportant doesn't mean that you have a problem with it arriving only once so that is a mistake, but I think reliability, and all that you know requiring all. That is a mistake.

J

Yeah some of the notifications, a timeout notification close about notifications. They wouldn't exist in UDP and the rest is just the old slide. So this hasn't changed.

J

So that's what that is. You know this was just more efficient, condensed way of writing transport features. It isn't before post APR repeated here. A your obsessive, very explicitly. I did ask about fallback to tear this, but I think this answer that I got from Tom animating. This I I mean this service to discussion. In my opinion, I, don't I think it's I agree. You know I, don't think that's the point in discussing it fall back because it should be something that's explicit anyway. It's a it's a choice. All the way.

J

I agree with that easy.

J

Yeah well, what else do you guys need from Fond du min set is like just the end of it already I.

L

Prank Rommel: is this an API specification? It sorry.

M

Sorry, sorry, sorry.

L

So there's a there's, an interesting question on this slide. That I think has nothing to do with this document. That I just wanted to raise up and say we should think about, is so there's an implicit assumption in um sort of the way that you've ordered the protocols and then set that like there's UDP and then there's TCP in the nursery you've ordered them essentially in order of additional features that you can get with them and then there's like TCP.

D

L

The floor, and then UDP is the subfloor and from a um from the standpoint of you know, you have a feature set. You need to meet. It I think. That's absolutely a right way to look at it. There's this second sort of externality, which is what can you get through the network which we should think about somehow and this when you talk about it, fall I can t OS or HTTP it's like yeah. You might have to go, that's like so it.

L

The order is like more complicated services with less complicated services it could be in the subfloor is TLS with HTTP I'm going to encrypt it, and you know let you figure it out, and then there is TCP and then there's UDP right in terms of what gets you the network yeah that might be interesting to capture somewhere, but definitely not in this document. So I think this is getting pretty close to it to done so thanks, okay,.

C

Tony Pali Apple, so one thing just on what you would started out with I think that yeah, it's definitely good to emphasize that the configuration of what you might want to use out of the features does need to happen at the beginning before you end up creating things making that really explicit, because that's essentially a something that we've learned by doing the min set, which is something that any API will want to build on top of this needs to take into account, essentially says all of the feature space you want to use needs to be defined beforehand, because if you end up saying I want to do this now halfway through the connection that could be really bizarre, very awkward.

C

So Brian did you see the email I did sent out two lists earlier today? Okay, because you essentially repeated exactly word for word what I had said about this, so.

K

C

We didn't talk, I swear right, so I think I, like the document I, think it's a very good survey of this I think the main substantive change I would do to. It is just to kind of remove all of the references to fall back, because that's much more of a normative time-based claim about what we prefer, what the ordering is, and maybe the order is what you say- I think oftentimes. It would be for feature set, but there are other cases in which we may have different preferences.

C

There may be two completely other protocols that we have in the future, like quick, blah, blah blah, and so just for those who have not seen the proposal. Essentially, what I was saying was for the feature lists where we have you. Okay protocol is SCTP and fallback to TCP. Does this fall back to UDP? Does this instead just for each feature simply list? These are the protocols that implement this.

C

These are the protocols that can safely ignore it, and these are the protocols that will not work with this and the idea of what order we want or if even if we just choose one, is left to a later document. Something like the happy I abolish document, something like an API document.

A

What's supposed to be in dr. as.

J

A service to those who haven't see my reply to that email: I'll, repeat that I agree with everything.

J

F

Easy so Spencer Dawkins is a curious. Individual I actually got up to say about 80% of what Tommy just said without me, actually saying anything which is even better than getting somebody to say something without sending them an email. This is why I'm an area director I'm just yeah.

A

So I have a couple of questions. First,.

N

A

What is on the list of things that you believe have to happen before you can say this document is complete.

J

Incorporate Tommy's commands give it another proofread. Okay,.

A

So it sounds like there's there's.

J

A

Content that you think needs to be generated, other.

J

Than Michael, do you stand? Do you have any okay can I see.

D

A show of hands.

A

In the room of the folks who read the latest version, okay, so it looks like about. Maybe eight can I see a show hands of people who have read the min sets in the last six months, a version it's more like 15, all right, okay, so that looks like.

B

A

Are reading it that's good, so let me just ask the room: is there any comment on so Michael taking the making the revision that was discussed this morning and then taking the document to a working group? Last call: does anybody have any concerns about that that they want to discuss.

A

Okay, let the minutes show everybody loves. That idea is tail here.

I

Yeah, oh okay,.

K

The remote, the negative Microsoft um is there any other active decision other than characteristics of the transport layer things like do. You need this to run on a very small constrained device. The reason I asked that is that, once you add that into the possible axis of decision, then other quarter code transports may come into play like coop specific.

D

K

From this point of view, or from a point of view of the classical transport layer, its application, but actually it's being treated as a transport in many communities- and you know some of those internal to like yeah or or other transports- quote-unquote transport like that. So it has that been explicitly. uh You have to be out of or.

J

Some reason well I mean this is so the men's that is based on the preceding document, which is the users document which also relates to this RFC. That gives a survey of things that we decided here to call transports. I don't know. If co-op is the part of that RFC I, don't I, don't think it is so I think we just at some point said we're gonna analyze. What was it TCP, UDP UDP light, MP TCP, leopard SCTP theäôve was the list that was just well.

J

Anta means that it's just a matter of taking taking bodies, and these are the document and saying you know how do you do fall backs and how do you, which of this stuff? Do you really want to expose okay? So maybe for a second.

K

Everything another protocol is.

J

Now, a bit of but.

K

Yeah because I did hear somebody mention in the future, maybe we'll consider quick, which is obviously missing in this one but yeah if it should be added in the next revision. I suppose at that time, it's something built on top of UDP, not very different from what co-op can be or the.

J

K

At that point it might become.

J

My hope I mean this is not explicit in the document and I would like to have at least a paragraph in the introduction saying this. My hope is that, because SCTP is a very, very broad protocol, it does a lot of things that you're not probably gonna very gonna, get very, very many things from newer protocols that that needs to be the D to be in the API that this document doesn't cover, so might be, won't feature here or there, but not a huge amount.

J

And so that's that's also. The case for quick-quick isn't a part of the end of the analysis, because when when Creek is still even now, it's still a moving target was this makes it hard to analyze.

A

J

A

Think that let me I think that the goal the main set was to identify the minimum set of functions, that a taps API should support to allow applications to get sort of basic functionality of what the existing idea protocols provide and the idea being. They need more specialized stuff that the protocols can provide. Then you do it outside of the API. It seems to me that if Co app is a candidate transport protocol, the way that you're thinking of using it yeah.

K

A

Useful test of min set is to say, are the set of functions that min set says you have to support things that Co app can provide and if co-op, and if there are answers where it's no, that might be an indication that co-op isn't really a fully functional transport protocol. It doesn't mean sort of the minimum bar of what you can do for a transport protocol or if the answer is.

K

A

You think that that it is acceptable, then maybe we've got stuff in min set that doesn't belong there, so I don't think it's so. The first step was like, let's survey, all the existing transport protocols to try to figure out what the functions are and then distill from that. What like the really necessary functions are, and then we come up with sort of the template and now, if you're going to build a new transport well, this is these are the things that you need to do.

A

Well, maybe co-op is doing that, and so it would be great to use that as a test form in set to say. Oh you've expressed things in a way that don't make sense to us or there's stuff, that's missing or stuff that we could never provide, but we think it's sufficient anyways. So that would be a really nice discussion to have on the list. Yeah going.

K

Through that, the motto I I have, in my mind of how this could work would be similar to motto: applied: ni RTF, like I, CCR, GE and CFR GE. They are sort of like places where you go to have a crypto advice, which applies to many potentially several working groups. Tonight you have similar for congestion control several working groups in idea of mission control issues.

K

This could be sort of that source of I, guess, advice for people work in different transports, and so you should probably go off and shop or expose this work in explicitly things like poor work looks like core and and others that are working on where to sequence, while others that are working on on transport issues. It's.

A

A great idea, I agree: you seem like, maybe something we should bring up it, dispatched where, where what's the forum where it would have reached the coop folks other than you happen to wandering in the core.

K

The core working group for that particular protocol is.

A

There anybody anybody in the room, who's active in that group, Hamas.

O

Was she consulting yes, actually I, look at prom, literally flipping it over I'm, really following this and very interesting, for instance, how would I run co-op over can FD in a car or run it over SMS and what sort of a sim and my other need, how thin of a sim I'm going to need such that I can be the co-op application can use this API and not know that one side is on.

O

Can FD and the other side is this, this co-op bus server up in the automotive cloud going through a canopy IP gateway where it gets stuck on UDP or whatever needs to get stuck on to go that way. So that is what my looking at my my layering history to do this, that how is this API now gonna make a possible where I'm working with Macs that can't possibly run a transport for the limit either, because if I is too limited or the Mac is too limited, Aaron probably would so on.

O

You know, that's our stuff. So does this give me? Have you done this completely enough such that now we can say co-op, you don't care! What the transport is, you just call this API, it's a matter of the shims underneath which make this decision. That I think is your goal, so.

A

If you looked at min set to see, does it doesn't I've not.

O

Looked at the last graph, the previous graph, it was not there. This last graph I've been very busy doing some like crypto stuff um work, but I got to get back into it. My will that.

A

Would be a great reality test exactly.

O

Yet that is my personal view and I'll be giving feedback along the way on this, as thank you right.

C

Tommy Polly, Apple, yeah I, think this discussion is very good. I mean I think for the purpose of this document.

C

The focus on the existing transports that we've already surveyed is the right way to kind of distill that set, but as we go on to discuss API stuff, of course, you know any abstract API for taps should incorporate any of these other pseudo transport options to abstract the transport layer away sufficiently and I could imagine essentially using this min set format or wherever we come up with here as a prototype for future documents that essentially define the mapping of X Y, Z, other protocol transport or a layer two thing as something that could be represented beneath these basic functionalities.

C

So we may also want to think of that of how like we could end up with like mapping documents in the future that look like this. You.

A

Nice to have one for quick, yeah.

P

Me I could live in. So a completely different comment: I've been thinking about how to map this, to post, sockets right and in post sockets. We have like those boxes which all have some kind of interface, and then we also have this one magic box, which is like the configuration rate, and you have separated the interfaces. You have into think that are related to connection establishment and things that are related to data transfer, but maybe it would be nice to take out of both of those categories.

P

Everything that's rather something that is configuration and it's not essential to transferring data or getting the connection set up and have like a surplus in there, because that would kind of make easier to.

J

Make it to configuration so afters maintenance category? That's that's I, call it maintenance, but it is what I met this with configuration stuff. What's.

P

In the maintenance category, I, don't know it by heart.

D

A

Is a subset of configuration.

J

There's configuration that.

A

Gets all you want to have you open all connections.

J

Set up to add on to call the whole same configuration.

M

C

um Tommy Paulie so Michael in some of our emails about post, sockets and I. Guess we're going to get to that presentation.

K

C

So we can just transition. I had been bringing up the like the idea that there were pieces of functionality that are kind of like core transport functionality, and then I was saying like and then there's kind of, like the protocol, specific things that seemed to be fitting on the side and perhaps I think that's the distinction that mirror is making this like things like I, want to initiate a connection things I want to accept a connection. I want to read: I want to right.

C

Those are absolutely fundamental things and they're certain maintenance, things that are absolutely fundamental to the live interaction and then some of these other things like you're, saying I, want to disable Nagel I want to do this, I want to set the timeouts those, like anything that you say this needs to be expressed as a requirement before the.

K

C

Gets created, he is kind of that configuration bucket, so you could add a distinction there that this is. This is something that they like, for example. No, so no application would need to say by the way, I want to call initiate connection they're all going to do that, but they will have to tell you I want to do this functionality or I. Give you this functionality for me, so that needs to be said up front as a configuration right.

J

So yeah I mean you need to distinguish between what needs to be set up front and not but other than that I. Don't personally like drawing a line and saying this is kind of not so important functionality, this core functionality, you.

K

Understand that number act.

J

But we're back to the old ways right you can send and receive, and you have maybe the things that and you miss out on some services that would be good to have like, for instance, in your example that was to check some configuration bit. I, don't know on what basis you would say. This is not the core functionality right this, because it's tasting it's.

P

Not really about or.

J

P

More or less important, it's really about fundamentals, so without dysfunction you cannot communicate. You need to call it some at some point in your way. Through all the function calls you do, but like.

Q

You know, then,.

J

You have just send receive pretty much.

J

Unusual or like I'm I, don't see the point. It's.

P

The point is to to map to to get the mapping between what we do in post socket and rise.

J

But I don't want to change min set for post occurs. I think you should change post office for min set because well because the mins that describes the functions that are there right if.

R

They say these.

J

Are not call things, and then it just makes the mins that just I mean poster is just just. Do the call things? That's not the right approach, no.

P

No, it's like it's! It's not that we're not doing it it's just like in which box they go right and like if we change, if you change it and post okay, it's a completely different proposal.

J

Sorry I mean I, read: post sockets, I know the boxes yeah.

P

J

What's the box so.

P

That there is a box which is called configuration, it's called configuration and like and I think it would be important to know which function should go in that box.

L

10 minutes over the.

A

A

Keeping keeping on.

L

You know my tradition of restating things that Tommy has said so there's I think there's a useful distinction in the distinction that you're making with respect to there are things that you need to do before setup and there's things that you need to do after setup. That's a very clear line in sort of a configuration space. It's like you cannot reselect a protocol. There are also things you do like you can't um once you've you've decided, which transport port you're going to use in TCP.

L

You cannot change that the destination transport port without having a new connection right. So these are. This- is sort of a connection set up versus connection maintenance side. I. Think that sort of the to rephrase what Miria said I think that the disconnect is this is necessarily bottom up. Right, like min set, is, is designed to be bottom up in order to get us a yeah.

D

L

So let me ask my Jo question again: is this an API specification and let me answer it? Yes, it is. It is a specification for the API in which you implement API. Is that you hand off to the application? Yes.

J

Swayne, this is how I there's always you could easy to write posts? Yes,.

Q

Oh yes, so, for instance,.

J

When we say that certain things need to be known at the beginning- and you can you.

N

Just said, you cannot change what's well.

J

With post sockets, you might be able you create, because.

N

F

Can where you just create a new than.

J

You guys tutored on a connection open your new connection, whatever I.

F

N

Me yes, I right.

J

Yeah, the bottom layer they get basic they want to build is on is the set of the services that the.

L

J

L

That's what that describe.

J

L

Pro sockets, essentially any post hoc encirclement ation, is essentially an inter layer. No.

S

I think I don't think that there's always a clear to hire. They.

L

S

Like so why I don't see it would be, it would be so I think.

L

What Miri was saying is that it would be useful if the configure a sort of a knobs that we have here were organized in the document in a slightly better way. But, like fundamentally, you know most likely. Postdocs and implementation is an inter layer, and you can I mean just throw another layer in there and adapt it so I. Don't think we have a big problem here. Right, I.

J

Mean I'm not religious about this, you know its distinction, I made about maintenance and actually set up and tell whatever it's. So we can't discuss about how we organize the texture briefly,.

A

T

Think I go confused, I.

A

Like what Michael.

T

Said and I didn't quite understand the disconnect with the pulse sockets but I guess we can see that up on the list. Yeah.

A

Well, we have an agenda in amman, po socket, so maybe we will have an opportunity to revisit them. Thank.

E

You, my god, I.

J

Think we're in the.

A

Folsom yeah they right we're already there. Yes, so, okay, Brian! Now it's your turn at the microphone and Michael can ask you questions.

L

All right, um hi, I'm, Brian Trammell, actually, for my curiosity, who here has read this document: okay, a few people good good, so I'm, assuming that you've either looked at 0, 2 or 0 3, and that you're kind of up to date. With this, this is not the hole we've already given the intro to post sockets like five times. This is the changes. This is the detail changes. So this is changes since 0 1, because 0 2 was I.

L

I accidentally submitted a draft that had a whole bunch of editors notes in it that should've been in there. So so it's like 0 2 and then like 30 seconds later, 0 3 came out and it actually breaks the data crackers diff tool. It doesn't know what it's supposed to difficut, it's pretty cool, um so you may recall from last time we had a quick framing discussion about how scary policies are. It turns out. Policies are still scary, so we decided to rename them and call them configurations.

L

Well, no, no, no! So there's actually there's there's a there's. A real reason for doing this is is policy in a lot of contexts implies something it's much higher level right, that's much! You know. I would like to have a connection that works anymore or something like this or I'd like to have a connection that works and I.

L

Don't want to use this interface because it costs too much unless I'm in this situation right- and this is like at a much higher level than one min set- is and then to I think we can realistically talk about at this point right. So what we can realistically talk about now are are much closer to the min set configuration stuff. So what what are the aspects of transport protocols that I'm willing and able to use? And then there's also this explicit separation where we took so in? Let's go here. This is the slightly updated thing.

L

So this is the magic configuration box over here on the side we put over on the side, so you wouldn't notice it. The this is a protostar protocol stack configuration which configures a protocol, stack instance which implements a transient, and we basically took all of the things that you might want to push down to a specific protocol. Stack instance and stuck those in a protocol stack configuration, so you could say, for example, my my configuration requires a reliable transport protocol.

L

If you happen to select TCP here's, what I want you to do, for congestion, control and timeouts and whatnot there right. So this is where the protocol specific tweaks live. So this is why I'm I'm not worried that we can take everything in min, set and put it somewhere. If it turns out to be specific to a protocol or a subset of protocols, we can just push it down into the PSC there's a new most of the work here that we actually got in for this Rev is in the API dynamics section.

L

So this is some ASCII art that attempts to show how all of this stuff fits together.

L

This is we used to have some examples which were just a sketch in go code and I got a pull request from somebody saying your go code doesn't compile like that's not the point, so we basically said: okay, let's work all this out in something that's clearly not meant to compile and then we'll bring some examples in an appendix after we have prototype implementations that we can that we can use that will actually have building code in order to head off those sorts of comments.

L

These are the abstractions like I, said before nothing. Much has changed here other than we've got a little bit more detail and understanding of what's going on with configurations. Does somebody want to interrupt me here.

F

Spencer Dawkins as a curious individual is actually on the previous slide. If that's okay, so the thing I was curious about here or was, if you were thinking that the PSE I guess would be something that's required or something so if I say I want a reliable stream and either end I, don't care about the details, you would be able to run with that. So.

L

There is an unwritten in this river, the document there is an unwritten principle which we need to write down. So in some of the discussions we've had before this meetings or in the hallway is for come clear that there are a few prints. There are few principles that we've kind of had in our heads when we're designing this, we haven't written down yet that we need to write down, and one of them is, if you just go here and you say: I create a carrier with initiate and I start sending messages at it and I.

L

Don't do anything. It does something City great. So you you don't have to create a gigantic block of XML or something else horrible and poke it into the Association before it works. You you, if yeah the it's a hard requirement that um initiate go just like in in in the current sockets API I. Just you know, I have some information and I just I in order to create a socket I create the socket and I go. That's a hard requirement of this arrangement.

L

The protocol stack configuration is really think of it as if you'd, if in a current operational situation, you would go and put something in at ccTLD in order to change something, then you're messing around with the protocol. It's that configuration, if you wouldn't think you're not thank.

F

You that's very helpful. What my concern was, you know we've got like I, don't know 20 or 30 years of applications that know way too much about the transport. Yes,.

K

F

I'm hoping that that knowledge doesn't get stuck someplace where the application could still find it. How and let me just still just continue to say if you're, if your point was that the you know the default, if you don't dork with it will work, you know it unless it won't work for you. You know something good will still happen, that that means that most protocols, both most applications, may not end up with a lot of protocol, specific knowledge that they don't need that they don't need. That would prevent us from doing SCTP deployment.

F

You know in not that, but you know.

L

In G, here's what we hope we hope and I actually Tommy has a little bit of experience with this, and it seems to to not be an incorrect hope. So far. We have this durable state here, which holds information on a per path basis. Transport protocols generate an enormous amount of self measurement information and can be used to essentially figure out for a given path in a given situation for a given carrier from a given application protocol. um This is the right thing to do right.

L

This is so you know we refer to configuration as the magic box. Actually Association is the magic box. Configuration is the box that you use in order to restrict the magic rate, how hard it is for an application to poke a PS en I. Don't think it can be any harder than as ECT LTE and set saw copped, but um if we want to make that harder, I think that's probably implementation specific.

L

You know you have to break a hash and give it a Bitcoin, and then you get to do something, um but I didn't actually mean that just clear but but yeah so yeah. This is exactly our goal. Exactly our goal is to you is to obviate the need for a lot of this dis around with socket options, because what you get by default is just better yeah.

F

And and thank you thank you for a very helpful answer and it's just because you said what I was hoping. You would say that doesn't make it right, but that you said well, I was hoping you would say, Thank You,.

C

Tommy Polly Apple. Just to that point briefly, I think one thing that Brian said is I. Think it's key for these protocol side configurations is that I think it needs to be expressed in the API that this is. If you use TCP, set this weird wacky option, if you use se TB and I, think it would need to be clear that taps may choose whatever it wants, and these are.

L

Just extra knobs, if you're a weird geeky TCP person, you could certainly put something up in the configuration that says: use TCP you create.

D

If you are a one-sided.

L

Oh- and this is another one of those principles that we've not articulated is it has to be possible to use post sockets on a single side of a connection and have just circly sockets on the other side of the connection. So in that case, either taps will look at the association information and figure out. Oh okay, I'm not dealing with something on the other side. Oh and another principle that we have is that you shouldn't have to implement. You must not have to implement transport protocol external signaling mechanisms in order to make this work.

L

This is not. This is not like dynamic network stack negotiation, because every project I know that's tried to do that, got to the dynamic network, stack negotiation, part and said: oh.

D

L

We're out of money, we got to go the so another one of these prints and that would basically, if you're, if you're, really in this situation. You know in that you're in that situation, you're always gonna, be in that situation.

L

You can always just poke something down into the thing that says: use TCP and again it's it's probably implementation specific as to how you express that, whether you have to trick it into doing that or whether you can just say like with like with um get out or like the the old get out our info hack, where you had to like you, know, get out our info and then poke IP prototype en to it in order to get it to us. Stp yeah I mean this is we?

L

This is the magic box, because that's the place where we put to do all those tricks.

C

The then the one other thing I actually came up here to say as an author. Another change in the draft just for people who are reading.

C

It is around messages and the fact that, because we like the mist of destruction but there's the whole problem with streams and like if I'm just going over plain TCP, what does my message mean and the solution that we came to there was that each side of a TCP stream is just one really big long message, and so it means that the API for reading and writing messages may come in multiple chunks and I'll. Just say at some point.

C

This message is done and it's kind of degenerate, but it allows us to put everything under that umbrella right without having to like have.

L

Yeah we had this, we had this other hack before that was worse. So this is the better hack.

M

Praveen baddest women in Microsoft, so protocol stack configuration right. So if this is, if this is part of the API surface and there's, there will obviously be system policy as well administrator configured. For example, how do you reconcile those two is it is, are both of those expressed in the same way. The other question I had was: is this an abstract representation like reliable semantics, or is this the application allowed to say I want to use TCP so, which way are you leaning towards so.

L

This is an abstract API, so we hope that concrete implementations of this are going to have compatible representations of configurations and protocol stack configurations. How the reconciling happens is that goes back to my taps.

L

The ITF 99 presentation of the jokey answer is: it looks a lot like cascading stylesheets, in that you have in it within any given system, a defined hierarchy of which overrides which, and then you just blow through the hierarchy, and you get an instance right so there's, hopefully we'll have inheritance rules that are less inscrutable than those used in CSS and I think we can get there because a lot of the externalities aren't the same, but they kept that the point being is that they essentially cascade right and as long as that cascading is predictable and I.

L

Think that is something that we're gonna have to drill down to on this abstract API is we're. Gonna have to make a choice about how that cascading works. On the abstract front, there might be applications.

M

That actually do not the API to be abstract, so we might have to have cases where, for example, the application explicitly chooses. It's, for example,.

L

M

Or quick right.

L

A

That we're gonna stop this discussion in five minutes, so you guys can find out amongst yourselves, but we're moving on so.

C

I think just the main point there is the abstract API should make it possible to do any of these protocols. I think it is absolutely necessary in some cases that they could say. I only want this one, but the bigger point is that if I want to reuse my code for using transports in other places, I don't have to rewrite it entirely. To say now: I want to go over this other transport I, just flipped the bit to say now. I want to use this one instead of this one yep thank.

L

You cool, let's take. Let's take the rest of that one to list, because I want to blow up the Charter in a second Kyle.

G

Rose so so I think you answered the first question: I was I had, which was you're not putting configuration protocol configuration into two buckets one which is generic and one which is specific. So you can you know you could you could potentially have something where you say you know whether you're using quick or TCP I want.

L

To use a particular so that all goes up here and the things that basically it's like, should you choose to select this protocol stack instance then, then, give this protocol stack instance a pointer to this information. This is very low level and should remain obviously so this is higher level. Okay,.

G

So two other quick points. One is around predictability. So there's you know when you're, when you're running a worldwide service that has specific. You know reliability and performance targets right if you're, if there's too much magic, and you can't predict exactly what the system is going to do, you might run into problems that are hard to diagnose, so the more magic you put in that's.

L

A really good point: can we take that offline because that's gonna be like half an.

G

Hour, okay, yeah and the last point I had was sometimes you don't. You want to override the magic in a non obvious way. Yeah.

K

G

Know during the during the hackathon we were trying to do, we were trying to do an experiment on aqm to decide. You know, like you decide whether ECM was providing benefit.

L

Or not and we could not create buffer.

G

Bloat right we were tried and and eventually we succeeded, but.

L

It took a long time. The intention is, if you know enough about which protocols talk, instances you're gonna select, you can basically like, and those protocol stack instances are implemented in such a way that they give you that nob, then you should be able to get to it. Okay, yeah. So it's up to the protocol stack instance implementation as to whether you wants to let you bloat it um so I think. All of this still is correct. We're basically working on details.

L

Now there are some of the things that are details are things that are implicit in the minds of the authors. That should be explicit in the draft. We're gonna write down some of those, the two things that we have on our lists to do beyond. Just you know, general document improvements are: we need a simple protocol, independent carrier state machine, so this is basically the state machine that met that defines what happens to the carrier when it's not bound to a protocol, stack instance, because once it bounced once it's bound to a protocol.

L

Second, since that protocol stack instances, state machine determines what goes on, but you might have a carrier that has lost all of its protocol stack instances but and is trying to reconnect or is in a particular state where it's still racing Tommy you'll talk more about that. The second one is exactly right: how do we represent certain transverses to their interactions in a generic right way? We don't want to be in a situation where we have to transport protocols that have are never retransmitted semantics.

L

So this is a thing that's outside the min set right is I, have a partially reliable protocol and I want to tell it how to be non partially reliable or how that personal reliability works, and so seq PPR. Has this never retransmitted semantics? How do we represent these sort of specific tweaks in a generic way and- and that actually seems like we're, gonna have to do some work. That looks a lot like the work that was done in min set and it's like above min set rate.

L

So it's that the things that are outside the minimal set, but still at that level of abstraction, and whether or not that's something that happens in the abstract, API or something that we say- okay well, this is something that we put off to a concrete configuration.

L

So we have some open issues. The drafts on github. The issues are here: if you read it and have a specific one, we we take pull requests. If you have a specific comment, please bring it up on the list, but and we can put in the issues but feel free to file issues against it as well. The question that we're gonna actually probably talk about later is: is it time to bring this work into the taps? Working group? Here's our view, there's a whole bunch of abstract bi, not application. No date.

L

These are not api's but they're api's right. We have a whole lot of these. Here's how we think all of this fits together. We have the requirements, basically come from Mensa. That's that that's the set of things that we that we can use to implement these abstract api's, but we think obviously and I, think that should be obvious in the design of post sockets. We think it's time to take a creative leap beyond that. In order to be able to do you know to have a transport layer API that gives us.

L

You know the advantage of the 30 years of experience that we have in this space. We don't want to correlate the existing options in the least bad way we want to go beyond that. We think that that creative leap is some combination of these right. Obviously, we think that post sockets, where we think the strengths of post sockets are or the interface that it provides up.

L

Other proposals have strengths in in how that should be implemented and how that works down, and we think that that combination should happen within the working group. However, that happens.

A

So I think what you're phrasing is pretty clearly not captured in the Charter, so I think you're proposing a charter change I might be, but because, because the Charter item number three is about experimental support mechanisms, you're not describing mechanisms right, you're talking about you're talking about an architecture, we have a. We have time to talk about this okay, later rates, so.

L

We should think about that. I.

A

Could be convinced either way, but you know I first push that doesn't seem like a mecha yep time. Can you go? Can you make a quick comment? Natural.

U

So Lincoln on that exact topic on request, come from and said it seems that sometimes we're kind of trying to say we in the past have done the right thing and we're prescribing what things should be done because we have min set. But at some point our users, whatever is above us, are the ones that are saying hey.

U

Can you provide this to us yeah, whether that is that's what I'm trying to capture with this creative leap, Yeah right or some other way, there's something else that is max said else now, I'm, not saying and approving that everything that every user requests is the right thing to do, but we should consider whatever our higher-layer is. One thing us to do in what we deliver. Ab now I'm, not saying that this means that we still need min said kind of to describe what the general said so far that we have observed so.

L

Let me let me rephrase that slide, then so requirements come from the layer above the degrees of freedom. We have to implement those requirements, complement set.

U

We have decided to limit ourselves in whatever shape or form whether we should, and even if we decide to limit ourselves to just a state that somewhere no no.

A

No, this was a quick comment if.

V

We doesn't think about.

A

V

Coming the more I think about it, the more I question the the possibility to abstract away from the transport layer, because we can't even do it in the ITF. Whenever we have a new application, we have endless discussion whether to use UDP, TCP, SCDP, quick or something else, and now we are trying to pass this task to an application, interface and I. Don't think that's viable! It's didn't work for the original sockets API I, don't see it working in any of the approaches we see here. I.

N

Respectfully, disagree. Thank you for coming yeah. Yes, my Keveza.

J

Thinking about why this, why this isn't in the current Charter? The reason? The reason the Charter is, why the way it is and it became so you know conservative- is that we we had these ups and downs of people saying you want to be top down. You want to support applications and I'm, actually not not not being able to agree on anything on that basis, and what the Charter now offers is something that you can implement now.

J

I would agree that this is I would agree that it's better to have a higher layer, so I mean I. This is what I like about about postdoc curse, that it tries to have a higher layer of it and represent things in slightly different ways. However, I am very concerned about implemented bility of the things there I mean some things can be implemented, but if I I mean I just when I read the document, it gave me many many hiccups in the sense of oh.

J

How is this not how it is going to implement I mean you know. I said you would need a system on the other side. That's because there were many sentences in there. That is that you couldn't actually implement this unless you have a system on the other side. So there are many many sentences in there that need fixing many statements that things just wouldn't really match and couldn't really be implemented.

J

I think if, if, if the means that is defining what the protocols now offer, and that would hope that this matches both future protocols off for pretty much mostly I will be, you know, give me any service that Creek offers that an application is to choose, except for certain security things where that isn't the main set any I got it's. It's not, there's not easy to find much that really you would have to have in the API.

J

As input but as input to this document, I mean what I, what I want that document to do is to be able to map down to men set, which is something you can implement. So you can actually see how this is implementable and yeah I mean the discussion goes in this direction, but the document doesn't currently match that. So it's just if destroying.

A

And we can come back to this at the end, if we have any time left we're so.

W

Hi, this is the second revision or the run version of the socket intense draft.

W

Before we start, I will give a quick over what a socket intense, isn't what the basic idea is by the way who has read the draft or some earlier version of it?

W

Okay, quite a few, so this revision of the draft was basically for reshaping it that it better fits within the overall ecosystem and really fit makes clear what piece it is where to fit it and what it's supposed to do, and it's essentially an input for automatic transport, option, selection or automatic transport selection. So it's an input for that magic, configuration box, Rhimes stated for post sockets and a way to express stuff in there.

W

It provides inputs of the application. It's a whether an application wants to optimize for bandwidth, latency or cost whether how the traffic characteristics are, how it tolerates packet, data or collect connection, while so beyond that I want reliability. The power of whether my carrier or my connection breaks down. Is this a real problem for the application or has the application semantics on its own to say? Oh just it was an HTTP download I'll do a partial get up after the next context.

W

I, don't care about my connection being lost, so you can choose a trend, a path that might break down I, don't care just do it and also stuff like host preferences. So saying? Oh, please, don't take a pass that is really expensive, and so this kind of input should help a system like post circuits or the policy that is somewhere sitting behind it. To do a sensible Church of paths to do sensible choose of transport protocol.

W

It's sensible shoes of endpoints, that's the basic idea of socket intents being that kind of input. So for one of the 101, it's really a pieces of information to tell the application what they know about their own communication, what they really know, and it should be intuitive in a way that it's not in the terms of the transport protocol, not in the terms of what I want to set as a flag on the transport protocol.

W

But in the terms of the application developer, I'm going to transfer about a gigabyte over the socket or I want to receive about a gigabyte of the socket.

W

It should be somehow generic that it's not bound to any transfer protocol or not bound to any application semantics. That's really in a way that could be done by many transport protocols by many applications can can be reused by many applications, and it should be completely protocol independent and the last. This is highlighted this this time, because it's exactly what Brian told about the post circuits it's best effort.

W

So if you specify a whole bunch of interns and saying well, my application wants to have a carrier that is as reliable as possible and I, don't like connection breakdowns. If there's only a pass that might break down, you still get a pass that might break it down. If there's a path that might less likely break down, you might get the past. That's met less likely to break down, so this policy behind that should try to match these intents to match these choices by the application, but it just had.

W

This only has another transfer that doesn't match the choices. Oh well, you get the best that's available and the basic idea about this is: it should be standardized, but it's an extendable set of hints that could be used for the transfer selection, so, instead of having for each platform for each API for each implementation, some other kind of bunch of keywords to tell the transfer, option selection or tell this policy thingy what the application is going to do.

W

You want to have one standardized set, and if you move over from programming in go to programming in rust or programming in C, it should be the same set of keywords. It's expressed in a different way for a different language, but it should be exactly the same set of keywords. You already know you can look up at since some central place.

W

So how does this integrate?

W

We did an implementation as socket options on BSD sockets, which turned to be out a nice thing to see that this actually works this concept of socket and turns, but it also turned to be out not so nice in how useful it is because you can only do it on a connection level.

W

So you can't say well, I, don't care about the delivery of this message or I can't say: I wants to do something like HTTP connection, caching and therefore I have to TCP connections, one over LTE one over Wi-Fi and specify that this message should be low latency. This message should through P high bandwidth, and then it gets measured, ly matched to the right, easy P connection. Something like this. You can't do with BSD circuits, but you should do with another transverse. So therefore socket and hence works with PC circuits.

W

The terms were Lukas pieces, but it's not as useful as it would be as with something like post softens in post, so that you could just use this as address configuration which was in the last process, I'm, not sure whether it's still the name the in the current draft, but we can take this later or you could specify the width needs us as a neat property.

W

So it's the concept with them. This is the, but you can take it in any city should take them in any system that they, like already told this, is, could write messages. Screams screams is what they, the posted people, just called carriers associations, but they just called associations and Association said, which is something about this, which bundles bunch of associations.

W

So how are they represented? So it's pretty simple yeah, just key value pairs. You have a pee just kilo case s, key string, Sam sings symbol, something to remind which I guess should be best. Captain Diana registry, sometimes somehow but I, think that's an interesting discussion item and have a wall value and the value should be have some fixed type. So then enum, if it's kind of category integer float, string or sequence of one of the above.

W

If this is needed, I have don't have a use case for the sequence yet, but it might be useful.

W

The initial types we forever a traffic category, which isn't you know, and it's applicable to streams, associations and Association, says size to be sent and received, which also makes sense on messages, the duration of the stream or some other transport, the bitrate sent and received, which also makes no sense on a message how bursty traffic will be, which might be interesting if you either to decide whether to schedule something over Wi-Fi or LTE.

W

If you want to save energy, how timely the delivery should be so, whether you rather want um so something like this be Nagle disabled or whether you want a low latency carrier, were they how disruption resilient if the application is so, whether you care about flaky, Wi-Fi or rather want to be scheduled on more expensive LTE connection, but that has a high likelihood of not breaking down all the cost preference.

W

So, whether you care about whether you sync, your traffic, is so important that it should be carried over something like data roaming or whether you say. Oh, it's an operating system update. We can do this at the moment when we have a cheap path available, so in this case even telling the socket API, if you don't have a cheap ass available. Just deny this request just fail. The creation of the connection- it's fine for me so with this I, come to the questions after the basic concept.

W

So it's this interface of this idea easy enough that it could be useful for an application developer and could be useful as an input to a transport system.

W

It's the structure sufficient and in can it express all fountain in the functional and non requirement performance preference, because you should can't use this to communicate, I'm literal idle channel, because it you need a reliable channel. If you need a reliable channel and then the transport system can't, if you UDP and.

W

What intents, what kinds of information are missing to make this useful of the first page of the questions.

J

My Michaelis, there was at least one thing on the previous slide that would never arise from just analyzing the transfer protocols and that's actually actually something. That was a bit of a concern to me when we started with this analysis that you might miss out on some things that seem important, like burstiness I think could be important for making a choice on the path that you want to use on some choices.

J

The depsite type system is going to make, but it wouldn't be wouldn't arise from looking at what the current transports offer all right, so I'm wondering if I mean I'm wondering what you specifically do with burstiness and I'm wondering if there are more of these things that we should have covered somewhere as features that actually don't. You know you don't get from just looking at what the protocols offer, but isn't the request. This is describing the applications. Traffic behavior.

V

What kind of a you know my opinion about: Transport automatic prints, code, layer, selections or I- won't repeat that I guess this is far too complex in the application that, because the application has to guess what the network or what the application interface is doing and I don't know what is thirstiness. We have to explain that to an application developer, and this is hard we've seen it 20 years ago with 80mm, you had to fill out a form like this.

V

What's your bitrate, what's your burst in US and people never got that and I I think what it would be better to expose the attributes of the underlying layer to the application and that the application decide how to do with with that. If you tell the application. Ok, this is an LTE link which might be costly. The application can decide ok, I'm now using a simpler codec or something like that. Instead of leaving it to some lower layer.

W

I also are happy to disagree with this I think then the point, the big difference between ATM- and this is here- the application tells what it knows. So it knows whether it will send bursts of data or whether it sends so what we rather smooth scream. So, if you're doing something like HTTP streaming, it will have its bursts, but it will mostly stay at its data rate. If you have a protocol like machine-to-machine communications will most likely just have either small bursts. I will send a few small messages. It's not about the exact burst rate.

W

It's not exactly about the exact band was here using it's just about saying: well, there will be rather big bursts. There will be rather small sources.

V

Take bit rate, for example, what's the bitrate of the opus codec? If it's var I have a bit rate, I can can give an upper limit, but it's not very useful because it won't reach the upper limit and pray. Often so bitrate received I, don't know so.

W

If you, if you have so again, let's consider service like YouTube, you rather know how the bitrate of the video is you're requesting or you can know this, and then you can tell it to through the transport whether you then later on have to decide to switch to lower bitrate. That's fine, but then then you can still tell the to the transport OS, which too low a bit. Bitrate I expect a lower bitrate on on the transport weather that this is really awful used for the transport in this situation.

W

In the current configuration, the application can't know so.

V

But the application has to know what bitrate is possible, so it can adapt. For example, YouTube will choose a different codec or a lower the codec bitrate if it experience some congestion. So my point is expose the attributes of the underlying layer to the application and that the application decides enter in principle and.

W

My point is: if you really want to do, if you go that down that road, you end up with all applications having to know all transport protocol they might potentially use and to know all the knobs of it, but I think still. Bose has a valid use case. I. Think if you're at the point where you have an application that you have exactly tailored to a transport protocol, then it's much better to expose the attributes of the transport through the application.

W

If you have an application that doesn't want to invest too much of the engineering into this part, then it would be nice to have a layer to take care about this problem.

M

Pravin Microsoft, one of the things I see missing here is there is no intent expressed for the send or receive pattern.

M

I think transport performance could be improved if the socket intent would say, for example, what the sending pattern is like, for example, to 4k, sends and then I wait for a message from the other side before I send again some pattern like that Express to the transport you could automatically do intelligent, like you know, disabling of niggling or figure out your delay tax strategy, and things like that, so I think having an ability to express intent of what's the pattern would be extremely useful.

M

D

M

C

2 that Tommy Polly apples so I mean thank you for presenting this. Thank you for doing this. I think it's a very difficult area to get right. I very much agree that oftentimes we're going to need to just expose the protocol knobs, because that's really what at the end of the day is there. So when we look at this kind of when we're talking about with post sockets, we have the configuration which is like this generic thing and there's like my protocol, specific configurations.

C

I think you know the protocol specific configurations are where we historically think of as socket options for that one transport. These things are the things that go into the top-level configuration in this great generalized world I think there are some things that are in that category. I think you mentioned like I, don't want an expensive path.

C

That's one of the things that we have been able to bring up into that top level configuration because you could say I, don't want cellular I, don't want this particular carriers network, and you can because you know that, but oftentimes people really just mean I want something general. So I would love to see us work on this as a working group and likely it will tear it apart, and most of these things like bitrate may need to be stripped out or something.

C

But I would like us to have the conversation, because this comes up all the time from people who are doing implementations of oh. We want an abstract way to do this and I think it'd be great to come down on each of these and some of the other options and say kind of definitively. This is something that can be expressed abstractly. This is something that can't and get our story straight, and so it may end up being a very difficult discussion and a tricky process, but I think we should go through it. So.

A

I just want to insert myself into the queue here that a question.

W

For you, Phil have you implemented this, so we implemented this on top of BSD sockets. But, to be honest, the only policies we really used was the traffic category and the send and receive size. This is the only stuff, but we really have experience and we really have evaluations on I.

A

Think that I think that is an excellent point which is sort of harkening back to the ATM discussions.

A

There's been many decades of categorization of what traffic, what applications want from the network and what the network can give to the applications and I think well, we have an opportunity to do here is some experimentation and that the whole idea of that taps API to allow you to you, know, mix and match transports based on map indications is we may actually be able to make some progress here, but I think that this table is a little bit of a tractive nuisance in the sense that many people have seen tables like this in the past and they weren't believable then, but I think that the real test is going to be what we can implement in turns out to be useful and I think Tommy's make making a good point, which is you know it's a let's view.

A

This is a work in progress, see what works. What doesn't, maybe some rosy table will leave me, will add some new ones, I think.

W

The intent of this table is really started this cash. It's not that I want this table in the final draft. It's just it's the strong.

A

Resemblance to the to the ATM traffic classification table, I think that is a set people off at least.

X

For me, yeah I think that and the RSVP stuff, : Baggins so yeah that the table makes me nervous and that this table makes me nervous and when you start trying to categories application behaviours, there's a it's very easy to rathole in a far too detailed way. That goes nowhere useful.

X

That said, if we you know, if we're building these sort of generic API is, there would seem to be a scope for passing in something about what the application knows about the data it has to the transport. So I think, and we need that ability I'm unconvinced about how we do it I think we should think we should at least think through that this. This is a feature of the future API right.

A

There are some semantics that come from the AVT world that might have something country, yeah I,.

X

That world I think is really good at going overly detailed and producing unusable things.

U

Like a soulless, LinkedIn I'm still grasping my head around a few these things, but the the first thing is talking about intents from the peace table that you had some of these things. If I read it correctly is what the application or whatever box is above this layer intends to do right, so it tends to send so much it tends to so. It's kind of communicating this is what I'm intending to do. Peace feel, please feel free to optimize.

U

Maybe around that some of the things you had in the box are not things that I intend to do, but things that I want the system to give me so I, don't think there should be mixed up. So the example would be timeliness. You intend to be timely. You don't intend to be hungry. You you, you want the system to give you some type of timeliness. You do intend to be bursty, or at least you expect to be bursty and I. Think in in the draft.

U

The wording changes when you once you get to those descriptions, so I think we should be careful to separate things that you're intending and just saying hinting at what you plan to do versus thing that you want the system to behave, number two point, and maybe more abstract is both for these things and four configurations that you're giving to a system.

U

In this case this one a particular if the system is to make a choice and do something for you is because you're you're asking or presenting a trade-off right, I'm, not gonna, tell the system, please perform the best possible I'm, hoping it will perform as well. That's that's what I'm intending to do, but if I, if I tell it that I want things to be- and this is bandwidth- is because I want you to make the creative for me, give me bandwidth versus latency, for example.

U

So at all points in all situations, you're expecting whatever you're asking to have a cost. So we should be clear when we're describing what cost you are expecting to potentially pay by requesting something, and this should be reflected both in configurations for this or for anything else that we're doing right. Obviously, otherwise, all applications will request. Give me the best service possible all the time right. That does not help us right. What helps us is somebody telling you I want bandwidth. Please sacrifice these other things for me to optimize for bandwidth and to be generic.

U

Obviously, if you're on a bunch of these things, then you could have like weighted averages like 50 percent bandwidth and 30 percent latency and 20 percent reliability whatever right. So you can come up with weird systems, but but just be clear that we're asking for trade-offs and that should be reflected in the commentation and then the API is the we design. That's.

A

P

Me and I can't agree more to not just that's absolutely right and also I'm wondering how much of these things we actually have to specify somewhere, because um it really depends on what protocol your interface supports like if the only protocol you support is UDP, then much of this is not useful. If you invent a completely new protocol, you need something completely different, so I'm not sure what what makes sense to specify here. I.

W

Must say this lot of this attributes comes rather from the path selection, ideas.

K

W

From the question of protocol selection, so I think most attributes only makes sense if you want to decide which takeoff to trade for a pass, not which trade of to take for protocol.

P

Also, your implementation of pass selection can be a limited right. If you have exactly one interface, there is not much selection to do yes,.

W

L

To path aware, networking there's.

W

The afternoon.

L

Hi Brian Trammell: can you go back to the ATM table again.

L

So everything nacho said plus so I think there's this. This.

L

Are you gonna take that what are you doing? Okay, um yeah, okay? So so this this year, there's there's, there's two issues with this. So I, like the the farther to the right you get, the happier I am the further to the left. You get the less happy. I am some yeah and then like from the out in is kind of like this. This sort of the heat map of how I like this table right so I, think there's a so there's a prop okay.

L

So you start trying to solve the problem of how do we take intents and express them in a way that you can do path and transport selection, which is mixing two things together that maybe shouldn't be mixed together and like expressing it.

L

This way makes it clear to me so I hadn't, really I've, also am I thinking, I've been thinking as a path and a transports stack as potentially partially equivalent, and this really underscores to me that that's wrong and we need to go and look at those sockets and make sure that that they were good there. The second thing is, there seems to be this.

L

We have the the started from a concept we'd like to be able to take some intent-based thing and use it to poke it at the transports back and have it do something nice for us, and this looks like it drilled all the way down to the bottom of that problem and kept throwing a bit and it's and when it you know when it before it hits the ground. It's it's doing really great stuff, and then it hits the ground and then so.

L

Finding the ground there I think, is a really important thing to do and say: okay, here's we go this far, then we stop and I don't know where that is, but I think that this work is most useful when it's higher layer and gets more problematic, the more lower layer it gets, because this also goes to sort of the discussion we were having a post sockets about granularities of of configuration and I think there are two extremely distinct viewpoints that are represented in this room, who I will call application chauvinists and transport chauvinists.

L

The application chauvinists are people who just want the transport to do something. I just want to write something fine and for the most part they don't pay attention to us anymore. They just use zmq and like wave their hands while they, when people ask them security questions and the the the shape of the API is that we produced or from a very transport chauvinistic area. It's like we're, not gonna trust you to do pacing, because you'll nuke, the internet right.

L

It's like there's still people who are very scared about user space transport stacks because they believe that this route user space, a route, non-root user, a distinction- is useful for security, which it just isn't um and I think that I think that this here. This really has shown that we need to kind of try and find the balance there right, because this is a very. This- is a very application chauvinist way to do it.

L

But does it in a way that, like selects an application chauvinist by the way, so I'm with you on that, I think that we should give application developers the things that they want. Not that things are, we think we can give them and.

L

Yeah, actually, you know what I'm gonna take the rest of this offline, because we should get moving all.

A

Right, cool thanks! Let me just paraphrase what a piece of what I think you said, which is I, think that implementation experimentation of this stuff is going to be critical because we don't really know what's gonna work and what's going to be useful.

Y

There's canasa when.

W

You've drill too far, so let me come back to the point of experimentation, so we did experimentation really with this option with this send and receive size, and we found out that it really helps for examples to decide whether you to do HTTP pipelining on a specific path or whether you want to do multi participate for this kind of decision. It really helps. It only affects about five percent of the use cases we really had where it made a different in 50% of the cases it was just doesn't matter and 50%.

W

So therefore, I think this bubbles past week, I can sit and say it is useful for these cases for the other ones. I just cannot tell.

Y

David Skinner's e Apple, so I'm gonna introduce the third type of shavon s, which is an engineering chauvinist which I am and it kind of reflects what you two were saying: I think there's value in creating a framework for socket intents saying like this is what a socket intent is: maybe have a registry, an eye Anna or here's where you put your new socket intents when you come up with them, I think there's value there, traffic categories, those have been implemented by Oasis for decades.

Y

Those have been proven to have value in some cases and apparently you're saying this has to be sent and received as well, and in that case I think it's great. You can document here's how you specify those and here's an example how they can be used. Here's how we're using them you get these benefits. The other ones. I would really recommend against specifying these things because you're like oh.

K

Y

How you specify burstiness- and everyone has a way to specify burstiness. No one specifies it, and no implementation does anything with it. So I, don't think I think it's premature to have a laundry list of all the things we could possibly want when we don't know what to do with them. I really like praveen's example that I think you can act on really messing with a delay. Dac and I call you play with Nagle, that's actionable! That's something!

Y

I can write code for how bursty you are as someone in the DC B's like what am I supposed to do with this, so I would push against to try to standardize things. So we have no idea what we would use them for.

A

Gouri the line was closed as a quick.

D

T

It can be quick.

A

T

Thank you, okay, so, um okay, so I, look after RSVP stuff, I'll, look after diffserv stuff. There is stuff in between and it'd be really nice. If the transport stack see some of this, but do you know it's really better when we see binary things, I want low, latency I want low cost. I want zero cost. I want I, am very, very bursty, but just not get into the details. So I like the list. I think the list should be bigger, but I think the data type value should mostly be binary. I.

D

X

The data type should be enums choose this, or this.

X

W

Think the last question I had is better for the for the last chartering and discussion. Part yeah.

D

D

C

All right, I'm, Tommy I'm, going to be talking about another aspect of kind of the it's like the guts of a taps implementation. Hopefully none of this ever gets exposed to anyone, but it's very relevant for how we actually make a system that can try. Multiple options actually happen whenever we're talking about trying multiple paths. Whenever we're talking about trying multiple protocols, Colin.

I

That's very quick. That's very quick.

X

It's a sort of scoping question, I, think and maybe you'll get to this. When you say racing, do you mean happy eyeballs or do you mean things like ice as well? I'm.

C

Coming from it from the happy eyeballs side.

C

It's so I'll explain, so let me get to the next site. I think I'll define the scope and people have brought up the you know. The term is the term racing the wrong thing, so we can have another term for a, but I didn't want to use happy eyeballs, because that's really v4 v6 dependent in people's minds. Yes, yes,.

X

It's just that when you say should be hidden me if happy eyeballs I would agree for ice, probably not okay, so.

C

I did this I. Did this particular thing as a draft update to the taps type guidelines document I had done last year. It included, among other things, this generic racing guidelines and I kind of distributed all out to just be that part, because I think everything else is captured elsewhere in things like post sockets. So this is a much more focused document.

C

It is trying to generalize happy eyeballs. So that's kind of angle. It's coming in on so happy. Eyeballs is RFC six, five, five five.

C

That is now we have abyss for that in the RFC editor queue, so that will soon be a new RFC number, which gives a little bit more advanced algorithm, and it has certain aspects that I've incorporated into document as well, and then also, this draft is trying to take a little bit stronger stance on saying that there are things that you really should do when you're building your options for how you're doing connection establishment racing, and so these are intentionally things I want to discuss and if you think they shouldn't be assured or a must please.

C

Let's have that discussion because I think they can be very important. So what is a connection option I think that gets to what Colin was asking I'm, referring to that here as a tuple, three tuple of your endpoint, your path and your protocol stack, which may be your protocols, your protocol options, let's say: I have TCP and TCP using TFO I would consider those two different rota calls right. So we can distinguish those just based on options. It defines a notation for how we can essentially look at how a connection is established.

C

I view it as a tree.

C

Essentially, the point of that is that it has a hierarchy of how we look at deriving these options if you just took all possible options and built them as like one big matrix, like a three-dimensional matrix of multiplying all of these things, not all of those make sense. If I do a DNS resolution on Wi-Fi I, probably shouldn't just arbitrarily assume that that endpoint, that I resolved works on another interface piece. Maybe it's a local resource. So this is an attempt to cull down that space of options.

C

So what are the ways that we have options? So we have paths branching in just like the path we can protocol stack branch. Oh, we can do endpoint, which I've been calling derived endpoint branching, because there's always the endpoint that you have at the beginning that the application wants to get to we'd better. Have that, but there may be stages such as Bonjour service resolution service, discovery or just normal hostname to address resolution that can be happening to generate new sub lists of endpoints.

A

Yes is a VPN full splitter, full VPN. Is that a path yeah.

F

C

That would be. That would be a path, so another term for path for people who are in PVD. Discussions is a PVD. So uh up half is not just an full interface. It's essentially just a self coherent set of routes, so a split tunnel VPN definitely falls into that. Yes, the derived endpoint I choose may drop me into a different path, but these are more about which, like when I'm, making an explicit choice at the beginning.

C

What do I want to say I want to force this attempt over a certain interface or do I, not care. So the document kind of is saying like there's a should between the ordering of these, that certain things like a resolved endpoint coming from one path, probably shouldn't be used on a completely different path, so that I think is a pretty strict one.

C

Similarly, if I have an endpoint that I have derived us very specific to a given protocol, let's say I have one protocol stack is using a proxy, and it has you know a new in point because of that that endpoint probably doesn't make sense if I'm not using that proxy or not using this protocol. That requires a different port, so I think those are very clear. We've been having discussions with some of the neat folks, um I.

C

Think the distinction between dua branch for path or dry branch for protocol stacks first is a little bit more flexible. I think it becomes simpler if you choose generally, which big path I'm going over first, but that is maybe more of just a recommendation. Rather than a must it's coming.

X

Hi : Perkins is v4 v6 a path or a derived.

C

So when I, at least when I think of paths as pvd's and in that regard, you can have your v6 and v4 kind of bundled, as this is like here in set of things you're doing.

C

Yes, they certainly have different properties. I think I would agree with you that when we look once we've actually connected and we're using our path properties, the path is definitely different between v4 and v6.

C

But when we're talking about the intent of the application to try to different path, options, saying that I want to try the v6 or v4 option doesn't really make sense until I've actually done. My resolution do I even have an address for v4 v6, but saying something like I want. A VPN or I want Wi-Fi on cell or Ethernet or something is something I can say up front.

X

C

X

Okay, yeah I, guess that makes us I'm still trying to think how it fits to a sort of natural versal style, where your trains find sort of interfaces. Protocols configurations the work.

X

So I'm just trying to figure out how it fits with natural vessel and when you're racing different options there is that a pathways that it had derived endpoints right. So what.

C

One thing I'll move on quickly, but one thing I want to point out is that when I'm discussing this, this is all about the kind of the initial connection establishment so not necessary time at the net reversal for ice, but like for things like MP TCP I mean which you know. Ultimately, we will be using multiple paths.

C

This is the algorithm they're, using essentially for the first leg just say: can I get any packets out once you can certainly use more paths and more addresses later on and I'm sure that you know doing ice stuff will end up using multiple things. This is about just like. Does this thing even work? The ice stuff is about determining that hmm the ice stuff.

X

Is about determining whether it works right so I think.

C

X

Is an area that we do need to explore more in this yeah.

C

X

Like a review from yeah I think Pafford derive 10 points or maybe not as clearly separated, as you might hope,.

C

So it also defines three styles of how you can actually do racing between your very options once you've built a list at each level. These should be relatively non-controversial. I think you could do things simultaneously and the draft essentially explains why you should probably never do that. You could do something based on a staggered delay or you could do something as failover I really prefer this option I only want to try the next option.

C

If my primary preference has completely failed, there's also the question of what does it mean to be established so traditionally happy eyeballs is something that ends once you have the three-way TCP handshake done and that's great, but depending on your use case, you may actually want to see. Is this really a usable path or usable connection am I going to be able to get my security? Handshake done?

C

Am I actually going to be able to receive useful data, maybe I'm doing zero, RTT work and so I don't want to just assume that something's going to work so interesting questions that an implementation has to decide there.

C

So that's kind of a summary of this draft. What I want to bring up here now is we have two drafts currently in taps that are very related on this topic. There is the neat draft on happy, eyeballs and then there's this draft. So what do they have in common? How we're going to reconcile them? I think there should probably we should combine them and only have one but I'd like to have that discussion out here in the open. So what do they have in common?

C

Both of them describe how you do happy eyeballs for protocol stacks. The needs when I believe kind of just goes into mainly that doesn't really go quite as much into the path racing, but they have the protocol stack racing in common. They both recommend a kind of historical database of what you've previously probed or what you've previously passively learned about your various inputs and paths, and that should influence the ordering and timing that you use.

C

So that's very important one and then also it's an important observation that both of them have that the combination of what options you're allowed to use come from your application preferences, plus your system policy, plus your historical data, and so there are three different sources of this okay. So what's different I'll go to the bottom one first, so there's the question of. Should we have this structured tree or do we really just generate one big list and then it's kind of free-for-all you let the implementation decide what to do.

C

I'm, fine, making this kind of more of a recommendation. I. Think if you structure things as a tree, it makes certain algorithm decisions a little bit easier. um The happy eyeballs timer racing between v4 and v6 address on Wi-Fi really has nothing to do with be happy. Eyeballs racing on a different path on a different interface. That's happening simultaneously. If you, as an implementation, want to kind of manage those all together, you can, but there be dragons down that road Brian.

C

Did you have something on that one or no okay, okay, so I think the bigger one I want to get into is there's kind of a fundamental difference in the drafts of. Are we what's our space of things to race?

C

So are we racing things that the application didn't ask for or not and I think that really makes a difference in your decisions, so in the draft of what I would like to propose is essentially that we are only going to be racing between things that the application has kind of whitelisted in, as my server probably supports this, as opposed to just trying to do probes, because the system likes this protocol or this other one.

C

We think that this one might work, for example like if I just want to try SCTP, because I want to make it work on the Internet.

C

You would likely want to try it, but then have a very quick failover, because it's probably not there, that seems to be a little bit harmful for the algorithm and for the Internet and I would prefer to see it's something that is more opt-in by the application.

C

So I'll just rush through these slides, so I think it gets to the overall question of when we're racing. What are we actually connecting to and these connection options?

C

They all come from either the application or they come from the network right. So when we're doing happy eyeballs the options were racing between our things that DNS told us. We know that these are real options when we're doing racing potentially between your quick and hb2. This is something that applications do all the time this is a LPN TLS is explicitly indicated. Someone out. There knows that the service works this or the application I just say: yeah I know I built that server for my application.

C

It supports these protocols on these ports, so we can have you know vague preferences, strict preferences. How are those actually determining what we're doing I think the vague ones are useful for tuning things, but we kind of we should I, don't think we should be using vague preferences to infer protocols that we don't have any explicit knowledge of them existing, and so this is a bold assertion. I'm we can talk over the email list, but I think it's an important thing to really clarify it as we're going forward on these algorithms right.

C

So one of the main points that we're trying to do here in taps is say: let's make our transport library flexible. We don't want to ossify on just everyone's just doing. Tcp I think that's great, but why is this really happening? So one thing that I think that we think of a lot is that okay, we have applications that are writing code that only worked for TCP. It only works over sockets.

C

This is definitely true when we see the ipv6 transition problems, because people wrote their code for v4 addresses only and that's a bad practice is that true for why it's hard to deploy new transport protocols or get applications to adopt their code into new transport protocols? Maybe but certain things like I I, see a lot of socket code out there that it really just deals with things in generic way. They just have a sock stream using a stream.

C

Just a stream couldn't I run that over any stream protocol, and so I would like to point out here. Is that sometimes actually the fact that our libraries are implementations like sockets, give us these generic vague options for protocol selection? But then we end up only doing one thing beneath it in that current implementation actually led to the ossification. The fact that we say sock stream is a stream. You can just write your stream on this, but then in practice that always meant TCP.

C

That means that applications have been deployed that they will get worse behavior if we start doing things under them that our stream that they never opted into, and so if we could end up deploying a cap system right that says, oh great taps works with quick, SCTP and tcp woohoo, and then the applications get used to that set as being the things that are there and the moment you try to add number four. They have to now wait for that racing or they're delayed or it just doesn't work.

C

So we don't want to get into that situation again. So the we can read this online I. Don't want to take up too much time here, but essentially saying that our API should work under with any protocol, and we all shouldn't have to change any of the application code. However, we shouldn't just be inferring, things I think is sufficient for a lot of these taps. Work to say all you can do is change the Flippa configuration but to turn on green light.

C

This protocol say I support this and that's a great goal, but we shouldn't be doing things that the applications don't expect in this racing. I think that ends up being harmful. That.

D

L

Great so I'll I'll point out that on this light, these right now seem to be slightly mutually exclusive. In that you say: okay, well, we're not going to and I think that's bad and I. Don't think it's a problem with that! I think I, don't think it's a problem with the expression of the thing. I think it's a fundamental problem that we need to find a way to solve.

L

It is you're, saying: okay, we're not going to have the library do unexpected things to you by being right by sort of essentially greasing the protocol space if you're familiar with the concept of greasing.

L

But then, if you have applications that are used to having to trick the API, you can't fix that without tricking the application I, don't know how to fix that. That's just a that might be a fundamental problem that we have to throw some brain at right.

C

And what I want to say is that it's actually not that big of a problem to change the application telling the aberration to rewrite their entire library, yeah.

K

C

Sockets is terrible telling them to change a bit like yes, these apps, yes,.

K

C

New versions, all the time, turning on a boolean whenever they turn on the new protocol on their server end is not that big of a deal. Okay,.

L

Okay, so basically the one of the the application, deployment and development cycle that dominated the sockets era is not the same as the current application development employment cycle, and we should pay attention to that. Okay, that's an it! That's a that's! An interesting insight, I'm less unhappy! Now it's not the thing I came up here to to say the thing that got me in the mic line was, as you were talking about. What should you raise much you're, not racing. Should you tree it or sequence?

L

It I think that the the distinction that you're making here or something we should pull into PO sockets. It's things that the application cares about. Then we know and not at the library thinks, um but one of the advantages of having a tree oriented as opposed to a sequence. Oriented thing is that you can use the same mechanism to do both current things that we know we want to do as well as to have the library potentially I'm gonna race, this psi and I'm never gonna use it because I want to know. We.

C

Actually, do that like when we're trying to probe 40s 1.3 great.

L

Ability we light it up another tree great, and then you store that information in the Association for the future. Okay, cool.

A

Okay, I'm gonna, lock the line and resonator hi.

X

Colin Perkins, so I think this has to be a tree, because you can always collapse it to a tree with just one set of branches which gives you a sequence, and- and there are tell you things where you need that that flexibility in terms of racing things that you don't expect, the the the issue which came out with a lot of the ice work, was that you got so many candidates that you ended up.

X

Congesting the link and people people are actually trying to cut down on the number of things they're racing, so adding adding more into this mix I think is likely to be a problem. So yeah I agree on that on this ossification point. A lot of the problem is we tried to be general, but it wasn't actually general there's only one thing there. You know if we from the start have multiple options. I think we avoid, if you say stream, and it could give you more than one thing I think you avoid some of that problem.

X

The other thing is give an explicit way of getting a protocol right, don't say stream and assume you get. You get TCP say you have have an option say: I want a TCP connection, exactly say.

C

That I want to interact with it as a stream, but the things that I allowed to use are TCP in yeah.

X

Yeah I I want to stream, but it has to be a TCP stream.

V

Welcome back most of our application layer, RSC's explicitly state protocol to be used exactly. Sometimes we have option to use different protocols, but most of the time we have use UDP for radios or use TCP for something else, use that.

E

Under server support, yes, we.

V

We have very few I, don't know of any application, RFC which simply states use a protocol which, with partial, write, reliability and quick interactivity or something yes, you state the protocol because we have discussed the pros and cons of the difficult of the different protocols. Another thing about the previous slides you're only talked about connection oriented sessions, but sometimes we have things like switching from UDP to TCP, for example, you want to do voice over UDP, of course, but sometimes firewalls only allow TCP and yes, in the super odd.

V

We have standardized this ice stuff, which is very complex and takes a lot of time, which is bad for a session setup time, and- and this is something you can't handle in some general library- you have to use it to it in the application. So.

C

Not necessarily um so I have experienced with doing things like, for example, like ike VPN negotiations run over UDP. We can fall over to TCP, where we need to, because there's an tolerance for that I think the solution there.

C

That gets back to the point of how are we determining what establishment means if you actually include some level of application protocol into that, like the application determines that I got useful work back, and this is going to work for me, then that determines it so I can use a UV protocol and, if I determine after some time, it's not the same algorithm for how you fall over to another path still applies, and we do employ this.

V

For UDP, you have to get some reply and sip you have to stun.

C

Message to show that you deeper.

A

C

V

C

A

X

You just you cut the line after I'm, not after.

Y

X

Just very quickly, following upon the the ice points and you you may have to hook this racing into the application state, absolutely yeah, so Soviet, so you're hooking into applications. Things like not all within the stack.

F

Spencer, Dawkins interested individual first observation I would have this. Not only do we have reordering problems with transport protocols, we even had them with a mic line and transport working groups apparently awesome good job. The thing I was gonna.

F

The thing I was going to ask about and I don't need an answer like in the next 18 months, so I'm, just yeah I'm, just telling you anytime, between now and and work ITF last call is fine, but I was also interested in your response to Brian about kind of what application development looks like these days compared to the socket, Sarah and I.

F

Believe you and I'm curious as an area director about what the tail looks like for that and like I say the don't don't answer that now, because the answer will change by the time. You know I mean you'll be further down the tail, but I would just like I would like for that to be cost somebody's mind before IETF last call when all the people that don't do it. That way, tell you tell us that they don't do it. That way, you say: does it make sense? You.

C

Said there's a fantastic point. My initial gut reaction would be that detail of people who aren't pushing new updates. Their code are probably not going to be adopting anything after socket anyway. At this point and.

F

And like I say, that's a fine, that's a fine! That's a fine answer! If it works.

B

No, no I think I'll. Take that question offline, but just for your environment I mean you take 23 minutes, say 8 percent accuracy.

D

T

Okay, we will use this computer.

A

So Cory, if we're gonna, if we're.

T

K

Have any discussion.

A

About the document approach in the working group, we're gonna need to end this pretty sharply I'm 20 minutes questions just.

T

Setting expectations, yeah I was gonna, be quick with my laptop, but I actually click anyway. So.

T

This draft is short and easy to read, and people have accused me of writing too much. This time, I wrote too little to figure out what else that we need to write next. So let me tell you what it's about and expect the NEET protocol stack. So it's about something that implements some of the things in taps with real running code. So this is a result that is generated code where we can signal things down to the stack and the stack can make decisions on behalf of the application in the ways we've talked about.

T

It supports a single API to transport. It's got explicit support for multi streaming, a multipath security.

T

It's got a policy based selection engine in the middle, which does the happy eyeballs candidate selection, and that may be the bit that's most interesting, currently I, don't say much in the draft. So if you want to see more about that ask questions or we all write more if people want it, this is the sort of thing that NEET does.

T

When you start an application- and you set up the neat system by initializing it, and then you pass a JSON string expressing what you might want, so here's a very general requirement you might have from the application. You want a reliable transport.

T

Things can be either required or they can be desired. So we have both levels of functionality. If you want, if it's required, it has to be provided or your connection fails. If it's desired, then it's something that goes into the ultimately into the candidate selection.

T

Whatever the properties you request from the application, nique proposes a set of candidates. In this case there could be different transports. Neke performs happy eyeballs and we we have a draft already on happy eyeballs and how that works, and then finally, a neat endpoint is used for communication, and then we enter in the medicines mod where you can adjust things, but they are only the connection. That's already been established.

T

So this is a diagram which kind of shows a little bit more, how it works internally, which is probably the most useful part of our draft, because that this particular draft brings together the functional parts. The app indicates, properties, the you call a set properties, call give it JSON. You could indicate things that require like transport and tight.

T

You can indicate properties like the dscp value or the abstract, dscp, or something even more abstract, like you just want high throughput or whatever, then you do an open call and finally, you can get back from the system what properties were actually set, because you need to be accountable, I think when we define this so that attach they can find out what happened when you did all the magic underneath to make up the connection that you finally chose. So underneath we have the neat user API, which is the main topic of the draft.

T

Then we have the neat logic, which is by far the biggest chunk of code. This is the piece of code that actually does things and from the neat logical, is a policy interface to a policy manager which is the piece of machinery in need that takes the properties input. It combines them with a policy information base which configuration information, this links properties with rules about how to use them, and it link them with profiles defining what it actually means. When you ask for something, so it expands the property list.

T

We also have the characteristics information base, which is learned. Material, you'll, see synergy with the things we've heard earlier in the taps meeting. This is things discovered from the network and it could be path, history, information, it could be Proust called history. It could also be explicit: signaling I'm, a great fan of PVD. So if you want to imagine PVD in here, we have a way of injecting pvd's into the system, so that network interfaces then become pvd's and you can learn about the characters pvd's.

T

Whatever you have a shared a lot of information, what's configured what was asked for either concretely or abstractly, and the information you've got about paths. Out of this comes something so: properties come down up, comes candidates and, as Collins already pointed out, whenever you do this, you end up with an infinite number of candidates.

T

So one of the things that the policy manager has to do is protest, a suitable list of candidates which come up to the neat system, and then it has to decide how to use these and that's a racing decision and it's easier when you've got a connection-oriented transport and you have a Kinect, it's rather fiddly when you've got UDP or dare to ground-based transport, because you don't actually have a Kinect, so you've got to define some notion about choosing which one, and sometimes that means you have to use this failover mode of racing, because really you just don't have a connection set up, but sometimes you have a protocol on top of UDP.

T

So this is a little bit more tricky yeah. You can do it and then you do the full ice thing. If you really wanted to, we don't know.

B

Who this policy interface, who is using.

T

This policy like who is setting setting deserve the the policy interface, is between the two parts of neat, the policy manager and the neat logic. So this is an internal interface, but it, but we thought it was a real interface that existed. Will you suggest properties and you get back candidates? The external interface is the policy input infobase, which is the horizontal line, and this this in our system is currently JSON files. That's.

B

Just the link goes missing, O's, not sure, I, think yeah and.

T

The pipe the coccidia for information base also appears as JSON, so um I was taught to be quick, so I'll carry on being quick, happy eyeball, simply selects, and when you've got these candidates, it chooses one of these possibilities and one of them wins. That was simple. It was easy and lots of difficulty in actually realizing this. So what happens when you ask for something? That's a little bit more abstract and maybe low latency and low latency is one of these services.

T

You might want, because you're doing a background file download and it's a concrete application requirement. My preference is kind of for things that come be expressed by applications and make the tap system do something, and then the application knows that that thing is going to be done. So when you ask for low latency you're asking for taps to put in the neat system a set of properties to implement low latency, then the application has to decide if they want low, latency and Dave.

T

If they do, they request that property and becomes a profile of things that they get I prefer that to try to say, I want to legislate. Less than X or I'm trying to send this much because these things I find very hard to actually match with things and I've played with RSVP for many years and I know. There are many many parameters here.

T

Simple word goes down the stack. This is actually a profile. The profile has to be expanded by the system. According to the policy information base, we get candidates back now we have probably two things to race. We have some notion of interfaces, preferably PVD, and we have the notion of which transport we should use.

T

So in this case, we have decisions made at two levels. We could argue which level we do first or whether we do them in combination. This is all good fun and I suspect in reality, we'll end up with different versions for slightly different views. This may be different versions for whether we're doing dare to ground versus connection-oriented. Maybe if we have slightly more PVD information, we get heavily into path selection, but ultimately we have to choose a path to work where the after views of Transport to work with- and we end up with this.

T

um So if you were clever you'd realize I didn't talk about anything which I will put in the session there I believe the session layer exists. I know it's the ITF, but there are functions: a book transport on below apps and yeah yeah. They really are yeah and the post sockets api talks about this I mean neat. We don't talk somewhat much about keeping such and connections alive over time. We don't talk about the mobility and moving of connections between one bear another or carrier in the sloppiest case.

T

This need to concentrate it, particularly on the things that we could actually implement now, and we have a concrete way of doing it. I have slight concerns about making things overly complicated when we do this adjustment, because I really want people to buy into the taps API and if we start bundling too much. I have fears and good news. Is we've bundled this much so far in our project and we're happy with this so and we can figure out how much is good.

T

The slightest callback based I think that to me is an essential part of the whole thing. This is probably a more fundamental change than any of the other changes and it's a change. It's already happened in the app development world, so whatever we do with the taps, API it has to be callback based. That means, when you make a read, call you're told first of all to go. Read you read when you make a write, call you write and it tells you it's written.

T

You can argue about whether it has to tell you every time it's written or whether it's tells you but all of it. But you get a callback. You also get a callback when it goes wrong, some sort of time outdoors pay, it's gone wrong and that's the same in neat for all the other messages. If you want to pick up an event for something happening, you get an event call it comes up. So this means you have to rewrite the programs in a different way. It turns out most people already writing that way.

T

So it's cool next steps, uh-huh trying to be quick. We have cold. So we would encourage people to play with our code or ask us how our code works. You don't have to Chloe that you can just simply ask you, can go to the github and get it um yeah. I fail to draw asking out by the deadline. I was I was feeling that diagram looked terrible.

T

So there's a red one if you haven't seen it, which has the ASCII art in um what does the working group do next, which is kind of passing the book back to you? People, because I think this space for some sort of document, which is architectural and defines some like API, maybe more than one API. So is that the way we go before I write a shedload of text? I would really love to know what sort of documents you want to produce and you.

A

Say something about your the example apps that you've got running on this sorry, can you say something about the example apps.

T

We just send bytes backwards and forwards, it's good and we we we have caught from Mozilla, which runs a version of Firefox on it, and it's got issues because Mozilla Firefox is a pretty big package too, on top of it, but it it sends packets backwards and forwards, and we have other apps as well.

P

Mia Kulemin, so I have um two concrete points say one is that this approach is like the main planet. Primitive is still a stream and I really think it should be a message and.

T

Well, datagrams are not streams ever mirror yeah.

P

So, but what you kind of do is like as humans, a stream and if you, if you have data, Gramps and I kind of merges into a message, but the base primitives should be message. That's hot! No.

T

P

T

P

It's not how a messages of cool.

T

Messages are what we are.

P

And the second part is that I think you have today, even if you say like I, don't know all this. What we have hidden in the association is something that you don't have to do in the transport layer. I think there's already thinks in the transport layer that have like long term state and that need to be covered. Yeah.

T

We have a long term state as regards path information, so we information about what transports. Where can we keep information about? What lacy was seeing on paths that soldiers stuff, but what I've avoided was was keeping things through that we would call socket handles relating to connections we'd set up in the past, because I think that belongs to layer. Above now, you could argue. We should specify that as well. Yeah.

P

So I mean like I guess. Our main use case is, if you take TLS, basically as part of the transport layer, then you need to handle session resumption as well and that layer but yeah.

T

P

Say that suite so we.

T

Do TLS, but magic happens when you handle the keys.

G

Cairo's, so I would have a minor quibble with the way that you phrase the API, the API, the requirement for the API model. You said event-driven, it seems like. So there are a number of different. There are a number of different models that are used widely in different languages.

G

Right I mean sometimes you have an event-driven model, but sometimes you want blocking, or you want to be able to program in an imperative style or you're, using continuation passing style or something like that right, I, so I think what you want is the API to be able to express a multitude of programming models, and it may be that that having a having an event based model is sufficient to implement all of those I know. What are your thoughts on that I? Think.

T

The event based model has got to be the way we go because and even when you get right down to the interface, we're actually sending blocks of packets, etc. And what.

G

I understand I mean the I guess. The point of having an event based model is so that you're not forcing blocking on people and you're, not forcing polling right yeah.

T

Okay, so, okay, so I can't send much to answer your question, but I mean the system we have is based on libuv. We have above this a socket Shin which rewrites at all the sockets. So clearly you can do many things are built. The callback beckus api, including reverting back to the previous version of this world, where we use sockets, so we actually have that was running code as well, so I think the callback based API could become the basis, and on top of that, you can build other approaches.

T

If you really want to do you do you want to respond.

C

I

C

For cutting a line, I think the main point is that we don't want blocking api's to be the native thing, but I mean event is one way to do a synchronous you can have completions like there are many models for asynchronous I think any of those are on the table. It should not be one model of a synchronicity that works here and yeah they're pretty much equivalent. Well.

G

And and well, the thing is that there are existing applications that are that are developed in a specific model right and so some are going to be using continuation passing style. Some are going to be using event loops. Some are going to be yet doing other things and it just. It would be good to make sure that that whatever the API here is, can support all of those models, and so I think just and it may be that that an event based system is perfectly sufficient for that right.

C

And they recognize that, like in the neat document, especially in the neat happy eyeballs document, it actually goes into a fairly detailed discussion of the libraries that are being used for venting. That's fine for an informational but I think anything that is produced as this is an answer. Kpi should absolutely not have any reference to how the implication should do it for a synchronicity.

V

All right second set, because you have to compose this network interaction with other interactions like form from the queue or something generally.

K

V

You use corvex I want to use an application data thing in my coop. I can not just the writing the written data or the received number, because else I have to use Global's and.

V

The other thing was.

T

That's fine and I think we should have this discussion going on. I mean one of the things I was trying to say was we we did it all in this way. We also did the socket layer manipulation on top, which reverted to socket I guess the way in which Mozilla actually called their internals of their product isn't something we want to talk about, but it's not quite the same as the way in which we envisaged it. So there's yet a third way is the API is being used. My.

V

Second point was about this low watermark stuff, where I can limit the queue size, but the queue size itself doesn't tell me very much because I don't know how fast my connection is. What I really want to know is what an Jacobsen called the packet sojourn time through the queue you can do this today with the socket API. If you have the buffer occupancy and the last received acknowledgement number and you can calculate the time packet, the last packet used to traverse the queueing time, and then you can get some estimate.

V

If your message will arrive in time and that's somewhat missing here is a way to determine the actual time. I pick it stays in the queue your real time person. Yes,.

T

Yes, the question.

A

V

T

Looks like it's framing.

K

Sort of packets online, your.

V

Multimedia is something.

T

Which is an application where you really want that sort of functional? My.

V

Use cases of the SIP proxy we had this problem in the wild.

T

V

The sip requests in the TCP Q and times out before it gets sent yeah.

B

This talks about this Policy Manager and there has been other discussions like racing and a payable and some other discussions here. What's your thought on that and policy manager like? How do you see like I mean? Do you see like this is a part essential part of this drought that you talk about it or you see like this draft can refer to some other things. That I mean okay.

K

So it doesn't clear on the policy.

B

Manager like how you need to use that okay.

T

B

T

Okay, so I see happy eyeballs as two parts I see some one part that generates a list of candidates tree of candidates. However, you wish to express it not all of which you will try and then a method for trying them. So in my own thinking, I see the happy eyeballs as how you choose amongst those lists, you're getting, but there's a question how you generate that list, because that list also includes preferences for bandwidth or cost or protocol or reliability, or maybe timing, etc.

T

So, there's another bit in here and how that interfaces with the stuff that's configured versus learned versus accepted from the application. So that's the bit where the policy major fits in and I drew it used as a big blob, I didn't say what it was, but that's kind of what he does is that something we want to define here.

B

Well, I I think we're discussing some of these things here, like within tens and for suckers and racing, so obviously there's a there's, something that we're gonna anyway talk about yeah and I, just like confused about like whether this is a fit to the draft or is it should be something something else yeah.

T

Maybe my final point is, since you kicked in, come back again and give me another chance and is maybe we're beginning to reach the point where we want some sort of architecture, which is why I try to draw big boxes on my diagrams. If we can come up with a architecture, we can all agree on them. Maybe we can start precision in the language we use for these things or maybe be slightly more agreement. If we have the same language.

A

Thank you Gauri. That's actually a really useful point. So the next agenda item is to talk about the document number three or number three.

A

So when we created when we wrote the Charter, let me say something before you comment on it.

A

So when we wrote the Charter personally, the model in my mind was we were going to spend a bunch of time wandering around looking at existing protocols trying to abstract the stuff that was going on inside of them. There was a lot of interest in energy, Ian sort of coming up with a happy eyeballs equivalent for transport protocols and, and so the text that came out of that discussion was doctor after we come up with min set was well we'll.

A

Do some experiments we'll build some stuff, we'll write some documents that describes what we built then maybe we'll recharter. We could certainly do that right. There's there are several activities going on I'm, actually personally extremely excited about the the level of energy and the fidelity. The work that's happening in this working group I think it's I was I was a little depressed for a while. The group was kind of quiet, but but you guys have really kind of lifted.

A

My spirits and but I think that Gauri I think summarized it rather nicely, which is you know we we could go in start writing rfcs about the stuff that people are doing or we could try to come up with an architecture that maybe synthesizes some of these pieces together and then write some RFC's that reflect that and I think there's a really interesting combination of perspectives here between sort of the neat perspective and the perspective, and that you know need, is very much supporting the existing technology.

A

Suite post sockets is trying to come up with something that will support stuff that hasn't been invented. Yet there's a lot of commonality between them, I think there's a lot of agreement, so my gut tells me that there is an architecture that can be extracted from this, but it ultimately it's going to come down to whether people in this room want to work on, creating that it may be that you know. Maybe people would rather just build their own stuff and so I don't know if we're gonna converge on that today.

A

But I'd like to hear a little bit from the folks in the room about what you think you'd like to do I mean: do you want to start if we're going to start taking a milestone? Three, the way it's right now, I'd say: we've got a set of candidate documents we can take at those in his working group documents and they'll be published with a title like something like taps candidate. You know post sockets, taps candidate, media taps candidate.

A

You know racing or something like that and just you know, they'll be sort of snapshots of a bunch of research work in progress around this general problem area, or we can try to tackle the nut, which is you know? How do we push these kind of interesting perspectives together where there are some obvious tensions in them.

L

Hi Brian Trammell. Would it be problematic for you to put like my fourth slide back up? Would that be easy or hard to do.

L

um So I I agree with you that those that's the choice before us is that we could, we could basically say okay well, this is the you know, we're doing experimentation with post sockets right now. We intend to do much more of it and we could basically frame this as yes or the third slide yeah that one um number three not number four show proof of soft proof with for a fast night. What? What's that even for go on back? Sorry?

L

Third slide, not fourth, one but I'll point out that that's basically an architecture right, those are relatively high level boxes, each of which can be implemented. I mean like each, which can be implemented within the system in a in a given way, but it's um the most abstract API. We can come up with. It still says something and it it basically.

L

You know this right here is Norka picture that you can hang the rest of this off of now, whether or not it's the etabs architecture or not, I mean obviously want some things under working group change control. um You know we discuss about it and we we tweak it and we change it and we, you know, put different um multiplicities on the lines and we change the colors and stuff and whatnot. But I would point out that we basically kind of already were there great. So that's that's the story of an architecture so.

A

I would be pleased to hear whether people who are working on other projects agree with your statement. That's.

L

What I textured that.

A

That makes sense to them that there's stuff inside yes.

L

Cool so feel free those people.

A

Weren't in a line.

P

Me I could have I'm not working on another project, but I'm. Not sure of the word architecture is actually the absolutely right. Word is probably more a framework or whatever, but the main point is this is on a higher level of exception than the need.

P

Interface is I, just wanted to say to need architecture. So it's it's not that it's like one or the other. It's just a different level of extraction and we should decide which level of extraction we want to document. We could even do both I'm, not sure if that's useful, but that's a decision. We have to make I.

A

Think if we do both, we either have to say how they're connected or we.

P

A

That these are two independent activities. No.

P

No I think they I really think they should be connected and I don't see them as independent. It's I.

A

Would like somebody for me to agree.

J

Macavoy I mean for two points. First, answering the request on somebody from need saying what they think about this I I think it's useful, useful architecture I agree with that as an architecture, I agree with how Maria says that being a higher layer.

J

Another point of why I actually went here is that the way you talk about the third item? We could basically, you know if we would leave it as the Charter is now it could just be more or less done with the documents we have now. If I read this correctly, I would disagree with that, because the point of the Charter I mean at least this was. My interpretation- is that this is about about how you will actually implement that stuff and the current documents yes, I mean happy hour.

J

Bowling is one part of it, but some part of it. There is more so I think we really need documents explaining how to implement that stuff and I mean I, appreciate this being good, but that's not all. That needs to be.

A

Yeah I'm. Sorry, if I was not trying to say that an architectural document satisfies charter item, three I was saying that that you know you could do. You could talk about implementations and the designs of those implementations without talking about architecture and about and without connecting the implementations together in some coherent fashion and.

K

I think that this.

A

Is an approach for doing that, but it'll take some work from everyone involved to sort of like mash their stuff in so it fits into this model. So you can see how they're related right.

K

I think that would be a useful thing.

A

I also agree, as the questions like I was asking Philip earlier, I think I'm. A meditation experience is extremely important for validating some of these ideas. Right.

F

A

That, okay, we have three minutes Gouri.

T

A

T

Disclosure I working brought both projects so, but I see this more as a framework and I see and the the postdoc is tufas higher level and okay.

A

T

Seems incompatible, but there are different levels to look at the problem, whether we want to look at both levels or whether we want to collapse it into just one. I, don't know.

R

Another instance are working, it I agree that we should try and merge and have the connection between them. I think that maybe we need something that is I think this is a good starting point for the architecture. Maybe this is a little bit too abstract and maybe need is a little bit, not enough abstract, so we just want one API. Maybe it's a little bit somewhere in between.

C

Tommy Polly, Apple um well I, do think this probably seems like we should be rewording, Turner or item three either way, so that should happen. I think with the terms of like. Do we have implementations or do we have architecture I? Think if we just stop with the implementations, that's interesting, but that's essentially we have today and we have a good amount of experience. I think we have enough experience in the room from various implementations. I also think from people who aren't speaking up to give their examples of how they do this.

C

When I've been working on the post, akut stuff, this is agreed at a very abstract level. I think it could be very interesting to say, let's all come together, work on the abstract definition. There's a neat implementation document like here's. How we decided to express this I could write a document on how I have decided to express it, and we call it all, maybe something like post luck or something abstract on top of that, with various levels of how to implement this depending on your system and that way we have both something.

C

That's like I think what I wanted to get out of post or at eye level architecture is something that is forward-looking enough, that it doesn't become irrelevant in ten years, but then we all should have also should have things that are concrete enough, that someone can implement them now, and those are two prongs that may end up with different documents. I didn't.

A

Think we're gonna have the architecture piece, so I'm actually kind of happy that that looks like these things together. Okay, we.

P

Could have it again, so um I also wanted to make one comment that we've been I'm going, hear a bot nap approach, so we've been analyzing what there? How can we generalize this and then going up, but I believe that somebody who reads the tapes documents later on this document would probably be the starting point and you would start from the more abstract, as it is a general thing, so that something we should keep in mind when we try to align things.

W

So I already see that the whole post sockets need the socket and stuff every sing. The people are now going towards common language and trying to at least find a common language and I think this is the way how this can merge, but I think it has to do em to merge in the DES separate documents and then converge to something later on with the common language we later on.

W

Half so I would rather be for keeping it separate and trying how it merges over time with the experimentation with the experiences and then find out over time what makes sense. What's the right, abstraction, I think this? We will see this later. I I think we can't decide this now.

A

C

One thing in response to that: Tommy I think one thing that could help speed up that process or make it easier- and we had mentioned this earlier- was to have a maybe a terminology document right, because we have a great survey of transport protocols. We have a great min set stuff, but there are these words that are floating around, that lots of people have different meanings behind of paths and endpoints and connections, and maybe just having one terminology document that all of these other ones, implementations and abstracts api's can reference.

C

So we don't have to worry about the differences there and get confused, would really speed up the process and I think we would end up realizing what we have in common much more quickly. Somebody.

A

Want to raise their hand to submit a draft for the next meeting.

A

I think the bar is low, just terminology, but a lot of it could be copy and paste are you was that it? That was that a thumb or was that a little hand house.

A

Alright I think we'll probably have to take this to the list. We don't time to talk about it now, but maybe we can have some hallway discussions this week and see if we can get some folks.

K

A

Think this was a really productive discussion. Thank you, everybody for waiting it out. If you didn't sign the blue sheets I'm as my bad for not nagging you about it, please do it. Those numbers are actually useful when we pick the room sizes for the next week. So for no other reason, if you want to have a seat in London sign blue sheet, thank you're ready. We're done.