Internet Engineering Task Force 92, 25 Mar 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: IETF92-NVO3-20150325-1520

Description

NVO3 meeting session at IETF92
2015/03/25 1520

A

All right we're going to started, welcome to NV 03 the second session. Today we have a shorter agenda. We've got one hour. We have two presentations, they are on a similar topic, which is Oh am in overlays, so hopefully the two together can encourage a good discussion afterward if we have extra time which our agenda has I. Think 20 minutes for we'll have some open discussion that we would like to focus on any questions around the data plane adoptions.

A

So we'll have time for that. If no one has anything to say on it, we can talk about other things or break up a few minutes early, so jumping through real fast, the intro slides. First of all, as always, this is an ietf meeting. So anything you say here is a contribution to the ITF. The note well describes what that means in more detail. So please be sure to be familiar with it.

A

We have blue sheets which have been started to be passed around. It's going to be obviously in a room this size, it's going to be tricky to get it everywhere, so I appreciate it. People can help each other out, passing it around without too much distraction. We do want to get everyone's name on it if possible. We have John and ignis, just as on Monday are taking notes here today. So thank you both and we are presenting these through WebEx.

A

So if you want to watch them on your screen or you're listening remotely, you can see them there today, like I, said shorter agenda one hour long we're going to hear first from Eric nordmark and then from Diego Garcia about some oam options, and so I'll take one moment to see. If anyone here wants to bash the agenda and nobody's complaining so I think Eric, we can get started.

A

Well, you can stand there, I'm.

B

Supposed to stand here, how.

A

It is one that we're going to be on video right, Fiona.

C

Mike there is a second camera out of that.

B

Particularly when you don't remember what you wrote on the slide, because you did the slides like a week ago and like I, don't know what's back there, but so um trace rod for overlays.

B

Go to the next slide, so there's three pieces here, and this is not an exclusion of anything else. We might be doing whether I am right, but this might actually be a useful tool in the OEM toolbox. So keep that in mind, as we think about you know different different forms of OM. One of my observations has been the traceroute on the internet.

B

Today tells me a bunch of stuff that maybe I shouldn't be allowed to see, but actually do you see them, maybe for historical reasons or maybe because it's actually useful right and what s with you as we do you overlay networks. We might actually lose some of this visibility by design as opposed to buy policy, and this is actually not that that hard to change right. We have actually prototype stuff for vehicle on a dusty stuff. So so this is sort of the typical path that you could see. This was sitting at work.

B

It looks similar from home. I have no idea what these routers are. So why is this useful? One might ask well because I can take this stuff and put it in a trouble ticket. All right and I can send it off to our IT department and they can go figure. It's the problem on our end or is it as they go out to the ISP, or is it somewhere else making forward the stuff along and if I have a temporary outage?

B

You know this is sort of the only way you can feed some useful information in other than saying I can't get there from here right. So I think that being able to take this stuff and say I can send this off. This is what I saw at 9am this morning and someone can figure out well. Was there something wrong on this particular router at that point in time, as opposed to having to look we're right? So that's the the the high-level thing so today there isn't much policy to enable people to block here in fin.

B

Isp wants to hide ur topology from this from IP trace route. They sort of have to either tunnel things which they do with mpls and whatever I clearly can't see everything I can't see the whatever optical trunks that are used underneath and whatever right mpls. There's a bunch of things I can't see, but I can see if they're a bit I could see more than a binary didn't get there and they could filter things out if they weren't as well, not send any ice in peace. Next, oh I have the clicker.

B

So when we come to tunnels the IETF, if you go look at it, we have actually developed two different tunnel models. One is called the pipe model and the other one is called a uniform model. This has been done both for diffserv as well as for TTL, and the pipe model is the one that's commonly destroyed or most commonly used for for env3. That means that the env II actually puts on a fixed TTL, that's configured in the outer header and then on eager. So you just throw that away. Oh.

C

That's all right, oh yeah, it doesn't really matter. We.

B

Can do them in this order to burn, ok and and with a uniform model, and you basically are counting the TTL across the whole path under layin and overlaid together and the behavior there is that the the ingress nve will set the outer TTL to be the inner or the original TTL minus one and likewise sort of copying in the other direction and decrementing on the egress. So these two behaviors are described, but people might mostly think about the the pipe model.

B

So so some of the issues that I could show up that are particular to to overlay networks and in particular, also to overlay networks carrying l2. Your Ethernet frames is that when you, when you have a a a DX land type network, well, if I'm going from one host to another they're on the same subnet or the simple case of them being on the same subnet, one idea, ping and traceroute I see nothing because the packets don't go anywhere. There are no IP hops involved at all right and yeah.

B

The ARP request may or may not. Timeout depends whether you are propagating ARP information in the control, plane or sort of doing flooding and whatever, but in order to see something, you actually need some access to the ingress and the e. So if I can you know get access to that thing, whether it's a virtual switch or whether it's a physical switch I would actually need to do several things.

B

One is check whether the underlay has an issue by doing a ping or trace route, but I also have to go check some set of tables that map my overlay traffic to the underlay and describe how their scent right, whether it's mappings from from vlans to the NI IDs, whether it's some sort of flooding tables or whatever MAC address to remote IP address of the nve right I need to go inspect all of those things and then I can tell it's this thing actually. Well.

B

That might not be sufficient, because I'm not likely certain that the packet even reaches that nve and that's very hard to tell because it's effectively in l2 service- and you know you can't really there's no widely available tools to tell that from a host.

B

So if I don't have that that access to their me, the only trouble ticket I can generate you saying it didn't work right and here's the IP address. I tried to get there, so we have those two models.

B

So so I guess it doesn't say that so basically, this means that I can actually get disability underlay. If I actually use the uniform tunnel model, because now I seen p errors, TTL exceeded that treasure. Our users will be sent from everyone along the path. Everybody will uniform the decrement, ppl, the ingress and egress and me included, but not sufficient, there's one more piece. We need it's about how we handle. I simpiy errors, and this is an area we played in before not for the TTL exceeded, but for other things.

B

So the idea is that so these desizing p errors sent by the underlay routers will be sent back to the source of that IP packet, all right, standard, ICMP or IP, forwarding behavior and that's the ingress mV. So the ingress mb gets these IP packets are sorry, gets these.

B

I simpiy errors and it can go look inside and say: oh, this is a DX lon, whatever envio 3m cap packet, that was in error, I can go extract where needed actually come from and send it back basically deliver an ICP error that makes sense for the original host.

B

A

B

At an example of how this would behave is- and it's there's one special piece here which we haven't done before, which has to do with mac addresses and mike headers, but first one the host censored and packed with the TTL of two it gets encapsulated and as part of that, the TTL gets decremented to one, and then it comes to the first underlay router the decrement and checks TTL and says this expired right and it sends back the the number three here is the shape of that I see the error.

B

It has the envio three, whatever VX town header in there and whatever it had inside, which is a mac and IP header. And then, when that packet comes back to the ingress MBE, it can form the icing Pierre at the bottom. That makes sense for the original host and I don't know if I put examples of what this stuff looks. I have some example.

B

So that's that's all there is to it right. Well, there is another one which is: you probably don't want this uniform TTL for all packets, so the host here says I'm on the same subnet as this destination host. There could be some protocols that assume something about TTL like TTL doesn't occur. Man if I send it with TTL one and gets there because we're on the same map that would break.

B

If you applied this uniform model all the time there could be other ones that use TTL, 255 and check out the receiver same thing right, so you have to do some ingress selection or you have to do some selection at ingress to figure out am I going to use the pipe model or the uniform model for this, and you could do that in any way you please write it could be based on and here's where it sort of you, the policy norm comes in saying, should I do this for this customer?

B

Is it for this port this particular originating IP mac address? Maybe I have some dedicated trace infrastructure that I run in the overlay for this right. You can do anything, you want something if you want to do it at a granularity where a host can do it. That is also running running other traffic. You want something in the packet to select this and there's two things that you can easily tweak in the packet and that you can set in the linux trace route.

B

One of them is the dscp or toss white, and the other one is the UDP source port that you use. So you can actually set up some things on the ingress and be saying okay. This customer is authorized to do this right. I will going to say that this customer can, if it uses this DCP value or whatever or the CDP source port I'm going to give those packets, the uniform, TTL processing model and there's some difference.

B

Is that some trade-offs in graph between that in terms of entropy and whatever I think I have just one slight love, so I think I'll go through that one before questions but and the the second piece is: if I want the behavior on the on the egress. I have to have some way: I'm, instructing the egress on a per packet basis. What TTL behavior do you want and the easiest way of doing this stuff we played around with different ones, but this is one is actually getting one bit in the in the B header.

B

That says, please apply uniform, TTL behavior at the and note that there's been discussions about sort of OEM bits with slightly different semantics, just a fly that it's not the same right. The OEM bit that's been discussed is this is an oem frame that should not be delivered to the unhold. That's not the behavior! We want here.

B

We just want the this TTL behavior, oh yeah, there was one you can do this, whether I simpiy errors, you can do it with our agenda packet too big, for instance, you can actually use it as a way of detecting that I think my underlay should be configured to be able to carry the encapsulated packets, but maybe it's misconfigured we've seen. Customers actually do this right and and it's hard to find out. Well, if not that hard, you send a small ping packet things work.

B

They run TCP and it doesn't tcp times up, because TCP sounds full-size back at well. Now you have to figure out okay. What should I look for right and you know other sort of unreachable whatever so in via con me prototype the stuff we found a bit at least at the time, wasn't being used for something else, then of its that's still the case with all the VX lon proposals, but and- and it basically says, here's the D capsule aiding behavior.

B

That's all that bit the semantics of that bit and there are some things that might surprise people when you do this. So the first one is your underlay and overlay are actually diff different, addressing realms right and you could think of this as separate verbs or whatever. But, like you know, your customer has net 10 right. Well, you can actually run n, bo3 or VX down whatever, where the underlay use is not 10 as well. It's perfectly valid because it's l2 or l3 they don't meet anywhere.

B

Well, when you do this, and that's not the example- but when you do this, the trace our output is going to show the underlay and overlay IP addresses. So you better know that when you enable this thing, don't think that you can ping those those underlay addresses is any useful way right, but there might even be the same addresses that you're using your overlay inverse K right. So as long as you know that okay I can do it over later, a drop which sees whatever overlay part of my path.

B

I can do this combined trade route that sees both and if I, take both of them and put them in the trouble ticket. Someone can actually figure out. What's going on. The other surprises what you see, as you have different IP versions. This notion I'm, getting and underlay ICMP error and generating an overlay, I CPR means taking IP addresses that are in the underlay and showing them overlay.

B

Well, that works well if it's v4 486 / before, because I can fit 32 bits in 128, which is what this example shows that basically, these six ICMP errors that have 96 bits of leading zeros or whatever it's something doesn't really matter what it is. It's just something that provides useful information for the guy who's. Gonna, look at the trouble ticket and I think that's it hey! Well, I guess.

B

The other thing is I think that this is a useful tool right and- and it might be something that people should consider, but as they look at different encapsulations, so questions or.

D

Comments David black um I mean we lay something that I think was said uh rather strongly at the sputters food bought this morning. Dscp is not a sitting in mechanism. Please don't use of this. Actually I actually think that your most common case, you actually don't need anything in the protocol, which is that, if you're using a virtual switch, that's co-located on the same box with your traffic source that can be set up by that can be set up but set up via via other means. There there there some interesting possibilities on thinking.

D

I had figure out how to make that one work. Yeah.

B

That's only one of the cases right, but also you do need, if you want to do this at the granularity of a mac address or an IP address, right, basically dedicate one of those in the overlay for doing the trays. It means that you sort of have to request people to allocate one of those for you right. The whoever controls the address space for the overlay has to do that, and by putting something in market in the packet of some form. You are avoiding requesting that your complexity, I.

D

Think I think keep looking because said this. This is not a good use. Dscp. Your discussion of how UDP effects, ecmp I think is I. Think it's fatal to use of the two usually use the UDP source port, because all you need is one is: is one one arm of a four-way ecmp to be out end and the OEM will never see it and the the trace? My goal is to find the traffic doesn't yeah.

B

Butbut tracer actually varies destination ports as well. So if you can't necessarily pick a particular on, if you do that right, but but because of the varying destination ports, if as long as that's included in me, if you can gosh, if.

D

You can get a trace for our tool.

D

That's prepared to keep retrying with a whole bunch of different ports and tell you which ones working which one which ones didn't you might have occluded something's wrong with libby, with with each with ECT cmp down there on the other thing is just a warning which we've had in the context of the endcap discussion for routing, which is watch out for nested in cap sooner or later, you're going to run out of payload size and ICMP and not be able to rebuild what you want to rebuild without state in the end ve on the way back.

D

B

We have 512 or five 776 bytes to play with, but this yes.

E

Sorry share on the line. I think what you're doing is a nice hack was really architecture it wrong, because you're you're violating all the layers, you just be thought a minute later to you're just going to uniform on my model. This is absolutely wrong. Well, there's a rag no, but there is a question about.

B

What is the purpose of layers right and.

C

One of them is providing.

B

E

Structure that underneath that.

B

Doesn't mean that the abstraction should get in the way of the ability.

E

This is a one hop and early whatever you wanted to you. Sorry overlay. It's a one hop over late. Whatever you want to do is one hung over, but.

B

This applies even when I have a combination of overlay and underlay hot saree, the same thing I'm being able to say these packets are marked with arcing a disability.

E

To the underlay, if you had terminated the little match, the layer, 2 and I've done an IT inside the excellent of that I agree. But this is wrong.

B

Okay, Sean opinion.

F

Really yeah, I was going to actually say something similar to him like if you go to slide 7 right do.

B

I have slide number Pedro. This.

F

Is an old gun one from them.

F

So you talk about the nbe, encapsulating the packet of the TTL decrement. So that's really not true right because the enemy, if it's a layer, 2 and EE, it doesn't decrement the TTL at all correct. That's.

B

One of the behaviors in terms of providing this despite this, this uniform ppl model write that as n cap and decap countless hops there as well. In addition to the underlay, all right be good and the reason I picked that one is that it seems useful to showing all parts of the path.

F

So if it won't work, if it's a non IP packet, if you're transporting well.

B

How would you mark the non IP packets right that clearly, you can't find an IP header there to go. Do anything with so you'll just need. You have to do that. The pipe model for non IP packets, but last time, I checked ninety-nine point nine percent of the packets that we carry here r.I.p, but but but it doesn't break those right. The key thing that still works.

F

Still works, yeah I.

B

Know that this is different, because people are used to thinking in layers and that's useful for some things all right, but that doesn't mean if you make it significantly harder to see. What's going on and and as a result, making it operationally difficult to deal with these networks. I think you know we should pause, write and think. Is that really what we want right?

B

F

E

F

E

F

Tends to be that you let the lower layers take care of themselves, and then you have someone that can query each lee. Yes, very and.

B

I can see cases running jwhite the.

F

Comp in sa correlations assuming the but.

B

It is somewhat hard I think, because I can see running a you know, even if I take in some of the failures I've seen right, even if I was running BFD. Doing continuous monitoring of my underlie path, making sure that that one didn't fail. I still have configuration issues when this stuff fails, because the mapping from l2 into the tunnel right is somehow misconfigured work, whether it's you know, sort of the port come of eel on TV and I, or some the flood list or whatever I used to do right.

B

That stuff is misconfigured and those things the underlay OEM doesn't help. I really need to connect the button, and the overlay one by itself doesn't tell me anything either and failed and I don't have a good way of connecting the dots all right. So.

F

If, if you had your end hosts sending out some kind of messages that looked like the data, they would go through the same data part and they would discover where the problem is right. There.

B

Would just be dropped on the force on all right, it'll.

F

Get dropped somewhere and because it's not received you will know, I know.

B

That they disappeared somewhere else. I could tell that it's not working, but I can't tell where it's not working so.

F

You're I mean what you mentioned. Was that concerned about the place where the mapping takes place from l to r us.

B

F

L-3 and that mapping takes place at the english point and.

B

Egress right, but he.

F

Grace and so you would know, but.

B

But as she start complex, more complex, topologies right, you might end up with multiple sort of chained and Bo three tunnels: ain't going through some routers that route between them whatever, and then it actually gets hotter as well. Yes, I could I could give some idea about at in the overlay. Did it get to this router that you know you did get over the first tunnel right I can tell that from the overlake high throw but I, then sort of so now, I know which tunnel forgot debug any on. So.

A

Just a quick note: we this conversation, we move on to our question and the limit it to the people that are currently standing just so that the next speaker isn't as well yeah. So.

F

Just follow the steam come quick yeah, if so, that the only so my biggest concern with this would be the scalability issues, because let's say you have a medium dm's using the underlay, each of those million VMS tries to start doing things like trace route. That underly needs to respond to, and you know how do you control something like that? How do.

B

I control the festivities.

F

B

An IP network I, don't know some.

C

You turn it off. I.

B

Baca lies in peace now, bark all treasure, I.

B

I can do it from home today. I can do it from here. I can do it from work. Icmp I mean Tracy artworks in IP and the routers don't fall over in the internet and there's plenty of people that I'm sure I dossing it right now as well right. So I don't know if that's any worse, but that's hi.

D

Tom Herbert, so on one of your earlier slides, you mentioned these two models: the pipe model versus the uniform GTL model yep. So my question is: should we actually consider the uniform tunnel model as kind of the operational model, and the example will give, I think, is david's example. If I have 2 n bees are co-located with a host with the VMS.

D

Ttl was relevant in that case if they set a TTL of one, that's one single tunnel, so therefore the packets will go. However, if my bm's introduce a routing loop in the overlay, then theoretically I can go through n squared hops in the underlay. Just to just have one packet eventually run out of TTL. So I'm wondering if this is like a problem that you might want to address here, but.

B

I don't think we can change it, because there are protocols like ipv6 neighbor discovery right that you know that run between hosts that are in the same subnet and they assume something about tikyo and the fact that we are building basically remote bridges. Using is this overlay technology that has to for those packets to still work it has to be default to being apart model. That's.

D

It so for a layer to network that might be true, but for layer, 3, we're.

C

D

Customer just a default route, the TTL could actually correspond to the underlay. Yes,.

B

If you're, not if you're, not if you're, not carrying l2 service over the env3 network, yes, that's correct, so.

D

I'm, just saying maybe this is, is.

C

D

C

D

B

Yeah, I think that that's a good discussion to have right, which is which is separable from the. How does this behave for for l2 right.

E

Sorry shower on the value again, so I want to give you one alternative solution which is architectural you're correct, so you can do, for example, a internet link, trace route or linked race, and then that can trigger a idea trace route at the NB and then you can breathe reply. The results, that's our cadet, very correct e.

B

Well, that's a different way of doing things. It's not a tool that people are used to today and.

E

Then again, learning yes,.

B

E

Want you want to twist the architecture and destroy all architecture soundness so that people are not know.

B

What is the purpose of art where we should take this to the bar, but the purpose of architecture is not something that it's a way of organizing our thoughts and having ways of creating obstructions. It's not a way to forbid thing. People from from.

E

Solving is architectes avail for reading people to do things insane things. It is a way of keeping everything sound and you know working in all different conditions and situations. So we.

A

Should we should continue this conversation, but not right? Now there is one more question in the back and I think we missed you as sort of a punishment for sitting in the back of the room since the.

B

Mic on the mics was not on our visa in the.

A

Previous session equipment, what do you.

G

So I wanted to add a couple more surprises. We that's.

A

Darrell Italy.

G

We've had a yeah, Darrell Bluett. Sorry, we had a lot of experience with deployments with trace, rot in hell three overlays and a couple of the gotchas that we found in the in the operational internet. Ours are often times we get much less than 500 bytes of data back like so in the cross. Af tray tea time expired. Coming back to the end we get not enough are.

B

You still seeing the only eight bytes of the of the transfer.

G

Coming a lot of errors, because.

B

We fixed that was added to Rada requirements in 1995 I, understand.

G

That this sometimes we.

C

Did that because, apparently, all right.

G

But I'm not pointing fingers, because it's almost every vendor as a few, you know as a as a device out there that does that. What we found impaired interesting is that when we do cross, I have often times we don't get the enough information back to determine the actual. You know IP address that we need to pass back to the bus back to the host okay.

B

So that's that's an additional supply. It's very good! I was assuming that that had gone.

G

Away, yeah and then the second surprise. Well, I mean you know that windows and stuff doesn't use UDP for this, and it's a real struggle when the tracer line itself doesn't understand the cross, they have information, you're passing back so a lot of times you get stars and stars and stars. We really wish you could get better information back to your to your users and then I think.

G

The other thing is that there was a lot of comments about about performance and stuff in here we've shown and implementations that are doing this- that do a small cash just for trace route on the on the encapsulator d caps later that handles this functions and if it it's just a small buffer and if it overflows, because too many people are doing it, then the people get more stars than they would have gotten otherwise, but it hasn't impacted. The env performance on the on the control plane are.

B

You saying that they remember that trace for cuts well,.

G

Data, you have to identify the traceroute, that's coming from the host right and you know when you get packets back to n ve, where to send that, where to send the the traceroute back and sell three that I'm talking about. No.

B

Not when you get the whole payload right, yes, if you only are getting eight eight bytes back sure, all right exactly.

G

So I got to figure out a way to have this, but what I'm assuring you is, the hacks can be done without compromising the performance of the than B and the final. The final thing that I'll mention is that the chip doesn't matter: okay, okay,.

B

That's you so thank you, make sure.

A

Diego you can use this to.

H

Hi, so well, what we're presenting is a bit complementary to what derek has been doing. It's a oam from the MV e to the MV and to do a bit of data but yeah. So basically, the idea is to to have the capability of doing traceroute and doe am from MV 12v, and some of the things that we wanted to cover with the generic overlay was having you know, yeah the first point: what eric mentioned that in the existing ping and traceroute does not work very well for overlays.

H

For the reasons explained before, and but some of the things you want to verify is yes, how do I find information from the underlay how to find information from the path that the packets are taking? How do I verify the behavior of the network when there's ecmp and multiple paths of the cmp in the way and how do I deal with the consistency of the control and data plan, a beta plane programming, especially when we're talking about an env that might be in a co-located with the server in the hypervisor?

H

But the controller might be an open flow controller or some external controller. You have to have some way of ensuring that that data path that you believe in BGP is one thing: has it been reflected down to two to the forwarding plane have some way of doing continuity, checks for verification, some fault, isolation as well, and performance in terms of delay and packet loss measurements. So, in terms of the requirements for the overlay framework, what we wanted to do is have the the overlay. Endpoint should be able to send them some form of frame.

H

That is follows the same data path and reaches the incest and reaches the end and ve, but at the same time somehow doesn't get accidentally leaked to the vm because or to the end device that is behind the end. Ii I think you know, while it might be acceptable in some production environments where every so often euro am packet, actually reaches the the vm that you were trying to target we're trying to avoid that as much as possible, and then by that means it's the the end.

H

Point should be able to differentiate, what's annoying packet and what's not so that it can parse it appropriately and reply back to the originator. So we have some information and, and the idea is that we are able to achieve OEM for both applications in the layer, 2 and layer, 3 overlays we're seeing that the MV 0 3 is moving to supporting not just layer 2 but towards also virtual distributed virtual routing environment.

H

So you know trying to build a the OEM in a way that supports both mechanisms and and again the other part is the fact that we could have some capability for doing trace paths, maybe not necessarily from the client endpoint and in fact we're not looking at it from the client endpoint, but rather from a central management system or the env itself.

H

As a network administrator I'm able to look at how I am going to be able to find the path that the packets are following, find if there's one dead link in a nice, EMP bundle or in an EMP fabric, how do I find that, particularly that is being troublesome, and sometimes you could forward packets, based on purely on you can try from an MV e to another MV using ping or some direct under late test, but you're still not fully verifying that your VX lat encapsulated packets are making it through that?

H

Indeed, the endpoint is properly configured and and part of what we're doing. Also with this framework is the ideas that to support multiple overlake technology, so you know not just VX land but MV GRE mpls over GRE mpls over UDP and some of the upcoming and capsule ations are being proposed as well. So I mean try to be.

A

H

So that we can I mean not not be the universal panacea, but have having some way of having something that is covering the different use cases that MV 0 3 is looking at so in terms of the packet that we were proposing is I mean having this is very similar or similar to what was being done and in some of the m LSP trade routes and LSP om that was previously specified in mpls, and we we were looking compared to what was presented some time ago in in a previous ITF meeting in terms of versions.

H

We added we actually added a version, a header to the packet itself, so we could support multiple evolving formats for the packet and you'll notice that there are timestamps and sequence numbers very early on in the packet, and this is to as much as possible have the capability of doing hardware, time time, stamping in the platforms that will eventually support it.

H

We tried not to do the timestamps stl these randomly in the packet, but rather in a fixed position, so that if a hardware implementation could potentially timestamp it directly and not have to look at it and punted to to a control plane to a supervisor. So and then what we are is the sum tlv formats for the type of encapsulation that we're trying to 20 a.m. and optional tlv is to identify the end system that so, if we're trying to figure out what's going on with a destination MAC address.

H

So there are two parts of the draft. One is verifying the tunnel from a from an env2 and vso from beat up to v tip and the other one is ok. Also once I get to this nation, can I try to find certain information about a particular endpoint, a particular host that I'm trying to reach, and I might not be having the the visibility to get there and so in terms of the overlay ping, in terms of how do we originate and detect this so in packet?

H

The idea was to have use some form of router alert bit in the VX lan heather and I'm not particularly set on it being one particular bit. We had a proposal and we have a draft implementation based on that, but you know there's been on the excellent GBE, there's its own OEM bit being defined. We could coexist with that bit and just use it. You know look at the interpretation of how the bid would be used and mpls over GRE or mpls of your DP.

H

We could do a router alert label that would keep things clean in that encapsulation and in the inner Heather encapsulation. Here is where it gets a bit trickier or a bit different for everything. That's tunneled to tunnel verification, Soviet up to vetem verification, and the idea is to actually have a full V explain compatible frame, so it has to be at least an ethernet header, and we were looking at doing actually an IP plus UDP headers as well, but the echo request and should have a destination MAC address.

H

That's like a well-known neck address that we can trap and we can prevent in case this. Oem packet arrives to an MV that doesn't know how to interpret the router alert bit. Then we don't want that packet to be forwarded to the end host, so we don't want to have it. We don't want to cause that packet to be flooded or so on. So we're trying to use one of the slow protocols MAC addresses, so that we can actually trap it and the same thing with the IP address.

H

If there's any chance that this packet would be routed, you know we're using at 127, 00 sedo or you know a local host address, so we don't send the packet accidentally to anyone, and this would allow us to properly identify the packet as the destined to that to the v-dub itself.

H

So we can get an idea of how trap it and send packet send a response back to the host and and in terms of a layer, three overlay the encapsulation would be similar, except we wouldn't be adding that additional inner ethernet header with a mac address reserved from iono and so.

H

Yeah in terms of again the the packet information we had a couple of cope, you know call side, dress is specifying the router alert bit in NV, GRE and VX lamb, and we had the mac address as being requested and we have the validate the control plane and data plane. The echo reply with the return code, so this is the encapsulation for the reply. The same idea again.

H

A similar mecha dress, similar destination, IP address and so on, and we would have different return codes in the packet so that we can able to look at what the different possibilities of errors. The different error types, different error codes were so the type of request that we're looking at in the moment. It's basically an echo request and an echo reply. The echo request could also be used to do to perform a trace route as well and we'll see how that worked and the reply mode.

H

There will be a no reply, so we can specify the type of reply that we want and we could specify do not reply mode so meaning we're just sending probes that somehow the receiver will keep.

H

Statistics will measure like they have a sequence number, so you can measure gaps in the sequence number to determine random packet loss or estimate packet loss and or you could do a reply via a routed, ipv4 packet or ipv6 packet back so not encapsulated into VX lan, just routed back to original to the originator or you could reply by the overlay segment in the return path by other is the return tunnel if they return tunnel is available.

H

So sometimes this is having the possibility of having to reply mode gives you the chance that maybe your outgoing tunnel is working, but they repeat the response tunnel is failing, so you can send the response to come back through and through a routed infrastructure and not tunneled. So you can see ok, I'm, seeing that my unidirectional path is working with my return path. It's the one, that's failing! So having that possibility opens up some. You know, wait it slightly more interesting troubleshooting options and then yeah.

H

We have a couple of return, calls as well and for the path traced procedure. I mean the idea is to be able to send the echo request with outer TTL again. This is originated from the Vita or the VN ve.

H

So we have the rec control over the TTL, we're not doing layer crossing there, but we would be setting the outer TTL to a particular value and we're able to send I get back and forth and for every increment we could also randomly or sequentially increment the source, UDP port, so that you can exercise a certain number of hash. Hash is for your ecmp fabric. I!

H

Don't guarantee that we're going to discover every possibility, CMP path, but if the user could request that for every TTL, increment I send 16 different UDP values and then I increment once I send 16 more values and so on, so you can get a good idea of sweeping the underlay and and the response for the TTL expires. There would be again dependent on part of the ICMP response. It's interesting to see the the point mentioned before that. Yes, people are, there are some implementations out there still replying with only 8 bits, so 8 bytes.

H

So we have to see you know how much information can we confirm 8 bytes if the 512, bytes or 500, and so bites were available? That's you know it's easy to determine the whole thing, but with only eight bytes, we'll have to see how we can do that, but there's short of also requiring the intermediate transient transit notes to implement a some version of this OEM which potentially they could do in later steps. The relying on the ICMP time exceeded message.

H

It's whatever we can have today and and basically in terms of a the other part, is no not just verifying the tunnels but verifying the end systems and what we specified. There were a couple of different TLDs where we request to ping or to verify the forwarding tables for a particular MAC address, IP address, Mac, IP combination or just a row packet that we specify as a certain number of bytes.

H

So, in that case, the idea is that we are able to look up against the layer, two tables or the routing tables of the end and ve, and we are able to get a response of whether that mac address exists at all. In the end, ve that we send the request to.

C

H

And again we have a couple of requests but yeah. Basically the next steps are, you know, looking forward to comments and feedbacks. We already got some comments on the mailing list and in person today, but we're so we're looking at working at that they.

E

Share on the value brother, this is another architecture, a wrong proposal. I'll tell you why. Why do you even need that OEM power relay overlays an adaptation layer? You are not going to forward based on overlay, just put an overlay, the excellent header ten-forward this packet- you cannot, it's like they say: I want an oem for eater type. This is a tap tation layer react the excellent at encapsulates your your l2 header into an IP packet. That's it! Why do you need an oem for the overlay I.

H

Mean there's some pretty much it. What do you.

E

Want to detect in the overlay in the VX lab, what do what error do you want to detect in the VX line.

H

Is the tunnel working at all.

E

Or yeah what tunnel that is not VX, nine is not the tunnel. The tunnel is IP.

H

I'm not sure I follow I mean.

E

You cannot forward the packet based on the excellent hitter.

B

So so so, when I was standing up there and and you said that this is wrong, like I, guess, I didn't quite understand what this was, but as far as I can tell what you're arguing is that everything that has to do with the uniform model is wrong.

B

Yes, because the uniform models has something about TTL, behavior and you're, arguing. As far as I can tell that that is architectural e incorrect impure whatever and we've had standards doing this stuff for mpls for ipv4 or ipv6 for a number of things for four decades.

E

B

E

Can't what can you do hear that you cannot do with ethernet, OEM and IPO IPO? Am that we have? What can you do here? Extra, that's extra to that I mean.

H

With easily whatever.

E

You want to do here, you can do it with ethernet, oh yeah, for the overlay and for with I do em for the underlay.

I

This this is why you 10 M, because an NDE might encapsulate to one or more two or more nvs on the other end, and while it's encapsulating to it, it wants to test to see if this encapsulation path is up and if it goes down, it wants to switch. That's why you need Oh, am packets to be originated by the NP and.

E

Say not originated by angry, you can originate by Utley, so you just want to test it check the mapping from the basically, you want to check the correct VX line. Header is added. That's what you want.

I

C

You can do.

E

C

I

You're testing the path from the encapsulating MV e to the d capsule eating at me and the packets are being originated there. So you can have a you, can do it when you.

E

Can do it the property, the IP joining.

A

I think we should probably this is a specific, should specific discussion. We should have there's a fundamental here around who gets to do troubleshooting if it's the attendance system keeps fading in and out tenant system or just the n ve owner I think maybe we should start from the point of view of what we want to facilitate and then talk about the architecture. So.

G

To them to that line, I have a couple of common terrorists. A couple of comments about this. um First of all, having done a lot of over the internet overlay troubleshooting over the last six seven years, I'll tell you that it's absolutely essential that you use the same UDP destination port that are using for data packets. When you try and decide if something's alive or not and you're paying, because very many times will be a middle box is blocking some port, but not another, and you know it.

H

G

Little trouble.

H

So we are, we are using exactly the same. Udp port does.

G

Misread the slides, no.

H

There's there's an inner UDP port inside transported all the way inside the overlay that could be, but the outer V X lon is definitely the same. We're not we're not touching.

G

That second point is: is that I I have to congratulate you of using a union of every idea when it comes to troubleshooting.

G

But I will say this is that one of the other problems with operationally is that, if the tool that you're trying to troubleshoot with is okay and somehow it so complicated that wasn't implemented, probably you're, really stuck and so I encourage you to find a very simple small subset of this, get some operational feedback and then add to it over over time. Yeah.

H

So that's the other thing that we were looking at after some of the conversations we have these days was to split it into one thing: that's verifying purely the tunnels and the other is and systems, but just less common yeah.

B

A clarifying question: Eric nordmark, so you said you have you so you have this rather alert or OEM bit so is that does it have exactly the same? Semantics is in the GP draft. How do you know and.

H

I as far as I read from the GP draft I could understand the ideas that is a packet I mean it would. It means it gets punted to control plane and in.

B

Particular in the DB drive it means that it never gets forwarded to the understand right. That's not the semantics.

H

Yeah, so that's that's my ID I don't want that packet to be forwarded. The the thing is: I cannot guarantee that it will happen for non impure VX land. It will not happen for non router alert bits aware implementation. So.

A

The Sharona myself I, I I'm actually but I'm sorry I'm, actually going to cut the conversation on this topic. It seems like there's a lot of good conversation. That's happened and a lot more that we need so we'll we'll get together with a few of you after this session, to chat more about what the next step would look like. But everyone should please read these drafts and comment on the mailing list. So thank you. Thank you.

A

So we have and I'm sorry I am really sorry to cut that conversation short, but we have about 10 minutes a little less than that left before we're supposed to be done here and I wanted to make sure there was a chance for some open discussion of the data plane, adoption that is in process right now.

A

If no one has comments on it, we could break up early. If you have any comments, questions or whatever this is a chance to to speak about it for a few minutes it just as a reminder we are in the process of adopting. We have three calls for adoption out right now for the VX line GPE for goo and for the.

A

Why am I forgetting the name of the last one genève geneve where's Jesse? He can yeah I.

B

Call it janea, we need a pronounced asian guy, but I, and do we pronounce it in French I mean there's like beer, the beer of working group that's pronounced in German right. So oh.

B

Yeah then, this boot, as well so I, guess sort of the sort of balance between industry in there I clarify. I think it would be good if we could. The ideal thing would be if we could come up with one encapsulation right but I. Don't not sure that how that were I can happen, and even if we do, we still have things that is getting deployed out there like the basic the advanced off.

B

So I don't know if it's useful to think about whether the participants here and sort of wearing a combination on the IDF hat and they're sort of we're, deploying, there's anyhow hat, which you know happens out in the real world sort of our amenable to thinking about other ways that we could. We could prune things down a bit and not end up with sort of these three potentially being deployed, plus whatever things are out there.

B

Existing and one thing that might help with our stuff is, is sort of looking at the the encapsulations considerations thing, and there are some thoughts about extensibility, there's many things about how to encode extensibility in the pocket so that we can make it make it a bit more future proof. If people want to build hardware that can use the extensibility mechanisms, which you know the encapsulation. The design team that was looking at this stuff actually came up with some good things that we wrote down would seem to sort of point out that unconstrained kill.

B

These are actually very hard to deal with in hardware sure, with the encapsulations, all of them the abilities that we can skip the whole thing right. We can pass over these extensions, but if you in the future say well, there's some security extensions that are useful, I should be.

B

It would be useful to be able to have the hardware actually parse and insert and actually do something is all about so and if people I don't know, if people have already looked at that stuff and say here's something that would make me tweak my encapsulation and if people are willing to do that, we might be able to get fewer of them. I mean I, don't have a problem with the working of adopting these things instead of refining them but I'm thinking about what is the world going to look like three years from now right?

B

Are we go or five years from now or whatever? How many of these things are going to be deployed out there? That's that's my real concern and I want to make sure that the IDF is part of this conversation, as opposed to being completely sidelined, because people deploy whatever they want and we don't get the benefit of sort of the additional review and sort of broader thinking in their yard.

A

Yeah, thank you so I just to respond. I guess I see this as a present process. If it works well, we want to adopt and document things that are going to be deployed and or are deployed now at some point we could do more than that. We could maybe come up with a one of them that we think is is going to be. The you know must implement standard or perhaps there's a way to combine ideas or whatever I'm not going to.

A

You know prejudice what the outcome would be, but I think trying to balance out this recognizing the people are going to deploy things and we need to document it on one hand versus it'd, be nice to kind of come together, more coherently and help the community on the other. I think that's a. I agree with your point. I think that's a path we're going to try to walk any other.

C

E

I this Jesse Mesa.

C

Might default on one specific point to Eric made not so much on the general process.

A

C

As far as the engine trans kill, these things so I actually happen to know, I think five implementation subnet at this point in commercials of switching silicon in the multi target range, including person, kill these so I just want to kind of challenge the conventional idea for a little bit. Stalker.

C

E

C

On context seem so it, it seems like we give up on one encapsulation, because there's a new one, new encoding or encapsulation every every few weeks or months, but at least have a defined mechanism to negotiate which encapsulation is used by the two NV is that will be a start and one small comment to Eric and the OEM, the other one about race route is overlays, inherently support multihoming, so that I think any address in advance. So there may be multiple ways to get to the same.

C

The only one but.

A

Let's yeah: let's keep the conversation on Mike if we can so.

I

To add to Sharon's comment, Dino.

A

I

Is dino sorry in the list working group we added an encapsulation type format to the mapping database and right now, its enumerated seven different, encapsulations and growing. So just let you know that the mapping database could tell you what v-type address to use as well as would encapsulation. So it's a form of negotiation, not back and forth. Just what I will support and you pick one of the ones that you would encapsulate it yeah.

A

So just sort of add to that. That means that the control plane has multiple data plane options as a requirement right. This.

F

Is elan go for mental? I think I want to make the same comment that I made at the virtual interim as well now trying to come up with one single encapsulation format.

F

That's a standard from my ETF is not going to help the industry, because you already have many different and you're going to create one more that the industry may or may not put up creating more problems than solving it where it might be useful, is in the TL vs we're still it's an early stage where it's possible there, where the IETF can come and define some some tea LVS that are useful for maybe future hardware to parse.

F

It has a request saying, and so this is a good time to start, but have that conversation and define some tlv that could be adopted as a standard within the IETF just.

A

To clarify you mean maybe using the same T LVS across different protocols. That's.

F

That's correct, so if you look at the tlv format, at least between two different protocols, I can see that it follows exactly the same format, so you can always go and standardize the tlv, even though the outer encapsulation that carries the TL vs might be different. You can still sanitize tlv former that could be useful in future, and it could I mean you. You can get interoperability as well, you can make it work, make them work together and also you can make the hardware that parses it to interpret it correctly consistently across the encapsulations.

A

And so will thank you, I will take the two people are standing here and, and then we'll, but I have to close the ques, because we actually really do have to quit on time today. The plenary isn't here next, but before we go on Tom, who has the blue sheets and and more importantly, if you haven't signed the blue sheets, please find them and do so sorry, good Tom.

D

Hard work so I just wanted to comment on that. So one.

F

Thing we are doing in goo.

D

Is actually defining several extensions, so we have a fragmentation extension coming up with a header, checksum and a few others. Security is actually very important and I think most of those could be generic, so, for instance, if you wanted to apply those to tlv right now, they're in flag field, so we can actually somehow separate out the base encapsulation from how you extend the protocol and say that, for instance, DLP's is one mechanism.

D

Flag fields would be another, but the content of those may be either the same or very similar, and that could be a common another common point.

C

B

Not my call so quick response to longer that yeah.

A

B

That's a good idea to think about different things that are common in terms of I guess not necessarily till the encoding but the actual payload and its semantics right and a it ties in very well with one of the observations that was made in the end caps design team, which is that the times when we had defined these sort of extensions and having actually use done right, they sort of have become useless, but it can get implemented in hardware or the middle boxes, just drop it on the floor or whatever right.

B

So maybe there's some sweet spot, whether it's some security, cookie or something that people might want where they want sort of better and protection in this packet infection that we could.

B

We can sort of say: okay, here's a starting point: let's pick this one and only define whatever how big it is right and that, if that helps sort of get across the hump that we have some hardware support for something that we can get into Nick's and switches right and then we can see well the extent it needs to be next, but maybe they're just doesn't software. But there.

A

B

That might actually help in terms of ensuring that this is likely TB be useful down. The road right, extensibility becomes.

A

Thank thank you for the comments. Everyone. Obviously we have actually the starts at the beginning of some good conversations here on these two topics. So please don't lose the energy. Let's take this to the mailing list and the hall pay and I appreciate it. We'll see, everybody soon will will announce an interim meeting at some point. Another virtual interim meeting and and you'll see some of the details on the mailing list. So thank you and I see one blue sheet coming up here.

A

If you haven't signed it signed it, please come up to the front and sign it there's another one over there all right. Thank you. Everyone.

A

Just one of them on the clipboard.

C

C