DASH Community Weekly, 18 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SONiC DASH Workgroup Community Meeting Jan 18 2023

Description

PR289 - Still needs volunteer
PR296 from AMD: Add DASH metering to Sonic HLD
https://github.com/Azure/DASH/pull/296
Slides reviewed
DASH moving to Linux Foundation
Q&A

A

Camera off this is our 52nd meeting together. That's like a year of meetings technically so a lot of different meetings together, I'm gonna stop the camera here. So last week we um talked about Chris's PR that we merged and we had prune 297 that got merged. We talked about metering a lot and we talked about the the new construct from AMD and we talked about moving to the Linux foundation. So I was wondering this week did. uh Did anyone have a PR first? They wanted to cover or something they wanted to cover.

B

Well, did we get the updates to with all the comments from last week.

A

Into the metering, doc yeah.

B

Did the updates actually happen based on their comments, I.

A

Believe they did prints, did they I can look through my email to look for the GitHub notifications, but um I know that a lot of comments came in and Prince was doing a lot of updating yeah.

C

I I updated based on our feedback and and the pr is also updated. I I'm just waiting for that that CLA thing to pass for the merge uh looks like after we migrate to Sony uh the.

A

Next thing, the next Foundation.

D

C

Not getting that.

B

So did you talk to the group about the next Foundation? What happened over the last week? Christina? That's.

A

What I was going to do here so um because we did, we did talk about you share my screen again. We did talk about moving to Linux foundation and then uh so. Over the last week, that's actually happened where we've uh we've migrated the dash repo to Sonic net and the new URL I'll post it, but it's going to be github.com forward. Slash sonicnet, forward, slash Dash, so here.

A

Any questions from anyone on the call was.

B

That completed I think it was.

A

Yeah did that for us, so.

B

Then, um when they does, this affect how they communicate to this.

E

The mailing list, Etc will change. uh It will be Sonic uh the next Foundation, something like that is yeah.

A

E

A

Yeah so right now we're a Sonic hyphen dash at Google groups.com, and that will change, uh but that that'll take a little more time. But in the meantime we can still use what we have Gerald for sure.

E

Yeah, it will change to uh Foundation.

F

So the old URL is still forwarding to the new one correct.

F

G

F

I hope we won't have to I mean I, guess ultimately, we'll have to change those hyperlinks in you know different documents that we have, which point to one or the other and then otherwise those those links will be broken.

B

We used to do a sanity check on all that, like we can at least soon that's all just works, but yeah probably we need people to go through it and see if it does work. Yeah.

A

And that's on my plate for sure that's an action item for me before the end of the week. So that's I've had that on my plate.

D

Yeah, it shouldn't be hard to do a grep command through all the MD files and look for URLs that are pointing to the old one.

D

um Yeah and a couple line said script could replace everything globally, I, don't mind, taking a look at that. It's kind of thanks Chris, my mindless got work.

A

Thanks thanks very much yeah.

B

Yeah, we don't want people losing their work so.

A

Right right so and- and we won't okay, any other questions on that- uh Community team, sorry, I'm, managing multiple windows. So, if I sound a little distracted.

B

We need to do the same thing for the dash under uh ocp site.

A

E

I thought that would remain unchanged.

E

um Since Psy is still under ocp, but I I.

B

But we didn't have that under that before I, don't think did we.

E

No um yeah, that's true: we just uh merged the dash API file to ocp sign.

E

And the PDF also is still hosted. uh The test framework is also still hosted inside for dash.

A

Yeah thanks Russia.

F

Was there ever a distinction between SCI for different kind of devices, because I was a Psy I mean he never said, Cipher switches and Cipher, dpus or site for Dash and so forth? Right so I mean are we? Are we really saying that you know that distinction is coming in.

E

B

A different, a whole different networking layer than size ever tackle before so somewhat different.

G

There was even distinction between devices before there was already PSI for gearboxes.

B

So just check it out and make sure that there's no links again if we don't have similar issues. If there's no issues, there's no issues but.

A

B

You you definitely come under different governance when you move stuff under ocp and Linux Foundation, okay,.

A

B

Other, even in the Linux Foundation, we have to update the governance there.

A

A

Okay and uh any PRS we want to cover today.

E

And Don had some.

A

Oh I'm in the wrong one here: okay, Anton, do you want to display or do you want me to.

H

um You can display only one priority is 315, I think that I actually added greens to you, but maybe after removing it's uh has changed.

H

So this is our extended, we're not doing Netflix cases uh right now after uh Gohan merged script. So it's passing on CI, so I need to fix the CLA.

H

I would like to have comments. Is it there.

A

So you have mercha on this one.

H

Well, this is the Greek festival here not to find it just suggestions.

A

A

This was the you know.

H

It's a test plan and the best cases so yeah.

A

Awesome, these are all brand new.

H

Yep and I also updated. You know the brv's test plan only 217, so right now it exclude eni this plan because you and I have been merged in the 217 right now includes only like new stuff, after merging I will update 217 once again, but just want to let everybody know that 2017. uh It contains also changes in some good documentation and some links and so on. So it's anyway, it's needed.

A

So Vietnam is the starting point: Implement Dash mechanisms, vmdbm blah blah so route; LPM, underlay state, flag, All, State, tracking Telemetry, monitoring, blah.

H

A

Yeah, if anyone wants to take a look at this sorry.

I

I mean I join us a little bit later. I saw today we're going to talk about the metering uh AMD folks, we'll talk about metering right, so we.

A

Can we can? We were just going over whether anyone had pull requests? They wanted to talk about. First.

I

A

So we were looking at uh Anton's uh 315 here. We can certainly talk about metering. If we want to shift over to that, Gohan did anyone else have an urgent PR. You wanted to cover.

J

I have a question um earlier with the site API for dash. What is the whole site API for that? How do I? How can I find everything that is supposed to be supported in a dash device? I even referenced some documents, but it's like bits and pieces. So if I go to extensions, um experimental under assign I can find some Dash apis, but then the underlay is from Sonic. Then I found another document that specifies underlay portion, but I don't see any like Port step even like I, don't know. Bytes TX bytes are excellent. Port.

J

Those stats are available on sorry, but not in the documents. So I wonder we don't have stats in a dash device or somebody missed them. So how can I find the one single place where I can see all the API for a dash device.

J

I

I think that here we have this right, so you know the other one I think is uh you know all the uh automatically generated that you know those attributes has to be supported right. So that's uh that's a key part of the overlay, so then underlay I think has been listed in this document.

A

Here, where I'm pointing to Gohan.

I

J

So here, for example, I don't see any port stats.

A

J

A

Top packet police report, here's port so.

J

We have no, that so is that means Dash device does not have any ports that.

I

Okay, I, don't know I press that did we I mean I, think the you know do we need to post stats there or so.

C

I think the initial idea was to have uh stats and counters based on eni um those things right. So we didn't.

J

C

Yeah I I think, um if that's something missing, then this is the place that we need to add uh like.

J

And where can I find a similar list of all the dash apis.

C

Right, that's all the extension headers um at least one um thing that I can suggest is um Marion has provided a app DB to psi mapping. So if you look at the same document, there is a mapping section so for the current scenarios, it's it's already covered, um so I think.

H

F

The mapping section yeah I, know if you're looking for PSY apis right as Gohan was mentioning. If they are Auto generated from behavior model that we are putting in, then we should just always basically go there and then just run this simple script and then just Auto generate it, because yeah.

G

F

You document that thing it'll go out of sync with, however,.

E

You know models.

F

Right so I think we we just basically are always looking at the stale data and the documentation. It's always be the right thing to to download, generate and then see. This thing know what it is right.

E

The API should be standard, though to refer to, and it should definitely as you're saying, uh should have the latest from the behavioral model and that's exactly what we had asked for about a week or so ago, during the break and Marianne generated a new set of apis based on the current behavioral model, and that's the one that's been reviewed and checked in now, but but the community will refer to the API header file inside. You know.

I

So I think that you know in the document for the overlay, you can just point the link to the latest. uh You know the master for the for the generated API right. So then, then you know we won't be able. uh So then we we get we'll get rid out of this. uh uh You know outdated these things right so she'll be always the latest.

B

Isn't it I mean you can get the apis? Are they self-documenting? Is there any any.

I

I mean the API is all the attributes right. So you know people look at the header files and those are what you need to be. You know all of them need to be supported.

A

So you're saying here in 3.2.9, we need to have a pointer to a link in the master that Marian updated inside.

I

I think you know you can still have the table, but also put another thing. Okay for the latest do I refer to wear plates right so because this.

C

I

I think it's a it's a it's going to be difficult to to maintain.

C

Yeah yeah yeah.

B

Foreign I I agree, but in the future, that's probably true, but I. Think at the beginning, it's nice to have these documents um and yeah I have another concern, though, is when we keep saying Marion compiled this like do we have a compiler? Do we have the the API generator? There is only.

I

B

Course, yeah, okay, so we can all do that.

I

E

I

Yeah I add the link there.

J

Do we have a similar way for the underlying I'm trying to find a way to get all the apis that the dash device should support or for the underlay I always need to reference to that readme file that somebody needs to update manually.

C

Now, for underlay we have this section right: I mean I, I'm, not sure what you're looking for, because all the supported underlay is part of this document and for overlay it's the auto generated files because underlay there are many other attributes inside which, uh which probably is not applicable to Dash, and that's why we explicitly called out here.

C

So if there is something.

G

Else we can always add to this.

C

J

Yeah, so if I merge this table with auto generated, one I will get the full yeah.

C

J

C

Expectation, that's right is.

J

There any way to Auto generate this table or I always need to refer to these documents that is manually created, foreign.

J

Way to get programmatically the underlay portion same like I, will do for overlay for overlay, allow Auto generated and I'll programmatically get get it so I can have regression against it, but for this one is there any way to get it? Programmatically or I always have.

C

To do that I'm not sure about that yeah I, don't think so.

F

There is an auto generated, I, don't think. There's an auto generated thing for the underlay, so underlays are are manually defined apis. They are generated like you know, you know, sort of like they are referred back to the you know some Visio document uh as opposed to the behavior model. But, however, these these documents are not no no way is auto-generated, as we have it for overly.

J

Yeah, because what I'm trying to do is basically regression where I can generate the whole list of API, supported and then check against the device, if it actually supports them, if they actually work or not- and you know, provide like data hey, there are 300 APS that needs to be supported.

J

Only 297 worked or something, and these three failed, so I was trying to get the whole API that is supposed to be supported and then from device I get the apis that is supported, I check them and you know, I can run some regressions that way.

J

Try again. Okay, I'll input this manually from this table. If that is.

K

Not I do besides.

J

This table go ahead besides this table and the auto generated one. Is there any other API that or any other place, that I should look for API, or this is all.

A

uh Prince or Gohan is this everything.

C

Yeah, this is everything, is I, mean I, don't have.

A

Payments that we know of.

C

Yeah but like like you mentioned, if there is something you find missing, then probably give it as a comment, and we can add to this list because even recently, we added two or three extra attributes. Yeah.

J

Yeah I mean if it's about it's about or if it's missing, that's fine I, just don't make sure I'm looking in the right places and there is no other place to.

C

J

C

Should be it, this should be. It.

D

C

Thank you very.

D

Much we're going to add Port stats to this table, then, is that a.

A

D

A

Gonna, go in and either I can do it or someone can do it and um you know, do a suggestion to the document of the pr so report stats, okay,.

A

Okay, do we want to do different PR shift over to metering which uh Gohan wants to talk about.

A

Going once going twice or PRS: okay, okay, maybe we can shift over and talk about metering then. So. uh Last week you joined a little bit late Gohan. So last week we talked about. Let me pull this up here.

A

So last week we talked about metering from Prince Gohan, section 1.5 in pr313, and then we also talked about pr296 from AMD. So which did you want to go into.

C

I think it's PR 296 for.

A

Me, okay, all right all right AMD! Would it be okay to talk about this for a bit and did you want me to present? Do you want me to present, or do you want.

K

To uh no I shared my screen.

A

K

uh Yeah, can you guys see the screen.

A

Yeah I can see.

K

All right, yeah, so uh I think. Last week we had the metering requirements that was uh presented by Prince, so that kind of gives the background for this uh proposal, uh which lays out the uh MTP objects for the uh metering based on the requirements that was uh presented uh so uh I think uh last time when we covered this beer, there were a bunch of questions uh around this, so just to give a higher level idea. We came up with a bunch of uh slides just to give overall picture.

K

uh So I think we can go through this uh slides first to discuss the uh uh some of the concepts and then go back to the PR2 uh look into the details. So uh so this is the dash outbound packet processing pipeline. So uh so there is an extra metering policy that is being introduced right after the routing lookup, and this is separate from the security policies uh which have been allowed in action.

K

So the metering point and the result of this metering policy is a metering bucket ID and uh the meeting bucket IDs come from routing policy and mapping and then, finally, uh final bucket ID uh is fed into the metering update block which actually uh updates the uh the metering bucket uh TX and RX bytes, based on the bucket ID derived in the pipeline beforehand.

K

uh So this is just a high level picture, so I guess the outbound packet processing pipeline is being. uh There is a small update to this. By introducing this policy block foreign.

K

Pipeline so so these are the different blocks, which are sorry.

F

Just one one quick question on the back: if you could please go back yeah, but there are two things: one is metering policy. Another is metering, update right. So what's the difference here.

K

So yeah I think this block was already present in the picture. So uh this is uh this is the block which actually updates the uh once the bucket ID is derivation happens from routing mapping and the policy blocks, and once the bucket ID is derived the actual metering update, as in the uh incrementing, the counters happening happens in the meeting update.

F

Instead, they happen in the meeting update, but they both refer to the same bucket ID. Is it correct.

K

Yeah so I think once we zoom into the pipeline it'll be uh much clearer. Oh yeah,.

L

Essentially, the policy derives the bucket ID and the metering update goes and updates the counters on that uh architects.

F

Okay, so you yeah so you're waiting for probably egress processing before you actually update the number of bytes. Is it.

K

Yeah so um yeah. That is one way to look at it, but I think uh overall, the placement of the table uh uh really doesn't it's not mentioned here, because I think that's implementation, specific, but semantically yeah! You want the bucket the final bucket ID before you actually go and update the meter table.

K

uh Yeah, so this is just a zoomed in view of the pipeline changes, so uh you have the eni table uh which now derives a meter policy ID. So a policy is attached to a particular eni and you also have the outbound routing table, which is now driving the meter policy enable and uh meter bucket ID. So the meter policy look the result of the meter policy. Lookup is also a meter bucket ID.

K

So if the policy is enabled, then the bucket ID coming from the meter policy table, takes precedence over the bucket ID coming from the routing. So this is in line with the requirements that were established uh last time right. So we want the policy route and mapping. This is the Precedence requirements according to the uh according to the Azure requirements, and also there is a override flag that uh mapping gives in which uh gives the uh final control over whether mapping Bucket over right, so that that part is coming here.

K

uh The mapping table also gives a meter bucket ID and an override Boolean. So if the mapping override Boolean is set, then the mapping bucket ID takes precedence over the policy or the routing bucket ID. So, um and going further in the pipeline, the final meter bucket ID is uh derived based on this decision graph and that goes and indexes into a meter table which is on a per eni basis, and you have the TX and RX byte accounting happening in that meter. Table.

I

Sure I think maybe last time we talked the txrx, maybe it's not very, very accurate, maybe impound and outbound. It will be more accurate.

K

Yeah so I think uh yeah I think we. uh uh The reason why the txrx terminology is used is because I think in dash. uh When we refer to inbound and outbound we are, uh we are referring to the first packet or the.

K

uh If the VM reaches out, uh then it is referred to as outbound and that itself can create a uh the initiative flow and the responder flow and bytes corresponding to the initiative flow is uh is accounted as TX bytes and the bytes are corresponding to the responder flow is accounted as RX bytes. uh Similarly, the inbound itself can create its own two flows if the Inca, if the first packet is an incoming packet to the VM and the txrx is coming from the initiator or the responder flow.

K

um That's why we thought just with disambiguate right. We can use txrx here and inbound output in the other case, so.

I

No, no sorry, this part, I, I, didn't understand. I think it may be more I mean trying to make it more consistent right. So because sometimes the flow is initiated by the uh you know from the internet, sometimes the initiated by the VM. Are you saying you know uh you know if the if the flow initiated by the VM, then the TX means the uh you know from the VM to the internet?

I

But if the flow is initiated by the internet, then the TX means that the the uh the the the the packages from Internet to the VM I thought. That was uh a little bit. uh What I mean I I in my in my thinking, I saw you know. Maybe we should be more consistent, saying, okay, so if we are counting, you know if it's a VM going out he's always uh the out uh outbound, verse and otherwise he's always inbound.

I

You know makes the direction to the VM so that in this case we can uh make sure you know when we say.

I

It's regardless of the uh the initial packet Direction.

K

Yeah I think uh the uh yeah I guess that uh the reason is I uh Prince also added a clarification here in terms of the.

C

G

C

Vijay I agree with Gohan, because uh if we always set outbound does TX right. There's no ambiguity there, um because if we say TX just t x right, then it depends on which direction the packet is coming and then from internet. So that means it will become TX right.

L

Yeah, that is fine, I, think right in terms of referring to TX as outbound and RX as inbound. Only probably only this thing is. We need to be careful, because the outbound pipeline is very different from the inbound pipeline right, so the same terminology outbound inbound, is used, but this Pi, this metering, derivation that we are talking about here- is purely outbound. For example, if you see in the pipeline, we are starting from an eni that cannot happen when, when the packet comes inbound right, we start from a VPC. We start from uh from.

L

Source IP and then.

H

L

Inbound routing comes the other way around, so the um I think we need to be careful about outbound, inbound I. Think if we agree that we understand the difference between outbound Pipeline and the inbound Pipeline, and it's not confusing to use the same terminology here then I think we can use it I think about this thing. Idea was: if you use outbound inbound STX RX, then it gets a little confusing on how the inbound metering gets uh derived. But that's yeah. That's yeah!.

F

Yeah but I think the distinction needs to needs to happen based on where these buckets or these meters are tied to right, because at one point we talk about Connections and then connections have flows, and then you know, but it's not. They are not tied to flows as I see it yeah they are tied to routes or, or you know, mapping entries right.

L

F

L

Isn't it that uh once we do this derivation right, this is the potentially the first packet path right after that, once you have established the flow you don't, you can just use the flow to Short Circuit everything.

F

Okay, they should go once the flow established that everything is short circuit, but those essentially have either in you know, inbound flow or outbound flow, because the flow is unidirectional.

I

Sorry sorry flow is the by Direction. Okay,.

G

F

Is no no yeah here? Here's the problem is the connection has two floors right. No, no.

I

Let me ask you this question right, so so here's a problem right so say you know you have one meter bucket and you map to two uh flows. One flow is initiate from the VM. Another flow is initiated from the internet right so.

G

I

Map to the same meter bucket, but in this case, if we are deriving that using that will, you know, count the txrx, uh you know differently, but before into the same package, so some you know TX, sometimes can't you know from VM to internet, sometimes comes from the internet to the VM using that will cause confusing.

L

So there's still this thing right there still uh for your uh the example. The take it with an example. Let's say the VM is establishing uh um a TCP connection, a web connection out right. So.

G

L

In this uh this thing there are uh potentially the way to look at. The connection is two flows right. There's one flow with desk Port 80 and some Source port, and then there is a reverse flow when the for the packet to come back, uh which I think we are referring to the r flow which is uh dashboard whatever is that Source Port we used and Source Port is 80 the traffic coming back right, yeah.

F

L

Is correct, yeah right, so that is the this thing.

I

But no, no, but there's another flow right, so one flow is initiated by the VM to the internet. Another flow is from the internet to the VM, but if your txrx, depending on the initial Direction They you know on one floor, the TX is counting as uh going out from the real other flow is THC is coming from Internet to the VM, but they fall into the same metering bucket. So in that case, the meter in the txc, sometimes you know count the packet from uh you know in two differentiate in two different direction.

I

Right so isn't.

K

That a problem.

E

If we have dxrx for inbound and import connection and txrx for outbound per connection, that will.

C

Clarify but I think I think the Gohan the way I look at it is you have a meter table or a bucket ID Associated to an eni right? That's the combination and, in that context, right any packet that is going out of the dni will be the txr outbound and yes, incoming is rx so because this is the eni and the meter bucket right. It's just not meter bucket alone.

I

Is the corresponding to one VM right.

G

Yeah yeah, so this William.

I

Could have a you know, uh out of all the traffic could have inbound traffic right.

F

So that's where the.

H

F

I

To right so yeah.

K

Right but I think the first initiator, uh so if the first packet happens to be an outbound packet, you have already established the reverse flow right. So when the incoming uh traffic from internet to VM in your example, will get accounted against the RX bytes in that same bucket ID. So it's not like it'll go to different bucket idols.

I

So your first first packet is from the VM to the internet right.

K

So VM to internet.

L

Is yeah so, let's say a VM to internet right, so just uh sorry I interrupted so let's say we have, uh but the traffic is uh 80 to Port 2000 right from Source Port 2000 to 80., so that one let's say we derive a bucket ID of 10 right. So in that all the TX bytes going from the uh from the VM.

I

L

The internet we accounted against TX and, on the same, whatever traffic coming from uh Port 80 to 2000 will go on the RX uh bytes. Yes, now, let's say you are going to understand your question. So let's say there is another connection coming from some Port 5000 to 8080, 8080 being on the um on the VM right. So if this is the case, we will derive another bucket ID 20, which is not shown here right.

L

That is the inbound uh inbound pipeline right that will come from either the peering or the or the inbound routing yeah right. So from there we derive a metering ID, let's say 20 right in that we will have t x, r x again right. The TX could still have the same notion that DX is going from the eni out and RX is uh received by the VM if it. If now, if you have to be clear, being VM Centric right, but it will be two different buckets.

L

I

Okay, so but I I think maybe the missing part is this: uh um you know it's a metering table right, so it doesn't tells you uh on what base is used to determine whether it's called TX or arcs. Maybe you know, because you only have two input: Emi ID and the meter bucket ID, and then you get two output TXI. Maybe you need a certain input to say: okay, what is the melody that we use to determine TX and arcs? That makes me right here right so.

K

Yeah yeah so I think so that that the direction itself coming comes from the connection right. So once that uh the connection has the TX and RX flows uh established. So there is a yeah you're right. There is a direction component uh in the actual implementation pipeline which goes into the meter table which actually tells whether to increment the TX or RX, based on which flow it hit.

E

Direction is available in the pipeline. It just needs to be used in this lookup line.

F

So no I don't get I, don't get the direction. Part! Sorry guys, I think what happens is that for when you basically have a bucket bucket, has RX and TX at certain point? What is it because in your example, when you mentioned about you, know going from VM to the web from Port 2000 to Port ad, you are essentially are, are counting uh TX that are getting transmitted out of eni and an RX is the one that is coming into the eni, which is the VM right.

F

But but you know if, if you flipped it into your second example, you know when the connection is originated from the web coming into 8080 on the eni, then you know, are the TX is still eni. Centric, essentially saying that okay, the or or they are essentially saying that they are web Centric? What is it.

K

Right so so I think the txrx here is based on the first packet right, so for the so that's where that uh inbound outbound versus txrx, the disintegration applies. So uh for the second case where the internet is reaches out to the VM. You have the first packet as Internet to VM case and the responder flow as the VM to internet case. So in that case, the bucket the same bucket is associated with both the flows.

K

uh So if it is so I guess it depends on how the direction is derived in the pipeline so which is fed into the meter table so I I. Think yeah, probably we can add some clarification there. When.

C

I think the concern is in. The second example: are you going to increment DX bytes for the packet coming from internet or the RX byte.

C

For the second example for the inbound packet.

F

C

F

One in which you said from 5000 to 8080 right, yeah,.

C

Which one are you? Are you thinking about.

K

Right so yeah, so on the because it's inbound and initiated it would be DX and RX. It would be flipped around uh if we.

C

Use it yeah I, think that's where the concern is right.

K

Like right, right, yeah.

F

Right, yeah I think this needs to be spelled it out because otherwise it just basically says you know I, don't think it's sufficient to say that we initiated because that information is lost. I, don't think this information is basically carried, or at least you know is known so to speak, so I think there's some. Something else needs to give right sure. So.

K

I think yeah. We can add clarification on that, but I think uh the scope of this discussion was. We were just focusing on the outbound uh site pipeline right. So inbound is something that uh I think we haven't actually looked at, but yeah I think. Once we look at inbound, we can add that clarification.

I

I keep hearing that you know the inbound is, uh is not discussed or is not is subject to further discussion. I, actually, don't understand. Certainly you know this is the covering both team by now, but I'm sure, at least that part. So what is well, each party is not covered. Why why the inbound is not covered here so.

K

uh Yeah, the inbound uh I mean the inbounder pipeline itself, looks uh different because in inbound we have the inbound routing table. Currently the inbound routing table is giving the bucket ID, but we also need to introduce the policy into the team, Bond Pipeline and also go through the cases right. So that is the part that Prince was also referring to that.

K

We need to just go through all the uh cases from the requirements point of view and make sure that uh the metering bucket and policy at the inbound routing level is indeed sufficient to cover all the requirements. So that is an exercise which is uh kind of pending, so we are trying to do it in phases right so just make sure that all the outbound requirements are satisfied and then come to the inbox.

I

When you see all the outbound requirement is satisfied, but here the the countries, both the txa arcs. Are you saying that currently we only cover the cases where the initiator of the connection is from the VM to the outside, but we do not cover the cases where the initiator is from the internet to the VM.

K

Yeah I think essentially, the outbound pipeline is when the initiator is from the VM to outside. So uh that's the part we discovered in this particular outbound pipeline.

I

So I I, okay, are you saying that current the metering proposal is only cover is only counting. All the flows that is initiated by the VM is not counting any flow that is initiated by the internet from the Via.

K

I think the thing is uh yeah we're just trying to go in phases right.

I

No I understand we were trying to go face it, but I just want to get clarity.

G

um What has been covered? What's not covered yeah.

I

So the for the flows that is from the internet to the VM, they are not counted and.

K

I mean it's not yeah, it's not covered in this, but eventually we will. We.

I

Should I understand eventually, but but I'm, just saying that, once we implement the proposal, any inflows that is initiated by the internet is not going to be counted.

L

Perfect, that's right.

I

Yeah: okay, okay,.

G

I

Make that clear right so I, I, I, I I, didn't realize that you know that part of the flow is not covered. I think the the language I see the you know in the in in The. Proposal is not clear enough to for me to understand the that some of the flows is not counted.

I

K

Okay, so that's the reason why we have mentioned this outbound packet processing- and here also it's mentioned as our bone metered bucket derivation but uh yeah. You.

I

Can add something? Let's get cardio because I I didn't understand this I, don't know like how many of the Community member understand that this part of some of the flows are not counted at all.

K

Yeah I mean not in this proposal, not yet, but it's uh we have to address that as well in the next one.

L

From the requirements uh Gohan I think we have left uh in the hld. The requirements also leaves the this thing, as we will I think we just have to look through. It looks similar, but uh the way we derive right here we are starting from the eni there. We don't start from the eni, we start from the inboard routing table. So there is some slight differences, I think uh enumerating all the cases that are there. I think uh in the hld also I think Prince left it as we will.

C

Do it one clarification there is uh even in the inbound flow it is tied to the eni. If we look at the the hld's last requirement, so we have the the type to the route Rule table and Route Rule table is part of the eni. So.

L

The derivation I meant the starting point where we start here the beginning of the graph units. That is what I meant for inbound. It doesn't start there. So all I'm trying to say is the pipeline is different for the inbound we have to.

C

L

At those cases there and then we can uh yeah.

C

I think we may need to add an another section for the dash inbound Pipeline and the meter bucket derivation I.

K

Think that that.

C

Will complete that yeah.

F

Yeah, correct I think you know one of the things that I'm basically grappling with here is that you know without actually thinking end to end or having a holistic view of it. We just cannot design what the bucket and what the buckets values are like txrx, as as I was trying to basically say before. How do you really determine um you know what is TX referring to? uh Is it TX to the eni, or you know, TX to the web right?

F

So since we have only thought about, let's say we're only working on right now, at least initially for the outbound, then then the definition May basically be revised or or certain things basically. So so what I'm really getting to is that at least at the design level?

F

We should, we should think end to end uh and and really taking into account of these scenarios before we start to really come out with these data structures, or at least start referring to certain uh fields which may end up need to be revised, because the fact that you know we have really not take into account of other things as well. I, don't know if all.

L

Of this thing makes sense: yes, and if I'm, are you saying that whether TX should count, uh uh for example, for inbound traffic in which direction TX should account and which are which direction RX should account? Is that what you are referring to.

F

Us now what I was saying is that, basically, you know just calling TX and RX on a bucket level without basically specifying what it is counting, whether it's inbound.

C

I thought that is clarified in the the requirement right. If we go back to the requirement, uh we clearly state that out, TX is outbound for an eni anti-metering bucket.

C

I mean like it doesn't matter which direction the the initial flow happened or or not, because if that is not the case, I think we have problem with.

G

I

C

I

No I think a prince I think. The problem with this statement is that why this this only cover the outbound flows, to see with the metering, but I mean it should I mean create it as a part of the VM initiative. uh Travel I think there's too many qualifiers in the uh in this statement. Can we get rid of those qualifier and say just planning say the statement that okay, so you know, TX means outbound traffic means VM to.

C

The okay to the outside and you.

I

Know just make a blanket State, yeah yeah.

C

I

Many too many qualifiers in the statement. Okay,.

C

Yeah I think that that makes sense I'll just rephrase that uh statement and and update the pr yeah.

J

Can I call it once again, guys cannot call it Eric's underscore inbound and TX underscore outbound, and it makes everybody happy. It's spelled in the name of the stat or something you have both ex and the inbound and RX so bound spells.

I

No, no, no, no, no txrx, and you know just using panel outbound. Why why we mentioned txrx at all.

J

Can we add the total and count the total and implement the total and finalize everything in total and then debate inbound outbound for as long as needed, but have a total done? That is.

I

J

I

No, no! No, why why why we add a third one, I mean I, don't get it I mean you know you should have. You know two each kind of One Direction and that's it right. So if you want Total as a map.

K

Yeah yeah I think that can happen at a higher Lane, but yeah I think uh we cannot. uh We need to just.

M

K

More clarification about what the xrx and probably just uh keep it unified for outbound and inbound, irrespective of who is the initiator right. That seems to be the feedback.

L

I think more so it has to be yeah um for the inbound. Also right. We need to make sure how it is being built today right, but for the serious pipeline. What they mean is uh if the accounting system, whatever is existing, is um I hope it is billing the correct we can go back and look at. uh We probably need to reconfirm that right for.

F

L

I

Sanjay, when you see existing, you mean yours.

L

Whatever is uh I, don't know, I think today, if it is being used as a bucket, ID um uh and Azure is already billing in some way right just to make sure that the txrx, the notion that we are talking about here is exactly how it is built. Otherwise it will be built the other way in your Azure Building Systems right now,.

I

So to clarify which direction is inbound? Which direction is outbound.

L

Correct, for example, if it is a peering thing, if there is a peering Rule and then they have inbound outbound, then the direction in which the traffic is going I think we need to make sure it's not eni. Centric uh I mean- or it is eni Centric like we are talking about here,.

I

Okay, so, who owns that is okay, yeah.

K

Yeah I think, once that clarification comes how we do it in the pipeline, we can yeah, it seems to be a trivial thing. Together, we.

L

M

I I'd like to just understand something better I thought I heard earlier. um You say that on the first packet of the flow you establish the bucket ID and then for all packets of the flow after that that bucket ID is used, but there's two counters in there one for each Direction.

M

But if a packet came in the opposite direction of the initial packet like there's also, you know bucket IDs based on, for example, the inbound route table and stuff. Would those not get you to the same bucket or they would get you to a totally different bucket? Like that, you, you have to use the bucket that was established on the very first packet.

L

So, when you say back at coming back, are you talking about the same connection or are you talking.

M

About yeah the same connection, so so, let's say the first pack of the connections outbound, and we follow this Pipeline on this slide and we get to a bucket ID. If a packet comes in the reverse, Direction I thought it was said that you would just use the same bucket ID, but with like the opposite counter got it. um But, however, if a packet came in the inbound direction, there's also a way to get to a bucket ID right, like the inbound route.

M

Entry would have a bucket ID and maybe there's a policy table, and whatever would that.

C

Not but but in that case the it already matches an existing connection right, so it won't.

M

Go I'm asking I'm asking I'm asking.

C

M

Sort of a high level question like: is there a way based on get on receiving a packet to get to a bucket ID like uh like sort of a a stateless way to get to a bucket ID or like? Do you have to use the bucket ID of the fir that you established from the very first packet.

L

So the but see I one thing, one thing: let's say in the ACL case: right, let's say we allow the this is not I mean it's a stateful. uh This thing right so let's say there is a allow statement in the outbound Direction right now, I'm leaving meeting aside right, if you just take ACL right. So if you allow the connection out, then the incoming the reverse traffic is also allowed. Irrespective.

M

Of what the rules.

L

Are in in the same way it's the same thing applies to the metering bucket too. So once we derive a metering bucket for a connection on the outbound Direction, the same metering is applied, uh it is accounted against the same connection. Now, if you have a new connection that is coming that might derive, the pipeline itself is very different, so the bucket ID that is derived the mechanism is very different than what is shown here so that would potentially I would expect it to derive a different bucket ID. Okay,.

M

Okay, that's my question so, depending on like which direction the first packet is, um you might get to a different bucket ID.

F

G

F

Yeah, great okay, so in in that case you know if, let's say I'm, going to the same port 80 on the same server but I'm initiating a new connection from my VM, and it is let's say you know it's uh Port 3000 to Port 80 now. So it's a new connection but to the same destination, would I be uh going to the same bucket ID or is it will be a different bucket ID.

L

K

That depends on how that yeah, how the bucket ID is derived in this pipeline. It could be that yeah if it's.

F

Perhaps let's ask this question, you know what does it depend upon uh and then in? In which case it would be the same, in which case it will be not the same.

L

Actually, here I think it would be the same. The reason is uh because the metering policy is based on the destination IP right, which.

G

L

A mapped to this, it should map to the same thing or it could come from a I mean if it is not uh uh it's not going to the internet or it's not the policy, but it is uh mapping entry mapping. Entry is also destination IP based so I think it would. uh The pipeline would derive the same bucket, uh not uh as, as you know, as a factor of where the bucket IDs are derived from right, yeah right so.

F

L

Be the same yeah.

F

G

F

Know your destination is based on uh on on the route entry or is actually the ultimate IP address.

L

uh So that depends on the I think that is the uh uh whatever the prince had mentioned right in terms of uh so there are three different ways and there is priority. So there is, uh there is a bucket ID that can be put on the route entry. There is a bucket. There is on the route entry, you could uh enable policy, and you could also have the route entry would point to a good one that any particular route entry could point to a mapping table and that mapping table destination IP.

L

The slash 32 could have a metering bucket idea Associated to it. So it depends on what case we are talking about. Are we talking about uh being inside the v-net intravenet case? Are we talking about intravenet uh different azr? Are we talking about uh peanut.

K

Peering, okay, I think yeah. So this pipeline is just kind of uh catering to all the requirements and based on the uh what is the exact scenario you have to configure the routing or mapping entry accordingly to derive the right bucket that you want right.

F

Okay got it no. Thank you.

A

Is this? Is this a decent stopping point guys, or do you want to keep going?

A

I

But what would be the next step? I think we we haven't followed through all that right, so we need to continue.

K

Yeah so I think we just wanted to introduce this uh other thing. uh I mean, like just uh I mean this is just kind of explaining how the bucket IDs are mapped. So probably, we've entered some of these things. uh Yeah I can probably just quickly go over this, so uh this is essentially showing where like. If you have that n uh bucket IDs, uh you can have one bucket ID in this particular example, which is referring to appeared being at one traffic bucket id2 refers to Pure, we met two traffic and so on right.

K

So now each of the bucket IDs are mapped to uh Enis, where they need to be accounted so in this example, the eni one you have bucket ID, 1 and 3, which is mapped to that. So when you map this bucket ID internally, what you're telling is that I need to account this uh these uh types of traffic on this particular eni, so these dotted boxes are essentially the hardware resource, so to speak, uh where the actual uh accounting will happen.

K

So uh now, okay, let's address the fact that why this particular mapping is required right. So if you, the billing, is done at upper eni level and uh but if you look at the bucket IDs, the the bucket IDs themselves can be associated with mappings, which are at a v-net level. uh Now, because the bucket IDs are associated at a mapping level, not all the mappings which are associated not all the bucket IDs, which are associated with the mapping, are actually required on every Ena right.

K

So, uh for example, you might have eni one v-net, a uh having a peering relationship with the v-net B and v-neck B might have a bunch of mappings and a whole bunch of bucket IDs, but only a small subset of those bucket IDs are actually relevant on eni.

I

One hey so, which is sorry um I, think people may start to get the job, but I think looks like this is not a very straightforward concept that you know people can comprehend in about. You know two or three minutes: I suggest you. Let's talk about that next.

G

Next meeting yeah yeah, it.

I

Seems here is like a really uh something that is not very straightforward to to come. Complex, yeah.

F

Okay, yeah sure I think that's so that's I think one other thing is that let's make sure that we basically talk about in the next one, but at the same time, if, if you are able to you, know, put some write-up a little bit on this one like on the pipeline and and these things that'll be great I think these slides are excellent, but if there is happens to be a little bit of a detailed description of that, uh that'll probably help as well, um but it in another thing is: you know thanks a lot really appreciate all the hard work on you know explaining this thing, but do you guys plan to share these slides?

F

K

Sure yeah we can share this theme.

I

K

I was planning to just put all these uh diagrams into the hid as well, but yeah we can share the slides. Definitely.

A

Okay, guys I'm gonna go ahead and stop the recording. It's.