DASH Community Weekly, 11 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SONiC DASH Workgroup Community Meeting Jan 11 2023

Description

Dockerfile.saithrift-client has a static tag in the FROM instruction PR289 - still needs volunteer

PR300: Chris Sommers: request to merge https://github.com/Azure/DASH/pull/300
Status = merged
PR297: https://github.com/Azure/DASH/pull/297
Status = merged

PR313: https://github.com/Azure/DASH/pull/313 [Sonic HLD] Metering requirements #313
Feedback to clearly layout requirement for metering
Section 1.5 Metering requirements

A bucket contains 2 counters, 1 inbound and 1 outbound
4K buckets per ENI, place this into scale document

Q: Is there an extra RX/TX shaping requirement?
A: No, this is counting only, not rate limiting.

PR296: https://github.com/Azure/DASH/pull/296
New Construct from AMD
Status = discuss further next week

DASH moving to Linux Foundation under SONiC

A

And 51 meetings together. So that's a lot um so I'm presenting you know kind of what we talked about last week and what I sent in the meeting notes over the weekend- and um you know Chris I, know you're on the phone, but you had mentioned you wanted to talk about PR 300 today, and is it something you want to talk about verbally, while I display.

B

uh Sure I presented it a few weeks ago, just introduce it to people. It's.

C

B

Two tutorials on how to write um test cases for the site, Challenger framework and they're just kind of exemplary tests for configuring. The device under test I don't show any traffic tests at this time. I'll, hopefully add more examples in the future, but it's really just a set of tutorials and diagrams and trivial test cases. So I'm hoping we can just merge that in so there's a resource for people to learn from it's, not an earth-shattering.

A

Okay sounds good. I can probably do that today. um You said you got a couple of looks good to me's and nothing else really so I can do that today. Okay, anyone else have a comment on this PR or had you had a chance to take a look.

D

Do you think I provided some comments to Chris separately, so yeah yeah.

A

B

So, thank you. Hannah yeah.

A

B

A

Can take care of that today, Chris after the call? Okay, all right.

E

Hey Christina, have we gone over um with the community that we've moved or in the process of moving um Dash under ocp, PSI and and the uh Sonic bits under the.

A

Linux Foundation Linux.

E

Foundation does everybody understand.

A

Yeah we've talked about it briefly, maybe four weeks ago or so, and we can talk about it more in today. Gerald, if you want to a brief talk.

E

Well, I just want them to be aware that the move is being made so um I mean um the merge of I think we talked about it in previous meetings emerge of the dash under the ocp side. That I think everybody understands and knows how is happening.

F

E

Then, on top of that, um we are moving the entire GitHub under Sonic proper properly and that work is just started and there's just Engineers on our side um doing. You know dotting the eyes crossing the t's on exactly how that's done, but we expect that to be done over the next week or two at the most, and so it won't change anything that we're doing uh at all. It's just that it's going to be sitting permanently in its proper place.

A

E

That the community's gotten Mormon here pardon go.

A

Ahead, yeah yeah now that the community's gotten more mature and moved forward, um yeah did.

E

Anyone have any questions we do want this to be more than you know, a Microsoft Project. We will be introducing you know in the future. Encryption and everything for other other use cases and um things like um equivalent to Outpost use cases, edge cases and and other educations that are General. So you know really want that to be done in a forum that um is not Microsoft.

E

Only and um you know, ownership it'll actually have the same governance as Sonic, Etc and um and we'll appreciate all the input that we get from being part of Linux Foundation, and it will also give us a a place to meet. um You know when we have those meetings. uh We can kind of coincide, meetings with Sonic meetings more properly Etc and uh same with ocp, which is the two places we normally meet.

A

Any q, a from the crowd here.

A

A

Sounds like not okay, so it sounds like no questions Gerald. So um what we're gonna do is just work on the back end of getting this work done here on our side, guys.

D

So I have one quick question um since I represent in the Sonic board as well, so um one that you know today what happens is that we have a TSC also um handles all those things which is andersonic uh chaired by liwa. So are we saying that you know since it's the governance is moving under Linux Foundation? The Tacs will also uh really figure out in terms of who are going to be the maintainers and reviewers and so forth as well.

E

um We we can talk about that I mean we're going to evolve exactly the way we are now and Lenny. uh Levi's team is already involved. With this.

E

We have Prince, we have Gohan and- and we were already involved with this, so we we will talk about those kind of things ongoing, but we wouldn't anyways and- and uh it's just more maternal, so um yeah there might be some some changes in the future to how we do approvals or how we review things or whatever, but that will be that will be done with this same crew here right I mean we'll talk about it and figure out what the best thing to do, and the Sonic people have already been involved, and, quite frankly, even most of the work that keysight's done um has been to you know, maintain um you know the the sonic environment and Society environment so that we didn't have to go through um a bunch of changes just because we move under them later so I.

E

Don't think that uh so changes might come, but it will come due to the people on this call. You know discussing things and figuring out what the best thing is for Dash, um and mostly you know, but the best thing is for Quality what the best thing is for moving faster.

E

um What the best thing is for getting more input around the industry, so um yeah changes might happen, but it'll happen with the people here.

D

Yeah absolutely I completely agree that you know the the people who are here are going to be driving since they have the best suited for that. um However, since we say that it's it'll be under the same governance, you know um governance, rules and so forth. We will probably start to adopt um those governance rules and start- probably uh you know, writing down who the maintainers and reviewers are so for. Everybody knows that. Okay, who to reach out to in terms of uh you, know what needs to get approved and so forth.

E

Yeah and that's and that's a good thing, because that's why we're moving it's a more mature environment that exactly we took part of it and many of you took part of for years to build that environment so we'll take advantage of you know if it's working like that which I think it is then of course we're going to take advantage of that and mature. You know Dash even more so yeah I think changes will happen, but it'll happen. There's enough people on this call part of Sonic and PSY. Already that you know we'll just discuss.

E

You know what what what, how it's done in those communities and we'll probably slowly start adopting. You know all the types of governance and reviewers and all that kind of stuff, and that's a good thing, just sounds good thanks, yeah.

A

A

Prince did we want to cover 10 minutes of q, a or anything around the private link service tunnel or guys on the call? Did you come to the call with any questions about those features that we talked about last week.

G

Requirements today so just.

A

So what I'm sorry broke up a little bit the? What requirements? Hello.

A

um It's kind of in and out for me is it for.

C

Anyone else thinking very bad yeah, it's breaking, but it's up.

E

Breaking Bad sound, like a team you're talking through a tin can of some sort.

G

G

E

Still not use one.

H

E

One back no I think maybe you have the log out log back in front so.

G

I think it's something yeah!

G

A

C

Can we look also at pr297, while he logs in back it's just a test block yeah?

C

If we can get that accepted, uh there are some feedback. We addressed it with the template for the test plan. We just kept a link to the Sonic one. Instead of doing a copy paste here at all.

A

I'll just go home.

A

So I'm just reading through this Chris Summers reviewed yesterday, why not use blah it's more accurate.

A

Oh there we go.

A

Okay, so you're asking for a merge here.

C

A

And we used the we reused, the existing diagram and it says fixed image: yeah, okay, yeah I can do that unless anyone has any uh objections.

A

Christina to merge today, I'll make a note.

A

Okay, I can do that.

I

Thank you very much. Thank.

A

You thanks for contributing.

A

See if we have prints.

G

A

Think we do yeah.

G

A

You hear me yeah yeah,.

G

A

A

Now, what were you saying you.

G

Wanted to talk about friends, so I want to talk about the second pair, the metering requirements.

A

2013., okay,.

G

Shall I go ahead or.

A

Yeah go ahead. Do.

G

You want me to.

A

Display or you yeah.

G

I can try to share okay.

G

All right, um can you see my screen.

A

You can okay.

J

A

J

Have a one question: I mean: should we close the service Timely or before we go into a? Is this one or.

A

I think we were waiting to see if there were additional q a Gohan. But if there aren't any, um you mean close the pr or close the discussion. I'm sorry.

J

Okay, okay, so if there's no feedback? Okay, that's fine! All right.

G

So go ahead on the service channel that PR is updated with the feedback from from last week's meeting. So if you need we can discuss or if we have offline comments, I can address in the pr hi.

J

Shy all right, so so we'll wait for some. uh You know some feedback to comments. If there's no feedback, we will get to we'll get that merged all right.

G

Okay yeah, so this is again a follow-up from a couple of weeks, back MD presented the metering proposal and one of the feedback we got from the community was to clearly lay out our requirements for metering, so um this PR is just capturing the requirements. The Proposal is still part of another PR, um but at a high level these are the requirements, so we can go one by one.

G

um So this, basically the metering and building are, uh is used interchangeably. So it's it's mainly for building the customers from uh for each of the different type of packet for uh packets and forwarding scenarios. So um the first one is like billing shall be per eni level. So uh so this is one key point to keep in mind like um it's a it's an eni level uh packet counters that we use for building the customers and uh and and the the application shall be able to query meeting back metering packet bytes. So it's it's.

G

Basically the bytes uh that we are interested in. um This is just just a generic statement that the buckets must be of you in 64 size and it starts from value 0 um and shall be counting. The number of bytes, so uh so, basically, we are looking at three different type of metering.

G

One is policy based metering, another is routable and the third one is mapping table based, so this will kind of cover all our requirements for building the customer, so uh why the policy based metering is uh for specific destination like if you are interested in counting the uh bytes or packets, going to a specific destination and that which need to be built separately. So that is what we uh we are talking about, like the policy based metering.

G

The next is, like you know, route table based metering um where the meeting bucket is associated to a route table and uh it can be used for scenarios like we need Perry, and then there is a mapping table where again like. If a specific destination need to be built, then we can associate the metering to the uh to the mapping table. So these are at a high level, three different type of metering requirements that we have so again like one is policy based for specific destination route table based, uh and then the mapping table based.

G

So um this takes into consideration some of the the packet forwarding like in the outbound direction. We we have the axles routing and and that's how the packet flows. So when it looks.

G

Yeah um yeah, if it gets disconnected I'll, join from my phone, um but uh when, when the packet flows through the different sections of pipeline like uh it can hit all these different um different scenarios and in such cases we have defined the order of priority where a policy takes precedence over routing and routing, takes presence over mapping. So this will be the order of preference if, if metering is associated uh to all these three sections of pipeline yeah, so that's uh that's this one, and uh next is like you uh in.

D

In case, like count, one bucket versus all three of them is that.

G

Yeah, so if, if a hit happens on uh all three of them, then this will be the order like uh we want to meter it based on the policy rather than but.

D

We don't want to duplicate order, meaning that you will. You will count it in on these. The counting will will happen in all three buckets. It's just that you will elude in this order. Is that is that what that means.

G

No, the counting should be happening only on.

D

G

Okay on single one, because we don't want to have duplicate, counts and uh over bill, the customer. Okay, you got it yeah thanks, so then um like if certain scenarios require that a mapping should take precedence over routing, um which is some of the scenarios that we have. uh In that case, we have an override flag in the mapping table uh with the with the metering bucket.

G

So what this means is like if, uh in the same order like, if, if you want a mapping to to be counted right, then we can have uh a generic override flag in the mapping table. That says that okay um count on the mapping table rather than the route, so when policy is enabled for so again like the policy will always take personal and uh that's what uh it's it's clearly stated here like. If, when policy is enabled for a route, it takes higher percentage, then routing and mapping meeting bucket.

G

But if there are cases that we need uh mapping to take preference, then it can take. It can use this override flag.

G

So here are the the different scenarios that implementation shall aggregate the counters on an eni per metering bucket. So this will be the combination for getting the billing data so again, like all traffic from an eni to peer v-net, private link, so I think all these different scenarios are described in this document. In the later part, what is appeared v-net? What is a private link destination? So this is again like some sort of mapping that we use for private link and that's where like.

G

If we have to uh count the private link destination, then we can use the metering Associated to a mapping and use the override flag.

G

um Then all traffic from an eni to internet destination that can use policy based metering.

G

So, okay, uh the other one, is traffic from eni to cross AZ that can be within same unit or cross region destination towards the peer unit. So all these different packet flows, we want to count or build separately right. So that's why we have to associate metering bucket in in different sections of the pipeline.

E

Foreign, that was an important statement you just made their prints just make sure that they understand us. We charge differently to based on where packets are going not always but like, for example, if you're going to a different region, you have to go through the land, maybe charge differently in some service cases or if you're going to the internet.

E

um You know we have to pay for the internet um peering, so you may be charged in a different way. So when you look at these things, these are things that we charge for and that's why we need the metering. If you couldn't do the metering, then we wouldn't be able to charge for it, and then that would break our services. So it's uh I think this is really you've done a good job um on this prints and whoever helped you, because it's pretty clear.

H

The only thing I would add to that is that uh we also measure by time. Is there a time component here to metering as well like by the hour.

G

No I think um not really yeah. This is once Associated to it out or uh or or mapping like once the flow is created, it will be, uh it will be counted for that flow.

J

Yeah that that is handled by the software, so you.

C

E

C

J

Have the camera so yeah.

E

J

E

When they collected Steve, that's the time, it's not it's not collected by the dpu, it's just when they collect it. They can aggregate the times that software will do it.

H

If you don't mind uh Prince, would you could you put a link here to this section in the chat? Please.

G

A

Should I I can I'll do it while Prince is talking, okay, yeah.

E

Yeah, thank you continue on Prince good good job. That was very clear. I, like it.

G

Yeah, thank you as well, so the next is like customer is built based on the number of bytes sent and received separately, so, which means this metering bucket should be able to count both RX and TX packet, right, inbound and outbound packet. So um so, when we associate a metering bucket to a to any of these tables, it should have both RX and TX counters, that uh um that are counting uh the bytes separately right. So that's this requirement um now the other.

J

Thing is this part: I I, don't understand, I saw the arcs and uh you know inbound the outbound. They will be associating with a different uh packet. No.

G

The same bucket, the same bucket, should have both uh outbound and inbound.

J

But they will be associated with different rules right. You know the policy rules, routing table rules, yeah.

G

So, for example, if them, if there is some once the uh once, this is uh uh evaluated and and if the mailing bucket is associated to this mapping, entry right for that particular flow, you have a bucket and that bucket should count both outbound packet. For that, and also the inbound.

G

For that particular flow, which is uh which is having a metering bucket.

J

How do they decide the direction.

G

uh It's the connection right so.

J

G

Yeah the connection.

D

Do we do we also count packets or just you know, even bytes, the.

G

Bytes I mean bytes is sufficient for our uh like for our billing requirements. Okay, um but.

D

For debugging purposes, let.

G

Me just quickly check if uh I.

F

I'm a little confused about what you're saying about inbound and outbound. Are there independent counters for inbound and outbound, or are they aggregated into the same counter?.

G

So the same metering bucket should have um an outbound and input like RX bytes.

F

And TX bytes yeah, so a bucket contains two counters one's in button ones, outbound and then based on the direction you choose, which one you'll count.

G

Yes, that's right.

F

Okay, okay, maybe we.

E

Just make that clear, add a line or something in the document just to make that clear. Since people keep asking questions on it.

C

F

Prince one bucket.

C

E

Yeah one bucket with a receiving transmit I think that'll make it clear to people.

G

Okay, yeah I will clarify that.

F

Yeah and also Prince um I, don't know if you've mentioned anything about scale like per eni. Like is there, do you have like sort of an estimate or even an order of magnitude, of how many buckets there may be for for an e and I.

G

um So this would be limited to the dpu uh scaling limits, but uh from a requirement perspective at a very high uh uh like the highest scale, I can probably get uh get the data and give a set of friends or let me see.

G

Do you know if we have a scaling limit on on this or um or we can check offline and update this.

K

I'm sorry: what was the question.

G

So do you know if we have a upper scaling limit on the on the number of buckets.

K

uh Number of pockets uh number.

G

Of metering, buckets yeah.

F

I mean these are like different classes of billing right, so is it like dozens or is it like Millions.

G

It will be thousands as far as I know, but I I would say like chicken, okay,.

K

Unite traffic will be in the base uh handled by two dpus. Were you copying them separately or so? What's your.

G

Yeah that will be counted separately. This is uh per DPO countries, yeah.

F

Like per e and I, like how many buckets would you expect for like typical scale, um I I I mean it's a very important um requirement, uh because it could could very well affect like how these are implemented.

L

Hence, do you want us to give a number from our serious implementation.

F

D

I think we should document this thing in our scale document, so so as to.

G

Say that oh something right, yeah, yeah, sure, I think here we can have right like uh yeah.

D

G

Okay, yeah yeah I'll.

B

G

uh Yeah I'll confirm that.

K

The other question is like you mentioned, that this is rx and the TX, so this is the more like a software required. This is not you asking for the hardware to say using the same Hardware instant for it. You just says the software Behavior you want like accounting for both arcs and TX right.

G

No, but that should be counted by the hardware and and provided to them.

K

It will be counted by Hardware, but uh could have a two bucket one for RX, one for TX right or you would say: hey I want to have a two con. Only one bucket for total, but I want to count RX and TX yeah.

G

So from the application side we'll be creating one bucket uh and and require that bucket to be counted for both RX and TX, but the internal implementation, if they have to use two different buckets for that, then it's a it's up to the implementation.

K

This is the bucket only for the the counting purpose or you plan to, because normally the meter also do the shaping. So do you have a separator RX TX shaping requirement? No.

G

This is only for the counting purposes for.

K

G

Purpose: yeah gotcha.

K

D

You yeah I hope people are not uh guys. I, hope people are not confusing between metering versus the rate limiting, because why are we bringing the shaping Concept in in the same context of metering? This metering is billing and it's not rate limiting.

K

Yeah because my normally I assume.

J

The media yeah, why? Why could you call her yes requirements.

D

J

I think we should we should we should.

D

Define, what the definition of metering is in this context here, because I think that there's probably historical reason why we call this near the meeting, because in networking terms and whenever you say metering, people think it's very limiting and then and then that's why I received question of shaping versus eight, limiting and so forthcoming. This is not rate limiting this is billing, so I think. Let's make sure that.

G

We I can get that clarified yeah in this yeah, but I think Gohan. We want to use symmetry because it's been.

J

Used I mean clarifies, is good yeah.

A

Okay, although it does say a metering, is essential for billing, the customers and below the high level requirements. Do we need to make it more clear than that yeah.

G

J

It clear it's not like a token bucket. You know, you're doing things like okay uh polishing those kind of things right. So let's just make that clear.

K

Okay, later on, he mentioned the bucket. That's why I was thinking in the token bucket.

D

I absolutely knew that you know the people will be confused here because of the fact that I, when I explain to some people within my group, Revolution is saying: oh, it's like you know, policing or rate limiting. You know, I said no. This is not. This is billing and then the entire conversation changes as a result. So, okay.

G

G

Okay, so that's.

I

One clarification question right like so: uh basically, the metering bucket derivation happens in the outbound policy evaluation stage uh and uh that bucket will be used for both the outbound and inbound side of the connection. Right like it will not happen in the inbox policy evaluation right.

G

So inbound policy evaluation has other another meter meeting like if.

B

J

G

The first packet coming so.

I

uh Okay, so, like you're saying, like metering bucket, can get no like for a given connection. You we have to evaluate both the inbound and outbound right, like so based.

G

On the outbound connection you have uh you, have you can create an inbound flow as well right.

I

So who initiates the connection first.

G

So this is on the outfound direction.

L

If the connection was inbound from the host side going out, so that is with going outbound right in the sense.

K

L

Flow is outbound, then, that initiator flow uh uh determines the bucket for both directions of the flow right. If the flow was incoming, that is, it is coming from external towards the host. Then that is an inbound flow. Then the power, the bucket that is evaluated on the in the inbound direction again, will evaluate both the forward and the reverse flows. For that connection, so, depending on the which direction the initiator flow is that where the which direction the flow got initiated, that determines uh which policies are evaluated for the metering?

L

Would that be the right thing to say.

G

Yeah that that's the right thing so I um Sanjay, are you saying like I? We can capture that explicitly in this or.

L

Yeah problem: if the okay there's some confusion between the inbound and the outbound, uh maybe we could put something like depends on where the flow was initiated, which direction the flow was initiated.

I

Yeah so Prince that that is the expected uh behavior. Is it.

G

I

It looks like it depends on who started the connection uh is going to decide uh like because we have to evaluate both inbound and outbound, uh irrespective of who initiated right, and it can actually both of them can in fact give the wiggling bucket, but it seems like uh we have to consider uh the source and then pick one of them.

I

G

F

So I, okay, Prince I, I I, have one more question. It's like um like what, if there's a possibility that you could increment the counter like in the processing pipeline, but then Downstream of that for some reason, congestion or some other reason the packet is dropped. Like is. Is there like a formal definition of like um like that?

F

If the packet is dropped, then the counter should be like that like uh like that, it's okay to count the packet and then drop it later or uh or is it like critical that if the packet is dropped, then the then the meter is not counted and I don't mean dropped because of like um you know, policy or anything like that. I mean, like you know, perhaps there's you know. Congestion and some Downstream resource uh is full.

G

Yeah, that's a good point, but um again I think we should not build for those packets that are dropped, but.

J

In general right, so that should be rare case right. So you know uh that's kind of uh so you know the you we shouldn't. Have uh you know job counter to go to understand the weather. You know the the appliances Trophy and customer packets, or not right so not due to the policy, but due to the you know, processing capability capacity of the pipeline right. So you know, if that happens, I guess you know you you you you would. You would generate some kind of learn because that's undesirable right. So, okay.

E

Yeah, if you look at this um once you go through the pulse, you know once you go through the rules, if you, if you pass the rules, there's no reason to drop a packet afterwards, that'd be like saying: I didn't find it forwarding.

F

No but yeah Gerald, like happened Gerald, like suppose, like uh I, don't know you're receiving traffic on both ports at line rate, but all of that traffic is being up routed out one for it. Like there's congestion cases where.

E

You don't have like infinite buffering. I, don't think you! You have to treat that case, because that can happen in any switch and you can lose packets in a network yeah. These counters are at the eight. Essentially they they logically sit at the Ingress of the network, regardless of whether it's in the appliance or not logically they're, just like they came off the server right after. If it was on a server, you wouldn't have lost anything um and, and and and the point is you can lose packets, and we don't do anything today about that.

E

F

E

F

Mean that makes sense, you're saying it's just like any other network drop like like it would have already been counted. Yeah, okay,.

E

And because this is logically at the edge of the network, that's the way it would have worked and we need to keep this done. Behavior.

F

G

So, let's move on to the next one. So um as we know that metering resource is limited, so there will be like some optimization in.

H

G

Application Side by um allocating meter ID when, um when required, anti-allocating when not in use right, so uh so the application shall associate this meeting buckets to eni assignment required and the allocate were not in use this uh kind of a generic optimization um part- and um here is one other case where application shall bind all Associated metering buckets to an eni um when, like when the metering bucket or um eni, is created or later when um enlar needs to meet around certain buckets.

G

But during the eni deletion, all Associated metering bucket binding should be auto remote. um So this what this means is like there is no explicit deletion or unbinding of metering bucket.

B

To an eni during eni.

H

G

Because that will be a lot of operations during the division, so.

D

So The Binding is auto removed, but also the buckets will also be deleted right. So.

G

The bucket may be used by another eni right.

D

If it is shared, but if it.

G

Is yeah? You have a.

D

Reference count or something.

G

J

That's handled by the application right, so the you know, it's all caging code will handle those kind of things.

G

J

D

Right right, so if it is shared, of course you know you will keep.

C

It around otherwise you'll.

D

You're going to remove it correct, yes,.

G

Yes, um the next is like in the inbound side, uh we have the route Rule table. That also has a metering bucket Association for inbound traffic, um but on the.

B

um On the burn, Direction.

G

The different scenarios- I I, didn't capture now, like uh it's dpd, so this.

J

Part I am trying to not quite understand because you cover the game button and all about you see the you know the packet will have a inbound and outbound. Then what is this uh uh building case so.

G

J

Is not covered.

G

So go ahead and like um like Sanjay was explaining before this metering bucket in the outbound Direction can also create the flow and count for the inbound for that Associated flow right, so yeah. This is for the packets that are coming back to the host, where there is no, um where the the initiation didn't happen from the host. The initiation is is happening based on the first inbound packet yeah.

G

So in that case we have a routing um routing Rule table right. Let me show you that this is the outbound route table and then we have a route Rule table for the inbound Direction, and here also we have a metering bucket, and this can count for the first in incoming packet and this metering bucket Associated to the eni, can be used for um the outbound packet that are that are created.

J

G

Yeah yeah yeah so that that's the inbound hello, I I meant in that section. But there are different scenarios like right from peered v-net inbound from internet to inbound, so that that's the scenarios that I didn't.

J

Isn't actually the same, you know if there's a yeah.

G

J

Then, what what is this? uh uh You know two EBD here.

G

So I can explain like a different inbound scenarios as well. Right like coming packet coming from uh internet I mean, but.

J

G

Mean it should be.

J

Same but yeah, but that this uh I think the the question is: uh is that similar? Like you know what you're describing the 117 you know, all these different scenarios, or or just you know, okay,.

G

So yeah you know it is.

J

Basically, this is a high level requirement right. So you know once you get this requirement, I guess uh you know. Will we look at what has been you know presented before you know the by by MD folks on the detailed implementation, and then there will be I guess there will be P4 pipeline that describe those things right. So then I guess this uh TBD things.

J

That's not really change any of those things any of those discussion- or this is many about the scenario right just for complete lists that you know there is a very same-bound scenarios. Then, yes, people need to it's nice that people is a is going to be aware of it, but it doesn't really change the design or implementation right. Yes,.

G

Yes, that's a that's right.

J

G

Doesn't change any requirement? Yeah?

G

Okay? So that's the requirements from a high level um and.

J

Then one question is 116: is you know, policy enabled for a route to take higher precedence? There's a routing and the mapping meter bucket, didn't quite understand this sentence.

J

So what do you mean by I thought it's many about the presence between the routing and the mapping right. So what what does it eat takes higher precedences in routing and the mapping return? You mean, you know when there's a policy enabled for around the the routing priority will be lower than the mapping party.

G

G

A

Yeah my understanding was, you looked, took a look at the mapping and if the, if there was a mapping that would take quote unquote priority is that right, Prince.

G

So uh no the the policy evaluation or the priorities like in the order of policy and then routing and then mapping.

G

So if there is a metering bucket Associated to a route table and there's also and and that route is pointing to a let's say, a v-net mapping table right and if the mapping table also has a meeting, then uh that flow should be counting the packets on the route table meter metering bucket, unless you have an override flag in the mapping table that that requires the meeting to be done on the uh on the on the mapping table based yeah.

G

So by default, is routing, takes presence over mapping.

G

Yeah so like I mentioned in the example like, if you have a uh say, we need to v-net case right.

G

So there is a uh uh there is some uh in the we need to win it like you, probably don't want to uh meter on the on the routing table, but certain mapping cables are certain packets going to a different PA like cross region or uh or different availability Zone. That has to be uh metered. But if that same eni is sending packet to a peer unit, then that meeting is part of the route table. And then you will count on that.

G

Irrespective of what is there in the meeting mapping table because you don't want to count uh or build customers based on the mapping table, but instead it needs to be muted, based on the on the route table, because it's going to appearing in it.

I

Yeah Prince one basic question so like in the outbound or in the pipeline we yeah I, understand we have this uh routing lookup like route table and then a mapping table right. So what is the difference between, like this policy based metering versus route table like where.

G

G

Okay, uh that's a good question so when the pipeline, but first hits the route tables right and if, uh if say the route table, has a metering bucket, that's one case we just explained about winning pairing. But if this route table is uh is a uh it's, a routing action is direct, which means it's going to an internet destination right.

G

um In that case, um you may want to build and that internet destination is like. Sometimes it's a um say. It's a default route like zero, slash, zero, going going to some destination right. In that case, you can associate a policy and the policy can have different destination IPS destination um that that need that needs to be built separately.

G

So the policy will is associated to the routing table itself, like if the routing table says that and uh that it needs to be metered based on certain policies. Then you can associate a policy to that. So in the pipeline it is part of the routing action.

I

Priority lookup with match on destination, prefix and each rule will say the meter bucket to use this yeah.

G

D

I see so in that case you know: where do you short circuit it because in because you said that you're going to count only once and then, if there happens to be a policy um uh that directs your to to internet route, then then, if you have counted it, does it mean that you know when you hit the routing table? You say that okay, there is some sort of information that you have already counted in. You no longer count it anymore right.

G

And none of these things we want to count you like, as a duplicate entry, so must be counted only once.

D

Okay, so in that case you know, I then I want to go back to the example of PNET to v-net, not the v-net pairing yeah, but on the v-net okay, so on the v9 generally, it should be counted on a mapping table right. Yes, okay! So then, in that case, what happens is that if you have a in the pipeline you're going to first hit the routing table and then you're going to hit the mapping table right?

D

And so so, since you have not hit the mapping table, would you not have counted it already on the routing table? And then then you know: would that not break your model so.

G

Unless you associate uh metering on that route entry, there's no meeting on that route table right.

D

Okay, so what you're saying is that if there is a route entry that essentially is uh within the same v-net like a VPC uh from one minute to win it, then you will not have any bucket associated with it and hence it will not basically um fall into it. Anyways it'll only go directly to the mapping. Okay, okay, thanks thanks.

D

By the way, since we are on this topic, you earlier, you mentioned um uh Prince- that you have a proposal PR. What's.

G

The number uh yeah- that's that's.

J

Two: two: nine six right so that that's uh that's uh from the AMD folks right.

J

A

296 is right here: okay,.

D

A

Do you need it in the chat window or can you find it.

J

A

Know if we I can find it no.

J

I think we understand the requirement now. Maybe I think we need to go back and look at that 296 to see how to understand it right so at least I think there is a in that period. There is a policy ID and there is a bucket ID. Is it the same or I? Don't think we mentioned the policy ID uh in the French document.

M

J

M

Is the prince requirement? It's mentioned as a policy right, the metering policy, so that is expressed as a metered policy, uh ID and meter policy uh rules.

J

Quite understand: didn't notice that policy, the French document.

J

Where did you mention the.

M

Policy, so the policy-based metering.

J

Or the policy-based meter, okay, okay,.

M

So uh yeah for to support this policy based metering requirement, uh a new uh uh schema is being introduced. That's what is being called as metering policy, and the metering policy can be uh like a prefix based priority, prefix based, and it has rules underneath.

M

So I think that part is covered in 296, but yeah you're, right, I, think given this we have context now about uh requirements. We can go over this again in detail. uh So I don't know if we have time today.

A

M

A

Have eight minutes.

J

You know I, look you know from because you know I saw the policy was all those actual rules that we have added. So therefore, my understanding of that the policy-based metering is that uh um you know all the actors will have a have action point into a bucket and we will do that. Come there, but sounds like you know. There's a policy metering is a new set of matching rules that point into the meter right.

L

J

L

They want to just flash your PR very quickly and then probably it will tie in the two. If you quickly go over the schema itself,.

A

L

We talking 296.

A

B

A

Do you want to you want me to look at it real, quick or do you want to present it.

M

uh Yeah I'm, just like, can you guys see my screen.

M

Yeah so I'll just go to the schema here yeah. So this is the metering policy yeah. So go ahead so, like you were asking. So this is a separate construct from the security policy uh which is it's. uh It looks similar to the security policy in the sense that it has a V4 V6 policy and it has a bunch of rules uh which is priority, prefix based and it derives the bucket ID. But uh the construct uh configuration uh object itself is different from a security policy.

J

um And in this case, IP prefix is always the destination based: okay, yeah, okay, yeah, because.

M

I think uh and.

J

The where, where, where this, uh this metering rules will land in the pipeline yeah so.

M

This the attachment point for this is that the eni level, so at perienna, you can attach a V4 meter policy or a V6 meter policy. No.

J

No I killed that part, but a member we have this. uh You know packet flow pipeline right, so this uh yeah. So the word does that? Okay, so let me ask this question right. So um do you also have a P4?

J

um You know um no.

M

Yeah, no, no, the implementation is not yet there, but we wanted to just establish the requirements and the MTP schema and the P4 proposal can come uh following that: okay,.

J

M

J

D

One big question uh so when, in previously in the requirements when Prince was talking about the policy, was he referring to the security policy or metering policy? No.

M

Meter policy so.

K

The reason why it.

M

Is right right, so the reason why it's the policy is uh separate for security and meter is because uh we believe that the uh the orchestrator also treats the policy separately as in it is two separate entities even from the controller point of view, uh so they push security policies separately and metering policies separately, so it kind of maps one is to one.

D

M

D

I think that, actually, you know, muddies things up in my mind, because I think when, when Prince was basically talking about these uh I guess the Flow versus policy route and mapping in terms of you know the priorities and then the order in which it is evaluating priest was referring back to this. You know the packet flow where he was pointing out to ACL and routing. So you know it's a natural tool for us to think that policy really referring to the this ACL uh and then.

J

Also I had the same confusing right so yeah yeah, so.

D

I think we need to really say that we have different. We are introducing a new construct here and it's a completely new thing which have never been introduced before, which is a meeting policy right, yeah I think.

G

That in the requirement, as well, that it's not active based form s, but it's different policy. Yeah.

J

And it is, it is a policy after alcohol or is this before the echo I I guess yeah.

G

Yeah in the outbound, this will be after the yeah go ahead. Yeah.

M

um Yeah, so the I think the placement of the policy uh table itself uh should not matter with respect to the security policy, because the security policy action is allowed denied, whereas this is a completely uh orthogonal action, so I think from pipeline point of view. It should not matter so we can keep it flexible.

J

I'm not sure either. Maybe maybe you can explain to me why it doesn't matter, for example, if the. If this metering is before the security policy, then you will count the package, that's supposed to be dropped by the security policy versus uh uh vice versa, all right so.

M

G

Think going back.

M

To the record yeah good, no.

G

Go ahead, yeah.

M

G

Point is like it in the pipeline, this uh the outbound policies are after attacking, so so these policies are like tied to the route policy, so should be after the.

J

Okay, yeah, then I guess in the in the 296 right so remember there was a doc. There was a diagram on that. The policy I don't know where I can't find I mean it's uh it's, maybe in the in the in the in the earlier part of the document. Isn't that we have this outbound inbound, the you know this pipeline. We should update that to see where.

C

Oh yeah yeah yeah, yeah, yeah yeah. This.

J

One yeah yeah right so then we should add the uh the act, the meter policy into this diagram right. So where is that that, let's just kind of where's the after rally.

M

Yeah I think yeah. Logically, it would sit after a routing.

J

Here: okay, okay, then we should update it.

A

Isn't it on the left there it's like down and across to the left. It says metering.

J

F

J

Where the actually the meter happens right so, but.

M

J

M

That's what the accounting happens: yeah against the bucket ID, so uh I guess what Gohan is asking where the policy is uh actually enforced. That would be after the routing logically in the pipeline, yeah, okay, sure, uh I, think uh yeah. We can look at that. Just update this pipeline diagram.

J

And to the IP prefix I guess the clarification will need is that uh these IP prefix, you know, for example, the meter reviews IP prefix.

J

Maybe is easier, but in the St and the pl scenario is that the IPv6 that has been rewrite to the prefix or is that ipv.

M

Right so I think this is the prefix in the packet. So from what we understand, this policy rule is um used only in the case of the internet uh facing route case so uh like for direct route case, in which case it would be the dip in the packet.

J

I sorting the Press document there is in the 11809. There is a all traffic from UI to a private link destination right. So in that case, uh what do we count? There.

L

So usually the BL is a mapping entry and there would be a bucket, usually I, think that's how it's deployed today, as We Know.

J

Okay, okay, but it still works to clarify that uh you know this. This policies IP address, which one failed to to.

J

um To map right so in case of uh IP address rewriting case.

M

Yeah I think uh one thing is yeah, probably when the P4 model uh uh proposal comes in, uh those things should be get ironed out as well, because there I guess we need to uh uh in terms of the behavior we'll have to specify what exactly is the IP address that is picked up for this policy lookup.

D

Yeah I think one of the things that really helps I think this came out in last review as well is the relationship right how things are related or interrelated.

D

um With respect to you know the current objects that we have, whether it's route table, whether C and I, whether it's mapping table whether it's Echo and if we are defining something you know as new things, such as Mutual policy. What's a relationship, you know: how do you derive it and then how do you really? How does it come into picture when does it? When does it form um and then and when does it get deleted and so forth right.

A

J

The pipeline picture so, okay, so let me ask this question right. So what what do we expect? uh You know we will have a draft for the people pipeline.

M

uh So I think yeah I think we need to start uh or we'll start looking at that as well. We haven't got any bandwidth to look at that, so it should be coming soon.

M

Yeah I can't commit to a time yet but yeah. We will uh also uh look at that proposal for the P4 pipeline.

L

So once we have a grade here, I think we'll start working on that coin agreed on this.

J

Oh yeah, oh I mean I, think you know, um you know the the the the requirements.

H

J

The requirement is part is not it's money about the clarification right. So these are the you know. All these scenarios we need to address wrestlers, there's not nothing to not agree on I mean uh yeah. So then your implementation, if it matches right so yeah I, think we may have some comments. uh You know you know. Some of the name is right. So you know those kind of things, but uh uh I think we we should. You know start working on that P4 pattern to get more calories just help to understanding yeah. So.

M

One thing is that so, along with this, uh the schema definitions. From my FTP point of view, we have also included some examples uh for.

J

M

Is there right so I think if how.

J

About this right, so maybe next I don't think this week. We have the uh you know time now, but the next week, let's go over that uh again right so sure yeah makes sense. Okay, all right! Okay! uh This is a nightling session, yeah sex! No, no, not.

A

So we're pretty far over for today um is it okay, if I go ahead and um wrap it up for the day, my engineering folks.

J

All right, thanks that.

A

Work for you guys.

J

I

A

Mean I love, I love these conversations yeah, but my brain is full and I have another meeting, um so I'll go ahead and um end the call or I'll stop the recording I'm going to stop now and Princeton.