DASH Community Weekly, 14 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SONiC DASH Workgroup Community Meeting Dec 14 2022

Description

Eddie Ruan: how to consume DASH programming - discuss

Added new test plans for VNET to VNET testing #297
Added functional testcases #302
Adding SAI api tests for CRUD operation #301
bmv2 does not support set and get - so have skipped this in bmv2 until it catches up
Adding baby hero test #303
Adding verification dataplane system test cases at scale
Adding underlay BGP testcase #304
Able to run this on hardware, not in bmv2

Do we plan to have BGP support in bmv2? Check in BMV2 meeting on Thursday.
SONiC has underlay BGP testing, leverage this too?
https://github.com/sonic-net/sonic-mgmt/tree/master/tests/bgp
Testing w/CP involved, vs simply DP - how to determine how/when to test on DPU/Appliance/SmartSwitch?

SAI Challenger docs & tutorial test-cases #300

Add DASH metering to SONiC HLD #296 - Vijay
3.2.9 Meter
Meter_policy_id
add P4 code to behavioral model? Yes.

Dockerfile.saithrift-client has a static tag in the FROM instruction PR289 - Needs volunteer

DASH ENI create/remove tests PR293 - Intel

A

Call on December 14th, and um so these are the notes we had last time from Nicole or from our call, and we had a different things we were talking about.

A

um There was a question from Eddie um I think he's on the call today, but um maybe we could talk or recap a little bit for Eddie from I believe Alibaba, maybe just um a brief overview on how to consume the dash apis when we get there he's newer to the group. So maybe we can give a small explanation once we get there is. Was that your question Eddie yeah.

B

So so, basically like uh this will come from the discussion and my currently like the we have like the net or no balancing. They are using the RT flows. So my understanding, the dash is, you know, basically will not uh limited. We are to using those apis, so pretty much like just uh you can consider remaining or generalizing current Rd flow. Those apis, so I want to like make some discussion in the group to see that would that be the crack I'm spending.

B

Basically, once we get to the the API Library, we can using that not in the sink D, in the like, say from application directly to programming the flow for getting the the high CPS and also approaches, or you have to go through the think D. So that's something I want to discuss in the group.

A

Okay and I have to admit admittee, um maybe someone with more Sonic or software knowledge might be able to help out.

C

Yeah I think I can uh I can put some thoughts here, so we have the apis defined through uh gnmi right, so um so there is uh also the the application DB schemas defined in the Sonic hld. So so, basically the expectation is the the controller or the external controller can program using the gnmi to populate the application. Db and it's not expected to to directly go through Cindy I mean if that helps answering, and we are also defining the Yang model. So um so.

B

Yeah yep yep yeah, that's for the more like for the application to give you the policies they say, for example, for my in that case, it give me the netting policies. My.

D

B

Is likely for after the first package handling like um because normally are handling this within like either my info or within my app then after that I need to programming the flows. I need some like this, a quick and the efficient way to handling it. So that's why I will come to see what, because the moment I go into the the sink view, which means I need to go through the radius or the these crabs right so or I can just using this API directly to from wherever I'm handling the first package.

C

So after the first packet is being handled, it's uh it's. The connection tracking right I mean.

B

Into the hardware so can I just program it directly from where I make a decision, because that probably I can get the most efficient way. To put this into the it's more like a similar to what it's currently say, you use Rd flow. Then you program into the hardware directory.

E

Eddie Eddie this is this- is John um I, think what you're describing is more like an OBS offload kind of model where.

B

E

Where you have like your own application, your own kind of policy decisions being made and you're decided.

E

On offloaded and which ones you don't want offloaded, I think what Dash is trying to do is it's trying to say that all of the known services for a hyperscale cloud provider like have been understood for a while that Dash can just directly Implement those in the data plane without having to have this same kind of a uh let's just say: OBS offload kind of model, and so uh you know like the the first packet of the flow is processed directly in the data plane and the connection.

E

The flow entry and everything is just inserted directly in the data plane um uh without you know, requiring you know going through like multiple layers of like apis and and.

B

E

B

Me understand you when you say the date plan. Do you mean that this is the safer downfall in the because normally like a data plan, say, for example, in the soc or the arm chip right or for the GPU right yeah.

E

I mean I think the data plane is like you know: each supplier will have their own implementation of it. It might be a hardware pipeline, it might be, it might include, you know, scores it. Could it could be fpgas I mean it could be anything but like Dash is defining like what's being done in.

B

F

B

Yeah yeah, that's fine, so that means that so that is still aligned with my thought. Like I said: I, don't necessarily go through the sync D which is in the arm. So, for example, I could because it really depend on like say which Hardware we were using right, if I using like, say some guys, which they have purely the P4 agent, so their their software, mostly in the soc on the chip or someone have a more like the powerful. Like data plan, then I can upload like say into the DPA those kind of things.

B

That's fine, but that's not part of the the the sync D or the Sonic approach.

E

No because, like there's like nothing in dash that requires like on a per flow basis, uh to go up to. You, know to there's no like apis, like on a per flow basis, like the definition of the dash data plane is that on a per flow basis, everything is handled directly, and it's only like, like you know, ACLS and routing and statistics and stuff that you know have like apis that go through the PSI uh apis.

B

Gotcha, okay, so then, basically the agreement is like say: we don't limit this API to be consumed by the thing, the only if somebody wants to use it that way, it's okay, but since it is a library it could be using all like anywhere could be in the soc for the uh the the TPU all could be even the DPA, which isn't kind of like a risk, five or any accelerator right. So so it's already up to the the custom consumer. The choice can is that a correct understanding, I.

E

Mean I mean it's up to the supplier and the consumer like if the support.

B

E

Wants to supply it, you know at that level, then they can if they want to supply it like, within the full context of Sonic and All of the containers and everything else, and they could Supply it that way. But I think like this. This project has really been about like what's the definition of that.

E

Get multiple suppliers to all provide and implementation of it.

G

Is only defining the packet Transformations needed and defining the apis for those packet transformation and what we need to program in the hardware, no matter how it is implemented in the market.

B

That's a good clarification because I just want to make sure we are not limited. Yes, okay,.

H

Thank you, yeah, and one of the things we discussed hi. One of the things we discussed in this group is, is test cases and we'll be getting into some pull requests shortly about that and we're trying to create a really full set of test cases that run at the API level without enough sitting on top and that will sort of substantiate your approach right, we'll have test cases hopefully say yeah.

H

This works at the sci layer um and then and the network interface layer put whatever you want on top of it normally Sonic, but possibly more than that or other than that. Yeah.

G

B

Reason I want to ask this one because we probably will move ahead before like I say this is all the dash into Upstream into like I see the upstreaming will be in the 2023.05. That's a dash info, so I assume that will be coming there. So it's like. So that's why I would say our side. We probably will be continue there, but once the API is ready, we'll convert into the in the dash API. But right now we are continued on our RT flow apis.

G

Apis are already Upstream to psi, and uh you know. Implementations, of course, is up to the vendors, but you know are the implementations could be uh present at any time because the site apis are now available.

E

Hey Eddie I'd, like to just clarify one thing like RTE flow apis are at the flow level. Yeah Dash apis are at like the policy level.

E

um There's like a there's, a difference like I, don't see how like, if you have an application today, that's using RTE flow apis that, like you'll just be able to replace you, know uh an RTE flow data plane with a dash data plane uh because there's not like per flow kind of apis and dash like that. The definition of Dash is to push all of the per flow kind of processing directly, underneath uh you know the PSI layer and not expose it. uh uh uh You know in that way. Okay.

B

I see I I think I make a confusing there. I thought like a dash is the flow apis, because so we're talking about the all these, like the the we need to be near all the load balancing. So if this is only the policy, then that makes sense. Yeah, yeah, okay, okay, because if you put it in the side level right so I didn't wear like it's a hey. This is the under the policy level.

E

Right right, it's just it's just you know the the psy apis like expose. You know like routing ACLS policy, you know policy kinds of things not per there's. There are no I, I should say not no per flow. Apis like there may be some per flow apis as a way of being able to like monitor like which flows are in the are in the data plane, but there's nothing that uh that you know has to perform at a per flow level. Above the sci layer.

B

Then, because I also get confused because, for example, in the ha discussion we are talking about the flow level. Sync. If.

I

B

Dash doesn't even aware of the flow, then this won't really up to the vendor right.

E

Right, there's been a lot of discussion about that, um but so, like really I agree with what you just said like aha, could be up to the vendor. I think that there was- and maybe still is like there's some um goals to try to make interoperability between uh uh suppliers or even, if there's not interoperability, to at least expose like.

E

What's the contents of what's being synced between, you know, Dash instances uh and so I think that's why you know there's a lot of discussion about the details of what's in in the ha messages it's around interoperability and and visibility, but like what you said is correct. I mean like um it's. You know it's like those details are not visible to the PSI API layer, okay,.

A

So thanks everyone for the explanation and conversation and wondering maybe if we can move on to PRS or something like that, if we closed out Eddie, did that that's.

B

Good okay, so I was just two to the dash as the policy level, so.

A

Again, yeah, yeah and and thanks everyone for helping to go through a thorough explanation, um did did anyone have PRS or something they'd like to cover? Today, I saw a few coming through.

F

Yes, so um with side Challenger getting accepted um last week, we have PRS now for test plans and test cases that we're holding off. So we just made bunch of them at once and we could go through them. If you want you.

A

Can start to 297.

F

First, okay,.

A

F

So this is, uh if we look at the files change. This is a test plan with a few test cases in it, and also it takes the template for test plans and test cases from Sonic management, and we put it in here as well. So the test plan template um just to make it easier. I'm gonna merge back into Sonic yeah and do the transition to follow the same template format and everything.

F

So this is this one.

A

And is there something specific I should be presenting, or do you guys want to present no.

F

There is nothing specific, it's just a template, it's a direct copy paste from Sonic plus we took it and modified and we added few test cases. You know test plan for winner to vinat, okay,.

F

It's pretty small, it's the test plan and then it's the best one template from Sony.

A

And we saw the we saw the comments and everything.

F

Yeah we're gonna address those before it's being accepted. So now it's.

A

F

Just so everybody's aware, and when we come back from holidays, awesome, wonderful.

A

Right, okay and.

F

Then we have three or two.

A

um 302, you said.

F

A

F

uh So 302 it's nothing but the test cases. So there are a few cases for I believe five of them uh functional, uh inbound outbound in bidirectional. What the add-on top, comparing with the cases that were existing before is the fact that one's the flow is established. It sends the response back uh just to make sure the response back is being accepted as well, um and these are the test cases uh mentioned in the test plan that we just contributed. So these are functional.

F

They have in line the site, config and everything, and it's um yeah, just inbound.

H

F

Ip uh basic functionality yeah then we have 301.

A

Now did this one, do you need reviewers or anything or what's the next action on this.

F

G

Out of curiosity.

F

Yeah, we'll be good on all of this. If people can provide feedback like I said so, we can address it and record.

A

F

In after the holidays, so.

A

Okay, all right and then you said, 301.

F

G

F

So 301 uh adds quite a number of test cases that are about uh I, would say 10 files or so with test cases. Each has multiple in them and it's a PSI operation. This does not require any traffic generator, it's just make sure the Cycles have been implemented and we're gonna add um as part of our learning process. So basically, you know what we are learning to use the site. We created these cases to make sure the Cycles are working.

G

F

This should be already integrated, if not I'm gonna, do it the into the cicd pipeline. Now what we have here. Currently it's create and delete operations.

F

We have added also setting yet and modifying of the attributes with different values. Unfortunately, bmv2 at this moment does not support setting yet so we.

C

F

G

F

May qualify them on Hardware that supports those calls, but for Community purposes are skip at this moment, till bmv2 catches up and adds set and get support once that happens, we'll just remove the tags and they will also be picked up to be run in cicd pipeline.

I

I mean for this kind of a sensor. Can you mention the pr description uh you know.

F

I

Being speak, I don't see those been describing the pr.

F

Subscription yeah I'll add a lot more comments. Yeah just.

A

C

Do you need that.

A

Done Gohan just for me to learn. Is it more than just the title.

F

No, no, no, it's a description, whatever I said now to be.

A

H

F

The description of the pr so people.

A

F

Get the details, even if you're not in this meeting and you know, come later or oh.

A

F

I'll do that too.

F

Then we have um I would say three or three.

A

Baby Hero Test, okay,.

F

Yeah, so this is using the um Hero Test kind of topology diagram and so on, but when you write 48, 000 uh IPS, so only one IP per ACL, but all the other skills number in um and at side level, it's configuring everything via PSI, using the config generator, and on top of this we start adding verification. So these are like data plan System test cases, so.

H

It's like what.

F

Is the latency? What will be the packet per second? And, of course you know, bmv2 is going to report 100 packets per second, whatever or Hardware will report. You know Millions, so um these are System test cases at us at scale. Doing verifications a bunch of them.

A

This is cool okay, yeah.

F

I I split it in very small PRS. You know there are many parts more just to make it easier to provide feedback and you know accept one decline, the other and so on and.

H

F

One this is a controversial one. I heard that a 304., so um there is a spec for uh underlay bgp with up to a thousand routes and then 2000 routes when you do hfl over and so on. uh We added a test case for underlay bgp. We.

H

F

Actually able to run it on Hardware that supports it, but yeah BMV to I have not heard any plans of supporting bgp and probably will add a marker on it to be like harder or something like this, not BMV too. So. People know that this.

J

F

On BMW and it will work on harder, but this will be and we'll keep enhancing this wait. This will be underlay bgp cases in here.

K

K

F

Anyone know if there are any plans for bgp ever to be in dmv2.

A

Let me see if anyone from AMD some or and videos on the call, sorry so I don't see.

J

You know before we even go into um bmv2 I believe in society. Api is there that we have integrated to be. You know, call him in the underlay and how we're going to call this thing, because so far, all the overlay side apis are essentially making those you know before articles right. So what's what's the interface here for those cones.

F

So for underlay we are using a classic Sonic, so this cases in theory could run very well on a Sonic switch. My understanding is that all underlay portion for dash it is served by Classic Sonic. In a way there is no change there, so in theory you could take these cases and run the mono switch and if it's desired, I can actually make a demo like that or I can run this.

F

um Let's say on a hardware platform that supports this on a dash dpu as well as run it on a Sonic switch, and it should work.

J

But your assumption here is that your assumption here is that you you're going to be configuring all these underlay on a dpu, um because you, when you basically set bmv2 I, assume that you are talking about just one physical device present in an appliance use case, for example, where there is no switch. And but you do need to configure some underlay.

J

Isn't that going to be, you know going to work or no.

I

I think I think probably I think probably they need to have a you know, test the description right. So I I, don't know like uh I. Look at the briefly look up here. I, don't know like what what test do you actually draw it's unclear to me.

F

Right, I, I love the description I mean the feedback is noted from the previous PR uh I'll detail, all these in the description of the pr, but specifically to all the underlay cases. My understanding- and you know correct me if I'm wrong- is that all the underlay it's inherited from Classic Sonic.

C

Yeah I think that that's right, I I, have an action item also to update the underlay section for the Sonic hld, which I'm working on so I'll. Add those PSI attributes that we are going to use for underlay routing like in it. It is uh the correct understanding that we are going to use the regular Classic Sonic yeah.

J

So in in that case, you know, I mean I mean.

I

I think I I think you know. Basically, if I understand correctly, you know these underlay rounding it's uh already covered by the standard Sonic test.

F

Yes, that functional level, so all the cases from Sony will apply here since it's the same thing, but uh Dash has certain like scale requirements like, for example, a thousand bgp routes which a Sonic device may have higher requirements, and when you have h a and one fell over, it needs to be able to load out up to 2000 routes. I believe that it's in the spec, so things that are Dash specific in terms of number scale.

F

Those are we can add in here and that I think they have value yeah bgp functionality is the fact that it sends the packets and the communicate.

I

And no I I think I. Think the you know the the sonic touchpad already have those you know, bgp setup, all these things right, so we should reuse it here. I mean it seems like this. One is another pgp setup that which is not reusing. What we have in the Sonic is.

A

It at scale Gohan, is it it's fast.

I

Part that the the Sonic uh test didn't cover right so but I think in terms of skills. The Sony. Can you know um the test? Can change your skill with me. Currently we test more routes and what the current delay you know 1K routes right. So therefore you know we can dance uh downsize that that test I mean they, but the infrastructure is there right.

I

So what I'm saying is that if we want to add some bgp test, we should go to the Sonic management to add those tests, leverage the existing infrastructure there and the modify and then add some more. You know, features there right so, instead of creating a new hash infrastructure to test it, just under the bgp for Sonic, it's a whole lot of work right. So.

F

Yeah I mean, with this I totally agree anyway. I believe the intent is that in few months or how long it will take all this will be migrated under Sonic.

I

No, but no no I mean my grade is one thing right, but other things that you know this one start a new test infrastructure on bgp right which, which is already there in the Sonic management I mean if you, if you as you, you know that that's uh that's a you know your new setup, all these things right. So why not leverage what we already have in Sonic and you know uh doing those tests, there's a plenty.

G

I

Yeah yeah, we have the bgb test. If you go through the test test, so I think there's a yeah bgp test.

C

I

C

In fact running this test track yeah, so it's like no. We are in fact running these tests, so it is possible that we can use these tests for, uh uh for all the under latest cases.

I

You know some scenarios I, don't think the Sonic currently covered right, but we have the infrastructure. I mean should usually we use the infrastructure. I mean this one seems like setting up a new infrastructure just passed to the bgp.

F

No I I agree with you and I think it's a good idea. I'll take it down, probably from here and I'll, actually make it in Sonic management. I think you are right for the underlay portion with Sonic management to have them there. So I'll make a PR in Sonic management for it. Okay, okay, great.

A

G

I think I have.

J

A quick question: okay,.

A

J

Have a I have a quick question on this particular topic. Right I mean we're talking about Sonic, part and so forth. Right, it's good, um but if you, if you believe that even in in just in the dash uh part itself where you have just a single dpu Appliance, there is no switch Asic and then your you do need to program certain underlay functionality right and you want to test end to end.

J

um What's your suggestion, do you want to bring in all the existing test cases from the Sonic management and then you know really try it out in the dash.

J

um Instead of you know, redefining those things I I I see this thing that we are punting it into the Sonic side of the things, but we do need to do sometimes. You know this end-to-end testing for that. What's your suggestion here right.

I

You should I think there's a a couple of things right. So what why is the you know for this one with this specific one? This is uh uh with uh control. Plans involved right, so it's not just the uh you know purely the data plan testing. This is more about the the bgp, the control plan testing right, so you need to have a you know: bgb Stacks running on Sonic. You need to have all this Sonic integration in order to to to finish a test. It's not like!

I

Okay, because the previous uh you know the um is Heritage or whatever the test we have discussed in the dash is purely about calling the side uh shift. Api- or you know this um Challenger layer right so that and then you know and sending the packet to verify the data path right. So I think that one we can continue to do it right. So you know even adding some. You know uh underlay layer, API, call and verify those things. I mean I. Think we will discuss all those things right.

I

So this this one is is different. This way, is uh you know um the control plane vault? Therefore at least I I? Don't think we have, uh um you know have have to have covered this case say: okay, this case also covered the you know here. My suggestion is that you know, since this is more like a control plan related, you know you already have a an underlay. We did it, so we already have the Sonic management infrastructure to cover those. So why not.

J

Look, you know, okay,.

I

J

No, no I Now understand your rational. Well. Thank you. I think that that explains it so now, in other words, for anything that we we require any underlying functionality testing we are essentially doing through some. You know static routes and so forth, which we carried out anyways right now on the dash side, and that will continue to suffice for as far as you know, all the other testing is concerned, but.

I

We'll just go ahead.

J

And do that yeah, okay,.

I

I I think you know in terms of uh the PSI API those data plan, functionalities, I, think the you know the Swift The Challenger. The framework is support, not just the calling the overlay, but also calling the underlay right. So therefore, you know can continue to have those things. You know data plan testings there yeah okay got it thanks.

A

No guys I do need to shift gears and hand over to Prince. If that's okay,.

H

I had one more PR.

A

Yes, sorry: what um yeah? That's fine! um You want to do one more Chris.

H

If that's okay, yeah.

A

And then that should leave plenty of time for uh Prince yeah I'll try to make it brief. I.

H

A

Which one is it.

H

uh 300 magic number 300.

A

Okey-Dokey, okay,.

H

So um I can actually um share my screen. Okay,.

A

So that's okay, make it easy on me.

H

Okay, uh it's no fun you can hold, which page to click on so hi everyone. This one um I've wanted to start creating some tutorial content for using side Challenger, because um even our own experience, you know, there's an onboarding.

H

um You know curve to ascend and we're we're putting lots of test cases out and so I wanted to start working on like educational content and and some tutorial examples that build a bit of knowledge base and it's.

H

This is my start at it: it's not thorough or complete, and what what I'm doing is focusing on tests which show you how to configure um device devices using config files and config scheme Etc and I'll get to traffic testing at another time, and we already have test cases that are running that I want to do is start backfilling with the knowledge base on how they work, and so um I also wanted to document the side Challenger test bed for dash.

H

So I started with that. Just explaining the framework and this this is basically now committed a readme file, and you can see this content looks familiar from some of the slide decks over the last few months.

H

So I just committed that to a readme Okay so discussing everything, that's the main, that's one of the main Contents I wanted to share, and the other is this tutorial file, so the tutorial readme and it goes through a bunch of little test cases that I wrote just explain things and walks you through progressively how to write, how to configure the device and using the site, Challenger format. And um so you have to look at this kind of as a step-by-step set of lessons. Yeah there's all these different ones showing different techniques.

H

Okay- and uh you know- don't want to try to go through it all here, but I think people find it interesting, and um this is actually the pr where you'll see all the files that are changed. It looks like a lot, but a lot of it's just test cases. These are little test cases, Json files and python files, that's the main content and then the readme I just showed you is that tutorial readme a couple little tweaks here and there to some existing files just to sort of make everything consistent.

H

So um hopefully people can take a look at that and these all these test cases right now, they're already being run in the ICD pipeline. The pr will run them so that they're not just Standalone lessons that get stale over time, they're being run by the CI CD, with every push with every run so they're going to be kept fresh I, don't like stale content building up. So that's the pr. Hopefully, people can take a look at.

A

Find time yeah, maybe I can make a stretch goal for Yusuf to do like a little video of one of a run through of one of the test cases, and we can attach it or post it.

H

A

A two to three minute: video.

H

Yeah, if he wants to pull down clone the repo and and run and go through I'd, be happy to work with him. You know offline okay.

H

That would be fun that.

A

Would be a good.

H

Yeah some people are.

A

Visual Learners and some people are, you know, auditory Learners or reading Learners. So that's awesome, Chris. Thank you. I've.

H

Already got I've already got fans.

A

Yeah awesome yeah.

K

That's a great job, actually Chris yeah I, really like such kind of uh documents, because it's really helpful for starters, especially.

H

Yeah, yeah and Andre, you know you can take any of this stuff and put it into the side, challenge repo and adapt it if you like, you know and I'd be happy to see if there's anything useful here they can get wider usage. That's great.

K

Thank you for that. Yeah, okay,.

A

And now I'm going to hand off to Prince. Thank.

C

You Chris yeah thanks uh Christina. So uh does the pr 296 submitted by Vijay for the metering proposal so I'll? Let Vijay speak about the the pr.

A

L

Hey yeah so uh can I just share my screen.

A

Please do yeah unless you want me to do it.

L

L

Are you guys able to see you yeah yeah uh yeah, so this appear basically in discussion with Prince and Microsoft uh folks, we just uh are adding the dash uh metering feature to the dash, uh so I'll just go with the changes. These are basically the changes in the Sonic agility and also introduces a bunch of uh new schema objects in that DB. So I'll just first look at uh cover all the uh schema changes in FTB that we are introducing for the metering feature um and just open it. Yeah.

L

Yeah, so first thing is that we're introducing a new metered policy and uh meter rule objects, along with the dash meter object. So the policy is basically uh allows uh allows you to configure IP prefix, which is a priority based match rules and from there you will be able to derive a bucket ID and, along with that, there is also uh something called as the dash meter.

L

So this object is basically given a bucket ID or associating it with an eni, and you can also give like an optional metadata stream that the orchestrator can use to pass in some context about the uh what the bucket ID means- and this is basically uh used to account for the traffic uh on uh on that bucket on that particular eni, and there are TX and RX counters which are number of transmitted bytes and received bytes.

L

So these are read-only attributes, so you would be able to fetch the the statistics on that bucket using this dash meter object.

L

So, apart from that, we are also uh introducing attachment points for uh these buckets in the mapping table. I think these uh some of these were present already. uh So this whatever bucket is getting created here, can get attached to uh multiple points. One is in the mapping table and then there is the uh route table and it also can get attached to the uh policy rule right.

L

So, along with that, the route also has an uh Boolean which will allow the policy to be enabled or not so, basically uh uh when, if there is a route hit, uh then the route itself can give a bucket or it can enable the policy lookup, which will tell uh the packet will go ahead. Do a policy lookup and pick up the bucket ID from the metering policy, so this is just I just wanted to go over the schema.

L

First, uh you can just look at the description that we have added for the feature so uh yeah just kind of uh this is a description of what exactly uh we have described earlier in the uh schema. So there are route table, entries, mapping, entries and meter policy rules. They are configured with the global metered bucket values. The bucket values are globally unique across all v-nets uh in a given region. So uh so these meter buckets are mapped to per eni specific metering buckets.

L

So uh the global metering bucket is associated with the gni by doing the dash meter en I create, which is basically Emi, comma meter bucket ID. That's the entry that you create, which will associate a particular bucket ID with that particular Emi, and the accounting gets started whenever you pick up this bucket ID in the pipeline.

L

So and if, uh if there is a, if there is a routable entry or a mapping, policy rule which is hit- and you pick up a meter bucket ID in the pipeline and that particular bucket ID is not provisioned, then that doesn't get metered.

L

uh So uh when the year the per year, 9 bucket ID also needs to be a deprovisioned or deleted when traffic no longer needs to be metered for this combination uh and then the of course, the whole idea of doing all this is to do a get on the for Emi, comma, bucket, ID or Emi comma start to fetch all the meter, TX and RX statistics. So the way uh this Statistics is used in Azure is for billing on a perennial basis.

L

Each metering bucket currently accounts for number of bytes number of packets is optional. uh Coming to the meter policy, it consists of a bunch of metering rules. It's similar uh semantic like somewhat like ACS.

I

Sorry I mean what is This Global meter, uh I'm I'm, not sure I understand it. So meter is just some kind of colors right. So what is This Global meter Park pocket? What's this me is.

A

It per e and I or Global or per you know, is that what.

L

It is yeah, so the uh the metering bucket itself is considered as a global resource, but since we need to account for the uh the accounting happens at upper eni level, we need to associate that uh bucket ID with the particular eni. uh So that is where there is the um there's a separate create.

L

That needs to be done on a uh on the dash meter, eni comma meter, bucket ID, but the way we are looking at it is that uh bucket IDs are like a global resource like because the packet IDs itself can be like a 32-bit value, and this 32-bit value can be associated with whichever against, whichever eni, where it needs to be accounted.

D

So, for example, go and there could be a bucket ID, which relates to a let's say, a private link uh that private link, the the same bucket ID Associated to the private link, but on different Enis. uh The counts are different, so the same bucket ID is associated to eni uh one and the same bucket that is assigned to eni 2.. The two counts will be different depending on when two hours.

I

Is this meter represents a single uh counter or represent a set of counters that you know could be um mapping to different UI I.

L

Mean itself is a single counter, so uh basically there's a TX under RX contract associated with a single bucket. So in terms of there's the hardware resource, the way you can think of it is the hardware itself has like a bunch of buckets, say: 16k, 32k, 64k and so on. Now you kind of take these buckets and you're associating it to particular eni, where the accounting needs to be done is.

I

A particular rule right, so what do you mean by associated with a particular UI.

D

So, on a year night, there are some set of uh prefixes all right, I think let if I can uh rephrase that so this bucket ID is like a class right. It's a class ID, it's like a class now for for this class of traffic. Each Ena has its own. This thing on how much it uh let's say how much traffic it's sent to that particular class right. So the global bucket ID that Vijay is referring to is the class ID. Is that identification for the class?

D

And there is a perspective from each eni on how much traffic was sent for each of these classes and on that eni, comma bucket ID? There is one set of counters, TX and RX counters right, but the class itself uh can have different. uh Each eni can have a different counter. I mean different values for that class, because it's Ena, comma bucket.

D

I

D

Definition itself happens, yeah, where the class is derived is either from the mapping entry or from the route entry, or from this uh more uh flexible metering policy, which gives a prefix versus a class foreign.

L

H

You say earlier, these were read-only values.

L

Yes, the counter themselves are read only because uh the in the pipeline, whenever we uh this particular any packet, hits this particular bucket. Then it gets accounted against this bucket and then the idea is to read it back. So there's no way.

H

Is there any way to clear it.

I

Do do you have some specific examples on that? uh You know this.

L

I

Meter, class and uh um and this map you use um I, don't know if it's I'm only one that is to do not quite understand the um you know.

L

The yeah, so we have added a few examples uh below so before, going to that I just wanted to just finish this one sure so I think yeah. This is basically uh just describing the meter policy. The fact that it contains a bunch of rules and uh each rule consists of Ip prefix map to a global meter bucket ID.

L

So the policy itself is the attachment point for the meter policy is applied at an eni and, of course it can be shared across EMS. uh So there is one uh fact here is that the metering Behavior Uh it's possible that in the pipeline the packet can get hit in the route or mapping, people or metering policy, because all of these tables in the pipeline are deriving the bucket ID. So we kind of need a way to kind of tie breaker or to choose if multiple tables in the pipeline uh derive the bucket ID.

L

So these are based on the requirements. That one thing is that when the route table entry has a policy lookup enabled and if the policy is a hit, then the policy meter bucket is picked over the routable entry meter bucket. So this typically, um if it should not happen because like if there's a policy if the route itself has the control to enable the meter policy lookup.

L

So now, if uh route is for whatever reason, if route itself is giving a meter bucket ID- and it is also enabling a policy lookup- and it happens that the policy lookup is a hit, then you get two bucket IDs in the pipeline. You just need a way to pick one so here. Basically, what we're saying is that the policy bucket is winning so uh and going forward in the pipeline.

L

There can be a subsequent mapping lookup, which can also give a global metering bucket, and here the mapping bucket ID is overwriting uh previous uh previously derived bucket ID only if it is a private endpoint. So if it's a private link mapping, then it is overriding it again. All this is based on what the Azure requirements and uh they've just made these rules right. So.

J

There are some requirements, I mean sorry. I have never seen any Azure requirements from metering, at least not in any of this community talks right. Are there any requirements that were presented before.

A

L

A

In the chat window, just what I had derived from prior conversations and um there's a link there, it's much more generic than what you have written.

J

No I I find this one honestly very complicated I, don't know why does it have to be this complicated? There are so many things basically that are thrown in here policy class bucket ID eni I mean all of those things. You know if I I mean if we have a first a definition of this one, and then we can say that how they are related between.

C

Actually, we can, we can add a section for.

J

C

All the requirements but yeah yeah.

J

C

J

I can actually understand the flow because so far it is you know: I I, it's I'm having a very hard I, don't know about others, but I am having a hard time really.

I

Fast, if I think you are not alone, I mean I, I, I, I, I. Think I also have a hard time to understand the uh yeah. This is, it seems very complicated. I mean I I'm, not saying that you know you know it should be. I mean I'm just saying that I yeah I think because I I don't understand how this works and even for this, some basic concept, like Global metering, bucket ID, the class idea. I, don't think I understand that.

I

What is actually uh there so I I, don't I, don't understand the behavior model here.

L

We are so perhaps we can add one uh like a pipeline diagram with the behavioral model, just showing how it's derived at various points in the pipeline. That's not something we have right now, uh it's more like described in text so uh yeah. That is something we can add. Yeah.

L

Yeah, let's go through the example.

L

All right, okay, so I think this is basically whatever is the winner to v-net case, where we, uh we have just uh added all the uh meter policies, yeah metering stuff into the existing example. uh So here uh like for this particular eni uh there, you can define a meter policy ID and the meter policy itself uh is defined here. So basically, this meter policy is saying that uh hey. This is a ipv 4 meter policy and the rules.

L

There is one single rule in this example which says that match on this prefix and pick up the metering bucket twenty thousand- and uh you can have multiple rules here, all inside the same policy with which can all derive different metering, buckets and.

D

uh There are prior it's a with priority, it will just add it's like a ACL with priorities and there are rules.

L

Right so the as far as the meeting buckets itself, there are like four buckets in this example: one thousand one thousand one one thousand two and twenty thousand- uh and all these are associated with this particular eni, which is defined here uh now we can go and see how these each of the metering buckets themselves are attached right. So here we have. uh The metering bucket thousand is attached to the uh this particular route, which is matching uh which is matching on 30.0 16.

L

and uh metering. Bucket thousand is also associated with the 40.0 16., and then we have uh this mapping entry, which is uh deriving to me during Market 1002., and we have the meeting bucket 1001, which is attached to this mapping entry right. So now let us go through the case uh like here.

I

Before we go to the case, I mean I I think there are two things right, so why is that? uh um You know this uh Dash side, abstraction right? What kind of uh you know data plans? Metering objects were providing to the uh as a as this Dash side layer. Another one is the specific Azure case. How do we complete? How do we leverage the uh you know that that the apis to to satisfy the Azure requirement?

I

I think these are the two separate scenes right so um I think maybe um you know I I get what trying to, but maybe we're trying to combine these two and then I think you know I'm sure you know Azure use cases where it's complicated, so you think it's a little, maybe a little bit harder for the people to understand. You know um what what is the Azure use case right, but maybe one one thing is that what is this? You know the the data plans, the side apis that we're going to provide?

I

What is the model there and yeah? So I I know that you know these two things are, you know obviously connected right. You have this requirement. You divide these apis right, but it's still separate right, so you know maybe people you know like honey, maybe they're interested and understand. Okay, what is the size apis? What is the model that I need to support in terms of uh you know creating those mirrors and then later you know.

I

At least you can understand that he counters down okay. What is the requirement to write so they they um then be able to figure out? Okay, why? We have this API design right so um that I.

L

I

L

Yeah so I think the people, so that's I appearance itself is uh like generated from the uh well from the P4 Dash P4 uh tables right so I think once we have the once we bring this in into the dash P4 uh bmv2. Then I guess this IAP. It's also will be clear, but I guess what we are just presenting here is uh uh just the app DB and the schema and how it will be programmed and pushed down. I think the dash P4 changes is has to come separately.

I

No I I get that yeah.

J

Yeah I think this is I mean one of the things that essentially perhaps myself and and Gohan are asking. Is it's still still in terms of really the relationship right I mean, of course we we have. We see that okay, you know there is a data model here that you're trying to present, but even at the top level, even before we go into this thing, so some of the new Concepts that you brought in like either classes or bucket ID, or even some policy ID and all of those things.

J

How are they related and how are they shared between certain other things that we are used to like eni and v-net right, because some of the things that you I am I'm trying to grab some relationship here right and if it's it's easily captured somewhere in the document, then it will help me really go through the flow right.

J

L

I think we should uh yeah.

D

I think we can add one section like uh Prince I also said right. Just to briefly this thing: um yeah yeah.

A

So I think here, if we can pick this, so we have two minutes left I'm wondering if we should pick this up in the next um Dash call when we pick it back up you guys or do you want to keep going today, um I don't know who has 10 o'clocks or not.

L

So Christina, if I might I, can I just want to finish the example.

A

Yeah: okay, like.

L

I think that for uh a few folks, if they go through this example, it actually talks about how the packet flows and what gets picked up right. So, okay, great yeah, so just quickly, like I, think for this example configuration these are uh so, let's take a packet if you take a packet which is testing to ten one one right. So um here we have the LPM lookup which uh which hits this 16 entry.

L

And uh here the action is uh uh map routing and it goes to the mapping table which hits the 1011 and uh here because the meter bucket is associated with uh 1001 and it hits the mapping. 1001 is associated with the mapping entry 1011, then we'll pick up this metering bucket and it's used for accounting, the traffic so and this meter bucket is associated with this eni. So whatever traffic hits this mapping entry is accounted against.

L

This see a nice meter bucket, so the idea is, you would be able to query on the cni comma 1001, which is hey. uh Give me how many bytes uh uh went to this mapping entry. uh Ten one one on this eni and it'll, be able to tell that hey you transmitted. Some X number of bytes, hence you will be built at this particular rate or whatever right.

I

Maybe this part is something I don't understand because a particular route we hit a route. He already, uh you know, classified to a particular UI right so saying.

C

But Gohan it's not the mapping table is not right. The.

I

Mapping table is not yeah.

C

Mapping table is not associated to any na.

I

Oh, it was a mapping table is not so. This meter U is associated with the mapping table, not the route.

C

L

So it can be associated with the mapping route or policy. There are three points where it can be Associated, so I think these are the different cases where.

I

Oh, when we say the mapping table is to say to PM mapping table which is not associated with a particular UI.

G

I

Okay, well, that helps yeah.

L

Yeah, so this is uh the next case is also uh similar, where it hits a routing entry and again it hits a mapping table which is the meeting bucket 1002. It's uh picked up uh because it's hitting the mapping entry uh so uh in the other case, where it's direct routing, there's a metering bucket only associated with the routes. Obviously it comes with the uh the metering bucket 1000, which is associated with 30.0 16, is picked up um and.

I

Here, so let me let me, let me ask you for this route right so because a route has to Associated dni there, then what does it mean that the meter also have about I have Associated dni here, and so this part, you know, um because you know, mapping table I, understand that right. So there's no CD UI, um but for the route it already have a enite right.

I

So then, what if the meter is not config to enable that the bucket ID on the UI but the route say Okay pointing to the speeder, is that is.

L

I

A bad indication or.

L

Yeah that would be considered an invent. So that's what we have pointed out earlier right, so I think it's mentioned also about that. If it happens to hit a meeting bucket in the pipeline, where it's not actually associated with an Emi, then it won't be getting metered against.

C

J

L

Does it matter.

K

J

Does it matter because you know a mapping table associated with v-net right and in if, as long as you basically say that if the bucket is associated with a v-net, why does it matter for the eni right? Why do you have to really associate the buckariety with the DNA.

L

J

D

J

D

Yeah, goodness no I think that's okay, I reckons I can just quickly say since we're running out of time too. So, roughly about this thing, the main issue is uh for implementing this in Hardware right, I think we can't, uh because the this thing gets replicated for every each of these classes gets replicated for every year now, so we can't all put all the uh all the buckets in all the year.

D

Nice rights, it's a hardware resource issue, so there is a finite set of resources that an implementation would have in the hardware for each now that is replicated per enx finite set of buckets where you can count against right.

D

uh Because of that there is, uh you know not all the packets in the system are associated to every enact. There is a subset of it, which is valid for each vni, and that is where the association comes.

J

Can buckets be shared across the eni.

D

uh Shared across the bucket ID can be shared, but the bucket itself is per unit. It's a bucket comma eni, because it it matters for billing to say that, for this eni I sent it to a class that I sent it to a private link, but from which year Knight did I send that right. So that accounting happens per eni, but the bucket type, the class ID itself can be common. There can be a class ID 2000 for a cosmos or some other private link right, and that 2000 is the on eni one. There is.

D

uh There was traffic sent to 2001 that class, so many bytes, and so many whatever the TX and RX, so many bytes from eni 2 to the same class. There is a different uh count of packets that were centered. It's built separately, built different.

J

How do how do how will people use classes I mean? Where does it map to.

D

J

D

Is uh for billing right? The controller says uh for uh this Cosmos, whatever, let's say some private link going for some service. The bucket ID is three thousand, and then uh the billing uh infra picks up this 3000 from eni one and eni 2, I, guess but I think the this more authoritative prints can explain how the yeah.

L

I mean right so one way to think about it is the bucket ID, the semantics of what this bucket idea uh accounts for is already uh Associated and uh attached to by the controller. So the controller notes the fact that hey this bucket ID 50 000 represents uh private link traffic, which is attached associated with this mapping entry, and it needs to be accounted against. The cni.

A

So I am really enjoying this conversation, but I'm also five minutes late to a different call.

C

Yeah I think we can. We can come back to this uh I think so in a separate meeting. But by the time we will have more clarity on the requirements and what the expectations are. Yeah, yeah.

A

I

Think, okay, are we having a meeting next week or is it canceled.

A

um I was I'm, I was gonna, um cancel the next two Gohan, because it's the holidays. Do you want me to hold one?

A

If you want me to hold one I'll, do it but I'm thinking most people might be out.

I

I, don't know Prince, you got do you do do do do do. Do you think you will cover this next week or we will find another time.

C

um I think next week would be a little bit.

A

I

A

I

All right, all right: okay,.

A

um Okay, well, we will pick this back up and if I can just show you one thing: Vijay.

J

A

Know I wrote this a long time ago about the counters and um you know what I understood about the counters and how they're assigned what you know the rules, the mapping blah blah. um If you can look at this and and align this with what you have um I put I put it I'll email it to you. The link, but I also put it in the chat window. But um if you could, if we could work together on yeah, okay, great.

L

Good because I think we learned uh probably another kind of a picture and.

A

You can reuse any of this. You can reuse any of this that you want, because I gleaned this from a lot of meetings that we've had so.

L

A

Right, so thank you. Everyone for coming I'm, gonna, stop the recording and.