DASH Behavioral Model WG, 6 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DASH Behavioral Model WG Apr 6 2023

Description

PR358 - Support SAi attr generation from annotattions

Marian provided overview as to why we are doing this:
Need to add more metadata to SAI headers to explain relations between objects.
New method for generating attributes for keys and action parameters.
If you place this directly into P4 code in the form of annotations,

Merge PR246 - Marian to sync to main, then notify Kristina

PR308 - does not compile for DPDK backend (expression is not implemented)
Andy added IF/DEF, does not handle multiplication
Marian will fix for NVGRE
Andy will find link for specific lines of code
Here is the link to some sample code with #ifdef to handle this kind of code: DASH/dash_vxlan.p4 at main · sonic-net/DASH (github.com)

A

To the behavioral model meeting for one of the April 6th I'm going to go ahead and turn off my camera. I hope you guys can all see my screen. um So what we covered the last go around. Was this huge um bmv2 for service tunnel and we had pull request 308 and I captured a few Snips of the different changes to the different areas.

A

You know here and uh you know just different Snips- that Murray and showed us last time and I believe I sent these out in meeting notes, and so that's what we did last time, the Q a around that PR.

A

It was a pretty big one, and then today we were going to talk about PR 358, which I just had it up on the screen there. This one here support PSI, attribute generation from annotations. So unless and then, if you guys have something else, we can tail on after that. If that works for you and get a head nod or a yes.

B

A

Cool so I'm gonna stop presenting and let uh this will be Marion go ahead.

C

Yeah thanks mm-hmm, so I'll start with the with the background why we are doing this initially, uh there was a need to to add a a bit more metadata to cite headers that explain, uh relations between objects and stuff, like that, um mainly it was for the CRM purposes um or critical resource monitoring uh to help explain in which cases we need to also provide more uh data to study availability, queries to understand how much resources we have left um but adding this metadata.

C

My initial uh attempt at this was to just try and guess uh where uh what are the rules for that? uh But that's not really. um That's not something that we can continue going that road uh uh for a long time, because there there will be more and more rules and they will all be embedded in code for the uh header generation. So uh we want to move away from uh that approach.

C

So this is the first iteration of that uh I'm, introducing a new method for uh generating High attributes, uh Four Keys and for Action drivers from uh from annotations. So the reason is that, if, if you put, if you put this stuff directly in P4 code in a form of annotations, uh let me find.

C

That yeah, so here are two examples.

C

um So again, three examples. um First one would be we want to have this. um For example, this uh parameter vni value. We want to find out what its type is, so it's um it can be derived from the uh from from the size, but it's not always true. Sometimes it has some special type like I, will be adding annotations to further uh further keys and parameters. So let me give you another example. So in this case this VM vni has side type of PSI u32, but what if we had?

C

Let me just view this file. What if we had something like like this.

C

How can we guess that this is the MAC address if what is best to um to P4 Json file is only the size, meaning that it is just it's just saying that this primer is 48 bits more. So previously we had a rule if the value is uh for the uh has the size of 48 bits, it's always a MAC address.

C

So we would need to put this kind of kind of things and also like if it's um I don't know, uh 32 bits, it might be an IP address, so it gets complicated uh more and more every time.

C

So what we want to do is to be able to kind of do same thing as inside to Define all of the metadata right over here, and the example would be like that. So we have a parameter, so we can add all of the needed metadata uh within the PSI structured annotation. Another example would be this: one sign IP, address family uh attribute or parameter that we set for the ACL group saying that this is either V6 vcl or V4 ACL group.

C

um So how would you derive uh this type from three to bits? um We tell it. We tell the uh API generating script directly to type and we can also add another metadata like is this a resource type uh saying that? Yes, it's a resource type so that when we will query availability, we can query separately or before, and V6 and stuff like that. So.

D

I'm sorry, um a couple couple of questions yeah, so uh yeah, Marion, sorry and the the question I have is what is: is resource type signify here uh yeah? So this.

C

Is this is coming from PSI apis, it's a tag.

C

Let me just open an example. So if you go to site API and open stuff like.

C

C

So regular um style route entry has an attribute, for instance, IP address family, which is read-only attribute, but it also has this kind of tag saying that this is a resource type, meaning that it is a classifier, so that you can. You can pass this attribute when you're acquiring how many routes the system has left uh that you can program yeah.

D

So you have some Hardware resource: okay, yeah.

C

Yeah, so you can provide a more detailed query and ask for specific resource within the object map. Okay,.

D

Got it now? The the second thing I just basically want to ask, is when you define a or when you specify a attribute, that is only applicable to a definition that just follows: right after that, or what? If, what, if I had? Let's say, you know uh two um parameters here: both of them happen to basically should basically take the same. Annotation can I, define one and then be applicable to both of them, or do I need to replicate this thing. You know for the for the second one that follows immediately after this.

C

um Can you give an example? I will understand. Okay,.

D

So, for example, uh yeah, for example, the the previous one in the VM BNI right, let's say in the VM vni in the previous one, where yeah so just no, no, the one, the one below that so.

B

D

Yeah, so basically you you have a sci type, says PSI, U and 32 right.

D

That's it right, but let's say uh just below that. You had another pick 24 right and defined, and you wanted that PSI. U in 32, also to be. You know, applicable to that one as well right, yeah, yeah,.

C

D

Have to do it repeat again or what what what would happen.

C

So annotation is for one parameter or one key.

B

Okay, yeah and by the way this is a P4 language, syntax specification. It's not something. We have control over okay and you know the annotations bind to the to the uh you know identifier following it, but you know if there's a lot of them or if it gets messy, you can always start using macros, but then you know that presents its own problems right, yeah,.

D

B

I completely agree: I.

D

Think now I can see more better, because what I know I see this thing that you know when you define this classifier for The annotation, that is, there is no uh comma. After that, it just follows: um yeah right, exactly yeah, it's.

C

Like a documentation over uh this.

E

C

Documentation over this one, so I'm trying to follow sorry- and here it's a better outline, because this is uh the oxygen comment right right over this attribute, but it is uh semantically uh meaningful, like all this x means something same thing here. So this is our uh documentation, but it's also a meaningful thing from which we generate a an API, so I plan on first.

D

You know what you know, what threw me off uh Marion is because, since they are both in separate lines, I mean it would I, don't know, but I mean a room probably make more of a readable part. If you had both annotation as well as the type sorry, the definition of.

C

The variable on the same line- yeah, maybe you're right, maybe like I- can just put them in the same line. If it's.

D

C

D

C

Right so I plan on adding annotations to all the keys and parameters uh in this following PRS, but this one is mostly about the infrastructure, and this is just examples uh how how they can be used. We have two parameters. We have also key or another thing that you can specify so so The annotation itself is a key value data structure, so you can expand that and um so, for instance, if our attribute is referencing, another object um like in this case we um we are annotating this ACL group ID.

C

The name is taken from Metro, but basically the key is ACL group ID. So the the thing is that um this is an object type and previously again we had to like derive uh the fact that this is an object type from um how it is named and uh derived the object type from the name itself.

C

So we want to alleviate all of that. So we'll just have uh object types specified directly, so it says objects because theoretically it could be more than one in PSI, but really in our case, for now it's just one but I I left an option like you can do it. You can provide more than one if it will be necessary, so it can be extended.

C

um So these were the examples of how to use them. What what actually happens is that um we have one simple, routine added to this uh generation code. It seems that simply you provide it with two things. First, one is the pointer to the portion of the Json file uh that uh describes either a key or parameter. Second is a dictionary that will be filled with all the data and it just parses the Json retrieves. All of this information, like you, could see in type.

C

Objects and moving on the more stuff we will need to add to the documentation, like some other things, maybe like special handling for enums. What would be the type or field or stuff like that? So all of that will be added in here.

C

um So it's very easy to extend I just take another key copy it to the attribute dictionary and that's it.

C

um So that's about that's about how how we parse this stuff from the Json and it's um it's used in two places for now, which I plan to deprecate at some point when we will fully move to that implementation. So previously what we had is um basically, as I said, we derived everything manually. Now we have an option. If, if we have structured annotations, we take all the information from there, but the old way is still available until we fully migrate to the annotations.

C

uh So this is for keys and for uh parameters as well and another thing that I had to do just to keep everything simple I normalized, the naming of um of those dictionaries for keys. So they had stuff like psyche types, LPM type, cellist type- it's not needed that it can all be just called type, meaning a type and site, because we have, we still have a much light to figure out what has been matched.

C

So that's another thing that you'll see in this code, but that's just a unification of uh uh of dictionary keys that we generate apis from so now. Attributes and keys will have dictionaries looking uh the same way, because both of them result in the same attribute and uh attribute inside doesn't really reflect if it was a key or an action rather yeah. So my plan is to First, merge that and then I will keep extending all of the tables and actions with this metadata and we'll see.

C

Maybe uh maybe even more information will be added, so it should be very, very like intuitive for anyone. If there is a new P4 code added, uh you will be able to just follow the example. uh Just add this annotation provide site type, maybe provide other things like if it's object or other uh other needed information to generate proper apis.

B

Yeah Martin I think this is really good. I'm, really happy that you finally got to this um I think it's a really good step forward to make the P4 code the single source of truth and not have heuristics buried in a magical code generator.

B

The one thing I would ask is, if we maybe start a repeat, even if it's just really Slim, it just lists the annotations and the proper usage or yeah.

C

You're right I'll add it to this pull request. You all right, I will list all of the available annotations. That.

B

You can use yeah yeah and at one time we talked about kind of a style guide or some kind of a something that describes how the P4 needs to be written. In order to you know, produce the result, and this might actually be that you know one time there was a little more, um let's say, hidden, hidden rules that you knew.

C

There were rules yeah, it was rules based. No, we can. uh We can go away from.

B

That, okay, good this- this is better.

B

And you know just for for people to know that this annotation technique is used in a lot of different places. In you know a lot of Open Source projects like Sonic pins, Etc uh stratum used it. It does make the code a little harder to read. You know, but that's that's the uh you know downside, but it's not that big of a downside.

C

Yeah, so if you don't have questions, I got like two more things: I wanted to bring up about other peers, so there is, let's start with uh with a simpler one,.

C

um So the bulk ad removed I, of course, pushed it again to to make all of the uh uh realistic checks class. So we can merge this one uh good yeah. Is there any comments? uh Please leave even there and.

B

It's been a while is that one uh any issue have you synced it to Maine lately? Are you concerned, because it's quite old and things creep in.

C

I can do that, but this is, but it is very, very contained and uh because it just implements two functions in this CPT file, so I'm not really worried about it, but I can I can sync to to latest Masters and also where was it.

C

This one I was trying to align to dpdk backend. uh This is the service tunnel from last week.

C

Still has an issue uh I don't know. Maybe Andy I will need your help to look at that this kind of code compiles for bmv2, but the pdk, backend uh Frozen error uh that it. This kind of expression is not implemented in the compiler.

F

C

F

Added an if def near that code, I noticed it when I was doing some changes it handles if statements General if statements inside of action bodies but not multiplication, and so these I just converted into simple. If statements.

C

uh So uh so, if it's enough support, okay got it, so I will fix that. It's for nvg re. Maybe I'll need to uh to repeat that, uh or rather I'll.

F

Find I'll find a link to specific lines of code that show this and send it to you. Yeah.

C

Yeah, if you fixed it, can just take it.

F

Yeah, it's an annoying difference. Okay,.

C

They're pretty easy to handle.

F

C

That's it thanks.

A

Okay, so you want me to merge 246 after about a half hour, is what you're saying uh 246.. So we just were on a minute ago, where.

C

You were going to merge sync to me just in case. Let me uh oops that's the wrong one. So just in case, let me first uh probably as Chris mentioned I.

A

C

It to the latest and then I will let you know. Okay sounds.

A

Good yeah, quick question yeah so.

F

uh I, is it true that you, you may have already said this and I missed it? Sorry, uh you are relying on like open source p4c, taking these key and action, Predator annotations and putting them in a p for info file or some other output file. Today,.

C

Yeah yeah, that's that is correct. Okay,.

B

Accents and Andy that's in the front end by the way yeah for a couple years. Yeah.

F

I know I know you were instrumental uh industry, luminary, Chris, Summers and yeah I'm.

B

So happy this is the first usage I've seen of the of the uh key value form of this, so yay.

F

That's the kind of work that got you appointed co-chair of the P4 API working group.

A

Yes, well, you guys might find this funny. I want to um Azure Linux champ award and I'm. Like do I do Linux, apparently I do.

A

It's this project, yeah that was funny I was I, was I was surprised to be nominated because I didn't know. So it was pretty funny.

B

Yay good for you.

A

B

Me yeah any any any. um You know uh good press you get internally in Microsoft just makes us.

A

All look better yeah exactly um so what was I gonna say: um Andy I saw your note in the chat window here, I'm, not sure if everybody else saw it, but um it looks like uh looks like people are working on the p4c, dpdk backend and as a lower price. Just to let everyone know, because we were going to see if we were going to compile in that so yep.

F

The compile's fine it, but but the next step you hope, is it loads into the dvda software switch, which it doesn't suck feeling the onion here. Yep.

A

F

uh The the people, the people that work on the open source, p4cd key back end um appreciate the interesting new test cases you're, providing them.

A

Good good fall, so does anyone else have anything or should we um end for the day.

A

G

Can't wait seven.

A

G

Ahead sorry, um I've been Marion Bush uh displaying the 358 PR I saw the eni attributes uh over there just curious if the eni table takes in the direction field. At this time, husky.

C

G

No definitely it doesn't so basically uh in terms of the term trying to determine you know the table, lookups Etc, which could be different for inbound and outbound apples.

G

Chronicles mainly.

C

um Well, the the attributes themselves were parameters, allow you to separately Define ACLS per in boundary and the ACLS for outbound.

G

C

um Acls are provided for both directions, separate.

G

Right right, yeah that will add more a little bit more burden on the ACLS um just trying to see if that can be alleviated using eni table. But we can discuss it next time if needed or I can bring you offline as well.

E

I mean SEO and ACR groups. All that stuff is SEO groups, SEO rules, that's totally independent of obni and directions right. Any.

G

Directions right, I,.

E

Mean they're, they don't care right now when you apply it, that's why you decide in the in Direction, but that's the only place the direction should be set on and I, don't think we should change that.

G

I I actually agree that I don't want to add a direction to eni um yeah. Just from the echo standpoint, if the icons could be different rules for in inbound and outbound for same uh connection, right.

E

But that'd be then a different rule within a different group.

G

E

You have flexibility there right.

G

H

Another way to put it is that for a given packet, you are applying either in Bound, ACL or outbound ACLs, and the metadata that you carry. You need to carry both inbound and outbound ACL group IDs.

H

So that's why given again I look up, do you need to assign both inbound and outbound ACL groups in DNA attributes, because at given time it's only one, that's going to be executing.

H

So I think that's a probably probably more of a question is about I. Don't know if I was clear.

G

Yeah I think if we have run out of time, then we can discuss this next time.

G

Question okay, I'm sure sure.

A

I will write that down.

A

Okay, then I will end the call today.

A

Thank you, everyone for coming. Thank you, Marion for the pr and I'll wait to hear from you on merging and doing all that. Thank you. Everyone for coming.

B