DASH Community Weekly, 12 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SONiC DASH Workgroup Community Meeting Apr 12 2023

Description

PR289 - Still needs volunteer - Dockerfile.saithrift-client has a static tag in the FROM instruction

PR350 - Add protobuf description for dash app_db (NorthBound)
8ccc883 Add excel for AppDB memory estimation - ask for Excel walkthrough? @chris to check the Excel sheet
14957c0 Merge branch 'main' into protobuf_table

PR358 - Support SAi attr generation from annotattions
Please add a description paragraph that can cover what is this change and why it is needed, this helps during review of the PRs.

PR336 - [ACL]: Optimize DASH ACL by introducing cluster tag
Merged #336 into main

The intention is… if the attribute is not provided, it is like any other ACL rule. If not provided, wildcard match. Match on ANY tag or NO tag (reword)?
@Pterosaur will change it in PR: #359

PR346 - pending check back ~
Looks like there were a lot of changes into main/SAI that changed PTF stuff, need some more time to fix integration with latest SAI master.
Change broke automated framework for flexible # of ports

A

Meeting this week is the 64th one so kind of a lot um on the left side. What I'm showing is what we talked about last week, where we had the protobuf discussion and the optimization for protobuf and I sent these these notes in these Little Snips of the presentation by Patrick, star or Z last week, and so it was a good hour-long conversation and it looks like you know.

A

Over the last seven days, there's been uh this one added here where added an Excel sheet for memory estimation, and um then there was a merge, so that was number I think it was three fifty so I wasn't sure if you guys saw all of that information happen in this PR.

A

But all that stuff happened in the last few days he did say that a diagram would be a whole nother thing and um they're still investigating the data flow.

A

So that's where we are with this one and it looks like we have some spell check that needs to be fixed. This is seat on he doesn't look like he's on. We can always help him with that.

A

So it looks like he marked it ready for review an hour ago, so that would be from I guess Chris a prince. Do you need to review this? Is this? Do he and you work together or is it murian.

B

No I think um I I have already discussed separately with the so so I'm aware of these changes. So.

A

B

It is better the community can review.

A

B

Right, yeah I think this is uh okay, it's mainly on the Northbound side right, so there's no switch to the sub pumps.

A

A

To anyone else have questions or comments about this one.

C

Well, it looks like looks like there's a merge conflict with the another document probably needs to be resolved right just due to um parallel efforts.

C

Okay message at the bottom means two people were working on that file. At the same time, and and now this this PR needs to catch up to it.

A

Does he just need to rebate or.

C

A

C

Would say probably needs to be looked at carefully by the author, so that the resulting merge is what he wants right when you have two documents, especially document files like this click resolve conflicts, you might get what you want. You.

A

Might not oh yeah I have had that happen to me it doesn't work for me. Yes,.

C

It's uh roulette wheel, so yeah, that's just a minor thing that needs to be done. Mm-Hmm, the Excel I would ask, is the Excel file? Is that just basically the the live spreadsheet that you've done? Screenshots of in um you know in the meeting or is there? Is there any additional content in that spreadsheet that we want to get walked through.

A

We could ask him to do that when to manage the call here is he on I don't see him on, but we could ask him to do that. So I can make a note.

B

uh On what Doug I can follow up by how um what's that? uh What's.

A

The ask I I can accept a walkthrough of the Excel if we need it.

C

If there's anything in there that we want to walk through since it's not a text file that we can just stare at in this meeting, sometimes it's nice to have someone walk us through if.

D

There's anything we.

C

Should know if not that's fine, just uh normal thing.

A

um Chris, do you want to go, take a peek at it and see if you can find it based on the conversation and see if we need to walk through yeah I mean I, can download it and try okay, cool or somebody? Okay, okay, thank you.

A

So we did that one um and then reshma had just made a comment on 358..

A

um Maybe we could add a description. Paragraph I, don't know if that.

E

A

E

um Because you know, I had a generic suggestion as well for all the PRS, uh if you can add a paragraph of uh what the changes and why it is needed, a small paragraph or Pierce, because here we don't have hlds, we are normally in Sonic. We we uh present the hld and everybody contributes uh in suggests the changes we go directly into the PRS without too much context. It will be difficult. Some description will help on the top mm-hmm.

A

Okay got it and then it looks like we went ahead and merged 336., and so this one they're actually quite a few notes inside of it. I, don't know if everybody saw the notes, so I'll just put it in front of us, so we can all see it and since it's merged I'm not going to be able to find it.

C

You can click on closed.

A

Yeah I know, but oh it's easier. Sorry, it's just easier to hear it for me anyway, go to the end.

A

And so this one.

A

So Marion helped out here provided some info, and then we had um a little bit a little bit more information this last week, where um you know, after considering it might be better to generate separate apis for sourced and desk tags.

A

That's the implementation to keep two tables, so that was kind of the last thing that happened in the last couple of days, and hopefully Z saw that and then Marian approved the changes and Prince merged.

A

A

Any comments on that before we move on for today.

F

Yeah there was a slight discrepancy between the the tag has prefix, which is good. Basically, it's a list of prefix perfect, the previous or the this one yeah the generation at at least I, don't think things I, don't know that things have will change the generation for the all. The way is still IP, as opposed to prefix.

F

um Marion is that something you could change.

F

I, don't know that they made it part of his change, though I haven't checked out.

A

And I did see that I forgot to ask about it. Oh sorry, Marion.

G

Which key are we talking about.

F

So if you take the ACL rule um before the ACL tag, you have um IP destination and IP Source right. Those are not prefixes they're, just straight IPS, which there should be prefixes to start with, but they never were moved to prefixes. Now the tag with the tag we basically get a little prefix. So that's good, but I wonder if we could also correct the previous definition, but the non when we don't use Tag to still to have prefixes as well.

G

I think they are prefixes I will I will check that they're supposed to be great well.

F

I know they're supposed to, but currently they are not right. So I don't know if they changed that in espr or not, but they clearly were not prefixes I mean I'm, showing that here right, I mean Christina is showing that here.

F

If you could take a look and it'd be nice to have prefixes there to make it and.

A

F

This great assistant here right.

B

G

Let's change the place they used at least they used to be a list of prefix.

F

No then they were, they were never prefixes they were supposed to, but they were never prefixes.

G

The latest version of Psy apis has.

G

Sip and dip as prefixes. If you look at this Eye Master, which was generated from P4 code, so it was, it had to to be changed at some point.

F

So I'll verify that because I rebased uh a couple weeks back and it was still prefixes there, so I'll take back.

G

Did you enable the dash match defined by the way which is used for site API? There is a difference between um apis that are used for Hardware side, implementations and the API that was used for bmb2 for some time, and there was a defined separate, those two and it uh it defines different uh match types for uh set and.

F

So that could be because I'm not sure I'm aware of that is that documented somewhere.

G

No, there is just a defining ACL that before you need to enable.

F

Okay, so by default it's not enabled.

G

By default, it's not.

F

Well, that would explain then: okay, that's about I mean if it's a hidden, compile it's going to be a problem for people, because this IPI then is not going to be consistent in Cross vendors right I mean I, don't understand, it may be a limitation or maybe the bw2 model, but I think that's asking for confusion here.

G

Well, we had to be able to compile bmv2 uh I I think we over. We have overcome this issue. I'll need to check. Actually uh the was the one that was supposed to align those two. He was working on that um so I can check with him and we can get rid of testify.

F

I'll be good all right in the meantime, I will check the Define to see if I can move to prefix like transparently thanks.

A

A

Thank you and the last that I had was um Vladimir, had sit there for a lot of for PR 346. There were a lot of changes um in main side change packet test framework and uh we needed some more time. This is a few days ago and so freshman or as a lot of me here, I, don't see him it's probably late where he is so uh reshma did you know anything about this? One.

A

Or uh or anyone else.

C

There were some lingering bugs that he was you know. Test case was having issues I think he has a comment there yeah, he seems to.

A

So he just needs more time: yeah. Okay, great those are the updates to the open ones that I know about for the week and then I hadn't really had a chance to look at new. So um did anyone else have PR's? They wanted to go over from this list.

C

The Mario and that one there that top one Mario and that's just a spell check right failure. You just need to put back ticks around that keyword, I think.

G

Yeah Chris, could you please explain how I uh the key, how I can add a custom keyword that uh won't fail? The spell check anymore.

C

Yeah just go to dot wordlist.txt, it's it's! You know, I! Think it's in.

A

The beginning here right, it's word list.

H

A

Yeah, where is it Chris? It's right here.

C

Dot Wordly, yeah and.

A

Then I think you add it in down here. Just add it in the right spot. Yeah.

C

A

And um I'll, let you explain.

G

It's in the project root director, yes,.

A

Yeah yeah, that's.

G

A

It whenever I do it, that's where I of.

C

Course, I'm not great at it. If you, if you put back ticks around like you know a code snippet or something it won't spell check that I believe so changes the style of the document. But you know in your in your table. You add in your readme your your list of keywords. You could put back ticks around those two. That's another way and I'll ignore them.

G

uh Yeah I understand that I think it'll still add it to the uh word list. Yeah.

C

Pretty soon the word list will be so long. We can feed it into chat. Dtp it'll be its own, intelligent thing.

A

Yeah I was having problems because I kept editing this main one and then expecting it to be in my branch and I still kept coming up with deltas and it was driving me insane uh and then finally figured it out. It's.

C

All about context right.

A

Yeah so anyway, yeah it's right there in the root um anything else from the group here today. So we have Alberto, we've got Canon Virginia.

A

Okay, so anything else from the rest, you guys this is really all I had for today and.

H

I know it's spring break week: yeah Christina one one point I had that on PR 336. There are some questions. There were some comments uh and I'm hoping you know. If we, if we can get some clarification on those things.

E

Did you now just discussed that earlier, um okay.

A

Were there more comments other than the ones we just covered.

H

Yeah this one, for example, you just passed by.

H

The swisha Intel, um if you go yeah this one.

A

Oh here, okay, could you please have a scenario where, uh like LPM match here,.

E

You want to describe it yeah.

A

Maybe describe.

H

It yeah um so I think what we are seeing is that uh um currently the way uh the P4 definition for this is that there is LPM match on the destination prefix or Source prefix, and assign a tag right, so that match as an assumption of one prefix matching and one tag assignment so question was that if you can have a use case where there could be overlapping prefixes and each of them matching to a different tag, how to handle that scenario.

A

Okay, so I myself can't answer that, but um anyone else on the call.

H

So the current current definition of table, at least or maybe it's understanding from my side, that I don't see, uh matching overlapping, prefixes and possibly producing result for multiple tags and I. Think Prince confirmed last time right that the the use case, the scenario is valid, uh that this can very well happen. Yeah.

B

Yeah, that's what I.

D

Are these tags purely control playing entities? Are they expected to be represented in the data plane in any way.

H

So from my understanding, it's representative data plane as well right.

B

Can depend on the the implementations like how they want to utilize this tag, but uh the idea is like it will be. uh It will be pushed to the southbound to the to the venture, implementations.

D

But but is the take, is the tag effectively just a name for a list of source IP or desktop prefixes? And that's it.

B

Yeah and then that tag is used in different, like you know, actual tables.

D

So it's just in their in Direction mechanisms right for the control for the control plane to use to refer to the same list of source, AP prefixes multiple times without repeating it. That's right.

B

H

Right, so maybe that is, that is a good clarification that this this represents only control, plane and then princess is your response that vendor can handle.

E

It is not only control plane right. The tag is going to be pushed down into the data plane as well, but just as the previous conversation was that it is used as a name uh to represent some other suggest addresses, but it is still a data plane. I.

D

I I, wouldn't I, would clarify it. It sounds like whether or not this tag is ever represented in the fast path. Is a implementation choice and need not be in the fast path at all. It sounds like.

E

If it is not represented in the data plane, then we will exhaust the number of prefixes all.

D

E

D

That's up to implementers to get right or wrong. That's not the concern of the control plane. Apis.

E

I mean that is you useful in in that direction, so.

B

It also um like we. We definitely want to optimize from the data plane side as well.

D

B

uh Because you, if, if it's not optimized and if it uses the the most part of the memory right by repeating this utilization usage, then it is indeed a concern to the control plane as well. So.

D

Here's a clarifying here's, a question that would clarify it for me, then suppose you've got a single Dash axle one stage and a list of source IP prefixes is given it. You know, there's a thousand Source IP prefix list, that's given the tag 57. and then there's a hundred rules that have that 57 tag in the source. Ip. That's that's allowed.

D

Okay, so that means that list of thousand sources is in a Thousand Rules. By that interaction of fusing 57 is the now the data plane. There could be another source, IP prefix list, with tag 58 that has some of the same prefixes in it right, but not all the same, not the identical list. So at data plane time a source type, a single packet could have a source IP prefixes that should get tagged, 57 and 58 logically or it matches 57.58 right.

D

Single Source, IB prefix could be in tagless 57 and tagless 58 and.

H

D

So that could be an arbitrary number of tags that match this packet. Source IP prefix.

D

As many as there are tags, potentially true.

B

So far, so good yeah.

D

So a data plane a fast path, that's tried to say oh I'm, going to identify the full list of tags for this packet. Source IP prefix could come up with a thousand tags or ten thousand tags that match that Source packet, Source IP, address.

B

I mean we have the the scale limits in this PR.

D

B

But potentially, okay yeah. It is possible that the prefix can have multiple tags.

F

On the bottom line is that when you do a lookup you can have, you can end up with different tags anyway and there's no priority between tags. It's not rated at all.

F

So I don't know how you're going to make your ACL rule match. But that's just me.

F

In the airplane.

G

D

I mean I forgot when I got caught up if a data plane chooses to try to enumerate as an intermediate step of packet processing. Oh this packet, Source IP, address matches these n Source tags. That's an implementation choice, but an implementation need not do that and probably shouldn't if it's going to go fast.

D

But that's that's like implementation detail. As far as Dash is concerned,.

G

B

I think that what I agree like if, if the implementation doesn't utilize, this uh tag, then yeah it's up to.

D

E

What Prince has added is something that will help you to you know help with the implementation as well so I, don't see. I mean I missed some of the conversation, but uh I. Don't know why we should not use this and it is applicable to the data thing.

D

The tag will be in the control plane. Whether it's in the data plane isn't is an implementer's choice.

E

That is a choice, but we would prefer to implement it in the data plane too, if possible, because.

D

I I question that assumption, but there are implementations that could use it. There's implementations that would.

E

It would be no help, of course, got it um sure we can discuss that, whether we can use it um uh not, uh but uh here the overlapping prefixes will end up getting the another tag right.

D

A single packet could have a source IP address that matches a thousand tags or two yeah or.

F

Nothing a tag is just a list of prefix yep tag, lists have I, mean tag, one tag two tag, and they have no relationship whatsoever. They could.

D

Overlap arbitrarily in their previous absolutely.

F

Yep as a matter, especially because they're prefix so I mean you could have a tag that just is a catch-all, a slash, zero.

D

Yep, oh I, I I've, been pushing and talking to implementers about this I have been stressing these facts repeatedly, so you're just preaching to my choir. That's what makes the problem difficult in the data plane.

F

I mean the API is fixable enough for you to do whatever you want. I mean it's control plane. Has that data, whether or not you want to use it and they are playing it's up to you, implementation so.

D

Yeah, but but any any fast path, implementation that assumes that as packet Source IP will match, only one tag is wrong.

B

Now, that's absolutely that's definitely wrong, but I think here the the idea is like if we give so in this example. Also like prefix ABC belongs to tag. One and CDE belongs to tag two and now C is a prefix. C has both tag 1 and tag 2 right. So when a packet comes with C that, then, if the implementation can um use some big bit vector or something like that, to specify that it is stack one and tag two and then the actual rules can be matched against that yeah big map right.

B

So all those things can be right, but.

D

But any implementation that uses a bit Vector for that needs to know how many tags there can be and allow the bit Vector to be long enough to handle.

B

Right and that's why we gave the scale numbers right and.

D

Yeah yeah, but that's a big bit Vector right. It's hundreds of bits.

B

I'm just telling like how it can the data plane can also potentially utilize this to optimize their implementations right instead of yeah.

D

But it's but it's a data plane Choice. It needs to do the correct thing and.

B

Again like, of course, it's a data plan choice, but the problem is when we have like these kinds of scaled actual rules and if the data plane implementation is not optimal, and we see that okay, it's taking you know, 90 of the memory for just this axle. That's not right. That's where control.

D

Plane there are various. There are many trade-offs in implementing the data plane that implements these rule matching correctly and there's not just one way to do it of.

B

Course, of course, so as long as the implementation can confirm that it's uh it's optimized and.

D

That there's many correct ways and there's a thousand times more wrong ways to do it. Thank you.

F

By the under the vendor, controls to its own control, plane anyway below the side API and it controls its data plane, so the vendor will do whatever he can yep.

F

It's too bad, though, that we don't have a capability database overall I mean it's not the first time that you know, I think you can probably database would be nice.

B

Yeah but I think this is more of a.

A

A

Or what database is it.

F

Capability database.

A

Oh capability duties, okay, we talked about that.

A

B

Were you saying Prince no I'm saying like this is uh more of an optimization choice. Right like this yeah.

H

I mean this sounds good I think I got the clarification. I really wanted. Thank.

A

You so this is there another one you had in here.

H

The other one uh is more of probably just uh wordings: okay,.

E

H

Yeah I think you just passed the other one. What.

A

Does it mean by match on all Source tag or no tag.

H

I think this was clarified that uh it's not like matching all Source tags, but it's matching any Source tag or no tag. So it's just.

A

Yeah we got that uh-huh, so hopefully someone will reward that for us, okay and I'll I'll make a note. I can I I can always put a suggestion in the pr.

A

And then was there something else, I'm, sorry that.

H

This is yeah. This is all I had that.

A

Was it okay, all right, um yeah, no problem and then I noticed I see a lot of mirror here. um Vladimir. Did you want to chat about 346 or are we done.

I

uh Yeah correctly supplements.

A

Yeah yeah I'll open it here, I just gotta, get to it.

A

A

That's not right and.

I

Wait yeah, so this this pull request um still need more changes because, like a lot of changes has been done into the side repo into the side. Pt of these cases recently so need some stuff needs to be resolved on the side side report first and then we can come back to this pull request, so we can probably just pull request into the pending States or something like.

E

This, oh I, think so Baltimore uh is this: uh is this pending on the fix uh that's yet to be made inside, which brought the dash video.

I

Yeah yeah, mainly this one.

E

Oh okay, okay,.

A

End of April or something go out of here.

E

There was a change that went into PSI that sort of number of the uh Auto generated framework changes for flexible number of ports. We are trying to see how to fix that. I think that actually.

A

So should we wait like till the end of the month and check back on this one, maybe Vladimir sure.

I

Yeah, like this.

E

Okay, it will try to prioritize fixing that data.

A

Okay, all right, and uh so that's that's what I have for today, unless you guys have anything else.

A

I, don't know if it's spring break where you guys are.

E

For me it's next week, oh it's next week, but a lot of folks have it this week, yeah.

A

I, don't know who has it this I, don't have kids, so I, don't really keep track of that, and so, if there's any time where there's family or other commitments, just let me know and I can shift the dash schedule around. So just let me know thank you.

A

Thank you. Well then, if we're finished, um I can go ahead and end. The call um we we do have I do have a contact who wants who has some ideas about, ha and so we may have an aha work group either this coming Tuesday or the one after that. Just FYI to you guys here on the call who are also involved in ha no promises. Quite yet, but I do have a contributor who's thinking about contributing. You know their ideas, okay! Well, that's all I have thank you for coming.

A

I'll send out the notes and everything like normal.

C

Thanks thanks Christina all.

A

C

A

A good day, okay, thank you.