DASH Behavioral Model WG, 29 Jun 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DASH Behavioral Model WG weekly June 29 2023

Description

DASH Behavioral Model Workgroup call

A

On June 29th 2023 I'll go ahead and start recording and turn off the camera. Oh go ahead. Chris.

B

Yeah I'll take the screen for now, just because you can move things around and yep make it feel bigger. Yep.

B

um Can everyone see my screen now.

B

So um this this issue is pretty old but still relevant and it concerns underlay support in the dash behavioral model. You know routing and switching and routing, and until it contributed quite a number of test cases, PTF test cases that are skipped for the bnb2 they run on a real platform. My understanding is, they run on Evans because they were, you know.

C

B

For that platform, and um but they can't be really used by the Community model because model doesn't support those side apis, and these are the apis here, the Anton identified.

B

So we we have someone working on this to see if we can implement it and what it requires. A few things one is to write P4 code that implements this.

B

The other is to tie it into the fixed site apis and it's a bit of a departure, because currently anything that's in the P4 code generates PSI, headers and code to connect it all to libsi and the libsi itself, and what we, what we need is a way to use P4 tables to emulate these apis, but not to generate any new site headers, because the headers are already fixed in the PSI repo.

B

These aren't generated from Dash code they're fixed, so we're exploring this and I wanted to. You know mention that to the community and we're working on this one. First, as a first POC create route entry where the route entry, the next hop ID for the purpose of this experiment is just a bmv2 port number, zero or one- and the point of this is to do a very primitive routing table and to um prove out the way to get the lib side generated without messing with the side. Headers, that's basically the bottom line.

B

What we're trying to do now is an experiment, and if that bears fruit, we would look into Implement all of these and try to actually pass some of these test cases.

B

Also in discussions we've we've heard that for the Smart Switch project, I think the Sonic Engineers have said we don't need.

B

We don't need routing in the dash dpu, because the switch is just going to work with the dpu and hairpin mode, meaning any packet that goes in a port will come out that same port and we don't need to Route it so okay, I wanted to this is, and it's on the level of you know say: word of mouth I haven't seen an official Proclamation, so I wanted to ask what do people know about that?

B

What's the interest in Flushing out these is that still important for the dash project itself or just important for a fully implemented. Sonic switch on a dpu well.

D

So I have a question: yes, how about in The Appliance case right.

C

D

Exactly yeah so, where you, where we don't have a npu or switch Asic I understand you know, yeah! That's.

B

Why I'm raising the question is? Is the this is like on the level of comprehensive, say, informal conversations where we've been told when Microsoft said we don't need to and I I, don't remember the details. I don't want to get into the details, but we need to like explore this because I know people have put work into implementing these underlay. Is that going to be in vain is?

B

Is there going to be an appliance use case that we're actually going to carry through, or is everything moved over to Just Smart Switch, where this this becomes less important? So it's a rhetorical question kind.

C

D

B

Microsoft but there might.

D

E

One of them.

D

Yeah one other thing I want to float here after discussing with some people lately. Is that what happens is that you know there is a you know explicit allocation of a dpu to uh eni right. So what really happens is that you know um when uh packet arrives at the switch Asic.

D

um You know. The way smart switches are basically built is that you know in front of these tpus. So there are a bunch of dpus right. So there's not one dpu. So it's going to be like multiple dpu, so like let's say, six or eight dpus are behind a switch Asic. The packet arrives at the switch async and the switch async has to basically make a decision that, based on the eni I mean of course.

D

First, you have to drive an eni out of the contents of the back end, but then afterwards you have to say oh by the way you know now, I know the eni and now I have to basically decide to which dpu the cni belongs to, because there is an explicit mapping and I need to forward that one to the right dpu order of the six or eight dpos that are behind me.

D

That, of course, you know also is a decision that before that, perhaps you know in any of the underlying apis, if you call it underlay API, so to speak, for the switch AC to say that where do I should forward it right well for the processing. So again this food for thought, yeah.

B

A

B

The appliance mode- that's already being done right and so with the software really had to change much it.

C

Yeah those decisions on how you know there were some presentations in the smartness summit, where um how a dpu works and how it is controlled, whether it is inside the sheet, metal or outside you know, on its own, uh they uh what we learned that the management of That Remains, the Same so um yeah regardless. If for this topic that you want to implement these um API you you want to support this and add the code for it in BMV, too. I think it will be.

C

It will be very good, as uh you know, um and and then it can be utilized in any ways for sure.

F

Even if anything, there are a lot of test cases using writing, I mean I, guess resume. You probably you and your team probably wrote those anyway. So that's more coverage for the beer into itself for dash itself right right.

C

F

A fairly like you say um it's um previous it's! The writing here is very basic. You know it's just a it's, mostly an RPM lookup to get the header and the the Ethernet Mac and that's about it. There's no ecmp, there's no load balancing. None of that. Sorry right.

A

And I think what oh go ahead.

F

Sorry, there's just one thing that.

A

F

Will run into is that your data plane has to be somehow aware whether or not you do as a post processing you do writing or not, because the basic PDF test cases they don't do routing at all.

F

Oh so maybe if you don't find a rod, maybe you should say well I'm not doing anything I suppose, but.

B

In other words, the implicit thing is, if there's no routing, then just pair pinning off the same port by default, instead of dropping it right, it's.

F

More than the port right, it's also your ethernet header. That's going to be modified.

F

Thank you unique. Your next hub basically defines your destination Mac, but about it anyway. It's you know it's minor, I, suppose.

C

Right basic uh routing is still needed in in the you know like, even if it is without ecmb, and even if it is using ecmp, then you know uh it could be utilized in many different ways in different form factors other than the Smart, Switch, Appliance or otherwise connected to multiple uh tours- and you know, multiple other uplinks uh switches Etc but um yeah. If you're planning to implement this and support that in BMV too I think it's not a bad idea.

B

Okay, I, don't think we're gonna get into acmp, though, for sure that would be beyond the scope of anything.

G

B

But if we decide for basic test cases, it sounds to me like people are still in favor of doing this. If we can make progress.

A

B

A

And I think that you know Mukesh is on the call and um I don't have Prince or anyone on the call, but we do have an environment in Azure I'm, just saying for Azure, not for any other Cloud, but um we do have an environment called Canary where we could shift once this. You know things like this are done. We can shift over from you know what we're doing today over into Dash pretty easily. We could convert and try it um so I think I have a vested interest in doing it.

D

Yeah, so I I just want to go back to this question based on my comment.

D

I think you know so for Chris, I think just to I mean backward while the discussion uh was happening in where I guess you mentioned that who, wouldn't that be the same, if there was a appliance side versus the Smart Switch, so I would I would just you know say that in the appliance case you know, the dpu is accessible to the front front panel board and the packets are out into the dpu based on the routing right based on I, guess the looking looking at the outside header and the outer header- and then you know just through through uh through the routing from the previous hub- that's how the packets are covered.

D

But when it comes to Smart Switch, the packets are forwarded in a different way. So there is, there is a difference in terms of how our decision is made in terms of selecting which TPU to send it out to so there is a subtle difference. There I mean am I, am I reading something totally different here.

B

It sounds I'll go ahead, move cash, I mean we need an expert to answer that.

E

Yeah so, basically I think these apis refer to the packet that is going out of the dpu, not packets, coming into the dpu, because this just determines the routing entries on the dpu that determines how the packet transmitted out of the DQ, which Port does it go out and Yeah like that. Yes,.

D

E

D

E

D

The fact that okay, you know like, if, if you, if you have something that you need to really Implement on the Smart Switch as a as underlying API, but yeah you're right actually, the underlying apis makes more sense when it's for the outgoing traffic on the dpu, as opposed to the incoming part. But.

B

I will just you.

E

D

To emulate something in in totality, how would you do that right so.

E

So for the outgoing packet actually, uh as far as I know, there's not much difference between Smart Switch versus uh Appliance uh in either case. My understanding is that full Flex routing is not required, but having said that, we would still need to support some of these apis because we need to know which Mac address to send the packet to I. Think uh earlier, Winstone first mentioned that the next stop and the desk Mac that we need to Simply is still important.

E

So from that perspective, these apis would still be required. Only thing, probably, is that there wouldn't be like a bunch of routes, and then the forwarding happens based on one of those routes. None of those things might happen. This is probably going to be one single route, uh but it would still point to some next stop and then next up, we'll say which Mac address and so on. So no path so would still be valid.

F

Well, I mean writing to know your header is none it's optional for dash, because there is already an appliance configuration that actually gives you that header to start with.

E

uh So that plans I'm not sure if the appliance is actually part of the ocp style, yet I think that Appliance table was just created for the temporarily, for bmv2 is my understanding. I, don't think it exists in the real ocp side.

F

I, don't have the history on that I don't know, but if you have more details there, but maybe we need some clarification. What should be.

E

Yeah, my understanding is that the appliance was just created as a shortcut uh just to get things working here in the dash report between the actual uh PSI production environment. There's not going to be an application, but the fact.

F

Is if you have only one rod, there's really no need to do a lookup per se, an RPM lookup on.

A

The inside going inside so it's.

G

E

F

E

F

Api Define a rod lookup right, which is very heavy, probably is not something you want to do. If it's only one right, then pretty much, your header is fixed right, yeah.

E

F

E

You need somewhere to determine the market, so you could have like have a fixed next up or something in your forwarding path. You don't need an LPN lookup. Would you still need some uh lookup to give you the MAC address.

C

Yeah the table look up, even if it is there, it can utilize fewer entries as needed, uh but that's a standard method of lookup and you know implementing um uh updating next top and all those things so even in the future.

C

If the appliance is one from Factor, Smart Switch is one form factor if the dpu is used by itself, Standalone in any case uh it it, it would be useful in the cloud environment uh you know uh outside um different other use, cases like Enterprise or whatever it may be, and it's a standard way of implementing rather than a hand modifying the destination Mac address yeah. It shouldn't consume too much of Hardware resource because there'll be fewer entries and that's fine just touch it standard. Yeah.

F

But it's not really about Hardware resource. It's about your data plan performance right! It's it's yet another thing to do that you may, if you don't need to do you, don't need to do right.

C

F

C

F

It's only one right: you can optimize for that, as opposed to doing generic lookup sure, but if it's a step that is not needed, it's not needed.

B

Well, I think that was the context of of this. You know I'll call rumor, it's not really a rumor, but off the Record discussion is: is that there's a performance penalty and a memory penalty for doing the general lookup? And it's it's really not needed?

B

Why and post that on the data plane, so so I guess to re-ask the question: if that's really the direction that things are headed and vendors, don't need to implement the the full Sonic underlay like we had been laboring to believe and everyone's going to get kind of a free pass on that, because it's not needed and it will prove the memory and performance footprint. Then how much should we spend on making these test cases if they're going to become not really crucial?.

F

I guess the question is we need a header? So do we reuse this with the Assumption? We do we reuse those known apis, but we know there is only one. So we just it's a bunch of calls, but at the end of the day you have one set of data, or do we make another one like the one? We basically have temporary right now, I, don't know.

B

Oh so we'll um this is a good discussion, just sort of as a reality check we'll continue to proceed. It is an internship project and you know some limited Direct, that's kind of EX. You know experimental project for us, but we'll try to make some progress on and see how things go. If nothing else we're going to um you know we're going to come up with a few techniques for how to accommodate side fixed headers into the dash libside generator. That kind of thing so it'll produce some useful outputs. It already has I.

A

B

It's spawning some good conversations here where we're revisiting this. You know almost a year, a year later, almost yeah.

A

And I can think of two cases in Azure where I could use this. You know just if I have two different um implementations inside Canary at least.

B

Well, now we're talking about BMV too, so um when you're saying you could use it in Canary, do you mean on a real dpu?

B

Okay? So from that point of view, you would want production gpus to implement full underlay anyway. Well.

A

The canary is not exactly production, it's kind of in the middle.

B

Right but it behaves as if it were a production device. So that's probably something discussion needs to happen with vendors, one right expectation of software completeness: okay, yeah. Well. This is useful.

A

It's really useful, thank you and.

B

I I feedback from.

A

Everyone yeah, yeah and I'll talk to um riff when I see him and talk to him about this.

B

A

So cool, um thank you yeah! No, thank you. So did the rest of you guys have anything else for today.

A

So um I know that I'm just pursuing my um what's. It called my pursuit of P4 knowledge, so I'm just trying to find at least an hour a week to um learn something and figure it out myself. So that's what I've been doing.

D

A

D

A quick question before you probably wrap up you.

C

D

uh Just a quick question on a comment that was made at the smartening summit.

C

D

Believe by Gerald.

C

D

That is, um it was said that uh the dash is going GA in Microsoft. You know in a month or so right.

A

I, don't think Dash is going GA I think he said that um the um serious project.

D

A

Going GA right is, am I splitting hairs here um so James's project with the serious Appliance 48 42 Ru rack, with the two servers and the dpus slotted into it. That is supposed to go. Ga in July.

D

Okay dates, move and.

A

And I'm not supposed to use that name externally. It's an internal project name so don't get mad, but um but Mark rosinovich used it in um ignite last September, and so so what was your question? I'm sorry I didn't want to talk over you.

D

Oh no I said that so this, this uh CBS dpu, sorry CS Appliance, has I believe eight dpus in it is it I think it's six six okay and uh um the software that it is running is perhaps.

A

Yeah but yeah being ported over two dash but pre-dash there's a lot of balls in the air.

B

That was formerly known as Sirius. You know.

A

Yeah formerly known as this year, but yeah keep going. uh What was the question.

A

Is there more questions.

D

What does what does pre-dash mean.

A

Christina, no, no, it I think what they did. This is before I joined the project, so I was brought in um probably a year into what they were doing so Gerald was working with James and uh a team before I joined and they thought well to do pre, pre and early development and make it go fast fast fast. They would go ahead and take something we had existing. We should be Rotterdam, modify it a bit and move forward with development on that to move towards Sirius. So that's that idea and then um so that's James!

A

That's do you know James Grantham, you guys I, think he's joined the call a couple times: okay, yeah and then, and then they brought me in sorry. I'm checking the chat here. um So then they brought me in as well and then I was to once serious goes GA. Then I move forward and do Dash Smart Switch. So um so it was not exactly Rotterdam, but it is they. They used a baseline of Rotterdam to speed up development of Sirius and move toward the Smart Switch.

A

If that makes sense, yep yep, thanks yep and so James works with a lot of different internal teams to get the software um to where he needs it to be so he can execute and go ga in July.

A

So if that's helpful, it looks like um yeah, it looks like riff joined, High riff. We were just about wrapping up.

A

That's: okay, that's okay! So everyone knows riffs from the Sonic team and um you know if everyone, this is everyone else, and in this meeting we talk about the behavioral model and development of the behavioral model and um I can share my screen on this side, just to show you, but um we actually have a GitHub project here.

A

Github allows you to make projects reference, so you can, you know, put action items and things you're working on in a behavioral model so that we can, you know, run tests as if we had Hardware things like that- and we were talking about this one in particular the underly PSI API for bmv2. So that's what we were talking about today before you join the call and trying to decide if we still need what was it Chris that um outbound routing yeah.

B

We still need to pull underlay outbound routing. Yes, next.

E

B

A

Yeah, it's been a while, since we had Revisited this one, so we were trying to determine as we move from the appliance towards the Smart Switch, because you know when you have the Smart Switch you you don't hair, pinning the traffic um we're trying to and we came to a conclusion. We think we need it Chris. Maybe you can yeah.

B

We still need to look up Mac address Etc, so there needs to be some. The lookups may be very small, may not be a giant LPM table like a generalized routing, but we still need some way to configure some minimum number of Entry. So plan of record is just to use the existing PSI underlay apis. Even if they're you know not fully exploited unless some other proposal comes up from this.

G

A

See so yeah, so we were just trying to think how, because the ports are different on the appliance versus the Smart Switch, how different would it be, and so, if you have any context on that or can think about it for next time, that'd be awesome.

G

I thought people had some like, uh you can specify the port, but maybe not that's another thing.

B

I'm, not quite sure, I I heard you very well riffs. Could you repeat that oh.

G

uh I saw P4 uh e said before we can actually specify like which product it goes to, uh but yeah it may be like different uh devices, um so it turns out to be different, but well.

B

The issue is not how it's implemented other than for the behavior model. Yes we're using P4, but um the question is: what are the site apis support that we need to configure this? This even minimal routing. You know our standard site apis and those are listed in pr236. You might want to take a look at that because you know yep.

G

Sounds good! Thank you. Yeah.

A

I'll put the link here when I send out notes, but all right guys um did anyone have anything else or should we adjourn for the day?

A

I'm gonna do Andy's seven second rule: where did Andy go.

B

And uh speaking of Andy, if you're learning P4 and these peaceful.

F

B

Because of people are guided to get repo, that's great.

A

Yeah yeah I've been looking through it. Okay. Well, you guys I'm gonna, stop the recording and.