DASH Community Weekly, 15 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DASH Workgroup Community Meeting 20220615 (June 15, 2022)

Description

June 15, 2022
Presentation of SONiC HLD

A

Today we have prince online to talk about the sonic hld and the yang so prince. Are you ready to.

B

C

Sharing my screen.

C

All right, okay! So, let's get started um today, we will discuss about the hld for sonic dash, um basically bringing up sonic um on an appliance or a smart switch.

C

So this is covering the overall scenarios and the schema definitions we will go over um in detail or on the hld, so yeah feel free to uh stop me anytime. If you have questions- and I have to thank those who have contributed to the hld, as you can see here- different folks have contributed to hld. So I will probably reach to that section and ask them to explain on that.

C

Okay, so, on the requirements part, we have divided it into two phases. So first one is the we need to win it scenario and route and lpm support under ipv4 and v6 state, volatile, tcp, state tracking, on flows and telemetry and monitoring. We are not considering in this design, at least for now the private link service tunnel and overlay ipv6.

C

So that's on the requirement side. These are some details like on the device metadata and swss how to bring up the device in in an appliance or dpu mode. So, typically, as most of you know, the sonic currently has different subtypes like door router leaf router, which is all defined for the switch or networking devices. But since this is something new, we have come up with these conventions yeah.

C

So any questions on the requirements.

C

All right so coming down to the cli so um since most of the configurations to sonic, dy device, configurations etc will be all driven by the controller. We're not planning to provide lots of config options to cli, so mainly um targeting for a support, show and clear commands. So this is also some feedback. We got from partner teams um so basically for the dash show, commands and clear commands for some kind of debugging operations. That's the plan currently.

C

Warm restart is an another area that we are not particularly considering for the phase one so, okay, coming down to the scale this is, we have other documents that james, christina and all put together in the other section. So this is just a recap of what are the scaling that we are targeting to support so number of v nets. um Enis are like 64 per card, and these routes echoes very nice captured here. The cipa mappings is like 10 million and total active connections that we are looking for is like 1 million in bi-directional.

C

So these are the scaling numbers for reference here. So that way, our design um has to consider these kinds of uh big scaling numbers.

C

So one of the thing that uh before we go into the design considerations, so I put forth uh these um this section mainly to uh to uh to capture that uh the regular sonic design is slightly different from this dash project, because we are looking at the high performance high scaling use cases, so the current sonic is has to be like you know, some restructuring or some redesign is required for for handling such kinds of scales and performance. So that's why these uh design considerations are good, and this is very important.

C

One like this is very critical for our design. um So let's go over one by one and uh and I think we can- we can spend some time um parsing this yeah.

C

So um first of all, the implementation must support like single and bulk uh update of lpm and see a mapping table so basically like we are looking at um routes and 10 million mappings. So of course, one programming it one by one is definitely time consuming so we'll plan to support bulk update in all layers like gnmi, um the the side layer, the orchestra layer at the establishes layer. So all those sections we are targeting for bulk updates um so mainly like in the startup.

C

It is possible to have scaled configurations right like during that appliance coming up. Then we can have these kinds of uh scale configurations applying um like you know in very short period of time. So once the system is stable, we expect mapping updates, like uh like the cipa mapping, updates uh for a v-net to occur like a hundred mappings per second and a normal operation. Route updates, like routes, are 100k per ni.

C

So this can happen like every 30 second, so these kinds of um constant updates and shown care should be expected and the design should be able to handle that um now. Another thing is like if an adding or a deletion of vm by the customer can have update to the eni and routes in in such bulk context. Right. So when a new vm is created these many routes and if it is an associated to a new v-net, then you can expect to have a lot of mappings also created.

C

So that's on the add-on delete of vm and uh the other key point is about the actual operations right. So uh the rules um adding and deleting per group for a stage the expectation is to handle it atomically because we should not have a transient drop or forward cases, because if, let's say the rules are applied, one by one, there would be like some transient period where the packet forwarding or drop is not expected according to the expectations yeah.

C

So so for a particular group. We plan to have uh the installation of that uh echo rules um to be handled. Atomically, yeah.

D

Any questions I see chris has faced.

E

D

Yeah um thanks prince. Do you as far as scale like these bulk updates, these large large vault updates? Have you done any? Have you done any experiments like even with a virtual side, back end to to look at the performance of just sonic by itself, without even with respect to the uh device physical device.

C

C

So, as you know like uh maybe once we come down, I will explain how the current flow is and what we are trying to improve. So basically, there is a um there is a research ongoing with how uh how much of db updates can happen um in in like in one apa call, um so we I don't, have the current numbers that's being worked upon, so we will be updating this actually uh based on our findings. Yeah.

D

Yeah, I think it'd be good just to have a reality.

C

D

Check well, we can do a hundred thousand updates on this size of processor and this amount of memory and it runs reasonably well or it runs, slows a turtle and it can't possibly work this way right good to have a early warning. If there's something way out of bounds here, sure thanks.

C

Okay, now coming down the the implementations must support, also the ability to get all the active rules and groups based on some group id. So that's based on the schema definition we can. uh We can go over that.

C

Then, um okay, so the other thing. Okay, this is just for a reference like in the normal operations. Mapping can change often and then followed by routes and then leased four apples because actually is like customer driven. So if there is a change, then only get it can uh get propagated or a new set of accounts will be applied, but otherwise uh this is the order of uh of churn that we expect.

C

uh The next is like ian isa. So uh there's one point which we have to consider is during an eni creation like right, there will be like a mappings associated routes associated hackles, so we cannot have an eni to be programmed in a partial way right. So uh so the the the intention here is to have uh the controller, apply all the configurations of the dna and then later bring the eni uh uh like administratively up. So it's like the port uh up and down uh kind of uh scenario.

C

So the eni, if it is down uh the packets designed to this email, shall be dropped so once it is up, only the the packet forwarding happens. So here um one thing to note is the order of operation right. The configurations of mappings or radicals, so these are all controlled by the controller like sonic- will not be enforcing any order of operation.

C

So what it's just to honor the uh the order that is set forth by controller and then once the administrative state is set to up by controller, then uh sonic will just enable the eni yeah.

C

uh Then um here, like eni and uh or wiener delete, um we are expecting or we are planning to support a mapping. Deletion of all mappings and routes in single apa calls, instead of like again um doing bulk updates or single uh mapping delete. If, if you say like delete map all then um we can internally go ahead and delete all the mappings associated to that.

C

uh At this point, uh this poi, uh the ad and daily tps, are item potential. According to the gmail standard, however, our team is also analyzing. What is our uh implementation so so, for example, deleting an object. uh What what item button here, I refer is deleting an object. Doesn't that doesn't exist exist shall not return an error, so it will be just accepting the api and then returning success.

C

um Now this is uh spur the gnmi spec, so we will also be validating that and- and if there are any um other observations, we'll I'll rephrase this yeah uh during a delete operation. If there are dependencies that that's one case right like if vnet is planned to be deleted and if mappings are still present, then the implementation shall return error and not perform any force deletions or deleting the dependencies implicitly then, during bulk operation, if any part or subset of apa fails.

C

The current implementation of gmail shall return error for the entire api, so there will not be any partial um updates happening in in forever for a single ap call yeah, then implementation must have flexible memory allocation for eni and not reserve max scale during initial create.

C

So this is uh some um some feedback from our internal discussion, so basically uh the the card or the the plans when we create an eni, it should not uh freehand a reserve like these 100k routes or such things so that, like for small vns, we may not scale up to that. So this is to allow some sort of our subscription.

C

um Then. Finally, the implementation must not have silent failures for apis like if, if an ap is accepted by the from the controller, then then and and returning success, it is not expected to fail in the back end and and um have um inconsistency between the controller and the sonic um implementation. So these are the main considerations that that are that are considered for this design.

C

So any questions or.

F

Yeah, I have a. I have a couple of questions. uh Friends say. Thank you so much. I think this is good when you say there shouldn't be any silent failures right, but at the same point you were saying that, okay, if, if you know in the delete case, if I, if I read it correctly or understood correctly, if you um you, should accept it and not really return an error right. Even though.

C

Something doesn't exist right, so that's what so uh here it it is not causing an inconsistency between the controller and the device right. The the controller has uh asked the sonic implementation to delete some something and which is a which is a anyways not present in um in in the in the plan. So the point is like here.

C

What I meant is, if let's say a v-net is uh supposed to be created or deleted and and we accept the ap, but the v-net still exists in the in the device or the the ena still access that that's not uh expected so, for example, same way with the ad like. If the object already pres exist in the device and if controller is doing any duplicate ad, that's fine still right, because uh the states are in sync between the two sure. Maybe I can rephrase uh if it is not.

F

Yeah, I think, yeah. I feel if you can rephrase that item potentially you know I think everybody understands, but if you try to do multiple ads, then you know like okay, it's just basically that were to be happen right, but I think, if you um you know, sometimes the deletion part is that you're trying to delete something and then it doesn't exist. Should you even return an error right so and then, if you, if you're saying you know, you must not silently fail, then you know essentially here you're, silently failing so to speak.

C

Okay got it so I think um yeah. Maybe I can. I can take that feedback and update this, uh that it's orthogonal to it.

F

Yeah sure, thanks yeah and the other question I have is uh since this, these requirements or considerations are putting certain requirements of how the api is going to literally look like right and what kind of apis you are expecting, whether it's the northbound or the southbound like psy or versus the gnmi, so that that also, in my view, is also putting requirements on how we are going to define the behavior models right. Since, yes, we are generating yeah, since we are generating psi apis using those behavior models.

F

That essentially, is an input to the behavior model. I think we should probably have a little mention of that. One here saying that okay, these requirements, essentially we're going to distill into how we're going to write the behavior models.

D

Yeah hana, I'm really glad you brought that up that I was going to say something similar, especially with 0.9 here in the design considerations.

D

There's a certain amount of orchestration and sequencing here like what it implies to me is you need to make a lot of table entries and then, at the very la at the very last step open the floodgates you know enable it administratively.

D

Is there even a psi uh dash site primitive for that at this point in time, is everything going to be controllable through psi as listed here or there may be some out of band or some other mechanism which I think we want to avoid and if we're just generating the scio from p4 tables automatically?

D

Is there an allowance for do? We need a table? That's just like an administrative state table, or is there a attribute for administrative for an eni that lets you do this effectively to the psi interface? It's kind of a question for marion you know do. Is everything here covered by the current psy, auto generation.

G

We don't have admin state to actually drop packets um if it's down, but we can edit it's not really a problem. Yeah.

D

Do do we have a consensus or can people think about? Do we want to make sure that everything listed here can be controlled um through psi with the same the right semantics, and we want to avoid ex unmentioned uh interfaces or unspecified interfaces for doing this.

F

D

F

You know yeah, I know I agree that uh wherever it is uh required and necessary or applicable, I should say for psy: we should. We should definitely spell it out. Yeah.

D

I mean I guess to to put it bluntly: we want to avoid band-aids after the fact right, right, secret, apis and special fixes. Everything should just be spelled out in the apis, and they all should be pretty up front. Okay, thanks.

H

Yeah, especially especially when it comes to administrative up after setting up everything right, you don't want any other side channel. You know yeah.

D

Options here yeah, I didn't saw you can't do it. That would be the another rule. All right.

C

That's right, yeah.

D

Thank you very much.

C

For this sure, I I think um we can also look offline to to define this if that that helps yeah.

C

Okay, so coming down, we have the packet flow. This is again explained in detail in other sections of this dash repo, so um I just put here for um for reference basically in the outbound directions.

C

We have the vm the packet coming from the vm that goes through axles and then routing um then having the encapsulation and going to the network. So the outbound.

C

Examples are given at the at the end, so we we will go over that in in this discussion, and then I have the inbound packet processing where the packet is coming from the network that goes to routing then echoes and then doing an encapsulation and going to the vm. So this is uh mainly for reference. It's not very detailed in this document, but I'm I'm sure like christina, has a good document elsewhere.

C

um So with this packet flow pipeline in mind, going to the modules design. um So, as most of you would know like in sonic, we have the config db as well as the app db. So in this design we are planning to have the dash objects programmed to the app db with the prefix dash. So this way, any regular configurations, like the underlay configurations, are all isolated from the dash um and we don't have to have uh um some stale configurations like during config, save, restore, etc.

C

So that's the intention why we are not putting all these entries to the db. So this is a a new requirement to the jnmi, because the current jnmi design has the configurations directly put to the config dp based on the young model, but we have put the requirement for genomic to uh to program the app db uh if it is like dash.

C

So so that's the the plan uh with that. There are not much of config dp objects that are being changed for sonic so, except for the device metadata that I have captured here in that requirement about appliance and dpu. So those are the only changes expected in the config dp coming to the app db.

C

So here is the object at a high level, so we have eni and then we have vnet. So these are the tables that are associated to an eni. Sorry is there any question.

I

Yeah prince, I have.

J

C

J

Yeah, so uh thank you, based on what you described earlier, where you're not gonna, put anything into the config db. So is that correct? The expectation is if the dpu reboot it then we'll get basically have to reconnect to the controller and get a phone config put on.

C

The controller again right: okay, thanks you're right so but, like I said uh because it's in the app db we will uh um in future we are planning to support the warm report of the card yeah. So in that one boot we will not expect uh the controller to reprogram everything, but in call boot. Yes,.

D

Will there be any way to import or export a configuration based on like a text file representation like a json file.

C

ah Not at least not considered here because if it is called boot, it's like a fresh start of the device right. So uh the reason why at least I haven't considered, that is because we will plan to provide one boot. If all that configurations has to be restored, then we can employ that model.

D

Yeah, it might facilitate testing at some point.

C

A

That would be a lot more work prince sir.

C

uh For because it's, I think it will be more helpful for like some test purposes, but for config restoration. I don't think we can.

C

We can have that unless there is some hooks in the gnmi to load from adjacent and oh.

A

C

D

I was thinking more of something that would not necessarily be related to gnmi, but it would be a very simple mapping from the the db, the redis db.

C

Yeah yeah, I think that we can of course uh that's supported like today. If you have a json file, I think uh maybe we will go to that example. If that uh is part of an adjacent file, you can just directly apply to the db. Yes,.

D

Okay, great and- and you know this is an ongoing discussion- we've been having. I don't want to distract the meeting. Let's have that meeting sometime.

C

That's smaller.

D

Meeting, thank you sure.

C

Thanks friends: okay, so um at the container level here we see that eni has these objects. um The routable ackl and associated dna and v-net will have the mapping table.

C

um So uh that's just giving a high level reference, and now we have that each definitions for each of these objects like we are dash vnet, and then we have dash qos. So it's the key. Is we net name? And then you have all the associated attributes, then you have the eni which which has ena as the key and then associated attributes. I would try to define all these. What these attributes are intended for and also the good reference is the other document that that is in the dash report, the config model.

C

So that will have some examples as well.

F

I have a quick question on so in the eni um you know in what conditions an eni is associated with multiple different v-nets, uh because you define that. Okay, you know the list of v-necks that the cni is.

C

Associated right, yes, that's.

F

The appearing scenario.

C

Right, so if you are p, if you have like v net p array, you can say like this e n, I has uh v net associations to let's say we need one. We need two. We need three. So that means it's kind of appearing uh scenario.

F

C

F

This is something that we are not tackling as part of v-net to v-neck, because it's just on the intra-v-net right now the use case we are doing once we do the in appearing, then this would become probably more evidence-based.

C

Clean appearing is part of the phase one requirement, so it is considered here.

F

Okay, yeah, oh um no, I think I was. I was just looking at those seven different use cases in which there was v-net was separated in the v-net pairing and we thought we were only doing intra-unit part right now.

C

uh Okay, maybe I can, I can correctly hear v-netween, it means like v-net and uh uh and we need pairing yeah.

C

Yeah, um okay um yeah, then yeah we have, the articles like articles are, uh um are um having like a groups and rules so uh so.

F

Is is there a? Is there a typo here, because I, if I read the key for acl, should that ena mac and his stage has the key as opposed to state.

C

Oh ena, mac and stage is the key yeah that this is a typo yeah yeah yeah. That's right, yeah! It's not the state of course yeah! So yeah thanks uh I'll I'll correct that.

F

Or I might basically go and send you the pr and then you can uh yeah.

C

Sure sure please go ahead, yeah, uh so so yeah, so actually we have like in in and out tackles uh associated to ena, and then there could be like multiple stages. So that's why the stage is um is planned as an uh as a key, and then you have the group association for that and and each group will have a group id and and a grid that the controller can reference to, and then the rules are associated to the groups. So here um how we ensure the the atomic behavior is like.

C

If, let's say there is a new set of factor rules uh to be uh to be programmed, then the controller can program a new, completely new set of active rules and then in one shot commit that tackle group to a to an eni. So that way the previous one gets replaced with the new one.

C

So uh so that's the intention with eccle and um the definitions are or the definitions of attributes are here. So some of the key points here is like the active rule, has protocol source address destination address and let's say, if you look at the protocols, it's a coma separated set of values and if nothing is mentioned, it's like wild card so uh option. These are like optional fields like if protocol has.

C

Like no values as specified, it means the rule should apply on all the all the different protocol types yeah, so capture that here.

B

C

And then we have the list of source, ips and destinations, yeah.

C

All right so coming down to routing type, this is being referenced in the mapping and in the route table, so we have different routing types. uh It can be like direct, we netween a director in a 10 cap, so I will explain those uh in detail in the upcoming sections, so we are not having the routing type private, link, private link, industry and service tunnels in the in the current phase.

C

So uh we'll be more looking towards these four actions like direct, we need to indirect and win at 10 cap, and then you can have different actions. Actions can be map routing, uh direct static, encapsulation, and some of these are again part of the the private link scenario. So we will not be considering that in phase one, however, I'm putting it for completion, yeah and cap type can be vxlan gre so like based on the action type. The other fields uh need to be populated.

C

So that's why some of these fields are like optional, um but but it means like in the yang I have uh defined. So there is a yang file, also in the and as a pr. So here the dependencies are.

C

Clearly described but, however, in the scheme it is very, it is hard to define. So basically what it means is, let's say if the action type is vxlan, then these two fields are mandatory um uh or actually yeah. So that's the point, but in in general these are like optional fields from an implementation perspective, yeah, okay, routing type appliance is just like giving some the source ip of the plans and also the the vnis that needs special treatment for setting the directions yeah coming down.

C

These are the two main tables like the routable, that's associated to eni, and then you have the mapping table that's associated to vnet, and these are the different attributes in the route table and again, like some of the cases I have clearly defined here, like overlay, ship is needed if routing type is serviced and also we are not having the implementation for this in the phase one um yeah and for each of the different uh routing type attributes will take its own meaning. So that's why these are defined over here.

C

But let's go to some examples. Maybe that will clarify.

C

Mapping table, I think this is the cap mapping table. So ip address is the ca, and then you have the p addresses centrally.

C

There is also a routing type required in the mapping table which we can def, uh which we can go over uh later.

C

I think one question in the in the pr, as a comment was why we need we have routing type in the mapping table yeah uh now the on the module interaction. Let me zoom.

C

Yeah, so the plan is to have the controller uh using grpc api and then um we'll have a gnmi container. uh Basically, that accepts all these get set, calls and we'll write to the db, um which will be the app dps based on the current design, and then we have uh the sws's that subscribe to these app db objects and then we'll have a dash or cajun that writes to the sing-d or the asicdb.

C

So this is at a very high level how the the api flow will happen.

K

Friends, if there are failures uh during the programming right this iap's programming, we don't regret that to the controller right.

C

Right, that's all the thing, so I think uh that's a good point currently from the back end. If, uh if the failures, how to propagate is a is something that um that we need to uh find like basically, some sort of error, state or the state db objects that the controller can also do get operations to see how what what is the state of that object like if it is uh programmed or not programmed, so we have uh today the uh some some state object that reflects the the final state right.

C

So similarly something um has to be done. Yeah. I think that is also something uh one thing too.

F

I think you know, pins have pins, have introduced uh something.

A

That we can probably absolutely leverage.

F

Exactly so abstract db.

A

F

Then you can actually utilize that particular area to to really communicate the error errors to the controller.

C

Right, I I think those are being considered, so we just need to work with the gen, my team, to how they can read from that uh the different tables to uh to yeah to return back to the controller yeah.

A

And it looks like there's an echo question from j park in the chat window. Is there a special way to specify go ahead.

C

I I I'm not seeing the chat window so.

A

I've got it, I've got it yeah, it just says uh jay. Do you want to ask your question if you're online.

L

Yeah, so I was just wondering if there's a special way to specify all port ranges versus uh and prefixes, or do we need to add uh like zero, two six, five, five, three five and uh 0.0.0.0 slash zero prefix to specify all address ranges and port ranges.

C

Okay, so I think I have defined that in the yang, but let's take a look.

C

Yeah, I think I should give some good examples here yeah, but the intention is like here: you can have the source port and destination port as as list, and then you can specify the range as the pattern like here.

C

Yeah, maybe I think I can put some examples here if that helps yeah.

L

Yeah sounds good, I think it looks like we need to specify from like zero, two six, five, five, three five, if you want to uh add all port ranges right.

C

Yes, yes, okay, do.

A

You want to do a pr against this j, or do you just want to write it down prince.

C

This is a pr which we haven't merged it, yet I think uh there are lots of good feedback from winkert and others, so I I'll have to address all those uh before we can merge, yeah so yeah. I I think if you can put that as a comment in the in the pr that would be good.

A

Shouldn't you do that jay can't think I'll.

M

Just yeah just to just a quick comment there I think your 2 to the 16th uh is is wrong. It should be 6, 5, 5, three five.

C

A

You're so funny john you're, very precise.

A

H

C

Yeah, okay, I think I.

M

E

C

It from uh some place, let me check what was what was it defined there yeah.

D

One more question: um you mentioned a little earlier talking about combinations of parameters that were hard. Let's say, restrictions in the.

B

D

Expressed in the sonic db schema, what's your expectation of the validation that we do at the psi interface, for example, if certain things are mutually exclusive or there's certain requirements that are expressed in must statements. Do we need to implement that at the site boundary, or is the controller going to do that level of validation or both.

C

I don't think that is at the silo required, because that validation happens at the northbound.

D

Okay, it does mean that we're doing testing at the psi level. We could get the device into a state. It doesn't make sense right if we programmed it yeah yeah. Some reason.

I

Okay, prince the ref, counting, etc. When you reference uh some of the objects that you are trying to program using the controller and if that has dependencies on the local control plane right so that reference count check and the interdependency checks you're doing in the arc agent. Isn't it.

C

Orchesin has uh dependency, but at the db layer also, we can do some checks right. Let's say: if mappings are there associated to vnet, then it doesn't need to be on a reference count basis. If there is one existing, then we can return failure right.

C

Is it uh like at the.

K

Organization there.

C

Of course there will be like reference counting um in place, but at the general layer I don't think there will be reference counting, but it will just check if the objects are present.

I

C

C

Okay, so we have the uh we have touched upon this overall design. um However, like.

B

I'm sorry, can I ask a question about this. uh This picture, and just uh you know for myself to make sure I understand and to recap so the auto generation from the behavioral model is it generating the psi api dash module or just the interface the the library that the psi api dash module will use to interact with the sdk.

C

That's a question: uh maybe marion can answer better here we are using the psi ap in this. Based on this uh flow. It is the psi ap that the uh that uh or cajun would be called, but on the auto generation part marian. Can you can you help answer.

G

Yes, we're generating only the interface. That's it.

B

So the interface between that module in yellow the psi api dash and the sdk.

G

No, not between the sdk, the interface itself. uh The implementation is on vendors, so it does not know anything about usb.

B

Right, no, but but the interface so this api that the the psi api dash is the api which is actually between those two modules, the yellow one and the white one. The the psy api dash and the asic sdk.

G

B

Okay, so so the mapping between the objects in the server and the site api is it implemented into that yellow module.

G

I don't really.

B

G

Which object you initially were talking about.

B

So so uh so the controller is going to use the the gnmi dash api, okay to configure the device. The the server will map the gmi messages into object in to the db to.

A

B

Right so the effect of any call of the gmi apis is going to be some objects being created, deleted or modified into the database.

B

At this point, those objects need to be picked up by syncd and somehow translate it into calls to the psi api that we have. So what.

G

B

G

Are you talking about app db or a6b, because there are two layers.

M

Right, I I want to understand that that, because I like it's- not it's not clear to me- is this: what's running on the host cpu in a smart switch or is this running on each and every dpu in the smart switch.

D

Each and every dpu has a full has this sonic stack subset shown here in this drawing.

M

Okay, so like in the in the in the web, documentation, there's a diagram that says high level architecture that shows a gnmi container in multiple of these sink d containers.

M

Is that just like some logical diagram, that's spread across multiple dpu instances or is, or is that what's yeah.

C

I I think, john, that that is a different uh architecture itself, where we have like different zinc d containers across gpu. So I think this we can just focus on the planes and um and and clarify.

M

C

The flow because that that's a whole different uh design, discussion right so.

M

I I understand like on the appliance. This would be the diagram for each and every dpu yeah um and the smart switch. You think it's different, or you think that this also.

C

I think that's that's a design we can, uh we can either have a single uh replicated or we can have its own sonic instance, but that's that doesn't change the flow right. So.

M

Okay, all right all right, I I.

C

M

It's not the right discussion for this meeting, but I I would like to understand. Like yeah, I.

C

Think small church, maybe uh christina or gerald, can um can help answer and we can have maybe a separate uh discussion for that. Okay, okay,.

N

C

We have, are we trying to.

N

Say are we trying to ask the question on whether sonic should run on every dpu in this sonic? No.

M

No, no, no! No! No! No! No gerald! I mean I totally understand that sonic will run on every dpu but, like I just don't know like where these containers run, do they run on the hosts uh sonic the switch sonic, or do they run on the dpu sonic um that that's my that's my question. It's just like the split this.

A

Is the conversation gohan where they can move around depending on the analysis? You know once um once we have something going. I think carol.

N

uh I think it can work in both ways. The the problem is, we have no performance data to say definitively that this should move down to the dpu or it should stay up at the switch. I think the desire would be hey for that piece. If it was on the switch you'd have one instance, it might be easier to to manage the device that way, but that's just a desire. We don't know if that scale. So we don't know if that would perform, and okay.

B

N

Kind of been left open until we do something like we don't have an implementation to actually know whether it would perform or not. I worry it won't perform, but I'm open to like if it does perform and okay.

L

N

And it's fine, then I would have no problem with it, but we we're not at that stage. Okay,.

M

So but as you know, for us to make progress like we can at least assume for the appliance, and at least the starting point of the smart switch is that this diagram exists on each and every uh uh dpu. Yes, and that's mandatory for.

N

Any appliance I mean they would always.

M

N

With the full set of containers.

M

Yes, okay, understood. Thank you.

B

Okay, I'm sorry! I just want to go briefly back, given that this is going to run in wherever every dpu.

B

The mapping between the the high level api and the objects that that api generates in the db and the psi api dash is, is the mapping performed by the yellow uh the yellow process. They are called sciatica. No.

C

I think this is the uh the psy api definitions like uh like what the orc agent here will translate from the the application db object.

D

You want to go to the next diagram. Maybe I can step in and help a little bit. It's a little more detail and I drew this diagram to try to help myself understand it and prints accept it into the document. So, mario, what happens? The flow is the grpc gmi call gets made. The gmi server makes an appdb entry.

D

Then there's a subscription notification to the orc agent. That then transforms those high-level, app db's into asic db or spy conceptual objects in the asic db.

D

Then then, the sonic stack, the sync d container subscribes to those changes and propagates them and just basically makes calls almost transparently to the equivalent psi apis. So if there's a table- and that's all done in sonic- not even the vendor doesn't even vendors- don't even need to write any code up to that point. Right, microsoft, it's kind of taking on the orchestration and the gmi, so you really just have to handle everything from the the c level call in the psi. Api down and marion did a real job in the behavioral model of writing.

D

Congenita templates translate those cycles into p4. Runtime calls it's kind of a neat thing. You can look at that as inspiration.

B

Yes, but but so if I understand correctly, the app db is at the abstraction level or of the gmi. The skdb is at the abstraction level of the psi api. The translation is performed by work, but york agent.

B

Yes, okay, great! Thank you very much for clarifying, but at this point I have a follow-up question, which is, as we generate the psi api shouldn't. We also auto generate the schema for the azik db, because that schema depends on our behavioral model.

C

Now the schema is out to define like basically, if you are making a an orc agent call to based on this iapa, it will be. um It will be internally taken care.

C

If you are talking about the schema for asktp, we don't have to explicitly define that.

G

The thing is that uh it is sometimes the case that um one, then, that more than one primitive in behavioral model, more than one primitive insight will be mapped to one entry in the schema. So it's not exactly one to one. It may be many to one and that's why it is done manually because it's uh what we have in the fdb is uh closer to how the user will see things, and then we may do some more complex translations and also handle dependencies and so on. Before we get to psi representation.

B

Yeah no, but but it's okay, but but is asic db already the size representation or there is a. Is there a further translation based on what chris said as a db is basically.

G

Basically, this is your representation yeah, but asic db does not have schema. A6Db is fully serialized based on site api, so it's kind of like what you mean it is auto generated, but it's already in place. You don't need to do any work for that.

G

B

About it I mean I, I understand what you're saying I didn't understand, how it works, but they will look into it, but uh this clarifies thank you very much.

A

Yeah I want to check in prince. We've got eight minutes left. um Do you think we'll need to go into a second session or.

C

No, I think uh I can try to cover the the remaining section and if you need uh we can plan so hey.

F

um Sorry, I just have one question like so um diagram right that is two dbs asic and half db. So for every object, for example like route, uh we will have two two installed in one in each of the db so yeah when the full scale config is loaded like, for example, I saw like 100k uh routes per engine, so it's like 6.4 million rounds so like we will be having like 13 million routes in this db. So like have we considered like the memory usage.

C

Yeah absolutely so we we have the memory section down, so we will. We are working on like what will be the scaled uh memory usage with with that many routes. We are working on that so uh I'll come to that memory section and what we are planning to do I'll touch upon that yeah yeah, so yeah. So in the industry of time.

C

Let's go to the sonic uh next section, the sonic host container, so uh not all uh containers that will be enabled in the appliance scenario so we'll have like some tmd and um dhcp, etc. Containers that will be disabled. So basically the section captures what will what all will be active? And here.

J

I have a question on the contender. um I notice you have telemetry markers and able that is in sonic that is currently hosting the gnmi server. But then you also have the dash uh gmi uh container earlier. So is that an extension to the telemetry container? Is that a new kind of so that's the that's? The new.

C

Design going on with the gm.

J

Right, uh whether.

C

B

To have both in the same container, or they will have it.

C

B

Have it in two different containers that will be.

C

A design that um that's that's part of the gnmia.

J

Okay, so you say you're saying either you consolidate, then that would be part of telemetry container if it's a separate, telemetry container will be turned off in that k for dash telemetry.

C

Is still required for dash so whether it is part of gnmi or whether it is part of separate both will be available.

J

Okay, but but mainly handle the the gmr spec, but for telemetry purpose versus the last configuration aspect. Yes,.

C

J

Right. Thank you.

C

Okay, so on the overlay dash orchestrion, we have the um we have. It um briefly touched above. uh Basically, there is a dash org that will do the translation from appdb to uh to psi, and then there is this: uh a regular underlay, the.

B

C

The dp as well like creating the port, some um traps like bgp, has to be enabled, so these are the existing underlay psi apis that we will be using um in the dpu world as well yeah. So that's this!

C

um It's a small section of underlay, routing or- and here is the memory footprint like here- is where for to the previous comment, we will be adding more data here, based on the scalability so currently with all base dockers and no configuration it's 1.3, gig and- and we have the upper limit, like probably uh five or six gig, which we are trying trying to uh to see how how much would be the usage uh with the scaled configuration that this section will be updated. Based on that findings, yeah.

F

Yeah and also uh prince the the container list that you had it above uh yeah probably needs to synchronize with this this list as well, because this has fewer containers than the one that you're asking for as a yes column from the previous list. Right so.

C

Oh, did I miss some containers yeah.

F

I don't know there was a gnmi container. I didn't see a list.

C

Because that is the that is still in development, so I don't.

F

Have the container.

C

Yet yeah yeah, okay.

F

So yeah then, once you have it then you'll have to probably yeah.

C

I think the the only missing one would be the gnmi, because that's yeah.

K

Prince, do we need any snmp operations for uh dash? Why do we need a snmp container.

C

So snmp get is a is still a monitoring part uh in our uh deployment, so we get so they do still do like you know. Sometimes the pa reachability of the blinds such like it or the loopback reachability. Those things are still handled through snmp, okay, yeah, okay and the cli section here, like I discussed before, like we have the show and the clear command so put um here so in case like for some purpose. If we have to clear all the configurations on the on the appliance, then we can leverage these commands now.

C

Coming to the example configuration so here is what I talked about like the lookup, how how it can happen. So let's say we create a vnet and then an eni, and it can say, like the cni, has the v net one association.

C

So uh so you create a vnet, you create a vni and then specify the routing type. Let's say: uh map routing, okay: this is a list of action, but let's consider only one case here and then we have the other routing type as we know direct, and then we have the v net in cap. So let let's assume there are three different routing type defined.

C

You can ignore this private link for now, and you have a routing table um of like for different prefixes uh and the action type. So here, 10 1, 0, 16 24, I'm defining these different um tables based on the previous definition. So I have like three different route tables or like four different route um tables for different prefixes, and then I have a mapping table for different ca addresses. So with these configurations, let's take an example uh for a packet destined to 10, 1 1 1..

C

So if so like, I said this is the out bound direction. If I quickly go back here on the outbound direction, uh you have the axles routing. So this is the flow, so I don't have the alkyl example, but in the routing let's come back here uh it. It hits for the 10 1 0 16 entry, because that's the most preferred uh matching um and then it says that action is vnet. So.

C

Okay, I think I need to oh, no, no, no! So ten one uh one falls here right, because this this one is actually uh incorrect. It is ten one one, zero slash. Twenty four alpha: okay,.

F

C

F

So that otherwise you know this would have been basically correct. Okay,.

C

So I I'll fix that, so this should be like.

C

No, no, no, actually, it is correct. Like 10 1 1 1 will fall into this only right, 10, 1 1 1, because it won't fall.

K

Here, yeah, that's correct, prince.

C

That's correct, yeah, yeah. Sorry. I think it is correct.

C

Okay, okay, so it goes to vnet, and then it says that the map routing, so it goes to the mapping table for vnet1 and in the vnet1 mapping table. Where is that here we have the mapping table and it will find for 10111 as the prefix. What is the? uh What is the next uh action so so so think about it. First look into the map routing table.

C

Then it's the routing table says that go to the mapping table for the next action and it goes to the mapping table and in the mapping table it says the routing type is next is vxlan. We need encapsulation so and then you have the pa address from here now going to the veena 10 cap. Here is the routing action, routing type v net n cap, and for that the action is static and cap and use vxlan.

C

uh So this is the action, so final action will be a vxlan encap and what is the vni and pa address? That's derived from the mapping table and the vnet. So so that is the intention of this section like how the different encapsulation can happen. Now. This is a one more thing, I think three and four are straightforward. So one more q explanation for this scenario: 10 1, 0 1. um It goes to the routing table here: 10 1, 0 1.

C

It says um v, net 1 and I have defined in the schema what this overlay ip means. So when it goes to win at 1, it's similar to the previous one, but it has an attribute 1006 so which means, even if the ca or the the destination is 10 1, 0 1 you look up to the v net. One should happen based on this overlay ip. So you are uh so basically here uh with our ap specified.

C

So now the lookup will go to the we net one, so we net one, but it will not look for. In the previous case, it was looking based on the packets destination ip, but in in this case, because the routing table is specified as specific overlayp.

C

It goes to this vnet1 for this destination, even though it is 10101, it goes to 1006, and then it says that okay, it has been a 10 cap as the action which is same as before, but use underlay ips to this uh ipv6 so see the difference. So that's the intention with that lookup.

F

No, I I don't understand how it decides. The mapping table for 1006 should be the head.

C

Because it has been defined in the routing table.

F

Yeah so 1006 in the routing table and then yeah. So then, what you're basically saying is that in the mapping tables key is actually a v-net and overlapping, because this prefix, then it has to basically so this prefix is essentially saying that this is actually an overlapping prefix or somebody else. It's a different prefix has a different definition versus the one. This one right.

C

uh For somebody else, meaning.

F

No meaning that, for the previous example.

C

F

C

In in the routing, if there is no uh overlap is specified, then it will use the packets uh or the uh uh or or the destination ipa.

C

So my point is like there are: these are like complex routing and lookup scenarios, uh but these are the different modes of operations or different yeah.

F

So plus, I think what you know uh prince, I know in the interest of time. I wasn't basically trying to look for this one, but in the in the second example, when you say the hit is dot. 24 24. Is this a typo here, because there is no lookup entry um which has dot 24, slash 24 in the router.

C

I think this is 10 1 0.. I I think this is definitely attack, because this one should be 10 1, 0, 0, 24.

F

Okay, so yeah, I was trying to look for this one and I I can't, but.

M

F

You have an overlay ip, I was. I wasn't sure how you hit the overlay ip to really decide that you're going to enter into a different mapping table in order to really do.

H

F

Right action so.

A

Okay, I wonder if we should have a q a session. You know this is a lot of content and I wonder if we should give people a week to ingest it and then then take like 45 minutes for q a next week.

A

Would that be okay, prince yeah? I'm fine.

C

I think uh yeah. I think that uh we can also add to the pr if there are more questions.

A

Yeah, absolutely absolutely yeah. So if the community could take the action item to add to the pr, then we could take the action to deliver a q. A uh based on that. uh Would that work for everybody.

A

I can't see everyone's.

B

A

Yes, yes, yeah yeah,.

N

Yes, yes, yeah. I think everybody's gonna have questions. I even have some questions.

A

Yeah this is a lot of content and we'll have to coordinate with prince or if you have someone else from the sonic team, if you're busy um yeah, maybe we should put that on the agenda for next week.

A

Okay, okay, great! um Are we through the the content for today prints or should we keep.

C

A

This is wonderful, it's just. This is just huge prince. It's so great. um I think I think there's going to be a lot of questions, but this is a massive amount of work. Thank you so much, and what do you think gerald.

N

No, I think it's great and it definitely deserves another.

B

N

There's gonna be lots of questions like I said I even have some questions but uh yeah, so it should be the next meeting yeah.

A

N

A

It is great work.

N

So you know people have been asking for this for a long time, and this is a good start.

A

Yeah yeah everybody get your questions into the pr contact, uh prints and we'll we'll try and cover this next weekend. But I know you had a question I'll, try and get to it offline for you. I'm gonna stop the recording and post it to the youtube channel and um it's 1007, so I'm late for another call, of course, um and I hope you all have a a good day and a good week. Thank you. Everyone thanks and thank.

F

A