DASH Behavioral Model WG, 23 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DASH Behavioral Model WG Weekly Meeting (June 23, 2022)

Description

June 23, 2022
Created new work items
Discussed need for testing

A

uh Welcome to the behavioral model meeting this week june 23rd, um I don't have my action items up from last time, I'm having trouble with my screen, but we can go ahead and get started. Maureen go ahead. If you wanted to get started.

B

Again again, christina, can you repeat.

A

Oh, I said we could get started um if you're ready, sure.

B

Is there anything specific for me for today.

A

B

A

Pull my notes from last week, I'm having a little bit of trouble. So while I try and get those going, um we can. We can talk about what we remember from last week.

C

I was wondering maureen: do you want to comment on the poll request you just submitted to rename the pipeline.

B

um Yeah right, so this is the item that was in the backlog for quite some time. This number, let me see where is it number 13?

B

We wanted to get rid of the let's call it the working name that uh originally came from microsoft and use the well established name dash, basically rename everything serious to dash.

C

How about the directory.

B

um Yeah, I noticed that a minute ago. I also did that, and there is a comment from you that we again may need some.

B

What do you call it some synchronization between two requests? Oh, this is about ci.

C

Yeah the the fact that you're renaming the direct now the directory and all the files and um and also some of the content and I've got a you know. Pretty big merge request poll request from the one I reviewed yesterday.

C

Probably the best thing is for us to accept yours and just get out of the way and I'll just synchronize to it in the pull request as needed.

B

Yeah, so if I, if I'm not mistaken, there will be two changes that you will need to do to your branch in order to uh merge with my uh renaming. First, one is the working directory, as you may have noticed. Now it is a dash pipeline yeah and second is one of the make targets, the the very first one where you make where you basically compile the p4 code.

B

um The target now is changed to the dash pipeline.json. I don't believe it affects anything else, because the auto generated apis and library, all of that it was either like sciol or already had dash in in it. So all the serious name was contained in p4, so I would expect two changes that you will need to do. Is the working directory and uh uh the first compilation target.

C

Yeah, I can, I can figure out, hopefully the when I do. A merge merge in the main it'll gets pretty good about renaming files and things there might be a little cleanup.

C

Hopefully it won't be too catastrophic, but you know it's only so many files.

B

Yeah and then uh second thing that I want to add, if you may have noticed this is still a draft pull request. Reason being that I agreed with christina and michael to also since we are doing this kind of a cleanup, uh we are sub, we are going to uh submit more information into readme in each of those directories. So please expect that to be done uh a bit later today and then the pull request will be ready.

C

Okay, so I have a pretty large readme in my pull request for the main top file. Are you going to modify that much because I'm uh doing quite a bit to document everything? What's the path to yours under serious pipeline readme yeah it'd be nice. If you looked at my pull requests.

B

Okay, got it right so then, maybe maybe I will need to split mine uh just renaming first, you can do yours and then I will update readme files.

C

Yeah, that might be nice because I've put quite a bit of work into that right. It'd be nice to uh combine our work, not collide a bit thanks.

C

Okay, I've always done some optimization today, people, I I I've, got it I'm using two different docker images: one that's pre-built with p4c, you can just get it from docker hub p4c is a docker image. You can get online, so I pull that down first and compile the code right away and you get the p4 will compile and either pass or fail in like 25 seconds.

C

So you get very quick feedback in the pipeline or locally, and the docker image for that is really small comparatively so I've got everything I'm starting to tune this a bit step by step.

C

But yeah, if you do the rename I'll try to merge with that after, after your pull request is accepted.

C

um I I I guess that's enough on that topic. um I know. There's two topics on my mind: besides, whatever might be on the agenda for today, maybe three topics.

A

Okay, um go ahead. Press.

C

Okay, just just to summarize them one is: um I want to start getting some kind of a some goal set for what we're going to test and when, or at least at least what have some goals, because we want to start doing meaningful pipeline tests, not just this dummy test. We did that. We showed yesterday all right so start actually having tests that exercise some of the features so we're going to be committing some resources to putting in more test work, and we need to have a pipeline with known.

C

You know known capabilities, and we need to figure out okay. What do we want to test? First? What order?

C

For example, we want to test, you know v-net service, different pieces of it, and you know today we have a bunch of p4 code, but we don't have much evidence that it does what we want.

C

So that's one item can.

A

We should we add this item to the backlog for us here.

C

Yeah, I would say um you know test plan for the pipeline at the highest level. We can always break it down, um and another item is.

A

Hold on you, gotta give me a chance to hit enter here.

A

And so so here in the comments we basically want to begin testing features yeah.

C

And you know I can we can have a discussion on that to figure out how people would like to go about that. I have some ideas. Okay, another topic is: when are we gonna? Have that's just an open question? When are we gonna have the non stand bmv2 when we're going to have a modified version that we can start testing stateful traffic.

B

So that's oh yeah yeah! We are going to be um doing that. I have a dedicated meeting on that on for that on friday to see what we have left to actually uh merge stateful stuff into bmw.

C

Okay and a follow-on topic is how to integrate that build workflow into this into this project. Right now, we're pulling vmb2 from docker hub or building it. How are we going to do the build from scratch bmb2 in this pipeline and integrate with all this infrastructure and automation right?

C

um I know that you have a there's a fork of this right marion going on, and meanwhile I've done all these fair amount of well say some refactoring right. My pull request. Hopefully, you'll have a chance to look into it and flag any concerns, but you know there's going to be a reconciliation phase. We have to decide what order we want to. You know accept these merge requests and how much?

C

How much um work is that going to cost people to kind of sync up to it?

C

Timing's kind of everything and right now we're going to rename everything it's kind of a one-time penalty, but then there'll be maybe another penalty. If, if we accept my merge request pr, you know 27, what's that, what's the impact on any forks, you have right now, where you're modifying v2 and how's that all going to work.

C

So that's that's a topic and, and then the other topic that comes to mind is the one gerald prompted yesterday.

C

How are we going to test the tcp state machine so to speak?

C

um And I don't know if there's actually a tcp state machine embodied in the pipeline itself or if it's kind of an you know say an emergent property of the pipeline, because I don't recall seeing truly a state machine, but there's something that's embodying it.

C

Are we going to have a white box test of that step by step where we stimulate the pipeline with packets and read the states one step at a time or we're going to do a black box test where we throw packets at it and very carefully orchestrated and then observe the response capture the result or whatever and and say yes, this this acts. The way it ought to assuming the state machine is correct.

C

I think that's a pretty heavy discussion. I mean just started yesterday.

B

Yeah, that's what gerald brought up.

C

Yeah and it his proposal, sort of presumes, there's an interface to read the state and even deeper that there's any internal mechanism, that's tracking the state explicitly as such right- and that was, I think, um I think, that's a whole discussion area. Is it a requirement for the neighbor model or even a vendor implementation to provide state by state?

C

Let's call it telemetry.

B

Yeah, so I can uh give my uh opinion on that, but uh probably it's better to take it when gerald will be here. I can just simply make a statement of fact that we, as of now uh neither in the behavioral model nor in the api we do not expose flow id itself.

B

We don't have an access to the flow right, so if there will be such a requirement, that's the like again as a statement. The fact that's the big missing piece.

B

A

Am I capturing these okay guys.

C

Yeah, I think so, just as long as we have even that bullets we we know that we have to address them at some time.

A

Yeah yeah and I'll fill in the assignees yeah go ahead.

C

So marion on the question of you know our sequencing, our poll requests and, ultimately the impact on the bmb2 that you know the custom one or, let's say the enhanced one. Probably you want to have that dedicated meeting and just think of that as an item right, maybe look at horror guests and go gosh chris. If we accept this, it's really gonna, you know um cause a train wreck in what I've got going on or maybe it'll merge smoothly, but we ought at least um try to accommodate each other.

C

So we don't cause too much uh excitement. I'd also like to to talk about a little bit about the docker file that you created in the docker image. I think there's a lot of room for optimization and I'm already starting that, but at some point either I'll have to dig in even deeper to reverse engineer some of the things you did or toss some ideas out to you and see how you think about it. I wrote about it in my pool: I've updated the documentation in my pull request.

C

The readme talks about docker optimization right near the top in the to-do list. If you could take a look at that and think about it, I even have some more information following that. I think the docker image has things in it. That can be, um you know, simplified, and maybe we can pull things from docker hub, not have to reinvent certain things.

C

So that's that's sort of a broad topic. You probably need to have your meeting about behavior, modeled and think about these things with whoever's. Moving with you it's as far as a test plan. My question to everyone here and maybe especially to marion, is: uh can we come up?

C

Can we agree on a list of basic tests and and what order that would give us a nice um say, a little progress in verifying certain parts of the dash overlay and not only how to you know how to configure it, but then what packet to send to it and if you're doing this sort of the legacy way marion, I imagine you'd write a scopy packet, send it in look at it and say yeah. That makes sense right, just as a manual test like get started and then eventually turn it into a ptf test.

C

With the same kind of workflow, and you you program, the side tables right now with a c client test client.

C

Is that a reasonable assumption, martin, how you might go about it if you were just to do it yeah right after yesterday, so in order to- and I think what keysight would like to do- is get help in understanding what tests make sense and if, for example, marion or someone else were to write best in whatever convenient way you like you know, without throwing curveballs at you like having to learn xcsc or whatever just do it in whatever way makes sense, because we can take that as a starting point and then, let's say, convert it to the form we're proposing that can eventually, you know, incorporate into line rate tests and it'll allow us to program the psi interface.

C

Even if it's a c program or c plus plus program, it can eventually be turned into cythrift, because the contents are going to be the same. It's just an rpc instead does that seem like a reasonable way to go about it.

B

Yeah sure yeah.

C

I mean I want to make the best use. I would like to request your help, but make the best use of your time just to do like you know, figure out what tests make sense, how to how to configure it, throw a packet at it. Even if you just capture as a readme or you know some something that says- here's the scappy command, I used here's the packet, I got back it's correct and we can take that and turn it into an automated test. It's just kind of um you know mechanical process at that time.

C

At that point, and you don't have to get caught up in the automation aspects, if you think that's a reasonable division of labor and a good way to use our respective skills.

B

C

Okay, so you know I I don't know if we need to have a group discussion on you know listing out the tests we ought to do, or maybe someone can make a proposal and say you know, I'm gonna do a few tests like this, and then we can try to turn those into automation, workflow and once we get some examples like that, it'll be easy to start, adding them right, it'll be kind of a rinse and repeat because people will see the pattern and it'll be really easy to make more and more um do you want to make a proposal, marianne or an offer of what you think you can do and when to get that process started.

D

Quick question to chris, um so chris here you are looking at only overlay um set of features or um on your content.

C

There's not a lot of underlay in the current bmb2 model right, it doesn't really have someone can correct me, but mario. I think we had this discussion once before.

B

Yeah right we were talking about behavioral model for dash. We are not really interested in behavioral model for underlay site, because it is tracked separately in the science community.

D

B

Have their set of tests etc?.

D

I agree um whatever underlay that is needed um for dash. You know just that part.

C

Well, there's the underlay in the final production versions of the dpu ipu right that underlay come from sonic right, but the behavioral model we're doing right. Now, really only has this the the overlay services correct, I mean there's no basic, router or other features in there.

D

So the overlay, I think cameron can of course add because uh this is his change, uh but overlay features are added on top of the existing bmv2. That also has underlay from sonic's side side.

B

What do you mean on top of it's? It's not really you. There is no really bmv 2 implementation of underlying site.

C

Put it this way, the bmb2 is just the engine that executes our p4 code. The p4 code has to have whatever pipeline features. You want underlay an overlay. Nothing in this serious pipeline directory implements underlay features that correct marion.

B

C

So you look for an l3 router, it's not in there. You look for an l2 switch, it's not in there. So there's no.

B

Basically, right what what size has right now is pretty much. Everything is documented in visio diagrams how the how the box should behave with respect to again, switching and routing and whatnot. There is no programmed way for the behavioral model in underlying psi.

B

It's all just just visio, basically,.

C

That makes sense, rasmus rashma.

D

um Yeah sure, uh just uh I I thought that uh you know uh bmv, two soft switch has been there for some time and then we um basically are extending it to add overlay features.

B

D

B

Not officially supported.

D

Not officially supported, I agree, yeah.

B

C

And if someone do that integration work, I don't know how easy or hard it would probably be quite a bit of work to take the bmv to switch.p4 or some equivalent and graft serious pipeline on top of it. But I think that's probably a pretty big task and I'm not sure that was ever a goal or requirement of this project.

C

So I think what we're trying to do is separate mentally separate these. We have the overlay features written in p4 literally and that's what this working group is focused on.

C

We assume there's an underlay on the final production version of hardware that will be tested in a sonic-like environment, but we have no way of using bmv2 to test underlay.

C

Now, if there's a p4dpdk soft switch, presumably it's it's the whole thing right: it's the underlay services we all know and love with serious pipeline or dash pipeline bolted. On top of it, then that could be tested.

C

So once once we have that we could test it in the ci pipeline and we could use existing ptf tests and better. Yet, ultimately, I'd like to be able to write tests that run it much faster in a line rate with significant flows using the hcsc or even hardware.

C

um I mean we can have that discussion when we get closer to that or we can have a separate discussion to start planning that out. Yes for overlay.

D

Right so for now, um our plan also is to you know at least we had discussed uh using ptf with um vmv2 overlay. You know when we have those test cases etc.

C

Well, we can, we can use um a combination of ptf and other approach, like I showed yesterday, which wasn't ptf strictly. It was a pie test, uh sorry, but it wasn't a ptf test and the reason is ptf doesn't support line rate.

C

So, if we're going to start creating a new body of tests, at least for our contribution, I'd like to focus on tests that ultimately can translate into line rate and have a path to that without rewriting the scripts- and you know maybe some ptf tests, but it doesn't have it doesn't get you past uh packet at a time.

C

So you know want to use our our resources to write tests that ultimately will translate into hardware tests, and you know you get two for one that way.

D

Yeah we can discuss um this in more detail in the other song.

C

Yeah, I think you know we can talk offline about that. I think I think the main thing is getting the test case, somehow represented in a way that could be turned into ptf.

C

If you want to use that framework or different framework that can go the line rate and figure out a way to you know get more than one outcome from the same initial effort, the effort being what's the test packet or packet streams, look like. What's the configuration of the data plane, all that should be common, regardless of which framework. So I think the real effort is in coming up with the test cases in general and then translating those into whatever script makes sense.

C

You know we can and we can cater to both types of or several types of frameworks. If we have chosen the test case as well, it's like marion has to drop martin. Is there anything you want to um ask or say, regarding this conversation before you go.

B

um Not really, I I got your message that we need to complete the first test with the full, with the complete configuration plus running packet, that has that is supposed to go through transformation.

B

uh This is my goal um I will eta is I'd, say roughly one to two weeks. I haven't started that yet um and yeah that's it.

A

I missed that, did you say one or two months or one or two weeks.

B

A

Weeks weeks.

C

Yeah that'd be great. That would be great, mario and that'll start planting the seeds for for doing more test cases, and it's a separate topic. I still need help with the sidewiff server. I hope I'm working on optimization today, because it was too slow um I'll be out reaching out for help with the scythethrift server from whoever is expert at that.

C

Thanks maureen.

B

Okay, yeah thanks need to drop earlier today. Thank you thanks. Everyone. Thank you. Bye.

D

I have to drop as well.

A

Okay, all right um cassine, it looked like you finished something up on your um er here.

E

A

So how should I update your hello, hi, yeah yeah.

E

Okay, so yeah, I think we have opened the pr so uh once the review is done, we can close that.

A

Okay, so just keep it where it is now, then.

E

Yeah yeah, I think that would be fine.

A

And so we're not really blocked we're back on track. Okay, yes, three got it. Okay, um oopsie is mar, is mario here, mario baldi, I'm only working off of one screen, so I'm having to flip guys. I'm sorry.

F

Yeah yeah I'm here.

A

How's the connection tracking going.

F

Yeah I'm working on it. I I've implemented some of the features that are in in contrail and I'm still working with some others. Thank.

A

You thank you. Thank you. Okay, all right I'll, just put still working on it check back in a couple weeks. Okay, um anyone else have anything.

A

Okay, all right I'll, stop the recording! um Thank you for your time this week and I'll post it to the youtube channel and um I'll clean up these uh new backlog items as we move forward and put more information into them. So uh thank you. Everybody.

C

Thanks christina- and I thank you, I can help you flush out some of those new items. If you want that, would.

A

Be fantastic: okay thanks! You guys have a good day.

C

A

E