Istio Networking Working Group, 9 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Istio Networking WG meeting - 2019-05-09

Description

- Discuss ways to reduce DNS overhead of Envoy STRICT_DNS clusters. Should we recommend against "resolution: DNS" except where explicitly needed? Should we allow unambiguous FQDNs?
- IPv6 update
- CNI update
- New process for the installer
- Istio 1.2 update and github epics for 1.2 items
- Testplan for Code mauve

A

All right, hello, everybody and welcome to this your networking community meeting today we will discuss a bit about DNS and ways to reduce the NS overhead and John I hope you're here with us, because you want to cover this particular github issue. I am sharing my screen, but it might be better if you actually share your screen and you describe a bit the problem.

B

Okay, let me share my screen, so basically, the issue that we're seeing is that pretty much any large customer has mentioned that they have way too many DNS requests and that they need to fill up their.

B

Their DNS servers and the the root cause of this is how envoy works so for Hany, strict DNS cluster, which is going to be the bootstrap clusters we create for pilot and tracing as well as anything with resolution DNS for service entries, so for each one of these clusters envoy will send a DNS request every five seconds by default.

B

So if you have a I think I haven't here like a strawman cluster with a thousand pods with ten server centuries, its envoy will generate 14,000 requests per second to the DNS server, which is pretty absurdly high.

B

So there's a few suggestions I have in here on ways. We can reduce these. The most obvious one is reducing the frequency of requests. I think five seconds seems extremely aggressive. I think those details for DNS over five minutes, at least and especially for a kubernetes services, because career noise services, the IP address, will never change unless you completely delete and recreate the service, even in during an update.

B

The IP address will not change so querying every five seconds from every single envoy for the DNS or the IP address of something that never changes is just doesn't make them so.

C

My understanding so I think like a five minute default. Ttl for kubernetes is the right thing to do, and what I don't understand is the implications for non kubernetes based platforms, I.

D

Would suspect it's exactly the same thing that same kubernetes, because we have talking about the beeps, not endpoints, so even raw PM's and and other things we still hit a stable address, not about them. Yeah.

B

If you're not using on, would you send a request and your client caches the dns request for finance or ever right, so I, don't see any compelling reason why envoys should behave differently even for non-korean services, so.

C

And you know I think the other thing is that there is there. We could set the default to five minutes, but there does be a way to override that default, so they're getting awkward some platforms, people still have recourse right.

B

Now, there's the global setting for the DNS refresh rate and it defaults to five seconds which is envoys default. This does not apply to the bootstrapping Fink, though we should probably change that. But that means that, even if you set it, then we're still going to be making requests for pilot and tracing every five seconds.

D

Well, overseas of the correct fix will be four, am going to just respect TTL from DNS server, which is what told DNS clients are supposed to do.

D

More complicated yeah.

B

That is a more complicated fix, but it is. It is weird to me that they do that as well. So can.

A

You elaborate on the reason why they don't want to do that. Otherwise, it's like complicated to fix. Oh.

B

I I think the intent of this strict DNS cluster is like you're using this as like the load, balancing pool and so I I guess, I'm not super familiar with with this, but I think like something like console where they use. Dns has like the service discovery, so they want to keep it.

D

So it's to do is the apt.

B

Mm-Hmm, okay, so yeah, that's some motivation. I have sent a message on this slack channel asking them if there was any negative impacts of increasing this from five seconds, and they said it's basically just up to you on how will in York have you know state law respond.

C

So so I guess the thing I didn't understand earlier about the the global setting is: can we can we tailor it just for the if we do it for five minutes? If we change the call to five minutes, what else is that gonna catch, but any.

B

Service entry, yeah.

C

B

You can create a service entry for an internal kubernetes service and for google calm, and it would apply to both alone.

D

Only only only cities that have load balance appreciate this beginners, yes, yeah, because evolution is actually done by avoid by by the application itself, and that is respecting TTL and everything else. If.

E

You did delete the community service and then recreate it on boy would keep the old pre created IP address until that time. Five minutes later. Yes, that is cracked yeah. So for.

B

A for instance for the pilot connection in the really weird scenario that you delete your pilot service. You recreate it then. Yes, there's going to be a period of time up to you, nuts, refresh rate that you may not be able to connect pilot, but.

D

You'll not be able to connect, because if you did it deployment, you will also get a certificate to Lussier yeah other.

B

Things would break if you.

D

If you do this kind of test, two disjoint I, don't see any downside to this I mean yes, there's some corner cases, but if someone wants they can change the default and override it. I, don't think five minutes. It's any problem with that. Do.

B

You think five minutes is the default. We should set quite.

D

Reasonable it'll cover most production use case is quite nicely. Okay and.

B

Maybe file about this.

D

To look over I mean and way to make sure that they just follow the standards because exist for a reason: okay,.

C

Sounds good? Do we want to do this for one one, six, which means like today.

B

It's the default, so they want to change it today. They can change a setting, but.

D

For sadist or epiphany we should do. We should use this five minutes for both default and for hideous.

B

Yeah, and so we should make this confirmable and the the bootstrap as well with the same setting, is that the consensus here, the bootstrap, would be referring to Pilate and tracing configuration. Yes,.

D

I mean is that that's got to go through the injector. So it's final, which also gets this stuff from the value from the global value signs to.

F

These strict inner square is only ever resolved. Subs know.

D

If for the bootstrap, yes for the, maybe.

D

So in the case you define a service with with load balanced attack. Dns, then it will resolve for normal resolutions that'sthat's how employee will do load balancing for those things and if you do it, if you, if you happen to set the address of telemetry or or I, think tracing to be a real DNS name that will not be a leap, will be a you know, normal external load balancers that me to use whatever.

D

So long term ABC should be a setting in in in in pair in in destination, rule and specific to the Zipkin or whatever service you have discovering I mean well okay, this mission will really, but is it worth adding it to the proper API I.

B

Would think most people would were just given five minutes or not, but.

D

Imagine it's it's of anybody. Use key is where you you are talking with an external service which depends on actual DNS doing the road. Balancing cause. Use consoles is ya, know, gonna, wear whatever all gonna sell.

G

F

The external things I would think that the respecting the TTL would be the right right weight handling that yeah.

D

But you can this employees fixing this stuff yeah.

F

Sorry, so is there an issue for envoy to do that? I.

B

Don't think, okay, all right cool.

D

G

D

One more second to preview side issue. We should still have the annotations that, because we have video with the convention, is that every settings that we have what most of them would can be overridden by rotation per foot. Yeah.

H

D

We should have one to cover this particular case where, if you have one specific or law that needs fasteners and we don't need to so.

F

Sorry, the annotation would be overriding on the client pod. Okay, see.

D

Also, we typically do a specified client on rotations that goes transmitted to pirated pirate. It takes into account interesting.

F

B

You okay, so the next issue is throughout our documentation, samples, etc. We use DNS resolution on our service entries for even very simple cases like you know, expose Google or calm, which causes all these unnecessary. You know DNS requests, should we change our samples and our standard to using resolution. None instead of DNS when it's not necessary.

D

That's interesting one, those impact will be sent and voting will just will not be invoked in load balancing decisions, but that client, who makes a decision on on balancing right, which we give.

B

Us so some more context on on the issues. What happens on a request? Is that say you make the service entry just for google.com, you do curl Google resolves the DNS is some IP address, and then it sends that request to envoy, which is also doing the same DNS resolution. So we kind of this double DNS resolution for no real gain, and so the client isn't going to do the resolution, no matter what so, it seems to me that it doesn't make sense to do it twice. I'll.

D

Treat it's worse than that, because a client made it off to one IP address and way, Mary's off to a different IP address and the original IP and sync it talks with that IP address, but can be ready to talk with a different IP I. Think that's a bargain mean it's not even intended. Behavior I think.

A

That's a very good catch and we should definitely change those except for the examples where we use the address directly because.

B

A

B

Yes, I think there's one example where, like we send it to like different regions or something so like in those cases, the this would be needed, I think for the standard examples we can clean it, muffin yeah for the others. It should be done, though, right I think it should be on be none like none should be the default. You shouldn't look switch the DNS. If there is a compelling reason, you need it right.

D

So for HTTP services, how it will behave, you have the hostname and the destination rule matches on or service entry matches on hostname and they need to apply resolution and to use original IP to to forward to whatever the client result. But it'll still use a hostname as young-ji. Okay, Janey's fine does.

A

This work for mesh expansion, when you have multiple pilots garrison.

D

B

Okay, so the the last point here this is maybe more of a minor optimisation. Now that we're going from five seconds to five minutes, but do the way that the resolution dns resolution search works and terere Nettie's. If you do a request for say, Google comm, it's actually gonna make six DNS requests because first it looks in google comm namespace Lester, while I press all these things before dad she goes, and so one way to avoid this would be to do a request for google.com dot, and so the dot at the end is I.

B

Don't know if it's called unambiguous, fully qualified Danny.

I

B

I made that up, but that will make it so it doesn't actually go through the full search, but we reject these in galle and pilot. So is there a reason that we should reject these, as should we allow them instead, and should we even recommend them or use them in our bootstrap? For example, it's.

G

Clearly above and it's definitely recommended by DN a-- standards,.

D

And they're clearly for Indianism solution or deletion, yeah.

D

Of course part. There is no question. The second one I think yes,.

J

One or two yeah.

B

Yeah all this will definitely one one or two, not one, not one that would also using the unambiguous domain name. I think would also resolve an issue where I think you can kind of do some hijacking. If you create a service, say google service and the name space comma, then the DNS request for google.com would resolve to that instead of the rule, but that's probably a minor security.

D

Okay, that's a very good.

G

Right, that's a kubernetes for.

D

But that'll make it candidate for one at one sink and the fact that we don't follow the DNS standard, which is all the standard in the books, probably also mean that is a validation part, could also be fixed, involving.

B

What do you mean by the DNS standard? I mean I've, never in told yes like a week ago. I never even knew about this period at the end, make.

A

This, like a critical baggage, I, think it's a great fine, but not something. That means like.

I

A

It's not a bug right, it gives definitely optimize things, but it's no.

D

No suppose that we reject valid DNS names is about coming google.com, dot, isvara DNS name. Oh.

B

Yeah I think that that is true right.

A

But I mean it's a bob, but it's not like that bad.

B

Is it recommended to use these, though, because I've I've never even seen these uses? Why yeah.

C

It's actually a customer issue. It's.

K

Not recommended with it's part of the era's it's like for 19.

A

Question is why nobody tried to use those in your configs right. Yes,.

B

That's my question too: I'm, not a DNS expert I saw this nice I tried it out and it reduced the D request. A lot so I'm wondering.

D

B

There a downside to this I.

D

Can tell you mrs. a meaning.

I

Observed is that you.

D

Should you should not try to do is hope that a number of tries those domains first and I. Think if anybody said your DNS resolver to to use top-level do not know.

B

Cuba nari search space, it's gonna break a lot of other things. Yeah.

D

I'm big supporter of standards, so HTTP in the guiness standards are the ones that are.

B

Okay, so something like the action items are changed. The default to five minutes make sure the bootstrap also uses the same. Config recommend resolution none and allow unambiguous qualified domain names. Is there anything else thing in the comments where.

F

You also own the Envoy issue, yeah.

B

A

Do you mind writing down also the decisions here? Yes,.

H

A

B

You guys make me an editor. Potentially sorry did you make me an editor on the networking group talk, so you don't accept my recommendation. Oh.

A

You did enjoy the team, it's your team drive access. We.

D

Can take it up, man, okay, yeah, I'm, sure.

A

Because you should have had edit by now, okay good, so anything else on this any comment or anybody else would like to comment on any of these topics.

A

No okay, good, so a team eyes, I notice you joined then I. Remember last time you wanted to provide an update and there was no time would you like to do that now sure.

L

I think I would. Let me just refresh my memory I think I was just gonna, give an update on ipv6 yeah.

I

L

Yeah so Sergei bit Sparky and he got he got all the the main feature. Prs merged and I was adding something. I have a PR for ant and simple tests. We added the functionality the same ipv6 tables functionality to CNI.

L

That's that hasn't merged yet, but basically end-to-end simple, full with my musics is passing with with the test case changes and the test case. Changes were just really to not do stupid things with IP addresses, so no assuming ipv4 all the time. So we we have turned on the the CI for that.

L

So Costin would like us to try to implement a model with in the installer so where we have the ipv6 n 10, simple testing triggered by the installer, make files and then I guess, there's a plan to move to having the end-to-end testing pull the ins like circle, see I, pulled the installer repo and then use the change set from the sto repo to build container images and then use the Installer to to do the installation.

L

That seems like a multi-step process, because you'd have to make files first Yoda to leverage for the end-to-end test to leverage the installer. So there's there's a lot of I think changes that need to happen there.

L

So in the meanwhile I've got a intend, simple test on ku Badham din. So that's the doctor and docker variant that runs with ipv6 on circle CI and that's in this in the CNI repo right now so I we we can I'm gonna I want to get that figure out. What the long term strategy for the Installer testing is so I can get that integrated there. So that's! Basically the update I can.

D

I can also few of your points here. Doug has a PR I, don't know if it worked or not. That is our being the new installer to Easter Easter for some insert tests. Okay, desserts current is a priority to me to to switch all the or most of the testing in history Co to use an installer, because.

I

D

To know some installer on the second thing, I think you already have a Pierrot podium in installer to integrate the cni project.

L

D

We need to finish it up with a few comments there and and then we can merge it and the plan is to have the Installer test, multiple profiles, and definitely we want to get your dint.

C

D

Because we already have kind, we have mini cube. We want to the Installer to be testing as many different environments and CI series, as it is possible right.

L

Yeah, the the din of stuff for kuba 84 KDC, is what it's called that the only really the main reason I'm using. That is because it existed before and it had IP and it already had the ipv6 integration. It's kind looks like there's been some progress with ipv6 integration, but I it's not merged, but.

D

Even even if kind has support for every six I think it's very valuable to have to have at least one test for each environment, so I want to order. The Mini Cooper test running as long as mini cube exists because some people may choose for whatever reason to use mini good. Some people may choose for I, don't like to be kind of monoculture and have just or just kind.

K

D

K

D

Will still be unkind, but it's very good to have at least one test with all the other options that exist right.

L

Okay, yeah, so just to make sure it's clear. Also, the the CNI pr2 is tio installer that that actually doesn't have any dependency. Ipv6 doesn't have any dependency on that, so it's so the the stuff I would be proposing would be a separate PR and it was a like totally just the test, not not a new installer functionality, because the SPO installer doesn't have to do anything for ipv6. It's just going to be on an ipv6 infra and it would just be. How do we trigger the KDC?

L

D

Prepared and if you can do it soon, I'm hoping to get after the one one, six, five whatever 86 is. How would I want to ask the release manager to start cutting the candidate for one 11.2 and so I can get more experience. I mean you can start early testing one on the new installer. So probably can wait. I mean a few more days to get one on five out. Oh, you want to have to listen same time, but the idea to have the cni parts integrated.

L

Yeah I'll try to get the CNI done first and because the the CI part for ipv6 in the Installer, that's gonna, take a little longer because I I gotta take all that circle, see how you.

D

L

And put it into a make file, so let.

D

Me know if I can help I'll be happy, and one thing I want to mention on this subject: I do not people noticed distressing environment working group. We have a new process to install the demo and to install demo profiles or kind of specific configurations of issue with customize, with single line. Cube CTL apply K github installer, and it would be great to have one of such profiles that is defaulting with CNI okay, so people can have kind of prepackaged. Pre-Tested I mean whatever we test.

D

We should also support users to install it as a profile. Basically, I, don't know what is a proper name for it, but for now we'll call it profile.

A

D

Was this was in the you know, meeting notes for environment and it's a wiki page in the Installer? Okay.

A

And it will have to update those set up such so.

D

The documentation will get some decorators and people who know about dogs will need to translate the reynolds with the proposal and the wiki page to get people started, but once it is tested and if the community accepts it, there's always a to be documented master of wanted to release. They have one month to write the dog. So it's not, but testing comes first. So.

H

A

Try to link the.

G

A

All right, okay, great things- I mean very much team. I have also a couple more announcements to make so I'm, not sure if everybody has seen the communication from francois related to release dates for one which is June 19th. That's the current plan and there is a by consequence. There is a release, one branch that will be cut on May 21st.

A

What does this mean? That means that every every feature or bug fix after May 21st that will need to be released into one too we'll need a special approval from the release managers. So, ideally, if you have PRS, you should try to get the merge in before that date and because, if not, there be like more more work to do also, let me actually share the list of the current github epics.

A

So it's linked from the from the meeting notes document, and these are the epics that I currently tagged for milestone. One two and you see there are like UX improvements refinements to namespace isolation. This is basically refinements to the sidecar API. There are a bunch of issues related to security of East. You know, there's a big one for each geo, multi cluster, which has way too many issues attached to it.

A

I'll show in a minute and obviously I I, also added the ipv6 support team answer game, mainly because it's almost ready right so I think we can safely target forum for 1.2. Let me just go inside the ipv6 support and show to everybody how this looks like. So this is a big epic of grouping for all ipv6 features and look how many issues were fixed in that area. Alright, so all these are closed or fixed. Are they close, are fixed, I, think they are closed by being fixed, or is that correcting what it shows here?

A

L

Think they should be yeah I'm, not really something I haven't describes any of those issues.

A

Ideally, they should show us well.

I

I think a lot of them are fix, but we probably have that some might have been closed. Just I think stale BOTS hit a lot of those okay, but I think a lot of them are fixed, so.

J

A minute actually, this problem, oh yeah,.

A

Doesn't what may have happened like the stale bot 20 and closed all your issues and your property or PR probably did not like fix them.

I

A

Way to work around the stale bot is to mark issues as enhancements kind enhancements. In theory, there.

I

A

A longer time like there should be a few months until that issue is being back Thursday right and there is like you have a new one that you know. Somebody complaining that there are too many ipv6 DNS queries.

K

Can I add a small current to what just said about my my communication, because I I'm not sure it's very clear if you have been in the past like if you have been working on an issue that was tagged as one dot to verify? If you really wanted to learn in one day to verify, it's currently still marked with this milestone because most of the 1.2, my stones, I, have been moved to 1.3. So that's part of the of the communication and I think it's very important to to not yeah.

A

Yeah so for some of these actually moved myself just today, for instance, let's take this one. This one was originally like: it had no milestone. Oh, it still doesn't have a milestone. So let's say we go here and we target to milestone 1.2 and if you are working on an issue and it shows march on 1.3, but you actually want it in one. Please go ahead and change that.

M

And a quick question: if you could go back a page yeah.

A

Do this go back to the epics there.

M

You go there's an epic for the environments, work group that we work through and I, don't think it's captured here. How are you filtering? Is there something that we need to set? Yes,.

A

Yes, so what I show here is just epics that are tagged with area networking, okay,.

I

A

Go here and my to filter a big and area networking if I change it to be environments as well, it will pick up your know it. Okay, now I change them all. What happened? Oh you guys have no issues in I know why? Because all your environment epics are probably in a different mind, stone or something yeah.

M

We'll get that straightened. Epics thank Sandra. You had mentioned the security sup and you know some of this stuff is cross-cutting and especially that multi cluster gaps and issues you know I think most working groups are going to be right.

A

M

That I do agree when you need to trim this a little bit. Okay,.

A

Let me actually show you also what we've been doing in the past. There are some cross working groups epics and issues, so we generally tagged them as both okay.

M

A

So in here, for instance, let's say multi cluster is being you know, has work done in both groups. We tagged them as post and some issues will be networking specific. Some issues would be environment, specific and, of course, there'll be security issues as well. Right sure this is more like you don't wait to filter it. It raises visibility on. You know how we're progressing towards the milestone.

A

I want to also show like the UX improvements, for instance, because that sounds very you know generic, but it has some of the things we been wanting for a long time like the elimination of dependency on port name for protocol identification. That is the part where you need to name your for with HTTP something or TCP something to be able to infer the protocol or this one which requirement to declare the container port in all diamo files. But most people don't have that because it's an optional field in kubernetes, but this sure requires it.

A

So all these are tags for one two, however I'm a bit concerned, and some of them might not make it just because I'm looking at the issue- and this is not assigned to anybody so I'm gonna- follow up on this- to see exact. Who are the owners or, if you, for instance, you know you're working on a specific area or item or issue, and your name is not there. Please put your name there, because that will probably raise the level of confidence. This gets done in one to also I.

A

First, this for security of East Europe. This one doesn't have yet issues inside it. It has just a list. So if there are like multiple people working on these topics here, which could create like separate issues for each of them, does that make sense.

M

Yeah and where that totally works for me and then I think my questions for you would be already's. P0 is stuff. We need to be working on at a p1 and p2. Is that your take.

A

Yes, definitely so p0 is the must-have write p1, it's it's not mandatory.

A

The thing is we I think we again mark too many things at p0 and given that we have the cold mode initiatives as well, some things might not make it, so we I think we might need to revisit the urgency and the criticality of some of these issues for one.

D

But in practical terms, I believe this. The plan is to have this release be date, driven meaning that, on whatever date was discussed, said that will be cartons and box exfoliant and yeah.

K

Yeah I think yes will automatically.

D

Go to one to three things.

K

D

Could is a strategy.

K

Yes, like on the on the date, the branch will be kept. Most of the ducks will be accepted during the first time before the release yeah. If it's really really a bug, there will be description. Obviously it's not like if there is like yeah, a great picture that can hands- and you are like at least on this- we will discuss, but mostly that's exactly what was described after other fries burger could be accepted, which really means that.

D

It doesn't matter what but I mean if the code is in for the feature by the date. Is there no problem? Otherwise, probably.

N

Would have to sleep.

C

Sorry, sorry, sorry to be cranky about this yeah.

N

C

We slam a feature in that's not ready before the freeze. We will addicted. It's gotta be well tested in every session.

D

To make that if something is ready and by by the please date is good. But we don't want to kind of put p0 and says that it must go in in the last moment and.

K

It will probably be a description around the the I'm trying to keep an eye and like a huge test, brought on the on the priorities right now and like that would be description about what is a physio on in the day preceding the the freeze. Obviously, that's.

M

Their call that if there's no test cases and it's a feature, it's probably get reverted, probably wondering yes, yeah.

A

And maybe I can use this opportunity to introduce Francois, who will likely be the release manager for one to along with lean yeah tomorrow.

C

A

So that will be like you know, voted tomorrow in the TOC, but I disagree. So just in case you, you suddenly need to find out facilities manager, it's very likely it would be France. Finally,.

C

Francois doesn't like beer you'll have to find some other way to practice dodge real quick. How did you escape France all reports to me guys.

M

New our works now, okay,.

O

Josh, you, you are a manager, I didn't even know that.

D

Why did you know about this I.

O

Don't know Josh is a manager. I do know about this.

M

Captain Kirk shirt on so clearly he's a leader.

C

Joshy by the way, it's not it's, not I'd William Shatner's John Belushi, but it is a Captain Kirk shirt yesterday and I.

D

Think, whatever words that nobody will be release minute for more than what it is so so far we stick to so that's.

A

Okay yeah so in case again in case there is some epic or some big area. I have I, don't have here. For instance, I don't have one for CNI I think it would be great to have one for CNI, especially if it's you know.

O

A

Show you know the just show bit when when we will write the release notes, we can have. You know a big big big feature and then, like the smaller features as well.

D

For Siena I think it's for do it was got completely in one but one and we are not running any major changes. I'm accepted ITB, six stuff. No, we just.

L

Yeah, that's I was gonna, probably lump it under that ipv6 support one, maybe or what we've been tracking the issues and the other repo. So the DNI repo yeah.

M

And also there there's an install kind of epic with a lot of work that needs to be done. Where did sort of product at least.

A

For the environments, working groups, territory.

I

A

Be great, if you could that one there, it just helped us be organized. Oh wait.

M

We already have that big, but so what's your request and I'll make a happen.

A

L

I, don't have a request, I was gonna, just say, I that epic exists exists and it I think it was under the install working group or environments.

D

We should ask us if you have the necessary, after after other testing, one smoochie discusses if we should make CNI the default is a or recommended or something similar. Since this modernist or again, it's it's, not the user, which one which profiles they just reinstall, but we should discuss it commendations for this. So.

J

So so Andhra just to clarify so the big things for CNI is the install, but that's being tracked under the install epoch, the ipv6, which is being tracked under the ipv6 you're. Looking at now, and then there was a separate effort which won't make one point to to possibly have envoy be out. You know in its own container, alright so um pot, but that's not gonna make one point to so so I about in summary, is I. Don't think we need a separate seeing Hadi, I, think of any type or covered under the other yeah.

L

J

L

A

Maybe, since we have on a few more minutes, I wonder how are you doing with the code move because there are like every working group will need to have. You know the test plan and the test cases right?

A

I haven't been participating in the latest discussion, so like Josh or costing. Do we have like some requirements that are percolating down? We need to take care of.

D

Requirements for sure yeah.

C

D

C

So I think every working group is supposed to create a test plan.

H

A functional test plan.

C

And then I think Oz and some other other people are working on tooling to actually associate the test plan with specific test cases in whether they pass or fail so Louis a first stab, actually a networking test plan, but you know I think he like got like you, keep put in like half an hour right now time. So it's it's far from concrete. This was something else.

C

I was gonna, bring up from the TOC tomorrow to basically say: look we need to start having something like a weekly meeting with working group leads to really get movement on things like the test plan. Some areas have filled it up. Others have so I'll link in the tab to this. This document right now. Okay, if.

D

You remember in all possibilities we had after these countries were cut. We had a manual testing where everyone is working groups and everyone working. These two are supposed to do manual testing, based on a spreadsheet.

D

This test plan I mean it's wonderful to have the spreadsheet, but it will require everyone to start. You know kind of covering everything. This is a test plan. Yes,.

I

D

Least man worried because again we cannot automate everything and ideally with an automated test, but it's an improvement over what we have the previous releases with with the manual testing.

A

Okay, so that means we might need some like communication sooner than the next meeting, which is in two weeks.

A

Yeah because we may need to like basically people will need to volunteer to help with the tests here right. So we all need to review the document and see you know what exactly is being proposed. What is it that we don't have covered yet yeah.

C

If you want to open it, you can do that now, just the linking okay.

A

C

An idea what's going on so every every group has got a tab here and I think the one that's gonna open these. Actually, the networking tab so.

I

You can get.

C

An idea and the idea so just think of high-level bits of functionality. You know CNI is a great one envision that earlier and- and you know, key functional tests being vague here. Sorry.

C

So for each of these areas there will be many specific test cases: okay,.

A

So this is more like high-level, then yeah.

C

D

Low level, in the sense that each feature has a list of tests that are required, mm-hm.

A

And do we need to implement these ones using the new test framework like? Are there.

C

What it's, what, whichever whichever test framework, is most appropriate, certainly I, would say new integration test framework is more appropriate than the old one, but some.

I

Of these might be better, it's unit tests.

C

Some of them might be, some of them might be better as performance tests and.

D

Some of them, what scalability or what kind of requires the scalability framework.

A

So it's the wrong thing. It's very interesting because I see actually see very few features test being covered so for wrong thing. We we just showed the prefix match here. It's an example. Just a minute.

D

Yes, oh yes, oh good, taste.

A

Detailed dancing somewhere about the routing features a multi cluster.

C

Yeah, so it's it's the responsibility of this working group to complete this plan.

A

Okay got it so I'll try to follow up with Louie or.

C

Even take time out of this meeting and everyone can propose, you know, or both yeah.

A

But it's just don't worry about if we like, we meet again in two weeks right. That's already passed me.

A

And it looks like we're not doing well in yes,.

C

I agree that we should put more effort into this. Everyone can can work on this too, on their own offline, but the code mob end date is not the same as the one two enemies, a codemod. What has been stand, multiple races.

O

C

O

Approved code Mach related item into 1.2 yeah.

C

That's right, if it's a test case I, think we're just gonna merge that yeah.

O

B

So should should should I be like going through here and adding features or should I propose a feature and then andra or someone else just pop them in that has it I.

C

Like that, what do you think just.

A

Add it yes, don't like do we don't you know when it's about this coverage, you don't need the Provos I. Think.

C

We need triple RDBMS about that.

C

D

The process was that touching, okay put our notation somewhere and so.

A

Let's see yeah, let's try to do them to add things here and then maybe we'll review it in two weeks. How about that?

A

Okay, all right! So are there more. You know, questions in people's minds or big things to discuss. We have ten more minutes. We can also adjourn a bit earlier.

A

Going once going twice, oh all right! Well, thank you very much for joining and see you in two weeks right thanks.

I

O

I