Kubernetes SIG Network, 25 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Network Bi-Weekly meeting 20210325

Description

Kubernetes SIG Network Bi-Weekly meeting 20210325

A

Welcome everybody to the network plumbing working group meeting for thursday march 25th 2021 over to you doug for regular business.

B

All right, thank you, dan um all right, it looks like we do not have any uh member or maintainer candidates, but we do have a project candidate um uh tall thanks for joining uh a while back tall had um given a presentation um at the multis maintainers meeting, I'm pretty sure was the one um where he had come up with a uh multis cuddle utility um that had a number of features, but the the one that stuck out to me at least, was that it provided a way to uh bootstrap.

B

Your cluster, with with multis tall, has also worked on a number of other that attached f related um work, um so um one of which is called nap k-n-a-p, which uh is like an abstraction layer for net attached f's um and you know, provided a way to uh like dynamically uh generate net, attach devs in a opinionated fashion was pretty cool um but anyways.

B

That's uh just my preamble um tomo suggested that uh uh well that maybe we could uh have some maintenance help for multis cuddle instead of having it as part of the main uh multis repo, it could have its own lifecycle and uh development trajectory as well.

B

um So we thought network plumbing working group might be a good home for it. um Tall is there: is that a good intro or is there anything else you'd like to add.

C

um Hi uh thanks. That's a good intro um yeah. I just uh added the links, uh so you guys can check check it out. I did uh the tool itself because I didn't really know what its future would be. It works fine for my uses, but they're, probably things that could be added. I do make sure that it's updated and tested with latest versions of multis and latest versions of kubernetes, but otherwise um I haven't been adding uh features. You can look at the readme, the link I can share my screen too.

C

I don't think I need to I'm sure you guys can open links and you can see what features it has so other than installing multis and by the way it installs multis programmatically, uh doing the same thing that you would do if you use the manifest are included in the in the multis repo that are the uh recommended way to install. It does, however, now only use the the default installation manifest.

C

I know we have a bunch there, there's ones for cryo and um I think two other environments, so uh this could obviously be fixed by just adding some sort of cli flag. That tells you which version you want to install um yeah. It's a work in progress, there's more features to be added, but um if it does get uh accepted- and uh you know, I'm stepping up to uh maintain it and make sure that it's up to date and uh in line with uh the multis project.

C

So even though it'll be a separate repository, um I I'll I'll make sure that it's it's in line with the main repository. I I do have questions, though about testing and how you do integration testing for multis, and if you want, if you would want the multis cuddle utility included.

B

Too uh yeah, I would definitely be interested in uh adding tests on either side or both if it makes sense to uh do. You know an end-to-end test um so that changes in either repository um would be reflected or maybe even something like uh like a nightly run so that we'd get some. You know uh some like canary type output that uh something something had gone awry um but yeah. It's.

B

We usually do it project by project and it's up to the uh maintainers uh and as per the rest of the universe, it seems that uh github action seems to be uh the darling of the moment, um but yeah I'd be happy to work with you to uh um get that kind of stuff together for sure.

C

D

So I got a quick question, so I think um network plumbing working group is would be the right place for this, because that was one of the initial statements for questions um but a question: why would this not go directly into the multis repo, because I mean if it were to be used for other things? I could see it as its own, but since it's directly tied to multis and wouldn't you want it like right there so as new features are added to multis people would say. Oh, I gotta update this too.

D

To make sure it's added, or it's just more of a question.

C

um I so billy I would tend to agree with that. I think that um one of the things that multis might be missing right now is a download button right. If you go to multis's homepage, there's you know instructions on how to install it and, of course it would. It would download the image from.

C

uh I think it's from docker hub or quay. I forget where uh they're stored by default, but um uh it would definitely be friendlier for people, I think, to have a command line utility to get a quick entry, but um um yeah.

D

Okay, so I guess it's more of a dog tomo question and I'm not pushing one or the other just seems to me that if it's directly tied like that, it might be useful, have it directly within multis. So just thoughts on that.

D

E

I'm thinking that so currently the. uh How do I say so, looking at the istio or looking at the other stuff, so they are like, for example, the cuban so well, for example, the uh istio is, uh as far as I know they are under the east organizations.

E

There is lots of lots of repository and they each deposit. They have their tools, I think so the and then also they are, even though they are cube, kubernetes, the cube adam.

E

As far as I know, maybe they also have the separate the repository and I'm I'm just thinking that so and then looking the, uh how do I say I would like to uh divide, as in the small granularity, I mean that so the mouth uses the network attachment definition client and which is in the separate repository in the same organizations, so they are the most so that they as far as they remembered I nowadays I don't touch with the uh modest couple, so they are.

E

If I saying wrong, please uh they correct me, but also the uh mountain scuttle is helping to introducing their models or they are touching with the nether touch depth.

E

So this means uh the mouth cutter may downloading the binary, but not the uh directory called mountus api so that maybe the multiscuttle may call net attached definition, client, so yeah, I'm so there, maybe that's why the so that we could saying that from the functionality point of view we could be making a separate repository, and then this also the uh how do I say this makes the easy main thing for uh updating some vulnerable vulnerabilities, the uh library right.

E

So if some some library is the target and then we may need to only one or two repository- not stay a big repository. So that's that's the uh my opinion.

D

Okay, I'm just afraid it might slowly get out of sync, because features are added to multis and it's kind of a side side repo, so it doesn't get hit or updated as often, but I can go either way. It was just more just one to open it up for discussion.

C

uh I think tomorrow has a good point: good points there. um um You know the multis ecosystem might continue growing uh beyond just a cube cuddle. There might be other things as well. So um you know if this is the first time that the ecosystem is growing with other tools. It could make sense to make this a rule of thumb that uh you know new tooling, would be a separate repository um as long I I would say this is fine from a user perspective.

C

As long as you know, we update the main multis uh readme that uh would direct you to that project to download as well, and it's probably worth discussing too looking at the motors documentation and to see if it's worth adding examples using uh multiscore.

E

So uh how about to so I'm just thinking that yeah we are going to have the uh some. How do I say, top web pages for kubernetes network pumping uh working group, because now, as far as I counted the india two months ago, 17 or 15 repository is under the network prompting working group so that maybe we could add the uh new web pages, for how do I say top pages for the amount of words this stuff and then we could add the assam.

E

uh How do I say more generic general document for beginners to get in touch with our component? Not only the amount of scatter, also the uh mouse network policy or srb stuff as well, so they are so so they from the uh motors repository side, the eye drag to how they say, I'd like to describe the arrhythmia describing only the of things and then the and then now nowadays the uh we also uh have there's some questions about the uh the uh quick start, their description, so yeah. Maybe now is a good time to moving.

E

So so, let's, let's say, let's think about the first- create datum uh top pages for all these stuff and they're. Adding the quick start of this stuff and then removing the quick start from the modus readme file. Maybe this may solve the id stuff and also there we, we could add the quick start for each tools. I mean that they are my cheap cutter, multinet commands to easy to showing the ip address and then also the easy to uh see the how to install srb uh device browsing all this stuff as well.

E

So what do you think about that? I'll? Do.

D

Go ahead, billy, I'm.

B

Just gonna say.

D

Yeah, I think that's a good idea. I still I mean- maybe not have a full like paragraph or page talking about it, but at least I I think you still from multis need a back pointer to the to the other readme just so that someone who's new and trying to figure out maltese, who drops right into multis can find this and other like quick start. So, yes, I I totally agree with you tomo. That would be good to move the.

D

Maybe the quick start up a level, but I still think you need some pointers in your readme to both the quick start and this multis cuddle project so that they can find their way over there.

B

Yeah and I'll give a plus one that I think that this uh contributes to um like uh lowering the barrier of uh of entry, and I think it does follow a model that we see with these other tools, like your steel, cuddle and cute cuddle, that um you know, here's a here's, a utility, that's specific to what you're working on and instead of being like hey, I have to introduce you to four different concepts in order for you to use this, it's like here's, a util, here's, the high level things that you need to do it.

B

I think that those are really good and, as tomo mentioned, we definitely you know, um need some. You know higher level entry-level um documentation, and I think that this definitely contributes to that. um I um yeah going back to the um to the repo question. I think that maybe the thing to do is to start with it in its own repository.

B

If, for some reason we see that there's like a lot of drift between the two, um then maybe we would think about a tighter integration, but I kind of like the idea of letting them have.

B

uh I think that the like command line utility is going to have kind of a different set of dependencies, and one thing that we may um consider doing too is improving the use of multis as a library um to like meet the needs of something like this um as need be, and then, for example, there are certain things that we already do have you know modularized and useful as a library like the metatap net attached, f, client library, for example, right.

B

Anyways, that's my my thoughts on it but yeah for sure. I think that one of the goals would be to improve the introductory documentation on whatever level that happens to be so, if that's a new documentation that comes out of the network work group, um if it's just the like introductory paragraphs or the quick start on multis, that's great but yeah. I like the idea of that, and I do like the idea, especially of having, like a network plumbing working group, introduction to this, that um maybe gives you a like opinionated path for like here's.

B

You know how you may move forward with this, something that we so one of the kind of the challenges with multis that we've sort of changed in our documentation approach over the years and the requirements for the quick start and the quick start really is a quick start.

B

It gives you an example of how to install multis, and we figure that probably in most production scenarios, you're going to need to customize that installation to match your situation, something that we used to do in the quick start is we would install flannel for you as a default network and it's the default network that is kind of the most challenging part from the um from in the multistocks, and it's like one of our most popular community questions is: how do I do this with uh calico? How do I do this with flannel?

B

How do I do this with this and the other thing, um and so we do have our reference deployments um repository, but maybe there's um you know. I could certainly see uh multis cuddle kind of easing the pain for some of that stuff. Where you know you could go to do, multis cuddle, um you know provision or whatever it happens to be, and it could tell you hey, you don't have a default network.

B

Do you want to visit this webpage and we'll give you some options of what you can do here, um something like that um which you know the multis daemon set as it is now it's like a little bit of a chore to uh actually figure out that information like it's like, um not so straightforward. It doesn't just tell you that you don't have a default network. It.

B

Yeah, um so these are some of the like pain points, and I think that there's something that could help out there yeah.

E

Oh good, so let me add the additional the uh feedback from the community side, so so the uh yeah uh so so currently motors have the quick start, the section for the stuff- and sometimes this is there some you guys asking me about the hey they're following the quick start and then we'd like to do a maintain or upgrading or some various the uh life life cycle, and how to do that so yeah. So to me at least the quick start is not just a quick start, not the perfect step to everywhere.

E

You know, so. The uh the quick start is just that for testing the models of this stuff. I mean that, let's so let I install the kubernetes is a small cluster in the wrap or some stuff and then attract at that time. The let's use the quick start, but in case with some production case, of course the production needs to be. Concerning lots of stuff. I mean that maybe they have the different directive for cni and then also the different cni conf directory and then at that time, of course, the uh for each deployment.

E

They need to be changing the configuration and then the quickstart is not supporting such they're complicated or the.

E

Diverse configuration point of view so sodium. I hope that the uh the such the amount of scuttle things is helping these guys to deploy that this stuff uh is the uh the one one possibility, so that.

C

So uh thanks I'll add the two comments, so first uh uh so doug. As you pointed out- and you know I- I don't have that much uh experience installing multis on uh uh various environments but you're. Looking at the the install manifest here, you're right, um they should probably be customizable. You know if people want to work with different name spaces or uh certain container params.

C

So my thought is that the cube cuddle, install command, which right now is very simple, can add a lot of flags that let you customize how you parameterize that installation right. So you could uh they'll be, of course, defaults, but um that those defaults could be changed. So that might be a friendlier way to install rather than editing manifests that um you know always worry people that they're kind of moving out of what the the standard file is right. um Absolutely, and I mean.

B

And a number of things I think, could be auto detected too, like tomo mentioned, like hey, where what's your cni copter? What's your cni vendor stuff, like that, um um I think that it would make it would increase the like a successful, install percentage for sure.

C

Well, well, great, you know I'll I'll rely on on you guys to open issues for me and and tell me uh or or contribute prs.

C

I think the code is really straightforward um if you want to improve uh another point uh going back to the repo question, um I just noticed that um the multis cni repo has a godot mod at the main directory, and I imagine that is for the operator right operators, um so so that is kind of a non-starter already for including both the ctl, because, as uh you noted doug, you know they might have different requirements. So if we do include multi-ctl, you know it could be a subdirectory with its own godot mod right.

C

It starts to become a little bit non-standard in terms of uh how go laying repositories are usually organized, which is not necessarily a problem, but I'm saying that if, if we decide to put multiple uh outputs to the same repository, then we probably want to create subdirectories with each with their own godot mod, and then you get into the question.

C

um So this is a question I have so um is: does malta cni the repo itself right now expose a go api that maybe other projects can use, so people can import, say the project and call apis or or is that just not a use case that happens uh so.

B

It certainly happens uh from srov perspective, um and it's used that way and also tomo has done some kind of forward uh looking uh uh work there by using uh go package.in um to to make that work a little bit more smoothly.

B

So it is possible, but I I would say that it's one of those things that on my list has always been. uh This uh could use uh some more improvement. Also, another thing that um I have uh considered previously- and um you know I'm not sure how much this has for legs, but um in the context of a container, runtime or maybe even cni 2.0.

B

Is there a possibility of like a more direct integration of multis into um something like that, as opposed to just running it as a cni binary, so that you could you could do the work uh you know like in a more integrated fashion with um cni itself or a container runtime. um So yeah, that's a convoluted way of saying. Yes, you can use it that way, but doesn't need improvements probably, and it is used that way as well in srv. That's the that's the gist! Well,.

C

Well, I think that's a strong argument, then, for separating the repositories, because um you get into versioning problems right, because if anybody is importing from this repository, uh the go.mod file will have the version via the tag. But then you know, if I, for example, need to release a new version of malta's cuddle.

C

I would be tied to that tag too, so we would be stepping on each other right in uncomfortable ways.

E

Well, but they are still wondering that the- uh how do I say uh the having the lots of components in the one repository is the lots of dependencies here right. So this means the. uh So, even though there are someone is trying to they're changing the cnr side, then maybe they need to touching the other report, their their command as well. So this is kind of the monolithic repository, not the monolithic binary, but the monolithic repository is the kind of the painful at least to me.

C

Yeah, I I tend to agree I'm worried about the versioning, especially because with multiscuttle I use the truly excellent go releaser tool to release my binaries, and it does so much of the work for me. But the go releaser tool really relies on the fact that you're pointing it to a repository. So I think it could work with subdirectories as well, but it gets the versioning from the get tags so uh yeah. I I just think it would be cumbersome.

C

It would require more coordination if the uh if we're working in the same uh uh same repo and and by the way, just I'll notice something quickly. I'm sure most of you know, but uh with go the go. Compiler is smart enough that, even if there are all these requirements listed in go.mod, it's not going to be automatically installing them. It only installs what it needs in order to compile the code that you're actually using.

C

So so in that respect, having a go dot mod that has that in a list of 100 different requirements from all the sub projects is but.

E

Itself is not a problem, but on the other side, for example, the mountains or the kubernetes, they still keeping the bender directory under these stuff. I mean that once we are changing the goal mode and then, of course, we do a go mode, bender command to introducing via the this stuff, so they are yeah. If you, if you adding the bender directory and the cube uh the multiscuttle, of course, the you via this code, their code is close to this stuff as well. So so right.

C

E

So they are so that's why they are they're staying sunday. I I also thinking that the mothers cut so how they say uh well. This is not not the real the uh issue, but also you know so nowadays, microservice is the getting the familiar or the famous right. Maybe we should do the same things in the command line or some stuff microservice repository.

C

Oh yeah, oh and and by the way, just looking again at the api and the interconnections between the different projects, so multiscuttle actually does not import anything from the multicni. So this.

E

Is just the independent from the models right.

C

Right but it does import from the network attachment definition, client uh repository so just to.

B

Point out that right, that's that's awesome! That's the that's the way I think. In most cases it should work. Is that yeah that you that's that's probably where we want to library eyes as much as we can basically right.

C

I guess my point is that we, I think multis already has kind of a mini ecosystem. I mean technically the network attachment definition. Client is independent of multis right. It could potentially be used for other things. um So I think we we already seem to have a multi-repo uh ecosystem here, yeah yeah right.

E

Yeah and then they are from the uh so mult network policy repository, that is, the uh pro provider network policy features for net, attach dev, and then this also includes the net attached dev client, as well as the other stuff, so so that we we have the rivalry and also the mouse network policy itself of the client, client library for march uh network policy, cld stuff. So.

E

Just for your question, yeah.

C

It seems like we're aligned with that that it should be for various reasons it's best if it's a separate repository.

D

Sorry, if I sent y'all down a rat hole.

C

No, it's it's an important discussion because we're, I think, we're kind of setting the.

C

Setting the rule of thumb for the future as well here so.

D

So I, um with the time change um not in europe yet and I've got conflicts now, so I gotta drop for another meeting. So if we're taking a vote, then I plus one it, but if you're gonna push it off, then I'll I'll vote next time, but unfortunately.

B

Yeah, okay, catch billy. I don't think we have quorum actually so yeah, okay, all right catch, you later, um all right, so yeah. I think, since we only have two orgs represented today that we can't actually make an official vote, um I give my unofficial um vote for a yes, so maybe we can make some uh forward strides and then we also have another technicality tall that to officially make you a maintainer, um your name has to appear on one more meeting.

B

So at any rate, I think that the thing to do is: I think that we've made some key decisions and I'm not hearing anyone. That's you know against the idea. I mean this seems very relevant, um so um I think we can move forward and we can just get the technicalities worked out um during the next session in two weeks.

C

Sounds great to me.

B

All right awesome.

A

Due to the time change, does anybody else have conflicts?

A

If not, I guess we keep going all.

B

Right, cool um yeah, so um um I think that covers a regular business and, as I mentioned snow quorum, so oh come on, go for it.

E

uh Okay, so uh so just a question about the uh so I serving the most network policy repository, not only the cld, also the implementation and I'd I'd like to know how many people using the mouse network policy, so just just yeah.

E

uh um Nowadays I, when I'm looking at the mouse network policy depository, there is no issues and no pull requests and then they are. I yeah. If, uh if there someone is using this stuff- and I really want to know their feedback and their issue or some stuff and then at least I'd like to ask you about in the nether prompting working group community, that's do you do you know someone using the mouse network policy is also welcome.

E

So, okay, so if, if you know someone using their mouse network policy, please let me know offline or sending an email or sending the slack anything at all. Welcome. Thank you.

B

Thanks so much I yeah, I'm realizing that, probably that time zone uh oddity has um not helped participation this week, because yeah it looks like we have um no europeans on the call today, um thanks so much and then mine mine was just a quick note to that um just about um some future looking stuff but um uh and a thanks to abdullah uh again for his presentation and I've still yet to make any strides on the multi-side with some implementation there.

B

But um as I move forward, which hopefully happens within you, know the next few months kind of a time frame for his dynamic attachments that uh I'll feed it back to the group and um see how we're looking um or see if there's any spec updates that are necessary.

B

I know we really considered this case in the beginning, so I think that we should be good, but um just wanted to bring that up and then hopefully, um at some point here I know tomorrow has had a big focus on um a multi-network policy, but a return to uh service abstraction as well, and that's all I've got.

A

All right thanks doug, and that is the end of the agenda. Does anybody else? Have anything they'd like to talk about.

F

Well, if no one says something I'll say something so I started, I sent it to doug now I started to sort of this more presentation now to look at the semantics of kubernetes networking with id you're sort of to see does the semantics change of sort of, when you add in more networks and when you add in overlapping, address spaces and so on, and my conclusion is that it really doesn't. uh So I can present that at some point in the future sure sounds good.

F

I'd like to hear see that I can send what I have to you now and we can bounce it back and forth a couple of times sure.

B

Also just saw that in my inbox thank you. There.

A

All right anything else.

A

If not thanks everybody and have a great rest of your day,.

G

Bye bye, bye. Thank you. Bye, everybody thanks again paul thanks, bye.