Kubernetes SIG Network, 15 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Network bi-weekly meeting for 20210415

Description

Kubernetes SIG Network bi-weekly meeting for 20210415

A

We're recording this is kubernetes sig network meeting from april 15 2021. um I didn't ask in advance, but tim or bridget do you have triage ready to go.

B

I do excellent because I do not.

B

I didn't get all the way through it before the meeting today, though, which there's a bunch to talk about.

A

B

Right, let's do.

A

B

B

C

B

We did all right uh all right, so in reverse chronological order, uh I did filter out a few before so. If anybody looked more than an hour ago, you probably will miss some uh here's, a user who's reporting a problem with graceful termination, but it wasn't exactly clear what the issue was. So I I followed up asking for a bit more um explanation, but I could use a volunteer to follow up when this person responds uh to figure out. If this is actually a bug or just a misunderstanding, what happens.

A

uh Send us on my.

A

B

B

All right, uh http lifecycle, hook tests flake on multi-node clusters. The interesting part here is this one sig windows, so it sounds like there's some assumption. There's some comments on here from antonio and discussion about the networking model, which does not require that a host network be able to reach other pods unless the platform supports host networking mode which windows does not so there's already a response. um So what is the right answer here?

B

So it sounds like there's something baked into the test somewhere. That is assuming that this can happen. uh Yeah.

D

Well, those particular tests are so qubit itself is dialing in the http request. Whenever the pre-stop or post start play cycle event happens, so basically qubit has to be able to connect to different pods on different nodes. In this scenario, so, given.

E

The way the test is written.

D

Yes, exactly, I have sent a request which forces both parts to be on the same node, which conforms to the network mode networking model that is already public and documented.

D

But it's still interesting that in some scenarios it still works, even though maybe it shouldn't, since there is no pod with host network enabled- and apparently that also happens on linux, with flannel overlay.

D

From what uh someone told me that they've tested in particular.

F

Well, how they check the comment from team is: the cubelet in linux should be able to reach the bots.

D

Yeah that says, with host network, enabled right.

F

Yeah, but I mean horizontal, enable means that the is the cubit process I mean it doesn't have to be a a pot. It has to be something running in the in the root space no and cumulative running in the good name, space.

F

So let me verify with the people here, so the cubelet process is going to have the same network than a host a pod with host network. True.

B

That is the way almost everybody implements it. Although I don't know that that's a requirement, I thought it was a check right. Don't we like checking the kubrick, I mean for a while there were people running cubelet as a container on the machine, so at least in the past it has worked.

B

It wasn't clear to me um if that was running in host network or not. My memories are a little fuzzy there um anyway, I don't think there's anything specifically that says: cubelet has to be able, or has to run in the root namespace just that it has to be able to reach the pods.

F

What I intend to do is if we can do this correlation is: if host network should reach other parts, the queue get should or must reach the other parts in other nodes too or not.

F

That is the answer that we need to give to crowd.

B

Yeah, yes, that's true.

F

Sorry say that one more time, antonio, the the the well, we need to answer this if tubelet must reach other parts in other nodes.

B

It is not a requirement that it must be able to only if the platform supports host network, so in linux it must.

B

uh I guess that is the definition of a platform yeah. No, no.

G

No, not even on linux, no, you don't need cubelet to reach other pods.

F

But that's the other thing is a pot with horse network. Is the same that a process in the host yeah. So then, if the, if the conclusion is that a cubelet and linus must reach other parts, you know their nodes.

G

I wanna be very specific because people might have uh pseudo-multi-tenancy around nodes and where nodes cannot like access, each other's and so on. Pods needs to access each chat, each others we're allowed.

G

No, not node must be able to route services traffic, but there is nothing that says every cubelet must be able to access every pod on the cluster.

F

But then it's impossible that a podcast network is able to reach all the pods. You know.

G

What I'm trying to say is that two processes can be running on root, name spaces. One of them is able to access other parts, and one of them can't all right. That's fine! There.

B

Could be firewalls there or there could be other routing issues there. The the requirement is pods in host. Namespace must be able to reach other pods, not cubelet, even if most of the time that is true both ways.

F

Okay, so I I was okay, then I got what I mean.

B

I'm not sure that was the intent of the way it was written, but that is the fallout of the way it is written.

B

um Okay, so let's take this one uh who's assigned to this. If antonio, are you assigned? No, I'm assigned okay uh I'll, look at it later and see if there's more to respond, cue cuddle describe endpoint slice panics this one's a fun one. uh I just kept it here, so I could stick robin's.

H

uh There's already a bug fix, assign it to me, but there's already a pr that should fix this.

I

J

Seems like we should triage, accept it too right. Yes,.

B

We should just have a shortcut.

B

Okay cube proxy, not creating hns endpoints. I don't even know what hns is.

J

It's the it's a.

B

J

H

K

H

K

L

I see so this is a coop proxy bug somewhere, where um for some reason it doesn't actually create those endpoints on a certain version of windows. I suspect it's probably related to like a windows, patch version or something I'm not sure, uh but I think it's assigned to me and it is james yeah awesome. Thank you.

B

Ipvs error log occupation with external traffic policy local, so there was some good triage on this one already. It sounds like if there are no end points for an ipvs service. The kernel prints out a helpful error message, of course, if it prints it out over and over again, it's not so helpful.

B

uh Lars was thinking up and ricardo thinking up uh ways we could get around this. um Who is it assigned to it's assigned to? Nobody can assign it to one of either lars or ricardo or somebody else if they want to take it.

K

Yeah, I can take.

B

K

B

K

Is it worth the effort to add and remove the ip tables rules for um when it's empty, we add a rule and when it gets an end point locally, we remove.

K

B

K

B

K

B

Yeah, it might be, uh I don't know what else there are other other ip tables rules being written in there. So if it's, if it's easy, then yes, it's worth it. If it's really really hard, then no, it's not, and the truth is it's probably somewhere in between yeah uh flaky test services for should function for service endpoints using host network.

F

Season is assigned to me right, your magic.

I

Accepted all right.

B

uh Do I have yes there we go.

B

Another flaky test services should function from pod to service with host network udp.

L

Wait is that one of the ones where, depending on where the pod lands it works, so it's like if it's on the node I feel like yeah. This is this one.

F

Assigned to me, I think that there are two or three of them.

L

Awesome: it's like a whole class of tests that are poorly defined, that we just call flakes.

B

uh Dual stack tests on windows.

L

Is this recently due to that fallout from the dual stack issue.

J

M

I think nisha was working on this yeah anisha is just adding tests. Yeah.

B

So it's assigned already we triage it.

K

I think it's assigned to a you, could just drop the triage anisha's working on this he's.

N

Well aware of it, um we have a bunch of those tests that are currently listed as linux only and it would just be nice if they were more.

B

Sure, okay, uh cloud provider, service controller race with node sync and service updates: this is a particularly fun one, because I don't think it's actually specific to this case, um but the example I thought was interesting to read it's possible that in the process of deleting a load balancer, we also end up recreating the load balancer and then leaking it, because the delete was already observed.

B

And I'm not sure off the top of my head. What the right fix here is some sort of interlock between controllers.

G

My memory is fuzzy, but I do believe it was fixed, but by finalizers like services, load balancers are only deleted after service finalizers are called. I am. That was at least two years back, so I'm fuzzy.

B

Do you want to dig into it uh sure.

B

Kubernetes bypasses external firewall ports. This was a really fun one to read, uh apparently we're magical and we can bypass firewalls um there's information missing, so I've already asked for more help. There jay also uh commented a little bit so I'll. Unless somebody really wants this one I'll just assign myself, I mean, I think, that one's probably just going to wind up getting closed right, because I it can't possibly be another one right. I my suspicion is, you know the firewall rule is insufficiently precise or something.

N

Either that or kubernetes really is magic. That would be worth a blog post.

B

Our packets are so strong, they bypass firewalls.

B

Okay last one then doctor shim defecation. uh I think this is mostly about documentation. Every time we get to the end. Yes, we need to uh write up somewhere, how an end user, who was using docker, shim or using um cubenet before, can recreate that behavior with bog standard cni. Does anybody know? Does such a dock exist.

J

Dan, I'm pretty sure it doesn't, uh but you can assign that one to either me or casey calendrello, squeed or just assign both of us. Why not.

B

You got it all right, I I turn over the reins.

G

Thank you tim. uh uh Can we do one more. The there is a probe failure like you, have it open as a tap and I'm just waiting for some feedback and uh either you look at it and we provide feedback later or we do it now, but I'll look at it. Yeah thank.

A

A

So I was next I wanted to share. um I sent an email.

A

Mailing list: that's what it's called: how about this sig network 2021 annual report that we need to fill out.

A

um I have put up a draft of this with all the questions and started, putting some comments on it myself, but I think the intention here is for everybody to contribute to this conversation a little bit and then we can agree as a sig on what we want to put in stone for our annual report.

N

I have a question.

A

N

A question about this: I see that we're concerned possibly about not having done a community-wide update, and I didn't dig in far enough to find out what that actually is. But is it just like some members of this group report somewhere else in some form? Yes, if that, if that's the case, could we get around that by saying? Well, we haven't done one in a while, but we've scheduled it and I volunteer to help with making it scheduled and happen.

N

If we wanted to make a forward-looking statement there.

A

uh Yeah, that's great, I mean we should definitely schedule one. It's been a long time since at least the last one I was aware of um so I I think this is. This document is more just about making us think about those sort of things um you know. I.

B

Clearly, I think that's our only way. That's our only way out of the uh obligations that we hold.

A

Is performing them yeah.

B

Yes, or at least committing to do so imminently, yeah.

J

And actually, I think we have quite a bit to talk about in this update. Yeah we've been, it was.

B

A busy year.

J

Yeah for sure there were times when we didn't, and so it's kind of like well, we could, you know not really say anything say we're here, but.

N

Since this was since this was by the way this I understand that this was due april 1st and we are late. Do we have a specific date that you're going to ask everyone to please put comments in etc by so that you can submit it by a predetermined date?.

N

Are both valuable but like when? Are you actually planning on pulling the trigger.

B

Can we say we should have this turned in by uh this time? Two weeks by this next meeting is that is that fair? Is that too long.

A

um And I think at this point we are overdue, like the the date to have. This done was uh last week, yeah, so um the sooner the better. But you know we should make sure we're getting it right. So if it we think it takes two weeks to gather up that information and well.

B

A

B

Obligations, the truth is that a lot of stuff doesn't get done until right before it's due. So if we put it for two weeks out, a lot of people are gonna. Do it thursday morning.

B

So if we put it one week out, maybe they'll do it a week instead of two weeks, I was out the first half of this week. This is my own excuse, but uh it's in my it's in my box to to do. um Should we set like a week from today.

A

Yeah, I think I think, let's see that.

A

And yeah I mean I totally agree with your waiting for the last minute assessment. That is what I did here.

L

A

uh Okay, what, if.

L

We just like copied this into a google doc and stayed 10 minutes after this meeting and just set a few of us, and just did it right now so and then casey you could finish polishing. It then we're out any of us came that might be the most efficient thing. I've been waiting another week.

B

I don't object to that, except that I don't have the 10 minutes after this meeting so yeah. I.

N

Have to have to catch you I have so.

L

How about the ten last minutes of this meeting, because that's what we did for sig windows? We just did it this.

B

Is kind of I I'm told we have a very full agenda today, so there may not be a good last 10 minutes.

G

Also, a track record of comments is probably something you want to show and something like that.

A

G

I mean, I think.

A

um Github should be a fine enough format for um having this conversation, we can I'm happy to set up another half hour or 15 minutes sometime for people who are interested to chat it out, live as well.

L

F

L

I'm happy to do that. I you know if, if you spend like two days, randomly asynchronously messaging people trying to get information and it's more efficient, I'm happy to come at least on in some of the stuff. I I know about.

B

Maybe we can, um I don't know how about how does tomorrow look for people regroup in the we have your usual thing: the cube proxy.

L

You can just double dip, we can just do it as part of that. You all want to just come, and just do that, like as part of that thing or do this report.

B

I can show up for the first part: okay, tomorrow's tomorrow, 8 30. yeah all right. So let's do it. Let's do this, um there's the open pr! Now anybody who wants to take a look take a look asynchronously tomorrow, there's the cube proxy meeting at 8, 30 pacific.

B

uh We can talk about it more there and goal being this time next week. We're done.

A

Okay, that's very good to me, see you tomorrow.

B

A

Next oops, that's the wrong tab! um Yeah! Next we have j ricardo doug to talk about keep proxy.

O

Go ahead, you put our name on that.

L

Oh yeah, okay, all right! Okay. uh What yeah can we oh yeah, the flag, deprecation thing, so we were talking about this friday? um Is it okay for us to finally start deprecating those flags? Does anybody we'll do all the details of it, but is it in general? Are people okay with that? Even if so can we take some? Is there anything? Can we just do it, sorry, which flags? I don't know the agenda?

L

You know all those flags and coupe proxies, don't work, um which is most of them. They're, like no ops.

B

uh Yeah, the no ops seem fair, like we haven't, they should be aged out by now right, yeah. Well, you can use them to generate a coupe proxy config.

L

Which is, I guess, I think, or something but other than that yeah.

B

uh I'm in favor of getting rid.

L

B

If they're not working right.

L

Okay, yeah, so, okay, so we'll just move forward with that on this cap and the other part of the cap is making sort of some strict like failure scenarios. I guess where you know somebody puts the flag in the flag, doesn't do anything. It's gonna break right um right. So as long as people are just aware of that, so it might break people with ansible scripts that have you know what I mean.

B

Like or whatever you know, but all of these flags were already marked as deprecated right.

B

Yes, they should have been were they.

L

um Is that what the answer.

B

Hinges on whether they are marked as duplicated, I think they were but yeah. The deprecation policy has a rule for deprecating flags to admin facing tools, and I think it's one release that you have to have a deprecation warning.

L

All right um cool, so that's good enough! I think I think that's enough for us to move forward with.

P

P

Cool, I don't think yeah, that's a that's! The only.

A

Question right thanks: jay uh ricardo and you're next yeah.

O

I'm gonna be really fast. First of all, sorry about leaving my mic open. I click on mute, but it keeps on building sorry, then so it's uh I have started some some effort about moving q proxy outside repo. I was chatting a little bit with antonio and also with our group in in into proxy, and I have opened so far. Four pr's me and casey.

O

We traveled last last week with some trying to trying to do that: q proxy, rendering so yeah, even if we are doing some capping thing, I think it's uh it's worth to move that outside of the ripple right.

O

So I've opened four pr's, I'm mostly doing some small small movements here, but and but one that that's pretty big, and I would like to ask you folks to take a look into that review. The big one. It's probably just you can approve team, because it's uh it messes with api and other things, mostly moving constants outside the api machinery theme, but yeah. I guess it is I've, seen also an email.

O

I'm gonna probably keep updating you about that, uh and the last thing is: if you people think that we should open a cab for the last part which is moving package slash proxy to outside of the repo or not. I have an opinion that probably we shouldn't, because I always listen that if you are rendering kubernetes kubernetes, you are going to suffer and have some pain and we are not responsible for that.

O

But on the other hand, it's probably going to break some cmi's vein during kubernetes.

B

And capture how we communicate even within the broader kubernetes world right, there are other folks who might have feelings about this. Who aren't here? Who don't come to our regular meetings um but would see a cap. So I would say yes um and specifically to discuss the point about whether it should go live under staging or whether it should actually really move out.

O

Okay, roger that about the the helper, the helper functions, I don't know if it's worth to write a cap for them as well, because most of them are like package, ipvs packaging, iptables package cctl.

O

So it's uh it's all up to you, but dprs. They are already open there. If, if someone want to take a look into that, I am trying mostly to make like small movements with them.

L

That's it so are folks, okay with this here and it's just a matter of doing the cap or is there anyone who's like? Oh, this is a bad idea because we're like could do it. You know what I mean it, but it's not really much point doing it. If there's gonna be, if people don't want us to do it, but is everybody else? Okay with it?

L

B

Or at least sort of in principle: okay, I'm very I'm happy with it in principle. I think it's a great idea in principle, even if we only get to the very last step and then don't actually move it we'll have broken all those dependencies and fix the the organization of it.

B

But I do think it's probably worthwhile I'm just going to have to go look at the individual pr's and make sure that there's nothing that stands out to me as like. Oh god I missed that before. But I don't think of.

O

O

There are some some some other things that are harder to to take a look like the async package that is used by cubelet q proxy, and I don't know where to put that. But that would be probably.

Q

Being with the exceptions.

B

Yeah we have, um there was at some point a request to move some of those more complicated packages off to kubernetes utils.

B

um I remember reading a pull request, for I don't think it was async, but it was something similar to that in in scope, and we found some issues that were like actual real bugs with making it a like a standalone api um and it turned out. It was just the person who was proposing it didn't have that much energy and had walked away from it and I they're still bugs those bugs still exist, um but they're since they're entirely internal they're, less of a huge deal.

O

Yeah, the async one cannot be moved to to youtube because it renders another kubernetes repo. Okay, that's client go, but I am moving one. That's called slice and I am like fighting with the old history because it's a pre-client because pre license agreement and pre pre merge message. So when I've moved with all of the history, I've got like a lot of punches from pro like you cannot commit. You cannot do that. So my question about that.

O

One specifically, that's on that's on the midi notes, but if we should keep the history about that that package or if we should just move because there are things from 2015 that all those rules weren't applied yet.

B

Yeah, uh I think we should keep the history, even if we end up adding one commit that says, remove all of the files that we don't need to move this between repos and then we have just.

E

A bunch of you know not so useful. The bots won't let the pr merge, because the old commits are are not up to snuff we. So we I had done a a a pr to move bounded frequency runner, which is probably one of the things that that you were looking at in async uh ricardo uh to to utils, and you had approved it, but it couldn't merge. And then we were like oh we'll deal with this after freeze and then I keep forgetting and vegetable actually just closed it. This morning,.

B

um Well, I I there are still a small number of people who can manually approve merges. So if we have to do that, we we can.

G

Okay, I have an opinion around the hotel. Util is a very dangerous place to have anything right. It's.

B

One thing to be tripped: it was supposed to be a collection of really trivial stuff yeah and it's becoming increasingly less trivial.

G

I, the problem with util is you're telling people you can vendor that which means you're, creating a contract around that api and I'm not talking about api rest api, I'm talking about golang, api, yep and, let's just say the ground at least the apis I've looked at, is not exactly the cleanest right and it has things like huh so with this. These are things we can always observe and the way we use them.

G

But if we tell the world that you can use these things, it becomes a whole different game and I'm not sure if the entire project is meant to serve people by providing packages they can use other than clango.

J

G

J

Like bright line for these kind of things, like component details and stuff, like that, that, like we don't intend for other projects,.

B

So the the line I mean, unfortunately, the way the go ecosystem works is, if you can see it and it's not in an internal directory. It's fair game which we've at least written in in various places like don't vendor kubernetes kubernetes, because we won't support you right. um All of the other repos, though, are less uh well defined. Less bright, liney util was explicitly done so that people could take useful stuff out of the kubernetes ecosystem and use it and like part of the the meaning there was.

B

We will support this whenever we bring something into utils, we hold it to a generally higher bar. um None of it is perfect. um There's some things in there that I look at too and I go. I wish we hadn't done that, but um we do hold it to a higher bar for testing and for documentation and for api cleanliness than we do for internal stuff. um You know something like async is big enough, that it might just warrant its own repo.

G

My problem, my problem with offering these things for people is, you have no control on what these things can vendor in so let's say we are in a situation. We can use these things in cooper nets kubernetes, but they vendor some weird thing and we're okay with that, and we understand it now. If we offer it to external people, then it becomes a shared like even the vent, like even the stuff that you vendor into you. The stuff that you build into your tools becomes something we need to worry about. I'm just yeah, not.

E

G

Yeah yeah copy the code. Now it's like, if you need the code to copy it right.

B

Utils is supposed to be like almost import free, there's supposed to be very few imports there um and so and there's a rule that it can't import other kubernetes repos so like. If we're really vendoring client go which boggles my mind, I'd have to go figure out what the heck we're doing. There um then yeah. That is not a candidate right.

B

um This is part of like the logging stuff that I've been doing in my free time is to get rid of vlogging dependencies out of places like util, because unfortunately, kubernetes has caused g-log and k-log to proliferate across the go ecosystem.

B

So I agree with you. I'm agreeing something like async is big enough and subtle enough that we don't want to copy it internally. We want to find a place where we can share it amongst each other and vendoring. Kubernetes is not a starter, so we have to find a place for it. Even if we put a big thing in the readme that says this is really project internal, please don't mess with it.

B

Go doesn't have a way to represent that.

I

It's also very important that if you don't.

G

Say it clearly, people will consider it fair game.

B

G

B

Like we don't version tag these repos anything in any way, so that people who are vendoring them are never going to get a 1.0.0 version. They're always just going to get o point something something something plus get.

B

All right, anyway, ricardo, thank you for and and jay thank you for taking on the hard work of starting that I'll go. Those pr's are in my queue. I will definitely look at them and have a hard think about how to break those dependencies right thanks. It's all.

P

Ricardo, I'm not just.

R

A

Thanks ricardo uh dan winship, I think you're next on dual stereo yeah.

E

It was just a quick thing, so we had talked a while back about doing a cap for making the apis or virtual stack, and then I didn't get around to filing it, and that is now filed um still.

D

Figuring some bits.

A

E

Out but it could you know people can start commenting. I realized that, since it's going to be behind a feature gate that means that it talks about some pain and flapping connector or services and stuff like that, but that's mostly for the early adopters once the feature goes ga it should all work pretty smoothly.

E

Anyway. That's it.

B

I, like the bug you filed by the way uh on the points ownership that was a good one.

A

uh Rob you're up next.

H

Okay, yeah mine is really simple. It's just. uh We have a fair amount of people for gateway api in lots of different time zones and so we're trying to find times that work for as many people as possible.

H

So if you're interested in attending those meetings being part of the working group, I just fill out that straw poll with when you're available and we'll try and find time's work. That's all we've got.

A

S

Yeah, so this is a quick uh question for the team. uh This.

E

S

That came up on the cluster network policy and we discussed this in this week's policy api's subgroup meeting as to where to host these crds or the resources.

S

And- and uh you know there are two options one is you know it is part of the networking api group, or you know we host them into a kubernetes sig repo specific for cluster network policy. So I think the general feeling is that we will host it in a separate repo under kubernetes.

S

uh The question is that, should it be specific to cluster network policy resources, or should it be a more generic network policy repo? Because there are discussions around, uh you know v2 network policy which might have their own crds. So should they all live together, or should it be specific.

B

I vaguely remember discussing this: what did we decide that it was better to stick it in crds than to put it in the networking group alongside the existing network policy.

S

uh I think the problem was that you know if we want to do it as a v1 alpha 1 api.

S

There would be some issues because we already deprecated uh and removed the v1 alpha 1 for network policy or the networking group so yeah. I think antonio had some uh comments around that. um So maybe it sounds like it's. It is a good idea to have it as a separate reaper.

S

uh You know, maybe also you know just follow what gateway api folks have done and maybe we'll evolve quickly, uh but then the question is: should it be specific or or rather how specific that repo should be because although those different resources are related, but they are not related.

B

Yeah, it's use. The goal of api groups was to associate things that are related and that would be used by the same sets of users and not necessarily to be just grouped by sig, although, ultimately, that's kind of what happened right, we end up with networking and storage and other groups.

B

um Do we have any belief or our understanding of what the other things would be? So one.

S

One example that comes to my mind, I think government is working on something like a dns policy resource. uh I know jay ricardo. We are all talking about a v2 for uh network policy, which may be a different resource by itself kind of an evolution for the network policy v1. Okay. Now they are all not necessarily related to each other. Rather I mean it's not something that will you know graduate together, but again they fall under the umbrella of network policy.

S

B

I think those are related enough to be in one group.

S

Okay, so maybe we'll uh start the work on uh getting a repo for for a standard, kubernetes.

H

A small follow-up question here: would it be in the same networking dot x, dot kxio that gateway api is using and is there any kind of uh you know we we have previously deprecated versions of api groups it for core types.

H

I don't know if we have any concept of that for official crds like this, but having the same group span across repos like this could be interesting.

F

But I think that they so the the the the summary is the last meeting we say if we need to implement something new, most probably has to be a b2 in network policy, and I think that the request is to create do the same, that you are doing with the category api create a new.

F

It's a new group, no.

H

Right so for gateway, we have a generic api group, that's that could be used for any networking resources. Just networking dot x, dash case dot, io that we're using which theoretically, you could use too. I'm just not sure. If that's what we want or not.

B

So I think, there's two issues one: should we try to coordinate one group across multiple efforts and repos uh and two should it still be using x case, like I, I believe at some point in gateway. We said we we need to decide at some point in the future, whether it's going to stay as xkates or kate's right yeah.

B

So um you know the the distinction between x, kate's and kate's. Being things under x. Kate's are not subject to the kubernetes api review process and things under case I o are um my feeling would be that we should probably move it under kate's that I owe for a gateway. um I don't know how coordinating across different repos would work. We need to have some coordination scheme, but it could work.

B

Don't you think.

H

I would how would api review work?

H

I I guess you know for crds, you know we have a pretty a pretty standard process within kubernetes release cycle, but crds kind of live with their own release cycle their own everything and not the same kind of formal processes for review.

B

uh Yeah you, you you'd, have to file a bug with api reviewers and get somebody like myself or jordan, or david or daniel, to take a look at it and say that they're happy with it.

F

But that's for for the, if you have it in three and you have a out of three no.

B

You can have it out of three also.

F

But that's not mandatory.

B

If you want to use a something dot, kate's that I owe suffix, it is right. That's why we created x-dash case that I owe so that people could be as be a sig owned api without being an official api. So getting the the extra level of review just gives it a little bit more uh gravity. I guess.

B

So then the question is: could we coordinate one namespace like that across multiple projects we would need. We need something somewhere to say you know, hey, I'm using the fubar name. Nobody else can use it right. Some some, I don't say a spreadsheet, but like a git repo somewhere that lays claim to names within that namespace.

F

Do you know if there are? Are there other groups doing that I mean startups? They used to have a lot of projects outside. I.

B

Don't know if there are other groups doing it. It's a good question.

F

I will ask, I would like to ask.

B

Storage would be the likely if anybody's doing it, it's probably storage. um So maybe we should ask them what they've done with this and just follow the pattern. If they created multiple namespaces, then maybe we should just do that.

B

I think we can there's no limit on the prefix too right, so we can say that I o and something else.networking.com, but given these these projects are all network policy focused like they really do, feel like the same group.

H

So what you're saying is policy would be one group and gateway would be a separate group.

B

Yes, I mean: that's, that's one way to do it right, it could be, it could be gateway.networking.case.io and policy.networking.com um yeah, but within within policy there would be three separate projects creating different crds yeah. Now, there's a new issue that just popped into my head. There, like, I know, you're, not allowed to share an api group between crds and built-ins, but I don't know if that applies to uh suffix sharing, I'm presuming it doesn't, but I've never tried.

B

So we should, we should try a few things and then talk to storage, so uh who's taking the action there. Do you want to reach out to storage folks.

B

Who open this topic? I forget. Okay, that was me. I'd be shake.

S

B

You uh do you know any of the storage people. Do you want me, I can. I can do it. Let me tell you if you can point me to someone I'll uh I'll pick up.

C

B

C

Abhishek, if you need help with that too uh tim feel free to cc on. That's me on that stuff. It's andrew stoick, okay, thanks great yeah.

H

And- and I I'm happy to be part of that and or start that discussion, uh okay, it could affect gateway as well.

B

Yes, all right I'll I'll, send an email or something or I'll start a slack even better.

L

I guess andrew.

B

L

Just can you add a note to our policy agenda for monday to just check up on that.

C

Yeah, no for sure we were going to do it this week in slack anyway, so this is better thanks for bringing it up. I will check.

L

Maybe we'll follow up on it I'll be shaking since you're out and maybe maybe the netball group can figure out who does it monday.

S

If you want yeah feel feel free to drag me in, but thanks for helping out.

E

A

Thanks everybody, um I think, let's see uh from sj.

M

Yes, I'm here a quick note about the cap for all board services support. This is a work in progress. I've dropped a link in the agenda, but thanks to everyone that already reviewed it and and offered alternative approaches as well it'll be great to get some more comments on it, mostly about the use cases for all port services around.

M

Do we need it for cluster ip services? Is it just? Is low balancer support sufficient and also about the approaches if any of them could break um any use cases for it? So yeah, it's still very much in progress, but it'll be great to get more discussion on it.

M

Yeah just wanted to say that that's all.

A

And then, uh just in time, we're at the last topic um from clawdu.

D

Yeah we've actually discussed about this in the triage at the beginning,.

F

L

I have a quick question before we drop is: does anybody know or care about this whole client go multiple endpoints thing of it being able to support multiple endpoints for api servers.

L

um Like does anybody know who would work on that or if people care about doing that, it would allow you know coupe proxy and everything else to not have to go through load balancers and it's an issue that we were triaging in the net paul sub project or in the coop proxy sub project, and I was like I asked if we could close it and then somebody yelled at me. So I figure.

L

Maybe they want us to hunt down the solution, but I don't know whether do you have to file caps for client code like I don't even know how it works. I assume it would be pretty easy. You just put a damn array and a round robin into the thing. If you wanted to do.

B

It right uh I mean I, I I don't know exactly what the implications are, but the environment variables that we've always used are singular, and so anybody who's, parsing those environment variables today would break.

B

If we changed the format of it, we didn't define it as an array, so we'd have to like there's multiple levels of discovery that happened there for the uh the um the the endpoint ips right like uh if we're talking about the ips that come out of a cube, config versus the ips that you find in the um uh what's, it called anywhere environment, yeah, any yes in the environment. Thank.

G

You and then my assumption is the client go. Endpoint uses the service ip, so it's load balanced, because the service, the coopernet is the full service will have all the end points of the api server.

F

Yeah they want this because they want the the use cases I want to to use this with for two process example, because you cannot have a service until you have to process it. Oh and I don't want to put a lot of water in the control plane. You know that's the use case, so I I just take the cry and go into proxy to to go to these three api sets.

P

That's a chicken or.

L

P

L

Specific part of it.

G

L

B

G

Use case yes, yeah.

B

Yeah, it seems it seems like a reasonable corner case to handle multiple for that. But I don't know the client go stuff well enough to know how hard it would be.

F

I I talked with this in internally with some people from fbi, aka machinery, and they they were thinking something about that and there is a pr that got stuck with an implementation, but I don't know it's crying. Oh, I don't think that until you have, you know everything sort out and super.

F

That's super street. I don't think that.

L

um What should we do as we're going through issues like with stuff like this? That's just not going to be solved, but it's not an invalid request. Like is there a label for, like you know like a bike, shed label or something like where you could say like okay? Well, we can, let's just keep talking about this forever.

B

Well, no, because we don't I mean, there's, I think, there's two categories there there's the ones that we don't know quite what to do yet. We think they're valid. Let's leave them open and see if we can discuss them later and there's the ones where we agree are valid, but we're never going to do it. So, let's just close it, because why have 6000 open bugs.

L

How do we prove to someone that we're never gonna? Do it like? What do we what's the bar there like? Do we just do I just I'm happy to just bring him up like every like here, every once in a while and just be like so for this one specifically, should I just say that, because I feel the same way like antonio mentioned, like I don't feel like it's ever going to happen, but.

B

I don't I mean if you, if you really think it's never going to happen, and the folks on api machinery agree like we should just close it.

E

B

I, like bridget's language, that she just posted in the chat right like this. This issue is a real issue and we understand it, but we're probably not going to fix it because the implications of fixing it are just too big, yeah, okay,.

S

N

S

N

Go ahead, oh, I was just saying you might want to put some language in there about blast radius and surface area like fixing it, for you sounds great, but there are going to be knock-on effects that we don't want to occur.

G

It's not even a fix, it's a feature request right. Another thing I need you to like: I need us grounded a bit. We have the luxury of clouds, so low balancers. Are there yeah bare metal, it's harder and I'm guessing whoever pushing for this? It's a very metal deployment because that's where you suffer the most.

B

Yeah and to be clear in in this case, I don't object to leaving it open um if we, if we think that you know given somebody with sufficient uh uh incentive that they would actually implement it in a way that we could convince david edens to accept, then I'm cool with that.

G

I'm not saying we shouldn't, we should say we're, not gonna fix, I'm just saying: let's keep it, let's think about it all right. If it's one of those things that will require a large service area, then we need. uh We need to schedule it in a way and people understand that yeah it's a corner case and we would like to add it, but it will take time.

B

How long yeah, unfortunately, these are the sorts of issues where we ask people to file a cap, and then we spend three months pounding on the cap and then they get frustrated and walk away from very often, and it's not that we should change the process. It's just you know. This is what happens sometimes.

G

I mean if you can take the bonding right.

G

A person like me who works mostly in cloud environment, something like that is like ah no all right. A vintage moment of somebody who's using this on edge or uh bare metal might be fundamentally different right, so I don't want to start.

L

G

Single foot because yeah.

L

That's why I want a label for these things, so we can at least just not have them as part of our triage like okay. This is a thing over 90 comments. It's a separate, it's a! We should have a word for this and a label for it. You know I mean that's what I'm saying well, there's.

B

As I say, there's priority backlog, which is a maybe as close as we've got to this, go ahead.

O

Yeah I I was going to say that probably this could be something as like. I am an on-premises, very metal user, and this could probably be like best practices for on premises like you can run an aj proxy and keep alive, and this is the way that we suggest you doing this. It's not like the end of the world. I know that that there is a case of of the edge computing thing, but for mostly the bare metal one, probably we should say hey we can.

O

We can do better, like writing, how we think it's the best architecture for this case.

O

I can at least like try to make that make make some some documentation about that and see and ask hey. Does this solve this to you instead of like doing.

B

I mean that's, that's great, I think it it doesn't. Enhan doesn't address the larger question of you know, there's always going to be a stream of these yeah. We agree with it, but it's just too complicated to tackle right now or we welcome anybody to do this, but understand that it's going to be.

B

You know, like slaying a dragon, um do we close the issue and pretend it never existed or do we leave it open and let the number of open issues just continue to grow uh over time right, we're at 2000, open issues right now,.

O

Oh okay, yeah in this case, maybe they won't.

B

Fix his uh or maybe we need a different way of denoting like ideas that we would love to do, but we used to have a we used to have a version, two accumulator bug where every time somebody came up with one of these like this would be awesome, but it's so challenging. It requires a v2 and just go. Add your ideas to that, and then it just became like a it's like screaming into the wind. It's like a venting place.

B

Where did that go? It's gone. I think the issue itself might still be around, but like nobody takes it seriously, it's not it like. There isn't, there's no plans to make a v2 that breaks things in any fundamental way. So a lot of those ideas are just non-starter ideas, they're just too big to be worthwhile.

T

Reach out to themes, teams were worth tracking them.

F

We're working on that thing.

B

I occasionally go back and I read through that issue and I live in my little dream world of how nice it would be if we did all the things in this bug and then I come back to reality and realize I've got 1900 other bugs to go work on yes, pearl six right, python, 3000.!

B

All right. I think we're done.

N

T

N

Year of ipv6, on the desktop, or at least on the cloud.

G

Yes, antonio, I had you on the field on the people that I will buy them a drink when I see them you're off that list.

G

O

Q

N