Kubernetes SIG CLI, 24 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kurbernetes SIG CLI 20220824

Description

Kubernetes SIG CLI bi-weekly meeting on August 24th, 2022.

Agenda and Notes: https://docs.google.com/document/d/1r0YElcXt6G5mOWxwZiXgGu_X6he3F--wKwg-9UBc29I/edit#bookmark=kix.xpquezpen15w

A

Okay, hello. uh Welcome to the august 24th edition of the kubernetes six cli bi-weekly meeting, I'm sean sullivan I'll be your host. Today, uh we've got some announcements, uh pretty big ones, about uh the release of version 125 uh specifically, and we have a couple topics that we're gonna get to.

A

So let's kick off with the announcements and let's get into version. 125 combiner is the name of the release so that release went live yesterday on tuesday, the 23rd on schedule, which is nice, uh there's a blog post which I'll show here is that visible to everybody, the the blog post? Now, okay, great yes, it is okay, so kubernetes uh version 125 combiner we've got our nice little logo here and here are the major themes.

A

I wanted to point out a couple to us, while I'm on here, which is related to us, which is that the server-side unknown field validation went to beta, which means that it's default by now on the api server- and this is part of an entire effort to move uh a lot of the validation from the client to the server.

A

um And so I think, if you want to know more about the the forces behind that, I think jordan liggett has been pushing this particular effort and the implementer of this he's out on maternity leave.

A

So so you won't be able to talk to him specifically, but this this relates to us this particular part.

A

um Another part which we may want to remember is that, uh since the femoral containers are graduating to stable, that's uh control, debug uses these femoral containers um and then another one we may want to consider keep in mind is that crd validation?

A

It sounds like the api machinery is moving more towards instead of validating web hooks wants to move to this common expression, language.

A

Which would avoid the uh the network traffic to go to the validating web books? um So please link into this. If you want to know more about that, um and so, let's get back to, we have release notes if you'd like to click through those those we have here, which also specify some of the major themes and enhancements features deprecations, and so so mate was helping us uh us out about the the code. Freeze um would match.

B

Would you like to.

A

Okay, so watch it put a link in there yeah. I put the link.

C

To the announcement, the third okay, here we go exactly that's the one.

A

That freeze has been lifted and it's now thought so right on put your code in after three weeks of being frozen.

A

Okay, so on to our next announcement, which is uh the contributor summit, kubecon registration has uh opened, and I put the link in there. So, if you're interested in, if you're going to kubecon in detroit and october 24th is the contributor summit, that's on a monday uh the day before the rest of the kubecon convention um and if you'd like to attend the contributor contributor summit, that's the link. You'll want to click through.

A

um So there's a couple other smaller items. I think this next one was actually in the blog post on kubernetes 125, which has to do with uh we're now redirecting uh from k-8s.gcr dot io to registry.kaiso.

A

So, instead of and so the gcr dot, io uh url may proxy your three 302 to redirect to the to the second one, the registry.k8.io- um and I think I I put in there september 8th, but it may have actually happened with the release yesterday.

A

So let me see if I saw I thought I saw something yeah here. It is. The the container registry service has been moved from k-8s dot, gcr dot, io to registry that case ir, k-8 dot, io.

A

um And if you're interested in the the next release team- and you want to start off as a shadow- then uh there's a whole bunch of links here- that you can click through to to look into that opportunity, and so so uh so eddie you've been you've done release teamwork. Did you find that valuable.

D

Yeah the release team was a great experience um if anyone is really wanting to get more involved upstream um and maybe not specifically sig cli- it's definitely a good on-ramp to all the parts of the project.

D

It does take a lot of work, though, and it's don't my caution to new folks to the project is: don't um don't sign up for everything every opportunity that comes along like pick a few things and stick with them and get good, but yeah highly recommend the release team.

A

Cool okay, so why don't we move on to introductions uh where we provide some space and time for those who are new or haven't been around for a while or anyone who'd like to get to know their colleagues in the six cli better? You can introduce yourself and say hi and of course, if you don't want to that's, not necessary either.

A

Okay, why don't we move on to the the topics, then uh mate? Would you like to uh talk about this first uh concurrency issue with.

C

Yeah, I I see martin is on the call and he paying me about the topic I'll I'll. Let him speak and I can uh provide uh some guidelines. I've been thinking about it for a bit and have some ideas so um I'll. Let him speak and I can.

B

Can you hear me okay,.

A

B

A

Thanks for joining us.

B

Cool and so I'm working on the acs, also known as stack rocks operator at red hut, and we found a flake in our test suite which, after debugging for like a month, turned out to be a race condition and due to the rest, client getter that we were providing to the cli runtime package resource package and it turns out this package requires this client getter to return a new client every time, because the client is mutable and basically specific to every um so that sort of the config it provides, is mutable and and specific to every client.

B

That's instantiated, but the implementation we used, which was part of the helm operator, library and provided the same instance every time and when there are multiple clients.

B

Concurrently using it bad things happened, and so there's a few ways to fix it. The the implementation was already fixed by cloning, the config, but it seems to me that there might be many more users of this interface and we should either document the the good way to do it, but to use it to provide such implementation of the best client center.

B

However, this will likely not fix any of the implementations which are already written because if they work, then why? Why read the doctrine right of something you've already done? So maybe we should consider cloning, the rest conflict ourselves or doing something even more crazy, like invoking it and making sure it actually returns a new, a new instance. Every time this this interface seems to get used or like something like this interface seems to be used in in other places as well. So I just wanted to hear communities and ideas about organization.

A

So, just just so, I understand correctly the the standard implementation of this rest, client, getter interface, the config flags.

A

It does it correctly right it copies before it returns, and so um so that's why we haven't seen that it's these other implementations of the interface that and it wasn't. uh It wasn't documented that that it needed to be uh cloned.

C

Okay, uh so a little bit of background, as I was looking through that issue and then uh processing that in my head over the past couple of days, um the cli runtime, when we initially extracted that, from uh from the current keep cuddle code, it was meant for for the cli.

C

So that's where the current flow and the current way of doing things is implemented this way, because it was the easiest and the simplest approach that we could take, and it was mostly the the approach that was already implemented for cube cuddle and just like martin pointed out. It's fine.

C

If you're talking about clies, which are meant to execute whatever functionality it has complete and rarely they will um rarely, they have issues when you will be running multiple threats in a in those simple cli.

C

Of course, the uh the end goal isn't uh for cli to be only meant for um the cli runtime be only for for writing cue card like commands and and definitely starting with the um with the docs, clearly pointing out that this is not thread safe and that you and what you should be doing to ensure that stress safeness.

C

In parallel with that, um I would really like to see I'm seeing two possible ways of addressing this problem and I'm not sure which one would be the best, because I haven't looked that deeply through code and which is feasible in the first place. So the the things that I was thinking of is either creating entirely new implementation.

C

That will be threat safe and the downside is that we will have to maintain two implementations and sadly, the uh the new implementation would not be primarily used in cube cuddle, although it could eventually or if possible, um add or fix the current code such that it it is thread safe. I'm not 100 sure if fixing the current implementation is possible.

C

uh Martin pointed out that the builder pattern that we are happily relying on for uh for constructing the builder uh will be the most problematic bit and I'm not sure if that's usable, but maybe we could, uh in the least invasive way, fix and address this problem.

C

So if anyone is interested in working on this, I'm more than happy to review the prs and see what's possible, uh the cli runtime like we talked about it. A couple meetings back is due for uh for every factor and it wasn't touched uh since the day we extracted that as it is from us.

C

I don't know it was like two three years ago, maybe even more so it's about time. We uh we pick it up and address the problems. There's there's been, there's been a couple of them with regards to the structure overall and the ability to expand it, and this is another case for uh for an overhaul for cbt right, runtime, not sure if anyone has any other opinions or suggestions they want to share with the group.

A

So so I'm having still a little bit of problems grabbing all the words, maybe it's on on my end, um if, if someone else here has better is able to hear matcha better, please add some more notes.

C

I'll update the notes in the agenda.

A

Okay, cool. Thank you.

B

So yet another thing I thought about part of the problem seems to be that this rest config is partially things that, like are conceptually immutable and could apply to any client that is being created concurrently and part of it are specific to a given resource that the client is operating operating on currently, so I don't know if we want to go that route, but maybe refactoring the the rust client to get rid of this would be a good idea.

B

Like there's all kinds of stuff in the rest, conflict there's like tls configuration, there's partial paths in the api.

B

It seems like a organically grown thing.

C

Unfortunately, it is, um I'm not sure if we will be able to touch the conflict, because that particular bead of bit of code is owned by the big idea and missionary and we're we're we're likely not going to touch or modify this particular bit um and yeah. As you said, there is some, let's just just organic, we grew and not everything we can do, and maybe just starting with the fact that we will be explicit about what the method is doing.

C

How to do it in a in a non-modifying way would be a good starting point.

A

And, and so one thing that we can all agree on is that we can add some better documentation. uh Does that sound reasonable for that particular interface for as a major note to any implemented implementer for that interface?.

A

A

So, are we ready to move on to the next topic, or should we or are there other aspects of this? We need to discuss.

A

Okay, why don't we move on to uh the next uh applied prune issue? uh I know katrina is here. Would you like to to introduce this issue.

E

Yeah, so um this is one that is very similar to something that came up a while ago. um Somebody this is the second time someone's felt strongly enough about this, to attempt to make a pr for it uh not just open an issue which there also is um so.

E

I was looking at this and I I thought that the original proposal was um a little more aggressive than we can be, but that um after thinking about it, I think there is a change that we could make with a reasonable deprecation process that would still be valuable to users.

E

So the thing that is causing pain is that there is a built-in allow list um that is used to identify which kinds get pruned uh when you use the dash dash print flag without the explicit white list flag, so it just gets used, um no matter what type of resources you're passing in and um when you use the dash n flag or whether you use the end flag or not, it always uses the same list of kinds and that's what people find surprising, because two of the kinds, I think it's just two uh that are in that default list are not namespaced.

E

So you can run a command with dash n that then prunes a bunch of um global uh resources, um persistent volumes.

A

Airspace and persistence, name spaces.

E

Themselves right so, printing, a namespace is catastrophic um and in this case you've got the dash and flag surpr specified. So people find that super surprising and dangerous, which I tend to agree with. um So my proposal would be that what we can do is uh start emitting, a warning uh that the uh if we can go scroll down to the proposal there, uh one more.

A

E

Yes, there we go so um if, and only if you uh prune is given the name space flag and is not given the white list flag and then we should emit a warning saying something like um key patrol ply is no longer going to prune non-name spaces resources by default when the namespace lag is given so explain what the change is going to be um and uh tell them that they can, if they want to preserve the current behavior, which I think is another reason why this is okay to do.

E

um They can specify the pruned white list flag themselves with the same list that you know that we're using internally as the defaults, if that is actually what they wanted um for the future. So if, for some reason they are managing a set of namespaces and volumes and of some specific namespaces all in one group, they will still be able to do that.

E

Then, after the deprecation period passes just change the default. Allow list so that um the if the name space flag is specified, you get a default, allow list that doesn't have non-new spaces resources in it. So it just removes percent volume and name space from the defaults so again like if they, if they specify the print whitelist flag, explicitly to give us that list of kinds, then uh there would be no behavior change at all, so this uh just makes it safer in the case where they pass the dash and flag by improving the defaults.

E

So um because this is a behavior change and because uh we never discussed a specific proposal, but we discussed related ones. I thought we couldn't do it. um I wanted to bring it up with the group um to see if there's a grants with um my thinking here or not, um and whether uh the uh cadence is uh appropriate. If we, if we do want to make this change over what period we can do, it.

A

So there's one uh detail that we'll also have to keep in mind is that the you can actually specify default namespace in the coop config file, which has the exact same um output or result as a minus n on the command line.

E

um Does it in the case of apply like if you think of the scenario, there is a scenario that has actually been discussed in this issue, where, uh if you have uh name spaces specified in in line in your config files um and it conflicts with the flag, then uh we actually error out and refuse to apply those con conflicting spaces, whereas uh if it is just a default in the cubeconfig, that is not the case. The namespaces will just be used from the files and it will continue on.

A

Okay, yeah that'd be something to I. I was. I thought that it had the same exact output as putting it on the command line, but maybe if there is some slight differences, um yeah I'll, let me do some uh some experiments to make sure that what I said was true and if not I'll get back to you.

C

There's one tiny lit that I, that I notice and if you are there's a hard-coded list of resources that I very don't like um I'll, probably instead of doing this hard coding, the that particular list to read that if that particular information is from discovery, we have that information. Whether a particular resource is a non-namespace or namespace one, because it's fine for the built-ins, but as soon as you start working with crd and even if, in our case, moving in namespace, as you said, katrina will be disastrous in a similar fashion.

C

There can be a third-party crd that is on namespace, which will have a similar effect in this particular cluster.

E

Right but the only way we would end up touching a third party um because of the way this feature is designed. It's not designed great to be honest, but um is that somebody explicitly told us they used the pre-whiteless flag and they explicitly told us to prune it. So at that point we don't care if it's name-spaced or not, because we have to prune it because they told us to.

C

E

I do agree with you, though, that that hard coded list in principle is not very nice.

C

uh Yeah I'm trying to get rid of those lists. I know that we have them a couple of places and ideally I would like not to have them anywhere at all. Yeah.

E

Yeah and and this one is it's there's a lot of things to to not like about the the way that pruning works. um That's that's one of them for sure, there's a tool that I that I've worked on open source um for my company that uses prune under the hood, but to make it safer um and to make it worth with custom resources.

E

It actually uses discovery, just like you're, saying mache, to figure out what namespace and non-namespace resources exist and then based on whether it's doing it has like very separate namespace and non-namespaced operations, because that's its opinion like you, should do that. That's the way to be safe. um It's so it constructs dynamically. The appropriate allow list um to use with the commands to feed in, and it never ever ever uses these defaults that I'm that I'm um proposing to change, which I think nobody should ever use.

E

But if somebody does, I would like to make it safer.

D

I have to give the pr a read so I'll keep the tab open, but I am always in favor of aggressive changes to make people not shoot themselves in the foot so preemptive plus one for me.

E

Right, so I guess that's to the second to the second part there. If we agree that we can do this, do we need a deprecation period? Can we accept the pr um with a warning that this has changed? That would be the most aggressive version eddie um and if we do need deprecation period, how long do we think for this sort of change?

E

It is in alpha technically, but it's been in alpha since, like 1.2 or something ridiculous, so there are certainly people um that are using it, and uh that could be a reason to treat it more carefully than the alpha policy says.

C

Yeah, especially that we've been um we've been advertising a fly very heavily I'll, probably even per particular vlog.

C

I would probably try to keep it a little bit longer around, especially that just recently somebody pointed out that me changing the version command and the default in the version that it should be done in a more extended way that we are deprecating and then changing the defaults and again deprecating, and only then changing so I'll, probably uh follow the same guidance, especially that when it comes to a person, I'm less worried that somebody is relying and if they are lying, they should be relying on something like json yum output, to have a machine, personal and not a human readable thing.

C

But for this particular one I would probably uh be a little bit more careful and add this one extra release and, if possible, the uh a blog post describing why? What or what? If someone is capable of doing it so yeah, but all in all, definitely yeah.

E

It should be uh strictly safer, at least like it's going to not delete things um if somebody misses the change, uh so I think that's good, um but are you saying that we should follow like the beta uh deprecation cycle, even though it's alpha sort of thing because of the computer.

C

Yeah yeah I'll, probably a little bit safer on that one. It never hurts. I I've personally went through a ton of hoops with an alpha api in the past, just because people started using it and it was barely available for a two or three releases, whereas this one is, as you mentioned, for more than 20 releases, so yeah, it's, it won't hurt and it'll be on the safe.

E

All right so warning this release and then wait I'll, look up the policy and see what exactly it is and then we'll change the behavior.

C

Yeah, thank you.

E

um There's also sort of a tack-on item, because I was looking at this. I noticed that the flag's name, which again I'm not proposing changing anything about the behavior of the flag. That's going to be that that's not affected, but the flag's name is prune whitelist, which is not using inclusive language, and I was wondering if we had buy-in to actually just sort of alias that and then follow the deprecation period to get rid of the old one.

C

Yeah, I don't see, I don't see a reason. What you're proposing the new name.

E

um The standard substitution is allow list, so I I would propose just move to that. Okay yeah list.

A

And if we, if we end up building a better prune, then we wouldn't even need that flag right. If we were.

A

The fact that we've hard-coded the gbks is a massive red flag and that we have even have this uh flag to specify, which particular gbk's to prune really points to major underlying problems with prune.

E

Yeah I agree and, like I said I, I've had to do a bunch of stuff personally to work around this in the past, um but I think that, given that there's no cap even existing, like that's something I'm interested in working on personally but there's not even a cap, yet um that's partially my fault. I should get on that, um but uh I think getting the inclusive language in in something that we can definitely do in the next release. And it's so easy for us to do that. And I think it's worthwhile.

A

So I'll uh I'll ask the the sig: now I don't I mean we don't have to go into a rabbit, hole uh a whole tangent, but how does everybody feel about a significant refactor and change? We we've done an experiment in the cli utils, using an inventory object to keep references to all the objects that have been applied.

A

That's a particular uh way to to solve this problem instead of uh instead of trying to use these proven white lists, and instead of um you know, trying to, we can actually only prune now in name spaces that are in the current apply. So if you, if you move everything to a new name space and apply, it won't get rid of any of the previous, it won't prune any of the previous objects in the previous namespace, because it doesn't even know about that previous namespace.

A

There has to be at least one object that specifies that previous namespace there's there's several uh pretty serious um drawbacks to the current prune all having to do with.

A

How do you specify what the set of previously applied objects were, and so the way we do it now is with the we do with the label or we do it with minus minus all which isn't minus minus all and and it's it ends up being causing these problems with the prune allow list and the name spaces, whereas if you specifically say and keep track of in metadata for one of the objects, we call, we call it an inventory object in the cli tills.

A

Then you specifically know, regardless of whether it's what namespace it's in, what the gdk is. You know whether it's a crd, um we we keep track specifically of what's been applied and we have. We look at that inventory object and the references within it in order to determine what that previous set of applied objects is um so this this would be a major undertaking and would would change, uh could could end up changing quite a few things um and would almost certainly get a lot of pushback from other parts of the community.

A

If we wanted to go down that particular path and follow the same mechanism, that's being used in the experiment, cla tills, but yeah- I don't. I don't want to hijack this meeting and go down that whole rabbit hole of that. But I I think that we we hopefully we can come to some consensus as a sig to determine whether or not we really want to fix prune at a with major refactors to actually solve the underlying problem of how we specify the previous set of applied objects.

C

A

C

We, when phil initially um kick-started the cli utils from what I remember that was one of the goals to to experiment, see what's possible how far we can improve the current print mechanism and my expectation, I didn't have any particular dates yet for that. But my expectation was that we will eventually get some results out of those experiments um which those results would be then presented in the 60 li as a set of options of oh. We did these this and that experiment.

C

These were the results of this experiment experiment. This is how we can improve the situation. These are the dangers, and these are the potential downsides of the approaches that we are going to take and then collectively we can pick which uh pros and cons we can accept and then eventually start the passwords changing the print mechanism.

C

That was the. That was the impression that I had in the past when, when the cli usually uh kick started, nothing came out of it. So I assume that since you're bringing this up, uh you would be able to put together uh a presentation outlining all that and then we can.

C

We can make a more informed decision as.

A

Okay, yeah, that's that that was. uh That was my impression as well that uh so I I know that I haven't gotten around to that yet, um but I it was on my plate to actually present to the sig kind of what we've learned from the experiment in the cli tools.

A

um You know where we ran into issues and- and you know what the things that we actually saw worked and- and let us all have a discussion on whether or not we want to create a cap to actually fix coop cuddle with any of these learnings with any of these particular uh methods that that we experimented with and and even some that we didn't so I mean we we considered.

A

uh I mentioned this to katrina already we considered a method called tombstoning which, um which is very declarative, but it's also uh kind of it also has its drawbacks, where you specifically mangle a particular yaml to like maybe put tildes around its name and you get rid of its spec so that when they apply, looks at it, it knows that you specifically want to delete this object because you've mangled the yaml in a particular way.

A

um So so, there's a lot of people who don't particularly like that method, but there's there's several different methods that we had considered and there was really only one that we ran with and and had experience with, uh with the inventory object that I was describing before, but I'll I will uh present to the to the sing.

A

Maybe I could do it as in a contributor at the contributor summit as a as a full session to say: hey, here's what we learned. I don't know if that's a good forum for it, but um in the near term I do plan on presenting to the rest of the sig. uh What we've learned.

E

You did give a brief presentation at last year's north america kubecon during our maintainer session yeah.

A

I I talked to all all of you, uh you and uh and eddie uh in in person. I think oh no.

D

No you're right.

A

I I actually I I think I talked about the problems with with prune, but not necessarily what the possible solutions were at the last los angeles, kubecon.

E

Okay, um I I also agree that we need to do something around this and getting the information back from the experiments is just like. Okay,.

A

Obviously, something.

E

That sounds great. um I would argue that we actually do need a cap about this, like we need to decide what to do with this. It's labeled alpha. We can't just leave it like that forever. I mean we have so far, but that's really not great.

A

um And there's also yeah go ahead. Sorry, I I interrupted.

E

um The other thing to consider is that the current implementation, besides having a allow list, which is not a great approach, it also is based on looking for the um cube control. Client-Side applies annotation on objects, so if uh we eventually completely get rid of client-side apply in favor of server-side apply which doesn't have that annotation, then we're going to have a serious problem where it's just all of a sudden gonna break.

E

So I think we do need to do something to it like we can't it's not an option to just leave it the way that it is for indefinite.

A

So the uh that particular code path, which looks for the last applied configuration annotation that was put there because we didn't want to prune objects that had been created with coupe cuddle, create or couple run so server side apply, hadn't even come, you know, wasn't even there when that was put in there.

A

That's my understanding, and so actually just removing it to accommodate server side, apply or or specifically looking for not just the last supplied configuration annotation, but also metadata from server side. Apply, would also be a pretty straightforward way to make pruning work with server side apply. I think.

E

Yeah there's a concept of ownership like did apply actually own. This object and now server side uh makes that concept more sophisticated uh where ownership becomes at the field level rather than the object level. So I think there's a question there like how like what should we look at to determine ownership if we stick more or less with the current approach and just make it less bad, like? I think, that's that's what the cap needs to decide like did.

A

E

Experiments yield something that we can actually move towards, or do we need to graduate the current approach as much as we don't love it like? What's the best that we could do with this, um given that people have been using it for, for you know, however, many years, but we can't just remove it so like what? What can we do to make it stable.

A

So so so my my personal opinion is that there's going to be a lot of pushback and there's kind of everybody's going to have a lot of opinions, and it's going to be.

A

um It's not going to be easy to make progress, um and I think that there may be a step where we actually decide whether or not we create a cap.

A

To be honest, I think that there's the danger that, if there's so much pushback in the community and even pushback within the sig, I wouldn't be in that camp. But if there is then you know you can put a lot of time into a cap that goes nowhere and it may be there's a pre-step to determine whether or not that's even uh going to be acceptable to the community.

E

A

E

Point is that we can't do nothing um so whether or not we.

C

E

Ambitious is kind of a separate question like that would be a detail of the proposal in the cap, and the cap would be saying, look there's this problem. We actually have to solve it one way or the other.

D

A

E

And explicit to everyone uh involved in this discussion that, like I know up and leaving it, the way it is actually is not workable. It's going to break things just if we just leave it like that people will break.

A

Well, it's it's been that way for 23 versions right so.

E

Well, but my point about server side of fly right: we need to work on this command. It will break if we leave it, as is.

A

Okay, I'll take that, uh let me put that reading notes is.

C

In a similar fashion, um I actually thought about generalizing the idea of we can't just leave perma beta karma alpha black and what sick architecture did, and that was implemented. I don't know it might have been two years ago. I remember that the crunch of api was affected by it. What they did was um they explicitly said and put a timeline on every api that you have literally six months from today uh to either promote the api or it will be deprecated and entirely removed.

C

I guess that we should think about having something in a similar way, even though this uh architecture focuses primarily on the on the cube api server apis.

C

um If you think about it, every single command, every single flag is in some way api, and if you go through the deprecation or in general api uh policies, the cli has its own section and we should probably think about it um and and figure out a way in the general way how uh how cute kind of should start treating those commands, because there are places where, where we have certain flags or certain functionalities and uh perma, beta or kind of or even purple, alpha state, and we should be explicit that oh we're we're gonna work on it or we're gonna, entirely drop it and a clear path towards one or the other would be a good signal.

C

And if, if there's someone interested in in pushing this forward good push this forward, if not we're going to drop it, we can't fix everything everywhere for everyone. So that's that's. That would be those uh uh uh some safeguards for us as a stick to ensure that stuff are either moving forward or after some uh period are removed from testing, and we don't have to deal with them.

A

Okay, well I'll. Take that as an action item for me to create the presentation on on what we've learned um on some of the ways forward to fundamentally better the apply prune.

A

E

um Would you mind if I made a cap, it wouldn't.

C

Have like the solution.

E

Part in it, it would like have um remove it entirely fix it in place. Use experimental, see uh sorry learnings from sixth seal experiment like at that very high level. Just to make the argument that we need to do something as sort of a starting point.

A

Yeah I mean I I you know I'll do whatever I can to avoid doing a cap. So if somebody else.

E

Yeah I don't mind uh championing that I like, like I said I, I maintain a tool that that relies on this feature, and um would you know as and I'm very aware of the ways in which it is broken and not aware, but also and um sorry, the ways it's broken. But I'm also um aware of how valuable it can be and would rather that projects like mine and many others in the community don't have to uh figure out independent implementations, because because we abandon it upstream.

A

Yeah, I think it's fundamentally broken. uh As for the reason I mentioned before, of the limitations on how to specify what the previous applied set is, and I think that you're gonna get just like, so I think that you and mache have the final word uh on on it, because you guys are the tls for six eli but I'll.

A

I you know, I it's been my experience that everybody and his brother has a different opinion, and is it it's going to be a um yeah it'll, be a you know, quite a political act to to try to get a fundamental change that really fixes this. um I I'd like I'd, love to see it happen and, and you know, I think that we shouldn't underestimate the power that that we have, because this is, you know what what we own.

C

A

E

Hoping that framing it as like, we need to do something here. Are our options um instead of like an option, is to leave it in permanent alpha because from the six perspective like, I think, that's unacceptable. So we need to. We need to to pick a way forward um explicitly, instead of sort of leaving it in limbo, I'm hoping that framing would help, but I I agree with you sean that that's been controversial.

E

I saw the previous attempt at this and I yes, there was a lot of discussion and then it just dropped so uh yeah. I'm aware that this will be difficult, but thank you for the warning.

A

Okay, um so does anybody want to uh so so? Can we move on to stand-ups.

A

Okay, does anybody want to uh volunteer for any standups.

E

I've been talking a lot, but I can do a quicker one of top of mind for for customize. um There is an issue that we have that's sort of in line with uh the general sig and even actually general kubernetes initiative to improve code quality and uh make sure that we have adequate testing of all the features. We're supporting there's one particular sort of group of features that are under tested and customized.

E

Where basically enables you to specify a remote customization um and then it will process it. So the standard thing is, you have all your config locally reference and other customization of your tree, but it's all local. um But yes, this feature lets you like point to github, say and we'll pull down a repo for you and and build from there, so uh that whole sort of area is under tested.

E

We want to improve that and we actually um put a code freeze on the loader that implements that functionality um until there's better testing in place, so somebody has stepped up to try to test one class of the under tested code, which is the use of ssh protocol to to retrieve those customizations from github.

E

The problem is that we don't have in our in our ci um the ability to actually uh pull from github other than via https, because github so uh to rewind a step here. We have two different ways that we run tests. We use pro for a few things and then the majority at this point are running in github actions.

E

The github actions by default gives you a token that you can use that allows you to clone a repo, a public repo over https, but it doesn't give you anything to be able to to do a clone by a by ssh, which um we need to do to be able to test this. So we were exploring some different options, and um here we go. Let me see if I can pull up the pr real, quick.

E

um One of them would be to commit an ssh key into the test, suite um which seems kind of sketchy, um because that's like that's an identity, so if you it might be attached only to a read-only token right now, but there's no guarantee that that will continue to be the case. So from that perspective I don't love it. um Then.

E

The other uh approach is to uh switch the ci over to a sort of a more dangerous version of github workflows that allows secrets to be read from the workflow um and just so that we can expose this one secret. That is currently a read-only token.

E

So anyway, I don't expect to actually resolve that on on this call, the reason I'm mentioning it as stand-up is that if anyone has experience with this sort of thing running um either just more advanced github actions or uh specifically with running uh or in prow, actually that could be an option if pro has a safe way to do this that I like, we couldn't dig out of the docks.

E

Something rings a bell for. You please uh reach out to me on slack, because we're trying to figure this out and I'll drop once I find it here, a link to the discussion on the customized repo for anyone interested to review. If there's somebody help us out um planning on posting this in sync testing to see if I could get some opinions there, any suggestions uh before to get help as well would be welcome.

D

Katrina, have you um have you asked anyone at github about this yet.

E

No, I can't say I know anybody at github.

D

Okay, we talked to the github actions pms a bit so I'll pass this on to them, see if they have any ideas for you.

E

Oh, that would be awesome. Thank you. I found the link. Here's the in the chat, I'll put it in the notes as well.

A

Yeah, so let's let me know if those notes um actually accurately reflect uh your your comments. Katrina.

E

Thanks I'll take a look.

A

And thanks for for giving us the update, um so I think we could we've got uh three minutes and maybe we'll end this three minutes early. If we don't have anything else, does anybody else have anything that they'd like to share with the sig.

A

Okay, thank you very much for for joining us and we will see you within the next two weeks for the bi-weekly meeting. Unless we have the bug scrub next week or if one of that the one week off okay, we'll look at the schedule, see you later.

D

D

Hey sean, don't jump off after.