Kubernetes Cloud Provider Special Interest Group, 27 Jan 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Cloud Provider Extraction/Migration Subproject - 2022-01-27

Description

KEP Review

A

This meeting is being recorded.

B

Welcome to the january 27th 2022 sig cloud provider, cloud provider, extraction working group meeting: this meeting is part of the cncf and, as such uh we adhere to their standards for behavior so uh which amount to please be uh considered, inclusive and polite to all your fellow members and contributors. uh So yeah, please do so.

B

uh We have a pretty short agenda item uh from andrew, I think, there's the question of which caps do we want to get reviewed before the kept deadline, uh and I think the first of those is for the cl uh cloud controller manager, my leader migration.

B

um So uh I think the big thing there is. We need to review it for uh adherence to what we believe to be the ga requirements.

B

uh Before I go into any depth jared. Did you have any comments you wanted to make on that? One.

A

uh We already have this discussion last time, so yeah like oh, I have all I did is you know for everybody put what everybody said in the recap.

B

Okay, um there was one interesting point uh which came up during the review of jared's uh ga requirements uh and I think it's uh pertinent to the whole sig. Interestingly, there is a requirement for ga that uh any apis be.

B

What's the term comply, um I'm thinking conforming conformant, uh that's actually a little challenging for our sig. uh Since cloud provider, specific uh specificities are not supposed to be tested, in conformance.

C

um Is this an api though, like it's a, I know we have a type with a kind, but it's not a resource. So does it count as no api.

A

B

Well, yeah, I mean it's, it's sort of an interesting one, it uses a resource and but arguably that resource is agnostic, uh but the some of the coding around it. Isn't it's a great question. I think we just need to be very careful uh about the performance piece and make sure that we're okay yeah. I think that I I I think that if we need to, we can ask for an exception and make a good case. It's just that.

B

I I think we need to be sensitive to that when we go for the ga requirements, be.

D

B

B

Other than that, I think you know the general thought is we want to make sure that we have at least one automated test of uh an h, a upgrade.

B

Where the ccm code is used separately, we may also want a unit test that actually confirms that the behavior that we want out of this is being enforced, and I don't think those need to be the same test.

B

But I do think that needs to be spelled out in the ga testing requirements.

C

I think we also need like rollback testing, but I know rollback testing is really hard to automate so like even like uh just doing it manually a few times and just jotting down that we did. A rollback test is probably something we need.

B

That's fair, I actually feel like. If we do upgrade testing, uh then the system that does the upgrade testing can probably do the rollback, testing, um they're sort of part and parcel of one another, but we can we can deal with that when we get there. The other thing I will say which I've mentioned before is uh this has to go to ga, but it doesn't actually have to go to ga this soon yeah. So I think it is good.

B

I mean it's good to get the ga requirements set if we don't actually successfully uh get the migration to ga this uh release. I I think it's still good to have gotten the requirements done, but we can see where we land.

D

Just all right, just a question for andrew there, when you talk about rollback testing, you're you're, talking like rolling back from ccm back to kcm, or something like that right.

C

Yeah and like even even ideally, between a kubernetes version where, like this got, enabled and then disabled, along with the component migration yeah.

B

Okay- and I will say I mean if you read through the cap it the cap talks about rollback, so I mean it does make sense. The the entire steps needed to go from migrating from uh something like the service controller running in the kcm to running in the ccm. The the cap also speaks about running it in the ccm and then rolling back to it running in the kcm, so it is actually covered by the cap, and s I mean I would say would have to be tested anyway.

B

But given that it's in the cat properly in the cap, I do think it needs to be even more reason. It needs to be properly tested.

D

And I'm making a note here to go back and reread these things, because I don't but you're talking about the this um leader extra leader migration cap right.

B

Correct yeah, because.

D

Everything was the leader.

B

Migration camp I mean granted. I say this, I I remember it being there, but I haven't actually read that cap. In probably six months.

D

I mean yeah like I, I don't recognize it when I look at it now, so I'm thinking I should go like spend some time reading. It.

B

All right- and then uh there were also someone says minor tweaks to the cubelet credential provider- is that from you.

C

Yeah I just yeah it'd, be great yeah it'd, be great to just get a lgtm from someone from the sig. That's not me um really. I I think the only thing I tweaked so like we, we set the beta criteria like uh last release. Sorry one sec.

C

Sorry um we set the beta criteria like a release or two ago, but then like we never got to them. So I think this release is just marking that we're gonna work on it in 124 keeping the beta criteria's the same, but I did add one extra beta criteria, saying that we need two working implementations which I think we already have right like we have the gcp and aws one out there. So I like that's kind of a free one, but just putting it there as a formality.

C

B

Quick on that subject, uh I know kermit was working on the windows gcp credential provider. uh Do we actually specify in the cap that we that someone has to have shown that the credential provider works on windows.

C

No, I think we just say like we want two implementations out there, that we know work to some degree because.

E

Who's looking into windows for us, so as soon as they I don't know, get something I can report back.

B

Okay sure now I think that kermit had built it, but I don't know that we have automated tests around it and I do feel like uh for beta. That should probably be a requirement.

B

C

Sound like sorry, I actually did I added adobe testing so for beta, I said that we need integration or uv tests on with one working implementation and then two working references of like actual providers, but I was going to defer like extensive edu testing for ga, because I feel like that might be a bit involved inside like the yeah in terms of some of our scripts.

C

I support doing.

E

Everything we can to get it to beta.

B

I'm all for getting it to beta asap, but I I was under the impression that the cloud provider gcp automated testing had this turned on, at least for linux.

B

So so we can double check with kermit. So let's let's follow up, but the nice thing is all we have like. There's two things: there's testing anything specific to the credential provider, but a large amount of the testing can just be turning the credential provider on and then running the ee tests, and I think that may be the state we're in in cloud provider. Gcp.

C

Okay, actually yeah. I was only thinking about the kubernetes kubernetes test, but yeah, it's possible. That jsp1 has this enabled, which would be a huge win.

B

C

B

I think that's where we're at, but I don't think they're turned on for windows, so I think.

A

B

Be good to just have either us or amazon, or someone have a run, a running e to e test. That proves that there's nothing specifically about cubelet on windows or credential provider on windows. That causes a problem.

B

Okay- and I expect there isn't, but it would. I think it would just be good to have that confirmed.

C

Okay, so so maybe like that's a good decision to make like for beta, we just have to ha we for beta. We want to make sure that the on the implementation side, we have e2e tests um and then for ga, we can say, like the whole kubernetes suite, like kubernetes core suite, runs on the cringe provider as well, which is going to be like a larger effort, but give us more time to work on that.

C

B

Awesome, thank you. Andrew um did. Anyone else have any other caps. They wanted to go over.

B

So um as a side note nick and I met earlier this week, uh and so I think we tentatively and correct me if I'm being dishonest here nick, but I think we tentatively have a plan for getting some of the alpha stuff needed for web hooks in the ccm done in this release.

E

Yeah, that's your plan. Yeah all right.

B

Cool and then um I'm behind, but the other thing. I think that is vitally important- that we need to get done, and this one's on me is enable uh no cp on cube up.

B

And I think that's needed for the the entire cloud provider feature uh testing effort, because that I think that's going to uh sort of highlight a series of places where we're not getting the testing. We need.

D

And what exactly is no cp? I'm not following. Oh no cloud.

B

Provider, oh okay, gotcha cool, so we have. We have two feature gates that enforced no cloud, uh no cloud provider, one does it I mean andrew can speak to this better than I can, but one of them enforces uh node cl no cloud provider on the uh main binaries, so cubelet cube up or sorry cubelet, cube, api server and cube controller manager, and I believe the other one enforces that there's no credential that the credential provider isn't using anything cloud provider specific right, andrew.

D

So one of those is like disabled cloud provider feature gate. What do you recall what the other one's called? I just I'll do some reading on it.

C

I think it's called disable cubic credential providers.

D

Thanks appreciate that.

B

Right and so those are theirs, you know those are currently uh feature gates in alpha. uh They get turned on in beta, which basically means all of our testing breaks, and we know this for sure. So um I think it would be good to have one set up one test suite proud job. That basically shows uh what'll happen when those get turned on, um but the weird part is that they break they don't bring. I know this sounds like a weird thing to say they don't break in the correct way.

B

Currently so right now, if you turn at least and- and let me be specific- they don't break in the right way for google. So right now, what happens? If you turn them on in with the cube up scripts is the cube up, will enable cloud provider and then the feature gate just kills the binary, which is correct, but not what we want to test. What we want to test is you turn off the credential, the key we've pacified to turn off the credential provider.

B

We use the feature gate to enforce that and then we see which tests fail because the cloud provider isn't there not because the binaries aren't running.

D

Gotcha I mean that makes perfect sense to me.

B

Cool um yeah, so anyone have anything else. They want to chat over.

B

Nope now I'm going to stop, recording and give everyone 10 minutes back thanks all.