Kubernetes SIG Windows, 22 Oct 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Windows 20191022

Description

Kubernetes SIG Windows 20191022

A

Okay, DUI start at Michael or stag I'm.

B

Glad since you have that according to everything, so go ahead, okay,.

A

Anyway, hello, this is October 22nd and our weekly sig windows meet up, and so this is being recorded be sure to follow the kubernetes community guidelines and in all of your discussions, we've got the agenda posted a channel ahead of time. It looks a bit short today, so if anyone's got other topics, they want to add, you know if you just drop a note in the chat and and then we'll be sure to call on you.

A

And so I guess, let's get started with that Oh with one one announcement so last week was the enhancements. Freeze. I did need to get a brief extension for the runtime class work that I had proposed that was approved and then we got the kept merged. So that's on track. So I'll just put a note on that.

A

So so I'll be moving forward there and so I'll just add: that's the agenda and talk more about it later. Are there any other exceptions that anyone's aware of for caps or is everything final I.

B

Mean the CSI kept it. We we're gonna, keep it in alpha I write deep, but yeah.

C

B

Then the GMA say they asked us to keep it in beta for a more site costs or so that didn't need an update. So I think your cube is the only one right, Patrick I, don't know of any others. Okay,.

C

What about the comedian stuff, yeah.

B

We couldn't find resources for that. So I, don't know you know. For now my you may be able to work on it and maybe I then just graduated with, like you know, kind of slipping it that through, but we don't with no resources right now and I believe Ben and his team are people that are looking to see who they can find some resources outside of band. That's fully booked, but another decision. Yet.

A

Okay, Adaline is doing some investigations to see if, if we can replace one of their current test passes with one using qadian, which would help so, okay, let me go back to the top of the list there. So are you on the call Kalia.

D

A

So I see you added something about local traffic policy. Yeah.

D

So, basically, what we're seeing right now, not what we're seeing so for 116 we introduced this feature, preserved dip, so that's signified by an annotation and so based on that. What we do is we preserve the destination IP and we disabled a routing mesh for load, balancer ingress IPS.

D

So this is what we have an internal customer at Microsoft. That is using this for their health probes. But the issue is we didn't disable routing mesh for node port based on this, so I can make that fix and it'll be in 117, but they need. Basically, they can only use released binaries, so I can't give them a private and I was hoping that I could back port this so that I could disable routing mesh for node port, because what happens is when they hit the node port.

D

I mean it gets load balanced across all of the nodes. So it's not really an accurate indicator of if the node is often healthy or if the pod is up and healthy, so they black hole a lot of traffic.

D

A

What's the case that works, the.

D

Case that works is for load balancer, ingress IP, so it just be a one line: change where I also disable the routing mesh for node port.

D

So yeah, that's what I'd be interested in backporting I, don't know if it's 115 or 160 and I'll have to check when I first introduced that oh I think they're using 115. So it's like a back port to 115 or even 116, just something that's gonna be faster than the 117 release, and so that also kind of ties in to the local traffic policy.

D

So, basically, for that we need DSR, and we have an issue filed on this right. We need DSR, which is hopefully going to be supported in vibranium or is supported in vibranium, and we also need to disable routing mesh for that.

D

So there is the thing where I can at least partially fulfill that and disable routing mesh, as it were like if that flag is set, or we can just wait and say: that's fully unsupported right now and wait until DSR is available and then from that I would want to know if we'd be back porting, that type of change to previous releases. Once the release is out.

A

Okay, so I'm gonna screen the issue right now so so this particular open issue has been moved to 17 yeah.

D

And I think that would.

A

D

A

I mean no matter what we have to check in a fix there first, but this issue doesn't talk about node tours well,.

D

That's the definition of a.

A

D

The definition that poor load bouncer and for node port type services, we would disable routing mesh.

A

Okay and it's a sanitation specific to Windows, or is it there with Linux as well notice.

D

Annotation is specific to Windows we kind of not get in last minute for 115 and the only customer. That's using that right now, it's internal or as far as I'm concerned is okay.

A

So so I'm concerned about here is the guidance that I've had before is that is that, generally, the dot releases are not for new features. If there was something that was, you know intended to work but was broken. Then that's where they'll accepts on the back ports, and so like one example was we had a disk calculation that was bad. It was causing pods to be wrongly evicted. That was.

E

A

For a back port versions, 14 and 15, but we checked the fix directly into sixteen first and then back ported yeah.

D

I know we had a bug where, if the node port was the same as some other yeah, it's usually for bugs right, yeah.

A

Is there test testing in place for this, because that's the other question that.

D

Testing in place for this I don't think: we've even really unknownst this as a feature except in the changelog. So okay, so the first thing is: I get disabled, a routing mesh for node for it. If that annotation is used and I think I would consider that a bug fix. So we can hold off on the traffic policy, but.

A

Okay, well, I mean I, don't think these two changes are are mutually exclusive of each other, so I think the right thing to do here is is if the intent was for node port and the look.

A

Sir ingress to both be covered under the same flag, go ahead and submit a PR for that 217 first and then we can look at the scope of it there with the release team and see if that's something that would be acceptable or not so.

D

Would this be fixing prison? Okay, so preserve tip also has that we preserve the destination IP and the traffic policy says we preserve the client IP, so these are kind of I mean they're related, but.

A

D

A

So you're saying you would need another. Annotation is what you're saying no.

D

We don't need another annotation, so basically DSR handles the preserve the like handles preserving the source, IP right so.

E

D

A condition of the external traffic policy or nodes local, whatever that's called so we need DSR for that now preserved if needs DSR and the preserved if flag. So it preserves both the client, IP and the destination IP and disables routing mesh. So as a feature it encompasses more than just the other one, but.

A

Dsr is going to need Windows, Server, 1909 I. Guess that's what it's going to be called.

C

D

It gonna need something.

A

Should later than that, no.

D

I, don't think it will need anything later than that the thing is.

D

Like DSR has different parts of it supported, you know so like right now, DSR is supported for some scenarios, but not other, and the complete support, I think, is what we're waiting on.

A

It's partially supported in 1903.

F

Right now and it will be 49.

A

Okay, so I mean so I guess so. What's the next step here, I'm still not real clear on that? Okay.

D

So, for me, I think the long-term is to support external traffic policy, but in the short term, would.

A

That resolve six to zero, four six or six to zero. Four six, a bug-fix rather than a feature, six.

D

Is that the one is that the one that was filed? That's marked for 117.

A

Yeah, that's in the meeting notes. I feel.

D

Like that, one would be more of a feature because we're gonna implement that in I mean it's an existing feature. Kubernetes, that's not currently implemented in Windows, so I, don't know how you would classify that.

C

Well, it's an it's, not an enhancement issue right, so it's not a future technically yeah.

D

C

To be a kubernetes kubernetes issue, just like.

D

Yeah, it's not anything new that is being designed, so I think it would be a bug fix, but the more pressing issue is the preserve dip, which is a feature I suppose that we added without a cap, but we added it and so there's a bug where we also need to disable routing masternode for it. The problem is we're, gonna have to explain, preserve dip and then we'll probably also have to provide I mean I sense that there's going to be some resistance. There.

A

A

So I think what would be most helpful here is, if you could get a specific issue opened, describing the change that you want to do for the prefer to preserve diff, because that's going to be different from what from the issue that you linked in the notes. And so let's get a discussion done at the right scope, they're describing to fix I, think it's played to the PR that emerged that I have linked there.

A

So go ahead and get a specific issue filed for that, and then let's get emerged into seventeen and then and then we could. We can talk about making a determination after we see what what's. Actually there I think it's definitely worth adding tests to this, because what you're describing is something that's going to be. Could change based on the Windows version there and so I think we need some way to measure that yeah.

D

Ok, I'll create the issue and then I'll create the PR for 117.

A

A

See Bend yeah, it's nuts there did. You have any other questions on that.

G

No I mean this was just the first I've heard about the preservative feature. So I was just trying to understand what it was and give some reference or other people who are similarly curious.

A

Why Julia billing fare in the dock.

A

But that's just something: okay, if you could just fix the link in there, then that puts awesome. Okay, so deep here next on the list, hey.

C

So what I wanted to talk about was around GMS a testing, so I put.

E

C

The pieces for e to e right now is in the end-to-end tests modified everything in the web hook and all the enhancements that Jeremy did in order to make sure that you know the necessary 80s extensions are in place, are all checked in so.

E

C

Was wondering if at some point in 1:17 Adelina can give the GMs a test a shot and 2n? And then, if it works, Stapley, maybe a few times, then we can remove this KMS a desk life from the EQ industry.

A

So is this in a staging job, yet.

F

No, it's not a staging job, yet mm-hmm, so I got it there, but we still have to talk a little bit about where we would put this this job. Would we keep it in staging batch ones so that that one runs every six hours, I think now or four hours I remember. But the point is that, with all the it's an extra machine that we need to deploy, so what do we want? This for every run.

C

Staking is fine for now.

B

We're changing GM sa stuff, very often right. So is there any type of run that we do outside of the every six hours.

F

There's a master run, that's sequential! That's a little bit, but just just the question: cuz I am a little fuzzy around the gym is saying so.

F

You deploy an extra node as the vc controller, the main controller, but that's schedulable right right. Oh so it's basically just like any other node.

C

Yes, it's just that the test itself cannot be scheduled on that note, but other than that other tests can run other.

F

Question, oh then I guess we could just minister it because now we're running with three machines per cluster, we're just splitting them into agent pools I! Think we can. We can keep it.

A

C

You have any concerns with that, like everything should be runnable on the DC, just as any other windows, yeah.

A

Absolutely it's just another member, like you said you just can't run the actual GMS eight.

F

Agent pools into.

F

The controller.

F

Keep the same number of VM, so it shouldn't be a problem.

C

F

A

Yeah, yes, so I could look at the results next week and then move forward yeah, because when we graduate that to stable we're going to want to cover it in it, in that same test passes everything else.

A

Okay, anything else on that.

A

A

And so the last thing I had on there was I had a link to the runtime class stuff.

A

So at this point we've got the two kept merged as as implementable, but there's quite a bit of work. That needs to be done here and so the first thing that that I was going to be working on was fixing. Some of the test passes that we need over in over an aks engine such that we can get container D deployed in order to be able to test this stuff, and so that's going to take me probably a week or so, but there's a couple other items in there that could be done in parallel.

A

If anyone wants to wants to take that long, the first one was, we got an agreement that it's okay to add a new flag describing what the windows build is from the cubelet, and so we within the cap I've got the source code linked, but it should be. You know a relatively straightforward thing to add and then add unit tests for, and so, if someone you know wants to take that on I'd be very grateful. It's it's a separate task. They can just be kind of done and tested in isolation.

A

It doesn't actually require container D at all in order for that to go. So if anyone wants to take that, you know just just pay me a slack, and you know let us know.

A

The other group of changes is actually changing the CRI definition to add the runtime class Handler and that's something that I think can be unit tested with with the CRI test framework, but I'm not sure whether or not the CRI API repo actually has tests that work with no CRI implementation and so may be possible to make the API changes without a working implementation.

A

That's something I haven't looked at yet I, don't think we have a zoom of familiar with that section.

A

Well, I get short of it if there's anyone that wants to start looking at the CRI API changes, I'm pinging some folks here at Microsoft, to see if I need somebody to start looking at it. Otherwise I'll be looking at in probably a week or two well, those anyway I just want to call those out as areas where. If someone wants to get started on those earlier, you'll be happy to have to help so.

A

Okay and then deep ooh yeah, it's.

C

Just like an update on the PR, we were wondering if it seems like it's LG T, and this is the PR from.

C

The one around pushing test images up, we were just curious. What are the next steps on that? Probably just getting it approved.

A

Do you want to talk about that cardio.

H

I'm sorry, which are first.

F

Part one from add windows, container images support. Let me paste the link.

H

So basically, this has been reviewed by quite a few people, and it's got some energy team energy TM. The next would be to have someone from and some more windows nodes to the image promoted that they have and configure the the jobs also use the remove the URL config options at the moment regarding image promoter, it's something that's that has to be done in community sleep. Oh, it's currently, student progress, I have a couple of requests which also helps with that I.

C

Was just curious red was cuz. We were tracking it a bit from the CSI side of things cuz. We need to push a couple of containers to other other peers linked to this any chance. Claudia yeah.

H

There are three parts to this particular spot: one or two or three: you can track them by it's both okay.

A

Okay, all right, that's everything on the list and we're out of time any other last-minute updates from anyone.

B

All right have a great day. Everybody and Patrick send me the link with the download. Thank you. Everyone.

A