Kubernetes SIG Windows, 14 Jan 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Windows 20200114

Description

Kubernetes SIG Windows 20200114

A

Hello, everybody and welcome to another sip windows, meetup it's their 14th of January and were back on track, getting started for the new release of Canaries, as always is a recorded meeting. So please adhere to the scene, CF code of conduct. Alright, let's get started. I know you guys did not discuss the one agenda item I added last week: I think you may have waited for me if I'm not mistaken.

A

So so, let's talk a little bit about that right now, unless partying ago, based on the 118 plan, you know start the first.

B

Yeah I think the 118 stuff is gonna, be pretty quick and so I want to just get that out of the way. Let's.

A

B

So your enhancement, you freeze, your caps- are: do need to be merged by January 28th code, freeze, March 5th, pretty similar schedule length. So the last few releases I didn't want to mention. There's a couple: a couple issues that we kept getting pinged on, both on like slack and also on the issues themselves. One of them is around the CPU limits, so I've got a P I finished. Writing some tests for that yesterday and I get a PR open for that.

B

So if anyone can take a look, that would be greatly appreciated and then the next one it's on the list, that's sort of related is the one about the cubelet metrics being pretty slow, I think we're probably just going to go ahead and cache those so that way, overlapping requests don't consume all the CPU.

B

What I was going to work with mark on that next, because the tests I just added for basically figured out how to write tests for that, while I was working on CPU limit and so we're gonna we're going to pair up on on that problem and see what what we can do there as far as features graduating this cycle, we don't have any changes, plan to run as username and so I think we can just go ahead and promote that unless someone has some objections on that make.

A

C

D

C

Other thing for the maybe the high priority think Patrick yeah we've been getting a lot of requests about why graceful, they're, graceful termination and the shutdown period isn't honored I think we should update the docks with. There are a couple of registry settings. You can add to a container that will have docker give you an extended shutdown, duration, I, think we should document that, because we've been getting a lot of requests for that. Okay.

B

Cool yeah, yes I'm at them, yeah I, think that makes sense, but yeah we can do those just at least as a dock PR. For now, if nothing else, yeah.

B

Okay and I see someone just added something about GMS a to the list, so.

A

I think deep was in our agreement that we're gonna keep GMS a to beta for two releases and then promoted to stable, and I think we're there right now right so.

B

It's deep on the call yeah.

A

B

Did we have any remaining work around the admission controller there before we get this to stable, like we need to set up testing on that branch or anything we.

A

I think we added testing that also Jeremy added and, if there's anything else missing, are you muted or Jeremy I'll know if you know what the study the status is on, that yeah.

E

It should be fine, I mean on the I write. The testing that we wrote does actually end up testing the web book well hook doesn't work. The test won't pass.

A

We'll come back and revisit it just in case Steve comes later on: okay,.

C

A

A lot a lot I'm not here able to confirm.

B

A

Then we wanted to put Cuba diem to bed all right, so I don't know so so I think Ben. You had asked a question a couple of times: do we need a new cap, or do you want to amend the caps in the process changes now? What did you get an answer on that? No, that.

F

Was I think what I thought? Maybe we discuss here, yeah.

B

Yeah so I think we should look at Ben's talk and talk about this because.

C

B

We want to change the design we shouldn't do that between alpha and beta. We should do that, and but it's it as alpha for for release the.

A

Problem with me, this person is perception right, so I don't want to keep it as alpha one more release, I would, rather, if you have, it, is better more people will use it more people have a little more confidence on it.

A

I know changing the design is like starting fresh, but you know changing the design based on lessons. We learned from alpha towards Greta right, and these are opportunity, unless you know for sure that we could change it again later, who this may be unlikely?

A

My personal preference is to market beta just.

B

I can get more usage.

A

B

Let's look at the design stuff and come back to that. I'm also trying to get some details on how privileged containers would work and that may require changes as well and so I'd like us to to kind of think things through that angle.

B

In addition, but.

A

That make sense.

B

But yeah average sorry about that in more detail in a few minutes.

B

Well, we've gotten anonymous and Yan cat.

B

Okay, and so after those bugs that mark and I mentioned we're going to continue down the container D path, Lantau and- and you and and again I'll have those tests running on on GCP as well. So we do have a test signal. There I think there's still a few bugs.

B

We need to work through and getting some of the stuff up on other portions of the azure, like the azure to see and I done so to continue work on that, but we're putting some of those bug fixes at a higher priority than that, because customers are are raising issues about those for the issues that are already or sorry for the releases that are already out there.

B

So, rather than waiting a couple cycles for container D to be declared stable, I think some of these fixes need to come before that, and so that's kind of the order. I were working on stuff in and CSI proxy did you have any updates on that deep or I? Don't think KK could make it at this time.

G

Yes, so the work is in progress. I think Kiki started pretty much a new branch where his he's good he's, integrating it with I sure disc CSI driver. So so you know pretty solid progress on the disk volume and the file system. Api is there kind of more incompletion, so alpha and 1aq should be. Let me look at it. Okay,.

B

Jen, do we have a cap open or something that needs to change already, or is it already.

G

The cat was already filed so basically we'll make them men commence to the some of the EPA's.

B

Okay, excellent, but there any.

G

Other question around, like other any updates around the privileged container up front for Windows. That's yeah.

B

I'm still trying to I'm still trying to get some more details on that.

B

So I talked to the people that were working on that and they've got a plan proposed and they're trying to get somebody to to work on it. You know like over the next next few months and so I'm trying to see if I can get kind of, like a public version of that doc out for review it so kind of the the main that impact is.

B

That is something that would only work on an on a new version of Windows Server, and so it's something that we'd probably be able to start testing at the earliest for 119.

B

Just because I don't see there being anything before code freeze and you know and I, don't know for sure. If it's something that would be available, you know between March and June or if it's even late or if it's later than that so I'm trying to get some more details on that. So we can use that to to reference in our planning.

B

When we're talking about these privileged proxies and stuff like that, because yeah then maybe we start on a road map to figure out, you know what we would want to change in those and or sunset them later all right.

G

Makes sense yeah, so then we continue working on this a proxy until we hear more about those clients and then if the whole functionality is there, we can make a call at that point. Yeah yeah.

B

But I think for my initial look at it. I do think that something like the CSI proxy could be run in a privileged container.

B

G

B

So basically simplify the deployment a bit, but you wouldn't necessarily have to change the VAP. Is that it's using that's my initial read but yeah you'd still need more detail.

B

Okay, should we move on to your dock Ben sure.

B

Actually did I put the right link on there. Let's.

F

See no, it looks like it's a github link.

F

As the one that I shared out and.

F

um At least you had that link there I'm updating it. Okay,.

B

F

It should be this and the right thing now to a Google Doc, but this is the same thing that I, like this past past week last week, yeah.

F

So it's just kind of a summary of the the strategy of using this win s. Proxy I could to be able to run well so, along with cube, ATM, to have an approach to running.

F

And cube proxy in a way, that's more, it mirrors Linux does things with like host network odds.

H

F

Think I heard the Nash talking about is like super faint.

H

Find the gaps between Windows and Linux and we have been trying to propose a bootstrap provider that can work similar to how the Linux cloud and it works so basically have the cloud, and it is today giving us the cube, config and basically the join token in order to join the worker. Node right, you're.

F

Talking about some custom, API stuff right now just want to be clear. This is just about I mean we can talk about that too, but it it's kind of broadening the scope of the discussion, but I.

H

We are doing a few bits and pieces everywhere and, in my opinion, if you keep doing this, we will never be able to basically Gorge keep the same flow as Linux right. So.

H

Not having a cloud in it, for example, because when I look at this, you have provided a mission deployment yamo, and in that you are running partial scrip and we are doing things out of ordinary way. I think.

F

You I think you're still looking at the whole the link to the github about coaster, API and there's some thing.

H

F

H

F

Like I mean it's about, win us as.

F

We just win us in Cuba D I'm, not talking about a cluster API work, I.

H

Think this is another thing that I feel like privileged container is the right way to solve the problem, because Venus will keep. We have to keep adding kpi's across its inverse and across its es. If we go through this route, I I've been putting pressure on the base team to basically timeline on the privilege, container and they've been telling every iteration so I. Think if you put pressure I, don't know what do you think.

G

H

B

So so the winners proxy can just run an arbitrary process, and so this would give you a way to potentially ship something and then run it on the host. It's not as clean as running in a true container, but it lets you run arbitrary code, and so you know for trying to move back to using the same sort of cube, ATM phases that are used by both cluster API and cube ATM itself. This is much more consistent because it still relies on getting configurations from from kubernetes and CR DS.

B

It still schedules things with the demon set and what I think what I'm hoping is feasible is that we could use things like the CSI proxy and win s today on Windows Server, 2000 19. So that way, we've got a solution for people that want to use cube ATM with that LTS version, and then, when a new version of Windows is out, that hat offers a privileged container support. We could say you no longer need the proxy or you can just run migrate, the proxy and do it into a privileged container.

B

If you don't want to change your code right away, the.

H

Problem there is so we have to like basically flannel to proxy, so every API that we are trying to make a call or, if you add any new feature to flannel or if you have to write that code and no.

F

Like witness, isn't proxying syscalls yeah.

B

F

Just it it, it exposes an API to launch processes like from a host service, so outside of a container, it does have some api's for doing other kinds of things, but the main one that we're using a really the only one we're using is for launching processes outside of the container yeah. It's.

B

So in the POC that you did bend flannel ships in the container, but when s locates it on the post from using the host path and then executes it right, we.

F

Copy it using a volume, a host Pat volume, mount we copy it from the container to the host file system, but yeah.

H

F

Think that might just be other features that were built for other use cases I.

H

See that's what threw me off because he has a lot of business calls implemented and so you're saying that you run the container as a process on the host right.

H

So none of those a china set sales calls is used now.

B

And so I think that gets to one of the questions that peter raised in the doc, which is, if we're just using this to basically launch stuff. Should we possibly consider forking or adding flags to remove that extra step from when s so that there's less surface to maintain their attack? Yeah.

A

And actually I wanna add on top of that, if possible, after we're done with the bootstrapping, to even need to keep winds open with an in pipe so kind of just shut it down. Well,.

F

At least well, so both flannel and cue proxy will contain you running and since those processes are still managed by win us win, us still needs to be running yeah.

B

So if you want update your your Q proxy or your final config, if we Ness is intere, it then you're broken got.

A

It so there's no way for us to leave me touch surface yeah, but.

F

Yeah I mean that's the other I think big thing, but as my argument for that is just that, it's the same attack service pretty much as privileged containers, in-house Network pods on Linux. Today, with maybe the caveat of it's a little less tested on it's a little less like well.

A

The one advantage of of privileged part is that they can use a pod security policy that will disable those well, in our case, is there a way that we can limit that yeah.

F

So, since the the way that you use win us from a container is via host, like the name pipe, you can limit access to the name. Pipes can.

A

We add a security policy to limit that exit.

F

Yeah yeah yeah, so.

A

If we can add that as well, that would be part of our cabin our implementation, then that gives you up to par from a security standpoint of Linux. Well,.

F

Linux doesn't ship with any pod security policies. Well is not in the defaults that I've seen so yeah. We would actually be like more restrictive and in pod, security policies are kind of like once you enable them. They start to get. They make everything a lot more complicated yeah. They.

A

Do make a lot of things complicated? That's why you're seeing other tools sitting on top of kubernetes time to manage those things, but I mean the biggest thing for us. Is you just need to give them the available tools? So if someone doesn't want to go down this path, obviously they don't have to use cube, ATM or write API. They can just manually bootstrap everything, but if they do choose to use them, they could be restrictive. The same way that could be on Linux or so yeah.

C

A

So that make sense, if you can add that to the dock and that'd be great.

A

Don't come back to Peters argument, I mean me to glance it. Is there anything else you can do on that on what oh yeah babe, where Patrick you mentioned, the Peter also had their concern about.

F

Removing features think so.

E

Forking it and removing the unwanted things yeah.

F

Yeah I mean I, guess it it might be a good idea, it's just yeah more work or something or we have to I. Don't.

B

Have a strong argument, one way or the other for it just I.

H

Have a concern on Vince just because of that, then it has a lot of interface.

C

H

That's what that's, what threw me off then I looked at Vince code and I mean if you are go just using the process service API. Do we really need that much of code and Vince is something that I mean, but we need to bring the rancher person into the discussion and talk to him.

H

F

I mean why or why do we? What do we need to discuss just a I think.

B

What we're getting at here is I mean ranchers, obviously got that there for their use case. Yeah.

F

B

So the I think the question is: do we make a you, a forked version or similar project with a slightly narrower scope and basically maintain that as a sig? Or do we basically share the same thing with with Rancher I mean it may be that that with your other POC, maybe they decide to that? They don't even want to maintain that H&S calls in there if they consolidate on the same approach, so I think would be reasonable to at least see if we can reach out to them and see what they think about it. Yeah.

F

Yeah there's some other just like trivial things that I actually do want to change in it. If we were to go like make this official, just like there's some Rancher branding in the like windows service that it creates that just would be confusing to users. Probably I have.

H

A quick question on the host network.

H

F

Could only hear like some of that if.

H

You're going to, can you hear me now yeah you're, going to shim the process level api's? Why do you need a network.

F

What do you mean? Need a network.

H

Of this host network, oh.

F

Oh, so that's so I think what you're referring to is like when you create the flannel Damon said it has to be on the like. It's a host network container that you create, and that's just because kubernetes like the cubelet will refuse to schedule. Pods when CNI hasn't been initialized, looks for like specific files to be written to disk, and but it will allow host network pods to be created.

F

So, despite the fact that we're not actually using hosts networking, because that doesn't work on Windows like the same way it it kind of allows, tells Kubla to allow this pod to start before. Cni, okay,.

H

So it's just this network is not used for user at all. It's just a.

F

No yeah there's nothing. There's like the only thing that runs in that container will be the winners client process which just talks over that name. Pipe. Okay, so yeah yeah.

B

And so, if we want to avoid using host network as a workaround, there, then we'd have to instead basically go create a new feature. That is a niklas pod.

F

F

So it sounds like there's general, you know no strong objections other than yay around whether we recreate win us very, like recreate a smaller utility to do the same thing.

H

Can be launched and if the client can talk, h-honest calls and basically program it that is attack. Surface I am worried about and.

F

That's what a pod security policy is side I believe mitigates.

H

We don't have right.

B

Well, they're gonna be on Windows. It would be a pod security policy applied to the cluster. It's not implemented at the Q blue of ultimate. It's done at the API server level.

F

But this is like you know, on Linux, by default, you can like mount the root disk and, like you know, hose your whole machine. If you want to it's the same story there like by default, it's very insecure, but we have, we can have like recommendations for how to make a production-ready configuration.

H

Yeah I would be much more keeping that discussion with Minister I'll be much more happy if there's no surface attack from itchiness calls, but let's talk to them and see I mean.

B

Do you have any other ideas for approaches that would work on ship to assist in ich you.

H

Know this processor is I like this.

H

It has like Lord of hls rapper course and client can issue calls to the host, basically through a Genesis. This little bit boring.

A

Which is probably also engage with the security virtual team in the kubernetes community, and let them know that this is coming and what it looks like I know. Kubernetes goes through penetration testing reports tours one this summer, you know discovered, like 30-plus vulnerabilities it worthwhile to make sure they're aware of this, so potentially could get some focus on the next one, but also get their opinion, so so been it possible once you actually get the document in the cab up and running. Let's make sure that six security is also part of the review cycle sounds.

F

Good and so I guess the agreement is that we will modify the existing cab for.

A

In my opinion- and it doesn't have to be the one- is that we actually mark the cube ADM for beta, modify the cap with the introduction of the new capabilities and make sure the right people review it from signaled six security.

H

They might put a block on this if he, because this host network is, we changed the requirement of networking for kubernetes, based on the assumption that we don't have hosts networking support. So we need to add something in the kit that says that we are using that as a for Chrome browser than we starting the support of host network.

F

F

A

Alright, so so we have, you know relatively a couple weeks to to get this done before the deadline. So maybe, if you get something up for next week or cannon or review it- and you are some time to to finalize this I'm- pretty sure that six security will need up at least a week to go through this.

F

A

Cool Thank, You, Ben yeah.

B

Okay, I think we're out of time. Did you have anything else? Michael I'm.

A

Nothing else all.

B

Right, I'm gonna trip over your signal thanks. Everyone thank.

A

You all have a great rest of your day right.