Kubernetes SIG Windows, 25 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Windows 20190625

Description

Kubernetes SIG Windows 20190625

A

Hello, everybody and welcome to another sick windows, meetup, it's the 25th of June and you're being recorded so make sure you adhere to the kubernetes code of conduct all right, let's get started. Thank you all for attending. So looking at our meeting notes, there's kind of two areas that we wanted to invest some time. The first one and this is gonna.

A

Take the bulk of our time is to go, look at some of the critical issues and bugs and identify some of the ones that do wanna fix for 116 and or obviously we want to continue down the path of doing work around the run as username can be the cube ATM and we're not gonna take off the throttle on some of those areas, but also when I increase the stability of six windows, specifically based on tickets, that people have filed.

A

So so one kind of take a look at that and then the second item is around the C cluster lifecycle collaboration, some notion of whether that I definitely talk about that as well, that you know that okay, excellent so Patrick you wanna drive the the.

B

Bag ones- oh yeah, yes, sure I can do that. I, probably won't stop it after about 15 minutes or so I. Don't think we'd want to go much beyond that. Just because I want to leave room for for the earth for other items. At the end there and.

A

By the way, your document that you would link varies closed, so I had to ask access I'm sure everybody has had to do the same Oh, weird.

B

Yes, I got that one from sig node. Are you on the kubernetes group yeah and.

C

Maybe you have to be in Sigma, it's user group, whatever Oh weird.

B

Well, I can just scroll through it real quick, so you to show you what what it is just spend a minute doing that first.

B

So, basically, what there but they're proposing to do- and this is actually something that Sikh architecture has already done- is they're trying to basically define a work flow around the labels, where they're trying to sort of triage your things based on how important it is and then eventually use that to drive some of the prioritization around what their sig is working on and so kind of the the key things I got out of it were they want to use a priority field which is already there as a label of critical urgent for things that could like break production important soon or things that not causing a production failure, but basically they think the feature is broken and then backlog they've got kind of the most nebulous definition there right now because like on the one hand, they say well, if it's backlog then see if it's something that the reporter would be willing to contribute but later on, then I notice, they're saying well.

B

If it's backlog should we just close it immediately? If we don't think it's important enough to fix like in the current release,.

B

And that was what they were saying here. Is it's like, and so I'm, not real sure what the what the best option is there that's just kind of what they're proposing and that's similar to what sig architectures done.

B

So, in addition to the two, the three priority labels they're talking about using these triage fields there to basically track which items have sort of enough information, and so things that are marked needs information not or not. Reproducible, or if it's more of a question rather than an actionable item, then they've got tags for those and that's kind of how they've what they've proposed to do. I feel because of the way we've got things spread out on the board.

B

I think that if we could try to set goals to put a few things in the backlog for release your base with that release, tag and sort of focus around that I'm, you know personally I feel like we'd, be getting some of the same.

B

Some of the same benefits they're talking about in terms of looking at the items that are marked important, soon and critical urgent, because we're basically sort of picking those up front. So whether or not we use a label to me doesn't really add a lot of value versus if it's on the board.

B

A

That the labels are important for other people that are looking at this because they might not look at our board, but they might look at the tickets themselves right. They could see something that's priority or gender parity important, soon versus a backlog, and they can set some formal expectation of when this is gonna. Come show me I like this, and that's you know that that's all of software engineering works anywhere.

B

Okay, yeah and so you're looking at this, like there's only so get the current backlog. 16 items I only see one of them so far, and that's actually the tracking issue for Windows support. Should we just close that I mean it's done, but anyway, I.

C

Think that's cube. Atm! Isn't it.

B

Yes, yes, you're right! Sorry, okay, okay, so leave that one there so.

B

Yeah we could do that. I guess. My my only problem is that if there's not any any automation or reporting that relies on those tags, then I'm not sure if people know what they are. That's just that's my what I'm afraid of on that I know what was everyone else think.

C

My sense is that, like we don't have a ton of changes that we're really driving I mean maybe I'm, just not thinking of it, but like we don't have a ton of changes that we're really driving out right now, aside from like cube ATM, it's mostly like the things that end up being meeting prioritization I like bugs.

D

C

Yeah I guess I'm, not sure that those are gonna be well yeah, just like it also doesn't seem like we have a ton of people just kind of doing like drive-by bug, fixes or try by feature. It seems like we tend to have like high-touch, and maybe that's just means that it's okay to you know like have this project board be out of the main place, I.

D

C

Think we have people just kinda like coming in and browsing our github issues so.

B

If we did have these better label, do you think it would help if it helped new contributors? Sort of you know identify the first thing to work on or.

A

B

That a separate so.

A

That's a thing right, so what a new contributor want is how easy it is to fix a bug right or low hanging fruit. That's what they want to get started. We don't really want to put a priority urgent or something that obviously it's true that if something is in the backlog, it means it's a nice to have. If you can stomach, wear them if it doesn't find so that's one way for them to identify something they can work on for very would be easier for us to identify them.

A

A pull-up bar are easy to fix and easy to get their feet, wet on the project, precision and start on something that maybe in the backlog was a huge item. Mm-Hmm I think our goals as a savior is we want to see what can we do every release by release right so right now we're looking at 116.

A

What kind of tickets are important for us to fix and how many like them, and just by putting them in the project board, then someone else that comes and externally will see that if they also see a priority tag, that's clearly they don't know. Hey this definitely will make it through I'm glad the sake is put in the right priorities. Mm-Hmm.

B

A

From that perspective, that the combination of the two works very well, if you only do one just because it does take a lot of time to do this- also okay, if they see that hey we're targeting these on the backlog for 160, so I think the most important thing is to kind of prune on backlog for 116 and see what makes sense to stay there versus leave. And then, let's see, if there's anything important in the backlog overall, that we need to bring it 116. Yeah.

B

A

Do that we should probably get started otherwise, yeah.

B

Yeah yeah yeah yeah. We can do that and then I guess. We could try to make sure that afterwards, the stuff that's marked in there, we can get the right priorities based on whether its production blocking or not it for critical versus important. Well, so I guess I start from the top here. So we've got this note that you had added there about getting images and built built ended for conformance built in the master tree rather than in a separate tree. Yeah.

B

So I think the main way that we're fixing some of this stuff is with the effort that Claudia is driving around agin host, but kind of the overall effort is that we still need Windows build hosts there.

B

So is that something that that we think we should drive for 16 or do we defer? We.

A

Can't define it unless it's something that's who can wait until Claudia's work is done, and then we can see if it makes sense to pull this in or so.

B

Yeah, that's right. Okay, so I think that right now like week, I need to focus on getting him just built first, but yep.

A

That's their first, my.

B

Own okay, so this Windows vSphere ete test, no tits I, see something open by Ben. Are you working on that Ben know.

C

So these are like there's a couple of these I think where it's like, basically like kernel, panics or like stuff, where, like the plague, become unhealthy kind of thing where it seems like I've I've tried to trace it as much as possible, but I can I think it's like something in the kernel. That's going around and.

E

See them after the latest Windows patches. Do you still see them yeah.

C

I'm school, so I haven't why I I still definitely am having node crashes. I haven't been investigating them anymore because they just kind of happen pretty frequently so I didn't know which one was supposed to even be fixed. So.

E

The VSP one- and that was definitely there was there- was a it is already fixed. So.

B

Rather than trying to sell it here can I just assign this and then we'll move on, and if it's not actionable, then move it out of the back move it out of 16 sure yeah.

A

There's a set of these bugs that the been filed- some of them are networking related, and you know also. We had one that came from Google on the Windows Defender I. Think all of those were on some kind of investigation to see if it's people can fix it. Yes, 16 and if we kept for the right reasons, obviously push whether that's a fixed company Windows at a later time, or something like that. Yes, all right.

B

Okay, so this one is still under investigation. I'll sign this one to myself to see if I can get over, they stayed on it. Yeah.

E

7B is a release date that we have an email thread with Google and the file system. Folks I think they replied, 7p I'll, pull that thread and authority or above the Inbox sounds.

B

Good thanks so all the way of those two there. So we could decide if they're in or out what was this one banners.

C

Life-Image yeah I think this should be abandon. Probably it's I don't know. There's like as far as I know, there's a one guy Google who controls making the pause images like the official ones. He doesn't respond to anything so I think it's just like it's never gonna be I, don't know. Now without some major revisions of the process. Are we gonna get that into like the core images?

C

A

Right before we give out, can we just put more time on slack? Maybe you can add me and Patrick and not add on on it and see yeah.

C

Right like that, would be the one approaches we should escalate it to kind of like yeah.

A

I just told me yeah so like they need to think of Windows as a first-class citizen, you need support, and especially with the work that you need around the updated go images in the windows. Sig tools I mean that it worthwhile to have them all right. Well,.

B

We've got an open PR that hasn't been reviewed here for that issue. Yeah, that's the one I've got highlighted right.

C

Yeah, okay I was confusing. Those two was one just the issue and when it's, the poll requests yeah, okay, yeah.

A

I mean Google took the reins for a lot of these things, but if they're not moving at the pace that we want, you can escalate and get them to yeah.

C

I was also just also unsure, since Microsoft is set up like the pipelines for recreating those one pause images. If we want it to even reconcile it now or if we just want it well,.

A

For the pause image itself and I like.

C

Should it live in the community or should it just be? Is it okay, if it just is like maintained by Microsoft well.

B

I think we need a community one that aligns with master. That's really what the question is here. I think.

A

B

Publish what superflat but yeah.

A

That's what I don't wanted his wall rights? It wasn't just. He wanted best permitted to come from in the--, okay,.

B

So this one okay, so the ple g1 I mean that seems to be a symptom. It's not really uh do we know what the root cause is here and.

C

I, don't think I'd traced it to something is another one where I was talking to someone.

F

C

About this one, if I recall but uh yeah I, don't know if we ever got, it was just like some hangings. This cause I think but.

E

C

See it also, it's I, guess I: it's I can take the action item, just kind of try and verify if any of these are still happening or if they've been fixed, that'll be great.

A

Are you able to run the pipeline here anymore? I know you had some issues in the past working to fix some of those yeah.

C

They just they're pretty chronically failing because I think of node crashes.

A

Okay, so obviously we need to look into those, not crisis. If it's because of some of these things, you know, as we scale with customers in the field, they're gonna hit them yeah.

C

D

The memory dump after the crash interesting information there- that's I.

C

Think what that's how I've been making these bugs is just looking at the Crashdown. That's oh, you mean.

B

Okay, so the next one on the list kind.

A

B

Yeah, and so we brought this one up with with Jordan for reviewing it because it was, and so the feedback we got on changing air behaviors. Is that that his argument was that once you've shipped an API behavior, then you would basically you don't want to change it. It's kind of what what the argument was there.

A

B

A

Bet, I guess: that's our community goal is that once you're up once you enter beta, you don't change your EP ice. Please not easily yeah.

C

I thought I actually closed this one too, because I feel like this is another one where it's just like: okay I, you know like there's arguments on both sides, but I think yeah. The consensus is just yeah. It is close. This close this.

B

C

B

Okay, I'll just get that out of the there okay, oh I, didn't that's what the red means. I guess. Alright,.

B

And then this is the same image issue. Again, that's highlighted there. What was this one? So there's one from Claudia says replace image used by Windows only test with the.

D

New server there is a test which basically tests that images can be put on from docker hub now, we've put the e3 team, busybox image cetera, and they made me put the to do their to replace with a nano second image, but is so far below the priority list that yeah no idea.

D

B

So it sounds like not 4:16.

B

And then this next one and like this images, I'll move that out there for now.

B

So the feature gated tests in non staging stable windows test runs deep.

F

So yeah I took a look at this, but I think we kept coming across. You know various tests that were marked in a certain way for a certain feature that we're that kind of prevented the overall feature. Gating mechanism from taking effect I mean I, can try to explore this a bit more. But that's where I was anistar. Do.

B

You think it's a priority for 16 or not probably.

F

Not because I think that existing gating mechanism, that Adelina has in place is pretty good. Okay,.

B

B

So the empty durval yems stuff, so that was, is this a test? Failure no.

A

That's just the permissions one I said probably right. Well,.

C

It's like the et test tests, the permissions of directories and they're, not I, mean they basically end up cascading on Windows, right and so like. You need the far directory to have certain permissions for the empty dirt volumes.

E

C

Have those permissions and I don't know it's a whole mess. I don't know right, I mean we basically just like made a workaround of like changing the permissions of like a parent directory to get this test to pass, but I'm sure that's not the reason. The third testing it. So we.

D

Haven't seen this usual I think.

C

It's because of the like provisioning tool use actually creates the VAR directory that it comes with different permissions. Then maybe on your images but yeah, it's like I, don't know, I'll take a look at this one too, and try and see if it if it needs to be pushed forward or if it just is like should be closed. Yeah.

B

So I guess net on this is if we want that directory to have more right permissions than its parent and that's something that's fixable yeah.

C

I think maybe it comes down to like there are operations being done out with go that just don't? Actually it doesn't touch Ackles and so correct. Yeah yeah.

B

And so like, if there's a behavior that we're missing, that that that's, that could be fixed because, like we've done, there's some stuff over insists whatever for for windows to set those Ackles. So if you need to do something like break inheritance and set explicit Ackles, you can do it.

G

It's actually related to not every user can access that directory. Since it's the permission issue.

D

G

That would actually not be a test issue, but just a journal like whether all the users in the continua can access the volume yeah.

B

I think this is one that we may need to fix, but let's confirm on that.

B

So next one around stats performance I want to leave that one here, because I've got I've. Had people complaining about it, saying that if they're running more than ten pods, they can't complete a stats call and so I think that one's pretty important and I was gonna, see if I could get ping Fei to look at it.

B

A

Mean to say, there's customers that will just give the ten pause.

B

Yeah yeah there's people that deployed is more than four gigs of memory. Nothing I was surprised at how little memory we could get get to pass that could forwards tests on QA diem, stuff I'll leave there. So this is kind of the last one.

B

I wanted to talk about, there's a whole bunch or Bruna's user name ones here so I talked to James and he's not going to be able to take that on in the next few weeks, but Claudio actually started rebasing this PR right here and so he's gonna he's going to give that one a go. So I want to make sure we leave that there and container D I'm continuing to work on that and I'm actually I think I'm going to get a few more people from Microsoft helping me with that.

B

So I'm going to leave it here until I get a confirmation on that. But that'll probably be my priority because some of these other stability things don't really matter when we cheat. If we change run times and so I want to understand whether or not we're going to have that 4:16 before before I before I, go and commit to fixing things specific to dakashin.

B

A

By the way, the question on the run is using them if I remember correctly, James got it to the point where there's just some API changes are needed to be made and run the test again and might be ready to merge. Is that correct? It.

B

Was ready to merge at one point, but because the code, a master, has diverged since then that needs to be rebased and then he needs to rename the fields based on the API review. We did with Jordan like it.

F

Address some of the things that Jordan typically takes on so I pointed them out, put comments so I think it's around the validation and things like that better typical. If you comment so I pointed.

B

Them out in the.

F

Prs, it should be a matter of this review addressing them. Okay,.

B

Okay, so we've got just four four minutes left, so the other thing I wanted to talk about briefly was I. Did some investigation around WS, l2 and kind, which is a tool? That's sig testing is a cluster cluster lifecycle been using? It stands for kubernetes in docker and basically, what it does is.

B

It brings up a bunch of kubernetes nodes in separate containers, and so this lets you test scenarios like calico and flannel and stuff, like that, all it's very very little footprint it actually works fine under wsl to and so I've got Doc's coming for that.

B

So as we're doing work around cube ATM, this may be one of the easiest ways to actually test that, because just because it requires fewer resources and so on, cube ATM and cluster API. Did you want to talk about that dinesh? Or do you want to wait until we've got some other things figured out on your side.

E

B

As far so I know that Kalia was working with said, cluster lifecycle and cube ATM. She gonna be continuing that work and yes,.

E

B

E

The work is complete. She already raised the PR, so it's only the documentation. That's remaining yeah.

A

E

We are internally removing all our automation and using cube ATM in our place in that place. So, okay, so she will continue to work and fix bugs and finish things.

B

And do you have somebody that's going to be working on cluster API as well, nothing to find what's needed for Windows? Yes,.

E

B

E

And I also want to I, don't know what is the dates for 116 I'm, also looking at replacing the transparent proxy in NY, so adding support for transparent proxy in Windows.

A

Yes, oh I know that people don't folks and bin might know them at four and a few others were also working on that some Microsoft folks. For that. So it's a part of the same effort. I know yes, yes,.

E

Yes and it's like it's, we just added the platform support the platform. Support is going to ship in I, don't know exactly the ship date I'm working on that current release, so it will probably another three four months once that is, but we want to start the work on the on one side.

E

So I think we have the pivotal and service fabric trim in Windows, so we have a team channel where we are going to look into it, but that's something we will definitely look in 1/16 I, don't know the dates, so is it ok to put it in and push it out if we can't meet it by 1:16 yeah? This is these open source community.

A

Who continuously push items because of a there? Are the priorities or things don't line up, but yeah yeah 116 will support it, make sure there's a cap so I think actual renewed I. Don't.

C

Think this is really a kubernetes feature.

A

B

Just a related project.

C

Exciting for all windows, kubernetes users, yeah.

B

Ok, alright, that's all I got.

F

For the CSI proxy work as well in 116 and add it to the backlog: ok, ok,.

B

E

It's gonna be focus on 116 or not.

B

Yeah I'm planning on it, but it's contingent on getting a plan for container D to ship in 116 or 117. Ok, so what we can keep talking about that you're you're on the same threads I am with the people contributing a container d. So ok.

E

And what are the dates for 116? Do we have that.

B

They're not finalized yet, but they have appointed a release team and so usually they finalize the dates within a week or two. Okay, all right guys. It takes our one I get a drop for another meeting. I have.

A

A great day.