Kubernetes SIG Windows, 12 Mar 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Windows 20190312

Description

Kubernetes SIG Windows 20190312

A

Hello, everybody and welcome to another safe windows, Meetup I. Thank you all for attending a couple of high-level updates and then we'll dive into into. There are meeting notes. So the first thing for congratulations. You guys you guys did it. We were able to finish every single PR in time for the code. Freeze that includes the group managed service account PR, that Windows node PRS.

A

We were able to do an update to the cap that included all of the support limitations, not supported features that could approve by Brian and Erin and as well, and we got our entire test Crete to be green.

A

You know phenomenal to have green tests across gcp and Azure and, to a certain extent, almost all green in vSphere as well. A couple of things that came out of that there's. A couple of back checks that benoit's have filed for that stemmed from some of the VCO testing and Microsoft is following up on those, so so stay tuned for more updates on that.

A

When we get them and then we were able to send an email and and get everybody educated on the fact that we're getting close to GA, that include SiC architecture, who we are gonna, go to and ask for final approval on Thursday, as well as the folks from sync release and and the rest of the folks are in charge of the CI signal and getting everything for our release.

A

So so kind of next steps is the the 314th were. The cigar collection meeting would try to get our final approval to go to GA I've already started working on some marketing Docs and we created with Craig as a section in the blog post for 1.14 to include the windows, node support and, more importantly, there's a whole bunch of you guys are help contributing in our documentation to validate and that the entire feature set in Windows. Node is well documented, with step-by-step instructions for opera doors for end users, as well as for the network administrator.

A

If you guys do get the chance, the the dogs are already linked in the actually pull it back up, they've already linked in an earlier date. For our in our meeting notes but I'm going to post it in today's as well, please please, if you have a chance head and read the dogs provide comments. I know that Adelina have asked that someone from cloud base needs to look at the ovn of yes section, especially around ovm kubernetes, making sure that you have the right classification in there and the right instructions.

A

So we definitely will appreciate that be aware that, probably, at the end of day that dogs update is going to be frozen and locked and all your comments need to come to a PR, so we'll link the PR from that document once that's ready, but the time to add feedback, that's easily changeable is, is today and right now.

A

Yes, the the last part on this is there was one PR out of a bug that came up on Thursday at code freeze day, and that was a mounting issue with SMB. We we work to seek storage a little over the weekend, but mainly yesterday, he went through four or five iterations, where we caught some bugs in the PR and such as the changes, but were able to finalize that in it was merged.

A

Last night and I'm attending seek release later on to make sure that it's it's gonna, make it in two or three four 1.14. The general guideline from seek release is that anything does milestone in and merges now is very likely gonna be approved and let in next Tuesday there's going to be is so all day and after that cherry-picks have to be manual. But even though cherry-picks are not manual still there's a risk with up here, so I'm gonna attend sick release and validate that we're good to go there.

A

Any questions on PRS or our dogs, or anything else that that's in our pipeline right now, all right, cool yeah, don't forget, read the docs. Alright, let's, let's talk a little bit about the testing update I know that we had a few tests that failed the other day Adelina. You want to talk to us a little about that. Please yeah.

B

So basically there was a time out in one the 114 tests.

B

B

One of the nodes going down it just went not ready state. Basically, doctor didn't didn't answer. I preserved that cluster. I investigated still trying to correlate that doctor failure with something but after a manual reboot of the node, while the test words was we're running sorry running, look you've, let started so.

B

Basically, I am on the lookout. If I ever see anything like that until we figure out what's what's happening with doctor, what in which conditions it misbehaves if I see- and you know this is not ready- I'll intervene manually with the Gingka flakes that test will be rerun. Well,.

A

The one risk there is anytime you intervene manually, maybe were were masking a failure. So it's very important to understand why why something fails I mean we should intervene manually so can get the test to be green. Well,.

B

A

Definitely understand why something is failing, and this required manual intervention, so.

B

It's it was Barker on the note I'm pretty sure at this point, doctor fail, I didn't see any bumps against a anything like that.

B

I'm pretty sure there was a failure somewhere on the node in Windows I'm. Still looking in sis logs to correlated, however, the logs for the cubelet were clean. I mean there was no prior failure with anything to just pawn. The only thing it it had in the logs would that be a BL eg, not ready. That's we're.

C

B

C

Well, on our doors, periodically I'm actually troubleshooting it this morning, mm-hmm.

B

But it's again we had this issue wild bag. This is linked to docker. It's not cubelet. It's not right, for example, in internal I've, never seen this okay.

A

B

And there we have the darker version: it's different on flannel. It's all.

C

Manual needs so trying to like understand how it all works, but something that could be involved is like the connection you know like here's, the docker, shim client, that's in cubelet and then there's a docker shim server, that's also running in cubelet and they're communicating over a named pipe on Windows. This is like, as far as my.

A

C

Got to today, but I was like I actually didn't know how all of this worked before yeah I don't have any finger to point but yeah. That's where I've gotten to in understanding this bud.

B

And with regards to the others, the matched container back off- yes, that has two recent failures that even slip through the flake events. So when we added we changed the test for that, we made it pretty clear why we that specific change to improve the passing rate on a case engine with a Jersey and I. So this test is a problem only Anika s, engine with agency ni, because the way Asia's and I works, it's just a bit slower in.

B

So in the preparing a pod for networking and so on and so forth, it's just the interesting I so, for example, and Pathak made this abundantly clear, even in the PR and within discussions that so, for example, if you look on GC or in this sphere, even in flannel, that fest has no problems.

B

It's strictly relate the pressure. Cyanide.

A

This is a crash bag of tests. Right not is that the ones that you're talking about that we did to PR zone yeah.

B

It will still flakes once in a while will snowflake.

A

Did that passing staging I know, Patrick was waiting for all of that to pass in staging because before he promotes them to regular past. Did that pass him not yet it.

B

Passed the stage, so it was passing the staging we promoted it. So, for example, if you look at staging that test, work like for the last.

B

Then again master it worked and runs. Then it failed two times in a row. It is just lately, but again it's because of the way a sure works agency and I works with it with those PRS. All we can to to minimize the flakes the yards on. The test is just how it works. I am testing and staging the new agency and I and then slowly promoting it.

B

Maybe that's that's behaving better, but I want to be absolutely sure that works before I end up will putting a bunch of reds and the on the board got it.

B

But, as I said in any discussion, this is only on because of a certain irony effects. Engine the rest have no problems and regarding testing I do have one question I see on this fear, that's like to be all green, so I guess most of the problems. The flakes have been solve, energy, see I, guess as well. Do you guys use flight events yeah.

C

We use 10 figures right now, I.

B

Was wondering NGC because you said on GC your flakes are because usually nodes will go down, will reboot or something there right.

C

Are you talking about on GC with oh.

B

C

B

D

We seem to have addressed I worked around as a node for the time being, I. Don't you do probably knows if we run with like attempts, you.

B

Don't yeah I, just look! That's why I was asking well, if you could add I mean if you still see any flaky test you you might consider adding it just choose. So it's not the. It doesn't add too much penalty to the total time of the test.

B

As I say, you might consider adding it. So you have it all been though I see for the first couple of runs. Okay, it's what no okay.

E

Yeah we we can add, we still have problems with actual the actual flakes with our mission as creation. Sometimes we're trying to figure that out first before, like I, think we were trying to figure that out before adding the chance, because I'm I actually mask various a. We have.

B

B

Well, that's that's all for me on the test, as I said, we're investigating that darker failure and I'll repeat Indian times with regards to that, as that spelling is very important for the right people to know it. It's just hey guys and it's from the CNI.

A

Okay sounds good.

E

About the failures you mentioned before so I, don't know why I felt with the plague now I, usually on the Nexus just means doctor is really slow, not responding in time.

A

E

So far, the actual only cause I've seen on Linux. So it's worth checking where the doctor was actually functioning you're sounding at that time. So.

B

Yeah I said I said: I saw, I tries to do backward because I saw it rebooting. Restarting sorry. The service was starting a couple of times so I know it's darker, I just don't know why? Yet.

F

B

Yeah I looked at a loss. Unfortunately, mmm let me see if I can grab them again, unfortunately, the buffer logs. So in the event log there's not much in terms of a message I just see, and let's get on to the meeting in outpace that you see what I'm talking about, because I have to connect to that particular machine.

B

C

Have a No now, that's in this state and, like doctor, responds just fine like if I use the doctor CLI. Everything is like super fast, which makes me think it's like the docker shim server is actually maybe the thing or the client are somehow like hanging or just being really slow.

E

A

Right, well, you guys can take it offline and see if there's something that.

G

We can do to.

A

Help there by the way reminder no engineering stand ups anymore, so we finished last week that was a drive to get us to code free. So we don't need to. We don't need to have this anymore, all right. So there's a couple of things around issue: three as really quickly who added the DNS conformance issue.

B

I'm sorry, which issue so.

A

I'm, looking at our Doc's, someone mention my screen here as well, so someone added under a sheet triage, DNS, conformance I.

H

E

H

Networking, yes, sir targeted for being promoted to conformance some of its.

B

B

So yeah, oh you.

A

Know what I added this because I'll go after Claudio commented, so I guess I guess. The biggest thing here is to make sure that you know we stay on top of it. So they are passing all windows, and then you know I'm, assuming that the Alina you and Claude you are going to basically monitor it mutates a lot into conformance would be.

G

A

And if they not passing on Windows at least go back and let them know that they should be following a certain specific prophecies. Yeah.

B

Yeah, so this this one will never pass on Windows the way it is right now. So that's why cloud you advise you just admin ox only on etcetera.

H

So who's the like the authority on who decides like which tests get promoted to conformance and who proposed this there's.

A

There's a conformance I, don't know if they decide any of that stuff in that in the in that meeting group or.

G

A

But I would believe that individual seeks can say, hey we added a new test and we think it will be very valuable to add it to conformance and they might actually do it themselves, I'm, pretty sure that that's how usually works so it's kind of harder to get everybody to to do. That is actually good. Note I'll, add it to my to my notes, I'm presenting to the community meeting in a car in a couple of weeks. So I'll add a note here around tests. If you add conformance test talk to us, but.

G

David, can you precisely tell me why is will does not pass in Windows.

H

Like, for example, the DNS resolution that uses NC.

G

H

Or the DNS resolution that we took that we like we split up the DNS tests.

G

Marshall I think we should bring this to talking in the next Network meeting and tell them that basically I think it's driven by SiC network. In my opinion, we as there was a discussion on that. We need to add these tests to conformance in signal one of the signal work meeting.

G

So let's bring it there and then power tell them that hey just make it Linux only for now and then once we add those things we like it, so the.

B

History of this test was that we would there were. We now have three DNS s. They were all in one conformance test. We split them for partial domain names, fully qualified domain names and the ABC hosts. They were all in the same test, and now, basically, it makes sense that trying to probably move it back into conformance, because we we split them no.

G

But there was a clear discussion at Illinois when this happened. Like six seven months ago, we clearly said we took this to Signet work and they basically we are. They approved this fear to remove it from conformance, and we said right, I think we should pull up the agenda meetings and meeting notes and talk to them again. There was a clear decision that these are not conformance tests.

G

A

Right so so, in the interest of time, we'll take an action item that here that the nation David will talk to to city Network about this, and the fact that this may not be a conformance test. Yes,.

G

A

But I will also take the action item to educate everybody that if they are going to add new tests, they should let us know. Well they do that or not. It means to be seen, but at least we'll be identifying it to fail test. That's that's! That's! If they don't do proactively will happen, reactively all right. The last item is a kubernetes login on Windows, so I don't know exactly who posted it, because I don't see. The full name here is.

G

Such in from Microsoft.

A

All right so so, basically, the the post was around logging right so and the very valid requests here. So in a case you can get all the logs, but if you're running natively on vicia, for example, what options do we have around all of the login? That's happened in kubernetes. Is there I that we only have a couple of minutes? Is there anybody? That's that's willing to experiment on this and bring some proposals in a future meeting.

A

Anybody wants to own this.

A

Is such none are call by the way at all.

I

Yes, this compose a whole new key confuse your the managed service is running Windows, just FYI, okay,.

A

C

It's kind of confusing what what he's talking about? What he's trying to do like just talking about log forwarding in general, yeah.

A

I think that's what he wants. He wants log forwarding like.

C

There's fluent e that works on Windows, so you could use something like that. I think it's just a matter of configuring. It Ben.

A

You know I replied to him just.

C

A

Don't wanna since you're recommending it just oh hey! Oh yeah,.

C

I can I didn't reply. I was just like I, don't even know what he's trying to do, but.

A

But asking the following thing: if he does get toward to come back and post some documentation on that right sure.

C

A

Actually, a good point on that is a la dissection TBD in our Doc's, around logging and log forwarding and Craig and I will figure out. If you can have someone to fill out in a temple. Yeah.

F

We should have, we add it to the Czar cool.

A

All right, that's all the stuff that I have today. We probably have time for one quick question and I need to jump to seek release.

I

Windows is that Lee sitting here as well? He.

A

Won't do the log rotation.

I

Well, the key right there should probably be something longer. Tater set up just the case. Yeah.

E

I'm juicy, it is we're just doing our own local rotation right now. It will probably be easier if we can just feed a loss into so when those even log, so that we don't have to come up with some custom solution for that. If.

A

You feed those logs into Windows Event log. They were just populate it like crazy. It might not be a scalable solution. Yes,.

C

Stuff like Genesis I, am which I know a sure is using and we're using a different thing for vSphere, but they'll handle like the rotation and truncation at least I can.

E

Use it with caution like we don't keep anything. This might very well crash your application, so we stop it. There.

E

D

E

Other suggestions and we're happy to switch to using that as well. Okay,.

A

I'll talk to Craig and can see if I can get someone to actually hunt down all the options and give us a good recommendation that you can give us a community and and fee figure out from there. Let's.

H

Put a little project voice, yeah.

A

I'll do that all right cool? Well, thank you all for for attending. Thank you all for your hard work. Please look at the docs again and we'll talk to you guys soon. Sorry for the delay in joining I'll. Try to figure it out will happen for next time.

C