Kubernetes SIG Windows, 12 Jun 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Windows 20180612

Description

Kubernetes SIG Windows 20180612

A

All right, everybody.

B

And welcome to another seen windows Meetup. We have a wide variety of topics today. Yes, we're trying to finalize all the items for 1:30 11 release, there's actually a lot of work, that's happening and then there's a couple of things we want to discuss. First, first and foremost, is the DNS changes that need to be made for with regards to the CNI plugin, and then, after that, I'll give some updates on on some of the recent work.

B

That's been that's been happening on on sick windows, so partly you wanna start, the DNS discussion or I couldn't give actually a high level overview. So some of the discussion started from a ticket that link filed where the cni plugin was called by the cubelet for every single container. If he had multiple containers in a pod that that kind of violated the cni spec and also it incurred an additional parfum provement.

B

So so the pr to make that change was asked to be reverted until possible until itís v is out, because the dns was using the specific capability of being called every time. A new container was started up, so I can replicate dns entries across all their containers in a pod.

B

So so that's kind of the high level of that and let you know that folks are living more familiar with the change start talking about this Olivier as well.

C

So modern can give you all way of. Why me when we push the our changes, when we did the windows CNI right, so basically we want.

C

Today, CNI has only added in point and removing one, and we wanted to have a gate endpoint, but I, don't know where we stand in terms of when the gate endpoint will be removed, implemented in the single spec. So we were using add in add in point as the idempotent call to basically get more information and that's what we did so really two different things. One is I, don't know when the CNS CNI Peck will implement end point and if there's any discussion on that, second is: when we implement the namespace support.

C

We can definitely try to remove this call.

D

Just don't think that yet is being implemented already in CNI spec, one v4 and I think he's gonna get through this probably soon. The only thing is that if we replace that we'd get is still not gonna work on this, you actually win. The CNI call at the end point actually hot attachment point for the container.

C

D

But my advice is that it was not mentioned anywhere and when I looked at the agents in my plugin, actually they were complaining that the corporate calls multiple times at Siena, plug-in and I said: hey, doesn't look right, I mean even the agency, like also saying that this is shouldn't be like that when.

C

We added the CNI for Windows. There was a huge discussion on this item. Remember we asked in that discussion. We asked hey when gate is implemented will remove the said. There was specific comment added on the code to if.

D

But still, this is a machine H&S, because the dns does not get replicated, but I can't address that issue by making this couplet called only that CNI once for infra container, but apparently due to the HNIC issue. They which should call the in fact the CNI for every container from- and this should be stated somewhere. At least we should have like a matrix with what supported in rs3 rs4 or which windows variations are we gonna target should with robust support 417 online, or should we continue with supporting both 1709 and 1803.

C

17:09 and 1803 is supported with the way that we have implemented now right. What sorry.

E

To interrupt you, but it would be I, don't know, would be nice idea to document it. What is needed for 1709 1903 and probably whenever the next release will come. So people know what to expect when they run run. The actual couplet I'll. Give you another example, for example the symlinks which work on the different version, but.

C

But just I just want don't want to change the PR now that that particular Pierre wanted to revert us.

C

It words today with I understand that there is a platform limitation that you have to make another call to replicate the deepness register information to the shade shared pod, the shade container in a cord which is sharing the same space that will be fixed as part of the when we bring in the namespace support in the platform and that's what we are working on now right, but I agree with the documentation that we should call out the documentation say. Ok, this is the reason we are doing.

C

Two calls, and here are the reasons, and we would like that to be stay like that for 709 and 1803, and then we can move to the namespace once named, support is available in whatever the neutral is called 1805 or 1806 whatever. That is, we will document it.

E

Because I have a documentation, would some checks in the crew billet I, don't know if checks in the public makes sense for the rest of the people.

C

Allow us to put window specific version checks. Is there something like that already for Linux to the check, I need.

E

To check, but it's basically, if you could probably fail a function, you need to log it and basically just log it into with the complete itself, and maybe since it's all work old, maybe we could add that yes,.

C

Our I began adding it in that system too. So if we have a shim code where we do have version checks so.

B

I think thanks: I was talking about that in one of the PRS or one yeah I seem to recall a discussion where someone talked about having the ability to do an OS check between 70, no wine or vanilla server 16, so that could differentiate capabilities so somebody, so maybe that's that something you can sync with them. It's.

E

A good mean we should also at this, for example, when you do Kubla think what couplet versions. So we know when somebody's running, for example, 1709.

E

F

Think we wait on the version checking in there know some topics, the CI itself. Could it be called many times even today and I'm not sure whether you're not a standard from say eyes back. There would be a cat command a tedium days of point for release today. It is not ready yet from simple ideas, but I think it'll be ready. So if we upgrade either say I'm to zero point, four then committed union could minutes way. We could use cat the cat command instead of current setup commands.

F

So I think if we switch to get on the same wave shouldn't care about the double checking in code. Name is: if we do really needed a logic, then we can do this in the IBM. Not increment is.

E

I'm not referring just to the networking part so just for example, doing a get or something I mean we need to think about storage. In the end, what types of storage can be supported on Windows.

F

And partying TCO I have seen that not piache said half the deaths for the top API, so the the toxin box could Saturday a verse and switch domain and then the containers sharing the phonetic tag from the sandbox. Then dish receives him in SS. Second, but as Windows a the it chef Ezio's, which called the current simple a dozen workers buy the item. The stock price version reveal works, window dead center, so I think after this is effect, then the DSD Theo should be Victor together.

A

Some bottom and Dinesh dinner with this H&S issue is: is this something.

F

So when should we expect the HIV Theo fixed and in witch doctor, it is the way it factor why.

C

Why isn't this today's way of setting this thing through CN is not a right option? Why you're saying that this that's? How it's play right.

F

The problem is as a common hated from the year.

F

The gas data flow pattern, this for option, Patrick, elated and I think the brought the the first way is the right way and I also committed.

E

Personal dinesh, the problem is we're not following the linux model and I. Think Colleen can add a bit more on this one, because it's basically just using one command and my rattle.

D

Actually, when you cooperate with Cole the CNI only for doing for containers who have nothing to do with the others containers from within the pod, so at least, if there's an initially that I think it should be isolated to the CNI side, because at the moment there are required workarounds in corporate and in the scene itself. At the same time, the completion cannot lead the scene I only for them for container and then at the CNI level. You should see that hey. This is a call for the intra container.

D

This is Windows Server, 1709 or 18:03. I know that I should iterate through all the containers from within that pod, because you actually have all the details, and then you just do attach endpoint for that, because the Demont is not right since complete calls. Actually the CNI plugin and says that it is other container is actually my infer container, but that container is nothing for container of the pod.

D

It's just a normal container is a cur just calling with random data and from within the public cannot expect that that's variable and if you wanna make sure you can't expect that the infra container ID that gets passed to the CNI section in fact contain of the pod.

F

Yeah and and the the TR have already murdered, so we should mix this way, but at the same time, since the right deal a chance, ETS occurrence of abortion, we should see a reporter, the the TR. So so so if we will make to work for current windows version.

C

So can I just do clarify a few things right so that the cubelet is starting, the trading the in front in ER, and we are adding a.

F

C

Point to the containers through CNI and are using to that film. Are you passing and we program the intro container with all the innocent emissions and then when we try to create the like a workload container inside that pod, we basically try to expect that the information is the expectation and that's what the next.

C

What I don't understand is with the CRI implemented itself is not there right. I mean why we should be going through docker and not just directly set it through CNI n. Whatever mate works, the DNS server set through CNI. Why doesn't it work in this model? Is I still.

F

This is because the they is a policy from particle. So if you said policy, then, if you set different policies, then the the giant servers and such comments made different from the from cube later. So, if you configure the DNS servers from their plug-in, then the then the list of servers are static. So and you you didn't, didn't notice the guest policy provided from horseback and.

F

Non DCO is under. There is also cat terminal, yes from horseback, which it called a pod config. So yoga could set up a list of different servers, also, two domains from horseback and the DOS game service may different from other day. Service, for example, could be of the interest on the host. So if we want to fully confirm to the the get policy, then we we should last cube later past those as those guests audience key to the container.

F

Am I clear today.

C

Even if the cube in the Senate, then it's the time of creating container it can pass those information to see my right yeah.

C

That's what you tried and issues with itchiness is what you saying.

F

From from the from say, see, I, can you run him? Interface cooperate, have a list of game servers and such domains so so cube later past those games context to control time. You cooperate as a seat. Doc, shame talk to him. Well, we have set up the guests for the for the sandbox and they container. We have shared the sin network stack with the sandbox for Linux containers against a set up differently because they each read the file. So so far in containers created actually talk. Shame touch overrides the fire.

C

F

For Windows 10 dangerous, they no such list, so my peer touch to assess interest and sacraments from the stalkers API, okay, the th servers and such domains to the sandbox container same career, the containers, the shares the same network with the sandbox and about should receive aimed at hooks Inc in the config utter at the windows days. It's just easier. This is this, isn't working yet at it should work later if we should be fixed.

F

F

F

Sorry I didn't hear clearly.

C

Marvin is saying, is dr. shim during them? Fine can will write the scene icon, say and pass it as an input to CNI right yeah.

F

You pass, the second is another part of CI parameters.

F

So from the fans back up, sparse or a forgiving mother set have network interface, it just passes the the essential since such a sandbox, ID or network namespace, and the way some capabilities, for example, map. But then, oh dear setting yeah.

G

I mean cubelet if I understand, and that has to be fixed in cubelet or wherever it's basically invoking this.

F

Yeah, it should be effect.

A

Okay, so let me make sure I'm saying that right, so are you saying that we need to make a change to dr. shim? That's Windows specific. In order to set these up then.

C

Understand from Bank a what what we were thinking is, the CNI can be modified by cue blood and runtime. Venous information can be set on that, and that should work is what we were thinking, but I would like to understand from thank you. What is this concern on that see that when he says it Genesis an issue, there are two two parts: I would split this into two two parts right, one is the shared car.

C

The shade container, getting the information related to DNS from the inflow container is something that will be resolved when we add the namespace support in Windows as a platform. Until then, we have a workaround that is already implemented in CNI, where we are making to add calls and getting that information and doing some things with that. Right. I would was that to stay asses for 1703 and eighty, not 1709 and 1803, and we will make a change as part of bringing in the namespace at work now.

C

The second thing is, we can look at how this information dynamically is passed from Kuebler to.

C

To CNI and at the end of it to H&S right so what we were thinking is similar to how linux is doing Rizal Khan. We can implement a mechanism where cubelet dynamically updates that see any configuration file with the set of DNS suffixes and be honest servers, and that should automatically work. That's how we were thinking, but let me I think if I understand correctly thin-faced saying there is no input to see and I call it self and we'll be doing something different for Windows and instead do something else's.

C

I didn't look at his PR, so let me look at how he is in machining and then that part has to be little more discussed and come up with a proposal.

D

As far as I remember, I think thanks I submitted the PR that it's the DNS policies in Genesis fixes to the creation config of the container of Windhoek regime. So he gets passed to the CRI not to the CNI.

F

F

The windows called.

F

Him as I said they, the reader fire phone in container, so touch him rewrite the file for Windows, then no such file. So you might yeah I just set the the giant config t-talk IPS. So so then the the dachshund box will set the chance. It's hot come Trager. Then the containers a it.

F

Then the container should receive the same Kieth config from the sandbox from the sandbox. So then to the different from the lien container.

F

Butter butter, since they a chance, Ezio's containers thousand receives the game config from sandbox. So we need to beauties that have ten servers for four containers. So that's the reason we should refer to the should revert to the Chia and called the sample again for each containers, so those containers could receive of the da server from sample IBM.

F

So do we have a name Eliot for the workflow I.

A

Mean I think what I'd like to propose. Here's I mean I think that sounds like the right workflow, but we should have Dinesh, look and see why this DNS settings we're not getting passed through. So it sounds like that's part of the OCI spec for Windows containers right.

A

Yeah because I mean that sounds like the Queen is fixed, because it'll be very be very clear in the code where the DNS servers are coming from. It won't be needing to manually modify files on disk, which, just generally sounds like a difficult maintenance problem.

F

Down have approved the reverse. Er do adhere anymore comments for the rebirth, dia. If not a way. We wish you the.

A

Okay, so we need to get this one. This one reviewed, oh okay, I see I was just approved. Okay, so so they'll revert it back to the previous behavior, but we're still going to be stuck where the DNS suffix in servers are pulled from the CNI config file. Instead, I think we can do that as a temporary workaround, but I'll talk I'll work with the nation', we'll we'll look at see. If we can get this done, the right way or the other way through the host config instead and come back next week.

A

Does that sound good? So.

C

The only question I have is.

C

You can pass the DNS entry, that's something that Linux will be going to.

C

Configuration right, it clearly says that business information is passed as bad of the ad in.

C

Future to be I felt like this was like a workaround that we initially came up with the darker shim, but now we should, since in it clearly says that this information is passed as input as the network configuration pass, doesn't mean to the endpoint call put Linux move to this I feel like this should be the right way to go forward.

C

F

They'd apply for a link container but as window coupe later also support docker one point xi, but dock one party, eleven thousand or setting the game server together with network that into you host. So we should waited copulated deprecating, taco, babe version, one point eleven, so so a if they are really quite on time to you to make the change it for for Lincoln inner. No.

C

But I say that this is the like this. This is what the right thing to do and we should that's what windows is moving towards and I feel like I understand that the backward compatibility and support has to be there, and we can support that. But I really want to push forward for CNI, taking over the DNS information and not like keep supporting the old model right I mean the the right thing to do here is cubelet.

C

Passing that information to CNI and CNI programming, as part of a ten point call is what I personally feel we should do.

A

Is there any reason we can't do that on Windows before it's done on Linux? That's.

C

What I want to propose, but that means that that's the model that we are moving towards, that means that we should pass as part of the JSON when, as part of the Adhan coil, just add that information pass, the town and leonis will take over and implement so.

D

There's only one thing that they want to propose this victim of all the workarounds that they incorporate at the moment at the senior level, so the cupola should call the CNI only for them for a container and then at the senior level, just inspect hey this is in for container is an unsupported version. Why should I have to call for all the containers from within the best specific port.

C

D

C

D

That's gonna be better when others five is gonna get out, you just handle it at the CNI level in they won't have to modify again that complete food.

G

C

I but I strongly think we should not be making CRI pass that set up a mission rather be given to CNI the q blood should program CNI and see, and I should take over the entire DNS information just completely program in.

F

But from current Canon runtime with empower or other container on higher settings, oh it coming from all containers and it directly from socket API think sample. I! Wish you happy! So even even you you are using different nope again. The game items should be same way with same TN policy. So hey I think it should be a common logic within kool-aid, not in CI I.

C

Would argue that it should be in CNI, but again let me let me let me talk to Martin and we both can't like will look at your PR and then we can some proposal there, but I strongly think Linux will be moving like I. Don't know at least tell me if that's not true in expect clearly says this. This is the going forward model that we want to support and I am assuming Linux will move towards that and I really want with the platform support of names.

C

Please I would like to move to this model from our spy onwards. I want to support this, but I hear you on that. We need to support the older models and I feel like CNI would be the right place to handle that versioning change and say. Okay. This is this. This version go with the container and this version go with in point so, but let me get you guys, I think give our see. Does.

E

It make sense to wait another week and maybe talk to other guys from the community. Yes,.

C

Instead of me from Linux community, someone can in like respond to what is the model that we are gonna going forward? Support in Linux I mean. Is it like truth.

E

Beyond sick network- and you could probably yeah.

B

We will probably have to post on sick networks, so one of you guys can draft a nice message and post it on sick network and see if they can give us some guidance there like getting them to come to our meetings. Probably gonna be hard, but you can initiate the discussion on their own channel alan.

C

Is it possible for you to guide and like lead and I, can give my windows information? There sure sure we can start an email thread and then begin I really think I at least we are designing. Our entire thing are based on CNI spec, like the v2 spec is just everything based on C and I suspect. A video of readiness, API surface, so I know that that's the model Linux is moving towards when.

E

He turns, does anyone know when Cena version 4 will be a Python code, I.

C

Don't know need.

E

To ask also yeah.

B

So should you move on to the next couple of topics while we take this offline and possibly get some day down next week,.

A

Yeah I think that I think that's good and I just tried to just summarize the two options down here at the bottom. So if anyone has, if you could review that, if there's a problems, please add a comment and I'll fix it.

A

Okay, should we talk about the port range, real, quick or was there any other hot topics.

B

And then that I know off.

A

Okay, so this there's a community proposal here for adding service port ranges, and we were pinged on this and asked to review it. And basically, it looks like they're talking about using cube proxy to program IP tables to forward, rather than just a single port, be able to pass a whole range all at once and I.

A

Don't know if there's any didn't read it close enough to see if there are restrictions on whether it could easily be resolved back to a you know: proper subnet, you know with the clean bitmap it was arbitrary port ranges or or so on here. So I think. The bigger question is whether or not this is something that we would be able to implement on Windows or not.

C

The single port, you load balance across multiple ports and any piece linked to that.

A

Yes, and have it up on the screen here, oops I gotta find the chat window.

A

But basically, the the proposals, instead of specifying a specific port they're, saying, give a start port plus a range. So instead of saying forward, port 80 I could say forward port. You know 1,024 with a range of a hundred, so that would forward 1,024 through 1120 for all to the same pod, I.

C

C

We are going to reserve that much of port on the host.

C

It can like, on a top level somebody cue props. You can convert that to a single set of vfp route and use it oh yeah. They have mentioned that.

A

C

Can I get back to you on Windows specific implementation like I, think I want to say: BFP supports fort rangers. What I need to get back to you, but it is something that we have to expose. There are two ways we can do it either.

C

We can make Q proxy split that into multiple ports and program as individual port rules a program, the portage so definitely Q proxy- can handle it for Windows and we can bring in the platform support of taking the port range itself if it doesn't support already, but that's something we can definitely look at I'll come and down this okay.

A

Excellent Thanks.

A

C

A

Want to look at that or have any other topics.

B

So I have a couple of things: I wanted to do brief. Everybody really quickly. So.

B

Essentially, a couple of the peers that one in were around adding some more of the cubelet stat summary details, and we have one thing: that's not there yet and that's network stats and the PR was filed for that that PR 64 9:01 and we would definitely not going to get to that I'm, assuming nobody from Microsoft or get to that ahead of time and face care, was the one that filed it so afraid that was you so I'm assuming this is probably gonna, be 112 or or something so I want everybody to know.

B

So most of the statistics from the table are gonna, be there except that and then let me see here- and there was another discussion.

B

Me see sorry I'm. Looking for my notes here, I want to bring up.

B

Okay and the other PR, the one in was the appear on the docker containers, resolved config path, so that was empty for Windows, so so that the piie are winning, and now we have, inter clustered in as a resolution of Windows, so good job of everybody that that code are working. I.

B

Believe, though, that the few folks that did try that they couldn't get it to work with with some of the SKUs of docker, so the guidance there was a if you are gonna try this make sure you use docker ee with the right version, so it does have a dependency on specific version and use that for kubernetes, so don't just use any vanilla, docker version and definitely has to be the Enterprise Edition.

B

Alright, that's it that stuff that the Hat I want to make sure everybody's aware of.

A

Okay, so I had and we'll close the DNS discussion next week. So thanks a lot hi.

E

A quick update from me um if somebody wants to try out to VN and obvious without hyper-v support or I, did PR on or being what ninety is sensible playbooks. So if, when everyone anyone wants to try it on clothes, feel free feel free to ping me if you encounter any issue. Okay,.

B

Yeah, are we still are you supporting that through those uncivil scripts that you guys have if.

E

So it's basically changed the installer, it's a better installer or if everything goes well during testing from the community, we might end up signing the Installer okay.

B

Recorded support.

E

So 1709 I tested with 1709, but it should also work on.

B

1803, would you recommend that over there, because I know you have the github open, V switch over, you know kubernetes page, and you have the answer for playbooks. There should I not be using those. If we want to try this out.

E

You should be trying I mean only this clip is the link to the Installer for exam. Okay,.

B

Yeah, because that one still has the hyper-v feature as a known issue: I guess: you'll update it. Okay.

E

This place will pure, which will remove it. Okay got.

B

It thank you very.

E

Good, a good feedback all.

B

Right. Thank you. One.

E

More question I, don't know if pop is here, we were wondering if you could post the vagrant, that you did on the roving and kubernetes talk if it actually makes sense for, for you.

B

Yeah you mean the stuff that Bob worked on. Yes, okay, yeah, just you want you and Bob talked together and like a belief he did. He can definitely post that if, if you guys both agree that it's stable and good, so why don't you sync with Bob offline I, don't see him today, okay,.

E

Thanks! That's all for me, cool thanks.

C

B

Thank you. Everybody have a great rest of your day. Alright,.

A

Thanks everyone. Thank you. Thank.

C