Istio Networking Working Group, 27 Sep 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Istio Networking WG meeting - 2018-09-27

Description

- Istio CNI proposal: https://github.com/tiswanso/istio-cni

A

All right so I think we can get started hi everybody and welcome to this your networking community meeting. We have one item on the agenda, but a very interesting one. So it's a proposal. Mr.

A

tonight proposal by by team- and there have been already a bunch of email- is exchanged on the subject, so I think there'd be quite an interesting and exciting discussion and I wonder team if we could start by giving an overview, maybe not everybody in the meeting had the time to read the document, so that would be helpful to like to get a good discussion and which we feel free to present yeah.

B

A

B

B

Rafael's presentation last week, last or two weeks ago, on the pod number controller I started investigating this P&I more because I thought it fit a little bit better and it's called some of the problems that we were talking about. Last time with the synchronization and in my colleague, Robert Lee and I have been discussing the CNI as well as a solution for the IP tables set up also in the context of like ipv6 and dual stack and stuff.

B

So I went ahead and created of TOC sorry I'm sharing the actual news, so yeah I went ahead and created a TOC with a quick implementation based on the sample, CNI plugin, which is really basic and then and borrowed a lot of the install stuff that calico goes so you'll see references to calico and the implementation deals details, but really the this doesn't depend on calico in any way. So it's just a implementation. That's requires just that.

B

The kubernetes cluster has the cni plugin enabled and it and then your normal eye, pan plug-in or whatever you've got for actual, actually doing pod network networking. This plugin will just inject or the installation will just inject. The configuration of this is TOC and I plug-in into the plug-in chain at the end, and it's sort of just a pass-through. So all the API events- it's really just returning the previous result of the of the prior plugin in the Train in the chain and just doing the handling, the command command.

B

Add action only and doing the IP tables set up for the redirect to the sidecar is: do proxy, so I quickly, I think it may be it's better to visually show what happens during the usage and then we can about implementation a little bit more. If anybody has questions so I wrote up a in the readme, a quick set of info on how to use this in a normal cube, Adam install you shouldn't have to actually even change any of these options.

B

Unless you install this, you know in a different name space other than sto system. So the way it works is there's a helmet that has sorry not a house or just a regular manifest that will create a couple. The cluster role and role binding for a service account that is usable by the actual plugin sorry.

A

To interrupt, could you please increase the phone if it's.

B

C

A

D

E

Go and you can just command+ what yeah, if.

A

It works if not I would just move closer.

B

So yeah, so the manifest is very similar to the Calico, manifest in in the aspect of just creating a service account and a demon set. So the service account has really just get privileges on the pods and then the demons that is is really just the Installer. So this is the thing that goes and copies the binaries into the offender for the CNI plugin and it caught, and it injects the configuration into the config file. That's set up in the in the CNI config directory.

D

One question: so the demon city is going to run one process in each node, yeah.

B

D

B

D

Watching for pod creation and Xen is doing the initial ization.

B

So this is the the the this is just for installing the plug-in on to each node, so the plug-in actually runs as a in exact from couplet.

B

A

This you know means.

D

A

Manage the network interfaces so to create the network interface, give it an IP address, so I think here. This is like what I'll do after right. It's a chain receiving.

F

This to be installed, yeah.

A

This does the install but I think Austin was asking.

D

If it, if it is easier, I mean, is it possible to also run it permanent, a me to have some process that is resident, and it's also watching for other stuff? No yeah? No, you can.

F

Have a demons that run a process? That's there all this.

G

Is not, this is just make second, this is not yeah and I is invoked.

A

B

F

B

Have some other process, the demons? That's that's calico does actually so. The the calico demon set deployment is as multiple containers in the pod and they they have that's where they run their no TJ in in, and this is just they also run the install container there. But so we don't have a need for the no general agent right now so I just have yeah.

D

That's what comes yeah.

B

A

So I have a question, so your proposal works in addition to existing plugins that manage the the networks and the interfaces and the IP addresses right exactly.

B

Yeah, so I can I was going to show that actually works so in this is a green screen here, and I can actually be able to increase the phone on this one. This this is just. This is a host. That's all in the one right here. I have another screen where I can show the gke setup, but so this is the pre pre condition right now for the plugins not running yet.

B

So this is just a regular calico config and you can see, there's a generic header for which CNI version and then a plug-in list, and so even calico uses a chained approach. So they don't just install this single, singular thing: they they have a port map plug-in as well.

B

They always seem to use, but the way that this works is the couplet calls this in exactly this order, so each each one of these types is a refers to a binary in the cni bender, and so each each pot event or pod creation event that the couplet will call in every plug-in in this chain. It'll pass for the for, like the port MAP plugin, the result from the Calico plugin would would be in the the message that it gets invoked with or the parameters that I gave them both with so similarly I'm SOI.

B

So what this install script does it's just installed? It sets up another entry in this list with the configuration of the s TOC and I plugin and.

A

Sorry so these are executed in order right, the plugins exactly.

B

On the reverse order on belief, but.

A

They are executed in order, but somehow there is like is there and is it a chain or its each of them executed individually? Is there any like input, output, I think the last one has to provide the IP address right, yeah standard out, but other than that exactly.

B

So there so the executable for port MAP would see the result of what this I think. This calico plugin and.

H

B

The port map plug-in may add, or, or you know, change something in there, but it can like similar to what I've done it can just take that for the result that it saw from the previous one and just return it if it did if it added no new IP address or anything like that, so that you can kind of just be invisible a little bit in the chain here and not disturb what the other plugins are doing. Okay,.

A

I mean this is this is great and if I have to see an ID plugins then is there? Is there a way to control the order like I have one for, let's say, network policy and I want another one for a disease.

B

You would need to control the order by this. This list order. Okay,.

A

So it all goes in the same file. Alright, thanks so much part.

B

Of this but CNI specification, so the way that to install the I'm, gonna, I'm gonna just apply this manifest to to go ahead and install the plug-in, and we should see so it injected at the end of this, in this install script is just using JQ to to inject on this.

B

So it's just created another list, entry with all the print with with the reference to the executable and then the parameters that for the plugin. So that's that's how it gets configured, and this would be this happens on a per day basis. So that's why it's a demon set, so all those all those containers that I just referred to they're running on each node and they they run it's just a batch script that installs these these configs and also there's a in the bin directory.

B

We see that there's an it's d-o-t and I executable, and then I just put the VoIP tables sh script that sets up IP tables, and so this on command adds. This is the OC and I will clot figure out whether the pod is a an envoy, enabled sidecar pod and then run if it is it'll run this executable, the sorry, the issue iptables sh with the right parameters, so I can show that I'll, just I'll do like the book in flow example. So I have this setup with its do already.

B

Is Dios installed in sto system and I've? There's no book info I've already got the book in fallout rule set up, so we don't have to play with that.

B

All right, my mr. gets screwed up.

B

So book info with the side cards being created, is it's getting set up right now and then just I forgot to show earlier that this is just the demon said sitting there so that that bash script that did the install I just have a a while one or while sleep in there. So it doesn't. So. Would you that so.

H

A DS without the ended container to do the IPA table.

B

C

H

A deploy, yeah.

B

Exactly so the the way this would work it doesn't. It replaces the innate container. So we don't need to have that in the auto sidecar inject. So.

A

Are you, are you running IP tables from inside the cni plugin like.

D

Touch a stool, a space does.

I

B

Well, yes, NS enters into the names, so it runs. Ns enter with IP table to into the the parameters that see an eye plugin get from the from google. It includes the Cabrini's pod net namespace, so I don't have to look that up in any special way, and so it just uses that and a center into the namespace and then runs IP tables. The thing that has to happen, though, is the kubernetes.

B

The plugin access is the cool API, so it has to get the pot info to get the all the container ports for the pot. Oh I see.

J

B

And then- and it also uses that pot info to determine whether the pod is has an envoy sidecar, so I, that's kind of hard coded right now, just check okay is this pod doesn't have a pot container named, is do proxy and is the number of containers greater than one okay.

H

B

I assume it's misty, Oh sigh, you know user container, so.

A

Here is that the YP table can't we create like visa from.

C

A

I

B

Whole thread went to that direction so immediately. Okay, I see.

A

D

Have any executive, Aussies, yeah.

B

Yeah no I so I we were thinking about it more like from ipv6 and dual stack support level, but yeah.

H

B

Right to the, how do we get envoy out of the sidecar yeah.

D

B

I, maybe six.

D

We also need to fix the denials that we haven't seen those hockey they were script. Don't you yeah.

B

Yeah so so this is the you can see the template for the sidecar auto-injector has no in it containers section and then, when we look at an individual container.

B

H

A question where you're doing that, so that's this plugin would always watch out all the paths. On this particular note. That's not belonging to is still system namespace and as soon as it reaches running assume it has the Envoy proxy. You would I mean how do you? How do you ensure you are doing it as soon as at the right timing as soon as the policy becoming active, yeah.

B

Great question, so that's that's inherent in kubernetes.

B

Scheduling so the way when the when the scheduler creates the is gonna, create the pod there's the cool it gets an event to create the network namespace even before the pod is it's only after the pause container started, I, think and it it doesn't. Even it doesn't even attempt download the images or anything for the other other containers in the pot until this plug-in chain runs. So some.

A

It's not inherent.

B

Synchronization there on so.

H

Faces a wish Romney. It should already be config because you are doing it really early in the cycle before the image is you can download it right, yeah.

D

The main difference between this dis approach and the other one with one set, because the other one couldn't want, is that it runs before anything you started so yes, yeah.

A

Yeah addresses.

K

A

Right place to do the work, really yeah and so I'm, not sure if I like, if it's clear, for you mean that this is not like the kubernetes controllers approach, when you have a process running watching for resources in kubernetes, this is something that gets invoked by the cubelets. So it's really for each fault. You get an invocation right. It wasn't super clear.

H

A

H

Him just made it super clear, yeah.

A

D

Remind people from the two or three weeks ago or more that we discussed the other problem where we're in voice carta can be slower than the application stuff that and we may get up a few seconds where things are not in a good water and the idea of moving some cash. You know pre putting some configuration in the unit container, so that's actual envoi when every runs it. It also has some warmed up configuration easily some some warming that can be done. So that was the plan at that points.

D

To modify the image container to do something to to help as a proxy to start faster right.

F

L

D

It's a condition of cyanide in have we started again the whole interest that we have in in running and voi or integrating was a sidecar container with with with a CNI plug in this import to address to address those kind of problems. Does that I keep asking about? You know I.

A

Can do that, though, because you're my has to be downloaded and with the download remember.

G

A

D

A

Three feet somewhere else: yeah Modi.

D

Can we can fish from pilots and regulation? We can have something of just HSN shark illumination from pilot s then went whenever weight, starting it already emissions towards a five-song right, so.

A

You're talking not about when envoy is injected into the pot when em boy is running so.

F

We're saying that right.

A

F

Don't need to do webhook injection of one go into the poverty.

A

Alimony in a container.

F

D

But before we do that with another summer, some options 20 psi programs. We are seeing right now.

M

So but this requires like changing the couplet, the command line right that P, that which is something that we don't have access to the I.

F

See my gives, you means to basically inject a step of behavior into the cubelets flow of initializing.

M

This is a new C&I plugin that I want you to invoke.

M

A

To deploy it in the that's directory and it will be loaded by by kubernetes api server, so the only thing that so when you start your baby, I server you have an option where you say: where is this en I plug in and if you put equal equals c and I it goes to the directory. That's Tim showed in the beginning and logs everything that's in there. Well so.

C

A

Need to change the you don't need to restart your server right.

B

You're exactly right, the part where it's a little clue is that it looks at only the first. This is maybe a kubernetes specific thing, but it only looks at the first file in this directory line. Alexa graphically so it's you, you have to to add the plug the chain approach. You have to be able to detect the first file and then make your config into that file. So it's a single file.

B

F

Either way this problem spaces well understood right. There are a lot of TMI matters today, having this mm-hmm nobodyís.

M

Questions like does it require I mean what happens in a managed kubernetes environment like can you just inject your own C&I plug-in or does it require you to have handle over the case over to you.

B

You have to be able to have access to you know to this file system location on the host. So when you so I sort of run that container as.

B

Maniacally mount this by this path into the container, and then that has to be you, so you basically have to know this path. These are command line options in couplet, so you you do kind of like for each hosted to raise environment. These things could change right. So you.

E

Yeah your higher-level questions. Yes, all the hosted grades operating support.

M

Scene, are you adding your own CNI plugin? All this is those that they have already configured as.

E

Far as I'm aware, every hosted kubernetes offering support, Springer and CNI the question.

A

Is whether you need to actually you need to restart? Do you need.

H

A

Your API so does this. Does the kubernetes infra read that file every time when you pod is create it or it's rigid one in once in the beginning and then relies on the list.

B

Or if it's, if it's able to detect changes to the file system or you know, but it doesn't, it definitely takes effect. Every single I could like, like I, showed you in this demo Marigny.

H

B

These pots, you know ups and running and kubernetes couplets running and I change that that Steel City not I'll, changed it and it didn't restart couplet it just you know, exact yeah. This.

D

Is like so and I know believe anyone is proposing to you know kind of drop. The existing injection complete I mean this will be an option for clusters where you have C anion, and this is possible and if it is not possible, we may shift our recommendation of.

F

D

Can lose out? Do it otherwise fall back and you don't have a perfect experience, but it's still working so by.

A

The way like I just found an example on the web. You can write the cni plug-in using bash in a few steps right. There.

D

If you, if you are able.

A

D

A

But like it's a fairly simple sphere, right.

B

Yeah I think I have on this host. That's done, but maybe not, it is pretty simple. I can focus on it, concurrently, yeah, so I.

F

Mean just just I reached out to Tim Harkin and some other folks in the kubernetes world who have a lot of experience with C&I, so we're gonna try to get some feedback from them about maintainability or reliable any concerns with c, and I as a mechanism for doing that, so hopefully we'll get some of that feedback or at least know where the pitfalls are. This will be good to have that and I think I see spikes on the call.

F

You would obviously be good to talk to the other folks doing the other parts of what see my is doing like calico or or psyllium. You know when we were going to be recommending solutions where those two things were stacked like what what are the best practices cuz. Clearly that's gonna happen. I.

I

Think, oh yeah, unless someone else wants to be at the end of the chain. Well, then they.

E

D

E

I mean it doesn't really I, don't think it's really material like we're in the chain. We run right well, as.

F

E

After the initial networking and.

F

Then, while we're doing IP tables, that's probably true, if we're doing something other than IP tables like let's say these user space, IP or EVPs and there's a more detailed conversation later right.

E

But I mean but we're that all goes with config that we publish that says here here: you're using the C&I plugin yeah.

F

So we'll just make sure we work with everybody to make sure that we fit in.

D

What was the other discussion we have you give and so forth? It's certainly better to be in the CNI world. Yes, right.

A

So in here we'll just replace whatever the CNI does, with iptables.

G

D

A

Or maybe there be other solutions right.

D

K

Seems on unambiguously better than what we have today right, actually.

A

Like us people, because we had the other proposal using the pod network controller right.

H

A

I see Rafael and Andrew I, don't see them online, so we kind of need to converge as a community to one solution. So I they're.

D

Very similar in implementation, it's I think this just adds an extra step. I, don't know it said all you share. How much would you share that? How much here, but.

B

It's really simple code, I mean it's just. The only thing that is really is doing is accessing the coop tube API. They get the power right. I think that's.

F

That's a big difference right. Yes,.

B

But not having.

F

To read like the cube, API server and reading from pod info and getting the Annette hooks that way, because right now Raphael is that has to pole, yeah.

D

Main difference.

N

He was reading the pods.

D

N

It's really pod info maps in the filesystem.

D

N

Are you reading the body info from no he's connected to a case.

D

Every Indian from American implementation is an input, maybe I don't know, but it's so.

O

O

B

Yeah you so you, the one thing I would like the you know, the kubernetes parameters be more complete and it would be a little. You know a lot more detail in the what we get from kubernetes. So we really only get a few things in the front: actually Cooper net actual internet II. So what we use sorry I'm, trying to find the code here, but.

D

B

So this is the the handler for adding a pot and the there's being.

I

B

To just grab get all the parameters, so there's kubernetes arguments that we get from Cooper from directly from the parameters into the into this command arc, and that includes the pod pod namespace.

B

So this is the net namespace and it has like the container, ID and stuff like that, so I, so I just basically used that to connect to III used that in the parameters from the CNI network config. So you can see that there's a coop config here that the install script creates that coop config from that service account that it that the manifest sends up and and then gives it to this, the plugin, the plug-in config so I in in that I. That's that's!

B

How I connect to the kubernetes api server and then it just grab gets somebody silicone, look I! Think.

F

I get the idea I. We would like to not have to connect the qadi server to be.

B

F

B

Require some change to the what CNI gives us so.

P

When you speak to what you need from CNI, because I'm, a maintainer on that project and I can try represent your interests at our next meeting. Yeah.

F

P

Not sure we actually.

F

Need parameters passed in C&I cubelet, copies files into the pod before starting? Yes, that contain pod metadata, which we could read yeah.

D

Instead of going to the API server, okay.

M

So I have one more question which I think I'm probably struggling to understand, so the cubelet, which runs on the host node, needs to access to this file right, which needs to be on slash, e-tec, slash, some CNI dot d, which is on the host file system and, if you're running on a managed, kubernetes cluster, for example, gke and you don't know Damon said T, you get access to the host file system directly, yeah.

B

D

I

Privileged then,.

M

You get access to the horse fight system, yeah, yes, including.

Q

The wallet master.

M

Doesn't this thing need adding that file to the master as well.

E

Course, on the master.

B

The sto system control plane at all in any.

F

Way, those don't usually run the pasture, but yeah. We might in fact they're deliberately partitioned from it for security reason right. Okay, so.

M

Yeah, you know just earn a privileged a mint set on that's well, you had the file targeted, yeah.

H

Looking at you, llamo Farah I saw you, have a host networker equals true I. Don't see you have privileges. True. Is that required.

B

H

I was just saying: I, don't see anything you're, it's your install! It's just yeah I'm we're trying to understand the exactly requirement. I just like to try it in a cloud we were basically based on what Sharon was just raising. So I saw you have this host network equals true, but I don't see, privilege, true I'm trying to is that accurate, based on what you've been decoy yeah.

B

At least with gke, so this is there's I I just uploaded today, a new version with gke manifest, and it you, if you, if you use a if you instantly run the manifest under a service account that has cluster admin. This works.

N

H

Admin and hopes now with the true okay.

F

Look make sense right, I mean yeah.

H

That that's true.

F

H

Don't want anybody mess this up so.

R

Tim, mrs. John slightly.

H

R

About the rumen moving these four IP tables, I was looking some similar approach to search essentially rational functions in two parts: yeah thanks for saying that, and your stuff looks a lot better than what I did so that's great well, what I did was basically somebody mentioned earlier: I create a V there and I, just like a a new network namespace to route all traffic in and out of it, and that even which route leads for IP tables. So you don't mean.

H

R

You can force all traffic through a new network namespace it's between the node switch and e0 did.

A

You run and boil a new network namespace and just basically created the V spread between the like container and and we're running in the other Nutter Nate says: I didn't.

R

Use on voyage I created my own little sample vnf. To do that so exactly that, yes, so I used I used network plug-in device plug-in to do this, which was a lot hackie. I think what Kim's done makes it a lot easier and I think I might I'll try to take Tim's coded. Do exactly the same thing.

D

R

Think that's a challenge because, with a CNF you usually just move low-level packets around you're, not having sockets they it gets a lot faster. You can do a lot faster. They can do things like DP, DK and stuff. So you've got.

K

R

Latency there's.

F

Something we needed to do this, this kind of stuff like oh, we can bind to an interface and capture start start.

I

Noting that it came today all right so.

D

Nothing to test the thing we don't yeah I, don't.

I

Know it's not like it.

F

Means table many user space.

D

You can just find to support preserve it's in that works. A neutral, socket permission is perfectly. We need to verify that you know. Z have zero. So whatever it's.

R

One of the questions- if you can do that, then you can go to my tea tables. You don't worry, but what applications are running on the pot it becomes irrelevant.

H

R

Can thank you, stick in any any proxy for this ng x or anything just becomes a standard ash and proxy. No.

F

M

So it's a VPS bed and then you basically have an envoy running on each of those things yeah. So.

F

There are kind of three distinct problems here. Right one is the CNI initialization there's: how do we do Network capture and then how do we schedule the proxy? Have we won the proxy that we want to run in.

M

F

Contact form or on a run it, the polymath.

M

Name in separate Network main space, you're gonna be out of the pod right. That means you don't have access or secrets, and so it's so.

R

You're in the pod, in the pod, it's just I, just I check I did was I change. The network namespace not the rest. The namespaces.

M

R

D

You can also run is aboard and have the secrets visible only in sides and VoIP would, which is such a security benefit, because you sometimes it's better you if you have the via telnet part there you can actually get out of the pod completely and have complete isolation between the sidecar and application.

R

Still longer site cars now I saw it lying right, yeah.

B

That seems like, because you can still use communities to manage your envoy pods right, yeah.

F

So that's the fourth question, which is, if that part, is actually debatable right, yeah, whether we actually strictly need that or not because.

M

That's the resource, accounting as a separate issue, or that.

F

Most and I wrote this up in the doctors and most operators don't actually care yeah.

L

F

It's only in situations where you're doing multi-tenancy on a node that the resource accounting is actually going became a serious concern. Well, most operators care about is that they can add it that the envoys are doing what they're supposed to be doing from a security perspective.

D

And I mean technically there ways to kind of work around this I mean we can because you can adjust the CPU per plumbing. Oh, it's.

M

Not allocation is attribution, you know.

D

Attribution in our location, both of the our program does allocation is also big. It's even a bigger problem because you don't want to have all the CPUs taken by bipods and not have any CPU for sidecars so but I see what can be solved. Yeah.

F

Ideal way, the existing Coover, these mechanisms for resource consumption attribution in.

D

Notation you would.

F

Be in play, even if the Envoy was logically invisible to the application code inside the pod, so we need to just find some recommendation for that. Yeah I think we have options.

D

A

So like coming back to the point, you enumerated right, the first one is like what what this proposal address is where.

F

A

Do we do this, like redirection of the traffic and the cycle.

F

Structure yeah.

A

So this is, it seems, like people are excited about this solution like is there anybody who disagrees or think that, because I'm personally excited about this and I want to know like if there's somebody who is not excited or.

H

Yeah so quick question: Tim is your steps ready for somebody else to try? No, you guys try it's GK. Obviously I would like to try it on the other clouds too.

B

H

Are excited yeah.

B

I would yeah I think they are so yeah, please if you want to take it thing me. If you have problems, that's that I think that there it's pretty simple and there from the code I can probably bring somebody up to speed in you know an hour or two okay,.

A

Okay, so we'll wait to hear from you Lin from Shira more like all the interested parties, but hopefully like the the major roadblocks with that right, so I think we might like it's very likely. We will go ahead with this, like going with the CNI doing the redirection inside this unit, login right and then next we can tackle like how we capture that traffic right now it's IP table. Maybe we can do it via for the visa, but I would actually like to see more from JA like what.

H

We did exactly.

A

I'm super curious about that option, but I mean I have a it's more of a gut feeling. I, don't know that that means.

F

Right so just it's, then you can you can clarify Marie. You want to jump in on this. Hey.

P

Yeah I, it sounds good to us like we're using si and I. If we can make a plug-in, that's cross-platform, that's great! Okay!.

F

So and I think there's a couple things that need to happen if we're gonna kind of consistently like move forward on this as an organization right. So when we need to write a document, we should get this into the sto org right, so that people can collaborate on our under those auspices. Assuming that we all agree, then we would essentially sponsor that work.

F

Three who's gonna work on it like who has been with what different skills our people don't be able to bring to the table here, obviously those things out, and then you know that design dog should probably come with phases right where we start with the simplest thing that Tim has and then we go all the way up to wild and crazy user space, IP stacks and other fun funk like DPT day and at some point, there's gonna have to be performance, testing interesting.

D

F

Right and that this I think is that there's a couple of different tracks within the project: yeah right, there's the spokes. You can work on the resource, attribution side and it's supposed to work on that. The networking side and that's some. What they're gonna be folks we're going to work on the hot dog right side right all right.

E

Do we have all that stuff flow? Can we not then hitch? Those I think we can and I think they can go and set up tracks, and we just need people to say that they're gonna own those bits totally yeah. It's like we're. Definitely interested in hot reload and I've been talking with you, some about that. We yeah in.

D

Order to make sure we have to prioritize the term at least the things we discussed the optimization. So we don't. We put the startup time of the site, yeah.

F

That's a critical thing: it's.

D

Been a pretty can capture that.

F

Is one of the party of all the things that we have once we have basic capture working? That would be the next most important thing for the health of the project. Yeah. It's because otherwise we're gonna go, spend resources right now, I'm doing like container sequence startups for the injection stuff yeah like Nate was gonna like yeah.

E

Sure it's so like sure, I'm just version PR that helps to address some of that. It's well that the kubernetes fix like we're. Always gonna have problems yes, yep, so there's there needs to be a design there as well, because there's a lot of complexity, around policy and security. Don't.

A

Need that anymore right, if.

D

E

Startup is inherently a problem of security and policy like.

H

There's there's.

E

A bunch of different stuff that we need to address there right, like.

D

You can't just.

E

Allow traffic in and out, while you're waiting, probably to be configured, for example. Yes,.

D

Exactly that no c'mon yeah.

F

So from the Google side, right now, most of our folks are super focused on fixing the kind of the config scale issues all right. We're spending a lot of time on that that work is obviously entirely orthogonal to this. I can get consulting help from the gke and kubernetes experts, on whatever predicting on the resource, attribution side and yeah.

H

I can get Tim to.

F

Consult on this, so we we like, we know what we're doing is solid. Hopefully, I can go chat with the Calico folks. The cilium focusing gets some guidance from them too, and they're obviously out there doing, combination part pitches anyway, so I think they've been motivated to talk about it. Spike I, don't know if you want to say anything on that subject: I think he dropped out of the call earlier. Okay, maybe I go to a real meeting.

D

Are we targeting this for one one? Maybe I mean that is the basic stuff. That is the other thing that we can do the other thing that we can do here at Google.

F

Is we can go around our customers and ask them you know, do some actual product management work like I can go and follow up on that and get some feedback. My having talked to the already, particularly by the security stuff, I think they'd be broadly in favor I think they actually prefer it Oh.

M

Riding on ways outside of t-pod god, yes, so.

F

What we're hearing is that the kind of security model which is the sidecar is part of your application is not actually what operators want. What they really want. Is they want to verify that the sidecar is doing what they expect right in terms of secret management right, and they don't want that to be under the control of the application right so will be of the operators the cluster is gonna.

F

Be able to do is audit that sidecars are running, that they're, not violating trust domains and that they're running in a manner that they control and not the application, and so, in fact, to some degree or other side cars being injected into the pod and then not totally controlling that process mix is actually a downside right now,.

M

Are these night cuts gonna run outside the pod namespace? Well,.

F

If the application developers have no control over where they won and the only requirement we have to meet is the operator requirement.

M

All they are moistener, but then we still run the.

M

F

Not actually priority.

D

That we can also improve the way we walked out of right. Now we put an annotation to not have the injection. If we go this way, we can say hey if you have a class that permission on your service accounts and you can opt out. Otherwise, you are in so instead of things that you know, we just put an annotation you're out, we can make it more secure and enforceable. Basically, I mean.

M

This is not a resort connectivity issue that people have.

M

Build is also resolve any of the health check issues. I, don't know if couplet is actually gonna talk to the yes.

F

Thank you because we can. We now control the lifecycle, the health tracker for using demon, sir.

E

Sorry, it lets us control.

H

F

Health checking separately from Paulette health- second, yes, okay, yeah and the timing of it basically.

D

We can start the sidecar, make sure wait until it is completely initialized and nothing will run.

M

Scoob Anita's help track will still go through an way into the application. Yes,.

E

Yeah but, for example, we could make envoy startup part of the CNI hook started.

A

Ready to take on the traffic, you don't start them as part of the CNI hawk-like.

F

Well becomes available, but.

M

Because he'll check completely, if you're not running as part of the pod namespace by effectively trying to like putting a hole in that hole, it variable stuff which says like if you're coming from the Kubler with this IP since for any other demon set and so on, we have a you know the IP address that the cooperate is actually gonna use that in put stuff in the IP tables, I said that traffic is not captured, in other words, which actually allows us to like get it on. The health check issue completely right.

D

E

I mean, but there's still an inherent just envoy, help check issue to like that. Does it that only addresses part of the problem right envoy still has to be ready to serve in a fabric that.

F

E

Still broken right so because clean.

F

These separate those two things yes and that's really important, particularly when it comes to food or diagnosing issues like if envoys gone sideways. What does the cluster operator do so.

A

Louie speaking a bit of timing and sequencing I, don't think it makes sense to delay one one and they we have. This whole thing ready, I. Think.

F

A

That is actually nothing too much pending in one one, because, though I think we will drop the incremental legacy s so basically one one will you can be ready soon. So.

F

The things that in theory were in our car, install yeah, Joey and Paula.

A

Decomposition, yes, so.

F

C

A

We're ready with one one and that what I want to propose is actually to make this feature exactly in this current form with IP table redirection, and we still have the injection in this in the pot part of one one. So.

F

That that comes back to my question able to work on it.

F

Alright, so health checking and timing issues, I think, would be if somebody could pick it up. Great first step is to get a design doc right.

A

Let me ask you a question, so when you say house checking and other things are, they still would inject it cycle or with the full of endpoints. It.

E

Doesn't say it doesn't really matter, I, don't think.

F

E

Injecting at all I think that would be a big usability work.

F

A

Agreed completely I'm just wondering like again like who's available.

H

A

I'd like us to delete incremental things and from from my point of view where we have something that it's a big step forward already like even with this robot right and then we can like worry about the other things. For one too, because I.

F

Know and I think that's, let's get it design, not together. Let's list at the face of the project, let's figure out who can commit to doing what and why yeah those people do the work and if it fits in 1.1 it does and if it doesn't and later Friday.

D

We should do some prioritization.

F

Okay, the trader, if we deliver say the the health checking feature or that prevents Nate from gonna have to do bunch of other stuff. And what else are we saving? By doing that, we should figure out where.

N

D

N

D

Nine already, but there is zero, VPN work that he's doing. That's also probably what we want to get in one yeah.

L

D

Part in one one is that we need to do stabilisation, testing and a lot of other stuff set take months, and it's far more difficult in sets. That's wonderful! We have the code when 90% of the work will be testing it and blow testing it. We.

A

Need everybody to participate? You know like if we start like adding our new features like moving to a full of a.

F

Nice thing about this is this can actually be delivered entirely. Independent, oh yeah,.

B

We could this yeah.

E

So so, like trim and I created an SEO ecosystem, github word the other day to start putting some projects in we put this in I would say straight is Jo org sure if.

F

You're, all in agreement that happen.

D

You know should it that.

F

Should definitely be in a separate Rebecca right.

A

D

I seen that I see the point that it can be done independently, so this is technically yeah completely orthogonal to the person of Ischia, so we can decouple their releases yeah.

E

Yeah, the one thing that may want we want to land in upstream is just the option to remove the in the container as part of the home chart. That would be the only thing that I would think.

H

Yeah I have a quick question costing you talk about the CEO VPN. Is that different than this you have something else. No.

I

It is a yeah from freedom that you are you doing. Hey.

F

Folks, volunteering to help open up the node stock and put your name this way to track it and then we'll come back once we yeah.

B

R

Q

B

Q

You want to want peace, do.

F

You think you want to work on yeah.

A

And thank you very much for volunteering.

L

H

L

Need a design document: do you think it's a one design document or a couple for some of these theories? I.

F

Think it's worthwhile having a high level and.

L

F

Of the different pieces of the project and almost do more detail ones phase by phase right, but we roughly know the different pieces and we just need to blow our description of them and what their dependencies and interactions might be in a delivery schedule associated with them like if we're gonna do like you know the bridging right there probably be a more detailed design doc for that, but I think we can write like a couple of paragraph outline of that sure, and then we could kind of figure out where they're gonna order. It's.

D

A benefit three dose and time misdemeanor kind of yeah.

A

Volunteers, so do that the.

F

A

F

Can write an outline and I can plug the people who have more expertise in some of these areas, but Tim did the original work I. Think Tim should Polly yeah primary over.

M

B

Yeah I could start it and- and you know, make those sections or I feel like V's, bridging and all that stuff, but yeah I was thinking more of them on the incremental change of my this, but yeah that that's I have to you know see if we can yeah okay, I'll.

F

Work with Tim Tim can be the permeable for he gets credit. You first mover advantage ten and then we'll all pile on to make sure that we fleshed out the ideas and make sure things are covered. Then we'll do another pass to just flesh out details and get our broader review and shouldn't take too long in the.

D

Meantime, though, I think.

F

We should forge ahead with the repo I. Don't seem to hold back so.

A

Tim, do you need any like help? You probably need to talk with Martin Shriram, oh so one of them I think and create the repo or somebody as somebody who's admin can create the repoint. Then you just be happy to set up with your integration.

B

D

My mother yeah.

B

That's gonna be worth the work. You know, yeah.

F

This is also gonna, be a bunch of a bunch of stuff that has to happen.

H

In this project inside it's your repo, oh, do we need a separate projects, so are we sinking sto, / sto,.

E

Yes, in an oil and.

J

D

J

It's not exactly.

B

A

Once in a year and the meeting in time like do, we have burning things, all right is.

E

This, where I, throw that under the bus and say let's talk about note up crazy.

J

And we can talk in the environment.

A

J

A

I mean I I'm yeah. We have to keep it awesome it in time. Yeah.

H

Just good advertising, we just add to the agenda for the environment group meeting we're going to show a demo. Some of the multi. Can cluster configuration automation. So if you guys are interested, please join next. Wednesday I think it's a one Houston yeah! We.

F

Should get a virtual network ID on the schedule for the next one? Oh, yes,.

A

Actually, yeah alright.

D

A

You very much thank you very much Tim for I think this is brilliant and hope to see you again in two weeks in.

I

One week, environments.

J

Meeting we can't wait.