Kubernetes Office Hours, 20 Jun 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Office Hours (West Coast Edition) 20180620

Description

Join us the third Wednesday of every month: https://github.com/kubernetes/community/blob/master/events/office-hours.md

A

All right, we are live if you're in the hash office, our YouTube channel. Let us know how the audio is. That would really help us out so welcome everybody to the afternoon edition aka West Coast edition of the kubernetes office hours. This is your monthly live stream where we have a wonderful panel of experts.

A

Did you see all up here, we'll be answering your user questions from hash office hours on the kubernetes, slack and kubernetes users will talk about what's happening in the kubernetes landscape and do all sorts of fun things so with that before we start, why don't we have everyone introduce themselves?

A

Let's go Bob chef, Mario, Solly Stefan in order. Ok, ok! Well,.

B

My name is Bob Dylan I am a research cloud administrator with the University of Michigan. Sort of my specialty is on prime deployments bare-metal networking and all the crazy stuff that crops up in between.

C

My name is Jeffrey Sica I also work for the University of Michigan, and these days, I'm kind of all over the place. So I'm mainly a developer, though.

D

Hey guys, I am Mario Lauria I work with George and Jeff to create community here in Ann, Arbor Michigan and the surrounding areas, I, work from little weather, site, reliability, engineer and I'm, mostly doing bare metal and networking and storage and other things around Letty's realm of production hi. My name is Sally Ross I'm, the sig lead first big auto-scaling and a general kubernetes contributor, I work at Red Hat on kubernetes and on open chef and so I have some also some miscellaneous collection of production information.

E

Hi I'm I'm Stephan Pradhan I work for weworks I'm, part of the developer experience team here, I work mostly around integrations around kubernetes with other cool stuff like East EO, and we also run our own open source projects around communities and also I'm into that yeah. My background is the goal developer mostly.

A

Awesome and I'm sorry and I'm, Jorge, Castro and I'll, be your host today, I work a hefty as a community manager and I help organize and take notes and feel the questions, and things like that. So before we begin first of all check the topic and hash office hours. That will give you the links to the live stream when the dates are and the times are. We run this on the third Wednesday of every month and before we again, let's just talk about some Sun.

A

Some quick house rules ground rules to get everyone started. So if you're in the channel, this is a judgment-free zone, so everyone's had to start from somewhere. So if you see someone who has a question that you think it's too simple, that's fine just--how just help them out and what we will do our best. To answer all of your questions. We don't have SSH access to your cluster, so there might be situations where we can't really help.

A

You live debug, something, but at a minimum we can at least try to get you to where you need to go or show you where you need to look to help. You solve your problem, for example. This morning we found an example that was undocumented and we were able to file that as an issue in kubernetes. So this helps us get feedback from you as users on the things that we need to get better at um panelists, you're encouraged to expand on your answers with your experience and their pro tips, and things like that.

A

So, while you can't answer the question, if we're talking about a specific area feel free to add any expertise that you feel would help their audience, you can help us out while we're answering questions if you've seen this problem before or you know something about, it feel free to just whack in URLs for documentation or for maybe blog post.

A

So if you have any information that can help the person feel free to just put it in the slack channel, so we can help each other out and also we'll be looking for URLs in the channel that we could read out. If that helps everybody, you can post your questions in the slack Channel or if you've posted one already on Stack Overflow just give us the URL in the slack Channel. If it's a generic question, we have a thread on discussed that kubernetes on IL, where we collect or we're collecting questions.

A

You can also help us out by tweeting spreading the word paying it forward. You could subscribe to the channel here and you'll get a ton of kubernetes content. That's just very useful in in learning how to use this stuff. Like we said, all of these sessions are recorded and if you're using this as a work resource and it's useful for you give some feedback on how we could do better. So we can make this show in useful for as many people as possible.

A

If you do want to sit on this panel, these are all volunteers so feel free to. Let us know if you have a specific area of expertise. We, you know we have a lot of generalists here, but if you're, if you know we've always finding that we need a storage person, for example, that sort of thing. So if you have expertise or if you've done some really awesome of kubernetes and you want to help pay, it forward feel free to contact me and we can have you sit on the panel.

A

The commitment is one hour a month, but if you can only do it a certain times, well we'll work with you to get you on here. If that's fine, you can help us take notes. If you want we'll toss that hack MD URL there in the channel and we're always looking for marketing help, especially getting the word out to users, there's like 40,000 people's and kubernetes users and we're always looking on how we can help them at the end here will be holding a raffle for kubernetes t-shirt.

A

So the way it works is, if you ask your question and we address it on the air. Jeff will put you in a little raffle and at the end, well we'll figure out who won and then I'll give you a code. You get a free, kubernetes t-shirt, which none of us are wearing today and, lastly, feel free to hang out on hash office hours afterwards.

A

The main channel could be very large and intimidating very busy. So we like to keep track of questions to happen in this channel.

A

So if it's in between office hours and you run into an issue whacking in the channel and then we'll get to it as we get through a Q, what what Bob and Jeff do before each session is they'll go through that months worth of questions, and if we find something that is unanswered will usually do our best to help and then worst case, we'll send you to the right sig or to the right place or you can get out so with that. Is everybody ready to begin all right great?

A

So our first question comes from oxygen thanks for joining says. If you have an API deployment that will be used by your internal application, but also externally through ingress, is it recommended to use two services, one that will allow an S, lookup DMS and one for the incas controller could not find a way for the ingress service to give pot IPS? Or is there a better way to go hard to formulate it hope that made sense.

D

So I'm I'm not entirely sure what they're asking here haha in general, you should be able to use the same service. You should be able to point to the same service from like ingress and use it internally. Like there's I, don't.

E

Think, there's any reason that.

D

You should be able to just reuse an existing service as the target of ingress.

D

Perhaps can you clarify unless anybody else has any feedback here? Can you like clarify with like some like pseudo Gamal or like pseudo kubernetes api object to like kind of clarify like what you want to do and and maybe like the part, that's not working. You.

C

Know like the equivalent of pseudo code.

D

C

The one thing I was also gonna ask is, if they tried getting to the service endpoint through the internal DNS name,.

B

Yeah like that, that should that should matter, if it's going through the ingress controller for at least for the outside stuff, yeah.

D

B

Like if you're.

D

Inside DNS should work. Fine, presumably assuming you're running you DNS, and if you're outside the ingress name, should work fine as well. It.

E

Also depends a lot of what ingress you choose right. Even we have this nice interface. It depends a lot if you use nginx, that to get directly to the pod will not use the cluster IP service. If you something like eesti, oh, they have their own stuff like a guitar gateway, their own routing. So it depends a lot to diagnose such problem. It depends a lot of what ingress controller reaches.

E

But they are usually the cluster eyepiece that we should work inside and we use that in your ingress. Diffusion, mm-hmm I see typing. Yes,.

A

So oxygen we're gonna go and let you think on that for a minute and type and then we'll come back we'll enter these two questions here and then we'll get back to you. That gives you time there to to. Let us know what, so, what do we need? What ingress ease controller he's using and Solly would like to see a little.

E

Code or something like that, yeah of.

A

What you're trying to do so? Well, that's happening, and those are you that are listening at any time. If there's a way, if you need something clarified or something like that, just type it in the channel will always come back to you if need be.

A

Mc says: I have two real basic questions. If you don't mind answering, is it possible to do a label, selective key equals value or other key equals value? I could see in the docs examples of doing key equals value or other value, but no mention of an or statement across multiple labels.

B

And I think you can still do that with set-based selectors I.

D

Mean I think it's I think it's it's and so you can say like, like, like the person said, you can say: label in 1, comma, 2 and label it forth and other label in 3, comma 4, but I don't think you.

B

Can say: oh yeah, not.

D

Across those labels, yeah I, think like so so in in set based label selectors for people who are not familiar so like the traditional label, selectors that a lot of people are used to are just basic of quality label selectors and it's like label 1 equals value. 1 label 2 equals value 2, so instead place label selectors. We can have different kind of predicates for our labels that are go beyond just equality, but I believe those are still like. All the predicates.

E

D

Together right, so you have to use it. You have to make those as two different requests and so right here in the docks. The sentence in the case of multiple requirements almost be satisfied, so the common soldiers, so that separator acts as a logical, and so so you can do some fun stuff with asset based labels like yours, but you can't you can't more things.

A

Okay- and let us know if that answered your question moving on to the second question: if I have two containers in a pot, is there a clean way of ordering shut down, ie I want container to to ideally shut down after container 1. Is there a better way to do this as a pre stopped cryptic container 2 that waits until container 1 stops before responding to a paying or curl, or something like that.

E

So think this is the perfect gift case for Enid containers when you execute them one by one. So let's say your main container: the runs in a pod needs some database back-end to be up. You can create an init container and do a while there and check if that back-end is up or not. If it's up it will unblock, and you can actually go to the stage where you run your actual container in the pod. So with Enid containers, you can have this sequential block system tour castrates, something like that. Okay,.

A

And do we have a link that we can plop in for Annette containers, yeah.

D

I I'm, not in me, perhaps if you can clarify a little bit what you're asking because I'm not entirely sure Annette containers solved their use case here, the way I'm reading it is like they have like container a and container B right and the pod gets us in there. Both long running containers and the pod gets a shutdown they're looking for ordered shutdown, yeah.

E

B

It's kind of the opposite of a net container.

E

B

Was actually something there was an issue raised on this reason, I think called like deferred containers or something like that to essentially have like an it for shutdown.

E

B

I, don't believe that functionality is in there right now, but in in general, just.

D

Kind of like as a design decision you want to be careful about, assuming you can have any kind of ordering, because there's no guarantee that one of your containers won't accidentally crash at any point in time so like in the happy paths or you can try to assume some ordering. But your program should probably deal with the idea that one container like stop esters are not being said it like. The idea. Is you want an extra container around to make sure it likes a collects lacks last bit of logs from your program.

D

I would probably have some sort of socket or like a while loop and and it could receive the graceful shutdown instruction and then ignore it until it stops seeing the a lot. As you see, hustling yeah.

A

He he or she goes on to say container one flushes data to container two with container to having persistence. So, ideally, two stays up until one goes away, but you're, probably right. They should be separate pods, but I.

D

Mean I mean if it's actually a sidecar container, you could you could just yeah, you could receive the graceful shutdown signal. So kubernetes sends a different signal or, like the initial hey, by the way you should start getting shut down and the actual like nope you're ending now so there's a great school shutdown period and so yeah it could. It could receive that great tool.

D

Shutdown signal wait till do like some sort of health check on the other container and wait until that health check ends and make sure so that it can make sure it's exported the last bit of the data or whatever.

A

So how new be welcome, asked what I think oxygen is asking talking about. Our first question here is and I have this question myself. Is that sometimes you want to direct? You want direct external to your pods, an example would be no sequel, DB stateful set where you have an external app that needs to connect to all the pods separately. I've, never found a good way to do that, and then.

D

A

D

Could probably use a service one of the external services or something with a service type load, balancer yeah.

B

You need to use service type load, balancer and then in your label, selector for the service target. Like each thing you spin up will have the actual Lake staple set name in it and with this or goal in there. So you can target each one individually. If you want. The name is headless service.

E

B

E

The documentation so search for catalyst service, that's.

B

For the internal cluster IP part, the.

D

Other the other thing you can do is you. You could basically write your own ingress controller. That would do that. It's not like or ingress like thing right like it's, not it's actually a fairly common pattern and I'm surprised that I haven't seen a more general solution for it that just kind of deals with this particular use case because I, a number of people asked me about it. But, like writing, a very simple controller that you know: configures nginx or something to pour through individual names. Ten points shouldn't wouldn't be too hard.

A

Okay and oxygen, let us know if that also answers your question. If you indeed had the same question as how newbie, which maybe I did you guys get the the reply there? Where he's using the Google ingress controller at 4:15,.

A

So, okay, just waiting for people to type to see if that answered the question for them.

A

Okay, moving on, if C says thanks a lot, pre stop hook sounds like it's the way to go: Mario posted in the the lifecycle Docs, so the lifecycle box that you posted with container hooks.

D

Hooks yeah, okay, awesome.

D

Do we miss a question from hell newbie with persistent volumes.

A

B

I missed a question before.

A

That's okay, we're gonna! Do this one! Next! Sorry, if I miss your question at any time, please repeat it or if you lack questioned in big, bold or capital letters, so I could see it. So we'll do serious. I hope, I got your name right question and then we'll come back to how new be here. um Surya asks. Can we use both security context, deny and pod security policy and admission controllers, or do we need to only use either one of them.

C

I don't know 100%, but I think the API allows for both I haven't done it, though so.

A

Anyone have any experience in this one.

A

Jeff, could you dig and look just see if the API supports it or not? I.

E

Want to set up a cluster and.

A

Test it now, but theoretically find out if it works, quick hack,.

C

Slash conjure up, go.

A

Yes, so, as you.

E

Wait 14 how.

A

Movie comes back with thanks for the answer, but I'm not sure how using a headless service addresses the need for reaching individual pods inside my kubernetes cluster. But maybe you can post an example. So.

B

Hitting the individual pods inside your cluster from outsider inside, because you can't hit the individual ones the inside, because they will all be prefixed with the stateful set name.

B

From outside yeah, what you wind up having to do is you have to create a service for each instance of those of service type load balancer and then basically modify your selector to for each one to target them individually. If you look at the labels on pas, that's spun up via stateful set you'll see that they have their full stateful set name as a label in there. So you can target them all individually. That's as a kubernetes I haven't see the one not I.

D

Don't want that ten, if you're going from before then that labels, not there okay, so it's a little bit harder but yeah that that would be the way to do it without writing anything. You know.

B

The other thing you can do too is, if you do want to try like exposing as like a node port service, you can set the external policy at the external traffic policy so that way, only the nodes that are running in this is the Patil answer, but it'll only work under very specific use cases and honestly is probably a bad suggestion, as I think about it.

B

A

I tried no port, but that generates a lot of problems on the client side. Yeah.

B

So I still think you got to use like service type load, balancer and target each one individually.

A

And it looks like the panel is nodding: yeah, that's probably the the most basic solution for that. Okay, all right! Well, he's typing their oxygen says: yes, that answers it. I'll just connect to the service or build my own controller. That's George! Yes,.

C

A

C

Yes, you can have both security context deny and pod security policy set at the same time assembly go ahead. Oh yeah, yeah! Just can you repeat the question?

C

Sorry so I'll repeat the question: can we use both a security context, deny and a quad security policy in admission controllers, or do you need to use only either one of them close the question and you can use both because the security context and I will deny pods that try to set certain security context fields, whereas pod security policy actually checks to see if on creation or modification of the pod it is allowed to based on what security context is requesting, so one is: is it modifying a security context?

C

One is: is it requesting it or not? Now, that's not me splitting up a cluster in testing, but just based on what I'm reading you can do both sure so.

A

A

Let me just did that answer your question.

A

Okay, so while we do that- let's- let's summarize what's happening here in chat here for those of you that are on video, so how many comes back with like so four three pods I need three different services and bob says yes want to target each pod individually and they'll get their own IP externally. Can you walk us through this real, quick Bob.

B

Sorry I was luenell of typing and spitting up an instance really close.

A

Okay, all right, no.

B

I just need I, don't remember the label name off the top of my head today, I'm looking that up right now, it's on the is on the staple setbacks. If.

D

It was and Steve thought kubernetes dot, io /, pod name there. We go it's a little bit of a mouthful to say out loud, but I swear. It makes sense when you read it yeah. Let's.

A

See audience let's see if we get each of the panelists spinning up a cluster, that's.

D

Hot it's under pod identity. The odd pod name label heading con documentation, so you create.

E

A service that has.

D

A selector that selects that particular label, which is you, know, kind of cumbersome you could have. You could have a script or a controller controller scripts that did it automatically.

A

Okay and Syria says they get it, but they're still confused. If you have follow-up questions feel free to to stick wacom in the channel. There I think that's I, don't think I've missed any questions. So I think that Ethan did okay.

C

A

C

Whole new be asked one earlier about persistent volumes. Did we answer that it was a no.

D

No, we didn't know, we.

A

C

Scroll up to 414 and I can answer that. One okay.

A

I got you okay. How mu B asks I have a persistent volume host, Pat, persistent vol claim and a pod running a DB when I go to the file system? I, don't see any files and I suspect that data is still there even after I delete everything, probably because of our taint policy. Does anyone know how I can clean it up? Is there a way to look at which files are in my PVC.

C

So the short answer is, if you're, using something like host path. The only way you can clean it up as a manual step, there's no like Auto cleaning files off of your host once the volume of the volume clean is gone, I am very confused. Why you wouldn't see the files on your host system if they are being passed through, though,.

D

Kind of prequels which, what what type of persistent volumes are you using? Can you post path? Oh, he does say hoes path. Yeah I had a good Jeff there. That's very weird wonder: something's not set up currently.

C

So if it is host path, the files should definitely be persisting, regardless of the pod going away or the claim going away or the volume itself going away. Actually it's just creating or attaching a folder to the pod in a very, very simple term.

C

So if, if that isn't happening, then there's a chance that the volume is actually not being created and adhered to, and it's just storing it within the container itself. Yeah.

A

He says thanks, that's exactly what's happening: I Alice, ty'chell, the fastest and I, see nothing, but when I spend new pods I see that they're getting stuff that should not be there.

D

Interesting I'd.

D

C

Love to see the ammo for the the volume and the claim.

A

All right, how many, if that's possible, you might have to I, don't know what people's policies are. Networking pacing around EML files, but oh, he could provide all that yeah. That's great, feel free to toss it in a gist. That's like just get up, calm or toss it in yeah.

D

You could throw it in and discuss like you like people.

A

Yeah yeah, let me put the thread. We have an office. We actually have an office hours thread for today. If you want to stick stuff in here. This is the related discussion for today's session, and this is what we'd be posting the notes from later, as well, so feel free to whack them in there and we'll we'll sort that out I get the thumbs up for somebody all right awesome. So it looks like he's. Let's give him some time to do that.

A

Courses so did I miss any more. Let me look I. Don't.

B

Think so, okay.

A

So let's go ahead and get people chance to ask questions. I see lots of furious typing. So if you have any other questions or you like some clarification on some of the answers that we're giving please let us know, but first, uh what have we got going around this week? Any any big things I saw that Code slush is happening, I, don't know, and 1.11 I saw something scroll by about a slush, I'm sorry code, thaw for 111 that Josh Burke has posted.

A

So computers are now pushing a pull request against masters. So I guess that means 1.11 is do really soonish I want to say next week, ish the.

B

Thing I'm super excited about 1.11 is the IP vs based for like in cluster service load, balancing, instead of just using IP tables yeah. So for.

D

For people who are like interested in like requesting features from SIG's now, it's also probably a good time to start talking to the sig you're interested in. If there's something that you would like to see or some feedback, you would like to give that.

E

D

Know 1.12 is about to open soon because of the code thought. So this is probably a good time to you know this is some of the things may have started their planning already or doing exciting sure.

A

So see, burners gonna ask the question that we actually talked about this morning, but it doesn't hurt to repeat it again see burner thanks for coming. He says: I have a question about using initializers on pods, how do I deploy a pod initializer without deadlocking and then a link there to their stack overflow question. We talked about this briefly this morning and there is an answer posted. What do we do? Def.

C

Should covered it be able to cover it again, yeah.

A

C

If your initializer is only relying on one pod, no matter what you're gonna have a deadlock, the best way to handle it is you actually create an admission controller webhook, and then you have an H a service hosted within kubernetes itself. So if a host or several hosts go down so long, as you know, the pod is on a host. That's up it'll still resolve the initializer so that make sense to the panel, because I figure, if it makes sense to them it should be alright. If you're going deep into initializes.

B

We've got several people typing now, so you might have.

A

More people are furiously typing furiously.

D

Wow, while we're waiting for people to type I, think he'll newbie also kind of ask the follow-up question there, which is: what is the file system? Where does the file system get mounted to during container creation, which is actually a very interesting question? I believe what happens? It's been a while since I looked at this particular two code path. Is that there's a special there's, a special volumes directory for each pod and container and I? Think things get like fine mounted into that special volumes directory?

D

It's a little bit of a funky set up, but if you look for like I, think it's gonna speak kubernetes volumes by default or something like that there should be. You should be able to like see that directory and all the volumes on there.

D

People can feel free to correct. You know it's been a while, since I've looked at that no you're, absolutely right and with using a provider, you know, IE port works, dry, yeast and other third-party provider for storage. They will sometimes storage in a directory that they've decided on. So maybe bar lid port works block right, so yeah, but you are definitely right. That's still the way things are done.

A

Okay and how movies posted there there yeah mail there I'm gonna, go ahead and link that up to the right thread and I'll work on reformatting it after.

A

So, let's, let's do that and let me give some time for the panel here to digest that ya know. So, let's, let's catch people up here who might just be joining the stream, so we are in the middle of figuring out why how newbies file show up when they should not win a pot is destroyed and he recreates a new one. Is that correct and while we're doing that, if you're just joining the stream, the way it works is we're answering? You live questions on hash office hours on the kubernetes slack.

A

If you ask a question and we talk about it on the air you're automatically entered into the t-shirt, drawing which will be having at the end of the show we can get a swanky kubernetes t-shirt. We have about a half hour left to go so the more questions. The better we're gonna give the panel here some time to digest a lota llamo that I that we just got.

C

So one thing that I noticed just glancing through, I don't actually see the amal for the claim. I see that there's what I call it a claim: template yeah.

B

That's gonna, that's part of the Stifel set.

A

So does so we're getting whoa we're getting a bunch of oh boy.

B

Yeah is there a way to collapse that post on second? Can you turn that into like a gesture code, and so we can sort of minimize it? Yes,.

A

B

Snippet please.

A

All right, dude is one of you want to taint. Oh go through the mo file. While we, the rest of us, continue answering questions and then we come back to you. Oh could.

B

You also like post a like describe it or output, the mo of the actually created stateful set, so he can see what it's actually doing.

B

Stick that into the thread problem.

A

D

A

So, okay, well, that's getting sorted!

A

A

Let's, let's move on there to events! These next question we all saw who so who's who's, doing the yeah mole digging that Baba Jeff could everyone else. You.

B

Guys, okay, so you're already played him in slack. Okay, excellent! All.

A

Right so Eve C, sorry, if I mispronounced that ask any best practice recommendations for shipping logs off from containers to elastic in this case, currently using a file beat demon sets for container logs on the nodes. But it's hard to know what log formats are going to come through. Ie multi lines get mangled.

C

E

Ahead, yeah I'm, usually using friendiy for these sort of operations and friendly, is actually pretty smart about that you will not get with using friendly multi-line parser will not get mixed logs out of there. It also helps, and in memory and cue on each instance that is, it also runs as a daemon set, but it has an in-memory and Q and NQ on its own, and it will not bombard your elasticsearch if, let's say one application goes: hey, hey we're and logs, like 100 megabytes of logs and stuff, like that.

E

Also our feature that I used and I find it very useful is to usefully indeed to extract some kind of metrics from your application logs and expose that to Prometheus. If you don't want to actually instrument your application, but they have three and fluent is like in my mind the best way yeah.

A

That's my bit or fluent D just to make sure there.

E

Are there are two things free and friend is the get the whole package has a lots of filters of their fluent bit is, is, is written in C and has a is, is less of a burden on each node? Okay, it uses less resources in terms of CPU memory whatever, but it has a multi-line filter, but it doesn't have all those filters that free and it comes with. Okay,.

A

And Devon bear like sad that he's using fluent very happily and a follow-up question. There's fluid affluent bay, auto detect a multi-line log messages, or do you have to configure it to understand a Java stack trace, for example, as a single message for.

E

For the Java logs there is a prebuilt filter that comes with plain D. You don't have to install it separately, so yeah detects that okay.

A

Awesome Oh anybody else, any other comments. All right are we coming. Are we coming back to.

A

I'm Ruby's question or should I move on we're.

C

Waiting some more info, okay,.

A

Gotcha, okay, so moving on here question one from oxygen ass: is there a way to see what Hatter slash ports, the Google ingress controller is getting when people connect.

D

Not that I know, but I would, from the client side, tried to interpret what headers are being sent from your application or livery you're using see what's going on there I don't know Google and controller if there is a way to actually see those okay.

A

And then they have a follow-up question if we want to talk to a third party API server- and we have several micro services talking to it, how do we find out what IP to give them, so they can whitelist it Google's default gateways, external IP, I guess and is a static, so I don't need them to update it later.

D

Yeah we need more provincial API server by third party. Does he mean its external, because that would be probably an external service or external name to get to that would be best I'm, not really exactly should always trying to do there so give it an example. That would be great yep.

B

And just okay, the IPS of the what its exiting out as its netting out is essentially.

D

B

Know IP e filtering.

E

D

I, honestly, don't know for.

E

D

E

Jk can reserve a global IP and you can use then that IP will not change. I did that from terraform a couple of times so for.

A

The varying output.

E

For the l7 load balancer, but not Ingres, not English, yeah.

B

E

There are Egret.

D

Solutions, I'm.

B

D

B

A

D

Let's wait for mation I will say there are Google folks who who work on this stuff, that hang around the various channels. So if you can find, if you can track them down, you can actually probably ask some of the people who work on this stuff to that place.

A

So let us let us know how that works out for you, though, because I do like to follow up. We can at least point you in the right direction. Suresh asks just a related question. How does initializer differ from an admission controller as I understand, with initializer we inject a mcpot and with a mission controller, is, do inject, employ proxy inside the pod. Most of these things, injecting a container inside the pod, so.

C

So I can give the general answer. The admission controller is just something that checks to see hey. Can this thing be scheduled based on these rules? Usually you build an initializer. If you want to then say, hey modify or update something on the fly once this thing gets scheduled or perform another action on the cluster. If this thing gets dented okay or updated.

A

And you know, their common looks like you all are generally not need.

D

It but you're not that the the big difference is that admission controllers are completely synchronous late. So you they're, basically saying on a particular API server event apply this filter and the filter I believe potentially like mutate stuff, but like apply this filter and once the filter completes pass it on to the rest of the filters and then eventually through, like the actual backend of the API server to the story.

E

D

Not whereas initializer puts like a marker on that says by the way something is going to want to do something to this later and then that thing can go and do a bunch of asynchronous work. It could make calls and stuff like that, without necessarily blocking up the API server from doing other work right.

C

The one thing is, it will block the thing from being scheduled still until all of the initializer chain completes yeah.

D

And it'll anything, that's not typically looking for, like pre initialized objects will will not see those objects right. So if you put the initializer of a pod or whatever scheduling.

E

D

It allows you to do much more anything work without necessarily having a performance impact on the API server yep.

A

Okay, Suresh I hope that answers your question thanks for coming now before we move on to the next one.

A

Devon Barrett's posted a hound chart to fluent bit Devon. If you could post that in the main channel outside of that thread, it would be handy for those of us that are listening thanks for that. Moving on Jolin welcome, says hiya. If he had a moment's, if few few moments for a quick question, would it be possible to get multi arch images specifically cute proxy and second part arm support has recently been unstable there. Any efforts to improve this so we had is that around is ed around we had a Don earlier today.

A

This is, you could have a complicated answer, because a lot of the core stuff is but a lot of the containers aren't you.

B

Might be better off hopping over into the arm 64 channel? Yes, so that.

A

I, don't think a lot of people know this channel exists, so we'll just mention it. That's. There is a channel just for armed people, hash arm 64, and you will find lots of armed people there, including people that are working at armed cloud providers and things like that you're, probably looking for Edie I hope he's listening. Let's see, I hope that answers your question.

A

Moving on any luck with the llamo snippet there or you y'all, still gathering information.

B

Still gathering information, okay,.

A

A

Looks like Joe Joe Thompson is adding some good information here about accident direct pod for the outside. Thanks for your help, moving onto end brand and I think we answered this one earlier, but we've hurt to repeat it: hi guys, sorry, but I- just joined late I asked this earlier, but did not see an answer. It's in the video which just posted afterwards. So we did answer this somewhere. I, don't think it was available at the time. Kubernetes performance question I understand that all API calls must go through the master node for processing.

A

Well, one could scale up a master vertically. Is there any advantage of scaling horizontally ie more masters? It is my understanding that even in a multi master cluster there's one master which handles the API calls and the others are just redundant. Spares is my understanding incorrect.

D

There is one controller vote, so master you should. We should actually clarify like mess.

D

Master is two different things like actually that we just commonly put together : 8, but there's controller managers, which are the actual control groups that do reconciling knows those are those are and like a one active and multiple ones that can fall back, and so they kind of have a least thing going on, with a periodically check to see if the one that's supposed to be doing stuff is actually doing it, not one of the other one kicks in, but for API servers you can actually run.

E

D

Api servers all talking to different @tv instances and they're. There they're all active at once, and they can all be used.

E

So you should use a load balancer in front of all the API servers and, though, depending on your load balancer to just load, one particular policy. But you should use a load balancer, not target directly a single API instance, and that's it.

A

And it looks like Joe Thompson also agrees with you as far as being encouraged to put a load balancer in front of your your masters, also something that we should you point out as the sig arch people will will Mountain we're kind of moving away from the term masters and being more specific when it comes to the part of the control plane that you're talking about so just something to remember.

A

They're moving on Surya has a question with a load of llamo, but we're I don't want to do too long, yeah mo questions in a row.

A

So let's, let's put that one on hold for now and we'll get back to it, we're still waiting on how newbies more information right, okay, so moving on, while that information is probably a Fitbit asks thanks for coming hi, everyone is there production-ready way to handle SSL certificates through nginx ingress I found out today that I need to use a out for TLB to support WebSockets, but now I can't use AWS, managed certificates, I.

D

Need to see more information later, 40 lb and WebSockets and certificates aren't mutually exclusive like so so. Websockets is actually supported by the enginetech scenario. Fingers controller I, don't really know how just managed is biggest work per se, but you know, as long as you have certificates and have as I still set up, WebSockets should be able to serve over HTTP, and you know open up successfully, so I'd like to see a little bit more information on that. If someone else on the panel wants to supplement that that'd be great.

A

Let's see it looks like a lot of people are typing thanks, those of you that are helping out how newbie, with this problem, there's.

C

um So far, just as a hell, newbie update, oh, never mind it might so. The folder that he was listing out in the chat is different from the folder that is posted in the discuss. Pv. Yes,.

B

There's a lot of yam while going back and forth. So it's now in a thread.

A

A

So it looks like we have like art, so are we are we using the disgust on it at all, or is it just a slack thread.

B

It started out as disgust then moved over to slack as questions we're going back and forth. Oh boy, okay, alright, we will probably delete.

A

B

So I'm gonna.

A

Go ahead and delete that so we don't get confused if that's a K hub newbie.

D

I'm waiting only after this question of maybe other questions by home, newbie you'll become a hell master.

A

Yes, if he's held newbie, are you home, fanboy number one Mario that.

D

Would be correct, yeah, okay,.

A

I'm going to close the topic on disgust, so we don't get confused, did.

C

We get think--but.

A

Yeah we were asking for more information on on that. Okay. So let's read: let's: let's go back here a little bit think bit as hi. Everyone is our production ready way to handle SSL certificates for nginx ingress I found out today that I need to use an l4 ELB to support WebSockets, but now I can't use AWS manage certificates even see what like sad, that they're using jet Stax cert manager to manage SSL certs for nginx ingress. It's not marketed as production ready, but we've not ran into any issues so far. So.

D

Yeah in this subject, area no production ready is, is very, very floppy. It's not there isn't anything. That's out there that is truly really production ready. These things are just trying to work with, let's encrypt and automate driving starts based on engine next ingress and new verses. So that was that was the one thing. Is that but also I'm a little interested as what he's trying to actually achieve in terms of from an end client into his cluster or what his application is doing?

D

It sounds like he's trying to use WebSockets and you shouldn't really be getting too many issues with WebSockets, although, if he's enable ws I don't know, I'll be EOB, orders with WebSockets I know, nginx ingress, which he does mentioned. It doesn't have support for was okay, so I'm gonna, probably.

D

C

Through this, alright.

A

So it looks like we have time for one, maybe two more so do we want to go back and try to answer serious admission controllers, PSP, there's there's.

D

Also, a later question: vise area, yeah.

A

Very small one: let's do the small one first here she asked: can we convert single master Kate's cluster to an H, a cluster.

E

If we're talking about cube a DM I think there is a way to do that. You create a separate etcd cluster and you expand using it on every master node. Then you point to that: etcd back-end cluster. So it's a way. It's not automated. You cannot run just upgrade to H a but yeah.

A

E

A

Is this is one of those areas or Q? Batman is in development. um The people that you want to talk to are.

A

Alanna I just gotta get the Sikh cluster lifecycle information, so these people are currently the ones driving cube admin and they're, always looking for feedback.

A

The H a stuff has been very, it's been getting a lot of work lately and it's it's an area that is high churn, so you know without making it sound like I'm, just throwing you over to them. They are literally looking at solving these use. Cases like right now so now would be a good time for you to get involved and talk to them. They're always looking for people who are actually using cube admin in in the real world and and are looking for feedback. So please please contact them because they will help you out.

A

There looks like we're still working with how newbie to grab is diamo. Thank you can see. That's Joe Thompson for your help.

A

Let's see, we maybe have time for one more I think what we're probably gonna do here is stick her out after and see if we can help fix how new be and, if not, maybe either move the question over to Stack, Overflow or somewhere or discuss and get more eyeballs on it. So anyone have one final question that they want to ask before Jeff figures out the raffle for the t-shirts be highly coveted, go brownies, t-shirt.

A

So one thing we should probably do in the future is is stick geist github.com in the topic or something so that it's obvious to people where to stick or to sticky ammo. So, okay, so it looks like some people are typing, we'll see. If we have one question: I'll just go ahead and outro us here thanks all the following companies for supporting the community by allowing them to participate. So that's Giants worm hefty o liquid web Red Hat we've works, the University of Michigan packet dotnet and the CNC F. So far in the past.

A

In the past few months, as we said, these are all volunteers. So, if you want to be on this panel want to help us out, you have a expertise, we'd love to have you the more people we can get, the better. We can rotate people out. Bob can actually rest for once. Bob and Jeff can actually like not have to come every single time, and with that I highly encourage you to stick around and hash office hours will be so in July.

A

I'll be gone, cuz I'll, be at Google next, so I think Jeffrey will be doing the stream and it looks like we have a winner which is suresh, Vishnoi, sorry, I, probably totally butchered that, but I will ping yuan slack. Congratulations, you've won a kubernetes t-shirt and thanks for everybody, who's participating, and we will see everyone on the third Wednesday of next month, which is I.

C

Think it was July, 8th, tene great and with that. Thank you very much. Everyone.