Kubernetes Office Hours, 20 Feb 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Office Hours 20190220

Description

Third Wednesday of every month we do an hourly livestream where we try to answer as many user questions as we can!

Post your question to this thread for us to check it out: https://discuss.kubernetes.io/t/kubernetes-office-hours-for-20-february/4711

More info here: https://github.com/kubernetes/community/blob/master/events/office-hours.md

A

All right there we go everybody we're live for our monthly kubernetes office hours, the live stream, where the third Wednesday of every month we get on the internet and answer as many of your user questions as possible. All right, those of you that are in the channel, if you can, let us know how the stream sounds. That would really help us out.

A

We are office hours in the kubernetes slack and we'll be popping over to hash kubernetes users every once in a while to check out questions, but we do have slight change of plans this month. The slight invite the slack inviter thing is currently offline, so you can't just join our slack and ask questions so what we do in the description for this live stream below. We have a link to a discuss thread where you can just ask your questions there and we'll be looking at that thread to answer it.

A

So if you don't have access to the kubernetes slack, go ahead and post your question there and we'll be monitoring those are you the participate? When you ask your question at the very end, we will run a raffle and give away a kubernetes t-shirt that raffle is guaranteed to be random. Ok, I'll, be your host Jorge Castro and we'll begin with our panel before we begin I'd like to thank the companies that let their engineers volunteer for this.

A

So a big shout out to Giants warm stock X packet, push it push or calm Red Hat, Samsung SDS, we've works, VMware, zing Huawei and the University of Michigan and special thanks to Google for sponsoring the t-shirt. Giveaway ok panel introduce yourselves real, quick.

B

Well, I guess: I'll start. My name is Bob Cohen I'm, a research cloud administrator for the University of Michigan sort of the stuff that we dive into is very much kubernetes on Prem. We also deal with things like Federation research workloads, a batch job. So that's more Jeff specialty. He should be right below me in the zoom window right there, wherever, wherever he's in the zoom window mm-hmm, they sell pests off to him.

C

My name is Jeffrey CK I also work for the University of Michigan.

C

My title is senior research database administrator which doesn't really capture what I do, but I mainly do things with running HPC and research, workloads on kubernetes and I'm, also a contributor in cig, UI and kind of cig scheduling, but mainly for HPC and batch type workloads.

C

Someone else please go.

A

Joel, you can go hi.

D

I'm Joel speed, I work for pusher we're a tools company based in London, my I'm, a cloud engineer: I work, my new icky vanetti he's been doing lots of stuff to do with custom controllers and extending committees, admission controllers and stuff. Lately, I've also done a lot of stuff with authentication and authorization in past I.

E

Got next Priya bossy, James forum, developer advocate and product owner, mainly I've, been working in Austin security a bit and also in custom controllers and custom resources, also been looking into admission control lately and in general deployments on a RBS, edger and on-prem.

F

Million for me, works worked on various different things in the past, including to Bodmin in the early days, and some networking stuff I'm, currently mostly working on ich es control, which is a CLI for break it up. Ich es clusters very easily I've also done runs around like developer tools like Scott Holden are the tech tools that help you build containers lawfully in push them to wherever universe, they're not deployed so yeah kind of done a whole range of different things, and these.

A

Are resident gene ops person as well yeah and I'm Jorge Castro, be your host I was a hefty au alum and now I'm at VMware. Alright, so we're gonna get started here. Let me just give you all the rundown how this is gonna work, basically post your question into the channel. This is a judgement free zone, so everyone had to start somewhere. So it's a simple question: that's fine! We'll do our best to answer your questions, but the panel also doesn't have access to your cluster. So live good luck.

A

Debugging is kind of off topic like we don't really know your network, but we'll do our best to get you moving in the right direction. A panelist you're encouraged to expand on your answers with your experiences and pro tips and all that kind of good stuff audience. You can help us out by pacing URLs to the official Doc's blogs or any related information feel free to whack it into the channel, and we will address it post your questions to the discuss thread which I posted earlier in the channel.

A

Here you can post them to slack channel, but there are a lot of you out there who don't have access to the slack right now, while the advisor is down so there's the URL for that, if not, it is in the youtube description for this live stream, so we'll be, will be checking that out.

A

All these sessions are recorded and available on YouTube they'll be available about an hour after this livestream. um If you want to sit in another's panel, we always could use more people to help get a little rotation going. It's like the same, just bunch of white dudes all the time, so we definitely want to get more diverse set of people, so it would be nice for some of these people to not have to come every single month. So if you want to pay it forward, we really love it.

A

We could set up a rotation for you. The commitment is one hour a month and that's that and we're always looking for marketing help. If you could tweet this out, that would be really great and then, at the end, we'll hold out a rally or a raffle for a t-shirt.

A

So is that with that, is everybody ready Oh before we start I do want to mention one more thing. There was a CVE for runs, see this month. That's posted there. Everyone who's using kubernetes and production, does need to check that out and that we did have over the past six months. We have had a few of them. So please do your due diligence as an operator and check out the Seavey's they're, always posted on the blog and on the announce section and discuss kubernetes io or the kubernetes dev.

A

So please check that out and with that, what's our where's. Our first question am I just reading from the top here, okay thumbs up so I will lack I hope, I pronounced that right. Welcome, says: we are working on refactoring of service catalog migration from API server to CR DS. Current implementation uses field selector, which is not supported on CR DS.

A

We have two possible solutions, replace it with a proper indexer for a particular Informer, and then they link to their POC or use the label selector and decorate resources with additional labels its redundant, because you have this information under spec and also in labels, but example prowl is using this approach is this solution. We have additional problem with keeping consistency between labels and fields in the spec. Can you provide some guidance on which solution is better from the performance point of view, long-term support and kubernetes, etc?.

E

Reusing labels for the same thing, there will be at some point, I think there is work on on making spec fields selectable too, but that will take a few versions of communities what you could do and what I'm looking currently into in is Auto fitting the labels based on the spec using open policy agent, for example. It's very easy to write some reco that just takes the spec and duplicates it's there, and then you don't have any issues with the syncing, because it's done by by the policy that you write.

D

Personally, I would go the indexer approach that you've been to in the PRC link. It looks pretty sensible to me it's probably the easiest way to filter and personally I'm not fan at the duplicating of the spec into labels, but it's everyone's choices: how they approach it. I.

A

D

A

Working their POC URL into the into the chat now and.

E

It's it is ugly like the the duplication and labels I kind of hate. It I just hope that you don't need it at some point anymore. We can just get rid of it without implementing some stuff. Do.

B

You know, like the PR issue, that's being discussed, I'm gonna, look at my head.

A

Does anybody else on the panel have an opinion here, I.

C

Would have gone with duplicating the labels just because that's what a lot of other different projects are doing. I, don't like it, but it's it's a pattern.

A

You seem adequately whelmed at the pattern, all right any other comments on this one. If you find the PR we can, we can whack it in the chat there for reference all right. Moving on Alex welcome Alex says: is there a way to set up a job template where I loop over an array to set the args for a command.

B

Do you want to call this one Jeff yeah.

C

I, don't actually think so I said I think like it would have to be an entry point like some sort of entry point script.

C

Please someone correct me if I'm wrong.

B

No I think you're correct there right now and it sort of assumes, like you, can't pass her like individual variables to each item in the you know in the job. Well, I believe that's a being worked on or like I forgot. There was like an index or something people wanted to pass out of that information could be passed in there.

C

Yeah so I know there was work being done with like batch style computing and being able to pass what the index of that job running was, but it wasn't like being able to pass individual arguments on top of that. So you couldn't do like Monte Carlo, just based on I'd, like an array of values within the job template yeah.

B

No, this is specific, just passing like the the index of the pot itself. So if you have like a parallel of four they'd have like you know, zero one, two and three, so what you could at least do something around that yeah.

E

Otherwise, you would maybe need to pull the arcs somewhere like out of a queue that just doesn't have the arctan anymore and then the next one just takes the next. If you don't need, ordered and.

C

I think that goes back to having some sort of an entry point script in the container that'll be aware of where it is in the job queue and then pull the right arrgh. So, unfortunately, not ok,.

A

And now would be a good time to mention is if we answer the question and there's maybe some questions that we have for, you feel free to also just post a follow-up or if you need some clarity or something or if we make an assumption. That is incorrect, feel free to correct correct us. We have no problem going back and and teasing stuff out. I think one time we spend an entire episode answering a question and it took us the entire episode going back to it.

A

Every once in a while, Matt welcome to office hours says working with adding to I scuzzy volumes to a container I have to my yam of config but I'm unsure if they are mirrored to the mop point.

B

uh He explained like what do you mean by mirrored to the mount point.

A

Matt, if you're listening, we'll just come back to you, there I just type, you can just type it in line. You don't have to you, don't have to do a reply or whatever and we'll come back to you. Simon gotcha, like hey Simon, good, to see you says, questions some pros and cons for running infrastructure as code with gke. What have you tested and what has worked good?

A

What has worked well, let's do that.

A

For structure and code in gke.

A

You look like he might have and then he muted again quickly.

F

Alright, this we read the question yeah.

A

Some pros and cons for running infrastructures, code of gke, and what have you tested knows what has worked well.

F

That seems like, like a super broad question. Reate so infrastructure is code, is generally a good idea. We kind of tend to call it get ups these days, you know when it comes to kubernetes work, whether to search right. So if you wanted to use something to bring up Dookie clusters without you know before you actually use like a DDoS operator to deploy things into kubernetes, you know your options. Kind of someone limit that you have terraform or or g-cloud COI or you could drive the API directly or you could.

F

You could probably also use the Google deployment manager forum correct. That was the name of it like forms kind of the most popular things within the community. Really, so that's that's the go-to solution for that to bring up the clusters that put themselves. But after that you could install simply like reworks flux which lets you point. It point the cluster at the repo and get all your deployments going from there so yeah. That's that! That's like a very high-level overview. I, don't know what people when they ask rows and columns yeah.

A

Yeah so he's I think.

F

This is sorry, yeah I can't see anything they generally. As an idea of of doing is. It is a question of doing infrastructure as code versus doing something else or is this? Are they asking for tool comparison yeah.

A

So he says we're specifically setting up the cluster not deploying to the cluster I've looked at Google deployment, manager, terraform and handsome all, but can't figure out, which way would be the best way put a the best way cell.

G

Phone is probably by far the most popular for that: okay.

F

Yeah so answer ball as far as giv guys go, I actually I mean I haven't met anybody using it. I, don't really know how well that gets used. I know that ansible has a bunch of features around the AWS. Api is built-in and I've not not heard anyone using those to successfully write.

F

It's kind of its kind of in between limiting ansible is much more powerful at provisioning machines and writing files and calling out commands not so much talking to api's and terraform is kind of a better choice and as far as Google deployment energy is concerned, it's kind of a good tool, but it's it's not so well documented from what I've seen. Oh, it's parallel to find much documentation or are geeky, and you kind of have to do a slightly weird thing with it, but it comes to actually do things with gkd and deployment manager.

F

It's it's something that it's kind of different from everything else. So Tara forums can like the most popular community shows or for that those yeah.

B

I'm, just back up what Leah said, I would opt for terraform and I agree with everything you said about ansible, it's much better shot like suited for like managing machines or managing what you install and manage on machines and not talking to api's.

A

Okay looks like matt has given us more information, I'm gonna go ahead and start it. Someone recommended I, start reposting the question, so you can read it on the actual stream is so Matt's working with adding I scuzzy volumes to a container add to my Emma config but I'm sure, if they're married to the mob point. So if I lose one of my storage appliances, I don't lose my data and the pod does not go down. That was the information so.

B

A

And then a ton of ton of llamó here ya.

B

Know that's sort of a those going to be handled by your storage appliance itself and it will be able to handle the failover. The I scuzzy target.

A

E

Is there? Is there.

A

Anything on the Internet mm.

E

Out like trying to double mount, it wouldn't wouldn't work, you would kind of mess with was mirroring there.

A

Okay, Matt that should answer your question and would also like to say +1 to terraform, but Matt goes back. Is there a way to mirror these or.

B

You'd pretty much have to handle that yourself in your application have to be aware just like this mount goes down and use this other one actually I, don't know how Cabrini's would actually handle that itself. I haven't I, haven't tried.

E

Yeah, it sounds like. Maybe you should rethink how dependent you are on that on that mount and just maybe retry with another one I have a list that you go through yeah.

A

And then they mentioned, is there wait a mirror with Alvey amor Z pools, no.

B

You're looking at like which storage provisioner you want to use and like I, don't think there isn't even a VM volume, provisioner.

A

Okay, people are still typing. If you have a follow up, Matt feel free to post it we'll move on to one bats. Welcome I, think I've. Seen you before a question: does anybody see bigger use of admission controllers to enforce company policies on Deb's? If so, what typical scenarios do you see everyone's all nodding: okay, Opa Opa? What is that give me a like open.

B

Policy agent, I.

C

E

Posted it in this thread that I mentioned before, okay.

A

E

It's it's kind of developing as a standard I think the azure team had a nice admission controller based implementation. That is now upstream in the OPA repo mm-hmm.

A

E

Currently, actually, testing it out looks super nice.

E

It's basically an engine that implement where um implements your policies in rego, which is the language for defining policies.

E

It works pretty well, but you need to like pull in data. If it's about about kind of excess and and company policy and processes, you might need to have some data based on which you decide things, and there is ways to import data into the into the policies. Well,.

A

Thanks for your question, wombat and that one actually came from the forum, so those of you that are just listening in on the livestream slack invites are not working. So we've put a discus thread in the description for this live stream, so feel free to just ask your question in there or asking them a channel and we'll go ahead and queue it up and warm around. Other questions, we'll pick random ones from kubernetes users or dis has.

B

This question actually came up regarding, like Oh Oh peon, more granular stuff on discuss, I posted an example from the roster.

A

Awesome and then what we'll do is we'll go back, we'll, go back and post those answers there and the thread. So we have one nice. Let's just use the thread as our notes. What do you guys? What do y'all think about that all right anything else on OPA before we move on all right, sam c hope.

A

I got that one right asked question: what an office hours is it possible to access the evicted pod resources, one of my friends, installs, Jenkins and kubernetes, but after one day that Jenkins pod gets evicted and would like to get the secrets back and Joel I believe you has to follow up on what kind of secrets they were talking about?

A

D

It looks like what they've done is install Jenkins and then manually configured it so I'm, guessing all their secrets, end up in a nice little config files on the disk and then, as soon as the pod goes away, it's gone once the one supposed evicted there's no way to get it back, that you can't get the volume over, because that would be garbage collected. So I was just suggesting in there I system volume claim.

D

If you want to configure it by the UI and add the config stored in that or just configure it using Fibonacci secrets and conflict maps. It's probably the better way to go.

A

All right any comments on this one I.

C

Brought notes in our hat can be that more or less mirrored what Joel said so excellent.

A

Excellent, all right, yone underscore dl welcome to office hours as hey thanks for your doing no problem. Is there a better way to use project in multiple environments than using home and it's variables I mean, for example, with config map and environment variables. It's easier to have just one values that llamo file and make sure everything in the project is based on it. Then, using a config map deployment as environments FYI have 10 projects and I can use a single big values. Yeah more for environments with helm, to manage all my projects.

A

I'll go ahead and reap ace. The text here on the stream, so people can see it Rico will get to your question here in a minute and Simon as well we're just building up a queue here, so please keep them coming. Don't wait for us just keep asking asking your questions. Alright, this.

B

One we're starting to see more usage of like customize, where you can sort of have like a yellow file with all these specific things that you need to template in for that explicit environment I'm. Personally, not a big fan of helm but C's bio. A good chunk of the community mmhmm.

D

E

I think both both can work depending on. How are you used to to work I? Think having a big yeah Yama file is customized or a big values you know is, is not a not a taboo and it's it's actually ok to keep all your conflict and in kind of a config file and then applying it to all the charts. This way, it is you keeping it central, and sometimes you might need very use that overlap between charts, there's similar to how how we would do it.

C

A mirror 100%, the only thing I'll add, is in our use cases. We keep all of our secrets in our CI pipelines, so they're not like in the values I am, will file they're just placeholders and they get filled in by the CI pipeline yeah.

E

We we have actually a conflict map and a secret and those gets kind of merged into the values before applied a key way of doing stuff. I.

A

Will try to get a home folks I'm! Just writing myself. A note here. Try to get mad for you know, maybe in the afternoon session more.

B

Michigan people more.

A

Michigan people we're not geographically diverse in this show and for all right, any other. Any other comments on this templating or the values that you know. My bed.

A

All right moving on those of you that are listening, please feel free to answer questions we're down to seven in the queue so keep them coming. All right, Simon asks I, read something about bonsai cloud, doing an off module where you can use the same off the same off to all clusters on all providers. Have you seen tested or worked with it.

E

Not sure what it does I guess it is either that poke authorizer or a no IDC implementation, not sure if there is any, is there any limitations on any provider as to what to set as IDC or picasa riser Authenticator, because otherwise I would just even Aussie would work.

B

Somewhere restrictive well like what you can set for a lake. Your options are starting the thing up, but yeah.

D

B

Guess only supports a GED yeah.

A

Okay, you know what I'm gonna see if that T gik folks will be interested in this.

B

Now he linked to it in the thread: yep I got it. I'm gonna tie.

A

B

Let's take a look at this thing: yeah.

A

So I'm gonna I'm gonna slap that in there and I'm gonna see if Jerry.

A

Maybe this will be the year we get a second camera, so we can have two things going on the stream. At the same time, I.

E

A

E

And it's, but it's I, think it's putting a proxy be front in front of the API and that's how it's: how its provider, agnostic.

D

Although you shouldn't really so it's using full of codecs on in the backend, which, in theory.

B

D

Community should be able to connect to its obviously depending on whether your provider allows you to set the IGC config or not, but that does then have lots of different upstream options for what you can use the author. It's something you could build yourself by the looks of it. Yes,.

A

Related question: have we have we figured out what's up with Jax lately? Is that still gonna be a thing? I know they were like with the Rihan idea, mergers and core OS and stuff? It's.

B

Been spun out a project, okay,.

D

Yeah so I actually am now a maintainer of Dex I.

B

D

Last month, so it's in its own dog, so Dex IDP, /, Dex and there's a team of about four of us at the moment doing the maintenance on project. Let's swim, so it will, it will survive. Awesome, I think.

E

A

Happy was abandoned for such.

E

D

A

E

A

E

The proxy right or to proxy yes,.

A

E

Good work there that.

A

Is awesome thing that scientist Dax is working quite good? Let's see if he has a follow-up, he does. How are they able to get akx aks to connect the Dex if it's not possible to change IDC in the API server.

E

Think what they do is they put an aussie in front of the api of aks, because.

D

C

E

The api, they don't have private private api's and then they they allow everything in in the api of aks but disallow or do the authorization in front of it. Ok,.

A

Then he mentions there's also vouch proxy. He says he likes that one as well looks like an option. Oh that used to be lasso. Okay and our John has a follow-on question. So what Dex I can use my ami credentials to login to the kubernetes dashboard.

D

Salsa food, so if you're using Dex or any ID see it's not so you can get the ID token, it's a jaunt jessamine token and you can paste that into the committed dashboard, which then has like a token field. When you see the login screen.

D

Alternatively, the oauth2 proxy, which I also maintained and has some support for doing this in communities, so you can put that in front of the dashboard. It will ensure that you're logged in and then put the token in a authorization header which the kunais dashboard then proxies through to the API server when you're making requests. So that way you can make sure people are logged in, and you also get a bunch of our back and stuff in the case that called so Simon Says. If that's correct,.

A

That seems really scary. Make me feel better yeah.

E

D

Scary, sorry, I.

E

Think the data see in front of the API thing yeah.

D

E

That's what you meant the proxy in front of the at the dashboard I, don't think! That's an issue! Okay! You should actually have a proxy in front of all your admin like if you have a premier soccer fan out, put yours proxy in front. Okay,.

A

All right people are furiously typing, and while we had them here, any any follow-up questions about Dex and on might as well just ask them and we'll get back to them about.

C

That but I will say, there's a effort going on this year to actually make more of a thing with dashboard to actually delegate all off to the API server. So if you log in essentially at proxies off from the dashboard to the API server, so anything, the API server supports you're able to do through dashboard instead of having some third-party plugin okay proxy in front of it and.

A

Joel is whacking URLs, therefore, for single sign-on from the dashboard experience shameless plug for himself hey if you're volunteering, your time you get to do the chain, let's plug one thing. I will do, though, I'm starting to notice I'm going to collect all the links that we're talking about today and kind of roll them up and post them in the discuss thread. So we have them and.

E

On the question of a DFS, I have used text or do text, at least against a DFS three I think, which has already see a support that does work. You just need to be take care of the right certificates, if, like some, some ID FS doesn't isn't set up with the right certificates, and then text does complain.

A

The same says: Dex doesn't support groups with AD FS.

E

Yeah best to use or IDC there I think direct ad doesn't have that support. But if you.

D

E

Fs that doesn't work.

D

It Dex doesn't have support for groups in a lot of the implementations. Recently, we at all there's a PR open to add Google Groups, for instance, although I would sell Knights PR welcome if you're interested and.

A

Then he's posting more information with that. Thanks to the person who PM me saying that the stream actually had inverse colors and our kubernetes symbol was not blue I apologize to you all I have fixed that I, don't know so by default, OBS had like reversed colors I, don't understand, okay, anything else to do with off, while we're here before we move on to rico's question just.

D

Because mentioned it he's saying that the IDC provided doesn't have group support, and that was because of the way that groups are handled by Google, so we're actually farming the Google stuff onto a separate connector and then Oh IDC should be able to support groups the way it should once Google's in its own separate connector.

A

Okay, I'll give Simon a chance there to type. If he's typing.

A

We're a little bit over halfway, then those of you that are just joining us thanks for joining us. Unfortunately, the slack invite system is down. So, if you're not already on the kubernetes slack, we have no way to get you in there. So if you check out the stream description below, there's a link to the discuss thread where you can just post your question and we will get to it in the queue so far, we have one question in the queue and waiting for more so feel free to do that.

A

In the meantime, it seems like we have a lot of questions about Dex, keep on asking them we'll get to Rico's questioning and then what are we loop back to that? So keep on asking your questions. Okay, Rico asks I've already asked a question in sig windows. Your answer would be really helpful. Does anyone know when calico BGP will be supported by Windows Server with kubernetes? The official documentation says calico bgp is not yet implemented for.

B

That I think you're gonna have to reach out to the Calico folks, yeah they're.

E

Pretty responsive.

B

On there slack so.

E

Also on Twitter, like just just ping Andy or someone on Twitter and answer you Andy.

A

So those does do they have they have a channel in slack right. Let me check yeah, they should have a chair, so this isn't a Windows question like a Windows person would have no right like if you're at Microsoft. You probably don't know this unless you're working, calico.

E

Specific yeah, yeah I'm, just figuring out I'm, not aware that there has been even an alpha for a calico and windows. We.

A

Definitely don't have a channel in the kubernetes slack CTMS CN CF slack baby. Let me look.

B

So what are you looking for? Just if the Calico folks have like a channel in slack? Oh, it's it's in like they have their own slack. Alright,.

B

It's slacked up project calico, org I will drop a link in on.

A

It right here a well know: Oh double post, but that's good to know and then someone's asking a follow-up about psyllium. Let's just get it out of the way here, but I could see the smile on Bob's face basura Kasmir, consider using psyllium for our clusters. We know the digital ocean is using it as well. You have some inside if it is used widely and maybe why yes or no I know they just had a major release and an action books, dough, yeah, I.

B

A

B

Out that information, while you talk about I, can't speak to how whether they're being used by again, definitely plus one it for being awesome. That's I also love anything everything about EBP, F and and that offload to the lower-level part of part of the Linux subsystem.

B

The nice thing is, they also have a whole slew of integrations, and you can actually do things like network policies based on DNS names and a whole slew of other fun stuff, mm-hm.

E

Yeah I guess most of us and a lot of people are looking into it running it in production. I haven't seen. Many do is doing because they had PvE BPF before that also in their own infrastructure. I think we are testing it. We need some overlay on some I'm sana infrastructures and their sodium doesn't come with an own overlay or these not with the GPS I. Think Susie is working on a BGP bird implementation for psyllium. That.

B

They do nice yeah, they do have a way of plugging into I, think it's or have a thing with cube, router for handling BGP and essentially it's just handling uh like some of the rules and some of the other off yeah.

E

I haven't used to broader, yet would be something to test yep.

B

Using cube router for BGP and cilium yeah.

B

E

I have personally.

B

Not not done that integration we're just using vanilla, psyllium is first off. We don't have to integrate with BGP, but did you move to 1.4 yet or no we're still in 1.3 just.

A

B

That's a problem for next week, Bob.

A

Alright I hope that answers your questions. I see people typing so keep on typing, keep on asking and we'll move back to it. Cuz pursue Eric, I, guess those a feature set looks promising. We'll definitely put it in a test series thanks for your input. Rico's also says: thanks for your help. You're welcome stick around so we can do the raffle here which Jeffrey is working on right. That's it all right moving on to the next set of questions here, so this covers all the questions that we've asked in the office hours channel.

A

We have a set of questions for from kubernetes users, but I'd like to give people who are listening to the live stream priority, if possible so feel free to if I miss your questions start to retype it. If you have a follow up, we can move back to talk about off, looks like Mac sky.

B

I think we yeah, we went over max guys, question sure.

A

Let's, where is it some looking at it here we go okay, so max guy posts. Let me just repost it here. Sorry I keep forgetting how to do this.

A

He says some: oh I, DC providers like gitlab, don't include groups in the JWT token, but they can be fetched from the Oh IDC user info endpoints any way to solve that other than webhook off.

E

We use all zero for that, because then you can just write some some small script that catches it and could it's put it in the new JWT for you decks, you could build it into it. I think we.

B

E

Sounded yeah Kiki, okay, I'm.

A

Gonna go ahead and start posting URLs here.

D

Yeah had a quick double check, the other to see provider index and it doesn't support this yet, but it wouldn't be too difficult to add, probably only like 20 lines code. So maybe that's something that should be done. There is some pls open around adding user info stuff at the moment index, but I, don't think they're quite related to this.

A

All right looks like the max guy is typing and then we'll move on to Simon's next question.

A

And notes are gonna be great for this I I think, even if you don't watch the show, just a list of URLs and stuff you talked about, could be really handy for people max guy says: I did get it working with oz0, but for open source solutions, we'll check out key cloak flanks, any other any other things other than key cloak. That are open source it you might want to toss them their first key close pretty much. Our only recommendation. That's.

B

The only one I can I can it's the one that we personally use. That's yeah I can like I've.

E

Looked for other ones, but currently there is not big, no bigot alternative I think decks would be closest and is very lightweight and easier to extend. I would.

B

A

E

Away from writing a bit, maybe just adding it to decks.

D

If you do feel like I did not feel free to add me in the PR or something and.

A

Already you're interested in doing that, alright, we'll give it a second there for the stream to catch up. If anybody else has those recommendations of Aniki cloak and then we will move on, we.

A

Can, let's see, let me just check the forum real, quick, just a would cost yeah.

A

All right, moving on to Simon see you give Simon a t-shirt nines getting maximum value out of this show, that's what we like to see anytime. They come back. It's always great. Thanks for coming, Simon Says I've been seeing an issue in Azure cubes for a plus core OS plus Canal+. This TO WEAR I'm, seeing an error on three different clusters. At the same time, following error, creating internal and then you read the error, which is still on the screen. Good.

A

The cluster has nothing in common, but it happens at the same time only solution, I found so far as to restart cute proxy or the whole cluster. The only thing I found is something like this in the following, and then he posts this gist here.

B

Rosalie I don't know where, in that stack, the problem would actually I.

E

Connect hurricanes and yeah.

A

Yep, so for sure, so if we can't figure this one out, I'm gonna ping, Craig and aks, maybe they have an engineer they can flop. On that.

B

A

G

B

Spray there it's not using oh no, they.

E

Could try another networking solution like that as you're seeing I have what is it etter, oui bien there.

B

E

There that there's your native adapter, he needs network policy and then calico policy. On top, that's what we run, for example, so you could I think that's also what is now running in aks, Kelly, Kulick, ethical policy and as your network, okay,.

A

So instead of flannel calico, it's a sure and calico yeah.

E

Or you could also add your and psyllium if you want it. Okay, no.

A

E

A

Yeah that we've met also has policy support.

A

B

Have to say that's one of the cool things about kubernetes is like you can technically have multiple CNI adapters and one run for the policy stuff and one to actually handle the networking yeah.

A

B

A

We have any general ideas of what could possibly be going on here.

B

Honestly, not really without like that, is it just yeah.

A

E

Mean the thing is that it's that it's at the same time soon it might be something that is as your infrastructure, specific and not implementation specific. So it might that's why I'm saying maybe it's DNS, because it seems it's. It's calling a private E&S, this psycho injector that a CIO or.

F

E

A Republicans yeah and then maybe.

F

E

A

F

He was like not or whatever and somehow cube proxy points the wrong place to begin with, and then you restore it it once. It is better place. Yeah.

E

Cube proxy does have some issues depending on, if you're, an IP tables or IP vs mode and your version of communities, if you're an IP vs, definitely update at least one twelve five or one thirteen one, because it deadlocks sometimes.

A

And then they mention I've changed to accelerated networking now, since last time maybe helps if it's vxy and related.

B

Is there anything in the logs on cube proxy, like you could turn the debugging on it and try and see yeah.

A

He's gonna check the version in the meantime feel free to ask more questions and.

F

I mean one other thing: you could do my music, no, no you're good yeah. One other thing you could do is like before you restart cube proxy. Take a look at VIP tables rules, so you're gonna kind of check out, and you know whether you can manually reach that address and and see the you dr. Manley tried to reach a pod like the actual Easter service.

F

And see you gave the other night the father's responding take a look what's in I P tables, whether the address of the pod is what that virtual IP is pointing at or it's something else and maybe compare the AP tables rules before and after q proxy restarting you can use IP tables, of course, I PBS less clear way. So I don't know how you would travel to that site. Ebs, but I think default is still kill. The IP tables who know yeah.

E

And he says if the IP is reachable but DNS has issues, it seems.

F

The IP of this service, all the party he is typing.

B

B

Are you using coordinates for your deenis are using cube DNS.

B

E

Cortinas time odds they usually around five seconds. 30 seconds shouldn't be Dakota. Typical accordion has eschewed yeah I.

F

Would just isolate the problem and try to debug it outside these two contexts, which is generally see you know, networking isn't in the same state like create some services and some people haven't said, and you know Obama load test and seen see everything is working as looks for adding ISTE. Oh yeah yeah. You know.

A

Yep they're still.

F

Typing and then I'm going to ask him in signature to make me out.

E

And it does seem to be something as you're related, so maybe yeah, because.

A

If you still have the problem without is, do then you're kind of ruling out things right, especially.

E

If it's exactly at the same time in all clusters, it can't be kind of a race condition of a single component right.

F

I mean this is what really what it really takes when you, when you want to run, set up the cluster yourself instead of using the service.

A

Alright he's still typing, we have about 11 minutes left, so we're gonna start to wrap it up. So you have questions in your joining us. Look at the link in the description for the live stream and feel free to ask your question I'm, looking at them now or ask in the hash office hours on slack those of you that have access to the kubernetes slacks and let me just check users one more time while Simon keeps adding more information.

A

Alright, so while this is doing on, why don't we run the I'll? Give it another 15 seconds for any questions while Simon keeps typing and then we'll run the raffle? No quick Jeff you could just let me know, actually posted the channel actually just announce who it is, and that will we'll figure it out from there. So we can clap with them.

A

So Cakery services, three different clusters and three different be nets, but with the same ability, zones and regions and that looks like Bob's responding there.

A

Yeah Bob brings up a point eliminating the components one by one.

E

Yeah and maybe take it up as Escher supports there, they usually very effective and help.

A

Out all right, so the winner of our raffle, is hold on I'm scrolling up to see if there here is Alex, which asks what question that Alex asked.

B

He was the first one, the very.

A

First, one was.

B

First or second, one second.

C

One about the job template the.

A

Job template Alex, are you on you have to you have to be on the claim, your prize'? He.

B

Posted it the other day it was before, like office hours started. Okay,.

A

Alright, we I'll I'll PM him and, if not, then we'll give away an extra shirt next week. How does that sound for everybody? Alright, last call for questions everybody. Well, while we're waiting for questions here.

A

Let me just do a quick shout out to everybody.

A

Is there anything else the panel would like to talk about.

A

You look tired. What time is it over there?

A

All you're needed.

F

It's not very late in the afternoon. That's it's only like you know, 10 to 3. It's just yeah.

A

It's it's February, London kind of look to your face. It is also cold here all right, while we wrap it up again, I like to thank the following companies for supporting the community with these developer volunteers, Giants, warm stock, X, packets, pusher, calm, Red, Hat, Samsung SDS, we've works, VMware, Xing Huawei and the University of Michigan and special thanks to people always for sponsoring the t-shirt giveaway. We have one more question from long that says this example is not working and it's the example. The rewrite example on kubernetes not github that IO. What is this?

A

E

It may mean ping Manuela who's, the maintainer, okay, a.

B

Lot of times, the the examples will honestly fall out of date with how rapidly things move.

E

Especially because I think I do remember a time there, all the examples were moved twice between repos and some of them are just way way way too old. Do.

A

You have there yeah.

E

I'll check the.

A

Github handle for eyelet outlets, or, ideally, a patient file, an issue that isn't just bringing a developer directly.

D

I'm just gonna say, make sure you're on the latest version because they changed the whole rewrite thing in the latest version that was released a couple of weeks ago and that documentation is up-to-date with the latest version. I was basically watching this a couple of weeks ago. So, if you're, two weeks off, that's.

A

Why it's broken and.

E

I think man well in our Channel and ok.

A

Awesome great all right, Alex I will follow up with you for your t-shirt, any last questions going once going twice. We do have another one of these coming up in six ish hours or so see the topic in office hours and watch this channel on YouTube. We'll go live again with that Thank You panel. Any last minute things are we all waving, let's just always all right. So thanks everybody we'll be back in a few hours and thank you again for joining thanks. Everyone.