Red Hat OpenShift Ask an OpenShift Admin | Red Hat Livestreaming, 15 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Ask an OpenShift Admin (E71) | External-DNS and Certificate Management

Description

We’re jumping back into the stream to discuss a couple of features available that are intended to simplify the life of OpenShift Administrators and Application Developers. We’re going to be talking about the cert-manager tech preview capability that allows teams to manage certificates as a kubernetes resource as well as the external dns operator; a recently ga’d feature that makes kubernetes resources discoverable through DNS!

Want to learn more?
External DNS Operator - https://docs.openshift.com/container-platform/4.10/networking/external_dns_operator/nw-installing-external-dns-operator.html
Cert-Manager - https://docs.openshift.com/container-platform/4.10/security/cert_manager_operator/index.html

Try Red Hat OpenShift: https://red.ht/3uVxkya

Follow us:
Andrew Sullivan https://twitter.com/practicalandrew
Jonathan Rickard https://twitter.com/jrocktx1
OpenShift https://twitter.com/openshift

Subscribe to Red Hat's YouTube channel: https://www.youtube.com/redhat/?sub_confirmation=1

About Ask an OpenShift Admin:
Do you have questions? We want to answer them! Every Wednesday at 11am ET, hosts Andrew Sullivan and Jonathan Rickard sit down to answer your questions about Red Hat OpenShift.

About OpenShift:
Red Hat OpenShift is an open source container application platform based on the Kubernetes container orchestrator for enterprise application development and deployment.
OpenShift on redhat.com https://red.ht/3rYz5ZS
OpenShift on GitHub https://github.com/openshift/

#RedHat #OpenShift #Kubernetes #ContainerPlatform

A

A

Let's get into the big topic of open source, something that we.

B

A

Cultured, what is that process that a developer or, let's say.

B

As the kubernetes.

A

Ecosystem really.

C

Hello and welcome to this week's ask an open shift. Administrator office hours live stream here on red hat live streaming. I am andrew sullivan your host and I am joined as always by mr johnny ricard, hello, sir. What's.

A

C

How you doing how was pto it was, it was phenomenal. I I told uh my team the other day that this is the first time in probably 15 years, like back like way way way back um that I I just I completely disconnected.

C

I can't say that I was 99 disconnected. I was uh because I I kept slack just in case any of the folks who, who report to me needed to reach out to me or anything like that, but um I I turned off email on my phone. I didn't have google chat. I didn't like none of that yeah. It was.

A

Nice yeah, sometimes you need that man like uh it's hard to completely disconnect but yeah it's sometimes you just need to be able to say all right. You know what I'm gonna turn these notifications off and I'm just gonna sleep through the night for once.

C

Yeah yeah well, and it's um it's funny because I I you know, I'm one of those people who I try and have not inbox zero, like zero emails in my inbox. But if I have unread emails in my inbox to me, that's I I need. I have action items, so I I'm struggling right, I'm paying the price now, because uh I've got like 30 threads or something like that that are unread, which is probably 100, plus emails.

C

So yeah, oh christian's, here, yeah, hey uh welcome back there, uh mr dead to me! um No! uh No! No! I I'm super happy for anybody who doesn't know uh christian uh last week. No a week and a half ago left red hat. He joined code fresh as their lead, developer, evangelist so happy to have him join us uh happy for him in his new position. I understand he will still be joining us occasionally. um You know.

C

We've talked about him joining us here on the stream, I'm sure he'll be wanting to join the get-ups guide for the to the galaxy. um uh I hear you might be joining that stream, um occasionally as a guest, johnny.

A

Yeah, I'm gonna I'll, be on there tomorrow to uh I'll, be uh uh hillary's punching back. You know for my sre comment. The last time she was on so I'll be getting the payback.

C

You know it's uh no pressure when it's somebody else's stream right there exactly.

A

C

Just go up and- and you know spout off at the mouth and that's that I.

A

Just do what I do. You know, there's nothing yeah, no punishment at all yeah! I was looking at uh christian's post on uh twitter about it. Like you know his code fresh, uh you know, care package is dope. He got all kinds of cool swag, so I'm I was super jealous when I saw it like he's, got all kinds of good stuff. Now.

C

Come on, we we get fedoras right. No, no, wait! That's.

A

True, that's true: we do get a fedora, but he got like I mean it was like a whole thing like he got socks I mean he was all dressed up. He could have gone out completely killed fresh and it would have been nice. Hey.

C

Speaking of speaking of swag, uh I have a meeting this afternoon to review designs for streaming swag. uh So I I think you have a conflict. I don't know if you'll be able to join but uh yeah for anybody in the audience. uh We're we're so close, we're so close to having swag for the live streams. So uh yeah um keep keep an eye out for that. We'll uh we'll be talking about it. I'm sure it'll go up on the cool stuff store um once it's available, yeah for.

A

Sure that's gonna be awesome.

C

Yeah, uh do we have a slack channel? um No, not not really, um so there is technically a discord associated with red hat live streaming, uh but if I'm being completely honest, it's really not used. um You know, technically all of the chat from all of the live streams gets sent over there.

C

So if you ever wanted to look back at any of the chat or anything like that, it's there, but um really the best way to reach out to both johnny- and I is the the kubernetes slack if you just go to kubernetes.slack.com and uh if you don't have an account create one log in there there's an openshift user's channel, um I I'm always in there. I check it. um You know multiple times throughout the day you can.

C

You can always send me messages on there, so that that's a good way to reach out to not just johnny and I but kind of the broader openshift community and then the normal. You know social media, twitter, reddit, etc. Yeah. All of my usernames are the same, which is the same as twitch, which is practical, andrew all one word uh and then of course, email um you're. More than welcome to reach out to uh reach out to me via email, andrew.sullivan redhat.com, I usually include johnny on any responses which is just j-o-n-n-y at uh redhat.com.

C

He got a nice easy one.

A

I did, I did got hit patient zero got in early.

C

You've been here longer than me, you've been here what seven years, I think so.

A

Well, next month, it's gonna be six actually.

C

Well: okay: yeah! Congratulations! Thank you! Yeah yeah! I celebrate uh four years in a month or two so.

A

Nice yeah man, it flies by, like I everybody says that you know everybody says it's just you know time flies, but it's like it there's just so much going on and things change so much that like dude, it's so hard to just focus on things sometimes but like it does fly by and it's been a whirlwind.

C

Oh yeah and covid, and everything has kind of made it worse because it's just like th this is I won't. This is basically the highlight of my week like this is the stream is how I use to break up my week and am I either preparing for or recovering from the stream and but all the days start to look the same.

C

um Let's see yeah I'll I'll put my email address in the chat. Come all uh icq numbers christian. I haven't thought about icq in a solid 20 years,.

A

Oh, I see my friend kevin's on here. What's up kevin.

C

Yeah and uh christian didn't point out there that he's also on the kubernetes slack. That's that's now how he and I chat most of the time.

D

C

Why am I not why I cannot chat on twitch um so I'll post this onto youtube, so andrew.sullivan redhead.com, so kamal? I just posted that on to uh on there. Hopefully it'll get rebroadcast across, because my twitch is not.

D

It's just not working right now. Oh.

C

D

uh Let's see andrew.

C

Here we go posting links, anything that has a link onto youtube, doesn't get rebroadcast, so the redhat.com caused it to not get resent to all the other platforms. So posting it on twitch is what what does.

B

C

uh Dislike walled, gardens, yeah um yeah, I assume you're talking about icq there. uh I hope none.

C

B

C

Me all right so I'll, quit, rambling um and and we'll kind of move forward here.

C

Excuse me, while I cough uh so. This is one of the office hours series of live streams here on red hat tv or excuse me, red hat live streaming, uh which means that we're here for you for our audience any questions you have kind of about anything, openshift related, whether or not it's something relevant to today's topic, which is going to be we'll talk about that. Just a second external ds, insert manager uh you're welcome to ask that in chat.

C

We'll do our best to answer those questions we're here every wednesday at 11 a.m, eastern for anywhere from 60 to 90ish minutes, depending on the topic and all that other stuff. So you can always ask questions we'll do our best to answer them.

C

If we don't know the answer- which, uh let's be honest, happens uh a fair number, a fair amount of times uh johnny and I both have uh we we like to think we have all the right contacts, um or at least we know where to where to go and ask um so we'll reach out to engineering, to product management, to whoever we need to and do our best to get those answers uh and if we can't get them while we're on the stream here.

C

um So that's why johnny's really really good at finding those answers, um while we're on the stream. So if we can't get it all in the stream we'll follow up in a later stream and.

A

I cry really loud too so people usually answer the emails, yeah.

C

It's uh we've gotten really good at slack right and, unfortunately, our engineering teams are really good about in the slack info bar, like you know, here's how to directly ping the team and get our attention type of stuff. So uh craig, what does the disc column represent when running a creek control stats?

C

Is that the local ephemeral storage for the running container? uh That's a good question. I don't know, um I don't think I've ever looked at that. Have you done it either yeah I would yeah. I don't know. um I don't want to speculate because it'll probably be wrong.

A

Yeah, let me I'm going to see if I can get out there really quick.

C

um Let's see officially ucsc yep and christian uh christian and I discovered because we we primarily communicate through gifts and memes that uh unfortunately, kubernetes slack does not have giffy integration, all right uh top of mind topics. uh So for anybody who is new to the stream or uh maybe it's just been a few weeks because I feel like since uh may it's like mid-june and I think we've been off air three of the last six weeks due to conferences and pto and all kinds of other stuff.

C

So um in case you've forgotten top of mind. Topics are things that johnny and I have identified over the last uh week or two you know kind of since the last stream that we think might be important to you to our audience. uh So we try and bring it up. We try and make sure that you're aware of things that are going on um and please don't hesitate to ask questions we'll do our best to answer about those as well. uh Do you have a vsphere hour planned? Wasn't there some csi requirements?

C

um I will say: yes, um we are always talking with our vsphere peers, um so both robbie jerome, as well as dean lewis, have been on the stream before so we try and bring them back on periodically so that we can talk about various uh various vmware related things, although I will say that in this instance, the red hat vmware csi driver is a downstream of the vmware csi driver. So while the requirements are 99 the same, there will be some differences, um but that's actually a good idea. um I hope none.

C

We should probably bring um the the product management and the engineering folks for csi in general onto the stream, because there's been a lot of changes that are happening there, um but yeah in general. um Just as an aside, so I think the versions might be changing a little bit, but the way that I understood it to be is that right now with openshift 4.10, the vsphere, the red hat vsphere csr driver. So that's the one that's shipped by red hat.

C

Is it's not mandatory? So if you upgraded your cluster from 4.84.9 into 4.10, you can keep using the vmware csi driver the upstream one, but at some point we will have to that will become a mandatory. The red hat, vmware csi driver will become mandatory, it'll, be a seamless transition once you get over there. We'll also do things like update or excuse me migrate.

C

Any entry volumes into the red hat csi driver and the reason for that is quite simply because the entry driver is going away so before that entry driver gets removed from kubernetes, we'll have to make sure all of those things get get migrated over.

C

Can we have more than one service network per openshift cluster? uh I don't believe so. um I don't remember. uh Let's, let's try something here. Let me share my screen share screen. Let's go with this guy.

A

Yeah, I don't, I don't think so without something like uh like a mesh or something like that. You know what I mean yeah.

D

A

See hopefully, this.

C

D

C

To see if you could go a little bigger, I don't.

A

C

I can because it's oh yeah.

D

You're in the present- um let's do this. I know I gotta make it big unless so so, if we do an oc, explain.

B

uh Open shift install explain, install config dot.

B

D

And service citer, but we don't want server side. We want service network.

C

So currently, only one entry is supported, so no, you can only have a single subnet defined as the uh for the service network there or a single entry for the service network. So even though it's an array um there's only one entry allowed, so just just for reference there.

C

um So what I did was go into so uh all of the openshift and kubecuddle commands um have an explain, sub command or an explain option and from there we can kind of walk through the tree of things that exist in that object to get descriptions of them.

C

So I just did for openshift install and the install config the networking.service network to do that. But we could do the same thing with like an oc, explain. You know service and oh, of course,.

D

C

Let's see explain service, I actually have to be connected to a cluster for that to work, and you see it spits back here all the fields that are a part of that particular object. So I can do service dot, spec.

D

Oops, if I could type risk.spec.

C

Here and it'll tell me all of the fields that are available in here, so you can walk through. You can see all the valid values you can get a description, so this also works from within the console.

C

So you can go down to administration and then custom resource definitions and we'll search for service.

C

Am I missing something? Why is there not just a service? Oh because I'm looking at crds, that's. Why um and service is a core object. Anyways, we'll pick on one of these and in here, if I do like, I can't create a new one. Of course I had to pick one.

D

Build here uh instances.

C

Create build um so over here in the gui. We have the same sort of thing right, so kind metadata, so here's spec, so I can go through and select additional trusted ca, and I think that also works. I think there's a way up here. Yeah api explorer um I've got a service here and then I can go. Do the same thing create service.

C

I was just looking in the wrong place I had to go back to when ali was on and ali was showing me how to do this so yeah, so here's spec and it'll it'll walk through. I don't know why that's not working, but um you can walk through and you can see all the information about the object and it's just a helpful little thing to help you uh or to go through, and you know see all the uh fields all right. um In fact, I ran free ipa on openshift christian.

C

It's a well-known fact that, like you, are just a uh a glutton for punishment, because you're also the one who spent, I think, two days figuring out how to deploy olm onto microshift, um which pro tip um technically possible, not at all friendly. So for anybody who thinks that you can use microshift to get a super small like full, full openshift deployment that that doesn't work, remember openshift is just there to provide the openshift api, but not necessarily all of the features of openshift.

C

All right um top of mind topics. Sorry, I got distracted by questions, which is a good thing. uh So a couple of things here um so first I want to talk about kcp, um so oops. Let's take this and go here in this blog post, so we have an introduction to kcp.

C

So this is something that we've started talking about in bits and pieces. If you have kept an eye on our what's next presentations.

C

What's next is started to talk about this as well, and I think I briefly mentioned this back when 4.10 was first announced or maybe when we did the what's next presentation, but kcp is a way that we can provide a so it's kind of two things is the way that I'll describe it so one we can abstract access to clusters for users so effectively what what it does is. It gives a a kcp user.

C

What looks like a cluster right, a full kubernetes api endpoint that they have full admin privileges to and then on the back end. So what's backing that api endpoint is one or more clusters that are distributed across anywhere. You need them to be so it has a couple of really interesting ramifications, so one for you know app teams, application, admins, etc. Who like hey, I need to deploy my own crds or whatnot.

C

They can do that and two it's not tied it's it's decoupled from any one, specific cluster, so their resource, and now I'm struggling to remember what they call the resources in here workspace.

C

So their workspace is theirs and the workload doesn't know that it might be running across like three different clusters and if one cluster fails, then it will just like you have in a a single cluster. Then it will do its thing to reschedule workload across any surviving clusters. um So it's a really interesting concept. It's still uh very much in development and all that. So I, if that sounds interesting to you. I would definitely encourage you to check it out.

C

It's something that I'm looking forward to testing out. You know someday when I have time as a way for us to again abstract away a lot of the details of the underlying infrastructure from an application perspective. They don't really care what infrastructure they're running on. So long as they can access the resources, so this just takes that one step further, so that as administrators we can, you know not stop worrying about a specific cluster and instead worry about our empire of clusters.

C

Very nice yeah. Hopefully that made sense. um You wanted to talk about open source summit. uh Next week's going to be a little special for you right, yeah.

A

This is my first uh conference going it'll be the first conference I go to as a red hatter, so um open source summit is in austin next week, uh 21st through the 24th. um So if you're in the austin area and you're going to the summit, hopefully I'll catch, you there so stop by the red hat booth, I should be around hopefully lurking make it all kind of I'm loud. So you'll you'll probably hear me like 50 feet away. So.

C

You did you say: you're.

A

Going to be streaming from.

C

There next week, yeah.

A

I'm going to try and stream from there yeah, so uh yeah I'll, go, try and find a quiet place, bathroom stall, something like that and.

C

Yeah that uh when I used to do a podcast we used to do it from like. We would wait until the expo floor would close, and then we would record a podcast and usually in the background you could hear all the beeping and all the noise from you know the folks all the behind the scenes.

C

Folks doing their thing so yeah, I don't know, I don't know if it's still a thing it used to be remember like organizations like companies would have these massive booths and there was a trend where they would put a conference room like on the second story of their booth, and I always I never cared for that because it was. It was still super loud, so you basically had to shout at the people that you were talking to, but nice.

A

Yeah yeah, that's I mean I'm loud anyways so like I actually try and like tone my volume weight down, because I I naturally yell, I'm probably deaf.

C

Oh, I I know I am way too much time spent in data centers, my god.

A

Or, or at sea on a ship you know just, I think it's finally taking its toll so.

C

We're children, you know yes,.

B

C

Afraid to scream right near here, yeah, uh let's see I'm just digging through chat here, um running database and openshift, uh so sj wang. So, yes, you can run databases in openshift. um There there's multiple schools of thought and opinion on that so running a traditional database. You know so something like mysql or my sql, depending on. Remember, I'm an equal opportunity, offender with mispronouncing things so I'll mispronounce it both ways. So that way, everybody is unhappy with me: uh anyways oracle, microsoft, sql server!

C

You know all of those things absolutely you can host them in there just be cognizant of it's a container platform and what that impact is to your database.

C

So I'm most familiar with things like mysql, and you know, if you have you know, sharding or log shipping or any of those types of things in place. You just want to again be cognizant that it's a container platform and containers aren't migrated, they're, terminated and restarted.

C

So how that might impact your application? That's using that database as well as any sort of data integrity. Things.

A

And there's some cloud native ones too right like a cockpit cb and uh you know: crunchy's got their stuff with posters.

C

Yep and that's exactly what I was gonna say um so yeah you you, um you already knew what I was going to say. Johnny, you read my mind of be on the lookout for yeah. There's lots of cloud native databases that are now popping up. Cockroach is definitely a big name in that space. Mongodb is another big name in that space, crunchydb, so on and so forth. So I it really just depends on your application. What your needs are and to be clear. Those are not nosql databases right, they are a traditional.

C

You know, or I can't say traditional. They are in rdbms they're, just a cloud native rdbms that takes all of the necessary actions. It abstracts them away into the product so that it protects itself. It scales correctly. All of those things for the user, so yeah, the nosql databases, object, databases all that other stuff they've existed for a long time and uh and they work great inside of containerized platforms too. So.

C

It's a subnet of a route used when a route created same as the service subnet, so routes point at services. So you you have to create a service first and then that service will have an ip address on the service subnet and then, when you create a route that internally points to the service in order to identify which pods it's going to direct traffic to so it doesn't, the route itself doesn't have an ip it's exposed through the ingress controller, and then the endgress controller will have a load balancer and the load.

C

Balancer is what has an ip associated with it. So ask an openshift emperor that sounds a little uh authoritarian for my tastes: um a live stream about andrea and avi ingress controller integration with ocp, uh yes gamal, um so I think it was dean.

C

Dean was on last time. We touched on that subject, so um we'll see if we can dig up that link um here, while we're on the stream and uh post that and yeah dean touched on that subject, we should definitely bring it up again and dig into some details.

C

uh Is there any other way to manage dns without operators? um Yes, probably it depends on how kubernetes native you want to be right.

C

So if we're talking external dns and I'll touch more on this in a moment when we get to that section, but if we're talking external dns, then you could do something as simple, as you know, have a web service that pings, you know out to an external dns. You know some sort of automation to do something.

C

Operators, I think, are probably the easiest, though, because you can set up watches for when services happen or when actions happen. You know, if you use the external dns operator, it has a lot of different functionality inside of there, so it is possible, um but it can be more or less difficult.

C

Can you explain how storage works when a creator, when a container is created from an image assuming that no volumes are defined in the container file peter?

C

So I think what you're saying is, if you're, if you are using a kind of vanilla, pod definition, so one where there is not a a specific volume, you know pvc associated with it. um So I think, there's two I'm trying to think of how to start and how to phrase this so there's three types of storage that we'll find associated with a con with any single pod definition or a pod instance. So one is going to be the container image itself.

C

So when you, when you start the container the host that it's assigned to, will pull down the container image and it will store it on that host and, of course the container itself will have a copy on right layer. On top of that.

C

So any modifications that happen to that to while that container is running, will happen in that copy on right layer, and that is separate from the second type of storage which I'm going to call ephemeral storage.

C

So to me, when I talk about ephemeral, storage associated with a container, I'm thinking of things like an empty dir or you know the other type of local storage that can be used by the application, but is not the copy on right layer associated with the container. That is also ephemeral but andrew, for whatever reason differentiates those and then the third one, as you alluded to, is like a pvc right. I'm I'm connecting an external piece of storage into my container image and mounting it wherever inside of there.

C

So there is ways to control that ephemeral storage, so how much capacity is being used by the emptiers and all that using quotas? So definitely a good one to look into. Hopefully that answered your question peter. If, if not, please feel free to keep chiming in on chat, empty dress to be defined in the container file, though right, yes, you would have a um a volume right, a mount inside of there and you would just define it as empty yeah.

C

So if none of that exists inside of there, then any- and even if it does any rights that happen outside of one of those explicitly named points happens inside of that copy on write layer.

C

So I usually bring this up with folks, because you know: okay, I deploy a container right, doesn't matter what it is I deploy a container and it I I haven't explicitly named like var log as a mount point, so any logging data that's happening in that container image or in that running container is happening to that copy, on write layer and that copy on right layer is going to be hosted or in on the host the var lib containers subdirectory.

C

So this means that if you have, you know a a single container, that's generating like terabytes of log data, if it's being stored locally, which in that instance, it would be. You know. Essentially you have the potential for not just filling up the capacity associated with that with the the local hostess, but also impacting any performance.

C

um So this is one of those reasons why, um like we, we generally discourage. You know workload from running on control, plane notes, because lcd uses the same disk by default right that that one operating system disk. So if you have some random pod running on there, that's writing a bunch of logs or churning through a bunch of temp space, doing whatever it's doing that can impact the latency, the iops, the capacity that's available for etcd. To do its thing, you can work around that right.

C

So do things like mount an explicit disk on var lab container that type of stuff, so that way, and all of that's in the documentation on how to have a secondary disk for that.

A

So hey andrew finish the top of mind before we answer any more questions. Just so we can kind of get through the.

C

Yeah, let's see was there, there was only two other things, so one I will briefly mention for anybody who is using acm that 2.5 was released. um I think I'm like a week late on that right. I think I think it actually released. Last week we just weren't streaming when.

B

I was on pto, so.

C

um There is a blog post about that. um I will ask stephanie to please post a link for that, um so there is a blog post that goes through some details. We also have um jimmy alvarez joining us next week here on the ask: an openshift admin we'll be talking about acm and acm 2.5 lots of cool stuff that happened inside of there I'll also do a plug for the cloud multiplier live stream, which happens, I think every other monday, so they they that is hosted by or that is done by the acm team.

C

um So they've done some really cool stuff in their two episodes. I think that they've had so far so please uh give them a watch. And last but not least, uh if you are a helm, user, red hat is performing a survey. So there's a link for that again I'll ask stephanie to please post that into the chat.

C

So there is a helm survey that uh just go respond. It'll help us identify roadmap, things that we can do in the future to help make helm better and uh ultimately be able to do more. For for everybody all right. That was. That was the last two top of mind topics so.

A

C

A

Of questions today, this is good.

C

Yeah yeah, please keep them coming. We we love questions. This is why we're here.

A

And just so so everybody can everybody can hear it if there's just people listening um when we say container file, that is, you know the same as docker files so like um when you know, like the big push to pod man, we've kind of gone to uh you know container file versus docker file. So um just so you know it's it's there. It's the same thing.

C

Actually, that's a good point. um I think I've missed. I mixed up my definitions there, um because uh reading back, I see peter is asking empty address to be defined in the container file, um not necessarily so the empty door would be defined in the pod definition.

C

The container file, which builds the image you can define a mount point in there, but the actual like where the, where the storage space for that mount point comes from, would be defined at the pod level.

C

I'm I'm pausing for confirmation or denial johnny, no.

A

Yeah, I think you got it in that modzilla just so you know uh yeah, I I read your your statement literally and so that's what I was responding to so yeah. Definitely what I understand is correct, but it's in the pods back for the empty dirt. Yeah.

C

um See was there any others? I see you're busy in chat yeah, there's a lot going.

A

On um is one of the questions is: is openshift 4.10 fully supported on hyper-v.

C

So yes, with a asterisk, uh so with openshift we have kind of um I'm gonna, I'm gonna call it two layers or two levels here um so one and um if we can share my screen again, let's see.

D

C

Docs.Openshift.Com- um and I just did a shortcut to the 4.10 docs as soon as it loads come on internet.

C

My kids are out of school. I would blame them for consuming all of the internet, but we have gigabit. So it shouldn't be an issue yeah.

D

I don't why, why is it unresponsive.

A

And just real, quick uh pilani had asked a question: um how to fix the issue of error from server air dialing back in x509 certificate signed by unknown authority. um The first thing I would check is your certificate signing request. So just do a oc get csr and then grep minus I on pending and if you have any pending csrs, then just do oc, adm certificate approve and then id, and then you should be good to go. The other.

B

C

A

Is make sure that your coop config is in the right place you know or that you have it exported correctly yep.

C

All right so platform support for openshift. So if, if we go to the documentation here- and we have this installing section and you can look through- and there is somewhere down here- I thought um yeah there's this list of like all of these different platforms. Basically, all the things that are called out here: alibaba, aws, azure, azure stack, gcp, ibm cloud, etc.

C

All of these are explicitly fully and wholly supported, because they're documented.

C

However openshift is also supported on any platform that is certified for rel. So if I do, I'm just going to do a search here for rel.

D

Certified hypervisor.

C

And what should come up here is a kcs right. What hypervisors are certified to run red hat enterprise linux and we see here so we have um red hat virtualization right, hyper-v, esx, esxi ahv, so even though hyper-v and ahv are not explicitly documented over here in my tabco.

C

So even though they're not explicitly documented over here, because they're certified platforms for rel, they are still supported with a slight caveat, so they are considered non-tested infrastructures and what that means, and I'll post this kcs link here in just a moment what that means is that there's a kcs- and I am, of course, not logged in there's a kcs that describes sort of what that means so effectively. What it means is that we don't test it. So it is possible that you could encounter an issue that is a result of the underlying platform.

C

So our support will do everything they can to to help you to support you with openshift.

C

But if it's determined that hey this might be an issue, that's happening because you're on a platform that we don't test that we're unfamiliar with, we might need to recreate it on one of those platforms so that we can rule it out type of thing.

C

So it's still supported fully supported. um We just have to take, um I won't say, take extra measures, but we just have to be aware of that. So hyper-v falls into that category. uh Ironically, openshift fertilization falls into that category, at least for now, um and then there's a few others as well ahv, I will say um I think I I I just said: ahv is one of those, but ahv actually is a tested platform.

C

It's just it's unique because it's a tested platform, but for the moment um there is not a upi ipi installation, so it just falls into the installing on any platform install method, but that'll that'll be changing.

C

I think I I hope I answered that. uh Why is there no tool in red hat portal to generate um certificates in a standard way? um I'm gonna I'll I'll? Let you tackle that one johnny when we get to our topic. All right uh odf is based on rook steph. Yes, it is um as well as nuba yep I'll put that in.

A

C

Yep uh how to use helm survey, I'm just catching up on chat here, you're, usually ahead of me. So I'm did. I miss anything. No.

A

I think I think we've got everything for now: okay, um yeah, I think the last. The last question was uh about the the x-509 okay.

C

All right um so yeah, uh do you? Do you wanna? Do you wanna do search manager? First, you want me to do external dns. First, it.

A

Really doesn't matter, I can do either way.

A

Do you want to take a breather? Do you want me to.

C

Sure you want to go I'll I'll, sit here and drink. My water, okay,.

A

Awesome all right, so, um let me see so start manager is an operator that is part of 4.10. uh It's it's tech preview. uh So that's one thing you're having some issues like just know that it's it's tech preview, um it's actually pretty easy to um pretty easy to install. I'm sorry, I'm trying to like share my screen and talk at the same time, which is obviously not going well, I'm terrible at multitasking.

A

C

Can't type and talk at the same time that was really funny.

A

All right, so um so, basically, what I'm gonna do is I'm just gonna go over like the two types of well there's a bunch of types of uh of uh issuers and stuff like that that you can use, but the main ones that we're gonna go over today are the self-signed certificate, and so what that does is essentially it's.

A

You know self-explanatory right, where it's we're going to create a a root certificate and then we're going to use that same certificate to sign other certificates within the cluster um and then what we're going to do is we're going to use an acme provider using the http lookup. So I did kind of cheat and I put a dns entry out there first. So that way it can actually, you know, resolve the http address and then create the certificate.

A

There was a question earlier about the difference between an ingress and a route and one of the things like in openshift. I think since, like 310, uh you know, if you create an ingress object, it will also create a route object. It's just part of the the api, um so in cert manager upstream they don't. As far as I know have any like full-on support for open shift routes, I was looking through their github issues. Last night it looks like they were.

A

There was a lot of conversation around it, but they ended up moving away from it, um but yeah. So where I I did mess around with it today. So using the acme provider, I can uh create a route and then have uh the certificate get generated, and then um you know get applied to the route. So we'll see all that though, so the first thing we're going to do is go over to operators and I'm assuming everybody can see my screen.

A

So it's a good yep, all right, so manager and then just install so standard stuff. It's going to create the same space, openshift search manager. So there's two types of issuers with cert manager and it is um there's the issuer, which is a namespace scoped issuer. So essentially, what that does is that gives you as a user.

A

The ability to um you know self-serve uh certificates, for your applications and then there's a cluster issuer, so you can do um you can have like a self-signed cluster issuer certificate. So that way you can have your developers generate a csr submit that to you and then you as the infrastructure admin can go in and approve that certificate, and then it gets distributed down to their uh to their name space.

A

So we're gonna do we're going to do both the first. The stop sign is going to be a cluster issuer and then the http lookup is going to be the um the namespace issuer. So click on this guy and then.

A

Open shift cert manager all right so now. What I'm going to do is I'm going to create the first cluster issuer.

A

So I'm just going to call it self-signed and then respect.

B

C

Can you be more specific with your question? Is this like certified operators um so just while johnny's typing at a high level, there's um three types of operators, there's from openshift's perspective, there's kind of core red hat shipped operators?

C

So those are ones that if you go to operator catalog, if you check the red red hat catalog all of those and then there is certified partner operators um and those are ones that us and and partners have worked together to create and they are jointly supported and then there's community operators um which can come from anywhere and they um they're they're not supported. But there is some basic validation to ensure that they work on openshift.

A

All right cool, so now what we're going to do is so now that we've got our cluster. It's work, we're just going to generate a certificate, and so what happens when we generate the cert is it creates a secret in this namespace and so um we'll actually call back to that secret for our actual signing certificate, uh which we'll use to sign our downstream certs. So just click on create certificate and then the form view on this is a little bit easier.

C

Are you using um sorry, I wasn't paying attention? I was reading chat when you first started are using um acme here or using so.

A

This this is self-signed, so this is literally like, like I'm generating, as you know, key pair and everything, okay.

A

All right so, like I'm just going to call the um the actual certificate. That's just going to be the sub sign ca root. um The issuer ref is going to be. I do this every single time. I cannot remember the name of the issuer that I said I created.

A

All right, so the issuer, the cluster issuer, is still signed. Okay,.

C

And to be clear you you could use this with your own, like intermediate ca, exactly if you have a corporate ca, you could get an intermediate and then use that to sign all of the all of these openshift certificates. Exactly.

A

And if you've done, like you know like with every other kubernetes object right like these kind of stack on top of each other, but um like once when I finish this, what I'll do is I'll show you um all the different options that we have boot so.

B

A

The secret that's going to get created once the search are are generated. um You know if we wanted to add email addresses and urls and stuff like that. This this is ca. This is one of those basic constraints. So if you ever look at a certificate- and it says you know um under the constraints section- it says sca- that's what this is actually uh setting- um let's see subject.

A

So if we wanted to put all the subject stuff like the location country, anything we could put that in there. If we wanted to update the usages, we could do that here. um The default is 90 days, so I'm just going to leave that the same and then common name. I'm just going to call this. The self sign.

A

All right and then we're going to do ec bsa and then set it to 256.

A

And then we should be good to go.

A

All right, so I probably I figured something so we'll just give this a second. So we still, we have our cluster issuer right. It's ready to go. We have our certificate. I've obviously messed something up here. So let's take a look.

A

Is your certificate of super does not exist.

A

B

C

Give it the name of the certificate object instead,.

A

Yeah I was, I was just.

C

A

The dm runaway yup exactly so that should be okay. Now.

B

A

All right, I'm just gonna blow this away and start over since I can't seem to.

A

A

Is your ref and this is self-signed.

C

Ravi pointed out that uh you may have a space at the end of the secret name.

A

Oh good call uh cluster issuer.

C

Tell you, while you're uh doing that I'll tell you what I this audience has already helped me immensely because, like the when we had rom on and we were going through, assisted installer, you remember how we encountered that issue with oh yeah, multiple nics that had ips on the same subnet, and somebody pointed out. Oh it's, this yeah, that's come up like four times since then, and and everybody's like. Oh, that was exactly it. How did you know I'm like? Well, I didn't here's where in the live stream, we figured that out.

C

So thank you to our audience. You'll help us exactly.

A

I'm trying to fly here so I'm not doing myself any favors.

A

All right, let's see and then private key you see dsa.

A

There we go okay, it was most likely the space at the end of the secret. If I, if I did, have it so so now, if you think about this right so now, we've got this issuer right. We're saying we're telling the search manager that we want to use the self-signed uh issuer we've generated this base certificate here, uh which is going to be our ca, cert that we're going to use to sign all of our other certs from this point on so now.

A

What I'm going to do is I'm going to create another cluster issuer and I'm just going to call this one. um You know cluster ca and then we're talking before about the different variances or the different types of ca that we could use right. So we could either use acne. We could have our own external ca self-signed. We could have used hashcourt, vault and then there's this video or banaffi. I've never actually heard of this until last night.

A

So there's a lot of different um cas that we can use and with acme it does both http lookup and dns lookup, so um whichever one you've already got configured uh but because I'm doing self signed, I'm gonna go back down to this.

A

Oh, and this is actually.

C

Kind of a random question: do you know if it'll work against like a uh like red hat, idm certificate management or active directory, or anything like that?.

A

I don't know I um I was thinking about that. I wish I would have set up an ibm last night just to test it out, but yeah. I didn't even get that far. How dare you I know.

C

You could you could sleep when you're dead you'll be. I know I know.

A

All right, so I'm I've, definitely because I can't remember anything. um So this is a cert.

A

So, let's keep here, the secret name gets used quite a bit, um so it's it'd be great. If I could remember it, but I just for some reason cannot.

A

But what we're doing is we're going to set this compare here and then we're going to a lot. This is essentially saying use that root c, that we've that we created earlier to sign any subsequent any any certificate after this.

A

A

Yeah, okay, cool and get rid of this space.

B

A

All right so cool, so now we're ready to go so now, I'm just going to create another certificate, really quick and then this certificate is actually just going to get signed by um or the one that we had before.

B

A

So let me go back to this. We'll call this demo so.

C

You created the ca. You issued a certificate for the ca and now you're issuing certificates for applications exactly.

A

Exactly and so actually, if I wanted the right way to do, this would be um let me just go ahead and create a new project.

A

I'll call it self sign demo.

A

A

I'll create a certificate.

A

Certificate name demo, and then this issue reference. This is going back or the issue of reference. This is going back up to that next level, right where we're going back to the cluster issuer and we're going to um where I will remember um we're going to use the um that that ca that we just used or that cd certificate that we just created.

A

I can't remember the name: it's just killing me that I should write this down.

A

All right and then so this secret name- this is gonna, be the name of the secret. That's in our namespace to look up so demo.

A

No sir key pair, it's not a ca. um If I wanted to do like a renew before so, I could say: hey renew this certificate before the 15 day mark right or at the 15 day mark.

A

If I wanted to start to last longer than 90 days, I could put that here um the common name I'm just going to put it doesn't matter, but for the dns names. This is where your subject- alt name, fill this. This is where this would come in. So then I'm just going to call this app um sign, demo, dot, ads dot.

A

All right, so obviously you can have you know other pns names or if you're going to use ips, you could use the ip and stuff here I need to set this to rsa and then uh not this actually I'll set this to ptsi and then pkcs1 and then 2048 all right cool.

A

So this is going to stay interesting because I'm dumb and can't remember anything. So let me go back up here and get this name.

A

And then cluster issuer is cluster c. Okay,.

B

There you go gca.

B

B

All right, cool.

C

Dwayne, I yes um pretty much anything can be an ansible playbook right. um I I don't know um johnny. Maybe you know can, can you do this through get ops as well.

A

There there are some helm things that you can do. There's I mean there's there's a lot of things that you could do. I definitely think that you could do a majority of it through home, um the self-signed stuff, the the issue you'll have with self-signed.

A

Is you know it's it's a it's an untrusted certificate, so you'll have to pull down the cert and then put that in your local key store right and um so that's either through uh you know, scp-ica trust and stuff like that or um if you do like the uh acme, resolver or the external stuff with vault, then yeah, almost all that stuff can be done in in uh you know, either through helm or definitely through ansible.

A

All right, so so what happens when we create that certificate? Is it generates a certificate request and then that certificate request gets approved and then we end up getting this gener uh this um this certificate that ends up going into a uh secret in the same name space, and it's in here demo start key pair. So if we look now, I've got the certificate. This is all base64 encoded yeah. So it's all there.

A

If I were to reveal the values, then I can take the certificate and I can actually extract these and put them in my local um uh key store and then you know be able to browse through the application using uh https. So I'm not going to do that just so, we can get over to the http side. Or did you want to do the dns one? First, like.

C

It didn't matter to me and yeah, so you would. You would mount that secret into your your application pod and then you would use that for its its server. So.

A

What you have to do, because the routes, don't you can't mount a secret into a route? So then, what you end up having to do, and let me stop sharing this I'm going to share my terminal really quick. Oh.

C

Right right, so you so you're creating a certificate to use by the endgress controller, exactly opposed to creating a certificate, but but you could create a certificate and use it with like a load, balancer service right, an external load, balancer metal lb or something like that.

A

Yup, exactly uh not this all right. Can you see my terminal yep? Okay, so essentially what happens is if you look at.

A

What you'd have to do is you'd end up having to create an ingress control or an ingress, and then in the ingress you add these annotations, like so you're determined, you're, saying who my uh cluster issuer is. You know you can you can put duration or renew before? If you wanted to um and then essentially create this ingress and then what will happen is the ingress will create the route and then the route will pick up the search off of the ingress um with the http.

A

It's a little bit cooler, because if you look at the ingredients now- um and I just learned about this- I didn't realize the ingredient or openshift had an ingress class name or like a default ingress class name. So I I don't know why it just never occurred to me that it did have one. um But if you put this ingress class name in there, what'll happen is when you create the certificate and you create this ingress.

A

It it'll go through it'll, send the request out to acme, or until that's encrypt. Let's encrypt will you know via a um it'll? Go through it's challenged, it'll go through the order and you can actually watch all of them go from like uh pending to issuing to approved and then suddenly your route has a uh it has a certificate on it. That's that's valid.

A

So what we're going to do real, quick is I'm going to create a project.

A

Just so, I'm making sure that I'm tracking the right one I'm going to create a new project.

A

All right cool, so now what I'm going to do is I'm going to create an issuer right. So we were talking about cluster insurance which we're using the self-signed certificate for and then we're talking about issuers which are name spacecraft. So I'm going to create a namespace scoped issuer. That's going to use, let's encrypt prod, all right! So then, if you look, it's just we're saying: hey here's the acme server. So this is just the prod url. uh You know my email. This is the secret that it's gonna create.

A

So this will have all the the acme certificate challenges and stuff like that kind of stored in it and then we're gonna use the openshift default ingress class.

A

So then, if I do http1.

A

We can see I've got my my issuer, it's ready to go so now. What I'm going to do is I'm going to get a nap.

A

And then I'm going to look at this ingredient to make sure that I don't have any fat finger typos in here, because you know I've only made about 15 of them since I've been online. So, um but if we go through the ingress object here, we're just going to name this. uh This ingress or this route rather www demo.

A

um We're telling that we're declaring an annotation to use the let's encrypt prod issuer that we just created. uh We're gonna use the openshift default. We're saying: hey, create a tls for this hostname and then put everything in this secret.

C

So effectively, what we're doing here is we're creating a certificate that is specific to this full domain name route. Instead of using a wildcard certificate exactly start out, apps.

A

Exactly and and I was going to do wild card, but the reason why I did this because generally, at least in my experience in production, we don't use wild card certificates right a lot of times. We use actual name, scoped certificates and dns entries and stuff like that. So you know like if you're a dev and staging and stuff like that wildcard is fine but like when you actually get to production. You want to know what's actually out there and using your your ca, yeah.

A

It was about to happen.

B

A

So if we see we have an ingress.

A

And what that does is that's going to automatically create a route.

C

Yeah, I see the path there is the dot well known, acme, challenge yup.

A

C

A

Then, what's going to happen is well so what's what's what's going to happen? Is ot certificate?

A

Once this thing comes up and it gets approved, then there will be another route that comes up with that matches. This uh that matches the ingress.

A

But it takes, it does take a minute.

A

We can see the certificate request, so we can see that the certificate request has been approved.

A

If we look at the challenges, we can see that the challenge is still pending, so this is this is, um and I know why this isn't going to work. I um the challenge is spending so what's going on is, let's encrypt is actually going out to dns and it's trying to find that name. So I could do like a curl on it, um but because I was being clever.

A

I changed the name ndns.

A

I did write this one down so.

A

All right, I think, that's.

B

A

This is the only thing. Oh so I will complain a little bit here. The only downside is, if you delete things it doesn't. Well generally, it didn't already build the secret, but if the secret was already it already existed and you deleted that ingress or the certificate it doesn't create or doesn't delete the secret automatically. So it that's kind of painful.

C

Yeah, I can see positives and negatives to that. Some folks may want to keep those for various reasons.

A

All right so then we're going to do request.

A

A

This should actually take off here pretty soon. If I I'm just kidding I'm dumb.

A

Oh, this is where I outsmarted myself here. So I'm I'm. This is excellent, because this is live.

C

See this is why you got a test in production.

A

I know I know so. What's funny is I tested on my other cluster all night and so like I was like. I should change all this right now, but I had faith in myself that I I knew better.

A

All right and then oc get challenge, so that should be. This should actually be happening pretty soon.

A

Okay, here we go so get challenged, so we can see there's nothing in there. So if we do once you get certificate request, all right we're good to go once you get certs now, if we do oxygen secrets, we'll see our secret name right. So we see now we have our secret. So if I were to do an oc get mines, oh yama secret, that dude we can see now that I've got my my my key. I've got my in my cert right and then my ca for for watching crypt pride.

A

That's all stored in the secret back in the um the other name, space yep well,.

B

So yeah I was going.

A

To say I forgot I did this when I was an issuer so because I did this one as a namespace scope that secret stored here. But if I were to do this in a cluster game space or a cluster issuer, it would be in the openshift sort manager. So then that really just kind of it does all the automation for this. This part was actually super easy, um the dns stuff.

A

If you wanted to do dns, lookups or dns uh challenges you, you can configure roles, and I am policies and stuff like that within aws and it'll do a lot of that stuff for you, but it it was a little bit more involved than yeah. I figured we had time for it. So if I didn't once you get route now, I can see I'm sorry I'll. Do it to get ingross, that's there and then curl.

A

And the rot will show up.

A

What's happening.

A

B

There's there's no route there.

A

Yeah, I did create the object.

A

But the certificate is there.

A

Right, there's that.

A

Oh, oh, my god.

A

Mommy, this is me being dumb. I've literally outsmarted myself today.

A

A

Let's see, delete ingredients.

A

All right so now so what did happen? I don't know if you caught that andrew it's I had the service name wrong, so I I created the app as as www, but my my template had just demo so.

A

And this should be already, it should be pretty quick, yeah.

B

A

One hand all right so there's our route.

A

There's our ingress our certificate's approved so now.

B

There you go yep.

C

Now we're good to go. Curl did not complain, which is what we wanted.

A

Yep exactly so there you go.

C

Yeah so, uh let's see catching up on chat here, a couple of things: um I'm gonna go slightly backwards, um zachary, um no, no, so we're using openshift um so similar or same underlying technology, cryo creo, but openshift, and you can use podman to create container and images and all that other stuff. Four uh palani. Thank you for all the help. uh Yeah glad to hear you got that sorted out, I'm wondering if maybe just it needed some time after the certificates are approved to refresh services and everything keep it uh modzilla.

C

Is there a good way to use self-signed search with containers without building a new image that has the root ca included? Is there an injector or something like that?

C

um So I guess it depends on what you would be using those for.

C

So, by default right, when openshift creates a certificate using the internal ca for openshift, it will automatically add those the certificates for that and add them and all that other stuff, so that internal ca should work, which is all self-signed stuff. If you have like you know in in your lab or your organization is quote unquote, self-signed right, um I think you would want to do a cluster level certificate um so like in your install config.yml, when you add additional cas when you do that that automatically gets added to all of the pods.

C

um So there is an injector there. I don't know the magic behind the scenes, um but that does work, so that would be an option to be able to do that like if you have a enterprise-wide corporate self-signed again, I don't know why I'm using air quotes. It is literally self-signed, it's just done by your organization and therefore universally trusted right.

A

Yeah, so so sammy, so basically the first thing I did is other than all the screw-ups right, like the the first thing I did is I created a cluster issuer for a self-signed certificate right and then all that's doing. Is it's just telling open shift that I'm going to use this cluster issuer to sign certificates and not to worry about any external ca? Then I created a certificate that I actually identified as the ca and um that generated a a secret.

A

The next thing I did is I created another cluster issuer uh to use that certificate that was just generated so that way I could sign all certificates after that, so that it's it's general like if you're going to create a self-censor for your domain right now, you would generate the csr, the key and the cert all off that same thing, and then you would sign everything else after that, with that certificate, that's exactly what we just did and then the last thing I did is I just created a certificate for an app, and um uh you know just just showed that hey.

A

I can use this downstream ca to sign this certificate. um The next thing I did is, I used the acme issuer, which is a it could be cluster or namespace scope. But in this case I did a namespace scope and what that did is it just says: hey. I want to use acme prod or let's encrypt prod, to sign my certificates.

A

Now I'm using the http lookup, so I went ahead and created the dns entry for my my route before I you know started all this, even though I forgot about it and um what acme does is it does a challenge, and so, as you generate your certificate request, it goes out and it looks for that um that http server and make sure that it's there and then uh once it once it verifies that it's there.

A

It approves the certificate, it creates a secret with all the tls cert and the tls key, and then it puts all of the uh let's encrypt goodness into another secret. So that way it can be referenced throughout the namespace.

A

After that, I applied that to an ingress and what that did. Is it allowed the ingress to go and get the certificate, associate the certificate to the the route or to the ingress itself, and then once it all came up and it it did the openshift fist bump it created a route which made it available from outside the cluster and I'll put all this in the blog. So that way, it's a lot clearer than me. Mumbling and stumbling.

C

I don't know what you're talking about. I never do that. This is good. This is good yeah. So, um let's see uh any other chat here, uh dwayne they're in ingress first uh trying to get a roadmap to study by um duane. I think it depends on what specifically you're looking at um most folks have probably heard me say before that: openshift is like a fractal right. Every time you find an answer to a question.

C

It opens up a whole new set of questions that have answers that open up whole new sets of questions and so on and so forth. So I think it would be helpful to understand what it is that you're trying to do so helm is useful for like deploying applications and all that other stuff.

C

It's fully supported by open shifts, and it is, I won't say an alternative to, but it is an a method in addition to operators for deploying applications.

D

C

All right um so yeah we can switch over. I can do the external dns stuff. It should hopefully be quick.

A

Nice yeah I'm looking forward to this. This would have been in hindsight when I was uh we were talking before. The show should have built on like what you're doing with the external dns stuff, like updating an entry like that. That would have been a cool demo actually.

C

They're separate but related, if that makes.

B

C

So um yeah, so external dns is a and where's my documentation here. So external dns operator is used for creating entries in external dns systems, so within openshift it's only or and in openshift 4.10. It's only supported with the hyperscalers so think. Aws, azure and google, specifically the upstream external dns operator, works with a bunch of a bunch of other stuff, and I I have tested it and like I've, deployed the upstream bits for external dns and I've been able to do in my lab using the rfc uh option.

C

So there's a a configuration here so when we create the external dns, um so there's a a source or excuse me not a source, a provider for um the rfc spec, so I've been able to update like bind with my um dns entries and stuff like that.

C

So I I briefly considered using like metal lb and the upstream external dns operator to show how that works but decided against it, because this is what's officially supported and it's it's a I'll say it's also much easier with the hyperscaler because, like everything, just kind of works out of the box, so relatively simple, just like what johnny did we use operator hub, we can search for the what is it ns, so external dns operator.

C

I will say that this is inaccurate, so you don't need to do that. It will actually create those for you, so we can ignore that once it's installed, we have the external dns operator here and while I tested this yesterday with a completely different cluster, I haven't tested it with this cluster. Aside from installing installing the operator here so we'll see what happens, it could be fun.

C

So you'll note that there's really only one crd- that's defined here so we'll need a little bit of information before we get started so one my cluster is deployed into aws, so you'll want to make sure that either the user that you deployed openshift with has enough permissions to create and update dns entries or you'll have to add credentials for that. So if we were to look over here in the external dns, the the configuration parameters you'll see here, there's like a credentials thing so you can create.

C

You can add your aws credentials in there and it's the gosh. No, I don't even remember having a brain moment here. The aws access key and secret access key. So you can add those into that secret and then add that into the definition for this and that's how it would have specific. You know permissions. So if whatever your a iam role etc, doesn't have those you can, you can add a separate account or if you don't want to add them to that user, all right. So with my cluster here, I'm in aws.

C

I've basically done nothing in this cluster except add in the operator here, so with my operator deployed. I want to go to the external dnses, oh and I need one other thing, and that is. I want my zone id.

C

D

Pull back up a little bit.

C

On my uh my admin host here, I've got the aws cli configured, so I'm going to use this command. Aws route 53 list hosted zones, and this is, I have a account, that's linked into some of our engineering stuff. So it's super noisy, like there's hundreds of entries in there, so I created this quick little grab command just to show the one that we want to use here and what I'm looking for is this id. So we have hosted zone and then this specific identifier is what we're going to use and I am using.

C

Guacamole here, which means that it's a slight pain in order to copy and.

D

Paste things out so we'll do that real, quick and save it off.

C

To the side, and with that out of the way, that's pretty much all we need to know so I already know the domain name that I'm going to use. I just retrieved the zone id and from here we can create our external dns.

C

So I'm going to use the yaml view. You can absolutely use the form view here. I just find it a little bit cumbersome to dig through all of these different options, so kind of what we were saying before.

C

There is only a subset of these things here. It's like infoblox, bluecat, etc. There is a lot more in the external dns, so.

D

C

Do a quick search here we go to the somewhere around here, yeah we'll take the first hit yeah. So if we look in here, you can see that there's a huge number of providers in here, so the rfc 2136, is what I've used previously locally with bind to do. Updates yeah thanks for taking me to the rfc, I should have known better.

D

I think it's down here is where the actual link is to show you how to do it. uh Rfc, 2136, yeah, so.

C

This is how you would configure it, and it's just like if you've ever configured like isc dhcp and bind to do dynamic, dns updates for dhcp clients, it's effectively the same thing right. You create a zone that has a key associated with it, and then you you're, basically configuring openshift kubernetes to use that same sort of mechanism to reach out and create those uh those entries anyways all right.

C

So I'm going to use the yaml view not just because I already have everything created and I can just copy and paste but again because it's easier for me to kind of see and explain.

C

Let's walk through what I've done here. So one I'm giving it a, and I don't want that. One.

D

I take that back. I want this one.

C

So I am create I'm giving it a name. So in this instance I have my domain aoa for ask an openshift admin. uh This one is going to key off of routes, we'll look at that in just a second and then, of course, it's aws.

C

So the spec is where all the magic happens. Domains we've got the domain name and we are doing a filter here, so it is going to match specifically for aoa.devcluster.openshift.com inside of the aws zone.

C

So I shouldn't need to include this zone identifier, because I have that identified there, but I'm going to do it anyways just to be explicit and and double sure, because I don't want to accidentally mess up anything that the engineering folks are doing right.

C

Andrew does not want to be responsible for ci failures and therefore delays to shipping product. That's just just my preference, so a couple of other things that we've got set down here, so one we've got a source I'm going to set it so that it keys off of routes inside of the cluster, and because of that I need to specify a default or a router name that we wanted to use so with a default openshift cluster there's one and its name is default.

C

So if we come over here to the to the documentation, docs walk through all of this stuff. So you can see this down here and you can also go in here to the options and it walks through all this stuff as well. So here's our filter and here's our source. So this one is keying off of services and I'll show that in just a moment as well, so here's the route one and then the last thing or the last two things that we've got in here are the label filter.

C

So essentially I'm going to add a label to any routes that I want to that. I want to create a dns entry, for that looks like this right and then the fqdn template is what it uses um in order to create that so it'll use the name of the object and then append it into here.

C

So I don't think that is required. No, that is required if you don't use the allow hosts. So there is an option and I think I've got it in yeah in the other one. The second example that I'll show in a second where you can specify the hostname entry that you want to create the dns entry that you want to create.

C

If you have that turned on, then this fqdn template is optional. If it's not turned on and it's not by default, then this template is required so that it can generate the dns entry all right. So with that I'll create my external dns.

C

If I want you, I can come in here and you can see it deployed the pod and if we go to logs and switch over to the operator log, essentially we can watch what's going on here so by default. This is not a super, informative log I'll say: basically it it just tells you hey it's in sync with what's happening, it's in sync with the cluster, so I you can increase verbosity but um and get more information about. You know: hey it's creating this specific dns entry based off of this route.

C

That type of information- I just haven't done it by default.

D

C

Right, so I want to switch over to my project here and inside of this project. You'll notice, a couple of things so one I have deployed a couple of applications, so I've got this hello kubernetes as well as my simple deployment, and if we come down here to networking, I have services for each one of those, and I also have I don't have any routes at this point.

C

So if I just create a route and we'll call this one.

B

Low without a w.

C

And we'll point it at our hello kubernetes and specify the port and create that. So when I look at that right, I'm and I'll be able to access that application. Everything is good right, so here here's my route, 53 zone. We can see aoa.dev cluster. If I do a refresh here, you can see we still have our five records. Star.Apps api api, end aoa.

C

um Sorry, that's the start of uh authority, so my typical three dns entries for any openshift cluster- that's inside of here. So this is working where'd. It go. Oh I closed it. This is working because of our wildcard, so very much to johnny's point. Before of you know: hey not. Everybody uses wildcard dns certs, especially in production, not everybody or dns certs, not certificates, not everybody uses wildcard dns in production either.

C

So what I want to do is I'm going to mimic like what would it be or what would it look like if I didn't have that wildcard dns and in this instance, all I need to do is go in and add that label.

C

So I'm going to add our label and copy and paste it from over. Here.

C

So this is the label that I specified inside of our uh external dns object definition and when I hit save and then do a reload and what we should see after a moment, it may take it a little. It usually takes it a little bit for some reason, but what we will effectively see is it is going to create a dns entry for this host definition.

B

Maybe come on little.

A

D

C

Crash loop back off, oh but y.

D

Containers with unready status what's happening friend.

D

Failed to wait for external dns controller timed out waiting for cash to be synced. Well, that's not good.

C

Of course, this would I I did this two dozen times yesterday.

A

Yeah, I'm checking status.rt.

C

No, you know what it is um so yesterday with my test cluster, I did um so. What do I want dns?

C

I did both of these prerequisites and it actually creates its own, as we just saw its own namespace, but apparently it does need to have these additional resources applied.

D

And, of course not uh okay, so maybe it does need both of them.

B

C

So yeah it's um creating in the namespace, so so there's an external dns namespace. Now so, let's close out of you so there's now an external dns and an external dns operator.

C

So the external dns is the one that I just created and then we applied those permissions right. So these here role, binding cluster role, cluster role, binding and now what I would expect.

D

Is if we come over here to our pods, we'll help it along a little bit.

C

So I I think it yeah see, so it does actually need those instructions, even though it doesn't deploy to that namespace, which is interesting.

C

So my bad, please ignore my earlier statements where you don't need to follow the directions that are clearly there. I thought it was an artifact from uh pre-ga anyway, so now that that's in place, we can come back to our logs and look at our operator yep reconciling external dns.

C

So now we should be able to go to route 53, and now you see we have seven records, including our hello.

D

And sullive, and if I copy this guy.

C

Here we end up exactly where we were before just without the wild card, so one thing you can do is if we go.

D

D

No, I want my crd.

C

So one thing that uh is interesting and I'll be honest. I haven't tested this before so I don't know what's gonna happen. I believe that what we can do here is.

D

C

um Label filter all right, so basically without the label, filter it'll match all routes and, as a result of that, what it's going to do is all right. Thank you. What it's going to do is is it is going to create dns entries for all of our routes as soon as it goes through. Another reconciliation cycle and the result of that is, and what I'm going to do is just to prove that it should work the way that we think it works, I'm going to delete our yeah there. Now we've got 23 records.

C

D

That's awesome.

C

All of these right, here's all of our routes and if I were to come here and do in so you get route a here's all of our routes and stuff like that.

C

um So now what I'm going to do and hopefully not break everything- is I'm going to delete my default star.apps.

C

Get out of here.

D

Delete record yes, all right so now, startup apps is gone, but this guy should still work and it does.

C

Now it it's possible that it's cached, so what we can do is create a new route.

D

And I'm going to go back into my a and solve namespace here, go to routes, I'm going to create a route! I'm going to name this simple, because.

C

It's going to point at our simple service we'll hit create here. So now we've got this simple dash and sullive we'll come.

D

Here and we'll have to wait for that to be updated, so we should see this jump to 23 again.

C

Like I said it does take it a second or two to update those for whatever reason it's like it does the sync resynchronization on a timer, oh yeah, but until it's done with that, what we should expect to see here is this: oh there it did work um so yeah it. It does exactly what you would expect.

C

That is awesome, so we can do the same thing with services, um so I will.

D

Copy out this one.

D

We'll do our.

C

Do this here, so I can do the same thing with a service, so you can see. I just named this one for services, so same deal get out of here, help domains. I just want to do a domain filter specifically for that I'll, add in our zone identifier. Here this time, I'm going to do it for services of type load, balancer again with that label. For um so that way, it's only specific services, um although I think there's only a couple.

D

Of for this get go, let's see get service.

D

A crap dash, I load balancer.

C

um So yes, so there's only three load balance services.

C

But, rather than creating a bunch of extra stuff, actually yeah we'll let it create it. I don't.

D

Care doesn't hurt me yolo, you know.

C

And then we have this host name, hostname annotation, so the hostname annotation allows us or is what enables us to customize or specify the exact hostname that we want so in the docs- and I don't remember if it's in our docs or in the upstream docs, it does call out that this can be slightly risky because it does exactly what it says right. It's going to allow the user to customize a domain name that may not may not be valid. That may redirect to something else. That can do a lot of other things.

C

So you know, as with everything you know, grain of salt moment of caution, but so we'll hit create there and.

D

When we come down to oops, I want route 53.

C

So now we've got 30 records inside of here, and some of these are going to look interesting because.

D

They should be our.

D

D

So there's our services that just got created inside of here.

C

So what I haven't quite figured out and when I was testing this day, I I think this is a result of how my cluster is deployed. It's just a default ipi deployment, so I believe it creates internal load balancers for those load balance service types inside of openshift, so it's an aws cluster. When I create a service type load balancer, it creates a internal only load balancer.

C

So when I browse to this from my desktop here, this doesn't actually work, but what I strongly suspect is that if I were to be inside of aws, this would work as expected, but anyways the the goal here is that it it does exactly what it says it does right um it. uh It creates that load balance service or that dns entry based off of that service and again on-prem with something like metal lb.

C

So remember a few weeks ago, when we showed metal lb, the two of those in combination means that it's now super easy for me to do things like expose again a mysql on port, 3306 or or all of those types of things.

C

So that way, I don't I'm not limited to just http https um through the routes and the ingress controllers.

A

That is awesome, that is man. That's super slick I like that.

C

So yeah pretty straightforward and uh the good news is when you delete these. It does clean up. So if I'm here to my service and I delete the external dns, what we.

D

Should see here is I got you duane.

C

After uh after a second after it goes through and does its uh remediation, everything should just pop right back in.

B

C

Yeah, I thought it was really cool and you can see the cluster is working just fine. Now that I deleted that star.apps, you know wildcard dns entry because it created dns entries for every one of my routes, so that way it uh it just works.

D

Yeah, I don't know why it takes so long to to rectify or to uh remediate, but that's.

A

All I got that's awesome all right, so we had a question from daniel norris um about uh he was asking. Is there a way to tie all the routes automatically to use start manager? um So I think if you have automation that can do that, like if you're creating, uh if you're using like helm, to create your route or if you're, using a template to create your route, then you can add the annotation within that um or any type of like you know, whatever automation, you're using ansible, helm, etc.

A

uh But right now I don't know if there's an actual like. I don't know, if there's something that you can actually put in the route itself um or on the even on the router itself to to actually generate the search automatically.

A

um I think you'd have to either modify your ingress and then build off that template, or you know you put the stuff in in uh in a route. So I'm going to actually go back out to engineering and see if there's some, if there's a plan for a roadmap for route support, it didn't look like it on the upstream, but that doesn't mean that we wouldn't put anything in uh to support our use case. um The other thing is um there: is a operator called cert util. Thank you, stephanie you're awesome.

A

uh There's an operator called certain utils that was written by red hat uh uh community practice, and essentially it does exactly that. So what it it'll go and it'll search for the secrets inside that namespace and then, if the secret is, um uh you know, I can't remember if it's like. I can't remember the name of the cert type but anyways or the secret type. I think it's okay um anyways.

A

It looks for a certain secret type and if it finds it, then what it'll do is it'll it'll strip out the certificates and it'll apply it to the route. So it's just another operator, it's just layers and layers and layers of operators. But that's that's exactly what it's designed to do um and I know that that was some of the functionality that they were trying to get put in the cert manager. But it just didn't land.

A

So hopefully that answered your question. I know that was kind of a lot there. So.

C

All right, I am on I'm on mute as I'm typing away, um responding to uh wayne here in chat, uh so I I did post for anybody who's interested. um So we do have a short link, a short url for all of the live streams. So we have a youtube playlist on the red hat youtube channel, it's on the openshift channel as well. But if you go to red.ht aoa, ask an openshift admin. It'll! Take you right to that playlist! You can see all 70 something. Where are we 71 today, yeah 71.

C

yeah, so you can see all of those episodes um on there all right. um So I I think we're I don't have anything else to show. If there's any questions feel free to ask questions we'll do our best to address those here before we stop the stream anything we can't um answer or if anything comes to mind right. If you're not watching us live, if you think of something afterwards feel free to send us a message, so you can reach me at andrew.sullivan redhat.com or on social media.

C

Practical andrew, just like you've seen me here in chat. um You can reach johnny johnny at redhat.com or jrocktx1 at on on twitter on reddit on all of the places so yeah yeah, we love to hear from you, so don't hesitate to reach out. um I I can't promise that I'm always timely in my responses, um my inboxes. I need to figure out how to like.

C

I wonder if I wonder if I can like in gmail, because we use google services, can I create a filter that will add like a a star or a label or something if it comes from external sure, because you know like 98 of the email we get is from internal folks, so so that way, it'd be easy to identify yeah bubble it up, anyways, maybe I'll, look into that um yeah, so anyways we'll be back next week.

C

um So next week we will be joined by jimmy alvarez to talk about acm 2.5 lots of really cool stuff there. I'm I have a side project where I'm working on doing a whole lab environment for red, hatters and rhpds.

C

That involves a lot of acm stuff. So I've learned a heck of a lot more about acm than I used to and it's really cool. So if you haven't had a chance to look at acm, yet um definitely encourage you to do that.

C

um Let's see I'm trying to think uh the 29th we've got a topic, but I don't remember what it is now.

A

It was, uh I don't know the name like that, the cute name for right, but it's essentially um we're taking open shift admins from the field right. We're gonna bring one of them on and we're gonna talk to him about his experience and stuff like that. So awesome.

C

Awesome, oh yeah, that's uh we're bringing on one of the accelerators right.

B

C

Awesome yeah so uh for anybody who's not familiar with the red hat accelerators program, um so they're like uh they're customer partner, folks um who are advocates for red hats. um So we we reached out to the uh to the accelerators team and we asked you know hey: does anybody want to come on and and share your experiences so not not like commons, but a little bit like commons, so that that'll be happening on the 29th so yeah?

C

Just for anybody who who is interested in any of the topics we've got coming up, please subscribe to whatever channel you happen to be on that way, you'll get those alerts. When we go on on the air, I would also say you can go to red.ht livestream and on that page is the calendar, so we try and keep that calendar updated with. What's going on, not just here on ask an openshift admin, but across all of the different live streams.

C

So you can subscribe to that. It's a google calendar and then you can see all the stuff we've got going on and I'll say that uh for me personally, I because there's always a lot of stuff going on. I subscribe to that, but then I copy over specific events that I care about um to my personal calendar. So that way I don't get all that noise, so just another option to keep track of or help limit, and we were talking about the other week with hillary right alert. Fatigue.

C

Calendar fatigue all right I'll quit my rambling. Thank you. Everybody. Thank you, johnny for joining. Thank you, stephanie, for all your help. In the background you've been busy today, keeping up with links and all that other stuff. So thank you so much to our audience. As always, we really appreciate you joining us today. um Chat has been phenomenal. We love all the questions again, anything anything that comes up don't hesitate to reach out to us uh all of that being said, johnny uh last words.

A

I'm glad you're back, you know, I'm glad you had some good pto stephanie you're awesome as always um yeah. If you guys can go to youtube and you know like and subscribe it's it's. uh It just shows that we're doing the right, stuff and uh yeah looking forward to next week. So I'll see you later.

A