Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 14 Nov 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building cert-manager: Migrating Thousands of Users from kube-lego to use CRDs - James Munnelly

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).

Building cert-manager: Migrating Thousands of Users from kube-lego to use CRDs - James Munnelly, Jetstack (Intermediate Skill Level)

cert-manager is a new project, built to replace kube-lego and make x509 certificates first class citizens in Kubernetes. Using custom resource definitions to introduce the concept of Issuers into a cluster, end-users can request signed TLS certificates from an ACME server (e.g. Let’s Encrypt), a signing key pair, Hashicorp Vault, or your organisations custom CA through its extensible design. This talk presents cert-manager and demonstrates its new features over its predecessor, and specifically our approach to migrate thousands of users from kube-lego to the new custom resource backed system, without hindering future cert-manager functionality or effecting production users. At the end, we’ll go over the roadmap and future plans for the project, as well as how you can get involved!

About James
James is a Solutions Engineer at Jetstack, which involves helping customers bend and break Kubernetes to their will. He's created a number of extensions to Kubernetes core, including cert-manager (a kube-lego successor), Navigator (DBaaS for Kubernetes), a simple cloud provider for bare metal clusters as well as various contributions to the upstream API machinery.- Wrangling Kubernetes API Internals https://skillsmatter.com/skillscasts/10599-wrangling-kubernetes-api-internals - Extending the Kubernetes API: what the docs don't tell you https://kccncna17.sched.com/event/CU6r/extending-the-kubernetes-api-what-the-docs-dont-tell-you-i-james-munnelly-jetstack
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Cool yeah, hello, welcome everyone we'll get started, keep everything moving. We've got a lot to get through. So thank you for coming along. This is my tour concert manager how we are moving everyone from cube, Lego and yeah, going over some of the new features, our roadmap and showing some of this off and the history of it all. So, first of all, yeah Who am I. I've got a big blank white space. There I think I was meant to put some text there.

A

So I've been working on cube lettuce, extensions now for quite a long time as you'll see I've been using Cuban it for a few years and I work with jet stack as a solutions engineer, which means I'm, helping customers achieve their goals, with Cuba Nettie's to keep a generic who a jet stack, we're a uk-based remote company, we're a consultancy that help enterprises and large companies moving over to kubernetes and adopting all of this new cloud native technology that you see out there and also building little tools to obviously ease the pain such as this.

A

So what is certain manager? First I think I, always first wolf. Some history cube Lego so who here has used cube, Lego cool nice, yeah, so cube? Lego was one of the first qubit, like external kubernetes controllers. I think this is me off the top of my head, but I think the only one before there was one of the actual ingress controllers, something like the nginx ingress controller.

A

It was made a very long time ago, I think nearly three years now by christian who sing over here yeah, it was it kind of solved one problem, one problem: only people wanted to get TLS certificates signed from let's encrypt a new service where you get your free TLS and it's seemingly used absolutely everywhere from what I've seen the prowl.

A

So if you ever submit any code to kubernetes and you go and check to see how those tests ran, prowl secures its endpoints with cube, Lego I think Kate's do IO, or at least parts of it and many other organizations, including yourselves and yeah. We've had 20 million image pools, although it says 10 million plus there that's just docker hubs, not being particularly accurate. Yeah we've had 20 million image pools over the years.

A

Now, seemingly only eight stars, but maybe that's just a reflection of how people use docker hub so yeah I mean it's become huge and it's been a very successful project, but obviously with a lot of people using it. There's a lot to do. There's a lot to support.

A

This is a little funny extra slide, I added in last night. So if you look, this is in 2016, so maybe it's only two years old take that back. This is Christian at the top in March, saying who's just gotten started on doing this. This is before I knew, Christian or jet stack or anything and I happen to have gotten started on my own she's, acting secrets or cube. Acne don't go and use it now. It's been way too long, fixing small bugs and not enough time fixing the big ones, but it's just funnily enough.

A

We happen to have started work on a very similar project. At the same time, and now we work together and we're doing it all again, yeah.

A

So cert manager, on the other hand, is trying to build out more of a general-purpose certificate management platform for Cuba Nettie's. So there's lots of pain points in Cuba Nettie's. One of them definitely is TLS not just on the edge, but between services securing core system components, there's a lot to it and beyond that. If you look at a traditional organization, they have quite a lot of different policies for how you can go and get your certificates.

A

You might have to send emails to your CA team if it's worse, I've heard some real real horror stories and we want to try and improve that, and we want to do that via kubernetes, because people are using kubernetes there's a lot of primitives there, especially with all these extension, API servers and so on, and so we want to be looking at building our further to being able to do things like revocation of certificates, auditing who's, requesting them policy as to who can request them as well and different separation of concerns.

A

Who are your admins, who decide these things and who are your developers? You just need to consume certificates. What does your organization say is the max length of time that a certificate should be valid for and so on, and we're trying to really provide an abstraction over all of the wide variety of CAS out there. So we're moving beyond just let's encrypt and to support other issuers too, and this is kind of the line I think similar to our we have storage class and so on.

A

I think we can almost introduce something like a certificate class, so I want a publicly trusted certificate and the definition of that can be different depending on the interaction environment you're deployed into this is further down the right. So currently we support acne, let's encrypt, which is obviously a bit of a given. We had to support that in order to ever provide any kind of a alternative. We have CA support, so this is load up.

A

A simple sign in key pair into the API server and sir manager will manage that CA for you, you know, sign renew issue certificates and you get all the niceties they get with anything else.

A

Just with your simple signing key CA key pair and that's very useful as a building block for a number of other things, too, and as of yesterday, if anyone's got a very keen eye on the repository emerged in vote, support which was added by a contributor who spent the last couple of months putting this together and so a big THANK YOU to him vie des jardins yeah.

A

So if anyone wants to, if anyone does use hash code volt and they want to start issuing certificates from there, you can now do that and give it a go. It's early experimental support, but we're work on it, and this kind of to me shows that we can support more than just let's encrypt. This isn't just a one one controller problem and that's kind of really trying to push this general-purpose certificate management side of things. It's not just about securing your ingresses anymore, so yeah we integrate with multiple CAS.

A

We have structured, API types now, so anyone who's used, cube, leggo will know you have one way to interact with it and it's through adding an annotation. Just you know secure this, which is great, it's really nice and simple, but as soon as you need a bit more flexibility, it's not quite there. So, for example, big feature request is DNS validation, support in cube Lego for Acme and, to be honest, it was very difficult for us to add in a clean way that I would allow you to be dynamic enough between different domains.

A

To do this and there's a number of other issues, but yeah we didn't have the configuration service surface that we needed in order to add the features we wanted so for doing this yeah we get more advanced configuration and in future 2 we can actually maintain that API through conversions and so on. So when it comes to new versions of cert manager coming out, even if that breaks the API schema, we can migrate people and help them out with that.

A

So a very brief one versus the other. This is a table pulled straight from the docs configuration. You know, you've got annotations vs., CR, DS, CAS, cube Lego will only ever support me. Oh, this needs updating to include volt now, as well as I said just merged yesterday, in terms of actual support for kubernetes, we I've only validated cube, Lego up to cube Nettie's, 1.8 may or may not work beyond there. I don't stand on stage and say it because then I'll be on the hook to continue to support that further.

A

We really do believe that sir manager is where you should be moving to, and obviously, as you've seen this talk, we've tried to make that migration as seamless as possible, so it is possible for you, debugging and the likes anyone it's used. Cube Lego has probably at some point, also looked at the locks for cube Lego to work out what is going on I personally I believe you shouldn't really need to look at the locks unless something catastrophic is happening.

A

If you've got a problem with your deployment, your autoscaler, it's rare, you go and look at cube controller managers logs. You tend to just describe an object, look at the advanced api there, cuba Nettie's systems in place for this, and we want to be using those and you'll see soon. We've got much better support for things like multi-tenancy, so having various different teams, each with their own, say, acne accounts or vault servers or volt, Pai backends or whatever else yeah and one big one.

A

If anyone who's tried to use cube, Lego with anything other than a GCE or nginx ingress controller is we have pretty universal support for ingress controllers? Now there are a few optimizations in kind of that code path. So that's quite a big win. It saves having so many questions on how do I get this there and this there and we don't have to have a hard-coded list of names like we didn't keep Lego so yeah. Why do this as well? There are many ways to get certificates.

A

You can just cube CTL create secret and that's fine, but it's you know it's manual, it's it's very cumbersome for new, always a calendar event or something it's not really ideal. If you want to integrate with something like volt anyone, who's tried that you'll be and adding custom and knit containers sidecars and things and again you need to start thinking about making sure these things are up today. Making sure that they're requesting the right types of certificate and volt does have some support for this. But it's not generally like across all of these different CAS.

A

It's not generally consumable and easy to specify yeah or you can directly embed, like the CA, API, I suppose into your application, but no one really wants to be doing that I, don't think, and then yeah keep that go, keep cert manager, obviously so yeah custom resources have been very, very useful for us. They've allowed us to actually abstract away all of the details of these CAS and we provide that translation. Now you say you want a certificate that looks like this.

A

It's then up to us to go and negotiate that and do whatever else in order to get that, we can report back any problems back to you and so on.

A

By doing it and you'll see soon with some of the new resource types we've introduced, it means you can actually have a multi-tenant environment where different people requests different certificate from different places, and you can use the likes of our back to restrict, who can actually reference certain types of issuers through kind of standardized cuba, Nettie's things like namespaces, so having an issuer constrained to a single namespace, you'll, see soon and yeah.

A

By doing it this way, instead of having its embedded, you know all over your all over your system, all over your deployment, it can all be managed centrally. So now you know when you've got a problem with certificates. You do a cube. Ct I'll describe certificates, maybe with all namespaces on there, and you can get a brief overview of everything and if you've got problems, you can have Prometheus symmetric, you can have Prometheus alerts. You can have all sorts to actually alert you of these things beforehand and you can do it integrate it all.

A

With your standard alerting flow, however, you like and yeah that's with the addition of the Prometheus metrics endpoint in there, and also the likes of cube state matrix that can go and inspect the contents of an API server and expose that all as metrics, so yeah I keep on about this multi-tenancy and things so going through some of these resource types that we have. We have issuers and cluster issuers, and this to us is the representation of a CA in cuba Nettie's.

A

So by doing this, instead of previously with cube lego, if you wanted to switch from say a let's encrypt staging to let's encrypt production endpoint, you would update a flag on cube lego and, if you wanted to support both I mean you'd struggle, I think maybe some people run to instances of cube Lego it things can happen, and it's not ideal. Really, it's not designed for this sort of thing. It was very simple: we need to get certificates from, let's encrypt, for this ingress.

A

By doing it. This way to you can actually have your organization or administrators to find these issuers. So all your developers need to say is I want a certificate. They don't need to think about where that comes from be that Verisign, let's encrypt or whatever else, they say, give me a certificate and they get a signed certificate if that kind of certificate is valid or acceptable within your organizational policies. So there's a lot around policy that we're going to be building out and I think a lot of it.

A

Yeah you're, see soon and certificates then reference these issuers in order to obtain certificates from them, and that will look something like this in the simplest case with a CA issuer, so we define it. This is within a single namespace, so this issuer can only be used by other tenants in the default space, and here we're just referencing a simple signing key pair, so syrup manager will now use this key pair to sign an issue certificates when a certificate reference it.

A

There is an equivalent cluster issue, a resource type similar to how we have roles and cluster roles. You have cluster issuers which apply across the whole cluster and that's kind of because a lot of users do just want. You know to access let's and crit production, not everyone's running huge multi-tenant environments with many teams and stand on the other. So we do support that and yeah. You often see someone creating something like a let's encrypt, prod cluster issuer.

A

Here it's a names based one, but it's the same sort of thing or a let's encrypt staging one. That applies cluster wide, to kind of provide, let's encrypt as a service to the users of your cluster, and you can see here. In the acne case. We define the acne server that we want to use your email address and then a reference to where we store the private key.

A

So you can migrate your accounts across if you need to from cube Lego- and we have that documented in our migration guide and then we've only got an example of HTTP o1 here, but your CDN s, o1.

A

We can enable different kinds of providers as well, which is kind of allowing user to use HTTP validation or DNS validation, or, as you see, say, your organization's CloudFlare account. So now you don't need your developers to go and add txt records to validate things. Sir manager still does that for you, but they also never get the opportunity to even get access to those API keys because it's sensitive, you can't put your necessarily a cloud.

A

Dns administrator account in the hands of everyone within your organization, so yeah certificates, so certificates represent a certificate request within Cuba Nettie's and they reference issuers or cluster issuers in order to retrieve the certificates from that source. So that has fields like DNS names, key usages, although that's a pull request. Oh and a number of other fields, it's kind of representing the whole CSR eventually and so magically they go along convert that into an actual CS are before any validations authorizations.

A

Whatever else it is required and issued that certificate and keep it up-to-date within the bounds that you can define on that certificate or even on the issuer itself, saying renew this certificate before 30 days before it expires or 10 days or say, I want this certificate to be valid for 20 days only whatever else it is, and with the acne case, as we see in a second, we also define some extra config there on how to solve it, so whether or not I want HTTP or DNS.

A

So this here is an example referencing that CA issuer we saw earlier. So we have our secret name, so the ultimate end goal of cert manager is, you say: I want a certificate that looks like this.

A

We will ultimately give you a secret Akoo, vanetti secret resource with that signed keeper within it, so that is kind of the bound, because we offer often get people asking why my nginx controller isn't now serving with a certificate that isn't that's kind of a step beyond we deal with getting your your signed certificate into a secret, so we have a secret name field and you can see here we have a common name, DNS name, so these are subject alternative names and we can continue to add more and more to this resource and there's quite a few pull requests out there to do just that, and here we have another example very similar of requesting certificate from an acne issuer.

A

Now for acne we do need a few extra bits of configuration. So right now that's defined in line on the certificate block, and this will tell it to go and validate these two domains by using the ingress class nginx, so it'll go and create ingress resources similar to how cube Lego did in order to solve that validation, and we can configure it all here in future. If this becomes cumbersome for users to actually define, we can actually come up with ways to automatically default.

A

This kind of configuration we are going for a flexible, API first and then from there provide optimizations and things on top to make this simpler and easier to consume, which you'll see surely yeah on my next slide. So simplifying all of this, because it's nice and everything it's good to have this flexibility, but it can become painful too I mean you've, got the same list of domains twice. It makes sense from a technical standpoint, but from a developer's standpoint they just had to write their domains twice and that's not ideal.

A

So we want to simplify that complexity, and this is was really a big key thing that had to happen and in order to migrate these key blogger users, because they're not used to having to define all of this different type of configuration. So we built this little part component of cert manager called ingress shim. This was just before cube con North America in December, because many users do just want a let's encrypt certificate. They we don't want to alienate these users. If we do, people will stay with cube.

A

Lego will have to continue maintaining and putting new features into that. If they do that, so we need to provide a seamless transition across and actually there is a large body of users who just want secure ingresses. They don't want all of these extra features that I'm doubting today so yeah.

A

We have an extra controller that watches all ingress resources, similar to how cube Lego did and when it sees certain annotations on those ingress resources, it will go along and go and create a corresponding certificate resource, so we're not reinventing cert manager again just to do this. One thing: it's a simple kind of controller pattern we watch ingress is we go and create certificates, and if you are great date, your ingress will it go and update that certificate and then further down the line.

A

Cert manager will then pick up that configuration and maybe reissue your certificate or whatever is appropriate. So you no longer have to think about how you create those certificate resources. You create your ingress and say I just want this signed, yeah, secure, I should say and yeah it's nice easy to use, and it simplifies things you're no longer having to explain this and when you do need to debug, you can fall back to that. You can just describe your resources still and see the events coming through and yeah.

A

It means we can do this, so this neq, blogger user may have seen it means we can just add this annotation on to our standard in grocery source and everything else happens see shortly and yeah. Sorry, this is what gets created off the back of it so from here. This is all your user needs to deal with in the API server. This is what gets created and that can be automatically configured with defaults as configured by your administrator or by users.

A

So I've talked a lot. Someone told me: I should have done the demo at the beginning. Who knows we'll, never know so I'm going to demonstrate how you can actually use all of this to get wild card certificates, which is a new feature in let's encrypt you can get wild card certificates for your ingress resources without any extra configuration, it's just the same as you did with cube Lego. So this is demo gods. I'm doing this live on conference Wi-Fi, so first things first. In order to do this, I am going to be.

A

Let me know if this is a bit small in order to do this, because this is a part of our alpha release, which would be graduating very soon of not 0.3. I need to install sir manager first of all, using the hell chart in the repository, so here I do a helm using helm. So, whilst that's going here, I've done a helm installed, I've specified some defaults here as well. So I've said use the issue a let's encrypt prod as a default.

A

So if I don't know what issue are you if you haven't specified explicitly which issuer to use on an ingress use this one name? Let's encrypt prod, it's a cluster issuer and also use the DNS. Oh one challenge provider and my configured CloudFlare account in order to do that and we'll see that sure. Just here we have our issuer named.

A

Let cluster issuer named, let's encrypt prod and here I've configured an ACME server, email address, a private key and I've enabled HTTP o1 and configured one single DNS, o1 provider, which is my cloud for our account, and you see here we give it a name and that's the same name as we've configured as a default. So everything should just pick up and yeah. My API key has already been creating the API, so I can go along, keep CTL, create, cube, Khan, / issuer.

A

Wi-Fi is holding up and now I can describe my cluster issuers and we can see here. The Acme account was registered with the acne server, so we have a standard easy way to D to debug. What's going on and get an idea of what's happening here, this can improve as well the way that we render this information with some new features coming in Cuba, Nettie's, 111 and yeah won't go too much into Excel run out of time.

A

So we've got that now, I'm going to deploy my little demo application, which is nginx in case anyone's never heard of it, so keep CTL create once again deployment great just make sure that's running excellent and actually, in the background, although I haven't created my ingress yet so that's up and running great and I now want to go and create my ingress, and here I've got a domain name. Kubernetes.

A

Today, I went on a buying spree of all cube entities, rated things, so we're going to get a wildcard certificate for this I will quickly check I'm not going to cheat and make sure I haven't already got a wildcard certificate for this I was testing this all out this morning. No good cool, so I'm now going to go ahead and create this ingress. And what we'll see in the background happening.

A

What we should hopefully see is a certificate resource being created four seconds ago, so that doesn't mean our certificate actually exists. Yet, if I do a get secret again, it's gonna Oh wrong. Namespace.

A

It's already issued brilliant okay. um So if I do here, keep CTO describe certs, you can see yeah. Oh yes, sorry! No, that is correct. I probably want to change this. We can see here. It's gone along. It's created a new order without the Acme server, it's attempted to validate that it's validated it. It's then gone along and issued the certificate obtained it and put it into the secret resource.

A

So it's kind of uneventful really this demo, because it just worked, but that's good right and you can see here- we've got status conditions that say, you know type ready true. Admittedly, this is inverted. This is, as I said, the alpha release, I. Think I'm, probably gonna change it.

A

As long as it looks like it's failed when it doesn't type validation failed is false, so it means it didn't fail, made sense in my mind at the time, but the nice messages here it does say order validated and we go along now, and you should also all be able to do this yourself cube con cuba, Nettie's dot today and there we go and we can quickly check this out and we'll see. If you can't see I can't make this any bigger.

A

We have a wildcard certificate and that's valid until the first of August, which is around ninety days and that's about it. This is a very uneventful demo because it worked, but that's exactly what we wanted and.

A

Thank you and yeah and, as you can see, it's just using exactly the same configuration surface that we used with cube Lego, but with all these new features- and this obviously works. If I didn't have a asterisk, we could have just gone for a standard domain that works just fine. We can have multiple drains, we could have whatever else, and if we adjust this in Gresham this one little component of SAP manager will go and update that certificate resource and everything else kicks off.

A

So great we have new features, they're easy to use and it works brilliant, no need to get back to my slides cool. So what's next insert manager I'm wrapping up now, first of all, defining policy on these issuers, so there anyone can go and get a certificate for any domain. In my cloud, flower account I want to be able to find policy. So who can do it? That's the first thing and also kind of what has to be on a certain resource.

A

What ask what a certificate has to have so maybe a required key usage or a required expiry date, or something like that. Advanced resource validation, so this is one thing: people if anyone uses any kind of CID operator they might have seen and if you go and create an invalid resource. Sometimes bad things can happen.

A

We want this to feel very native, so we've got some work to actually add a validating webhook and if anyone's heard of them to make that just a seamless user experience things like a slash, approve endpoint, so you can then start using our back to constrain, who can approve certificate requests within your organization and we can then set up things like auto approvers, if required, there's be talk of a gr, PC issuer interface. So, right now, as I say, we support CAS, the CA issuer.

A

Acme and volt people sometimes have internal systems for this sort of thing. Maybe you want to send off an email, because you want to use all these nice stuff, but with a team of people behind it. I, don't want to do that, but that's exactly it maybe having some kind of a pluggable issuer interface, so someone else can bring it along and they get all the other features of cert manager with their own see a potential integration with Sto, so they have their own component.

A

Sdo CA, which issue certificates to secure mesh, there's looking at it I think so. Managers well poised to actually be a drop-in replacement for that component potentially, and that means you can actually see well get better insight into what's going on within the machine. Within that see, a standardized there's been some talk about using it for renewing cube atoms self, hosted TLS certificates, I've not tested it yet, which is why it's on the roadmap, but I think it should just work using the CA issuer.

A

So you can keep yourself hosted, certificates up to date and also integration with kubernetes system components like webhooks, which creates a bootstrap problem for point two but yeah, so we can actually start securing other kubernetes components for other extension developers too, and volt issuers supports being crossed off because, as I said, it was merged yesterday. So great so yeah in summary, we've had 38 contributors is becoming a real community project. I'm really proud of that.

A

With eight hundred and thirty-four commits I forgot to put on here, we hit a thousand stars that was a couple of days ago, just before cube con and we're up to eleven releases and I'll be doing another release later today, with the volt support shameless plug. We are obviously hiring everybody else here. So if you're interested in the sort of thing you know she'll come and say hello yeah. If anyone there's been some talk about having a bit of an informal development meeting at some point today.

A

So if anyone is interested, you want you've got more questions like roadmap related or you want to get involved in a particular area or if you've used, sir manager and something's painful kind of express that shout at me, yeah if you want to hang around at the end, I'm thinking maybe go over to some of the tables over there. So we can have a bit of a discussion about it. Maybe move it to lunch. Let's discuss out of interest come about finish. Who is who's already used? Cert manager, cool nice awesome?

A

Well, thank you for coming along I hope. You learned what you wanted to learn from this talk this session and yeah. Thank you, I, don't know if I've got time for questions.

A

Looks like I've got five and a half minutes for questions so yeah. Anyone.

A

I'll repeat out.

A

If you cuban at easter net today domain, you know that's one that I owns in my cloud. Flower account. It is publicly accessible, though yeah. Is that what you mean.

A

A

So the question there is: is it possible to use this for private domains so with the DNS validation provided your route DNS zone is publicly available or a root DNS zone is publicly available and authority. It should work just fine. You should be able to use a DNS, the DNS challenge mechanism, to do that. They'll update your public name servers and, let's encrypt, will then be able to hit that and retrieve the txt record. In order to validate it, you wouldn't be able to use HTTP validation, obviously, because these are private yeah.

A

Let me come and chat and we can try and dig into what's up there.

A

Good question: what DNS providers do we support? We've got about five or six at the minute, we're basically cherry-picking them from the xeno lego library for an audience looked into it. So it's pretty easy for us to add. Dns providers in its Google, Cloud, DNS, CloudFlare, I, think Akamai the other day route. 53 we've got a nice documentation site on here which should go into details but yeah, so we have a list of them. It's pretty easy for us to add new ones, though so create an issue. We've got about five more in the works.

A

Good question: so, if someone goes along and modifies that TLS secret either removing a key or maybe replacing with a new one, sir manager will then observe that change what it does every time it observes any kind of a change. Is it just validates? The certificate is valid for the desired certificate state and if it isn't, it will then trigger the reissuance flow and so yeah. It will switch back basically.

A

Yeah, so we don't have a huge number of extensions on there right now. Ultimately, though, there is one point in the code where we take a certificate spec and convert it into a CSR, so the addition of extra fields is fairly simple.

A

Yeah I can't remember if I repeated the question, what other fields do we support was that question.

A

Yeah so, as I said, can we do SSL offloading with? This? Was the question as I said, before kind of we basically put the secret, thus, if cut into a secret resource, so if you're using the GCE ingress controller, for example, that will take that secret resource and it will go and send it to Google Cloud, and then you get your SSL offload, so we don't deal with actually encrypting your traffic. We deal with getting certificates, encrypting your traffic or how you consume.

A

Those certificates is kind of a separate concern, so your ingress controllers concern really for the ingress case. So, yes, we do support SSL offload, but you will need to write something or your ingress controller needs to go and take that secret and put it in the appropriate place.

B

A

Okay, so I think the questionnaire is with cube Lego. It was often recreating a private key for every time it renewed or issued a certificate. So I didn't actually realize that in cube, Lego first of all, I'm also not going to fix it, but certain managers, I do know, does if there's already a TLS key in that secret, it will use that to go, and you know sign the CSR. So you should be happy they're there. Any other questions.

A

Yes, so the question there is once that secret updates: do we have a best practice like a recommended method for actually kicking your pod to say you know sick huh for reload, it's a good question. I would say no, whilst I'm on stage I don't have a best practice for it. A lot of people have seizures with ingress controllers, which are using some kind of a watch, and they do it.

A

You do have now shared hidden namespaces, which is new in Cuba Nettie's, and you could probably use that just have a little watcher that does something in fires. A signal, yeah I, don't have a github link to send you I'm, afraid yeah I think we have probably out of time now, but yeah I'll be around here, and if anyone does want to discuss future plans or get involved with development, then come say hello and we'll work out a time.