Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2021, 29 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Unraveling the Mystery of Versioning CRDs - Harsh Thakur, Civo

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Unraveling the Mystery of Versioning CRDs - Harsh Thakur, Civo

CRDs have given birth to a lot of projects by extending Kubernetes. As these projects grow, the definitions of the resource start to evolve and maybe completely change, thus requiring versioning of CRDs. Versioning of CRDs can be tough as the end users need to be provided with seamless upgrades, zero downtime and backwards compatibility. Have you asked yourself questions like: “Do I need a conversion webhook if I am just changing the validations?” , “How do I migrate the existing CRs to the new version?” .This talk aims to explain the CRD lifecycle and help users quickly catch up with the changes happening in the CRD ecosystem.

A

Hello, everyone, my name is harsh and today I'll be talking about city version.

A

Little bit about me, I'm a developer civil cloud, which is a blazingly, fast, managed humanities provider. I'm also maintainer at open ebs, which is the cncf sandbox storage project. I'm also a hashicorp ambassador, and I really love playing chess.

A

So getting right into it.

A

Cid versioning is a lot like any other api versioning conventions, so you typically do not want to add a required field in the same version or change the name and types of the existing fields, because that would break the end user specs and the reasons for wanting to change names and types of fields are to simplify user inputs for better validations or simply because you want to evolve to the new api conventions.

A

For example, uh kubernetes sub resource status has evolved from using faces to using conditions.

A

There are times when you don't really need to upgrade as well and that's when you want to add optional fields, as that would wouldn't break the end user, specs and uh change validations. And by that I mean validations which don't break the existing defaults of the specs.

A

So when it comes to cid versioning, there are two views for it as an end user. We are always exposed to the server version, and server versions can be more than one, whereas the stored version is what's really stored uh in hcd, and it can only be one so. The api can read the stored version uh from at cd and represent it in multiple versions to us, and we look at how it does that.

A

But essentially all you need to know is that store version can always only be one.

A

So when it comes to conversion strategy, there are two of them.

A

The first one is really a last mile change where you want to just bump up the api version, and there are no schema changes in your uh crds.

A

The second one is what we really uh think about when we think about convert conversion strategy, and we look at uh how it works.

A

So I have a little demo and in this demo, I'll have two versions of v1 and v2 and in the beginning, v1 would be the stored version and when user requests v2 uh view of the object. What happens is the api reads: the object from at cd in v1?

A

It has a conversion web book which we have written and I'll show that in a moment, but the conversion workbook would then throw back the object to api in v2, as it has some conversion functions and the api then returns back uh output to the user.

A

Now, what I've done is I've scaffolded this project using cube builder, I even played, along with the basic example of guest book.

A

All I've done is I've added this field, full name in the v1 of the spec and in vito of the spec I've added uh I've split the field into full name and last name, first name and last name other than that. What I've done is I've added this uh cube builder marker, which I realized I'm on the wrong branch. So let me switch up to v1 and I've added this uh q builder marker, which says that v1 is supposed to be the stored version.

A

And other than that, I have the conversion web book in place and we look at what this uh conversion package does in a moment. But all I really want to show at this moment is uh the conversion functions where I read the fields, uh make it into one field and then assign the object metadata and that's typically how conversion would look like, and in case you have functions which lose data between versions.

A

You would want to store them into the annotations of the field of the cr and that's because uh the versioning follows the concept of lossless conversion, where you want the users to roll back to the older version as well, because when we introduce a new version, we set the old version as a stored version, as that would give time to the users to upgrade to the new version.

A

And that's the reason uh we have assigned the v1 as the stored version. Whilst v2 is not, and we look at that in cr and cid and hope it will make a lot more sense. Then we just words.

A

So if you look at this.

A

We have the v1 and it has served and storage is true, and this is how you typically introduce a new version. You set the older version to be the stored version and the new version is just introduced and without adding any markers right by default. It's it's set as the sew version.

A

So now I'm going to create a simple example.

A

So, as an end user, I can create uh right now: objects in v1 and v2 and we'll do that. So in the v1. I have a simple example: uh guess book sample and the I have given I'm giving it the older field or full name as it is the v1 spec.

A

And ksq uh alias for cube, cutting.

A

So we've created this uh example. Now, let's try to view it.

A

A

So we can see that uh the object is written in v2 instead of v1, even though the stored version is v1 and that's because uh clients like cube cuddle, always prefer the newer version higher version.

A

So that's the reason we got back the object in v2, but if we really want to look at how the object looks in v1, we can still do that.

A

So now you can see the object in v1 and this really encourages clients to start using v2 as they see the object in v2 always.

A

And now uh we look at what what I meant previously by the conversion, what the conversion package previously did.

A

So uh in the beginning, when the conversion package wasn't there, we had to write conversion functions between each of the ap api versions and what that meant is suppose. We had three uh c cd versions: v1 v2 and v3.

A

uh We had to write conversion function for converting v1 to v2 and v3 v2 to v1 and v3 and v3 to v1 and v2, and that's like 6 conversion functions. But with this hub and spoke model, what we can do is we can have a centrally placed hub version and any any conversion that needs to happen. It needs to happen via this hub version.

A

So in that previous example, I spoke about, uh if v2 wants to convert itself to v3, it will first have to convert itself to v1 and then to v3.

A

So it only means that you have to write uh four conversion functions in this case, because uh v1 needs to convert this into v2 and vt, and v2 and v3 convert themselves to v1.

A

So for the next part, what we've just seen was the conversion uh happening from v1 vo to v2, but there are existing objects like in the one I just created and was created in v1, so we need to bump that up to v2, as well as as that's a cr, not a crd, uh and it's analogous to classes and objects in object-oriented programming.

A

So in order to do that, this is component known as storage version migrator, which does that for us it probes the api discovery document every 10 minutes and looks at the new desired storage version and bumps up the existing objects to that version, or you could manually just patch all the objects and edit the specs or something and do it or have a custom program to do it.

A

So I'm just going to do that right. Now, I'm going to switch up uh to v2 and I'm going to install and in v2 what I've done is. Oh I've just set the new v2 as the stored version.

A

And I'll show you that.

A

So, as we can see, v1 is still being served, but now the story is set to false and v2 is storage version now and clients can now create objects, and so they can create objects in v1 and v2, but uh in the end of day uh objects inside uh hd would be stored in v2 and to check that storage version. Migrator has the component known, as uh has a cr known as storage version migration, and we can take a look at that.

A

Sorry, storage version, migrations and if I can find facebook in here, we can see that the status is succeeded. So this means that all the objects in our in that cd are now migrated to e2.

A

So now that we have done that, the next part, what we'll be doing is, uh since uh all the objects have been migrated to v2 and all the new objects will also be in v2. Since that's the new stored version, uh we can now deprecate the. uh We cannot deprecate v1.

A

And in order to do that.

A

I've done is I've added this uh marker.

A

Answer version which basically said served equals false and we'll look at another crd as well, uh and before I uh jump ahead, I did forget to mention that this file is also scaffolded using cube builder and in order to declare that v1 is the hub version.

A

uh I just had this one liner, where I mentioned the type which is guest book, which is the v1 or v1 type of v1, of the object and hub is just a interface which we'll be using in order for the conversion package to work so that we can do something like this.

A

And now that we have installed, let's look at our crd now.

A

Did I not install it? Oh yeah, I have not installed.

A

Now, let's view that.

A

Yep we can see that served as falls and storage was false and uh served and storage are true. Now what this means is that we can no longer view our objects uh in v1 uh and I'll show that so by default we again get v2. As always, a higher version is preferred, but when we try to view it in v1, it just doesn't work which makes sense, because now we have replicated the version.

A

We have followed this whole process of uh deprecating, a version where we introduced uh the newer version to be served. Whilst the older version is still the storage version, and then we set the new version to be the storage version and in the end we stopped serving the old version. Now we can also proceed and uh stop serving in the next release. We can stop so uh we can remove it from the crd itself and also drop the conversion by book.

A

Support or if you choose to you, can still keep it and uh clients wouldn't have to have this intermediate uh conversion.

A

When you try to do releases, uh that's because of the hub and spoke model, so they can directly convert from v1 to v3 instead of having to first upgrade to v2 and then do v3.

A

I've provided all the references uh and a lot of the scaffolding, I've I'll document that inside the github repo and uh that's basically what I follow from cube builder, docs uh other than that there's official documentation about crd versioning in kubernetes, docs and there's the github repo for the demo and I've also linked the storage version migrator component and for not non-cube, builder or operator sdk projects uh which may not even use a controller runtime. uh I have provided an example, or someone has already made of about how the conversion works in this older projects.

A

You can reach out to me at twitter, linkedin, medium or my email and I'll be glad to answer any questions or discuss anything more. Thank you.