Cloud Native Computing Foundation Crossplane Community Day EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: The How & Why for Building Crossplane Providers - Krish Chowdhary, Red Hat

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

The How & Why for Building Crossplane Providers - Krish Chowdhary, Red Hat

Provisioning and managing cloud resources like infrastructure, services, and applications are one of the core features offered by Crossplane. We achieve this goal by creating independent Providers which expose Custom Resource Definitions in a Kubernetes cluster. Although the development of these Providers is community-driven, there are overarching design patterns that exist in all the Providers. In this talk, we will be going over the fundamentals of Crossplane Providers, the architecture, and how to contribute. Additionally, we will talk about the how and why for creating a Provider, what drives the use case, and when is it a good idea to create a Provider as opposed to other options like Helm Charts, Operators, or Kustomize resources. We will wrap up the talk by bootstrapping the development of a new Provider, as well as a small demo for creating a new resource.

A

The how and why for building crossplan providers, hello, everyone, my name is krish and today we're going to be going over uh how and and why you might want to build your own crossland provider, but first things. First, a little bit about me, I'm a software engineer at red hat in the office of the cto and I'm a maintainer on the provider aws and provider in cluster projects under crossplane.

A

I'm also a member of the kubernetes 6 storage, where I actively contribute to a few projects. My twitter is chris ciao underscore, and my github is chris chow.

A

uh So in this talk, we're going to go over the what where, when why and who for crossplan providers as well as in the second part, the how for how you can create a provider. This will include talking about the main components and then eventually live coding, a new feature for the provider in cluster.

A

So, let's start with what so, let's quickly go over, what crossplane is so the crossband project allows you to provision cloud resources from within your kubernetes. Cluster cross button is responsible for managing the entire life cycle of these resources, including update, delete, create and observe on the note, observing cross. Plane providers expose information about these resources on an ongoing basis through kubernetes fields on the crs, as well as events so from the cross plane. Docs.

A

We have that providers are packages that enable crossplane to provision infrastructure on an external service. They bring in crds that map one to one with external infrastructure resources, as well as the controllers to manage these resources.

A

So that's a good definition, but what are providers really right? So essentially, providers are similar to operators, but you utilize, the cross plane operator for installation and management uh and later on, abstraction uh providers use a lot of the same tooling as operators such as cube builder control, runtime, controller, runtime and controller tools.

A

The main difference, though, is that providers are really designed to reference some external resource. It's responsible for mapping one-to-one with some other resource and representing that resource. Well, so, for a developer, the really nice thing about providers is that you don't have to worry about things like our back deployment and bootstrapping. Since there's already solid tooling that exists to handle those uh so the where so, where are providers located?

A

All crossbend providers are open, source and available on github under usually either the crossplane or crosspin control organizations.

A

Examples are the provider sql which orchestrates sql servers by creating users, grants and roles the provider. Helm is similar so that lets you manage and deploy helm charts using custom resources, namely a release resource.

A

Lastly, the provider aws is responsible for provisioning resources on aws, such as databases on rds or even kubernetes clusters using eks, and it's really responsible for representing those infrastructure resources on aws, so the when so when does it make sense to create a provider? Well, as I mentioned, providers are designed to interface with some external api and manage the crud lifecycle for those resources.

A

So the focus then, is really on high fidelity resources. Right. If you have some external resources that you can represent with high fidelity on a granular level, it makes sense to create a provider. If you're looking to create your own abstractions, then you should probably opt to use an operator as opposed to a provider.

A

Another reason why you might want to use a provider is, if you want to use compositions, so the so crossplan has a powerful composition engine that allows you to combine several resources and the best way right now to utilize. This is to create a provider that manages your resources, so why? uh Why should you creator contribute to provider? uh Crosspin is a cncf sandbox project, hopefully soon will be incubating, and many of the providers are open source and anybody can contribute their shared development and maintenance of common resources.

A

Every organization shouldn't have to write their own version of the provider aws. We should be able to all contribute, as a community, to one project that we can all benefit from cloud. Vendors can also expose their apis in kubernetes through a common interface of a cross, plane provider and, lastly, you should contribute or create a provider because of the streamlined development and consumption process, as I mentioned before, crossfit helps a lot by handling a lot of the messy parts so who who is really creating and maintaining these providers today?

A

Well, there's three main groups from my perspective: there's the community organizations and vendors.

A

So from the vendor side, there's engineers from organizations like alibaba, ibm, aws and equinix that contribute to the development of their respective providers, there's also contributors and engineers from organizations like upbound, red hat, squiz and accenture that are contributing to the active development and maintenance of providers and, lastly, with any open source project, there's a lot of developers in the open source community who contribute code back to providers and the main crossplane project.

A

So let's get to the interesting part: how how can we create a crossplan provider? Well, the best way right now is to use the provider template, which is a template that helps you bootstrap your development and it exposes some sample resources and the basic structure of a controller.

A

So the best way right now is to hit the little green button that you see called use. This template and that'll take you to this screen, which allows you to define the repository name and owner for your provider uh and just like that, you have your provider, and so now we're gonna go through a little bit more in depth into the provider in cluster that we just created, uh followed by a live coding. Demo of us creating a redis managed resource in the provider, written cluster.

A

So this is our freshly created and cloned version of the provider in cluster, based on the current structure of the provider template. So let's dig a little deeper into the various components of the provider and cluster and break down what they mean and and how they affect our development.

A

So from the top down. Let's take a look at the apis folder. First, the api folder contains first, the generate.go file, which describes the workflow for how we generate crds, along with necessary files for kubernetes and crossplane.

A

The template.go is similar. It describes uh all of the schemes which we want to add uh into our kubernetes runtime. So in this example, we have two schemes here. First, there's template v1 alpha 1 and sample v1 alpha 1..

A

So let's take a look at sample v1 alpha 1., so within the sample folder there is another subfolder called view on alpha1, and this contains a doc.go file which specifies the group name and version name for this api group. Within this we describe our uh we describe our sample resource which, which is described in the types.go file.

A

The name of this file doesn't matter, but, more importantly, it's important that we have the necessary annotations, as well as the structs, with our type meta and object meta. This is the actual api api type that this provider exposes.

A

There's other generated files here, along with the register.go, which describes the type metadata, as well as the initial scheme, builder registration.

A

Next up, let's take a look at the command directory. This is the main dot go file. That's actually run inside of our docker image when the provider is created.

A

This includes uh all of the command line, argument parsing, as well as the initial setup. So here we can see, we initially run uh our apis when we add them all to the scheme, and we also set up all of our controllers.

A

Aside from that in here, we describe our manager as well as the signal handler within the example directory. We have examples of the different custom resources that our provider exposes. This is a good practice, because new users can then see what the different resources are, as well as examples that they can run directly.

A

Within the internal directory, we're able to see the different controllers for our provider under some projects, you may see that instead of internal, this folder is called pkg for package, but in the provider template it's currently referred to as internal.

A

So within this we have our template.go file, which contains the main setup function for all of our controllers. So, for example, we iterate over our config setup function and our my type setup function. So this sets up both controllers, as I previously mentioned within, let's say the config one, for example, we'll see the setup function which describes how we initialize the controller that reconciles provider configs uh and so in this example. There isn't much.

A

But if we take a look at uh my my type uh we'll see, there is a setup function as well as a connector which describes how we produce an external client when we initially want to set up this controller.

A

And additionally, we have our external struct, which has the create, update and delete methods uh for our external resource, and the observe method is responsible for seeing the or examining the current state of our managed resource.

A

The observe function returns a managed to external observation, which contains references if the resource exists, if it's up to date as well as connection details, this is vital because if at any point a resource no longer exists or it isn't up to date, we'll run the create or update process as needed.

A

So, for example, for create we'll see that it creates our resource and returns successfully without an error if the if the resource was created, uh the same goes for update and delete uh when a resource is deleted, we'll call delete and um the reconciler will continue to repeat for the managed resource until the observe method returns. Resource exists, equals false.

A

There's also a my type underscore test file, which contains all of our tests for the observe, connect, create, update and delete functions.

A

Next up we have the package directory, which contains our crds that our provider exposes so in this example, there's not many crds, but in another provider such as the aws provider, that has dozens of resources, you'll see quite a few crds here.

A

So that's all the main files that you really need to look at as a developer.

A

Everything else is handled opaquely by crossplane for you, and it's really that simple to get started in the first part of this talk, we explored what crossplan providers are as well as when it makes sense to create a provider and how to actually create a provider in this part of the talk, we're going to go through the process of actually defining a new managed resource for the provider and cluster at a high level. The goal of the provider in cluster is to define resources in cluster that follow the same interface as other cross.

A

Pin providers such as the provider, aws, gcp and azure, so the first part of creating a new manage resource is defining the api. We do this inside of the api's directory. More specifically, we're going to create a new folder for cache v1 alpha 1.. Inside of this, we define a doc.go file which contains the group name, which is cache.incluster.crossplain.io as well as the version which is the one alpha one we're also going to scaffold out an empty resource for redis, which contains a spec and status struct.

A

At this point, this is all empty, but we're going to come back to this in a minute next up we're going to define our register.go function, which defines all of our type metadata, and it also has an init function which registers this resource, our redis resource. With the scheme builder.

A

This allows us to run our code generation which creates our deep copy, managed and managed list files, and so these are used internally by kubernetes and crossplane when we run our code generation. It also creates our crd for the redis resource at this point. There's something exciting here, but as we build this out slowly, this will get filled in with all of our different fields.

A

uh Now, let's actually build our uh controller out a little bit.

A

So at this point our controller doesn't really have any logic yet, but we've set it up within this regis.go file and we can see that we have a connector struct here which describes how we connect to whatever api we're using, in our case, we're just connecting to the kubernetes api server, and we also define our reconciler, which has all of our crud operations such as observe, create update and delete.

A

At this point, we can also add the controller to the setup function and what this does is it sets up or creates all the controllers, with the loggers and rate limiters when the the pod is started for the in cluster provider?

A

Next, we can start to scaffold out the controller. So before you actually write your logic, it's important to define. You know the different uh functions that your client needs to have, and so in our case our client will have a creator update function as well as parse input, secret and two functions for deleting the deployment and redis service.

A

At this point, these methods aren't defined yet, but, as we build this out, we'll slowly see that things come together.

A

We're also going to define four functions here for creating our kubernetes resources for our redis deployment containers, as well as the service itself.

A

So now we can head back to the api and we can see that we've added a few new fields here. So at this point, for simplicity's sake, we have three fields, namely password secret, ref, config variables and memory limit memory limit states, the maximum amount of memory that we want to make available to the redis pod in our deployment.

A

A config variable allows the users to inject arbitrary environment variables into the redis instance and, lastly, password secret ref references, the secret, uh which will be which will contain the password for our redis instance. uh If no reference is given here, then we will expose redis without a password.

A

Once we define these attributes, we're going to run our code generation again, and this will update deep copy as well as our crt.

A

Now we can actually start the process of implementing our client. So here uh we define parse input secret uh as a function which gets our secret and then extracts the password data from it.

A

Delete redis deployment just validates that the deployment exists, and then it deletes it. The same thing exists. The same thing happens for our delete: redis service function.

A

Next, up uh inside of our insider make redis deployment, we define our apps v1 deployment resource, which contains the object, meta and spec for our deployment.

A

The containers are defined in another function, down below which first sets up all of our environment variables, as well as our memory limit, and then we define the pod as a having container, which contains the bitnami redis image, uh as well as which exp, which exposes uh the default redis port 6379 uh with the maximum memory and environment variables.

A

We also have a liveliness pope to validate when the pod or container is actually started with our client defined. We can actually start the process of now building out the controller itself, and so here we can see that we've defined our connector function as extracting the provider config reference getting the cube config from there and creating a new client with that specific cube config.

A

Our reconciler then has a reference to the redis client and it's responsible for observing the current state of our managed resource. So, namely we check if the deployment exists, if the deployment is ready, if it's currently in the progress of scaling up and we also validate if the service exists, and if any of these resources don't exist, the controller will fail and and it'll get put back in the retry loop for our reconciler.

A

If everything exists, and it looks good, we'll return successfully, that the resource exists and it's up to date, we'll also expose the ip address of redis at a specific key in our secret, namely the endpoint key. The create function actually runs the process of creating our redis instance, so this means deploying the deployment and service resources.

A

At the end, once everything is created in our secret, we will expose the port and password. So in our case the port is always 6379 and the password depends on the input secret if it exists.

A

uh Update in in the redis resource at this point is a no up and delete simply just deletes the deployment in the service.

A

So with that our managed resource is implemented and we can define some examples in the examples cache folder, and so this will have our sample readiness resource with the password reference, as well as our provider config. So just like that it was that easy to create a managed resource.

A

All in all, this process took me about an hour or less than an hour, and we have a v1 alpha, 1 release of red sn cluster, so we can add some additional features, there's still quite a bit of work to do such as a redis cluster or additional testing for redis, because at this point we haven't written any tests, but we have enough now, where somebody can? You know, provision redis and cluster using this provider.

A

So that concludes this presentation. Thank you for watching and if you have any questions, feel free to ping me on slack or add me on any of their socials.

A

My linkedin is chris ciao, which is the same as my github and my my twitter is chris chow underscore thank you for your time and uh have a good kubecon.