Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2022, 1 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitOpsify Everything: When Crossplane Meets Argo CD - Ying Mo & Ken Murray, IBM

Description

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

GitOpsify Everything: When Crossplane Meets Argo CD - Ying Mo & Ken Murray, IBM

Argo CD is a popular CNCF incubating project that implements GitOps on Kubernetes to drive continuous delivery of Cloud Native applications. Using Git at its core, it's been widely adopted across the industry. However, real-world organizations may still have many traditional IT systems, they may have invested heavily on management automation for these systems. How can they benefit from GitOps? This session will share the magic to combine Argo CD and existing non cloud native IT automation assets by crafting a bridge using Crossplane, another popular CNCF incubating project. It works seamlessly with existing automation technologies such as Ansible and Terraform, with a large user base and mature eco-systems. This empowers you to GitOpsify everything, cloud native or non cloud native, to manage hybrid technologies using GitOps consistently. The session will also share some best practices and lessons learned that you may consider when you start the GitOps transition with your IT systems.

A

Hi, my name is karen murray, I'm a ci city engineer at ibm, working within the cloud pack for watson, ai ops team. My area of expertise is in the adoption of git ops in the development and operation of deployment pipelines.

A

Today, I'm joined with my colleague ying mo to present on the topic of git ups and crossplane, and its application across both traditional non-cloud native it environments and cloud native applications.

A

So an overview of the agenda um first stage here, I'm going to talk about the adoption of crossplane and argo cd um in ci cd pipelines, I'm going to give a generic view of a deployment pipeline as it exists today and and and how we use our deployment pipelines in the uh infrastructure, provisioning and life cycle management, but also in application deployments.

A

I'm going to talk about cross plane as an abstraction platform for infrastructure provisioning, so basically using githubs to both manage our infrastructure and also application deployments.

A

uh Yingmo will then discuss ongoing development work in the area of ansible cross plane provider for more traditional I.t systems. um He'll discuss the the provider in terms of of how it works and how people can use it, how it can be adopted and and then finally, a deep dive across the and across the ansible provider. Runtime life cycle, some best practices in the in the use of of the provider and then finally, comparing the the answer provider with the uh the ansible operator.

A

um So if we just look at a typical deployment pipeline, a very generic view, um you typically would have a build here coming in on the left-hand side of the plumbing pipeline, and the first stage would typically be the acquisition of infrastructure onto which you would run your deployments.

A

um The second stage would typically be some kind of configuration you might want to do on that infrastructure. So in terms of a kubernetes cluster, for example, you might want to set up some storage or some prerequisites before you do your application deployment um application deployment. Then you would deploy your your software components onto that infrastructure.

A

You may want to do some post deploy checks to ensure that the you know your applications have got are deployed to a sufficient level before you would trigger some downstream test pipelines.

A

You may want to export some deployment logs um trigger then downstream test pipelines export those test pipeline results to some location that could be. You know downloaded by individuals at a later point and then finally, you may want to do some notification out to the wider teams that you know your deployments have have completed the stages in such a pipeline. That would be typically, as I call it get ops ready would be those that would be more typical um to application deployment and- and you know.

B

A

Application of kubernetes manifests right on onto a cluster. So in terms of this generic pipeline view, here, we could look at perhaps configuration application deployment be more critical to the typical. You know view of get ops and the application of the adopts infrastructure management in particular would would not be applicable as it stands today with to git ups.

A

Infrastructure management typically involves the use of custom infrastructure, apis, okay, so provisioning of infrastructure. You would typically point to that infrastructure providers apis to create those those those environments which typically don't lend themselves quite well to the adoption of git ops, um so infrastructure, provisioning and lifecycle management. Then, if we just focus a little bit in on the infrastructure section of our pipeline acquiring the infrastructure and today what we do in our pipelines is that we maintain a pool of kubernetes clusters, so this pool is available for deployment at any time.

A

It speeds up the our complete deployment life cycle in a sense that we we have clusters prepared before they're actually needed for deployment. So we maintain a set of pre-provision clusters in a pool when we request a cluster from our deployment pipeline. We go to our pool manager and it basically picks a cluster available for the deployment pipeline.

A

Our pool manager then also tops up that pool, so it communicates via custom apis to ibm's infrastructure provider to to create um clusters, so in this case we're using red hat um openshift clusters, um so we we communicate with our infrastructure provider over their apis to to create a cluster at a particular config level um and it's placed into our pool ready for ready for deployment at a later stage.

A

um So, as I said, the pool manager is is set at a provisioned level right. So we have a set level um of x number of clusters we need um for our deployment pipeline. um It supports variation in cluster specifications right so different um specifications of our worker nodes, for example. um You know cpu memory um and storage, um and the key point here I guess, is that the infrastructure, the interface to the infrastructure provider um is, is custom, interface right, it's it's non-cloud native. um So it's quite interesting.

A

If we could think of infrastructure and management like we do application management, um could we you know, adopt and leverage the benefits of get ops um and span that across both infrastructure and application uh deployment?

A

So with the the use of crosstrain we can achieve just that, um crossplaying provides an abstract, an abstraction platform um for infrastructure provisioning. um So we have rolled out um a provider for our our for our infrastructure um management, um crossband provider. um It defines a cluster as a very simple resource type, so the the complexities of managing our clusters for our end users are, is really massed masks to to to our users. They don't need to be aware of the the api specification they just need to.

A

You know be aware of of the kind of type of cluster and the configuration of the cluster within the ocp resource. The openshift platform resource definition here on, on the left hand side. So you can see here we specify things like the cluster name, the diversion of the open shift.

A

We we want to provision things like the site, what data center we want that cluster to be deployed into, and then some very you know, high level configuration for the workers within the cluster, um so the user basically just needs to populate out these parameters here and the provider communicates across to our cloud provider to um to provision the cluster.

A

The provider also provides the cluster endpoint and credentials, then written to secrets on our control plane cluster. So the control plane is essentially our cluster running argo cd.

A

um So it is our our master cluster for that that basically performs the uh the provisioning of our infrastructure and running our deployment pipeline as well um as with any type of you know, kubernetes resource type. We get status um again via the provider back onto the ocp resources. So, um on our control plan cluster, we can list all the clusters we've got deployed in our pool and get the status of those as we would with any other kubernetes resource type.

A

um So, in the definition of our resources, then for cross plane, we're using helm, um helm provides a very good mechanism by which we can override and set configurations for our clusters.

A

So the template basically allows for the passing of um customized attributes um for our cluster deployment. So here on the left-hand side, for example, you can see we're taking default values from a values file, um but we're then also taking um like actually like the cluster name and we're overwriting those um to to create the the overall ocp definition and we'll see that on the next slide.

A

um So here we can see the default values and for our clusters, so this will be in one particular cluster pool type. This pool type is basically holding clusters of ocp 4.10.3 and in site xyz, and this is the default configuration for the the workers.

A

The cluster pool yaml then, is basically overriding the cluster name, each time it. It creates a new instance, our new cluster instance in our pool. So when we want to create another cluster, we essentially just add another line um here into our cluster pool yaml um and, and that then merges with the named template and to produce the overall cluster instance here, on the far right hand, side.

A

So how can we bring this together then um to manage infrastructure applications using git ups? um So this is a kind of just a high-level view of the flows.

A

Then, um if we start with github github on, on the left hand, side um on the the blue, the blue kind of trend here we're looking at the default uh values coming in for the clusters, our cluster pool yaml, which defines the instances of the the clusters that we want to you know, exist in our pool, as I said in the previous slide, that's merged with the named template and it basically roots through argo cd, which syncs off of the git repository holding those artifacts, true to provider and post requests across to our infrastructure provider, to generate the clusters in the pool.

A

So if we want to, as I said in the previous slide, if we want to create clusters, we essentially just add a new cluster entry into the cluster pool yaml. And similarly, if we want to delete a cluster, it just gets removed from that cluster cool yaml um and argo cd detects the update um or the commit um to representing that removal or addition of a cluster and and and reconciles that, um via the cross-plane provider um onto our cluster pool um application deployment.

A

Then um again you, you know the cloud native right, so it's using the traditional cloud native path. Here we have our values.yaml um again helm, helm, driven um our application templates, which you know contains all of the the templates for our application deployment and as well as the green line, which essentially is again syncing off of github, our github repository for application deployment. True argo cd and this time straight on to the cluster, that's chosen for deployment.

A

So, as I said at the towards the start of the presentation, um our deployment pipeline will choose a cluster from the pool for a particular deployment run and that becomes the active cluster then for deployment. um So we deploy straight onto that cluster again using uh our argo cd.

A

um So some of the benefits of of doing it this way right so managing both infrastructure and applications using git ops. We have a common approach, so we leverage the same argo cd approach for both managing our infrastructure, so you can think of infrastructure as code, it's managed and maintained within github. um Some of the benefits there, obviously right, but with anything I suppose in github is that we've got it versioned and audited.

A

um We've got an audit trail of when clusters get created and and removed from our pool um single sources, shoot, of course, um for deployment infrastructure. um It avoids configuration drift okay. So when we look at our cluster pools, we know definitively that those clusters map back to our definition within within github.

A

um There can be no configuration drift um in the sense of of individuals, creating clusters and placing them in the pool that, may um you know, contravene our configuration templates for those clusters in that given pool um cluster state is continuously reconciled, and this is an important aspect.

A

If clusters get inadvertently deleted from our pool, you know argo cd will see that um and it will detect the change and it will drive um the the cluster pool back to its defined state as per raspberry github.

A

um So again, that's a very important characteristic for us, not not only in terms of application management and to ensure that your applications are mapping back to the single sources shoot, but obviously also for our infrastructure.

A

um So I'm going to hand over to ying mo now and adimo is going to discuss some ongoing work in the development of an ansible provider.

A

Again more focused on you know: mapping git ups across to traditional I.t systems. um So again continuing on with you know, applying the the concepts of argo, cd and and crossplane uh to the to the non-cloud world. um So so over to you.

B

Thanks ken, so my name is moyin and I'm from ibm as a software engineer. Next, I'm gonna share with you. How argue cd can be extended to which traditional I.t system, by leveraging cross-play so a little bit background.

B

As we all know, both argo cd and cross plan are designed for cloud native and it can only be run in kubernetes.

B

However, this may not be true for many real world organizations where they may have I.t systems that are traditional based, such as application run on vm or bare metal, and they may have already heavily invested on management automation for those kind of systems for management. Automation, design has a popular automation. Technology has a large user-based and mature ecosystem.

B

It is widely adopted by many of such organizations to automate the management for different varieties of I.t systems, ranging from cloud native to non-cloud native.

B

So the question here is how we can drive githubs for those systems consistently, no matter it is cloud native or non-cloud native here, I'd like to introduce an open source project that is currently working progress, the cross-plan provider for ansible.

B

It is targeted to extend a cross-plan scope by enabling its integration with ansible to build a bridge between the cloud native and non-cloud-native world, so that can open a door to drive and reuse existing automation, assets to manage hybrid technologies using the same way consistently and help organizations transition to cloud native while keep their existing investments.

B

Let's take a quick look at how it looks so this is diagram that illustrates how ansible provider works at a high level.

B

Basically, ansible provider is driven at its core by a kubernetes controller, called single run controller where it watches the ansible run kubernetes custom resource, along with a reference to the provided config. That includes all the information to run in super contents.

B

The controller usually retrieves the ansible contents remotely from many places such as ancient galaxy, automation, herb or get ripple, and then it will store the content into a working directory so that can be launched by a um embedded and civil runner come online.

B

So here are some examples to demonstrate how to use it. For example, we can define an ancient run to call an ansible row or an ansible playbook, and both of them are usually retrieved from a remote place which can be defined as the requirements in the provided config.

B

So as it is shown here, this one is retrieved from ansible galaxy and this one is retrieved from a good ripple and if it is a private gear ripple that requires credential to access, it can also define the credentials information here.

B

We can also pass variables to customize the ansible run like this, and for some simple cases you can even define an inline playbook directly within the ansible run resource which does not require any remote retrieval.

B

Next, let's take a little bit deep dive into the ansible provider design so that you can better understand what happens inside the provider.

B

First, let's take a quick look at how crosspen provider manages its resource so to manage resource, or usually we call it external resource on targeted system. It requires us to define managed resource that includes the desired state and is watched by the cross-plan managed resource reconciler on local system to manage the external resource.

B

We will have to operate on the managed resource where the changes will be detected by the managed resource reconciler. Then the reconsider will operate on external resource on target system to reflect the changes that we made on the managed resource to be more accurate.

B

This is a typical reconciliation loop that is consistent, uh consists of five phases.

B

It starts from the connect phase that is usually used to establish the connection to the tactical system, then observe, delete, create and update that map to the resource cloud operations.

B

During the observed phase, the provider will usually detect if the resource on target system exists or not.

B

If it does not exist, it will advance to the crate phase to create a resource and if it does exist, but not up to date, it will then go to the update phase to update a resource so that the desired state and the actual state are always in sync.

B

Once it's finished, it will requeue to wait for the next loop and if user deletes the managed resource, that means we will never need external resources on targeted system anymore.

B

Then this will trigger the delete phase where the provider will help us delete the results on targeted system. So this is what we call as cross plan manage the resource life cycle.

B

Now it tends to the small run life cycle. You may see that it is actually a subset of the cross plan manage the resource lifecycle generally in ansible provider. There are two types of lifecycle for the ansible run that are supported.

B

In the first type, the provider uses observe to handle the case when the managed resource as a run, is present and use delete to handle the case when the managed resource symbol run, is absent and both observe and delete will call the same set of measurable contents in a second type.

B

The provider will run the ansible contents in observe first, but using the check mode, which is essentially a dry run, and if any change is detected after the dry run, the provider will then trigger, create or update to kick off the actual run of the same set of ansible contents.

B

Also, both two types will use: connect to prepare the runtime environment for the ansible run, for example, to retrieve the ansible contents from a remote place.

B

Okay, now you learned how small run will be driven by following the life cycle.

B

Ansible provider implements the life cycle to decide when to run the answer, but in order to do the actual work to manage the resource on target system, it relies on the ansible contents.

B

Now, let's see what are the best practices for ansible users who creates the ansible contents so that can align with life cycle much better.

B

So here are a couple of rules. The first rule is to keep your ansible rules or playbooks adaptant.

B

It is always a best practice to write ansible rules or playbooks in an adapted way for ansible provider. This is required because the same ansible contents will be run many times in the reconciliation loop. You should guarantee that each simple run should produce the same results if the desired state is not changed.

B

The second rule is to ensure your ansible rules or playbooks to support state field.

B

It is also a common practice in many ansible modules that support state field and behave differently according to the state value and for ansible provider. This is required because the same ansible contents will be used to handle both the case when the ansible run resource is present and absent.

B

Next, let's take a quick look at the difference between ansible provider and ansible operator, which is another similar technology created by a red hat that can bring ansible into kubernetes world.

B

So as listed here, there are a few major differences. First ansible operator is mainly designed for managing combination resources, while as a provider can be used to manage both kubernetes and non-kubernetes resources.

B

And second and simple operator requires you to bundle ansible contents that manage resource into a container image at build time before you deploy it and ansible provider on the other side works as a driver without having to build ansible content into it, but instead to prepare and call those usable contents at runtime and third ansible operator is usually implemented as a co-located kubernetes application to operate on in-cluster resources, while ansible provider usually runs on one kubernetes cluster then are based on the resources on another remote system, which can be either kubernetes or non-kubernetes.

B

Lastly, all the discussions around the ansible provider was extracted from the design dock, which I put the link here. Just for your reference, you can.

A

B

To get a more detailed version, and since it is an open source project that is evolving rapidly, I would encourage you to join us and feel free to open good issue or submit pr if you have any good idea on how to improve it.

B

Okay, that's all for this session and thanks for watching.