Cloud Native Computing Foundation GitOpsCon NA 2022, 1 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Simplifying Edge Deployments Using EMCO and GitOps - Igor DC & Adarsh Vincent Chittilappilly, Intel

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Simplifying Edge Deployments Using EMCO and GitOps - Igor DC & Adarsh Vincent Chittilappilly, Intel

Multi-cluster, multi-app edge deployments may involve multiple applications on different cluster types located in different environments. This adds complexity to management, security, and consistency. These complexities can be addressed by using EMCO (Edge Multi-Cluster Orchestrator). With EMCO, GitOps can be leveraged to communicate with Kubernetes clusters over the git protocol, integrating with a variety of public clouds. EMCO acts as the intelligent agent that writes resources to the git location, helping in management, consistency, and security (since no direct communication happens to these clusters). - What goes in the repository: o Kubernetes resources as rendered from Helm Charts. o Cloud-specific configuration and resource files (Kustomization, FluxCD system files, RootSync configurations). - Security considerations: o Token-based authentication for Flux, Azure and Anthos. o Kubernetes Secrets concept for storing and passing sensitive information. o HTTPS APIs ensure data privacy. - The different git libraries in use (and future): o Git2go for interacting with git. o Azure and Anthos APIs o Gitea for local git server

A

Hi everyone uh welcome to this github's Con presentation.

A

My name is I work as a cloud software engineer at Intel, so I've been working with Intel uh for the past one year and I've been engaged most of with github's related work. We are doing cool it stuff and I'm glad to be part of it.

B

Thanks, others I definitely agree with you when it comes to cool Edge, stuff, um anyways, I'm, Igor, DC software engineer at Intel been doing on and off work around Edge and Cloud, currently developer and maintainer of Amco, and me and others will be here talking a little bit about githubs and a little bit about about Emco but fundamentally Emco with gitops, and that at the end, we'll have a demo as well. So I hope you enjoy.

A

For the last couple of years, your dental have been working uh on MCO as a solution for Edge deployments and for the last couple of months uh we've been looking at githubs and integrating it to MCO, and we have realized like how powerful Amco and get Ops is to solve this problem of edge deployment.

A

And that's what we are going to talk about today.

A

So, just as a recap, before we jump into a problem statement, uh we would like to just recap what githubs is. This is a basic guitar flow. We have the developer on the left hand, side so developer puts in the source code, which is like resource yamls Etc and through some CI it goes to the caters resource uh get Triple and on the cluster we have like few config agents like flux or Argo, which continuously monitors this git repo for resources or any changes.

A

So as soon as it finds any resources, or it sees any changes in the resources it will, it will pull it up and deploy to the cluster. So this is a basic gist of what git options. Now, let us see like what technically it's defined, as so technically, it's defined as a set of practices to manage infrastructure and application configurations using it. So good is like the important Point here.

A

The git is a single source of Truth, so the entire state of the system is is Monitor and tracked in Gate itself, and as I mentioned earlier, uh we have like an agents like flaxseed or Argo CD, which are different in the cluster to continuously monitor this repo and which does the job of doing the deployment of applications.

A

So we can say like what you see is what you get pun intended.

A

Let's understand the problem statement in a real world scenario for an edge deployment. uh There would be like n number of clusters. For example, let us consider cluster number one. So, let's assume this cluster is deployed in Azure, for instance, so this cluster can have like app one app two and so on. Like five apps.

A

The same way, we can have another cluster, for example cluster 2, and let us assume that this is deployed in AWS and it has three apps app to app for app3. Similarly, we have cluster 3. This can be an on-premise cluster and it can have like four apps on it and so on. We can have like n number of clusters with n unique different apps, so we can see from this illustration itself like how complex it can become a multi-cluster multi-app deployment can go out of hands pretty quickly.

A

So it's such a sort of a scenario uh brings in some complexities with it as well. uh So one such complexity is management.

A

So, just as we saw like such a huge deployment, a lot of apps requires an orchestrator in place which can manage the deployment of these applications in a smooth manner, and also we require something which could monitor the status of these applications. Like till the deployment go, find other applications working properly is the health code of those applications and also in certain scenarios. Certain applications would require certain sort of resources like specific Hardware, so we also have to ensure that those apps get deployed to those type of clusters.

A

The other complexity that we could encounter is security, so we need like a secure way for this deployment to go through. The API calls that we make should be secure and also any sort of credentials or any sensitive data that we might require for this deployments to go through should also be maintained in a secure way.

A

The third complexity that may encounter is consistency. So, as I mentioned, your consistency is a key for any deployment. We must have this assurity that what we desire is what we get, and even if this drift that happens in the system, like the the versions, are old, uh the entire configuration should be able to remediate this issue and ensure that consistency is always maintained.

B

To address the edge deployment, complexities that were just outlined, I would like to introduce you to Amco or the edge multi-closure orchestrator.

B

This is the Linux Foundation networking project currently in its sandbox stage, and some of the things we can do with it are, for example, intent-based Department of cloud native applications, so Emco is designed to have an intent based, API and then to be pluggable to fulfill those intents.

B

Its main focus is cloud native applications on the edge um you can deploy to a set of two sets of kubernetes clusters does a multi-cluster nature of it, but those kubernetes clusters can be reached in different ways through different implementations of backends, namely the typical Cube config based approach with direct API authentication, as well as github's based approaches.

B

um Flexibility, modularity and scalability of Emco are major selling points of it, and specifically, when it comes to the modularity, we can attach different backends, like I just said, regarding how to reach those kubernetes clusters, namely guitars for particular implementations out there. Both you know who know names from the public Cloud industry as well as lesser known, implementations of such through Amco. We can. We have we get access to single pane of view for all the Clusters we have, as well as the applications there in deployed.

B

um So such applications are what we call composite applications. They really are collections of applications that get deployed as a single package, with logic uh to to to represent how to distribute that composed application. Those components of the application across the multiple clusters and Amco provides that engine and the language to achieve that distribution of such composite applications.

A

So now let us understand the basic uh MCO flow with githubs, so this flow is basically to deploy resources to three clusters. So, as shown in the diagram, we have three clusters: cluster, one cluster, two and cluster three. So cluster one is like a direct cluster. It has no github's component in it. uh Cluster 2 and cluster 3 are githubs managed. So in this cluster 2 uses flux CD as a config agent and cluster 3 is like a Azure Arc manage cluster.

A

So basically we we would onboard this cluster to Azure Arc, and thus it's Azure manage cluster. We have a git server here, so this git server can be any git server. It can be like a local git server. It can be GitHub, gitlab and Etc. Then we have MCO here. So we can deploy MCO to any of this can be co-located with any of these clusters or we can deploy and code to a different cluster also, so we have shown like three important components of MCO.

A

There are more components than this, but cluster is star orchestrator, resource, synchronizer and mongodb. These are like few important components of him. Now. Let us assume, let us look at a scenario we have user and the user want to deploy an app to these clusters. So the first step that you would take is basically registering these clusters with MCO.

A

So for cluster one, since like it's a direct cluster, it will be the cube. Config will be the registration key, so the user will provide that and for cluster 2 and cluster 3. Since they are GitHub space cluster, the interaction would go through git server, so we require some good credentials and in term in in case of cluster 3, we also require few Azure credentials.

A

So all these credentials and Cube config is provided by user one, a user to the cluster register, and this then gets stored in the mongodb in the step. Two, uh the user calls instantiate so basically wants to instantiate app one on this cluster, so he calls that API command to orchestrator.

A

An orchestrator then gives this command to resourcing so resource synchronizer does the job of basically applying the resources and, in case of github's, writing the resource files to git server so that what will happen The, Next, Step, so resourcing gets the credentials from the mongodb in case of the direct cluster. I directly applies the resources in case of the github's cluster cluster 2 and cluster 3. It writes the resources to the git server for cluster 3, which is like an Azure cluster. uh An Azure config should also be created, and that's what we see here.

A

It calls that Azure API to create that config.

A

Now. The next step is step five. So in step five for cluster 3, which is our Azure, managed cluster, it will configure the the flux CD on that cluster and then, after the flux series uh configured, it will start pulling resources from the kit server and seeing the resources and thus like. After all these five steps, all the all these three clusters have app one uh deployed and installed on them.

B

In terms of github's back-end support by Amco, we currently offer three Upstream. We have base basic Flex V2, which allows us to use githubs with any kind of kubernetes cluster. As long as we run that Flex V2 controller and operator inside we have Azure Arc, which happens to internally also use flux V2, but in this Azure Arc it drops extension for Amco is flexor 2, plus the additional extensions to extract the maximum functionality from from Azure, and then we also have anthos.

B

um So we can run githubs for you know for Google cloud, and this is this: is this entos config management, including repo sync reposync, is a structure that in enters, we can use to scope deployments to particular namespaces and also to apply role-based Access Control there using um the the repo syncs respective service account in kubernetes. So these are the three ones we have flux, V2, just a generic flux, Azure Arc, which is also based on Flex, but for Azure. We do additional extensions and then entos config management.

A

So git server is like one of the important components of the GitHub support. uh Usually we depend on uh look uh so git servers like GitHub gitlab Etc, but it's also designed to have a local git server and MCO comes bundled with a local git server.

A

uh This git server is is powered by git T and the UI is very similar to what we see for GitHub. We can see here it's very similar to what GitHub provides and not the best USPS of having your own git server. Is that you, the user, will have full control over the data, and this ensures that there is no privacy leaks or anything like that, and also another issue that can happen with uh git service like GitHub, is API 8 limiting. uh So you also won't face this issue.

A

If you have your own git server,.

B

So Amco with githubs, we can think of git as a source of Truth for these clusters that are synchronizing with the git repos, and then we can also think of Amco as the entity.

B

Defining that truth to be put into the git repos guitars is a great addition to Amco, because it really aligns very well when it comes to making the most out of them, because existing functionalities, such as on-demand instantiation of applications, intelligent placement of workloads on clusters, including clusters of different Natures, from different interfaces, including githubs, uh the customization of resources, post instantiation of the application. That's something Emco can do so.

B

We deploy the application and it can customize it later and githubs also attaches very well in that point, and then finally, automation of service, mesh and other connectivity, and you know, networking and the security infrastructure, the uh automated distribution of security certificates and things of such Amco and githubs go along very well together.

A

So, just to recap, the complexities we discussed earlier- uh we're doing a recap here, so this three complexities that would face an energy deployment over management, security and consistency.

A

Let us see how MCO, plus key tops can solve the complexities that we discussed earlier. The first complexity that we mentioned was management Now.

B

A

Githubs helps to deploy these resources very easily, so deployment is taken care of by githubs.

A

Now what MCO adds on to this is intelligent placement of resources.

A

So considering the scenario where a certain application requires a certain amount of CPU or Ram, so it should be placed in such a cluster which have enough resources to handle that app's needs that is taken care of by Amco.

A

Also, what MCO provides is like a One-Stop solution for monitoring so to find out if the apps are healthy in the cluster, if they were deployed properly, if they're updated properly. All of this is taken care of by MCO.

A

What also Amco does is application dependency management between clusters so consider an example uh where we have app one which is deployed on cluster one, but then there's app2, which depends on this app one, so it can only be deployed after app one is deployed. So all of this is taken care of by MCO, so we can mark this problem as solved.

A

The second complex series that we discussed was security, seamlessly integrates with istio and service mesh. Due to this authentication of users becomes very easy and we can do that in a very secure manner, also by using istio. All the API calls between the MCO services are secured. Also, the calls from MCO the git server uses https protocol, which also means it's secured. So this ensures that no one can sniff to the API calls and get data from it.

A

Now the flux, config agents that we discussed they are all deployed in the Clusters. So this helps us to avoid the risk of storing credentials anywhere other than the Clusters. So we don't need to store the credentials in the kit trip or anywhere else. Everything is confined in the Clusters.

A

So we can also say, like we have solved this complexity of security.

A

The third complexity that we were bugged with was consistency.

A

Now what MCO provides our guarantees is consistent intents and what gitops guarantees is consistent, rendering the combination of both this ensures that what we desire is what we get and that to in a consistent manner.

A

Amco also has a very unique directory structure which ensures that the apps are properly deployed to the right cluster every time.

A

So we can also consider this consist this complexity problem solved just recap in our initial problem statement, so we have like n clusters with n different types of app, so with all the discussion that we made so far, we recognize that MCO plus GitHub is a perfect solution and by having them as a Central Center of this deployment, we can easily simplify the deployments of these applications on any complex cluster.

A

Hello, everyone uh so we'll bring a small demo to Showcase how MCO, with githubs can be used to apply uh applications to the Clusters uh so on it inside we have three cluster in total and right hand side. We have the cluster which has MCO installed in it, and then we have the two target clusters uh so we'll first start by uh installing flux in the Target cluster, so I'll make use of the boot trap mechanism to install uh flux.

A

uh The important thing to note here is the path, so we are considering this as cluster 2., uh so this is the actual path where the resources will be uh synced. We will just apply this uh command, so it's connecting to GitHub. It will write the resources to GitHub and then uh it also try use that to sync the flux components to the same cluster.

A

Okay, so now it's done. uh The second step is to uh install monitor.

A

So what monitor does is basically, it will monitor the resources in this cluster and uh we'll report it to the git repository and the service earning in MCO will monitor uh the status from the kit repository and note it down so make use of the helm, install command uh to install Monitor, and there are a few parameters to it. But the important one is the cluster name, which again is uh the same one as I mean discuss Japan. It should be the one with cluster 2 in it.

A

So we can see the monitor is now deployed. We can just quickly check uh they're all deployed properly.

A

Okay, you can see I'm monitors deployed in under an MCO name. Space uh and the flux become uh components are just deployed in the flux system namespace.

A

uh Similarly, for the other Target cluster, we'll do the same steps. uh Install flux, install monitor, I've, already done it uh to save time so now, moving on uh so Emco comes bundle with this test examples, uh so one just Test example is the test flux example.

A

uh This has a few files in it uh yaml files and a setup script. So the setup script is where we specify uh with apps to deploy to withhold clusters.

A

uh So we can see here. uh So we have two clusters and we are saying for uh collect. D is deployed in cluster one and cluster 2 and operator in cluster one.

A

It also comes handy with a test all in one Crypt, so we'll be using that to apply the resources, we'll call apply to it, and this will uh instantiate The Logical Cloud, as well as the deployment.

A

So the deployment is in progress. Okay, so deployment succeeded. uh So we can just quickly see uh what all steps were taken. uh Zoological cloud is the first step that gets instantiated, so it got succeeded and the second step is basically deployment uh that also got succeeded.

A

Now let us uh check the git Repository, so in git repository we have the two folders for two and in that with other files for flux, and this is cluster one. uh So, under this contact, ID contact context, ID path. You have apps folder, which has the two applications that we wanted deployed uh to collect, DS all Collective files and similarly operated our operator files.

A

And uh similarly we have cluster 2 as well. If you will cluster 2, plus only at Collective so similar to the cluster one case, we have all the files in it here.

A

Now quickly check the uh clusters till they get deployed.

A

Yeah, so this is cluster one. It has operator and collect D. So orbital also has few TD Parts with it, so everything got deployed here and similarly for the other cluster.

A

It only had a collect dnit. uh We only had wanted collecting cluster two and that's the only thing that got deployed here.

A

You quickly see how status gets dragged, so uh it's a different branch for each cluster. So in cluster one, if you go to Cluster folder, you will see the status so status for both collect DN operator and similarly in the branch call cluster 2. The status for cluster 2 is meant foreign.

A

A

So deletion went through as expected, we can see logical automation, succeeded, deployment, option, succeeded, okay, Okay, so let again go to the uh folder.

A

A

Okay, so we can see okay, so Operator just got written in front of our eyes just check again yeah, so everything got deleted from the git repository. It means like the resources should also get deleted from the Clusters yep, so it's not terminating state, which means it will get terminated soon.

A

uh Let's take cluster one yep, so everything got deleted in this cluster. Let's again check uh cluster two yeah so yeah. So all the resources got deleted. uh So this was our short demo on MCO with githubs uh thanks for watching. Thank you.