Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 15 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Sponsored Session: AWS & Weaveworks - Enable Hybrid Cloud Kubernetes with GitOps, EKS, and EKS-D

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Sponsored Session: AWS & Weaveworks - Enable Hybrid Cloud Kubernetes with GitOps, EKS, and EKS-D
Speakers: Paul Curtis

One of the biggest advantages Kubernetes has to offer is that it is agnostic to infrastructure and capable of managing diverse workloads running on different compute resources. This allows organizations to take advantage of scalable and cost-efficient cloud resources as well as harness a secure and fully controlled on-premise environment.

Streamlined processes and automation are key for application and operation teams when managing clusters across environments while maintaining security and policy checks.

In this demo you will learn how to:
· Use GitOps model-driven automation to increase reliability and stability across EKS (cloud) and EKS-D (on premise)
· Simplify multi-cluster management with GitOps
· Enable developers to push code to production minutes from Git
· Improve utilization and capacity management through hybrid Kubernetes platforms

Tech utilized: Kubernetes, EKS, EKS-D, Git, LinkerD, GitOps, Weave Kubernetes Platform

A

Hi, this is paul from weaveworks today in this video I'm going to demonstrate a simple build pipeline for promotion from development to production using eks and eksd.

A

Before we go, though, we're going to take a quick look at get ops again, just so that everyone in understands what they're, seeing the principles of git ops are very simple. Everything in kubernetes is declarative, so you declare everything. That's in your cluster.

A

You use get to keep a versioned state of everything. That's in your cluster. This allows you to monitor what's going on and it is the authoritative source of truth, tooling, that weave works is written over time, both commercially and in the open source applies. These changes to the cluster after checking with git and software agents, continuously reconcile what you've declared in git with what is actually running in the cluster at any given time.

A

For this demo, there are two clusters: the first cluster, the development cluster is running eksd in equinix metal. It's a single node, very simple cluster. That only has flux the get ops enabled tool installed on it.

A

The second cluster is a bit bigger. It's running in eks, using the weave kubernetes platform. This cluster is going to be where production containers are deployed. The connection between these two clusters is a single get repository.

A

The manifest that the developer places into the git repository for their development cluster are put into the dev branch in production.

A

The production cluster draws its manifest from the main branch of the same git repository to promote containers from the development environment to the production environment, the developer issues, a pull request.

A

This pull request, then, is reviewed and when it is merged, the new containers that the developer has completed are deployed into production.

A

Our production environment also includes two other features: we've installed a service mesh linker d and the get ops, progressive delivery controller flagger in order to do a canary deployment for new containers that are deployed into production.

A

Okay, let's begin with the development cluster, the development cluster, the manifests for that cluster are contained in the dev branch of this git. Repository in this repository are the manifests, helm, charts, customized templates required to run the developers application now, we've just heard from marketing that they want to change the background color of this particular application from green to blue, so the developer goes in finds their manifest for this particular application and we're just going to change it here manually from the color green to the color blue.

A

Keep in mind that flux is running in this cluster so that when the developer commits their change, they will automatically have the application the new, manifest and application deployed for them so running in the cluster you'll. See here are our pods our front end and flux version two. So the notification controller, the customized controller, the helm, controller and the source controller are all running here.

A

When it detects a change to the get repo, it will start the new pods, as you can see happening right now and as those pods come on and running, the kubernetes will have done a blue-green progression here and we will see that our color of our ui has changed to.

A

Blue and when all the pods are up and running and the old pods have gone away, we should see nothing but blue. As you can see, it is shifting between the two pods that are running okay. So now we've completed our deployment and we're very happy with it.

A

So at this point the developer says: okay, we're ready to go we're going to send this off to the devops team or to the ops team to actually deploy this into production.

A

So what we're going to do here, rather than just deploying directly, is we're going to create a pull request from the dev branch to the main branch and, as you can see, the only thing I changed here was the color of the ui.

A

So I create my pull request and I say you know put in my note and create that pull request now in production, nothing has happened, but this pull request is ready to.

A

Go meanwhile, this poll request is waiting to be deployed into the production cluster, so let's switch over to the production cluster side of this okay. Let's move on to our production cluster.

A

Our production cluster is using the weave kubernetes platform, which is a completely git ops, enabled platform it packages up a set of standard cluster components, as well as providing isolation to applications through the use of workspaces.

A

So in this production cluster, the application that we have is here at the very top. So the first thing we have to do is we have to go off and merge our pull request.

A

So our pull request is in we like it: okay merged pull request. I think I'm okay with that and we go ahead and do that now in our cluster we have configured a canary deployment for our application that canary deployment again because everything is get ops- is defined here and you can see it targets our specific application.

A

So we also in this cluster have linker d as our service mesh, so we should be able to monitor the change in the traffic as the service mesh begins, its progressive delivery, we can watch the pods change and we can do that very simply by watching what goes on here. The traffic can be monitored with linker d's built-in grafana dashboards.

A

So what we are going to see here in this gravana dashboard is we're going to see the traffic alter over time. So as the new pods are started, as you can see here, we will see the traffic begin to shift from the old pods to the new pods over time.

A

We've configured the duration for that in our canary definition for flagger. Now, here you can see the progression is beginning: we've set it to have intel intervals of five seconds and five percent of the traffic moving back and forth. So it'll take it about a minute to do this.

A

Meanwhile, you can see that the pods now there are two sets of them running our original set and the new set the benefit here of using a service, mesh and progressive delivery means that there is no downtime in any uh to any user, so they don't see the changeover.

A

The second thing is: is that the canary, as it's defined here, also has thresholds. These thresholds are used to determine the success of whether the progressive delivery and the new pods that have been deployed worked adequately.

A

These metrics here that you see come from prometheus and so can be any prometheus metric can be used to determine whether the progressive delivery passed or failed. So currently, let's see we're at forty percent and sixty percent.

A

Our threshold is fifty percent, which says that when you reach 50 50, I.e, half the traffic is going to the old, pods and half the traffic is going to the new pods. Then we're going to test and ensure that everything is running right.

A

We'll then shift the traffic over to the new pods and scale down the old pods. So if you look at this graphically, we can actually see where this is happening. So let's take a quick zoom in here and we can take a quick look at the traffic as it is moving.

A

As you can see, the original pods here in orange, the traffic was scaled down. The new pods here in blue are scaled up over time.

A

Once flagger has determined that the new pods are working adequately, it will then switch the pods. Then all the new pods become the primary and the old pods traffic will be scaled down and then the pods themselves will actually be scaled down to zero. So you can see that happening right now. The traffic is going back to the new version of the pods.

A

It's gone back up to a hundred percent, and if you look over, you can see that the pods, not only were they renamed, but the old pods are being scaled down to zero, and at this point now all of the traffic is now diverted to the new pods.

A

This was done completely with get and get ops, so our canary defined hearing it defined the progressive delivery. We used git and pull requests to promote from the development environment to the production environment.

A

We have the weave kubernetes platform, which is giving us a multi-tenant production quality, uh kubernetes infrastructure which, by the way, can run on anything. As you see, is running here on, hosted eks and our development environment is running in eksd.

A

All of this is achieved through get ops because get ops is platform independent. It will work on any kubernetes and it really doesn't matter where that kubernetes is.

A

Thank you very much for watching and I hope you're enjoying kubecon.