Cloud Native Computing Foundation Crossplane Community Day EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Scaling Kubernetes Global Balancer with Crossplane - Yury Tsarev, Absa Group

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Scaling Kubernetes Global Balancer with Crossplane - Yury Tsarev, Absa Group

Global load balancing, commonly referred to as GSLB (Global Server Load Balancing) solutions, have typically been the domain of proprietary network software and hardware vendors and installed and managed by siloed network teams. k8gb is a completely open source, cloud native, global load balancing solution for Kubernetes. k8gb focuses on load balancing traffic across geographically dispersed Kubernetes clusters using multiple load balancing strategies to meet requirements such as region failover for high availability. Absa runs more than a hundred GSLB enabled clusters and the number is continuously growing. To handle the configuration overhead Absa uses the combination of Crossplane XRD abstracting and simplifying k8gb deployment configuration for cluster operators, minimizing the potential human errors and enabling GitOps best practices for huge scale hybrid cloud deployment.

A

Hi everybody, I'm yuri, I work as principal engineer for apsa group and let's chat about crossplane and how it helped us to scale our kubernetes global load balancer so quickly about a cross-plane setup in apsa. Apsa is a a huge financial organization that serves multiple countries in an africa continent and we have a pretty uh reasonable scale and we operating over a hybrid setup, part of the workload on-prem and uh part of them in edibles, cross plane. Instances are currently deployed uh within our on-prem data, centers and kubernetes clusters.

A

We are using a set of uh course, plane providers, uh main ones are edible, aws and helm, and we use our special somewhat temperatures that we developed to automatically propagate credentials as kubernetes secrets. uh In automated way. uh We are someone.

A

We use crossplane compository source model and associated composite composite resource definitions, to create our very own platform and provide an abstracted resources to our technical customers, so main exercise composite resource definitions that we operate at the moment are as bucket to provide specially configured data storage object. Storage was grayscale instance, eks clusters and kgb for global load balancing, so we mostly focus on a kgb part today.

A

uh Meanwhile, the principles of cross plane based compositions are shared uh between these resources.

A

So what is kgb? It has open source project that we developed in apsa. It is a cloud-native global, kubernetes, global balancer. It is kubernetes native and it enables global service load, balancing function for workloads that are deployed on top of kubernetes clusters.

A

Typically in geographically dispersed data, centers or different cloud reach, regions like different aw, it was regions and, as is a common reference setup.

A

uh One of the distinctions uh uh points uh of a kgb project is a absence of a single point of failure. So we do not have any control clusters and no instance where uh traffic is going through. So there is no single uh bottleneck and the system is uh architected.

A

The way to be really reliable at a global scale and the it is all based on a dns protocol that is battle tested by internet and is a good match for global load. Balancing traffic steering we faced obvious kgb deployment challenges because we operate on more than 120 kubernetes clusters and this number continuously growing, and there is an obvious challenge of repetitive helm configuration where we given. We are packaging kgb as a helm. Chart we're providing uh flexible enough configuration configuration parameters to enable uh customization of every part of a kgb is a distributed system.

A

Meanwhile, our special specific setup is relatively stable, so we need to parameterize, not all of them, but a minimum amount, and that's how crossplaying helps with us. It is its abstracted resources. So here is an example of a.

A

Crossplane uh composite resource claim, uh which provides an interface for end users, a little the technical customers or.

A

You know whatever services uh our human is interfaces with the kubernetes api. We exposing just uh a five config required uh configuration endpoints to drive the kgb installation, our multiple clusters, so it provides very effective, abstract abstraction.

A

A very minimal configuration only required params are getting exposed and you decide what kind of these params are with the help of cross plane obstruction power. uh This setup is very good for github, so via uh uh we can easily automate. uh The application of this xrc manifests uh uh visa tools like cargo, cd or flux, and we don't need any client-side implanting like a with k, because the xrc yamls are very minimalistic and we can just drop them in into github's triple and let argo or flux applied it to the target clusters.

A

So let's uh perform a very quick demo and demonstration how it all works. So we already have a couple of clusters where crossplane is already deployed. We have a empty kgb namespace which we will use to deploy a kgb installation.

A

Let's quickly look how x-rays are composed, so we have a it packaged as a standard x-ray uh definition. So it's a composite research definition and that's how we expose the required parameters. It is a open ip history schema and the heavy lifting is done in a composition where we actually provides the default parameters and then patching them from x-ray down to the specific resource instantiation.

A

We have examples, resource claims, so let's be the european one. So that's exactly this minimalistic uh abstraction uh that was demonstrated in a slides. So let's create a kgb installation where it's stacked with eust1 and it's going to talk by configuration with africa. So it is as easy as applied with keep control.

A

And we can check the status of obstructed resource with the standard control commands it's not yet ready. Meanwhile, we can check uh and demonstrate that underlying that it is uh under the hooded standard health facility. uh Thanks to uh cross playing provider hell, we can check subtracted resource again. It should be already ready and it's true and we can quickly check the fault fee of the service, fair parade.

A

And let's quickly describe this spot and double check that the environment variables were properly propagated down to the port configuration, and here we go cluster geotech as expected label s0, we can do the same.

A

Same application of resource in another cluster in another data center, the only difference is uh in this specific instantiation is it's totally the same it just the geodetics are flipped, because this uh cluster resize is in another geographical location.

A

So we can quickly check the plot there. It should be already ready and let's make the same, end-to-end check.

A

As in another velocity.

A

Here we go so we have a properly configured system uh in a box of clusters and both data centers and that's how we minimalize the setup and make it very concise and declarative to uh dive into this cross, plane, composition, concept. You can visit a a very good official documentation for which described composition in a very practical way.

A

There is a a very nice video of a quarkxplane developer, providing you ideas of cross-plane composition to be used to build your own cloud platform. That's pretty much what we do within amsa uh kgb, io our official uh website for uh our kubernetes global balancer in case you you're interested in this topic.

A

Please visit it and visit our github repo and provide feedback, and you can grab these slides right in sketch. So you can click all these links and follow through. So thank you so much. That was uh an example how we using cross plane uh and in subtraction power to scale a distributed system as an example kgb, and you can use the same approach to scale and compose your own unique platform abstractions on top of kubernetes.

A

Thank you so much for watching and have a great rest of the conference. Goodbye.