Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Application Autoscaling Made Easy With Kubernetes Event-Driven Autoscaling - Tom Kerkhove, Codit

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Application Autoscaling Made Easy With Kubernetes Event-Driven Autoscaling - Tom Kerkhove, Codit

Deploying applications to Kubernetes is one thing, but how do you scale them? Horizontal Pod Autoscalers to the rescue! But when you want to scale on external providers such as Kafka, Redis or another dependency it is not a walk in the park anymore. Enter Kubernetes Event-Driven Autoscaling (KEDA) which allows you to take your existing apps and easily configure how it should scale without having to worry about the magic! Using Kafka, Azure, AWS, GCP, Prometheus, etc? Don't worry, we've got your back! Join this talk to learn what KEDA is, how simple it is to get started and why it makes application autoscaling so easy!

A

Good morning, good evening, wherever you are welcome to this talk about kubernetes event, driven auto scaling, where I'll show you how you can do application auto scaling, which makes it super simple on kubernetes so first before we dive into it, so my name is tom carkova, uh I'm an azure architect at a company called called it, and I am one of the co-maintainers of cada and if you want to reach out to me, you can always find me on github, or you can also pick me on twitter.

A

We are always open for feedback, so don't hesitate to reach out so before we get started. Let's have a look at how you can auto scale applications on kubernetes without cana, so plain old, kubernetes fresh cluster. How would that work?

A

So, if you have a look, imagine we have four deployments, which represents applications. What you would typically do is you use a horizontal part autoscaler which allows you to scale on cpu and memory, so this works fine in vanilla, kubernetes. Now, imagine you want to scale on one or more of these external dependencies at the top of the slide. How would that look like so over over on the kubernetes side? You would use what's called an external metric which basically defines these metrics from outside of the cluster.

A

Now before you can use these external metrics, you would need to use a metric adapter, so a metric adapter basically pulls the metric from one of those systems makes them available for you to automatically scale on now. There is a caveat, however, you can only use one metric adapter, so if you want to use multiple of these systems, you'll have to choose one and make sure all the metrics are available in that one too. So in this case, imagine you want to auto scale on prometheus kafka and azure monitor.

A

What you could do is then send all metrics from kafka to prometheus, send all metrics from azure monitor to prometheus by using a tool like prometer.

A

Now all of this is a bit much and we with gada figured. We can make this a lot simpler so that you don't have to worry about all of this auto scaling infrastructure.

A

So with cada, we basically have a variety of scalars and secret sources so that you can automatically scale deployments jobs or practically anything inside your cluster. That has a slash scale super resource, so it could be also a resource from a crv from another tool that you're using.

A

Then you just choose one of the 30 or more built-in scalers that we have or you can build your own or you can use an external scale from the community and you scale your application by using that now, one of the beauties that we have is you can also optimize for cost and scale all the way down to zero, so that your cluster resources are freeing up to those that actually need it.

A

Then, when cada sees that there is new work coming in, we will basically scale you from zero to n instances to keep up with this security is important. So that's also why we strive to have good authentication, which is simple, reusable and manageable. That's why we also allow you to manage that separately from the scaling definition.

A

If you want scope to a namespace or scope to the whole cluster, which you will see later on uh today, we're very happy to be a sandbox project and we're actually in the process of um proposing to become an incubation project. So if you are actually already using cada, it would be my pleasure to talk to you so that we can convince the cncf to graduate now. One of our main mantras is that it allows you to to focus on the application and not the scaling internals itself.

A

So we really want to do as much as possible for you so that you don't have to worry about anything now. If we remove everything again and we have a look at what it looks like with keda, basically, instead of adding hpas, what you will do is you will install cada in your cluster? It has all the scalars out of the box.

A

You don't have to worry about anything anymore and then what you do is you deploy scaled objects or, if you're, using jobs, the scale job the scaled job or object, basically defines what you want to scale when it has to scale and how far it has to scale then for the rest, uh basically keda interprets these by using an operator, and then it handles everything for you.

A

So it has a scale controller that basically checks the external dependencies, and then it manages the auto scaling for you and under the hood it will actually use an hpa as well so fairly simple. You just install k-na, you create a scaled object or a scaled job and all the rest is managed for you no longer need to worry about external metrics, one or more metric adapters. No, we manage it for you.

A

So how does it work under the hood? For us, it is very uh important that we do not reinvent the wheel. So that's why we are extending kubernetes, so cada only handles the zero to one and the one to zero scaling for all the rest. We basically rely on the hpa and serve custom external metrics, sorry to the hpa, so we are actually using a metric adapter ourselves, but we make sure that we support all the systems that you need across all the cloud platforms.

A

Now, like I mentioned, we have a lot of these built-in scalers. We also allow you to use add-ons which are not maintained by us, but maybe by another vendor another product or maybe yourself, and we make it super simple to get it installed, either by using helm or the operator framework. So you can also find this on the operator hub now. What scalars do we support? Well, actually, these aren't just a couple of them, but we basically support all the major cloud, vendors and all the major products in the ecosystem.

A

If we are missing uh some, don't hesitate to uh open an issue or maybe even contribute the scale to us, and then you can also auto scale uh your applications with them.

A

Now I've mentioned um security before a bit. What we want to try to do is that we have as less secrets in the cluster and that we also reduce the duplication of the secrets.

A

Now your deployments will have to authenticate to the systems anyway, but we also want you uh to have the controls to use um separation of concern and give an identity to the workload, give an identity to cada for auto scaling. So you can manage the permissions differently. You can rotate them, but you don't have to. We don't force you to do it, although it is a best practice, sometimes you can't- or um it is not uh the best fit at the time.

A

Now. What we do is we give you the trigger authentication and the cluster trigger authentication crds, which basically allow you to reuse the authentication by either using environment variables on the scale target kubernetes secrets, but we also support other secret stores, like hashicorp fault or, if you're, on a cloud platform.

A

We support some of their no secret authentication offerings like azure, managed, identity and aws spot identity. So by using those you have no secret authentication. Nothing to worry about. You simply rely on your cloud provider to manage that now. Let's have a closer look and show this in action.

A

So what I'll show you here is um one of our examples that we have where we will use a.net core worker, which is processing an azure service per sq and then we'll basically add auto scaling with data. So you can see how easy it is to get started now before we dive into it. I have a service bus namespace here in the azure portal.

A

You can see that I have one queue, which is called orders at the bottom and then, if we have a look at the access policies, so the different identities. In this case, because we will use connection string, you can see that we have an identity for the auto scaler, which is cada and requires manage permissions.

A

We have one for the portal and the generator which you just use for the sake of the demo, and then we also have one for the application. So that's the workload we that we run, which only has listen permissions so because we use we will use the trigger authentication.

A

Application can be scoped to just listen permissions and the auto scaler manage. If we would reuse the same connection string, then our application would require to have more permissions.

A

We will not do this here, um but this is just to show that you can separate those now, as you can see uh on this portal, we have no messages on the queue we have a clean sheet and we will get started so in here. I have um basically one deployment running as you can see. It is the order processor, it is super simple and it is purely demo where. So, if we have a look at the deployment, you'll see that we just run the image from github's container registry for cada core.

A

We mentioned that it is using connection string authentication if you want to run the same demo with managed identity for azure. You can just go to the github report follow along, um but for the sake of the demo, we pass the connection string of the queue and we give it the name of the queue and then we also have the secret here with the connection string.

A

So if I would go here and say, give me the parts we will have one instance- and I will watch the logs for that one and just show that it is up and running so on the bottom right here. I have an order generator to cue some example work. So if I now queue one message, you will see on the left side that it got immediately processed straight away.

A

Now. What we'll do is we'll queue, a lot more work. So let's say we will queue. 250 orders, so now the generator will start generating the traffic and on the left, you see that our order processing is slowly processing those messages, but it is not that fast.

A

So in the portal here you see that it's going way up and it is almost never going down because it is that slow. So this is a perfect case for cana. Okay, so what we'll do is we will break in here? We will clean and we will just watch the deployments here now.

A

If you want to get started with cana, it's very simple: you can do helm, install data oops, just as simple as this. In this case, we installed it to the cada namespace uh data system namespace.

A

um However, I already did it so if we just check you see that it is up and running, and if I do a k get all indicator system, we can see that it basically installs uh the cada operator and the k dot metrics api server. Okay, if we do a k, get c or d you'll see that we also add some crds over here oops over here um to authenticate now. The version that I have installed doesn't have the cluster trigger authentication. Yet if you would use the latest version, you can do it now.

A

What does it look like to get things auto scaled? So I have another file here which has a scaled object and the trigger authentication. So, let's start with the trigger authentication. So with the trigger authentication, we have the various options to authenticate.

A

In this case, I will refer to kubernetes secret, which you can see here below, so it has the base64 encoded connection string, which is not super secure, um but for the sake of the demo, we will use this and we just refer to the to the secret. We say which key it should use and then the parameter name now the parameter name is specific to the trigger the authentication trigger that we want to use which brings me to the documentation.

A

So if you go to cada.sh- and you want to see which scalars we support- you can basically see the full list of already here and if we go and have a look, you see every single scaler that we have. Okay, if I go to azure service bus, which is the tricky that we will use, it basically gives you all the uh information you need, and in our case we are using the connection, authentication parameter for our trigger authentication.

A

So this is what we will use in the meantime. You see that our message, processor still hasn't picked up, so we still have time now. If you look at the scaled object, this is basically where we define everything related to the scaling itself. So first we define what the scale target is. So in this case we use just a name which implies that we will scale a deployment.

A

So this is an exact match of the deployment that we have already up and running. Picking up those messages.

A

Now we also define um what the maximum replica account is. So basically we're telling cada to go all the way to 25.

A

You can also define a minimum replica account if you want to always have five instances up and running. This is where you can configure this now. I left that out because we will use zero, which is the default meaning. If the queue is empty, we will scale down to zero instances, and then it is super simple, so you can have one or more triggers. In this case we have one trigger, which is of type azure service bus. We give the name of the queue and say hey.

A

We want to start scaling if the queue has five messages or more and in terms of authentication. We just reference our our trigger authentication resource that we have in the cluster.

A

So this is just as simple as it is. We have an existing deployment running, we diff, we just point to it. We say how many instances we want and when we want to scale, if you add multiple triggers, it will basically start scaling if one of them meets the criteria which is important.

A

So what I will do now is, I will um apply oops um that skill object and, as you can see, it adds the scaled object trigger authentication our secret. So if you would do give me the scaled objects, you can also see that it was. It is targeting the deployment, the name of the deployment maximum of 25 and it is ready to scale and it is actually active so on the right.

A

You now already see that it is scaling from one to four and it will start spinning up the parts to process them as we go. So if we now have a look you'll see, it starts slowly. um Picking up the work and then in the meantime you see it already one two eight now for the trigger authentication authentication.

A

We have the same experience, so you can see all the trigger authentication resources in your cluster. You can also see that we are, in this case pointing to a secret uh and not all identity, for example. So in terms of the operators, they also have the the controls of getting more information as we want, and of course, if you want to have a closer look, you can always use describe to understand. What's going on and have more information.

A

In the meantime, if we check our q is already empty, so we have basically fanned out to process the whole cube and if we go back, you see that data already scaled all the way down to zero. So if we do, I get parts, there's basically nothing to show here, because nothing is up and running.

A

If we now queue back up two messages of 10 sorry, you will see that in a second cada will see those messages again and we'll start uh scheduling the work again.

A

So this is just as simple as how you can do auto scaling with cada. You have an existing deployment, you just say how to scale it and how far and then we will handle everything for you, you don't have to worry about hpa, metric, adapter, etc. If you want to, you can see the hpa that we use under the hood. You can also influence the hpa by tweaking it how it should work.

A

You can see in our documentation what that should look like, so you can also have control there if you have to, but generally we say that you don't have to okay. So if you're, not a kubernetes expert, you can also do auto scaling fairly easily.

A

Now, if we have a look at the community that we have around cada, is uh it's growing uh fairly a lot? So we have more than three k stars on github and more than 100 contributors from various companies like, of course, the maintainers, but also people like ibm, shutterstock and other companies, and we do bi-weekly community stand-ups on tuesday.

A

So if you are interested or have questions or doubts, please make sure to join one of those and we're happy to discuss them, and you can find more information on our website now, most importantly, who is using cada? Is it already mature enough? Do people already trust trusted and I'm happy to say? Yes, we have various products that are relying on cada to do. The auto scaling, for example, apache, airflow and astronomers are auto scaling the workflows with cada.

A

We have serverless um tools like azure functions and fission, who are relying on cada to do the application, auto scaling, and then we have all this uh like dapper and k native would use it as well, but also in terms of end users. We have a growing catalog, including major uh companies like alibaba cloud and microsoft, and also others like coralux, cora logic, sorry and the shader hyperion roads works and others, and we're actually constantly adding more and more of them.

A

Now, if you're interested in, why alibaba cloud decided to use scada for alden application, auto scaling, you can go to the cncf blog, where we did an article with the kind people from alibaba cloud talking about why they chose for cada and how it helps them do auto scaling very easily.

A

Now another example was azure function, so uh I will not dive into the demo here, but it is just as simple as using the azure function, core tools um you can deploy, azure functions to kubernetes and behind the scenes they are automatically installing cada for you creating the scaled object for you, so they are using cada to manage the auto scaling for their serverless workloads without you even having to worry about it.

A

So if you want to give this a go, it's just as simple as I'm doing these three lines and you are up and running on your kubernetes cluster, automatically scaling those azure functions.

A

Now, in terms of roadmap, um we are constantly evolving, but one of the major aspects that is missing today is http workloads. Http is a little bit harder to scale, because all the all the things that we mentioned before, for example, the queue you exactly know how much work is on the queue while with http.

A

It is synchronous, communication and you don't know how many calls will be there in five minutes, for example. um So it is a bit harder.

A

So now we have an an alpha uh version of this up and running on on github, basically, uh where we have an add-on scaler, which you can deploy separately, which is aiming to deliver scale to zero uh for http uh without a dependency on prometheus, and this is fairly important for us, because, if you're using a cloud vendor, you could also use their telemetry and observability stack and it's a bit ridiculous to just spin up prometheus to do auto scaling. So we want you to choose.

A

If you have prometheus already, you can use keda today and use the prometheus scaler or, if you don't have prometheus, but you still want to scale http. What you can do is use this add-on. You can deploy it and you can use the new crd, which is called http scaled object. Where you basically define what you want to scale.

A

You define the amount of replicas and then we will basically do all the hard work we support today ingress, but we're also planning on supporting gateway api service, meshes by relying on the service mesh interface and service to service communication.

A

So if this is important for you, please go and give it a try, share your feedback, and let us know what you think, but we are super focused on this, because this is very important. Okay now, how does it work in the current version? So you bring your own workload.

A

You bring your own service because you know how to expose your service and basically we'll use core cada and extend it. So we will have.

A

We have a data http operator to manage the new crd and basically, we use an interceptor between your ingress and your service so that all the traffic goes through the interceptor to measure the traffic, but also to hold the requests, because if you support scale to zero um with http calls which are synchronous, you have to hold the request until the service has spun up and then you, then you can forward that request through.

A

um But as I mentioned, this is alpha, but we are very eager to hear what you think and also what the scenarios are, that you're looking to solve and we can work with you to make it better now in terms of what else we are planning to do. So we are constantly adding new scales and new secret sources to authenticate.

A

For example, at the time of the recording, we are adding azure pipelines, so you can automatically scale build agents so that if you have a lot of pipelines or builds that are pending, you basically scan out on kubernetes to serve that built, pool uh sorry to build queue. And then you can basically scale as you want.

A

If you have any other needs, we are happy to add them. Frankly, um we have recently added kubernetes events to cada. Now we are also going to add cloud events in case you want to expose them outside of the cluster or integrate with existing tools to gain more insights, we're also working on adding uh building a community around the external scalar, so that you can also use the the great scalars from other people and also making it better to discover them.

A

That's why, for example, we're working with the artifact hub to expose cada scalars in there as well.

A

So if you today have an external scaler, you can now go to the artifact hub, add a new source and then it will automatically be listed on artifact hub and then lastly, we're working on cncf graduation to incubation, which you can find on github as well, but we are fairly agile uh where we have a public roadmap, and if we have people who have certain needs that are crucial, then we are happy to work with you and see what we can do, so don't hesitate and open those feature requests.

A

Now, if we go back to our kubernetes cluster, to summarize, um you can basically use scada in the whole, auto scaling bigger picture of kubernetes, where we have cada for the application auto scaling, so we can scale our app across nodes. But, of course this is not enough.

A

You can still use the cluster auto scaler to basically resize the whole cluster so that your application can even scale further, but at some point your cluster will be full cluster. Auto scaler will not be able to keep up and then based on the cloud provider that you use. You can also scale beyond the cluster and overflow the workload. So in this case, if you use an azure kubernetes service cluster, you can use the virtual nodes to overflow the capacity to azure container instances for that serverless kubernetes experience.

A

Now the beauty of data is also you can scale down to zero, which is also important from a cost perspective, because if our orders app goes all the way down to zero implicitly, you can fully rebalance the cluster. You can get rid of azure container instances.

A

You can remove one node and you can basically save a lot of resources in your cluster and with that I would like to thank you uh for your time and open up for q a and here, if you have any questions. Thank you for joining.