Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2022, 2 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Event-driven Autoscaling with KEDA - Zbynek Roubalik & Jorge Turrado

Description

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Kubernetes Event-driven Autoscaling with KEDA - Zbynek Roubalik, Red Hat & Jorge Turrado, Docplanner Tech

Nowadays, the popularity of event-driven applications is rising, they enable us to design and develop scalable, distributed, and flexible systems. Kubernetes platform brings the distributed and flexible aspect, though it doesn't provide any built-in way to deal with event-driven scaling properly. Scaling based on CPU and/or memory usage doesn’t fit well with event-driven processes. The majority of autoscaling solutions are usually complex, and their scopes are too attached to a specific provider. KEDA came to solve these problems, providing a simple way to gather the metrics from external sources and translate them into Kubernetes metrics to drive the event-driven autoscaling. During this session, we will introduce KEDA: what it is, how it works, show it in action, and discuss future development.

A

Hello, everybody uh we're gonna, get starting our next session, so hi everybody, my name is danielle. I'm cncf ambassador, I'm so happy to be here as a moderate this session today. So I'm really happy. I mean uh back to the return I mean in person event since pandemi.

A

So today uh we're gonna talk a little bit about next session. As you can see, the kubernetes evangelion, auto scary cater. So please welcome our next great speaker, uh jorge from uh dark planner. I'm sorry, my bad pronunciation, spanish and there's v-neck from the red head and the principal software engineer and a please welcome.

B

Okay, okay, first of all, thanks to all for staying here, we will try to teach us to be funny and just share the knowledge we are gonna talk about, kubernetes event, drive and auto scaling, aka, keda and first of all, who am I I'm jorge dorado, I'm an experiment of planner tech, I'm one of kedah maintainer board and also microsoft, mvp in developer technologies, and you can see my social media networks there.

B

Yes, if you want to write me, make me any question: you want no problem at all and now my fault spinyek will introduce pizza, hey.

C

My name is vinayak robotic, I know the name is pretty hard to pronounce, but both guys did a pretty good job. So that's fine, I'm from czech republic, I'm a software internet third hat and I'm working on openshift serverless, which is native stuff and I'm also kidding maintainer and yeah. We can go ahead so today we will talk about kedar. So this is the agenda and we can start so. Basically, what is kedar? What is the project? What is the main goal of the project if there is just one sentence?

C

One thing that you need to know after this presentation is that we are really trying to make kubernetes even driven, auto scaling that simple. So probably maybe we can finish the presentation. What do you think? Okay.

B

We have a lot of attendees, please work.

C

B

They want to learn a bit more. Let's continue. I.

C

Don't know, let's try to do it: okay, okay, okay, so I'll try something. So what is the use case? So let's say that you have very simple application: it's a consumer application that consume messages from some external service.

C

Let's say you have a kafka consumer application that consume messages from kafka topic, and you would like to auto scale this application, so how you can do that on kubernetes. You have hpa right, but with hpa the options are you can scale the application based on cpu or scaled application based on memory, but now it is even driven world right. Everything is 7-driven, so this might not be the best solution. How to drive the auto scaling, because the cpu or memory usage might not correlate with the needs for for the auto scaling.

C

So here comes kedah for the rescue. So basically, this is the example: application just redesigned to use scada. You don't have to change the application. You don't have to change the deployment you just plug it in and cada will scrape metrics from the external service. So, in this case it is the kafka topic and based on the number of unprocessed messages, it will automatically scare our application and if there are no messages, it can also scale down to zero. This is something that hpa cannot do.

C

There is like a alpha version that already in the kubernetes, but but currently you can do that with hpa. So this is the main goal of of the project, and basically this is just a quote from one of the users that he mentioned, because there were ways how to do the auto scaling based on custom metrics, but the the stuff to actually actually do that and to configure the custom. Metrics is very, very complex and we just try to do the same way but very easily.

B

Yeah- and we can imagine the common scenario before keda- I guess that you have been working with kubernetes and probably in some scenarios where you need a higher scale based on, for instance, h3ometrics aws matrix, whatever not important, as you could know, or if you don't know, I should explain it to you. No problem in kubernetes we have three different api metrics ipm one is a metric that is already locked by the normal metric server. The second one is custom matrix.

B

Third, one is external metrics, usually adapters like monitor from infuse or others blocks customers. What's the problem? Imagine that you are starting your project. You are okay, I have a rabbit. That's all, let's install directly the rabbit adapter yeah for scaling based on. What's the problem. Imagine that oh, but I would like to.

D

B

Metrics for it from insurmountable or from another provider monitoring system. The problem is that you cannot because the ap, the custom metrics api is already low, but your rabbits.

B

So you have two options: move all your things to prometheus scrap, scraping the metrics from different sources and just use it for the same, and with that change, you could use the normal approach of deployment plus hpa. Okay. What can I solve why kedah is useful, at least in my opinion, who could imagine that I don't get a useful okay. The point is that we have more or less the same scenario here we have the deployment, but we replaced the hpa with the scale audio.

B

The scale object is just a crd that get a register in kubernetes api apis, which don't worry. We will see one example of them, but basically it's another object that says to kera how we want to scale our application.

B

So kera will read this scale object and internally, we'll prepare all the needed things just for expose those metrics in a single network circuit so get a wheel, lock external metrics api for serving the metrics, but with keva we could serve all the different upstream. We could have a scalar based on service paths and other based on prometheus kafka and a lot of different scalers that get a support.

B

Just for giving some highlights about kela, maybe in the other side no problem, the only intention of cada is make the autoscaling simple. We don't want to manage any other thing than outer skin. The scope is really limited.

B

Keva started as a partnership between red hat and microsoft three years ago could be yeah more or less three years ago. Right now is a project of cncf, because kedah was donated to cncf and right now it's a science.

C

B

Incubation yeah.

B

It was a it was donated two years ago and after version 2.0 now right now it's an incubating project and currently the current version is 2.7, and usually we release new versions every three months, every three months, just for giving some highlights about uh how keller works and is get a project done by a group of friends. Of course, I expect that I consider you as a friend I.

C

Will think about it? Oh.

B

My god, oh you are praising my heart, but apart from us you can see we have more or less a bit less than 5 000 stars on github, more than 190 different contributors and several users and inside our contributors. There are it's, not a small companies, maybe the smallest companies. There is mine, but others are smallest companies like red hat microsoft and abn super small who knows about this right and also our users are quite different and we are super proud about them.

B

If you use kedah, don't worry you can be listed, we will be proud of you also, and yes, we're going the extension in the bottom. You can see that dot sh a community. You can see all the information about the community about how we manage the project because we try to be closest as possible to community, because it's a community project, not a business project from a company and every two weeks we have our open meetup with all interested people.

C

Okay, so let's talk about um the concepts and architecture, how can I design and how what is actually doing so? As we mentioned, we are auto scaling deployments kubernetes deployments. You can also spawn kubernetes jobs based on the events uh and also you can target a custom resource if the custom resource implements specific thing, but you can also target your custom resources so, for example, argo rollout, uh as we said before, we have.

C

It was you we have 50, plus scalers, so different services, aws azure, rabbit kafka, you name it and basically the main concept is really with scale based on the events in the target system. uh Important aspect is that itself does not manipulate the data you need to handle the data transfer to your application. We just do the scaling okay, so uh this is like, let's say the architecture, so there are two main components. There is operator that monitors the custom resources and then we have the metrics server, which provides the metrics to the hpa.

C

So under the hood kedar for each skill object, cada creates hpa. That does the scaling from zero to n and operator. Does the scaling from one from no from one to n and operator? Does the scaling from zero to one? So that way we can achieve the full scalability okay.

C

So this is the example of uh scale object. As you can see, it's pretty simple. We just need to tell it what workload you you would like to. You would like to scale. This is the lagdos scale target.

C

Then you define the minimum maximum replicas and then there is a trigger section and in the trigger section you can you can specify multiple triggers in this case it is just a kafka, so you specify okay, this is my kafka broker. This is the consumer group. This is the topic and the lag so based on the based on the lag it will. It will automatically scan my application. There are multiple additional options, but this is just the highlight how you can how you can do that, and this is skill job example.

C

So this is the other crd so for for spawning new communities.

B

C

Okay, okay, so basically it's a very similar thing here here you put your standard, kubernetes job specification and based on the evidence on the system. It creates a new new kubernetes job, so it's very good for batch processing and especially for processing of long running executions, because if you would like to if you would like to scale your application that um handles some long process, the hpa thing might not be might not be ideal because you know once you process the messages from the system and the processing starts.

C

The metrics are already already down so hpa might scale down your application in the middle of processing. But if you scale jobs, you can just spawn a new new jobs to do the processing all right, yeah some advanced features.

B

So basically we said or again I can say.

E

B

Now we try to don't reinvent the wheel. What that means, if kubernetes already has a super good piece of software of software name hpa controller, we just try to reuse it I mean we can expose the metric and then configure the hpa to request this metric. But why couldn't we extend that functionality in get catastrophe?

B

Why not? So because we can, because we can implement some other features like a fallback mechanism. What is a format? Imagine that your kaka broker your promise, your server. Whatever is done, it's done, so you cannot reach that. What do you want? In that case, you can specify using the dci feature set in the fallback replicas.

B

Okay, if in case of any upstream is available for five times in a row just scale to or not the scale do just consider that scaling that figure as a requirement of into the set replica, for instance, if you have more than one trigger, and only one phase, that trigger will request that amount of replicas and the other trigger could expose the other value, so the hpa will proceed if there.

B

Another important thing is that I hope that you don't need to customize the hpr, the hpa behavior, because it's a pain, but maybe you need to do it because one.

B

Specifying those values in a section in the schedule and get away with that faster the xpa, but you can do it if you need to customize, because you need to scale down a scale out or feeling faster or slower, you can do it. So it's a good thing. One point super nice and trust me. I have been on call today and this feature. Thank you. So trust me. It's a super good knife, the possibility, the capability of pausing, the auto scaling. What that means!

B

Imagine that you are under maintenance and you need to scale to zero in other mechanisms or with other tools. You need to barely remove the hpa, because if you would simply scan to zero your deployment, the hpa will scale up and we'll scale out the deployment. Again. With this feature, you can just add in an annotation in your state object, don't worry, we will see how to do it in the demo. You can say okay, never mind.

B

I want to have this fit a lot of instances and I take the rings because I know what I'm doing it's hard way for doing maintenance or just for dealing with problems. So for me it's one of the the best features that we have uh obviously kedah, as majority of software nowadays expose prometheus metrics, because if you don't do it, you are doing the things really wrong.

B

In my opinion, at least inside kubernetes and right now we have more than 50 scalers, but the most important thing the scalers are not opinionated scalers, that that means that they don't have any logic. They just go to the upstream and request the metric. If you need to do any logic based on the metric just no, I would like to have that metric plus one well, whatever not important the target, but you can have more advances.

B

Scenarios cannot support being extended, implementing your own grpc server, implementing the external scalar interface or if you are more comfortable with the rest api, you can just use the matrix matrix apis trader and just implement your own api rest api and just expose the value that you want and get the value from your work. You can do it so you can extend kedah easily to fulfill your requirements.

B

One example: one sample of these extensions is the http app. We tried it's not the objective of this meeting, but just for.

F

E

Oh, my god, oh yeah,.

B

I always break the microphones.

C

Sorry, you speak a lot, probably.

B

C

Let me handle that okay, nice.

B

Sorry, er http atom is just a an external metric server that we use for extending a and scaling base of ntl and http it's in beta. I guess, but it's an example. So there is an example of external metric implemented inside cada and in the last line, don't worry, this slides will be served, so you don't need to just copy in your blog. Your blog, don't worry is the link to the it's to the scale, object definition just for watching all the all the capabilities inside the scale object, and what about anti-authentication?

B

This is important topic in my opinion, is one of the most important things in kera, because keda usually needs based on the work that keda does usually needs. High privilege on different systems get a needs to the permission for listing kafka brokers for listing messages, so it needs some privilege that are quite risky if there are not good manners. So for for solving that, first of all, keda allows to reduce the credentials. I mean you don't need to just copy and paste potential every player and every place that you are there. You can specify together.

B

Okay, just go to the warlord and take this environment variable where it's already the the secret.

C

B

Yeah, I can dance no problem.

F

B

All and how can how can do that or how can go farther and reuse more queda has another two theories: another custom resources, one is trigger, authentication and the other is cluster trigger authentication. You could imagine the difference. One is name spaces and the other is just a cluster scope applied and basically using them, we can extend even more the security capabilities. Why? Because we can use, we can specify some secret from okay took the secret from the deployment or from that secret, but also from from both identities.

B

I don't know if you know what are both identities. Basically, port identities is a mechanism from several cloud providers for using even emma system managed identities in the in their site. So it's the most secure thing: it's like a system managed identities in azure or aam in aws, so it's the most secure option. But if you don't, if you don't use them, you can use hasik or bolt directly integrated inside kera, not using any other integration that can go to hashicorp, vault and just read from them and also from azure keyboard.

C

Yeah just to add that to this, the cluster authentication is good because, for example, you have an administrator who has some credentials and they don't want to share it with the developers. So he can, you know, define the cluster to go to authentication object and the developer just reference in the scaled object to this static notification object. So we can do this stuff as well.

B

Okay, um just for showing some five highlights.

C

So I will, I will maybe show the character it will show the parameters because I don't like it yeah right, yeah sure yeah. So again, this is like this is like the scale object. So, as you can see again, very simple, just the skill target, if you don't specify the api or kind it's automatically deployment, but you can define, as I said before, you can use the custom resources, for example, as a route. So then you specify the api api and api version of kind, then the minimum maximum replicas and the trigger section.

C

So you see it's just a couple of lines and you can explain the.

B

Stuff you catch me drinking.

C

G

B

We need to prepare better for the next time. Yeah. Okay, in this side, is the common hpa, nothing new under the sun under the sky, the common hpa, but it seems- or they seem quite similar. The problem is that for using this side for using this apa, you need to do this. Also, in the adapter I mean you need to configure the adapter just for renaming the metrics and exposing them in the name space. You will need them, so the hpa is quite similar, but the work and then the under the hood is higher.

B

It's really higher with kedah. You don't need to do this. To do this, you only need to spawn kedah in the cluster, install them and that's all.

C

Yeah, so if, for example, you would like to scale your one deployment, you need to define the rules. If you would like to add another deployment or auto scale it, you will need to again reconfigure the whole adapter. It is not like the crd, thingy, okay, and what about now yeah demo time so yeah talk is cheap. Yeah right, I can talk. Maybe, and you can.

C

So, as you can see, this is like the no.

B

They cannot see it. Oh.

C

They don't see why.

C

Okay, I don't know so. This is the deployment right.

B

Yeah, okay yeah: I can do.

C

It don't worry.

B

Okay, yeah, basically just for improving the demo. I will do it there just for not making a bit bully them all always presenting there. You can see there I for just for improving the process we have in. We start in a scenario I have already installed. Cada cada is already installing the cluster and I already installed a helm chart a head to answer. Obviously help chart a rabbit q. I install a rabbit server just for using rabbit scalar sample, but you can after the demo.

B

I will show you the where you can download the whole sample, but basically we have already deployed.

B

This file in the cluster this file just has a secret with the secret of the for connecting rabbit host. Has a super simple deployment with a mistake. The annotation is not there, but ignore it. It's just a sample like a rabbit customer, don't worry, eat the cues messages from the queue and do absolutely nothing with them, but the most important thing please scroll below is this. This is the scale object in this skill object. We are basically specifying the steady scalar charger drive. Basically what what we want to scale nothing important.

B

These values are for just for improving the speed of the demo and the max replica count. We want to have 30 replicas at maximum and the triggers. This is the place where we are. We are going to introduce the different scalars, the different rules that we want to use just for scaling in this case. Important is an array, so you don't need to scale based on only one you can specify.

B

All of you want without any limit without any limit, yeah, just double checking and the trigger authentication. Why trigger authentication just for specifying where are? Where is the connection string from for their rabbit server and we define the trigger authentication and we have specified it inside the trigger just for saying this trigger should use this authentication so go there and take the parameters. The secret parameters from there.

B

And just for doing the things totally transparent, I will deploy this job. This job is super simple. Just for enqueuing. It.

C

Generates the load basically.

C

You should mirror the screen right. You should.

D

B

As you can see, I'm not the best with mac os, okay,.

C

You are microsoft, fanboy, right, yeah,.

B

My bosses are more microsoft, haters yeah, that's good.

C

You have a typo over there.

G

C

Yeah, that's good.

H

C

I can hold it. Oh thanks so jorge just created the the publisher job, which is the stuff for generating the load.

B

C

You can see he can pre.

B

And right now we are going to watch that is what's cube, ctl yeah, yeah.

C

B

C

Watching the consumer application so he's watching the workloads, as you can see, it's already auto scale down to zero. I don't know why. I know.

B

Because yeah I make up a joke for him. It's a.

C

B

I added the annotation that I explained it during the during the presentation just for pausing the autoscaling and sewing, so we need so. Okay, sorry, I will remove it locally because otherwise it could be a pain.

C

Best demo ever.

B

As you can see, the annotation is there is this, so I will try to remove it. Oh my god. Sorry nice.

B

Okay, no problem and now automatically the autoscaling has started one instance, because we were in a zero scenario. We were just scaled to zero. So remember, the operator has increased from zero to one and now the hpa will take care. We take the control and we'll scale this workload. So in a few moments we could see that okay, right now, we have four instances.

C

Yeah because we are generating the load, so we are generating messaging to the rabbitmq and the application is probably processing them and it will be scroll down to zero right after a couple of.

B

Yeah after we will reach 30 instances and suddenly we will scale to zero again.

C

Okay, so maybe in the meantime we can check the additional stuff right because we don't have too much time. Yeah.

B

In the meantime, oh, my god.

B

This is the url. Basically, this url, this demo is there it's a public demo that we have. We use a rabbit and if you go there, you will see step by step how to reproduce this demo. Just for try in your under infrastructure, you could try it in mini cube in local cluster, never mind, no problem, that's why we try to use rabbit. They don't pay us for using rabbit, trust me or at least not to me. No, but basically that's the the url. You can find it in github.

B

Slash catacore, slash rabbit, okay, no, the scream is not mirrored.

C

So can you move the slides here? Yes,.

B

For sure, as you can.

C

See already autoscale down to zero, so it is pretty simple and maybe we can go further right.

D

C

So I'm gonna go to the slides.

B

That sounds easier than it is.

C

C

Okay presentation: you don't need to do the presentation right, nice, okay, again! No! No! No! No.

B

We are bored about you.

B

I will say that this is.

C

Yeah awesome, so what about the future of the project? Because you know we can already auto scale the stuff, but still we would like to stay focused on one thing. uh So there are several things that we would like to improve. So one of them is caching the actual metric values in the inductor.

C

So then you can save basically the traffic to your to your external service, so, for example, to do kafka broker or then maybe we can do some nice analysis on the on the values and maybe do some predictions, maybe plug some aiml stuff into and monitor, monitor the basically the metric values and then based on that maybe start scaling a little bit faster, a little bit in advance.

C

uh Another another thing because, as said at the beginning, though, there is the limitation that there could be only one extension point for metric server in the kubernetes cluster. So that means that you can have only one installation perform cluster, so we will need to somehow solve this. So this is another like a big thing that we would like to do.

C

Another cool thing is cloud events, because cloud events are everywhere right now, so we would like to you know, maybe expose some cloud events and do some cool stuff. So then you can, and you can integrate uh integrate, basically uh carry out with your portfolio, and maybe you know, based on the code events based on the statistics about scaling. You can do some additional stuff um yeah and that's probably it so. Do you have anything else.

B

Yeah, I have noticed that I don't know when we delete the one of important features that supports arm architecture yeah in the latest version. It's not critical, but if you are on aws and you are using graviton nodes, you could use also there. So do you have any question.

A

Yeah, so you know that so we have five minutes. So we have. We.

F

A

Million- and we have a bunch of t-shirts here, so if you have any questions and good.

C

Questions yeah t-shirt, bad questions, go.

A

Out that is a regular one, so yeah just uh bearing in mind. So we have a one virtual question: we're gonna maybe address that first. Okay, so can you scale a part according to the load of the difference path.

C

uh You you need to you need to somehow get the metrics from the different ports. So once you expose the metrics from the referring port, maybe from through prometheus, then you can scrape the parameters. Scale, scrape the metrics from the portfolios. So basically to do the scaling. We always need to have the basically the source so yeah this is doable. I suppose.

G

Cool yeah, thanks for the answering and questions okay yeah, I'm gonna, do prosperous.

F

I think the start at the start you mentioned you can create multiple scaled objects for the same deployment. Nope.

C

F

There could be only one.

C

Scale object per deployment, okay,.

F

C

Can specify multiple triggers in one skill object so if there could be like a trigger for kafka for prometheus for rabbit or cpu, targeting the same uh same deployment and then hpa works. This way that basically, it selects the the greatest value to drive the scaling. So basically.

F

So so it takes by greater greater value. There's no like.

C

Exactly five between.

F

Them so I can, for example, do something like uh scale by metrics and then say uh wednesday. I have an event at 4 pm. I want to scale it way up and it will not fight yeah. Okay, very interesting, thank you, but.

C

There must be always only scale object targeting the deployment, you cannot combine skill objects and your hpas. It could be only one one object. So if.

F

You know it, but it manages yeah yeah. Thank you.

A

Thank you so much and I'm gonna go to back. There go for it.

D

Thank you. um How does it work? For example, I mean in this scenario that you are upgrading an application, so you are triggering triggering a kubernetes deployment and is a scale to zero. What happened.

C

Nothing because uh because if you update the deployment uh cadaver after a few seconds, it will notice that the deployment has changed like the the replica number doesn't uh equal. What is basically provided by the metric server, so it will again scale through what should we do.

D

But that does that's the thing I'm wondering, for example, if you have a scale to zero and then you are constantly checking, for example, the number of messages in rabbit and q, and then you upgrade the version of that application.

D

There's! No! No! No! No okay and monitor.

B

All the time, the app stream, so if you don't have any messages in the queue, will scale your deployment into zero and cannot still be monitoring the the queue.

H

Yeah, when there.

B

Is any message in the queue will be keda put a scale again into one, and then you don't need to do any more more than deploy the same object once at the beginning.

C

And because you are always targeting the same deployment, it doesn't mean that it was changing. The version was changed, but it's still still the same deployment, so it doesn't care like so it should work. Okay, thank you.

A

Thank you, so I.

D

Think that we have one last.

G

A

B

Don't worry, you can meet us after the meeting.

H

B

Question directly to us: don't worry if you don't have time, we will be here.

E

Hey uh so does you know I'm thinking specifically around kafka, but it probably applies elsewhere. um Does kida support like partition level um scaling.

C

uh No itself does not scale kafka, it scales just the consumer application, but, for example, if you are consuming uh messages from kafka, it doesn't make sense to scale out the number of consumers that is larger than the partition, so yeah it's kept, so basically it will scale only to the maximum number of partitions. You can rewrite this, so we can. There is an optional value. I would like to scale to more consumers, but it doesn't make sense, but.

E

Yeah, this is covered. Okay, cool thanks.

C

Maybe you can do another question. There was one a day: okay,.

H

Hi hi, I saw in the demo that you were like you, put 18 replicas, and then he went up super fast. Do you have like throttling? Because.

C

This is the stuff that you define on the hpl level, so this is like the scale. It's called scaling behavior. So then you can define like the older options. This is.

G

C

That's provided by hp8 can be like fast.

H

um Can also be by external services. My limit can be based on the metric of external service. I'm not sure I get the question. Sorry in case like I am, I don't know putting in a amazon limit. I am using a delay services. I am putting like a batch of bots running.

E

H

Get the limit of throttling of eight or less? Could I put like a limit on that based on that limit by checking one metric from aws and to see if I reach in the limit of the throttling or not? I don't know, if is playing well.

C

Maybe we can discuss this offline because I'm not sure okay, I got properly or do I understand. Okay,.

A

Yeah, thank you so much yeah. So thank you. So a great presentation demo today and thanks for attending so enjoy the rest of the cube, call and officials yeah.