Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2022, 12 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: KEDA - Real Time And Serverless Scaling In Kubernetes - Zbynek Roubalik & Jeff Hollan

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

KEDA - Real Time And Serverless Scaling In Kubernetes - Zbynek Roubalik, Red Hat & Jeff Hollan, Snowflake

Event driven architectures are exploding in popularity, often coupled with the desire to make them real time. These applications enable us to design and develop scalable, distributed, and flexible systems. Kubernetes brings flexibility and a distributed platform, but it doesn't provide any built-in way to deal with event-driven scaling properly and in real time. KEDA is one of the fastest growing CNCF projects that solved these needs. Scaling based on CPU and/or memory usage doesn’t fit well with event-driven processes. Current autoscaling solutions are usually complex, and their scope is too attached to a specific provider. KEDA provides a simple way to gather the metrics from external sources (such as queues, streams, databases) and translates them into Kubernetes metrics to drive event-driven autoscaling. During this session, two of the current KEDA maintainers and creators will introduce KEDA: what it is, how it works (with demos), and discuss future development plans.

A

Okay, we are at 2 30 on the dot, so I assume we're good to go. We are good to go. Oh, hey! There's the sign begin, hello! Everyone! Thank you. So much for coming uh I'm very excited to be here today with savionik as well talking about Keda, which is the real time and serverless scaling solution for kubernetes a cncf incubations project, we'll jump right into it.

A

We only have half an hour plus a little bit and we want to leave plenty of time for Q a thank you to all of you who might be joining virtually hello to you as well. I can feel your strength through that little camera in the back.

B

A

You feel it too, oh all right! Well, maybe after we get through intros so very quickly, who are we and why are we so excited about Keda, so I'm Jeff Holland I am a director of product right now it's snowflake the data Cloud company and prior to snowflake, though I worked at Microsoft for about 10 years and part of my time at Microsoft involved, a partnership with red hat. uh When we created and founded cada is a project. This would have been I, don't know three or four years ago.

A

At this point you can find me on the socials at Twitter or LinkedIn. If you have any questions or just want to connect more than happy to chat with you there that Microsoft I was working on a bunch of the serverless tech like Azure functions, Azure, container, apps and so scale was a key part of that which is part of what inspired us to be like. Oh, how could we make scale better in kubernetes all up.

B

A

B

Jeff, uh hello, everyone again I'm very happy to be here on stage Jeff he's great and to talk about keraso. My name is binyak rubarik I'm based in bernocheck Republic, which is in Central Europe. In case you don't know, uh I'm software engineer, working on red hat and as Jeff mentioned, we are quite cooperating on Keda for some time, uh so I'm kind of maintainer and also active in a commutative Community I'm part of the TOC board and my main focus is kinetic functions- is a good project, so you should check it out.

B

Definitely, but now, let's talk about era, great.

A

And it's crazy that you flew halfway around the world and are facing jet lag, but you were here plenty of time in advance for the session. I flew from Seattle and I ran in here five minutes ago. So credit to you, I.

B

Guess you did a great job, so yeah.

A

uh Yes, uh you can imagine how good of a PM I am if I can't even show up if I like what, if what is it PM, if not somebody who creates slides and speaking of sites, we'll jump right into it, so we're going to talk a little bit about what cada is we're going to show a demo fairly early on and then we're going to show you a little bit of behind the scenes of how Kata works and we've got some really exciting stuff to share in terms of what the community is building moving forward, as I mentioned, we'll have questions at the end, so if you can hang on to them there, we will answer questions then so, starting with framing cada I like to to use this analogy, which is, let's compare two two scaling stories and here's the task.

A

Let's say that I have so my my task right now is there's a cubecon party happening tonight. If you're not invited to one come reach out to me, I'll give you an invite. There's like seven of them happening, probably more and here's the goal. You need to provide enough Pizza to feed as many people as show up at a cubecon party.

A

Now you have two strategies here: one is you show up with one pizza to start with and you put it down on the table and then you just look at that pizza and you wait until that pizza box is empty and when the pizza box is empty, you're like okay, well, I guess we need more than one box. So then you leave and you come back with two boxes of pizza, and then you watch those two boxes of pizza and you wait till they're empty.

A

And then you come back with four and then you come back with eight. You could do that. It would work. The party attendees will be ticked off at you, because if there's a hundred or two hundred or three hundred people there they're going to be waiting an hour two hours plus while you run back and forth getting a bunch.

B

Of people I'll be on the other party back then.

A

You know yeah, you would have left, you would.

A

About or whatever, when you're number two, this is what a product manager would hopefully do, I'm going to find out how many people are projected to come to this party and then I'm going to make an informed decision based on how many pizzas I think I need based on that. So if there's a hundred people I think okay, 10 pizzas is maybe enough.

A

Maybe I'll go with 12 to be safe and so I'll show up to the party with 12 boxes of pizza and if there's a thousand people I'll show up with 120 boxes of pizza. Now, if you were in charge of creative writing pizza for a party which strategy you're going to go with, I think everyone's going to go with strategy number two. But the funny thing is when it comes to scaling our applications. We so often go with strategy number one.

A

We deploy an application and then we just watch the CPU in the memory and we wait for all the CPU in the memory to get eaten up and they're like okay, well, I guess we need some more and so we'll give it a little bit more and then we just sit there and watch and it is not an efficient way to do things. So I am pleased to share that cada is the better Pizza way to scale your container because kada yeah? Thank you. It's a beautiful analog. It makes me very hungry.

A

I love pizza, so cada makes it so that your kubernetes cluster is much smarter at thinking about how to scale things, because it will Scale based on the events, what is causing the consumption in the CPU and memory, so it integrates with over 55 different event: sources from information from Prometheus event, sources like rabbitmq, Kafka, AWS, sqs, Azure, event, hubs, gcp, Pub, sub postgres. You can go to cada.sh and see a massive list of scalars and that list grows two or three every single time. We do a cater release.

A

So this is a really useful pattern because it makes it so you can scale your workloads based on the events, the cause of the consumption and not the reactive way that we've been trained to scale so long before, and what I love about Kate is just how seamlessly you can pop it into any architecture in any kubernetes cluster. So let's show you this in action before Xavier neck walks through some of the architecture, so we're going to do a quick demo here. This is my hello.

A

Cada demo I want to make it as simple as possible. We have a rabbitmq cue, there's a lot of fantastic queuing. You could swap this out with Kafka or whatever you want I'm just using rabbitmq, because it's the easiest for me to to install on a cluster and I'm going to deploy a container which is just consuming messages from that queue fairly simple. Now what we're going to do is we're going to add cada to the mix and cada is going to do two things. One! It's going to go! Ask rabbitmq!

A

How many events do you have that need to be processed? How many people are waiting in line to get into this party and then based on that information? Cada is going to scale our application based on those events, and it does that scaling through the horizontal parado scaler, the HPA, that we all know and love. It's not doing its own magical scaling thing behind the scenes. We'll talk about that in the architecture, so you'll see how Kato will scale this one thing I want to call out here right before I show the demo.

A

The application has no idea that cada exists just like in kubernetes. It's not aware that it's running in kubernetes or if it's in a laptop it just wakes up and says start giving me messages so Kate is able to pop right in there. I didn't change my code in any way, but it's able to enrich the scaling logic based on knowing in this case, how many messages are on that queue. So, let's see it in action. This is the deployment manifest that I'm using beautiful, beautiful yaml.

A

You can see I just have a very simple container here. It's complaining because I'm not requesting any cores or memory. That's the squiggly yellow lines here, but I'm just deploying a container and what cada is having me Define is this custom object next to it, which is this cada scaled object, and this is where I tell tell cada about the event source and what I want it to scale so I say: I want you to scale this rabbitmq container and I'm going to scale it based on events coming from rabbitmq and I mentioned.

A

There's like 60 of these, that I could choose to scale it on I'm, telling it to kind of Target about five messages per container. That's like saying how many pizzas per people so I'm saying every container go for about like five messages. I tell it the queue to listen to and I can even describe an authentication parameter so that Kata knows how to securely ask rabbitmq how many people are in line okay. So this is what I've already deployed in my cluster. So let's see that here so I can now.

A

At this point, my Q is empty I'm going to go ahead and say: sorry, let me zoom in here, I, don't know you can see the last time. I ran this, which has spoilers okay. So right now my contain container is running one beautiful thing: it's scaled all the way to zero. This container is actually not consuming any CPU in my cluster, because cada is smart enough that it's looking at that rabbitmq thing and it's saying, there's no messages here.

A

Why waste the CPU Cycles, so cada has helped the scale all the way down to zero. Now what I'm going to do here is I'm, going to add a quick job which is going to publish I, can't remember a thousand ten thousand messages onto this rabbitmq queue. So now, let's watch this in real time and you'll notice, even before I finish talking. Cada now sees oh shoot, there's actually a lot of stuff going on here. There's a lot of people who need some CPU so very quickly before the first CPU cycle, even got hit.

A

Cada has already helped me scale up to you can see. I now have four instances of this container up and running, and even before, I finish that sentence. Now it's gone to eight and it will continue to scale and scale and scale because I told it hey if there's a thousand messages in the queue I want you to Target about five five messages per container. So it's now helping get to that state where it's able to handle the workload as I described it.

A

So you can see very, very quickly, I've been able to horizontally scale my services now. What I want to do quickly here as well and show you what happens if something goes wrong, so I have defined one more piece here, which is some fallback logic that says if the connection to rabbitmq fails, what do I want to do? Does it now scale all the way to zero and my application blows up?

A

Well, I I wrote this fallback logic to say you know if something goes wrong, just run it for run it for replicas, that's a pretty safe spot to be so. Let's make something go wrong. Let's uninstall rabbitmq from this cluster that we've been talking to, and so you can see here my rabbitmq just got terminated and as soon as it got terminated, cater realized. uh Oh I can't I'm trying to ask rabbitmq how many messages it has I'm, not getting any answer back. Who knows why it might have failed?

A

In my case it failed because of user error. But what you'll see here is I finished talking is that it's going to end up in that fallback mode. Hopefully it will well. The containers are now unhappy because they also can't talk to rabbitmq, but you can see here if I I'll switch to a view where you don't have to look at all of the errors. It's now trying to run with four instances, so it it's even Kate has helped us fall back into a state where it's like. Okay, something went wrong.

A

I can fall back now. I I won't walk through this now, but if I just add rabbit and Q back into the mix, everything will go back into a healthy State. It'll go back to consuming whatever messages might still be there and and going there. So hopefully that made sense. I had a simple container, I told Kata scale this container, based on events from rabbitmq.

A

It very very rapidly got to the right amount of containers, I needed to consume all those messages, and then I even have some nice features here, like fallback to make sure if something goes wrong. My whole application isn't just floundering, because this doesn't know what to do so. I think at this point we might even be at a healthy State, we'll get there soon enough. So that's a quick hello world demo very, very kind of simple to see. Any container can now point to cada.

C

A

With that, sibiana can walk us through a little bit of the architecture.

B

Yeah, thank you Jeff. That was great awesome demo, so I hope that by now you have some understanding how Qatar works or what it does actually. So we can take a look at our architecture, so Keda is built on top of kubernetes, so we are trying to use as much components and Concepts from kubernetes. So to not so do not do the you know the stuff that yeah.

A

Then things that already exist exactly.

B

So um and the main component is scaled object or skill job. This is the main resource where we Define the metadata for scaling Us in that in the demo. So let's say that if we go back to Remo what have done if I create a scaled object in my cluster, there are two main components of Keta: the first one is scale operator or controller. This is this box and the second main component is scada, Matrix, adapter or kinematic server.

B

So once Jeff created the scaled object, the cada operator spotted this resource and based on the metadata definition, it creates HPA with the with the proper metadata, with the proper scaling Etc and then it uh it checks. The scale object again and see what kind of scalar has been used for for this skill objects.

B

So in this case it was rapidmq, so it creates a scalar object, which is basically an adapter that talks to the external service and queries for for the metrics, so it it creates the scalar talks to do external service and pause the metrics. And finally, because S as Jeff mentioned, we rely on HPA for the auto scaling, but HPA cannot scale down to zero. It is not able to do that, so we need to do that from the from the kerasite. So, okay operator does the scaling from zero to one.

B

We call this phase an activation phase and you can even specify specific threshold for this so in in a moment that there is like the threshold in on the on the Queue. It will scale up in this case when we are on one replica of our Target application, then the second main component kicks in and basically it's a it's a matrix, adapter that's registered on kubernetes endpoint and it fits the external metrics from the scalar to the horizontal Port Auto scaler.

B

So there is a horizontal scale control in kubernetes and it asks for metrics and we are providing them through the through this interface. Again, it's a standard, kubernetes stuff, so we are trying to to be transparent in this. So I think that that's the that's the main logic quite simple and.

A

The one thing just to call out and I know: I mentioned this a few times I love about cada. Is you can pop in that operator without impacting anything else, so you might have already 50 containers running in a cluster. You go and install the cada operator at first nothing's, going to change, and maybe now two of those 50 are like yeah scale, these based on events. Those two are still going to be scaled via the HPA Sabi convention, but now they're going to be enriched with all this extra data from cada.

A

So this makes it a really nice design to incrementally adopt, as you want, without worrying about, like oh shoot, do I have to now have like a cada, flavored kubernetes cluster. It doesn't work like that. It's it's very much just doing its thing on the side, you pull it in when you need, and you ignore it when you don't yeah.

B

So there are two main resources to define the scaling, so the first one is scaled object. Basically with scaled object, you can Target any deployment, stateful set or any custom resource that implements this specific sub resource. It's called scale, so we can target targeted here and the second main part of the skilled object is basically the trigger section. So in this case it is a Kafka trigger section. So it says: okay, let's, let's scale this deployment based on the messages in this Kafka topic.

B

We can have multiple triggers in this in this trigger section, and in this case we are sending all the Matrix from from this particular trigger, so it could be Kafka with mq Prometheus all targeting the same deployment and we are sending the metrics over to HPA and HPA. Does the decision on on the scaling itself, so it basically at the moment it selects the highest value from from all the from all the metrics foreign. The second component is called scale job. This one is is very good for for long running processes.

B

So basically it does not scale an application or deployment, but it uh but its schedule. A kubernetes job based on based on the events happening in the external system, so it's the very same trigger section so again the the Kafka topic with video threshold and instead of specifying a Target, we can put playing kubernetes job definition and it will. It will create the kubernetes jobs. Based on that. This is very important aspect, because if you have some long running application that it process some data from the external system, it takes hours to process actual stuff.

B

If you use the HP approach, the HPA may actually kill your application during the processing because the metrics are already gone, but the application needs more time to process the stuff. So in this case you should use skill jobs and to spawn a skill jobs space based on that metrics.

B

This is just an example, because what we are trying to achieve with Keda is is Simplicity. It's you, it's Simplicity from the user perspective. So as a user I don't want to deal with, you know configuring all the stuff on kubernetes I just want to plug it in create scaled object and do the scaling. So this is example: on the left hand side you can see a scaled object on the right hand, side you can see basically the HPA with with some custom metrics. So you can. You can actually implement this on your own.

B

You can Target this very same endpoint endpoint in kubernetes, but you will do you will need to do all the all the stuff that you don't want to do. So we are trying to to make this very simple uh yeah and let's talk about some Advanced features, so you have already seen the fallback in the demo. It's cool uh also. What is what is nice? This is HPA scaling Behavior. This is Upstream kubernetes, Upstream, HPA staff.

B

um It's it's very important because sometimes the metrics may go up and down up and down, so the replica number, maybe maybe plugging as well. So you you would like to you know, avoid this situation. So in the scale Behavior, you can specify a window which will be used for selecting the Target Target or pick a number. uh Other cool feature is posing or photo scaling. So imagine that you have your deployments, your applications, your scaling.

B

Everything is perfect, but you would like to do some maintenance and you don't want to remove the scale object from your cluster. What you can do you can just annotate the skilled object with The annotation and pause the auto scaling for for the time for the for the outage or forever.

B

um We also think that auto scaling is, is nice? It's fun if you have it in a cluster, but if you have many, many scale objects in your cluster. It's hard, you know to get uh beyond the picture what's happening in the cluster. Actually so observability is is very important for us. So at the moment can I expose this some Prometheus metrics about what is happening.

B

What is what it does, how it does the scaling, but we are still trying to improve improve on this area and yeah, and the last thing I would like to highlight is that, as you have mentioned, we have like 50 plus different scalars, but if, if those carers are not right for you, you can Implement your own and you can use uh it's called external scale interface, it's basically a grpc interface, and if, if you implement this interface, you can Implement your own scalar with your own logic and just plug in together and cada is written and grow.

B

If you don't like grow, you can write it's dot net Java whatever and just use the RPC to to talk. Look at that to do the scaling for you. If you want to know more about the spec. This is the link for to the recommendation, and last but not least, is the Authentication. It's important right. So because, if you, if you, if you need to talk to some external Services, you need to store the credentials somehow and most likely you don't want to store them directly in scale object in gamma.

B

So you would like to have some reference, so there was a reference on the trigger authentication object, so it basically can hold references to Secrets vaults. uh You can even use spot identity and it's namespace based, so you can have a one particular object. Try to take out integration object in a namespace and reference it in multiple scale objects or you can use cluster-wise trigger authentication, and this is useful. You, for example, you are admin on the on the cluster.

B

You can set up the crosstalk authentication object in one end space with all the credentials and the users that will do the actual scaling that will create the skill objects. Can just reference to this. Cluster authentication object without even knowing what are the credentials. So you are not exposing the data to more more users than necessary.

B

Okay. So that's the current state, that's the what what we have with Keta. It works. It's awesome, but what's next, so the first point I would like to highlight is that at the moment we are just feeding the metrics to HPA.

B

So if you recall that was the two main components get operator and ketametric server, so we are just sending the metrics over to H to HPA through the metric server uh every time that HPA asks, but this might not be the best case if you have tons of scaled objects in our cluster, and maybe all of them are getting the same external service. So, for example, the same parameter server so there could be. There could be like a very high load on this on the server.

B

So once we are able to Cache cache the metric values in the in the Matrix adapter, we are able to tell okay. They are not to ask for metrics every time, but you know store them and give give some give some value later.

B

The other cool stuff that we can do with the with the caching is we can do some magic with the actual numbers so once we have, but once we know what are the actual metrics coming in the in some period of time, we can do some, maybe just some smoothing some some Ai and our models. Etc.

B

um Do you want to follow up.

A

Yeah on the uh for custom logic, with evaluating multiple triggers, so in my example, my container just had a single trigger listed in that list, which was rabbitmq.

A

You can actually provide multiple of those you could say like rabbitmq and CPU and memory, and this cron task or something else now the challenge with that is when it creates it behind the scenes and it actually goes and registers all that with the kubernetes auto scaler, the HPA, it's going to be publishing all those metrics and we leave it up to the HPA to decide when to whether or not to make a scaling decision and usually today, I think Dominic was mentioning. It just goes.

A

Whatever number is the biggest: that's the one that's going to prioritize and so moving forward. We also have some asked to say: like hey: can I have some more customized logic here? Maybe I want to take the average of a few different uh sources. Maybe I want to prioritize one over the other, and so doing more of that logic. On the cada side that we can give the HPA just the info that it needs to make the scaling decision on the cloud event sides have been mentioned. These last two are actually related to the aspect.

A

You know we integrate very cleanly with Prometheus today, both from a emitting Prometheus metrics for you to operate, but also scaling based on a Prometheus time series query. We also want to integrate with Cloud events so that, if you uh cada itself can emit Cloud events. So when something happens, K10 it's Cloud events, but also there's an issue of understanding hey. How could we scale Things based on cloud events that might be coming from other systems, similar story with open Telemetry?

A

Having that same there's an open, Telemetry scalar being worked on right now, so you can make scaling decisions based on Telemetry from open, Telemetry and deeper integration with that and then. Finally, this open interface for predictive Auto scaling.

B

Yeah, that's another cool feature. Basically, because we have, we are getting the metrics from the external system, so we can. We can try to do something with them. So we are thinking about introducing some interface that we can plug some aim model, for example, and based on the incoming metrics. We can predict the actual actual state, so we can say that we would like to Auto scale the application. You know every Monday because of the of the prediction.

B

At the moment we have one one, one scalar that is doing this stuff, but it is like a talking to some external service, but they would like to do it like an open way and another important thing is environmental impact. I know these days, it's it's very important stuff. As we know all the situations happening around the world, so we are actually uh cooperating with with the cncf environmental sustainability group.

B

That's trying to solve issues related to this to this to this program, so we have done a POC actually uh that will integrate with Co emission intensity or power consumption or similar data and use the data to actually modify the cadaskaring decision. So in this in this, uh in this specific POC, what we have done is that, based on the carbon impact we can we can cap the actual maximum replica numbers that we will. We will Auto scale to so.

B

Basically, if, if the, if the, if the, if the carbon stuff is, is looking very bad, so let's not scale to 100 replicas, but just you know to to, for example, 10 or 90 replicas, or something like that. So this is the POC that we have done. uh It is not this approach, it is like a separate controller, but we would like to see in the future something like this directly in the in the scaled object. You can check the the POC here. There are leaks.

C

And links and recordings and.

B

There are also some some talks that keep going about about this stuff, so you can, you can check them in the schedule. Yeah.

A

And all the slides that I'm showing here are uploaded to to scad as well, so you can click the links if you want. This is something I, I really love. This is something that has been uh I. I've spent a lot of time in in the serverless space and I know a long time ask has been hey. I love the elastic scale of my serverless compute I would love to be able to control that scale so that I'm utilizing energy efficiently.

A

Maybe if the data center is under heavy load or there's a lot of carbon emission delay like don't schedule my jobs right now, this notion of being able to have scale rules not just based on the events themselves, but the environmental impact in the environmental environment that those are running in I is super important. So all of these things that we've talked about in terms of what's next all of these are great opportunities for contributors.

A

Huge thanks to the community. I've been blown away. If you go to cada.sh- and you just start scrolling about halfway through you'll, just hit a wall of logos of just like all of the amazing companies and people and organizations, who've been involved in helping make K to great. So for those of you who are here, who are a part of that Community already who've been contributing opening issues providing feedback huge. Thank you from sibionek myself and the rest of the maintainers who can't be here and for those of you who are interested.

A

This is a really great project to go collaborate with, because it really is it just does one thing: it's not a full-fledged serverless platform, it's not doing TLS termination and, like that's great there's a space for that go, contribute to those as well, but Kate. It's like I'm, just going to do scaling I'm going to do it really really well I'm, going to make sure you show up to the party with the right amount of pizzas and everyone is happy so thanks everyone so much for your help and and with this and I.

A

Think at that point. Yes, before we go to questions, please submit your feedback on the.

C

A

So if you want to snap the QR code- and let us know what you thought- hopefully they'll invite us back for more kubecons in the future, we can share some of the more exciting features or let us know if there's things you'd like to see done differently. Yeah.

B

Thank you very much and do you have any questions.

A

We'll go right here in the front: I don't see any mics, so just speak up and then I'll repeat it so that everybody can hear. Oh, oh yeah, or if Savannah wants to run around.

D

A

D

I have a question yeah. My name is Dan and I. Have a questions like what is the interval that you set for the appropriate Prometheus server for like a scraping the metrics, because uh usually I use like 15 seconds and.

E

D

Impacts scaling, sorry, another questions about like creative: how? How does it compare to K native sure.

A

So all uh activity you can answer.

B

Both questions yes go for it, uh so the polling interval- that's that's there. That's for the activation phase, only that's only for the operator, so this is for the initial initial skating. So this is for the zero to one. Then it is up to HPA to ask for for this interval and HPA. The default value is finalist taken 15 seconds, but what we are trying to do we are with the with the cache.

B

We would like to also also create like this interval for for the HPA also, so we will like send the metrics only in the specific time, but at the moment it is just for the uh for the initial one.

A

Yeah- and you can see like here, I for my demo I said- ask every five seconds to rabbitmq. You can configure this to 15 seconds 30 seconds 60 seconds, but today we're going to explain once it goes to one. Then the HPA will just ask on something period like what's the latest metric and Kata will go and fetch it and bring it back for the HPA.

B

Yeah and the other question yeah, it was like it was like yeah. That's a good question. I quite often get this question. um It could be a long talk but I'm trying to do it in a short way. So basically I don't see anything as a as a like competitors, but it's more like a complimentary tool. So because, uh in a messaging systems there could be like two ways: how to deliver data.

B

It could be pool based approach or push-based approach, so the pull-based approach is basically that the application needs to handle the connection to the external source. So let's say we have application that talks to the rabbitmq. So in my application, I need to open the connection and manage all the data delivery to the application. This is the full based approach. So this is where cada works on.

C

B

Other hand, the push-based approach is that the application doesn't need to care about. Like opening the connection Etc, it is just waiting for incoming requests, so both approaches as like pros and cons. You know it depends on the on the use case that you have so, for example, if you would like to Auto scale your application based on incoming HTTP requests, because HTTP requests by its nature are push based, it's very hard to achieve it with uh with uh with with HPA and with Canada, with this approach.

B

So in this case I would I would choose K native, but also K, native and cada. They are like it's just Auto scale. We are trying to do the auto scaling that simple, uh with with Keda it's more like a serverless platform, so you can do more stuff. You can do like separate configuration. Eventing stuff so really depends on the on the use case. It's our complementary tools, excellent.

A

Yeah I think yeah back here, like the third or fourth I. Think was the second hand up, at least when we asked for questions.

E

Thanks I I have a question about: uh what's the suitable workload type uh that is cater is most appropriate for because I assume those events in the event, queue should be homogeneous, but what if they are not what if they are heterogeneous jobs and um in your example, there was like four jobs or four events handled by each pod. But what? If? It's, not that even.

A

Yeah, so on this one, it is like the the the the ideal scenario for keita is where you can have horizontal scale: they're not like they're, not having to uh depend on each other, they're kind of independently horizontally, scalable and I would even say, especially for scaled objects. If processing a single message can happen within some reasonable amount of time, I don't know what the number you'd put on it like two to three minutes or something. Cada is great.

A

When you have longer jobs like let's say you want to transcode a video and that transcoding job might take three hours, that's still where you want to use the scale jobs pattern where you're spinning up one job per video in this example and spinning it up. So if you do have a scenario that might be a little bit more uh heterogeneous, as you mentioned, like there's exceptions here and there I guess, the only other one I would mention too, and this came up recently on our slack Channel Shameless, plug to code of the slack Channel.

A

Somebody was like. Oh can I Scale, based on the content of the message. So it's not just that I want to know. Are there a thousand messages to be processed but I want to know? Are there a thousand messages and based on the content, to your point, like maybe one of those message takes three hours and the other messages only take three seconds. Kata today isn't gonna know the the intricacies of that gesture you're going to need to put something else in the middle yeah. I know, K native.

B

This is basically the use case. 4K native I would say a community Eventing because you need to know the the specific what's inside the data actually because nature doesn't know about Rita, it doesn't care about data, it doesn't handle the data delivery, so I would use some different systems like anything or any other Eventing solution. That requires more complex logic.

A

Yeah, so some variance is okay, but if it's like two hours for one message and 30 seconds for the next kata's gonna have a hard time figuring, not the right stuff. Today, today, okay I think we have time for one. Maybe two more questions see when I call it. You choose I, don't know who would have been there.

E

So how do you scale Keda itself.

D

B

You don't scale.

A

The scaler needs no scale.

B

Yeah uh yeah this is this is issue because we already are relying on kubernetes API and unfortunately there could be only one and one like let's say thing that connects to the tourist endpoint, providing it's providing the the data you know. So there could be one only one metric server per cluster, so we can have multiple replicas of the very same, very same metric server, but you cannot plug any other tool. So you can. You can scale up the number of metric servers, but uh kubernetes by default, always ask the one.

B

You know so it's we are trying to. We are trying to do this like the scale internally. So basically, we are spawning more processes. Etc, but um I don't see any big reason for for the for the like actual scaling of Cato.

A

Yep, the primary bottleneck, though, is that custom metrics adapter you can only have one uh and it's okay, that's cada, even if we spent multiple processes, that's still.

B

uh Yeah, this is a usual ask that I would like to have multi-pocket installations in my cluster. That's not possible because of this limitation that you can have only one metric server. So even if I have like multiple installations, they will share the very same metric server. We are trying to solve this, those Upstream so so to have some kind of proxy in in between, like between the end point and but it's still still stuff, that we need to figure out how to do that.

A

So if anyone here has contacts with the custom, metrics adapter team- um yes I know- we've talked with them in the past, but so.

B

A

More, it's got to be a short one before we get kicked out.

C

Yeah is there a time you've switched to using gcp's Cloud functions or AWS lambdas.

A

Oh, so this is just an opinion piece at this point, because neither of these necessarily would relate directly to cada gcp Cloud functions or AWS Lambda. You know what I expect I used to be the the product lead for Azure function, so I would have said neither Azure functions now: I'm in a multi-cloud snowflake world and I, say they're all beautiful they're.

C

A

Would say: choose the function provider where most of your other stuff is so, if you're already using gcp for a bunch of other stuff, go use. The gcp function thing: if you're using AWS go use the AWS thing, if you're not sure, try to get it to work on snowflake. That's all I'll say we don't have Cloud functions, but just do it anyways. Thank you thanks. So much everyone we'll be around they'll. Probably kick us out. So if you season Ecker myself around, let us know, but thank you again so much for coming this.