Cloud Native Computing Foundation KnativeCon EU 2022, 19 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: What is the Knative Asynchronous Component?- Angelo Danducci II & Michael Maximilien

Description

Lightning Talk: What is the Knative Asynchronous Component? - Angelo Danducci II & Michael Maximilien, IBM

Currently, all Knative services are called in a synchronous fashion. However, in many use cases, a blocking request / response primitive is not sufficient. In particular, for data processing and AI use cases, a blocking invocation approach is sub-optimal. The execution of these services is often long running and surpasses the timeouts for responses, or result in the client having to manage a multitude of pending blocking requests. A more natural invocation pattern is to allow for “fire and forget” or asynchronous invocations, where services are called in an async manner. Doing so allows the client not to block as the service execution is unraveled. The Knative async-component aims to achieve exactly this invocation pattern. Best of all, it does so in a natural and progressive manner that makes any service asynchronous with a simple label and lets the service’s caller decide when to invoke the service synchronously or asynchronously. The project is still in incubation but once it reaches beta-level, we can encourage Knative users with similar async use cases to download and try it in their own Knative clusters.

A

Okay, thank you everybody. um So this talk is with uh anjiro. He is not here, uh he's a colleague at ibm, so I'm speaking on his behalf. uh So first is this is angelo. So if you want to contact him, he's pretty much writing the code, but, to be honest, there's also a few other contributors. A lot of people have contributed to the to to this, like evan matt in the past, and then also be married belinda. Who was actually the lead for this? She did a lot of the work unfortunately she's at github.

A

Maybe fortunately I mean, I guess you know things happen. uh Anyhow. I basically review code these days yeah. I know I know I know so. What's the problem statement, I think salah boy right. He talked about this. The issue is that a lot of times in uh serverless environment, you wanna call a service, but you don't wanna necessarily wait for the response and the reason is maybe it takes a long amount of time for the computation to happen or some network communication that needs to be done.

A

So there's there's a series of use cases I'll mention some of this. So there's that situation, where you don't necessarily want request response, you want fire and forget right and there's other type of models for serverless. So this is just one aspect of it. Okay, and for this particular case, where you have this problem, the asynchronous component can actually help you and it will show you that. So what are the use cases, for instance, when you're doing ai, where you have to run a model and compute or actually train the model?

A

For instance, you could be doing that and you may need to you know crunching a lot of data before you actually get the result. So you want to request, do all this computation and then you have the response at the end. That's just one use case. uh Obviously, if you're dealing with you know, for instance, at ibm, we do that a lot where we're dealing with existing technology. That may be uh doing.

A

You know all kinds of computation, maybe like spinning off a team with a bunch of people working and then you get a response right. In that case, you don't want to wait right. So that's the main thing. So let me show you the demo it's about, I would say about one minute and let me put my passcode a minute and 40 seconds, but I will narrate it so.

A

Here's an application, this application, if you notice it, has networking canadian dev ingress class set to the async.

A

If you try to use this application- and you have the canadian async component installed on your cluster, if you call this application by default, it will just wait for 10 seconds. So that's what the application does right. It just waits for 10 seconds. So imagine it's more than 10 seconds like 10 minutes 10 hours and you can see it's waiting uh and just a curl, but if you now curl it uh and you pass a header.

A

This is the new innovation that we added right prefer async respond async in this case, you'll see that it returns immediately with a 202 which in http, is that you know it accepted it.

A

Okay, so the computation will happen and you can check it for it later and of course you have to set that annotation on your application, but we've also made it even better whereby you can set your cluster to always have async so that all the app all the services that are deployed on that cluster will be async and that's what we're showing you here. uh So your application doesn't have to have that header and it will all it will be uh um async if you pass the the the header.

A

Now, how do you change your service to have always async or not uh your cluster? Sorry, you have to patch the config map the config network. This works with all the different network, ingresses, so estu, contour and so on.

A

So this is pretty much it for the demo. Let me now switch to slides to just give you a big uh big picture view of the architecture. This is maybe the one that's the best to explain. The key thing to realize here is that we are making very few changes to the k native architecture. Everything in green are the things we're adding.

A

So the way to think of it is when, when you make a request for to a service, so you're basically say curling that service instead of going to the service, it goes to a queue, and then there is a consumer and a producer right. The producer is putting stuff on the queue and then the consumer is consuming it basic computer science, 101.

A

Now, of course, how do you make it scale and does it scale? I guess that's a question. I'm sure a lot of you are asking. Well, we did some initial testing and this is what we found out. So if you do synchronous, meaning you don't have async you get about 153 requests per minute uh per minute.

A

If you have the async, we get 392x improvement now. Obviously, people like evan will tell you immediately. How did you test it which applica this is the initial result, I'd love for you to try it and let us know, but you get a significant improvement because you're not waiting right things are just putting on a queue and then the consumer, um the the consumer, can consume it as fast as it can so, depending on we're just decoupling, it's again, there's no magic.

A

It's basic computer science, but if you do this to your native cluster, you all of a sudden have asynchronous requests to all your services. That's the beauty of it. So if you have any questions, please go to the async component. Try it ping, uh um you know angelo and myself and uh we'll try to and we'll try to help you. So thank you for your attention.