Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 23 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: FaaS is Not Only the Serverless: Stream Processing with Serverless - Jun Makishi & Kensaku Komatsu

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

FaaS is Not Only the Serverless: Stream Processing with Serverless - Jun Makishi & Kensaku Komatsu, NTT Communications

Function as a Service is the popular solution of Serverless, which runs a short-lived function. In contrast, we will show a new Serverless: run long-lived function to deal with stream data. Our proposal is unique since it starts multiple Serverless functions for single stream, and let them keep processing the stream one after another. This enables a new use case like enriching an application with AI-powered analytics using video stream in real-time. NTT Communications has developed a new platform with Kubernetes and gRPC. We will throw a live demo to send voice stream from browser to this platform and run media processing functions in real-time. The functions will be open sourced, and all participants can try it out from tomorrow.

https://sched.co/MPeU

A

Okay, hi folks, Anthony, okay, so presentation so that we will get stretch and the scientific coming to this forum and I very appreciate to have a Tech Talk at a great event. Creek on today. We'll talk about the farce is not only the services. This presentation is, we will expire, but a media stream processing with subtle restrictions and the my name is classical cover to working at NTT communications company in Japan and I'm, a technical manager hi.

B

Guys, my name is Jim Akashi senior architect for communications.

A

Services mega processing using WebRTC, so that today we will talk about our experience and the package. We talk about a service real time, media processing platform for RTC interfaces, and this platform is working on an top of geneticist GK we are using and so that our way we talk about showed this, but one today and today's topic, we will cover the kinds of frameworks and the before diving into the technical architecture. Topic in the detail. I will talk about there.

A

Some kind of business aspect of our project, I will have a quick talk about a topic.

A

Other said in the titles right, we are not providing viable to supportive services that enables.

A

Teton in a peer-to-peer video communication between client application, client and the we are providing as be careful, the grant application for the JavaScript and, worse and and white, and we are also providing some current functionality to make a p2p communication between the true diversity result.

A

Our customized providing some business application for the sonnets and the typical use case is our education and shown in this right. Sko Peter comparison, consumer use cases. Our customer is now providing on top of our platform using our community. So.

A

Currently, our customers have some advice for the wishlist other services. That means that our custom is not such certain way. To always the video communication. Our customer is now having a wish to make a salad, video or audio media processing process chain. On top of a platform, for example, a recording is an alkine cases and the voice, recognition and objective shown rough severe under so long. A la cocina is now happy wish to our rewrites such a media processing for the statuses and to make it realized set your features.

A

Other said we are now really focused on the quests I. Don't worry now bad using violence. The computing power is not enough to let it be all right side. She features, I explained it before and so that we make use of the cloud computing power now for the web. Rtc.

A

That current home services does not provide the safety features because a viewer- you already know that currently interfaces for they're very dedicated for such as RPC, and that means HTTP over ass and some in some cases are WebSocket.

A

Reason have provided that the web RTC is no so that second challenge is that meta stream processing requires a long time system, and that means I mean eight five over five minutes over certain units or so, and it is quite a different from the current model, because thicker RPC required up to thirty seconds or so, and so that currently service architecture Sanford warm that not a bright to our demands so that we decided to build our own platforms services and we get straighted proton services as a trial from this January called media pipeline factory, and this is a bubble diagram of our architecture and details about this.

A

Jim will talk about it later, but our system is now working on top of the kubernetes and kubernetes, and there are some component or working Cerberus architecture model and the nation that we deploying our way back to Gateway as a continent on top of analysis. So that, and we are not providing such a service, we are 10 media processing platform using our whereabouts gateway.

A

And before getting into the technical.

A

This is the boat service and, with this accessible developer can easily make their own pipeline scenario. And how do you see that a.

A

Client application will be sent the weather at each traffic, that is Mary our video audio stream I our Weber to get way running on top of the kubernetes and, if developer I want to add salmon features high partial value on top of it, pretty quick, for example, if the developer as the coding feature choose a file right, icons.

A

In this way, a recording studio where everything and in the same time.

A

And same time, by using the branch not.

A

And we can add some voice recognition feature and that is using Google's speech API and after that we can add some translational feature to factions.

A

In this way, developer can compose some scenario for the media processing and we can show you the some demonstration applications.

A

A

And this web application is Alfred voice, recognition and the automatic transmission scenario and I think I. You may she said some time delay is happening. What this to the state and the other state our platform is so that if the certification required the scenario and a platform simply spawns continent at which nigut and so that this delay time spanning time of a platform- and please say konnichiwa- was something.

A

Both of the Commission desert here and the translation research here and the in this way our developer can easily the media program application with a person.

A

Okay and the in the demonstration we couldn't define the continent such as fire right boost the condition of component functions with a translation function, but not only the predefined functions developer can deploy the castle functions with average DK and with such a in person under outer pips tree functionality is provided so that with a developer, can easily develop their own custom functions and even tip white. On top of our platform,.

A

And from from this right Jun, we talked about a detailed technical architecture.

B

B

So again, my name is Jim McKee. She an architect of this new service platform and as Ken showed. This platform is to help users focus on their media processing logic, just by simply composing their own pipeline, like he showed in the UI and not only focus on the media processing logic. We also want to freedom from being worried about infrastructure and operation.

B

So for the rest of all time, I will describe the solutions and internals we built here, hoping that my I talk over the tips and thoughts to those who want to build a platform like this.

B

So this slide shows overview of our platform. Mediastream is a data which latency, jitter and other matters and in big data technology context. There is some good solution to take care unbounded data in real time, but we couldn't find good of the self solution to take care of media streaming like this and particularly non for WebRTC.

B

So we decided to build a new platform on top of kubernetes and creating media processing pipeline with containers in this morning. In the keynote Bryant said that, but kubernetes is a platform to create a platform, and we did exactly the same thing.

B

So serverless I must first talk about why we define our platform as a service when we talk about service. Typically we're talking about function of the service with functional function, as a service user only needs to care about function, which is a business logic and even when to fire. The function and most important user only needs to pay for the time the function consume. The CPU resource we want to provide us exactly same developer.

B

Experience with our platform event is typically short-lived, for example, in AWS it could be HTTP requests, s3 upload as an s topic notification to invoke a lambda, but in contrast, in our case we define media streaming as an event and real-time media processing as a function. So every single media streaming is a single event for our platform, and this brought some challenges.

B

So first I will look into the functions in Ken's demo. He showed two functions speech to take his transcript and translation function, and we cascaded like that. We believe that a media processing pipeline could consist of multiple functions and we want to give a good program ability here. So we decided to create container per function, which is exactly a Microsoft's architecture and we Cascade gateway to those function.

B

Containers we take care of those container management, so this abstraction gives a flexibility to user, that user can bring their function and just swap the function container to enable it also, we don't share containers across multiple streaming session, which means if we have a fader on a one session, it wouldn't affect the other sessions.

B

Also state wouldn't be accumulated beyond a session, and this brought a big stability to the platform and one more thing about the scale, because we can just scale I mean we can just scale this platform by simply creating the containers. So it's horizontal scalable and next one is event.

B

Our event is media stream, which, when we're even starts when media storage stream data flows in and event ends when that media streaming stops, which means event is long-lived in our case and so do the functions. So we need to take care of this lifetime of long-lived function.

B

We created our own controller to spawn containers when we receive a new session request and we terminate the containers when we detected session close event to keep a simple. We interact only the communities API to take care, the lifecycle just spoiling the containers and terminating the containers.

B

With this lifecycle management users, application walkthrough could be described like this I used service working group specification to describe this example of audio recording case, which is another if scenario from cancelled.

B

It is pretty simple that, on receiving session open event, function will do their initialization and on receiving a streaming data, they will start processing their logic. In this case, one function would do the audio decoding and another function would save the stream to an mp3 file and on receiving session close the event. We simply interrupt long-running function and terminate the containers.

B

We have a contract for contain functions that we always send signal signal to functions on receiving a closed event, so that function can gracefully finish their processing logic. With this approach, any other scenario could result in this kind of diagram, and this simplicity brought help us build a reliable system to mention about scene, CF, service, working groups, specification I, couldn't find a way to define, describe the relationship between functions and now also I couldn't find a way to define the event to trigger interruption to the existing function.

B

Maybe this is a new challenges here, but I'm, not family of the working group. So I will appreciate, if you can correct me, if I'm missing something and by the way, I love to input. This use case to the working group, if it's interesting to them.

B

Last topic for the server this is K native somebody thought that why not K native? Well simply it was not there when we started the project, but as far as I know, kinetic now supports ERP system. So maybe it's time to look into this, but I still see some challenges here we could create our custom event using kinetic eventing to take care. The web RTC session events and we could use tentative servings to scale functions, but I couldn't find the exact way to wire a function to another function while they are keep running.

B

So maybe this is a challenge that we should need to solve. If any of you are familiar with K native I'd love to discuss about it because I think a native is promising and worth merging.

B

Also I am happy to contribute to the community with our findings so.

B

Let me change the gear to talk more about kubernetes. Let me ask a question: how many people in this room run kubernetes in production?

B

Ok, pretty much, then how many of you create more than 100 pots a day. Oh nice I would like to discuss about your challenges, so we create more than that. We directly call pot API to create containers. We don't use replication controller, because it's not that much.

B

It doesn't last that much I mean one session, for example, for the video conference could last 30 minutes or so so what it doesn't make sense to use a replication controller for our case and we create containers procession, which means, if you have a mini session request, we need to create many contact containers pots and which brought a new challenge on the performance.

B

I forgot to put y-scale to this graph, but our path home is still in trial phase and with the limited user, but even with that, we had experience to create more than 100 pots in few minutes. So it was a pretty big challenge and we need to deal with the spike and also we need to spunk on pot and serve it in the fi fo manner, first enforcer manner.

B

Another challenge here is to think multiple containers for a same session. If multiple containers get ready state it very different timing, if there's a big gap, then we could lose the the streaming data on this. We don't buffer. Of course, another challenge which comes from creating many pots is observe a bit of the observability. This is a mouthful distributed tracing creating a pot plays an important part of our workflow, and but our workflow also has other operations like DB, transaction and authentication as usual.

B

So we want to observe the whole end-to-end, workflow and drill down to see the operation, how long it took to create a pot or how long it took to make the pot to be ready state. It means that we need to correlate the route span to the inside of the function function inside the container.

B

We did this by propagating the span context to container environment variable and have container extract the span context from the environment and create a new span attached into the root span, and this is the the image of the our tracing data.

B

And merge tenancy is obviously another challenge. I don't have much time to describe everything here. One thing to mention here is we added a our original hydration logic in the application the year to check that which containers can be connected with each other? Besides that other equalities features, security features.

B

Okay, so let me then describe some of the integration we did, which could be interesting to you guys.

B

First, the RPC. We use the RPC service training and we define throat a buffer to propagate, not only the payload media pedal, but we also propagate media capability metadata. This was good to bring inter functional ability, but not only that it was good to seamlessly support the gstreamer library. This streamer is a most one of the most popular media processing regularly, and it requires a caps in the buffer. I mean the payload.

B

With this schema, we can just simply input the data to the gstreamer library, and you can do some media manipulation or any other stuffs using the library.

B

Well, er PC works well from our perspective and to add a comment waiting for a custom transport support to test some other of transport like a UDP I, don't know if it would make sense or not, though, and we use cloud events to control our workflow.

B

We use colobus to define our web RTC events and place event together in the middle dispatch, the events and so that our controllers would be loosely coupled to be honest, I don't see much benefit on using cloud events so far, because there's no less tools in the market now, but yeah I strongly expect to have some observability support in the cloud events like you been tracing, not only the specification but the implementations in the tools and, lastly, telemetry. It's not fancy stuff, but it's one of the most important integration.

B

As always, we have a service ID session ID and we want to group all of the logs metrics tracing with this ID. We did this by assigning that ID to current as metadata and use metadata agent to bind the ID on when sending the data to the backend.

B

Also, one more critical thing is to create a dashboard to make all of the telemetry immediately actionable.

B

Well, I can spend more 30 minutes more, have had this pain point in the solution, but I know I'm running out my time. So I will save this to the next chance. If you guys are interested in please reach out to me and I'd love to discuss about this.

B

Ok recap: we built we wanted to build a server-side, cloud-based, real-time media processing for WebRTC, and we built a service, real-time media processing platform using kubernetes. In other words, we completely sit on kubernetes shoulders. This platform is our for developer user development experience. We want user to focus on their media processing logic and we want to help turn turning their application to a real-time communication application easily.

B

Lastly, you can find the code used for the demo in the github links here and you can use those code and also the tricks. I showed to create a platform like this, or you can even sign up to our prior service to try out the new user experience on media processing. All of the code in the github is already pre-installed of platform, so you can immediately try it out before building your own okay.

B

That's all! Thank you so much for your attention and have a great day.