Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2020 - Virtual, 4 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Service Mesh and Serverless Chatbots with Linkerd, K8s and OpenFaas - Sergio Méndez

Description

Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Service Mesh and Serverless Chatbots with Linkerd, K8s and OpenFaas - Sergio Méndez, Universidad San Carlos de Guatemala

This session will present the different challenges for telco companies when they deploy support chatbots for clients, this is based on a real experience of working with chatbots in a telco company Telefónica based on Guatemala and some countries in Central America. The session also present a simple architecture, the workflow and the way to implement service mesh with Linkerd, kubernetes and Openfaas, and how they interact together to implement canary deployments to create a simple and fast strategy to manage chatbots as a serverless functions. At the end of the session the speaker present a short demo on how can implement a canary, blue/green deployments for serverless chatbots with some support intelligence. This session is the different that a previous one that i talked at Oscon 2019, there i talked about architectures here is focused on Service Mesh and Serverless implementation.

https://sched.co/Zemu

A

Hi welcome to the cubic coin: u and cloud native com 2020. So let's get start so the title of this session is service, mesh and serverless chatbot with link rdks and openfast. I am sergio mendez about me. I'm an operating system, professor and entrepreneur. I have a brand called cloud society, so it's about joining open source uh communities and that kind of things. So I am a clown native enthusiast. I am working as a sre at weiss line.

A

I am a meet up, organizer of a cloud90 plus gt and it's really awesome to share cloud native technologies with the people. So let's get start the it infrastructure and evolution. So there are several steps in the evolution of the technologies or maybe the different steps that you are using to deploy your application.

A

So, for example, maybe you are deploying an application in a monolithic architecture, so you suppose that you are using a house, so you are close to the bare metal and you are installing, like a kind of hypervisor and you are deploying your application in a virtual machine. So maybe you have.

A

Maybe you have some control in or talking about of the operating system. So maybe you are using containers so that containers doesn't use a hypervisor that is associated with that kind of virtualization in developmental. So maybe you are packaging or the logic of your application inside a container. So you don't. You have less control in difference with the monolithic architecture, because you are maybe to playing there your application inside the container. Are you only have that control in the container? So in the serverless you don't have control about the hosts.

A

You only deploying a small part of code and that's it so there's no server there. So I mean the basic concepts.

A

First, the cloud native, so cloud native, is taking approach to building applications using that cloud computing model, so you are paying for that services in the cloud. So in a second concept about cloud native, so in that cloud computing model you are using containers, maybe micro services or service meshes and taking advantage of that cloud computing model.

A

So that's essentially the cloud native thing and I can mention the cloud native apps that uses the 12 factor to the to build their application so uh serverless. Well, you are taking approach of software solutions that you are deploying only the small part of the function that you are running in some transient servers. So you are paying what you use in that moment.

A

So um in that environment could be very elastic, so you don't have to operate or to pay attention to the servers, but because of the concept that don't have servers so serverless has a high level abstraction of the service of generally deployed that small part of of the code. So you don't have to worry about the servers anymore. So that's the really cool thing about serverless, so serverless provides you some features, for example, the event driven thing so the serverless and start when you are receiving some requests in the platform.

A

So when you are receiving that request, the the service turned off and after a few minutes um that service dies so only lips during the time of the request. So it provides you some kind of streaming data on an out scaling, kind of service and fault tolerance.

A

So the good things and the bad things are, the good things are cheaper. It's cheaper than traditional cloud services. It's scalable during those reduce the up skulls. You are focusing more than they just experience depth. So in the bots there is a bender locking thing because you have to adapt, adapt your logic or your software to them. Serverless service that your cloud provider gives you.

A

So there is a learning curve to understand how your cloud provides their implement serverless, there's not suitable for long-term tasks, because that serverless functions only leaves for a short time so, um depending on your provider, maybe don't support the language that you want to create your server expansion so about controlling the system and you don't have more control or well. You have less control on that on that infrastructure, so you don't have access to the server. So that kind of thing.

A

So you are only deploying your code in that platform, so you can use serverless in the in internet of things so to create chatbots assistants or for api. Some kind of streaming so implement service for image, manipulation or even processing or multi-language applications, or to implement some kind of ci cd, depending on the case.

A

So what is service mesh, so service mesh essentially is like the kind of service that lives in the service area, so lips like a side container of a side card together with your service, so this side card, which you're looking here this your primal application and the side cards of the cycle receives all the traffic as a proxy. So that's like side cars performance as a as a proxy so decide to reside all the traffic can decide what to do with that traffic to forward to the original application or drop the traffic.

A

So in this graph you can see, for example, in the kubernetes context, a bot could have more than one container, so one container is the service mesh that resides the traffic and forward to to um the original microservice. So the service meshes has two components: the data plane and the control plane.

A

So it's really awesome so in the data plane is leaving your application, so the green square could represent your original container on the and the blue squared could represent the sidecar or the proxy element of the sidecar that receives the traffic so when they receive the traffic forward or ask to the control plane what to do with that traffic or generates metrics and apply some policy and show that information in the ui. So linker d is an ultralight service mesh.

A

So it's really really easy to use to implement in kubernetes, so gives you that kind of observability or a loyalty or implement diet security between your different micro services without code change. So this is really awesome.

A

So has the same architecture, though, when we are talking to the basic concept of the implementation of service mesh, so has some control plane and data planes so has different components, for example, implement a tab, identity, or maybe some validators just an api. So you can access that api with the cli.

A

Has some web panel to show the graphics, or maybe it's recording some metrics information showing the graphs using and in the data plane? Is your original application with that the linker, the proxy or the side card that receives the traffic and that throws to the control plane to decide what to do with that traffic?

A

So what linker they do so linker team gives you that observability arrivability and security thing so with the observability gives you that power to generate gold and metrics like success rates, latency throughput.

A

I showed you in the in the panel that service dependencies or topologies to implement that kind of uh timeouts or secret breaking when you are traffic splitting that kind of things to implement security and secure the communication between your containers using tls and that kind of thing.

A

So why should I care about um who can use service meshes could be used by c sras and architects um because they implement observability and some security primitives, because they are very critical to cloud native applications or architectures, and the funny part is that you don't have to modify your application to get that benefit. So that's the really cool thing about linkerd, so openfast is an open source. Serverless platform can about open funds. Is that gives you that platform to implement service service mesh in uh to create that event driven function?

A

So so these functions are created as some microservices packages with docker and deploying to kubernetes, but there are different versions on how you can deploy open fast. For example, there is a version that you don't use, kubernetes only container d and there's another version that you can deploy open files using aws fire gate and open fast, provide you that auto scaling and metrics for this. Your serverless function without limits of execution like lambdas on that kind of thing.

A

So the features uh are the openfast provides you that that easy ui portal uh to package the future functions to provide you portability to move your functions, uh provide you a client to manage that functions and some jump. Don't confu for configurate some options in your functions and provide you that out scaling from zero to like lambda or it depends how you configure your function.

A

So there is a reference architecture about open, fast, so open files internally use a lot of so we're there, for example, nuts, to create that some cues permit you to recollect some metrics um to showing through graphing and different components. So it's really awesome.

A

So in a serverless real use case scenario yeah, I can mention that in my past job, in a token company, we were creating um um chatbot tools, so this tool was the mission of this tool was to create custom, chat, bots and they are executing using slack telegram, whatsapp or facebook.

A

So, right now, I am trying to remain solve observability on soft canary deployments to that chat, but so, and this telco company and their requirements of their application is to prevent the brand unlocking. So they agreed to use open source technologies um to implement some fast integration about these technologies. uh The short time of development time to implement devops and ci cd uh to spend less money and smart weight could be the performance, the resilient system and observability. So this software, like link, rd and open fast, provide this thing.

A

So essentially, the solution has open fast, that serverless open source platform link rd to as a service mesh to implement observability and the canary or blue-green deployments and kubernetes to orchestrate the different containers created by openfast and scale and provide that that scaling thing in the system so about this is like a common kind of execution diagram now how the chatbot works. So I can mention the artificial intelligence theory about agents.

A

So when we are talking about agents, uh the agent uh interacts with an environment so has some perceptions of the environment, capturing by that sensors and this agent decide what to do with that perceptions to action to the environment. So.

A

It does that actions with the factors, so the things that provides that perception in the parliament are are the incident messengers, all the chats all the texts and that kind of things that receive the agent in the form as a container they're playing kubernetes, um deploying using open files and linker did recollect some metrics and do that traffic splitting for the application.

A

So it's really awesome in that way so to be in context uh canary on bluegrass deployment. So the blue, green employment. You are deploying basically two versions at the same time, so you are splitting the traffic between uh version a and the position b and the canary deployment you are gradually shifting a traffic between a version, a and b for some service, so in the architecture of the demo, so I'm gonna implement so um blue blue green deployment of canon deployment with 50 and 50 of the traffic for each service.

A

So to implement this, you have to use a dummy function called root. So I in this demo I call it chatbot root and the chatbot ring and the chatbot blue, but it's the same logic.

A

The benefits of this solution is that this zero vendor locking there is a fast integration with technologies. You are using technology that gives you that ability to provide that cicd platform and you are spending less money in a smart way and creating an auto scaling application using open source. So you you don't spec, some vendor locking, is very silent and you implement observability without modified your application, so the demo shows uh claudia a slack chatbot that is leaving my slack meetup.

A

So I'm gonna show you the templates of open, fast, great functions, uh some integration, some functionality of lingerie, some canary deployments of blue green deployments, uh to show you how link rd works and the observability thing of linker d. So here is a repository that I gonna deploy this demo. So if you want to check it, so it's really awesome.

A

So, let's start with the demo. So in this part I can show you the link rd the link at the dashboard. I suppose that I have configured the cube ctl to connect to the cluster. So now here is the link. Rd dashboard is really really really cool. So in this namespace of kubernetes, the openfast event is publish all my functions, so you can expect to find the chatbot green, like I said, the blue, green and the chat bot route. That is the dummy service.

A

um This blue and and green service called the micro service every time when they wrote when I write and cubic on you in the chatbot so called the microservices, and I am doing a traffic splitting between these services. So let's write in the chatbot so, for example cubicon: u um this cubicon? U again, this is cubic.

A

U or maybe hi. So maybe you are seeing like there is passing the traffic like more or less 50 and 50. This is the blue chat, bot and the grid chat. But then this is the idea of the container that is executing this logic. So there is alternating the request between 50 and 50. So let's generate more traffic that calls my microservice service.

A

So, let's reload this part of the panel, so you can, I can show you the dependency of of the different service, so chatbot green calls microservice every time that I write and cubic on you and the blue does the same. So I can show you the traffic split for this thing. So in real time I want I'm gonna, try to show you if they change, for example, high or hello cubicon. U, there is changing in that time.

A

So it's really awesome. It's changing so hi yeah. So it's changing in real time. So let me show you another thing. For example, this blue is calling microservice, so the blue calls microservice. Let me try so cubicle? U so cubicon? U again.

A

So maybe cubic u yeah! This right now is changing. So he become. U.

A

So let me again cue account you.

A

So, in the same time, well delays a little bit to show well to show the change in the graphics. So here you can see all the requests in real, almost in real time or in real time.

A

How is deploying the different things? So I want to show you the graphana, the graphana dashboard. So here is the graphana dashboard and shows you some information about the different microservices and the information that is recollecting.

A

So it's pretty pretty awesome. um So here is the open, fast portal. So in this portal well talking about open fast, I can see my different serverless functions and the url to access that function. For example, here's an example or function that returns the information of the host, and it's really really nice.

A

They are counting how many times you are requesting this this function and you can deploy new function functions here. You can. You can also delete the function and that kind of things um I want to show you more or less the the code, the code of well. Let me access to this server here and to show you more or less the structure of this function. So here's the demo, um let me check this.

A

A

In this directory, I can show you the different parts of this, for example, in the stack um there is the gateway that we are accessing. This is this.

A

This portal is the gateway, so the images that we are using um they match that we are publishing in in the docker hub. So let's see that different templates, so, for example, well here is the python3 flask and template so essentially gives you a docker docker file that you can modify a template um and and in the index function with the example of how to execute that, like so essentially container that is executed like a serverless function.

A

So it's really awesome and to finalize here's a um a repository, it's not the final repository final repositories, the url that I'm showing in the presentation. So here the openfast has some cli. For example, here you download the template and you are creating the project to create your serverless functions.

A

In this part, you are deploying your serverless function with the fast cli, so here's the chatbot, green, blue and root here are here. You can see that we are injecting the sidecar to the original function. So essentially you are getting the jump off of that deployment that you are adding some code to inject another container as a sidecar.

A

So essentially the service is already deployed and inserted with a side card inside so to implement the traffic. Splitting is very easy like this, for example, this file, you can see that they match this service and- and you can describe in how many traffic goes through every service, for example, 50 and 50., and that's the way that is very easy to implement.

A

um They open source, serverless functions and some observability you see on linker d and open fast. So that's it. That's the demo so just to continue well, I have here I have some references that you can visit um on the repo, so you can follow me in the social networks with third erm gpl. You can find me link it in twitter and that kind of social networks and thanks. So I think that we passed to the questions and thank you very much thanks.