Cloud Native Computing Foundation ServiceMeshCon EU 2022, 19 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multi-region Service Mesh: How Koyeb Built and Operates One Using Kuma and Envoy - Yann Léger, Koyeb

Description

Multi-region Service Mesh: How Koyeb Built and Operates One Using Kuma and Envoy - Yann Léger, Koyeb

Service mesh is all over the place but how do you build and operate a service mesh across regions and at scale? This talk will cover how Koyeb, a serverless cloud provider, built a globally distributed mesh to provide easy-to-use inter-service connectivity across multiple cloud providers and continents. To achieve this, they built a custom stack with a multi-region service mesh using Kuma, which is an open-source control plane for service mesh orchestrating Envoy proxies. Yann will walk you through how they built distributed connectivity inside the platform, the key decisions they had to make, and what their architecture looks like. They now have a purpose-built stack based on Nomad, Firecracker, and Kuma.

A

So yeah, just to give you some background, um I'm the co-founder and ceo of kojeb. I basically spent the last decade building cloud service providers uh in europe. Mostly so I started uh with some engineering at a cloud service provider called outscale in 2017.

A

We were kind of trying to bring the feature of aws user to the french and european markets. um We I spent six years building, another cloud service provider called scaleway and in 2020 um I started uh koeb with my two co-founders and basically I'm expecting to spend the upcoming decade building other cloud service providers, because I tend to apparently do that to maybe to to know a little bit more. Who is in the audience. I would like to ask you who has already set up a service machine production.

A

Is that the case for everyone or okay? So we have a few people, but not everybody.

A

I I saw a dozen of people for the people who are following form remotely.

A

And who has a network engineering background, maybe.

A

Not more actually even lesser five people, because so from my perspective, like uh service measure, are kind of uh the modern network engineering I would say, and we'll go into a little bit of how we implemented it at kohib and what it encompasses uh and what it also brings to uh to the user.

A

To uh give you some context on what we are doing so at koreab. We are basically our uh our cloud service provider and we are providing a completely managed platform and we basically allow developers to deploy full stack application in minutes, um and we do that in a way where we basically don't want you to learn anything specific to the cloud provider. So we don't want you to learn anything specific to our platform and we take care of both the we take care of basically everything infrastructure.

A

So you can deploy both container and native code, in which case we will case. We will build the application for you and deploy it in production and uh coming back to our service max subjects. We also completely abstract the networking part and the service mesh.

A

So from a white perspective, that's what it looks like. So we have this koip serverless platform, which abstracts several different functions. We abstract both cloud providers, orchestration, continuous integration, continuous delivery, networking um with both a global components and a service mesh, and also some monitoring and storage. And today, obviously, we will focus on on two parts. So one is the service mesh and the other one is the global part and how we basically deploy globally.

A

The key question we will try to answer is how are requests processed when you deploy an application on on on coib, as mentioned, we have two different components: we have our global components because we are providing a platform where you have 250 edge, location built in and 25 car locations.

A

So that's the global parts and the second part is the service mesh part where which deals with the load balancing parts with our completely, which provides a completely encrypted network to the end user and also provide service discovery from so, as I mentioned from our perspective, as a goal was to provide a completely zero configuration deployment options of services, and we want that to be easily in multiple regions.

A

So we want developers to be able to deploy in two or twenty location without having to think about how to make them say yes, different microservices um communicate together. So uh the first goal is to have multi-region services um and to provide uh completely transparent inter and intra-region networking, and obviously this needs to be completely secured. So there is two components in that. So one is everything needs to be encrypted because we are also running our um across the internet um and we want to be also completely multi-tenant because we're a cloud service provider.

A

So we have multiple customers on the same machines which we'll see bring some challenges, because not all technologies are designed for this and with some scale in mind on the multi-tenancy side and the the last part we're bringing is the edge acceleration to make it more concrete. I will just give you a short demo, which is a pre-recorded demo on how we deploy our next year's service um in paris on koeb uh and what it looks like and it works. uh So here we're basically deploying an application which is called hello.

A

I think it's going to be deployed from github, so uh it's a demo application. We provide it's going to build automatically. We can select how many machines are running which size of machines we are using in which location it's going to run and that's where the magic happens, uh because we are taking care of the build process and we automatically provision uh are completely ready to use service mesh. So you have you have at the top a public url which is tls encrypted by by default. You can add your own custom domains later.

A

If you want- and you also have a private domain which is ready to use so if you deploy multiple services, they will be able to communicate together as we'll see in a minute.

A

So the build succeeded, which is a good news, because it's a pre-recorded demo uh and uh it's going basically so and that's the demo you'll see so you see uh uh through which edge location uh it's going and on which car location it's ending uh and it's paris in paris because it was recorded in paris yesterday.

A

So you can choose a test. So if you go to demo.com.coe.app.net.com.

A

uh You'll land on on this demo application and that's what I what I got yesterday when I landed um so actually uh and that's where the internet magic happens and the bgp magic happens. You might not go through the same exact age location. So yesterday night I was going through madrid two days this morning and the cab I was going through marseille.

A

So that's another story. I will not focus deeply on how bgp works, but it's mostly business negotiation which don't provide always the best latency.

A

And the question we will try to answer is how we go through to this app and how we will reach it. So the first step um is basically uh the edge. So we, basically when you type the url, you have the dns resolution, which is going to return. You three different unicasted ips.

A

Your brother is going to use the first one and wherever you are in the world, it's going to be always the same three ips, so that's bgp and bdp anycasted is doing that and in this schema you have like a user in valencia and a user in new york city.

A

So you, the user in valencia, is going to go to the nearest education, which might be madrid or marseille depending on peering agreements, mainly, but it's supposed to be kind of the nearest geographically, hopefully, and the second user is going to go through to through new york city and what it brings is. Basically, the tls connection is going to be terminated at the edge. You will be able to provide http 3 for all people who have browsers who support it, and you will get also caching at the edge.

A

The second step um is, our edge network is always going to route you to the nearest core location where your service is actually running. So sorry, not where your service is actually running. So it's the nearest core location before routing you to the core location where your service is actually running. So remember. We deployed the service in paris.

A

So if you are in uh new york, you're going to go through the edge location in new york, then to through the core location in new york and then to paris, because that's we'll see how it works behind the scene. But that's the way we implemented the mesh.

A

And to go to this core location, where we have the complete service mesh running, I will just give you a few uh more details on the global context of the technological stack we are using so we're using uh for the orchestration side nomad. um So we are in our technology, who is basically uh communicating with all these components: um firecracker for the virtualization side and and to deploy micro vms.

A

It's running on top of viral servers and for the networking side we are using um kumar and voice who are dealing with the service mission and discovery.

A

So what's kuma and basically bring to us is that they provide completely fully automated service deployment. We get all the basics which you'd expect uh as a as an end user. So you get uh secure private connection. You don't need to think about encryption between your different services.

A

You get completely automated genius provisioning for your internal services, uh layer, 4, layer, 7, load, balancing and it's really completely transparent as a as an end user.

A

So we have this on each location. We have the stack running so it's it looks like this. You have like uh bermuda service, the hypervisors uh with a chorus and micro vms on top of the hypervisors, and the container you deployed are running inside of the micro vms.

A

So pretty that's pretty standard uh in terms of service, mesh deployments and kuma is basically taking care of all the service mesh control plane. So what it does it's going to broadcast the configuration to all instances of um of kumar which are running which are embedding envoy technically and it's broadcasting the configuration? It's not broadcasting it, but it's distributing the configuration where it needs it to be so on on the control plane side you have kuma who is dealing with all this configuration and on the data plane side.

A

You have the envoy which are embedded into managed by kuma and that's the key, that's the key thing, and so going back to our service, which is located in paris.

A

uh The question is: where does it go through, so we uh stopped uh at the edge earlier and we said like it was going to the nearest core location. Then we have two components in the. uh What we call the data pass, the first one as um and it's both of these components are not standard actually to uh um to kuma.

A

uh So it's uh things we had to add up um the first one is what we call the glb, so it's a global load balancer, which is going to identify where the cell and which call location the service, is running.

A

So, if you landed in in new york, this is the component which is going to tell you uh human services in paris, and it's going to root this traffic to paris, and then we have a second component, which is the ingress gateway and it's going to identify and which exact machine and the core location the service is running, um and actually I I I was told that uh ingress gateway component is something which came up pretty recently in kumar, so we are not using it because we uh we had to um we needed it earlier and it was not yet released.

A

So we have our own implementation, which is dealing with our multiple customers and multiple meshes. So that's um an overview of the world database. So on the left, you have the edge servers which are connecting with mtls to the glb, which is located in the core location. So that's a scenario where you learn directly on the right call location.

A

The gab is going to send it to the ingress gateway in the same location and the ingress gateway is going to root the traffic to the right sidecar, where actually your micro vm is running aside.

A

If we go to scenario where you have two services in the same location, so uh in paris, uh in this case we will we the use cases, you have our web application and you have an authentication service. So there are both containers. The authentication service could be a rest service, for instance, and what it brings is basically it's going to provision completely automated dns provisioning, and it's also going to bring you native layer forward, branching and as an end user.

A

As a developer, you, you can do a curl on oath, that's 3000, and it's going to be completely transparent for you, so your web service is going to be able to reach the old micro services without having to to deal with any of the complexity.

A

So in this case it's it's pretty simple, like the telepaths from communication inside of our region is going to be direct. So it's not going to go through another component. It's going to go from the from each service car to the second one um and if you're, basically in a scenario where you're deploying across multiple regions, um there is a two over there, I'm not sure what it means. I have two minutes left, maybe so.

A

What I was going to say like if, if we are deploying these two services in different regions, it's going to rely on another component, which is called the zone ingress, which is uh actually pretty similar to uh the ingest gateway we had earlier, but it's for inter-region.

A

So it cannot go directly to because it cannot go directly to the sidecar of the uh located in new york. It's going to go through the zoning rest, which is located in new york before reaching the right sidecar in new york.

A

When we built this platform, we had to choose the right technologies and we had two key requirements uh which were mostly filled by uh kuma at the time.

A

The first one is multi-tenancy and we rely on open source technology, so the only one which has a which was able to play this was kuma and the multi-zone factor.

A

So that's why we ended up with uh with basically kumar. We still have several challenges and limitation with um with this technology, so one is multi-tenancy scaling. We have like about uh 3000 different service mesh already and um because each customer has a his own service mesh.

A

That's one key point of our implementation and basically, when you add up latency in the current implementation, it's going to when you add up meshes sorry, it's going to add up latency, so we ended up having like sidecars who were taking five minutes to to boot. So it's that's part of the challenge we are trying to tackle and solve.

A

um We have several performance issues here and we also have the challenge of uh memory of override from sidecars, which is a huge concern for when you deploy tiny micro services like functions or services, with 120 megabytes of ram um and the two things we are looking for, which are uh part of the technology already um outbound, ips and tcp udp support which we need to implement.

A

So we are basically looking to expose way more features than we have now uh in the in the future. We already have built-in observability um and the platform uh which is going to be released this week actually, um and we are basically expanding right now to 25 core locations um and that's going to be it for now. I I've skipped really quickly on the next on on the last uh slides. uh Thank you very much.

A

So we're going to basically announce a pubic preview today of the platform uh it's already available, you can sign up and that's uh that's all, and if there are questions, I'm happy to take them. If I have some time.