Cloud Native Computing Foundation EnvoyCon 2019 (San Diego), 27 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Envoy Namespaces - Operating an Envoy-based Service Mesh at a Fraction of the Cost - Thomas Graf

Description

Join us for Kubernetes Forums Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Envoy Namespaces - Operating an Envoy-based Service Mesh at a Fraction of the Cost - Thomas Graf, Cilium / Isovalent

The most common architecture currently includes running Envoy as a sidecar proxy inside of application pods. This provides an excellent resource and security isolation but comes at a steep resource consumption cost. Each individual sidecar proxy is running as a separate process and is duplicating all required resources.

This session will introduce the concept of namespaces to Envoy. Similar to namespacing in the Linux kernel which serves as the foundation for containerization, namespaces for Envoy allow to isolate resources and thus share an Envoy instance among multiple application pods running on a single node without losing any of the isolation properties. We’ll look at how a service mesh can be operated at a fraction of the required resources while still providing virtualized logical Envoy instances which present themselves to Envoy control planes as if they were running as a sidecar.

A

Hello good morning, my name is Thomas Groff I'm, one of the creators of psyllium and co-founder and CTO of Sol and the company behind psyllium. Today today, I'm talking about envoy namespaces, which is probably a new term, because I I, basically just literally made it up, usually I talk about kernel-level vpf stuff. Today, my topic is more on the control plane, scalability good, so let's dive in I would like to set the context.

A

First of this talk so I'm talking about scalability, but it's more, the scalability of the control plane managing envoy less about on Y itself. The reason why that's important to us is we are maintaining Sulaiman.

A

We are working with many some users, and many of them are running at really large scale, because that's a specialty of solemn silom provides advanced networking and security for cloud native workloads, which means that, as these users also have intent and desire to either run or plan, to run a service match based and on what we typically see these users struggle if some of the control plane aspects.

A

So what I'm talking about today is how we are helping these users, our scale service matches or annoy based matches, doesn't have to be a service too much in particular. So this is how you typically deploy envoy in combination with solium in a mesh configuration service to service, so not no edge.

A

No North vows basically poured upon service to service, and so on, like envoy, is running as a sidecar container inside the application pod as a separate container, and there is an IP tables rule which will redirect all traffic into envoy solemn is outside running BPF code to do all of the network, all of the policy using the PFA, then, on top of that, you run a control plane right. This could be easy or it could be something else and all of your side.

A

Cars will connect up to that control, plane to basically receive instructions on how to do routing how to actually, for example, send metrics and so on. If you look at this picture, it's pretty similar to this picture.

A

Can you see the the kind of the how this looks similar like it's basically set up for a perfect data scenario, where you have many many many many endpoints hitting a centralized resource and that's what we're typically seeing so I'm gonna walk through some examples that we have I've seen here. So let's look at the like kind of a moderate scale example both for a low turn and a high drone example.

A

The low drone example would be when, when workloads are not actually not changing much for that scenario, we're assuming basically one part event per minute per node, which means that one part per minute is in being pitted being added, were being removed from the node on each node and then the hi churn one that will be the one when you're actually scaling up and down or you're, adding notes or you're actually running batch jobs in that environment. You have 20 parts event per minute per node, still not a massive amount for this module scale.

A

We're kind of looking at 500 notes with 30 parts per note, so nothing massive like moderate scale. The two resources that we've been seeing that have caused many issues is network traffic and CPU usage of the control plane, not on. Why we're talking about the control plane of this from a CPU perspective, it's 90 percent, if just it's just plotted off encoding, just encoding all of the communication that needs to go to all of the Envoy instances and all of that control.

A

Plane traffic needs to be on the network, so it will cause network traffic so at low turn, we see relatively relatively moderate modern network traffic and CPU like 11 megabytes per second is already it's not nothing, but something that your network can handle and you need about 12v CPU. So you already need a pretty sizable VM, but you can still can still be a single VM if you want to for hydro we're already going into several hundred megabytes per second- and this is this- is still the moderate scale Kloster.

A

This is nothing hyper scale yet, and you need several hundred V CPUs already. We can go further and actually go into kind of the cluster size that some of our users have multiple thousand nodes. For example, five clusters 1,000 nodes, each 30 pots, preneur, look at the numbers like it just simply does not scale up. This is not envoy. This is the control plane that can simply not communicate with all envoy instances in an efficient matter, you're running into thousands of V CPUs that you need at peak. So what we typically see.

A

What often see is that when you scale up and add a lot of notes, let's say you actually add a couple of hundred notes to your cluster. Your control, plane, CPU, will spike and is to control. Plane becomes unresponsive for for several seconds or even minutes, so we've been looking at how we could potentially resolve this, and what we have been thinking about is and been working on is a shared envoy model.

A

This means what if we could actually have instead of one envoy per pod, have one envoy per note and share that envoy with all the parts all the application parts very similar to how applications already share the same Linux kernel. On that same note, right, we don't have a specific network stack or a specific storage storage sack for each part, the kernel is there for all the applications and the kernel has a concept of namespaces, which means the kernel can do resource segmentation and so on, based on these namespaces.

A

So bringing that same contact concept into envoy allows to scale up work very important, I'm, not proposing or we're not proposing to replace this model or replace the cycle model. If this disk can work out very, very well at really large scale, but it also has disadvantages. It means that envoy needs to be aware of, of having multiple serving multiple tenants. Envoy now needs to do resource management right, because it's serving multiple tenants, pod CPU limits will no longer apply to your own voice, ID card and so on.

A

So there's other issues that have to be solved. At the same time, we see this as this can be a solution for your higher scale needs. So, let's look at that so on the right are now basically the numbers where we have still 15,000 total end points, but because we only have 500 notes, we have 500 on voice.

A

Obviously this does not impose any problems anymore for the two metrics that we have been tracking network traffic and CPU even more interesting one is to high scale one which, even in a high churn environments, it basically goes down to a couple of dozen V CPUs or not thousands of them. That's why we have been successfully successful scaling these users using this model. So that's the background. Let's look quickly into some of the details. How this actually looks like I won't have time to actually go over everything.

A

If you want to learn more about this feel free to talk to me after the session so injecting a shared envoy. If we don't have an envoy in the south as a sidecar in the part, how do we actually inject envoy if you use solium, we can inject envoy automatically on demand outside of the pot, which means any traffic entering or leaving a part can be redirected through envoy using BPF.

A

We can do this in two ways. We can do this on a more traditional network level, so similar to an IP tables redirect rule like for each Network packet, but even more interestingly, we can do this at connect system call level, which means that adds your application issues. A connect system call to open a TCP connection. We will translate the IP address in there automatically to go to the sidecar proxy running or that node, which means there is no network Dina. There's no network mangling at all.

A

We'll have you basically do the redirection when the connection is set up once for the connection and if you're doing a persistent HTTP connection between your app and the Envoy sidecar, you pay this cost once and then it's been, you have no cost beyond that. It also means that there is no requirements to modify any aspect of the network stack in the part, so you don't need any any sort of privileges, noting that the application part remains entirely unmodified. The application part can even be privileged itself.

A

It cannot remove the redirection to the side core and so on. So it has a couple of additional benefits. So how does envoy fit into this? Actually already pretty good at managing multiple tenants right? We have. We can have multiple listener configurations, which means we can. We can actually run one listener per pod per application. Running on that note to, for example, have different routing configuration, different filters and so on what we are currently adding at what we.

A

What we need to complete is the resource management side, meaning that, if you have a shared Envoy instance, one part cannot consume all of the available resources for the SharePoint one, and the second aspect that we're working on is to actually make the control plane, for example, sto aware of Asia envoy model right now with the concept we have right now.

A

What we're using right now is concept of virtual envoys, where even though you're running a single envoy per note, that single envoy instance will register itself multiple times as a virtual envoys to is do so from an SEO perspective. It's still 15 or 20 thousand envoys, but you're actually only running a couple of hundreds, one per note longer term. We would actually like to make it you're aware of like the shared envoy model as well. So we can, we can read, we can reduce the communication, that's required from control pay to unwind.

A

With that all of this development is currently happening or in the SiC envoy on the cilium slack is if, if you are interested in that feel free to join our Salim slack ping us there, there are multiple users there kind of working on setting this up.

A

If you want to learn more about the cilium project itself, it's open source, so you can go to the get up page and learn more about psyllium, and obviously we also have a Twitter handle I think we have a couple of seconds left for questions and if that is not enough, feel free to catch me outside after after the session. Yes,.

A

Yeah, so the question was gonna whether I can talk a bit about the resource isolation. The first objective there is that we simply make sure that they can set boundaries, for example like each each workload can only use, let's say 10% of the actual total resources allocated to envoy and then the second. The second aspect we were working on is that you can limit the number of requests, but the furnace aspect is the most is to list emits the most important one, but that's that's literally like what we're working on right.

A

Now, we don't have like that. We, this is not fully done yet yeah. Yes, we have a design document like join the join this, the second, when we can have pointers. Alright I think we're running out of time. Thanks a lot.