Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Proxyless Service Mesh with gRPC - Menghan Li, Google

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Proxyless Service Mesh with gRPC - Menghan Li, Google

gRPC is a popular choice for building microservices. A service mesh is a dedicated infrastructure layer for communications between microservices, with features such as service discovery, load balancing, application security and observability. A service mesh typically uses sidecar proxies alongside the service instances to provide these features. In this session, we will introduce the new exciting development of service mesh features in gRPC, and show how to build a proxyless gRPC service mesh without the complexity of managing sidecar proxies. This talk will cover the architecture followed by a demo on how this works in practice.

https://sched.co/ekI8

A

Hi this is mohan, I'm a grpc maintainer. In this talk I will talk about grpc and service mesh and the service mesh support within trpc. So, first of all what is grpc. I think many of you must have known this. So I'm not going to talk too much about that. In short, jrpc is an rpc framework that is good to use for building microservices, but in practice for microservices you will need more than just the rpc framework.

A

For example, before you can send rpcs, you will need to find the new services while sending rpcs you will care about load, balancing and security before service mesh support is added into grpc. Grpc provides limited support for those problems like that's when the answer is over. There are two very simple pick: first and wrong: ruling balances. Qs tls is also supported in jpg, but it's only configurable at startup time for observability there's no building, but there's open sensors plugin.

A

Of course, you can write your own plugins to get those advanced features that you will need, but writing and maintaining plug-ins can be a lot of work so because of those problems is a need to integrate the rpc with service mesh.

A

Let's take a look at how service mesh solves this problem, so service mesh is an extra infrastructure layer added to the deployment to control how different parts of the system interact with each other. There will be one service mesh control plan, that is, uh that is responsible for all the configurations.

A

The service mesh part of like the service mesh part deployed with selection or get the configuration from the control plan, and then it will decide where the traffic should go and also do all the other features like security and observability.

A

So in this more concrete example, this is how typically service mesh is deployed. The the application, the jrpc application, will be deployed with a cycle proxy. The cycle proxy gets the configuration from the control plan and there will be one connection between the drpc application and the proxy, so all the traffic will be sent to the property first, the proxy then, based on the configuration it got from the control plan, decide where the traffic should go.

A

One thing you can see in this model is that much of the sophistication within the drpc is not being used. For example, there's only only one connection between the application and the proxy, so the connection management feature within jrpg is not being used.

A

There are some other potential problems caused by the proxy. One of them is performance, so there will be one proxy on the client side and one on the server side. This can add overhead. This can add latency right and also also proxy could become the bottleneck of the system if the applications are fast enough.

A

Another problem is that the proxies are standalone binaries that multiple that need to be deployed along with the application. So this means you also need to manage the life cycle of the proxies. You will need to deploy them to upgrade them and also to help check them.

A

A third problem is, you will not have end to end security, because the proxies in between need to enter into intercept the traffic and end-to-end security could be important in some situations.

A

So because of those problems, we were thinking if we can add service mesh support into grpc, so we don't need those proxies, and this leads to a proxy, less jrpc service mesh in this deployment, it's very similar to the proxy based deployment.

A

The only difference is that there's no proxy in between so the drpc service will talk to the control plan directly to get the configuration and also the gipc service will talk to each other directly without going through proxies and the module we added into grp could can understand the conflict, the configuration from the control plan, so it will do all the features that was done by the proxy there's. Actually, one more thing to decide before we come over uh in the proxy based model.

A

You can pick the proxy and the control plane in pair, but this will be a native spot of of service mesh within the rpc, so we actually need to decide which service mesh to support and and as always, what matters most is not. The implementation is the api, and this is the api between the application and the control plan where the configuration is exchanged.

A

So we want to pick an open and popular api that has very strong community support, so there will be multiple implementations to choose from. This will help, which will also help prevent, prevent vendor, locking and eventually the apis we pick is xds xds apis were developed for envoy and envoy is used by many popular service mesh implementations at the proxy, so this makes xds the de facto standard for data planning apis for service mesh.

A

This is an overview of the xds apis. I'm not I'm not going to get into many details of those the the protocol, but I want to give an overview because it helps understand the demo at the end. From bottom up, the first thing is endpoint: endpoint is a server a server instance. It contains the server address and also whether the server is healthy above the quality above endpoint is locality. The quality is a group of endpoints, you can understand it as a zone and a special thing that the quality can do is priority.

A

So let's say we give this locality p0, and this is p1. The the lower price, the lower priority locality is only used when the higher priority is not healthy and we'll have a demo to show this at the end above locality is cluster, a cluster. You can see it as a deployment and by default it can be. You can have deployments for different services. You can even you can also have deployments for different versions of the same service.

A

So this can be helpful if you have two versions of the same service and you are migrating from one to the other. Also cluster is where load balancing is configured, so this blue box will contain call due to low passing between localities and uneven for endpoints within locality above locality. Is the rule? Is the road routes, so this is where request routing is happening. This is where you will do pass matching head imaging.

A

You can send specific traffic to certain cluster or even you can do a traffic splitting between two clusters- and this is also where timeout, retry and other kind of uh features for the for a certain route is configured above root. Is the listener and virtual ip.

A

So this is the part that makes more sense from a proxy's point of view, because, as a proxy, all the traffic comes from a listener from a virtual ip, and this is where the configuration for this for the for the traffic from this listener is configured, but this doesn't doesn't apply very well inside in the rpc. So I'm not going to talk too much about that.

A

Many of the xds features are already available inside the rpc and to enable those features you have to do three things. The first thing is to pull in the xds dependency. Excuse. Dependency is not part of the drpg core, because we don't want to have unnecessary code for those people not needing xds, so you'll have to explicitly add the dependency.

A

A second thing is you'll have to change the resolver scheme to xds. This tells the grpc client to use nexus. Resolver and relovable will then trigger all the other components like the load balancers for xds, a third things you are provide a bootstrap file. This file has the xds server address need, and you can also specify other configurations like you can set a node id to identify this kind.

A

There are some limitations of the proxy less of grpc service mesh. The first thing is: there's still a feature gap between what grpc can do and what the proxies like envoy can do. The xds api is a very rich set of services and we are still actually actively working on adding those missing features.

A

A second thing is that you will still need to deploy a booster file with application, but comparing to deploying a proxy this should be much simpler.

A

Next is that there is a big ecosystem around envoy, especially regarding to observability, but fortunately many of those plugins are also available as grpc interceptors and open sense and status handlers like open sensors and also we are actively working on observability supporting xds.

A

The last thing is that you will need to recompile recompile the applications because, as I said, you will need to explicitly add the xts dependency, but this should not be a problem if you already have a cicd system, and another thing I want to say is that you don't need to migrate all your applications to proxy like proxy. Less at once, even within the same application you can have, you can set different resolver schemes for different grpc channels, so the the channels using xds resolver will use lexis modules with india, pc and those use.

A

Let's say I use ds. Resolver will still go through the proxy if you configure it that way, so it is pretty easy to set up a mix and match proxy and proxy less deployment, and also another thing I want to say is that the goal of proxy last year, pc service mesh, is not to put the proxies like envoy out of business, it's more like to provide an alternative to coexistence with the proxies.

A

If you have non-grpc applications, you will for sure still use the proxies and even for the rpc applications in some situations, it might make more sense to go through a proxy.

A

So, as of october in grpc release 1.33, our xts client already supports the four main xcs services, and also with support reporting via lis note that this means this includes xcs version, v2 and v3 in terms of features. We support the load balancing within localities and revising follow qualities. We also recently added support for pass matching head matching and traffic splitting, so those are the things that we are actually working on: we're working on timeout circuit breaking for injection, we're also working on adding excess support on the server side.

A

Two other big features are security and of the ability you can watch the progress on github. It also you're also very welcome to contribute.

A

This is a list of resources that put together, I included the drpc design docs here in case some of you might be interested in how the xcs implementation is done within their pc.

A

So that's enough of talking and we'll do a demo to show how this works in practice. In this demo, I will use grpc wallet. This is an example implementation we created to show all the features and for the control plan. I will use traffic vector.

A

This is google's managed service, mesh control plan and I'm not going to show you exactly how the configuration for the control plane looks like our mode to show you the the service setup and that and then show you what the client can do so some context for the grpc wallet application, so javascript wallet is awarded for special coin called grpc coin. We all have three services here: the account service. This is where the user information is stored, the stat service. This is where the price for jfc calling is reported and the wallet service.

A

This is where the the number of japanese coins for each user is stored. So the client can make a price call to the stat service like the green arrows, so the client will send the user information with the rpc. Then the stats service will verify with account service for the information and then based down the response. It will send the price back to the client.

A

The time can also talk to warrior service to get a balance for a user sam, similar to the the price call. The client will also send user information, and the wallet service will verify with the account service and all, and then the right service will send an rpc to the stat service to get the price so based on those two responses, the water service will send the balance response with number of jeff's coins and the total worth of the jfc coin to the client.

A

The first demo we're going to show is traffic splitting. So in this demo the client will talk to the warrior service to get the balance and we will create two deployments of the warrior service, v1 and v2. Imagine that you are migrating from v1 to v2 and before you before. You are confident that v2 is is stable enough.

A

You want to split traffic between v and v2, so we config the rpc fetch balance to send 60 of the traffic to v1 and 40 to v2 and and uh as the youtube the v2 service getting stable and stable. You can greatly increase subjectivity.

A

So so this is our wallet client here and we're going to run this command to tell the client the client to just talk to the wallet service, to get the balance and note that we set the xcs scheme uh redouble scheme here to to use the xts recover and also we tell you to make the unary rpcs in a for loop. So we can see the the different responses from different backends.

A

If we run that we'll see those responses from v1.

A

And there's one from v2.

A

So you'll see that the distribution is not exactly 60 and 40, but you will see that there are more traffic from v2 than from v1 than v2, because the algorithm we used here is not it's random best way to wrong ruling. So it's not exactly the conflicting you said, but you can, but that's definitely more traffic from v1 than v2 okay. So the next demo is header matching in this demo. We'll use the client to talk to the stat service to get the price, and we have.

A

We also have two deployments of the stat service, the stat service for normal users and the statuses for only four premium users. The premiums gets the price update with a higher frequency, uh the client who will send the user information with rpc. It also include whether this user is a premium user or not, and then, based on the header will route the premium request to only the status premium service.

A

A

So around this command.

A

So around this command, to tell to tell the client to talk to the stat service to get the price and also we still set the xcs reload scheme and we are using bob as the user in this command. Bob is a normal user.

A

So if we run that we can see this, the response comes back from the stat service. If we run it again, it should still come from the stat service, but a different backend. So if we change this to user alice, we'll see that responses from a stats premium, cluster and and then also the updates- is for much more frequency. Frequent.

A

So that's header matching and the third demo we're going to show is failover. In fact, failover is the priority feature of localities. I just I talked about in the xcs overviews overview side in this demo, we're going to run the client in u.s central and we're going to start multiple services, some of them in u.s, central, some of them in u.s west.

A

uh The the logic here is: if the client and the server are in the same zone, the the server from the samsung will have the higher priorities, so your essential servers will have priority zero and u.s west servers will have priority. One we want to see is that all the traffic will go to u.s central at first, so none of the us west server should receive any traffic and then we're going to turn down the servers from us central and then the traffic will go to u.s west, and uh so this is the setup I.

A

So this is the wallet client, and this is the server from us central. This is also us central. This is us west. This is us as well. So if we run this command, tell the tell it to talk to the stat service to get priced and also run the unity rpc in a for loop.

A

We'll see both the u.s central servers are getting traffic, but the u.s west servers are getting nothing. So if we kill one of the u.s central servers, you'll see the the other u.s center, so we're still getting all the traffic, but the u.s west are getting nothing. So if we kill this one as well, we'll see that u.s rest of us are getting the traffic now, and this fell over happens fast, because the u.s west servers already extended like to the client. They are just a lower priority.

A

So now, if we restart the u.s central servers, the traffic should go back to u.s central in a minute, and this is slow, because the control plan need to do health check of those servers and the traffic will only go there windows. So when those servers are healthy, so you can see, one of the us syndrome is already recovered and the other one is taking a little bit long.

A

And and now and now the traffic are all recovered from for the u.s central service and also it's doing raw opening within those two servers.

A

A

All right with that, that's the end of this presentation. So if you have any questions you can find me and also you can find the grpc team here and if you have any questions, let me know and.