Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting

⏯

youtube image

►

From YouTube: gRPC Communication Patterns – A Deep Dive - Kasun Indrasiri & Danesh Kuruppu, WSO2

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

gRPC Communication Patterns – A Deep Dive - Kasun Indrasiri & Danesh Kuruppu, WSO2

Real-World microservices implementations often use a variety of communication protocols and standards to build different services. gRPC has emerged as an efficient, reliable, and robust way to build inter-microservice communication owing to its rich service definition capabilities with protocol buffers, a variety of language support, and high performance. In this session, we will discuss how gRPC can be used to build inter-microservices communications. In this talk, we’ll explore fundamental gRPC communication patterns; Unary RPC, Server-side Streaming, Client-side Streaming, and Bi-directional Streaming, Understanding when and how to use gRPC Interceptors, Deadlines, Cancellation, Error Handling, Multiplexing, and Metadata, a network analysis of how gRPC communication works behind the scenes.

https://sched.co/ekCZ

A

Hi, everyone welcome to cubecon session on grpc communication patterns. I'm custom industry and my colleague danish group will also join me in conducting this session. So in this session we are going to have a closer look at some of the most commonly used grpc communication pattern and we'll have a look at how they are implemented internally.

A

I'm sure most of you have heard of grpco even use vrpc in production, but if you are new to grpc it's a modern inter-process communication technology that allows you to build distributed applications so that you can design a microservices-based application using grpc and remote consumers can consume it over the network as easy as making a local function call.

A

So grpc is based on a contract first development approach where so that you come up with your own service definition using protocol buffers. So that's where you define all your business operations of your application and then you can generate server side and client side stop code so that you, uh you can establish the communication over the grpc channel uh internally grpc uses binary messaging using uh protocol buffers and uh which runs on top of http 2..

A

So grpc is an efficient, strongly typed polyglot communication protocol that allows you to build a request response style, synchronous communication as well as you can use duplex streaming use messaging in grpc as well.

A

So if you look at the applications of jrpc, so it is often used alongside other technologies such as restful services, uh graphql and even technology such as nets on the event given communication space, so most of the internal service communication can be built using uh grpc, while uh it is most commonly common to use rest and graphql as external facing communication.

A

uh However, it is also possible to expose the rpc service to your consumers direct directory using an api gateway.

A

Now, let's have a closer look at the rpc flow of grpc, so here we consider the same uh application that we discussed earlier. So here we have the production for service and the consumer application.

A

So let's have a closer look at how messaging or remote method indication works in this particular user. So now we have the stop code generated at the client side and the server side so from the client application. I invoke uh the remote method uh in this case. We simply invoke the stubs get product method using my client application code and when you invoke that sub is responsible for converting that message in fact, encode that message and build the outgoing uh protocol performance.

A

So in this case, we create message headers to message pages, uh so obviously we are sending a course request to the service application and also as the path we have, the name of the service and the remote method that we need, and also we have things such as printing type as part of the message headers and as the message payload, we have the encoded message. So this is where we use protocol buffers to encode the language, specific data structures into the protocol buffer wireframe.

A

Then the message is sent over the http 2 connection and and at the server side you can.

A

The server application looks at the path values and find the corresponding stop. Then it is sent over to the corresponding stub and stop unpacks the message and converts the message to the language, specific data structures and invoke the actual implementation of the remote function. So in this case the remote function is invoked at this this point and then the response is sent back from the service drpg service. So this response follows the same path as with the request.

A

Now, if you look at uh how these things are implemented at the http 2 level, so uh suppose you have a consumer, client application and a server application. Then client creates http 2 connection, uh which means it creates a impact. It creates a grpc channel uh so on behind the scenes, it creates an http 2 connection. So once you have the grpcs channel, you can send one or more rpc request over the same channel.

A

So in this case, uh these different rpcs are mapped to rpc codes are mapped to streams in http 2, so here we have rpc4 and rpc3 rpc5 running on stream, 3 and stream 4.. The same applies for the response path as well and when it comes to message frames here we are sending headers and data frames, so it is. This is where all the grpc headers are sent and data is the place that you send all the business specific payload of the rpc request.

A

Now, uh let's have a closer look at request and response messages in prpc.

A

So if you look at the rest request message here, we have the request, headers and a frame, a message frame known as length. Prefix message, so this is where you send all your business payload, so this can be a single message or multiple message based on the message pattern that you are going to use and we will be exploring a lot on length, prefix message in upcoming, slides and at the end of the request, you have to send the end of stream class. So this is.

A

This is a another frame, a data frame similar to bench, prefix message, but it's an empty frame.

A

So that marks the end of the request, the request flow and if you look at the response message, you have the response, headers and length prefix message again same same as the request path and to mark the end of the response message we use, we use trailing header. So, unlike the request path here, we use a header frame.

A

This contains all the trailing headers which mark the end of the string.

A

Now, let's try to uh understand some of the communication patterns and dive into the internal implementation of each and every pattern. So, let's start off with the unary or simple rpc. So, as you know, a simple rpc is all about sending a single request to the service and you expect a single response from the service.

A

So if you look at the implementation of this, so when clients in the single rpc call, it sends a set of headers one length, prefix message and end of stream flag, an empty data frame and in the response path, you have headers response, headers, a lm prefix message, a single message and trailing header. So this is very straightforward.

A

Now, if you look at server streaming scenario, so here we have a single request, one rpc location, but you get multiple responses as the multiple messages as the response.

A

So in this case, request path is very similar to simple accuracy, but in the response part you can see we are sending headers uh and multiple length. Prefix messages followed by a trailing uh trailing header in the client streaming. It's the same thing, but in this case uh request as as part of the request, we are sending multiple request messages.

A

Therefore, we have multiple links, messages followed by your industry and as a response, you get a single length message with headers and credit and if you look at a more complicated scenario such as uh by that bi-directional streaming rpc, so in this case uh in bi-direction and rpc, we send a stream of request and stream of responses.

A

So you can understand this further by looking at this example, here we are sending a series of order, requests to process to be processed by the service, and once those orders are processed server, send service sends back a stream of shipments.

A

So if you look at the implementation of this again, you can see there are. uh There are multiple length messages you have header and end of string and in the response path. Also, you can dehydrate some multiple means, prefix message, and when it comes to the implementation of both client and the service side, you can look. You can create your business project uh by looking at the end of stream flags uh in both request uh part and the response form.

A

Now, uh let's dive deep into request and response, headers and danish will take you through the rest of the.

B

Session, thank you. uh So uh in our previous slides we talked about how message flow in different messaging patterns. uh In this section, we are going to look uh into deep into the request and response messages.

B

First, let's look at headers. So when we talk about headers uh in grpc, there are two types of headers: uh one is uh call definition address, uh called definition headers and the other type is custom head metadata.

B

All definition, headers are predefined, headers supporters by http 2.. If you look at the table, uh there are a bunch of called definition. Headers. uh Some are prefixed with same column, so those are called reserved headers.

B

uh So one of those headers are one is method, so in grpc, the http method, always post and the other one is part that contains the service name and this remote method- and there are some others like authorizations and schemes as well, uh and the other type of headers are custom. Metadata.

B

Custom metadata is an arbitrary key value pass which is defined by the application label there so metadata. We use metadata normally to share informations about grpc call, for example, authentic authentication, headers, etc.

B

So you can see there are a couple of headers which are prefixed using grpc hyphen, uh that is, uh those headers are defined in grpc core implementation.

B

So your pc, timeout, your pc encoders, are such those headers, and so, if you are defining custom headers, we you need to avoid this prefix in your custom, metadata and also in the content type. uh We need to file the prefix application, slash grpc if it is not been given error.

B

The next thing we need to talk about is blend. Prefix messages by definition, message framing is an approach we use to construct information such that the intended audience can easily extract that. So in grpc we uses a message: framing technique called length, prefix frame length, prefix framing, is an approach. We that writes a size of the message before writing the message itself.

B

If you look at the diagram, we show in right side, so you we are generating the encoded byte array uh that and we are compute the size of it and in these four bytes we append the size of that. So in grpc, uh four bytes are allocated to set the size of the message and the size is written as big indian integer.

B

So and also you can see one byte in front of that size, four bytes that represent the compression frag. So if it is zero, that means this message is not compressed. If it is one that means this method is compressed.

B

The compression uh algorithm is defined, uh defined and passed in the uh request message: request headers, so the other thing I need we need to talk about is that how we and how grpc encode binary message by default? The rpg uses protocol buffers to encode this message: the protocol buffer encodes, the message based on the structures defined in this service country. So if you look at the look at the definition, so you can see, uh you can see a proto protocol definition.

B

So in our example, we have order id message and it has one field called id and from that message uh we are there generating the binary uh message binary. So, if you extract the binary you can see, there are different type values, a tag value passed. So, in our case, we have one order message with one field: that means we have only one tag value there, so the tag value pair is mapped to a message fee. So at the end of the message, we pass zero to indicate that it's the end of the message.

B

So uh when we go in deep to the tag and value pair, so the tag is derived using uh field index which is defined in the service contract and the wire type. The wire type is directly mapped to the field type.

B

So in our case uh it is a string type that uh the string like mapped to length delimited. That means the wire type uh value is 2. So from that we derive the tag value and the value of the field is encoded using different technique based on the wire type.

B

In this case, it is string. That means we are using upf 8 encoding to encode the value. If it is an integer, then we may use point it depending on the type actually.

B

uh The next major thing we need to talk about when it comes to grpc is error handling. So it is a first class concept in grpc. That means every rpc call.

B

The response will be either payload message or an error. The error includes a status code which is predefined and it is unified across all languages and also we they pass us status message. We describe the error and also these errors are sent as a response trailing headers. So I capture uh in the first table. We I capture those two headers comes in the trailing as trailing headers. One is the rpg status. Another one is trpc message, india, uh so let's say uh the request computed successfully.

B

In that case, the grpc status will be zero. That means okay. uh If it is an error popped up in the service side, uh the corresponding error is going as the grpc status and grpc message describe. What's the error, this is so when it comes to error handling. uh There are a couple of uh best practices we follow uh the first one is. We do not include error details in the response, payload in most cases, uh so that means all the error. Details are always going through the training headers.

B

There are some of the situation. We cannot follow this. uh Let's say you are using a streaming example and you you need to pass an error detail to the client without uh without stopping the stream. In that case, you need to add error details in the response pair. Otherwise, most of the cases you can send the errors by the training headers.

B

uh The other thing is that server side when we are when we have an error, it's better. We can return all the errors to the client caller. So unless there's the internal state of compromise, in other cases, we can ascend it to the call most of the cases.

B

uh The other thing I need to have emphasize is the deadline deadline, allows both clients and services to know when to abort the operations. The clients are the one who initialized the call uh that uh so once the initial initialize client is setting the deadlines, so deadlines uh is normally set as absolute time. We specify the time which we are brought the operations.

B

So when, when the client in initialize the call uh the deadline informations also go inside uh inside the call inside the request message as the header and when it comes in come to the service, they first look at what is the deadline? Param, uh the uh according to the deadline? Param? They decide whether to pursue the operation or it and send an error to the client.

B

So if the server is calling another service, it's, but it's it's important to propagate those deadline. Information to the other services as.

B

The interceptors are also a main point in grgrpc applications. The mechanism is to execute some common logic. Before and after the execution of the remote function, we can apply it both to the server and client side and, depending on the message pattern we use, we need to use uh different interceptors like urinary, interceptors, streaming, interceptors, etc.

B

The useful uh the main use of these interceptors are for the login scenarios and authentication scenarios, or we need to capture some metrics for those use cases we use interceptors.

B

uh When we come to the evening from implementing services, uh the service questioning also plays an important role. So let's say you have a service running and you need to uh update the service, so the service, the services, should strive to remain backward compatible with the odd triad.

B

So if we have a better versioning strategy that will allow us to introduce breaking changes to the grpc service, so in grpc all these service version is done, you'll see in the package name, so we are appending the version number to the package name, how it works is, uh as we told earlier, uh grpc call is underneath a post request and the part of the part of the request is derived using the package name, service name and the method name.

B

So if you append the version to the package name, that means whenever the version number change, that that will create a different context.

B

So the all client will not affect when we deploy both services in this in the environment, so that if the client needs to.

B

Migrate to the new version, they need to get the correct version of the proto definition and generate the stuff of it.

B

The final thing we need to discuss in this session is to extending service definition. Let's say uh there can be situation where we generate uh the application with different messaging patterns. You need to extend your service definitions and add some custom options in protocol buffer definition.

B

They they provide us this facility to add custom options in different levels of contract, so it can be service level, it can be method level, it can be free levels, etc. So so I captured a few of these scenarios.

B

We may use it. So if you talk about one uh scenario, so let's say we have a service and it's secured using uh external or provider. So you need to pass this or provide the url in the service contract itself.

B

So there you can define the service options to provide the stream or provide the url, and we can use that uh of custom options in my inside our service definition uh same likewise, you can uh add custom options inside a method level and also in the field level as well.

B

Okay, uh that's it we need to cover in this session. So um in this session we mainly refers to the grpc, often running book and all the use cases and the source codes are in this guitar people so that we need to cover. So thank you for listening.