Cloud Native Computing Foundation gRPC Conf 2020, 5 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Real World Migration from HTTP to gRPC in Ruby - Nao Minami, Wantedly, Inc.

Description

Real World Migration from HTTP to gRPC in Ruby - Nao Minami, Wantedly, Inc.

As of today we are running about 100 microservices at Wantedly, built with Ruby, Go, Python, Node, and Rust on our production Kubernetes cluster. While we’ve been historically using HTTP/1.1 for majority of inter microservices process communication, since the last year we have started introducing gRPC for some of our microservices which requires low latency and high throughput. This talk will cover the details of our migration approach and the performance improvement we’ve achieved as a result. Average latency got decreased by 50%, and 90%tile latency was decreased to a seventh. We will discuss our migration steps we’ve took, technical and organizational challenges we’ve faced during the migration, and also tools and libraries we’ve made in order for smooth transitions for servers originally built with Ruby on Rails.

A

Thanks for joining my session today, I'd like to talk about iowa and why we migrated into micro services, communication method from http to jbc, especially in our rubric microservice.

A

My name is nao and I work advantage from tokyo and I work as a software engineer at the infrastructure team. My team has two responsibilities: sre and development infrastructure.

A

Why I worked on introducing of grpc in the microservices in this talk, I'd like to talk about those three things.

A

The first one is why we decided to introduce grpc and the second one is about how we migrate it from http to jpc, and the third one is what we achieved by using jpc.

A

Okay, let's start from y at first as a background, let me introduce my company's products. We have several products advantage. Visit is a product for javascript candidate matching advantage. People is a product for professional contact management by this product. We offer our user experience like linkedin based in agil.

A

These products are on one platform, which means they share the same set of users, profile data, so use that profile. Data is very important for our products.

A

Our products are built with more than 100 micro services. They are written in ruby, girl, python, non.js, etc and run on the kubernetes cluster. We have been using h1 for majority of my microservices.

A

As I said, user profile data is very important for our products and it is managed by a service. We call user service, so this service is very important for us now. This service is written in a movie by historical reason, and today I focus on this service.

A

Let's dig into the details of user size, user services and microservice used to get or update user's profile data.

A

This substrate fundamental data so used from many micro services and the high throughput is required, and this services latency often directly affects the user experience. The low latency is also required.

A

This service was widely used in the microservices, but there was a problem.

A

So what was the problem? The problem was the latency caused by using h1 without keep arrive. Hn does not perform well for several reasons.

A

For example, dns program and the tcp 3 handshake is necessary, increase their latency and the tcp strong stuff is also a problem.

A

We reduced the server side latency as much as possible, but it was not enough to meet the increasing demand, so we decided to introduce jrpc.

A

Ygrpc, as you know, there are some rpc frameworks involved and we could choose a suitable one. In that situation, we've decided to use grpc by those three reasons.

A

The first reason is high performance, grpc is written h2, and so it can use run, drive real-time communication streams. This gives us low, latency and high throughput.

A

The second reason is protocol buffers grpc uses protocol buffers. It provides a single source of truth of grpc service definition.

A

It is a very good point and as an additional good point, we were already using protocol buffers over h1 before using jpc, so it was a very natural choice for us and the third reason is multiple ranking support.

A

We have micro services written in several languages, such as ruby python, go nos.js, etc and the grpc support disadvantages. It was a very important point for these reasons. We chose grpc.

A

Ok, the next topic is how we migrate it from http to jpc how we migrate it. Looking back, I think there are two steps. The first step is preparation and the second step is migration.

A

Let's start from the preparation step for using jvc, there are four required preparations, schema management, monitoring, productive development, environment and load balancing right now. The jvc ecosystem is not so mentioned enough, so especially in ruby. So we have to think about those things.

A

The first one is schema management without the monorail, which means under the system maintain multiple robberies. We have to share the same profiles across the multiple repositories, so how should we share profiles?

A

A few approaches we can take here, for example, copy and paste manually in this approach. It's very hard to update multiple laboratories continuously download profiles automatically in each battery.

A

In this approach, you still need to generate code manually for each repository, so you will suffer from the differences in each device environment, for example, version of proxy presence over plugins, etc.

A

So we decided to make a central repository for profiles apis. This is the central repository for schema management, api source, open files for our company and do various tasks in the ca pipelines.

A

How to use aps report3 in this repository code is automatically generated in the series pipelines, ruby go and node.js code is generated in for each profile, then generated code is distributed via packet managers such as bandwagon, commodores and npm.

A

With this workflow developers don't need to generate code by themselves and can focus on the implementation of features.

A

This is an example code in apis, ruby code generated and published at rubygems by using bandra. The robot can fetch and use a generated ruby code.

A

So, with this workflow how we drop jp servers on the client in ruby for jpc servers, we implement a cloud which enhance the generated grpc service class based from apis by implementing instance methods for each ipc method. We can use jpc servers written in ruby and for jrpg clients. We use a subgraph first from apis by using a stub object like this, we can communicate with grpc servers in ruby code.

A

This is the basic usage of apis and we drop grpc servers and clients like this.

A

Okay and the next topic is monitoring.

A

As you know, application monitoring is required in production.

A

As a background, we were using various source products such as feedback, numeric, animation, etc. For monitoring our h1 servers, we wanted to continue to use them for grpc servers.

A

The hard monitor, grpc servers for this purpose. We can use grpc interceptors. We need to send metrics of hdfc request and with grpc intercepts that we can implement such features.

A

Jrpc intercepts, as a user activist, the current interceptors and server interceptors client interceptors are executed on the jvc client around the user code and server interceptors are executed on the jvc server. Before the request is passed onto the user's application logic, it is a perfect way to implement command patterns, for example, logging monitoring, authentication, etc.

A

So we can use jpc interceptors for monitoring, but jpc interceptors for each such products are not actually provided at this time. So we have to implement some grpc interceptors by ourselves.

A

We implemented them and made them available to the rubric community as open source software. The published interceptors are shown here. For example, we implemented the grpc interceptor from neuric.

A

With this channel we can send the metric matrix to numeric in each request and the other one is a jpc interceptor for open sensors. With this gel, we can send the metrics to some such products supporting open sensors and another one. Is the jpc interceptor for access logging.

A

With this gen we can print access routes in a specified format by this grpc interceptors. We monitor our grpc servers.

A

Ok, the next topic is productive development environment. As you know, it is very important topic.

A

Now we need to create a product development environment for jpc by ourselves for h1 servers. The ecosystem is major and the many open source source libraries exist, so we can use them, but this is not the case with grpt servers, especially in ruby.

A

We need to create a productive development environment by ourselves, so how to create success environment for the productivity we use a common utility library named subsix. We want to use some useful features for you, for example, hot reloading and utility for protocol buffers, and we want to avoid repeating the same configuration in multiple micro services.

A

So we implemented a common utility library named servicex. Servicex is used by all jpc microservices in our company.

A

I'll show you some useful features of servicex for jbc servers. We implemented jpc server command. This is used like this. We execute bundle, xx, jvc server, then jfc server process starts run.

A

It prints some rocks and the third reason known about 6046.

A

With this command, the rpc service class is automatically loaded, and the sum of jvc intercepts are automatically set. Afterwarding feature is also implemented with this command developers can just focus on the implementation of the grpc service classes and next for javascript clients we implemented servicex jpc start form method.

A

This is just like this.

A

With this method, some jbc intercepts that are automatically set and the user agent is automatically configured so their developers don't need to do complicated configuration, so they can just focus on implementing the business logic and the next. By using protocol buffers in ruby, we implement the utd library named pb with vb. We can create photograph objects more easily.

A

For example, as shown here, we can create a timestamp object from a string object in the easy way without bb we have to convert a string object to a time. Object by time.first method, then pass it to the google.timestamp class as an argument.

A

On the other hand, with bb, we don't need to do such complicated things. By passing, a string object to bb.27 stamp method. We can get a protocol timestamp object easily, and the fibo.method is also useful.

A

With this method, we can create user-defined prototype object from hash object, as shown here. The values of hash objects are automatically converted to broadband objects, so we can create nested prototype objects in the easy way.

A

Maybe it's included in subsets, but we made it available to the ruby community as open source software. So if you are interested, please try it that's all for services, and the next topic is vmrays for productivity. We decided to use grpc combined with parts of program rates.

A

At the background, we migrated to migrated from ruby on race and wanted to keep using possible features in it. So we decided to use jerpush jam with rav jam and we found that it's very useful to build a jpc server with portable uploading rate.

A

So we recommend this configuration if you are in the same situation and then next, let's think about the load balancing. What should we consider for it for grpc, f4, load, balancers and very useful?

A

As you know, jrpc is the rhythm h2, which is designed to have a single long live tcp connection, so in forward balances. Once that connection is established, there's no more browsing to be done.

A

Our request will get into a single destination. This is not a good situation, so we use steel for s7 load, balancing in kubernetes classes, as shown here and by continents. Configured by steel are injected as cycle proxies and used for l7 application level. Road balancing by this configuration, the load is evenly distributed.

A

Okay, I explained the preparation step next I'll, introduce our migration step so how we migrate it in the migration, a basic strategy that tries more at first, we made a prototype as a profile concept to frequency impact, and then next we used grpc for our part of the traffic in production and gradually increase the percentage of the jvc.

A

This is a schematic. Diagram of plc have a development vanettes cluster. This is separate from production faster and we tried jrpc in this cluster by using a development cluster, we could try jrpc without worrying about affecting production environment.

A

This is the result of poc.

A

This diagram shows the distribution of latency, as shown here, latency decreased dramatically in all cases. By seeing this result, we decided to try jpc in a production cluster so how we tried grpc in production.

A

This is a schematic diagram of our trial observation in the production cluster. At first, we used jvc for only a small part of the traffic.

A

By trying small, we could minimize the preparation cost to minimize risk and the next we gradually increase increase the percentage of grpc by gradually increasing the traffic. We could control the risk and finally, we could achieve the complete migration.

A

With these steps, we migrated to grpc okay on the next. Let me introduce the result achieved by using jpc.

A

As a result, we've achieved the performance improvement by using jpc average latency decrease dramatically.

A

Average latency decreased by about 20 milliseconds per request, and the impact was more remarkable when many requests to user service were involved in one web transaction, for example. As soon as shown here in the microservices average frequency decreased by a half the brand one is the latency when using h1 and the later one is the latency, when using jpc average latency to decreased from 130 milliseconds to 65 music by switching to jrpc in this microservice, multiple request user service were involved, so the impact was remarkable.

A

So by using jrpc we could achieve a performance improvement and improve the user experience.

A

Next, let me introduce what we are doing since migration.

A

After the success of the migration from http to jvc and the achievement of performance improvement, the use of grpc is increasing throughout the organization.

A

Now almost all clients of user service are using jvc and the new microservices are being developed using grpc and the jvc started to be used in other languages, for example, gold, not js, etc.

A

So jrpc is becoming vitally used in the organization.

A

This is the summary of this talk at first, I talked about why we chose jrpc and the answer is what we needed: a high performance average framework and the jvc was performant enough, and another reason is that we by already using protocol first, so it was a natural choice and we also needed multiple languages support because we had micro services seats written in several languages, jvg supported them.

A

It was a very important point, and next I talked about how we migrated from http to jrpc, by using jpc, we have created a workflow for protocol buffers using our aps repository, and we have also developed some libraries for productive development environment of jrpc and published some of them as oss and in migration step. The basic strategy were try small, so we tried in the smoke and the quick way. Then we could achieve the migration.

A

At last I talked about some results of migration from http to jpc. By using jpc, we could achieve the performance improvement. Now the use of jpc is increasing throughout the organization. That's all thank you for listening.