Jakarta EE EclipseCon 2020, 26 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Make your microservices as much robust as mission critical systems by MicroProfile and Jakarta EE

Description

One of the most important things when building microservices for the enterprise mission critical systems is high reliability. Even if a new deployed service has a bug, it is required that the other services should not be affected by this bad service.

This session will provide how to build such a robust system with MicroProfile functions such as Fault Tolerance, Metrics, and OpenTracing. Although there are a lot of tuning knobs in Fault Tolerance (timeout, circuitbreaker, retry, fallback, and so on), it is very difficult to find the best values for each parameter of each microservice.

This session will provide a demonstration to show how to tune these parameters in cooperation with Istio, and various other tools, such as Grafana and Kiali, to monitor microservices, and the attendees will learn how to find the best parameter values.

Speaker(s):
Kenji Kazumura (Fujitsu Limited)

A

Hello: everyone, my name, is kenji kazamura. Today I talk about microservice, specifically the for the enterprise system, which usually require robust and reliable quality.

A

Who am I I'm working for fujitsu for several years and I'm responsible for development of launcher, which is an implementation of micro profile and enterprise application server, which is an implementation of jakarta ee? I'm a member of some committee of jakarta e working group and also a member of jcpec.

A

So today's agenda, first of all, I introduced the current status of microservice on the enterprise systems. After that I provide some guideline how to make a robust system with micro service, and then I will show some demos with micro profile and steel and some tools, such as grafana or care.

A

So this is the jakarta ee developer survey. One of the question is: what architecture approach do you plan to use for implementing your java system in the cloud and 39 percent people answer microservice and 23 percent on the hybrid?

A

I think these numbers are not bad, because the target of this survey is developers but in reality there is not so many micro service systems in the enterprise mission, critical area.

A

So I think the enterprise people are usually conservative and they are reluctant to adopt a new technology after the technology are matured.

A

They start to use this technology to their systems, so in case of micro service reliability is one of the key points for the adoption in the enterprise system, because the existing system is designed the concept that failures should not happen, but, on the other hand, micro service is designed for uh resilient on the failures.

A

So there is a gap between the existing service and microsoft and we need to bridge this gap.

A

So how do we make service robust.

A

When considering the reliability in microservice, we should know the importance of each service is not same. That is some service required very high sa, but some are not. That is you need to discrimi distinguish which service are important and which service are not, and one more thing to be considered is when adding new service.

A

This service must not affect the existing.

A

Specifically, the high sl service.

A

So, given that, given this consideration when using microservice, I think there are three points to be considered.

A

First, one is relationships among service, as I mentioned the previous slide.

A

When something wrong with one service, you need to know which service are affected by the bad service and you need to isolate this bad service, and the second thing is, I recommend, to use for tutorials to each service, but the problem is: it is very difficult to find out the best value of the parameters of ft, because ft provides a lot of features and lot of parameters, and it is difficult to find out the proper values if you do not set the proper values to the ft.

A

The system uh getting worse and third one is observability, so the only setting the fortrans is not enough. You need to keep looking a lot of metrics of service and the system is working well with your setting for trans parameters.

A

So here I pick up three major four trans features: timeout fallback and circuit breaker, so we use timeout. So you can use the timeout for the service, which requires timely response, and if the service has no alternative method, you should use just timeout, but in case of fallback, this service has some alternative method: alternative service.

A

You should use fallback and circuit breaker if the service has no alternative and this size may affect other service. In such a case, you should use circuit breaker, so I will explain more details with a concrete example: food delivery system.

A

This system is not real system. This is a fake system and available only this session.

A

This system provide the customer, the food and delivery. The food to the customer, the first of all customers log into the portal service and put a service asks uh account service and if something wrong with account service, we just time out, because this service has no alternative.

A

So we just uh sorry page to the customer and just time out and the customer ordered the food, for example, burger or pizza, or whatever the portal service requests to the order and order dispatch. The request to the each shops, burger or pizza depends on the customer's request and the shop service, such as burger service requests to the delivery company.

A

So there are some choice and, for example, broomstick or duke bike, and here we can use a fallback. For example, uh if a broomstick has some scene long in this. In such a case, vargas service switch from broomstick to bike, because the customer doesn't care the which this very company delivers the food. So in this case we can use fullback, but here we cannot use fallback because, for example, if something something wrong with bargas systems.

A

We cannot change from burger to pizza because the customer wants to eat bug, but not pizza. So in such a case we cannot switch from bullets pizza. In this case we use, we should do the circuit breaker and we use this system and for the demonstration.

A

I will show the way to observe the service status by using grafana or query, and I also show how portraying's value affects the service and in this demo I use microprofile in launch, which is an implementation of micro profile.

A

So I use six service in this demo portal order, burger pizza room and bike, and each service has a entry point and these service communicate each other by a rest api to simplify the demo. The customer specify only two parameters: shop, name and full name like this.

A

So so, let's look uh quickly the program code or this demo program. This is a portal uh service and there are two metrics here: burger timer and pizza timer. uh It is for uh to measure the response time of burger service and pizza times.

A

I will use this matrix later in this demo and what is doing here is just called the order. Service order is a rest client and before calling the order, the timer start and after the order service, timer stop and next order service.

A

This is also just calling burger service or pizza service. It depends on the customer request and baga and pisa are less client and burger services and bhagavatam is also just called duke broom it. This is a restaurant, so it's the program is very simple: each service called the other service by a rest, client.

A

So and these six services are learning as kubernetes port bike, bloom bagger or the pizza port and also istio service, also running other backgrounds. So I use six application and steel to order. The food.

A

Use http requests.

A

Like this, for example, shop, eagle, bugger and food hamburg so and.

A

Response is like this burger queen, accept the old hamburger and duke plum dilbert, and if you want to order the pizza it's similar way, so you specify pizza and margarite and response is same similar as response.

A

And next I show you the query window.

A

A

And hide name it unknown unknown is promoted itself. So I just ignore this.

A

This is a clue window and.

A

So you can see the you can understand the relationship of the mic service with this kind of graph, so portal called older service and other service called vaga or pizza and the bala side is called bloom. So you can easily understand the service relationship.

A

Okay, next, I put.

A

Some bug intentionally here, I put some bug in broomstick program.

A

A

So every three times it's london reads: sleeps five seconds and I put the timeout and the callback time at one second and call back.

A

So fallback cross is here and it's simply called a duke bike instead of duke broom, and let's see what will happen but before that.

A

In order to see the metrics, I set up the grafana dashboard and add panel and first select a metrics application, and bugger time mean.

A

So this is a phrase time.

A

Set gauge so this matrix is.

A

I created in the application here this matrix.

A

And you and one more matrix.

A

Application and bug timer. Second: this is a ascentive information.

A

A

A

Okay and let's see what will happen so in this case, I use some script to call the bugger service in this script. Just call bugger service repeatedly.

A

So let's do all logger service.

A

So if you see some request, use dukebroom and some requests use, nukebike and first look: the qre console.

A

Okay, so you can see uh barrier service request room, but in addition that bargain service uk is also bike service, because some request, timeout and burger service requests to the bike as a fallback and let's see the metrics okay, the average of the response time is 0.36.

A

Second, it's not bad, but uh when looking at the percentage formation, this is 99.9 percentile and this is 99.99 percent and this is 98, and this is 95 75 and maybe 50 tile.

A

So if you look at the percentile, it's uh almost 1.1 second or 1.3 second- and uh this is uh expected behavior, because I set one second for timeouts, so this is uh expected uh result.

A

So the question here is: what value should we set for timeout?

A

The answer is, first of all, you need to set sla or sao to this service and, if you don't care the percentile information, you just look at the average time and this value and to see if this value satisfies the sla urs array or seo.

A

And if you care about percentile information, you need to also the this kind of information and you adjust the timeout value until the response time satisfies your sla.

A

So next time I put another bug into here peter, but before that I developed the back.

A

A

For tournaments.

A

A

And in this time I put the uh back into the pizza service.

A

So in this case every two times randomly sleep 10 seconds and at first I just put the timeout and retry and let's see what will happened.

A

And before I request to the system I put another matrix so select application and this time add pizza, matrix pizza time me. So this is average.

A

A

A

Matrix application and pizza bit.

A

Second: this is the past entire.

A

A

A

Okay and this time I use the other script like this. We call two types of http request. One is for the burger and one for the pizza and call repeatedly, and let's see what will happen.

A

So let's see the matrix, so it's time it's getting time to reflect the matrix.

A

A

Yeah, okay, yeah and the average of the response time is 1.4 second and 1.2 seconds, and it's a little bit worse, but the problem here, if you look at the percentile, almost 2.8 second or 3.3 second or 3.8 second, but it is okay, because pizza service has a bug and this results are expected.

A

But uh problem is here burger response time, also very bad. Although bag has no bug the response. Time is very bad.

A

This means that pizza service affects the burger service. In this case, we should use circuit.

A

A

So the meaning of these parameter is 25 of 4 request, failed, the circuits become open and when the circuit is open, the request do not wait. Instead, return immediately with a circuit breaker exception and all the service catch, the circuit breaker exception and return to the service unavailable code. This is 503 is hcp code.

A

So, let's see what will happen.

A

A

And again request.

A

So, let's see first carry console again.

A

Okay, so you can see here uh some of the requests returned: five aks five excess code because circuit breaker exception are thrown and let's see the.

A

Matrix average time is getting better and still pisa has uh um uh bad number, but it's okay, because pizza has a bigger and burger the response. Time is getting better. So in this session we understand that sex breaker is very useful.

A

A

Higher reliability of microservice is necessary to be succeeded in the enterprise market and one of the key. The enterprise microservice is observability and microprofile and steel, and some tools, such as grafana kiana tre, provides a way to observe microsoft, and this is the end of my presentation. Thank you very.

A

A

So we have three minutes for qa.

A

A

No questions so.

A

A

Is there any audience.

A

A

Okay, so uh I I show you some uh the reality of the enterprise microsoft situation, so thank you for participate, uh my session, uh it's a great pleasure for uh presented in xcom 2020.. Thank you very much. Bye.