Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2017 (Austin), 15 Dec 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Enable your Microservices with Advanced Resiliency and Fault Tolerance Leveraging Istio

Description

Enable your Microservices with Advanced Resiliency and Fault Tolerance Leveraging Istio [I] - Animesh Singh & Tommy Li, IBM

Building and packaging microservices is one part of the story. Given a highly salable and distributed microservices deployment is going to face failures at different layers, how do we make these microservices resilient and fault tolerant? How do we enforce policy decisions such as fine-grained access control and rate limits? How do we enabled timeouts/retries, health checks etc? Even though some language specific frameworks address these issues, the implementation is often framework or language specific.

If the underlying framework or language changes, the resiliency features need to be reimplemented or ported over. And in some cases, applications also have the responsibility of implementing the code and configuration required for resiliency and fault tolerance. A Service-mesh architecture attempts to solve these issues by extracting the common resiliency features needed by a microservice framework away from the applications and frameworks and into the platform itself. Istio provides an easy way to create this service mesh.

In this talk we will discuss how to build, deploy, connect your Java microservices leveraging Istio service mesh. We then show how to configure and use circuit breakers, timeouts/retries, rate limits and other advanced resiliency features from Istio without changing the application code.

About Tommy Li
Tommy Li is a software developer for IBM focusing on Cloud, Container, and Infrastructure technology. He has worked on various Developer Journeys on Kubernetes, Microservice, and Hybrid Cloud to provide use cases on cloud-computing solutions. He is also passionate about Machine Learning and big Data.

About Animesh Singh
Animesh Singh is an STSM and Lead for IBM Cloud, Containers and InfrastructureDeveloper Technology. He is also a member of IBM Academy of Technology, and has been an active open source IBM champion. Over the course of last few years, he has been leading major initiatives for IBM Cloud and Bluemix and currently works with developers to design and develop cloud-computing solutions around Kubernetes, Docker, Serverless, OpenWhisk, OpenStack and Cloud Foundry. He has also been leading cutting edge technical Cloud products and projects for IBM Customers (US and around the globe) in Telco, Banking, Aviation and Healthcare Industries. Animesh has numerous patents to his credit and runs Bay Area meetups and user group son Cloud Foundry and Serverless technologies, and is an active speaker in conferences around the globe.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Hello: everyone thanks for joining. My name is animation, and this is my colleague Tommy Lee or together. We work for IBM, Cloudant containers, developer, Technology, Group, and today we are here to talk about how you can build resilient and fault, tolerant micro services or leveraging iste ió.

A

Now before I go into some of the details, how many people are actually using service, mesh or sto here very few I didn't see any hands raised, in fact, so I think it will make sense to actually you know, go back and talk a bit about the motivation of the project acceptor of what led to it, but essentially in 2014 there was a reactive manifesto which came out more than 20,000 people have signed it.

A

If you haven't read it, I would HIGHLY encourage, go and read it very nicely, but the key theme there was that today's application demands can simply be not met by yesterday's application architectures, essentially, today what we want systems which are responsive, resilient, elastic and message driven now, if we focus on the resiliency piece office, what it says is they are significantly more tolerant of failure and then failure does occur. They actually meet it with elegance rather than disaster. That's essentially one of the two characteristics for a cloud native application.

A

Typically, during the same time, microservices revolution was happening or it started somewhere around that same time frame and essentially, as we are all aware, you take a monolithic application, break it down into single functions which are typically around business capabilities, and you have small teams which are actually developing, maintaining and deploying those micro services. Now, typically, micro services are encapsulated inside containers. There is a one-to-one relationship most of the times. Everyone's container journey starts with one container and it's easy in the beginning, but soon the growth becomes overwhelming.

A

We need container and micro services management enter container Orchestrator. Now, if you look at a survey which was done last year, where we respondents, what are they expecting most from a container orchestrate or scheduling cluster management and discovery where the top three functionalities identified?

A

Now, if you look at the continuous stack and all the layers, definitely popular container orchestrate is like unity, swarm Meisel. They actually said at layer 5 of this.

A

We are all aware of the cuban --'tis architecture, any interaction we do with communities through our UI CLI api goes to cuban aires master, and then we have slave nodes, which are essentially responsible for running your workloads. They actually run the cubelet queue proxy docker demons and they're, then serving your workloads on top of those slave machines. Within IBM we have our own IBM cloud container service, which is based on chip annuities. We manage its a managed service, so we manage the kubernetes master. It's single tenant, the slave nodes are actually managed by the customer.

A

We do provide triggers for OS updates patches regarding hypervisor, Blanca, daemon, etc, and customers can actually scale up and down based on their need. We actually use the logging and monitoring data from the slave nodes to provide the management, but we don't have any access to the persistent data which is residing on customers cluster.

A

What we did was we actually also created a lot of developer patterns to show how communities and micro services work great together, so they are essentially on-ramps to technology and they actually help developers be productive with cloud data in AI.

A

You can actually find a lot of art, available patterns on developer.com, slash code, slash patterns and anything you need versus getting started with the technology bead, the architecture, diagram, step by step instructions, the actual code in the github repo, it's all there, so some of the micro services pattern we created for all the Java when spring boot fans out here. How can you actually run Springwood micro services on top of a cuban? It is clustered in addition in this particular pattern.

A

We also show how can you use functions using the very popular open source, open risk, which is part of the Apache foundation?

A

There is another Java ie framework which is becoming quite popular, which is micro, profile, it's being backed by Red, Hat, IBM, Fugit, C and others, and we'll talk in bit detail about this particular micro profile micro services framework. So there is a developer pattern to actually use that and if you're in a polyglot land where you actually are having micro services written in multiple languages, there is a pattern to get. You started there as well. So what we see is Cubana DS is great for micro services.

A

We need anything is and that's the question. A lot of people were asking kubernetes and micro service has already handle a lot of your use cases. Now. What we saw on that slide was the top three functionalities which a container Orchestrator gives you scheduling, cluster management and discovery, but is that enough? We need to build fully reactive and resilient micro services, and for that we need mechanisms to actually avoid fault. So that means, when you are doing Canada testing, a/b testing problem out new versions. You need to be able to selectively doubt traffic.

A

You need fault isolation, so that you have ways to create circuit breakers. You are enforcing bulkheads, accept, detect faults when they do happen. For that you need create metrics capability and then, if failure does happen, you are able to recover from it in a graceful manner. In short, what we need is strong visibility, fault, tolerance, traffic control and a way to incur and for security and policies entered service mesh.

A

So what essentially, is the service mesh a way to think about? It is think of it as a network of devices. So, essentially, you have a network in your data center, which is connecting to machines in this case. Instead of connecting to machines, you are connecting a couple of micro services together and all the tasks which typically the routers and the switches would handle that's all offloaded to the service mesh.

A

Now, how do you actually build a service mesh and your site cause so sidecars actually are gatekeepers which actually sit within your port, along with your micro service, and they are responsible for intercepting every incoming and outgoing traffic and because they are able to do that, they also provide you richer, outing, load-balancing, plus they are collecting a lot of day. They can pass to the metric systems.

A

Sto, essentially, is an implementation of the service mesh launched earlier in the year. Ibm Google and lyft came together and if you look at the architecture of Sto, essentially three key components are their pilot, which is actually responsible for configuring. These two deployments and measuring that all these configuration are propagated to all the components in the system, all the routing and resiliency rules which we create. They actually go in history of pilot mixer, which is responsible for policy decisions like a Sears rate. Limiting authorization.

A

Acceptor mixer is also actually responsible for giving you the dashboards, which give you the great metrics capability and then finally proxy, which is essentially the sidecar, which is the heart of the service architecture that is based on envoy, and it mediates all lane bound and outgoing traffic and it's responsible for enforcing all the policy decisions. All the routing decisions, load-balancing decisions when your application does get deployed.

A

This is how it looks you have is to your control plane with all these components and then finally, your Easter data plane, which is essentially hosting your application, taking a bit more into the architecture. As I said, all the traffic entering and leaving is being intercepted by Envoy envoy, essentially is a layer. 7 proxy, which was developed by lyft very high performance proxy, are able to actually handle up to 5 million requests per. Second, that's essentially the heart of this. You then have ingress proxy, which is the gateway to your application.

A

You can actually use envoy for that as well. A lot of people actually use your cube communities, ingress controllers and then nginx also announced they have added support in also, you can use engine X instead of Envoy and all these different protocols, G RPC HTTP to HTTPS, T or currently.

A

So how do we make micro services resilient with this to you?

A

So, let's see the capabilities which is to you is providing traffic control and visibility. Let's go into them a so, for example, when you are rolling out a new version of your micro service, all right, you don't want to redirect all the traffic to that you want to selectively control. In this case, a rule like just send one percent of the traffic to this new version will actually ensure that if there is some port in your new version, you can actually roll it back in a timely manner.

A

You can steer the traffic based on the content based on whether the user is using an iPhone or an Android or a particular browser, or the request is coming from a particular geographic location. You can steer it to a particular version of your micro-service. Then I talked about visibility, which is essentially the key.

A

How do you actually look what's going into your service mesh, so Steve comes, but by default with Jeff on a--, with Prometheus back-end, as well as Zipkin, which actually gives you consistent make metrics across the whole fleet of micro services, which you have deployed.

A

Essentially, since boy is intercepting every incoming and outgoing traffic, it's also able to store that data and be able to transfer that idea to a mixer which is based on a plugin based architecture and Prometheus and graph wanna they're actually plug-ins for this, and you can actually bring your own dashboard into this mix and definitely Zipkin, which gives you the request tracing capabilities and which is very important. You are able to look look at the source of the request. You are able to look into the request.

A

Headers you can see which services are lagging which are slow. All that information you can get from the zerkin dashboard. So around the same time Steve was launched. We launched a developer patent. How can you actually manage micro to services traffic using history on communities, so we took the sample booking for application, but we also what we did was we added a relational database component to it when the sample application came out, it was a static application.

A

Everything was being written to the local file system, so what it allowed us to do was a make the application dynamic B. We can also show that for aggress traffic, where a certain protocol is not supported by history or, for example, in this case you are using JDBC to connect to a relational database. How does this to handle it?

A

This is essentially what's happening in that particular pattern. So, essentially, what we are showing is at how you can selectively route traffic to different versions of the micro service. Now, if you would have seen booking for many times, but if we don't book info actually gives you a product page which gives you the details of the book, then there are reviews micro service. There are three versions of it which essentially are reviews of the book and then finally, you have ratings, which are like how many stars this is getting.

A

So in this case we are showing that we are splitting the traffic 5050 across the version two and three, and nothing is going to be one. Similarly, if a particular user is coming, we can actually select where to redirect him. The other thing which we can do is that we can limit the access to a particular destination micro-service. So in this case, when we see, we probably want to say that reviews version 3 doesn't and cannot talk to ratings micro-service, so you can actually base me make decisions based on that.

A

So now, let's look into fault tolerance and if you still are questioning why we need fault tolerance, because things will go wrong, how many of you were on the rainey street yesterday night I was like one of these and I was shivering. There right say contradict things.

A

Things will go wrong and essentially, before going into this, let me talk about some of the definitions which we will use and I want to make sure you know everybody is aware of some of these terms, which are being used in the resiliency features, so circuit, breaker and I'm going to read it verbatim, because this is a great definition. So the basic idea behind a circuit breaker is very simple.

A

You essentially wrap your function, call inside the circuit, breaker object and what it does is once the failure reaches a certain threshold, the circuit breaker trips, so that any other further call to the circuit breaker returns with the error without overloading you're protected method. So that's the idea behind circuit breaker. Then we have bulkhead right. So that's a pattern which is talked a lot about in the micro services use case now. Essentially, if you see in the industrial world, a bulkhead was used in ships or aircrafts actually to partition them into sections.

A

So if a hole does happen in the hull of the ship, in this case, you know the motor will only fail one component or one compartment and the ship wouldn't sink. So that's the idea, which is also carried forward into the micro services patterns.

A

Now, let's test the resiliency of the sample application right. So within IBM we actually have developed of two IBM. Research essentially is developing it's in very early stage, but that actually gives you a great visualization whether your application is resilient or not. So it gives you a control panel to inject false. You can actually inject false in your sample application and then visualize the traffic flow using the view panel.

A

In the first case, what we see is we actually abort the ratings microservice, which was the last micro service, and in this case the system is behaving as if the ratings micro service is not available. What we see is reviews micro service has written correctly to come back and just show the reviews. If the ratings is not available, but if you actually add some delay when your ratings micro service is delayed, there is timeout happening. In that case, the application crashes. It is not able to handle and know the reviews.

A

Micro service just shows that it's not even able to process and show you the reviews, which is the wrong behavior. So this tool actually allows you to test fault tolerance and it's actually pulling the data from Prometheus and zip cane and giving you those visualizations, and you can actually click on the details and be able to see what are the response times where it is lagging, except with that Tommy would give a very quick demo of the sample booking for application and this tool.

A

As I said, the tool is in very early stage, so it doesn't work. We do have a recording. We have a fault, tolerant and, and then I'll come further and talk about how do we actually handle resiliency in application frameworks themselves? And then how does this you handle it with that Tommy.

B

I am Tommy I'm, going to show you to pull info samples and how to use that tools so for the purpose of this demo already have Pio and input deployed on my Crestor.

B

So, as you can see right, this is a info page and you have the details, micro service, showing the details, and we view my post service, of showing the reviews and also the rating micro service will help you help some of the we be some emotional to review micro service to show the review stars.

B

As you can see, this free version will review. This is the version one of three years. This wasn't free of the reviews. It is version two of the reviews right now, I'm going to apply on steal traffic, routing rules to allow all traffic to version two of the reviews, so I will run a steal, control, command.

B

So right now all the traffic should route to the version. Two of the reviews, micro-service.

B

As you can see in the matter, how many time I refresh you'll always see it, they don't review with the black stars.

B

So right now, I'm going to show you the tools. These two is called steal. Analytics is developed by being internalist. Well, it does is equal to pry force injection into the sto pilot and also help you visualize, like some of the traffic's, using the logging data from Roman theists. So let me apply some traffic to our point. Four page.

B

Shows right now is all the login data for the past thirty seconds from fermentis. So you can see all the ingress traffic is going to the product page and product page is calling the details and reviewing microservice to get the details and we've used. While we also caught a rating to get the rating stars.

B

Right so right now, I'm going to pour all the traffic to the reading microservice.

A

Which is going to the ratings microservice? Hopefully, if it is working fine, if the Reapers are able to go through, would be able to see that the application is.

B

So the truth does take some time to apply the rules, so give it a second. So as you can see right for the past thirty second, the traffic to the rating is decreasing because we put all the connections and we refresh it as we can see the review micro-service, it's good at handling five hundred steps test at code from the rating service. So it's still working fine is healthy. However, your pride delays on that breathing micro-service.

B

Now I'm going to a prior 20 second delay to 100% of the connection.

A

So what you should have expected.

A

B

As you can see, the red parts are the message from review to prada page and you go on the product page. You can see that page is loading because we are having delays. Print delays on the reading micro service and review doesn't know how to handle that. That's why we turn the power for status. Go back to this other page and that's the end of the demo. We give it back to Animesh thanks.

A

Similarly, you can actually add annotations.

A

A

From that perspective, a lot these capabilities actually come from the framework itself right essentially circuit breaker as well in action, essentially create an annotation.

B

A

Radar inclusive, it is also another adaptation which is very useful in the mic to provide ears, which is all that. So, yes, things go wrong. Things will go wrong, but what do we do is something wrong instead of doing X, to be too moist white wall. That is an annotation which you can add that back in the fight. Okay, this particular thing is it's trying all its female. Let's do this, so what happens? It is flying missed you into this mixed. No, the application libraries themselves are giving you these hot tourist beaches.

A

What would you do right so I can play, gives you a way to actually disable all the animation based? Of course, even if you have animation the video for you can actually set up variable, and by that you will be able to disable everything else, and just you a takeover in this case, except of all evolved and something which is more weight into the application. What you want to do if something goes wrong, we might have question right. What, if both are enabled right?

A

So is a free trial tryouts and you missed us saying why try out something I also have to thank you. So this case is 15 right. There are policy that 10 second or metallic.

A

So again, the reason here is to me is to in these cases is yes, great, looks which I'll mention more, but just was to give it who company's application lending it 43 are calling modification. You want to be like all these poor for his features.

A

If you have services, they didn't buy all PHP that look very constant its it is to actually allows you to do it initially, so you can actually act what for this particular application without any cues the poor, so you can create simple things like time off speech which I publish it in circuit breakers in control, the tension, full-size and Express code.

A

So it's sampled. In this case you can create a policy to show there are X plus y connections. What we're saying is that only X maximum predictions are about why our technique anything beyond that should be important.

A

Essentially, if a particular port is not responding in theater policy to detect it out of the pool so that subsequent requests are not being sent to that particular.

A

Time aux and me tries within this word.

B

So you go through our IPM pattern, page to see like um these examples, it's using Java micro service. So what this examples is yes, fine, major Java micro-service is called speaker schedule, web applications sessions and vote and all these micro servers who communicate directly to the east, to ingress and for the vote micro service.

B

He also used the cloud and database image to store, is holding data and optimal running on a steel and here's the github page of this pattern, and what we're going to show right now is the Turkish breaker feature or an sto, and we will use it to simulate a load balancing pull attractions.

B

And I also have this example running on another cluster, so once you deploy, it is shape of the Syria page similar like this.

A

There you have speakers, you have their schedules, you can read their talks exactly right.

B

So you can vote anything and you can go to the web page and see all the voting data. This data is stored in cloud and database image.

B

So right now everything is working properly. But, however, if you look on the row, our new version of the cloud and image, if that's broken, what happens so right now, let me inject another cloud. Another version of the Cullen image.

B

As you can see, the crowd ntp second, is on. The version of the image, however, is spoken properly, so you would keep having five 300 status code back to this bloke micro-service. As you can see, it's not working properly. So it's creating error to this micro service.

B

So what I'm going to do is going to apply practiced on circuit breaker roof wait does is when it detects server error from any other parts, each other from the load balancing pool. So you will keep all the traffic only passing to the healthy parts.

B

So let me a prior is do control.

B

So right now you try to keep poking so the first time you see the boat is creating error. But after this error the circuit breaker rule is designed to eject that um broken image out does um low balanced input. So from now on, every vote you vote is gonna, be healthy. Returning to understand scope and that's the end of this arm, demo I will give it back to Animesh.

A

So these are positive, which was here in that particular case. So if you wanna try it out idols services while actually at that, but including the slides, are atmospheric looming or so to once and ended by saying that next good reactive Michael services services, which are responsive the ceiling elastic a systemic. Let's use this.