Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 14 Nov 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Istio Tells me my Service has Slow Response Time, Now What? - Endre Sara & Enlin Xu

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).

Istio Tells me my Service has Slow Response Time, Now What? - Endre Sara & Enlin Xu, Turbonomic (Any Skill Level)

With telemetry service provided by Istio, containerized application performance monitoring has never been easier. But what do you do with slow response time when you have application deploying at scale? Slow response time is just a symptom, the root causes might be incorrect CPU/Mem limit configuration, incorrect number of replicas, underlying infrastructure congestion, or, a combination of the above. The corresponding way to solve it could be vertical scale the container, horizontal scale the pod, or reschedule the pod to a different node. In this presentation, we are going to share our lesson and knowledge about managing containerized application performance by leveraging the telemetry information from Istio to assure the application SLA

About Endre
Endre is the VP of Advanced Engineering at Turbonomic, directing a team of developers focusing on new technologies and opportunities to extend Turbonomic's existing capabilities. Previously, Endre was a VP at Goldman Sachs, leading the Systems and Application management team and Network Management team driving the management strategy, design and implementation for Goldman Sachs globally. Endre holds an M.E. in Electrical Engineering from the Technical University of Budapest and a Ph.D. in Electrical Engineering from Stevens Institute of Technology.

About Enlin
Enlin Xu is a proud graduate of Columbia University and has been a software engineer in Turbonomic since 2011. He is now a Senior Engineering Manager that leads the engineering effort for Cloud Native technology design and integration in Turbonomic. Before coming to US, Enlin graduated from Hong Kong University of Science and Technology, obtaining a B.Eng in Electric and Electronic Engineering. During his years at Turbonomic, Enlin has been inventor of three granted patents in resource management space
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Hello: everyone thanks for coming. It's a Friday afternoon and I'm sure that there's a lot of other options between the air and more writing the Tivoli so appreciate you showing off I, want to talk about this too, and specifically, some problems that we are trying to highlight and actually suggest some solutions. For this. My name is anda shiro.

A

I am the vp of advanced engineering at to economic I've been with uber onyx, since we started the company, but now for about nine years now and excited to talk a bit about the company and then jump into sto, just a warning that you know I'm actually, gonna do a live demo and apparently you have to sacrifice a goat to have a successful demo and the problem with Friday is that you have to sacrifice a goat every day of the week.

A

So I'm like up to my fifth God now, I hope that I'm always going to work well see but anyway, so a bit about who is economic. So the the goal of our software from day one was to assure the performance of the applications, providing virtual automation by understanding how to keep the whole stack healthy, how to make sure that we assure the performance of the applications by understanding what they're?

A

How is it policy perform, how it consumes resources, how those resources are available from a multi cloud infrastructure, and how do we drive orchestration to continuously scale and allocate resources to meet this demand, both in a kubernetes cluster but across the full stack, both in terms of virtualization public cloud, as well as the underlying fabric, wherever it runs? We've been doing this since 2009 actually launched the product in 2010, not very still at that time, but obviously it's a bunch of things. So talk a bit about sto.

A

This is probably not new, just just a show of hands how many of you have deployed this do in production, so so, as obviously, people have been managing traffic routing trafficking in their environment across various applications for a long time with micro services, with a larger number of components and a more elastic, agile environment. This is becoming increasingly complex, so one of the the key benefits is very sort of image to be able to take control.

A

All of this is to primarily be able to manage policies across multiple proxies across multiple application components, enforce these policies and also provide visibility and an enhanced resiliency into a complex mesh of services. The way both users communicate to an ingress, as well as application, communic components communicate with each other.

A

Typically, the pattern is that you deploy a a smart proxy alongside with your services, and you have a common control plane that is responsible for distributing the configuration, as well as collecting instrumentation, also providing security for each of the services, and, with this comes fairly broad telemetry as the actual proxies around the traffic, they feed data through the mixer board as the initiate the coop as well as as they complete the course. So you got really detailed information about how each transaction has performed, how how many transactions you're performing. What is the response?

A

Time of each of these transactions? Really good visibility, a lot of traceability into it. So at this point you've got, you know everything about all of your your application components, how they communicate, how many transactions they perform. You know you can tell if any one of them is slow, but the question is: what do you do with it now? I've got thousands of nodes, tens of thousands of micro services coming up and a community each other.

A

What do I do so? This is this is where we start. There are a number of questions that has to be answered that are not answered by a service mesh. Well, you've got really good control over how your traffic is being managed and you've got all this instrumentation and you've got the ability to change the weight of traffic routes. You can scale the applications, but it doesn't solve. It doesn't actually assure the performance of the application.

A

It doesn't answer the question of how many instances do I need to be able to satisfy the application demand, not just at 10 o'clock, but actually every minute of the day. Every second of the day, if you will do I, need to scale my application horizontally or vertically, do I need more nodes in my cluster. Do I need to scale them upward, then how many containers do I place at this particular point in time, in a node and and how do I manage the underlying infrastructure?

A

How close these micro services these pods should be to each other in the context of their communication, as well as all the resources they need to consume, so a number of really exciting challenges and fairly complicated at scale, especially if you want to answer this question continuously and actually do something about it, especially just to illustrate the the the scale some predictions that east-west traffic is going to be a significant majority of runs in your data center over the next couple of years, and of course this depends on your specific business on your specific application, but in general, with the evolution of micro services, the obviously the traffic patterns are significant changing and it's it's it's important to be able to understand and manage this communication between the components.

A

For example, you can have applications that communicate with each other, but they might be co-located. They might be in different available disowns. They might be in different regions, they could be spread cross, a pram or a public cloud provider, and and and these traffic patterns can continuously change. So, while you could try to address this problem by static rules, labels forcing these spots to run on the same node in a large, dynamic environment, this becomes humanly impossible.

A

So what we're proposing is I need for a dynamic affinity that is driven by software and it is continuously able to manage application performance in the context of how traffic flows, as well as how applications are performing and, of course, how they consume all of the other resources. So it's nice that I can put these pods close to each other, but there are a number of other really important factors that you also have to consider.

A

So what? If I have a noisy, neighbor and and I have say two pods on the same node that are impacting each other? Obviously, I want to prevent this problem. But how do I do this in a way that I don't sacrifice my application response time is this: if this particular micro service is communicating with someone else and I don't get the same performance on a different node or similarly, what if a node is overloaded I, don't want to store my other containers. I can fairly easily move this container to a different node.

A

But how do you find the right trade-offs between the the compute storage I/o resources that they consume and the way they communicate with each other and there's a number of other interesting examples in terms of resource fragmentation, where you can have long waiting, pods that cannot get scheduled? You can easily solve this problem by moving existing pods over so that you can place additional pods just illustrating a couple of examples where you can solve individually each one of these problems, but to really assure the performance of the applications.

A

You need to consider all of these things together, at the same time, for every single component in your infrastructure to truly be able to make sure that you're providing level of service that you intended to provide. So what do we have?

A

Kubernetes provides a really cool I'll to healing capabilities. I can say that you know I want X number of instances, replicas of this pod running environment as well always make sure they have number of replicas that I specified, but that in itself is not going to assure the performance of the application. This is actually a screenshot from one of our test applications. We are running the web application measuring the response time and graphing it in graph owner and obviously the fact that I'm Auto healing the environment doesn't assure the performance. This is a over.

A

The second response time, which is in this case, is not acceptable for a particular web service.

A

You can auto scale your application. You can make sure that you're running X number of containers, but that in itself, especially if it's based on a single metric like CPU threshold, will also not assure that your performance, your response time, is what your SLA requires, and it is also reactive. You know there are a number of things that you have to consider to solve this problem and and and in general, how do I continuously scale the environment? How do I manage how these things coexist, especially in a shared environment?

A

We've got a number of large enterprise customers who are providing a shared multi cloud infrastructure, and there are many application developer teams that are deploying into the same set of clusters. The question often comes up of do I provision for the first case, I reserve- everybody, in which case it's very expensive, do I provision for the average, as in for a best-effort service.

A

Typically, at the end of the day, I feel urate out of error messages and, in general, how the scare, the underlying infrastructure and trying to manage this manually application of a application application developer by application developer is really painful. It's almost impossible to guess what is the appropriate memory request or limit that you need to specify without knowing what the actual incoming transaction rate is going to be at any point in time, and especially if the application scales horizontally at the same time?

A

So really, our point is that you have to continuously resolve this trade-off, make decisions on where to place application, where applications can truly run anywhere on any cloud how to scale these applications with the understanding of the complete stack and and what I mean by full stack is that it's really exciting to be able to easily deploy applications into a kubernetes cluster to be able to route traffic through a service mesh.

A

But at the end of the abus, runs on some infrastructure, be that public or private cloud and and and you cannot manage that, independently, with the understanding without the understanding of the underlying infrastructure. But there's no single human who can look at both the underlying storage fabric or the choice of the storage tiers in a public cloud. The network fabric, you know for all technique cards, while you're also trying to enforce policies at the service mesh layer. So really our proposition is that you need software to be able to solve this problem.

A

You need to be able to understand the complete stack, make decisions to solve all the previous trade off that I mentioned continuously in real time at scale, without relying on humans to operate the environment at the second by second or minute by minute level.

A

So I want to illustrate this with a couple of demos, and hopefully we have some time for questions as well, so we are running actually several kubernetes clusters. Mostly one point, nine one point: nine, two: seven zero zero point, seven point one with some Prometheus and refiner and we are actually leveraging to economic to manage this complete stack of applications and infrastructure and effectively preventing application performance degradations.

A

So if I switch over.

A

Okay, so this is actually the user interface of two anomic just to illustrate a pass simple way of solving this problem.

A

Turbo is actually able to tap into several public API instrumentation points, you're leveraging the instrumentation exposed by steel, but actually currently getting that through Prometheus we're managing kubernetes through the kubernetes api is either running on a juror or running on a on-prem, VM, very environment.

A

But we've got a fairly large selection of of combination of layers, of stack types of applications in infrastructure that you could run on, be that sto or or Cisco tetration we're at any hypervisor, as well as any cloud between typically most of our customers running on either AWS or nature, or some on Prem cloud and and by simply interacting with the public api's of these different components. We are building a single common abstraction, a single common representation of the complete stack stitching it together continuously in real time understanding.

A

What are the applications that are running an environment for services they expose to end-users or to each other? What containers they running on in what ports they are being all the way down to which, in this case, they're running a non-prime virtual environment?

A

What virtual machine they're running go on what host, what data stores, they're, leveraging and really the goal of our software is to resolve these trade-offs continuously by driving a set of actions that are both deciding on how to scale horizontally vertically applications how to reallocate resources such as reschedule a pod from one node to another and, of course, how to scale the underlying node itself to drill into one of these examples and kind of nailing it down to sto.

A

So this is actually a fairly simple application that we deployed. You know a client and a server application. This happens to be the client part, it's actually running in in a a port with an sto proxy so that we have the ability to both instrument and to control the traffic.

A

This particular port is running on a kubernetes node, which happens to be hosted on one of our HP servers and really the what the instrumentation allows us to do is to to look at how much network traffic is is generated by this application.

A

Okay, I guess it wasn't enough goods anyway, so so the point is to be able to decide where these pods should be running and.

A

If I go over here.

A

It is to be able to take any one of these pods and and place them closer to to their bodies. Now I am demonstrating the actual ability to control this in you know manually, but really our system is designed to automate these actions and continuously control the environment. What most of our customers are doing is they are basically letting our software to automatically control the environment.

A

So all these decisions that our software is making based on the real-time changing demand are actually executed by our software, closing the loop in a Cobra's cluster on a public health or non premium vironment by actually taking these actions and, of course may be changing them as the workload demand, changes and and and to dos there's a couple of other use cases. It's of course, not enough just to manage the applications in terms of their placement, but also in terms of their scaling.

A

So what we have here is another application that we are generating relatively high amount of traffic and both through our user interface, we can see the transaction rate and the response time of the application.

A

This is also actually coming through sto and Prometheus, which we can also see through graph owner and and what our software is identifying as an action is to basically provision an additional instance of this container, which then, in turn will scale the container out and what's really important, is that, while I was showing how bad the current response time is, that is not the only factor in deciding how to scale the applications. We are looking at the combination of the available resources in the cluster.

A

How far can I scale this application and in general, how many instances do I need to be able to assure the performance so so understanding the complete stack informs the decisions that our software mix and and closes the loop much more comprehensively than if you simply took a single metric or some calculations on a combination of metrics and and scale the pod, either manually or through some methods.

A

So that's most of the demo. Actually so I, don't if there's any questions or comments, anybody might have and I think there should be a microphone somewhere in the middle.

A

Okay, in that case, hang around for a bit longer and thank you for coming.