Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2018 (Seattle), 15 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: PID Autoscaling Strava's Linkerd Service Mesh Using Prometheus Data - J Evans

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Lightning Talk: PID Autoscaling Strava's Linkerd Service Mesh Using Prometheus Data - J Evans, Strava

Strava’s traffic fluctuates alongside our athletes’ weekly exercise habits. Our service infrastructure must dynamically scale to meet constantly changing demands efficiently and cost effectively. This is becoming increasingly important as we augment our ensemble of microservices to provide our community with a richer feature set. In this presentation, J Evans will describe how Strava leverages Linkerd, and its seamless integrations into open source monitoring tools, to maintain observability and enforce consistency throughout their service mesh. Specifically, he will demonstrate how Strava uses Prometheus to aggregate metrics at the control plane (Linkerd) and application layer to fine tune a proportional-integral-derivative (PID) auto scaling system capable of maintaining high resource utilization across the Strava platform.

To Learn More: https://sched.co/GrVb

A

Well, hello, thank you for staying this late. Has anybody here ever used? Rava, alright, cool lot, sigh hands me too, so I'm gonna be talking about pit auto-scaling and how we use it so, first off what is a pit controller from control theory? A controller is any algorithm designed to maintain desired state of an error prone system. There's a couple key components: the setpoint would be the desired state you want to maintain the process variable would be the actual observed state. The error would be the difference between the two.

A

So if you want to maintain eighty percent CP utilization and you're somewhere around that, but not exactly and then the control variable would be what you can change in the system to minimize the error and get where you want to be and what's special about paid controllers are the three components of air that are being used. So I'll kind of explain why the all three using all three is important.

A

So here we have a stable system and then I introduced some error by changing the setpoint and if you're using proportional, only control, then the error response will only be proportional to the size of a error and in such cases you may have overcompensation oscillation and it may not even converge where you want it to so, to at least account for the convergence problem. We can add an integral component, and that looks at the accumulation of error over time and will eventually take care of any steady-state error in an arid system.

A

But we still want to take care of this oscillation as well. So for that we add the derivative component that looks at the rate of error change. So at the beginning of this curve you see a quick decrease in error, that's associated with a very negative derivative component, which would act to counterbalance or dampen the proportional amount which gets us to our setpoint relatively quickly. So then, how do we use pit auto-scaling at Strava?

A

So we deploy a lot of our infrastructure on meso s--, and this would just be an application container running as an ace. Those tasks and the shape of those deployments will be orchestrated with marathon, so here's just one sample deployment and it be driven by a paid controller service, and in this example, let's propose that we have a set point of latency of 500 milliseconds. So we expect the request to be serviced in 500 milliseconds. That gives us our set point now.

A

We need the process variable, so we can observe what's happening and for that we would use linker D. It's a great piece of software that has a lot of benefits, including automatic load, balancing across all of its replicas to automatic retries circuit breaking, but I want to focus on the Prometheus integration, so a linker D can automatically export metrics, Prometheus style, metrics based on the services it's proxying, and we use that, along with some application level, metrics to get a good sense of what our topologies are doing in production.

A

So that gives us everything we need to have a feedback loop for our pit controller, so traffic will come in, say we have increased traffic or spiked. Then the observed latency may spike, as well as seen by linker D, and that would result in error in the pit controller service, which would try to correct that error with an auto scaling event say: changing the control variable, the number of instances from 3 to 5. So now we are scaling against these 5 replicas, as opposed to the 3 before and theoretically.

A

That should bring latency back down where we want it, and this is kind of a holistic picture, an entire mesas cluster. We may run so there's many micro services, all proxied through linker D and all of those proxies and all the services are sending metrics data to Prometheus, which in turn can be used by the pit controller to make auto-scaling decisions and make sure we're not under provisioned, over-provisioned, etc and I got through about four minutes without even saying the word kubernetes. So I'm going to fix that now.

A

You can also do something similar with kubernetes a couple. Key differences would be. It wouldn't necessarily be a marathon deployment, but you'd have a kubernetes deployment object and then here I've kind of Illustrated running in a host-based linker D model. You may opt to do a sidecar pattern, so the application container and the sidecar are running in the same pod. And finally, you can't use our super awesome, Pig controller.

A

Just yet you would use the kubernetes horizontal pod autoscaler, which uses a different algorithm and a prometheus implementation of the custom metric server and there's a link down there. That goes into more details, and that's about all say on this I will say there is a talk by some an engineer at buoyant I think his name is Thomas that we'll be talking about essentially running scaling and kubernetes, so you can get more details there. Thank you.