Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2017 (Berlin), 6 Apr 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: Prometheus: From Berlin to Bonanza - Brian Brazil, CEO, Robust Perception

Description

Keynote: Prometheus: From Berlin to Bonanza - Brian Brazil, CEO, Robust Perception

From its humble beginnings right here in Berlin in 2012, the Prometheus monitoring system has grown a substantial community with a comprehensive set of integrations. This talk will go over the core ideas behind Prometheus, give a brief tour of its end-to-end feature set and show how these combine with other CNCF projects to allow you to scale your systems and culture in a dynamic cloud native world.

About Brian Brazil
Brian Brazil is a core developer of Prometheus, and the founder of Robust Perception. He has developed and maintains components and features across the Prometheus ecosystem including the Python and Java clients, and many exporters. He wrote many of the best practices and guidelines for those looking to use Prometheus, and publishes regularly on the Reliable Insights blog.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

I'm Brian Brussels, obviously enough, and instead we're going to look at at Prometheus, which was came here from Berlin. So I am one of the four core developers of Prometheus, hardly the one who isn't from Berlin, so I'm from Dublin I've founded a company called robust perception, we're doing to support and consulting Klingon for mateys I've worked in many open-source projects and I did my time in the Google ad mines as well, and so to give the actual history of the medium started off by Matt Prowse.

A

Who was an executor at the time, he's no Google again and Julius, and they were working sounds out of time and in 2012 it started. Then in 2013 it was devolved. More was in sound sound, expanded support, bazooka, which is kind of like it was roughly Heroku till around and as well good developing further starting gets the go job and ruby clients going, but we're still talking something. It's just inside sounds out in Berlin in 2014, other companies started using it.

A

So myself, I was working in an Irish startup called box ever and Johannes fish in docker, as well with using it. The project matured a lot we got a new storage engine and much more efficient I got new text for mash should still have, and then in 2015 we publicly release because the project was always on github, but we just spreading by word of mouth and we're worried that someone else would post attack and use first. What actually happened is we posted hacker news like after getting like things like documentation in place, and nothing happens.

A

Two weeks later, some person we never heard of posted to hacker news. I woke up like half eight in the morning. Why are we number 16 on Hacker News? That's how it goes in forcefully and an adoption increased, quite lost and then in 2016 as well.

A

We joined the CN CF as we continued to grow quite a bit because the thing is: we've gone from a project was started in one company with two people to a project with over 300 contributors just in the core, and there are also I think we're up to about 150 third-party integrations we're not actually sure how many there are and there's at least 137, because that's how many port numbers around case, at least let's check this morning, just mailing this IRC everything is getting.

A

We estimate there's over 500 companies using for natesan productions, because no one company is behind Prometheus. We can't actually challenge the explicit products we estimate over 500 using it really in production might be as many as a thousand.

A

We also a sign of many many companies wanting to fund developing the prometheus you've got core OS you've got, we've works, you've got get lab, they don't code near us perception, SoundCloud and lots of other contributors, another company, so this is a nice healthy ecosystem showing that how, in the space of five years, you can build a really strong project used by many people.

A

So the question then of thought prometheus is itself because we all heard the open-source story before so. Prometheus is a metric slaughtering system, so looking at metrics, not logs so complimentary to things like flu and deer, the Alex attack so time series database, a query language.

A

We have client libraries, an ecosystem, and the thing is important that we have a cloud native approach to monitoring services, because you need a different focus for things in the cloud in terms of architecture, we can talk out to kubernetes, find out where all your pods are and scrape those then scrape things like see, advice or Cassandra.

A

My sequel, your batch jobs, pull that in stored, very efficiently locally runs rules and alerts; Center, that's the alert manager, page or duty, op C, whatever you're, using and view your graphs in the open source cabana in terms of philosophy. If you look at things 1020 years ago, things were all manually managed and then we started moving to things like chef and publish and now we're in a new era again with kubernetes and just being the paradigm shift.

A

But a lot of monitoring is still stuck in the old days like how many of you had someone said. We need to alert when a pod dies. Why would you want the last alert if the new ones created life moves on, or you actually want to care about? Is my users can't see their cat? Videos like that's are really important.

A

Alert load average is high, is not an important alert right because unless it's actually impacting users, why would you be calling and waking up a high-paid engineer in the middle of the night to say load average is high like yes, this number is higher than this number, but the cat videos are ok that so we kind of have to have a change in views. How we receive these things as a sort of thing you need to look at so in terms of getting data, and so creative supports main service discoveries.

A

One of the popular ones is for kubernetes, so we can discover everything kubernetes knows about and all your labels annotations. We can pull them in because for medias in kubernetes have similar data model to that regard. They both have arbitrary key value pairs. You can do with things and then you can map those and radius labels. So whatever system part of a paradigm you have decided for for your kubernetes enables your command annotations.

A

You can map that into what makes sense, not even just your organization but for your team, because it turns out, everyone has different opinions on how to name things and 3ds will automatically pick those up. Those changes, then, inside your applications we have multiple client.

A

Libraries like go, Java, Python and Ruby are the official ones is another ten, like all the main program languages are covered and if someone else's instruments or like we get that sort for free now, obviously not everyone is instrumenting directly with community slides libraries is unfortunate about, hopefully, I will improve over time and because they are not specific to Prometheus. You can get them to talk at the graphite or anything else as well. We're completely open, but it's also many exporters for all the standard stuff see advisor for all your container doctor rocket stats.

A

Are there my sequel, SNMP JMX for all your fun Java applications and, of course, minecraft and factorial yep? Yes, it's keys.

A

Now the factorial one was a bit of run, but the minecraft ones is actually a friend of mine in Dublin because he wanted to know maybe how many ticks per second we're going on his server because he really likes to put lots of mods in so this stuff has real-world fake road applications.

A

As well like as Prometheus is also well supported and well part of the cloud native ecosystem, so kubernetes, as I said, we can pull data from kubernetes for service discovery and kubernetes itself is also instrumented directly with our client libraries so that all works linker D has metrics in our format and for gr we see and flu indeed, during the integrations as well. Now new projects being added. Since you know, I wrote my slides well in advance two months ago means I now need to update a wish at cord.

A

Dns directly exposes previous metrics dr itself. Also instrumented with primitives I'm, not sure with us and rocket, doesn't have anything. Let's talk to them.

A

Well, then, so you have all this data coming from all these systems. What every cloud native put it'll be older stuff with DMX or SNMP data and the prom query language is ensuring complete language that lets. You do basically anything you want. If you don't believe me, go to Google and search towards Conway's life Prometheus and aside from the silly examples you if you want to try to cut whether a disk will fill in four hours rather than when they'll hit that five percent full.

A

Are you at two T Drive and then take two years to actually fill? You can do that if you want to get 95% latency across the entire data center, you can do that. Whatever you can represent mathematically, you can put in prompt ul, so you're not limited to live to alerting on things like load average, so you can actually alert on well. The cat videos are fine, but the dog videos aren't I know. There's a religious split, they're, an important thing as well is you can grasp on it? You can alert on us.

A

There is no distinction in for medias, so you can have better alerts that don't wake up your engineers in the middle of the night, so we can focus on solving real problems and improving open source further, rather than have to deal with sleep deposition. Here's one example- and this is a more complicated example.

A

What you normally have- and this is a staff from C advisor- called Canaan- your CPU usage seconds, total I, just pull all the the docker and stats for how much CPU each container is using, but then we're going to aggregate it up and so much off by image. So this isn't by data center. This isn't by region, this isn't by application or anything else normal. This is a cross-cutting look at all the docker images that are there and seeing how much CPU is being used and give me the last two talk five.

A

So this could tell you hey these docker images he wasn't using most CPU, maybe we should add some optimization flags and those to save a little bit of power. This is an example. Just a small example of analytics. You can do beyond just doing simple, alerting and graphing, because as well once you have your alerts, you want a principle that not every results in a page. If you can think if we go back to the older world, if you have a rack and the rack fails, you don't need force killers for every single machine.

A

You know it should be handy, one alert per rack or what would our data centers saying? Hey, here's, the racks. You need to go pokin things and you can find up sensible in the same way. So our Newark manager has grouping so alerts that are coming from the same data, centers same rack or whatever makes sense to you.

A

You can group them together to a single notification, choose which team they're going to and thrust with those notifications, so if it picks up only half the machines in one cycle and half and the other you're only going to get two notifications, or maybe even with just one, which is getting closer to that ideal of one actual page for incident. And if you want to send em to your taking system, you can do that as well.

A

There's also been designed to work in the real world because we know things like availability zones fail, especially Us East one, and you know you need to be reliable. So it's designed that dirt manager is using an AP and may working. So, even if you've never partition, both of those alert managers will be trying as hard as they can to send you alerts. So you might end up with twice the notifications on your pager. At least you notice something going wrong.

A

So, as I've shown you from radius, it's a metrics monitoring system, we don't care about individual events, we're looking at the system as a whole. It's a time series database storing all this data locally, so you can access it reliably, even when everything else is going apart. The query language get relating client libraries to let you get the data in an ecosystem of at least 150 different applications to do that data, and we view this all combines to allow you to monitor your cloud native applications in the same fashion.

A

Now, on top of this, this isn't just at Google and other projects. Today, there's also actually a polytheist track that is on today and tomorrow in a 0-6, just a few talks might be of interest. The first one is my own, which actually at 12 o'clock a party the schedule change the last two months and, if you're looking on the internal details of how all the different monitoring and instrumentation systems deal with the simple question of how many requests that I get in the last five minutes, so that yes, a 45 minutes, talk on counting.

A

So, if you're deep into monitoring this one's for you now, if the top talk about previous I would recommend here is Fabian talk, I'm, alerting in the cloud native environments looking further on what is the best way to have alerts that are actually actionable by an engineer and work. Waking up someone for how you can approach those using prompt scale informations. So that's one to look at and then David is talking about agravada and is because profile is not always enough to have do so.

A

How you integrate with our API is to do things beyond tomorrow, there's a few there. We go so Julius in the morning if you're ever wanted to know what long-term storage infirmities and how we make them talk to each other. This is a long requested feature and in fact said when the companies we works has started to develop a cloud service based on this and he's talking about how that's being built up and how it's been used, because it's the long, long requested, Alexandro and so not everyone is tailed native.

A

Unfortunately, some people still have to do things like switches and NetApp's, and you know other hardware like that. So he's going to talk about SNMP and logs and has the integration with hardware like that, isn't kubernetes, because how would you run a switch on kubernetes, wouldn't really work and Bjorn is one of the other core developers as well he's going to talk about configuring communities for high performance, so I've actually got some nice changes in the next release as well.

A

1/6 for our memory usage is going to go down by I, think 20, 30 % reason for that and by tuning some parameters. So please go and we always gonna give a nice big, deep dive as well and finally, not in a 0 6 and c04, with registration acquired, Alexandra and Frederick from core OS have a workshop that two-hour workshop. That will take you from instrumenting your applications to using the core OS committees operator, to seem that data inside for radius and refine and and all together on kubernetes.

A

So if you want to actually try out Prometheus, this is the workshop to go to. Thank you very much. Oh sorry, also thanks.