Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2016 (Seattle), 16 Nov 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: Distributed Systems Simplified Using Kubernetes by Brandon Philips, CTO, CoreOS

Description

Keynote: Distributed Systems Simplified Using Kubernetes - Brandon Philips, CTO, CoreOS

Understand how to better control clusters with etcd, a cornerstone of a variety of systems and the Kubernetes’s default storage backend. It simplifies Kubernetes cluster availability by handling reliable state management. See how etcd stores and replicates data across the entire cluster, and can recover from hardware failure and network partitions.

About Brandon Philips
Brandon Philips is helping to build modern Linux server infrastructure at CoreOS as CTO. Prior to CoreOS, he worked at Rackspace hacking on cloud monitoring and was a Linux kernel developer at SUSE. As a graduate of Oregon State's Open Source Lab he is passionate about open source technologies.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

How's everyone doing closing event: I can tell the the audience is just doing really high-energy today, closing out I, don't know what it is. There's something in the air all right, so I'm going to do today is I'm going to talk about how to simplify, running and operating distributed systems on top of kubernetes really the goal is I.

A

Have five minutes and I want to just talk about ways that core OS is thinking about how to help people wanting to run kubernetes inside of their environments and then running applications on top use, some really common design patterns to make that easier to do so, we know already today that scaling stateless applications on top of kubernetes is really straightforward. I think that's why a lot of us got excited that first moment where you run a container, and you tell the cluster all right. I've been currently running.

A

One copy of my container I want to scale up to three and then magically everything just works out. It ends up in your load. Balancers, it's awesome, but scaling complex applications is actually quite a bit trickier. So today, kubernetes has a lot of the fundamental foundation in order to run applications like databases quite well. We have the idea of persistent volumes. We have the idea of a stateful set previously known as a pet set, so you can take in container that needs to persist the disk and that's not a problem.

A

So it's pretty easy to do that, but where we start to get into more complicated, things is where we want to think about running, distributed systems, perhaps distributed databases or distributed monitoring systems or distributed caches, and this is where you start to have to think about more complex parts of the problem than just getting some bytes to disk. Like classic database, you have resize and upgrade you have reconfiguration. You have backup, and you have healing that you need to think about in order to make this actually happen.

A

So the basic idea here is that we want to extend kubernetes in order to declare a more complex application and we can model which is saying you know what I want to just add more capacity by adding more containers running. So our hope would be that let's say that I want to run a database that has read replicas and it's clustered I want to say all right. Currently I'm running I want to run three whatever's of my database and I want to have a couple of reed.

A

Replicas and I want to run this version, and so a few days ago we introduced this concept of operators, and one of the operators that we introduced was for Etsy d and really an operator is just trying to encode a bunch of human, operational knowledge and software to reliably manage an application like a distributed system like Etsy D, so Etsy D, if you're not familiar, is the primary data store of kubernetes.

A

It is introduced by Korolev since 2013 and it's distributed because we're trying to ensure that the kubernetes cluster that you're running stays up even in the face of individual machine failure, and so with the Ltd operator. You're able to do just this deploy a distributed system. This distributed database by simply saying I want three databases in the cluster and I want to have this version of the software and what it does is it goes through this constant cycle of observe analyze and act.

A

It looks at what the current state is figures out, what the current state is versus, what you asked it to do, and then talks to the Etsy da pis and the kubernetes api s and in order to make the state true. So what I have here is a is a mini coop cluster I have a net CV cluster running on top, just as described in the slides there's a cluster. That is three instances, so you can see.

A

Size is three and we can see that that's true if I go and find all the pods that are labeled in my cluster as having a 2d cluster as demo cluster and then what I'll do is I'll just destroy one of the instances of Etsy D inside of the cluster. This is a really bad destroy because it means I lost state I like lost the disk. This was running on and immediately after I destroyed it.

A

You can see that demo cluster 12 went away, but demo cluster 13 came out because the operator worked with kubernetes and worked with that CD in order to get the cluster into the correct state. Does a lot of other things like rolling upgrades, backups, etc. But that's the fundamental thing it's coordinating between kubernetes and the distributed database. It's all open source, it's a github under Etsy the operator and our end goal here is to make it really easy to make a chase setup. So kubernetes possibility, particularly with self-hosted setups, like I, just demonstrated yesterday.

A

Now the other cool thing is prometheus operators, which Fabien talked about a little bit and it makes it really trivial for me to monitor this distributed database.

A

So what I've done is I have a service monitor for this STD cluster, again I'm looking for things labeled demo, cluster I come in I have a Prometheus instance that is searching for all the things: labeled demo cluster and actually launching a Prometheus instance, and if we come in here and oops and if we refresh the metrics, which I just press the wrong button, if we refresh the metrics, what we should see is right here you can notice all the networking went out when I deleted that pod.

A

So all the monitoring and the operators behaving as designed next steps, go, read the blog, try out this new code and build and discuss with remaining at the conference and the event tonight. What you'd like to see next out of operators, Thanks.