Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2016 (Seattle), 16 Nov 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk - Say what? You're Running the Storage Platform IN Kubernetes?

Description

Lightning Talk - Say what? You're Running the Storage Platform IN Kubernetes? - Stephen Watt, Red Hat

GlusterFS is an open source, scale out, distributed filesystem that is becoming popular as a shared storage solution for containers. This talk is about how the GlusterFS community containerized GlusterFS and enabled it to run directly in Kubernetes. This is fairly novel as actual storage platforms (as opposed to storage adapters or volume plugins) are a *very new* type of workload for Kubernetes. This approach allows a single Kubernetes cluster to run both the application containers and the containers for the storage platform that provides the volumes consumed by the applications. Besides a demo, this talk will also describe the tradeoffs between running the platform as ReplicaSets vs. PetSets and usage of the platform with the new GlusterFS Dynamic Provisioner features in Kubernetes v1.4.

About Steve Watt
Steve Watt is a Consulting Software Engineer at Red Hat and is a member of the Kubernetes Storage Special Interest Group (SIG). Steve leads Red Hat’s Platform (OpenShift and RHEL Atomic) and Storage (Gluster and Ceph) engineering initiatives for Container Storage.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

So hi folks, my name's Steve watts, I work for Red Hat, an office of technology on emerging technologies and over the last two years, I've been predominantly focused on containers and storage and so I'm also part of the kubernetes storage. Sig been working on that for about the whole two years, and so you know in the storage sig, we've done a lot of work on connecting kubernetes. You know using kubernetes with storage platforms, so in 1/4 we had a whole lot of updates around dynamic provisioning and storage classes.

A

That's actually not what this talk is about. This talk is about running storage platforms in kubernetes, so I've got the simple task of in five minutes explaining how you ran a complex stateful system inside kubernetes.

A

So the example that I'm walking you through today is Gloucester efface right, so, glass, uruk's fess, is kind of an easy thing to use it's open source. But if you like it, this commercial support available. It's scale out, you know to petabyte scale. There's you know, software-defined storage. So that's not an appliance. It's actually really easy to contain our eyes and ears, but, most importantly, I'm. Not manufacturing, storage out of thin air in kubernetes, it runs on commodity infrastructure, but those infrastructure. Those servers actually need local disk.

A

And what I'll do is it sort of federates? The local disk exposes a file API out of that and that's what the dynamic provision is consumes: storage out of it's mature. It's got a well-established community, it's very viable, but specifically like once. You've got the storage platform running inside kubernetes. It's got all the support for the volume plugins and then dynamic provisioners already inside kubernetes. So if you want to basically mount cluster volumes within your pods or provision storage outside a cluster, you can do that.

A

So this is the actual deployment topology right. So imagine you have like I, don't know if you're on premise right. The main reason you do this yeah say: I have 20 node 100, no cluster. You need at least three of those nodes to be with cubelet hosts to have a local disk available and formatted local disk, at least one that supports more than one. But you need at least one and on that will on one of those hosts will run a Gluster efest pod right.

A

Just a containerized blusterer fish running inside a pod and mounting /dev right. I have /dev SD B as an example, but it could be anything and then we also have a restful service interface pod. So this is what the the kubernetes dynamic provisioner talks to you. This exposes a service, and when you set, you submit a claim against the glossary of fare storage class- and you say: hey I want a hundred gigabytes of storage.

A

It will talk to this interface and this interface will manage the communication with the actual glossary of Fay's file system, to go and provision that volume and then provide the information back to create a persistent volume. So you can use it. So that's sort of what demonstrated here so you know, starts off with an administrator. Once this thing is deployed right, the administrator would create a storage class if you're not familiar with it. This is, as I mentioned, part of cube one. For so you can create an arbitrary storage floss.

A

You could Gluster a face, but in that thing you point out, like hey, I want to use the kubernetes cluster of face provisioner and then once that's done, the developer says: hey 100 gigabytes from cluster. If a storage cloths provision for storage and then you use it in your pod, so I'm now gonna give you a demo of this.

A

Can I get a bit more help piping? This thing out, I.

A

Need that to the screen all right, all right there we go.

A

A

The first thing I'm going to show you is that I'm gonna do Cube control, get nodes, you're gonna, see a master, I've got three nodes here and I'm gonna go off and I have a bunch of files that you get this with the project when you clone the project, what I'm gonna do is Dow deploy the blaster nodes.

A

If you remember on the deployment side, you saw that there was a pod on each one of these nodes, so I'm actually going to basically do a CID where I'm going to replace the actual node name inside the file and submit the deployment, and that will actually start the cluster. A face pod, like you'll, see now I'll go and change it from node 0 to node 1 and again no now we are going to automate this probably into a daemon set.

A

We haven't done it yet, but this should be nothing technically present preventing us from this. So now, once we've done that we're going to create a service account, so this is for the restful service interface to communicate securely to the pods. We get a secret out of that. So we're going to store that in an environment variable, so you can see that there.

A

Okay and then we get icky going to add that information to a file to actually deploy the restful service, so the API server and the secret name. So once that's deployed you can do a cube, control get pods and you can see all those three Gluster efest pods are running as well as her keti, which is the name of our restful service, and you can actually see that there's a service endpoint for hickety. Now this is kind of what's interesting.

A

Now, once we've got the restful service that actually doesn't know about the Gloucester pods or anything about its deployment topology, so we're. Actually, when I do a cluster a list, you can see it's empty right. So now, what we're going to do is we're going to pass in the deployment topology to the to the restful interface and tell it about the Gloucester pods, which ones it has and what block devices are available. So you can see we've passed that in and it's automatically adding each node and each block device.

A

So there's three on each node here and now it is aware of this, but the problem is that this information is all stored in memory on State, but now it has a distributed file system and time up, okay and and so just about over and and so it can persist that that information on the distributed file system and then you're able to provision volumes directly out of that. So now it's redeploying itself and it'll.

A

It's the the service comes back up with its database on a distributed file system instead of in memory, and we can then provision the volumes and you like 30 seconds and I'm done all right and so I think I'm out of time. So I've got to get off the stage, but that's it all right! Thanks folks, don't go check out this website of.