Ceph Ceph Days NYC 2023, 8 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Ceph Days NYC 2023: An Introduction to MicroCeph

Description

Presented by: Chris MacNaughton | Canonical

Building up a Ceph cluster can be a bit tricky and time consuming, especially if it’s just for testing or a small home lab. To make this much easier, we’ve started working on microceph. It's a snap package that uses a small management daemon that allows for very easy clustering of multiple systems which, combined with an easy bootstrap process, allows for setting up a Ceph cluster in just a few minutes!

A

I'm Chris uh I'll be talking about Microsoft. So what is it? um It is a snap and I know. Some people are a bit leery about snatch from canonical um it's. A clustering demon uh uses a DQ light, which is a sequel light, compatible.

A

That works too. uh It's a sqlite compatible database using graph for consensus. uh The management Daemon allows us to build a cluster of arbitrary nodes. We then use it to manage the placement of all of the various F daemons. You guys are all aware of monitors managers osds the metadata servers, almost sorry. What was that almost the rados gateways? It's in progress. Now it's really opinionated about what you put where that's likely to change soon, but for now it does what it does and you get Seth.

A

um So why did we do that? A couple of different use cases we really wanted to support a single node introduce F. You can run this on your laptop. It runs as a strictly confined snap, which means it doesn't have full root on your system. It has very very limited access to things you have to give it permissions to do things, it's kind of like having your phone running Seth instead of the server. In addition to that, we really wanted to be able to run small Edge clusters deploying thousands of them.

A

Instead of you know thousands of very small clusters rather than a few big ones. You know we all like talking about petabyte scale clusters, but we kind of like some small terabyte scale clusters too. uh We wanted it to be quick to deploy very predictable, and you know the idea being that average deployer isn't actually a SEF expert, uh you'll think the 7-Eleven, you know someone who's, the assistant manager can actually go in, take their little tiny rack in the back of a few nooks and run stuff on it.

A

This lets them do whatever they want to do with Seth, instead of having to hope that they can find swords somewhere. uh There's almost no infra overhead that clustering demon is very minimal. It's a little go demon apparently we're all fans of go now so yay um and it really needed to be repeatable.

A

You can you know script it fairly easily get all of this. uh We're actually gonna have a demo. Hopefully now guys in the back are going to hook me up as we're gonna actually do a self-deploy in about three and a half four minutes.

A

A

Yay sorry I can't actually Zoom it in, but hopefully it's all legible, all I've done to prepare the system is to install the snap. It's that Microsoft One, the fourth one down we boots drop a cluster. This is setting up that clustering demon, uh getting it ready with DQ light and things pretty quick doesn't actually do much so cheating uh we tell it to add some nodes. I've actually got four machines here to play with. We get a little token.

A

That token has all the data. You need to add these nodes into this cluster and we're going to hopefully go over to the next Tab and add a couple more of them. These will be my first three nodes running manager, monitors and the MDS servers actually.

A

It's a little bit slower as it's actually sinking, State doing things with raft and fun. Clustery Magic.

A

We can do them concurrently, there's nothing that stops us from doing that. At the end of this, we actually will have a ceph cluster with no storage, which is you know not that interesting, but it's a nice first step, foreign.

A

I do have this one set up too the red Skateway I mentioned it's a progress. There is actually code in place to do it, we're working on all being happy with it. You know deciding that we like what it's doing and how it's doing it. Okay, so we can get our status. You might notice that microsoft.sest status, it is confined you're, not running this as you on the system, you're not running it as a ceph user or root, but inside that confined space, you have your monitor, managers Etc, but no disks.

A

So, let's add some disks.

A

I did run into a slightly annoying bug that made me not want to do it concurrently in this demo. Sorry it takes it a little bit longer, but we're gonna get our one OSD showing up shortly. You might notice that the adding the disk doesn't actually add into ceph in a blocking fashion, which is citing.

A

Some of the the clustering work is actually around um letting us manage demons and everything in a central fashion. So you could say Okay. Add that OSD from that machine over there uh put a monitor over there put a Skateway over there foreign. So we actually should have shortly three osds up and in.

A

Let's go and enable anybody else, Gateway leave us up next.

A

So this will be fun. The next demo is actually playing with red house Gateway, just a little bit in a slightly smaller version, but the same concept so you'll have you'll. Take this and be able to see for the other one, a nice handing off point.

A

So hey we have that all right, I think we're on the next demo, which is actually of that red house Gateway that we just set up again pretty minimal setup. It's pretty quick and easy.

A

All right so similar setup, it's a slightly smaller one, but it still, you know, suffices to let us play with the redness Gateway.

A

I did accelerate this one. A little I didn't feel like taking five minutes to demo a little bit of S3 playing with. um So again, we have you know no data in there in the pools we set up riddles, Gateway, uh there's actually config on here to do um The, Nest three commands.

A

You can see that there's a red house Gateway running there on this node, it's not in a container or any of that it's bound by Linux namespacing, though nonetheless,.

A

Okay so we've uh not got a user, we're gonna, add a user I believe that's what we're doing next and then creating a bucket and writing and reading an object.

A

Slightly safer, though, live demo, but unfortunately means a bit of waiting, occasionally.

A

B

We've created our user.

A

And the credentials should actually be set up in the S3 command. Now that does that part.

A

All right, so we've uh listed things added a bucket: oh hey, the bucket, the goose, now yay looks right to it and then read from it to make sure it actually works.

A

All right, okay! So, what's next uh before we're happy to call this anything like something you should play with you can today, please do uh we want automated encryption uh so very easy, like seamless to a user encryption of the osds, uh you need to actually build a configure your network. Instead of we tell you what to do and we're going to be doing stuff upgrades, so Point releases and major upgrades in a very automated easy to use fashion that does just work. It doesn't eat your data.

A

uh It is very much a new thing. We've been working on it now for about six months. Please try it out, tell us what you think tell us. What doesn't work tell us what works kind of all layers on feedback.

A

uh How can you try it out microsoft.com? It is on GitHub as well canonical Microsoft.

A

C

C

What's it using.

A

So the the demon that it's running is a little management demon that so the synapse ships with all of this F bits, it's the Ubuntu packages, Force F, uh when you say you know the first three that start: okay, I, don't have enough monitors start a monitor and it just runs that locally. It knows what the whole cluster state is. It knows what disks are added to What machines, ecstasf, which I'm a little annoyed about that design choice. But we'll come back to that.

C

Yeah, that's really cool. Actually I wish I had this four days ago that I wouldn't have had to use this cluster I was like, if only there was some packaged Microsoft thing. I could use I thought about using the um what's the in this F source code. This is one of the make things it's like quick start or something, but that's it yeah yeah, but that's a nightmare. Yeah.

A

C

A

A

Just a fun tidbit, it doesn't use ldm, it actually even uses the direct block device. It's a little easier to constrain that properly we're doing lvm on a hostess difficult to correctly constrain so kind of fun. One. There.

B

Have you ever tried to create Microsoft uh deployments and then have them talk to each other?

B

A

Was just curious that said: we've done a lot of things elsewhere and you know our playing with Seth with uh mirroring gateways mirroring RBD volumes, so it.

B

Really wouldn't surprise.

A

Me, if that's on the menu fairly quickly sure.

B

Yeah, um so in CBT it will will like deploy stuff into the temp and we can do like you know. Big cluster deployments, like uh you know, kind of like a real deployment, but not really uh but I'm super curious with something like this. If you could kind of uh do a single machine deployment and then extend it, it'd be be cool to see.

A

So one of the things that we're looking at is really well supporting that single machine use case and then barely seamlessly scaling up. So you know, you're, playing with on your laptop you've, got some stuff fundamentally, the same experience as I'm on a server in a data center doing the whole racket once so. It really shouldn't be A different experience and you should be able to go from one to the other, even though you probably shouldn't. Let's go from your laptop to a data center.

B

A

All right, thank you. Everyone.