Red Hat OpenShift Data Services Office Hours, 26 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Data Services Office Hour: What's New with ODF 4.8

Description

Join data experts Chris Blum, Michelle DiPalma and data novice Chris Short every other week for a hands-on Office Hour about Red Hat OpenShift Data Science. Be ready with your questions and to learn a few things along the way.

What's New with ODF 4.8 - select features from https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.8/html-single/4.8_release_notes/index#New-features

Twitch: https://red.ht/twitch

#RedHat #Data #Storage

A

A

A

A

A

A

Good morning good afternoon good evening and welcome to another special edition of the data services office hour, I am chris short host with the most of red hat live streaming and I'm joined by the one and only michelle de palma this morning, michelle. How are things up there in canada today.

B

Fabulous fabulous.

A

B

Just fabulous the weather is beautiful, everything's great yeah, we're gonna, have a good show, so I'm excited I got stuff to show you.

A

Awesome, I'm excited to see this because, as we all know uh of the idiots in the room, I am the one. That's a storage idiot. um So yes, uh I'm curious to see what's new in odf48 and you are going to show us is what I hear.

B

Yes, so here I'm going to share my screen.

B

All right, can you see everything yep.

A

I see everything everything: okay,.

B

So we start I'll start with access to redhawk.com. Just so people see how I get to where I'm going. I'm not even logged in actually at this point, um so let's go look at the release, notes always a good place to start for what's new and I'm not going to go over all of them, because I.

A

Kind of realized.

B

And people can read people you can read.

A

People can read yeah.

B

Right, okay, so um 4.8 release notes. I just wanted to point out a few things: I'm just going to new features. So um if you go through this document and you look at what's, bolded you'll just get the gist of what it is. That's new I want to today. I want to discuss briefly compact uh deployment, general availability, I'll, show you how it's done. Yeah.

A

I know contact's.

B

Really cool actually and.

A

B

Easy, it won't take very long at all just to demonstrate like to show you what I did and then the.

A

One, it would be definitely easy: okay,.

B

A

B

Right, like um the one I want to spend time on, is the caching policy for object buckets. So if you don't mind, I would I'm gonna spend the bulk of the show talking about and showing you um refreshing mcg ideas, multi-cloud gate, object, gateway ideas and then showing specifically how the caching policy gets used, because I actually think it's kind of cool.

A

B

This is just fun stuff, so.

A

I've had a fun experience the past few weeks with just web caching in general, and it like, has really kind of opened my world to like. There is a lot of edge out there right like trying to make the experience the same for somebody in south africa as it is for somebody in san francisco is challenging.

B

A

B

So these caching.

A

Policies are here to help you get data close to the edge. I feel like right and.

B

They require a little thought actually.

A

Yeah, like some planning and you know like yeah,.

B

Yeah classes of.

A

B

And I'll show you the documentation, I'll go through it and and it's good because it's it's completely accurate, but I thought I could complement it by there's having sort of like okay, if you, if you take the ten thousand foot view. This is why this is set up this way, and this is how you think about it, and then you can customize it for your deployment. So I think I did this last time. The volume encryption volume.

A

Encryption part yeah. You showed that off so.

B

I should so there's so there's more as we go down and we can talk about, maybe, if someone's interested in some of the um I think there's a replica ii there's some other stuff going on um if we want to do that in a future show. But I think that these two compact and caching are going to take up our time today.

A

Okay, okay, so let me just say real, quick folks for everybody out there watching this isn't office hour. So if you do have storage questions, post, odf or ocs related storage questions feel free to fire away, but we come with a topic prepared. This is what we're talking about now. That does not mean we talk about this the whole time, if you have a question and you need help with something feel free to ask.

B

I will do my best to answer it. There's a lot to um uh there's a lot to odf and a lot of different ways to configure it. So I'm by no means the expert in all of them, but usually.

A

We can find the answer. Yeah.

B

A

B

Oh I'm in the room, so let me show you something I wanted to show so, starting with compact. um I have this is when you go to install your configuration. There's. It's actually right here in the document. Wait a second! It's a configuring! Three node cluster you're told exactly what it is. You need to do um pretty much you're modifying your install dash config file, your yaml file, to do this. So do you!

B

I don't know how you um I always install on aws, because it's it's just what I have access to right now and I will have a nice massive. I have my golden basic install, config yaml and in here I will change stuff up according to what I'm doing so. This is my standard uh odf bait. If I know I'm going to deploy odf because I have to do a demo, I need to try something out. This is what I would typically deploy.

B

We have other instances that we support, but I I think I just like put this one in ages ago and I stayed with it. So we have our workers. I'm gonna have three of them. They're gonna be in five forex largest, and then I have the control plane down here and that's pretty much it. I could probably make this smaller. Maybe I don't know, but this works, so I I'm I stuck with it.

B

The change just like it says here in the documentation is we're going to take the platform to just brackets and the replicas to zero under worker, so I'm reducing my worker nodes, okay, so in addition, as you can imagine, um I have to make sure that uh the masters are scheduled like if you look down lower it'll. Take you through the direction of how to do that like so, it may not be sufficient to just eliminate your workers.

B

You also have to make sure that you can schedule pods on your masters, so the pods, your okay.

A

Your masternodes.

B

Are now going to become collapse, they're going to have a role of master and worker so that that's just open shift like if you're doing compact? That's how that's done the one thing I wanted to mention to anyone here is: let's say, you're doing edge compact and you want. You know you want odf. You need to make sure that your masters are big enough for odf.

A

B

So I have so I took a copy of the same file. I'll show you a different a second I reduced my workers, and all I did is I just made my masters bigger, but that was really the reason I mentioned this is but the first time I did it.

B

I didn't do this because I forgot, I didn't, have the 4x large and I go in I'm like oh great, my cluster's up, it's compact yay and then I go to install odf and I got a little warning when I went to install it and it was and I'm like what did what is this? It was a tech preview, pop-up, okay, that said, uh you're installing with minimal resources. This isn't tech preview. So I just want to be clear. Minimal resources is not compact.

A

No, so, let's, let's pause for a second there. I have seen a couple articles, a couple slack messages, a couple you know read: me's out there that have talked about like okd or even vanilla, kubernetes and like having really really really tiny masters and um really kind of not much bigger worker nodes or controlled, plane nodes, and you know worker nodes. Sorry, um like folks, you have to think about what this cluster is doing. The whole time right like if you're throwing metrics at it.

A

If you're throwing you know any kind of operators at it like it's got to be able to handle that workload and whether they be infranodes, control, plane, nodes or worker nodes, they can't have 20 gig disks. They need more room than 20 gigs. Folks, exactly I've seen several tutorials this week, where it's like, oh yeah, all you need is 20 gigs of disk on your master and I'm like how can you even get iops for ncd?

A

Out of that right, like I don't understand, like folks, you've got that cd has to perform good and your cluster will perform good. But if you throw too much on your control plane and you don't give it enough resources, it will struggle yeah.

B

Very well, some thought is required even for compact right, so think about what you're doing think about how large these machines need to be and so on and- and I thought it was at first. I panicked when I saw the minimal resources and then I I remembered I'm like okay, no, no, no it with this setup. I have now where the masters are large enough, so I can collapse the two roles of master and work or on on them and I go to install idf.

B

There are no warnings about anything, even though it's a compact cluster. Why? Because it actually has the resources it needs. They're just collapsed onto the master. So I just wanted to make that point. If you find yourself going to you install your contact cluster and then you go to install odf and you see a warning about minimal resources.

B

It means that the masters are not sized large enough, got it right, and if it's this, maybe you can fix that later. Maybe you can go ahead and install it and then just expand uh capacity later. But if it's like cpu and stuff like that, then you may have to go back and resize.

A

Yeah yeah, if you're.

B

A

Cpu or memory resources you're going to have a bad time right.

B

Right so here's the.

A

Diff on the two of them.

B

Yeah yeah yeah, so really so straightforward from platform. It goes from aw aws and a 4x, large and replicas three. I take it down to nothing. Zero and the type from uh underneath the control plane has changed to something much larger. That's it so. Okay,.

A

B

A

Mean there's a three node cluster now or it's still a circular.

B

A

A

B

Actually, have.

A

Two clusters.

B

Up right now I have my compact cluster because I wanted to show that to you and I also have um the my regular cluster, but.

A

I could do.

B

This all on a contact cluster- I just didn't- want to complicate my life here so.

A

B

Going to run a top because uh I'm having issues these days all right. So this is what a three note cluster looks like a compact cluster.

A

Nice, okay, yeah! I am five four excel. That's.

B

Good and here your rolls- and here your pods everything's, scheduled odf, is up on this. um Just just I mean I don't want to mess around with it too much, I'm as you can see, I'm testing things out, but here your stuff is it's ready. It can do anything on three notes. Wow.

A

Could you do I mean this is a personal question almost I feel like, but could you do a compact odf cluster on a larger open shift cluster like if I had like an eight node openshift cluster happen to sit in my basement.

A

um But yeah, like I'm kind of carriers, can you do a compact install on a larger set of clusters, or do we have to have this mixed mode master worker.

B

So when you, when you go to the part where you go to create the storage cluster, you get choices there right. So you you could decide to um pick the smaller nodes in your setup. You actually do get some some choices.

B

So if you want to do minimal resources, though it's still tech preview, you can do it, but it's tech preview and it and it's actually, I think it's in it- should be in the release stock, as well as further down to talk about tech preview, dev preview, so you can try it out um and also something else. That's available in this release that I wasn't going to do today that we can do in future show is you can turn off different components?

B

So if you, if you want to run super small, you can say you know no mcg and I think uh no cfs like there are certain components. You can just say: don't when you go to create the storage cluster, you turn them off and you're running with the minimal things you need to have rook sephora.

A

B

You could, and in other cases you can also turn off rock stuff and just run mcg. So there's more lots more flexibility coming for, and I think it's probably driven by the edge cases right where you really.

A

Need to choose, you can choose.

B

What you want so this is what a compact cluster looks like with od the difference from the odf side. Is you don't even know? I don't think you would know if, uh if I didn't yeah.

A

Well and then, if you say.

B

Yeah, you don't see it. You.

A

Don't see anything different yeah, it all looks the same.

B

A

That's pretty awesome.

B

Yeah, so that was the compact thing, I'm sure there's more to talk about there, but that's pretty much my knowledge of compact. I haven't done more than that, but uh just.

A

The fact that you can get open shift and odf in three nodes, that's a big.

B

Win for everybody.

A

I feel like right.

B

Stuff yeah and there's more about it here, delivering the three note architecture.

A

Yeah, I dropped those links in chat for everybody to see.

B

A

In case, oh no problem, okay,.

B

All right, so the meat and potatoes of today- and I know it only gets a little bit here, but it's actually, I think it's pretty cool and it's a big deal. We should talk about it all right, so caching policy for object buckets. So, let's, uh let's back up for a second here, let me get rid of this one. um So this this link here takes you. Where is it that one takes right to chapter nine, about caching policy foundation?

B

So but I, but we have to back up because I'm sure no one remembers uh what they need to remember about mcg. Do you remember about data federation? It was a long time ago.

A

Oh, this was a long time ago.

B

A

I'm gonna go like I'm actually pulling up the open shift uh youtube page right now to see if I can go, find the episode, but yeah.

B

A

B

Okay, okay, so, let's back up for a second uh I'm going to close this is my regular hang on. Let me just close this, so I don't confuse myself there. We go all right. So here's my regular. This is a normal cluster right, uh not compact and uh I've installed, openshift, container storage everything's up to date, and I I'm fresh, I'm new. I haven't done anything other than this, so what I wanted to talk about was the backing store, the namespace store and bucket classes to complement what you're going to see in the documentation.

B

So just a refresher for everyone, the multi-cloud object gateway is a gateway. It is not an object, store itself, it's going to use other object stores and create virtualized buckets that achieve different goals. It sets policy on them and and so on. So whether you're talking about the regular standard mcg bucket, which would be when you mirror between several buckets or you tier across you, have like levels and you can mirror you can spread it does the deduplication and all that other stuff.

B

That's one kind of standard mcg bucket, there's also uh name space buckets, which is what we talked about in data federation, which are exactly what they sound like. So just like with database federation. You would have um you have multiple buckets and you say I want to read: I'm going to create a virtual bucket and I'm going to read from these buckets and I'm going to read and write to that one. But the buckets themselves may have information in them already um they can be accessed the way they've always been accessed.

B

So I mean really namespace buckets are just like data federation for databases right, so you federate across several yeah. So so it so it just there's a reason things get named the way they do, and um I think it's just.

A

B

Naming's hard, so in the case of mcg, because it's not an object store, it needs other stores. The standard.

A

B

It needs so. This is where this naming comes in a backing store backs up a standard bucket in mcg. A namespace store backs up a namespace bucket in mcg, so the naming will start to make sense as we go through it, but you it comes with a default one. We installed this one for you, cool and just just like, as you can imagine, with data federation or as well, when you go into your backing store. What are you going to need? Remember?

B

Entity is a gateway, it's not an object store, so it needs what it needs to know the provider and maybe some details about the provider. It's going to need credentials to connect and it's really going to need. What's the target bucket the target bucket yeah, so I can create another one.

B

So this is this: is the default one that comes with it, but I can create another one that has azure as a provider, and these are all so when you do object bucket claims and stuff like that, you can have these back in different backing stores so but a store, whether it's backing or namespace, is always going to have provider credentials and a target bucket minimum.

A

B

A

Looking at your.

B

A

I see s3 compatible, which means you can just have a minio instance pointing at a huge jbod. If you wanted, you know, like the sky's the limit here, yeah.

B

So going back to the multi-cloud, so so, when, if you're just setting up one store, you don't need all of this right, you wouldn't do it this way, but that's not the point. The point is to have more than one and be the gateway to more than one. So I'm going to I'm going to use the default backing store in this one, but uh I wanted to show you also under namespace stores when we go to create them they're this they're, the same. The information required for data federation is the same.

B

The difference is in how it's used, so it just like in databases. If I'm setting up a federated table, I don't want to mess with the original table. That's not the point. I just want to view. I want to view into them so as opposed to. If I were setting up an actual table, I would be messing with it. So it's the same case here when you have name space stores. You're saying uh are we gonna have read into this great. Give me my credentials. Give me my provider.

B

The information you need is the same, but it's not they're. Actually not gonna write to it unless you say so, and that is set in policy. So I'll show you that so the reason we have it's just to set up these different names, so I'll set one of these up for you now. Let's call it.

A

I've been tinkering.

B

I have and actually what I should do, I realized my naming was not so fab uh aws. This is gonna have a particular bucket, so I like to name it not the standard. Let's do the m de palma, bucket, okay,.

A

Makes sense just.

B

So you know yeah and think about it like really.

A

Breadcrumbs are great folks, absolutely.

B

I've already put the secret in as they give you a secret.

A

Yeah, you would have had to get the other one right.

B

And then this one I want my um to palm a bucket in m de palma bucket. There is uh there's already information right. I have some images from a previous demo, so this is the case where I'm not. I don't want uh mcg to go in and take over the whole bucket and all that I just want to read. What's in it right- and I might have more than one of these, so I would consume.

A

The bucket yeah I want to.

B

Consume the bucket- but I don't want to here here- I've got new world daily stuff. This is another bucket, we'll add this one in as well. So that has to be so as I do this that would then depalma bucket, let's uh secret.

A

The region yep.

B

Yep yep I've got all this. I've got one for azure hang on. Let's do.

A

Silly question, maybe not silly why ssl disabled by default.

B

Oh, I don't know, that's.

A

B

A

B

I mean it works for me.

A

Right like I'll, dig into that, because if you me personally being the person that I am with the background that I have, I think all traffic should be encrypted over the internet. But that's just me might not be the case for everybody, but.

B

Yeah, so I was gonna what, let's just call this azure bucket. I can't remember which one but it'll pop up down here hang on a second, I select my secret that I already put in in here.

B

uh It's 12.. I was happy to remember that so obviously name things properly, I'm not here azure bucket 12. Let's do that, so I know that there we go. You know where it's coming from. I know the bucket is I'm all organized in my virtual world here. So if I look here now, I'm under namespace stores, I've got two. So these point directly to two buckets: it contains all the credentials and information. I need to connect to those two buckets they're just stores.

B

These are the actual object stores that are going to be connected to that's it. For this part, we haven't done anything other than set it up. So in bucket class. Hang on a second here. We're gonna have to hide sure for a second okay um in bucket class. We get to start to see how we're gonna separate between standard buckets and name space buckets, so we're ignoring the default for now, because we don't need that. Okay, so what's a bucket class. So here here we go.

B

This is like the the meat of mcg lack of a it is, though so standard buckets and mcg are kind of what we've discussed before, which were um I'll. Show you one here, every every time you go into bucket class, you get you're going to do your general. What category is it in place and policy and the resources you want? So, let's just.

A

B

With this one for a second, if I go into placement policy and remember I selected standard here's where I get to choose how much tiering do I want what's going to happen in that tiering.

A

B

I can say spread great, let's pretend I'm going to do a spread and after that it makes me select the resources right now for a standard bucket. I only have one default uh backing store, I can go add more. We can go back and add more. I can do that. Look it even has it here create a bathroom store.

A

Oh wow, you just put one up right there, smart ui. I love architecture.

B

If I can remember.

A

B

I don't know backing store or something stream.

A

B

You go yeah so and then we use the same secret as before, because I'm going to and then I'll choose a different bucket. Let's do I don't know.

A

I saw like an eight and seven in there yeah.

B

Yeah, where am I some other bucket oops and there's a bathroom other.

A

B

Obviously, yeah I'll get it at seven. Oh wait! Yeah! Okay! That's a second! uh If I have trouble, oh, no! No! No.

A

B

No okay, so I've created another bathroom store, so see how they pop up here and I can choose them. So I can say between backing storm de palma and the default backing store, do a spread. I can go back and say: no, no, no do a mirror whatever, whatever whatever we want to do, but I have to I select them and life goes on, so I'm not actually going to do it right. Just.

A

For folks that are, you know not up to speed difference between spread and mirror.

B

Okay, so let's go back here to placement policy all right, so there actually are some really nice demos about the details of how sdg does spreading and mirroring and deduplication, and all that kind of cool stuff using uh crush algorithms or whatever like if they get into it. uh Much.

A

Oh they, oh! So it's like deep, okay got it.

B

Yeah right just at a high level.

A

B

A

The data everywhere so that you can recreate it, no matter what failure domains you're, trying to hit.

B

Correct with all kinds of like error, checking and so on, and so on and mir is just that a mirror would be a mirror. So you could. I mean we tend to think of mirror as just two, but if you have five uh totally.

A

B

Them you can do that too, and also your second tier can be something else. You don't have to keep them the same. You can have a stretch.

A

In the mirror, you can be as paranoid.

B

As you need to be, I think of it, I'm like how paranoid are you so so that's the idea here so in this case, uh so.

A

B

With mcg is is encrypted in transit, but everything written out is encrypted as well. So if, when I go to create this, if I went to look at it just with like through aws and look at it, I wouldn't be able to read it because it's all encrypted.

A

Got it all right, so rap scouting reef, says in chat, it sounds like spread. Is raid 0? Where mirror is raid 1.? Does that make sense.

B

A

Know vienna.

B

There's more and actually we can do.

A

There's yeah there's error, checking and blocking and all that yeah. I guess.

B

A

B

I would like to get someone, uh maybe the developers who worked on it to actually speak to it, because I'm not going to do it. Justice like there's more going on, but I'm not going to be able to like really explain it well. So so let we can so.

A

Yeah, so if you're doing spread and mirror you're, basically doing kind of like a raid 10 scenario, essentially like your striping and your mirroring, and you know- data replications everywhere, yeah.

B

Cool okay, so so remember this is um this kind of bucket in mcg is, is often used when the requirement policies of the business policy of a company requires encryption in transit and on and at rest.

B

So you know this is where it typically comes up where it's like. Oh it's object. It's got to be. These are our requirements. So, okay, we'll just do it like this um any other questions about this, because I want to.

A

No, no, that was a great one wrap's going on. Thank you. Yeah.

B

Okay, so so yeah, so that this is like standard kind of spread mirroring as he was saying, uh we choose our resources here. We can have as many as we want to configure each one of them ends up in a bucket right and and so on, but.

A

ah There is a question. Sorry, oh.

B

A

What's the difference between spreading then mirroring versus, mirroring and then spreading? Is it just an order of operations thing or is it.

B

I believe it is, but so I've had the same question. I'm like.

A

B

What are the scenarios where I want to do that? That is a.

A

B

I have not answered yet like what right like.

A

Yeah yeah no like when do I spread and mirror versus mirror than spreading. That is a good question.

B

Right so I don't have an answer for you because I have the same question and I will I will chase it down, because I should know that um yeah.

A

Great question yeah. Thank you.

B

Okay, so while standard bucket classes are in you're fresh in your mind, I want to compare them to name spaces, because this is not what you just saw even though uh place and policy and resources and the review all look the same. So in the case of namespaces, we have data in the buckets already and we have to go and configure a bunch of interesting things about this. Is this is federation?

B

So if you look at placement policy for a namespace bucket, you get a few choices, so we're choosing a policy here and let's go through them. The new one in in the ui is cache namespace, which will end with that. But so it's cool, so a single namespace store, so they're different um scenarios where you, where you would choose different ones, so the one I normally work with. Is this one, the multi-name space store so but um there could be cases where single might be very useful to you.

B

So again, an cg is not an object, store, it's going to be a gateway to an object store. So if I have a single namespace store single namespace or it's going to be, I choose, I haven't created any I choose one, oh did I actually. I guess I did um well.

A

B

Yeah, but these are careful- these are namespace stores and we did create them.

A

B

Gonna I so these have data in them.

A

Right right, this one.

B

Was on azure, this isn't the name de palma, I'm not going to necessarily write to them. We have to see where, if I'm in single label, so he'll go next. Okay. So in this particular case I didn't name it yet. I have to go back.

B

I have to give it a name: let's call it uh bucket bucket class single right and we go next. I choose it. The single name, space store. I've got I'm the the back ending. I shouldn't use the word back because it's used under source, the name space store. That's going to be used. We have chosen as aws m bucket, which, when we look here, we can see has stuff in it: yep, okay, just the word player and then but notice. I don't get them. I don't get a choice of many.

B

That would be the multi-name spacebar. I could be right here.

A

Hey you can't select multiples. You can only do that.

B

So so what here we go, so I just created this one, it's available. If I want to make an object market claim on it, it's all there remember I'm creating this bucket class for it. So this could be. I happen to do one in aws, but we can do one in azure, so you can you can create. This is when you have a case where you have some bucket somewhere.

B

That's got to be read only or or um excuse me. You want to be able to read the existing data from it right you um and you're going to write to it as well, so you're just dealing with one. There are cases where you may just want to have this kind of federation with one I don't typically use them. I typically do multi or now I'm going to do caching stuff, but there could be.

A

B

Where that is, if you think, in terms of federation, you want to be able to offer all of that, so um that was single uh multi. But how did I name them? Multi-Bucket class.

B

Yeah so multi: let's do this one: okay, okay, so I'm gonna choose multi, that's the! And now I get to choose more than one. So I can do this nice right and this is kind of like straight up federation. What we were doing before so um I have to I've chosen where I'm going to read from which could be many, many many and now I'm going to choose which one I'm going to write to. So I can. It doesn't.

A

B

Okay and then we.

A

B

And we go off so this is a very simple case, obviously, but no.

A

But like I could totally see regional areas right like I used to work at a newspaper company and we would get photos from people all the time. If I set up ftp servers for these photographers or sftp servers back then uh across the globe that tied to a bucket, I could just read in all the stuff into my. You know, cms and just be like all right, optimize images, optimize images and store, you know locally, so it can get uploaded to our cdn done. You know that would be brilliant.

B

That would be so hang on. Let's see if I mess up my aws creds. No, no, not my azure credits. Oh.

A

Your azure creds.

B

Right there, so I bet you I did so: let's go take a look b, class multi and it's a bucket class hang on a second um I'm sure. Okay, so I'm on the right place and my project there we go okay, so.

B

Let's see okay um b class multi, let's see.

A

B

Why is it having trouble.

B

Oh, my azure bucket 12 is not yet ready. Maybe temporary.

A

Error status, false temporary error.

B

That's me like: it means I did not enter something properly like that. That's probably what it.

A

B

A

Can I go to my portal.

B

A

You wanna, do you actually wanna do this on screen? Do you wanna turn your screen out for a second.

B

I might actually be in there already, but hang.

A

On I'm gonna stop saying.

B

Let me go check.

B

Just make sure I did the right thing, I probably made it up and I'm not actually in a bucket that exists or something like that, all right, I have to sign in probably yeah like it's always the process works. Fine. It's always like I. It's me that's what I have learned. Storage is hard.

A

I mean, let's be honest here: it's not trivial, that's why there's a show a whole series on the channel about it right like.

B

The whole series mcg bucket 12., okay, so let me go in and modify this. I think I should probably be able to share now. um Hang on. I have to find you again here. You are.

B

Okay, I'll put you up here.

B

Did I call do you shoot here, okay, azure book at 12? Do you see my screen? I.

A

B

A

B

At it all right so that azure breaking browse is the right resource. So I have to go back to my namespace store and look at this one. Let's go modify.

A

Creating still so that's.

B

Yeah, I know it should have been like bam. It should be good. I bet you my credit's, my secret.

A

Scroll down for a status message, there.

B

A

ah Account key, it does not exist.

B

Yeah yeah yeah, all right. I have to go off screen and fix it, so you guys can see my stuff. Give me a second okay.

A

B

Please, privacy, please.

B

um Account key account name, let's see what I did wrong here.

A

So, while you're doing that.

B

A

Ignore me for a second: you do you, uh you could even mute me if you wanted um so folks coming up later on the channel today. uh We have a bunch of afternoon shows for me at least uh afternoon, um at two we'll be sitting down with whaling. Dang he's the co-founder of stackrocks, and so this is at 2 p.m. Eastern 1800 utc it'll be on all the youtubes for you to watch later. If you're in europe, I'm sorry it doesn't align with your time zone, but you can watch it on demand on youtube.

A

uh But we're going to talk about like a lot we're going to cover the gamut of you know how stack rocks was created all the way over to you know what way is doing now inside the organization with acs and all the fun stuff we're doing there, and then, after that, immediately after that, we're going to be revisiting our back on the get ops guide to the galaxy. So our back is hard and I always encourage you to go and watch our back videos, because that's where you're going to run into your, like?

A

Oh no massive security hole whoops because I didn't understand this one thing right and uh so we're gonna revisit a previous episode and kind of continue on that. Meanwhile, back here on the data services office hour, we are ready to go.

B

A

He's ready done, I'm ready.

B

So what the just so you know the secret I created for azure was um I had um I had the field names that you use for aws.

A

The error we had.

B

Actually says account key account name, which is right.

A

B

For azure, so so this is now ready. Everything is ready. So if we go in.

A

B

Yep yep yep. We can scroll down.

A

Yeah, look, it just finished everything done. Yep.

B

Yep everything's happy so back to the hour. Okay,.

A

So at this point, we're all set.

B

Up so wait so the final ones we've got. A single we've got a single.

A

B

A

B

A

B

Yeah, a default which is not federated at all, so let's go back in the namespace. This is going to be uh cache there. We go all right. This is what's really really new, so.

A

This is, I think, bucket, will serve data from large from a large raw data out of a vocal caching hearing that.

B

Okay, so that's.

A

That needs some english word. It.

B

Needs an explanation.

B

You would use this kind of mcg federated bucket in the following scenarios. You have far away. You have a massive amount of object data in some bucket.

A

B

On the west coast.

A

Of the us, but I've got.

B

A

My eu data way over yonder yeah.

B

And you want to do caching near ncg near where the gateway.

A

Is so funny clusters yeah reasonable clusters? Yeah is that kind of.

B

Yeah and you want to be able to control where that caching goes yes, so so that would be, I imagine so so in my mind, I always think of it like okay, let's say, you've got some specials. You know.

A

B

Storage, you create a backing store on it, local to mcg, where you're running. Maybe it's compact right? You can see scenario where you've got a little bit.

A

I could totally see a compact data store in each.

B

Major region, and maybe you're not using that for the rest of your your uh odf install, but for this particular case where you need to cache, it needs to be super fast. You want that to be here. Well, we didn't. We only use the default in this demo, but you can imagine you can have many backing stores and that's for regular, normal mcg buckets, but here we're going to use a regular, normal mcg bucket as our local cache we're going to fetch from afar and then do it.

B

So yes, so it's about control like this is this gives you.

A

Right this is this: is the control yeah yeah.

B

So over here remember so hub namespace store. What's my far away bucket, I only get to choose one in this case like I'm like, because it's gonna be some large set of data in one area that I want to bring closer.

B

Let's do let's see if our azure bucket works, it should and then uh so I set that I set. What is my name space store far away and well? Where am I going to write to it locally?

B

Well, I think I created this one with aws, but I can use my defaults as well. So one is a namespace store, but my local cache is a backing store.

A

Right that makes sense, no that.

B

A

But like that default banking store, what what is it using.

B

Aw, in this case, it's awful in.

A

This case it's aws, okay,.

B

Yeah, so the default, and so much correctly if I'm wrong, but I believe the default backing store for mcg will depend on where you deploy. So, if you're on prem, it will actually use uh seth or rooksefrgw if you're vm uh on vmware yeah it'll do that as well.

B

But if you're, if you have a deployment in azure, it will be there if you're going to be uh in aws, it just defaults to the an aws store and it creates it creates the bucket it talks and everything, but you have control now it doesn't have to be that if you don't want it to be so I see this as very useful for hybrid cloud right you're on prem. You got data far away either. I don't know on some other part of the planet. You need to have a view into it.

B

You don't need all of it. You want to cache some of it locally, but you want to ensure that the cache you're using is nice and fast, and you want mcg to do it for you. So there you go so you get to set your time to live right, which you would normally do in the cache and then voila you create it.

B

But the thing that's important. The nuance here is that the hub, the far away part, is federated, so it has to be a name space store. The cache which is close to the gateway has to be a backing store. It has to be a normal backing store for uh mcg, so I wasn't sure if, if that was entirely clear to everybody, looking at it, you may be like what. Why do I have to do this here? So I just.

A

B

To make sure like that, that's the mindset and now you can think of different scenarios where you would use it right. So.

A

B

A

Is one of those scenarios where it's like, I kind of want a picture right like yeah.

B

A

Like we could have like flash cards almost of read many, you know right like.

B

And which, and which scenario goes with like, why would you.

A

Right, why would I use xyz kind of thing, yeah, yeah, correct.

B

Correct you're right so and just to complete this, if I go into object bucket claims- and I make stuff I can I can do you know- um let's do a single just to say with naming to it single right and I can choose something here.

B

I go into that's my storage class and then I choose from my bucket class, which one I want. So you see. However, so the object I could claim chooses a storage class which has to be nuba and then inside of the after that I get to choose the bucket class. I want. If I choose single, it's going to look it's going to use exactly what we configured, which I have to go back and look because I don't remember, but um they're done so if we go, does it give the information here? No, no.

B

This is all related to we'd have to actually go into it, so that was single. I've just created a few questions, questions questions.

A

No questions, no questions yet.

B

All right, so here we go here, comes the next one, we'll do a multi, nice yeah, just just to show you and then we can go so here you start to see them look at it. It tells you the secret it's using, so you can see how important your naming is going to get right so that you should be able to look at this at a glance and say: oh, I know I know what you're doing here kind of stuff so hang on. Let's just do the last.

A

B

Okay, something more useful than bucket cash. I hope people use uh and off we go so so that so the caching part is the new feature. I'm sorry. I've said that wrong. Having the cache available to you in this bucket class ui is the new feature there was caching before, but you couldn't. It wasn't quite like this.

A

You couldn't necessarily configure it as granularly as you could yeah.

B

Right right, so that I think what would be and I'll probably follow up with this is um some diagrams explaining like when you have the following scenario. This is where you would use you.

A

B

Name, space, buckethead and so on, but it's sort of it has some nuance to it and I I think for the caching one I mean I could see that being incredible, really useful in um in edge cases right. You got a little.

A

B

A

Going that's something that was.

B

My thought it was like that's useful on edge and then you're doing your work there right at the edge right, you're bringing in the data you need. You have hot cache, you work with it and then you write it back so um so those are the big. What's new, any compact questions, I I don't know, did we cover everything we wanted to cover in compact.

A

I don't know uh folks out there. If you have questions about compact, let us know um I feel like we did a pretty good job. There.

B

Okay and then so uh just everything I did is in the documentation um a lot of it. You can do all of this from the cli it just. um I wanted to show off the ui and it also looks better during devops. So uh this the chapter I'm in here is managing hybrid multi-cloud resources for those who are interested.

B

If you want to see more about um all about mcg how we do all of the different parts and what have you so, but I, the one that's coming out of release notes is chapter nine caching policy for object, pockets, but again, if you're looking into it, you don't understand it go, go back and look at the um the supporting documentation for how to use mcg, because once you have that in your head, I think it all kind of flows from there. So so they are there we go want to create.

B

We can create more stuff. We can look at some of the other stuff in the release. Notes, yeah,.

A

What else what else is in here? I'm I'm curious.

B

There's a ton of stuff in it there's so much like it's it's yeah I mean that's our so new features. Okay, so we did compact, we've done cash and policy. uh We've already done persistent volume, encryption, fake provision, stuff. Okay, you have to talk to chris blum about that. Guy.

A

I think provision for vmware is a whole like thing yeah, it's.

B

A whole thing right so and I would I would like to do some of the other things here like for ibm power systems, but I I right now. I don't have access to those kind of labs, so um maybe in the future we can do some stuff on that. But um let's see air gap stuff blah blah blah enhancements, we've got some new alerts to come out. um I would like to do replica 2 at some points for you that would be fun.

A

It's in here somewhere hang.

B

On um yeah yeah wait wait. I also want to see what dude- oh hey, hang on once again compression d compression individually yeah yeah. So we did this too added the ability to create names to suck it using the user interface um tech previews are always interesting. Maltese, it's there that would be an entire show and a half.

B

A

B

Yeah I mean yeah.

A

We should do a malta show to be honest with.

B

A

Not necessarily uh like just you, but I could bring in the expert from my team on multis, and we could do that. That.

B

A

A collab show that would be cool.

B

We could do a collab show and then like. If they do the most part, then we should be able to set up so that uh all of the storage talking goes over uh this segregated network, so that would be awesome. Actually that would be.

A

B

I think- and there are already um customers who do that and one- and there are a bunch of others who want to do it and know they have to do it because of that stuff. So what else is going on? I think we there was a component. I was looking for the. um I know it's here uh flexibility, so this is going to be really interesting, also for edge right. This idea here that you can turn things on and off turn off, rdw or turn it on and so on.

B

um I would like to test that out on a we could do like a super duper minimal, odf on a super super tiny edge, uh edge cluster and see how well that works run some performance testing. We could do that on your cluster.

A

That's true: we could.

B

Maybe we could do it right like see how small can we get and still have what we consider reasonable performance and just turn off pieces? You don't need anything.

A

Yeah, that would be a cool show yeah, but like I would want to do some setup. First, obviously, um I mean we could pull up my cluster right now. If you want and walk through it, but we only have 15 minutes.

B

I know so like so I yeah when I was thinking about like what else am I going to show like ah some of these things are really big so um and it's fine.

A

B

A

If we end early, that's okay, just as long as we answer people's questions and share more later, we're fine, so.

B

A

Flexibility and sizing- that's cool. I like that.

B

Yes, yeah and being able to really fine-tune what you want in your edge uh cluster is very, it's really important so and then yeah. So I always think of like small edge cluster, uh very, very minimal footprint for odf, and then um you can even just you have mcg and you're just like caching locally what you need on something very fast right. You do your work there and then you send it back or something like that. It's just uh more flexibility is better. So that's kind of what I have chris, that's great, no.

A

This was very informative, I mean we got questions, we got, you know some curiosity, we need. We realize that we need, to kind of you know, maybe have six or seven slides that say hey. This is when you would want to do these certain scenarios.

B

Yeah right, like.

A

I think that would make for a great blog post a great live stream, a great everything right like.

B

You could write the blog.

A

Post and we could just.

B

A

It on air, potentially, you know.

B

A

Kind of demonstrate and show off how to build the various things in the blog post, that'd be kind of.

B

We could also do a um and a more in-depth um explanation of how spread and mirroring is done. Inside of mcgee we've been done before we.

A

Could do that like if we.

B

Wrote it come on talk about it, that would be fun, and then maybe we can also uh talk about caching and and what's going on there as well like how that's all being managed internally. That would be interesting to have someone go through so into it.

B

Any new questions. Anyone uh we didn't come up with a poll.

A

No, but I do need to give you access to the thing that can give you polling, so we can do.

B

Later, I think.

A

Actually yeah, you might already have it, but I feel like I've shared that with you before but anyways. Maybe I didn't. I've shared a lot lately, just trying not to be a single point of failure, so we'll see um but yeah. Let's definitely talk for a few minutes after and uh folks, if you have any questions, feel free to reach out to me at chris short on twitter, my dms are open short at redhat.com. My email is open and I can get any kind of question you need answered. Trust me.

A

I have ways um I ask nicely: that's the.

B

A

A

Right all right, so thank you very much michelle. This was great. I'm glad we went over all this stuff because, like the tiering that that fascinates me right, like bringing data closer to people in compact clusters based off what region- or you know, geography or you know- data center they're in yeah- that's going to change some folks's world and that'll be awesome. So thank you very much for showing us that thank.

B

You thanks for coming.

A

Yeah, thank you for coming and have a great day. If I don't see you again later today, stay safe out there. Folks.

A