OpenZFS 2018 OpenZFS Developer Summit, 21 Sep 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: VZVOL by Rainbow

Description

From the OpenZFS Developer Summit 2018
Slides: https://docs.google.com/presentation/d/1BFgMUoUu5NxotqVbbn-ci-gAF3ylpzNyy9Dj1iwY_F0/edit?usp=sharing

A

Our next speaker and last speaker before lunch, she is a director of Infosys of security at the startup called pull request and she has create. Was that all move stuff? Sorry, so it moves up and she will be presenting about this new tool that she created called visible, which helps users to manage and operate with the divorce and on ZFS. So please welcome rainbow.

A

B

Sorry guys give me one second I.

B

B

So awkward silence, more awkward, silence, okay, cool how's that right doing you guys, having a good time so far cool so I'm rainbow. Most of you probably have no idea who I am that's: okay, cuz, I, don't really matter, but sorry about that bit of a snafu with the employer name when I first submitted to this conference, I was actually working for a company called pool request based in Austin.

B

Since that time, I actually joined the AI ops company, moog soft we're based actually here in San Francisco I'm out in Colorado, so I'm actually gonna meet half the people at my company for the first time in, like two days after this, so yeah I'm, rainbow I wrote a thing called VZ vol everyone else's talks. Let me close to Mike. Everyone else has talked so far been like really cool, really interesting, really high level, if you're expecting anything anywhere near that awesome here, you're about to be sorely disappointed, so get ready.

B

I've actually never presented in a conference before so. This is my first time and I'm slightly terrified. So let's do this. What is VZ vol? Have any of you? Buddy has anyone here heard of easy ball?

B

One person, two people the guys who I submitted to okay cool. You guys know what a Z balls are, though right: okay, cool, so um a little bit of background. Vz ball is a thing. I wrote to make my life easier when dealing with Z balls, so maybe I'll make your lives easier. Maybe it won't. This is probably waste your time, but we're gonna get started anyway.

B

Anybody here ever used IO cage on FreeBSD a couple- people, okay cool, so you can think of this like a really really bad knockoff of io cage for Z balls. So you know I mean I make music. My bands are bad ripoffs of other bands, my codes, bad ripoffs of other people's code too. So Who am I like I said I'm rainbow. A most notable thing about me is I've. Got rainbow hair. Wait. Note that that's that's the wrong rainbow, sorry that that's not the right one, either IIIi, don't know who put these slides together.

B

I went on Fiverr.

B

No, that's me so I'm rainbow I'm, the director of information security at a company called Mook soft. Like I said we're an artificial intelligence startup, we don't use ZFS, we we don't, we don't use FreeBSD these. This is something I did actually in my personal time and I'm trying to convince everybody else to kind of hop on. You know the BSD Solaris kind of bandwagon maybe get some ZFS action going on in there. But you know it's it's. That's not really. My wheelhouse in the company, so I can only poke people with a stick.

B

Often angrily Mook soft was gracious enough to actually you know, let me be here and take the time off and present today. So um can you guys say thank you dumb. A couple of them are watching.

B

So what is VZ Vall? It's a basically it's a shell script. I wrote it it's licensed under the two clause. Bsd license it's a zeevala management tool, it's actually in FreeBSD ports, so you can just PKG install VZ vall. If you want to give it a try, um pulls in a crap-ton of dependencies in and of itself, it's actually a zero dependency program, but it has some optional functions that are enabled by default on the port, to make your life easier and those do have dependencies so I'll get into that in just a second.

B

It's fairly easy to use I'll be doing a demonstration at the end of this. So you'll see it's just a couple of flags and you're you're up and running it's written in SH, posix-compliant, SH, no, no bash isms! Here, if any of you have ever followed me on Twitter, you know how much I hate bash isms. So it is properly posix-compliant. Sh it'll run on anything with a Bourne shell.

B

Originally it was it's really targeted for allowing you to make you subs evolves with you, no jails, VirtualBox VM. It's actually kind of how it got started. A little bit a story there I'll get to that in a second, and you know other virtualization environments like beehive. It's just it's meant to make your life easier. It's not magic as cool as that would be I can't do your taxes. There is no pot of gold. I'm, sorry I know the name rainbows little misleading.

B

There is it's! It's not World, Peace in a box, that'd be pretty cool too, but that kind of get away from what I actually wrote it for, and it's not gonna, do anything for a file system that isn't ZFS. So uh everybody else you're. Out of luck. Sorry, oh.

B

Sorry I have like no actual monitoring feedback so not used to it. So some features of easy vault, sorry for blowing out everybody's ears lips.

B

So it's it's easy to use. It creates delete some lists, yours evils, for you can manage them according to type there's actually two types in it. I week you can either create you know, normals evil, with nothing. Fancy added to it or it'll actually automatically create a shim VMDK for you. If you want to use yours evolves with VirtualBox, that's kind of a pain in the butt you gotta remember some weird view box manage command. Funnily enough, that's how this all started! I could never remember the command. I literally just wrote a shell script.

B

I was supposed to act as an alias and take two arguments, and then it became a couple thousand lines of code when I decided to expand it, you can also format the years eve all with various file system. So, instead of having to remember, you know, like the mkfs dot, you know EXT to command. That goes with it. It's just one flag that you got to it.

B

The code is fairly simple: it's fairly clean I did my best to not make it like an abominable eyesore, but you know it could always use a little bit of help and as a bonus it will not steal your soul. I can't absolutely guarantee that, but I've done I've done a pretty good job of making sure there is no soul stealing going on.

B

So that's why I wrote zvz vol, I I could never remember that this is terrible. Anyone at Oracle, you should feel bad you're bad, and you should feel bad.

B

So that that's actually why it was just supposed to wrap this, it's it it's um it's a demo time yet I'm, not I'm, not yeah, let's, let's, let's do the demo, so let me see if I can actually get it does this have internet access can I get a terminal up, yep cool.

B

B

Sorry, I don't normally use a Mac. I've got a ThinkPad p-51 with FreeBSD on it. So oh.

A

B

A

B

A

Think this hike is good.

A

B

I didn't want to like accidentally, you know, show off Matt's screen, but you know he's the one who brought his laptop to a to a conference and let everyone use it. So not my OPSEC problem so give me one. Second: I've got a shell into a box. I set up just for this.

B

If I can actually remember what the domain is for, it.

B

Typos are always fun.

B

Okay, so we're sheldon. So the first thing is like I, said: VZ vault, it's a it's a package, so at least on FreeBSD um I do build Linux packages. It is tested to work on Linux, but that is not a primary platform that I support, so any any any any extent bugs on Linux I, don't really test it on Linux myself. Anyone here runs it on Linux. You know give it a try file a bug report whatever. So here's what visible looks like.

B

So we got one up already and that's that's the output of list. It's basically just pulling in the information from from ZFS list and searching for XIV all tagged datasets. So it's it's not anything particularly fancy. It does have some. You know nice output. If you can see type raw, that just means that there's no no VMDK, so we're gonna create one on the fly using the tool. Somebody give me a name for the volume and give me a size.

B

Not gonna name it tank.

B

All right, we'll call.

B

And how big are we making it five gigs cool, so the thing about this first of all, is there a couple flags that go with it? Vz Vall does have to either be run under pseudo or as root right now you can't run it as your normal user. Obviously it does a little bit of weird stuff in the background, when you're running a pseudo to make sure it's setting permissions for the right user, so you can actually use the XIV. All yourself I need to clean up the demo fest stuff that permissions.

B

It gets a little wonky. There's um there's some code cleanup to do there, but um I could use some help so so the first flag, we're gonna provide, is tech, tea and that is just to say the type. So there's only two options: raw and I can't reference VirtualBox or VBox. So we're gonna, we're gonna, find out I, never use that functionality anymore. So.

B

Ssize, you can specify it in megabytes or gigabytes right now, so we're gonna do 5g. You can also do you know 102 for large m. If you want to do it and it specified Meg's I, don't know why do you want to do that? Those are the only two types of options it takes right now. I need to expand that and make it a little smarter about actually doing with appropriate size, and then tech V is just gonna, be the volume name. You have to specify that there's no way around it.

B

So it does some testing. It also creates the VMDK make sure that it's in the proper location and if we go ahead and list it again, this screens a little too small hold on.

B

Okay, well that you could. You can see this and I'll put a problem now with the path but um yeah. So you because you can see it, creates a a VMDK right uh right then, and there and and everything's happy. You can also do a couple other things like format it with a file system. As you can see right now, it says FS, you know none.

B

We can go ahead and let's see what file system should we try and form at Trinity? Yes,.

B

No, no explicitly zero support.

B

Okay, let me see if I can remember, it will remember the the appropriate flag, I.

B

Can't so I'm going to read the man page reaction of a man page.

B

And you can change the the pool you're adding it to it. Has some auto select a logic so it'll either, because this was I wrote this on FreeBSD originally it'll either yeah it'll pick the first by default, the first zpool that you have, if that one's name boot pool, which is normally the case on a full disk encryption, freebsd installation with ZFS it'll move to the second pool. But you can always override that and you probably should.

B

Now, if I remember there should be ah format option there.

B

B

So it'll ask you: if you want to destroy everything, it's gonna freak out a little bit and then it prompts you. You can override it with the TAC F flag. That's for more for automation like if you're using this as part of another script.

B

Otherwise yeah, probably let it ask you.

B

Now we can go back to lists and sorry for the terrible output. But if you look now under FS, it's showing you of us and it is a you FS system. So it's it's perfectly usable as a! U FS dis gun on just about anything.

B

Alright, so so that was the demo I'm gonna get to a little bit more information about this I realized. That was probably the worst demo you seen all day, but um not my fault, like I, said I'm new at presenting oh wait was that the only demo you've seen all day, oh cool I, have a really really short memory.

B

So a little bit about this again, we are on github, it's at github, calm, slash, rainbow hacker horse, /vz vol I'm going to be moving it to it's an organization page soon, right now we're at version. 0.70 more features are coming soon. There's some bug fixes that still have to be done. Oh god, please help I'm really bad at this.

B

Also, as of literally 20 minutes ago, we actually have a logo designed by my my lovely girlfriend Eileen and yes, it's a horse and it is dealing hard drives like their poker cards. So don't ask me why we thought that was a good idea, something about logical, logical drives all right. So that's kind of the conclusion of the worst presentation. You'll see all day, I made it through it without a panic attack. So boo, boo, anybody got any questions about this horrible monstrosity. I wrote.

B

Right there there is no, so the question was: does it do partitioning anything? That's part of the filesystem formatting tools. You can technically pass through right now, though, it's a little limited. That is a good question. I personally kind of hate partitions, so I would just create more than one XIV all if I needed that kind of situation. But it is something that you know I'm sure people would find useful, so I'm perfectly happy to accept any pull requests you guys have for that, and it might be something I.

B

Do the road map that'd be something definitely we should have for one Oh anyone else. Cuz these lights are really bright and I can't see right. There.

B

No, no! No, because the via the VMDK is sorry about that. The VMDK is only a little itty-bitty shim file that points to the zi ball, so it has absolutely nothing to do. In fact, you can't even manage that VMDK with VZ ball. It's automatically deleted when you delete this evil and it's automatically created. If you specify that you want a VirtualBox types evolve, that's that's it!

B

No! No! No! It hasn't absolutely nothing to it. Every part of this program works specifically on the Z balls themselves, and everything else is just kind of fluff around it.

B

Ok, so the question is: when you're creating a virtual box type VZ ball, does it create the does it automatically route the VMDK to a guest VM? Is that kind of where you're coming? What your question is? I know that that's on you dude, I I as little interaction with VirtualBox as possible. The only VirtualBox utility that gets used as part of VZ ball is that that VMDK create option. So no, it's not gonna do any of that. That's kind of out of scope.

B

If you want to attach that to your your VM, you you, you go right ahead and do that with with other tooling. That's not gonna, be part of it.

B

Right now know it kind of defaults to your the the default directory. I, probably should add a function for that. It should be a quick, easy thing to do, just add a flag and then an override for the variable that sets the path so that that could be something to add. If you want to go on the github page and drop that in as an issue, that's something I can do this weekend.

B

Yep whoo, where there's a question somewhere Oh Ellen.

B

B

See I actually didn't know that so no it doesn't set. It doesn't set that that flag it does. It sets a custom flag for FS type when you're formatting, but as far as anything, any other data that tripping over rounds on here as far as any other types of properties. No, it's not setting them I, actually wasn't aware that that property was a thing, so maybe you and I can talk later about the benefits of properly setting that and maybe that's something I can bring in for just in case. Anyone didn't hear.

B

Alan's question was whether or not it's at the whether it was a rosy ball or a GOM type xiv also- and the answer again was was no I didn't even know. That was a thing all right cool thanks for having me.