OpenZFS Leadership, 15 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: October 2020 OpenZFS Leadership Meeting

Description

At this month's meeting we discussed: DRAID code review; "healing resilver" terminology; zdb --cp; forced pool export

https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM

A

Welcome everyone to the uh october 2020 uh opencvs leadership, meeting uh the post developer summit edition. um We had the for those who are aware, hopefully you're all aware that we had the opencvs developer summit last week um and I posted all the videos uh on the website now on youtube. So you can find those if you missed any talks.

A

um There was not much on the agenda uh for today. um I just put on something to kind of discuss any um any things that came up by the conference. um If folks had any questions about any of the projects or the talks or um wanted to give feedback about the conference organization in general, we can do that cool. um So I guess I'll start with the things that I wrote up about the ais and then uh folks think about if there's anything else, that you want to talk about.

A

So um the first item was uh there was um a hackathon project around uh doing some more work for the zpool feature sets uh project so defining which features uh which which feature sets would uh be in there by default. So, like you'd be able to say you know, zeep will create o features, equals compat, 2019 or compatible 2019, or something like that, and then that would only enable features that were supported by all of the um tier one platforms.

A

On january, 1st 2019.

A

So we need code reviews uh both for the work from that to to kind of verify that those are the right sets of features and then also for the main um pr, which I think I need to take a look at the code.

A

um But I think other folks would also maybe like to take a look at that. I linked to the pr.

B

Do you know if that.

A

B

um The z-pool status nagging you to run an upgrade.

A

Yeah it should so it's it's both the set the like that be able to create it with dash, oh um and also uh I'm pretty sure. He said that he changed it uh per the design that we came up with a couple years ago to um like, when you run super status.

A

It uh it'll tell you basically, like are all the features that you requested enabled, but um if there's newer features that are not part of that feature set, then it doesn't mention that anything about it.

B

A

And same with, like upgrade yeah same with, like zip great dash, a it'll only enable features that are in the requested feature set.

B

Right so if you're in compat 2019 and you want to switch to compat 2020, you just z, pool set features, equals blah and then z pull upgrade dash a.

A

Yeah, uh I think so you might not even need the upgrade dash a after.

B

Right yeah, I should just set the flags yeah um yeah that sounds good, so yeah. We can definitely I'll take a look at that.

A

Cool thanks alan um other stuff that I wanted to bring up was the d-raid code review um and brian. Maybe you can just uh mention where you're at with that and and what you're looking for as far as the review process yeah so.

C

You're, probably.

A

Looking for another review from me,.

C

But aside from that yeah, that's where I'll start I got a review from that would be great, um but anybody else who wants to take a look at it that would be um welcome to um the code is all there. I will probably squash it one last time just to make it easier to review. There were some minor fixes so probably today or tomorrow, I'll push an updated version and squash everything um there's a lot of code there to review.

C

So a really detailed review would be a lot of work, so I'm not really asking for that. Necessarily, although it would be welcome, but if people want to pull it down and test it out make sure it works for them. I've seen a lot of testing already, but more is always welcome, but I think that would be a good spot to start just make sure it. um You know, looks good in terms of there's not confusing in terms of documentation and um that kind of thing detailed code reviews would be welcome, of course, but.

A

um Has has mark maybe or anyone else done, a detailed code review on it.

C

Yeah, um I don't think he's posted anything in the issue, but he's definitely gone over all the code with me on this. So he's uh probably needs to go over it again because there's been some bug fixes and I'm gonna keep working with him on it, um but we've both gone over the code. Quite a lot cool.

A

Yeah, that's good that at least it's like two people are familiar with all the code in detail.

C

Yes and it's based on you know other work. People have done it based on isaac's original work. So there's.

B

Changed a lot since then.

C

But a lot of people have had their hands in it, um and a lot of people have contributed fixes for it. So.

A

Yeah, it's good to have you know one or two people do detailed review of the final version right.

C

Yes, absolutely, um I would say if people want to focus on the particular areas you could pick one part of it. It touches a lot of the code base so like maybe just look at how the d raid implementation, core implementation works or how it integrates with the rest of the zfs code um or the.

D

C

Yeah documentation like reading the man page, do you understand how it works? What the trade-offs are that kind of stuff that would all be very welcome feedback.

C

Cool yeah, I suppose I can even post a link in the issue, maybe to uh mark's talk for people who want to get familiar with the details.

A

Yeah sounds good.

A

The next item that I had was the block reference tracking design doc.

A

I don't see um powell on the call, but I know we worked with him a little bit during the hackathon to just kind outline what uh what the like data structures and stuff would need to look like. So I think that's kind of the next step for that project is to is, for him to.

A

To take a look at uh to put together the design doc and then maybe present that at one of these meetings uh in a future month.

A

Cool um next uh alan, do you want to talk about this? One item that you added the terminology.

B

Yeah, just uh with mark maybe's talk talking about you know the different kinds of re-silver we have now and the fact that you know what we used to call uh uh sequential re-silver is just kind of the batch mode of the healing, very silver, and we have the new thing. That's actually a really sequential re-silver uh and maybe making sure our documentation uh clears that up and that you know we can use the right terminology so that we all know what we're talking about.

B

When we talk about it- uh and I don't know if, uh like I don't know what a the sequential rebuild stuff looks like when you do his equal status, but maybe making sure that uh and the kind of actual healing scrub are more clearly delineated and it's easier for people to tell what's going on yeah.

C

So, regarding the status they actually look very similar, so the output of equal status is almost identical, whether you're doing a sequential or a healing resolver, um and we tried to update the terminology in the main pages. But I think we probably missed some references here and there.

B

But that would be great yeah it'd be great. If someone could uh just look it over and make sure that we caught all the places where we talk about, we silver and we, you know, delineate the two types and uh maybe looking at the zepal status stuff and making sure that uh they're not so similar as to be confused with each other. But you know we don't need it to look too much different, uh obviously, but yeah, which.

A

Is something I thought of.

B

When I was uh reviewing uh mark's talk after.

C

A

Yeah, there may also be documentation on the websites yeah uh that we want to clear up um because, as I recall, we did update the man pages, at least, although that might have been not totally complete, but probably there's documentation on the websites. Talking about you know, scrubbing re-silver and whatnot that um we can reinforce the new terminology.

B

There yeah, I forget the name of it, but there's used to be a tunable anyway on freebsd to be able to disable the batch mode, but it might not call it batch mode or whatever uh to disable the range tree for the the healing we silver uh and just do it. You know the 2010 way um and make sure that the documentation of that uh reflects that it's not sequential receiver, that you're disabling.

C

B

C

Update those documentation with that, I think it.

B

Was called legacy.

C

B

C

B

Was the word I was trying to grab for, but I couldn't find it legacy scrub. Maybe nfs scan legacy is what it was currently called yeah that was it yeah. I couldn't think of what it was called, but legacy was the right thing to be able to search for.

D

Yeah, no, that yeah that was put in there, so uh people could stop being freaked out by how it behaved differently and after having tried it a few times. I just ignored it and completely forgot about.

C

B

Do we lose matt, oh there, he is.

A

Sorry, my internet just died. You're.

B

Back, I guess the next one was also me. The zdb uh copy out thing that uh jorgen suggested and I managed to whip up in during the hackathon um yeah. I posted the initial version. I gotta look at brian's feedback uh and I think we also want to do the long ops thing to it, but maybe that be a separate review just because it's not related, and it would be a bunch of work that I won't have time for as soon uh but yeah.

B

I want to get some test cases added and I uh jurgen found that I had missed, adding it to a little.

B

Call list, or whatever of uh the flags that shouldn't enable the dump all option and so or shouldn't get enabled when the dump all option is set, and so uh it was trying to copy out random things. If you uh ran some other mode like trying to look at the meta slabs or something anything that would enable dumbball was also trying to enable the the copy code and making it not work correctly.

B

But I fixed that part, but I do still need to add tests, and uh I think I didn't update the usage message in zdb. I remembered the man page, but not the usage message, that's baked into the binary and a couple other things there.

A

Cool thanks for doing that. Alan.

B

Yeah and then I had the ideas to extend that further. Currently, it just ties the copy out in one megabyte chunks, but I think it might make more sense to actually look at the denote and use the record size so that on error, you'd have the option of writing all zeros or something and being able to say copy up most of the file. Where you know one of the blocks check some fails or whatever and be able to recover the rest of the file.

B

At least you know, I've seen cases where it's like a 20, terabyte vmdk with two checksum errors, and you know if you try to just use cp, it says that file is not readable and you know you can do stuff with dd or something but uh having it just be something you use zdb for might be interesting.

B

B

Or even be able to do something kind of like uh we have that zfs send uh send invalid data or whatever it's called there's.

A

Another that's a module parameter. I think for that.

B

Yeah but having zdb be able to do that, for you know, while copying out this, one file in its own uh version uh makes.

A

A lot more sense than having a.

B

Tunable, let you do that on the live system, so yeah yet another flag for cdb yep, but I think maybe I'll wait until I've done long ops to add that bit.

A

Yeah good idea, I think you can't you can also do it with um dash x. There's like a flag that lets you set tunables like you can do like dash x, blah blah variable name equals value.

A

um So I think that you could do it that way as opposed to a flag, um which is you know, maybe even hackier, but you don't have to like add new, um add a new.

B

Fly plumbing or anything like that: okay, yeah, yeah, we'll see uh what I can construct to do. That and it'll be interesting to be able to make a test uh for that too.

A

Yeah, you definitely could.

B

Yeah, like we can make a test image that purposely is corrupted and since it's a stripe there won't be another copy or whatever something like that.

A

Cool well thanks for doing that, and congratulations uh for those that didn't stick around for the hackathon awards. um The allen won the first place in the open in the hackathon for his work on z, on zdb dash, cp.

A

Cool well, that was all that I had on the agenda.

A

Do folks have questions? Are there other things that we should have on our radar of action items, or are there questions or comments about the developer summit or any other upcoming work?.

B

um I expect hopefully, this week, we'll post our review for uh the forced export feature. uh Excellent will's got that uh rebased on master. Now uh we originally did the work a while ago for 0.7, but we've got it rebased on master now and quite a few improvements along the way as well, but it looks pretty good. It works very well on freebsd, where you can force them out to file system, but it also works on linux.

B

A

That do the file systems like stay mounted but kind of hung. There.

B

uh I think yeah they end up in some kind of state like that. I think the main thing we had to do for linux was change. The zfs vfs uh opting um z unmounted uh into a tri-state instead of a boolean to cover the fact that you know we're going to try to unmount it it might fail, in which case we need to not remove it from the list of file systems. Zfs knows about so that it doesn't end up uh in this weird state.

B

Where then zfs will fail while trying to do the export, because there's still this one file system, that's in use all right.

B

I think it adds a tunable to be able to force it anyway or something I forget exactly what the difference is, but uh because the use case uh for some of the people using it is they're just doing send receives so the file system might even be mounted, but they still need to be able to force export and so there's a tunable to enable the more aggressive mode or whatever, whereas the default would be.

B

You get told that file system is in use, go use, lsof or whatever, to kill off everything, that's using it, and then you can amount it, but in the with the tuna or the module parameter, you can just say: leave it in the you know.

B

Let zfs export anyway and leave that file system in that kind of zombie state.

B

A

Look forward to taking a look at that code.

C

Yeah me too I'll be interested to take a look at it.

B

Yeah, uh luckily, it doesn't involve nearly as many changes to send receive as it used to uh to be able to unwind ascend that was in or a receive that was in progress when the pool was suspended uh right.

B

You know, which was mostly the fact that the original version changed all that before redacted send and receive was import merged and so the since it all had to be rototilled again anyway, it was easier to try to come up with something that was a lot smaller death, gotcha.

A

Cool other questions about stuff from the developer summit.

A

If not, then we're all going to get quite a bit of time back today.

A

All right: well, I imagine that folks are maybe a little bit um open. Zfs sat my brain is kind of saturated with open zfs from uh last week's conference, so I think understandably, people have gotten a lot of opportunities to talk about their open, dfs stuff last week, so not too surprising that there's not much to discuss today, um we'll be back four weeks from today. Let.

B

Me just double check.

A

I thought one more thing uh I was looking.

B

At uh there was some stuff paul dodeck did years ago to make uh zedifest list faster.

B

If you only wanted the name of the data set and you were sorting by the name of the data set, um I looked at it more closely and it looks like we could also get the grid and the create txg for no cost as well, without having to look at the the properties that put the snapshot, uh and so I'm looking at the ways that we'd be able to do zfs list of name and grid sorted by name without actually making it slower uh and so I'll post. A review for that.

B

When I get that working, uh but my prototype seems to work so far, meaning that you can get uh you know, zfs list, dasho, name, comma, guide s name without having to load the properties of every one of the snapshots, which makes a big difference. When you have 16 000 snapshots across a thousand data sets or whatever.

A

A

When you're just getting the name, um you could just get that from the zap object, without looking at the data set at all right.

B

A

Which case you wouldn't have access to the guide so.

B

Maybe the optimization wasn't made uh what the difference is, so the name is what was already there and then what I was extending was for the uh to be able to get the grid because we get the data set already, because we have to check if it's uh within the range of transaction group numbers.

D

B

So we have that, but we don't go and do the next level. I'm trying to remember exactly well.

A

That makes sense to if we're already getting the data set, then to provide you know whatever we can from there from.

B

There, uh instead of restricting only to the name and then you can get that much more information, but without having to call the um to look at the properties yeah, so basically everything you can get from. I think it's dsl get stats fast or something like that, which is basically directly accessing the dsl fizz key that you already have without actually turning that into the object set and getting the zap of all the actual properties to get everything else.

A

Yeah, there's probably a lot of I mean you can get the vast majority of the properties. If you have the data set to you, I mean you can't the only thing that you can't get are, I think, user properties and um like zpl sizes, evol, related properties.

A

Like the version number and whatnot.

B

A

So almost everything.

B

Yeah we can get dsl data set fast stat because we have the data set uh quickly, but otherwise it ends up calling the object stat one which uh is much much slower because first, you have to actually uh do the obset from data set.

A

Yeah thing gets you like all the accounting and whatnot, for example.

A

So you know you would have like all the stuff that you have in the default listing of like, if you just do z, pool zfs list dash chi snapshot.

A

um That may not have anything other than stuff that you.

B

Can get from the.

A

B

Yeah, and so we can make listing, uh take advantage of that that'd be useful yeah. That would be really cool.

A

All right um we'll go again any any more questions comments projects folks would like to discuss.

A

All right, in that case, happy october, and we'll see you four weeks from today on november 10th at the later 1 p.m time and enjoy your opening fast hacking until then bye everyone. Thank you.