OpenZFS Leadership, 14 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: December 2022 OpenZFS Leadership Meeting

Description

Agenda: unmapped prefetch; bandwidth quotas

full notes: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit#

A

All right, let's get started, uh welcome to the December 2022 openza solution. Meeting I see a couple of items on the agenda.

A

um Do you have any announcements? So, let's get started there uh first, one unmapped prefetch review request is that you logged under.

B

Yes, it's me uh just added I already announced it in in chat and personally to few people, but last week I completed a map prefix NPR, which I started during the summit hackathon. It appeared bigger project than I originally hoped, but by chip here to be not bad, but it's based on much more invasive Arc cleanup.

B

So it's not exactly trivial to review other than it's in two separate patches, but I'm not sure I can split it smaller, smaller chunks, but I think both are good to have, because that code is pretty old and needed cleanup anyway, and the final result I got with it is pretty amazingness for me uh like. If somebody don't need compression and don't need much of data, cache hits during sequential operations and dispatch particularly doubles, read speed from one point: it or introduces prefix, which was for many years disabled for uncashed data.

B

If you said primary guest metadata and second, it avoids one memory copy out of two. So, instead of quipping data from Arc to debuff from debuff Cache user space, now it Maps directly buffers from Arc to debuff Cache, since we know that buffer will be evicted very quickly, we don't need to use ABD for that, and science is on Cache.

B

We can just map the buffers directly and that's only one memory copy, so I was instead of two gigabytes per second I got four gigabytes per second, which is probably the speed of single memory copy, the One stream. That's great yeah and that's same happens, fluorite just for right.

B

It's happened on different thread, so right performance is about same four gigabytes per second, yes, those lower CPU usage, I guess again, instead of two memory group is only one and it's very like seriously small and clean patch comparing to uh alternative or direct patch, which is much more invasive, I think, but I think they could coexist together like if people who want to use completely uh zero copy redo rides, they could use or direct patch in between I think.

B

This patch could also be mapped to a direct just maybe some uh property could switch which for directs use should be used and obviously that or direct zero copy one can be used in some situations and sanctions require it requires alignment, buffers and other things. So maybe when it can do the things my patch could do the things so I think it would be a good part to complete the picture so I'd like to ask for testing the reviews, so it passes the tests, but and I did my tests.

B

That kind of all I can do for now.

A

Cool and I just to make sure I I caught it in for folks who maybe need a little tldr. um The the Improvement here is when you have um set the primary cache to not include the data.

B

Yeah data- that's probably the most.

A

Primary cash metadata and then you're reading a file sequentially, and so it's being pre-fetched, um and uh because we know that it's not going to be going into the ark we can like optimize, reduce the memory. Copying is that yeah.

B

Well, by default Arc for any allocations above four kilobytes of vbsd and above, like half kilobyte on Linux, as ifs tries to allocate ABD which practically chunks it in the four kilobyte chance. So it means we cannot directly map it into game view cache it's a trade-off for using more bigger arcs up to the whole memory size without KVA. You can contention uh this, but since we know that buffers will be evicted very soon, we can bypass ABG allocation, allocate buffers linearly.

B

It's practically a fuel line, change to make Arc do that, it's trivial and after that, there existing code pass already when we are creating a buffer for debuff cache a code just see that okay, we have linear buffer already in our just instead of coping it just directly. Maps the buffers no shares a buffer.

A

Yeah in this um applies only if the data is not compressed.

B

Is that right, yeah? If it's compressed, then it if it falls back and uses traditional ABD cache? Okay, sometimes there is no much place to win well, I could I was thinking. Maybe I could still use linear buffer if compressional algorithm or something would benefit from linear buffer, but I'm not aware about that. So right now, if it sees a buffer is compressed, it immediately fall back to existing Behavior. It will still the buffer still beneficial better than it was before. It still will be prefetched.

B

Just there is practically no extra copy because there is decompression instead.

C

Yeah well, actually, I think. Maybe we do want a linear buffer in that case, because if it is compressed, I I'm quite sure we end up copying it to a linear buffer to decompress it, because all the decompression algorithms can't take an ADB. They only take a linear buffer, uh and so, if we skip making an ADB and just always have a linear buffer for the compressed case, there that probably saves a nun copy as well.

B

So I remember I, discussed with people about a compressional grid, whether we can do compression into a Chunky's buffer, but if it supplies to decompression also then yeah. Probably we could do that. Yeah.

C

I'm pretty sure, right now, if the sort, the the compressed buffer that we're going to try to decompress is in ABD, we have to linearize it before we feed it to uh like lz4 or Z standard to decompress. It.

A

Yeah I think it's true, um but I investigated this several years back and the performance win that I could get by like not linear right by, like keeping it linear um was extremely minimal.

A

um I think it has to do with, like caching in.

B

Fact this probably depends on the cache size. I have the buffer size.

A

Like maybe, with record size equals 16 mag, you would see an improvement, but.

B

It still should be Improvement on memory bandwiz side to your first copy, the additional.

C

B

It but yeah I should probably profile that it should be like one line change to allow that.

A

Right yeah, try it out and see if you can get any Improvement on the performance there.

B

Oh and for right, it should always avoid scorpion, yes, because we are compressing from already linear buffer. Always it's just after the writing. We're doing Corporation copying the EBD that part-time skipping.

A

C

If I understand correctly, the other big win here is before, if you had primary cash equals metadata, you just basically got no prefetch, because we had no way to take care.

B

C

The data before.

B

Yeah, no, it actually I think I was actually who introduced it years ago. I, don't remember how many, maybe five plus long ago, uh when I hit that I blocked prefecture in that situation, because, like preface happens on debuff Cache layer and if we don't have heat on the buffers- and we don't know when to evict- that's why in this fight shall introduced I make made Arc know which buffers are uncashable and introduced, one more cache state for data to be evicted as soon as possible, and they leaked it once per second.

B

Ever since that was not prefetched, I was thinking how to call it I just called Egypt, but then it's the state appeared to be called archivity. They have function, archivity, that's a bummer, so it ended up. Arkham cast.

B

Less critical but I'm sure for hard disk pools. It should be huge improvements.

B

We can compensating latencies and also there I fixed support for small block sub block reads if you're doing sequential 16 16k reads, for example, from 128 kilobyte blocks previously for each read again from disk now it Aggregates that just doing one copy keeps it in Cache for a while after you read all of them's Analytics and again for it's 16k may be a bad example, but if somebody said uh block size or record size to 16 Meg, who knows what the application reads at 16 map granularity?

B

So it should be another big win, I think so, just all wheels from every side. I need reviewers. For that now people are welcome.

A

Cool any other questions about the prefetch moments.

A

That's cool! uh The next thing on the agenda is the Zed configuration.

C

Yeah, so we started a pull request a while back and it's it's complete now, uh so it uses Vita properties to allow you to configure the ZFS event Damon on Linux to basically control the thresholds of how many checksum or how many I o errors in how much time uh before we consider a disk to be the source of the problem and offline it and kick in a spare neat, uh and it's because yeah right now, those values are just hard coded into Zed on Linux, because it was kind of inherited from the FMA or whatever on Solaris, where there was a system-wide way to configure it and on Linux it would just statically set an NV list with those values, but instead now we get them from pervita properties and we'll have a second pull request shortly.

C

That will extend vdeb properties so that the root V Dev, the one with the name it matches the pool name, uh we'll also have a zap. It doesn't currently so um like before we make a zap if it doesn't exist when we import the pool uh and The Inheritance we come up with. There means that you could set these properties at the pools, uh bdev property, so that new diss, you add, will inherit the value you set rather than you know, having to remember to set it on each V Dev.

C

As you add, and remove v-divs.

A

C

D

C

To get some quick review on that to uh to get that merged.

A

A

All right, I think that was all of the agenda items on the agenda. um Other topics or questions today.

C

um Pavel and I've been looking at one of the things we talked about at the dev Summit, the doing I O rate limiting uh per data set and trying to think through which.

C

Thing you would actually want to be limiting, uh for example, if you want to just limit the read megabytes per second per a data set. Would that be? You know at the the top layer, like the the zpl, where you're saying you know the the read and write calls we get from the application? We're limiting that or do we want to look lower and be like you know, just because you did a 4K read doesn't mean that it didn't actually generate a lot more reads from the hardware right.

C

If the record size is larger, then your 4K bead maybe pulled a whole um 128k off disk and then what about all the metadata and.

A

Yeah, well, even ignoring that, like caching, would have a big effect on that like. If, if it is cached, then it would count at the zpl level, but not actually doing any iot disk.

C

Right and you know kind of just like quotas with the compression- it's a question of in that case. uh Should that be the provider's gain that the fact that they managed to get it cashed for you and you get the throughput that you were told you would get anyway uh or in some of those cases is like you know: should the user get the the win, the fact that what the requesting was already in cash um and yeah we're definitely interested in what other people's opinions on that type of thing are.

E

So I can share uh about my banking on this topic, so uh uh in my opinion, we should provide something which is predictable so either we should be consistent and just limit the application traffic or try to limit uh like the disk traffic, which is, uh but when we go to this traffic extremely hard to um this won't be predictable for the user, because we have to then account for all the in inflation related to like grade Z, all kinds of I, even like metadata traffic and stuff like that.

E

So even if you do like smaller requests, it can be inflated, uh and it's it's hard to like predict by the application by the user. uh How much throughput you you will use Etc. uh On the other hand, um when we measure the application throughput. Of course it won't be reflected in this traffic because again, there's a lot of factors, but I think that uh we shouldn't try to do something in the middle. So let's say we.

E

We monitor because doing this on application uh like trying to monitor application traffic, so monitoring just read the right system calls or uh v-ops. uh um We basically.

E

It's more predictable for uh for the user. Let's say: I will use D Trace to monitor my application, so I can see. Okay I have this limit and it matches what I get from the application.

E

uh If we will try to like try to do something in the middle, so uh let's say we do smaller read, but we know it will be turned into whole record size read.

E

So we try to inflate that and account for that, then, for me, as the user, it's hard to uh to predict, uh what's my limit exactly so um so yeah, so so I think our conclusion was to uh to just monitor this at the v-up level and basically uh limit the application traffic and not try to limit disk traffic, which could be much harder and also it's much harder for other reasons, because we are already too late like uh it's already in transaction group, so we may not be able to actually delay some of those requests, so we would need to like I, don't know somehow penalize the application later.

E

So it's it's getting messy yeah.

A

For rights for async rights, that seems very difficult, yeah.

C

Yeah and just you know, uh once all the rights are mixed together into a transaction group, we can't uh make some of them slower and not end up delaying others and so on uh and yeah, like we kind of looked at trying to do it completely logical, like The, Logical data or completely physical, um but even on the rights. It's like the ideal place to rate limit stuff.

C

There is in, like the txt, try a sign or whatever it's called Uh before it's in the transaction group, where we do the throttling for dirty data and so on, because other throttling already takes place there and it makes sense. um But you know that's not at that point. We don't know how much actual physical I o this.

C

This Zio is going to cause, and so we can't actually tune the limit there and yeah like Pavel said, even when we looked at trying to just guess and be like oh you're doing the Zio, and we happen to know that the record size for this object is, is 128k, so we're going to round your read request up. It gets very confusing to the user when their performance kind of Skips all over the place uh versus if they just get.

C

You know the solid 500 iops at the their quota is set to then that seems maybe not as ideal for load balancing but much more predictable and easy to reason about from both the user and the administrator's point of view. Yeah there's.

E

The result I think we cannot predict like no price compression stuff like this, so yeah, but uh but I think from the user's perspective it would be, it would be uh good to know like uh okay I can verify that I can, let's say, use D, trace and see. Okay, that's my application throughput or iops or whatever. So this is the limit I will put so because this is my normal workload. Let's say right.

C

And we're doing it logically, it would map directly to the the per data set case, stats that we have now for the reads and rights and and the iOS there.

A

um If we're talking about doing that the zpl layer, then you could also do it at the VFS there like um and have this work for other file systems, but but it would probably be an implementation. That's specific to your operating system. I was only does anybody know of like this being done.

C

A

Other operating.

C

Systems, so there is one that the VFS layer in FreeBSD called rctl or resource control.

E

But uh from our experiments it doesn't work well, it's now way off well.

C

Especially, it tries to do things like limit CPU and it leads to all the people using up all their CPU quoted in the first couple of milliseconds of the second and then everybody waiting and causes really weird load effects and so on. But uh in this case the the target.

E

Things work is Linux, but in terms of like doing this or the VFS, of course uh that cut that this could be done. But one of the arguments to do it in ZFS is that we could get this inheritance so uh yeah, which we.

A

I'm not saying we shouldn't do it in ZFS, I was just kind of like you know, in terms of the landscape of what other options there are um and then you know comparing kind of how those are controlled and how they're used like are they used? A lot sounds like the one on free there. There is one on previous D, but it doesn't work very well. So maybe nobody uses it.

A

So you know looking at like why why that doesn't work well and trying to improve on it or if there's something like that is used then, but we want to do a better version of it like hooking. Existing applications like or Administration tools into our new thing um would be helpful.

E

Mm-Hmm uh also, uh if we do it at the zpl level. uh uh Well, we want to start from like uh uh from limiting the data traffic, but uh I also uh can imagine that we would like to add along the way, uh limiting metadata traffic. So all the like uh file.

A

Creation rename.

C

Right like I'd, imagine, maybe we can't charge you the bytes that it would cost to do a stat or an MK dirt, but we can definitely charge you an IOP for it. Every time yeah.

E

Yes, yes, so with all of those, we could have like, uh like total counters right, that uh that limit. So, for example, I, don't really care if those are reads, writes or stats or whatever I just care that I don't want more than this amount of file system operations right or something like this.

C

But yeah we were just interested in anybody else's thoughts on you know their use case for this, uh and you know practical considerations as far as uh being able to to make this useful to Beyond just the scope that we have for the first customer.

D

Hey everybody: it's first time showing up here, I've been using ZFS for about you know a decade at UCSF running lots of servers, lots of petabytes uh my use case would much more fit. The model of I'd want to prioritize uh data sets, and so, rather than rather than you know, limit total iops I. Would you know if no, if the system isn't busy, I want to give as many iaps as possible to whoever wants it right? That's my business and so a model where you're prioritizing what's coming in and so filling a transaction group.

D

Maybe you know maybe I could say you know 40 almost like nice, nice thing right, so forty percent would go to the highest level if, if there's contention, 30 the middle level, 20 lowest level priority, you know just something like that might fit better in terms of how I would sell this to my customers.

D

um So that's just all I can provide in terms of feedback about kind of usability or value. To me.

C

Yeah we've had similar thoughts about you, know relative priorities and so on, although I'm not sure how you would Express that in a way that would scale to you know a large number of data sets or users.

E

Yes, like uh with uh with this kind of uh architecture, we would need to definitely the model would be definitely much more complex in order to to monitor like uh the whole pool globally and know exactly what's going on in all data sets and try to like. Based on this knowledge, then uh limit uh limit resources for each data set based on priority.

E

It just much more complex, basically, but not not impossible. I guess yeah.

D

No, it definitely as I was thinking about. I was saying it more as um like nicing. So it's just you know it's. It's gonna, be a relative priority group right and so everything that's coming in that's above a certain threshold. You know essentially comes in through one slot, um but.

E

Just just so uh just so, we uh like uh understand the um like consequence on doing this at the zpl layer. So when we limit uh the throughput or iOS or basically operations, we have no idea if this can be actually consumed by the pull or not right. So uh we don't have this knowledge. We have no idea if our read will turn into like multiple reads or our right will turn into multiple rights Etc.

E

So we cannot, uh so we can easily uh like send too much or too little, but uh uh and it's hard to it's hard to tell so in in ZFS. There is already throttling to uh to to provide this back pressure if we just try to send too much, but that it's not based on like priority of the data set, but uh for this it would be hard because we we don't really know the pool limits and we don't really know how our application Level traffic translates to disk traffic.

E

C

um Right, like Matt, said: if it's a cash hit, it turns out, it doesn't generate any disk. Io.

B

C

B

From other perspective, I think it would be more useful to track uh from the original separation between application, uh stats and I disk stats. I think uh we obviously needs to get closer to disks, but I'm thinking, for example, in case of reads: if somebody tries to do more iOS and he has uh guarantees for uh quality of series four, we could just marks those IOS as lower priority.

B

uh Just reduce some priority level, so somehow else move it to their iOS to scheduler, zero, zero scheduler and then that layer de prioritize them to run slower for right. Obviously it would be more complicated, uh but then again we have our protein mechanism.

B

uh We share it now base it only on size of of dirty the amount of dirty data in Arc, and we already have a problem. If we have two applications running one in bigger blocks, one in smaller blocks, a smaller one will be much heavier penalized than uh bigger blocks. We already need some sort of redesign there, so if we could add into the same function uh returning delay time uh or other additional factors, not just all buffers are put into the same linear queue and then respectively, wait for the time.

B

Maybe we could then just depending on priority, depending on how much user written in a previous X seconds minutes whatever you'd get different coefficients in that function.

B

So if we still get full throughput of the disk, so we'll get always get the same amount of dirty. Arc is just somebody with a lower priority who already went over. The quota can get throttled heavier.

C

So, instead of a quota, make it more like a reservation where.

B

C

The first x iops that you do go through at the normal priority, but anything over that gets de-prioritized well.

B

That's: what's the cellular ability operators do that I! Guess that's everybody who who wants to not just limit somebody's treatment, but do some fair share? That's that's the approach, I'm, not sure. What's the point of giving exactly X megabyte per second, if we still have one reason, it's idle! Why not?

B

Because we have some background processes like Scrub, but maybe we could account that somehow to.

C

Nice and interesting slightly different, take on it where we just flag it as lower priority and rather than installing it.

C

Although I guess it means coming up with more queuing mechanisms in the Zio level to to sort in and prioritize the cios. Although I can see where that'd be useful in a couple different places as well.

A

um How were you guys what use case were you thinking of for the strict quota which.

C

A

C

uh Multi-Tenant thing a bunch of different uh lxd containers, basically and wanting to make sure that one Noisy Neighbor doesn't uh tell everybody and just to keep it simple. Just we'll just set a like a megabytes per second in an iops per second limit on each of them, uh and that will keep any one person from dominating the system uh too much and not too worried about fair share or anything just that one person doesn't make it terrible for everyone.

A

And so you would like, you would kind of have a rough guess of like well. I know that you know the system can sort of do like two gigabytes per second of I o and I have like a hundred.

A

You know, um clients and so I'm, going to do and I'm going to over subscribe, like 10x, maybe so I'm going to do give them each like.

C

Each one gets 15 megabytes a second or something a Meg.

A

Per second or whatever, yeah yeah, so it'd be very kind of hand. Wavy and you could you know you get into situations where you can't get your whole quota, because it's over subscribed by everyone else. You get into situations where the systems underutilized, because, like I'm, you know a few people are hitting their quotas and nobody else is using it. But for your for your use case, those are all kind of fine yeah.

C

And you know the the goal was to not make it overly complicated, where it's going to take too long to build, although maybe doing it more like a reservation rather than a quota where you know, when you go over you just de-prioritize instead, uh maybe isn't making it too much more complicated yeah. It.

A

Depends on what the.

C

Queuing looks like yeah.

A

It depends on how they're de-prioritized.

C

A

But that does seem like uh not too hard wait to like describe what you want, but.

E

The prioritize from what they understand the reads right right, but uh can we do that the same with rights.

C

What's right, I think maybe we can do something like you know, don't assign it to the transaction group if, if there's anything else of standing still or something.

A

C

A

Easily it's hard to.

E

No, so then it's quota right, if you don't assign to the transaction group.

A

Or it's like you get really super slowed down. You know where, if you're not getting assigned to the txt until the next dxg or it ends up very hacky, where you can do select something like. Oh, if the txg is at more than 50 of the max dirty data, then I'm not going to add any more of your iOS or something like that. But I think it'd be hard to come up with anything that has kind of consistent performance.

C

A

C

That happens well, if we're prioritizing like as we work through the list of cios, then if we're getting to assigning to.

A

It so the transaction we've already for rights. You don't have the Zios ah right.

D

A

D

um Doing all of that stuff, yeah.

C

A

Could sort of that you could do it much better for synchronous rights right because synchronous rights, at least some sort of I o, has to happen in open context.

E

Yeah, but uh they wonder if, if we cannot, um if we cannot do that for all kinds of iOS, if it makes sense to do it at all, because if, if it will do it for like most of the iOS but still asynchrites, we cannot really address them and uh and one consumer still can send a lot of Rights and just saturate the pool then re-prioritizing the rest. I, don't know it doesn't really address the problem. So.

E

Under utilizing the pool um yeah, that's really concern and I wonder if this is not just a balance of being able to uh utilize the pull fully and being able to provide productive predictability to those all the tenants on the system.

B

I think the most terrific way do it on a dirty buffer throttling place. It's just currently throttling calls done in one sequential queue if we could somehow split it that single queue between several, what we call Consumers some entities who should be applications that should be balanced.

B

Okay, you, your specific, uh whatever client, consumer, 10 of dirty, Arc, buffers start trolling, or something like that. Like maybe Implement, two separate drop links, one system white, just not overflow, the memory and one more user specific which start trotting earlier when it got 10 percent of goatee buffers consumer just by itself.

B

So, if nothing, if nobody else uses a pool, it will still get most of throughput, or at least not hundred percent. Depending on how tight do we limit it and pull characteristics, but most of pull Bando is, but if there will be other users who will push data into Arc more actively, this one will obviously get penalized heavier.

B

Discussion how to turn it into exact numbers if it sends and so on, but.

E

Having those skills like per data set seems.

E

B

Oh, what shotgun gravity of all this quotas we are discussing that are expected? Is data set or.

C

Data set, although with inheritance like like this quote, is basically where you know: if you set a quote on the parent, then the children have to respect their quota as well.

B

And that's actually, one of lock contentions I saw from my benchmarks the high hopes so yeah, something better would be good. Yes,.

E

So we are considering like not doing inheritance initially, uh but of course it would be nice if you could have like a let's say group of customers that have some kind of quota and then each customer have smaller individual quota, but you can do over provisioning in this just for this group of customers. So.

E

And, of course, all the all the data sets created below for this customer uh are treated as as a one. Basically, a single limit s.

E

But doing this at the um doing this for data set instead of like process or user or uh actually uh remove some problems, let's say that uh you would like to slow down uh withholding range locks on files.

E

This should be okay, because all the customers of all the users of those files needs to be slowed down. If we would do the the limiting like per per uid or per jail or something like this and potentially you can have users with different limits trying to access the same file, then you you have to do slow down differently.

E

But if you do it like this, then yeah you have to you can easily slow down all the consumers of the of the file, because it's based on data set limit.

A

Yeah seems like a tricky problem overall yeah.

E

Yes, it is tricky problem and actually based on on some of the attempts like in FreeBSD with rctl uh I. Think it's uh I I personally, would prefer to have like something which is predictable, measurable, like something that I can.

E

Easily figure out uh or or test if I set this limit I know what exactly does it mean.

C

Yeah because I think, especially from the administrator or user's point of view uh getting random, speedups or slowdowns uh because of things you don't control like the right, amplification or metadata or caching or even compression, uh is probably quite confusing uh versus and I think in most cases of a multi-tenant situation. Where you know the provider is doing this, they would like to keep the wins from compression and caching to themselves uh to get more over provisioning or whatever, rather than necessarily passes on to the customer.

C

A

Yeah so I mean maybe the next steps for this are to write up like what exactly you would propose. I mean I like the idea of coming up with something that's like, maybe not the best thing that we could imagine, but something that is easy enough to understand and easy enough to implement that it's useful um and uh and then maybe getting feedback on. Is this really useful right?

A

We can build something simple, but if, if people don't use it because it's not really what they want, then you know this wasn't really that useful right, like some people, might have a hard requirement of like I need to be able to get the full like if I'm, the only user I need to get the full full throughput of the system um like if there's no contention, then I should have the same performance as I did before, um and that's kind of a different requirement versus like what you're talking about where you would just set a quote on each user or each data set.

C

C

Yeah definitely interested in feedback from more people to to see how we can both keep it simple and useful.

A

A

um Any other topics for today.

F

uh It's me Tino here from Germany I. Just wanted also some feedback for my pull request for the chart to reworking um The Benchmark numbers are nice and uh I don't get any reviews I have no idea what what to do. Next, it's a bit more! uh It's about 65! So it's difficult to keep it always rebased. It's.

F

Maybe you can take a fast look and and assign.

A

F

A

Yeah I, just looked at the numbers in your PR I mean the numbers all look good thanks for Gathering. All that data on those different systems um is the.

A

um Is the old Implement so that you're changing the implementation of shot two which is used by the shot 356 in shot 512.

A

um Check some algorithms is that right.

F

uh Yes, this is the request. uh I put a link into into the the meeting chat and you will get the Benchmark numbers on top of the pull request.

F

Then the numbers are on on an army Horizon system on some Intel system and and make many Minds when, when you scroll down also rinse uh did some some more tests and so on it works it shoots it's about. It's all open, SSA code, Apache, License, assembly stuff, is also an open SSL.

F

So I think it should be okay, but I don't get any review or some more comments on it. So I just want to ask.

A

Yeah, it looked like um rich, took a look at it, a while back right.

F

That was the first uh thing. Maybe um the the real implementation was done, I think in in September the before this. There are some some some points that that needs fixed, uh and these are fixed to go currently so.

F

The test from the CI are always okay, I think in in the last time,.

A

Yeah I think we need to figure out like what what review.

F

Some more time, so it's not uh for five minutes now, but I just wanted to call.

A

Yeah um and I think we should try to figure out like what level of review do we want to see for this. You know there's 28 000 lines of code. Presumably most of that is the stuff that you just copied over from um from.

B

Openness right.

A

Yes, I: don't think that we're going to review that um so.

A

um I mean I, in my opinion, I'm kind of like that's fine, we'll just trust that they did it right, like that's, obviously used even more heavily than ZFS is so um uh we should review like the infrastructure stuff.

A

The you said that you changed some of the interfaces.

A

Yes, um so we should take a look at that I'm just kind of skimming through the code to see you know it doesn't look like there's a ton of that.

A

Use this sha stuff yeah.

A

um So I mean I, don't I think it's not a tremendous amount of work to review this. um We just need to find folks who are interested in. You know check some algorithm stuff. Let's take a look. um I haven't looked at this stuff in a long time, um but I think that we I'll definitely ask someone to maybe Mark um to bug people about code reviewing.

A

um It looks like we have a couple of candidates rich and um they see comments from one other person. Oh yeah, um Sebastian, brain Slayer. um We can ping them.

F

uh He uses it already. When did he wear out here? It's a router based system for forearm arm, seven I think.

A

Yeah I think um him and I think Rich would probably be able to do a good job on the center. He's done a lot of work on the compression algorithms um and he might be interested in this as well.

F

Okay, just wanted to ask.

A

Yeah thanks a lot for this work. I think um are there before and after.

F

When you take the Benchmark numbers of the break three hash supporting uh stuff a moment, I can I can put it on.

A

Just because I may have to motivate people if they can see like. Oh you know this actually improves performance or on certain things. Yes, a lot a.

F

A

Because I, don't remember the like the shot and eye stuff is that new yeah, oh yeah, that.

C

One depends on the CPU having the instruction yeah.

A

But I thought that we didn't have that before foreign.

F

Implementation of course, and then you can yes, so let's see the difference, of course.

A

um So in the ones, the old one uh it has the shot, 256 and shaft 512 generic, but not.

F

This optimized.

A

Ones are all of those optimized ones new, or did some of those exist before.

F

They are on you yeah. The old one got also only some some open Messages, a generic optimism, optimization for for I. Think only uh 64-bit systems hinted.

A

All right, so, basically, this is, you know we can think of this as adding the um you know, the.

D

A

The what do we call it? Not SSE but um vectorized, adding the vectorized instruction, um optimized versions of shot, 56 and shot 512 or.

F

For a lot of platforms, of course, yeah for.

A

A lot of platforms, including you know, the AVX, the shot and I and the um arm V8 CE stuff, which looks like great performance.

F

Also Power, 8 and so on are implemented. Oh.

A

A

So yeah I think that that I mean that's a huge moments win um for folks who are using shot, 256 or 512 um on those platforms which are you know the most commonly used platforms, so uh I think. Maybe we can um try to highlight that performance Improvement to motivate folks like hey. We need code reviews, so we can get this in so we can get. You know what is it like?

A

F

The VIN on top more than 5x performance I, could make better I could I put the winnings on top of my employee requests, so I think it's okay in this way for motivating other people.

A

F

A

All right, thanks, um I, think we're almost out of time, any any volunteers or I'll just um hope that I can bug Rich into doing it. He's not here.

D

A

Defend himself so all right cool thanks uh any other topics as we wrap up the meeting today, foreign.

A

Which is after the new year, just after the new year, uh January 3rd, um see you then bye.