Node.js Benchmarking Working Group, 10 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Node.js Benchmarking WG Meeting

Description

https://github.com/nodejs/bootstrap/issues/114

A

I'm having some cold.

B

Okay, so welcome to the benchmarking nodejs team meeting for September 10th 2019 will sort of follow our agenda. We don't have too much on it, but I guess and we were having a conversation on compression and some of the work that have done so I think will add that to the agenda, um but first like any announcements that people want to share in general before we get started.

B

Nope, okay, so let's, let's get back to that I'll just give a little context for people who are watching the recording lunamon mentioned that his team had been doing some work on some additional NPM modules. To do compression and I just mentioned that you know: we've done some experimentation at IBM to see. If we could, you know, modify core node.js, so you could, you know, specify a shared library which would provide an alternate Lib library to do compression.

B

We just hadn't gotten enough cycles to do the benchmarking, to show that you know when it was off. It was an acceptable overhead, so we were talking a little bit about that. I think Gabriel! You! You had something you want at it and we were just.

C

One thing so in the in the in the fast use case right, III I, agree that that that constantly that having having like an indirection on every call into into deflate is, is a penalty, but there's there there is even even in fast. There should be a break-even point, because after all, you called the function so as to compress stuff. So if the actual payload code that does the actual compression is much much faster, then then, then at some point it offsets the overhead.

C

So there is a point at which at which you could have just used, regular Zed, Lib and gotten, or the built in said, they've been gotten the same performance as as you do by by paying the overhead and then doing the faster Zed live past that point, it's always better to use the faster Zed live. However, you quickly approach the point where the function is taking long. So there's that so there may be like a nonzero window between the break-even point and at the end of the useful lifetime of the function.

C

That is a performance benefit right.

B

C

Direction for C.

B

I'm not personally so worried about the weather when you turn it on, it makes sense in all cases cuz some cases it won't and you just don't turn it on. My bigger worry was that when you integrate it into the codebase so like what a'mma has now is like a separate library that you can use or not use, but you got to change your code to use it right.

B

What I wanted to see if we could do was like integrate it directly into node, so on the command line, you would say, use this shared library instead, but that's gonna. That will add some overhead and what's important. Is that we'd be able to show that, hopefully that that actually, oh, that the overhead? That's there is small enough that it doesn't matter yeah.

C

B

Too big people are gonna say well, no, we shouldn't include it because it's you know, gonna hurt people and we can't actually turn that off right, yeah.

C

Yeah, at the very least or in the ideal case, we wouldn't be providing people with an additional knob that they can turn right now and for some people it may make sense to turn that towards an external Zed, Lib yeah.

B

That's what we had like what we experimented was with like a command line that says, use this library. If you don't, if you don't, you know the default, is you don't specify that and internal to node itself that basically had a check that says when you made? Is that a call to any one of those Ebla calls that says? Could you enable this no just call exactly what it would have called before.

C

B

C

B

An if and a call yes.

C

It does yeah yeah, yeah, yeah and.

B

So you know the hope would be that if you're actually compressing something, that's nonzero work right, yeah.

C

Exactly that's right! Yes, so.

B

So you know, hopefully compression is and is enough like enough of a piece of work, that a call overhead is not actually that significant yeah.

C

And especially, a decision overhead, if it's just a simple as a single decision, they're managed all right, because at that point at that point, it's pretty cut-and-dried right. Because, okay, yes, you pay, you pay only for the decision and then and then the fact that it's an indirection after the decision has already been made is part of the performance of the library. You can just bundle that in with the with the performance of the actual compression, you know the price of the indirection yeah.

B

Cuz I can't remember: I'll have to go look at the code. I can't remember. If we I don't think we used an indirection table because that basically means you always have an indirection right. Exactly.

C

Yes, yes, you don't want to do that. Yes,.

B

Yeah yeah I think we just added the. If that says you know if on go over here, otherwise, just continue on the way you were.

C

Exactly yes, yes, so then you can find you can find the the the overhead to to the case where, where the flag is on plus the price of checking the flag, exactly.

B

A

Just library is it compile time you specify the information, no.

B

It's run it's it's basically run time. It's like a dash dash zeal, apply Lib, you say equals to this and it's a shared library. So if you specify that it will dynamically load it and you know, look up all the read all the pointers. You need to do the interactions or to do the forwarding. Basically so.

A

Another thing to consider I mean perform is definitely one thing. Another thing to consider is the security because of this dynamically sharing with some library. How do you? How do you make sure that it is doing the right thing and not because now you can link with any library as long as there is an API there, you can tell.

D

B

The command line that this is the one I want to use yeah.

E

B

E

Think, depending.

B

On your environment, you can point that to a place where you know only, you have access to write a library right, yeah.

C

I mean I mean we already.

A

C

Already loaned we already low lots of code, I mean once once the note process is up and running, it is assumed to be secure, so security is pretty much deferred to the administrator. No.

A

But the current all the libraries are statically linked in the binary right in node binary. There is no.

E

A

The system, libraries, but all the lines really everything is now we are static linking with with a binary it. So that's that kind of opens that vector right for the security opening. It does. That's the next part.

B

There there are options you can build, that that do dynamic, linking the thing that the objections I've heard in the past is that you don't want to like look for a library and automatically load it if exists in a standard place. Yeah.

C

Exactly yeah, it's not good yeah.

B

Then people can basically like, without the the person who's running the code at the library and it'll run without and be used without them, knowing as as specifying the library path on the command line. I think addresses that in that, if I can change, your command line, I can run pretty much any code. I want anyway, yeah.

C

A

Who provides you so think about in fast environment? When you upload the function, you specify normally, you specify what nor, unless example, is the JavaScript code. You say this is where my node, this is the version of know what I want to use, and this is my function to run now now you have a regional component to upload where my library is going to be, who provides that library, yeah.

B

So so that kind of case I might be that you sorry you're out of luck like unless.

D

B

Infrastructure says well one of the options I want to give. You is faster yeah and then.

C

D

C

You can like certain providers, they allow you to customize the environment right in which your function runs like like ahead of time. Right, like you, can have like like an execution environment, and you can specify I want this, and this and this right so, like you, sorry, but I'm, not sure how much granularity you have over that. You know also another thing I just thought about is since we're talking about the dynamic library we could potentially.

C

We could potentially render this as an add-on right because add-ons are dynamic. Libraries too, it's just here's the here here, here's an add-on: it's got some symbols, here's the symbol table go around with it, you know and then, and then we, instead of having a command line, we would be. We would be using the require mechanism to load the library, but otherwise it would be. It would be the same. You you load the library and then you pass off the the function. Pointers to to to note core and you're, basically telling node core.

C

You know, here's a here's a you know here: here's a bunch of function, pointers they satisfy this particular use case right who's, not the ones you would normally use and thereby you turn on the indirection. You know yeah.

B

That's true, I think I think when we looked at the other one. It's be interesting to think all through this. You I think you're right there. What what I think! In our case, we had like an existing Z library so and I think there are existing Z, Lib libraries, so it's in in in some cases you just don't want to have to even write a wrapper right. You just want to be able to oh.

C

For sure yeah yeah yeah the existing.

B

C

The reason why I brought this up is that is that the in the in the fast case, you may not have control over the node.js command line right as the fast infrastructure may support loading native add-ons, because that's a fairly popular sort of thing to do with nodejs right, in which case out control the command line. You could still tell nodejs to use this particular binary at runtime, but you would use a JavaScript interface to pass it right.

B

C

And and again des that, the security would be at least as secure as as loading a native add-on of any kind which has security implications, as we all know, but people still.

B

C

That this would just be another add-on in that sense, do.

B

The faz environments have compilers generally installed I, don't.

C

A

B

Would be the one the one wrinkle is. The other thing like for me is: some of them are interesting like it's also. Sometimes it's gonna be tart to the height to the hardware. So it's not purely a software thing, you need a card or you need. You know some Co accelerator like in.

D

B

Case that you mentioned, you need to probably have a certain level of the instruction set available right, yeah, yeah.

C

Well, I mean, but that could all be checked right. I mean we're now we're talking about the native add-on right, the native, add or I can actually check these things because it has fairly low level access and a.

A

Little text any edit or checks you have, it adds the instruction and that time is pain. So there's order, no more man, anytime, you add you, may you can make it as efficient as possible. You know all secure and everything.

D

A

The short-lived function and that's a if one use kids, which is growing exponentially now it within the JavaScript world in the node.js world,.

C

Okay, okay, but let me ask you this: for compression in particular: how short-lived can you be before it's useless or.

A

Think about look at our all the micro benchmarks, all the input we have for the compression they're, so small. None of them are realistic. Right, they're they're, like what 10 bytes 15 bytes when naughty 101 K bytes he's just running n number of time, but the input itself is very, very strong, very smoky.

C

What is it, what is it in real life like how do people use compression in fast? You know what are the chunks that they are sending as.

A

A function so think about it as a function. So what is going to impact? It probably will not impact the execution, the function, what is going to impact the startup time, which is nothing but the node core startup time so.

E

A

Start up time takes place. A tech 30 millisecond is going to take 40 millisecond, maybe 50 minutes right.

C

You, if you add, if you add 20 milliseconds to the product time but but but you you double the throughput from traumatic from from 10 megabytes per second to 20, megabytes per second after you've paid that price. It may still be worth it right because the function can run for multiple seconds and it can do a lot of compression in that time. All right! Yes,.

E

C

Much compression do you do in a fast environment with a single execution of the function. You know how long does that function run after after the whole thing starts on the.

A

Fast itself is meant to be run for the show very, very short-lived, so you don't so you are not supposed to run a huge function right, not like a web server application. Think about.

C

A

Small think about anagram or think about images.

C

Yeah, okay I get that okay, but then what do? What can you accomplish in such a short time with with with compression? What can you compress like the PNG image or or or you know what? What sort of realistic use case is there for running a compression algorithm for 30 milliseconds yeah.

A

So what I was trying to my point was that it's not that there are no use cases. What my point is that there it'd be increase in the startup time, the node runtime. So when we think about the load performance, we are not thinking about the compression algorithm itself right. We are talking about the node dodgiest runtime startup time, and that would be is. Can we make it small enough? So it is not. You know impacting the startup time we already I, think current or with latest run latest version.

A

We have a 30 milliseconds startup time on on xeon hardware right. Our goal right now is to get you to 10 milliseconds. If that is a goal, and if we add this code, I will go to achieve that goal or not achieve either and again. This is nothing to do with the actual compression or crypto or.

C

A

Right here, okay.

C

But but since we're talking about compression right, if you, if you look at the, if you look at the picture from from the end users perspective right, they can compress. Let's say they can compress 200k in in 20 milliseconds with the infrastructure, as it is now startup time and everything you call the function, it's a cold start. You spend it 200k. It takes 20 milliseconds to compress, and then you get it back right now.

C

If, if, if, if we can make, if we can make a change so that in those 20 milliseconds, you can compress 300k right, even though the startup time is, is much greater, it's a much greater proportion of that.

C

You know it's still from the end users perspective: okay, I've sent 300k it's taken just as long right and I got it back. Okay, it's one and a half times faster, I, don't care that the startup time is now a greater proportion of the total time spent running the function. It still wanna know I'm faster from their perspective. Yes,.

A

So think about in our case, where we are not doing a function not doing compression, but now there is this. Some code is going to get executed as part of the startup time, which is nothing to be the compression of stuff. Oh yeah,.

C

Okay, so you yeah, you don't want. Okay, you want it.

A

Slowed down in the case which which not using the compression but and.

C

If you, if you, if you do this as an add-on, then you're not slowing that down the case when you're not using the compression, because it's an add-on only those only those functions which require it will actually be slowed down by it. Similarly, with with Michaels interface, only those functions which have the the command line parameter will be slowed down by it. Who's.

A

Going to do require Lord.

C

Lord core I require is part of the function definition. You call require this superfast Zen, Lib and then and.

A

So what I was saying that who's going to call so in typical application? The user application calls require in North core. If you take away the Z leap and put in external library, as we were talking about, then who's someone needs to do a require right. Maybe it is dependent on some command and options, but there is that cool which going to check for the command-line options. Well,.

B

That the difference to is that like I, think Gabriel was suggesting a require, but the require may actually be too late. Yeah command-line option can be early enough. Now it may not matter. You may be able to switch engines further along yeah.

C

Yeah yeah, almost certainly right, I, mean think about it think think about think about nodes ad lib right. Let's say you have just for simplicity sake. You have a global static point or to deflate right yeah. It's it's set to no. Normally it's set to no right. So so you basically say: if Global static, deflate, equal, equal null, then deflate right and that that's their that's the flag, check price you're, paying no matter what right.

C

But if it's, if it's not null, then then you know star global static, deflate right, so then you're paying then you're paying the price of of loading, the library and in directing, because it's not statically linked and all that right. But but that's that's what it looks like from the deflate perspective: you're only paying that price, if that, if the pointer is not not okay, otherwise you're paying a really tiny prices of comparing something to zero right. It's like one more instruction right, compare compare that global! Okay! I'm!

C

Sorry not one more instruction, because you know you have to fetch it put it in the processor, then compare it and then and then get the result. So, but but it's only like it can't be more than like ten instructions to check whether that global static value is null or not.

C

Now how that globo static value becomes non null, we have several choices: several design choices it either becomes non now by virtue of the fact that something was specified in the command line and the price of loading the library was paid before the first call to deflate was ever made or I require was present in the definition of the function which caused us to set that value to something non know by some as yet undefined, no jsapi, but either way. The result is the same.

C

It's it's now non null and therefore you use that instead of using the the statically linked deflate, but you only do that if the customer said do that right.

C

Otherwise, no note note core the only the only addition in note chord. The only impact is those ten instructions to compare that pointer. To know I mean we could even say this is probably wrong, but we could even say like if likely that a dot equals now, then then, then, what.

B

We want to do something like that yeah, so.

C

So cuz, you know for most use cases, it's gonna be no, so therefore you want to wrap it unlikely to do to help out the I guess, the branch or whoever gets fat.

B

You want to make it as cheap as possible if you haven't turned it on exactly.

C

That's so we can even wrap that check in in in and if, if likely and and and then it's it's it's up to it's up to, though it's up to those who, who are heavy users of compression to decide whether paying the price of the additional startup time of the additional indirection during each call is worth it for them. And you know we can run the experiment to decide whether whether after we add all this, there are any use cases left for whom it makes sense.

C

You know, because I I can't imagine I can't imagine that that it won't. It will never make sense right, because, if you're compressing anything significant, your function is going to run for at least 300 milliseconds right. You compress anything useful in this world right and so so.

C

Okay, instead of the 30 millisecond startup time with within those 300 milliseconds, it's gonna be a 50 millisecond, startup time right, plus 250 milliseconds of useful work right, but that's still 250 milliseconds of hardware-accelerated work, whereas before you had 270 milliseconds of software work right, so you can still achieve at least a couple of X there. I can't imagine not, but anyway, we'd have to run the experiment. Yeah.

D

So Michael, yellow volunteer here looks like.

B

So basically, yeah I mean I I'll dig up what we had I, don't know if it'll still it probably won't apply to it won't apply to note, but it you know, I can dig that up and then you know the real work. I think is, you know doing the experiments to say. Can we show that it's not not a big overhead yeah and you know getting it landed and stuff, because I think as long as we can make it show the case where we truly don't believe the overhead is going to be a problem.

B

It unlocks a bunch of opportunities for people to do things where it's not like. You have to change your code. It's like okay. You've got some existing code, even HTTP, which I think does some compression and decompression right behind the scenes. What benefit.

C

Yeah yeah, oh for sure, yeah HTTP is a huge huge candidate for zenlea, yeah and and I. Think I. Think the bigger picture here is also that that more and more of these, these, like these calm infrastructure components like like compression and hashing and and and these kinds of things they are becoming increasingly hardware-accelerated and and and and some of the some of the paradigms that that we've been using so far, which which is pretty much to run these things in sync and to use a thread.

C

If you know if it becomes too heavy for the main thread.

C

This is going to change, because hardware can have its own threading hardware can be implicitly asynchronous right, so and and and and- and so you you get a synchrony without having to do your own, threading and so forth and like with with like when you have like what is that called Cindy right single instruction, multiple data, you know that isn't, there's also implicit a stream right, because you have no benefit if you're doing it synchronously, because you're feeding exactly one data and then asking for the result right and so so so the so the so.

C

The concept whereby you have streams that run in parallel has to go much deeper into the stack than it has so far right. So far, streaming has been has been an abstraction imposed upon what is otherwise a synchronous algorithm. But now the algorithm itself becomes asynchronous and unable to accept multiple streams, implicitly without any, without any sort of abstraction on top.

C

So so, for this has to be woven, deeper and deeper into the stack right and and and in add to this, the fact that that there are different kinds of hardware like you can do, compression with FPGA. You can do compression with all of these have their own libraries, some of which may may you may only get the benefit from if you you expose the streaming interface rather than the synchronous interface. So so so sooner or later you know we'll need to. We need to accommodate this increasingly heterogeneous sort of computing landscape.

C

You know where you can, you can have acceleration for for for domain-specific applications and that's.

B

Certainly why we were looking at this, was we already had hardware which had acceleration for Z Lib, and it's like? How do we make useful for for for people using node without having to change.

A

B

A

B

A

Don't work with our I said library what we, what I try to do, keep the JavaScript API very much similar to how the node.js silly provides random Iying, but using the npm install mechanism and it's very easy to integrate that way for sure, if you use just like that, so as long as you keep that javascript api same then underlying you can take, you can add any library right so.

B

That's probably good as long as you're the one calling z lab right, if you didn't have any dependencies or anything that are using it, it'll.

C

Be harder to get.

B

Them to you or library versus yeah.

C

For sure yeah yeah yeah exactly yeah, so yeah you're right this. This only applies to to the to the to the calling project. Yeah yeah.

A

And so well, but one good thing about its explicitly mentioning that way is one good thing is even if, in the fast world K, if you take the same code into the fast world, you can then specify when you're selecting either a machine or infrastructure. You can specify the type of hardware your to run this function, because I'm using this specific npm install package, which takes advantage of certain type of hardware, yeah I,.

B

Know you could definitely yes, you could it's, it does give you good flexibility, it's just the how you get people to use a challenge. Sometimes yeah.

A

It's a marketing, you need to kind of promote it right. The people know about it in the C of NPM packages for sure.

B

A

B

That sounds good I, think we've got I'll, dig up the code, and then it sounds like Gabriel you're interested, because we're we stalled out was doing the benchmarking, which is as how it was how this all relates back to this group. Is we sort of stalled out and doing the benchmarking that would help us show that it wasn't? You know that it was okay to land. It.

C

Okay, now before before we give up on this topic or before we move on from this topic, if you do, if you do provide that code, so I can look at it will I have the hardware on which to actually test it, because now.

E

You know if I do my left, it'll.

C

Just fall back.

E

C

Serve as like, as like a proof of concept like an inspiration, basically for how to how to swap out Zed, live on the fly, it should yeah like.

B

After you know, it's been a couple years, but you know the goal was that you could specify any one of several z, Lib libraries, okay,.

C

Good good, they you know, okay, you might have.

B

To write a little wrapper or whatever, but basically it should be like it was meant to be pretty much compatible with the standard z lab libraries are out there, so I think there were already ones that use that were provided for Intel hardware accelerators, like I, think there were some cards that Intel had so you might have some existing Z Lib libraries that it actually tested out with as well yeah.

B

Okay I mean a simplest case. We could basically make one which wraps the existing zealot, that's used in know right so yeah.

C

B

Offloaded and you can just test that it won't be any faster, in fact, it'll be a little slower yeah.

C

All right, I mean I can always test like systems ad lib, and we already kind of map that, but we have it at build time rather than run time right.

B

Yeah we could test exactly. We could test that one well, I really thought the initial testing would be is not convinced, not not convincing. You can make it faster but convincing that yeah when you don't turn it on there's just no problem.

C

Yeah yeah: it's not much yeah yeah, there's no penalty yeah, because.

B

I think, as long as you show that case, and then you say there is hardware that can be beneficial to our customers at particular times, I think people aren't going to say. Oh well, don't do that. You know convince me, you really can have a benefit. It's acceptable yeah as long as yeah we're heads except well now, internally, it's it's very good to be able to say here's a tangible use case. That's accelerated by four times right. So if we couldn't.

A

Come up with that.

B

That'd be great I gotta, just plug in here.

B

Okay, so, although I might take me a little bit of time, but I'll dig that stuff up and we can start a discussion on that.

C

Sounds like fun, yeah.

B

Okay, the other thing on the issue on the agenda was to 25, which is investigate time taken number of benchmarks in core yeah. It's gonna open that yeah.

C

I, have a I have a spreadsheet about that that I shared with you, we were gonna, we were gonna sort of decide which or help decide or put it to the community as to which benchmarks are more important. If I remember correctly, it's been a while well.

B

It was around figuring out if there's a smaller set that makes sense for us to run like we've talked about running it some periodically or once like I forget what the intervals we were talking about, but basically trying to have some subset, which could run, or even you know, a collect data so that you could go back and look at it when you needed to yeah.

C

So what do you think have you had a chance to look at the spreadsheet I'm, not sure if I shared it is.

B

It yeah is it something you can paste into the issue, yeah.

C

It's a Google spreadsheet, but but only you and who don't have access to it for now, which is why I didn't paste any big issue, because it wouldn't be very useful to anybody else. So I. You should have like an invitation to edit when.

B

Did you send that out.

C

Let me see here: well, I, don't know what, because it sent it from the from the interface, but I can send it again as a link.

A

Maybe we can add that link into the issue itself, so that I.

B

Would just paste it in there anyway so that we know where it is and then, if somebody wants oxide, if somebody clicks on it and says, hey I want access, it shows in trust, and we can.

C

Yeah and give them permission yeah. Okay, that's a good point. Okay, let's do that? Okay, 225, you said yep.

B

Here, I can paste the link in the chat. If that helps.

E

Many desktops will be there. We go okay.

C

C

Yeah, of course, yeah I hit paste and, of course, please see the link to the issue.

E

So basically, like.

E

C

C

Okay, so it's gonna be pretty terse and cryptic for now, because I I was I kind of wanted to run the spreadsheet by you to see. If this is the structure that we want for it yeah so there you are.

B

Right, okay, so.

C

Yeah this this is basically it then and, and- and you know, I was I was actually thinking of of chip talkers, you know TSE meeting spreadsheet. Something similar, you know is, you know, call it votes, but I might call it like you or.

B

Something like this, what I'm thinking is what we want is like, which ones do people use actively yeah cuz? Those would probably be a good subset to have right, yeah.

C

B

So it could be like, can you mark, you know, try and get as many people to fill out. Have you used this or not? The only question like this, isn't that this isn't a huge list like I'm one? Well, it's pretty big I'm wondering if this.

C

B

A SurveyMonkey survey or something oh yeah,.

C

Okay, yeah cuz.

B

C

B

For us a lot more easily right.

C

Yeah yeah yeah yeah yeah, that's a good point. Yeah me I've, never done what a loser. I can dart, yeah and and I guess they. They aggravate for you and stuff yeah.

B

Like you would basically get a results that say you know: 20 people said yes to this one. Five to this one yeah yeah.

C

Cuz otherwise this is this is an exhaustive list. As of master, so right people should see their benchmarks here if, if ever they use them, there's even one that that that isn't yet one master, it's the one for the for the weak refs. Where is nappy, because you know I'm working on that weak references as as cleanup hooks, so I even have a benchmark here. That's not really a benchmark yet, but anyway, I can remove that line yeah, so so, okay, so then I can turn this into into a SurveyMonkey survey and go from there.

C

If I figure out how to do that because say.

B

We can, if you you know, if that's easy, it might be easier to just do that, the other. We can also ask the foundation to help us because they do have an account to do some surveys. Oh cool.

C

The other question I have is so like the voting system that we should use. I mean I know this is super super splitting hairs, but should I just make like a set of check boxes and then people check the ones they like or should I do like a score like from one to ten or or whatever, because I think you can do that or you can be like a read. Disagree somewhat agree all that stuff. The.

B

First I mean that is there I mean the thing that popped into my head was like. Have you used or run this yourself right.

C

So I guess yeah.

D

C

B

C

Bit basically yeah.

B

C

Yeah, that's gonna, be a tally for each benchmark. Right.

A

Maybe we can specifically use in the functional use words. Also, the performance use. Does people use it for the functionality and not really care so much about the performance of that, but the feature of their function? Things like.

E

A

Give us some direction, which are the really people kid, the performance functions and the performance of a particular right.

E

A

Also, on top of that, I was thinking then, looking at that list can we narrow down ourselves, something which is just a JavaScript function. They are really you do require, but then mostly it is just running in v8 mode. You are not really using any of the node.js functionality right and, and the performance of JavaScript is easy to gauge from how the v8 performance is the newer versions. But there are certain API for some silly crypto, which depends on certain libraries, which they don't really have.

A

It's not easy to go and test the performance with those libraries. You have to build yourself and kind of test the performance. Maybe it is possible those we can combined with the node.js specific, maybe the event loop related may be celebrated, and things like that. So there is a few.

B

Lotus the bottom like to 31 to 32 and 233 RZ lip specific tasks, yeah.

A

So something which you cannot really people cannot really know the performance immediately. They have to go, build their own thing and then maybe run their API and build some small application to know the performance of deflate, for example. But.

C

My understanding was and correct me if I'm wrong, that that you know people who, whoever who used these benchmarks, whoever needed for a very good reason to run these benchmarks. They should already be aware of these performance implications, otherwise they wouldn't be running benchmarks right.

A

What I was thinking more of so launch is how you use node.js use for the something which we're doesn't give right like a file system, access.

B

Http axel HTTP.

A

B

That's all you should be note ap. Is they don't look as generic JavaScript pretty.

C

Much the only ones that are not I think are assert. Dp called these kinds of things. Yes,.

A

A deep equal doesn't it use. Jaws is pure JavaScript right.

B

But I suspect those are things: we've implemented a node, oh.

C

Yeah yeah yeah yeah, but but.

B

Now yeah, these are things where we've written code and note. If it's, if it's in the v8 its you know, we don't necessarily have you know coverage for those well.

C

Deep equal buffer for sure that that would that would probably be native at some point. Yeah.

B

I'm, pretty I think they all are at.

A

B

A

This one for each is it native to node or the this weird specific or the JavaScript physical.

A

So I'm thinking that if we can narrow it down further to me.

B

Okay, yes I think those we might have introduced specially at some point. Just because I don't know if there's any 4-inch code I mean so other than the es ones, then maybe I would say what I said is true.

C

Yeah I, don't know what about events. Events is an abstraction right or do, or is that also need.

B

It back right but I think that's like no events that are implemented at the node level. Oh.

C

Yeah, okay, oh FS, is the.

B

Specific es ones may have been added because at some point we were float when we float patches like we, don't necessarily have an exact copy of the eight, hmm so I think those may have gone in because we actually did break things at some point on some key things.

C

Yeah yeah, yeah, I, guess I guess to have their value and we'll find out how many people.

B

C

Because I mean.

B

Interesting, how many people say they they run each of these or have run them right in the past. It's kind of like I, think we should say it's like. Have you ever run these? You know which.

C

Is yeah? Okay, all right now that that's that's an important consideration! How shall we phrase this? Because.

E

That area right.

C

Because the votes, the votes will be influenced by how we phrase this so yeah.

E

C

Shall it be like which of these benchmarks? Have you found useful in the past or I think.

B

One, my first question might be just which of these benchmarks have you ever run when you know writing a PR writing / testing at PR.

B

Okay, then, week separately, we could have a second question which is like which of these do, you think, are most important to test like I guess. The problem is that when you change code, maybe that's not a good question because, like obviously, if you change the URL related to URLs, you should run the URL ones right.

B

C

Okay, yeah there they're all kind of important, but but still did that there could be there could be some bias towards one or the other, and and this would uncover that bias. You know.

B

Right, like the how many times have you used them, is a good one, because that that is a bias in terms of like how many people actually need to run a particular subset right. Oh yeah,.

C

C

What in the in the previous year or like? Because you know how many times have you used I mean we should give a window in.

B

The last year or say, yeah.

B

E

B

Like two questions last year or last ever.

C

Well, I, don't know it may be lost in the midst of memory, but yeah.

B

C

Exactly yeah, so, okay, just okay, one one thing that I just realized this with each of these questions we are. Each of these questions has or has a constant factor the length of the list of benchmarks.

B

Yes, it's long past.

C

Okay, so which of these yeah? Ok, so then that's a checkbox which of these do, you think, are most important. That's the checkbox, or we could do like a most important least important thing. At that point, we can do like a one to five thing. How many times have you used? Oh, this might be okay.

C

This is also probably gonna, be a one to five thing like very often sometimes you know never that kind of thing right, because I, don't think we'll want people filling out like integer validating text boxes, because that's just gonna take them forever. No.

B

Like I think it's I mean even we might just ask the one question which of these have you used.

C

Yeah yeah, cuz, I, guess I, guess they're, probably probably very important. If they've been used, they have some importance so that sort of speaks to the second question and I mean that's I mean the first question is a special case of the third question right: do you use them for what purpose right right, writing, testing a PR, so yeah I, guess I guess. The third question seems to be the one that that sort of captures the most yeah.

B

Just have you used them.

C

B

Ones have you used in the case like that might be the simplest and people can just check bubble ah there. We go right well,.

C

Yeah I mean we can set the default two to never like how many.

B

Price of exactly.

E

C

And and then they'll just just upload the ones that they actually use it yeah.

B

So that's true, and then you could have these sometimes yeah.

C

Exactly okay, all.

B

Right so it's three o'clock, but okay.

C

All right so I will hold this one: okay, cool all right, so yeah I'll try to turn this into a survey and then we'll go from. There sounds.

B

Good so I'm gonna say in the notes here: Gabriel Gabriel, shared survey discuss.

B

B

Guess through them, can you PR in the minutes.

A

Yes, I can do that. You.

B

Might add a little bit more detail or that you can remember, but yes, we can do that. Okay, Thanks, so I got a run. Talk to you later. Thank.

A

You you later doc, you later bye.