Jenkins Platform SIG, 10 Jul 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2020 07 10 GSoC Git Plugin Performance Project

Description

Jenkins Git plugin performance improvement project office hours July 10, 2020 for Google Summer of Code.

A

B

A

We shop, you have the show yeah.

C

Thing today, okay, so I have access to your machine mark and I wanted to ask what so general rules how I should use the Jenkins instance. If so I agree, I wanted to ask that first, so.

A

You are, you are welcome to use that Jenkins instance in any. What I would call reasonable way? I would appreciate it if you would not reverse engineer all of the credentials that are inside of it and take those away all right. That would be really that would be bad behavior, but I'm entrusting you, then that this is a it uses, Jenkins credentials and therefore there are techniques that would let you get get them back out. Please don't steal my credentials other than that. You are welcome to do it.

A

It is a destructible instance I reconstruct them all the time. I reserve the right to reconstruct them at any time, I, destroy them and restart them very frequently. So don't be surprised if sometime, you were doing something and all of the history of the thing gets destroyed so but but you are welcome to do your. You are welcome to use the resources that are here there you're welcome to define new jobs.

A

If you would like the job to persist over, you know over a potential restart, let me know, and I will archive it into the definition so that then the next time I restarted. There are also multiple copies of that. If you need access to a separate copy, if you'd, like a fresh copy, I run fresh copies on other machines and I could give you access to those other machines as well. I.

C

Think this is OK for me right now: I don't need a fresh copy. Ok I think mainly I would want to use it for running the benchmarks. Note there are quite a lot of machines there. I would only use it for that and also the second thing was the GMs reports again.

C

So we were thinking of adding that to that Jenkins, and so so my question was that I would have to I would have to what I would have to do is to access the Jenkins directly, the home, changing the directory of this instance and add the HPI JPI to the replace, add the JPI to the plugins directory key in the in the instance to integrate this time again inside the Jenkins. That's.

A

C

For my local instance, your.

A

Much you're doing much much too much hard work for that. What you do is you open? The plug-in manager find the available tab and click the jmh plugin.

C

And it's solid.

A

Don't you know yes, yes, you you describe the hard way to do it. Do it the easy way and we'll take care of it later, with making a permanent just.

B

A

Installing so you can use it, I think it's actually already there, but if it's not go ahead and install it, and you are welcome to install any other plugins, you need, if you install a plug-in and it it disrupts the instance. Okay, no problem, if you install a plug-in and I strongly disagree, I'll, let you know.

C

A

Yes, we will Chuck Norris plug-in thanks very much Justin. That's great! No I I do not have the Chuck Norris plug-in installed, nor do I have the Bruce Schneier plug-in installed, nor.

C

The next thing on the agenda so I need to pee. I belong back for the two or three benchmarks and I. What I assumed was when I raise that P at the benchmarks will run on DCI Jenkins IO, because the branch was G saw hi, something G image so, but the benchmark that stage is not being run by on this yeah Jenkins retired I was not sure. Why is that happening? I actually have the PR open. There, I was looking in the Jenkins file.

C

We do have the condition where we run all the we accept the branches which has a rosy sock, but still it is not running other PR number. This one.

C

So I tried, I, have had multiple commits in this PR two or three benchmarks. I added and there's not running I. Think I will not be able to show you that.

C

So whoa so I was actually not able to figure out what could be a reason for that. Is it because I don't know, I have actually I, don't have a clue. Why.

A

Was so so is the I just just let's, let's be sure, we confirm it here, okay, good! So could you go back to the to the the Jenkins file definition? I, wonder if what we've got there is a file, regular file expression. Then what we need is a regular expression.

C

A

Because it may just be that we're not matching the we're, not matching the branch name, yeah isn't ji-suk asterisk. That's a file named style, regular expression in order to match anything after the wouldn't need to be dot asterisk.

A

B

That would be 1:30.

A

I think so what what? What that regular expression says and I should have read that seen this when we code reviewed it right, sorry for not even thinking of it.

C

A

By zero or more times so, if you name the branch, gee sock orgy sock that would probably have matched it, but but we need I think we need an extra dot in there just looking at it now,.

B

A

B

A

Fix that or if you'd like I can commit it directly to the master branch to fix that. Okay, sorry.

A

Yeah well come on code reviews had some responsibility and I. Clearly I didn't didn't fill mine here.

D

C

Was actually looking at groovy documentation, I was doing it and I did not.

A

Yeah so another failing of code, reviewer Justin, that's good.

B

Cps is like its own generally groovy, but uh caveats: okay,.

A

It's clearly a DSL, it is a domain-specific language. It just happens to be based on groovy yeah.

C

Remember why I did this, but I'll look for it and I will probably add the information in the PR and they were like.

C

C

This is more of not something discussing about yeah, so I was I was thinking. I I did say that I would create last utility in the benchmarks model, which would validate those benchmarks functionally so so I've created a class which, right now it offers two kind of validation. The first is to check the number of references. If we are running some, an operation which is related to the fences we basically compare, number of references are correct or not, or a particular repository.

C

The second one is the size of the repository, that is, the dog gate site, so so so what I noticed was that I was actually using the file utility to calculate the size of a for the tree directory and, and it was if that operation moving jingling the benchmark itself. So I was a little then I thought of benchmarking. The files util dot.

C

The file you to start estimating the size of the directory API actually check what kind of contribution is it? What kind of overhead is it added to the whole benchmark and and I? Think it's not a surprise that it's just adding a mini error. So the benchmark results. It said that it's adding a millisecond to the the operation is taking of milliseconds or I, actually checked it for 300 MB size repository dot get directly. So it's just taking a mini seconds. I am results wise.

C

It wouldn't make a difference if it's a if it's a gate fetch, maybe but all the small operation it might I think it just is just an observation for myself. That I should not include anything even if I have to validate I would have to find a way where I do not use the benchmark itself to validate in a parameter.

A

And that seems fair. The a micro for the goal is to have it be as smaller unit as you can possibly test right. It's so so I think I think you're right. It's the concept seems to assume that the the thing the benchmark has written is doing valid things and we have to do it by inspection or by observation. That seems fair to me.

C

So the last topic today is the abstract, colesium. Suppose I've been exploring the ACMA place, so actually it was for the size estimated class I was looking at I. Think I'm, something in question. I was looking at how the mighty branch projects they they were working. How is it ketching the branches so so first I want to confirm something.

C

Just a second I.

C

Actually wanted to confirm a locking cache, lock, feed.

C

Where it's a biggie, this is a big class and it's very difficult to.

C

This branches yeah so I'll, actually talk about the locking of the cache after this, because this is this is what I can find right down at this first. So what I can see is that we use a functionality provided by Jacob Rev book, which is basically blocking the commit three to two. Actually, we want the branches and we want other references. So if we have, if someone is building up scanning, the P R, then building the project so a mighty ranch project. According to my understanding, it works like if I have.

C

If I have multiple branches, they put scan those branches and then, if they, each of them have they have a Jenkins I would run the Jenkins Island debate.

C

That is the very basic definition of a multi branch, and you have other differences like PRG, one who bailed upon we would have full requests and it would scan them and then run based on those four requests as well, so so so what I can see is that we, first of all what I want to ask that I see in this class is that get can see like it can also provide these functionalities individually, where VOC is actually, if there's a lot of things, it's not just well, I could see that it is parsing.

C

They commit one of the things. I could see, that's passing the commit while walking. We can do this so and then I then I saw that we have a functionality in CLI. Get called get REM pass come on, which would do the same thing, so the question I have is: why are we using a book? Is it because this is the gate? It's providing all these functionalities, but we don't have a class to do it like we don't have. We would have to add similar functionalities into a particular place, and that is Jake.

C

It is only providing us that is the reason why we using the book or is there another reason I'm not there right now, yeah.

A

I think it's again, it's it's Jake. Its interfaces are designed for and very well thought out for the way that Java programmers want to interact with things and so I think that when when this was implemented, it was detected that hey the jagged interfaces are just cleaner and easier to use than they get client plug-in interfaces. So let's use Jake it and and I think it was a good choice. It was a very wise and sensible choice.

A

It also avoids avoids the overhead of launching a separate sub process to interact with the with the git repository in small ways right. This thing can crack it open and let Jake it do a bunch of caching for it in the Java process that does that address your question. It.

C

C

So I think my main miles. The main thing I have was thinking about was that, if we're using something like this, do we want to benchmark the operations with almond hair? Then we have considerable amount of references. This is because I think we have seen how data less the more books or the number of reference leaves in the equation between the number of references and size of the repository. It's not it's not, of course it's linear in the sense that it increases the size of the repository. The number of references increase, the.

B

C

Crevice I saw those references, but since we would I'm actually I haven't seen the Rev walk in depth, but what is- but this is just a high-level thought that I have is that we could probably benchmark the rev both the functionalities, reusing from their book, compare them with house.

C

He like it works, although just what you've said I think it's not needed, but is it something we should explore because since this is something jagged is providing and with whatever experience, I've had I just I thought that, okay, if this is provided by Jake it and same thing, can be provided by get. How is it possible that it would perform see like it would performative in in areas which we haven't explored yet good.

A

Good question and I think it's: it's unlikely that in this use case we would find some place where we say. Oh, this is significantly enough faster to justify reworking the implementation to use CL I get.

A

But but it's it's a valid question that I I can't confidently say: I mean we know that we know that for small repositories, fetch is faster with jagged and there comes a crush hold where fetch become slower with jagged I would be shocked if, if the operations in je get for these revlon dove, things were dramatically slower, in particular because Jake it is pretty commonly used to implement things like the Garrett code review systems back-end and therefore it does an awful lot of rev, lock and so I would have assumed that the jagged authors had done quite a bit to make sure that that thing works well.

A

C

Okay, so maybe what I can do is to just throw this out beside a benchmark would not think with a lot of time to maybe benchmark a certain functionality under certain conditions. Oh yeah.

A

I think particularly if you can see a way to do it with without a lot of effort, I think that would be a real positive, another place to say: hey I'm gonna, try a benchmark of something, that's a higher level than just fetch. This is this is an and this spot is a this. This particular area is a place where we've got potential for cache contention, so you might even be at a higher level where you say: I'm gonna, try yeah! So yes, interesting. If you can do a benchmark without a lot of cost.

C

So when you, when you're talking about the cache contention, earmark you what you saying is that or a multi branch project when we created first, the first thing we do is we try to find the cache? If we find that, then we we create a client and then do all of this work and if we don't find it, we fetch the repository and then create a time. Then do that, so why we have a assuming that we have a cache.

C

So the contention it happens by we're locking it is the a is that the area where the issue comes or for multiple branches, no.

A

C

A

Least the cache contention claims that I've heard were were in the operating case, where the clone already exists. There's been a change on the remote repository and now each of the branches of that remote repository are being evaluated. Are there changes there and they tend to build a stack of of people who all want to access the cache cache copy? They each want to get their own lock on the cache copy. They lock it update it unlock it. Locket updated, unlock it, and at least that's what I thought was was had been in.

A

The report was hey if I have a hundred and fifty branches and with a hundred and fifty branches, if there's a change on one of those branches that gets detected by the github branch source plugin, it will invoke a check of all of them and if it will now process- and it may process, changes on many branches and they will contend with each other for access to the cache.

C

I think I need to read more about locking caching cache resources and because I am NOT, I haven't read too much about it. I just actually, there was a gear ji-suk idea, which was related to implementing a system which would distribute the caches right.

A

And- and this is not, this is not that this is decided like.

B

A

And in terms of in terms of your project, I- wouldn't I'm, not I'm, not personally concerned about this layer. If you want to shift your focus back to the lower layers that you initially described, that's fine too there is. There isn't a requirement that we have to improve this piece, we're looking for way looking for things to improve, and if you find that this could use improvement great. But if not okay,.

C

Okay, that's okay, I, just read about it and I'll, possibly try to I'm, not sure how to reproduce it. Is it something reproducible have you or is it? Is it a random? Eventually, it's not it's not like it's going to happen every time. So is it like if I have 150 branches, it is going to happen for sure or or is there something that happens randomly, and you know.

A

C

Why even happening so.

A

So I'm pretty sure that I can see it and I'm pretty sure that I can see it on that computer that you have access to if I run. If, if I, let's see well, actually is everybody okay, I could switch and share my screen and show you the job where you'll see it. If that will help you.

C

A

Here, let's just put this up and what I'm gonna do is bring up that job. Can you see my screen yeah? Oh woops, wrong, one. It would help if I use the correct.

A

This one, okay, so I, have a thing called bugs pipeline checks. Let's make this so that it's actually readable on the screen. Sighs there is that easier, bugs pipeline checks in bugs pipeline checks, I have a repository named Jenkins bugs and then it's represented by different protocols that access the same repository any one of those.

A

If you look here and we look at the scan repository log, it scans a hundred and fifty or so branches looking for changes and and I have scripts that cause changes in many of these repositories in in roughly linear time it just it causes a change and then sleeps and it causes another change. And what I will see is that when I look at one of these jobs running, it will show a long pause and the console output where it says this branch indexing.

C

A

And that branch, indexing and I think it's this branch indexing plus this and I have a suspicion that there's a long pause between this and this, but again I'm, not sure that this is in scope for your for the project. You're working on Rishabh, I'm, I show it to you. Your goal, I think, is to find areas to improve plug-in performance. I'm, not sure that this should be your first priority. Yet.

C

Okay, I get that I get, and this is not ever just I was interested in how what the issue was like actually they're, not quite understand it and land right now, I understand of an issue okay, so so so what I'm going to do is after I think I've read enough I'm going to start reading a pro dry for the estimator space task. I knew that the first stars pudding to us. The second is I paid benchmarks or to compare the walking commit just walking functionality, the being both delegate and jagged.

C

So these two things and I I, don't think we have any.

D

Other thing to this does say: Leggett provide their functionality or this only Jake. It has that I just check like Jake. It has that revoke and I. Think that is pretty awesome thing. Yeah just check for the good sailor, I.

C

Wish I miss checking, so there are couple commands with multiple commands which would provide the same functionality. I haven't taken it each one of them, but I am sure that we have commands, which would there's a command which would print the current graph there's a commands which would walk the commit history and it's the gate.

C

Rev book come on so yeah, so so I see I think more into the code where what operations I would need to bench would be useful to benchmark then address yeah, and the first thing I'm going to do is to raise the PR for to correct D junk inside as a great expression, yeah, so I think that's it Tom I say: okay, do anything you guys already cut. Yes, I will. So you could just end.

A

It you're okay, running the you'll, run the session next Wednesday and next Friday from your your zoom account. Yes,.

C

So so what I'm going to do is I'm going to I'm going to mail the link to each of the men to create an event. Google can I do that and of engine for signing.

D

D

There is nothing we could really.

C

D

So to do you need, like my help, for creating those empty like deposit is for the benchmarking purpose or you'll be using use.

C

We actually yeah. This is one potential discussion we actually amazed. So so what you're talking I think we should explain. The topic first topic is that we so for the bench benchmarking strategy. We will more focus on the size of the repository, and now we want to move on to parameters like the number of branches, commit history and, and so what home care is proposing is to create some repositories with the constant size but different number of branches, so in depth.

C

So when we use the benchmarks I have for these repositories, what we could find out would be how much the number of branches they create performance overhead, all the they contribute to the performance of get fetch.

C

So so that would be I think that's an interesting thing we could do and um car if you could, if you're, if you're, creating the repositories, I sure as I remember, you have created a repository with with 5000 branches, yep, so I think what would be I think what we could do is that we could have the repository with maybe ten branches. Then then we go to 50, then 500 and then 5,000 yeah.

D

So that was just waiting for up world from work for that particular thing. So I'll go this weekend, I'll be creating those separate repositories for you definitely.

C

Okay, thank thank you. So much for doing never.

C

Once you have those I layer and edit my benchmarks and we can run the experiment, then.

C

Okay with everyone- okay, so that's it for today. Thank you guys all right.

B

Wouldn't go great mark I'm sure it's gonna be awesome. Yeah.

A

Well and if not, it.

B

Will be colvett.

A

Nineteen makes things interesting, even if they're not awesome, they are certainly interesting.

A

Everybody, I'll post the recording link.