Jenkins Platform SIG, 19 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2020 08 19 GSoC Git Plugin Performance Project

Description

Jenkins git plugin performance improvement project meeting for Google Summer of Code. Pull request review status and branch source plugin changes were both discussed.

A

B

Hi, everyone today is for today's agenda. The first thing, first and foremost, thing is the design discussion for authentication. So we have to discuss two things. The first is what information do we need to send to other plugins to um to create uh an authenticated client connection, and the second is to which plugins do we send that information?

B

Because last, in the last meeting we uh we had a discussion that we could uh delegate the responsibility of uh getting the size of repository from the brand source plugins to the um to their base plugins if they could be called baseball if they get up plug in or they get flat.

B

So um so, while looking at the brand source plugins, I have uh seen that so with the github brand source plugin, I I saw something which, which kind of helped me in generalizing how we could send the information to each plugin so um usually in the brand source plugins the scm source. What it does is when it needs the credentials.

B

It looks for a class and for an example. uh The github brand source would look for a username credentials. Class username password credentials class that git lab transfers would look for a personal access, token type of class and then the context. So we need three things for authentication. First, is the um the type of credential we need. Second, is the context from where we need it, and the third is the credentials id which could be mapped to the credential store.

B

So so the second thing, which is the context, so these brand source plugins what they do is they get the owner of the sem source which is logical for them because they are brand source plugins. So they would get get that context and then find the credentials for um for that particular context. But what we need is we. We need to pass our own context because, for an example, a freestyle job will not be aware of any uh multi-branch project.

B

So we need to pass our own context and we need these plugins to understand that and to take that in account and then find the credentials. So the github uh branch source plugin already has a static method, so so the git lab, and so the gitlab brand source plugin did not have a static method. To look for the credentials it was, it was connected.

B

It was uh linked to the instance of this uh scn git sem source, so my extensions, the extensions we implement are static, so there was no way I could borrow those credentials which it searches in the sem source owner. It's us in the owner's contacts and then get them into our uh con into our extensions, and that would, I think, would not be correct as well. So so what I've done is I've done something similar. I borrowed what uh the gator brand source does.

B

It provides a general method where you provide me so that that that function asks for three things. It asks for um the context it asks for the. um I will show you through the code only so it asks for the context in and then it asks for the url uh where, for which we are asking this, it could be. It is generally the api url and third, is the credentials id.

B

So um so I've done something similar for the gitlab plugin. Instead of uh taking the owner owners from the sem source owners context. I have I I've provided I've. uh I've asked it to ask for the context, so it it is generalized and then it would um so it would then look go and look for um the personal.

B

So so, if aga, if a person is uh entering a gitlab url uh in the git plugin, it would provide the credentials a personal access token and that token then can be accessed by the gitlab brand source plugin. And then that would be used to um to create an authenticated connection with the client api and.

C

That's how yes, yes ma'am, so so you said: you'll use a personal access token, but doesn't that then preclude doing anything with a gitlab url that uses ssh, because in that case it only has a a private key credential.

C

I mean that's okay, it just so this this the technique you're describing, will work great for um http based get urls, but not for ssh based urls.

B

Yes, it would not market would not.

A

Work for that, but then.

B

But then um what I would one I would see is that so I I I look at how uh the um the current grand source plugins are taking the credentials and and if we look at the default credentials method to get the credentials for the git lab um brand source, they they are asking for the personal access token only and not for for so what you're saying is.

B

If we have uh so the ssh url, we would have that if we have a personal server, where we've, uh where we have a git lab server, hosted right and then we would want to connect to that server using ssh or if, in any case we would have a server where we would establish a ssh connection, and we would need a private key to do so.

B

C

But but it's perfectly okay, that the the ssh case needs special thought. That's that's perfectly. Fine. Solving the https case is already great.

B

Yes, and- and why I did not think about it is- is that I, whenever I I look at how this has to be done, I look at the I look at the current way. The brand source plugin is handling credentials, and that is why I just shadow the just the same process and I did not think about the ssh servers, but yes,.

C

B

Definitely a case we would have to.

C

Well and there's there's no choice there because there isn't a way to do a rest. Api call with an ssh token, with an ssh private key.

D

C

D

Can't be done, uh you'd have to like find a way to translate the url to https, but for custom, urls, you're kind of sunk right, exactly like you could maybe do it for gitlab.com but like if it's calm like right. What am I gonna do.

B

Okay, so um so now, with this implementation, what has happened is that we can be sure that we need in. If we talk about the git plugin, we do not need to send any other information to the plugins. I was also looking at the getty brand source plugin.

B

um I haven't looked too much into it, but I I as much as I was looking at how they are authenticating the connection. It's the same process. They are also in the brand source in the sem source class. They are trying to get the owners context and then um and then it is trying to create a connection. So so now, my uh so now we had two points to discuss.

B

The first was what information we need to send in, to which plugins so now the second uh decision to which plugins should we delegate this responsibility depends on so do we want to implement um a general method here, so the github brand source plugin does implement the method where it is asking for the context, and uh but it is perfectly fine for the branch source plugins to to say that I don't want to ask for the context. I know that I am going to be.

B

I am running in an scm source environment, so I would get the sem source owner's contacts and I would need that to provide the credentials. I do not need to provide a general method to get those credentials so should we shift this implementation, the responsibility to the git lab plugin, so that that the gitlab plugin would be a plugin which would not be limited to a multi branch project right it could. It could have uh end points which could be accessed by other plugins and it does not have.

B

It does not have the sole responsibility of working for a multi-branch project. That is what I'm saying so programmatically or technically. It doesn't matter to us if we are putting this implementation to uh the transfers plugin or to that plugin to the gitlab or data plugin, because um one way or the other, we would have both of them in stock. If the brand source plugin is installed, we would have the other one.

B

But yes, uh the vice versa would not work. We could have the getup plugin, but we would not have to get the transforms plugin, but I'm I'm not personally aware of of the use cases of the gear plug-in in the gate laptop. So that's something.

B

Maybe you could tell me if there's there are use cases so so the decision rests on this uh fact that we could. uh We could uh create a method which would just take the we just need the uh brand source plugins to take our context and then look for the credentials, because we will provide the credentials from our context because we would have the user who would choose the credentials and then only would access a private repository or gitlab repository.

B

So so we need that from the from any plugin. So since the github branch source plugin is already it has that method, and I I exactly did the same for the gitlab I I haven't tested it because I'm having some issues I want to discuss later, but uh but it's the same, it's uh lying behind the same thing, we're doing for the gitlab transfers plugin and it should work, but um so the decision rests there.

B

Should we move this implementation to the to those plugins or is it okay for us to um uh keep this in the branch source plugin.

B

So it would be only. I think, one case where we need to uh we need to see is that so the git for the git plugin it it doesn't matter if the plugin from where we are asking this information, if it so if a branch was plugin, would have multiple repositories, mapped to a single owner, git plugin. Would I I, as far as I know, would have one one owner one deposit right if you're talking about a project, so that might that we might think that that might be uh something which might create an issue.

B

But since we are providing our own context- and we know that the brand source plugin would take the credentials from our context, we just need the the functionality of that plugin to be able to connect with the client server client apis.

B

We need that the credentials, the the mechanism to scan the credentials pretty much the same in the git plugin or the github branches, but you know the git lab as well what I saw the way it scans the credentials and the type of credential it looks depends on the brand source plugin, but the the way they are scanning for the credentials is same so so for us. As far as I I understand it's, it doesn't matter to us if it's implemented in a branch source, plugin or or down below to the github plugin gitlab.

B

So um so yes, so I think what we need is first uh definitely interactive testing, um because uh so I have I've been facing some dependency issues because of which I was not able to upload the get lab brand source plugin to the same jenkins instance, where the latest version of my git plugin is installed.

B

I I before going moving to that point. Is there any um any concern any doubt or any anything? We would want to.

C

Discuss I'm I'm not concerned about whichever path you choose, so I think if, if it's, if it feels simpler and more elegant to you to do it, one way versus the other that that way is great.

C

The giddy plug-in for me has been is the closest to a clean room implementation, and I think you said it does not contain this needed function right. It doesn't have it yes and.

B

So just one second mark, I have looked at the scm source class. So usually, when I'm looking at a branch source plugin, I always go at the same source class, and then that is the point where I understand how the client is being created, what it needs- and I do the same thing now. I haven't looked at the other classes, where uh the gt plugin might be using a credentials function where it's taking the context and doing that I haven't looked at that. I will. My observation is just limited to the to the sem source.

C

Class, so would it and would it which of the which of the two alternatives, would make it easier for you? Is it indifferent to you which, which way you go or is there one that oh, this would be easier if I did it this way, if, if all the plugins look like github branch source, would that be easier or is it rather some other way.

B

For me it it doesn't make a difference till the point I so for so it's different for each plugin for an example for gitlab transfers. It doesn't matter if it's the gitlab transfers or the gitlab plugin, because the way I am creating the client it's it's directly, I'm not using any wrapper method provided by the brand source plugin I'm using directly the java implementation and instantiating the client like that. But when we talk about the github brand source plugin, they have provided a common utility class of a connector class where it takes some.

B

um It takes the repository url. It takes some information, takes the credentials and does all the work for us. So so maybe it's for us it.

B

I am not sure if it matters, if you're, if you're, finding a standard common uh decision that okay, we if we implement this extension, we are going to implement this extension at all branches plugins only or if you're not doing that, then we're doing it for the uh independent github plugin, all of the gitlab and so on.

B

Maybe we could look at cases if, for example, in the data branches, it's what I have already done it. I have tested it inside my local instance with the private credent with the credentials and private repository. It is working it's getting the size and as far as I'm concerned, I think I need the size and I'm getting it. So I I'm not sure if it matters too much what plugin we are using.

B

So um maybe we need to look at that from the perspective of so it doesn't even matter if we have a multi branch project or not, because so that dependency is also uh is not there, because these methods do not require the context of the brand source plugin.

B

It requires a context which we would provide, so we just need a git, a git plugin, uh a pipeline which is um trying to use git as the scm and then doing whatever it wants to so so it so I I think it would not matter to us if it's a brand source plugin or a normal product.

B

That's what are you using? The item context for uh that is what.

C

I I will show you exactly: I thought that was used to pass the credentials in right, that that's that's how you're, relying on them to to get the credentials for that job. Right.

B

Yes, so the lookup credentials, which is prior to the credentials api, it takes uh three things from us: it takes the type of credentials we need, it takes the context and and then it takes the url, we provide uri and then we match it.

B

D

Yes, yes, uh and you said the uh gitlab branch source plugin, not the git lab plugin is where this is. Your convenience method is right. I mean.

A

D

Where you've written this one, but did you say there was a pre-existing one or you were talking about this one online 751.

B

751 is the pre-existing uh method which is provided in the, which is in the get lab branch source plugin how it takes the credentials. So if we, if we, if we see it's, it's getting the sem owner context here, instead of doing that, I am providing it the context- and I I add this context here- that's that's just what I've done and rest of the um the way of looking at credentials.

B

It remains the same.

D

B

Yes, so so this this method can be static. This can be starting because it is not connected to any instance of the current class where it is entered the acm source class. It doesn't matter if we are using sem source object or not, but for this way of taking the credentials, it won't matter because we are trying to get the scm to be. This is not static, so we cannot use this method to pass down the credentials to the extension class extension implementation.

C

Yeah, so this what you're doing seems quite reasonable to me, you're saying hey in the case of gitlab scm source. I need to ask a question: what are what is the personal access token associated with this context, so that I can, I can use it and okay now, you indicated that you could choose to put this instead in the get lab plug-in. Was that that.

B

Yes, we could do that because we need two things to implement these methods, my x, the extension and this method, the credentials api and the github lab the github api git lab uh rapper java. uh So we would not know we would need those two things and we would have that in inside the get laptop.

B

So we can implement the same thing in that plug-in.

C

But there is, but there isn't an obvious place in the gitlab plug-in where this would naturally insert, whereas the scm source, it's a natural location, to insert it. That's where the github branch source has it right. Github branch source already provides this at this layer.

B

uh The github brand source plugin has um so it it contains these uh the scanning credentials method inside the general connector class, the utility class. I was talking about right and it borrows borrows it in the sem source or wherever it wants to. That is how it is working.

C

Yeah, so the concept of a connector is at the branch source level in the github branch source plug-in. That seems, like a a fair justification, say I'm doing the same way following the same pattern. Putting in the gitlab branch source.

D

Yeah and you're you're planning on using this when they set up the freestyle project or at run time.

C

So van like when it both running, it has to be at run time because it's usually.

B

Yes, it has a repository.

C

Size calculation, exactly.

D

Yes, exactly this use case is when people are setting up a brand source plug-in. I think this is used for jelly.

D

C

C

I thought that this was con would then be used. Oh, I don't know. Okay.

D

I think the I think what happens is that runtime um this is used uh to produce the output for the the ui, and then the user selects the credential they'd like to use. It finds the the candidates and then it uh present the ui presents the credentials that the user can use, the user selects the credential and then they save it. And then inside of the config xml of the of each underlying job in in a branch source, setup uh you'll see a credentials id.

D

uh Stored and persisted.

C

So justin is that is that, are you are you considering, then that, should we store the credentials id as part, somehow, as part of the the this the question to ask for the size of a repository, I'm not sure, I'm understanding how? How would we use the the information you've shared.

D

Yeah, I guess I'm wondering if well I have a couple of questions about it. One at runtime like are we potentially gonna expose credentials, we shouldn't uh if we detect the wrong credential, that user doesn't want to expose, um which that's a hard problem from usability perspective. I think that's hinting at the first question that you're asking is like: um are we gonna make people select which credential they want for size calculation?

D

I think that's kind of your question right.

C

Well, so the way richard has implemented it as far as I understood it, it's the the job has a repository defined for it, and that repository has a credential associated and he's relying on the called on github when he calls into the github branch source it will using that job. It will find the credential id for that job and use it. So so I don't think there's any leakage, because the credential id is absolutely associated with a specific job. Did I understand correctly with rishabh or correct me? Yes,.

B

So so, if, if I am, if I am creating a freestyle job uh and if I choose to check out the scm using git uh via a gitlab private uh repository, so with the repository, I would add credentials for that repository to do the process to do the thing to the checkout and that those credentials would be the personal access token in in case of get lab. Those would be the person access, token type of credentials, I'm choosing and then so I now the git plugin would have the um would connect these credentials.

B

With the repository user repository information we have and then we could send the credentials id of the particular job and the context of the job to this extension, which lives in the brand source plugin and then the brand source page would use the jobs context and the credentials id we've provided to get us the credentials.

B

Yes, that's that's how I have.

D

I see so it's only scanning credentials associated directly with that freestyle project.

B

Yes, yes, yes, okay,.

D

I'm less than the freestyle project mode. So that's where my questions are coming from too probably.

B

um So so now I'm one question I'm wondering is: if, if this is this function of scanning credentials is independent, it looks like it can be implemented anywhere because it just needs the context. It needs the url and then it's need the credential. But then okay, we need the information for the client api client and that we can find only uh either in the brand source plugins or the uh the github plugin or they hit flap.

B

So nevermind, it's just.

B

So uh so for me, I need to uh the github brand source, plugin extension. I have interactively tested it and confirmed that it's working, but with the gitlab one I have not been able to so I would so if we don't have any question, if there's no concern right now, I would like we would like to ask the question and relate it to the dependency issue, so so in the git plugin. We uh we depend on the gate line again, of course, and so, since the unsupported command is, is shipped in the latest release mark.

B

I assume that the git, the git plugin, would depend on the latest gate claim plug-in now.

C

Not yet you'll have to explicitly declare that, and you do that here by inserting a line version version. Yes right, yeah so and the the technique that's being used here is this particular file is using the jenkins plug-in bill of materials and that jenkins, plug-in bill of materials provides a default, get client value that was verified and that default value is lower than the one you need. So you'll need to declare you'll need to declare the version number explicitly there.

C

Then three months or six months from now, when the bill of materials is updated, we'll be able to remove the version declaration again: okay, yeah, okay. So sorry for the confusing nature of maven management.

B

That's okay. So currently in the pull request, I did add the version, because, without that my test would fail, but, uh but now, when I was trying to so I so the git lab brand source plugin is directly depending on the get plugin it desired it as a dependency in its form.

B

But my uh the issue comes when I'm trying to um so I'm again having um and the even enforcer plug-in upper dependency uh issues. While doing so, and uh so what I did was I just excluded all of the dependencies which are creating those upper dependency issues. I'm not sure if that's the right way to do it, but um so wherever I was finding a conflict in versions, I was just excluding them which the git, the latest git plugin, would have different versions and what the gitlab transfers plugin needs are different.

B

So it's different, so that was creating an issue.

C

So I would, I would expect that those that I don't know I that may need some further investigation because those exclusions yeah. So I I don't know I always have to work through very painfully these maven enforcer enforcer issues. So I'm a little surprised at those exclusions, but if does it run for you? Okay, when you attempt to load this in a run time.

B

uh No, it doesn't I'm still having enforcer issues. I am having it for the good time. So, if, if you I- I never understand this line, I always take it as as a conflict, and I try to exclude it. I I don't understand this thing. What does this mean? I understand that we have two versions of the same plugin, and that is why we need to remove.

B

One is, is that so is this what this line means that it says it requires upper bound dependencies for the git client and it's rating, the 3.2.1 version, and I am getting the 3.4.1 version.

B

The git plug-in has declared the 3.4.

B

one version, so I so initially I thought since the only since, since the gitlab brand sources, depending on the git plugin, it would get the git client plugin from the git plugin itself. So why would it create a conflict I? I could not understand.

D

Something else is probably also declaring it.

C

Well, except the the report here is actually saying that git lab branch source depends on git, plugin 4.3.1-snapshot, which depends on git client 3.2.1. I would have interpreted that to mean that somehow your 4-3-1 snapshot is inadvertently, declaring that it needs 3.2.1.

C

That surprises me, though, because you'd said it doesn't doesn't even compile for you. If you don't depend on 3.4 at least 3.4.0 right.

B

Okay, I think I I can look at the dependency tree, and maybe I should um work out my day like that. It was just that's how I so what I need to do right now is to uh to add this updated run source instance in this plugin in my jenkins instance, and then I could test it for a gitlab repo in a freestyle project.

B

So I also I'll try to solve this, and um I think after that, so again in the last meeting we had the conversation that the git lab requires a server name instead of the url, and the brand source plugin by default does not provide us a method to directly use the server url to get the client. But I was able to look at the java api implementation. There was a way- and I invented that in our extension, so that's solved.

B

Also, I uh using curl requests, I uh so we also I I I I had an issue in the last meeting that the size of the repository comes under statistics for a project and you and in the documentation it said it said that we need particular access.

B

A user would need access to particular access to that project. To get those statistics, I was able to get these statistics for a private git lab repo, for which I have developer access to. uh So I I I'm assuming that it would work for us in general cases for anyone, not just the owner. I had the issue that it might only be able to get the statistics for the owner of the repository of the project. That's not that's true! So, yes, so now the steps I need to implement.

B

First of all, the current pr, the 931. Now it has the unsupported command work, all that I shifted from pr 937 to 931.

B

So now in in 931, the git tool chooser is being instantiated in the git scm uh indicated in the in particularly in the method which is creating the client.

B

So if someone wants to run the plugin yeah sure there should be, they will be able to test the get to chooser with simple checking out uh a repository, a freestyle job in any way I what I need to do is, I need to add it to other places where the client is being created as well, particularly in the sem source.

B

I haven't done that there so for me, that's one commit I have to add, and the second then, would be once uh we have covered all the places where the get to chooser has been instantiated to add a similar safety switch in the global configuration page like we added for the redundant fetch issue, so then the user would have the ability to not use the git to choose.

B

So that's something I still have to do so. Do these two things I are missing for from that yeah, but I I think uh in in terms of get to choosing it's the design and its function, the function it should provide it is. It is doing that um then there's one more discussion which we haven't discussed, I, uh which was related to what the get to chooser is returning.

B

Currently it is returning a string which is the uh the git executables path, which is directly feed it to the the builder of the client, uh justin and fran have uh have recommended or they've said that we could, instead of using passing a string and if in case we're not able to recommend anything, I I the get to chooser sends none, and then I would. I have to add another check after that.

B

So if the recommendation is none then don't use the recommendation use what was uh given by the system provided by the system, the git plugin, the git plugins recommendation. So um instead of that, we could use the git tool, we could return a git tool. So I do. I do not understand the use case of doing because so, if, let's just go where the get you chooser is being called, I and I we'll see how the result is being used.

B

So so this is the pr 931 code, so the chooser is being called and the chooser's tool, which is the executable string is, is then directly added here.

B

So if we, if we return the git to git tool instead of a string, then we would it's. I think it's it's the same thing. We would have to then get the um executable from the git tool. It would store that. I, I guess the only advantage where we use a git tool is that the that, for the cases where we are not recommending anything, the git tool would go to the default installation.

B

But then that would be a concern for us might be a concern for us, because git tool then, would make another decision which we do not want to make. If we're using it too, because we have to consider before changing what kind of implementation we can suggest, we need to consider what the system and the user has um selected.

B

The git to chooser is now equipped to look at those decisions and then provide the result. But if I, if I instantiate a git tool- and I I add the recommendation- we want to give to the git tool and then we return the git tool in case we are not providing any implementation. The git tool would default to the default installation.

B

That's uh right.

B

I think yes, and that would then that might be an issue for us, because, if you're not able to recommend uh an installation that doesn't mean that we should switch to the default in installation, we should.

B

We should use the installation with the git plugin before the get to chooser is using the exact that same implementation, so that we do not break any possibilities.

B

That's what I thought with returning git tool instead of the string, fran and justin, do do we have a? Why should we use git to do you? Do you think there's there's there's an upside using it too. Instead of a string returning a.

D

String, I mean, I think, some of the challenges I had with it is that we have none scattered about the code a little bit um so, instead of using that, we could do something like using git tool, which is typed and then instead of having none, you could potentially have a constant like there's a pattern called null object pattern and what you do is you have a kind of a dummy of that type. That's that signifies that that it's done in this case that you're representing.

D

If we need to have some kind of null, then maybe you would do something like that. You would have a constant get tool. That's like revert to default and and then you would use that to determine whether it should be the other one. If you don't need that, and you already have access to the default git tool that was already selected, then you have another advantage that you can just send that back.

D

Potentially, I don't recall if you're, if at the end of the day, we're sending back a git tool anyways, and that would be another reason why I would kind of maybe say that we should use a git tool class. If that's the case.

D

Does that kind of make sense.

B

It does just so if you're not talk, if, let's just if you're, ignoring the git tool here, if you're ignoring this part of the code, the existing uh the git plug-in implementation, we do not use.

B

The executable path string and then we directly add it to the client here, the builder of the build building method for the client.

B

That is how it's it's done right now, but what you're saying is, as I understand that, and I I so my only concern is- would I have to update the git tool class, and I was I was thinking if I was just. I was just wondering if, if that's something we need to do, if it's, if it's worth that change changing the git to last, to add our case where, if we do not want to recommend anything, we add a constant.

D

B

Would you change the.

D

B

Maybe I am misunderstanding what you said, because I so if so so the case I'm talking about is justin that if I, uh if I do not want to recommend anything.

B

The git tool would have the git tool doesn't, as far as I know, it looks for a class either uh a descriptor so which could be so if, if it doesn't have that, if it's null, if we do not provide it, uh then it it defaults to the default installation which is uh inside the plugin. Now with that, my concern is that then we are making another decision without considering what the user and before reaching to this point the plugin has made.

B

uh Maybe what I can do is that I could possibly I haven't looked at. uh You know I haven't implemented this. I I can try uh instantiating the git tool and then look looking at how this could be done, and maybe then we could have a better discussion about how we could use a big tool or should we use a git tool or we should provide a students.

D

Yeah because I guess I don't think you would need to change the get tool class itself, you would just instantiate like if you needed to use the null object pattern like I don't know if I'm not super convinced that that's exactly what you need to do, but if you were to need to do something like that, and I think that's kind of where your concern is of about changing the get tool class, you would probably just create that cons.

D

I think you only care about that here, and so, if you only care about that here, it seems like that constant could live here and get scm, uh and you would use that to make your decision of whether you're gonna go back to the system default. I'm not sure if this is the same where same place. That fran was going in with this, but.

D

Yeah and then you would just use the executable from the git tool.

B

Yes- and I I think that seems like the better way you do it, I only did not do it because we were making some custom decisions which the git tool is not equipped to make, but then I I think I should look at the get 2 class first before making that comment, I haven't um uh I I do one thing we can discuss this on friday.

B

I can look at the gig 2 class and how we could, uh if we're not providing any installation, then how we could use the git tool and then give that message to the gift plugin.

B

I think that would be a better way to do this.

B

Then I think so. Another thing I need to do is after doing all of this. Apparently I need to calculate the time spent on the jobs currently without the get to chooser, so that after adding they get to choose it, we can compare jobs and uh approximately uh they reach to a conclusion. Okay, this is how this performance improvement has worked and to watch.

B

We need some metrics for that in those metrics would come from this experiment, um so I have enabled the time that I think it's called I'm not sure what the plugin is called, but it just time stamps that build uh the console uh logs, and uh I think I would uh so what I'm wondering is what kind of use case.

B

So what I was assuming was, I was taking the lazy part of just using mark's instance and not worrying about what kind of jobs I want to test here, what kind of from which jobs I needed. I was one I was just. I was thinking that to to get is to get real metrics or close. To that point, I need to measure jobs which are used in production or somewhat something like that, because if I do this do the same thing with my instance, what I would do is, I would look at a brand source.

B

uh Maybe I would create a multi branch project with a with varying sizes of repository. Could do that. I could do that and I could then see how how the plugin is. The performance is changing in the checkout step, how it's uh changing in the scanning process or for the multi branch look at those. So there are two ways to do it in what I was seeing.

B

My approach, my default approach was to just install the plugin, not ex not first uh expect from where the performance improvement is going to come and look at jobs with with huge workloads, multiple jobs, which marks instance, contain and then see the difference after I think they get to choose it, but I'm not sure if that's the right way to do it or if it's possible, because so to see considerable difference, we also need to vary sizes of repository. I was assuming that one of the project would have jenkins eye, but I could.

B

So what I'm asking here is what approach should should I take? Should I take the approach where I okay? I know the first area where we could see some improvement is the checkout phase, so I should create five projects with varying size of repository and then measure them measure profile them with my jfr in my local instance and then okay, I can say that this is the difference in time.

B

After the get tool, chooser has been added and then I could move on to other areas of the plugin for an example, the the sms source, the scanning of repositories and then building those repositories. I could look at those areas and then uh measure the differences like that.

C

So, at least yes, I'm I'm not understanding what you gain by using jfr on your local copy that time, stamping when I enable it, it looks like it does, report it at least two per second resolution, and and yes, so it does provide time. Second, one second resolution: uh let me do a quick check just to see if I change to the j get implementation. Does it still time stamp, but I think it does at least well enough to to see it. So I don't know that you need to use jfr locally on your environment.

A

You're welcome to create those varying.

C

Size repository or varying size, jobs on the on the the test environment or your local either is fine wherever wherever it helps, you get the data that you want that you need okay,.

B

D

So a question of like how you can do the slicing of before and after or I guess maybe I didn't understand the.

B

D

Overall question: I guess.

B

The overall question is: how should I, what how should I get the metrics to understand what kind of differences this class is made to the git plug? That's the the aim of this experiment, and I would use that in the final presentations and the.

B

I was actually thinking out loud when I was giving two approaches of doing it, so um I I guess I would I would default to using. I would create different projects with different size repositories, and I would like to do it on markz instead, instead of my local instance, because I am I've seen sometimes at my instance because of uh because of my personal resource it it, um the the reports are sometimes.

B

It's too varying the reports of different builds, the same for the same repository for the same configuration, the site, the uh the profiling which I do from profiling. I've seen that the local instance my local instance adds a lot of variance to those results, and I think, if we are providing these results might be important for us, because they kind of signify what this project did to the gate plug-in. So it the results should at least have some. We should have some confidence into those results.

B

What whatever? However, smaller increase will be added to the percentage in performance, improvement or larger it? Just the change doesn't matter, but it matters the results are we have consolidated results.

C

Yeah, so one one more thing you might do as you're establishing those jobs is intentionally declare them to only execute on agents with the label cloud that way you're avoiding the wide variability that is existing the hardware. That's behind my that's on my jenkins instance.

C

All the agents that are cloud instances are allocated on google cloud platform they're the same basic processor type, you'll get different operating systems, but they are the same fundamental processor type and and so you'll you'll you'll get on average a more common common throughput result than if you, if accidentally you run on my ancient raspberry pi as opposed to a thoroughly modern new computer running debian testing, or something like that or your bitcoin miner. No, I'm just kidding right or my bitcoin miner. That's right! Why didn't?

C

I think of that, because that's something I want to do with my electricity is mine. Bitcoin.

C

Of course, I have an entire rack of bitcoin miners and they've all got gpus. You know they have enough enough diversion. That's great. Go ahead.

B

Yes, so, um okay, so I'll I'll do that I'll, create projects and then that's how I'm going to get the metrics. So from my site, I think I've explained what I have to do from the mentors. We need the kit chooser to be reviewed, and then I think I I don't see any blocker in that process. If there is some, please I think that's obvious, it would come out in the review process.

B

I hope I haven't left out anything uh uh justin fran omkar any comments in the pia which, although I I made sure that I did include each of other suggestion. Only one thing was left that was uh justin and um fran's recommendation of using the git tool instead of the string. That's something we've left, but it's apart from that. I think we've included everything in that year.

B

If still there's something I would have to have that review.

B

I think that's it um so on friday I would I would try to uh in so I would should be done by instantiation and uh and adding the safety check, I'll, try to set up the projects and and if we are able to see some results, we will discuss that if that's possible.

B

So that's, I think, that's.

A

B

A

Yes, I think uh last time we discussed one of the point like uh for the gitlab, you needed the owner's permission level, so I think you missed that point.

B

Yes, yep, so I I tried to to clarify the doubt I had at that time. I tried so I used curl to actually I have uh I I am. I have developer access to a project, a private project in gitlab, so I was able to get the size, and so there are varying level of access provided to roles in git lab, and so they range from owner to a reporter.

B

I and I have developer's uh access and I was able to get the size so that so maybe I misread the documentation, and that is why I was saying that we need the owner's owner's rights to get the size statistics.

A

B

That's pretty great.

D

That's all yes, yes, yes, you basically just need read access to the repo. Basically right, any any role that has read access should be able to see the statistics.

D

Yes, that's perfect, that's what we need.

B

Yes, so thank you for spending more than half half an hour. It was just 8, 30 and say 55, I'm sorry! For taking more time, then we were scheduled too.