Jenkins Google Summer of Code Office Hours, 20 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Jenkins GSoC - GitHub Checks API Prototype Demo, 2020-05-20

Description

Kezhi demos his initial github-checks plugin for Jenkins. This is a google summer of code project that is currently in the community bonding phase. Project page and contact links: https://www.jenkins.io/projects/gsoc/2020/projects/github-checks/

Meeting notes:
https://docs.google.com/document/d/1TZLmu3nBPbwUjzLVYGnV_YtYvmzxzw6A4eEVYpbmi3Y/edit#

A

Repositories so here this is a sandbox repository I created for for testing our checks, plugin and so next. I think we should take a look at the class architecture, so it's here and uh so first we have to subscribe the event from github. So here this subscriber is provided. The green one is sub is provided by the github plugin, and we just to extend this subscriber this class, and then we can, you know, kind of subscribe, the events from github, and so so. The next thing we have to do besides subscribe the event from github.

A

We have to listen to the uh jenkins jobs. So here we have this job listener, extend the cue listener and also the wrong listeners. So we can do different jobs. Different, take different actions like create a check run when, when the one listener triggers on start- and we can- you know like complete the check run and when the job drop is uncomplete so and uh the extension point that provided now is this check result, and this kind is a- is an abstract class.

A

So I define some um method like the get name, get output and get annotations. uh So if other plugins implemented this class, I can use this method to get the the informations from from from the from those customers.

A

So I think I'll start my demo, uh so that would be uh clear, okay, so yeah here this is a subscriber, and here we have the ex applicable method. So it it decide whether this uh whether this subscriber is, is applicable for the current project and the events. We have to provide the the github events you want to subscribe here. We subscribe the installation repository event and the checkron and chigsu, which is here this installation repository event, and you see the payload here is added is because I installed.

A

I installed the app on my on my repository. Just now you can see the the the repository is sandbox and this this this id is an installation id. We have to use this to authenticate I design and then um repository or to you know, in order to use the api used to access the api.

A

So here I'm going to deliver this tail so here so here we can receive this event. The install in installation repository event and the up we update, update the installation is basically we map the installation id to remap the repository name to the instance installation id because later when we are either handling the jenkins jobs, we only know the repository name of the of the job. We don't know the installation id so we have to so.

A

I built I made a map here, so this is korea, so if uh so I'll show you, the check run I made uh yesterday and here is- uh is a: is it and.

A

So can you see it? Can you see it? No? No! So the the name is default jenkins wrong. This is because I have I've provided a default check around and the name and name is default jenkins wrong. So you can see here shows the default. Drink is wrong, but I haven't provided other things like output or any annotations.

A

So there's nothing here.

A

So if we trigger the build again.

A

A

Some problems with my.

A

A

Oh, I don't know what's the problem with with my jenkins instance here now, but I think we can still keep going up here. Oh because he has triggered the breakpoint. So here is our initialize.

A

So when initializing the job, we just you know, create a new check run, and here we can. You know like retrieve the the source user scm api, and here we retrieve the head of the of the current revision.

A

So if the head is a pro request, we can get the repository for name, and you can see. The full name is just like is a repository in like, and the headshot okay, this this takes a little longer. What oh? Yes, because yeah! Yes, because I didn't open my local network, I can't do it now, because uh my school network doesn't mean allowed to do that.

A

So if I open that I will be, I drop off the meeting, so I just show you I just uh so the headshot is: it will give you the headshot, the current revision, which is now I know the this, this proquest and it should be 1 8c8 and if we keep going there is installation id uh we can retrieve the installation id from from the from the map we created earlier.

A

So next we get the token it's this token allows us to access a github api and then then we use that- and here is how we retrieve information from the consumers.

A

uh We, you know just iterating all the consumers here the by invoking the uh all method and we create a check around here. So so we oh.

A

Okay, yeah here this method uh currently now. Currently, I just use a apache client to create a check run, and here I this this is uh here is a payload. This are some headers and in the headers we have to bear the token- and here is a is a body, so the name is a source get name, and it's just like the default uh defaults or checkron source. I provided here. You know like return.

A

The default chip snake, so here is a headshot and we set its status into ku, and here we send this way, send this post to github and you will create a new uh check run on github, and so the next thing next thing when the job has started almost the same, except here we just because because back back there we have added all the uh the creative backgrounds um and to the to attach to all the sources to our our uh jenkins run.

A

So here I add actions so later when the unstarted is triggered. Almost everything is almost the same, except here we just instead of creating new check runs, we just uh you know, get actions uh from the run and we update these check rounds. We created earlier so here here also.

A

Also, I just I just simply set the statements to in progress and here's the steps starting at and this and that's a time for this for this job. Also, I send this: send this payload to github. It will update the check runs.

A

So the next thing is uncomplete. Also like the unstart select on start uh here. It just will be complete, so you know when we so here we almost the same like the updating, the chi chrome. We here complete the check, run and also some payloads. Just the conclusion is success and the status is completed and completely added.

A

So here we send this payload to github. It will eventually become like this of the same city created started 32 million seconds ago.

A

So this is this. This checklist will create just now so.

A

Okay, so that's basically it yeah, that's the how the prototype currently works. Okay, thank you. It's really nice.

B

Thanks looks really good.

A

So currently, the the big, the biggest problem you can see here is that you know. Instead, we provide this uh aesthetically.

A

We have to retrieve the names the status and also when completing the job we have to. You know, get the conclusion from the consumers and we have to. We have to receive all these things and we have to retrieve the the the informations from consumers about this, the output.

A

You know like title summaries text annotations for for lines of code and the above things we can provide because the name when the users when the customers are defined their extension, uh extend the extension point that, to you know, implement the return name and the name is, is in our control, so the headshot also- and all this all these things we made don't need to retrieve them from the consumers. Just the below part. This.

A

This annotations object the start line the end line so annotation novel messages. So that's it.

B

Cool, um I posted a link in getter, um just showing you how they get how branch source plugin is looking up the installation ideas. um I don't think it's a group.

B

I don't think it's a great idea for us to just store the installation ids in a map um and relying on a webhook event.

A

Yes, yes, yes, that's that's not very clear because maybe you, if you use that map you have to you, know, process that map information, because the next time when you restart the jenkins instance, uh you you won't expect the the github to send you the repository installation event again.

B

Yeah, so this just looks it up.

A

B

Basically, okay.

A

So this is used to generate installation tokens.

B

Okay line 106 is where it locks up the tokens. It looks up the installation.

A

Looks amazing so it cures uh yeah. So I don't so using this. So we, if we have this method, does it mean that we don't need to? You know, store those installation, ids and I can just directly use this uh use this method to create a token and to allow me to authenticate as an installation.

B

uh Basically, yeah um not all the way, yes, but basically um so this is a credential, so you call the get password method on the credential and internally. The get password call will return this and it will also handle caching for you as well.

B

So if you scroll down to line.

C

B

Just down the bottom of your screen at the moment.

A

Okay, password.

B

Yes, so so, basically calling the get password will handle it for you, um when you retrieve the credential.

A

Still, I'm not so so this this uh this method, this just works. I like the like this line of code here you right so basically it works like this. Okay, yeah, exactly okay, so we still have have to you know, have some way to know the installation id.

B

No, um you just need the app id and the.

B

And the organization.

A

You made so you made a query to look up the installation id yeah. Okay, that's cool! So I think this answers these questions about. I think you have asked a question like this right here.

D

This this one of my questions was exactly this thing. What so? Because if you do it in a map uh dynamically and banking, this down, you don't see any yes, so it's better to query for it. Yeah.

A

Yes, you, you saw hoping yeah yeah. The mapping is a problem I want to discuss today so yeah. So I.

B

Think if you just change it to use the github branch source api for this, then you shouldn't have this problem at all. It should just be handled for you and we can always adjust the plug-in to make to. If we do need to expose more of this as an api, we can, if you need an api for giving the token.

C

B

For now, this is just handling getting you that the um this the beer token, um that you need to pass to the api.

A

C

That's cool, that's cool.

A

So I think, and so do you have still other questions about my prototype or we can keep going over our.

D

Agenda yeah, one thing um I was wondering so currently from the perspective of the warnings plug-in I'm just interested at the end of the build. So if the pin build finished, I think it's okay for my plugin to report the results somewhere. Yes, actually I I don't really care about the progress during a build.

A

Yes, so so, basically uh you you're, uh the the one is next generation. Plugin has, you know, uh get its reports during the during the publishing stage? I guess right.

C

A

So it's it's definitely before uh before the uncomplete.

A

You know before the uncompleted event triggered, so so, basically, what uh the consumers like the wording's next plugin next generation plugin has to do is uh because I have earlier attached the uh attach the actions to the wrong right and the wrong on the under the action bears a check around source.

A

So uh so what you need to do is to update the the the the fields in the source. So here this is the source, uh the the source I provided here, the extends they implement. The extension point so here this is, you know we need many more discussion here, because we have so many parameters that we want to retrieve from the consumers.

A

Maybe we should not. You know, implement this in the class. Maybe we should use some. You know like xmail or or some other things, but currently I use this as a class. You know. So if you, if you implement, get external id and get started and get conclusion, and this method will be invoked uh when the job is uncomplete and I can retrieve the info, so I can retrieve the uh the information the parameters from the consumers.

A

So does the answer question.

D

Yep, I think we need to talk about this a little bit in more detail, but this is not nothing we need to do today. Okay,.

A

So the biggest question is that we have so many we have so many parameters, so I'm not sure whether represents name as a object as classes is, uh is the right way to do so.

B

The classes is fine, I wouldn't, I wouldn't add everything yet just add the important ones and then um and then people can come asking for what they want or.

A

So the next thing is uh pipeline support. uh I I I I'm not very familiar with the pipeline, so uh I find some resources from the jenkins developer guide and I think, and so from what I have known now.

A

The pipeline uh we just need to you know like implement some builders and uh the the pipeline well like magically uh compatible for plugins. Is that.

A

B

A

In his platform.

A

I think I have a I have a post about make the parking pipeline campaign.

D

I think you have a link in the gita channel.

A

C

Make your pipeline.

B

It could be in the how to how to guides down the bottom.

B

How index are all hell2 guides? The last line.

C

C

A

A

Here is: what's it, this is all about the pipelines. I know from this post. So so what I'm thinking is that maybe we should you know like uh like uh those parts, these parts we may maybe we can, you know, implement them as builders or publishers, then this maybe uh makes them make this pipeline compatible.

B

Are you thinking of adding a step which allows people to report custom checks, yeah.

A

Yes, the goal is like you, you are lacking your okay, nothing, your posts about the so currently the users only can you know like uh update the information in status form like they can uh they can't uh create a crazy runs about the code like they can't use the reports from the warnings next generation plugins or coverage plugins, and they can report them to the github. That's because maybe they can maybe they can. You know they can't uh access this information in the pipeline script.

A

So that's that's one of my questions so uh does this necessary to you to make our or you know, make our parking pipeline compatible and or maybe uh mix mixing configurable by the users? uh That's also my problem.

A

D

Actually, I think so that we need to make this happen in the beginning. So I think the first thing is to have an api for plugins and if there is still some time we can think about it to create a generic step that can report some texts or some annotations, but I think the main focus should be on support for plugins, okay, and I I think this could be added later on to have some generic thing.

B

Yeah, I think this was on the nice to have stuff like it'd, be great to do, and.

A

So so we'll pull off these things. So another thing is also the general api yeah. I got some code stuff from 3ds proposal and.

A

A

He defines some general api, uh some code steps, so I'm also thinking about the general api. So here is the idea from key. So actually I think this this this method. Actually, you know like describe what our plugins have done, like publish report to github like subscriber trigger events like past report from the customers, uh but I'm I'm not sure whether we can this. You know this general api.

A

I, uh where it works so.

B

Yeah, I'm not too first in the jenkins extension points and creating the apis for other plugins to use.

A

So when I first think about the the general api is that when I saw because now we know for github, we use the github branch source api. So maybe we maybe wait. uh Maybe we can use the gitlab branch source api for the github and feedback uh bitbucket the source branch source for bitbuckets. So this is the first thought when I saw the general api- and I don't know whether that's feasible.

B

Could possibly, we could possibly publish our api separately to the implementation as well, because the github checks plug-in is probably it's going to have to depend on our branch source and a long list of other plugins? Probably we possibly might want to have our api plugin separate.

D

Okay, so now I understand the question yeah. Actually, I'm not sure um I also implemented a plug-in with which is requiring git, and I created an api plug-in to make it yeah extensible, for instance, use subversion, but I think this is something we can look at it afterwards. So first just create an implementation that works, and then we can see if we can extend it in an api.

B

Yeah, just publish it to the experimental update center. Well. First, because I mean it works and then we can publish it to experimental and then we can iterate.

D

And one thing that I'm not really sure about it is the the number of git uh api plugins. We have so we have a git branch source api. Yes, and what is that a github api? So I'm not. I.

A

Think you have listed them here right yeah, there is yeah, that's that's very confusing I'll show you the first one, the github api plugin is. This comes confused me at the first.

B

So that's a that's a typical api plug-in in jenkins for an external library, as it just just wraps, github api and provides it so there's only one copy running and we don't get class plus complex from different plugins running different versions.

D

And this is the url for github for us.

B

Yeah, so that handles all the api calls all the authentication and okay and most most of the caching. Although github branch source is doing some of it as well.

A

Here so this is the the page, the home page for the github.

C

A

The main plugin we need to depend on yes,.

D

A

Yes, we have to retrieve the source that the sm had from you by using the sm api.

D

Even if I so I'm not really familiar with the github branch sounds api, I'm just using the git systems in jenkins. So if I just want to use a git and github, uh this is the github plugin, or this is something differently again.

D

There is an another one. We call guitar plugin. What is this plugin doing as a mod github api? There is a github plugin yeah. The github plugin was a bit confusing.

B

D

um Yeah, okay, maybe I I should look afterwards how these plug.

A

Okay, french, okay, did you get our branch source? Api is kind of you know. Oh okay,.

C

Document here here is that.

A

That's the user guide that one yes, this is like the here. You can see that you create a github organization project and you know so this. No that's our our ci server looks like for now.

D

Yeah, but can I use your plugin without the github branch source plugin or is it intended.

D

Because I don't use the github branch source, api plugins so.

B

That that's what I was just mentioning about, possibly when we might want to separate out an api plug-in that doesn't depend on anything github related um and then then our plug-in just uh handles their extension points. Okay, just so that all the other plugins don't have to depend on github that want to contribute here.

A

The the biggest problem for if, if our plugin depends on the git plugin instead of the branch source plugin, the biggest problem, is that we we may not able to you, know like retrieve the the name, the the the name of the github repository from the asm api implemented by the git plugin.

A

So so that's something so we can do this step. These two steps we can, we can do it. We.

C

Can retrieve the source we can.

A

Choose ahead: okay, okay, so the github plugin we can have plugin is that we depend on because we have to subscribe the github event, so we just use a little part of it. The green, the green plus, is provided by the github plugin, and we just extend this and we can subscribe the github event.

D

Maybe that would be a good idea in your design document to have the actually plugins also here that we are required to use so which class from which plugin.

D

It would be a little helpful for me, because I'm not familiar with this github plugins yep also.

C

For other developers.

C

So there are things to discuss more.

A

So that's possibly about my uh the things I want to discuss.

C

D

um And one thing I I'm not sure about was the reaction that you are creating. Is it just so you can find again your intermediate results.

C

D

You are right in the end of your design document. I think, can you go to the yeah, give me this yeah and this action is just to store intermediate results or how or what are you? Why are you required to use it.

A

uh Yes, it's just I. I need to attach the actions to the run so and carry the checkpoint results to the uh to the wrong side. I create this action and this section is actually a wrong action, so this is like the it's like the warnings next generation. I think you have the result, action and result.

D

Yes, okay, but I'm showing something in jenkins. But the actions are just here to show something in jenkins and you don't show that I'm thinking about.

A

I think I think about this a little bit, because if we don't leave the the, if we don't need to store the installation id here- and maybe we can get rid of this without action. So that's that's why I'm currently using it the installation id and to attach it to long.

D

Okay, understand.

A

Okay, so to oxidize the other questions: okay about the status api and the the checks api yeah. It's also very confusing. I'm going to show you some.

C

C

Oh yes, no! No! No! That's not!.

A

A

So you can see here and we have two- you know like two checks: the first, the this one, the continuous integration. This one is, you is used in the status api. It just leaves one message this community school. So if you go into the details, you are going to the ocean of the jenkins instance, so you can hear this lotion of the drinking instance. But if you see it watch the detail of the checks and you have go to the the checks yeah, that's the difference.

D

Yes, but my question was: is it configurable so or is? Do we now get both actions or both results.

B

Yes, we get both the results. We should be able to suppress the other one. Okay, so um there's a way that there's a plug-in that turns it off. um So we should be able to do the same thing that plug-in does and disable it once we're stable enough.

A

The status api is, the status is created by the branch source plugin right yeah. I.

B

C

C

A

Questions so I guess.

B

A

B

um Where is it here? um I just posted the plugin that does it um into the get a channel, um so there's a so very easy to implement. Once we want to okay.

A

A

B

Well, assuming we can add a trait automatically, it's.

A

B

For the user to add it, but it should be doable. Otherwise we can update the plugin if we need.

C

A

So I guess that's all for our today's meeting.

D

Thank you helpful to understand everything.

A

Okay, I think the the in the next video we have to talk more about the the api design, the the milestones, some things like that.

D

Yeah, it's a good idea.