Jenkins Google Summer of Code Office Hours, 13 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Jenkins GSoC Office Hours, Jan 13, 2021

Description

Office hours discussing project ideas including REST API documentation generator, cloud events plugin, Tekton client plugin, Jenkinsfile operator, and more

A

Welcome to google summer of code: uh this is the jenkins office hours. It's the 13th of january, thanks for being here. uh Let me bring up the agenda uh topics and we're open to topics that you would like to add.

A

All right so jenkins project, google summer of.

A

Code, we just released jenkins2.263.2 today and jenkins 2.275 with security fixes, so it's been a busy day so far for me watching and being sure that all the parts and pieces are behaving as expected. There, forgive the my not being as quick on this one as I should be here. We go. Let's see the meetings.

B

Congratulations on like new release thing, I should say.

A

We we like that. We, like that a lot. Oh nope, that's the recordings. I need the notes.

A

huh I know that I have notes.

A

Well, I apologize, I am not finding the gsoc notes, we use a google doc file and this is really awkward. I don't find the oh and probably, in the event calendar just a minute.

A

Let me check there because I could just read that yes meeting minutes. Of course, if I would read the calendar, it would also help. Here we go all right, so I'm going to share my screen, I find that it helps me if we have a shared copy of the agenda. We look at it together and then we'll type notes so january 13 2021.

A

And let's make the text big enough to read and now the sharing thing share. My screen.

A

Okay, so you should see january 13 2021. Is that visible to you.

B

Yeah yeah, it's good.

C

B

A

All right so status updates uh discussion, so one that that, if you are interested, I could put a discussion of the most recently added project idea.

B

Yeah that would look cool like will it be.

A

Get credentials in a pipeline step.

A

But rather than me spending time talking about that, first, are there other questions that either of you have that you would like to put on the agenda.

D

E

Yeah um I studied the project um um just assessing.

E

um I started the project um that was um automatic specification generator for jenkins rest api, and I I have researched about quite a lot of things. So I have few questions and like on that project idea. There was one video uh which is referring to youtube um from 2020 discussing about that project, idea with the potential students at that time on 2020.

E

So um at that time, um olekser said that um um we need to first of all, a good step to get started in on that is to just work with get apis um with you using sas, pose to get annotations and just started making specifications, but um on the other hand, um at that time the another potential mentor which is christine. um I also said that you can use um open api to publish, I mean documentation, so I was little bit confused.

E

I mean um I have the results a lot, but I'm not sure again where to get started um on that.

A

And I think the answer there, as far as I could tell, is that neither of those is particularly preferred. Both are potential. So uh as far as mark understands.

A

Either path, either path will be helpful and may give you insights on what what will ultimately the path that you choose that you would choose in your project proposal.

A

Because I I don't think that kristen had done the full, the full steps there to confirm it, and I'm pretty sure that oleg hadn't done the full steps on the says pause thing either ah and oleg's here. So we can. We can ask him directly. That's great! So oleg segar was asking about the automatic specification generator for jenkins rest api and he'd started using well sega. You should explain. I shouldn't restate what you did.

E

Yeah sure so, hello, um all right, um so I studied that project um in which we have to extract rest, api from the sources and then publish the rest api um documentation. So for that um I, on the projector you have mentioned, you have to study java rest api and open api, slash swagger. So um for that I have watched the video, which is in the um projector idea, which is directed with the youtube in which you are talking about um that project with christine and the potential students in 2020.

E

So um so on on that at that time um you said um the first good step to get started to have a good understanding is to um generate specifications using suspose which is going to capture the annotations and through that you can just start it with the get api. But on the other hand, there is other part of the code also which do not have um annotations, for which I have to.

E

I need a com, I mean complete analysis of the project, or maybe some um annotations also to get started um so um and on the other hand, christine I mean when you said to christine at that time. um What's your perspective to get this project um done so she said we can use open api um open api to um generate the documentation, so I was little bit um for getting just started in that, so where from where should I get started you you know um research further?

E

Where I mean I, I watched all the resource resources that I can possibly search. Yeah yeah.

F

And uh thanks for doing this research so again, you're totally right uh by referring to these videos, uh so yeah cispos was just an example, because this pulse allows to capture easy beats, so items which are annotated and hence which you can easily extract from the code.

F

Our framework which we use for uh the rest api step here. It doesn't always provide explicit endpoints, uh though recently it improved. So if you navigate to the jenkins code base, you can see that many requests annotated by post by get and you can capture data from there and maybe one of the approaches is actually quantitative methods which I'm missing, because when you go through class, you can easily determine that the method is used for endpoints, for example, it has annotation or it has uh specific parameters like uh just query, parameter, annotation, etc.

F

All these methods immediately become a part of rest. In addition to that, you have getters, and the getters also documented step here. So by taking these three types, you can cover significant part of use cases for junkies rest api. Now it won't be complete, but at the same time for other use cases, maybe the good approach is to just annotate the end points properly.

F

If something is missing,.

E

E

So um so, um if I have to um take a another one step so should I get I mean study the code base of stapler or started with jenkins core.

F

I mean so my recommendation would be to actually start from a relatively small plugin, so you can please repeat that my recommendation would be to start from a plugin. So, for example, you can take a git plugin or maybe something smaller, so something which has a rest api and then you can just.

F

Dependency on this plugin add it to fastpass and do some scanning, so there are already some tools separating in this way. So you have seen references to pipeline documentation, generator right.

E

F

So I can find a link, so basically you could take a single plugin, not necessarily jenkins. Core junk score is just a big tonalize and you can take a plugin and try to build an example which is based on core very similar to python documentation, generator.

E

So should I just pick, let's say one any plugin and use adam get plugin, and then I try to generate this specification for that kind. Plugin for that recipe, is it.

F

Yeah, it would be one of the most simple ways: okay,.

E

F

Pipeline uh sorry, gift plugin might be very big, so you can take something smaller. It's one of the options.

E

Okay, and um in that I mean you, showed um that um in that time, you're using some who am, I page, which is showing the info about your um get info, okay, and so is that also a plugin that who am I should I get? I mean try to um generate specification for that. Who am I.

F

Well, you can do that, though, for my page is really straightforward. um Just second I'll open the code.

F

Now mark are you looking for the code.

A

I was not looking for the code, but I certainly can oh like just a minute.

F

I know I can do that.

A

F

A

So, oh not jenkins.io! It's jenkins, all right.

F

A

F

Want it yeah, it's just go to fire for my java, so yeah. Thank you. So is this. Is this the right page? Oh like yeah? So if you scroll down, you can see some examples, so you can see that it's exported beam, it's extension, so it allows to quickly map it uh to the endpoint because action. So it's just slash for my it's exported beam. It means that if you navigate to that using uh json or xml api, you get a response and you can see.

F

Fields which are being exported, for example, get name is authenticated, is an animus, and this is basically the information which would be exported best api. Yes,.

A

Oleg would it be okay if I brought up my jenkins to show show.

D

A

The experience interacting with for segar's benefit, so it's I, I think, sega. What you'll see is something like this. So this is my jenkins instance and I'm going to append on to the end of it. Who am I- and this now has the ap. This is the web page rendered and if I do api on the end oleg, I think that shows me. The rest. Api hints.

F

No, you need to go to json, for example,.

A

ah So if we go.

F

To the json three additional types but yeah api, slash, json and you get the api response. So basically, these three fields which we discussed.

A

ah Yes, okay and there it is without the pretty and there it is when done pretty great.

E

Okay, so I have to generate that um json specification right.

F

um Not json specification, but yeah. This is json response, so you can recommend in open api specification, so basically by processing connotations and, for example, by extracting javadoc's documentation. The way you can see this is not the best example, because there is no javadoc.

A

Well, but, and- and this would be- it would be valid for him to add javadoc to this to this class. If that helped right, I mean he could submit a pull request to. Who am I proposing to to javadoc it, but I think, like the point, is valid, that if the intent is to use javadoc, this class won't do that as it currently sits.

F

Yeah anyway, here is just an example, but.

F

E

So um I mean so basically um I have to the eventually I have to generate the documentation, but in order to generate the documentation, I need the um end point so through here from annotation, I'm just trying to capture the endpoint and try to pass that into and and from that end point I have to generate the um open api specification and from that specification we have to generate a documentation. Is it.

F

um Yes, so once we get our openness, api specification, documentation can.

E

F

For you, because there are tools for that.

E

Okay, okay, so so, basically I have to um capture the android and try to um generate the open api specification. From that end, point.

F

E

And the good step is user to get started with the um pick any small plugin and try to capture its um end point get endpoint and just generate open api specification for it.

F

Yeah, so you don't have to complete all the project uh during the application phase, so it's just a discovery so that you take a look at options and you can come up with a better proposal. That's it! So! Can you please repeat that sorry, so the recording phase happens in the summer before that you basically explore the projects, uh explore the opportunities and your main goal is to understand how to implement a particular project and come up with a proposal.

F

So, for example, here, if you see that this approach doesn't work, you can make another proposal.

F

You know that's perfectly fine and again uh yeah. This open api is a quite wide area, so there might be other topics and see them.

G

A

So the the goal to restate oh, like the goal, the goal in this phase is to prepare a persuasive.

A

Thorough project plan right.

A

Is that uh is that a fair way to say it all egg, that one of the one of the objectives.

F

Here is not a detailed project plan, but basically to form understanding what would be the deliverables. So, for example, you can take a look at a pipeline step, uh documentation generator. Maybe it could be something like that. Maybe something different but yeah. The goal for now is basically to see what are the opportunities and come up with such a proposal.

E

Okay, so, basically, currently I'm just trying to um research more and more. What are the um possible ways um not have to worry about the coding stuff? Is it I mean.

F

You can prototype, but yeah. The goal is to basically export very well so.

E

Okay, I see, I see okay.

F

You don't have to produce, let's see a solution for the problem uh to get accepted now.

F

Basically, your goal is to focus on the proposal and to collect information uh which you need and if creating some prototypes helps, then you're welcome to do that.

E

No okay yeah got it yeah. So um for now um um I mean for just um I mean just start more exploration. um Can you recommend me any plugin small plugin should I can I mean.

F

Well, so how I could do that, I would go to jenkins and just search for exported bean. Oh.

A

Looking in the in the jenkins github repository.

F

Yeah uh well, uh for example, there are plugins, which are quite popular now, like configuration, is called plugin, so it would be a good starting point.

A

Because in it there is probably an exported beam annotation that would indicate.

F

No, there is no exported bins, so there is one class configuration java which exposes cpi ah okay, but yeah. It has no data expert api, so yeah. This plugin doesn't work for that. I'm looking for alternative.

A

Proposal. Okay, so I mean, is it? Is it worth a look at the git plug-in or even maybe at the git client plug-in, as I wonder it? Well, you indicated that git plug-in might be too large whoops. It would help if I navigated correctly.

F

So we basically need a relatively small but at the same time a useful plugin, so yeah there is. There are many plugins which have exported bin annotations.

A

Yeah, I'm actually curious. I wonder if the if my my friend the platform laborer has an exported bean. I don't think it does.

A

No, it doesn't okay, but it doesn't. Does it have any post, it does have a post. So so is that might be one exam? No that's just in a in a test, so that doesn't help us. This is only a test implementation of some other post, okay, nope.

E

Yep, I have a question like what um the annotation exported being I mean, what does what is it? Is it representing that particular exported.

F

Yeah, it means uh that uh snapchat can export this class on get requests. So, basically, you can export data from this class when somebody needs metadata and that's probably not the most popular case, because in many plugins they just receive commands, they don't export the data.

E

Okay, so a step ladder is using it to export data right. Yes,.

E

E

Sorry I mean, if I confused, I mean I mean if.

C

F

E

F

Yeah they should.

C

F

Because you want uh yeah, so we still need to find the plugin.

A

So like, as I look at at the get object inside git client plugin, it's definitely got exported bean.

A

I don't see any post on it, though, should I post.

F

Well, uh you plug in definitely have some posts.

A

Yeah yeah, I absolutely the git plug-in I know does I was just thinking, get client plug-in as a smaller, smaller plug-in.

A

Okay. So let's look at.

A

Yeah so, for example, example here is.

A

Form validation inside inside one of the browsers, like.

F

Maybe they should take away that if there are not so many plugins exporting data, maybe it's not the best use case.

F

Yeah, so you could take one of pipeline plugins. Of course,.

C

F

Okay, I'll try to find something, but again I'm just looking at plugins, and I cannot find anything these interesting exported annotations, so yeah maybe actually take an engine score, make sense there, but.

E

So, um um okay.

A

So segar, you had described the the steps to attempt and I wanted to capture those, and I wasn't typing nearly fast enough. So I think what you had said was you. The first is use the who am I based on oleg's guidance class as an example, a in jenkins core and you had noted, extract the the api endpoints endpoints as or describe them as open api, or maybe I'm getting it backwards here. So.

A

It extract them and describe the api endpoints with open api.

A

Then I assume you've got to find some way to extract, so that would be an interactively done. Then you extract them.

A

The api endpoints programmatically, and that was where oleg was uh suggesting the pipeline step generator, is a is an example of something else that does that? Oh, like did, I understand correctly there, or am I going down the wrong path.

F

uh That's that's fine and then okay. Finally, I found plugin, which is good. It's a good average api plugin. It has a multiple exported beams to generate a report.

A

Okay, so code coverage, api, plugin.

E

So I mean okay.

A

A

E

So from this plugin I have to kind of extract the endpoint and generating the open api specification. Is it right for adjuster exploration, point of view.

A

Yes, I think I think you described it cigar that this would be this. This plug-in lets. You gives you an example of a place where you can use the exported bean and exported annotations to identify things that are providing data. Now, how oh, like I don't know how to reach this end point from inside jenkins itself, I'm sure it must be reachable and is that one.

F

From jenkins, you need to publish the first report.

C

F

And yeah after that, uh so we can, I see um maybe I should adjust this yeah.

A

Okay, yeah I'm happy to stop sharing in a second you'll.

D

F

F

Yes, yeah, okay, so here we have exported the beam. So what mark was uh there are multiple entries? You can see that, basically, all of them point to coverage result. So coverage result here. It's basically a model object and you can see if you go here.

D

F

Result then, you should have exported things for other data fields, so, for example, coverage 3, which is also exported been. So what happens? um If you go to this entry point, um you get average result and then, on the next level you get coverage three and average three includes other classes. So basically we will get a hierarchical structure.

F

Is a lot of data being degenerated, so here there will be json field results which includes more fields underneath how to access this data. So coverage result its model object. So it's available on the top.

F

Instance, I believe it's available through action.

F

So let's uh check this theory so yeah there is a coverage, action actions exported exported through api, and here you can see that there is a coverage result which is get resolved.

F

So what will happen here is if you navigate to coverage, result page and then go to result. You will get this report, which is a part of api.

F

I'm not sure do we have a coverage discovery discovered gp, set up anywhere.

F

I guess they drank this. Follow runner doesn't have it.

A

And I thought that I was using coverage api on on my instance, so I'm well you're. Looking I'm going to look a like.

F

Yeah, I think it is yeah, so here's our coverage report, so it's slash coverage, it's covered rejection and we have a report here, so you can see that we cannot access it actually through the web interface. But I believe that.

A

Yeah ci jenkins that I o may have maybe limiting intentionally the the apis. It's publishing.

F

Yeah, it uh believe it's a lot of apis but yeah. There is no rest api exposed for this data, so I believe it gets just queried through rest, so yeah, probably not the best example either.

F

Let's see if they need you to meet with. uh Actually you can see that it's already get resolved so because it goes to target.

F

So maybe we could just uh extract it.

F

Yeah, just yeah. You can see that api json works. So if you invoke it on your instance, you will get data here. Basically the data is filtered, so we get nothing.

F

So you can try this url a similar one.

E

Recording of this video is again is going to be at glitter right, so I I'm just going to watch it again. I'm to understand.

A

Absolutely there's there's a reason why we record these sessions. The recording will be available, probably within two or three hours of our ending here absolutely cigar.

E

F

Sorry for my explanation, yeah, I wasn't prepared.

A

Actually, that was that was marvelous oleg. Thank you very much for taking us there. So now, oleg the coverage, the coverage api, that you were showing that's different than the jacoko coverage that I'm using there are mold. That's an entirely different thing. Right, that's yeah! uh There are two plugins, it's just separate I see, and so the the coverage is. The newer have been great. Thank you.

E

So- um and I mean um I should explore the um coverage plug-in right- um yes, okay um and try to extract the um I'm and I'm like um I mean. Why are you saying we I mean I mean I mean not saying but um use adam. We need to extract the json tree, but in order to generate this specification we don't need the. I don't think that we don't need the response body. We need the api endpoint in order to generate the um open api specification.

E

um So basically we just need opening endpoint and from endpoint we're just gonna, somehow integrate the open api specification in our code to generate that specification from the endpoint.

F

Yeah, that's correct, so yeah. The main problem is to find this data because, as you might have seen, sometimes it's not trivial uh but yeah. In many cases you can just add additional notations.

E

Okay: okay, okay,.

F

So in jenkins, yes, step is basically talking directly to java, so unfortunately we are not using spring or quarkos or whatever uh for the jenkins core. So it's more difficult to capture.

F

Where the endpoints, but again uh yeah, I believe that string connotations in theory. We could adopt them um as an experiment.

A

So but but today's implementation really is based on the stapler framework and there are 15 years of code that is tied to the stapler framework, so so having segar and others who are exploring. This idea know that the rest api is is generated automatically in jenkins by stapler reading the java or reading the java objects. Did I state that correctly, oleg.

E

A

E

That's right, and in that video you said um stepler what is doing it is kind of a data mining. We just I mean doing two things basically kind of converting the rest api endpoints into java object or java objects in two end points kind of is it.

F

Yeah well, it does a lot more but yeah. Basically, it's binding binding for requests and for data. Okay, okay,.

C

Okay, so hey guys uh sorry, I just got to know about this meeting. I like I saw mark's message on the group and just joined, so if so, there are only like three minutes left to the meeting. If I wanted to discuss a few gsoc project ideas, if that's fine with everyone.

F

Yeah, maybe one question we spend so much time this uh question crest api, maybe much or has any questions because.

A

The audio quality.

F

A

Soft now himanchu no noise, but very soft.

B

Hello, oh yes got it: okay, okay, okay, okay, cool thanks thanks! Actually, I did think I could like change some settings in mic, so that must have affected it. So is there like time uh still time? Can I ask my question like is? It is related with plugin installation manager.

A

There is time- and I think, at least for me- I am willing to continue for at least another 15 minutes oleg. I don't know about you and kara, but I've got time.

D

And we could continue, I'm happy to continue as well.

B

I'm fine, uh that's good! Thank you so much also yeah, I was uh doing like I have been following this project, plugin installation manager, project idea. Then uh there's uh like I was doing what I should say.

B

I was preparing and preparing for this project idea and in this skills to study and improve this mention of java, like I would like to ask, uh is there any particular framework that you would like to suggest for this idea like spring or anything uh for this, like particular idea, uh plug-in instruction manager tool like uh you have mentioned java, but java is like so fast.

B

So that's fine.

F

So if you're looking uh for modern frameworks, uh it would be like yeah spring would be your option options. You could also consider quarkos if you want.

G

B

F

So this is also.

B

F

Could consider.

F

uh Yes, so basically I'll.

B

Yeah, please you're saying something.

F

So yeah, I just wanted to say that uh yeah this uh also, this is also quite popular framework, uh these days, especially for cloud native applications. It's uh support for uh building native images. So if you want to export this option, it could be doable.

B

Any other framework that you mentioned and I did not like uh for sure applications.

F

Yeah there are not so many frameworks, so yeah.

F

Yeah, if one of the problems with plugin installation manager that it has a pretty bad cli interface, so, for example, that you're working it uh to let's say pica cli so that you have multiple commands, use proper option management.

F

It would be a good way to consider and yeah. Then, if you want more modularity, you can use quartus or something on the top of that. But yeah. It's a separate improvement uh which just uh makes.

B

A cool difference so like we can go directly with uh only with java, uh like just with java, or we can have like, we can use focus to add more modularity to our code.

B

Yeah- and I I get that like yeah, it's a framework but like if I want to add some modularity. Thank you. Thank you.

F

B

B

Like you mentioned about json uh data structures, package management tools, so, like you were referring, were you referring to the tools rated uh which were already built for jenkins or some other tools uh in this case, which you like mentioned like when one must know about package management tools? So what kind of tools were you referring to?

B

Do you have any idea of any example.

F

You mean tools for jenkins, so in general there are many acli clients. uh There are tools like plugin manager. There are also a lot of developer tools, for example, plugin compatibility tests that uh parent forms different scanners.

F

So, if you're looking in the tools array, it's something would look for all these tools. Yeah, they basically stand alone. All of them would benefit from some improvements here and there. So if you're looking for relatively small projects, it could be an option.

B

Okay: okay, okay, thank you and yeah. That's like those were the two questions that I had. Thank you. Thank you.

B

A

So himanshu and segar feels like we've. We've addressed your questions vibhov. Shall we give you some time and let you discuss you? You had a few project ideas. You said that you wanted to offer.

C

Yeah, oh thank you mark, so uh basically I had a. I had a few ideas regarding uh like the first idea I had was the cloud events. Plugin uh basically jenkins should support like cloud events, uh so this would mean that uh you should be able to create cloud events, and you know jenkins should respond to them, uh maybe trigger jobs and stuff. So this was the first one.

C

Second, one was a project that is already there, a techton client plug-in and hoping if it could be part of gsoc and uh third one. uh I don't know if this even counts actually was the jenkins file runner operator.

C

This is something I uh prototyped and I think it was in december or november, so this one would be basically like running uh jenkins files in a serverless fashions, and this is based based on what oleg built and thanks to him for like building that awesome file, runner, which basically turns uh jenkins files into like serverless executions.

C

So uh I was ho so these were the three ideas I was hoping if we could discuss these and maybe choose or take them.

F

What are your thoughts? Three areas could be potentially feasible.

F

You just need to factor them to something which can be implemented during the jsop timeline, for example, cloud events most likely. Yes, because it's a new code cloud events have relatively simple api, so I think that a prototype could be created uh for a different client and jinx valrana reparator yeah. The main problem is to actually define the scope.

F

So what would a student do there.

C

So currently, uh the so for the tecton client plug-in current. uh I was there are a few things that that could be added. uh After addition of, like you know, uh first, they could add the remaining uh apis that aren't there in the plug-in. Then they could work on some features which make the uh plug-in more dynamic in nature such as uh pre-loading, some stuff and everything.

C

uh So this is this is what I was thinking with in terms of technology and uh about the uh jenkins file runner operator. I was thinking if it would be possible to have a ui in it. So maybe in the operator it runs. A like the operator will have like the instance of an operator will have an instance of a ui, also running alongside of the where the user can see like which all file runs have been run, and maybe execute file runs.

C

So this is this is the scope I was thinking about.

F

It's in principle, it's possible again yeah. There are examples of such implementation, for example, injecting sex. So I think that if you want to propose this project, maybe just do that. Okay,.

C

Okay, so should I create a so what I'll do then for the proposal I'll create a draft and then I'll share it with you guys soon, yeah.

D

The mailing list is a is a very good place to share your draft proposals and you shouldn't hesitate to make a number of proposals, especially since these all sound interesting um that doesn't mean you're, committing to mentoring on three g-stock projects or anything. It's just. This is for discussion and to see what students are interested in working on and we'll see what what um I guess get traction and and um how the process goes, but they all sound like very good proposals. So thank you for that.

C

Okay, that sounds good.

D

C

Now, actually like it's more about like the idea that that was there, and you know it should be implemented, it's not really about the mentoring part, even if the idea just gets executed. It's just really nice to see that so, just like kind of looking forward to that thanks carla.

D

F

So yeah cloud events may be interesting because it's um standalone project, so the student can create it from scratch, but other projects so they're also viable.

C

Okay, so uh I'll, so what I'll do first is I'll, create the first one on cloud events and then maybe we can start from there.

C

The rest we can look at like I'll, create them also, but I'm hoping the cloud events one like I'll I'll, get to help a little on it, because I'm a bit interested in uh having jenkins be discoverable by cloud events, and you know triggering jobs by cloud events itself, because then, if cloud events works for jenkins, uh someone from tecton or something they could, you know trigger uh jenkins job purely through cloud events that would be cool to see.

C

C

So I don't have anything else apart from this to discuss.

F

Then yeah, probably we thank everyone who participated today and please see you next week.

A

Recording will be available and the notes are already there thanks very much everybody.

F