Jenkins Google Summer of Code Office Hours, 25 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GSoC 2019 in Jenkins. Phase 1 demos, part 1

Description

During this session Google Summer of Code students will present the results they achieved during the first month of coding.

Agenda:
* Parichay Barpanda - Multibranch Pipeline support for GitLab SCM
* Abhyudaya Sharma - Performance testing framework and Role Strategy performance improvements
* Prastik Gyawali - Artifact Promotion plugin for Jenkins Pipeline

Talk abstracts: https://docs.google.com/document/d/1rEIzZ56CObCIcxtPlQoxn-7B9oAchB82Vh4zd61AJ98/edit?usp=sharing

A

Yeah well like welcome everyone who's watching. These demos today is urgent 26. We have just part of google Summer of Code demence by Jenkins students. We have two presentations today.

A

So appreciate we'll talk about multiplayer support for the trapezium. Then a video we'll talk about strategy, performance improvements and performance, training, test framework for Jenkins and then first it will talk about the ongoing progress for pipeline support in promoting this plug-in. I. Just make sure do is my screen.

A

B

A

This meeting has been a broadcast in b2. If you interested, there is a link, and if you want to ask any questions, please ask the questions and the meter chat. Jenkins project has a special chat, or some of its gene says: there's a JSOC I seek special interest group and using the demos, if you watching YouTube just the journey to this chat and ask questions back, we will make sure to ask to the presenters and yeah with the introduction.

A

So yeah as I said to be, is only part of the demos tomorrow we will have a second part at 4:00 p.m. UTC, so three presentations there and yeah. This is just the beginning. It's a just coding phase, so stay tuned for more demos and more presentations in the next few months, because there will be a lot of things to show.

C

Ok, so I I'll just turn off my camera. To save my bad word, okay,.

C

Can you guys see.

A

C

A

uh Blame to Steve your screen and your marketers presenter, so everything will be recorded correctly in b2.

C

But welcome to the presentation of multi blinds biplane support for gate slam, g-shock 2019 project under Jenkins organization. Okay, this is about me I'm, very shy. I'm from Odisha India I like to contribute to open source projects and I am also a big football fan, and these are my mentors. They help me with code reviews, I'll, give suggestions and also contributions to the code, so the goals of the project.

C

We are implementing a good lab, API plug-ins that perhaps a good lab for java api s--. Then we will create a good lab plugin that is lightweight and depends on get lab. Api to delegate or I mean get lab API plugin to delegate all the good lab API calls and with lab branchless plugin for multi branch, pipeline jobs and further organization, but difference pipeline jobs are for repositories with political branches either for creating new features or the multiple branches can be for creating new features or for creating noise requests and folded.

C

Organization is a collection of these multiple multi branch pipeline jobs for rare use it or or a can I'm. Having some background noise.

C

I get slapped blue ocean plugin for pipeline jobs, which is based on days on github because it was like we have presently. We have support for github and bit bucket, so this will add, get slab support in the ocean and we are also aiming to have a clear and efficient design of the plugins are trying to make a clean and bug 3 UI and also writing modular codes for future extension of these plugins.

C

This is the first release of project good lab API plug-in. Basically, labs get left for Java ApS. So why do we need a Piatt like okay? So there are two reasons for this: I mean the first reason is about why we need an API plug-in in Jenkins. This is in order to have a centralized control of API releases. This reduces the maintenance burden of the child, plugins that depend on the api's, so we can create releases from the one get lab. Api I mean the end login and all the rest.

C

Child Huggies can decide whether they want to upgrade or not. It depends on them and then the reason for creating the specific bit lab API plugin is that people want developers want. The API is to be stateless, I mean they just want to make API calls to an end point and not concerned with the API implementation. So this is a practice used by github plugins. So we are trying to do that in this and then well.

C

There are multiple reasons why we chose get lab for Java a place that can be seen in the wiki page. I'll leave a link below with lab branches plugin.

C

This is I'll just give a quick description of what the features they are and then we'll go directly to the demo. So.

C

This part is about. We have implemented with lab server configuration and basically configures our get lab server in Jenkins. There are. There are some new features, such as creating a good lab personal access token inside lincoln's itself. You do not need to go to a server to create access. Token indirectly, do it in your Jenkins and to allow duplicate server increase by providing a unique ID to them.

C

I mean you can have same with less ever, but multiple increase of that, and we are presently targeting to support, get slam server versions above 11 point oh, but since our API support have calls for support, API calls for so we'll about nine and, above so in future users requests or there are need, for. There is an equal when we can do minor changes to the code, and we can support lower versions.

C

Hazard and here, adding Jake asked support, which is called at the user, configure the UI without going actually to the UI and just right by using a while that that is quite convenient for users, especially using a continuous integration.

C

Then this part is about what how it helps our developers. We are creating a cleaner and I mean there is a gets lab plugin, but what we are trying to do here is I mean a cleaner and modern form is which supports API plugin of features such as token clear that etcetera the most. This plugin is quite inspired from the other plugins like github, and it's not plugin aspect.

C

So we have support for get slapped a PA, plugin incremental fields as well, which is used to create incremental releases that can be used in ideas to check whether the downstream plugins I mean the child plugins that depend on the plug-in work properly.

C

If it is a work in progress on a new release and maven checks and which emphasises good coding, side and Java 8 compatibility as well, we are using a trying to add as much as Java compatibility using screens API, which was introduced in Java 8 and with it, makes for a more modular and more readable, so we're using that that we are using Gentle one of the later a later version of Jenkins, which is two point.

C

One point: five zero, then the main reason is it implemented a persistent descriptor that automatically calls load, meaning it loads xml data from the disk when the object is tensioned ciated I mean, although the UI reflects the state of the object at runtime. So this this was added in this version or sentence. So we are depending on that, and we are right now discussing that, if, if it is possible to upgrade to the latest version of jenkins in the next period is okay. So this is the poor organization.

C

We are using iota Jenkins the plugins package, and here is a package which is it loves our country, achievement in sport related to the server configuration in our plugin the credentials, but it implements of IP it is.

C

It extends an ID credential which implements a person access token, which is specific to lab or here so this made the credential at least both gets populated with our base Fingaz lab person access token, and here the service we have server configurations, get lamp server, it represents of one, it is a abstract experiment and it represents one get lamp server entry. It has all the fields and etc and its life service.

C

It extends no global configuration and it also implements the first persistent escaping that I described how it helps to use the object state at runtime, and we have one which is this extension, which is circuit left. First access token it Creator, which basically helps us to create our token inside deck inside.

C

So this is what are you a this? This is the basic configuration of gait lab server. There is a unique name that is generated by which the little elf servers are filtered and you provide the heat lamp server, a URL which is by default you it well populated, with the link to the gate, lab Community, Edition cells.

C

So- and here is the credential early spots that you can for plate by selecting the drop down, and if we want to select your if you want to manage they'll get lapped by boots on your Jenkins server, so you can check this option right now. This is a work in progress, because this has a correlation with branches. So, after implementing the glances with well into this.

C

This is get lapto connector, so you provide you get lab server URL here and you can how you have two options: either use your credential, that is a persistent credential or user name/password reduction that persist in your Jenkins. You can use that or you can directly provide your credential to the UI.

C

I mean think it's configuration and create a token. It will give you an idea of that. Okay, so I'll go to the demo of how it works.

C

So this is my local Jenkins instance.

C

Go to manage them things configure systems.

C

So so I have a good lamp server configured. Yes, let's delete it.

C

Okay, so for the initial setup you will get get a hit like this state and you have to a do get observer here. So this is the unique name and server URL retention, so they. This is a previous idea of the token. But maybe you want something else, so we can. We can directly create our credential from here. We click add an advanced convert login to login password to token.

C

So this is the server that I'm growing, so I use. My I do not have a retention, so I'll use my log.

C

I, just communicating with the server to create a token for you.

C

Okay, so the potential is created with this particular ID. That's the b-29 see.

C

So I choose this p29 see we have a book, so this is a work in progress, I'm, just ignoring it right now, so I'll this well, you can just a is the connection here.

C

Okay, so potential is verified for user and if you see this, this is a invalid token.

C

C

So this was about how you configured using the UI, and so as I was talking about Jenkins for DES configuration. This is the yml file that you include. This is a sample of that, so it is, it contains two parts. This is this is the credentials you can define the scope, giver ID. You can define our token here.

C

So this is a person invalid token, but you you can either directly give your token here or you can use an environment variable for that Jenkins I mean Jake, asked, has a nice documentation about handling secrets and the repository and you you and the second part it it is. It deals with our plugin, which is creating a server entry with credential ID, which is referring to here, and if you want to manage hooks and a name of that ID, this is optional and the server URL, so I just show how it works.

C

This is my Jenkins flyable file.

C

The you can go to this. You path configuration as cold and yeah I paste the path and I'll apply. It says configuration applying I'll just refresh this page.

C

Okay, so so this is our week, we supplied the server you get left at all and this this this has created entry for that with the server name as well, and the credential ID select a time when it works.

C

Okay, so this is all that I worked in this phase of one evaluation, I mean even putting things, so these are the resources that basically is the source code source forward, and we key and the issues that we've solved and the open issue doesn't- and here is my blog link. If you want to know more about the progress and what all work I did you paint this refer to my blog I, weekly blogs,.

C

Okay, so this was me at the beginning of coding phase, but but slowly I learned from other plugins for days, Jenkins, documentation and metals code review. So this is all a lot of bugs and this one.

C

Okay, Thank You Jenkinson's, you, sir, for giving me this opportunity to work on a real-world project and.

A

Thank you too, so the project just started. Yes, so Markie or anybody else. Would you like to add something I.

D

Would like to add Prachi, but this is really really good and I am really proud of you. You did a lot of hard work and it shows so. Thank you very, very much.

C

Thanks Becky thank.

A

A

It looks now so in JSOC chatter that some discussions about which depth working in use for Jenkins but yeah, not that much specific to the project. Okay,.

C

B

Was the question on.

B

C

I'm not able to hear you sorry. This is my oh good to know. Yeah.

A

C

A

Yes, I was just asking about the dark theme and you will be using on your slide somewhere. So is it an existing theme for Jenkins or.

C

How do ya it's a it's a Google slide. They have an option, but acting. Oh, so yeah, it's just below site. Okay for the for the Jenkins. You I, you me yeah, okay, the screenshots okay yeah, so they did directly have a option to convert it to invert the color, so I data County, specifically Jenkins.

A

Okay, thank you. So if there is no other questions, we can just pass it to the second demo.

E

I'll, be there Radian yeah I share my screen. Okay,.

F

Hi, everyone and today I will be showing you a presentation on running gmh benchmarks for Jenkins plugins. So what so? Let's just start with what is JM it. Jeremy Tizen stands for Java micro benchmark harness, which is an open, JDK project. What jmh basically does is it runs the benchmark watch multiple times.

F

First, it transform up iterations to warm up the java virtual machine to allow compilation into a Java bytecode. After that, when multiple benchmarks have been run, it run, it can calculate various measurements like average time, the throughput that is, the number of operations etc and finally, to improve the accuracy of the measurements. Jmh runs the benchmarks on multiple folks of the gem image, which leads to reducing the errors in measurements.

F

Now, let's just first start with a brief overview of the of how this framework works. So what the framework is available through Jenkins test harness and you can easily use your Jenkins inside the benchmarks. We have a main profile in the plugin pump for running benchmarks, and you can even run your benchmarks directly on see. I didn't convey. Oh, there is a pipeline step for that, and you can also use configuration is code for setting of the benchmarks now to run a benchmark.

F

You basically have to run it directly from a unit test, all the benchmark classes that are annotated with a gem. Each benchmark are automatically found and a temporary Jenkins instance for each individual folk of the jmh benchmark is started.

F

So let's take a look at how a benchmark looks so, basically, what you have here is: you have to create a state class. These state classes are automatically instantiated by the image and these are the only way to pass information into any benchmark. So we have two methods here for setting up and cleaning up your resources. The first one is the setup method and we or their own method.

F

Here, then, your benchmark methods are annotated with benchmark and you take the state as a parameter to the benchmark function, and your benchmark code goes here now to configure the instance that have started for the benchmark. You can either use your Java code, your methods that you have in your plug-in, or you can use Yaman files from configuration s code. We do not yet support using local data, that is, the conflict or XML files for Jenkins own, but that is that is the future improvement.

F

Now, let's take a look at how you configure a pinch mark using configuration or skirt, so just like the previous one, instead of the MH benchmark state. Here we use the configuration, is code jam, its benchmark state and we need to provide. We need to override two methods. The first one is the path to your Yama file and the other one is the enclosing class, the class that contains the benchmark. So in this case it's the sample benchmark and you can continue like you did in the previous benchmark for viewing the benchmark reports.

F

The the benchmark reports are available through either a Java object. That's available inside your unit tests that you are running the benchmark from and analyzing it you can even fill the bills if they fail to reach a given threshold.

F

The reports are written to the disk after the benchmark, completes and Jason, and these reports can include multiple measurements like I said before, so you can measure the average time the the time, the single short time. That is the time without any warm-up, and many more things are for ease of visualization of the reports. There is a JMS report plugin for jenkins, or there is a JH visualizer website which you can pass the Doosan reports. So this is a sample report that was generated using geometric eliezer.

F

This is an actual pull request from a row strategy plugin, and you can see big difference. We can get from a small change now, let's take a look at how it all works. So basically it all starts from a Jenkins test harness which contains the benchmarking framework. This Jenkins test harness is made available through plug-in pump. That's 346 the latest release and configuration escort uses them to configure the benchmark using configurations code.

F

Then we have the pipeline library that has a run benchmarks, which runs benchmarks on CI Jenkins aheh as a part of your pull request builds. And finally, we have a row strategy plugin, which uses all of them, so this basically contains the poles of a benchmarks that were created during this coding period. The benchmarks are now run as a part of the pipeline, and now these will help us in the next phase is to improve the actual performance and measure the performance improvement of a particular change, some bugs and challenges that I faced.

F

So the first thing is the test trunk issuer from Jenkins test. Harness will refuse to be Marshall. That is something related to configuration, is code and JEP 200. The other thing is the jetty server on which Jenkins runs is leaving stray threads. So this causes the image benchmark to wait for 30 seconds for each fork, and then it forcefully kills the JVM another issue that I faced was the I used the deflections library originally for auto detection of benchmarks, but that seemed to cause each person conflicts with the Guara that was in Jenkins core.

F

So now that has been fixed, so the next steps would be more benchmarks for the road strategy. Plugin and I'll start working on improving the performance. Then we use the benchmarks for measuring the actual performance improvements and address feedback from the adopters next I'll just move on to running a simple benchmark from the road strategy plugin. So we have one benchmark here that is using configuration is code. This is the benchmark method.

F

This measures the time to create an ACL object. That is something related to the road strategy plugin and we have a benchmark, run hitter. So inside your ID, you can just run it like any other day unit test.

F

Let's just wait for it to compile.

F

This is configurational Scott setting up the instance and.

F

Now, as you can see, the first form of iteration has been completed on. It took point eight four of a microsecond for an operation.

F

Let's just wait for the bench up to complete and I will show you the results.

F

So, as you can see here the as you can see here a gem, it is running multiple iterations of the same benchmark. This same process would be carried out in another folk of the JVM, so in the meantime, I will show you a report that was created previously.

F

So this is what a report looks like now. This report can directly be fed into Gemma's visualizer, so we have a report and we just ordered her, and we can just see how fast it's going on.

F

So I would request you to please contribute to help improve this framework. You can either contact me through the grid chart or through Jenkins developer mailing list.

F

Last but not least, I would like to thank my mentors. Oleg manager runs a year and sukoon for their help and support throughout the coding phase and thanks everyone.

A

Thank you for the presentation, any questions.

A

No questions in the chat as well, but you this is not the first time when we have this presentation, meeting project meetings before so questions there and if.

A

You use it so as I mentor I would like to say that does a great job so for the first coding phase we didn't really expect to the framework to be fully booked iced. So we started from implementation of strategy playing in and it was completed, maybe in the first week of coding, or something like that. After that, we spend a lot of time for the choice and get moving code too.

A

Is this harness to plug-in poem passing through code reviews of my daniels, the command engravings and yeah I would say that maybe half of the discordant face a VHS spent not on the implementation of the framework itself, but on product eyes in bed. I guess it was quite in my experience one, and it's really great to see that, because now all these components already available to users in production components so yeah you can just take them and use them, and it's really easy for developers a lot to be integrated.

A

A couple of tests in my own plugin and yet I'm looking forward to is more adoption for this framework in Jenkins. So yeah thanks a lot of you did. It's a great progress and yeah for also there's a plug-in itself. We got a couple of performance fixes. We should improve the situation a lot in the next days.

A

Hopefully we will enter this code soon as well. Thanks a lot for your support, yeah. Thank you. So no new questions.

A

Okay, so pristiq I, already yeah.

E

A

E

E

Hello, everybody I'm, a prostate given D and currently I'm doing a project on the Jenkins as part of the google Summer of Code program, and my project is artifact promotion plugin for Jenkins pipeline, so I quickly give a short introduction about myself. I am a plastic currently I'm pursuing my engineering studies from NIT, Jaipur, India and I'm. Currently, in my second year of my engineering studies also, this is my first time participating in the google Summer of Code and also like my interest and inclination to its software.

E

Automation has brought me to contribute to Jenkins and I'm, very so, apart from that, I'm also interested in data structures and algorithms of complex systems and I I love participating in hackathons.

E

So that is all about myself and so talking about Mendes I've had four mentor is John Ross all again Matt's two of them were technical advisors, so, like in this presentation, I'll give you a short overview and introduction about my project, the updates and, and also the updates about what I've been doing in the past month and also about the future implementations and approaches to competing this project.

E

So that is what I will be sharing, so first of all, I quickly give a short overview of my project, so it all started with the promoted builds plug-in not supporting the pipeline structure, probably because it was a developed before the pipeline before the pipeline was introduced in Jenkins, so as a result of which the current implementation does not support pipeline. So this is the reason behind the inception of this project is to make the promoted builds compatible with the pipeline.

E

So like what would the new feature do the new feature would run and on demand promotion when the pipeline build is already completed inside the pipeline and and the promotion would be in a way a bit different than the conventional promotion in such a way that the new promoted build would instead trigger a new promotion job a new promotion job would be triggered. After all, the promotion, criterias conditions are met, and this promotion job would be containing all the bills and job information of the upstream jobs that actually triggered it.

E

So this standalone promotion job would have would be having all the information of the option, jobs that triggered it and all the conditions consequences related to it, and also after that, we also expect to attract the flow of those builds inside the Jenkins by using the free printing engine. However, like this, this was our initial planning. There were certain changes incorporated in the community warning phase and first coding theorist, so I'll talk about the changes. What would the actual changes?

E

I decided in this feed piece is, first of all, a new plugin, as the name suggests, artifact promotion plug-in a new plugin won't be made, but instead what would be happening is we'll just incorporate all these new features and we just merged that immerse them into the existing promoted bills.

E

Plugin and we'll be having a type of monolithic code for both for both for both freestyle and pipeline projects, and the reason behind this is because, even if a new plug-in is made even if the new plugin is made, a lot of dependencies will be still come from the existing promoted build program. So it is, we thought we discussed this with our mentors and we thought that it would be better if we incorporate all these changes into the same plugin and have both the functions inside the same plugin.

E

So and also what I like to say is in the existing promoted Bills of many of the modules extension points are tightly coupled, so they'll have to be refracted before using them for pipeline, so our main focus, the first focus was to refactor the existing extension points instead of detaching, as mentioned in the proposal, instead of detaching the existing of extension points and making a duplicate of those extension points will be refactoring.

E

The existing extension points already being placed in the promoted bills plugin and also will be making a new classes for other classes, like promotion processes, these classes. These are very much big modules of the promoted bills and we think that instead of refactoring just making an intact duplication of these modules would save a lot of time and help the plugin work smoothly.

E

So in the new workflow, the design for the pipe and promotion that was decided after all, these changes was that input the like previously like a currently what happens in promotion is input is taken from the web UI, but instead of that will be since we're using it, since we're aiming to use it in the pipeline will be taking the inputs from the declarative pipeline scripts. Making scripts and giving commands through the pipeline strip, and the same extension point that have been in that day is.

C

E

The Freestyle project, it would also be used in the pipeline promotion but after proper refactoring to support it with the pipeline by making them pipeline compatible, also, and also, as I've already said, that the new analyst classes would be introduced and the and after that, if the promotion, criterias conditions are met, we'd be triggering a new promotion Java as already discussed and pass the bill information to the new promotion job and use fingerprinting engine to track. So this is the final design for the pipeline.

E

After discussing about all the changes, so next, like this, a flowchart would give you a short description about like how the previous promoted builds and the new anticipated from that builds will look like the previously it will. Just the input would come from the web UI now it will be coming from DSL from the declarative pipeline and after that will be the factoring, the promoted modules, promotion, mode use and after that, we'll be triggering a new promoted job and after that, promotion would be deemed as successful.

E

So this is how the promotion would be working in the pipeline. So like talking about the progress of coding, if one I'll give you a short read about those, it was more or less focused on the refactoring of existing extension points. Also, there were some attempts made to introduce new pipeline promotion logic by making new promotion classes and also some DSL scripts. Steps of self-promotion was also introduced, but these are still in the development phase. Since these classes are highly evolving as we go on adding new classes, these classes will eventually evolve.

E

So these are still in the development stage and I had some of the extension points. Like promotion badge function, the mission descriptor war also defected, but they're still undergoing the unit testing, therefore getting merged into the promoted bills, and these are the uniqueness about these extension points- is that they highly depend on other modules, as I already mentioned, that they are tightly coupled to other mode deals.

E

So for that, due to the factors on something or something as promotion condition, descriptor, we had to again make changes to many of those dependent classes like upstream promotion condition self manual. These were also changed and also new classes. Interfaces like promotions run, I accept I would also add it, but still they are in the unit testing phase, since this coding phase was more or less related to the refactoring of the promotion logic.

E

A particular demo for this is I, don't have a particular demo right now so I, just without further ado, I go on now with the plants that III plan to execute in the next coding phase is to just complete all the refactorings that will not done in they're not completed in the first coding.

E

Phases are and also unit tests off, or do all those modules and introduction of new DSL steps have to be done in the next coding phase, and these are all the relevant links I'll if you wish I can put them on the grey chat. So this is this all from you today.

E

A

Thank you for the presentation, any questions from anybody.

A

It looks like we didn't get so many questions morning sessions so yeah. We see how we improve it next time.

A

And like the not so yeah, if you have any questions about the presented eminence, please go to the Jenkins edges or web page. Just a second I'll, probably just show it.

A

So if you see my screen, you can go to.

A

Cancel your projects, JSOC. If you go to this page, you can find all information about.

A

Ongoing projects for Jenkins I really started from seven projects, so one project has been cancelled by now, but there is a lot of projects and a lot of just information you can find. For example, if you want to know more about those strategic performance improvements and about performance testing frameworks, you can just go to this wiki page.

A

You can find all the information and if you want to ask any questions, each project page has links to get their channels and you can use these links in order to join project charts and to ask questions in many cases that they're just integrated is prodigious. For example, this one is a row strategy plug-in it's a either chat for all strategy, pretty much the same for other projects. So, for example, if you go for multi branch pipeline, forget her anga top, did you just get lab bench source plug-in as a link for promoting builds?

A

That is also project chat being used to, though the website has been all the data is available and just go here and find the demos what happens next for G sub projects it as I said tomorrow we will have another set of demos, so there will be a three demos, one for plug-in installation manager, SLI two by Natasha, another one will be about remote in over a patch after working on kubernetes by a long and they will go say demo for work, Mouse plug-in your improvements.

A

This is a major project for change and hold the U is for that. So, if you're interested to see what's going on in these projects, just join these demos and we in one month's. We look up another white session where we'll have more presentations and we will be able to present content which is more ready to release and hopefully, all the projects you have alpha or beta releases by this time.

A

Okay- and that's it for me, any last-minute questions, I.

D

Don't have any last-minute questions, but I would like to congratulate everybody. It's a very job, walk very good job. Well done.

A

You know indeed so yeah Jae Seok already always helps us to target some critical areas of Jenkins which didn't used to get much much attention before. For example, a sort of pipeline and promote its bills is one of the most vaulted Jarecki gets more than 100 fold, pretty much the same for multiple supporting big drop yeah at all strategic begin gets a lot of 4s collisions about performance issues.

A

So even if it's also a great project, especially since we all said, deliver the test framework for all 13 developers, so yeah thanks all for your work and see you next month.

A

Thank you really. Thank you. Liam I'll stop the broadcast. If somebody wants to stay on the call of today to discuss some topics, we can do that. Okay, thank you.