Jenkins Jenkins Contributor Summit - Oct 2021, 19 Oct 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Let Jenkins be part of the Kubernetes

Description

Share some ideas about how to integrate Jenkins as a backend engine into Kubernetes.

A

I just I just joined this meeting break out. True.

A

Okay, thanks everyone. I will start my session and my topic is: uh let jake's be part of the complete um yeah, I'm I'm come, I'm rick and I come from coopersburg community, but and I'm not going to um simply promote christopher. No, I am one I want to share my ideas about how to integrate chickens and communities.

A

So before I get started, I will introduce myself first- and this is my. uh This is a screenshot of my github profile and, as you can see, I I'm proud, probably participating in a lot of projects, and I I love open source. It's uh open source can help me to understand and involve a lot of things so yeah, I put a project.

A

uh The open source, best practice, project first and uh and the second one is the main character of this sensor. Is cooperatives? Okay?

A

So if you want to know more about me, you can visit this page and you can find the link of my twitter linkedin or on other assist.

A

Okay, and if you are already uh familiar with jks you you, you must know, uh there are um plenty of jigs plug plugins in the community, and now it should be over 1800 plugins. You can find from the community and just like the picture at the local on the core corner of this page, you can you can adopt a lot, many plugins to power, your chickens and yeah and make make your tickets like a powerful robot.

A

But the problem, the the problem- is how to uh how to use chickens in in your daily life, and I will give it a three typical um using case and.

A

And the the simplest way is stand alone. You you just start up starter chickens, maybe from a tomcat or from a dark container. Maybe you also can start it from a uh start it in the communist cluster and the the there is a very, very small barrier on this and on this case and butter you'll be armed. You will meet a big limitation about uh on it and the the good thing is, you know you don't know you don't need to know too many things about the chickens uh inside of rejections uh yeah.

A

This is good, but if you have a lot of other requirements about it, such as you want to learn drinkers in communities or you want to integrate chickens with sonic, global nexus or other scenes, or you just have a very, very specific requirement about your csd flow. You you, you might maybe hard to find a perfect plugin for for each.

A

So that's that's what I am and I'm talking about the limitation uh yeah. The second is the advanced way you you can do a lot of other things and a lot of advanced and but it it require you to have a high technical skills, maybe a plug-in development understanding.

A

Maybe you you need to know how you need to familiarize java development or other things. I will explain it later.

A

So in this way, basically, you can do everything do whatever you want, but you also need to uh learn a lot. A lot maybe um uh learns learners learning. Something is a good thing for for you, but for a team and you you might need to consider to recruit some skilled members and the third one is a community integration solution and like uh like kunis, uh I think for now a lot of teams already adopted companies in their environment, but.

A

It's hard to just used kubernetes and, as I mentioned before, as a standalone way, you you, you might need to do a lot of other things as uh to get um to make it be easier to use. So uh a community integration solution um might um must be better for you.

A

It can share the cost with the community and how to understand the share the cost of this community and like if, uh of course, you can integrate the jenkins with uh uh sonarcube and access. You can install all of this uh system or components by yourself and, of course, you you need to maintain all these components by yourself or by your team. So if you mean, if you install like four components you you might need to uh familiar with all these components, so you can be able to maintain it and then yeah, maybe you.

A

You also need to uh write some address a plugin for it, but you also need need to understand how to write a checklist plugin. But um if you adopt a community integration solution, you can and take advantage of the core of the community. You can work with other people who, in in a active community and yeah, let us like a christopher community yeah, as um I think this stand alone is, um is a suitable for a small team or your team and uh yeah.

A

Just uh uh just the stars, use jinx as a csv csd system, and you you don't want to spend too many time or less too many members to uh to learn or maintain uh dragons uh yeah. This might be the easiest way for, and you course you. There are plenty of plugins that you can choose, as I mentioned before, they are all proven sonar 800 plugins, and um it is quite easy to integrate with other system.

A

If you uh try to search a keyword from a keyword from the uh git genius github uh organization, um for example, you can search a nexus, you can search uh griddle uh uh go along or other other keywords. You can get a long list for those plugins, but I don't think it's.

A

It is always easy to use those plugins, because.

A

Maybe this is the tool set for for community jenkins. Community can accept a plugin with a simple flow. Anyone can uh can contribute your plugin to jks community.

A

No matter you, you contribute either as a personal um action or maybe a standard for your company, and but another set of of this we use some plugins are lack of maintenance, as you can see, and some plugins match don't have they commit commit over one year or several years, and if I I remember correctly, the the backup, plugin and already don't have like me and over three years- I I can't remember it so this is the let's request you to.

A

Take a look at the plugin and about the last day of the release and the frequency frequency of the commit okay, so.

A

You can you can use it you as a standalone way, but you also need to.

A

Choose the plugins by yourself.

A

uh About the advanced way, and of course, as I mentioned, you can pour your jenkins in maybe those those ways and, for example, you can implement your business flow with shared library paper.

A

What is the third library problem? uh In my opinion, it's just a and.

A

Groovy screw scripts uh that that can extend jenkins babylon and you can, if you already know how you, if you and try to search group in google, you will you random group, is a script language which is very similar to java. So if you are familiar with java, you can let's groove it by yourself, but um in my experience it's not easy to make sure your shared library scripts and for how to say uh it is not easy to write a unit test or test it manually and yeah.

A

It is not easy, and if you, if you try to use shadow level in your business and um maybe you you will write a lot of scripts yeah but.

A

It might be hard to maintain.

A

uh Another way is that you can build up your your own jinx plugins, but.

A

It's very cool yeah you, you can build a your pro, your own plugin, and you can almost do everything you want, but before that, please try to try not to forget some requirements.

A

First, of course you need to uh familiar with java and the women yeah, because jenkins is written by java yeah. All almost uh almost all the plugins are written by java and the the second barrier is that you need to understand the dickens plug-in mechanism.

A

You can find some documents about how to write a simple things. Plugin or maybe you can find all also fun can find some videos about it, but.

A

The the next thing you also need to consider is uh how to publish your six plugin. You can um host your plugin in the community or you you can publish it in your private environment, no matter which one you you eat. Well, it will make you have more costs up on it and.

A

Yeah, the third one, maybe you you you want to integrate with other system like like sonarqube nexus or, however, and but uh and I think the open source plug-in and focus on to fix a common use cases.

A

If you have a specific particular use cases you you also need to consider to write or extend it.

A

Okay, okay um about the community integration solution, uh you you, you can adopt chicken sex or conspiracy. Of course, you can also find other um other solutions.

A

For jksex it's, uh and it is the best it's based on githubs and you can and you can maintain your environment with github's way. You can put your config files into your git repository and it's good and the chickens x can take chickens or tekton as as as its ci cng and but for jigsak's.

A

It does not have a.

A

It it does not have excellent ui, which uh uh it's only in on you.

A

You can only use a very simple dashboard and for for the ui part group's variables will have a and beautiful and powerful ui, and for those for those you users who who want to who love to use your button and maybe click click uh you can try, uh try to use it and uh for now uh christopher helps uh take jacob's pavlon as cesar engine, and currently we also have a working progress plan to support tactile, and basically it will be um similarly skit github actions.

A

uh You you just need to provide a simple, uh simple, ammo file. It can can build your tasks.

A

Okay, as we are talking about the integration, I think the most excess exciting thing is that chickens and kubernetes working together. Why I say that- and I I will- I will share three points about this. The first one is dynamic agent support um before we before we have kubernetes. Maybe you can install your chicken agent in a vm virtual machine or maybe because you that can can be dynamic, but for companies it will will be much easier. You can, you can have agent, just just like have a image.

A

For example, if you want to have a java environment, environment or java agent, you can have it just have a image of jtk, maybe open gtk and.

A

The plugin of kubernetes in jake's will start or launch agents as needed.

A

For example, you can you can't give the give a label name for like a java. It will find a port template and start it for you and destroy it if your pipeline was done.

A

The second one might be the uh secret and uh credential, and if you are some familiar with jenkins uh and chickens smell and will store some sensitive, sensitive uh information into credential, such as token username password something like that.

A

But as we all know, chicken smell store the almost almost everything as a as is uh affair yeah, which in a fair system and uh but better for communist users, they store their sensitive scenes in a secret.

A

So we can have a map. We can have a map in between secret and credential.

A

If you integrate things into combination, you can just turn and edit or create a secret, and then it will help you to have a credential to use the last one is dirty, and this is a pretty easy way to extend the coordinates. Of course, I will have the example in combinational depth to how to extend companies with chickens yeah. The quality with eyelash said you can store configuration and data into a fair, um but that that is not a cloud native way way.

A

I have a table which, which is part of the my opinion and such as in cooper spirit, helps we and we have to have certainly used public and the problem will be a map of a mapping of the chicken's job.

A

So if you create a theory plan, it will help you uh to create a dick's job and and of course you can modify or delete it and another one is papillary. It february represents a chicken spirit and we we also put all chickens configuration into a country map, and so you don't need to log into jks ui, and if you want to change the configuration you just need to just need to change the configuration in a comfort map um for virginia's build log and artifacts.

A

Currently we don't have a. uh We don't pro, provide the solution, but it's possible to use to store artifacts to s3 or build a log to investigations, and we can get a log directly from that search instead of jenkins.

A

So yeah I I I I already talked a lot about the conspiracy and uh and have a brief brief introduction for the what is a distributed operating system for cloud native application measurement using kubernetes. As is kernel, it provides a plug on the play.

A

Artifact allowance. Third part application to assemble integrate to this ecosystem. You can get more information from this. The two links, uh github dot com, slash, google square or cosplayer.l.

A

uh Yeah a couple sphere is the name of the the project. It is also the name of the community and, as I apache foundation promotes, I applio community is over code, so yeah.

A

This is a picture of what else and, as you can see, we can we support different rules, maybe for, for example, developer, tester or operators. A developer can write their code in in a git repository once the git repository have changes and our problem will build it and do some tests and build and push the image. Finally, we can deploy it to complete.

A

So why could spare adopts jackets as it's a cd engine and as we can see, uh jks you still still use the most popular csct solution and it's reliable and have a excellent black plug ball design is. It is easy to extend it and the for me. The most important thing is: it has a mature and actual community support. You can get a support or get help from the community from a forum or email list I'll just create a issue about the dynamic agents.

A

I already talked about it, so I I've already talked about the computer. So what it looks like this is a.

A

Screen shot of cool spell, you can recommend your covenants.

A

In the in your your dashboard, for example, you can you can change the configuration such as map or secrets from the left side. Just click the the manual configuration of check, all the series from here and, as you can see, I already enabled the dells components.

A

So you can see the local chickens.

A

uh For for this environment, I installed in a case ready. uh Actually it is quite easy. I will so learn. Serif later.

A

This is the picture of the devs projects and, and the adapt projects sounds like a for a small group or small team. You can create your own jobs projects and help different rules. Permission management.

A

And for construct helps uh we we have. We use reuse uh namespace in kubernetes to um to map uh to have a mapping between and each and jenkins folder. So.

A

This is a this is a picture of the store which is a source to image, and you can build a image just select from a source.

A

Git repository or you can build a image from a artifact, it might be jar over or maybe I'll see.

A

This is a screenshot of the pipeline and you can see the stages steps from the graphic page.

A

It is very similar with jake explosion, and actually, if you, if you want to have your own dashboard or something like that, you can, you can work with us and see how we achieve that.

A

Yeah, the the awesome things is that you can edit a standard, jack, stapling script in graphic way or in the script way.

A

A

If you, if you try to try to read the document of jks, you can see jacob's ex pro, provides a excellent and powerful command line. uh We also try to try to provide a command line to help our users to use and manage manage our devs system.

A

As you can see, you can create the later edit on even gc. Your public runs yeah.

A

Okay, this is a simple and experimental dashboard for in the terminal. I uh we got the um the idea. We inspired it from uh key my eyes, uh which is a awesome uh project. So you can, you can see the all namespace of google devs from the left left side and you can.

A

You can use the key as a shortcut and switch to switch different, namespace and check the prevalence. As you can see, we have a pro. We have tips for how to use it like you can tap r and it will start a public and you can also create a plan.

A

Yeah I didn't, I didn't, have that page to create public and but.

A

As I mentioned, we have a mapping between jenkins job and the jake's builder uh to a combination. This is the crdo tablet.

A

Yeah we we put the kids get reposter address into a spec.

A

There is back yeah, so I think it's. It is possible to to let our potential chickens, tickets and companies users to use this kindle solution, so we we are very open to accept any address or any thoughts on on this. So if you, if your team and also want to increase things with completes, maybe you can try to contact with us to improve this solution.

A

A

This is crdo papillons, which represents aging skills. We put some metadata into the annotations. As you can see, you can get the stage stages, the stages and the um stage states or the whole stage from the annotation, and this is.

A

Under you, you can also get the build number uh it's filed here.

A

Yeah, I already uh show you how we achieve my opinion and just have a crd to to do it, but we also have some potential issues.

A

I I don't know if you already know it or you, maybe your team might be already try to figure it. So the first one is support a unknown chicken's plugins and for now we have a. We have a project to to assemble the jenkins particular version of jenkins and the particular a particular list of plugins. But for some reasons we cannot support all plugins or potential plugins and the the most the the um the important thing is. If you install a some some um some plugins you met will meet the.

A

Compatible issues, so we lack of a extension mechanism.

A

The second is the clarity way. We don't. We we still.

A

Still don't have a excellent way to get the log of drinking, build our chickens and we we have was some kind of uh ap server or um which is similar with the prophecy.

A

So we, if you call that a ps4 and the episode will call the apl giggins, but we we don't want that because it relies on this state ball of jackets. Oh, we hope we can break that.

A

It is similar with his artifacts and.

A

Both this part be aware of the change from chicken sand.

A

Currently we we can create, we can modify, we can delete a chicken's job or build, but- um and we don't have a um better solution to get the states of a jacob's job or build.

A

Currently, we we we use api polling to do that, but it will cause some unnecessary uh api calling. So what? What is the possible solutions- and we currently we try to do some research, uh for example, uh define a crd for the external checklist, plugin support and yeah, like I said uh it is through under the search for this part of the.

A

We matter need to consider the backend and the front end part and, for example, if uh if a user wants to install a and like a dynamic parameter pipeline parameter plugin, and we meant to consider to support it, have a api or under the ui part which which allow users to choose the parameter from the ui.

A

Okay, the quantitative way: maybe we can get the build log from unit search, so we don't rely on the ipl chickens, so we just use together in a search and get artifacts from s3, and the third part is that we we don't actually don't want to have a direct api call and so event driving might be a solution. So if we have a plotting event, so we communicate with the event center and jake is also communicating with it. So we will deeper locate it.

A

So maybe we don't need to have a uh have a lot of jenkins api calling and.

A

So that's all I want to share with you and thank you guys. This is my github profile link.

A

So I guess maybe we don't have a. We don't have too many participate so.

A

Okay, that's all.

A

I will stop the recording.