Jenkins Google Summer of Code Office Hours, 30 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GSoC 2020 - Jenkins Windows Services: YAML Configuration Support project demo

Description

Presentation by Buddhika Chathuranga, a GSoC 2020 student in the Jenkins organization. On Windows machines, Jenkins server and client can be installed as Windows Services in order to get better robustness and manageability within the system. This is a functionality bundled into the Jenkins core directly. When installed as a service, Jenkins uses the Windows Service Wrapper executable (.NET, written in C#) which is being configured by XML config files. Currently, there are only a few configuration checks there (no XML Schema, limited validation, etc.), and it’s often that the service wrapper is misconfigured by Jenkins users. In this project I update Windows Service Wrapper to support YAML files as configuration inputs and to introduce better configuration validation during the service installation and startup. Usage of YAML should simplify configuration management in Jenkins, especially when automated tools are used.

Talk abstract: In the presentation I will talk about a brief description about Windows service wrapper. Then I will talk about project tasks. Under that first I will talk about YAML configuration support. Then about the new CLI and XML schema file. End of the talk I will demo on YAML configuration support and YAML verification with the JSON Schema.

References:

* Project page: https://www.jenkins.io/projects/gsoc/2020/projects/winsw-yaml-configs
* Repository: https://github.com/winsw/winsw
* YAML Support Documentation: https://github.com/winsw/winsw/blob/master/doc/yamlConfigFile.md

A

On to the next demo, which is jenkins windows services, yaml configuration support.

A

Thank you, okay. I'm going to just share my screen.

A

Okay, you see it. Thank you.

A

Okay, okay, so uh welcome everyone to uh jenkins industries, uh completion, support uh presentation so from sri lanka and I'm undergraduate of moto unicity- and this is my first time of I was out of code and I had a little bit of experience in web development and uh thank you alan.

A

So this was my background. Okay, uh now, let's move into windows service wrapper. So if you talk about windows, stepper, uh so windows desktop is a tool which we can use to run jenkins as service in windows. Machines and this services actually features actually bundled into the jenkins core itself, and we have to provide a lot of configurations into windows wrapper. So at the initial state of this project I mean before we start this led so project.

A

uh We will configure windows server by an xml configuration file, so you can find the project represented in the uh given link and if we talk about current configuration, actually the current configurations mean the configuration that we had uh at the initiate stage, because uh the thing we did in this project is update this configuration management. So, uh at the initial stage of this project uh we configure the windows service wrapper by an xml configuration file and xml configuration file should be uh name should be same as the explorable file name.

A

uh I mean the system execute file, name and xml file should be located in uh directory via windows. Server exportable is located, though in ancestrality. That's how we, uh how the approach that we had to find the configuration file and some problem that we had at the initial stage is. We can't specify the configuration file uh from command line interface and uh there are no it sme, schema, validation and uh yeah.

A

So the uh how configure management was at the start of this project. uh So if you interested to find a sample xml configuration file, you can find file in the given link.

A

So uh this is how a sample configuration file I mean the xml configuration file, looks like uh in windows series, but there are a lot of composition apart from this, and they are.

A

This image contains only a few of them, and so my project in google summer of code 2020 was a configuration as a yamaha configuration file. So if we talk about the project task that we had to complete in this as ml configuration support, is the major deliverable and a new cli and xml schema validation on other deliverables that I have to upload.

A

Actually, those second one added that the edit while we are doing the project and my proposal contains uh the configuration support part and while we are continuing with the project, uh those uh second and third deliverables comes, and those are awesome actually and uh okay. uh Let's talk about uh yamaha conversion support and why we decided to move into ml configuration. So you know yaml is less verbose and much more human readable than xml. I mean. Xml is a great language that we can use to manage configuration uh anyway.

A

uh Yaml is uh becoming more popular among configuration, management tools and also ml is uh lightweight than xml and also json. So anyway, xml is a really good tool anyway, and you know, yaml is becoming uh much more popular among the community, so I think it's really fair uh to uh provide yaml configuration support for industries that for available uh use.

A

Okay- and this is how uh sample yaml configuration file, looks like uh there are a lot of communication apart from this as well. So here are a few of them and obviously you can see it's uh much more human readable and that's kind of beginner friendly than xml.

A

Okay, and uh if you talk about the talks uh that we had to complete uh under yaml configuration support. So first thing that we have to do is uh passing ml file. So we had to uh convert demo file into an object graph, so status is released.

A

Actually, we released a feature uh preview release a few days uh ago, and uh you can uh find a pull request uh that I created in the giveaway, though submerged actually and the release has been released, and I use the third part library which is called yamal.net in order to pass the ml file, which is awesome library to pass yaml files in cshop and.

A

Though we uh decided to go into yammer configuration, so we will continue the xml support as well. uh So definitely we have to uh update windows wrapper as we can support both xml and yammer. So I had to update that as well, and uh I did that and released and you can find a full request regarding this task in the given link and uh uh if you talk about uh extension, support for windows, extrapolate configuration so as in jenkins, uh we had a few uh extensions in windows series trapezoid.

A

So currently there are two extensions uh which are run away, process killer and shared direct mapper. So uh we are providing uh also those uh extensions need uh configurations, external computation, so we are providing configurations to those extensions uh from uh the main configuration file that we are used to configure windows service, so those extensions needed yaml config support. So under this pull request, I have done the those uh that thing and uh it's also uh you can find that feature in the released as say and if you talk about schema validation.

A

So you know when we are writing a ml file. uh There are a lot of configuration that we have to provide. So it's really uh fair to provide some schema validation, support for users so because they can easily do uh specifying those uh configurations. So at the moment uh we have provide external uh sorry, json schema file to validate uh yaml configuration schema, so users can use some utility tool from visual studio marketplace in order to validate your yaml confusion, file using json, schema and yeah. Definitely we will uh integrate this feature into windows wrapper.

A

uh So then uh definitely uh we can validate the configuration file uh before uh windows extra using those configuration I mean at the moment. uh The configurations are used on demand. So if there are any uh mistaking configuration that you specify, uh definitely a service will be crashed. So definitely we have to validate and verify this configuration before using so up to some extent. Now we can do this because I have done a little bit of validations uh using this json schema, but then we have to.

A

uh We have to do uh a little bit uh in this validation part, so definitely uh in future. Even after so uh I am excited to do that task and you can file the json schema file in the given link and this is currently merged and you can find the pull request in the given link.

A

Okay, some unit tests and documentation. Things were done during this project and there are a lot to do in unit test and I think documentation is kind of okay and but but anyway, I have to do a lot in unit test coverage. So I definitely will work on that.

A

And also you can help us by contributing uh immunities and documentation, and so if we talk about the new command line interface, so current command line interface has only commands. I mean providing all commands and options. I mean all the details will be provided as arguments command line arguments, so we need to let users specify the config file from command line and we need to let user to skip the schema validation asset when they are providing the command to uh no source wrapper.

A

So we needed a much more user-friendly and much more manageable uh command line interface. So we started working on that and uh project has been created for that, and there are a few things to do in this task, and this will be merged into a version three patch that currently uh we are working on that and if you talk about the status about the stars is not much date and yeah.

A

Anyway, you can contribute to that and xml schema validation and via validate xml schema against an xsd file.

B

So there's open pull request for that and there.

A

Are a few things to complete in documentation, part and test coverage in xml, schema validation, so yeah? uh Those are the left in uh semi, schema, validation, part and uh yeah. uh That's uh all about my project and uh now, let's move into demonstration so.

A

I think you can see my uh command line interface. Yes, yes, ah thank you. Okay, okay. First, uh let's see uh how uh we can use uh the xaml configurations uh with windows extractor. So this is the uh configuration file uh that I'm going to use to provide the configurations into windows wrapper.

A

So this is the basic configuration file and I'm using this.

A

So we can first install the windows wrapper using, as is managed windows service manager. So, okay, now jenkins is installing. We can check in service manager.

A

And now let's uh go to localhost and let's see whether jenkins it's running on port 8081.

A

A

Refresh the service manager, okay, now it's running uh it says finally yeah it's uh working on uh working perfectly and uh yeah. That's a little demo about the current release. I mean actually that's in that way. We can use yaml configuration file that I just wanted to uh like that and uh okay, let me stop the jenkins server and.

A

Install that and okay, then, uh let's uh go into schema validation, part okay, so uh I mentioned that uh we can use uh utility tool uh by windows.

A

uh Sorry, we should study a marketplace in order to schema validation uh for yammer configuration file, so you can use this configuration file and you can use the tool called little tool called yammer by red redhead: hey you can you, you can use this uh tool and you can uh provide the schema file uh to um uh to uh this tool and uh you can use that to validate yaml schema files. So then users can easily do this. I mean yeah suggestion. Config suggestions is there and jenkins, and this is not showing an error.

A

If we go over this, it's a name property description schedule which are the compulsory uh configurations, are not there and, let's and let's uh add those as well and.

A

A

And uh executables, let's use java as excusable, and uh there is auto completion. I mean not actual tokens in this uh just generating the configure names uh also there. So now users can easily uh use this tool uh to validate their json file uh externally and now they can easily uh mandate their yaml configuration file using these two and if you talk about new command line interface. Actually uh this is not completed yet, but there are a few things that I can show in this.

A

Let me show uh so if we talk about the the current uh command line interface, so this is the way we provide the configurations. Sorry, details uh that need to uh window size start executable. So if we want to pass, uh let's say configuration file from the command line. Actually it's not. uh This feature is not there anyway. If we wanted to do that, then we have to use some kind of way like this, so we think it's better if we can provide with named arguments, something like that.

A

So that's why we move to a new command line interface, which is uh more user friendly. So if we see the help, these are the commands which are available, and now this command contains uh options which we can use to use easily while we are providing commands into windows stripper. So in this say so you can provide the configuration file. Let me copy the part of let's take the xml file. I copy the file path, and now this way.

A

We can uh pass commands and options into noises for experts, so c is uh represent the config file, so we can provide the config file using uh this tray, so windows live stream is uh starting with xml. So in this way we can use the new command line interface. So, but it's not the merge that you can't use it, uh but uh there is that's overpool request uh for this uh feature, so you can uh using that pr and build phone there. Okay, uh that's all about uh my demonstration.

A

So, as I mentioned before, as uh improvements, uh what we can do is actually we can integrate. We have to integrate json schema validation and there are a few things to update in a new cli and xml schema validation, which I will do after um 2020..

A

uh So those are all about my project presentation. So I need to thanks uh jenkins project or greg means, and my project mentors. They did a really good job in this. Actually I I didn't have the kind of.

C

System tool, development experience.

A

C

I had a little bit of web development experience, but I could finish.

B

This stuff successfully, with the help of my mentors and really thank them, especially like nana, show next turn and michael citoli, uh leaving me with this, and uh I need to thank uh the kings of getting this and uh you did a great job. Good awesome.

B

So that's all out of my project presentation and thank you very much.

B

Thank you would mentors like to add a few comments. Sure yeah thanks a lot for budika um yeah.

B

This project was slightly isolated uh from jenkins at the moment, because windows service wrapper is located in the separate github organization, but it has a serious video for jenkins because jenkins uses windows, service record for all kinds of windows, service management, including windows and stores, including jenkins core any kind of agent installation also plug in which provision, agents and the support for yaml configuration actually simplifies integration with configuration management tools, which is a great addition in the future, and I believe we will soon be able to adopt it uh once the future gets uh a bit more adoption and feedback and yeah.

B

He was working on a new area because has a lot of experience with web development. Development of system tools is slightly different area, but still all the main data.

A

Bills for the project have been landed, um they could got a good good test coverage and uh overall, the quality is quite good. We discovered a few major issues, but now they're fixed, and I believe that soon we will be able to make it available in the jenkins project.

A

So would you did a really good job and hopefully we'll be able to continue the project so that the remaining deliverables could be landed?

A

So thanks budika?

A

Thank you. Thank you, martin. Thank you. So much mike. Would you like to say something.

A

A

Okay, if he's talking, he's muted yeah.

A

Okay, so are there any questions from participants or from panelists.

A

Okay, we can move on then oleg. I guess: okay, yes, thanks buddhica for the presentation, okay! Well, our third presentation today is external fingerprint storage and I invite submit to start presenting.

A

Well thanks, martin.