Jenkins Infrastructure Project, 17 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: INFRA Weekly Meeting 2020 03 17

Description

Jenkins Infrastructure Project Meeting - 2020-03-17

Notes - http://bit.ly/2T0oZ9v

A

So hi everybody. Let's start this, this new infrastructure meeting. We have many four topics that we want to discuss. The first one I mean there is no real order, but the first one is at the IBM infrastructure and the current work um as far as I know. Maybe I missed something, but you are still. We still have access to the to temporary machines. Jim. Can you confirm this? Yes,.

B

The s/390 machines is yours, and that's not temporary. You guys have that.

A

B

Is temporary? That is we're talking to legal, and you guys, probably all know how well that.

C

B

Be a little while for like a hardcore machine dedicated you guys, but.

C

B

Guys have he's not going anywhere for the time being so feel free, okay,.

A

But you confirm the 390 it so we can start working it yeah.

B

So that the s/390 is yours, that's not one change, that's that's hosted and it's already taken care. The power will be eventually switch over to a full-time server. Okay,.

A

I created a new epoch for this specific work, which is infra 25:19. So if you are, and you can feel free to follow, that's basically what you are looking right now is some unstable script, or at least a script that we can use to configure that machines. As far as I know, Mark Waid already did some experiment with that specific machine. Is it right, yeah.

C

I'm, actually still using it and I have learned learned several things from be using at experience, so Jim I think you would advise me earlier that we need to use git LFS from from the standard download or from the download site, not from the package provided by the OS and I can confirm absolutely modern. Goodell FS is much better to use in the in the case, tests I'm running then, and the outdated get l FS, that's included with the OS.

C

Also open j9 is crucial on series. 390 I haven't tried, open JDK with hot spot on 390, adopt open JDK, but open j9 performs very reasonably. The bundle JD JDK is completely unacceptable. It's really painful yeah.

B

I think I think that does change once you get past Java, 8, Java eats and above ok. Anything above a should have the JIT built-in so that s 390 performance should be good. So do you switch to eleven? You feel free to use anything you want. It should work. I know for Java 8 it wasn't. It didn't get into the release or or I, don't know what happened, but you have to use adopt version. I took that performance.

A

Mark, if you can, if you could, share a script, so all the inscription, the description student figure that machine would be nice. So you can already.

C

A

Again, you said there isn't it.

C

A

So there is an epic, so if you put the link in the google doc.

B

A

And so usually you should you should find the right to get. You know. That's a big song, interesting too infra.

C

To five one nine, yes, oh, it would help if I typed correctly. Sorry.

A

So once once once the the agent is configured, we can add it to Sierra Jenkins, oh I, put all the instruction in one of the tickets should be pretty easy to find, but yeah, nothing more to add here.

A

Next epic is about moving Sierra Jenkins that are on anybody where s right now. The main work that has been done was to create specific mi that we can use from Sierra Jenkins at I/o. So the main, the main focus right now is to is to move all the agents or machined from Asher on Amazon I configure Seattle Jenkins Aradia with the right credentials, Tim jaw and Alex old maid backer images for Amazon, so it seems to be working now, I just configured into one, and it was for a few minutes before the meeting.

A

So I should be able to put more machines there and which used the number of our machine that we can provision I'm, not learning to remove that I'm still planning to keep it as hold by key. But I would like to use to keep that number of machine as low as possible. So once once Ubuntu machine is working correctly, I will put the Windows machine and I will probably also add the arm 64. So we can.

A

We can use specific instances running on the arm on Amazon, so could be useful if you want to do some Testaments architect.

C

After teacher yes mark and and the master will likely then continue on Azure for the foreseeable future, not that I care, because you keep me hidden away with DNS references but I assume the master stays on Azure for the for a while.

A

Yes, there is no plan right and are no plan right now to move the master it's working and maybe once we have so. The main reason for that is because moving the agents on Amazon is quite easy because we just have to specify the security group and we have to use public IPS anyway.

A

So the configuration is quite simple to put in place, but if we want to deploy more resources like to be discussed or whatever, we need to put in place some terraform for that there are form code for that, and it will require more work. So right now the focus is just to reduce. I should be able to use Amazon computer, but the next step is definitely would be a very happy if we could deploy a second to this cluster on Amazon and start using it Thanks any questions regarding the OAS nutrition.

B

Yeah one question is so we'll have arms 64 from AWS. There is no current offering or cloud offering for arm 32 right, I guess.

A

B

Okay, yeah I do a lot of people and I was you know we were talking in the IRC. A lot of people use arm 30. Have you guys seen any use of that? No.

A

Maybe heard I get some some ideas, but I I read out that we can find some stats about that. um Is it right?

A

There is no way that we can have that information.

C

We're not hearing you but you're, not muted. Oddly enough.

D

C

D

So we could get some information from studs and if somebody spends time might be the very people to have this information public at least.

D

But right now you can get anything from the website. I mean starts JSA, yeah.

C

Okay, so you think that there is data being gathered that might be available in the raw data store or like that. We could tell us if you're running an arm, 32 mm.

D

I believe so, because platform is part of the metadata being submitted. Also plugins, like support core plug-in, also correct to this information.

B

You guys also pull anything from docker hub I, don't know what kind of stats they give you in terms of it's just overall polls, or they tell you about our connection. We.

D

Don't have official images for arm right now.

C

D

This physics wouldn't be that high I believe for we have never promoted experimental arm images.

A

But it would be interesting to anywhere puts an emergency if people are interested about that and what would be the best way to say. But.

B

So so going forwards with like arm 32. If we wanted to produce image, we I know there was talks, I think, oh, like you, it was Oleg and Mark. You guys were talking about the windows guests running some of the hardware from houses or apartments. It was on the mailing list from a long time ago.

B

Is that something we would do for like arm 32 like we wanted to build on architecture we get like community support in terms of hosting, or you know, infrastructure for arm 32.

C

Yeah I think in terms of market interest. The the market interest is much stronger in the arm, 64 stuff that yes.

B

C

Providing I'm not worried about hosting I'm about arm 32, if, if we, if there's a press for and if there's a community drive for it at that time, we then see. For me it was a fun conversation, but I, don't think we'd be safe in relying on hardware contributed from basements that typically, we've tried to keep CIA Jenkins that I, oh, very safe in the end of what it uses as resources. Now,.

A

That make sense yeah, and one of the other reason why I'm, not comfortable to put machines that are hosted in in one person basement is because that question is down. There is no way to replace it or whatever, so prefer a more stable situation, because I don't know no.

B

You totally right and I think I think arm 64 like Jersey with all the new ratchet pies I, think three nup slide. Sorry Alex and I were talking about or 64-bit or arm 64, so I don't really know where other people are pulling like I guess arm. You know hardware from unless you're using some sort of cloud service.

A

Yep so I think every single said that you procedure I will I will just try to use in my arm City porn and we just put a new levels on Cielo Jenkins at I/o. So we can, when we can do some tests from there.

A

The next step each but I, want to discuss this button. What are nice? You know how's the Jenkins that I own so mesh in this, but I deploy I deployed one service at least beats that we can use and I'm currently work yeah, so but the biggest, the biggest limitation that I have right now is hobbies, need to contain all the files that we need that we can provides because it creates unnamed dislike and then I give md5sum for each file.

A

And then, if the remote nurse, as as the same pile with the same ash, then it gets distributes. I need a file. So me hope in still that, give you the file just for what to the closest doors, and so while it works for more most most of the mirrors, it does not work actor actually because yeah, every viewers only keep the fives for the last year. So that's and so, for example, all the files that we can download from archives action to the dial are not available.

A

So the plan is to deploy archives, which include Rio, entre beauties that contains every file. So if a forum or if a random barrage does not contain the file that we need, that specific archives always have that's the file that we want and that's that's the biggest limitation that I have right now. The second thing that I discover while doing some tests. Let me use a lot of actually access in our new house, so I cannot use the engine each container, let's say tropical whatever so I need to rely on Apache, um but yeah.

A

If you want, if you want to blow the work that is going there, um just go and drink it into a snapshots, I will just to building. Let's, please be PR any people being right now. So, once once this PR is merged, we should be able to to promote its and to do more more test.

A

Any questions regarding at the new off stuff.

A

One time two time, three time.

A

Just continue so that the fourth of week is the last to Peter. I want to also discusses. We got in the automated Jenkins release, so I made some work to refactor CI.

A

The GG are relieved that she I totally that IO, which is the Jenkins instance, used to build the trigger release and to trigger the packaging process, and there was one drug running on that specific Jenkins instance, which was used to configure the communities cluster, so I created separated Jenkins instance called call in Fredette CI drinking today, on that instance, is allowed to manage all communities in our resources.

A

So now that the job is separated, I can be more restrictive on releases. Yet thinking that I also only allow people who should be able to trigger release and to have access to that instance, at least a trigger job. But everybody else should be able to just read um the outputs. So next step is now to to push artifacts and to be sure that when we publish artifacts, we published artifacts on the experimental release line.

A

So we do not affect the current release process so once this is officially once I've added, this I will modify the publishing script to push. Artifacts took me hostage, egg Israel so to the I mean that the the the service that we used to distribute packages, so that is once once we publish to the to the experimental release line, and we really did that is officially working correctly.

A

We can start to use to to trigger releases based on the Jenkins, get repository and try to be as close as possible to the to the real process, because right now, I'm still I'm still releasing version on my own Fork and to be sure that I do not affect the current release process. Any question.

A

Nope then, let's continue.

A

And the last topic that I want to show you, which is a small team or but a tool that I wrote recently I will just share my screen. It's going to be zero.

A

Sorry about that.

A

A

Can you see my screen.

C

A

Yes, yes, you can see, connectivity is okay,.

A

So basically, it was a small project to automate the deployment of our application. The idea is just to have a small CLI that can where I can specify some rules and basis based on those rule. I can update a specific EML configuration, so I will just show you the current outputs, or is this so? The idea is assessment to the right code of the CLI, so we can either specify directory or a configuration file. If we specify the directory, it will just look at every file inside and apply the different groups.

A

So in this case, I have just a three example and the first one is to date, every Jenkins instance that we have so we just specify a source. So in this case, I want to retrieve the latest version from a mavin repository, which is the version to the 226.

A

We specify a condition, and so I want to be sure that the docker image with that specific type exists in these kills the reserved version that we can use and then it will update every target. So a target is just a file located on a git repository and I have to specify a specific key value. So in this case it will check that the Jenkins master mesh tag as the right volume.

A

So the read the latest in the latest run the version was already updated, but otherwise you just changed the value and just push the new version on directly. So that's why you saw some pr's on Jenkins if for a slash charts to automate the deployments so currently the sources that I can use this to either fetch a version from a mapping repository I can fetch version from I get up really so, for example, flip click inside it's also to mentally updated.

A

As long as you have a docker image that contained the write that had to write some tag, so the configuration so digitally object. Put this quite simple: the configuration is.

A

Charts so I just created the directory, how did CLI, daddy and so, for example, for a trillion sides I specify the source is coming from the github release, so I want the latest the latest version from the Jenkins and for a slash break insight.

A

Api I want to check that the docker image exists with the value generated from the from the tea-table release, and then I would just specify every file that I want to update to the files I'm the key value, a message which is used in the it commits I specify if I want to do a git repository located on get up, for example, but I can also just specified it's um as like each repository. If it's a github right now, it also opened a PR.

A

Otherwise you just push directly to a specific branch, and and that's it so right now it's work as I said it's working for Jenkins, plain insides, but it would be nice to refactor at least two updates. The docker image tag that we have in the Jenkins you for organization to use github releases if it's possible and and otherwise I'm, really looking for feedbacks on this module. If you have some.

A

A

So that was the last thing that I wanted to share. um So if you don't have any thing that you want to I.

D

Have a minor question: yes right now: I'm working on the roadmap proposal for Jenkins project, okay, and maybe the next meeting I, would like to discuss.

D

For the James introspective I can pull some from the December contributor summit materials, okay, but yeah we could discuss a bit more and also infrastructure. Related topic is about a road map engine, because one question in the proposal is whether road map JSON files should be provided from Jenkins ion as the source or whether they should be hosted on a separate repository and the sample righted by CDN like J's deliver. So if somebody from the interest actually is interested to discuss this topic again, I would suggest to cover it in the next meeting. Okay,.

C

So I, like the trade-off there, between hosting of the inside Jenkins, that I are hosting in a separate repositories that a bandwidth consideration and access control consideration so how it happens right now,.

D

Basically produced engine from the ocean so every time a user maybe needs to a page JavaScript on the client side, downloads a JSON file and renders it and this place said there is no immediate concern about bandwidth or whatever, because we didn't expect millions of users to go to this page. It's mostly about maintainability, because he road map was a separate repository.

D

It might be better to maintain easy to maintain from the administration side. Approval to request reviews is ability, because, with a separate repository, auto JSON file hidden somewhere in check the report, but yeah that's a minor thing.

A

Do you think that you will update.

D

That's one not I, think it will be. A moving target is several updates every week at least it my it's my wishful thinking, because we really want to have a dynamic, because.

A

The main my main concern, if you I mean the main advantage that I see to keep it under a jacket that I you it's in one player and you have everything is in one place, are just easier to to look at the Jensen. Fine, too lazy to whatever.

A

So personally, I would be more in favor to put it in Jenkins Jenkins value, repository.

D

So that's my current documentation I consider moving it outside. If somebody votes for that because from visibility standpoint, it would be preferable. Let's say that's how Jenks enhancement, proposals incremental bang.

D

On the secretary poll, we have some issues with the process, but in principle I think is better than their heaven to them, and also magenta say: oh, but if either trade dogs so right now, I am just doing a simple implementation cook it based on glowing roadmap Mallis, but for the teacher I mean I might be proposals to move with text or memory foam and then CDN and all the topics. My arrives: okay, okay,.

A

D

But yeah it's not something. We need additional effort from the infrastructure team.

A

Because we either either you provide either you want to have a specific application for it, and then you provide, let's say the car image that we can deploy in a community cluster and and that's all, we really just put it on Jenkins, Rio and yeah. So.

D

We already use a CDN delivered on the plug-in side, and basically we don't we don't you see an impedance I mean you use it so plug inside surfs images or might be I'm, not sure, maybe giving mobile network to that the latest fashion. But when it starts in September all the images reserved from CDN it was which CDN AJ is deliver because we were, we were serving Public Communication, so using CDN, which was providing out of the book support funky table to visit all this, the entire JavaScript ecosystem or works like that.

D

So that's why we didn't eat anything from the infrastructure team because we just used the features provided by existing CBN Vitra. Okay,.

A

So that could also be a reduction yeah.

D

I I do not expect somebody to solve a mission with Asia CDM CDN if you're concerned about that. No no.

A

We don't we don't. We don't have a Joseon exactly and yeah.

D

That's my even touch him that I.

A

Sent an email yesterday so the last it was four weeks ago when they said I should receive the contract and I tried so right now it's nowhere. I try to lose recurring cities, but yeah sent an email yesterday already before um yeah I think it was yesterday to see what was the current state and.

A

The reason why I don't want to use as your CDN or incident whatever is because if we use it for one side, we want to use it for every web site and depending on the website, it can be quite expensive. So that's why sorry irresponsible is better yeah, so did something that would like to have yeah but actually trying to find solution with fastly.

A

Yep thanks a lake for your intervention, any last pitch yeah.

B

One question I sorry, I didn't attend. Last week's meeting I did see the topic saying IBM sponsoring were you guys looking for? Was that just concerning this 390 and power resources or no.

A

No, it was, it was the BBC 64 ins and as for hundreds, okay, so, which is I mean, which is for me, sponsoring somehow.

C

Clearly, sponsoring you're putting putting money and do into the project that we use for compute resources, yeah.

B

Yeah I I was trying to talk internally but try and see if you guys could use IBM cloud for some x86 resources, but that conversation has not gone anywhere. So that's why I was wondering. There's like oh, you guys wanted x86 resources now.

A

Right now, right now, the main the right now is those architecture. That's the current state. Yeah thanks, have a good day, have a good weekend evening for some and have a good day for the others. Bye-Bye right.