Jenkins Evergreen, 7 Aug 2018

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Essentials: Open Planning (2018-08-07)

Description

This is a weekly meeting to discuss the progress and plan for Jenkins Essentials, an automatically updated Jenkins distribution.

Find us on GitHub: https://github.com/jenkins-infra/evergreen

Join our Gitter chat: https://gitter.im/jenkins-infra/evergreen

Jira board: https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=406

A

To the open, Jenkins essentials planning meeting, if there's a Tuesday, the 7th I think we're still waiting for Mandy to show up, but we can get started. Anyways, there's really only a couple things that I wanted to talk about in this meeting. It should be fairly straightforward and I think we'll wrap up early and the the first one that I wanted to check with you specifically Batista's around was some of the AWS work. I mean it was I'm, actually not sharing my screen, so you can't see what am I trying to I was.

B

Looking for the rest of your question, then yeah.

A

So there's a lot of these. Am I and AWS related tasks that I think are important before we, you know give it an alpha user. A watch stack button that they can work with and I understand. You've got some vacation coming up so I'm curious. Is this something that you think you'll wrap up before vacation or I should try to hang on order.

B

Figure out some of that this is this is a good question. So well depends so the finicky, if you're, referring to using a marketplace and so on and there's no idea where to start from no real idea stops from I mean I would take much more time that is left but I already studied yesterday for the removal of ami one, the one you see, that should actually be I.

B

Think you scroll down a bit, but it's in progress, because yesterday I was having a look at how to tackle that and so to give context to two people, possibly listening to what I'm what I'm talking about. We have an issue because we didn't want to really use an ami. We would actually bake ourselves. We want to try and find some official public one so that we don't have to maintain you bleach when there are security issues and so on.

B

So that's what context so I looked more deeply in the iaws marketplace and usually made use it and so on yesterday and there's from what I can tell no image that provides, for instance, docker out of the box, I mean already installed and available. You know be able to run docker run when you're starting up, but so that leads to my kind of question. I guess the only way we have to make this work for, given our requirements it actually to enrich to use even more user data to do what I'm was previously doing.

B

You know baked in the image itself to actually you know: yum update and iam, install docker and configure the I think resource or user or whatever is needed to actually start docker, and actually it's kind of a bit more waste for a chess. It would be similar, but there we would install Java and I think I'm, forgetting something we were again.

A

We were saying that we would like dr. an Java I'm, assuming that you talked about enriching this by providing an init script and a cloud formation template.

B

Absolutely there's already one for anyway, we already have a user, because we do you know doc around to actually. You know fool and run the essentials either various cloud flavor image, so that's already existing anyway on the master side. So, but it would kind of you know at some time and I would hope that on the ws generally, it's very fast, but you know we would have to add, maybe like 30 seconds or one minute or maybe worse, so people would be waiting for it and it could be slightly more flaky or something.

B

You know that that's what I wanted. That's, why I wanted to kind of avoid you know kind of blockers or by experience possible experience for users, but I, guess it's even more important that apply. We'd only indeed have to maintain an ami yourself so yeah that that's that's the gist of what I wanted to talk about. Yeah.

A

I think that's a reasonable approach, the the user data approach. It does give us some amount of flexibility to change this as we move along I'm. Very surprised that dr.

B

Fink doesn't.

A

Maintain a Tucker enabled double yeah. Well, I didn't.

B

Find I found, for instance, on a value aside. Some ets enabled you know images, but there those seems those. Those images seem to have some kind of VCS client or something which we do really wrote and le vent some kind of run time. So we don't really really want to. You know have that running and even probably not not even present, for obvious reasons and so on doc, aside, I didn't really find something, maybe I just missed it, but now maybe it's kind of consistent also with their move. The word soccer, Enterprise and so on.

B

So maybe this is only available somewhere somewhere else or I. Don't know. Maybe I should just ask ask someone else to have a look. Let us that's possible, I missed it. That.

A

Wasn't area I swear? There's a French contributor to the Jenkins project is very very in love with docker. Can't you ask him.

B

Yeah I can I can do that. I can do that. Indeed, belief is that we do have many people, yeah I, can't even reach up to some people who were previously a colleague and and they're so very deep into the Bucer, and you know that guy, but anyway, I'm talking about Dania. If we see if we stop trying to be hinting anyway,.

A

Yeah yeah, it's just something like I've, been I've, been testing out the darker cloud image and it works beautifully. I'm really excited about it. I just I can't wait to have a lunch stack button that we can put in a blog post or on the github readme. So we can start getting people testing this way, I'm really excited to get this yeah get the ami stuff done. Yeah.

B

By the way I think with in it might be getting enough to pick slightly too much. Well, maybe not I think at some point we might be needing some. You know Jenkins owns, but maybe this is this is going to be.

B

There is going to be you know, sponsorship and so on issues with this, but having some kind of edible, US official Jenkins account to host and to store I, think see, I think things like CloudFormation template, maybe or maybe not or maybe in the provider of data that marketplace owner or something I I didn't dig into that part parity, but I guess you would be needing. Some kind of you know account that would be providing some some thing in the marketplace. So.

A

Can you create a ticket and just describe the requirements of what we need on an AWS side of things, yeah I.

B

A

B

Really do that because I didn't dig into these how to create you know something I they made available in the marketplace, but I guess I would kind of sense. So.

A

I, don't think we need to make something available in the AWS marketplace. This is different from just providing a CloudFormation template with the launched stack button from my understanding, so.

C

This is my understanding, right now and and.

A

This is based off of possible your.

B

Specific information.

A

Is that yeah, basically, because.

B

A

B

Ever used I only ever used the WSC and I locally, triggering the you know, reading the CloudFormation chasin fire locally, so I don't know. If this is there there's something standard to be able to do that. You know in the cloud fully and maybe but yeah you already. You probably know already about that or my.

A

Understanding was that you, a person, is able to create a large stack button. That reference is a publicly available CloudFormation template, which is something that could be embedded in a github repository on a third party website right and that will watch. Maybe I just had.

B

To look I really I really thought it was something related to marketplace, but I, maybe I just you know, was rifting into something that was seemingly obvious, but is not so yeah and anyways thought getting back to the kind of issues that are still existing I. Think if we agreeing on the fact that I'm you're going to use user data and then I kind of the issue as a result, because it can be really hard because I know what can be done, the only issue with that as well.

B

For instance, it's not really shown master, because it's a one-shot thing. You know when you're presenting your master is.

C

B

Slightly more pity for agents, because this is going to be losing time, you know each time an agent is going to be provisioned and I hope it's not going to create flakiness on the agent side. You know provisioning agents each time and it's during each time. Tracker Java, I know before it's. Even so, you know because.

A

They can connect.

B

It to the master right.

A

Now the the installing of dr and java yeah yet actually happens on every single Azure p.m. it's provisioned for CI a-- jenkins. I it's very inefficient I acknowledge that it.

B

Is it is fairly.

A

Reliable I, don't we've had agents fail to provision, but they've failed to provision typically because have failed to create the VM, not because they hope to install the dependencies.

B

Right yeah, it makes sense, I mean and I'm not really worried about the fact that you know the us and people are progeny VMs on time and calling a pity gate or or um or whatever and there's local caches and so on. So it should work. You know ninety nine, five, nine percent and all the times. But yes, anyway, it's it's it's more like adding 30 seconds, or maybe one or two minutes to just have the agents coming online is going to be mostly on issues for demos.

B

We will hug that around for forgive.

A

Those so since this is.

B

Here I want to make.

A

Use of his time, because there's this conversation that we started to have on this pull request that I created is one of the topics I wanted to discuss in this.

A

They pull up the things specifically. Maybe while we're chatting about this Mandy will I get back from coffee or wherever she's run off to. She need to try and.

B

A

I'm really excited about that other pull request by the way batiste hiding the Freestyle and Maeva drums. That's cool.

B

That was not even created before and on the milestone. One, that's why I you know I can tell sometimes, when I am kidding about the difference between the things I'm doing on work, time and personal time when I get to choose which do you rather work on another feeling.

A

So this is yeah.

C

By the way, with as far as hiding me, even jobs, I mean what we actually want to do is not have to make them plug-in installed. Then that basically means making sure that the plugins that we do have installs are built against a suitably new version of parent. You know of Jenkins core and all of that that's entirely doable um and as far as not having freestyle shop types, we've there's a long-standing issue with the split plugins from core label and JIRA about perhaps moving at least the freestyle project.

C

Freestyle fill associate descriptors and some of that stuff into a plug-in.

C

Moving moving other stuff, like project or even abstract project into the plug-in, would be a much more serious task because there's so the things were that have been proper dependencies on it. For.

D

C

Historical teasingly strings. It would take us a long time to disentangle so.

A

C

A

Great sorry, the test meant that he's picked up on one of the things that I wanted to try out with jenga's essentials was I wanted to hide that stuff. From the new item view and I wanted to discourage the creation of new freestyle and main encounters. I agree, like generally I, would like to see them disappear from Jenkins at some point, or at least become fully optional in Jenkins, but we're definitely not there yet unfortunate.

B

Yeah because I mean this is not in opposition to what jesse was saying, I think because they were cutting off a bit at the beginning, but what I did is just kind of can be maybe seen just as a kind of a sheen in the intermediate time where you know we, we actually extract things and we remove that steam and we, actually, you know, don't know, do not even stole DT, freestyle, plug-in or whatever that will be named. Yeah.

C

Yeah I mean we should be sure you know it's a short-term solution. We can do something. Do you know about the mass extensions plug-in or hide extensions plug-in or something like that? Extension filter? Plugin extension filter, plugin yeah, so we can use things like that as a switch I'm.

B

Not sure I'm not sure we are started talking about the same thing here, because actually the task is already done, and the Freestyle and maven jobs already filtered out yeah.

C

I just meant it as a more general way doing what you did. You can use the extent for plug-in and then there's this matter of turning off different kinds of UI elements.

E

A

Is this is getting into a tangent Matt? That's detracting from the really big topic that we need to discuss. Yeah.

E

A

So the the dependency resolution work that I had done last week and I'm just gonna set up some context for the video or for anybody that that's watching. um One of the things that we need to do in Jenkins essentials is take the essentials gamal, which is a description of the plugins and versions thereof. That should be distributed with the with the jenkins essentials and generate a update level or an update, manifest.

A

In back-end, so basically we have you know some files. Let me just go to the essentials GMO in the in my pull request here, so we have some files that describe certain versions that a developer has said. You know this is ready to go into Jenkins essentials, say: there's essentials, plug-in with an incrementalist build at RC 20 and then through some process. We need to determine here's the full manifest of all the plugins and their dependencies. That will create a legitimate.

A

Jenkins distribution, and so that's what this plugin this plugin is one starting off is playing. This pull request. Number of 169 is really getting into and I wanted to start discussing.

A

Some of the behavior which was implemented here and the gist of the behavior is for plugins, which we do not have a explicit declaration in essential, as the yellow of what version were relying on the first version of this code is consulting the update center. When we resolve a dependency to say: okay, workflow a greedy example, we say we need 2.0 a workflow aggregator, but that's all that exists in essentials. Demo with 2.0 we're going to the update, Center and saying give me their.

A

You know: workflow a grader says: I depend on with float job, and so we go to the update, Center and we say, give me workflow job and that's the version that we're pulling there so Jesse I, there's a few turfs comments in here and I'd appreciate it. If you could expand on your thinking here, yeah.

C

First comments from yesterday or what a is able to type in a phone keyboard.

C

Attempted telepathy didn't work, yeah well, I mean I, think there are three possible policies, two of which are acceptable.

C

Yeah I mean what I would really recommend. Is that essentials diamo just test a list every plugin, because I think in practice, with the possible exception of blue ocean which, which is done via monolithic mega release, read.

D

C

Else runs on an independent release cycle and you, and it would be perfectly normal to just update one component when that avail. You know when that update is there and you want it to go, in which case you would just have the script.

C

It would just be generating exactly what you list essentials, diamo and it would still be parsing, manifests to make sure you didn't make a mistake and it would fail if you attempted to process a yamo file that had a consistent set of plugins, so either a plugin is missing a dependency or its dependency is too old, or it requires a new version, newer version of jenkins and what you're specifying like? Basically, all this stuff that jenkins cord that startup, you would be pre checking if that seems like the most straightforward option to me.

C

If you really want to keep it artificially short, then you can have it load in transitive dependencies, but the only way to make that the deterministic is to pick particular versions of each of this transitive dependencies as a function of diversions that are specified in the ML file and those you know in The. Associated actual manifests and in practice what that means is that you take the transitive closure of all of the plugins and for each transitive for each plugin. That's not list in the ammo file you collect.

C

All of the versions are requested from dependencies and you pick the highest such version.

C

If you do that, remember, there's no need to ever load the updates enter JSON because it would just it wouldn't matter you, don't you only care about the ml file and the manifests when you pick up versions according to whatever is published on the update center today, then you basically break abilities. You know for sure what essentials is publishing today.

C

You know it just causes. It basically destroys the the idea that essentials yamo is manifest of what goes into the product and, as you saw with workflow aggregator I mean it's almost meaningless, because you know it's been what like two years or something since we've got to really think that there's in the meantime, you know basically you're getting a random version of YouTube the pipeline plugins.

C

Whatever happened to be published to the public update center and sooner or later, you would want to publish a particular version say an incremental version, so you would wind up filling in those transitive versions in essentials, DML sooner or later anyway. So why not just start off by having them all written down.

A

My concern my primary concern with having them all written down. You know option one, as you described in the essentials demo is I. Don't think that that's gonna be practically manageable in in big.

A

In the case of all the pipeline developers actually like preparing a release or having an incremental release and going to the Evergreen repository and updating essentials ml, then I think it would work, but if it's Batista and I keeping track of the versions that are available and with keeping an eye on things and trying to rev them or do an automatic change of some form, then I, don't I, don't think manually curating. That list is gonna, be I.

A

Just don't see that happening like I imagined that we would just have stale the Jenkins essentials instances out there as a result of us not being able to keep up with releases that are hitting the update center.

C

Well, I have two points that one is that in certain cases, not the normal case. In certain cases, you do actually want to have developers explicitly push a particular version out there or, more specifically, a particular combination of virgin versions, because we deployed some sort of complex, multi, plugin change and you want to push it out there and start getting feedback from telemetry and so on, and you want everybody to have a consistent snapshot of the whole system. So you want to you know, file a pull request.

C

Did you that, as far as the more common gate that you know, developers are not really engaged with essentials in particular and they're, just dumping stuff out there and occasionally running maven release perform status quo yeah? That is quo situation and then I think you should be doing what Jenkins X does and have an automated tool. That creates an a new, explicit revision of essentials that you mow according to what stuff has come in and I. Think you should do that, because it makes it explicit that your new version of the world to users.

C

It means that you can, if you want to roll back or something it's a matter of get revert. You can also do things like in the in the future.

C

You can, you can imagine doing something like having a set of automated testing acceptance test harness smoked, ass I can toss some other stuff, which would run as a multi branch project against evergreen and it would reproducibly consume a particular version of essentials die camel so that the automated process, rather than just dumping stuff in master, would be creating a proposed update to a set of plugins that could be. You know it doesn't updates at once and before any.

C

If that goes live you have a complete test run that might take a few hours, but it goes through and verifies that things are not completely broken before it actually emerges. It.

A

So the the Jenkins X approach that you described, I'm I'm, struggling to see the difference between consulting like the ingest JSON, is what is going to be processed by the Evergreen back-end anyways I mean, if there's an automated process. That's running this script that I've written here that generates the ingest, a sign rate, no.

C

No, that the patches essentials document, what's the difference, because essentials, dot llamo, is what is versions, and so, as part of that change, you have validation that a particular proposed change to this. Okay.

D

C

Dot yeah mol is, in fact something that meets all automated quality criteria whatever they are, I mean everything from the obvious stuff that you know plug-in dependencies are satisfied to integration tests pass.

A

So in the case of workflow, aggregator I think that's a good one to pick on, because it's.

C

Just something that shouldn't exist- and you have no actual reason to include in essentials at all well we're close yeah, and so it does nothing.

A

C

Very helpful you only purpose or that plugin is like basically in the Jenkins 1x days, so that people could have a single check box that they could click in the available map. So it's basically totally irrelevant for essentials, where you're, assuming that this gets pre-installed anyway,.

A

So I'm thinking that and and patís feel free to jump in if you've got in this as well I'm thinking, some combination of both of these actually makes a lot of sense like for for the the pipeline suite of plugins listing all of those in essentials, diamo I think is really really valuable, but I don't want to make like is I. Don't want to manually, maintain an entire dependency graph with all the dependent versions in in our source tree right now, but this process makes sense to me, but T's had to go.

A

He did mention that he had to leave it as every time this process makes sense. To me, like all of these together makes sense to me: I, don't something mutually exclusive. I, don't think is, is gonna, be a good solution, but combining these makes a lot of sense to me. Well,.

C

Yeah I mean you have various options for tuning it I'm just trying to say how I think things ought to work mmm um and and I don't think yeah I mean I'm, not proposing manual curation by you and Batista's as a proper solution. There are various ways to automatically proposed updates to essentials, diamo.

C

Well, you know incrementals versions and update center versions. I mean you can you can decide whether just you can decide at some point whether you want to do automatic update of the latest incrementals and master? That's that's an easy tool to write they're, just an extension of an existing tool or if you only want to do automated updates to official releases and leave it up to developers to explicitly push incremental patches or something like that.

C

But my point is just that. However, you choose to prepare those updates that the actual fact of the update is recorded as an explicit git commit then then track yeah I agree that makes.

E

A

And this is a comments and I think I think I have an idea of of where I need to go with this script, and it's not just one script. I. Think there's there's some other tooling that I'm going to need to write yeah.

C

It's a script instead of workflows and policies, and so yeah.

A

Okay, well, that's been very. This is very helpful to have that clarification Jesse. Thank you. It.

D

Can express that in writing.

A

Okay, I definitely I definitely understand the terseness of funky words, so it looks like we won't have a mandate today and batiste had to leave early and leave all so yeah. Thank you for thank you for jumping on and participating and I'll I'll see you and everybody else later.