GitLab GitLab Runner Demos, 5 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Demo: GitLab Runner Group automated release gen 1

Description

Demo of the first iteration of https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29445

A

um Hello people, um I'm gonna, show you the new uh release process uh that I've been working on.

A

You probably haven't seen my comment yet with all the things if you do feel free to click through things and feel free to ask me questions in this. In this video I'm not gonna, go super deep into details. I'm gonna go over the general process, so people can have a mental picture of how things happen and then afterwards we can go into details as time goes by uh just very shortly.

A

The new uh the way the new release process works is.

A

A big pipeline.

A

Which basically creates uh tax and change works and branches in different projects?

A

The idea is that, at this time, all the projects that we have um are gonna be the same way they were before, so there have been no changes to the pipelines to make this. This release process work, so the runner pipeline, for example, is completely unchanged.

A

The only thing this process does is create change, work, create a branch, create the attack and wait for the attack to finish and then allow you to run the rest of the projects.

A

um If we go, for example, into the runner stable diagram job, uh we are going to see some walks and we are going to see. uh We can see all the changes we can see uh the the new tag. We can see some change, work, generation and stuff like that, and apparently it succeeded.

A

And then, after that, we have a stable Branch, that's not dry run, which actually creates a stable branch, and then we have attack and Tuck. That dry run then attack.

A

Then you have a job that waits for the attack pipeline to finish, and then we can do the same thing we already did for chart and for operator. The reason they depend on all the previous Runner jobs is because we need the runner images, so we can run the chart and operator Pipelines.

A

So in essence, let me open a runner and let me go to attack, for example, the last release we did. Basically, everything that was done here is, it sounds simple than it is, but everything we did was generate this change work and add that to a new tag, 15 10, 1 and push the attack from then on. The runner pipeline continues as normal as, if you created the tag manually.

A

um Something cool about these jobs is, they can be around as many times as you want and they will not produce any negative side effects I'm going to demonstrate that right now. Hopefully it was time it didn't work and we actually had a few extra commits, which I am really sorry about, but it is the way it is.

A

I'm talking about this, this shouldn't happen.

A

Anyways so I'm gonna do a quick, uh mock-up release. It's not gonna be a mock-up, it's just that the release was already done, so the jobs have already uh completed the jobs for all the tax for the projects, but we're still gonna create this pipeline as if we're creating a new release um so we're.

A

Let's say we are going to create the 15 10 1. and.

A

A

511 release and the 13-0 release for operator.

A

So um if I go into my releases project, which is this one which I'm quickly gonna, show you what it contains, it's just a few configuration files. That's it uh it consumes the other uh big release, project which is here and you can see it and that's rather huge.

A

But for now let's go to the releases project that contains uh configuration only and now, let's say we have created a 15, 10, stable branch and we have something inside of it. We have a config and we have already done 1510 0 and chart 51 0 and 15 yeah 51-0. There's no operator here since the time operator wasn't supported.

A

um Or other operator wasn't ready for automated releases. It's not that you need to do something special to have this thing supported, but I'm gonna go over that in a little bit. So now, let's add the new. uh The new version.

A

All I need to do is add the config here, add the new versions uh you can see. We have a runner project here or a specified and configured chart project and operator project now, I'm gonna, add Runner, with version of 15 10 1. and chart with version 0 511 and with an app version of 1510 1. Our version is something that's uh used for projects that use a runner, so both Helm and operator actually use Runner.

A

So we need to specify which Runner, they're, gonna use and I'm gonna do the same for operator and that's gonna be 1 13 0 with an up version of 15 10 1..

A

Now this is uh this is all that I need to have uh my new release going I'm gonna commit that and make it out runner 15 10 1 release.

A

And I'm gonna push that it's gonna be a force push just because I already had that, don't worry about it! I! Don't think anybody cares about this project being Force pushed at this point.

A

So if I go here- and this is my Branch with my commit in the release project now I'm gonna go to the pipeline and who can edit? This way is more confusing. So I just use the job dependencies View.

A

Now we can see some dry iron jobs surrounding and we can see that we have the 510 and 51 one tax.

A

So I'm gonna just quickly go over all the steps, so we fetch uh we fetch the the remote repo with the lead, stable Branch. Just so we can be sure that it's the latest with the lead attack. That's in case you have a walko uh project. Already, that's just clean up stuff, uh then we make sure it's all cleaned up, and then we generate the change walk somewhere around here yeah somewhere around here we update you can see, we have it commit that updates GitHub Runner version uh in the helm chart.

A

That's actually something that's configured in the chart and that's configured in changelog Dot release.yamo.

A

This is a config that per project and that's specified for a project. It's not required, but some projects require some Custom Custom changes so, for example, for Helm for every release. We need to update this version and doing that is done by having these actions here, which is just you know, update this file. We can use templating here. This is the release version. This is the release app version.

A

Then we can commit this file and then we can add it to the change, walk, simple, fairly straightforward, and then we can see that this is added here, not this specific release. But if you were on the latest release, it would be added same way as these releases.

A

So now that this is done, I'm gonna have to refresh this and all the dry run jobs passed so that most likely means I can proceed to running the runner stable branch which, if you are unfamiliar with our release process, is gonna, create the 1510, stable Branch or it's gonna, get it uh it's going to update the change work.

A

So if there's no stable branch is going to get created, the change work for the version is going to get generated as this was done before and if it's already existing, we are going to update the change walk.

A

The the thing about this is that the change work generation is rather smart, although I did it so that's questionable, but it's gonna try to do the best thing possible. So let's say that uh we have already created the stable Branch, but haven't done the release yet, which is the case in case. We have a feature for this on the seventh which, as you can see, is probably going to be in a couple of days um and yeah and let's say we have a feature freeze and we create the stable Branch.

A

We create the change, walk and it's all fine, but then we decide to pull something else into the 1510 stable Branch just before we release the attack. If we're on this job again on that stable Branch this this job, it's gonna, regenerate the change, walk and add only the new Mrs. So it's trying to be smart, but that's basically it and I can basically run this job as many times as I want.

A

If I look at it here, We're not gonna have any new commits. That's because we have nothing new in the stable branch if I were to pull. uh For example, cherry pick a merger Quest from Maine and cherry pick it here. It's gonna get added to the changeup.

A

Now that we have the stable, Branch ready.

A

We have a tagged iron which is going to create the attack or the attacks from the stable branch.

A

So for this use case, where we have 15.10.1, we are usually uh gonna create 15.10.1 from that stable branch.

A

Maybe something I missed is that this configuration here is is going to persist for the whole release. So whenever one of these jobs is run, it's run for all of these versions and tags. So that's why it's trying to be smart, so you can.

A

You can run these jobs as many times as you want, and as long as this release is correct as long as you have the correct versions here, even if they have already been released, this is not gonna have any side effects.

A

The point of that is that this pipeline can fail on so many different places and I want us to feel safe to uh maybe make some changes in the releaser or in the release config of this project, or something somewhere or just rerun or recreate the whole pipeline, and so we can do it freely and without any bad consequences.

A

So now, in our mock scenario, we're gonna earn this job, and this job is gonna. uh If we didn't already have the 15 10 one tag, it was going to create the 15 10 1 Duck.

A

You can see it's just succeeding and it's not gonna push anything because everything is up to date. Already uh it can see that uh sorry yeah they're under attack. It can see that the attack already exists in the remote, uh so it's not gonna push it and that's fine, and now we are gonna wait uh for for the attack pipeline, which has already uh has already succeeded a while ago. So this this job is just gonna go green immediately and now from here on.

A

We can uh just do the same thing for chart and operator and we can start stable Branch. We can start attack, we can start the diagram the attack and then we can wait for the Pipelines.

A

So in general, to summarize, if let's say you're a release, major and you are about to make a release in the general case, all you're gonna have to do.

A

Let's say: let's make it 15 11., let's make it 15 11., let's say: you're, making 15 11 you're gonna check out to the 1511 stable Branch you're gonna make this 1511 you're gonna make this 52 and you're gonna just add an operator here, that's 1 14, and this is going to be 11., and this is going to be 11 as well, and you're gonna commit that to the 15 11, stable branch and you're gonna, just click buttons uh on on this pipeline screen uh why the pipelines are oh manual now, because I, don't trust myself and I want us to have a solidified experience with this before all of this can run without any supervision.

A

That's gonna be a great time, but we're not there yet. So we have to click buttons manually.

A

um I think that's it and yeah. It does basically the same for every project. uh It's completely identical for every project. uh That's why adding operator was so simple. It took me uh a short amount of time, uh except for operator, uh so automation itself. Taking more time. The only different thing is.

A

The only different thing per each project is this configuration this configuration, that's it.

A

If you have any questions, let me know if you want a review uh this whole mess. Also, let me know I'm looking for reviewers bye,.