GitLab Solutions Architecture, 25 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Getting to Merge Trains - Ways in which GitLab CI Evaluates Your Pipeline Jobs for Execution

Description

Dispelling the mystery behind Pipelines for Merge Requests, Merged Result Pipelines & Merge Trains.

A

Hi, my name is tim poffenberger and I am a solutions architect at get lab, and I would like to take some time to talk through different ways that get labs. Ci evaluates your gitlab yaml and the options it provides and the intent behind this is to help dispel some of the mystery behind pipelines for merge, requests, merged, result, pipelines and merge trains. They all use a lot of the same words, and sometimes it can be a little bit confusing the heart of how I'm going to approach. This is we're going to run through this question.

A

How does git lab evaluate changes uh to your code repository to determine which jobs to run for a given pipeline?

A

Historically gitlab has utilized these only and accept parameters within jobs to help evaluate only um what what jobs to run for a given pipeline. How this has been evaluated in the past, though, is just the very last commit for a given merge request. So if you have a merge request and you have 20 changes, the very last commit is what's going to be looked at to determine which only and accept parameters are being met or not met from there. We introduced this merge request pipelines.

A

These are enabled when you use the workflow and rules syntax and they evaluate commits from all of a given merge. Request's commits. So if you have 20 commits it will look at all of those 20 commits to determine which rules and which jobs are applied applicable for a given pipeline.

A

This was a necessary step for us to take in order for us to achieve pipelines for merged results, so pipelines for merged results, build upon merge, request, pipelines and are enabled via settings change. This is a premium and ultimate only feature and only available to a merge request when it is not in draft mode and what it will do is it will evaluate commits from all of a given merge request commits, along with the merged changes from the upstream or target branch.

A

So if you have changes that are going on in your default branch, the merge request will evaluate both those changes, as well as the changes being applied to your merge request and then. Lastly, all of these are predicated on the need for merge trains so being able to cue multiple merge, requests and allow those merge requests to be executed in such a way. That is trustworthy so that you can have multiple merge, requests, ready to be merged back into that default branch um and all um receiving the previous merge request changes.

A

So, let's walk through an example: here's a source code repository it stores both application code as well as infrastructure. As code code, it's got a gitlab ymo file and a readme file. This merge request is also one commit behind the target branch. It's in draft mode and historically, this is how you default. Configuration files have been set up, so you have this. So what I'm going to be doing in this uh is I'm evaluating this tf test so anytime, I'm making changes into this terraform directory so or my gitlab yml file.

A

I want this test script to run anytime. I'm making changes into my app directory. I want this app test to run and then this always job is always going to run, regardless of what changes are going on, and so let me go ahead and make a change real, quick.

A

And I'm going to apply this and when I pull up my pipeline tab over here, what's going to happen, is it's gonna kick off a pipeline and this pipeline is then going to show me um exactly what I feared so again within this within this merge request. All of the changes all these directories have have been applied so hope. My hope was that the tf test job would run the app test job we run and this always job would run.

A

But unfortunately, what ended up happening is this always dropper on end because it only evaluated my very last commit here.

A

So let's go ahead and I'm going to comment this out and I'm going to scroll all the way to the bottom, because I've applied this same logic except via um the merged result, pipelines or I'm sorry pipelines for merge requests and I'm going to commit this change real, quick and then I'm going to walk you through it and then I'm going to apply a second change so within here we have this workflow. uh The workflow is really handy, because what it's going to do is it's going to prevent two pipelines from running.

A

If we didn't have this workflow set, what's going to happen, is this type of pipeline would run and a rules type pipeline would run.

A

And uh part of the reason that is being prevented is because we're actually looking for this merge request iid um within uh within here we're simply changing only to um rules and then we're making this a list instead of a key value pair and same exact files, and then this rules, when always for this always job.

A

Let me go ahead and add a second change and commit that change. And what we'll see is this pipeline come up again so right now it's at and now we have 342. So, let's pop this open and now we'll actually see, I didn't make any changes but beside this, um this minor change to this readme file, but um it noticed that the whole merge request had changes in the app directory and that terraform directory.

A

So it's going to go ahead and evaluate those jobs as well, which is exactly what I was hoping for: let's go ahead and head over to settings so within the project, settings general there's a merge request section and we're going to enable results for pipeline, merge pipelines and I'm going to save these changes.

A

And now I'm going to head back over here and I'm going to click on pipelines and I'm going to run another pipeline and so right now we can see that these are running as detached.

A

What this is saying is this is detached from um the the default branch where this is being merged into, and it's um it's just letting. You know that that is, you know happening. If I run this pipeline again, you can actually see that it's doing the exact same thing, even though I've enabled that setting, which is a little bit confusing because I have this- enabled merged result pipeline.

A

So I was hoping that it was going to show a new pipeline that was no longer detached, but it failed to do so and the reason it did that is because we're still in draft mode. We don't want upstream changes to to that default branch to that target branch to impact or break any of my ci cd pipelines, while I'm still in draft mode.

A

So I'm going to go ahead and mark this as ready and now that it's ready I'm going to run this pipeline again and let's see what happens now, you can actually see that this commit shaw has changed because now it's merging all those changes together which is really convenient. And now, if I go ahead and open up this pipeline, it's always job, and uh let me show you something real quick uh so right here, you can actually see that um I don't have.

A

um I have an app directory, a tf directory, a get lab, yml file and a readme file, but in my master branch I have this fancy file that I added. So when I look at this pipeline.

A

You'll, actually, what we're going to do is we're going to list out all of the changes within this always job ls, hyphen l.

A

So when this job shows up, you can actually see this fancy file was added uh because it merged all of that those changes which is really convenient.

A

um Lastly, what you can do within your settings is you can you can enable merge trains which would then allow your merged results to to be queued? So what this looks like is I'm going to enable merge trains and now historically now, when I look at this merger, merge request rather than it immediately saying, merge back into that target branch. You can start that merge strain and now I can add this to many.

A

Many concurrent merge requests and they will be queued whenever they're they're clicked start merge strain. That's it uh thanks for joining. Hopefully this was helpful. This was really helpful for me.