GitLab Incubation Engineering, 17 Mar 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Incubation Engineering Jamstack -- Weekly Demo March 17th 2022

Description

TL;DR: https://gitlab.com/gitlab-org/incubation-engineering/jamstack/meta/-/issues/38

Current WIP MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82920

Pipeline Wizard MR / Docs: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76128

A

So hi everyone thanks for joining um we're. Talking about the incubation engineering jam stack sdg update today and I'm gonna take this opportunity to give an update on the pipeline wizard, which has just recently been merged, and I'm gonna walk quickly through it. How it looks now at the moment and how to use it, and then I'm gonna talk about what I'm planning as in terms of next steps, um how I wanna use it with gitlab pages and how that's going to look like um so just to very quickly demo.

A

What it looks like right now.

A

um This is the mr that's now finally been merged, so yay and here's my local instance, where I'm using it in gitlab pages, which is its first use instance. So this is just uh some kind of demo repository that I have set up locally.

A

um I'm gonna go to two pages, um and now, instead of the the old link to the documentation, we're going to be presented with this um simple four-step onboarding process that makes the whole thing work so initially, all we need is a build image which for most front end pages, would be node.js.

A

The next one would be the installation steps which is usually npm, install or npm nci and, and the next one would be. How do I build the application? For example, if I have a nox.js application, it would probably be npm npm run build.

A

You can see how it sets up the github ci on the right hand, side. I've demoed that, before with a little hint as to what part of the ci yaml file we're, currently updating um deployment parameters would be main, um and the last step is actually the the ability to commit this new gitlab ci yaml file, which will then trigger the pipeline, deploy, um we can add a commit message or leave the default one.

A

I can select the branch I want this to commit this to, and I've had a conversation with earlier today about um how I would like in the future, to add the ability here to create an mr instead of directly committing to a branch, but that's something for a later iteration.

A

So this is just an mvc. um It has other features that are missing, such as filtering, but I'm going to talk about this in a minute. So if I click commit, it will obviously commit this, and this is where it stops at the moment, and this is the part that I'm working on right now, um I'm redirecting the user to to an interim state page, because what happens now I've clicked commit.

A

um Obviously a pipeline has been created and is now running and we have that information state in uh gitlab pages um in the gitlab pages metadata, where we know whether a pages site has been deployed, but that won't be true right until um until the page is actually deployed and live.

A

So now we're in in a weird interim state where um the pipeline is probably still running or it may be, maybe failed, because I did something wrong, so we don't have a deployed site yet so right now, if I go back to gitlab pages here, I'm being presented again with the onboarding state. So my little piece of work that I'm doing right now is adding another database column that basically just shows onboarding state. um It will be set to true using the user interface.

A

um So as soon as I'm clicking commit on that last thing um we have the information. Okay, that the user already went through this whole thing so present them with uh a better page that says basically: um okay, we know you set this up, but look in the pipeline for to know what's going on right now um so yeah. This is what I'm doing with gitlab. With regards to gitlab pages, it's a it's a minor, mr, but it's the first one that I've developed that has a database change.

A

So there's a little overhead with that regard with that regard, so there's a little migration in there.

A

This is what I've been working on this week, um but I'm going to use this uh whole gitlab pages um pipeline wizard to demonstrate how the wizard could be used in any other context, um because I've now set up the wizard with this whole configuration where I'm saying here, select your build image and installation steps. How does that look like in the back end and how can that be used in different contexts? Other than pages, so to do that?

A

I'm going to open my ide here. So how does the pipeline wizard look like um this is what I've committed just recently and what is finally merged the pipeline. Wizard is basically just a view component, it's front end, only um there's no back end, part to it, um so it's it's very flexible can be used in different places.

A

um So how am I using this in pages in gitlab pages? I have another view component that is being mounted on the front end whenever I click this. This pages link here using a little condition.

A

So this is all it needs. Basically, so the pipeline wizard is a new view component um and I'm passing it a template and we're going to look, have a look at that template in a minute and I'm passing it.

A

These props, which is basically just the project path which is coming from the back end and the default branch, which is also passed in from the backend and now the pipeline wizard, will do all the rest using the information from the template. Now we can have a look at how this template looks like if you you can see, it is being it's a yaml file that is being loaded from this place and there's a templates folder inside the pipeline, wizard folder, and we're going to have a look at that right now.

A

So I'm going to look at the templates, folder, here's the pages yama, and this is basically the whole um logic behind the pipeline wizard. So basically, what we're saying here is we're giving it the the whole wizard a title gets started with pages we're giving it a description.

A

So we have a little introductory text here and here's the whole part that creates that guides the user to through this process. So we have three three steps.

A

This is a step. This is a step, and this is a step um and you can see how I can click here through step, one um obviously with a required field step two and the step three and the fourth step is always the commit step.

A

um Now we have in each step, we define a set of inputs, um they are basically widgets, so they can look differently. Most widgets are just text inputs that widget could also be a list of text inputs or in the future right now. These are the only two widgets that exist in the future. We're going to have uh more widgets like um a drop down. um We could have a branch selector or anything that we basically could come up with now.

A

This input has also a target, which is basically the name of a variable inside this template. So with you see how build image is going to be included in this template, where the value of this input field is going to replace this build image variable inside the template. The nice thing is um the pipeline. Wizard will also include the comments that you include here in the template.

A

As you can see, it will show you all those comments, so you can help. You can inject comments uh to help the user uh once they've left the the wizard and are working in the gitlab ci ml file on their own.

A

For a little more information- and I I think the most complete documentation as to how this pipeline wizard is working, is in the mr description for this, mr um I'm, currently working also working on getting this whole instruction and documentation into the gitlab docs.

A

Although we currently don't have a proper place to put that, so I have to figure figure that one out where it's supposed to end up until then, uh I think the um mr for the pipeline wizard for the initial pipeline wizard is probably the most informative space for any documentation.

A

Right any questions so far.

B

Hey, uh I listed one uh question, so I'm not sure how how good the question is, but I'll just ask you anyway. You mentioned that this is purely front end. Yes, but it also creates a or it also commits to a branch. For me.

A

Yes, well, the.

A

The information as to how the pipeline wizard is set up and its logic how it's working um is happening all in the front end.

A

Now it does at the last bit um the very last step. The commit step is um basically a graphql query that sends this finished uh gitlab ci yaml file to the back end, um but it the backend, wouldn't know whether how you created this using the web ide or a static editor or whatever um you're, basically just sending the finished camera file to the back end. um The good thing is, um you really only have to set up the pipeline.

A

Wizard like this in the front end anywhere in the whole gitlab ui code base, you could just mount one pipeline wizard pass it a template file, yaml file, um give it the project path and default branch, and it's going to be happy.

B

Right so uh you're saying no new existing endpoints in the back end were introduced. It uses it.

A

There were no no in no uh endpoints introduced. It's basically just a fancy. Very fancy way of writing a yaml file in the front end.

A

B

A

B

Yeah no this this is. uh This is remarkable because uh we've discussed this before, but most of gitlab is just updating a yaml file right, uh there's, there's so much uh yes, so so many parts of our application of data yaml file, we were just going through some of the security configuration sections which are pretty much the same. I was talking about five minute production that I'm working on, which has a part or which involves updating yaml, I'm sure darby has his mobile pipelines that can consume this too. So this is excellent.

B

Thank you for creating this guinness.

A

Yeah, I mean really, it is just a way of writing any yaml. It doesn't even just apply to um the gitlab ci, although this is always the default value for the file name and probably some other places where it references gitlab crm, because it has been written for that.

A

But in the end it would be very, very, very easy um to use the same kind of framework to write, cube, kubernetes configuration files, helm, charts, anything, that's a yama could be done with this, so we I should have called it a yama wizard I didn't, but it is.

B

Yeah, so staying on the theme of reusability um after the last step of the wizard, where it creates this uh commit. um How do I, the consumer of this component, decide what the next step is or the next next endpoint is that it must go to the next view that it must render like? How do I.

A

B

What happens next.

A

um I am planning it that is not implemented yet and that's my the the for the mr after the next one um I plan on introducing an event basically at don um and then redirect. I don't know.

A

Or you can do a router push uh anything, so it's going to emit a done event. Probably um that you can then react to, and I mean feel free to to edit that or to create an mr um that gives more context like the the file.

A

The contents of the file that could be a payload of that dom event. I don't know.

C

I I have a question. This looks really great. It looks also so I've never seen this color before, but it looks super simple. So that's, I think, a great aspect.

C

I was wondering what happens if you have a pipeline already? um Is it supposed to work together or is it going to create a separate pipeline just for the pages or so the in.

A

The mvc like it is right now um assumes you either have no existing pipeline. That is probably the most common use case for pages that I want to use it for um or uh you want to overwrite it, so it doesn't parse an existing file. It just is going to to override.

A

So, as you can see, this project already has a gitlab crml file, so it just says in the last step: that's the only difference, commit changes to your file um and it's going to override them, but yeah, as I said, it won't incorporate whatever exists, although it is I'd, say a simple to medium, simple effort um to change that behavior, because we already do a lot of yaml, parsing and yama merging inside the editor, so the whole logic on how to do that is already there. So I already have a um to do.

B

A

Inside the code and an open issue to to support um just merging whatever you're doing into your existing gitlab crms, so that is a next iteration. It's planned. um It would probably be as simple as introducing a an api call here uh that basically reads the existing github crm file for that project, um although you'd have to think about what.

A

How do we work with different branches and that's the main reason why I haven't implemented it from the get-go, um but basically you just parse the existing yaml file and instead of writing new document, you'd probably do a powers, powers document uh and whatever is the value from from the remote.

A

So from the front-end side, it's probably uh not difficult at all. The biggest challenge is, like the user experience. What, if there's a file in the death branch, but not in the main branch? Are we using the main branch? um What should be the behavior be here? So I haven't really thought about that yet and that's the main reason why it's still a.

C

Makes sense, thank you.

A

Right yeah, so um hopefully, next week I can tell you some progress on the onboarding state column um in the pages metadata, which is just another um preparation step for using the pipeline wizard for pages.