GitLab GitLab Pages onboarding, 19 Jan 2022

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: GItLab Pages development environment setup

Description

I'm showing how to setup gitlab pages dev environment inside the gitlab development kit.

#gitlab_pages

A

Hi, I'm vlad I'm a back-end engineer on the release team here at gitlab uh and most of the time I'm working on club pages recently, and I just wanted to make a quick video, maybe series of videos explaining how to set up a dev environment for github pages, how to and yeah how everything works internally.

A

So this will be a short video just explaining how to set up a dev environment inside the gdk I'll share. My screen now.

A

Yeah, you should see the zoom window, uh so I have a gdk set up already and this is a pages project, but I will redo a lot of stuff there. uh So if you like, I will assume that you're completely new to gitlab development and know nothing about gitlab or gitlab pages.

A

So first thing you need to do is just google for github development kit uh you'll end up on this page and then you just need to go uh to the installation, and I use this one line installation.

A

I should mention that I yeah. Let's do it this way. I should mention that I use you mac uh hardware, uh so as many people here and it's probably easier on mac uh hardware to set up gitlab development kit. But if you don't use this or if you have any uh problems with uh installing github development kit, you can use this page.

A

So I just use this uh one liner.

A

I needed to repeat it a few times, but I guess in about 40 minutes or one hour, I got it working without much changes to my development environment, but I could have some stuff pre-installed, so you may have more problems than I did so in the end you just end up on this page.

A

I should log out.

A

And yeah: let's go to the console.

A

So if you just type jdk help, you'll see the default uh name and password I've already changed that you will be prompted to change it once you've used it yeah, so it will be root. Your password.

A

A

I'm not sure where I will be logged in, I probably yeah by default. You will be logged into this page and you see that I have a if you are familiar with gdk you'll see that this uh url is a bit different by default. It will be just localhost and port 3000 and I have a special ip and a different port. Different part is just because I have uh my 3000 port already in use, and I have it forwarded to my dev server.

A

So it was easier to just modify this port and I needed a different ip just because just to use my uh gitlab runner in the docker mode, because you can't use it with a localhost because it will try to connect from the docker to localhost and local hosting docker will be not what you want it to be, which is gitlab installation in docker. It will be obviously a container itself, so yeah, let's start with the gdkml file.

A

So if you are completely new to the jdk um yeah, uh this is where everything is controlled from, and I highly uh suggest you to modify only this one file: don't modify any other config file on your uh gitlab development kit, just because every time you run jdk update or gdk reconfigure, it will overwrite all other config files. So just if you need to modify anything modify it here, so this part on top is just because I have a separate, gdk installation.

A

I wanted to make this one from scratch, so I needed to override this port. If you don't have any jdk installation just ignore this, you don't need this. So then uh first thing you'll need to to actually build pages websites. You'll need a gitlab runner and uh to make it work with more complex stuff like jkl or anything you'll need uh how to code. You know the you'll need a docker uh object.

A

uh Sorry, basically, you just need to use uh docker uh and docker runner will not work with uh localhost isaac, as I explained before, so we need this hostname. I just yeah.

A

Let's, let me show the second page if you're not familiar with how with how our docs structured inside the gitlab development kit, there is a folder called dock, and inside this there is a folder called called how to, and there is a bunch of articles for special uh subjects. We are only interested in runner right now. There is or one for pages here- it's maybe not updated very often, uh but it can be useful for you but yeah right now. We are only interested in runner. I actually have it opened already, but yeah.

A

This page is super useful. uh It will guide you through how to install runner uh and how to make it work uh if you yeah- and there is a docker installation here. uh This is what you need for pages.

A

It is possible to make it, and my simple example will actually work with a shell executor as it suggested uh above, but yeah you probably will just need docker uh installation anyway. So, let's start with it uh yeah and the reason uh it will guide you through how to create a loopback interface. In my case, it's already there.

A

So if I type just in my console, if config you'll see that I already have somewhere here, I have a special address.

A

Yeah kind of hard to find: let's make it a bit different, take less uh 0.2 yeah. uh I already have it set up on loopback, so yeah I just added an uh another ip address to the my loopback interface. It's basically just another ls for localhost, but it's a different ip yeah and it will be understood from the docker installation so and yeah as a prerequisite. Obviously you need to install docker on your machine, so I just copy it from here.

A

Passed it here and then, as I typed above, I just need to run reconfigure, restart and tail. ah Let's go through some other config options here so for gitlab pages.

A

You just basically need to enable it uh because by default it's disabled uh and then I also uh turned off after update just because if you will be developing gitlab pages, you probably don't want uh gdp every gdk update to override everything, but this is optional. You may disable this.

A

Yeah and for runner uh yeah again, we just need to enable it, and then I just use a docker installation mode. So yeah, that's basically all I need to do.

A

You can't just take token from nowhere I'll explain how to get it a bit later and you can also follow the docs above, but I'll show this.

A

So we do gdk reconfigure.

A

Gdk restart I'll, just type it even before I configure finished.

A

It will take some time from here.

A

uh Yeah you can, if you're just watching the video you can probably scroll a few times. I won't be catching this into parts. So, while I'm waiting, you can scroll the video.

B

A

Okay, so now, as you see, uh a reconfiguration is finished and we are restarting gitlab, it will take about two minutes uh to actually get the web service running so uh yeah, but yeah. Okay, we have everything uh restarted.

A

You can see that we have a url, we can follow, but yeah I already have it opened, but yeah. Actually, let's just do this. Let's copy it from here.

A

Yeah now we see in this page because uh gitlab workhorse is already running, is just a simple proxy server written and go so it's it's booting up much much quicker than the gitlab server which is rails. You don't need to do anything as as it mentions it will reload this page. Every five seconds until rails is ready.

A

Yeah, I can probably go to the docks while we are waiting. um What is here this is the runner page uh and.

A

We actually only need uh yeah. I copied this already as we as you saw to the uh gdk yml file. Oh by the way I forgot to mention. uh Gdk yml file is located inside.

A

Yeah, I just realized that I should have mentioned this before uh so once you run uh gdk you'll have a uh directory called git lab development kit.

A

Yeah, in my case I just started in tmp directory uh and inside this file. You have gdk yml and jdk example. Ml example. Yml will contain all the options you can use uh with. Gdk uml uh yeah definitely should have started with this, but anyway, um now we have this uh runner.

A

uh We need to actually register it uh for it to work yeah, let's login into our server.

A

um Yeah, I need to reload this a couple times I mean not couple but.

A

I just go here.

A

A

And if you guessing is my password, if my password is one two, three four nine and so on, then you're right.

A

Okay, we logged in so register runner. You need to go to this menu to admin panel.

A

You can also register runner for a single project, but it will make it much easier if you just register it as a system runner for the whole jdk installation, so go to runners.

A

uh I'll delete the existing runner, um I will just. I was just trying uh to do this before recording the video but I'll show it how it's done from scratch. Okay,.

A

So on this uh page there is a command line somewhere, which registers runners.

A

We just need to find it.

A

Okay, it's not this so register, oh yeah, it's just uh below! No, it isn't anyway, it is somewhere here yeah. So we just need this uh command line uh and, as you see uh I'm doing this in uh gitlab development directory, uh there are a lot of stuff here, but also one of the things we need is this gitlab runner, config, tomo file, so I'll just cut it for you to see um gitlab runner yeah.

A

As you see, everything is already set up here, but uh there is a token which is like uh which isn't registered. So what we then do now is just register. Runner will use the same file uh as a config file.

A

Run. Untucked just means that it can run any job on the gitlab server. It will ask you for the url, so I'll just copy it from here.

A

Gitlab ci token, so you can take it from here just copy past it here, just keep name, skip tags, don't enter anything or you will not be able to run jobs here I'll I use docker.

A

uh You can use anything, but uh it's easier to just copy that and now, if we cut this file again, you see that we have actually two runners registered and probably this one will work, but every time you reconfigure your instance, it will be gone.

A

So we go here. uh We uncomment this line it. It won't be there if this file will be empty for you, but, uh as I mentioned, you can just go to gdk example copy some structure. Like some parameters, you need uh yeah and there is a token somewhere here yeah. As you see, this is the default token, which is used, so you can copy it here and you can copy uh the token from yeah from this installation uh to here, uh and then we just need to reconfigure our jdk once again.

A

I'm not sure a free configure actually does restart so I'll type, I'll type it every time, but maybe it actually does restart so yeah. Now we are waiting again and you can scroll the video for a minute again.

A

While I meet myself.

A

Am I even recording.

A

Zoom just complaining that I'm not recording myself.

A

Okay, so I'm recording and we are restarting.

A

A

A

Now we need to refresh this page. It will ask us for the password once again, because I forgot to press the remi. Remember me button and gdk just asks you for the password every time you rerun it restart it.

A

Okay, now you see it's been low uh yeah, it's loading.

A

And rails just takes some time to actually start the first request.

A

I mean to load all the code.

A

A

Okay, so we are again logging in once again.

A

Okay and you see that we have a runner which was active 40 seconds ago uh yeah and now we should have a working runner. So let's just go to the main page and create a new project.

A

So just press the button new project.

A

Create from template there are a number of pages products here are gatsby hugo jekyll. I uh totally recommend using just plain html.

A

Just because pipeline for those projects takes significant time to being processed, it's like a minute or two while a job for plain html is basically copying single file and it's much much quicker to you uh to test pages. So I'm going to use this one, let's name it pages project.

A

You know important thing here is to make it public because, depending on the level visibility level, uh project can uh be like there is a feature called access control for pages, which is uh verifying that users have access to the project via oauth workflow and above when I showed my gdk yml file. I did not do anything about the house process, so it should be disabled, but in case it will be enabled uh pages, will just uh not work properly, so just make it project public. It will make your life easier.

A

So, let's create it.

A

If you need to work with altification.

A

I guess you can find some some information about this inside the how two pages uh file yeah. There are some things here, but yeah. You need to know what to what you're doing and how this feature work works inside. So right now we are seeing that import is in progress. It's basically a sidekick job uh which is running this and I guess we already refresh refreshing page. So import is finished now yeah.

A

What we need to do next is to run the pipeline, because when you create a project pipe, it's not run automatically at the moment.

A

So let's go to cicd pipelines.

A

Wait a bit then.

A

And we need to run the pipeline.

A

Okay pipeline is created.

A

And I hope, when it will be loaded, job will be already running on the runner. Oh yeah, if you're interested what is happening and if you have any problems, just run gdk tail.

A

It will show you all the logs and if something is broken uh most of the time most of these problems will be here uh so yeah. If something is wrong with the runner, you'll see it.

A

If something is complaining that some part is in use or something it also will be here, and it's just ctrl c terminate to exit this so yeah. We have a pages job running it probably already completed.

A

Yeah succeeded, so we can go back refresh the pipeline.

A

Yeah just cancel the page loading. That's why we're seeing this message?

A

We see that we have two jobs, so the first one is uh what you saw. Actually, let me show you it uh how it works.

A

So now we're just viewing the main directory of the project.

A

Yeah, I never visited this page in this gdk. So that's why I saw this webpack compiling message now. We also need to wait until the tree is loaded.

A

Yeah, it all takes time so inside uh we have this git club ci yml file.

A

You can actually just google gitlab pages and it will explain everything about this file, but yeah now we're again waiting for the assets to be compiled.

A

Okay, this is finished, and now we just need to wait for the file to be loaded. Okay, it's a super simple file. So uh this is why you need docker image uh because yeah I mean docker runner, because uh the default template already assumes that you have a docker runner.

A

So there is a single job here uh stage deployed. It doesn't matter that it's called this way. uh You can even skip this uh script, nothing to do because uh actually, I'm not sure if I will be easy yeah. Actually, I should show this probably so uh inside this project we just have a public directory and inside it we have index html and style css files. So the way pages works is gitlab.

A

Pages works is that you can run any jobs, ci job which is called pages and which has artifacts exported inside the directory called public. So we already have it here, so we don't need to do anything. It's just been loaded with git clone. uh So what rhino will do? Is it will just uh send this public directory as a zip file to gitlab as an artifacts file?

A

You can actually see it here if you click the pages. The job.

A

Yeah, you can see the job artifacts, you can uh download them. uh Yeah.

A

Yeah you see that artifacts file is downloaded, it's been downloaded anyway uh and inside it it will be just this. uh This file tracks right here, index and style. Css, nothing, fancy inside it, uh so yeah and you can do any. You can build any site this way. uh So job is succeeded.

A

If we go back to the pipeline. Page you'll see another job here, which is pages, deploy it's actually kind of a fake job. It's not executed on the runner, uh it's just a sidekick worker which executes this job and it's just copies, zip.

A

This zip archive, I just shown you to special place uh and then pages demon can take it from there. I probably will explain this uh later and probably it will be another video uh showing how all this works internally, but yeah anyway.

A

Let's continue with our example. Actually everything is ready. We can now just see how everything works we go here. We have this uh url open this in the next tab and yeah. Here is the final example: we see the uh pages website here everything works out of the box uh yeah. Now I will just show you how to make a simple change to the pages.

A

Daemon. Actually so first thing first, is that uh we have this gitlab development kit directory. There are a bunch of stuff in here and there is a directory called gitlab pages if we just go there gitlab pages.

A

uh So first of all, this is a reference yeah. I can just show this. So if you go above, there is a inside gitlab directory. There is a co file called gitlab.

A

Pages version 1510: this is a tag uh inside gitlab repository gitlab pages repository, so.

A

These pages this uh commit hash, just corresponds to this text. So if you, if we show uh it show we and these pages version, you see this, there should be yeah. This is a commit hash. We just saw uh yeah. So, let's first things: first, let's just check out master.

A

Pull it from the origin.

A

You probably want to create a special branch. Add the remote by default. Remote will be https if you're a gitlab team member. You probably want to switch this to ssh remote. If you're a community contributor, you probably want to add another remote for your fork of gitlab pages but yeah for now. I just assume that you know that and you want to change something inside the gitlab pages daemon and actually see the result of it running.

A

So I'm gonna use my editor, which will be a golan in this example.

B

A

I'm just opening it opening it on the another screen. So that's why you don't see anything.

A

So yeah here's my golend, let's open the directory.

A

My case, it's tmp gitlab development, kit, gitlab pages.

A

Yeah, it's again opening on another screen. I will transfer it back here.

A

Okay, I'm just one to check yeah. I hope I I'm starting the right screen. Oh I'll need to re-record this and it will be painful anyway. uh Let's go to this uh go end, so I didn't know. I don't know what to change this uh easily in this demand to show this, but we have a main go file and let's just show, uh let's just change. What is printed uh when you run the.

A

Gitlab pages server, uh so yeah we have a main function. I believe it's somewhere yeah. It is here. We validate arguments fail to load config there. There is a log config function. Let's just go into this, uh go to.

A

What is wrong with yeah, I guess it's yeah. It's indexing the file but yeah. Let's just find it lock. Config.

A

Here we are: oh, it's actually a different function, but whatever uh no it's. It is the right function anyway. So here we can add something I don't know uh yeah. We have this message, so let's just make it visible. I don't know starting pages.

A

So how to actually see the results? First of all, you need to go to this directory, then just run go build.

A

It will download all the go modules, build every and you end up with a file called gitlab pages. It will be binary file located in this very directory.

A

So yeah here it is, you can actually just run the binary, but jdk already does this for you. Let's just run tests before we do this so.

A

This is how I usually just run tests and you can use a shortcut like go, build and go test. This just meets every director inside this, so it will uh run tests for every package inside gitlab pages yeah, it's building bunch of stuff again, but yeah. Now we see that everything is working most of the time when you will be working on github pages.

A

I suggest you to just run write tests and run this. It will make it much easier than to test than testing anything in the browser.

A

Okay, so all tests are passing now, uh let's copy this and run gdk tail uh gitlab pages.

A

So we see only what pages is printing here and inside our directory.

A

I have a zoom window.

A

Breaking everything for me so yeah, we have a tail here and we have a build directory here. So what you do is just type jdk or restart gitlab pages.

A

And we see in here what is happening so now the process is being terminated.

A

Oh, you don't see anything in here. um I wonder how we can see the results.

A

Actually, let's do this, let's just grip for pages.

A

Okay, there is a config file by the way, if you're interested how all this works inside uh yeah there is a config file describing uh how pages should be uh configured. I guess um cut.

A

Hitler pages.

A

Oh, it's actually in this so.

B

A

Actually, I don't know where log variables falls, but whatever you should see, but anyway, uh just to make uh the case gitlab pages um yeah, it doesn't print print much. I guess um yeah. So it's kind of a failed example. But here you should actually see the.

A

Yeah, I don't know here, you should actually see that page has done something, let's, let's actually make it do something. So we go back to our editor. Go here, add verbose.

A

True save gdk re, configure, jdk, restart kit, lab pages.

A

The configure will take time again.

A

But at least I hope we'll see that we actually did some changes to the gitlab pages daemon.

A

Yeah anyway, so you can just work from this directory inside your jdk push branches and develop here completely. Conveniently you don't need to use, go pass or anything like that today and yeah test. All the changes here test them right inside the gdk.

A

uh Just don't forget to uh make when you test something just run the go, build and gdk restart gitlab pages. It will use this binary. I've shown you yeah and you can easily test this. This way you can easily test all the modifications you do, the gitlab pages daemon and then just create a merge request.

A

It will run the pipeline for you. If you want to do. You can also run tests locally. You can run linters locally, uh but this is all kind of advanced, so yeah I'll try to make another video explaining how all this works internally inside the rails, application and inside gitlab pages application, but I would actually do it here, but I have a meeting in three minutes, so I probably end here without this example being finished, but yeah thanks for watching.