GitLab Verify Group, 22 Mar 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitLab CI Walkthrough

Description

Brendan (https://about.gitlab.com/company/team/#olearycrew) and Sarah (https://about.gitlab.com/company/team/#superSGP) take a walk through GitLab CI to find what is confusing for new users (Sarah started at GitLab recently 😃).

For more information on GitLab CI, see: https://about.gitlab.com/product/continuous-integration/.

To see our thoughts and contribute to how we can improve, see our Verify direction page: https://about.gitlab.com/direction/verify/ and public issue tracker: https://gitlab.com/gitlab-org/gitlab-ce/issues.

A

B

A

Now, recording our exciting adventure into the CI product sure.

C

Sure yeah so I I. Do you want me to share my screen, or do you want to share yours like how do you want to why.

A

Don't you share yours and I can watch that and then I can take notes in the background on my nose, great.

C

Great yeah so I think the general purpose was you wanted to just kind of walk through the part of the product that is verify see, see, I didn't get lab. Is that right or like? Are there specific questions you have maybe or no.

A

Mostly just to walk through from a product perspective, so this is like the beginning of my third week being on the verify team. As I start, you know working on bugs and looking into stuff I realized that, like you know, for instance, the pipeline exists in like five different places. The pipeline, widget and I don't necessarily know off the top of my head, which ones those are, and since you do, the product rusty seems like the right person to give me the overview of how it looks from a product perspective. Sure.

C

And I'll try, but you know I as I as I like to point out. I've got that science dog that doesn't know what he's doing as my avatar for my demo projects um but yeah, so yeah, we'll just kind of walk through I'll spend 10 minutes, maybe kind of talking at you about how I see the CI part of things and then and then from there. Hopefully we'll like have a lot of questions and I really appreciate you taking notes. Cuz I, think you're gonna have a lot of insight.

C

I always say that people that are new like have such a unique opportunity to contribute in a way that those of us that are you know been here longer can't, especially for me because, like I, have been using get lab since before I even was that cat lab so like I am blind to a lot of things that I need help with, and I'm really excited to. Have you on the team help me not a piece of blind of those things and see things that are obvious.

C

Ux and UI sticking points alright, so yeah I'm, just gonna kind of dive in this is get lab. Cops like Brendon demo, which is public.

C

Oh sorry, slash, Bren demo, which is public, it's just kind of where I play with some some different CI things, and so this is a little react, dashboard app that I deployed and get lab pages. What's the I. So if I look, this is behind the curtains, not CI stuff, but just to bring up the the page right. It's just a little. This thing called core UI that gives you a one of a thousand admin dashboard templates that are in the world in react right, but it's it's decent cuz!

C

It's a you know: decent-sized application, I, don't know that I'd call it a real-world application, but it's decent sizes to play with so the way that I get at CI, typically mister get lab, see. I am right! That's the thing! That's going to define the stages and jobs that are going to be included in my pipeline eventually now there's another thing that get lab.

C

Has that I'm not going to necessarily talk about unless you want to later called Auto DevOps, where I don't have to define and get lab C IMO technically, that is part of the configure stage at gate, lab I'm, not the verify stage, but of course it uses the verify stage as much of the operation side of the house does that's. Why, on our vision, page for verify, I called verify the Gateway to operations right, because that's how it's?

C

How release orchestrates the things that the gate lab early stage does its, how configure orchestrates a lot of what they do, especially on the dev ops, it's how security secure orchestrates, what they do and how a lot of its secures outlets are exposed to the user and so verify has this interesting kind of in flux? It's this interesting kind of inflection point between your writing the code and then your deploying and operating the code.

C

In fact, some folks would argue that verify should be on the dev side of house versus the other side of the house.

C

We have it as the first op stage, but yeah anyway, here I have not used that I've manually, orchestrated my jobs and stages with a gait lab CIE animal.

C

So here we can see like the basic structure of the animal, is jobs that are here and there's some stuff up above that I'll talk to you and each of those jobs is gonna, have it at a minimum, a stage and a script that it runs. So the stages are run in serial right like first here we see the stages defined as prepare build test deploy, so those stages are going to run, see I really like that. But then, if there are multiple jobs within a given stage, those run in parallel right.

C

So here we can see the prepare stage is going to run this one job called NPM and it's going to run. Npm install the build site. Job is gonna, run during the build stage and run NPM build, and then I've got some pretty much more complicated jobs here below that are going to do stuff with code quality and performance.

C

We'll come back to that after we look at the pipeline, maybe, and then I've got a special job named pages, which is a protected job name and get lab that deploys it into get lab pages which forget, live, comm is hosted on, get lined up IO and that's how I get this live app running.

C

Of course it's just aged. You know it's basically a static site hosting get our pages, and so this renders down in the end to just HTML Javascript that can be hosted here.

C

Okay, so that's overview that we can come back to it and talk about it more, but I don't want to spend too much time on any one thing. So every time I commit my code. That pipeline is going to run right and there's a lot of options. I can do around what runs on branches versus on master. In fact, we can see that here already here were writing something on the deep branch change, something of course it's here running a commit of on master.

C

You can see the stages are different right because I've got these stages below and let's see what that is so prepare build test, prepare build test, but then I don't have deploy right. So this deployment only happens on master and we I've seen that there I didn't point it out to you in the yellow. But that's why these look a little bit different right. I can see the status of all these. You can see, there's a couple different sizes that it's failed, which is obvious Pass, which is pretty obvious.

C

This pass, which is less obvious, maybe which means it passed, but with warnings right and so this SAS job is allowed to fail and still pass the pipeline, but it had some submarines involved.

C

So then, if I'm looking, if I want to look at an individual pipeline, let's, let's see what warnings happened on this pipeline right I can either click into a job from here and go right into the output of that job right. So, if I click on the SAS job, this is now the actual trace from when it ran that job and so we'll see the steps here. Right, like export docker run this container right.

C

Those are the things that were specified in my mo and then the output of the scripts right and all the stuff that happened and at the very bottom, hopefully we'll see what happened. I can't tell what happened because I understand it, but something bad happened. Maybe I've got a weirdly malformed it to react, elemental, some sort, I, don't know. Yeah.

A

It looks like fragment, isn't defined or something that we yeah.

C

I, you know, I also, don't know react, but that's that's neither here nor there.

C

Certainly, but the other way I could get back to this sorry was from the pipeline page I. Could click this and I'm gonna get more details around the pipeline as a whole? So here we see the same thing that I was hovering over to expand, but in one kind of view right, like everything that happened again, stages prepared build test jobs that run concurrently on the stages right. I can click back in here, I get to the same place.

C

We just were on that SAS job or I can also go to the failed jobs tab here, which will show me just the bottom trace from any job that failed. So here we can see the same thing. I can also see every job that that ran in a kind of a different view right. This is more top-down view versus a left-to-right view. I can see what commit this was based on and what the look the hash for that was I can see how long it took Tehran when it ran.

C

I can retry a failed job which this will fail again, because I think I'm, pretty sure. Cuz I've got some syntax error in there, but that's kind of the overview of the pipeline. So then I can also see every job that's running, and so this is just basically an expanded view of the pipeline. Plus we were looking at before. I can also schedule. So there's there's a number of things that can trigger jobs. Right, like I said a commit, is gonna trigger a job.

C

Obviously, on the branch that commits done, you can also have jobs trigger and on tag. You can have jobs triggered from an upstream pipeline. So if there's some upstream dependency here that then triggered this this pipeline to run I could and I could also have a schedule so, like maybe I, want to run every week. Some some job.

A

What kind of upstream would trigger a pipeline like I'm, not quite sure, I, understand how that works? Sure.

C

So so right now, we actually only have the upstream trigger in a downstream, which is little less intuitive, we're actually adding right now being able to watch an opportune pipeline, a trigger which is more, maybe more realistic. But, like let's say you had, you know on an iOS and Android app that you were deploying that needed to redeploy anytime. You updated your server-side API right that service ID. Those might be three different projects and get lab right, one that has like the server-side API code, one that has the iOS.

C

You know objective-c code and the one that has the Android code for the Android app. So you might say anytime, I change project, a server-side code trigger a you, know, rebuild and deploy of iOS and rebuild on deploy of Android or something right like you can imagine, there's lots of those kinds of that's what we mean when we say trigger we're, actually looking at changing those words, stock stream and downstream, but like again things that that build.

C

Maybe a larger system then would be contained. Typically in one project. Right, like micro-services, would be a great buzz word to use here, but like any anything that that really consists of more than just the project itself right, so everything that we've looked at so far has been very project. Centered centric on this one react project within gitlab, but obviously, when building the applications in the real world, it's not only just one little project that you get to deploy hmm and JavaScript.

C

It might be multiple projects that come together into a you know, overall application or overall system I, don't know how you name your world and so.

A

And then you and then you trigger those using the gitlab config yeah Moe. Is that correct? Are you.

C

Yep, so there's a really great doc that I love and the fact that I love- it is not a positive thing. I've actually talked with our team about and we're doing a lot about, Doc's team, but there's this configuration reference, it's great reference documentation right and so that that's that's what I always say: I say: I, Brendan O'leary, yet lab CI user of three years loves this document because I come here and control F, whatever I want, but the best for anybody else.

C

Maybe I could find some examples in it. Those. Why is why I pulled up yeah but look at the trigger syntax. I might say that.

C

You know on after deploy, go trigger some other projects on on branch on its stable branch right and so that that may be where I again trigger a downstream pipeline and then that actually shows up in the UI in a great way and I think I have one somewhere where you can see an upstream pipeline.

C

Triggering a downstream pipeline may be triggering another downstream pipeline, and so that's I think I do actually have that in Brendon demo, for the folks, following along at home in the buzzword, compliant micro-services group I have a parent and B downstream, where the pipeline and think for a parent. Yes, so we can see we build and then I trigger and then that triggers this downstream pipeline on the other project, which I can't go to.

A

Yeah, that's kind of interesting that you, you would really want to be able to click like right into the pipeline of the downstream one. Theoretically, right from here, yeah.

C

That would be a great one for us too right. Why can't I do I thought you could I, don't know, that's either broken or shouldn't be like that, so it's broken one way or another yeah, so that here we can see the build happening on the child or a parent. Yes, the apparent is supposed to go back.

C

There's there's something funky it's trying to do something for me: I will figure that one out I think some of that expansion might actually be coming in eleven nine, and so we could be in the middle of a release, candidate issue here and I'm, not sure that's definitely worth looking into. Yes, that's the that's the idea of upstream and downstream kind of relationships, and that's something that has traditionally been kind of hard to manage and get lab and we're trying to make more lovable, and that's we're seeing some of that.

C

As far as other things that you can do with it, let's, let's talk about like other in use cases, so another thing we saw was caching. Let's just go back to my react. App so I had that preparer job and all that preparer job was doing was npm install', but then it it was doing something special with that where we can see that, but I've defined a global cache here and I can define this as job level or.

C

It looks actually, but here you can see I'm caching, node modules right, and so what that means is if node modules exists at the end of my job, upload it to the cache and then pull it down before. I start my next thing right and so I'm I'm, not a developer, but if I was a developer, I'd be a no developer, so why my examples are going to be a node. Sorry, no.

A

C

Yeah so like here, my prepare stages is doing the mmm install, but then actually no models for everybody else. Right so I don't have to empty I'm install every time, and you can see in my build site job. It's just doing, npm build which those of us that are no developers and javascript people. No, it would fail if you didn't, have your modules, but but we're cashing them.

C

So it's okay and then there's another concept of artifacts, so just to kind of differentiate those two here we have the code quality job, that's kind of scanning for code quality.

C

It's using the get lab docker image to do that and you can have an artifact be kind of like any generic thing you want, and so sometimes folks, use and and the difference between artifact in cache, and this case beam that artifacts are kind of meant to outlive the pipeline right like maybe, if I needed, to pass something to the downstream pipeline, it could be an artifact, whereas the cache is only kind of like best-case and it's for this pipeline. Only and so artifact can be just kind of any path.

C

Like oh I want to wrap up this, you know set of jars into a tar gz that is on this right and then it there is, or it could be, the special report type. So we have a report type of artifact. That, in this case, is a very specific JSON format that then get lab, interprets and puts in the merger. Quest to surface like are different, a different report, output that we have, and so that can include things like code quality, so we can see there's no changes to code quality in this performance.

C

Right performance is another category of ours in verify, so we can see that the speed index went, went down by some maverick measures, measure and then unit tests right. So here we can see unit tests all passed, all one of them right and so that this report artifact, is how gitlab interprets that data right. So we can see the code, quality or artifact that performance artifact see the test. Oh probably is in this merge request.

C

In this merger quest on adding you know tests right so here we can see I'm just running unit tests that then output, a like je that XML for forget, lab to interpret.

C

Other idea so couple other concept I'll touch on, so we saw this image at the top again. This is a global image declaration, so Caitlin runner actually has a number of different executors that it can run as right. The most common is our docker executors and that's what like the shared runners on get lab, comma and so the beauty of the docker executors in the a little bit of luck about when we built the product when docker was becoming a thing and B I got to give Camille some credit on.

C

You know having a little bit of vision here and what he built, although he gets shy when I give him this credit, like it's kind of dealt with his daugher first mentality, and so the beauty is. The developer can just call that you know a base docker image from docker hub or wherever to do the build inside. Oh right, and this just labels you to go a lot faster than other tools where you might have to like. Oh I want to use no.10.

C

Now, instead of node, 8 I've got to go open a ticket for my DevOps team. To add a node, 10 runner or you know, process to to our build process, and so that's that's. What we're doing here, we're just saying generally use the node latest image when you're building, but then I can also override that. So you can see code quality is just using docker stable to to do that, and then also you can call out services.

C

So I, don't know how well you know darker, but service is here just means like another docker image to run alongside, and so we run docker and doctor along sides of them can run docker and docker.

C

It's very meta.

B

A

Know a ton about the details about docker I know generally yeah.

C

I mean yeah, you don't need to know a whole lot, I think to get service verify again huge power, verify that, like I, know enough to be dangerous with docker and it so I could be dangerous and verify. But then the executor can be other kinds. Executors right, like you, can have just a shell executor.

C

That's just running on a on some sort of compute platform that just literally the scripts you run or just going to show out that thing you can have dot an executor inside of kubernetes and I'm, an executor called docker plus machine, which is where you actually spin up an additional vm to then run inside of each time. You do a job which actually get lab comm is doing, but it's transparent to the user, but the beauty of that is your runner can be anywhere.

C

So like an example, I give of that is a semi silly project of mine, I have a I, have a rarely posted to blog, called CI CD. All the things where I try to use give up. Ci CD for look.

B

C

Really, it's posted my my certificate expired. That's not really opposed to it. So, like I got a thing called a pie hole at home, which is a Raspberry Pi that you use as like a DNS black hole for like tracking software like ad tracking stuff, where basically to take your whole house and make a black hole of like I. Don't want to be tracked by all this! These you know stuff! That's out there. You can say: okay put a DNS in the middle of all your devices anyway, that's neither here nor there, but I.

C

Of course, when I was doing it said, let's put it on get lab, and so the beauty of that is it's a Raspberry Pi sitting in my house, but I wanted to be able to deploy to it right and so like a tip again, another more typical or or less a less flexible platform might say. Oh well, now you've got a I've got to poke a hole in my router and like have get lab comm like connect in through to my home.

C

Somehow, but I didn't need to do that, because I just installed get lab runner on my Raspberry Pi inside my network and it's pinging gitlab calm to look for jobs right rather than the other way around. So it's a lot saver. So if I go to my CI CD settings here, we'll see my little raspberry pie, runner so I go to runners.

C

The runners available to me are all of gitlab coms, shared runners, but also this little friend called the pie, hole which is the raspberry pie in my house that I tagged with the thing called Prague, which is just a you, can tag it with anything like I called it prod, which means that all of the jobs that are tagged with prod, which are the only things I'm actually running here, will only run on that runner right, and so this basically says connect to that runner, because it's tagged prod and run this shell script right boom, and so, when a job executes there, it actually actually does not execute on get labs a cloud it executes on my Raspberry Pi at home, as you can see it running on piehole right.

C

So that's all just just running clones, the whole thing the raspberry and runs the script again, a little bit more, maybe of an advanced use case, but really valuable for people that want to like have like a bespoke runner, a runner that you know a lot of a lot of our users will have maybe a shared pool. Runners like we have on get lab comm on their self hosted instance, but then they may have specific runners to deploy right to production and now that runners more locked down so anyway, that is runners in five seconds.

C

A

I'm, a little still curious about runners I'm, not totally certain I understand exactly what a runner is hmm I know. That's it I'm, not a hundred percent sure like what it is. I know the one can install it locally on one's own machine. No! No, so is that just like, what's running like it's allotting the threads and like the computing memory to run the processes exactly.

C

Yeah, so get lab. Runner is a separate application right. That's just written in go I'm gonna, get to some train slides here. Sorry, there's a couple slides that will help me talk to me better about it yep. So it's it's just a little process right that runs somewhere and it's written and go so it can run on almost anything right. You can install it on your computer.

C

You can stall it anywhere, and so it is basically the process, that's sitting and waiting for jobs that match it from gitlab and then it takes and and grabs those jobs and executes them right. So yeah, it's kind of like just a little service. If you will that runs on on your computer or any computer, you want and then again it can. It can get more complicated when you say: okay, I want this runner to actually auto-scale more runners for it right.

C

So that's what like what we do on get live.com we've got a couple of runner like hosts that then spawn lots of other running hosts right like we're gonna dashboards.

C

Which is also public and I go to.

C

I, don't scale machine metrics.

C

We can see that, like all day long, here's how many jobs have been run in the last 24 hours on get live. Comm.

C

Somewhere I mean this is just total file system memory, nor everything where the number of instances I don't think that would be here right, but but basically, as these jobs are coming in and we scale up machines to to run them and then we destroy them when we're done, and so that's that's, obviously a much more complicating use case.

C

Then just install I don't know one machine, but it's it's what we, what we and many of our large enterprise customers do in order to like balance, what it lets you do is balance. How long does it take for a job to get started with how much am I paying for compute right, like? Obviously, you could just have a bunch of bulky servers that are really powerful sitting around waiting for jobs all day and that works too. But you know in a cloud need of this world.

C

You want to just have it scale up and down.

A

And so one thing, if like customers are saying, oh it's taking forever to run our jobs like one solution, is add more machine exactly.

C

And we'll compute it exactly and say: ok, let it let it run concurrently out here. Here's more yep runs the scripts inside. We saw that architecture.

C

Again, you can have shared and specific right so like again on get lab comm, everybody just gets the pool of shared runners right, and so we limit, like the number of minutes, depending on your tier, that you get to run against our compute and actually coming soon. You can buy more minutes, but what a lot of people do is they bring their own specific runners right and then they've brought their own compute and they don't have big kind of like own than their ability. The scale how fast and slow and where things.

C

Yeah, you literally just run. You know this binary that you get from from gitlab and then you register it against it lab and it, and it starts waiting for waiting for jobs.

C

Hopefully that helped sorry, yes,.

A

That helps no that helps a ton yeah.

C

I mean it's like a complicated, let my calm, it's not complicated, but it's like I know it's a lot of bespoke terminology. Maybe right, like you like, take a hold of your first coming on to what I bet and again I've always got to remind myself and like I'm, always working on my myself, like you've, been doing this a really long time. I'm like that, doesn't help the new people or like someone that wants to figure out um CI or, like the brand new user, didn't get live, combat right.

C

A

Yeah, it's definitely not I've spoken a little bit to some of the tech writing. People too. I know that.

B

You guys are working.

C

A

Of expanding the documentation because yeah, it's not necessarily the easiest thing to like dive right into what is interesting, I think cuz, there's two levels of it in the product right, there's like the configuration part and like building it, which is all very interesting and then there's the like user part where you know, if you're someone who's writing code all day and pushing it up.

A

The more dev side of the DevOps part of it is a totally different thing where a lot of your encounter is really going to be through that pipeline, widget or the bigger pipelines exactly.

C

Exactly yeah and then and then there's a lot to be done about that. I think I'm, just like to kind of get your friend thoughts flowing right like we've looked at nice, simple Brenden demo projects, but if we go look at more of a real-world use case like building get labs EE, it gets way more complicated way more quickly. Alright, so like the get lab c CI mo for for get lab itself is.

C

Eleven hundred and seventy lines- Netflix, you know, as we all know as programmers- is not necessarily an indication of good, because that that might mean that some stuff is not written a little oh right or not not not against the people who wrote the CMO, but like probably we're not giving primitives that, let you dry up code, better right and then the pipeline is obviously also very complicated and then basically always running right as you'll come to find.

C

As you start, committing more and more code right, I mean there's only oh, no there's a bunch like yeah, so the entire first page is basically running pipelines right. What's.

A

That pipeline icon there, like slanty one. So if you look.

C

At this council, yep I see I, see yeah, it's a good question that was cancelled, so someone cancelled it, which tip can happen manually like I'm, not going to do this. The fact I probably not have permissions to yeah but I've had permissions. I could can't so.

B

Some job in here, oh wow,.

A

This is huge yeah.

C

This is the test, so this is our spec right yeah, like 50 paralyzed, our specs on get lab.

C

There's some manual triggers right. So another thing we didn't talk about yeah is like I can have some that manually trigger right. So like this one in this case like to do a deploy a few, a full QA review, app I actually have to like request it right and say play.

C

These actually are supposed to collapse until one.

A

It's just numbered one out of 50. Yes,.

C

Yes and then yeah and then skipped you can see like it'll skip things sometimes or you can actually tell it to skip. But you know this is canceled. You see this whole pipeline is go just say another commit to that branch can cancel it too right. You can see like this person.

C

Can't era has committed a couple times in this branch, and so once there's another commit. We try to be smart about being efficient and stop the pipeline before, because something changed like around the pipeline fresh right. This is the theory behind that yeah we can see. This is a this. Is a convert, commit to master on get lab and it's gonna be not, incidentally, 39 jobs run on every master, commit for get lab.

B

Itself, mostly test.

C

Yes, heavy test right, oh yeah, so here's a here they rolled up together right so like our spec PG, is actually using our parallel key one where you, you can tell it to run up to fifty parallel jobs and then it creates those and gives you like index right like a variable, that's the index of it.

C

And so then you can smartly split up your your jobs into parallel, but we also want to do that automatically for people because like why, if we knew it was a rails app like why couldn't we just do that right.

B

C

Yeah, that's a lot of the stuff that I want to like tackle is like being like yeah I. Think the thing that thing I always say to folks is like I think we're get live. Ci CD verify what everyone call. It is called verify, instead their mark or watching and say what everyone call it they do matter. I.

A

Team page is my handbook update for onboarding as I went and I change it all. The CIC.

C

There should be one term for it like that's why I said they'd be mad at me and rightly so, confusing these lots of different terms for the same thing. So the thing verify I think verify is really good. Yeah kind of what I'm calling the middle-of-the-road, maybe not the right thing but like in the middle I, think it's great right like if you've kind of done some cio for and maybe you've touched to get loud before and maybe you're don't like jenkins, like I. Don't like it's.

C

It's that's great for like a lot of use cases, but then I think where there's a lot of the work and a lot of what we can do can make a lot better and I. Think a lot of people to celebrate that because, like it is in my mind, it is the best tool in the market.

C

I'm, like I, thought that before I work here so but I think we can gain a lot in the in kind of the beginning side of that spectrum and the end being folks that are getting started with CI or gain start with UFC I write like that stuff.

C

We've talked about a lot here like how can we onboard people easy ugly, like expose better these words pipeline job stage, runner like how can we make that like more clear and then also on the other side, like the super advanced use cases right like they're, you know more in who owns delivery for us and and all the you know, maintainer zuv, gitlab and folks that do have advanced, not just buzzword.

C

You micro services services that they're running folks that have massive mono repos right, like we've, got a lot of users and customers that are like you know: gaming companies or you know, building you know, building the end, not just an Android app but building the Android code base because they're a device manufacturer right like how do we make that calm, minor repair thing lovable, so I think on that those end of the spectrums is how I think about verify.

C

How can we make get on board right like because I think if you can get off what you're doing you can find a lot of positives quickly, but then also how do we solve at the same time? And there you know whenever you're designing software, these two things can conflict' with each other. So like it's also good to keep in mind, how can you help like a really advanced user, go even faster and that's kind of make.

A

Sense, I have one other question for you. Yes, people talking a lot and I know: we've been doing work on like secrets and masking things and that sort of environment variables, and so, where is does that tend to.

B

It's a great question that I didn't touch it on. Okay,.

C

So I have a little the thing that brought me to get lab comm originally, because I had a little side, slack app with a buddy I'm doing, which makes no money because spoiler alert people don't pay for slacks. They don't pay for you have, but it brought me to get live.com because I was pain, github 10 bucks a month for repository and making the money, and so once I brought to get lab comm for the pre repositories, which isn't nothing anymore. That's not a differentiator anymore, but was at the time.

C

I was like, then struggling with deploying it and I'm, not at the right computer, where I've got like my Heroku credentials on it right and so get lab, provides the ability also to like have like secrets. So I was like oh wait. I can orchestrate this whole thing with get lab.

C

Ci and I did orchestrate this whole pipeline with get live CI as a free user and get live.com like you, I didn't know, even though who I was then, but the like one thing: I needed was I needed to deploy to Heroku I needed like an API key right, and so here we can see. This is like using dpi, which is a Travis open-source tool to deploy to Heroku right as a provider Heroku you give it an API key it bundles it up all the way.

C

Heroku want something shipped it off to hurry, but this you can see it's just like an environmental variable right and so those are called CI CD variables. We used to call them secret variables. We don't anymore because you know by default they're passed as an environmental variable to your pipelines, so in a lot of ways, they're not secret in that way, but there are ways that you can protect them, and so here you can see.

C

I've got all of these variables, which I won't show on the video, but these are like my actual like variables that I need to like to deploy my app or run tests right and then so so these are stored and they're encrypted in get logs database, but right now actually I have poor hygiene on these and they would be because they're not protected, they would be sent to any branch.

C

So any developer, although it's just me and my friend developers- and this could you know echo this variable out right protected- is a flag that says only send this environmental variable when I'm running on a protected branch or tag right, and so that means like master, is typically protected. Right and so I might say only run this when on master that way, some bad actor or accidental actor. That would say echo of Oroku production.

C

You know key into the log that would have to go through a merger quest where I said yes, merge it right before it would actually run on master right. So that's where protected comes in really well, the thing we're adding seen is called mask right. That's the thing we've been working on, and so this is gonna say even if you do accidentally like again mostly, this is not.

C

This is mostly protecting against, like accidental secret leakage like if I accidentally, echo it or an error comes back right from dpi, where it's like this API keys invalid and it prints the API key for you right like if we see that, like just mask it as X's or whatever on in in the log. So that's what that's, what masks will do? Person is protected and then I can actually even be even more specific.

C

So we have, we haven't, talked about a concept of environments, but you can have different environments, you're, deploying to that's in the operations. Tab. I only have one here, one theoretically, I only have one right like production is my thing, and so I can say also only to pass this to QA right. Like so. Let's say there is a set of keys to deploy to QA and a set of keys of the point.

C

Production I can say both protect it and only send it to this environment right and so then it's like kind of doubly protected we're not doubly protected, but it's protected and scoped correctly.

C

That kind of makes sense it. We probably have those in get lab to like well see I'm, not see so, and then that setting is protected for only I, think maintainer right so I'm, not a maintainer of get lab community edition here.

C

So I don't even have the ability to do that right, like and so those variables are probably used throughout the get labs ce-1, but as a developer and not a maintainer, DFC I don't have the right to see those and then there's also a set of variables that get lab itself provides to you that are really valuable when you're trying to build some complex things, and so that's a lot of things like okay.

C

What's the Shah of this commit or what was the commit message here or what environment am I the point or what's the idea of the job? That's running or there's this very powerful thing called the CI job token. So this is a one-time token that we generate that lets. You authenticate against things like the get loud continue registry, so that if you're one I like push an image into our docker container industry, you don't have to like put your username and password in there.

C

You just say: hey use the CI job token it'll authenticate you to this projects registry and you can push in adjust right and so there's a lot of these Wow yeah, but they're just you know. They're just have been. You know the project name right. What namespace in my interest project? Where is the registry for this project? What version of gitlab is running right? This is really valuable.

C

We use this and some burgeoning stuff outside right, like like secure, publishes, docker images inline with gitlab versions, so we can say, pull the docker image for SAS of version 11.7, because this is get loud, 11.7 right, there's a lot of really advanced stuff. You can do with this, but comma-separated list of license features available. I get lab, usually what the user triggered this job right, and so that's what we saw sorry earlier in here.

C

Yeah, like the CI job name right, so it's like it says what script am I running right and it takes the CI job name, but then it also has this: like naps pack, our spec suite report path. That's a custom variable that we've set CI project. Making, though, is the is the name of this project at lab. Cr right, you can see that node index.

C

That's when you're doing this, that our spec terrorism right, we're, saying: okay, run the tests for this project index seven or whatever index we're in right, and so that's how you get to kind of it's super like pretty advanced use cases name.

A

Yeah check your thing to deal I think yeah.

C

Like I want to check out this boy, this one or I want to she had to do migration or whatever yeah, exactly very what you do complex things. Oh.

C

Yeah I want to deploy to environment specific. To this thing right so I've got the doctor of you. Domain I've got the environment that I'm deploying to right so that to deploy review apps. For this specific right like another example, a good example, one would be: are the website right which you've you've merged?

C

A

Yeah I think I think in some coming releases you and I are going to be working together on some release app stuff. Yes,.

C

There's some release up stuff. I want to do. That's, that's the really.

A

B

C

Let's try and do a to get you to be excited about.

A

C

No no I was just like John said to me, like we've got Sarah, she wants to dive in I was like well, I've got one.

B

This month to get up to speed and then there's.

C

What looks like have a release to like understand the code.

B

But I'm still like, where is.

A

This file exactly.

C

Like I, don't I mean I've checked out the code base before and I have lots of fun. So I have a lot of respect for our engineering teams. That's.

A

How you want to be consistent, at least once you figured out one tiny microcosm of it, then you can start being like well based on what I know. I can apply this to other microcosm when they're, not just like.

B

Oh, it's 303, oh yeah I've got another me I, don't know if you do I, don't.

A

But I think you do cuz I remembered you I.

C

Do I do it? Let me just let me just run this real, quick, it's it's with Kenny my boss, so it'll be okay, and this recording eventually made me say that just so yeah, so the review apps like here on get lab calm. When we make a change the website, we deploy a review app that you can click on right.

C

As you can see here, it does like the slug for this reference dot about source rocket, lab calm like deploy to that, basically right and in that I don't know if you've ever seen, one on a merger quest but like if I see like sort vacancies by outputting to charting library, there's some change, that josh is making pizzas charts messy and if I click view app.

C

It takes me to the changed app well, it doesn't work well, of course, in the demo because I'm demo, but the theory is so you can see it and then the thing you and I are gonna. Work on together is visual usability. Testing of that thing right.

A

So people can add notes and stuff exactly.

C

Alright I am gonna, run now and talk to Kenny, but thank you very much for this I'd love to post it to get that one filter, so you'll get the recording, I think or you may go. If you did it, the cloud I think it'll eventually get to drive, but I'll slack you about it all.

A

Right sounds good, so Kenny I said sorry for keeping you I will.

C