GitLab Developer Evangelism Team, 19 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Dutch GitLab Meetup: From Idea to Production: Using GitLab for your whole development lifecycle

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

A

First, uh thank you very much. Everyone for joining this workshop, we'll be talking about uh using gitlab from idea to production. That means from your for your entire development lifecycle. Right from you having an idea you want to work on right to getting your idea deployed on production will be looking at the different workflows you can use within gitlab and some of the processes that you can leverage on gitlab.

A

That will make your work and life much more easier. Yeah, like uh franciscus shared the other timer. I am amber cursed, dickhongo, I'm a developer, evangelist student program manager at gitlab before uh joining um before becoming the developer evangelism program manager.

A

I was support engineer at gitlab, I joined gitlab 2016 and in my previous role I was mainly supporting our enterprise users, helping them resolve issues, especially around kubernetes and other problems that he might have so in this uh session, we'll be looking at briefly about gitlab itself, I'm sure some of you might already be aware about the company and so on, and in my discussions with francisca she mentioned that I shouldn't come with any assumption about the attendees, knowing anything.

A

So I should try to be as explanatory and expressive as possible in sharing all the things I will share in this session, so I'll be going through what we call the gitlab flow of working within the gitlab workflow and also do a demo of what we call identity production using one of gitlab's prominent feature called moto devops, and I'm aware that almost all of us already have our google cloud account setup.

A

So in the demo we'll be seeing how you can with just a push or even for a freshly a fresh project, you can enable a kubernetes cluster and deploy your code as easy as possible. So we will be seeing that in the demo.

A

Okay, now briefly, gitlab is a fully remote company. uh I think we are currently close to one thousand five or four hundred employees working in different parts of the world uh I joined from nigeria, I'm originally from nigeria, so then moved to the netherlands after uh two years, working with gitlab- and uh I I joined when the company was still around a hundred people- that's in 2016 and now I've seen the company grow to 1300 and we are scattered all over the globe.

A

We only meet physically, maybe during certain offsides or when the company as a whole itself organizes. We were supposed to meet in prague last year, but kovid happened uh and hopefully we'll meet physically soon, maybe in the netherlands, so that we can meet all of you. Wonderful people in our community yeah. So basically, gitlab are pretty light, says what I would call open core. We have open source version which anyone can contribute to and see how the code works.

A

Then we have the enterprise version that we provide to companies, enterprises that for them to install in the on-premise within their own private network and those who don't necessarily want to maintain their own infrastructure can use our sas version, which is on gitlab.com. So those are the two versions.

A

If you, even if you want to install it on premise, you can decide to use the open call version, which is you can freely install and view as you will use as you wish, or you buy the enterprise license, which gives you much more functionality and one of the beautiful things about gitlab since inception is on between the second of every month.

A

There are new versions, we always call it the best release ever because every month, new features, new awesome contributions from the community go out for everyone to benefit from and um gitlab promotes uh what we call a single source of truth. Instead of as a company to have different tools, for you have a different tool to write your code, you have a different tool to manage your issues, to submit pull requests.

A

Mad requests a different tool to manage your artifacts, a different tool to at almost every stage of your software development, instead of youtube different to gitlab, serves as a complete devops platform from right from you thinking of an idea discussing it with your teammates straight to security and monitoring, real-time monitoring of your application.

A

Everything can be done within gitlab, reducing the need for you to manage different slas around and also having to incur much cost in maintaining all the different components uh that it's making your infrastructure work, and one thing that most applications tend to struggle with is the integration between all of them so with gitlab, everything is in one place and uh you can just install it or use the size version and life becomes easy for you.

A

We also support continuous everything. Now we have continuous integration, continuous delivery, continuous deployment, everything is inbuilt into into gitlab now and in terms of features.

A

Gitlab has grown to contain almost anything you need within different stages of development and we've broken the stages into manage plan, create verify package, secure, release, configure, monitor and protect, we'll be seeing some of this in uh you know. In our demo, but a couple of them are much more advanced and things that you might necessarily, you might only need in uh an enterprise environment, but the part I want to mention uh is the verified stage and the package stage.

A

If you look at the very first teacher, you will notice right from uh we talk about continuous integration. That means anytime. You need to test your code. You need to make sure area all the pushes that we had done by your team or you yourself is tested, and you have adequate test storage everything necessary down to running uh security, analysis on your code or secrecy detection.

A

Quite a lot of issues that happen lately is where you see employees mistakenly, pushing secrets to uh they get repository or in one way or the other passwords get leaked out or something gitlab helps you to be able to identify when someone mistakenly pushes a secret. So that way you are able to have much more secure and robust environment I'll, be sharing this slide so that you can subsequently at the end, so that you can be able to go through every part of this.

A

Subsequently, then I'll be moving to the gitlab flow, now git itself, it's a very robust platform and other uh version control systems. We have like spn, like maya mercuria. Everyone has the flow that people can use to be able to work, and it also comes with its own, but git's method of working is kind of complicated.

A

Then github came up with the idea of branching working in a feature driven environment that you have uh your main branch within git, which is mostly master, and most people are renaming it to me now then, anytime, you want to add the new features or make modifications instead of working on the master.

A

You create a branch out of your kitchen repository. Do all the necessary work you want to do then match back into the master. That's what most people have adopted now and github itself has promoted.

A

Gitlab takes it a bit further because that flow that github introduced didn't take into consideration. The fact that it's not necessarily what is in your master that always goes straight to your production or whichever environment you have. So. Instead, we introduced a multi-branch workflow, where you can have a specific branch that anything that goes into that branch like production.

A

Anything going into that branch is what is deployed to your production, so you might end up having a different set of code or different set of work in your master branch or any other feature that you have. But anytime, you are ready to deploy or you have a production ready code. You can simply merge it into the production branch and it gets deployed to production.

A

Now, then, what? If you have different environments? You have staging? You have probably have testing you. It might have canary, also different stages completely down to production, so gitlab, one of the flow we are promoting is where you use branches for different environments that you have. It might be branch it might be using tags so that any time code is written and probably you want to test you merge what is in your master into a pre-production branch.

A

It might be staging, let's call it staging here, so you do all the necessary testing and everything you need to do uh on the code to make sure. Okay, we've tested it, everything is fine, then you match from uh the staging branch to the production branch.

A

So that way you are able to isolate different code in different environments and it's easier for you to revert between the different branches or even cherry pick, setting certain features, probably uh there's a bunch of commits that have been done within a particular branch and you simply want just a commit to fix a bug or anything.

A

You can cherry pick from one branch to the other, which makes things a lot easier and less complicated. Then you can decide to do release.

A

Release base flow, where you can tag certain parts of your master branch for certain releases. Probably you, you are not pushing to produce to production. You are releasing your software as a specific version, so you can have the from the same code base be able to tag certain parts of your code or certain branches, as this is version two stable. This is version three. This is version four, and probably there are some bugs that were introduced earlier on in the code, and it affects defense branches.

A

You can change peak within all of them, which makes work working with gita and github itself, much more easier.

A

Now at gitlab there is uh not official, but there are kind of uh 11 rules of the gitlab fluid that our ceo seed wrote about once the first one is when working with kids, you don't commit stress to master, always use the future branch for new things or new features you want to introduce master becomes like the single source of truth that it's at the end of the day that things goes back to master and you should make sure and no time should any push or any commit that is done just be accepted.

A

I everything should be. Fine, testing should be done on almost every commit, not just the ones that are going to master, but everyone on any branch that is created then code reviews before margin to master. Yes, sometimes you can trust it's not just enough to just trust code written by anyone. It is very important that anytime, a new feature is being introduced or code.

A

A team member has created a commit to make sure it is probably reviewed by a teammate or someone else. That's a senior to make sure the code is okay and intact, and you you can also do deployments automatically either based on a branch or a tag or if you are not doing automatic deployments, you can do delivery, also by maybe specific branches or tags.

A

Then uh it should be that you set the tags, your team members or the uh or you as a company should set the tags for your features directly. Not the ci, automatically tagging and releases are advisable to be based as tags, not specific branches. So that way you can be able to tag specific sections of your code without creating the branches completely, and it is advisable that every body starts from the master.

A

It shouldn't be that every anyone that's just coming near to the team can just pick off any branch and just walk off the branch, but start from master work on your code or any feature that you want to do once you are done, you merge back into master master cells.

A

Like I said earlier, the single source of truth. Now it's also advisable that you fix bugs in your master. First then back to the branches. So that would be, since master is your single source of truth? You can easily everyone within the team can uh have an updated code piece, and most of us are fond of probably just writing cryptic or commit messages that we just want to just commit and move on. But it's more every time, it's advisable that whenever we are writing commit message, you should reflect the interest.

A

The reason why that commit is being done so that when the git history or the code base is really, we can know the reason why this change was introduced instead of having to go to the code itself to understand what exactly is going on now, that's uh about gitla gitzflow using githlab, then next one is the gitzlab workflow itself.

A

Gitlab, like I said earlier, is a system that encompasses the entire development life cycle right from you coming up with an idea of what you want to do to completing this, to releasing it to your users or deploy into production. It starts from the gitlab repository itself. You write your code. You commit then to gitlab ci gitlab ci. Is that stage where every code you've you write or you've written is tested?

A

Now it's tested not just for against your test streets, but the different test requirements, for example, license compliance. It might be that your company, due to certain reasons or due to some legal reasons, don't work with certain line census, maybe mit or any other licenses out there. So it can be implemented that your company don't want any library or dependencies that are using certain licenses within github ci or your test stage. You can be able to identify which test licenses have been introduced and move them out now.

A

You can also implement it in such a way that you want to be able to review these changes that made before it actually gets to probably staging or production itself. Here's what we call review app review app gives you like a preview of your application itself, so that you can see the change that has been made before you as a superior or as someone that has much permissions clicks on the match button to uh to access the changes that have been introduced.

A

Now we also have what we call the security dashboard, after all, the for your security team or uh quality assurance team. After all, the security scanning analysis and everything that has been done, we provide a dashboard where your security team can see. Okay, what are the different infringements or the mystics, or these secrets uh that were exposed and so on within this particular project, and issues can be created that okay, this needs to be fixed, or this issue has been detected.

A

That way, an issue can easily be identified. A numeric request can be created to fix it. It goes to trusting again if it's accepted it gets merged and so on then, at this stage, gitlab cd can either be continuous delivery or continuous deployment, where you can either set it that anytime, all tests have been accepted.

A

The code should just be deployed automatically, which is continuous deployment, or you need to manually intervene that okay, you still want to see it, even if everything is fine before a deployment is done. You manually, deploy yourself. This is like a 30 000 feet overview of the workflow itself. Now.

A

From the developer perspective or a team member perspective, it starts with an issue. Okay. Maybe there is a new uh request for these are websites. We need to add this image. It starts with an issue.

A

An issue has been created. Okay, this is what needs to be done, some platform called tickets, and so on this. This is what needs to be done. These are the requirements. A lot of discussions happens, comments and so on now, once a decision has been made has been made. Image request is created, github calls it pull requests in the magic quest. The developer writes his code.

A

Does everything then push all the changes that have been made to the code can be reviewed within the match? Requests. The match. Request can also be linked to the issue that okay, this match request actually resolves or closes this particular issue that has been raised now, once all the changes has been accepted. The next thing is for ci to run or uh test run, all the necessary test. Suites to check the code has been written to make sure it meets compliance, and everything is fine.

A

Now, once that is done, like I explained earlier, there's a review app where we can see uh a preview of the application, then, within the review app. The essence of the review app is for teammates or team members or you as a person to be able to see what the application looks like and confirm. Is this change that has been implemented exactly what was expected or there are still some changes that need to be done. Different reviews happening once that is completed and all every change has been made has been approved.

A

The code is matched and the cd continuous delivery of continuous deployment pipeline starts if it is continuous deployment once the match request has been approved that match it automatically goes and deploys to production.

A

At this stage now, you can also decide to implement monitoring uh at the last stage, where gitlab can be monitoring your application, probably for load of for performance and other things, and if there are certain issues or certain problems that arises, an issue can be introduced. That is why the line goes back here again when a new problem is discovered. A new issue is created straight to march requests all over again and the circle continues.

A

So I don't know if anyone has any question at this stage before I proceed.

A

A

Okay, awesome now. The next thing I want to talk about is auto develops now, most times almost all of us are have been are in this scenario, where you have a new project, you you're writing, probably php code. You have to write your test suites, you have to write your ci cd pipeline, specify exactly what needs to be done. What security checks need to be done, everything you have to make sure everything is prepared beforehand, but gitlab has this feature called ultra devops? What other devops does is?

A

We've created some predefined templates of jobs using our gitlab ci scripts to automatically be uh just help. You run your application through all the different life cycle, right from testing your code to building your code. If it needs to be built, probably into a docker image, to reviewing it to doing dynamic analysis, testing, to publishing into production, to testing for performance and everything necessary, you don't need to set up any ci. You don't need to do anything. All you need to do is just to push your code and it does everything now auto.

A

Devops is basically just uh a couple of uh templates. Gitlab cicd templates that have already been prepared in such a way that they are uh language agnostic, not tied to a particular language and it uses cloud native, build packs. Build packs are.

A

Libraries that can analyze your code to understand what language is it written in what are the necessary test, suites that are needed to test the application and what is needed to build the application, so we use build packs within autodevops to test your application to identify first in what language is he written? Okay, it's in ruby. Okay, definitely, ruby requires a atm file; it requires a rig file and so on it looks for them. It checks. If there is a tests that needs to be run against your code, it runs the test.

A

Then it builds your application. If it's ruby, it knows where the necessary entry points or the everything that's just you to build. Your application builds it before starting a deployment.

A

So most of our demo will revolve around gitlab auto device to show you how idea to production actually works. Now. Autodevops includes all of this automatically. That means it automatically builds your application to automatically test code quality, static, application, security, testing.

A

Every single thing is done automatically how it does. This is like I mentioned earlier, the templates we've created. We created templates for each of this stage and it runs a suite of commands against your code based on the type of code you've written and the one beautiful thing about ultra devops is the templates are provided publicly for you to be able to customize to your taste, because, no matter how agnostic a bunch of scripts are, they might not necessarily meet or suit your environment.

A

So you can easily uh pick up any parts of the auto level script edit it to your taste and do whatever it is. You like I'll, be showing you what some of these scripts looks like when we get to demo now?

A

Oh, I think the next thing is uh first proceed to a demo before the demo. I would like to ask again if any one of us have any.

A

Questions. Okay, sorry, can you hear me? I just saw a sign as if I lost network edit.

B

Just for a sec, your your sound was away.

A

Oh okay, probably it was my network connection. Google just showed me. I was offline for a bit yeah. So basically I was asking if anyone has any question based on everything I've said here before we get to our demo and in the demo, I'm looking at scenario where we will have a couple of labs and we'll do it interactively I'll. Do certain parts of it and uh franciscan mentioned last time. We wanted to be like a workshop format where, uh once I do them, you all can also experiment do it.

A

They will proceed to each of the stage. That was why I made my intro short so that we can have more time to work through different uh activities and the things we will do is we integrate our account with uh google cloud accounts with uh gitlab.

A

This is so we can enable github to automatically create communities clusters without we having to do it ourselves. Then we install the different uh components that gitlab needs to be able to deploy applications to the cluster and we'll be using this uh sample. Ruby application to see how you can use auto devops to deploy an application straight to the kubernetes cluster. Without you having to do any configuration, then uh the last thing we'll be seeing is make edits to the application and see how the deployment is done using the gitlab workflow itself.

A

So any question.

A

Let me show you the gitlab ultra devops.

A

A

Okay, it seems as if uh we are all on the same page. I can simply proceed with the rest of the presentation, but let me get the autodevops template first and show you what it looks like before we proceed.

A

Now the gitlab ci environment works with.

A

Emo files, mostly it comes as a dot gitlab dash, ci dot, yaml file and instead of you having to create yours. What autodevops does is it automatically includes this particular one into your environment, and it already has almost everything you need to deploy your application see for each of the stage.

A

It includes templates for each of the stitch that you require within your job and, if you probably need to tweak or to edit any of these templates any stage you can clone this particular project or come here and copy any of the ci file and make changes to it. You see, let's say, for example, you want to change the.

A

A

Let's say, for example, you want to change how the ci builds your application. You can simply come to this particular ci file. Make changes. Probably you have some custom way. You want your own application to be built. You make the changes here and it's good stress to work as you please.

A

Okay, back to our presentation, we have a simple ruby application that you can use to test uh auto devops to see how it works, and it has it's basically a docker file with uh a small ruby application that just says hello about the hello. Almost all of us are, I guess all of us are used to uh what I'll be doing here is. Are we starting a new project? I don't have it already.

A

Then I will now integrate that project with my google cloud account install the kubernetes cluster install all the necessary applications that is needed, then we'll deploy the application now. The first.

C

Thing I'll be.

A

Doing is to take a copy of this project, to my account.

A

Now this is mine in the menu here now we have the operations menu. Then kubernetes, where I think kubernetes is here, because it's much more easier to integrate the communities cluster with gitlab and run your application.

A

Now there are two options we have. You can either integrate it with uh a cluster that you have a certificate or for if you are adding it yourself or allow you that to create it or you use a gitlab agent to that is already within a cluster to do the integration.

A

Now, let's use the first one, which would be easier for us. There are two options you can either use it with an eks cluster as an eks cluster, or it will go gku I'll, be using a gke in this instance. It will ask me for my account. I have series of accounts here.

A

Now automatically gitlab will. uh I hope my phone size is not too small. Let me increase my font size.

A

Yeah now here I'll be specifying a name. I mean image.

A

Just name it this way, then you can specify an environment scope. Maybe you want multiple kubernetes clusters to be created, one each for one for staging one for production or one for dev.

A

However, the environment is uh is, but here I want almost all my uh all of my environments to use the same cluster so I'll be leaving the wildcat there. Then, if you have uh server projects within uh google cloud accounts you can specify which one you should use.

A

Then, if you do not set up your if you've not used google cloud before you can use the link here, I think it's attached to. I don't know if the campaign is still running where you can get some credits towards your google account and you can start with the free trial. If not, I doubt if any of us have a free trial, since probably most of you will have tried google cloud before, but you can uh create an account if you don't have one already, then the zone, I think amsterdam is eu west form.

A

Let me look for amsterdam.

A

Rest for here just select, hey, then how.

C

A

You want to create in your kubernetes cluster. We all know that kubernetes, um probably for those that are not aware of what kubernetes is kubernetes, is simply a distributed environment where you can run your applications, it's composed of nodes, which are machines. Here.

A

We can specify how many nodes we want to use within our cluster, and I will leave it at defaults. Three nodes, then just leave the standard here uh we are now using anthony's is a feature that google is providing we'll be using that then here.

C

A

Specify for gitlab to manage the cluster that means gitlab will create uh all the namespaces. All the service accounts, all everything that is necessary for your applications to run within the crosstalk. So this is also useful if uh you want. You have a group within your company, a gitlab group, and you have several projects and probably due to cost or whatever reason you don't want to create a lot of clusters.

A

You want to have a single cluster for all of your projects, so gitlab you can enable github to manage the cluster and it creates namespaces like a specific section, restricted section for each of your projects to run or to do all your deployments without affecting any other uh project.

A

Now also, you can leave this option enabled to create for your production environment. There isn't a single cluster for your production environment for staging and any other one. A namespace is created for each then I'll click on create so gitlab connects to.

A

Google cloud the api to create organizational kubernetes cluster we force is any of us trying this or do we have any questions at this stage.

A

Any challenges in doing this, or am I too fast.

D

I guess it's really really quite clear.

A

Okay, okay, thank you yeah. This would probably take a few uh a few minutes. uh So uh let me take you to uh the gitlab templates here to to see some of the ways uh we've already created. That can make it much more easier for you to be able to use auto devops. Now. One of the ways I was explaining earlier is within our build templates. Here we use cloud-native build packs. Now I think this is the website build buildbox.I.

A

If I'm correct, yes, so and basically what this does is. uh Basically, it uses build boxes, uh uh phenomenal oil technology- that's heroku, started with where they don't want to worry about how you build your application or how you test your application. You should just write your code, you push them and everything should be deployed automatically, so they have a suite of libraries that can enable you to test a language.

A

Maybe you have a team environment where people write different services or a service, oriented architecture and people are free to use different languages to write their applications, but you want to have like a central place where all these deployments are done. So it's going to be hectic for you to have to provide the different, tooling and different resources for each team to be able to work with it, but when using build packs, vmpass can help automatically identify in which language a particular project is written.

A

If it has any test that needs to be run, it does it and it can help you build images of your applications and uh and deploy. Let me check on this yup. It has been created now. This has automatically created the gke cluster for us, and here we can still specify what environment scope we wanted it to to use, and we here, if we have a costume domain, probably you have a domain you've created for your company or you just want to specify it.

A

This is necessary when your application has been deployed so that you can have and also for review apps. You want to see what your application looks like before you push it out there to production, it's necessary for you to have a base domain.

A

Now we also provide an environment here for you to monitor the health of your cluster and see how things are running. It's not any move yet because I have not installed any uh monitoring uh application on it like prometheus.

A

Then we provide you a section here where you can install things like ingress, which handles all the uh traffic or routing within your cluster, a search manager for ssl certificates, prometheus for monitoring and also a gitlab runner.

A

Gitlab itself, if you are using gitlab.com, gives you certain minutes for you to use to run your ci. So if you've run out of it or you just want to have your own runner private runner for it that you want to use to run all the uh jobs for you, this particular project or your group, you can install your runner also in your cluster, the same cluster.

A

Now, let's start by first installing our ingress.

A

You click install here to take a few seconds. Then it's when you have the ip generated by ingress provided by google automatically. Now you can either go to uh your dns provider. It might be google dns itself or your xerox, router 53, or maybe I using a free dns service like nip, which I'll be using here, just to make things a bit easier for us nip.

A

So you the ip address, that's provided for you here is what you will use to go and create the area called, preferably a wild card uh record, to make it easier for gitlab to automatically create subdomains for each of the defendants is doing.

A

um While this is running.

A

So here there are, it's still a couple of other things that you can deploy to your cluster. We have uh crossplane. I have not really looked into crosstalk, but I know it's probably around the microservices. We have jupiter or we have k native. So we have elastic stack. You can install it directly from here. We have fluent d and a lot of other things. So let's look up. Okay, yep. It has already installed ingress in the cluster and we have an endpoint.

A

So let me copy here and come to our details: dots nip, dot, io, then save. Then the next thing I'll proceed to installing the other components that I need like. uh I need a search manager to create ssl certificate for my services.

A

I need prometheus to do monitoring because I want at the end once my application is deployed. I want to see how my application is uh performing. All the different metrics have been recorded around to see all of them, uh then I'll also install the run on.

A

I will not be using the github run as itself, then I think that's basically I don't need everything else here. Basically, but if your application needs it, you can just install them straight from here. So mostly uh one caveat here is: when you run all of these things, you can still go into the cluster, probably if you are familiar with kubernetes itself, to probably make changes to, or for one reason or the other, you want to tweak or make changes to how some of these things are behaving in your cluster.

A

You can log into your cluster directly and make changes to them or install, or that is to use the cluster for other things. Since.

A

The cluster is within your account and you can always login to administrate yep, I like taking state by stage and not all at the same time,.

A

Well, it's running: how is everybody managing with the partial lockdown in the country? I've been staying home for more than 48 72 hours at a time since I came back to the.

D

D

B

Everyone on me.

B

Is everything clear, is it possible to follow along or or are we missing something?

B

No, it's! Okay, okay, just just a mute.

C

Okay, I was uh wondering abu bakr, I missed it. How do you link? uh Does the kubernetes cluster spin up on the gitlab account and then gitlab bills you for it? Or do you fill in an api key for your own google uh uh cloud platform account so it spins up on your account.

A

Okay, yeah, I think it's probably because I've been logged in already. That was why I didn't see the stage there that particular state where I clicked on uh google gke.

A

It asked me to log in- and you saw me selecting the probably noticed me selecting a google account to use my own google cloud accounts has already been set, so I have to select a project, a google cloud project so that google cloud project is already existing.

A

If you don't gitlab will ask you to go and probably create a project. Add your billing information and everything so that google cloud can build you. So we are just serving as an interface to make it easier for you to create all these things. But google cloud builds you for anything. You do on the google cloud side.

C

Yeah, okay, so so, basically you hook up your own account to a gitlab and they will use the ap api key to create all the resources needed.

A

Yes, so the last thing I'm installing here is the gitlab runner.

A

So almost it's almost the same process. If you want to use eks, uh it's just that you can't log in. uh I just logged in to provide uh gitlab with my with access to my google cloud account for eks. You have to go and create some rules and permissions and other things within your amazon.

A

Aws accounts, feed them feed all the credentials and all the other details into gitlab as necessary. Once you've done that gitlab now have access to be able to create the cluster yeah. My runner has been installed. I think everything I need here has been set. The next thing is to go back to my project.

A

Nothing is still happening. My ci is not running. It means I need to go into settings then ci, cd and here enable auto device by default. It's not enabled because we don't want suddenly for everybody's projections, just started running and crashing the around us. So that's why it's disabled by default. Once you enabled you now select the strategy you want for the deployment, should it be continuous deployment street production or you want to time it incremental rollout or you want to do continuous delivery.

A

There should be a manual process to actually do the deployment so for this for our use case here I'll leave it at the default, which is continuous uh deployment production. Once I click save here, gitlab will now automatically start the initial pipeline for us and you'll notice. It started with series of jobs. So if I click on running.

A

Now it starts first by attempting to build my application, then, in parallel it's checking a lot of other things: the code quality, it's running, license scanning, it's running tests against my application. So it's also using.

A

uh Let me see, is he running? Is he using my runner, the one I created uh yep yeah? This looks like it's running uh it's using the run. I access to install so I'm showing you this job blog, to show an example of where it uses cloud native, build packs to actually build our application. Let me see, can I see where it does it here.

A

Download, okay, it's actually building.

C

A

Yeah, let me go back a bit and check.

A

A

Yeah, I think it's automatically detected it's a ruby application and built it without necessarily going to the effort, but basically it has built the application at this stage, then it's running code quality, it's scanning it for any it's scanning. The container builds at this stage here for any security, vulnerability or anything, and the one I like most in this stage is the secret detection. Here, probably when I'm making an edit to it, I will try to introduce a vulnerability and see so that we can see how it handles that particular scenario.

A

Maybe by you putting a password within your your code or you forgot, and you pushed a dot emv file, the secret detection state will automatically check and uh let you know- and once it's done with the code quality and license scanning here it goes to uh setting up the dust environment in the review stage. We have the build test, review dust, production and performance up to the cleanup stage.

A

Now, while this is running, we can come to our operations here and see the different metrics of things that are happening within the kubernetes cluster that we created.

A

Okay, I think the metrics are not live yet. uh Let's go back and see the pipeline and wait for deployment a bit to to be live. Let's see, what could quality is checking okay,.

A

Sometimes you check my skype. Sorry for.

D

A

It's my family calling.

B

Not now mom, I'm busy.

A

Yeah, let me check which of the pipeline. It's.

A

It's almost done so any other question before these two.

A

Two pipelines complete license scanning and basically checks any license that we've provided or any dependency that we have for probably some license that we or you as a company, don't currently support.

C

Yeah, can you go into depth on on one of the, um for example, to see what you can configure? Are they all in ruby, like you mentioned, or.

A

No, the it's just my project that is ruby, my code, but you can specify uh with the configuration the kind of.

D

A

Sorry, let me get for for you more information, for example like uh license scanning.

A

I don't remember of head how the configuration is done well,.

A

A

The off the top of my head, I would say you if I remember vividly you provide a like a small configuration of the kind of licenses you support, then, when the scanning is happening it uses that, but I don't currently have more information about it off the top of my head, but I can share this link with you for you to probably review it in more detail.

C

Okay, is there also an overview of all these jobs? You can run because, for example, we often use sonar cube for a code analysis scanning. Could you plug that in there as well or.

A

I think there's a certain security scans that can be run. Most of them are provided in. Let me grab the overview of our entire security uh scanning that that we do yeah gitlab secure here. So there are certain instances where you can configure what kind of test you should run and what it should use depending on on what stage, but we've tried as much as possible to cover as much language as possible. Let's say in the testing here we support several languages and also different tools that can be useful for these comings.

A

Hope that answers your question.

C

Yeah one more sorry for that the the scans are are pretty clear. I guess like the type of scans you can do and what the uh languages are supported are these all uh the scanning tools are: are those things uh uh you guys gitlab created or are they a standalone open source tooling that you guys integrate into gitlab.

A

They are standalone open source tooling, that we integrated.

C

Oh yeah, I see clear and yeah for container scanning. I think yeah.

A

Yeah, if you, uh as you, can see here uh for the different languages, most of the tools are tools that are publicly available either on github or certain projects online.

C

Okay, thanks yeah, that that was uh definitely answering my question.

A

Okay, I I've also dropped the link in chat to this pages, for you all to review at your own leisure time,.

A

Okay: okay, the uv stage is complete. It's doing the dynamic application uh security tests also, and I think, at a stage what it's uh basically testing, instead of just the code itself, it also needs to test how it responds to different scenarios uh of dynamic requests or other attacks like csrf and other things.

A

A

Let me see if it's done, it's almost done with that. Also, while it's running we can ask any other questions. We have.

A

Is anyone trying it now or we are mostly just um following now, as you can see here, it's running several requests through proxies to the application to check for different things.

A

And if you remember the other time, I created uh an nip domain name with my ip address, see it's dynamically, creating certain subdomains within the application to run some of those tests, it's scanning for headers scanning for a lot of things to make sure the application is fully secure.

A

uh Let me reload page what job is currently running: yep, it's not deploying to production. Finally, all test has been run. None needs to push our application to.

A

D

So in the beginning you showed a yaml file, yeah blueprint for the pipeline. I understood yeah, are you always working in the user interface or is it also.

A

D

To work in the email directly.

A

D

If you have any.

A

Customization that you want to introduce like uh um let's say for okay, do you mean if you want to customize the pipeline for yourself the order devops for yourself or just working generally.

D

No, if you want to adjust pipeline for your project,.

A

Okay, yeah, you can clone your project, so I think in one live environment. I've used it before I close the whole yaml file put them in a project that is central in the instance gitlab instance. Then anytime, I need it. I simply include it in the ci file for the project.

A

I want to use them for so that multiple projects can benefit from this uh yaml file that I've customized and once you've done that once you don't like, since it's general to your application, you don't necessarily need to go and edit it every time, but entirely depends on your use case.

D

So you could do it both ways. You can just import the demo or copy it from another project and then the user interface is updated.

A

D

As a result of it, and then you can either do it in the ml file directly or yeah.

A

User interface.

D

A

Okay, it's doing all the necessary communities deployments, creating a service stateful set running anything necessary. If I have production a database. Okay, it's creating an ingress service here and uh yep application should be accessible. It has deployed my application and it's telling me that my application should be accessible at this url.

A

So if I open the url yep, we have the hello world here.

A

We have the hello world here, so our application is live running then, uh let's see which stage of the pipeline okay, everything is, has run now. What we, the next thing we can check is our security dashboard. Was there any vulnerability that was uh that was seen? None then we can also go to operation and see metrics with different metrics from our cluster.

D

A

As you can see here, we can see memory, usage, core usage and the different things from the cluster. All these are being taken live from the kubernetes.

D

A

Let's now go to back to our code, make some change. Maybe let's use the gitlab workflow itself. Now someone comes and say: okay, I have an issue. The.

A

Text of website should be changed to dodge. Now I assign it to myself. I put any label if it is necessary and.

A

Change the language of the website, for example, submit an issue and probably developer, sees this okay, I can work on this. It simply goes, creates a match request.

A

New match request: we don't uh we don't have any uh branches yet. So I can come to the repository here from the master, sorry branches.

A

I want to create a new branch from master and I name it. Language change.

A

Create branch then, from the new branch I want to create a match, request, make it a draft, because I don't have any change yet and I can say it closes.

A

A

Now I'm trying to reference the issue that I created here so that when I merge.

A

Now so that when I merge this particular uh project, this particular match request, it should automatically close my issue now.

A

A

Then, within gitlab we have what we call the web ide. I can automatically launch from this match request that I want to make a change. Let's review the rig file, what does it do it does?

A

It has a test file and in the appsec, it's basically testing, true against true for the purpose of this demo and in our server file in our gem file. What do we have just mini test and rake? Everything is fine, then, uh for the purpose of this demo, I want to create a dot emv file.

A

And just add some things to its testicle to test, for example, then, in my.

A

Yeah, let me add, commit to language change, add dot env file.

A

Then in my server file here I want to change what it says to the outside world, so the body should just be.

A

Is my spelling correct.

C

You need an extra a.

A

C

The last one is I'll: let mao.

C

A

C

A

Okay, no problem. Thank you.

B

A

B

A

We've changed the text, it says and uh changed greeting to dutch. Then we commit this. Also. Now our pipeline is running. Let's see, okay, we have two pipelines. So let me go to projects here and see. Cicd pipelines.

A

I should have committed only once, but no problem. Let me stop this one from running and.

A

Yeah, let me stop this one also from running so that we can see the tests that this and I waste too much time now my test field right. This one fail.

A

A

This was what I wanted to show earlier. You see here it's using build to automatically detect the uh the code, and here it's able to detect that it's a ruby application and it downloads all the necessary dependencies and other things to use to test our application and it tested here and it found.

A

Test test yep, it found something in my dot emv file.

A

Now let me go back the secret, detecting.

A

Thing you didn't maybe felt my test test is not a secret, so let me go and uh fix it.

A

A

Repository branches, language.

A

Change, basically, let me remove the dot energy file, so I don't waste too much time. uh Remove that here the file don't create a new mod request. We already have one commit and since the way gitlab ci runs- and these are initial pipeline here- because everything in this stage has to pass for it to go to review. This will no longer proceed. So I think, to see what time I would just cancel the rest of it, since you already have one that has failed.

A

The rest will not proceed because everything the review depends on everything here to pass. Now, let me go to pipeline for the new one. We have.

A

We'll notice the difference here here we don't have production it's because we are working in a different branch and not master. So what it would do it knows that. Okay, we are testing if we are creating a feature and testing it out and if everything goes fine we will now match to master. So it's not that's why the production uh stage is not here.

A

We don't take all the time any question.

A

I think we are almost at the end of our time, we're supposed to stop 7 to write franciscan. Am I right.

B

Yeah something like that, if there are not too many questions, we can squeeze up the time a little.

A

Okay, no problem: I think this is the only editor moving before merging to master.

A

So while it's running do we have any.

A

A

Okay, the test passed this time secret detection passed. Everything is passing.

A

It's running a total of 12 jobs and even when you are working within your own environment or using your own uh ci scripts are not on twitter verbs. It's always necessary, like we mentioned earlier, for every change for everything, because you can take a single change to break your code or to introduce something that will be disastrous to your code. Tests are always necessary for every commit that comes before it goes to master.

A

And one of the benefits of the security testing that we have for auto devops it's, it tries as much as possible to run all the different sorts of tests uh at the ends. I think I've made a mystic funds where I was writing a ruby application and uh I wouldn't use emv file uh in uh gitlab itself, but I had an a.t.m.e file to test my local system and at one point or the other, I think I forgot and I pushed I added it to my dot.

A

It get it no, but I don't know why it still went probably maybe at a particular stage I had already committed before doing. Git ignore uh so the whole emv file went live and I had to figure out exactly how to clean up the project, because the project was public so that other people wouldn't ha wouldn't have asked us to discrete.

A

D

Of it, I had to go.

A

Look for all the icons that I have the secrets there go change or recycle. The tokens and secrets to be safe, then fully implemented secret detection on the project, so that any time any secrets at all since I'll be using gitlab ci cd5 variables, which is, if you don't have a dedicated, uh maybe like votes, hashicorp, vault server, to manage your secrets or any other solution. You can add your secrets in a gitlab ci variable here and it will be provided to your ci cd environment whenever it wants to do anyone.

A

Probably I once worked on a project where, after all, the necessary tests are done on a php project. If he wants to deploy it to a server, it uses scp to just move them, but we need to configure ssh. We need to configure a lot of other things to do that, all the secrets that are needed for the ci to run properly. You can add them as ci variables here.

A

Let me go and check if our job on fine come on just license remaining.

A

Come on come on come on now. One thing that will happen here at this stage is we'll see how review apps function review apps. Is you being able to see sorry? Can you hear me.

D

Now we can hear you.

A

Okay, I don't know it's. Google appstam notified me now that I went offline. Thank you. I don't know what is wrong with my internet. Probably zigo is having one or two issues.

D

I wanted to ask because for this demo we now use the editor in gitlab. Yes, so there you have the checkbox to create the merge request yeah. But uh how will you instruct to to get the mercs request if you are using just regular get from from your own command line.

A

Yeah, um once you do your commit and you don't get push to gitlab and gitlab notices that you are working in a branch or you've pushed a new branch, we will automatically in your command line with only a link for you to click on to create a match request automatically.

A

So once you click on the link, it takes you to the comparison page. Okay, you have this new branch and you are. You want to merge to master all whichever branch, then you click on continue and it presents you with the magiquest page, where you can specify the title description and every other thing. So you will see a link in your command line to click on to automatically create a match.

D

A

Okay, uh this is done. Let's see what stage is running at this time now. One thing about uh having a match request is where's. My much request.

A

Oh, I think I did something wrong. Let me see uh I was creating a match request to the upstream projects, not my master. I think I missed that. Let me see no.

A

Yeah, I was creating a match request to my upstream, instead of instead of just my my own branch yeah. Well, it's deploying here this is going to waste our time. Let me let me close this close my face. Sorry for that.

A

And go back to the main project.

A

Should have looked well before I just submitted the magic quest, so it assumed probably I wanted to update my upstream. uh Let me create a new match request.

A

Language change, compare yep close this.

A

Okay, no wonder it was not showing me and yeah.

A

Oh, I think I'm making the same mistake again.

A

Just to mine yep now I want I'm asking this to close this.

A

Let me assign it to myself and submit.

A

Sorry for the confusion, okay, my pipeline is still running actually, uh since it's still on the same uh source branch. That's the uh that the pipeline started on the same language change branch. So if you notice something here in my mi, this is my match. Request from within the match request, I can see different uh tests and checks that it's doing on my application now here, you'll notice that it has deployed this to review language. This is an environment. It's created automatically for my review and I can view my application first before actually deploying.

A

As you can see here, you see it says ahoy alabama. Now, if you look at the url, it created.

A

A special subdomain just for my review. Let me go back to my mr and explain further at this stage at the review stage of my application, it automatically deployed my application to a particular port. I guess just for me to review and created the necessary services and url just for me to be able to review the application and see what the application looks like not to production this time. So I've.

D

A

Everything is fine and the whole thing is okay. It's going to still do the dynamic application security testing on my review to make sure the whole changes I've made.

A

We've not introduced anything that will make up the application break. So from this same place, I can see all the necessary reports of other reports like to manage licenses to view full reports or to see any potential vulnerabilities in the project. As you can see here, so you just can detected two potential vulnerabilities.

A

I can expand to see. What's the issue os command injection in rake, I can review them and also review this security, uh one that is specifying their potential vulnerability for our use case here. We don't necessarily need to just uh proceed to merge the branch and remember it closes my issue, the issue I created earlier it it closes it because I mentioned in the description of the magic that you should close the issue once the mad request is matched now. Let me click on.

A

I also selected that it should delete the source branch since I won't need it again delete it when you are done so now. A new pipeline starts and, as you can see here just from within my mod request, I can see the pipeline for my review pipeline.

A

Then I do the merge and the new pipeline for deploying to production since imagine to master also start, and I'm seeing it here without having to jump around uh different.

A

Pages, let me see my issue here. The issue is gone because it has been closed by the march request.

A

You see it shows here that this match request was done, and uh this was the one I created earlier and was linked to the other project when I confused the name, so it even mentioned in which match request was uh this issue mentioned a commit that was done in regards to this issue, and this issue was closed via this particular match request.

A

So if we go back to the merge request here, we'll see our pipeline still running and uh going through. If you notice here, we have there's a difference in the pipeline. Okay, our review apps here has been deleted because we are done with it and we are now deploying our application to production. If you remember earlier.

C

A

Deployed to review language change, but here now we are deploying to production and once we are done from here, we can also we can still get the link to view our application.

A

If we check it currently, it should still be hello world after the whole pipeline is done to become oh yeah. Animal.

A

Do we have any question? This is the end of our demo.

A

While we wait, I'm not sure you follow us we're patient enough to wait till the end of the pipeline, but while we wait, this is the end of our session against any question or any comments that any one of us have.

B

No questions, I think you were very clear.

A

Okay, thank you is any of us using gitlab at work.

A

Okay, there's a question in uh in the comments I think chris is asking: uh how would you go about setting up test stage with two running containers like app mysql?

A

I tried with docker in docker, but with that I need to wait for mysql container to be ready before the app container. Is there a way to handle it better.

A

ah I can't think I think, most times, probably just like you already know, you specified mysql as a service which still uses uh docker in docker, and it has to wait for the service to be ready before uh the application can create. I can't think of anything at the moment, but probably I will share it with my uh teammate internally and get back to you within uh the.

A

uh I don't know if you can share your email address somewhere or if you are in the meetup group. I can share your response with you and let me answer. Let me copy the question site then answer your second question now this. Your second question is the other thing I try to do to call a third party api to run additional tests on an app that api is ghosts. Inspector actually needs a publicly exposed ip address. So is there a way to get app ip from runner? um That's interesting.

A

It depends on how your runner is configured. If, let's say, for example, you are using a docker based runner, that's going to be difficult because, except you are.

A

That's going to be difficult to the cabbage runner, but if you are using ssh based runner, that's where the uh your honor itself is running. All the jobs within the machine itself are not using docker images.

A

That's the only way you can probably get and an ip address, but ordinarily, I'm not sure, except if the.

A

Let me also look back for you on this, but of the top of my head. It's using gitlab because gitlab is built to just uh gitlab automatically just creates the machines using we use google cloud, uh a google cloud, vm is automatically created, your job runs and the machine is yeah ephemeral. So it's it's lost after a while. So I'm not sure the ip address are in any way provided or made accessible to the to the.

A

So sorry, if I may ask, is it the ghost uh inspector that needs the ip address or is within your ci that you call.

A

A

Chris, hello, okay, yeah, probably one way to probably do it is uh to use.

A

I'll help you investigate and share more details with you, but on the top of my head. I'm thinking that if you there's a way you can use uh docker machine. If that's still obtainable, then probably here when the images you use to run your jobs, we have some pots exposed. If that is supported by supported by ghost inspector I used see before so that when you are running your job, you can call. uh I forgot this website address, you call to get the ip address of the server you are running on iconhazip.org.

A

It tells you the ipa address of the server that you are using. Then you can fit that ip address to the ghost inspector, or maybe for your specific use case of ghost inspector. You can have a dedicated runner that will allow communication uh from outside, but I'll definitely investigate. This is an interesting use case. I will investigate and share more details with you. What's the best way to get information to you, sharing it with francisca or or probably sharing it uh in the.

A

Yeah sharing it in the meetup page, what's the best way to get your response to you? Thank you. A lot for joining and.

A

We look forward to sharing more with you yeah.

C

I can haze it.

A

Thank you very much yeah. Thank you, chris yeah christian hitch on me talk. Definitely I will I'll guess the necessary answers and share share share them with you. Thank you so yeah, I don't know how to pronounce it thighs. I know jay is wine dutch, so the project is thighs. Thank you.

D

Yeah. Thank you. Okay. That's a nice first acquaintance to kid lab.

A

Yeah. Thank you very much for joining.

D

A

Bye, so I think I'll be stopping the recording now uh and uh thank you very much for everyone that have joined. I look forward to seeing you next time.