GitLab GitOps, 6 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Terraform-related features introduced in GitLab 13.2

Description

Terraform-related features introduced in GitLab 13.2

In this video, we go over two Terraform-related features introduced in GitLab 13.2:
1) Multiple Terraform plan support in Merge Requests
2) Read-only access to Terraform State API

Some useful links:
Cesar Saavedra's Twitter: @cesar_saavedr
Link to the repository containing this demo:
https://gitlab.com/tech-marketing/sandbox/gl-13-2-cs-demos/tf-mr
Link to the Epic to provide feedback about Terraform MR widget:
https://gitlab.com/groups/gitlab-org/-/epics/3441
GitLab Infrastructure as Code direction:
https://about.gitlab.com/direction/configure/infrastructure_as_code/
GitLab GitOps GTM Use Case:
https://about.gitlab.com/handbook/marketing/product-marketing/usecase-gtm/gitops/
IaC with Terraform and GitLab:
https://docs.gitlab.com/ee/user/infrastructure/
GitOps Topic page:
https://about.gitlab.com/topics/gitops/

A

Hello, my name is cesar cebedra. I'm a technical marketing manager here at gitlab, covering the cd and github's go to market use cases in this segment I'll cover uh the features, multiple terraform plans, supporting merge, requests and read-only access to terraform state api.

A

This is the agenda that I will follow for both of the for both of the features. The first one will be multiple terraform plan support in mrs first some context. Gear ups takes devops best practices used for application, development applies them to infrastructure. Automation, github is also one of our go-to-market use cases. We partner with hashicorp for some of our getups features.

A

Terraform is an open source infrastructures code tool created by hashicorp, which enables users to define and provision infrastructure using a declarative configuration language. The terraform plan command summarizes expected infrastructure changes. It generates an execution plan which is basically a set of necessary actions to achieve the desired end state for the infrastructure.

A

It's also a way to check uh the set of changes without making any changes to the real resources or the state, uh the uh and then um and for example, you know before committing change to version control system. In our case, a merge request is the place to run the terraform plan command.

A

The current state of this feature is that during a single terraform pipeline, several infrastructure environments might be affected previously. Gitlab enabled a quick overview of the expected changes in a merge request. Only for a single environment, starting with git lab 13.2, the terraform merge request. Widget supports multiple terraform artifact files. Like you see in this screen snapshot.

A

This is actually obtained from an mr and that with a pipeline that has run- and he has actually done terraform plan on three different environments, staging production and review, and you can actually access them as artifacts from the mr.

A

So why does it matter for customers?

A

You know when managing infrastructures code, many environments are typically touched production staging, for example, customers can now review what infrastructure changes will take place across multiple environments and emerge requests not be limited just to one, and also this will lower the risk of introducing errors uh into in infrastructure configurations into production, diminishing risk and eliminating inspected downtime. So what's in it for us, it helps us to better position. Our integration points with terraform.

A

It enriches our infrastructure scope story and it provides us with an entry point um into terraform accounts, all right, so, let's jump into the demo.

A

So this is a project that has a a uh get left ci pipeline right, uh the at the top at the top level, and uh there are three environments. Think of these of directors has three different environments: uh review staging of production, and the description for each of these directories is here.

A

Basically, the modules consist of all the terraforms that are reusable modules that will be reused by by each of the environments, production, review and staging and within each environment there are a set of terraform configuration files, as you can see here, they're all the same with minor modifications for each environment.

A

Obviously, so you can apply different values for each of the environments, and here we have the main ci configuration and let's go into there, so here the the key areas to remember or to to go over to be able to exercise this feature, you need to have a terraform plan step, which is right here.

A

Okay, uh you also need to be able to cache cache the uh the terraform direct terraform directories that are generated for each of the environments. You need to have the artifacts saved for each of the environments separately.

A

And we have a step for the actual generation of the plants for each of the environment. Here you have one for review, one for staging and one for production.

A

So with that said, let's go and check.

A

Let's create an issue. uh Let's say um we can say clean up some debug statements.

A

And let's submit the issue.

B

And then let's uh get started.

A

Working, let's work on it and then we're going to clean up we're going to open it and we're going to create a merch request.

A

And then that merge request we're going to open the web ide.

B

And let's go to the get lab pipeline and let's just make a small change here. We can just delete these lines here.

B

B

And then we commit to.

B

A

That branch and it's going to fire up a pipeline.

A

And the pipeline looks like this: we're going to be running terraform, init first to initialize the the terraform environment. We are talking to an http based state file which is being stored on the gitlab end and then we're going to go into the validate step. We're gonna, be we're gonna, be doing a terraform validate to make sure that all the changes we've applied.

A

um Terraform changes are, okay, are good in the build step, we're gonna be doing uh we're gonna be generating the plans for each of these environments. There we go so here if we go into the job itself.

A

um Okay, so that's the plan right there. As you can see, these are all the changes that will be happening and then let's go back.

A

Let's refresh this to make sure it's complete, so the merge is done. Let's go back to the mr. So here are the three terraform reports.

A

One for staging one from production and one for review and, for example, we can look at the production one and when we view the log it takes us to the basically the job output, the runner output.

A

Okay, this is basically the new feature. Is that now you know the plans for different environments can be uh added to the mr as artifacts. So.

B

A

uh Now the dmr is complete and we've checked that it's run. Okay, let's uh let's merge. So, let's mark us ready, let's merge.

A

All right so now that is merged.

A

There should be another pipeline already running right here, which has a deployment step.

A

Here you can see, there's a these two are manual, you can actually deploy staging or production.

A

And while that's running, we can go to the amazon, console and log in here.

A

Right now I have a staging environment which I was running the other day and now, let's reply to production.

A

And this is basically doing a terraform apply in this case we're using amazon ec2. So it's going against amazon and running it's basically instantiating a production environment on amazon, ec2.

A

Per the um per the configuration terraform configuration file specifications.

A

So it's going to add a vpc, it's gonna add an ec2 machine.

A

It's gonna has add a security group, it's gonna add uh you know, subnets everything that it needs uh to um and everything that it needs and it was specified in the configuration file.

A

A

A

Good, so the apply is complete. So let's go to.

A

Amazon and there should be now a production environment.

A

Besides the stage- and here it is.

B

A

A

Now I'm not going to deploy staging because I already have one running.

A

So, let's move on to the next uh feature, oh just in case before we move on to the next feature. These are the the steps to recreate what I just demoed to you.

A

Some interesting resources here that I want to leave you leave with you before we move on to the next feature. Is uh here's the link to the demo project that we just saw, and there is uh three interesting pro three projects that I found very interesting uh about the same topic.

A

There is one from emily ring that shows multiple terraform reports, four different environments. There is one for nick from nicolas click, which shows how to use terraform with aws, and there is. There is one for my cases, which shows multiple environments with terraform.

A

It shows a very nice directory organization with multiple uh pipelines and if you want to provide feedback for this specific.

A

Feature there's an epic that you can. You can add your input feedback to all right, so the next one is read-only access to terraform state api, so the current state is is very simple. Before this release git lab users without maintainer access could not interact with terraform commands, especially terraform plan, so this would fail here, and so, if you were not a maintainer, you could not.

A

You know, run production plan and see the reports.

A

What 13.2 introduces is uh now that, if a developer, if you have the developer role, you can actually, uh you know, have read-only access to the terraform save file api. I'm sorry, and in this case, that we use this user called sasha soft developer to demonstrate this specific feature. Why does it matter?

A

Developers are able to run terraform plan to see infrastructure configuration changes also developers as github stakeholders can effectively collaborate on infrastructure, configuration updates and for us it enriches our infrastructure's code story and enhances enhances our github's positioning of taking devops with best practices to infrastructure automation. These are the steps on how it works. You basically create a developer branch. You add sasha softw as a developer to your project, you log into sasha soft dev, and then you basically run the pipeline on the newly created branch and you'll see it work.

A

The terraform plan will execute for sasha, because she's able to access the state file uh on a read-only basis here are some interesting resources to follow up on that. On this specific feature, that's all I have thank you very much.