GitLab GitLab and Google Cloud, 6 Feb 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Setting Up OIDC to Get Credentials from Google Cloud

Description

Contents:
1:11 - Steps overview
2:00 - Creating the workload ID pool and provider
6:22 - Creating a service account
10:18 - Creating a blank GitLab project
11:18 - Writing a script to generate a GCP access token
14:14 - Setting CI/CD variables to point to a GCP account
16:57 - Configuring a pipeline to get credentials

Docs:
GitLab OIDC for GCP: https://docs.gitlab.com/ee/ci/cloud_services/google_cloud
GCP Workload Identity Pool and Provider: https://cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds#create_the_workload_identity_pool_and_provider

A

All right, so one of the items that you'll come across as you're going through the sa onboarding checklist is leveraging oidc, which stands for open identity, connect to retrieve credentials from Google Cloud within a CI pipeline.

A

The reason you might want to do this is, if you are trying to work with Google Cloud as part of your pipeline to say, create or destroy a server access, your storage or so on, and you want to do this in a secure manner. Ideally to do that, we can leverage credentials and access token that will be generated using open identity, connect and we'll take a look in this video. How we can do that now, just to give you an idea of what the end result is going to look like for this.

A

It's a pipeline that will perform a bunch of steps and we'll, at the end of it, print out a credential, and this will be the credential that you would then use to perform card operations in your Google Cloud environment.

A

So the checklist here will take you to this link, which describes in our documentation how you can configure oidc connect or oidc, rather with gcp, and we're just going to follow this step by step. There's three main sections here that we're going to follow at least how I think of it is we're first, creating a Google, Cloud workload, identity, pool and provider.

A

That's one part, the other page we're going to go to is creating a service account where that'll be the account that will be impersonating when performing actions in our Google Cloud account as part of our pipeline and then we'll actually create the pipeline itself in gitlab all right.

A

So, let's get started with actually creating the workload, identity, pool and provider. That's the first step to do this. A prerequisite that you will need is having a Google Cloud environment that you can work with, and I have generated one following these steps on the gitlab demo website.

A

You may have that or another environment, but in here I'm just going to click this link from the documentation which actually takes me to the Google Cloud Docs and on this page, which describes some information about workload, identity, Federation, I'm, going to scroll down to where it says, create the workload, identity, pool and provider.

A

That's what we want to do this link here that says: go to new workload, provider and pool will take us conveniently to the page that we need to go to and I'll move this over to this page, and in here we can start by just giving it a name. So this identity pool I will call it gitlab demo who- and you can give it a description if you want I'll, just click continue and now we need to add a provider to it. So we just clear this first section now.

A

The second part is to create a workload, identity provider and we're using oidc here. So that's what I'm going to select in this drop down here for select a provider I'll, give it a name and I'll follow this format. That's listed in the documentation, but I'll name it a little bit different, so I'll call it gitlab demo or slash gitlab right.

A

So I have a word or phrase uh Slash another word and then the ID we can enter that it may generate a an ID automatically for you or I can set one myself manually this way, and now we can set our issuer URL, since I will be using gitlab.com for this process, I'm going to use gitlab.com with https at the beginning and a trilling slash at the end as my issue URL so make sure it looks exactly like this with the again the drilling slash and the https at the beginning and I'll set the audiences to allowed audiences, because I want to manually specify this value here.

A

So in this case it's just the same thing as the issue. Url except I am not putting a trailing slash here right. You cannot have that here and this is for again the cloud hosted version, but if you have self-hosted, then you would use the address of that self-hosted gitlab instance. In the issue URL and your audience now we can click continue and we move on to setting our attributes.

A

When configuring your provider attributes, we want to allow it to basically tell gitlab that hey when we have some value in gcp, here's what it maps to on the gitlab side of things, so google.subject maps to assertion.sub.

A

Paste that in there get rid of that white space or a new line and I'll add another mapping here, actually so in documentation.

A

It says any attribute that you have on the Google side is just going to equal to assertion dot whatever that attribute is on the gitlab side, and in this case what we want is actually attribute dot user underscore login, so I'm going to put that in there attribute dot user underscore login and on the gitlab side, it'll be the same thing except we have assertion instead of attribute so attribute.user login and assertion.user underscore login here all right and that's it. So we have our attributes mapped properly, and now we can click save.

A

And now it's creating, and you should see that we now have a rule that I just created my identity pool. It is set to enabled it has the ID visible for me, the IIM principle that I will need later on, and we have our providers listed here as well. So we will need that as well all right now.

A

What we're gonna do is move on to the third step, which is create a service account. So we've just completed the process of creating the workload identity, pool we've created the workload identity provider and now we're going to create our service account.

A

This will be again for the pipeline to impersonate as it's performing operations, so we want to make sure that this account has all the right permissions needed for whatever processes, whatever you're doing in your gitlab pipeline.

A

Now, for this I'm going to go into service accounts and I suggest opening this in a new tab, so that you still because you will need to refer back to this pool information.

A

So when the service accounts in a new tab- and we can click create service account, you may see it at the top here since my windows, smaller click, the three dots there and create service account and I'll call this gitlab demo account name it whatever you like, and it's May should automatically generate an account ID for you. Otherwise fill that out as well, and you can optionally add a description.

A

I won't do that and I'll just create click create and continue, and now we can grant some permissions to this account.

A

I want this account to be able to impersonate itself, so it's described in the documentation here, but basically you want to make sure that we access, we provide it with or stuff that that basic permission so in this drop down here for selected roll, you can either type the filter or I'm just going to scroll down to it that there's service accounts here and in the service account section. We are looking for the service account token Creator permission that'll allow us to impersonate service accounts.

A

We want this account to impersonate itself, basically to be able to create oauth, 2 access, tokens or jwts, and so on. So click this service account token Creator and we can click continue and we can actually click done because we don't need to add any other users access or anything right.

A

So we click done, and now we can click into the gitlab demo account you'll see that an email has been generated, or this account we will need that later on and if you go to the permissions tab you'll see a whole bunch of permissions that have been given to this account.

A

Now you may see it, but in this case I'm actually not seeing it so I'm going to manually add another permission here. What I'm looking for is a row here. That's that has my IAM principle written. So this principle is set.

A

Url here should be a row in this principles column, but it's not so I'm going to manually. Add that here by scrolling up to the top here again in my service, account I clicked onto the permissions Tab and now click Grant access and in the new principles box.

A

I'm going to copy this value here from my from my identity pool that I created I'm, going to copy this I in principal value and paste that, in here in the service accounts principle value here, okay, and for that principle, I'm going to assign the rule the same rule that we talked about earlier, the service account token creator so make sure that's selected, and then we can click save and now I can see here there is a row for that principle. Set URL and we have the permissions set to it or service account token Creator.

A

So this looks good to me.

A

And now we are actually done with all the setups on the Google Cloud side of things, so we can now move on to actually creating a project in gitlab and leveraging the account that we've just created so I'll go in here and I will create a new project.

A

And we won't use a template for this. We can just use a blank project to start and I can call this something like gcp uh o, I c d y d c demo.

A

I will put this in my own namespace, so I'll just search for my username and specify that and we can leave everything, as is click create project and now we'll start to leverage the documentation here for generating our credentials.

A

So the first thing I'm going to do here is create my script, which will run these commands or set these values rather to get our access token that we need right, so I'll create a new file here and you can call it whatever you want as long as it ends in dot sh, so I'll just call it get credentials, dot, sh, right and in here we're going to basically just paste these values here.

A

The the text here so I'm going to copy this paste that in there then add a new line copy this paste that there and on a new line copy this and paste that there these commands are basically saying that we're generating a payload we're getting a Federated token out of that payload, and then we are generating an access token. Based on that Federated token, it's using a curl to do that when we have this access token, we are going to want to actually print it out for this use case.

A

So what I'm going to do is add another line and say: Echo dollar sign access, token, so I'll copy that and paste it there and you should see it highlighted in green because it's a variable.

A

We do need to modify this a little bit here so where it says Federated token I'm going to wrap this in curly brackets and add a dollar sign before the curly brackets. So now it's a variable, it's highlighted in green and we're able to basically just populate the value of this in here. That way.

A

Right now for this to actually work, what we would need to do is populate the value for this project number pool ID and provider id, but instead of actually hard coding it in here.

A

What we're going to do is we're going to leverage gitlab's, CI CD variables, so to do that I'm going to turn this into a variable I'm going to wrap this again, just as we did for the Federated token I'm going to wrap this in curly brackets and add a dollar sign and do the same thing for my pool, ID and or my provider id as well and there's a fourth one here in the last piece service account email. So I will do same for that as well.

A

So I have my dollar sign, curly brackets there and we're good to go. So our script is now ready to generate the credentials so I'll go ahead and commit my changes.

A

And once we have that we have two more things left to do: first, we're going to set the variables the cicd variables for this to use so I'll go ahead and do that I'll go to my cicd settings here from the settings. Tab and I'll. Actually I'll move this over here. So I have my variables readily available.

A

So in CI CD settings we can go into our variables and expand that I'm going to add four variables, the ones that we just highlighted. So we have our project number rule, ID provider, id and service account email to start. I'll start with the project. Number click add variable, set the project number as a key, and the value of this will be the project number. That is not the project name, but rather a separate number, which is kind of hard to find.

A

But the easiest way to look for it is, if you go to your identity, pool that you created in the IAM principle. There will be this URL here after where it says projects there is this number that is your project number, so copy that and paste that, as a value and click add variable.

A

The next variable I'll create is the pool ID and the value for this is going to be the all ID that we had so your lab demo pool Supply made. So that's my ID I'll. Add that as a variable, then we'll go on to the provider id which will be the provider we created and it's not the display name. So don't copy this.

A

Actually, if you recall, we set the ID as we replace the slash with a dash, so like I can confirm this by quick clicking edit on that provider and you'll, see here below the name of the provider that ID is listed. So I can copy that and paste that as the value again, we just replace the slash with the dash there. That's what your ID should be, and then click add variable and then finally, for the service account email.

A

That will be the email address that you saw in the details. Page of your service account. So if you click into again your service accounts- and we click on the service account that we just created your email will be listed right here, so I'll copy that paste it there get rid of those new lines and click add variable all right, so I should see four variables that I've just created now and we're good to go on the cicd variables side of things so now.

A

Finally, we can move on to actually creating the pipeline itself, so I'll go into the cicd section and I'll go to the editor.

A

And it's a very simple pipeline that we're going to configure so click configure pipeline there and I'm going to get rid of all this default text here, because we don't actually need that. We don't it's not a very complicated pipeline.

A

All we're going to do is I'm going to first give this it's just going to be one job, so I'll give it a name. I'll, say: credential get credentials, that's what I'm going to call it, but you can put anything there and now we need to specify the image that we're going to use, in this case we're going to use an image that has both curl and JQ installed already. So this one is dwdr-h-a-u.

A

Slash Alpine Dash, curl, Dash JQ all right and then we're going to specify the script that we want to run as part of this job, and there will be a couple things here, so I'm going to add a dash, so chmod plus X dot, slash and then the name of my skip script that I created, which is get credentials.sh.

A

And this is basically giving ourselves permissions to actually run the script that we created where we're getting the Google Cloud credentials and then we're going to actually run this get credentials script.

A

So that's pretty much it that's our entire pipeline that we have most of the work is done inside the script itself, so we're good to go now and we can click commit changes all right and now you should see when, if you go into Pipelines that we have a pipeline running.

A

We can click on that and click get credentials and I'll expand. This and you'll see here it actually already successfully Ran So.

A

It performed all the steps here and at the end of it generated the payload generated the Federated token, and then it finally generated the access token and printed that out for us right here. So that's pretty much. It we've now leveraged oidc or getting Google Cloud credentials.