Cloud Native Computing Foundation KCD Africa, 1 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to automate the setup of a Kubernetes cluster on GCP

Description

Using Ansible to install, setup, and configure a Google Kubernetes Cluster (GKE) on Google Cloud Platform (GCP). Infrastructure as Code (IaC) is paramount to maintain consistency across different environments. IaC resolves the environment drift issue where each environment has unique configurations that are not reproducible automatically. Ansible is the tool of choice to implement this tutorial. It helps us to create the necessary code to provision a basic Kubernetes cluster on GCP (GKE) automation.

A

Hello, everyone welcome to the inaugural kubernetes community day, africa. I hope you are enjoying the event so far. Let me introduce myself. My name is rafael natali and I will be talking about kubernetes, ansible automation, infrastructure as code and how all they work together. A little bit about myself. I am currently a senior devops engineer at marionette consulting.

A

I have been working with cloud and devops for about five years now across different industries and geographies. Before that I was a system administrator working mainly with middleware technologies.

A

Also, I am an aws and gcp certified professional and more relevant for today's topic. I am a certified kubernetes administrator. Okay, so let's get started. Infrastructure is code. Infrastructures code is an intrinsic part of the devops culture and best practices. The goal is to guarantee configuration, consistency and traceability across environments.

A

There are a plethora of tools that can help us to code. Our infrastructure ansible is one of the most well-known automation tools in the devops space. It's a simple and powerful tool that we're going to be using today for provision or kubernetes cluster on google cloud.

A

Okay, so the first step and to have our code up and running is to create a couple of folders for or ansible. I will create just an ansible folder and a couple of subfolders inventory and rows. Okay.

A

Inside of the inventory folder, I will create what we call inventory file in our case today, the inventory file will have the declaration of all the variables that we are going to be using uh through our code. I have a section here for gcp, related information.

A

Where I define my zone, my region, I will define my project id my service accounts and also a section where I will be defining some variables for our kubernetes cluster. So, for example, the name I can say kcd africa and the number of nodes. Let's create that cluster, with three nodes with 400 gigabytes of disk, uh solid state and machine type, n1 standard, two: okay, those are the variables that we are going to be using uh through out.

A

We will see uh when you start to go with the rows where we're going to be using that okay. So as for the inventory file, I will be using a file that I created beforehand uh just for security reasons. uh I will not be showing here, but has the the same variables that I show in the example before.

A

Okay, after the inventory file, we are going to be using the rows. The ansible rows to configure the ansible tasks to provision or cluster roles are basically a set of instructions, also known as tasks that ansible you will execute in the specific order to achieve the desired state.

A

It will be clear when we start to code it okay, so the first row I'm going to be create is a row that will create a gcp network, a vpc in my project, so I create a folder called network inside of the folders. The folder rows sorry tasks.

A

Okay, so I don't want to use the default network of my gcp project to host my kubernetes cluster. In that case, I am first creating a specific gcp network. Okay, the tasks is just a ymo file. That should must actually exist inside of the tasks. Folder name should be main data.

A

Okay, so in this task I just have a name for the task create ucp network, and here I declare the module that the ansible module that we are going to be using the ansible module, as we can see here in the documentation, is a piece of code that ansible uses to know what to do when connect to gcp.

A

uh So here we can see in the documentation all the parameters that I can use uh in case. In the case for the network, uh we have an example which is pretty much the same, that we're using here and all the return values. So when we create the gcp network, we can return. We can register this in firewall, for example, the name as we are going to see here in in the row.

A

Okay. So the module is a piece of code that ansible knows what you do in the gcp. I will come back to uh to the rows to the tasks later on. uh Before that I will just create another row for kubernetes cluster.

A

Okay, create a row the same as I did before. I just created another folder kubernetes created tasks. Inside of these tasks, I will create another main.yaml.

A

And now I will have two ansible modules: okay, so, in order to create a kubernetes cluster in gcp, I need to use two ansible modules. I need to use the google cloud gcp container cluster and after I need to use the google cloud gcp container notebook to create a notebook inside of the kubernetes cluster.

A

Okay, again, I will come back this later when the cluster is is creating.

A

And the last step is to create a instanceable playbook inside of the ansible folder. Here I will create a file called create kubernetes yemo again another even file.

A

And the playbook the ansible playbook is created is a file that we create to execute. The rows uh is a ordered list of tasks, or that's in that case rows that we use to execute the same code every time. The same way, the same manner is important to highlight that, as I mentioned, is an order, at least so.

A

When I declare here to rows, I will first create the network, and then I will create the kubernetes cluster, because I need the network created first and then create the kubernetes, because the carbonates exist inside of this network.

A

Okay, so now we use the ansible playbook command to execute this playbook and the playbook will call the two rows, the network and the kubernetes rows and will execute all the steps. So, as we can see here, it start the play create infer, which is this one.

A

It and simple find the rows and it finds the first row is the network, okay, so the network, the network, is in here natural tasks. Main.Tml.

A

So ansible knows that you need to use this ansible module the google cloud gcp compute network to create the network, so it will use the name network dash cluster name. You can see here. This is a jinja 2 type of variable, and I replace this value cluster name with the name of the cluster name that I specified previously in the inventory file. Remember I put kcd africa, so uh ansible will ask gcp to create a network called network dash kcd dash africa.

A

It will create all the sub networks automatically, because I set this to true in the project id project again ginger, true type, it's going to replace uh with the project id that I specify in the inventory file with the credential file again that I use in the inventory file. Okay, the state the state for this task is to be present, which means that I'm telling ansible that whenever it runs it needs to ensure that the network exists.

A

Okay. So for now I already run the first time it changes, which means that it already created, we can see here in the gcp console, see already created with all the subnetworks.

A

So that's why it's changed: okay, because previous it doesn't exist and now it exists if, for whatever reason, this network is removed, it's deleted and we run this uh ansible again. It will recreate okay and if we run danceable again and then symbol see that the network already exists, it doesn't do anything, it doesn't recreate or delete or do anything you just say: okay, the the state is present. So I I'm okay with that uh in this. Using this keyword register, I am registering all the outcomes of the this module.

A

Remember when I show you guys the return values here in the documentation, so all these values how to create sub network description name. All of this is registered in this variable. Okay, and this is important because when I am creating the kubernetes cluster, I use this okay.

A

Now, as we can see here in the playbook, it is running the task of the row kubernetes and the task kubernetes clay, kubernetes cluster, which is this first one which use these container. This module, the google cloud gcp container cluster again the same in the documentation. It creates the cluster.

A

The very first part of the the cluster there are several all the parameters that we can specify through the console we can specify here.

A

Okay, as you can see, a lot of things, uh so what I did here, I used the name, which is the same name that I specify in the adventure file the kcd africa. uh The number of nodes that I want this tree, the location is the zone.

A

Europe was 2 in my case here is the important part is the different part network which network I'm going to create the kubernetes cluster. I don't want to create in the default one. I want to create the one that I created before. So that's why I register here.

A

I use the variable network with the return value name to create this kubernetes cluster. So I associating my kubernetes cluster with my network. If I don't specify anything it will create in the default network, but I don't want that. uh My project id is coming from the inventory file. I'm authenticating using the service account again that I specify in the eventual file the state is present, so if forever, if for any reason, this cluster is deleted.

A

When I rerun this sensible, it will create- or if I run in a project id that doesn't or in a zone or a network that doesn't have this cluster, it will create okay to remove. We just need to create another row with another playbook and change this to absent.

A

So if it exists, it will change the status the state from present to absent. It will remove okay.

A

So this is the first part, let's jump to the console and see what's what is happening in our kubernetes cluster console? Okay, you see that we have a cluster, we name it. Kcd africa uh in europe, west 2c.

A

With all the information here.

A

For the network you see is creating in the network that we specified previously that's great now. The second part is the notebook. So this this module only creates uh the cluster, but doesn't have any nodes.

A

That's why we need the second module, the google cloud, gcp container, node pool again, I'm specifying a name, let's just jump, for the documentation just for you guys to see again.

A

uh In the same way, there are several parameters you can specify. I just choose a couple ones for this presentation for simplicity, so the name is node pool and the cluster name. The initial node count is the same that is specified before, and the cluster which cluster I am going to associate my notebook, I'm going to associate with the cluster that I create in the step before the kubernetes cluster, okay and I will create, with uh the disk size of 100 gigabytes, ssd, the machine and 102 and again present.

A

So I I want this to be present okay, so we can see uh here in the terminal that it already run so it already created the kubernetes cluster. It already creates the kubernetes node pull so two here. Okay, three are changed.

A

So if we go to the kubernetes the engine we can see the node is here: it's created.

A

It has all the three nodes for those who don't know these nodes are nothing else than three compute engines instance that gcp creates automatically for us. We don't need to to worry about that.

A

See those are the three nodes of my kubernetes cluster.

A

Now, just to show you.

A

What happens if I run this ansible playbook again.

A

Now all the resources that we want already exists, so I don't want ansible to do anything you see. So when we rerun this with uh this information and ansible checks that everything already exists in gcp, you just say: okay, okay, okay, so everything's, okay, because all these states are true, so I have a cluster. I have the node poll. I have the network, so everything's, okay, uh ansible doesn't need to do any change in the environment.

A

A

Just to show how do we connect, I just increase a little bit my screen and how do we, if you go here in the cluster.

A

I just connect. I will copy the gcloud command to connect with my cluster.

A

I copy here paste. Okay, my good config was configured and now I can just oops group cto get.

A

Notes- and here they are uh the same three notes that we saw in the console before. um If I do ctl get namespaces, I can see all the default uh namespaces get parts and exam uh group system do we have anything that could assist in rainy? Oh, we have great. Let's go that's good from this point on. You can manage the kubernetes cluster with kubctl no problems at all.

A

A

So that's what I wanted to show uh to you today. uh This presentation was just to demonstrate uh the use of infrastructure as code to install setup and configure a simple kubernetes cluster on google cloud platform and also to show some essential concepts uh from ansible like rows and playbooks, and how to configure and structure them to provision resources in the cloud.

A

Much much more is possible with ansible gcp kubernetes. Obviously this is just a starting point to you be able to organize your kubernetes configuration such as namespace deployments and policies as code okay. So thank you very much for being with me today and continue to join the kubernetes community africa bye.