Continuous Delivery Foundation Ortelius Project Meetings, 20 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Automating microservice configurations with Ansible

Description

Presented by Piha Adhikari at the May 20, 2022 Ortelius Visionaries Summit

One of the main principles of microservices is that they are each independently deployed. As a consequence of their distributed nature, microservice development and operations tend to be much more complex creating a barrier to adoption. If your teams have not moved from their monolithic silos and DevOps practices they may not fully see the benefits of a microservice architecture. Sticking with the "good old monolith" may sound better than dealing with the need to deploy hundreds of independent software pieces. In this talk, Piha will throw some light on how Ansible can help solve managing the deployment of hundreds of services, with their configuration changes, for you build and deploy automation.

A

Greetings: everyone, and thanks for joining today's microservices visionaries summit, my name is piha and I'm the guest speaker here to give a lighting talk on the automation of microservices configuration with ansible.

A

So, let's get started uh before we move into a real uh uh automation area. Let's just know a bit more about our organizing coming company, which is audelius. So a brief summary about this organization. The product artillery is uh is a as a micro service management platform and, as we all know, a bit that in microservices architecture we have a lot of decoupled pieces and there comes the problem here. Those problems are addressed by cataloging by decoupling pieces, like version details, cracking microservices, blast, radius and much more other information.

A

Autelius is also an incubating project under the governance of linux area and last, but not the least uh as part of this summary, uh we also want to mention that autelius mission is about simplifying the adoption of microservices through a world-class microservice management platform driven by supportive and diverse global source, open source community.

A

Okay. So before again, I start so some introduction about the topic and this session is about. As you know, about automation of configurations for microservices and as we all know, one of the main principles of microservices is to be independently deployable.

A

But what happens is uh in a way, it's good that we can deploy all the separate or small micro services independently, but when we see a big, a in big picture as a config as a consequence, microservices development and operation tend to be much more complex than any monolith.

A

Because of this distributed nature, if you, if we have not moved out yet from the working from silos, doing the deployment separately, one by one dependent on each other or so you know there comes a lot of cures, and that is why we need divorce practices to be here, some automation to be here and these operations uh there is a need, and so that is where I am going to bring some information to you today in this session.

A

uh So some introduction about myself, I am bihar dhikari.

A

I am currently working as a vice president, president, in the data analytics department in that first group uh organization in the uk, I'm based in london, I hail from india and yeah so recently in the recent experience I have been working with uh banks, so some multinational banks, and also some uh eco commerce, companies and plus some uh in some telecom area.

A

This is about my recent experience and personally in the recent years I have been a massive devops and enthusiast, and I've been working with the in these sectors, as already mentioned. Apart from this in my free time, you can find me cooking. I do online blogging and on weekends, doing some photography.

A

So this is all about me, okay. So let's get started on this as we already talked a bit about the problem, so we we can see. You know these dots, colorful dots on this uh slide. If you see so these small small dots are nothing but different micro services.

A

What happens? We are happy now we uh you know that all the micro services, because of the microservices architecture, maybe achieve decoupling and but at the same time, what happens like. If I see you know, let's assume, like I'm the release manager for the entire uh project, with the microservices and and then maybe in a single day, we need to deploy many microservices.

A

So then there comes a need. You know. Maybe there is one change. For example, I want to change. There is a new fresh environment and we have in that new fresh environment. Those details needs to be get added to all the deployment related things in for all the micro services. So how do we you know? Do we have to go to each and every micro service that the source code and make some changes, or we want to have some centralized configuration uh which we automate and you know which makes her life easy?

A

So, uh let's move further so yeah. So there are many um configuration automated configuration uh tools available, one of them being ansible.

A

uh So what is ansible some of you might already know, but we want to start with the step one here. I don't want anyone uh to just you know, bounce this information, ah so ensemble is nothing but an id automation, configuration management and uh provisioning tool. It uses playbook to deploy manageable test configure.

A

So in a way you can see the end-to-end cicd pipeline right from uh the part when we start, uh you know, building a code uh to then uh doing the configurations, testing, managing deployment, everything everything can be covered and then anything from full server environments uh to any kind of project, whether it's website, whether it's good application, back-end front-end, anything any kind of application, it supports all of them and it takes while by yaml so yeah.

A

So if you are familiar with the ml, uh yaml is a very easy to read and write a very simple way of having a code. So uh we'll talk a bit more about it. So this is uh what is ansible now further. uh We want to know why and simple. So as I just mentioned. Yes, a gamble is something very easy to read.

A

Its syntax is a pretty simple and ansible is an agentless one of the uh boon which we have about it. So if you can compare with other configuration um automation tools like, for example, if I talk about puppet or something so there, those are not agentless, we have to have a dedicated server, and, um but here uh in for when we go with ansible, uh we don't need to have ansible installed in the target, server or environment.

A

That's one of the best feature offensive, and then it's built on top of python so here which helps us to use any of the python functions and libraries most of the python functions I have been.

A

I have seen, are available and can be used, and then ssh is one of the ways available with ansible for secure connections, and then it follows architecture for sending configurations and very easy and fast way to set up and minimal requirements.

A

uh It has so I see it as a perfect solution uh to get started in terms of automating. Your microservices configuration further. How does ansible work so here? If you see uh in this first, let's talk about the diagram, so a person there using a machine. Let's call this uh machine, as uh you know the place where the ml file is present or playbook. You can call it, let's call it ncl playbookwebservers.tml and then from that playbook we want to connect to different web portals like web1 web 2 web 3.

A

These are the hosts different hosts and if you see the sample web server cml file there, so the syntax is also pretty simple, like we start html file with like three dash and then we add more information with single dash and name hosts uh here. The host um in the diagram are like web one, two three and likewise we just add at their tasks, subtask whatever we want to do like install blah blah blah restart this copy. This to this uh move, this file delete this file rename this file.

A

So all those things we can just add over here in this playbook. So this is a very basic example. Of course, uh we can have some better design also there, but that's about the advanced thing. Okay. So now, let's have a look of the architecture and civil architecture here, if you see so again a couple of things which are the main pillars of ensembl architecture, so users from the user side, if we see ansible there is something called host inventory.

A

So host inventory is a file which which has the detail of the host. For example, let's take some uh project like, for example, which has certain three environments. Just you know maybe dev test and production, so uh those three environments: they will the target environments their server details. We call them like host and in this host inventory will have. You know that dev server, some host environment details or um like that test environment and production environment then again playbook.

A

We just saw an example of playbook where ml file is there and we have tasks and subtask or something- and we just call those tasks, then talking about the modules. This is a very important part of ansible core modules, custom modules, so ansible works on basis of modules. I would recommend, if you are interested after this session, whenever you have time, go to the ansible documentation and there just look after the modules it has.

A

So there are different kind of modules for different areas, for example, if you want to go with the calendar time or something if you want to work with just uh operating system related things, so there are different models for different areas. Have a look of it. There are standard modules, there are modules uh where you can customize them, and then there are plugins available add-ons, uh for example like ebay logging, etc, etc, and then, in terms of connect connectivity- and there are again some standard functions available, how do we connect with the different hosts?

A

Okay, so moving further uh a playbook structure, so we did have a look of you know how playbook iml code like. Similarly here we have the flow diagram of it. So if you see from playbook uh the first ml file, which which is invoked when we start currency.

A

Functionality, so it will just start with playing uh connecting to the host and then playing the tasks, and then tasks will further invoke the modules, depending on what modules we have written over there. So it will just invoke those modules and those functions.

A

This is a simple uh example of playbook structure moving further, um so uh you know this is I'm presenting audelius uh typical pipeline cicd pipeline, which is in place, and uh this is here so that we get some idea how you know ansible gets pitched into a perfect micro services solution, so very high level uh flow of this step, one where we have some docker some contourization and with the help of that move further, we are uh here in this example, which triggers audelius to version microservices, track application, configuration management and build the maps which are required so after having everything counterized, then further, the audio source code is available with the right version with the right track and everything moving to the star uh step: three, where um the the scheduling is already done, uh which is based either on demand or on the series triggers, so those deployment gets triggered.

A

Based on that further the step, four, uh where we, the recording tracking of inventory across all clusters, is done and somewhere snowflake also comes into picture. If it's about clio cloud. So this is a typical uh ortillas pipeline.

A

So now here, if you see on top in terms of uh you know, you see jenkins there, you see like ham, circle, ci and all that, so this is a typical pipeline and if we see ansible somewhere here so I know, iii like a deploy hub is being used as a part of this and ansible playbook is triggered from there.

A

So, let's understand audilius and ansible relationship, so audelius uses ansible user uh that covers to run tasks locally, as well as all the remote servers, including windows machine which is brilliant. You know like linux as well as windows, can execute existing playbook to perform deployments yeah. We just talked about that. Then audilius called after ansible has executed a deployment and record information about a deployment. For example, what are the microservices deployed?

A

What are the environments where the deployment is then done and the logs which are generated so all those things all those things can be catered. I've been gathered here and then about the pre post and custom actions. This is a very important part where ansible is being used and um in the component level, where link happens, and then uh this is in two areas: pre-action and post action, so pre-action would be.

A

uh If we talk about kubernetes area, uh these playbook to scale kubernetes clusters prior to deployment when we want to scale up their servers and after the deployment again, we use them uh to run sql plus to load a sql into oracle db. So this is the area how or ortelius uh product is using n symbol. So again, as we know, you know, so these are the configurations which, in the audience, side which are required to scale up and do the liberated uh configuration.

A

uh So let's learn a bit more about creation of principle uh playbook um how it's done in what areas and there's a screenshot as well so uh autolia's product. You know dynamically, creates ansible playbook to on and it runs on the fly. As you can see here, the iron circle area and simple playbook are unit.camel.

A

So this is the unit and you can see ssh connection. You know to secure this is used and there are certain parameters provided like twitter to environment host name and one. So this is how the connectivity is made and creation of ansible happens um moving to further in terms of interaction.

A

So um again, here, as we already discussed, pre post and custom actions are the three main areas where ansible interaction is done from audilius and as we are using ansible to execute ham deployment of the product of a component to the kubernetes cluster. And if you see here that custom action is already mentioned and that ham chart is being used as a part of this okay, so that's all about a quick session on ansible how we can use it as a part of automating. The configurations for microservices um questions are welcome.

A

So if you have any questions, please post them out and thank you for joining us. Thank you and if you need more information on the audelius in the ansible side, how it's connectivity, integration and all so you can please go ahead. uh You know visiting this github link and if you want to know more about its upcoming events or something just go to the linkedin area and do follow linkedin. If you like the session and would like to have more sessions and in case um you know, uh feel free to get in touch with me.

A

I have mentioned my twitter and my linkedin, I'm more active on linkedin on my linkedin profile. So that's all. Thank you. So much for joining cheers. Bye,.

A