Continuous Delivery Foundation Continuous Delivery Summit 2019 - San Diego, 26 Nov 2019

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Adopting GitOps Best Practices at HeidelbergCement - Peishu Li, HeildelbergCement/Lehigh Hanson

Description

Adopting GitOps Best Practices at HeidelbergCement - Peishu Li, HeildelbergCement/Lehigh Hanson

Do you feel lost in Kubernetes' ever growing CI/CD landscape? What about GitOps tools? Ever heard of Tekton and want to know how you can leverage this powerful yet flexible building block to help solving your unique CI/CD challenges in a cloud native way? If any of these questions ever crossed your mind, you are not alone! In this talk, the presenters will tell your a story about what the DevOps team at HeidelbergCement did for their digital transformation project when they faced these challenges in pursuing CI/CD best practices. They will walk you through their tool evaluation process and explain why they settled on a hybrid GitOps approach that used Tekton, Argo CD along with their own custom webhook to meet their not so unique requirements. You will also see a live demo of how these pieces work together to provide a complete GitOps workflow.

A

All right, good, that's team! How do you I wake up it? So what a great day we have went through, I really really enjoy and, like the some by name, is Pedro D, I'm, an IT manager or look for the High Council and we are part of the Heidelberg cement group. So so this just talk I will share with you the story.

A

We have attacking cut offs or CCD pipeline to deliver back resources into kubernetes and based on the feedback we just heard from one of the panel's members that we should not I should take the best practice out of the title again, but really one of chances. That I was saying. This is absolutely the best way, but we just said we'll follow with the best energy practices so about get off. Okay.

A

So here's the agenda so starting off with the brief introduction with who we are and what we do and also the introduced in the high-level back social architecture of digital, peripheral right, NT, I'm gonna serve, spend the majority of time to talk about care of adoption story and then, in the end, I will show you a complete workflow of carafes and also a live demo.

A

That's a quick facts about our company, so Heidelberg is cement, is the largest building materials company in the world, so we're ranking number one in aggregate material aggregate and we learn material in cement and number three in Radian bags concrete right. The company was founded in 1874, which is a hundred forty five years old and higher quality Heidelberg Germany, and we operate in more than 50 countries worldwide and we have almost 60,000 employees starts background about a company and here's the high-level max architecture.

A

So you see that the central to the aresty we have what we call tidal foundation. So really a kubernetes oven, shaped culture. So we're running lots of about resources. Those are most of the state resources. You know. On top of that, also we have several social services, one with Rambo go DB, post grass and radish in a random pure and it like search and other still for data sources and occurs. So the I also have services running on premises, although some accesses no symbolical vacancy social wrapper as a REST API.

A

So we can consume piety as inside cluster, and the connectivity between the cluster running in packs of magical and also social running inside or terror center is established through the address side-to-side weekend. Okay, so we got this architecture in this background out of way. So it's time, let's talk about the kiddos right before jumping to get up. So let's have a look of what the CCD pattern looks like before we adopted all right, so this is very typical sense in the pipeline, probably familiar to most of the guys.

A

My conception is so we use a gather from darkness in right, so we used together to store called repo and also the home char, with configuration repo, and we start with just a single Fourier so that we have single Depot store both cold and Yamcha. Now so we use the gather has the building container registry. So we use that ready to store the talking images and also started with the key driver. Ci is just a mo file.

A

Oh here you can see the typical kit of CI. We have a build, and this is very simplified version, but build stage. We have a deploy stage if it's CI in a city so pure stage, we'll just build an image and push into the talk registry and for deployment stage. So we run, we run a script. Film upgrade command and with touch install track so to make a sort of a item podium web banners that authority Humira leaves into the red cluster right.

A

So this this setup works for a while, and but there are a couple problem we found the first one is that if you use a home, you probably don't notice that from health perspective, when you deploy your, they call human lives. Presently after the deployment object is clearly in psychiatric cluster. The home will consider success right, even so that you may found in reality that you may have some part just struggling to be ready to be ready status due to a number of a reason right, but he'll just say, I'm successful.

A

So you look at the CRC the pipeline. This runs successful. Harvard, you know apartment every everything you know. I was running, and second problem is that after deployment you in successful men, so the artifact deporting into your crib light clusters and nobody's a monitor there and was so, if say, for her reason, you scrub something you did pancetta early point out say if it giveth of something right back then, if I point my assault, Pokemon logo, a concert I was twenty production between there. What are you moment?

A

Externality just sounds in then just there's no choice of what's going on. Is you have the many Dupree but have been in right, the pipeline deployment to put it back right? So that's the slow, monitor and there's no sense to reconcile your your film channel, which is your kids, our state and the runtime object we just accessed a so that's is that where the we found, the get offs is the solution to this okay. So here's the so what is get up?

A

So let me ask you so how many you guys have used in Calif, so so so kiddo is a. Is a term coined by comedy car we've works so the so. What it is is that I take the definition is kind of confusing, so this is the together of my definition in the Damon's way. So what it is is first face the crib Nettie service advocated every practice right, so you usually use the kid as a single source of truth, to manage both every application code and a configuration right as that's the definition.

A

So, however, if you look at the remember the previous life just show you we pretty much feeling every definition right every every single given here, however, we don't consider what s together, Franko's right. So then I added a second important drug home, which is a good able to must have a chromatic controller which is responsible to constantly reconcile reconciled applications status, African stares against an the desired state right. So if you have a visual here's, here's the the pipeline I just reconstruct the pipeline I follow the kid of principle.

A

So, first of all, I didn't buy no Amanda enhancement, so we sever the helm child from the source code. Okay, so you have to report.

A

Therefore, also the we did is we remove the the deployment stage altogether and repress that get off controller Monica get a controller right. So during that process we have you evaluated a two leading negative tools, so Wainscott from with cough with flags from real works and always Argo CD from Intuit. So those are the probably most popular tools out there, so also the it's not specified in the in disparity, also for both of to the care less about a CI part which I've told later right. So it really just focus on the delivery right.

A

So if you look at this diagram, so you configure your either flux or audio CD to monitor your configuration repo right. So if you just say, then they compare. What do you have to put in internet roster? If any duration detect, you may be good every three minutes, so they run reconciled or right, so they make sure they are in sync, okay, so there is a so we have Y variables.

A

We frogs a Naga City, so the basically they both get the job done. So it's really our decision, which one we choose right. So we found both are good, but there are few differences. I want to point out which first of all these are go. Cd comes with just user. You are a dashboard weight, lesser attachments show you a demo and, however, if you use the wave frogs, then you have to subscribe to.

A

Their online version is called we've caught, but we've caused not just attachment ham and tons of other like ostrich in management or what we have not. We don't have that experience. So that's kind of when saying we have to consider right. Then, however, D with we practice, in addition to monitor your configuration, the chrome charge, Cavallini ripple also Martin your Parker registry. So whenever you have you take it some change there. The automatic is sinking a documentation with your configurable like home child example right.

A

So if you use a aqua CD, then you have to extend yourself I plan to add an extra staff to do that by Red Square, something that then the bastard is the differences that wanna call a dynamic configuration so something like HP, a repairing or Israel. So those technologies turns to be changing. Your deployed.

A

Artifacts like Department, for example, the Egyptian they made I'm, going to certain threshold being in rich they're, going to either scale of scale out of scale in so you're. The Ripper number is going to change right so that creates a contract with your the law. Scotty get off to write the kid of things of try to so hey wait a minute. Your conversion says you have to partial, interpret our source of fire path.

A

I try to deliver the registry, so they the yeah, if you as a way more I, didn't big more skills of this cover. So the way the handle is very different. So at a time the the refunds at such Google as a select channel. So the answer I found was that you just get rid of the.

A

Gamma Phi well, they're, sitting works. I found the Agra city has a more arrogant way, so there they use this soap use this ignore differences annotation inside their CEO T in the current application. Okay, so then you can use just a path to specify which block or blocks in which sections you want to go. You want the take care of a kind of controller to you. Go to those differences alright, so this was better in my opinion. So if I inches I have those expressions in focus, APA, review or.

A

So the end of that we choose the August city as our ankle of tools, so I just want to emphasize that oxid is not as very better that we first write both are very into the reason we choose that we just because they filled out this rhetoric, so we don't have cousin. We don't have. We've called subscriptions everything. Also we do care about the CIA and we've talked about us or user. Is there also we take care of those dynamic changes.

A

But well, our dunya so still have have issue with the to see iPod. So if you look at go back to this right, so we still use the CIA capacity.

A

I use category of your life James point our canvas from Google, which is Kempe Bill hawal, found that the chemical is quite slow in both of them back cells are red in parkour, so it gets slow slower and to the point that each each bill takes from 46 and all the way through 18 minutes, to build it's very, very painful, to watch to say the knees right then, because of these music services, with your elbow can deploy it, you everything everything you can, but for me five, six, ten, ten!

A

Ten, that's a lot of hours of mass right! So then we have we have. We have fixed that problem. So then we found that there's a period.

A

We only found that is terribly called Butyrka heard about that bill. Kid connected. You have three or five times faster than chemical, so we give me a shot and also, we said because it's very hot in my computer to work with Kevin I've, CI sono, saying when I give text animation and so tectonic Christian Christy pronounced securities dative building blocks right, so it's a spinoff from the Canadian built right. So so we give tension.

A

And fortunately we found this this in the catalog that to have a period temporary, so just go out there and we were blown away by the results of we see so sort of ten minutes or eighteen minutes. So we cut down to an extreme or two minutes. The subsurf computers really takes that run. When means with you see demo, so very happy that then I will stay about on. Yes.

A

So if you pray with Tecton you're found better, it's been especially when a time when you vary the solution, there's no simple way to trigger the tech time right right, so we don't know derived city or take that taking a community doing important work. We try to do the creative way, so then also there's no support for like a caveman, so how company trick did take tectonic table from the kid library. So then we found out, we made a decision that we have to read all kill that webhook right call that language.

A

So we come on system. We've read very simple: we that way right. So here's one enough. Finally, she looks night okay, so we get rid of the the CI catalepsy I have line altogether repairs with take town five line; okay, no take them so because we're using other cities. So we have take care of cigarettes. The parking image tag with the actual configuration, the various fire.

A

Basically, so we are extra of their tax for inside inside the Tecton poplar and they had considered different a set of tasks inside that we have was like on staff right now. Each pebble on each task, an associate with the resources critical resource will be your image resource, so they called hybrid resource. So this is a terminology, so this is final picture and also the for the web hook, so the web for the configure to for the called repo the protocol pointers web.

A

So whenever the other make a commitment or make a budget required, which is the poor request, so with sugar, this work, the wifehood will create the pipeline resources and the private run everything right. So basically we do cogeneration from the web hook. So, first of all we specify a naming convention. You found the existing whale, we just use it. If you don't, we just create from scratch.

A

So this is run very well, so the so it's time for the demo. Let's see in the demonstration I'll show you how or back services source code, repo and helm child has own repo, so we dem Convention. Basically the basically serves a name. Config is a configuration home chakra form. Then the charity text on UI, so Tecna has less unit. So you can see the build process in real time and also we have show you teeth. I'll go see the UI.

A

So if I made some change to the code, repo or heaven you'll see, the pipeline will find out by the way the cop have run run. You can see that in real time after the fringe build, they will save the changes and they were modified to tag the image tag from Takara version, Yama.

A

Alright, so when that happens, then I can see they were big enough and deploy. Do you pop into the acrobatic roster? So you can see that area. Thank you. So that's one demo.

A

Ok, so, first of all, this is the Argo city. You are right, so I have just my have two applications. So this is the Postgres circle. We just never have identity services, so identity services. So it's as single Sun also matching those issues talking for API to protect the API. So the the dinosaur has dependency on the post grant so for Postgres, also used together to manage the home chart, reviews the because off stream he'll chart so try expend this one.

A

A

So it says it shows you their health status right. It's healthier that you know as a weather synchronized. By the way you can switch between brand-new synchronization for production. If you don't want it automatically, you can do manage the port, so you can also for this demo I just coming us automatically cigarette I, then the and this for every artifact. They also has held radius right, so those are checkmarks. So if you see that we have our application, they have a application concert.

A

Then they have this a service department and then this is a pawn okay. So if I make a change it in my source code right, you will see the pipeline. This is the textile UI and, as you see the pipeline run, you can also give injury to fibrin run, see what's going on, then detention, you will see if I switch here, you're saying they created the kick of this simulation process. They'll create new power and ten minute, the old part right. So, let's give it a shot yeah.

A

So this is the max sources. Well, this is a real real world war by the way I just copy to my personal, get back account for demo. What's a real real world war all right, so we have this SV dollar core, API, okay.

A

So if I just change something on the floor, okay, so you know this is what you do code review yourself. Our code looks good I, just remove the comment right.

A

Make a commitment, okay, so right away, so this workflow will trigger the pipeline right right. So if you dig in the problem that way as I say, the pipeline consists of tasks like hyper run, also consist in right hand created so called task right, good cause, Bunyan copyin, so you can, because the pie will have two steps or so here's the first step they can take into you so they're yeah, that was the Tecton pipelines. So each task is really pop and then run a contender. So this part, actually you initialize yourself right.

A

So, okay, that's going on the first test of the bill. I! Look at that! So you can you can just like that's what are you saw James demo in the joking exit, so this is a real pipe from Tecton UI, which is.

A

So how'd it runs over time, so he takes about a minute. So then you're. The second test will check out with push T the changes right. It's already done right first day so six hours, so you change the tag. Okay, sound fish leave just give a couple seconds see that the happy get up. Cachola, Pixar automates single a trigger simulation goof. So you see the current view. Part with the particle on line is we're using the rolling object right, so they can contaminate your path.

A

Gonna say that.

A

All right, so so that's pretty much it, and so this is the entire Kadapa adoption. It's really a start. We just got started alright, so the journey it's not a destination, so docile other areas such as like advanced appointment like a canary I, was going through and a/b testing because scenario and difficult activity the child so supposed to have. They don't support a lot of the past, but they have extensions right. So in August Elena's our goal Ross all right.

A

If you use the wave frogs, it's a wave flavor, so the post doodles advance the formula. Ok, so question.

A

No gets all the answer. Your question- yeah, oh I, just I, just remembered some questions earlier. You guys when I was like how to keep or you like I'm back sources. Right I have to have time accent for many years. So of course this tectonics are using, but the if you use araga city axis, has this concept called application of your application. So after I have to the put poster art right. So every serving everything of you, everything like all the others. There are various synchronize.

A

So this is very good for like disaster recovery scenario, so in which it is something extremely catchy little. The word is negative because that monetary reform, a great back.

A