GitLab Ops Section, 2 Jul 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How IaC and AppDeploy are Converging

Description

GitOps Use Case - https://about.gitlab.com/handbook/marketing/product-marketing/usecase-gtm/gitops/
Recent Video - https://www.youtube.com/watch?v=wgvRXLeHFxs

A

All right, so my name is Kenny Johnston I'm here with Jason talking about how get-ups and infrastructure is code and continuous delivery relate to each other. Just those a bit of background get lab is launching a git ops, focused use case around the get-ups model and Jason I we're talking about how we have two separate teams that are working on potentially similar work, one in infrastructure as code in our configure group and one on continuous delivery in our progressive delivery group. So.

B

There was actually also just another really nice video that was recorded and posted from the folks who are working on that yeast case I'm talking about get ops and all the different elements, which is another really interesting. Video to watch if you're interested in the topic, but yeah.

A

B

Was thinking is and it's the way that William actually interested on that video is that the infrastructures code has been developing and is getting some great tools. Get-Ups is meant to be an evolution.

B

On top of that, where you say like hey, you know what, if you checked all of that in the source control and then it was automatically deployed your environments and was kept in sync and the way he described it was it's essentially taking a lot of the lessons of continuous delivery of software applications and applying that to the infrastructure as code advancements that have been made, which.

A

Makes a ton of sense the.

B

Way that us and in not just us but the market, is talking about get ops. These days is really infrastructure, as code focused, so they're like talking to operations, people who are thinking about these kinds of problems and what I've been noticing and what I wanted was hoping to try and get people thinking about. Is that I think there's actually a convergence beyond this initial step, coming where infrastructures code will be more and more automated and will be replicated out to environments and and provisioned in everything based on the state?

B

That's that's how they get I think that application deployments will also be moving in that same direction and in many ways, already sort of are already there, but I think there are also innovations happening on the get outside that could then be brought back, and so.

A

B

That these that we can get ourselves ahead of the game a little bit by potentially thing about these things, as eventually going to converge and already building something that treats infrastructure and the code that goes to that infrastructure as sort of a bundle that can be delivered together as part of your continuous delivery process and so right now.

B

It's not that we're intentionally building two different solutions or anything like that, but I, just see kind of like similar discussions happening in two different areas that it feels like would be really really interesting to have as a common discussion and talk about you know like wow. What would it look like in three years? If, if these do converge, and is it a good idea? Is it a bad idea? Are they? Is there anything we're?

B

Not thinking about and I just thought it would be really interesting to have those discussions and that I guess the other element is that, even if the technologies don't converging somehow remain separate, deployments or releases that are managing a deployment are always going to include some element of code and some element of application, deployment and and I really starting to think that line will get more and more blurred.

B

As you know, applications are deploying virtual infrastructure for themselves rather than having another team, that's handling it and it's just kind of like all, is rolling together through the environments. Okay,.

A

B

A

That I'll share with you one of the things we heard it I heard a doze I attended to talk about care form usage in the speaker was talking about how the seam he was applying the same 12 factor app principles that you applied to an app to allow it to be bundled into a micro service and more progressively delivered to infrastructures, code, and so I do see like that, that your point about them solidifying makes sense.

A

I, guess one way you could think about that is he's infrastructure, just another programming, language or digital artifact that can go through the same CI CD process as anything else and I think the answer is becoming more and more.

A

Yes, that's part of the principle of like CI CD based infrastructure, as code where you are are consistently deploying changes in a push fashion here, infrastructure I, wonder how much of what you're suggesting is maybe about adopting the mindset of pol based for your application itself to like, instead of Mia progressive delivery, it's more the pull based delivery system.

B

When I was talking about, there's innovations happening on the infrastructure delivery side that could potentially be brought back, and that's specifically what I was thinking of this, like maybe yeah pull based software delivery as well, is really kind of because if, if pull based is better for deploying your infrastructure, it's going to also be better for the Poynting software, because you get the same benefit. They talk about security benefits, they talk about consistency and everything. Yeah.

A

B

Application deployments as well and.

A

We typically think about those things like one of the benefits of the pol based is the immutability. And let me like describe this historical perspective, like it used to be that you handed your application over a wall to an ops team, and so there was a sense of immutability, because the ops team was never going to muck with your application code. But now we're moving to this DevOps world, where someone could muck with your application in production and having the pull base model completely, prevents that that you're always going to be checking.

B

A

Repo for the current state, so that does seem like oh yeah, I, don't hear that from customers or users, but a future problem as you adopt more and more of this, you know we're progressive, delivering developers tend to be get repo, focused, so they're not going to go jump into the pot or container or virtual machine to adjust their application code, but I like that that that model that the other thing that you said that is interesting to me and is not exactly how I think about it.

A

Is this like joint delivery, and so maybe it would help to walk through I. The pattern we've seen in the configure stage is that the infrastructure definition is kind of own well in larger organizations. Is it owned by a platform team that is supplying that platform to many applications, so the delivery rates could be completely different right.

A

You could adjust your platform, you know once a week but be delivering new applications to that platform, and that disconnection is actually valuable because it means you're kind of providing a layer of abstraction for your developers that they can just ship. Knowing that the platform will will behave in the way, they think it will behave, so that makes me less inclined to think of them as intertwined in a reship release, and it is both infrastructure and application changes. Yeah.

B

I think that's probably true, and a lot of the stuff I'm talking about is a little further ahead, where I'm imagining things might go, but I think already, there's probably certain elements that that platform team probably wants to just kind of say like well. You know. Yes, it's it's more on the infrastructure side, but we're okay with engineering teams kind of provisioning. These things themselves so like a static, IP addresses or something like that. Maybe.

A

They're like well.

B

You know like, let's, let's hand that over and.

A

We'll just say people can do that. Maybe.

B

Be PCs will allow maybe other parts of the network provisioning will say like yeah. You can put that in your application stack just to find it there and it makes sense. Don't worry about submitting your separate ticket to the platform team to make that, for you go ahead and define that all in code and I can imagine over especially like, if I think, talking three years, that more and more of the stack will be able to be defined in software with the application itself yeah, there will certainly still be some line right, like you.

B

Won't in your application, be able to like spin up a new data center.

A

B

Becomes like a requirement for certain things, you're building, but to a point that's further along than it is today. I can totally imagine that becoming part of the and I. Don't.

A

Even know what you.

B

Call it, but it's like the app and infrastructure staff yeah.

A

Yeah, that's a great point. I totally agree. Actually I think we already see that today, you know things like you know. The talk that we heard from those was about different layers of the I came up with, at the exact time they're like different layers of how they build up an application from an infrastructure perspective, and there are like base modules that are things like you know, include VP, C's and no sizes, and things like that that you built up, but the end service is probably owned by that team.

A

That uses is like comprised of these different components or even think about it from a kubernetes perspective like if you were defining how many pods or some like redundancy, you would do that at your service level and you employ the platform, that's provided by the platform team, but they're still Mongols together infrastructure and app code definition owned by that same team. So actually that makes total sense that yeah you're always going to wants to be shipping them together. Does that mean that, like? Are there like approach of progressive delivery that we have is.

A

The only thing, let me put it this way like you- can think of technology changes as as always evolving, and one of those steps is a necessary evolutionary step that every organization is going to pass through like we, we don't do any CD. Today we do CD, we do some progressive delivery. Then we do get up style application delivery, or is it get up style application delivery would supplant what we call CD style, progressive delivery. Today.

B

I think it's a little too early to tell in some ways like off the top of my head. There they're sort of they I guess outside of the Venn diagram into different parts of the Venn diagram, because they they they don't need to impact each other. Well, the part that was really interesting when you were talking you know, I was like wow.

B

That's actually really interesting is what would progressive delivery of infrastructure look like and what, if your application code you're defining the infrastructure along with it, and so for this new feature, there's a new ingress point, so it needs a static IP address, and could you bundle that in structure with the code that.

A

B

Just that feature, and so like as you're progressively rolling out that feature it's doing the infrastructure in provisioning. That goes with it, I think, probably yes, but how you would define that could be pretty tricky, but it's a really interesting thought that things could evolve. That way, in which case, then you end up going through through both both of them. I think I have more exposure and more career history on the progressive delivery side.

B

So for me that feels like a more a thing that everyone's going to go through more than the other, but but get-ups is also really interesting. I just don't know yet whether everyone will use get ups or if it's kind of certain environments, yeah.

A

And one of it might be and I'm thinking of the get-ups, the distinction between the two as push versus pull and I'd rather use those terms than like, because I think you can do progressive delivery while still doing get ops yeah and so like I'm progressively rolling out my change. You.

B

Could, yes, you can you can definitely do progressive delivery in a full fashion and that's no problem and.

A

So that's what I'm suggesting is it's the pull fashion going to be, though, like just preferred method, as opposed to it being a stopover, and the thing that I'm I'm stuck on is I think that the pole fashion works really well with kubernetes style applications and container based applications. But you can still do progressive delivery with you know traditional virtual machine style deployments, and so, if you're or have not migrated to a cloud native, you would not even have the option of doing pull based typically yeah.

B

I think there's two threads here that actually are pretty independent from each other one and it's pulling push deployments and get ups in theory is always pull based. I'm.

A

B

Based but I think you can also potentially have it get be the source of truth and still do pushes in certain ways and then so that's one track is a decision, whether he goes you pull and push and and how that works. But then the other interesting one is whether the application and code deployment are one thing or to England and both of those I think can be considered separately, but also interact in interesting ways. Yeah.

A

I think couldn't we support the two of them in separate repos, but the same thing in that they would have some dependency between the two of them today, like if I had a application, repo and an infrastructure, repo I could say whenever you're deploying new infrastructure, you would also need to redeploy the app back on top of it or something like that. Yeah.

B

You could use in the same way that you could put like the front end of your app in one project and another project.

B

It's always going to be somewhat simpler in the end, if you can get everything into one repo for your atomic unit but yeah, that's not always possible and kind of depends on the organizational structure as well. If you got separate teams- and that makes more sense, but if it really is in the future, one engineering team is defining the infrastructure and writing the code. They're, probably just gonna work in one repo yeah.

A

Okay and then maybe there's that I keep thinking about this platform team concept, but maybe there's a separate platform team. If there's not, but in at the end, you would wants to define it as close to the repo.

B

So wherever that line is in the future, where it's not part of the core platform, but it's something they exposed to the engineering teams to be able to provision, but is the infrastructure yeah.

A

What's the you, you highlighted those as they came over the word, you threats, what do you can you it that's more clear to what you perceive as a threat to get louder like we're, we're deficient or potentially not ahead.

B

A

I bet, you said threats there.

B

Are different topics I think that can be in fact, I think. The reason I'm thinking about this stuff is because I think that there isn't anybody else and there isn't any other product. That's got these really nicely meshed together in any way, and you.

A

Have an opportunity.

B

To potentially build something really cool that that doesn't.

A

B

To put the put it into boxes that was sort of what I was imagining as as the bad feature where you know in two years, we kind of look back and we're. Like, oh cool. You know, we've got a really nice infrastructure as code deployment tool built into the product and a really nice software deployment tool built into the product, and you.

A

B

We make them worse, work together nicely in the single application, but someone else will then come out with like here's one where they're just the same thing and support for that, and it will be better yeah.

A

Yeah and I well like the interesting thing, is our current capabilities and direction, and the in restrictions code is very.

A

You can combine what we're doing with terraform. For example, you can combine terraform with an application code in a single directory and you could use parent-child pipelines to deploy them separately. You can use the merge request, widget that is still push based. Our future kubernetes agent is going to be pull based, but you could still you could utilize that for both change pulling changes to the communities, cluster definition itself, as well as changes to them, I think really.

B

A

To the application and that.

B

Was another another I think bad outcome, or at least not ideal outcome, where you're sort of eventually these things kind of converge? And we have two solutions and you think yeah. You can use our configure solution to do your app and code infrastructure deployment or you could use our CD solutions through your app infrastructure deployments and it just creates a a decision to be made at the beginning. For somebody who's like oh gosh, which one do I pick. You.

A

Know yeah I guess I would think of that as though you would be deciding push versus pull, but we should be prescriptive about here's the places where we think you should be doing pull and that.

B

Is like maybe that's a more logical I, don't what you just said is has not been kind of like formalized and agreed upon that. That's like what these teams roles are. So maybe that's a potential outcome where it starts to make more sense. Yeah.

A

It's weird because the your team is working on both push and pull in different context when, when, when deploying infrastructure as code in that category for things like terraform and then it would be pulled for kubernetes, which is yeah, it's there's a little bit of murkiness in my head there, but I think.

B

Deployment modes for CD, where he set up its kubernetes also, but where you set up communities to monitor for an image and if the you can have a monitor for a tag and so, if you're, just updating the same tag with like the product tag be in production or something like that, then goo radios will pull that whenever that image is updated and so like in the auto deploy kubernetes deployment, you can set things up that way and that's owned by the CD team and that's a full model or you can use our ECS deployment, which is like push.

B

They will push whenever things go out, and so so both are both right now. Yeah.

A

So the two threads we said were push versus, pull and remind me the second one, the.

B

Second, one was whether or not infrastructure and code will kind of just like there'll, be one file that defines them both. What Tara form is today like you can put code, and you can put code and infrastructure in your terraform definition and it will deploy everything.

A

Yeah I was thinking like one desirable outcome. I guess would be us having a prescriptive idea for our users, like independent of how we organize and make sure we're clarifying as teams, but for our users I think push versus pull is a like. When should you use either, and we should be prescriptive about when we think one should be used or the other as we evolve more pull based methods, then the other one in when it comes to apps and infrastructure? It's more.

A

Is it our recommendation that those should be tied together, and if so, then we will hopefully build an experience that is not disjointed as a result versus. Is it our recommendation that they shouldn't and if we can come up with those two kind of like sets of conventions that we want to suggest to our users, then I think that will help our teams align on who's covering what and making sure that we have. That kind of like goal post yeah.

B

Yeah and it's all pretty nascent, so it's not like we, it's not an emergency decision, but it would be really great to have everybody thinking about this stuff already and kind of talking about it. I think he'll help guide us to an interesting place. Yeah.

A

And I think you're right, calling out that it's nascent means we can't just say, like you know, go: ask five practitioners, one of the things that Victor. That means, like you, guys. Five practitioners for the best terraform flow you'll get typically five different answers, they're starting to become some coalescing, but but I do think it would behoove us to investigate what the what we think is our current convention and William like I guess we can change that convention over time.

A

But as long as we all agree on what the current convention is about, helm, cool, I think this video is great. Hopefully others will watch it and we can continue the dialogue. Yeah.

B

A