GitLab Release Group, 3 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Think Big Release Management #6

Description

Welcome to the 6th Release Management Think Big

A

All right so we're here for our sixth think big for release, management and Hana. You added a couple of really good items for our agenda. We'll spend this session talking about deploy freezes and if we have some time we'll go ahead and dive into some of the feedback that we've received on our prototypes. So I'll, let you do the voiceover and it started awesome.

B

I just wanted to share some of the the findings and discussion points that the Shah and I went through last Friday, so I had to hear the notes and also the recording and I pretty much were disgusting. You, the whole deploy phrases, teams and VCS is, and it was awesome because Sean just gave me a lot of insight from a back-end point of view. That will definitely impact the user experience and the in the front-end. So Meghan is not here yet okay, but I'm already discussing some of those things with him.

B

A sink and I gotta hear some of the learnings. The first one is that yeah, we can definitely add a time zone to the mo file and also represent this to the user. You. Why? Because I had a question related to that. So if you set a deploy freeze in a period well, how can we communicate this to users? They are other side of the of the world and you want to hear Jackie doing a voice. This about the embassy yeah.

A

Sure so would we want this to be something inside the NBC or how do we feel like the technical implementation?

A

Would it be much of a lift to add this as the first cut, or should we just implement something in the amel file and then add this as the second iteration I'm very open for this I'd rather just give people the ability to have the option to send in deploy freeze, and if this is too much of an of a lift, then we can delay it, but it'd be great if we were able to create this context for globally distributed teams, because that's what this deployed freeze, there's gonna be servicing anyways.

B

And I think that was a point the Hsien mentioned about the figuration. What is it grown right, Shawn that it already supports time zone, not songs, I'm, sorry setting this for the mo file yeah.

C

So sir I guess with I mean Clark Ron's really designed, for you know server management from the perspective of a server, and so so you can put it so I guess. Some of the questions we had were around you know blocking out holidays, for example, am I getting too ahead, but because you can do that in a con, but it's you know you have to go and kind of to be tricky to put it in there, but as an MVC we can certainly support it.

C

I guess they I guess you know NB C. We can also support time zones. No I, don't see any problem with her in the mo I guess. The issue would be proposed in BC is, like so say, you've got. You know, an organization has got you know 10 or 50 or 100. You know projects and, and they want to go and put in a holiday into every project.

C

They'll have to actually go into every one, and then then it would be a so once they actually did that or you know, even if it was ten once they did, that then you'd have to inspect the mo of each one to to see whether it was in place or credo, etc, and so so so I think probably will need to eventually get to some type of page or dashboard or something that can consume all this yeah Milland presenter and then maybe have some type of overrides, like a calendar overlay or the ability and the ability was for an administrator to override something you know to put in a holiday for everyone suddenly or a blackout period, and maybe let jobs go through when it is of like a period but see they all quite a lot of features.

C

After any of they say, yeah.

A

So I feel like what you're hitting on with this dashboard view, would be what what we would hope to you accomplish with supporting releases at the group level. So.

C

A

Allowing that to be like the entry point for then somebody to we could say as an NBC that this is at least a linking options. So you can then link to your projects, repo and change this yellow file, rather than like uploading, a yellow file at the group and having that trickle down, because I feel like that option. Like instrumenting global blackout periods at the group level, would be across a lot of stages and I.

A

Don't think that that would be very efficient for us, but we'll have to think about how customers are expecting to interact with that, because.

C

Well, in fact, I wasn't that's a good point when you know I wasn't thinking about having again father got pushed down, I was thinking about kind of having a combination, so the mo files inside the project and then somewhere else, the pipeline checks am I allowed to run. You know some type of master calendar and that's all that that came out of the feedback from the infrastructure. People were saying they have this need. You know they.

C

They need to do calendars, but there's no, there's no reason we couldn't go forward with just starting with the mo file. I think that would totally work.

A

Yeah I suppose that this would be our opportunities. Then, like dog food with the infrastructure team, wrote.

C

A

Second and third iterations, whether that's a master calendar view or not, yeah.

C

Well, we'll say a date: they would be a good starting point, because so we have a high volume application, but we don't have a lot of applications. So that's actually you know we're just being used and you know in everything that's great, but we don't have the problem of trying to implement 14 or August kind of 10, but we don't have 100 projects, yeah and so yeah. I think that and I seem to be pretty interested in this feature and in fact, they've kind of written some stuff themselves.

C

A

Hi Nathan how's it going where just to recap to bring you up to where we're talking about we're kind of honing in homing in on what is the first MVC going to include in the amel file and making sure that the issue description reflects like the smallest change that will make so. The first kind of topic that were iterating on right now together is including in the ML file time zones, so that if someone is in a global team, they see the blackout period in their time zone rather than in.

A

This is PST at 8:00 a.m. time sandwich will be very you know we want the user to be have contacts there, so I think we've decided that will include that in the NB c--, and I can make sure that that the description reflects that in issue.

D

A

And then other things that we were kind of talking about and discussing where the future iterations after we introduced the mo file Sean brought the amazing point that for organizations that have hundreds of hundreds of projects- and they want to implement this feature across all these projects, having a single place to administer this and I think that incrementally supporting releases at the group level will help be like a landing page for them to then jump off to administer projects, and then there could be even like a future iteration of having you know the ability to configure a Yama file that then pushes down to projects in a group.

A

So that's I think where we're at right now.

C

They're also, we high in a we were talking also about how like what actually are. We are, we blacking out exactly and we're blacking out a release only to production, and then how do we define production? Can we, if there's a problem with production, do we have some way of overriding it does? How does it affect the existing deployments.

A

Yeah, so to answer the very first question: I think we kind of discussed this in the issue. It'll be for all environments, given that gitlab kind of doesn't distinguish.

C

The production.

A

Environment so it'll be all environments, but the very exciting and interesting part is: what do we do if there's a production failure and it's a deploy freeze, so we would need to like how do you auto do a rollback or how do you? How can we support automatic role, blacks, regardless of deployed freezes? I think is probably the answer. So it's like investing further in rollbacks so that the owner or administrator can override the deploy. Freeze and really, you know, redeploy the last successful release or the last successful pipeline yeah.

C

Well, could even be rolled by a strut yeah.

A

C

Because we don't want to have the situation where you know, there's a problem in production name and then what they have to push a new version of the Amal to to change it right.

D

One thought I'd said: while you were talking you're talking about how it's gonna be for all environments and I guess this goes back to my question about how this would look at the animal, because I'm kind of envisioning that each environment specification would have a sub. He called blackout dates, and so you could kind of configure it on a per environment basis. But maybe that's were you thinking more of a global blackout that would automatically apply to all environments.

C

Yeah, that's a good point: I didn't I hadn't thought about about the at the environment level like that that it's more and it'd be more intrigued to do, but well, we can kind of have it at both levels like have it. You know for the yam or fall as a whole and then potentially have you know entries at the at the at the lower level that over either higher level I don't know yeah.

D

And I was wondering if it even needs to be at the job level, um because right now we might call a job deploy right, which will you know? Association is a server and and push them files, but get land itself doesn't really know that that's actually a deployment. It doesn't really distinguish Mena testing job and a deployment job, so I think it might have to be at the job level, where you mark certain jobs to say this job can't execute during the deploy freeze, mm-hmm.

C

It makes sense so.

A

Would that really be on the runner side, then.

D

C

Think no I I'm not I'm, not entirely sure, but now I thought we put it in the I. Guess we put it in the em all, but rosewood then just not make it accessible to the runner. It wouldn't tell the run of it that there's a job available to be picked up.

C

And so I think well, I think we made it pretty clear in the issue that that we won't automatically pick it up at the end of the period like it just gets missed for for the embrace I yeah.

A

For then BC, then someone's got to manually go in there and like retake it up. Yeah.

E

A

Like the documentation should be, if you set it to ploy, freeze you'll need to go back in there at the end of your period and retry you're missed I'm, not sure. If that's what, if we should have like notifications, that say, hey your this has been. You know this was skipped like it's a to. Do that then create that's, then created you might want to consider. How that how that looks, hi.

C

Anna we'll talk about something on the UI around that route with to show that it have been skipped for a for that reason. Yeah.

B

Exactly so, I looked into the documentation to see where deployments are displayed to users, and we have this in the merge request. The deployments view the deployments detailed view. This did not so my suggestion. The embassy is that whatever we can identify that a job was skipped due to freeze the update information instead of just saying, skip to skip because of deployment fees, but also the merge request.

B

We add the information about when the deployment freeze kind of like when it's a yeah when the phrase is a currently active and then afterwards change the message so that the user you know knows that they have to manually, go there and retry that one step, because right now yeah we can definitely create a to-do, but with that, for example, any military users would generate a task to do. You know if that the the system admin is that whoever is assigned to a specific merge request, but they get in because.

A

I, don't think it'll be worth doing it to do for this, because I'm pretty sure we'll fast follow with a retry automatically feature we're like once it ends we'll run the pipeline again or I. Guess the job again right, so is that what we're gonna call this now that it's a blackout period or a deploy, freeze, / job like how do we want to or is it you.

C

Know said they come to slightly different meanings right because ideal.

A

C

Like hair periods, more general right, it's just like we're just not going to basically execute what's inside this job, whatever that might be because he again Hitler doesn't really know whether it's doing a deploy or not specifically, it could be anything could be just some cute control scripts or something in any stage. So.

C

A

Are you are you? Are you saying that, by nature of it being deployed, freeze we're kind of biasing this as a actual deployment or change to production, rather than it being just a no action occurring on this environment or no.

C

Exactly yeah and we don't really have an easy way of of these- have any net yeah.

A

Because people who took the surveys, which were took the survey there's like a hundred eight responses to it overwhelmingly, eighty percent cited that deployed freezes was the lexicon that they've used to really.

C

A

Identify a period of yep.

C

A

Execution, either to a staging environment or a pipeline being run so.

E

A

I think in their minds that the respondents at least, and not all of them are get lab users, but in their minds, was if a pipelines running that it's gonna result in a deployment. But that's obviously not always true.

C

I also thought of a scenario when we were discussing last Friday about to say: you've got this.

C

You know a big developer group engineering group, you know a big organization and then they have you know a deployed blackout, freeze or whatever on production, for whatever reason, whether it's it could be a regular thing or there could be a problem, and you know they want to just you know, hold things back for a day or something so, but during that period I don't know, I can easily see a use case where we wouldn't want to restrict creation of review environments, for example, because particularly if it's a there's some problem and there's a reason, there's a blackout because you know maybe the developers and the testers are trying to solve this problem.

C

And yet they would, you know, get the code ready for as soon as the blackout period is finished being then they do the actual release. So so that so having a kind of blanket. You know stoppage on anything deploying well, we can't really define anyway, so it.

A

Has to be like in the mo file I think that specificity per per stage or environment is appropriate. Yeah.

C

A

A global change is not what our customers would be. Looking for, yeah.

C

A

We interviewed a customer about this. They gave me two examples of deploy phrases that they use the first one and they call them moratoriums. So this first customer said when I am redoing my QA environment, because apparently they have to pull a copy down from production, anonymize the data, and they don't want people to playing that staging environment.

A

So they put a moratorium on that and they let people know that they have to spin up their own feature branch in order to test their code and then they can deploy to the test environment once it's available. The second use case is once they have like a company event which they would have weekly company events. They employed a moratorium on Friday to Monday what people would be deploying all through staging, which would be.

C

A

It would skip production and then they would have to go in and manually rerun the pipeline for all of those merges weird Plus, and they mentioned that that was kind of a pain, because then they would have a bunch of merge conflicts that I would have to reconcile. No, we.

C

A

To think through, like in these use cases, do we advise using merge trains with deploy freezes to avoid a time where your master wouldn't be green? Yeah.

C

You did yeah, we could.

C

But I'll say but I think that yeah the simplest, the simplest case is just to is just to have it at the at the job level and then the the is user-defined whether what that environment actually is.

D

Yeah we might want to I just thinking through all the different ways and that we could do that in one implication of doing it. That way, which I think makes most sense to me. But now we could theoretically have a pipeline that has multiple different deploy. Freezes like one job, has a certain deploy freeze. Another one has a different one like it's a edge case, but it would make it hard to show in the UI like determine what is this pipelines?

D

The pipeline doesn't really have a freeze its that job has a freeze and I know we are trying to. We are planning and showing kind of at the environment level. What the deploy freeze is that doesn't really line up with how we'd be specified it specifying in the animal would.

C

I, don't know if we can do it at the stage level, but would would at the stage level work I.

D

Think maybe yet I don't know if we must have.

C

The ability, yeah yeah.

D

I'd have to look into that, so that was another idea. Another a third idea I had to is that we could maybe tie into only except those are like conditions that a job is run and maybe right now we already have some special values like merge requests. You can say only run this job for merge requests, so we could say accept, run this job except if there's a current deploy, freeze and then that deploy freeze could be defined, maybe globally or even outside the amyl.

D

If we didn't want to manage that, deploy freeze inside the amal, but then we're still kind of a there's kind of a hybrid approach there yeah.

C

I mean the other stuff could work like you know. If we have something as we're talking some type of dashboard, we could just have a you know a settings area and then the animal could just say, use this use these settings or use this calendar, and then that could just be defined elsewhere. Yeah I.

E

Think that could be super powerful if, because I shot, I think you were matching this, but an example where there may be like an emergency where you need to like, let's blackout right now, folks right away if they had to go edit. This yeah well file to do that. That would be kind of a bummer but I think the dashboard you're talking about plus and maybe Nathan's idea of just adding, except if and then it can go check this kind of settings area right.

E

You have full power to it's there in the mo one time and yeah and.

D

Then also gives you the flexibility to kind of define, an environment level or a kind of a global level.

D

Blackout I know like you, could have an instance wide blackout, and then every project can just say can tag their jobs with except deploy, freeze or whatever we want to call it, and then you can manage it in one spot. Yeah.

A

So I wasn't following everything in that, because my animal knowledge is not like super clean, so I might need help with updating the actual ish description if the requirements are different than what we're saying, because reconciling the issue right now, it sounds like we should make the update, for. We want to be able to specify time zones but I think, like that's just more like technical proposal, details that we should just add, but it's out of like the acceptance criteria, we have today kind of Maps exactly what you guys are saying.

C

A

Doesn't yeah you.

C

Slope, though, because we have something have another place to tails, all that right, like what good would would we'd, maybe start with only in the ml and then move up to say something bigger, I'm.

A

Okay, with that yeah I think as long as they ask is we want people to be to set a deployment freeze for any given environment. So that's what that's where our customers have asked for, but we've just discussed that environment. It's kind of a tricky word, because the environment doesn't necessarily have the rules or the logic that would prevent a deployment. Yeah.

C

A

Actually, at the job level, but the customers looking to experience it at an environment or a stage, but we can.

A

We can provide, like the guidance in the system, to show that, like in the animal file for this given project, you're gonna specify when a deploy freeze is active and the deploy freeze is going to be applied to relevant pipelines. For this animal.

C

And so in so from the UI perspective, Nathan I guess what we will need to do is I guess we need to make sure all this is exposed to the API and then, whenever the page is, is loaded, they'll have to be a call to check the the amyl enough. That will be expensive, that there would be the simplest way and then put this banner or whatever it's kind of very saying. There's a blackout period.

D

Yeah, that's the one difficulty about storing the blackened information in the animal yeah is that we then have to reference it just to display in the UI, which is a little odd and possibly more the more I think about it. I kind of liked it yeah yeah, the more I think about the I kind of like the idea of storing the blackout, dates outside and then just referencing them from within yeah.

C

D

They're they're two different approaches and I'd kind of push for the other one earlier I'm. Switching no.

A

I think this is where, like originally Nathan, we only have this conversation with the UX experience, which is like hey. People are used to specifying this in the mo fire. We'll see. Icd is all in the ammo file, but now we're learning that well from a back-end perspective, it's going to be perfectly be as performant as it could be. If we store this outside of the and will file so I, don't I think it's okay! Well,.

C

We do have some precedents as well because like, for example, if this simple I'm just thinking, what would the simplest version of this be and would be, maybe add another one of those accordions in the configuration they have. You know like a period or something- and you know, and just have a simple. You know relatively simple form in there and because if you look, for example, a just like merge strains, for example, you know that behavior is a lot of the a lot of the behavior of the CI is not actually only in the amal.

C

It's it's also driven by settings in these. In these you know in this settings area, so it's not really a and then we would just have like it only accept or something pointing to to their page. I think that would work yeah.

E

Isn't it nice to to pull this out of the animal so that, if you need to make an emergency change, you don't have to.

C

So then, you would just have it the job level that it could be controlled by configuration yeah. So.

E

It's like set the mo up one time and then you you can on-the-fly change things that you, okay,.

D

Yeah that feels consistent to with how like CI variables, work, yeah.

A

D

Kind of this external data that you're referencing from the mo file well.

C

In fact, if we, if we also somehow who get into the CI variables, that could be the way to have the override right.

A

Look at you look at you.

C

Don't think that would that would do it I.

A

Like that to override the deploy freeze to that's a really good idea, so I'm gonna find that issue and comment actually Oshun her hey. Can you elaborate on so.

C

A

And then we can expand that in that proposal. As far as what I need to do to update this deploy, freeze, MVC, it sounds like we have the work that has to get done in the ammo file, and then we need to create another issue that talks about the settings page is that yeah.

C

Because because there'd cuz, the forum would because the forum has to have the form could be, you know relatively complicated right, because you know you gotta have repeating dates and then times within those dates. So well, you might just have okay, you might just have times because you might have a blackout period. You know every day from 7:00 p.m. to 8:00 in the morning or something. But then so you know regular chrome stuff, but then what we could just use the cron form every go field in this book.

C

You know just put cron format in there and then we also need to incorporate dates and, and so and then you might have repeating groups right and in all that would roll up to one setting one name to setting holiday calendar or something. So so relatively you know you do it just as a rails forum, but it'd be relatively complicated form. You know two or three levels.

D

Too, we also could, if we were interested in simplifying it, might make sense to not do the cron part as part one. We could even just have like a more of a calendar view or, like date, pickers oh yeah yeah. That would be less useful because you have to go in you'd have to like update a year's worth of blackout, dates and then come back and keep doing it, but that might be a short-term solution. You know that.

A

Would that be the lower hanging fruit to do that first and then implement the animals.

D

Well, we'll need to make the animal aware of the black updates, regardless for it for there to be any kind of functionality.

B

D

I think there will be some component of yeah Mille the editing so.

A

When I spoke to users, this is a feature. That's pretty like 60% are the technical team. The build managers that are implementing, deploy, freezes and 40% have to interact the deploy freezes. So this is one of our first feature sets where we start noticing the bifurcation of get labs users being front-end dominant versus wanting to use everything in the amel file. But since the split is still fairly, technical, I am almost leaning toward. We should do the investment in the animal file first, but I'm open to to your guys's thoughts.

A

I just wanted to share that. In our user interviews and in the survey we learned that build managers, a compromise like compose up the release, manager, persona about 60% and most of those are working with API is developer, centers or yellow files.

C

What would it be worth to not not necessarily do a POC, but to you know in the issue just kind of like go through. You know we're gonna, we're gonna change, this page, we're gonna and then kind of think it through. You know what.

B

C

Nbc or you know, spec or anything, but oh I mean we could even do a wireframe, maybe because yeah when we start to think about dates and times it can get complicated right when it comes to forms and yeah just to make sure we're not missing something because I think. If we, if we go one direction, it will probably end up saying on that direction.

C

A

Yeah and I want us to make the most informed when we validated with the users the yam will file with the option to have like a date picker. That was very, very salient with them. Oh really,.

C

A

Agreed that that would be a really great option, but we didn't have like we didn't build out mocks and we didn't put mocks in front of them. This is more like hey, conceptually would having a yellow file configuration satisfy your need for being able to set deployment freeze, but when we look at the second and third iterations I thought that we'd be expanding more into the UI anyways, so so.

C

I read somewhere and I, it was kind of when I was onboarding until I can't quite remember where it was there. At one stage there was a the gamma file was a UI or there was that you know it was, and so so I guess with. Maybe we should I should try and find that it, because that way, you know why did we make the decision to go all the yeah more you know and and away from from pages you know, there's probably flexibility but yeah I'd.

D

Assume the main thing would be like being able to version control it and treat it like code.

C

Coulis yeah, that's actually a very good point: yeah right, but.

D

The question is whether like do we is there any benefit in version controlling our deployment dates like I, don't know if this one this might be a piece of data that doesn't need to be in version control, yeah,.

C

I mean: when are we stopping information about how to build an application versus we're running when we're we're running the application in two different things?.

A

Like this question, that's a very valid point.

C

It's really really talked about operations right, yeah,.

D

I think, regardless it's probably worth writing down all these proposals like how, because I think there's two distinct way. It's one is kind of what we can talk about where you configure the dates outside and then reference it in the animal or we do everything in the amyl. We include the dates in the animal and then the question would be like: where did we put that it probably is worth digging a little further into this issue and actually putting all those proposals down just so, we can think through them and yeah.

C

I completely agree and even to to work through some examples. You know like the weekend example: the emergency example the holiday the holiday calendar example.

B

The ui/ux everything is gonna defer if we go like, for example, with the job only approach job. First, then it's completely different from the current proposal. So we nice to get yeah.

C

B

Educators, yeah.

A

Because I feel, like our current proposed a little high on I, like what you have built out in our prototypes, satisfies what our validation confirmed like as the needs and the problem that we were solving. But I do think that we were looking at just a sliver of what customers are using things like zvo labs to do with deploy freezes so like the biggest use case for moratoriums or deploy freezes, were reoccurring times or holidays.

A

So I wonder if we should just outline all the different proposals and then select for the MVC, what use case we're going to tackle first and confirm that how much it differs from the proposal that we had outlined today. Yeah.

C

A

I'm gonna comment out: you guys does that make. Does that sound, okay, mm-hmm.

A

What next I think we have like three minutes?

A

This meeting always goes by so fast. For me,.

B

B

Can you touch base just on the on the comments you already left on the other topics at Jackie, so, for example, setting with spoke yards at a time if we shoot I mean yeah, we're gonna, look at the whole thing again, but if we should consider those points, multiple periods at a time for the same environment, job or whatever, and if we're going to maybe this is something for us to validate for users or show upcoming blackout periods, if that's the case also user interface, yeah, but I think we do have a competition.

B

Sorry, how I'd like okay, nevermind.

A

You just talked yourself out of any voice over that someone had to do. You're like that that we're just gonna talk about it. So when I look at like the things that we currently have today, hi Anna, it looks like it's just the date: picker right from an MVC side, right, okay in the front end, and that date, picker was going to reuse, a component that we already have available to get lab library right and then it was going to be fed from the amal file like the date specified. Is that correct?

A

Was that the assumption ie me on the settings.

B

Page yeah yeah, but it really depends if we're really gonna set only date, but not time, for example, and if we're going to do anything related to time, I think we need to customize their component time altered a UI okay.

C

If that I are we using a library photo? Is it some completely custom component.

D

The date pickers yeah yeah we're using its a few boots for bootstrap view. That's basically the bootstrap date picker, and then we wrap that version in our own get lab UI library.

C

D

C

A Tom Tom component we're not using it.

D

That one doesn't right, no I, think it's just a date on the related I'm. Looking at that, so is it. The deploy freezes for environments. Nbc issue- is that the one so I'm looking at the designs for that but I, don't none of the designs have a date picker in them. I don't know.

B

No because there's only covers the the yamo and then in the front end is showing users when the deploy will happen. So it's really the notification part of it. Let's see gotcha.

D

C

This is, this is the driven from the mo right, yeah.

A

Yeah, the actual MVC issue didn't have a date picker in it. Okay, we're at time but I think if we can iterate on the MVC issue. I would like to get this buttoned up before we kick off 12.10. Obviously, because we're supposed to be shipping it then so, let's, let's hammer down on what makes sense and I'm inclined. Do you do what's the lowest hanging that enables our users to have deploy freezes and get lab so whatever?

A

Whatever you guys think, is the best technical implementation for that I'm happy to make sure the requirements meet that need. So just let me know what I can do to help make that clear inside the description, but I really look forward to hearing more about your proposals and different options, and let me know how high on and I can help okay.

D

So it's good, so I I'll, probably add some purples. Let's just straight to that issue. Does that make the most sense perfect.

A

D

A

Right, thank you guys. So much awesome.

C

Thanks everyone.