GitLab Release Stage, 11 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Discussion about cleaning up stale environments

Description

Discussing https://gitlab.com/gitlab-org/gitlab/-/issues/19724#note_313679002

Releaase:Progressive Delivery

A

Okay, so a little bit of history of what we're talking about and then you can see this is in the backlog. So this isn't something that needs immediate attention, but it's definitely something that we need to talk about, because we have a lot of pressure internally to get this done and I I want to make it lab employees happy.

A

So if we do come up with a plan, I would like to schedule it right now. It's humongous and we can't do anything with it. So this is the point of what we want to discuss. So the idea is providing a mechanism to clean up stale environments, and this was opened by our Docs team. Okay, and if you can see the motivation to this is that we currently have 500 environments.

A

95% of them don't actually have a deployment and they want it, which means their dude they're, doing nothing they're stale and we want a way to clean it off. So that's the motivation of why this is important and we have. We have done some work around this in previous milestones. So in 12.4 we provided an API that allows you to delete environments and basically anyone with some technical scripting knowledge can create a script that will, you know, go one by one and delete it. In addition, I think I.

B

Think that the deletion is not strict requirement, but like stopping environment, it's I think he at least it fulfills. There had an expectation stopping.

A

The environment, yes,.

B

Stopping a memory and actually cleaner, cleans up in Berman's board deletion is actually like a remove the database record completely better. Stopping is just like showing in different, so I think it's a I think it's enough for them at least because nobody cares if the record exists or not as long as the available environment, the index page is clean, I think it's enough for them. I guess: okay,.

C

Just just so there surely are. We talked about the the one like the record on the database off get lab, and then you have the actual environment on the line. Deployment server right, those are not being created like the last one is not being created by, for example, manual job right. That is it's just a record that is being created and that's being stale. Yes, gotcha.

A

Okay, so the first thing we did was the delete API and the second thing we did not long ago, I think maybe two milestones ago was we added a way to auto expire environments?

A

So if you set it up in the first place, it just gets deleted after the expiration date. So let's say you set up 30 days after 30 days, it gets deleted and then you don't need to deal with this. The problem is: what do you do for older environments that didn't have this configuration yeah.

B

So basically, the idea to approach this problem is that let's create an a future to set out of also the experiment or to the existing records. It's all the records, it's really easy, and so yeah actually I have a proposal to this issue. I can talk that later then yeah. Basically, the idea is just setting X parallel pillared to the existing records and yeah.

A

Okay, so that would that would do it after the fact. Would this be a configuration in the setting, of course, probably.

B

Yeah can I share my screen.

B

So the basic idea is, we can hear this issue well.

B

Just think as I'm gonna shoot, it.

C

By the way in between our environments with an expiration date deleted or stopped so only the record or both.

B

So this is a accessory. What's the question.

C

Sorry I asked like hey when we have environments with an expiration date, the new ones. When do you expired? Are they deleted, like including the database register and okay, just.

B

ah Yes, yes, so here we are saying that wwe.com, which is a problem problematic project today right and the problem, is that under the review folder, there are so many no deployment empty, empty environments like this one. How about this one- and this query dog team- wants to clean up these records right. They don't want to see this one in available page, but we can just move it a way to stop the one because I nobody cares. You know what existing is stopped and.

B

The basically in my proposal is that, let's let allow users to set X parallel appeared to these no no deployments yet environments empty environments. So maybe we could bought down here or something I, don't know they. You access up to give me to you, but we have a button here and there like schedule schedule or to stop in one week, two empty environments or something like that. It's just just like one button.

B

One button to update all existing records to set expertly appeared so existing is our existing feature, which released two milestones ago or automatically cleanup environments is going to catch up these empty environments. So.

C

This is gonna, be a one-time action right. This is only needed.

B

C

B

Yes, so that why we need to set the button here, the folder is that, because users do not want to set the ex peril appear to the production of a staging.

B

So this is a again this. Like a current UX proposed I said it's a bit: mmm it's it's kinda, not ideal. Probably it's just over engineering and what do users.

A

B

To do to just clean up under this, maybe folder, so why don't we just create a long button and then create.

C

A we can essentially create a call-out that appears once certain requirements are found within the project. Like hey, you have like a +10 and no deployments yet review environments. If that is the case, then we show a call out towards the user. It says, hey seems you have a lot of unused environments and you might want to consider stopping them and move them to the stop. You know like.

A

Permanently in the UI I do have an Shin Young is: is this something that would only need for the Dax team internally, or is this something that also external users will be using external.

B

Users can use this because this is not just a Gila problem, but yeah users has the same issue: yeah.

B

Maybe that phosphate Edition is just creative man you're a lot like one click, one one-off action button somewhere and the next we can just have like, like Demeter, said like automatically detect. If empty environments are powered up, we can just say like coal out and then like let users to clean up the things.

A

Yeah we could do that iteratively dimitri. What do you think.

C

Sorry, can you repeat the question so.

A

Shinya was suggesting that in the first iteration we create a button that we put here in the UI, and people can press and in the second iteration we do it with your suggestion. If you have over 10 environments that are it not that have no deployments, then the product will suggested.

C

So what is the? Why is wiser, needs to be iterative there in terms of scope, because it's a scope, scope decision. It seems.

A

C

Questions yeah.

A

C

Questions basically like, why can't we skipped was the second one. Is there a good reason to to be more minimal.

A

Have we lost you, Shinya yeah.

B

Yeah shake it. What is the second one, so you mean they.

A

Hurt and scope like, what's the difference between doing one shot for the end results which is tell when, if you have over 10 for example, environments, then then have this dialogue as opposed to the first iteration that offers the button.

B

B

Sorry I assume struggling Gander send down. Did you phrase.

C

Let me rephrase yes so alright, just asked like hey there's the idea to be iterative here, first place a button and then afterwards, in the second iteration place the call out message and make the button only appear if certain requirements are met. My question towards that is, why would it be necessary because the first option would create you, except as in we create a permanent UI action button which is not useful to 99% of our users, where the second one will be contextual right like if it?

C

If the context is right, then it will show the button. So the question is: can we do that immediately or is there there's like a huge scope involved? You know, you know, shouldn't foresee and end result image it. Okay,.

B

Okay, I guess your question so.

B

First of all, do you know that code or Dex perrolli feature for environments we shipped to mouth Chango?

B

B

How can I show you.

A

While Shinya is looking for it, what do you think of a compromise which is, if you have over 10 environments, that have no deployments only then show the button right.

C

B

Can't connect when about this old XP relief uture it. Basically, if user configures review very environment properly with this actually there's a Gila llamo syntax, that example. This is a Caleb's a llamo and is a configure that environment name.

B

Hold stop in one day. This is a.

C

B

We released two months ago and if user properly can configure cell review app with this new keyword, basically the number of stale review app no empty, but no people and all emitter empty environments. The number of empty environments won't increase because all ever means is gonna have our expel appeared. So all of them is going to automatically stopped and removed from this available. Tough. So.

B

Let's, let's consider that if user created a new project and then they set this keyword from the beginning, they don't need this button at all, because the reviews- yes whatever it is, it's going to be stuck automatically at this. The problem exists for only existing future, and that is a forgot to set this keyword or it did exist when they configure maybe app, and so basically, this.

B

What in such project, users want to what user want to do is set expel a period for existing no empty environments? This is the first step and the second days you just want to update Google of a ciggy llamo tea set Oh, stop it properly, and it with this two steps. This is, do not need.

A

A manual action anymore.

B

B

C

B

Following okay right, so this is the motivation why I said? Probably we need about here something like that. So in that sense, in this button, maybe we want to show dialog that.

B

With this button, you can set expel, appear, dirty old, empty environments, and later you should update the key log, say llamo to set auto soap in so that your pipeline, your project, won't have her one half empty environments anymore. So you don't need to manually click the bottom anymore.

B

Something like that. Is it clear.

C

Yes, that's that's. That sounds amazing to me and in line with what was to some sort of discussed previously.

C

So the question the question is: I think the remaining question is: can we do that in the first iteration create the call-out message with the button and the instructions how to do that in the scenario that we recognize towards the user? It is relevant for them to do so. So can we implement this button and only show it when certain requirements are met so where it makes sense to what to use.

B

C

A

B

mmm Like if there are ten empty environments- and we show bottom right.

C

B

It's a from back in the parks agree is possible. Yes, with.

C

Your explanation, in terms of like hey, if no out updates Amal is said, you know, then we can. We see that from the back end, like hey, give me all environments that are currently stale that are created with a yeah mode that didn't include auto update syntax and if those exceed a number of say ten, then we show this option.

B

A

B

Possible, yes, the tricky point is that how we define which involvement I did still, but we can creator I, don't know whatever condition like if the environment doesn't have any deployments and it exists for like a month, then we consider it as a still something like this condition is required. But, yes, it's technically possible. So.

A

C

I think it would.

B

B

Is not high I think we can just include in wanted ocean. The is the first generation there's.

A

A column here that says updated is that something that we can use the information from.

A

Like would that define something being not still.

B

Yeah, we can consider that if it's now updated for.

B

No sure because, like if its production environment and then, if it's not updated for months, it could be considered stale, but it might be not so mm. It's been difficult to define, which environment to distill. We.

A

Have a few conditions that have to be met and I think it really. We need to be very specific here so that we don't mean production, so hey I wouldn't want to do that. Any work on protected environments, I think. Second of all, we have to have no deployments yet so I assume production would have some kind of deployment to it and, third that it had not been updated in the last I. Don't know, yeah.

B

Yeah yeah yeah, if we consider, if the logic takes into account of, if there are any environments at deployment or not, then yeah, it makes sense.

C

A

C

Good so in this case, only in a very specific instances, this call-out will be shown. We will instruct the user to you know, fix this situation for now that is a temporary fix that will fit just the current situation and instruct them how to fix that in the future. So there's a there's a like to two parts to this call-out.

C

We refer to documentation, I, think that would be them the NPC there.

A

B

This also has a good point of that. User might not be aware of a new keyword Oh to stop in this thing. Yeah, it's a good good place to you, encourage them to define it.

A

C

A

Can we also add it to the audit log that someone manually has to delete these environments.

B

Maybe should be next is the ocean.

A

C

It won't there be auto records, be created as the person that will click the button like how I'm thinking about it's like, as if, as if the user batch you know, stops the environments like they do a batch known, and if you do it manually, there will be a record of that right.

B

Yeah, so this logic is basically stopped in mermen despots.

C

Yeah, so will there be records and is the iteration or this talking about or please correct me if wrong? It's like, instead of having, for example, in this case, we have three three, nine thousand or almost four thousand review. Apps say that you click the button. It will remove three thousand of them. Instead of praying three thousand records, it will just create one which states. You know this user deleted three thousand environments. If you want more information, this is the list I.

A

Think in terms of an audit we have to list them all so.

C

If that is the case, will they not already be listed with this magic? Won't we get that for free. We.

A

Need to log the delete action, I, don't know if we get that for free.

C

Shinya, you know yeah.

B

C

Okay, so that is something we need to look into, but we might get it for free or not yeah, but in any case, if so, then it's just a separate iteration.

A

There's only one thing that I'm a little bit concerned about in terms of this thing is how often we're going to pull the information, because, if we're deleting thousands of environments and polling might have a performance, it might impact the performance.

B

No, no, this doesn't impact any performance.

B

The initial purple cell, like this, is like in this proposal. I have a strong concern on performance, but since we like, we just said expelled instead of just deleting now, we can just say like let we're gonna schedule, deletion, we're gonna schedule to stop Berman's. So it doesn't like background worker there's a background worker to stop all like environments, they expert environment, so it it actually can handle a bunch of Umbra meds in scale. So they're no concern on performance is this approach.

A

But a user won't actively see anything being deleted anyway, because we're just scheduling it. Let's say we schedule for a week from now a week from now.

A

If I refresh it up button continuously at some point, I'll see them like going down one by one right. It goes how many in parallel and ham in serial are deleted.

B

It's a Philippa I have got the current background Parker in the process setting, but it's basically can stop many many environments at once. So mm-hm I, don't think users wait for a long time and when the right time comes, Wayans expired.

C

Is there a reason for us to like, if so what I understand that is if the user clicks the button, then a week from that moment, then the action begins to take action. Wiser the week needed like and I'm just like immediately yeah.

B

It's a good question and then so, basically, after an environment is created, it doesn't have any deployments, but it might have at some point like maybe like. Usually it gets a deployment within I, don't know ten minutes, but it depends on the pipelines. I think it might be a day or I.

B

Don't know a few days so like one week is the basically hmm safer option for them like at least if your environment is going to get deployment, it should happen within a week at least and otherwise, because that is as a still so there's the reason. We cannot immediately start the process. Otherwise it's going to mess up the usual continuous deployment process.

C

Gotcha, okay, that makes sense taking notes of this so update issue with what we've been discussing here.

C

Seems very doable by the way. If we get this out, I don't know when it can be scheduled. Yeah.

B

Same impression, I think this should be doable and then it's useful for doch doch team members. I am.

C

By the way, thinking of one additional thing that might be needed, that does not include or increase scope by too much in terms of front-end. This I think the button when pressed, does need a modal to reconfirm by the user that they actually wanted to leave. You know 3,000 models unless but I have to look into that, because it's an action scheduled a week from now it is like. Alright, this action has been scheduled a week from now.

C

Then it will happen, the user might be able to cancel it again like is it if it's an immediate action, a model might be needed. If it's not, then the the UI will update. Like hey somebody pressed this button. If it's the current user, it's occurring to use it, but also might be another user, so we don't get. You know multiple of the same thing scheduled like for example, say that 300 users use the same project and 300 users push this button.

C

You know you would like the first one to push the button, and then everybody is informed that this action will be scheduled basically or it's already been pressed once. What are your thoughts about? That? I have.

A

A feeling, if something is missing here like to indicate that we did something with this environment, so I don't know if, if we need to update the updated column here in the screen that you see here or have like a note column, that says scheduled to stop in or some kind of information, I.

C

C

Yeah we can do that in deployment like there.

A

Were some concerns about adding more columns here so I'm trying to think.

C

I would not add another column at all. I, don't.

A

Know what about.

C

Adding it in deployment like no deployments yet will be scheduled to be removed if no deployments are received, something like that. So.

A

Where would you see it like in the deployment column? Yes,.

C

B

Do I'll train you like it and do you know this problem is basically doc team's concern right, the very specific team and then, like the other, guys like I, don't know somewhere somewhere outside of Rome. The duct teams do not really care about like what's going to happen home the environments right so I think that showing the deletion progress were I, don't allow it to the other. Members could be here, I, don't know just disturbing them or it could be noise because it's just you know, stop being here empty environment, so it doesn't.

B

They don't need to pay edition Oh. mmm Well, that is up to you UX decision, but it's just my opinion. Yeah.

C

I see what you mean, so you say like instead of informing environments and every page that if there are no employments yet and this action has been pressed, if we do this, then we inform them in all those places that you know this environment will be deleted if it does not get any new deployments which might be unnecessary. Well, so there's there's two things.

C

I think I would like to inform users, at least in some way that this button has been pressed, and this action is in progress in one way doesn't really matter too much, but the other one is. Do we need to inform users of specific environments that are scheduled to be the leader? Yes, now, if the second one is not true like, if that is not per se, you know requirement, then I would say a mass like I'm, just a message that the process is in progress should be enough. As far as I see it.

C

What do you think for it.

A

Yeah I think as long as people we notify the users. It doesn't really matter.

C

Yeah, do you think we need to be specific per environment or it doesn't matter so much again.

A

Shinya is a point that this is specific to the Ducks team. So as long as we have some kind of notification that says all environments that have no deployments will be deleted on and that's it or scheduled for deletion.

C

Okay, oh I'll, get into that.

C

Yes seems good, let me see last thing is I want to have want one thing: clear say that we do that say that somebody pressed the button- let's let's walk through this. So if this call out message informs the user, hey your stale environments, would you like to stop them and also, if you want, you can update your CI llamo file with the following confirm your configuration.

C

You know: stop this from happening in the future, not needing to clean up by yourself. Then the user clicks the button. What happens next right, like they click the button, and then we inform them in some way that this thing is in progress. Also, do we inform others of this? Will the UI updates for everyone I think it should I would say what I expect to happen there is we press the button, the model or the call out message updates and it states hey?

C

This is in progress, it minimizes to a smaller one. So it isn't, as you know, intrusive, to the UI at.

B

Least, you don't see their ex perrolli in a way it's going to expire here this column. So it's not like.

C

What can I say that yeah.

B

It's field like in 23 hours, they say environment is going to be automatically stopped. So yeah, ok,.

C

So we get it for free, yep and.

B

C

This Stacey.

B

Somewhere somewhere here.

A

In disgust when the expiration is coming to happen, I suggest not to have it configurable just to set a time like in two days in a week a week, I think it's good, but I'd like to hear your opinion.

C

Yeah I'm fine with that we can call in the docs them see what they like.

A

Well, since this is like a two-year-old issue, I, don't think they're gonna mind waiting another week.

C

Yes sounds good. I'll update the issue in a little bit after this meeting.

A

Okay, I'll post the video to the NAS. It's unfiltered.

C

Thank you so much thanks. Thanks.