GitLab Delivery Team, 16 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2020-11-16 Delivery team weekly

Description

Delivery Team weekly meeting

A

A

Hey everything is fine.

B

I hope this isn't an indication of how your day's going vira.

C

Something like that, yes,.

B

Oh fantastic, well, happy monday, everyone, um let's find my tab. So uh myra you have the first announcement.

C

uh Yeah, just a heads up that I will be off on friday because a national holiday.

B

Awesome um yeah, I just wanted to say uh well done everyone like super great teamwork, uh maintaining mttp like lots of um potential delays and uh yeah. I loved seeing the kind of discussion about like ah okay mttp was relatively unaffected, so um that is super great work.

B

So, thanks to all of you who jumped in and pushed out releases later earlier, helped unblock things and generally it was super awesome teamwork there, so yeah very nice to see um also and then uh final announcement uh engagement survey has opened up today, so you'll have an email coming from cultureamp, so that's open for the next three weeks. I believe please do fill that in your feedback is super valuable.

B

It's been based on the questions that typically get asked of people when they leave the company as part of the like exit um survey, but it doesn't really make like sense to only ask people who've left like how they felt like career progression was in the company and opportunities and things it's much more valuable to get that feedback from you all when you're still here and we can make changes so um yeah. Please do uh fill that in and so we can continue doing all the good stuff.

B

A

I have a question sorry.

B

A

um Are they sending the emails right now or is it already delivered because I received nothing? I.

B

Think they will be sending them from today, I'm not sure if they're all coming in one go or whether they are staggered, they possibly are staggered with a big three weeks so yeah it will be soon. I think.

B

Of course, so um incidents and deployments so uh lots going on here um and so maybe the best way um would be.

B

Yeah, actually, maybe job you're. Actually, your comment there. It's maybe the good thing so job opened up. This super awesome, uh like request, basically to to change the way we handle um workflows for incidents, which is great so job. Do you want to talk us through kind of what what you were thinking with this? um With this proposal.

D

Yeah, the proposal is just uh to help make it so that we're not spending so much time in the incident review meeting, which happens once or twice twice a week once in each time zone. uh The idea here is that uh we're only gonna do synchronous incident reviews for s1 issues and for everything else, there's just a new label that you can add to any incident that says review requested if it gets that label, then we'll discuss it synchronously and we're also discussing um sort of like a spin.

D

The wheel type thing where, once uh once during the week, we'll select a random s2 from the last on call shift to see whether it has like everything filled out properly, including corrective actions. We moved the corrective action section above the fold now, so I think the most important thing for us is to ensure that we have that filled out before we close the incident issue and that and it could be that there are no corrective actions. But if there aren't, we should probably state the reason why.

D

And I guess the major point here is that release managers are going to own the release, blocking incidents that are opened.

B

Yes, exactly so from uh so now, so it's quite a few instant things, I've kind of tried to outline roughly what this process looks like for us, so we have um uh we. I think we are pretty comfortable, uh raising up s2 incidents for anything blocking deployments. um We so those are the ones we will own.

B

Maybe that expands later, but for now we will we will own anything to do with uh deployments, and um so that means sort of working with whoever we need to to resolve the um incident so dev escalation or engineers or call or qa whoever it is, um and then the additional side is like the the post incident piece.

B

So we will own the process for getting those things done in the simplest cases, it will be just something we can review and make a decision on, or we maybe don't need to do anything, um and if we also do need something, then we'll also pull in the people. We need to get the incidents uh like have a uh all the steps like five wise. If we need to do five wise, um get the corrective actions and all of that, so we will be owning. The whole thing.

B

um Does anyone have any concerns about us taking this on.

A

I was thinking about this, so what concerns me here is that well during the things that, when the deployment is blocked as a release manager, your your interest is just to make sure that things get unlocked. So you have all the time that is needed to deal with this. What I'm not sure about is that there is a.

A

There are a lot of things that happens after so you are driving the the even if we don't do a real um retrospective on it, just figuring out corrective actions and things like that, and usually you as soon as the thing are unblocked. You have this moment where you want.

A

You have to continue the deployment, and so you have all this pressure, and now you have also these things to take care of, and if you don't do I mean probably, you can also say yeah I'll do this tomorrow, but we all know how it works when you start um say yeah tomorrow and then procrastination and you forget about what happened basically,.

B

Yeah, absolutely um we'll have to find a workflow. Basically that works to cover that, um like certainly it doesn't have to be immediate, like uh I think we have, I don't know what the time frame actually is on an s2 um but like within a reasonable length of time, um for those, so we'll have to find a way to fit it into our workflow, like as a release manager, we own the process.

B

It doesn't have to be the release manager who fills in all the details, like it's totally fine to delegate to somebody, but it needs to be a like delegate and make sure this thing actually happens.

B

I mean these things. Do I suppose, like we could have a look like I'm very open to hearing suggestions and ideas and what we do to make it easier so that we don't just forget about these things, um and if anyone has any preferences for how we, um you know like in your day-to-day stuff, like how do you, how do you keep on top of sort of the slightly less urgent but still important tasks.

C

So how do we we identify deployment related incidents? I guess it is not clear to me what kind of I mean they are related to the deployment, but if a deployment causes a regression, is that also a deployment related incident.

B

It is if you need to raise it, so I think that's the the simple definition right now is as a release manager. If you need to raise an incident, and so there'll be times when you do need to raise them so either you will want to block future deployments um in which case you'll trigger an incident, or you will need to alert the engineer on call and then you would have triggered an instant or also if you need any help from dev escalation.

B

The way that process works is you need an incident to be able to do the dev escalation process. So if, as a release manager, you have to raise an incident in order to move uh deployments forwards, then that would be a deployment related incident yeah just to keep it super simple, we'll just worry about the ones that we personally are raising.

D

Yeah, I think that I was kind of classifying them as incidents that don't affect our slos but affect our ability to deploy. But myra you have. A good point is like there's that other classification of incidents which come from regressions that do impact our slos and I think I think the assumption is that it doesn't include those because those would probably be owned by the eoc. But then again, maybe we would help with that. So I don't know.

B

Yeah, I think we should be definitely ready to help with those um and if we're asked to take them, we can, but I think just to make it super simple. So we don't have to do too much of the um incident analysis to be able to decide. Is it our incident or not? We for now we'll begin just until we're comfortable with everything we'll just begin. If we raise it and it's related to deployment we'll own, it.

A

So this means that if let's say we have um an issue with a deployment that happens after the deployment is completed, this is not owned by us. So I'm thinking something like well yeah not only feature regression but maybe yeah some metrics that just go skyrocketing after the deployment is completed.

A

Even if this this, because this will still prevent us from deploying the next version, but maybe the the the alert system detected and is not something that we detect on our own.

B

I think those ones won't be assumed that we will definitely take them, um but you may be asked to, and you're certainly welcome to volunteer to, um and that will go down very well, but it won't be the. um It won't, be the automatic assumption that we will.

B

Cool, does it make sense that the release manager is the person owning this some we need to have some sort of rotation of people owning it, but are people happy that it's the release manager.

A

I don't think they change that much about what what is actually the status quo right. It's just that now, you own the the whole process, but it was already the case. So if you trigger an incident- and you are a list manager- you're already paying attention to everything that is happening- and you can't call it done until it's yeah, you are able to promote to deploy production again right.

A

So it's just a matter that now you also have to do the homework after it and making sure that yeah, whoever was involved into it, just fills the root cause analysis or whatever collective actions are identified. I mean it's it wasn't it wasn't written down explicitly but was already part of the of the process before.

B

Yeah, I agree- and I say like please, you know like um the process is a little bit simpler. So, as chad's noted like we no longer have mandatory incident reviews, which is great um we a lot more of this- is just async um and then we'll do a check-in on various ones to make sure we're kind of like uh meeting expectations on that, um but do delegate. You know if you're feeling completely swamped out and you've got ten of these things, then, like definitely ask ask people to help you out on that one cool.

B

So then, just to summarize briefly, this means that what our process looks like in terms of incidents. This isn't a numbered list because they're not necessarily in order, but we have s2 instance that we raise for anything that blocks an incident at a deployment. So anything at all that you need to gain support on or prevent other deploys going is an s2 instant.

B

We then have engineering call approval for any active, s1, s2 s3 incidents before a deployment, then we can use the chat ups override command, which gives us an ability to log the reason- and that goes on the instant, uh the release issue. So we have that track for compliance, um and then we have all the other bits. We talk about good, good, good, okay, and this will say we'll continue. This will be a little bit of a learning experience for us all.

B

um So please, you know, share feedback, um suggest improvements for this one awesome so on to the next topic, which is kind of more exciting um holiday budget. So.

B

So good to get those.

D

I don't mean to be a nit picky nelson here I just like I, I had a truck I saw this, mr earlier and like I had trouble understanding what they were getting at because they they removed. They mentioned the 100 and then they removed the sentence about. Oh, you can spend this 100 by yourself or whatever you want. They removed all that and then they say 45 dollars. What's going on there, anyone who is drinking.

A

I I read this three or four times today before I came to a conclusion, so my understanding is that it used to be a thousand hundred dollars.

A

It will not be that's that same amount for this year, because e-groups will prepare as walk that will be mailed globally to everyone, but they still want you to have a kind of virtual party. So for this year you will have uh 45 dollars a low ones for the virtual party, plus you get a swag from the e-groups next year. If everything goes well, you would you will go back to the old amount.

D

I totally make sense now I wish they made that clear because, like yeah to me, it sounded like they just reduced it from 100 to 45, but then you mentioned the swag. It makes sense like they're, just I guess, allocating 55 dollars for swag and then giving me the remainder.

E

Is it known what kind of swag it is or is it like a pair of really expensive socks.

F

Including shipping, probably I think shipping would be the most expensive part so.

D

I don't really want swag honestly like it feels like such a wasted mail swag, all the way to slovenia.

E

So so I joke about that, but um I strongly suspect that the bag that I got for my five-year anniversary that I'll have to pay import taxes over it, because I got a letter from the post saying uh hey. You know this is stuck in customs because they didn't mention a price. You know. Can you please provide us info about it? So I emailed them a whole bunch of paperwork, and I haven't heard anything since and my status is still in ups stations. You know it's being transferred by the postal office.

E

So I'm like I kind of wait when the bill arrives.

A

So yorick I actually got a couple of years ago when I was running the hour of code. The first edition I got a bill from 600., 600 euros of import taxes for textbook and other random things that they shipped to me, and I give them a bunch of paperwork, a lot of stuff to say this is just um these are toys, or these are just gifts for kids.

A

They are going to play in this thing and so and they removed the bill also because they sent me some tanuki coin and- and they said, oh coin, so it's money- these people are mailing money just went crazy over it, but yeah. It happens.

B

I'm I'm quite sure if, if genuinely people don't want swag like, I think it would probably be also reasonable to open up a issue or something to have an opt-out. I think that's probably fine. If you genuinely don't want swag, so I don't think you'll get more holiday budget, though.

D

Yeah fair enough.

B

You could sell your swag. um Does I don't know? What did you do anything uh like delivery team focus like last year like? Does anyone want to do something this year.

E

So I I can't speak for those outside of the netherlands. I don't know uh within the netherlands what we usually did had a company-wide uh meet-up. We had them every month, but then we had a dinner during christmas and we just expense that as usual, but um I mean one: that's not gonna happen because of cofit and second, because all the changes around um reimbursement for beat ups and such uh as far as understand just if possible. At this point- and so I think this year will probably be unique in that sense,.

A

Yeah, I did the same here in florence with cristiano. We just went out and we had a nice lunch together and what the the we did- a virtual pizza thing with the delivery team during the first wave of pandemic, but it was kind of weird because it was kind of like something like breakfast time for someone um kind of mid-afternoon for others. So it was. I mean it's, it's really hard to spend 45 bucks in something that either is a breakfast or uh afternoon early lunch or whatever.

A

That's. Maybe the challenge.

B

What I'll do is, I will I'll put out a doodle and we can see if there's any sort of availability time dates that people like optional or completely optional. If you would like to do something- and um I mean I wonder if there's almost like it- doesn't- have to be 45 dollars a pizza right, like maybe there's some online thing, we could do or something like that. But um if anyone wants to do so, we can see if there's actually a time that we that works so I'll, send something out awesome.

B

Is there anything else anybody would like to bring.

B