GitLab Release Stage, 1 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Progressive Delivery UX/PM meeting 2020-09-01

Description

Meeting to improve alignment between Product Management and UX for progressive Delivery

A

So I'm going to give a small intro.

B

A

What this issue is and how it started, so we just completed a competitor analysis on harness um and- and it raised some really interesting questions and linking this back to our previous uh competitive analysis and survey that we did with spinnaker.

A

It looks like they both of those tools, approach, cd in a very different manner than gitlab, and we're also losing business to both of those competitors. And the question is um so.

A

I had a hard time formulating the questions and that's where nadia came in to help me because the question, the big question is: why are we losing business and how can we get more customers to use cd within gitlab and not go to external tools, but going specifically into some of the their approaches to cd is that they treat cd as a first class citizen and the way that they do? That is with their several different methods, specifically for harness. There's two that come to mind. One is in the pipeline view.

A

They also show the jobs themselves um and you can like deep dive into it. It gives you on the job itself like full context. The logs. You can see the images in the issue itself here, but uh you can see like the logs in front of your face. You can see what's going on in the job, you can see metrics associated to it. So it's really really awesome.

A

You have like everything in one view and with us we kind of have this data, but you need to like click around and look for it and it's not very intuitive.

A

So that's one thing, and the second thing, which is something that I think vitica is working on, is that they allow you to do they kind of say. Cd is basically the same. It's like a template. You need to choose if you're doing canary, if you're doing a blue green, if you're doing some kind of uh other thing and then they just let you you know, select from a drop down what you want to do and enter the specific details.

A

So you don't have to build everything as one huge gitlab cieml file and just have the deploy as one job inside the the ci ammo. So the question is: do we can we treat cd as its own thing and not just a continuation of ci, and maybe these two approaches is the nice way to do it? um So that's kind of the background to the specific issue.

C

So it sounds like our hypothesis here is that our competitors, spinnaker and harness are providing a better I.e. First-Class citizen experience with cd, which is why we are losing our our cd business to those competitors. That's our hypothesis right! So what we need to do is we need to go out and prove that. So we want to understand what do people like, because just because they do that doesn't mean that they're doing it well or doesn't mean that there isn't room for improvement in what they're doing so. That's, oh yeah, but.

A

Since we already had some sorry since we have some work on spinnaker that we already did, we know that pipelines was one of the big advantages that people had mentioned in our previous survey, um so that kind of strengthens that.

C

Yeah yeah, I know I'm not saying like the hypothesis is wrong or anything it's just more of like making sure that we're covering it here in the discussion guide. Okay, so we're at so we're asking the questions that we need to in order to get the information so um yeah, I know totally so. We've got this spinnaker information, and now it sounds like we want to focus on harness.

C

Then this time around, to try to dive into more of the ins and outs and how people are using it. What do they like, what they don't like? What did they wish? It would do that it doesn't do and then compare and contrast that with gitlab.

A

So is that really the approach, or is our approach like just regardless to our competitors, even though, like we're losing business to them? Obviously, but how do we get cd to be its own individual thing? You yourself now that you finished the ci thingamajig, which I don't remember the name you were like. What's the difference between cis ncd and nadia was like looking for some docs on how to start with cd and everything's tied together with ci, it's all bundled together.

A

So the question is: is how do we take that component out and make it its own thing? I don't know if we actually need to focus on harness. In that sense, I just maybe yeah, maybe a good.

B

Question users need to to that, like, however, yeah like, maybe maybe they do, because ci and cd are really different and different types of organizations are using um each of them. But this is one of the questions that I would like to like ask, and this is maybe why it's good to take into account some of the hardness and spinnaker stuff is competition, but also like what you are, or they are saying, uh is that um I would like to understand actually for myself like, um like do people understand the difference?

B

Do they know which one do they need to select?

B

uh You know and those like and also experience of, setting up like pipelines, you know, like the templates that you were mentioning, I remember from exhibit labs years, that um those templating, where those help with uh configuring the pipelines, what big huge ideas, even in those years, so I it sounds like from company to company. That's a trend, that's something that people will think that people need.

B

I don't know if that's reality, because I don't remember us doing any meditation in those years, but this is what I would like to learn, because that sounds like making it's very simple also by the competitors, data that I've seen they have really nice visual ui that will allow you, okay, choose the type of the deployments that you're using and then we you don't even actually need to know what you need to do: ci or cd. We will guide you through that. Maybe we should not even bother, but you know.

D

Like yeah, that's that's the thing I wonder about as well like we have at one point like on one side, we have the the the question of like what are our competitors doing better like what is the attraction towards those products and then, on the other hand, we already kind of like think we need to separate those concepts and make them more accessible in some way or form.

D

But the reason for this once is unclear like, is that the reason why, like the competitor is better, maybe it isn't like, doesn't even do the users even care like it feels more like this kind of like?

D

Oh then, it is better in our own minds as conceptual things that we want to keep apart, like on one hand like I like the idea of this. That's the thing I just don't know if it makes sense to develop that idea.

A

So um so a few more background uh points um that came up regardless to this specific issue. So one of them is the fact that sometimes it's not the same people that are doing the ci bit and the cd bit so the cd bit is deployment right and deployment is production and production is sacred, so only some people have permissions to it, but ci is like more internal.

A

So it's for internal use cases and therefore the permissions schema is a little bit different and when I talk to enterprises- and this is more on jackie's side- but I've been to my share of uh conversations where this came up, where enterprises, specifically like banks and those kind of sectors, are looking for some way to um do job permissions.

A

So today we have like file permissions or or like specific roles, but if we would take the deploy part out of the ammo file, that would definitely solve the problem that the enterprises have in terms of the permission model, making it like safer.

D

I mean in a way users are already able to do this right with includes.

A

Are you talking about parent title, child pipeline.

D

uh No, no like say that you create like if you want to have separate permissions per file, and you would say each job has its own file.

A

D

A

It doesn't it's not part of the same yaml file, so then you need so the ammo file has a bunch of jobs right. It has your.

C

A

Your task you're deployed.

A

Okay, so the deploy is part of that same file right. The.

D

A

D

That say that you have three repositories and each repository contains uh a yaml file, the build file, the test file and the the deploy file. And then you have a fourth repository that includes all three ammo files into the main gitlab ciemo file. Then you could say towards the test. I I only want these developers to have access to that, the same for built to save for deploy and that way have permission management as a workaround. Basically,.

A

um I see what you're saying and it makes sense, but still the person who has permissions to the fourth repo needs permission to everything.

D

A

If they need to include it, I.

D

D

That is a interesting question.

B

I tell you this: if you two are already confused about this question, I can imagine how our users are not very clear.

C

So, what I'm? What I'm hearing is that we're very close to a solution, but we're not quite sure.

C

Like making it a first class citizen, I would agree that we need to do that, but I don't want. I feel like there's information missing. Just like dimitri said earlier. We need to know more. We need to understand, maybe more about how like, if I wanted to separate my cd and my ci, because they're due different people and nobody can touch production and, except for my anointed production people yeah, I'm going to use a different program because they only have access to that program. Is that the case? That's, my hypothesis.

C

Is that one of the cases that we could uncover by talking to people who do cd and ask them about how they're structured? Why do they use that tool and undercover? If we can work in the right permissions to how we have it structured?

C

Can we also offer them the same capabilities within our tool instead of having them use a separate tool altogether? So I think, there's still merit in talking with people who use competitive competitors in understanding. Why like? Why did you choose that one? Why do you continue to use that one? What are the things that work? Well for you, with with this particular tool, what are the things that you wish? It did that it doesn't do, and we can have good conversations around that.

C

We can also talk to people who use git lab either use it for cd or I don't know. I don't know if we should talk to people who don't use it for cd but would like to, um and maybe they use a competitor, I'm not sure that seems like it might be harder to find those people, but.

B

um That there are some quite some people that we talked with in the spin occur research, um gitlab.

D

B

And spinnaker from cd right, alright, so should should be hard, and I think we should do that is the.

A

Cd solution, it doesn't even.

C

A

B

Well, yeah yeah, but.

C

Do we want that.

B

Also, that's probably a question, probably I don't know but uh yeah. It also sounds to us like um lori, you said, uh like information is missing yeah. I think we even like need a little bit of problem validation like what exactly are we trying.

D

B

Why a bit of a generic research would be good here like well? I agree with you.

C

Yeah yeah definitely problem validation very a little bit more generative than our typical problem validation, because we there's more to uncover, I think and or you're probably right. You know everything that you've said you and dmitry have said: you're, probably right, but let's get some data behind it, just to make sure that we are on the right track and have the opportunity to ask more questions about other things that might help you to make some decisions for the ui and future wise later.

A

Yeah, I definitely agree. That's why I uh kind of asked for help on this one, because I think I don't even know what I'm trying to uncover. If that makes sense like, I know, there's something to uncover.

C

Yeah yeah, you did yeah, but and you do but it's it just looks different than normal, totally now you're doing fine. So do you want to shall we go through the discussion guide dimitri? Would you like to start at the top and go ahead and make changes as we talk through it, and then we can.

D

Yes, because I like, as I was typing with, I missed a lot of like oh, this is a hypothesis. Oh, this is a question I.

A

D

Going too fast, uh let me let me.

C

D

My screen, then we have at least some uh something to go about, should be this one.

D

um We got two times the same one all right, so I found the to start at the top a little bit the business decisions I was wondering about this, like this seems to, from my perspective point at like all right. Is there an opportunity? Canvas like what are the questions we have from a business perspective? This is more like this is the business decision. Why we're even doing this like.

A

I don't think media.

D

A

At this point, because we kind of don't even know the questions yet, but uh definitely along the way, I think the business decision here is: we have competitors, we're losing business to them. We want their money.

C

Yeah, so the business decision is that we'll uncover the changes that we need to make to our platform, our product to capture that lost business.

D

We want to recover you.

A

Can just write, show me the money and that's fine.

C

And you know ori, that's a good point about the opportunity campus. This is this is this is truly generative research like we don't really know what we're going to find out. So it's kind of hard to write the opportunity canvas at this stage. I could see after we do this, that you'll probably have everything that you need to write opportunity, campus yeah.

A

I don't even know like where, to start with with the problem description right, I think that's something that we're gonna well.

C

The problem is some we're doing something wrong or not enough and they're going to competitors right. That's that's really the problem and we just have to figure out what what it is that we're not doing or what it is that we're not we're doing wrong. So we can fix it. So we can start capturing some of that business.

D

Yeah all right, so it's basically kept your business uh because we're losing losing business like we want to recapture, because.

C

These are like you know,.

D

Investigate today's user experience and limitations of pipeline configuration deployment setup in gitlab, I don't know how that is like a business decision, or is it like? I'm wondering how this can be.

A

I have one thing to say that I learned a lot from the ci experience and that's that we found out that anyone that's using ci um in git lab is like 90 sure, to have retention next year, and we don't have that same number for cd. So I want to make sure that we can continue on stages, increase retention based on whatever comes up from here in terms of the business I'm not sure like.

A

If the features that we are going to figure out are going to be paid or free at this point, but I think it will definitely increase stickiness and retention.

D

That is increased retention similar to ci for cd features, going forward.

D

Capture additional cd business, where we are currently losing ground.

D

And then yeah, so I I think the rest is less business related right, like it's more like all right. How is what are the higher hypothesis in terms of the visual versus the other? uh You know like? Do we need to separate those things? I think that's all hypothesis based nothing of that is like business related. As far as I can see it all right onto our boxes.

D

um Our pipeline view, which was built for ci, is limiting users who are using them for releases. So this is, I think, a pretty good hypothesis, but I was saying here like what is it? What is the difference of using a pipeline for release versus mix usage versus only ci like when? Where do we? Where do we draw the line.

A

At the deployment.

D

Okay, deployment deployment, but also what I got out of the conversation we just had like five to ten minutes ago. It seems it is more permission related rather than conceptual, or do I take it.

A

I think permission is a specific use case. It's one.

B

Of the parts yeah well, some companies are just.

A

B

To do the cd today, it's like it's just one step too far. They want to like be manually validating what they're delivering to production. So some companies are even not ready to do cd today continuous delivery, like they don't want to deploy the productions, trade so yeah. There is part of the permission question, but yeah. It's also like what how far.

A

Yeah I'll even add to that um there's a lot of uh companies out there, including my previous one, where a we were deploying to on on-prem and not to cloud so continuous delivery didn't really exist. What you did was you created a build and you uploaded it to some cdn and told the users to take it from there. um So it's not really a cd and I'm sure, there's more companies.

A

You know with that same model um and the other one is that people are okay with automating everything as long as it's internal, but they still want those manual checks at the end. So.

B

Yeah, which is not an automated deployment anyway, then, if romania will check in so yeah.

A

C

This is, we should just have like. Maybe one would be good or or maybe two, because if you have a lot more like you should have a goal for each hypothesis and then like several objectives for each one of them. So it's almost like we're getting really specific here with our hypothesis hypotheses, where we don't really have to be. That specific here, like the hypothesis, is, if we add, features and increase the ease of use and the increase, the um uh trust there's another word there.

C

um In our cv feature, we will get more people using our cd feature. That is our hypothesis right. That is our reason to go and find out what are other people doing to do those things? How are they increasing trust and effectiveness and efficiency in in their cd option? And how can we do that? That's all we really have to say here.

B

Yeah, I agree with that. So.

C

Yeah demeter, I think those.

B

Two sub points are going into too much details because it depends on the type of the work we talk about, uh which would be which we will learn anyway in the process.

D

Yeah, but I I mean, like those um like it's it's nice- to have a general hypothesis that is encapsulating all the things that we want to know, but if it doesn't go into specifics, it's harder to kind of ask the right research questions.

C

You're right so.

D

C

That's why that doesn't go in the hypothesis, it's probably in our goal or our objective. So we want to understand you know. So maybe I see these as maybe go objectives because they are much more specific with the pipeline view like understand how our pipeline view limits, users like in which ways so they're they're, very much um objectives, though not not, um goals.

D

It's it's. You know it's like it's a thin line between hypothesis and and and goals.

C

I know you just have to think of like hypothesis as like your main topic, and then your goals are your paragraph topics and your objectives are the facts that are in each paragraph at least that's how they taught me how to write.

C

It was eighth grade. It was a very long time ago.

B

um I have to drop, uh who is, who will be the dri of this doc, just like on aligning on the next uh actions or, if any, help needed for me. Let me know I would love to collaborate, but.

A

I think I'll take it since uh dimitri is gonna, be off for a month anyway, yeah let.

B

Me know if you will need any help with setting up interviews with doing them or whatever uh uh I would love to support you here. Reid have to drop thanks. All thank you. Thank you. Thank you. Thank you. So much.

C

Yeah great, I can help too just let me know what you need and um if you want uh emily to help recruit we'll get that started too. Yeah cool cheers all right, bye, guys.

D

Bye, this is still here so.