GitLab Pipeline Authoring group, 5 Oct 2020

Previous Meeting

⏯

youtube image

►

From YouTube: CI Pipeline Authoring 3 year vision discussion

Description

Michael (Developer Evangelist), Nadia (Product Designer) and Dov (Product Manager) meet to discuss the 3 year vision for GitLab CI Pipeline Authoring experience. They discuss such ideas as a guided onboarding, automated CI configuration, better templates, pipeline analytics, and more.

A

Yeah, so this is a meeting to discuss the pipeline authoring three year vision, um I'm the ux designer for popular authoring, product designer, and we have our product manager dove here and michael who has lots of great ideas that we hope to tap into last time. We covered lots of crown just discussed all kinds of different ideas around improving the ci workflow in general, but I thought that it could be useful for us to meet again and dive a little deeper into the pipeline authoring experience itself.

A

So, if you look at the category direction for pipeline authoring, you'll see that there are a few, um a few kind of problem areas that we really want to focus on, starting with just improving the um the experience for creating your ci configuration um where we we want to have more robust templates system and things like that. We want to have more automation and then also, we would like to have some analytics that actually show you how the pipelines are performing based on the configuration provide. Some automatic suggestions maybe even apply those suggestions automatically.

A

So these are just some ideas to kind of get us started and maybe spark some ideas and conversations around it. I'm not sure what would be the best process here like we can just chat and talk, or we can try to put something in the mural. I'm not sure what we thought.

B

About I think yeah I I can. I can maybe give some guidance on how we can proceed from here, like as nadia mentioned just recapping.

B

Our team goal is to make the authoring experience as easy as possible for our users and one of the ways we measure, that is by measuring the first time to the end pipeline. So we know that as a first time user, you know working with yammers. Trying to you know, walk your way around. Getting the first green pipeline can be can take a while, and we believe that, if you do it as quickly as possible, then your experience will be easier, smoother, more likely.

B

You will use our product so which leads us to the second way that we can measure ourselves is by adoption. How will we gain more users and like one of the ways we can do, that is to and like templating as nadia mentioned in a great onboarding experience? So if we can like maybe think about how we can improve those two ways, measurement time to first gain and in case the adoption of our users, this would be like can give us. Maybe spark some more ideas around that areas, because.

C

I think that's great to like focus on the specific areas. um I think I've told that last week already, but to iterate on it again um when starting with ci cd um there's a certain delay, which means you either get to know it succeed. So it does not and waiting for the runners to execute the job which can be like on gitlet.com.

C

Sometimes it gets delayed because there's a lot lots of things in the job queue, um but I would say many people try it out at first glance even before, maybe installing it themselves, they might just use a sandbox environment and when the time to first success or maybe even to first failure, is too long they might they might skip on it.

C

There are thoughts on maybe simulating the ci configuration and having some knowledge points inside in a way that you can also detect that.

C

The syntax is wrong, which is life lending for yaml. This is already in progress. Another way would be.

C

You have to have to have a quick sandbox environment which does not um provide you with a fake job. um It provides you with a simulation, basically a business impact simulation somehow um to provide you quick feedback. um This can get quite complex because the web parts or the gitlab server parts cannot.

C

I think they cannot execute everything, and specifically the runner needs to take the portions which are then executed. um I think there were some discussions also on the runner side um to execute a yaml file locally and do some sandbox testing and linting, and things like that.

C

So we always need to keep in mind that there are two components involved like the the gitlab server side and also the runner, which has some extra magic involved to execute the job stamp.

C

I'm thinking of- and this is like more like a long-term vision to have a virtual runner somehow or we can just throw in configuration and includes templars resolve and everything, and you get a simulated result from it, especially um you have. You should have the option to maybe adjust certain thresholds or parameters for the job, then so you can like play around and see ah when I changed it. This on the right hand, side the exit status made, might change or like the dependencies or duck, is really complicated to understand and get a grasp into.

C

Maybe this simulation idea could solve some parts in that area. um The other thing is, and this is somehow solved with the templates. We already have to add more fine granular templates, um like the the hello world, template in a way that we kind of create job templates or top preparation templates similar to the security sastas includes for autodevops.

C

So that you kind of have the exit one example I'm using in my workshops, you just run it. It does exit one, it outputs, something it's broken and and then you have kind of an inline comment or pop-up, which tells you hey. Now the pipeline is broken because we kind of forced it to break.

C

Please change it to exit zero and you have it green again, which is kind of the the personal success level, and then you dive it further with so I'm using a golang tanuki, which kind of prints an ascii art this could be could be a way of providing hello, world templates or easy learning templates, because the the other templates with golang and c plus plus and everything, is all right. It's a good starting point. um Still.

C

We could also add more best practice templates, like c plus, plus with c c cash, python, specific stuff, nodejs stuff. This, I think some of the templates haven't been updated for a while, um and we also need to make sure that our users understand um that there's a drop down with the templates and they should use it because sometimes I'm opening the web ide and asking what it what is inside yeah. I don't know what's next and um making the drop-down more interactive somehow with like.

C

I don't know how to really do it in the from the uk's perspective, um but there should be the user journey to say: hey, maybe I've detected, you are in a golan project right now and it also selects the golang ci template for me, because it knows that there is like 90 percent go code inside the repository.

C

um This doesn't work for projects which are new, um but there could be some like auto detection and helping the users on the first steps from from guessing what they are trying to do.

C

I think in the first iteration it needs us to define certain use cases, but at some point it could be some code, intelligence or some learning involved.

C

Now, I'm stopping my monologue.

C

I want to hear your thoughts.

B

Yeah, I think, first of all, I think it makes sense like the the way that the drop down is designed and I had a chat with nadia. I I look at how, for instance, gitlab action actions. Have this really nice template. They have like less templates that are than us, but it looks like really like so easy to use and consume, and you know once you select it, you know exactly which template you're working on, and you have like this suggestions of like different actions and different jobs that you can do based on the template.

B

You you you just selected, so this is something that I think like we would definitely need to need to revisit in terms of like detecting.

B

What's in your repo, this is also something that I think we we discussed a bit just knowing like figuring out uh what you have on your code and by that providing you with some suggestions. I think, like those two points, are like exactly uh the kind of thoughts that, like me and nadia, had uh in the last couple of weeks, just by looking at competitors and thinking about how can we improve our solution so validating? Basically, that makes very sense what you just said.

A

So another thing: um another thing that uh we've been um discussing not as much as the templates, but something that is narrator- is introducing analytics some kind of analytics and uh suggestions based on your pipeline configuration so that um not only for um yeah. So I we're not sure exactly what it could look like.

A

You know we could have uh reports or um and exactly what are the metrics that uh are important for the developers to see like what kind of analytics uh would you be looking for, for example, one of the metrics that we can track is um the speed um for the pipeline and that um it's very useful, because then we can. um We can talk about the cost, efficiency uh and so on.

A

But what are some other metrics that we we may want to look into and what could it look like and what what are the you know? What are the pain points? The main pain points that we should try to address there.

C

I think so the duration of the drops um is the first iteration of the first mvc I would try. um We are collecting that already somehow we make it available through the rest api, um it kind of needs, alerting and monitoring, but that that's kind of a different stage, a different feature set.

C

One way like creating a report is one way um we should let I have to say that um it's, it's uh frustrating to wait for uh ci, pipeline output and, if you so, when I'm building a new pipeline for example, which I did lately for the ci monitoring webcast, um I was just adding jobs and then I I was thinking- oh, let's add the code, quality uh job or the into the stage, and it took seven minutes and it was like yeah. It's not really it's blocking my whole pipeline. What can I do about it?

C

And I said: okay, maybe let's um at asynchronous dependencies like back, which makes the whole picture more complicated.

C

You want to see the critical path, which is something you currently can only do with dark, but we also want to have it visualized and on the other side it would be super nice or convenient when we have like certain data points of previous job rounds in the pipeline, um and we can suggest um improvements based on that to the configuration and say hey. This job is running for almost between five and ten minutes. um Maybe you want to inspect the script part. This looks kind of I wouldn't say fishy, but um it looks.

C

Maybe you want to improve something inside or another thing is um you're using a default docker image like dbn latest or dbm buster, or something like that and you always install update, update, upgrade install, and this takes a quite a long time in the before script section. um We could, for example, suggest to say hey: do you want to create your own docker image with the build tools, pre-installed or maybe use a different version to speed up the pipeline, um because the environment should be the same?

C

um The thing is these best practices are quite opinionated, so we will kind of need to find a way to make or to to get accepted in in a specific sense. So we shouldn't push our users to do it or force them and say: hey you. You definitely need to do that more. Like your suggestion and saying hey like the black clippy on the right hand, side, if you remember office, um hey, I found something to improve. Maybe you want to talk about it or maybe you want to jump on it.

C

um The best thing would be um like to have these two tips and the link to the documentation like the ci efficiency documentation, which we lately uh released, um but yeah.

C

It would be super nice to have that inside the editor and from a like from a viewpoint perspective, I would prefer a graphical editor in contrary to the yaml edit, because, especially when I'm starting with the configuration, I'm not I'm not like super confident with yammer, I need to do it because puppet and tyra, and afterwards everyone uses yummy right now, travis started it and um github actions also changed from the previous htcl format, which was kind of more object, orient oriented to yaml, again, which I think they they merged azure devops with the actions format.

C

Somehow um it's super convenient, but the problem is, um I cannot remember how to create variables. Is it like with a dash? Is it an array? Syntax? Is it something else? This is super complicated for me um and especially when I'm trying to explain it to others in a workshop or in a training. um I think the first you you don't really know. If there should be a space, there should be something else um in in my perfect world, it would be abstracted away and saying hey. This is like the first job.

C

This is like in you know, from visio or from an editor. It's like a code block from a flowchart um and you're, defining your job and you're, defining like the docker image, but it can be like something default. I don't care about it in the first iteration which command line. Do you want to execute, and this could be like echo exit one, for instance, for learning um and the visual or like?

C

I don't know, if should if it should be a separated thing or like a learning course inside the editor. Somehow this guide here and say: hey now, you've got one job.

C

The next step would be, let's create some stages, the creators create stages and add a secondary job and then have some sort of interactive learning, um and the other benefit I see with having a graphical editor is somehow um you can also attract people who are not like that deep into development, technical engineering devops whatever, because they just want to use, for example, gitlab pages- to deploy a website.

C

They don't care about the yaml configuration at all. I would say some do it's. It shouldn't pose a stereotype here, um but I could imagine that, like a drag and drop editor, it should be like connecting the pieces. All together um could be a really nice feature to also attract like design management or non non-technical teams. To say, hey, I want to use gitlab ci cd. I don't care how the syntax is in the background. Let's generate it with tools jsonnet, whatever is there. You can always edit the the final result later on.

C

um In my opinion, there should be some sort of abstraction removing all the default settings. We don't need um and just having the first success and then maybe a full embedded course which guides you to build a sample project with unit tests with best practices applied for caching for for other things and maybe even have a deployment attached, because git port is there or something else we can can. We can add um somehow um yeah, I'm always thinking in the mix of education workshop and making things easier. um So again,.

B

Yeah, but it's always good because, like you know, you speak about education and to some extent we educate our users on how to use ci.

B

So it doesn't matter if you're teaching this in the course or you're teaching that to a working experience or working environment yeah I, the graphic editor, is something obviously that is on the roadmap actually started to walking on. I. I almost see this as a crash course for to to yammer, because, like eventually, you need to know a little bit of yaml you don't have to but like you'll have those big building blocks and it's like you'll have like a basic and advanced settings. So, like the basic settings, it's just like drag and drop.

B

You don't need to know anything, but then you have like this advanced mode where you can magically see all the ammo see what like you're opening the hood. You see. What's underneath the hood and like, if you feel comfortable, you can just dive into and like change few things in the ammo. If you don't you just close it and say: okay, I'm fine with how it is for now. Let's run it and let's, let's make it, do its magic without even doing any changes to to the armor.

B

So that was like my thoughts, like general thoughts on like how this should be uh like. I, I don't see it as a completely decoupled form from from the ammo. So it's supposed to be some sort of an abstraction layer, but for the advanced user we still need to provide them with all capabilities because, like there are some things you'll be able to do with the visual builder.

B

But there are some things that you probably won't be able to do so, like, ideally like the most common things are the things that we'll do with the visual builder and we'll add, maybe we'll add, more and more features into that. But there is a limit to how much we can do uh even with like, even with the visual builder. So once you feel that you're more, like you learn and when you you know what you want to do, then you feel comfortable to like see.

B

What's behind the hood open, see the yamaha definition and then like walk your way around it.

C

Yeah, I think I think, having this combined or attached um to one single editor, where you can switch um this. This would be super super nice um also because um oftentimes when you debug something or you do troubleshooting, you need to understand the yaml syntax um for some to some extent, and um it should be. It should be a way of easy going, easy, starting, maybe even encouraging after you have created your first pipeline or first job to inspect the yammer syntax and say hey when you click this button.

C

This keyword is getting added some sort of interactive learning mode um to like so just to uh remember, I think I started with gitlab ci three years ago four years ago. I don't remember exactly, but I couldn't figure out what is a keyword on the first level or in the top level, and what is not, I was like which shop name should I should I pick? Should it be all tests? Should it be job one? Is it a defined syntax, and I think these are like pain, points or points?

C

You can abstract with a visual editor, but you also should encourage people to try it out and maybe even copy paste the yammer configuration from it.

C

So I could imagine that the developers or teams are going to use the editor to try things out, generate the yammer configuration file out of it and then copy paste it into a generic one or something like that, or maybe even just trying out, defining a job um and using that yammer for something else. So, like kind of a simulation mode, you have an editor with lint live linting, and then you use it for something else.

A

Maybe what do you mean use it for something else.

C

Yeah something else in in a way that they have their own, um like sandbox environment. They are trying things out in their account um and they are actually working on an open source project or within a customer namespace in a group, and they have a central ci cd configuration templating.

C

Because for security reasons you might have like a remote ci cd configuration, so it's not allowed in the project itself. It only is managed by approved people in a central group project, and they can make suggestions with. Oh, when we like have this job, it will run five times faster and the same. The central responsible for the cicd configuration can approve can approve it then, because sometimes I hear it from customers or users. They don't want the developers to manage the ci configuration by themselves inside the project.

C

They want to have it removed from a security perspective and have a central project managing this. And if you, if you don't really have access to the central project, then simulating and running it. You probably want to do that in your own local instance, or your in your own namespace, like I keep testing the gitlab features in my own namespace and then maybe repurposing it or creating a merge request for the upstream project.

A

That makes sense. Okay, thank you. uh That was very helpful.

A

um I I personally uh got some clarity on some of the ideas that we've been discussing and uh have some ideas on what it could look like, for instance, the visual builder that we're currently working on. um I think we can further uh develop the the designs that we have to be more educational and to have more guidance for the first time users, because I think uh it is very nice that it's interactive- I don't know, michael if you've seen it or not, but we're currently working on um this.

A

This visual, it's not drag and drop, but you can interact buttons, you can add stages and jobs and there's uh it's like the split view where you see the yaml and you see the visual builder at the same time, which is nice so there's the opportunity for us to have this kind of interactive educational experience for sure.

B

Unofficial gitlab tools, I can send you a link.

C

Yeah, I'd love to have a look, so my feedback could be overwhelming. But um let's just.

B

It's just like yeah, it's just like a like a bit of it's not even a bet. It's like a pc like pc, so I send it link on the on the chat here. Okay, so there is the visual pipeline builder, where you can actually walk a bit with it. So.

C

This makes me really this makes that makes me happy thanks, I would say we should have uh like coffee, chats or some calls or think bigs. I don't know like this. More often.

C

But I leave you with like when you need me: please invite me um to have a thought or sharing the latest updates or something else. um I can also try to connect uh more people from from our community or from from our coffee chats um to provide additional feedback, or maybe even uh gitlab trainers. I know so they can provide more feedback on the learning curve.

A

C

Would be very helpful.

A

All right, thank you, so much yeah, uh let's stay in touch and I'll share this uh recording in the ci channel and feel free to cross post wherever.

C

A

You have a good.

C

A

Bye. Thank you. Bye.