Red Hat OpenShift Developer Experience Office Hours, 5 Oct 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Developer Experience Office Hour: New Pipelines features in the 4.9 OpenShift Console

Description

In this session, we'll do a deep dive into the new features available in the 4.9 Red Hat OpenShift Console when the OpenShift Pipelines Operator is installed. We are going to go over Tekton Hub Integration as well as Pipelines as Code. If we have additional time, we will review some features which were previously added in OpenShift 4.8

A

A

A

Good morning, good afternoon, good evening, wherever you're healing from welcome to another developer experience office hour, my name is chris short. I am host with the most and showrunner of red hat live streaming. I am joined by karthik and serena from the future serena. You want to do a little introduction, real, quick.

A

If you can find them, you.

B

Sure do yeah sorry hi. My name is serena nichols. I am considered serena from the future uh as we're, usually when I'm on here I'm talking about what's coming up in dev um experience in the console and I'm the product manager for openshift dev experience I hand it off to karthik.

C

uh Hello: everyone, I'm karthik, jeer, I'm a front-end engineer, working with dev console uit. So previously I've worked on some of the pieces around openshift, serverless and topology, and last two releases. I mostly focused on uh openshift pipelines. Ui set of things uh thanks chris and uh serena for having me here over to you.

A

No, that's it's great to have you karthik really. It is so sereno we're talking about some new stuff pipeline wise. We.

B

Are we are yeah so today we're going to do a deep dive into the features that are going to be available in four nine console when open pipelines, operators installed um we're gonna go over the tecton hub integration as well as pipelines as code, and then, if we have some extra time, we might do some more digging into some of the previous features. um But we're really happy to have karthik join us here, as he was one of the the lead for the the pipeline work in in the console.

B

So I think I'll just hand it right over to you. Karthik.

C

Sure let me share my screen.

B

A

Awesome, could you just increase the font a little bit on that or yeah the zoom, whatever thingy.

B

I think one thing too we wanted to discuss is like: why would users want techton hub to be integrated into into this kind of experience and when you're, using the pipeline builder you're able to add tasks to your pipeline currently or previous to four nine? It was either tasks that are in your name, space or cluster tasks that were on your um cluster. So this integration with text on hub allows you to grab anything that's in hub.com, so all the community tasks essentially and- um and now I currently I'll- share share it over to you.

C

Yeah before I get into the new look of the task, adding mechanism, so I'll uh talk a little bit more about. You know uh the previous experience for uh adding tasks to a pipeline builder. So previously we were having a simple drop down. uh If you click on add task, it will show a drop down that consists of all the task names and the badges in a drop down, and apart from the name and the badge, we don't show any other information to the user, so offer often using users will be able to.

C

uh You know, select the task which they uh uh sometimes they might choose a wrong task and uh realize later on and uh open the task and uh close it and again they have to repeat the process to select the proper task again. So the main goal of this uh tecton hub integration is uh to provide the user the ability to actually search the task and uh read through more information about the task and then add it to the pipeline builder.

C

So let me click on this add task now, so it opens up a quick search model and user can actually type on the search term. uh So the results are based on the best match found with against this uh search term.

C

So uh you can see uh all the tasks that are listed here, so the cluster tasks which are already available in the cluster- and we also have uh namespace tasks which are also available in the cluster, will be listed here.

C

Along with that, we are pulling all the tasks from the tecton hub interface, so the tecton hub library contains a lot of reusable tasks, so we are making use of that. We are pulling all the tasks into the pipeline builder, so user don't have to switch context between the openshift and tekton hub.

C

So, as you can see the badge here that differentiates between the tasks that are already in cluster and if you click on that, you can see it is already installed and we can see the description of the task and the categories and the tags for that particular task, and we can also see a badge here which says that red dad provided and community provided.

C

So this will differentiate between the tecton hub and the redact provided tasks. So let's take this task as an example, uh we haven't added this openshift client task into our namespace, so the button here says install and add.

C

So we also show this version drop down, uh wherein user can choose a version and install it to their cluster. So let me choose 0.1 and click on install and add. So, if you have noticed it showed a loading decorator for a second, so it was installing the task to the cluster.

C

So till that time you can't perform any other action on that task, so a loading decorator will be shown there and uh once the task is added to the pipeline builder, uh you can click on the task and uh add the parameters or the when expressions input, resources and all the uh configurable stuff and uh the.

B

Question just a quick question on that um when you're, adding that when you're adding a task here- you mentioned this, but we don't have the capability of adding something as a cluster task right, it's right now we only have the ability to add it in the current namespace.

C

Yes, correct: that is okay, so, okay, once we once we install it, it's going to, uh you know, add it to only this namespace and yeah. Okay, thank you, and one more thing to highlight here is previously: we were having the drop down list. So if user selects the wrong task, they they have to click click it again and they have to remove it from here and uh they need to change it.

C

But now, if you hover on this task- and it has this delete icon, so you can click on that and uh quickly change the structure of the task.

D

C

C

And one more thing I want to show here is: uh we are showing this installed button here or a badge here and uh whatever is installed in the cluster will be pre-selected here and you click on it and the whatever is installed will have the check mark and if the user wants to upgrade the task, so we'll show them a warning message because by updating this task, this will replace all the references in this particular namespace.

C

So all the pipelines that are using this task will get updated. So we are showing this error message or warning to the user, and once they hit on update oops.

C

Let me refresh.

B

It's not a demo without something little going on.

A

Yeah yep: it's not live tv unless something breaks right.

B

C

Yeah, so actually we have the ability to uh update the task and uh add uh the task to the pipeline builder, and uh one more thing I wanted to show here is uh so, as we are composing this pipeline through this pipeline builder, there is always a chance that you know the user wants to look at what is the underlying structure, so uh this model blocks the view of the underlying pipeline structure. So we are also.

C

I have that we have added this uh drag capability, so you can drag the model to a different location in your screen and you can also resize based on the size of your window. You, if you want to see more information and you can resize this.

C

B

Great so question around when you do the search, um what are when you're doing this search? What are you actually matching to in the back right.

C

So we are matching to all the text uh that includes the description, category tags and uh and then.

B

C

Name itself: okay, yeah and all the metadata will be matched against this search term. Excellent.

B

B

So that's it's actually interesting, because we we implemented the tecton hub integration, but in addition to that, we really made the experience for pipeline building so much better right because there's a discovery mechanism for your tasks, whereas before like you were talking, we just had a drop down. If you didn't know what the name of that task did you're kind of in an unknown state right, um so that's great awesome!

B

Okay, um I guess one other question I would have is what would happen if you, you installed a bunch of client here in this name, space did version .2. I think it was. If I remember right now, if you go to another namespace and want to reference openshift client, what would happen so if you yeah.

C

Okay, so I I have installed openshift client on the other namespace. So if I try to do that here in the new namespace, uh it will uh again show me install and add because this.

B

C

Doesn't have that.

B

Now is it possible on a cluster, to have two different versions of the same task.

C

Not in the name same namespace.

B

But in separate namespaces or are we I don't remember what happened with that? Yeah.

C

Because we don't change the name of the task, the name of the task is still the same and we are.

D

C

Create it as a namespace task, so the name will be unique, so the versions get updated. So we can't really uh have two different versions in the same name: space. That is a known limitation.

B

But can you have two, can I have v one in name space, one and v, two, my name space two or am I constrained? Okay, awesome, okay, great excellent.

B

While we're here, do you want to go over some of the the when I know we had talked originally about doing this later, but while we're in the pipeline builder, do you want to kind of go over some of the web expressions in the final tasks, maybe as well.

C

Yeah sure so let me compose a pipeline here. I'll choose a couple of tasks.

C

So so, when expressions are a way to, you know actually uh guard a particular task. So if you want this task to be executed on a condition, uh then when expressions are really helpful here so the moment I clicked on add when expression uh diamond symbol popped up here, and uh that says uh this task is guarded by even expression, and it also guides the user to you know, fill out the form.

C

uh So if you, if you open it- and uh if you don't fill out the form and the create button in the pipeline builder will be disabled and the error decorator here will guide the user to actually fill this form section. So here there are three uh fields that we need to fill.

C

One is input and the operator is nothing but in or not in its equals or not equals kind of uh and values, so input can be a parameter or the name of the pipeline, or it could be any variable that is available in this particular pipeline.

C

So we can also do dollar and parentheses to open uh the another code assistance feature here, so this will uh help us to fill in this form. So here I've chosen uh context, dot pipeline run, name so, and I can also use control plus space bar that opens up the same code assistance feature and here it shows all the context of the pipeline. So if I have parameters, let's say so I'll have a default value to stage, and I want to run this task only on the stage environment.

C

Then I can come here and click on control plus space bar. This will give me the params that I can use. So I I'll click on that and I can choose in and.

D

C

I'll give as maybe prod so I need to run this task only on prod.

C

So once I'm happy with the when expression I can now hit on create, so the visualization also has that, uh when expression decorator in front of that particular task.

C

Then I can start the pipeline and the parameters will be asked, so user can say uh prod. So let me first run it with the stage environment, so this task expression will fail and this task should be skipped.

A

I, like I, like that, like the way we've integrated this, it makes it very easy to pull those environment variables in that's very handy.

D

C

B

C

The other thing, too, is.

B

Like the just yeah, I was going to ask you if you go over the colors and the symbols kind of the icons that are shown there.

C

Yeah, so this task is kept, so it is grayed out and says when expression was not met. So if I try to.

C

Start the pipeline again with prod.

C

So this is executing the task because uh the win expression is met.

B

The other thing I'm just going to mention is that in each one of those little bubbles for each of the tasks karthik, we have like the underline.

C

B

I think that represents the number of steps that are in that task.

C

Yes, so those steps are shown inside this tooltip. Yes,.

B

Nice, okay, so for people who haven't seen it before just trying to give more more info there, so there can be multiples and then.

C

B

Are also the lines underneath the colored, as is the tooltip, also indicates the status of each one of those steps in the tasks as you're doing it right. So that's.

C

Great so, along with the param, not only the parameters, it can uh also uh have variables like workspaces, so results context. So if I have a workspace.

C

So this code assistance will reference that as well.

C

And we have this code assistance feature in all the parameters, so even here, if we want to change it, we can hit ctrl space bar that opens up this one.

C

D

C

Add this then expression to finally tasks as well.

C

So exact same procedure.

C

B

And just for an overview of the finally tasks, those those are kind of executed, no matter what happens to the previous steps, um previous tasks in the pipeline right. So one might somebody use that like for cleanup, possibly or something like that.

C

Yeah, possibly uh you know, maybe at the end they need to clean up a workspace or send some notifications, slack notifications to some other subsystem or users. So so this finally tasks will run uh no matter what happens to the regular pipeline tasks.

C

So it's guaranteed to be executed, but uh we can even guard that task with ex when expressions so we can conditionally run them.

D

B

B

Okay, um that's great chris, do you have any of there? I think we're.

A

B

To move on to pipelines as code- okay, great um yeah, so that was great. I think that was a great overview thanks carla for sharing that. I think the next piece that we wanted to go over was around uh pipelines is code right um which allows the ability to store delivery pipelines in a git repo as the source of truth and execute pipelines um from there right and lots of people have been.

B

I know waiting for this feature and- and it's already supported by openshift pipelines- it's just something that we are following up with integration into the console.

A

Question from chat can I go back and forth between the pipeline builder and the yaml view, like it'll pick up the changes in yaml? Yes, okay! So that's what I figured.

C

Okay, uh so I have already installed pipeliners code in this cluster, so just for demo purpose, and uh so this is the repository where you can see the there is a walk-through video created by ishmael and those who wants to try it. They can watch this video and there is also an installation guide. I followed this one so so we can create this. uh You know we can install this pipeline as code using this yaml file and then it creates a bunch of resources into your pipelines as code or namespace.

C

And we need to get the event listener, so the pipeline, as code will have a event listener that is already exposed.

C

So we can use that url to configure the web hook in the github repository.

A

I'm going to share this github repo with everybody real quick, oh, you already did serena. Thank you.

C

So I have used this url, and uh so I already created a github app that for testing pipeline s code, so in that github app, I've added the webhook urls here and also in my repository webhook.

C

So this is the repository I'm gonna use and in the settings I have added the same url event: listener url as a web hook. Here.

C

And if you look at the repository, it has dot tecton folder that houses the pipeline. That needs to be run on this event, so it has a pull request: dot yaml file- that is basically a pipeline run that does embedded pipeline spec is there so this pipeline has a simple three tasks: just echoing out some values, so fetch build and deploy.

C

B

You mind if I share: do you mind if I share your your repo as well of your demo? App is that okay.

C

B

Copied it okay, great thanks just in case people want to try it on their own sure.

C

And I've already created the secrets uh in this cluster. So all I need to do here is create a repository in the namespace, where I want to create this pipeline run. So in my case, I'll go to rs dev.

C

And click on pipelines, and previously we were having only pipelines list, but now, once the pipeline s code is installed now you will have a tabbed view and this will contain pipelines and the list of repositories and the create button here will also let you create pipeline and repositories.

C

So I'll click on the repository I'll paste, the repository cr.

C

So this repository cr actually ties the github repo and your namespace in the cluster and the branch and the even type.

C

So let me change the.

C

Github url and the namespace.

C

So and I'll leave this even type to pull request, so I want to trigger a pipeline run whenever a pull request is created well hit on create.

C

This takes me to the.

C

Repository details page, it has the basic details about the repository and we can change the labels annotations and it gives you the link to the github repo and what branch and pull request or even type- and it has the ammo view and all the pipelines that are created through this repository that will be linked here.

C

So now let me go back to my github repository and I'll create a new file.

A

I like how that said, patch three you've done this a few times.

C

Just two times.

A

All right merged closed.

C

Yeah I'm just I should take something.

B

So now that pipeline run should become, you should see a pipeline run automatically generated here.

D

Once the controller picks up those changes right.

B

And I love to say this all the time. Clearly he did a demo prep twice and it worked the two times before so yeah.

A

We know it worked twice.

E

And we make it work a third time.

A

Just on the console side hit the details or something just to make the page yeah, okay, cool.

C

Let's dab name space, put requests.

C

Okay, one check failed.

A

Oh, that's your previous work.

C

D

Oh, it's in progress.

B

Okay, it's in progress.

B

B

Well, chris, you said you said that people like to watch us have to resolve issues when we're demoing. So.

B

Is doing a great job yeah.

A

I mean this is all in the operator right like it's just got to pick up the changes.

A

You didn't write the wrong branch down or something did you just have a curiosity.

B

A

Yeah wow, I don't know what the deal is there.

C

Okay, so I think I tried it previously with uh another.

A

Four seconds ago, last round: oh huh it's that page now.

B

Just yeah there.

A

B

A

Okay, so it just happened: ta-da.

B

Oh no, this is his previous. Oh.

A

This is a different. This is a different.

B

Project yeah yeah.

C

Let me try again.

C

Okay, let me see what what.

A

Did you need to squash that.

C

Books are successful.

D

D

C

So if we want to debug, then uh so pipeline says the code is the controller, and this is the event listener. You can view the logs.

A

It says interceptor, stop triggering process.

C

Let me see if it is creating in some other name space ah see it got created here.

A

Oh yeah, where is here one.

C

Minute so this.

A

C

If the name space is not matching so.

B

Oh so rhdev, maybe there was a hyphen in it or not. A hyphen.

A

So jm and chat says the main branch on the github repo is named mastered. Your pipeline is expecting a pr on main.

A

Branch main here, though, is that master? It's just that's just the comment master. It is main there.

E

Yeah from master.

C

It isn't me so.

A

Yeah, it should, in theory, work.

C

Yeah it is creating the pipeline runs, but uh not in this namespace. This creating.

B

C

Pipeline says code.

B

How about in the ammo for this did you have our hd or did you have rh hyphen dove.

E

Weird very interesting.

A

I wonder if that's maybe a github symptom where it's like mane or mastered it's confused, but it built just elsewhere.

B

But it worked on the other one right and even.

A

B

You, like you, have it in your karthik project. It was working.

C

Properly yeah, it was work, it was working. Yes,.

B

C

B

Yeah, so if you go.

A

In there and go.

B

To pipeline runs, you can see those that were generated based on those. So it's interesting something.

A

Dev preview folks.

B

There's a typo somewhere that we just.

C

B

Catch me probably more likely right exactly.

A

Yeah reminds me to throw up the future stuff that we have.

B

D

B

So can we talk a little bit about what the advantage of um pipelines this code would be rather than having pipeline quote-unquote resources created and using the pipeline builder in your cluster.

B

Like what would the advantage of that be for somebody just because everything's got everything's in git, so it's single source source of truth, so you've got your pipeline definitions just stored in git.

B

And can you go again back to the pipelines, nav item.

B

If you have pipelines this code created you'll see that they see those repositories here, if you don't, if you have pipelines that were generated either in the ammo or through the pipeline builder, you'd find them in the pipelines. Tab.

C

B

Yeah and then either way, could you do a search and just show how you can look at all of your pipeline resources together, like through the nav um and then maybe even add pipeline resources to to your I'm sorry. So if you click on search in the nav and then just look for pipeline resources, plr, okay and yeah, uh the resource ones, yeah, the um I'm sorry the runs. Sorry yeah yeah.

C

Okay, you want me to add it to the navigation. I.

B

Was gonna have yeah add the pipeline runs to the nav yeah.

C

B

Then, if you just click on that and nav now it's easy to see all those pipeline runs quickly, but those.

C

B

Be showing yeah, and that would also be showing you pipeline, runs that are associated with a pipeline. That's from git or one that you built from pipeline builder yep. So.

A

Can you clone a pipeline as a question from chad like if you're.

C

So weak in the builder.

A

C

Yeah we can, I mean cloning, a pipeline. Okay,.

A

So yeah, I think you.

C

A

Pull out the yaml.

C

A

Reinstantiate it someplace else, that'd be the.

B

A

You know the way.

B

A

It right like pull from the repo, um but I see what you're saying you want to be able to like reuse these in cluster without doing too much.

B

Yeah, it would be interesting what the use cases is looking for because we have talked about should you be able to, and I think maybe this is what he's going to say, I'm not sure but start off with a new. I want to create a new pipeline based off of one that exists.

A

Today, exactly right.

B

So pull in and then I'm going to make some additions it'd be curious to know. If that's what he's looking for.

A

Because we do rules whatever yeah.

B

Yeah yeah because we do have that in the backlog so.

A

B

Are interested in that definitely do like, plus one or something.

A

Yeah, well you let us know your use case a little bit and uh maybe that gets bumped up. I don't know. I can't I'm not the dictator of that.

B

Well, it helps to know who's looking for.

A

B

For sure right so yeah, okay,.

A

So yeah it does sound like he's trying to clone something and modify as needed. So yeah.

B

Yeah and again so karthik, can you just do another quick, so we're saying this is an openshift 4.9. It will be coming out what version of openshift pipelines would we have at that point? Do you remember? I don't remember off the top.

C

1.6 will be, uh the version would be the version.

B

Okay! Okay! Okay! So that's when we would get that okay awesome, so that I think that kind of does it for the the kind of the deep dive in the ui piece around pipelines.

B

um I don't know so.

A

Waleed said that is the use case for creating based on an existing one, yeah, so perfect. Okay, that's great see where that sits in the backlog. Maybe.

B

Yeah I'll share that back with siama too, because product manager for pipelines itself itself, so we coordinate as priorities and how to get things into the console. But it's great to to know that somebody's looking for that. Somebody else is looking for that for sure.

A

I mean I could definitely see that in organizations across the globe, that being something right like that comes from like ansible, I feel, like you know, like the.

B

A

To do something and tweak it and then run it. You know.

B

B

Karthik are there any other, like other usability tests, that I mean usability, features that we could talk about.

C

So I think we have uh covered all the winners.

B

Oh everything before then yeah, the other thing that people have asked for that we don't have done yet is when you're looking at the pipeline logs there's been some requests around having that be colored.

A

B

A

Oh, like the output yeah.

B

Yeah yeah, because right now everything's just uh you know single color, um but it would also be interesting for anybody to pipe in through the through the um chat. If that's something that they feel is um so.

C

This output was there in 3.11, but uh info.x. We don't have that, so we already have a feature: request and pattern flight team working on uh building this log viewer with the support of nc colors in it. So hopefully we can have that soon. Yeah cool.

B

A

And then I think I always appreciate colors and logs. So it's just me yeah.

B

Yeah, I think that the hope there would be that all of our logs in openshift, not only the pipeline log viewer, would all be utilizing that same pattern fly component.

A

Exactly familiar pattern fly is the: what do we want to call it? The the template toolkit.

B

A

The open source.

B

Resource design system.

A

There you go open source designs.

B

Yeah right, like think of.

A

It as theme plus plus for your website, if you're used to like wordpress, applying a theme. This that's basically the framework on top of which we build this ui uh questions about analytics on pipeline execution.

B

We actually do have metrics today.

A

Nice, okay, cool.

B

Now, since this is probably not a long living cluster, I'm not sure how much we'll see here for metrics, but we can give.

C

B

C

Yeah, the metrics are there for uh the regular pipelines.

C

A

That next tab is here.

C

A

Not for pipelines is code.

B

A

B

Maybe kick off a few right.

D

Right right, yeah.

B

And then they're, obviously.

E

The pipes right.

A

There we go all right.

E

It takes a couple of seconds.

B

A

Yeah, it's a 30-second refresh too.

E

A

There we go there, we go stuff's.

D

A

So yeah you do have some analytical data there. I'm curious we'll lead, uh so he well he'd responded, for example, to see if there are some abnormally slow executions compared to previous.

B

A

Right, like that's a good point, so.

B

That in the bottom left there is pipeline, run duration. So that's for the entire pipeline run, so you would be able to see that from a historical perspective right like each one of those.

A

And it even breaks in which task yeah, that's cool.

C

Yeah and- and you can you can, you know- hide a task and see only right through the legends yeah. That's really.

B

A

It is yeah because then you just focus on the problem here. Yeah.

B

And that's all if I'm not mistaken, these are all pattern fly components too, so you know we should be having that same functionality in in any chart that you're, seeing throughout the console, which is great yeah, so I'll leave. It looks like.

B

It looks like this is: the bottom left chart is giving you what you're looking for. I think.

A

um Yeah that's for the entire pipeline and then each task has one. So that's good too. I think you're hitting that nail twice. Basically.

B

A

B

Notice, one thing here on the left is just that: maybe we don't go into enough granularity because it looks like we're saying zero minutes yet right, but that's probably a nice enhancement on our side that we could work on yeah um right because that's 30 seconds. But it's still saying a minute.

A

B

Minutes, yeah, yeah yeah, that's.

A

All yeah it looks like this is what elite's looking for yeah.

B

Right awesome awesome, that's great good! All right! Well, I think that's all we have today chris well.

A

That's awesome. I mean this is good.

B

A

I can't wait for this to ga right, like it's going to make.

B

A

Lives easier, um so thank you serena. So much. Thank you karthik for the wonderful demo and thank you audience for watching uh coming up. Next on the show on the channel, we have call for code for racial justice. Talking about the truth group truth loop project, uh which is an open source project to make people aware of legal ramifications and personal responsibilities.

A

That kind of thing um so tune in for that and then immediately after we will have red hat advanced cluster management presents, and the title of this show is a little bit of everything, because it literally is a little bit of everything from the rackham team. So please stick around for the next couple hours and uh we'll see you on the next live stream.

A

B

E