Continuous Delivery Foundation cdCon 2020, 20 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Screwdriver UI Walkthrough - Alan Dong, Verizon Media

Description

Screwdriver UI Walkthrough - Alan Dong, Verizon Media

In this session, it will cover the fundamental parts of Screwdriver regards 1) what is a pipeline 2) how to use a screwdriver to setup a pipeline from scratch 3) integrate with SCM (i.e. GitHub) 4) setup collections for personal preferences 5) how to get involved with Screwdriver.cd to get help and contribute back to the community It would be informative, and please join.

A

Hi everybody: this is alan doan. Today we're going to talk about screwdriver ui in this session, I'm going to walk you through the various features that screwdriver ui has so, let's get to it.

A

B

What is screwdriver screwdriver is an open source product built for continuous integration and continuous delivery. It's built on top of open source technology like kubernetes docker, node.js golan. If you're interested in there is another session, my teammates jason and tiffany host to give you a deeper dive into the architecture, design level of screwdriver.

B

Screwdriver follows the standard ci life cycle, just a quick touch up on what that is. Screwdriver is event driven. It listens to events from scm.

B

Scm means source control, management like github and bitbucket, for example, once a commit event is sent from webhook commit event will trigger the whole workflow graph to start.

B

And this is how you can write the configuration code in a yaml file that screwdriver engine can understand this graph. It shows you that it listens to both code, commit and pr merges to the branch that your ripple defaults to.

B

Now, let's focus on the screwdriver ui and I'm going to show you the first feature about user.

B

Onboarding, let's see screwdriver in action.

B

And now we're on a screwdriver ui, so first we need to sign in right now we allow both signs from your designated sems like github, and you can also sign in as a guest access.

B

I already have a login user with github, so I'm going to click on sign with github.

A

B

Is sso so after sso from github now we're logged in here.

B

This is what my dashboard looked like after I logged in and for brand new users. We can use this guest access to see what that would be look like, so we're showing the instruction to help you to get started. You can fork this repo, which is here and create pipeline with your ripple, so you can use a fork that, or I have created a.

A

Github report just for this purpose and let's take a look.

B

It's a very plain node.js application against a start and test, so I'm going to get this on boarded so for every screwdriver repository.

B

So for every repository you want to be on board and you have to have a screwdriver.yaml file which in this case it does not have. Let's take a look at this here. You'll see that and let's see how we can add that first, if you're familiar with screwdriver already, you can always create your own screwdriver yamo and commit.

B

But in this case we're gonna take advantage.

A

B

This script pipeline- oh, let me log in with my github repo.

A

And then click create pipeline, I'm going to copy.

B

This github url and create it here, and I have this all here- I will create my screwdriver yamu yourself. That means you're gonna, create your yama yourself or you can start with a node.js pipeline.

B

We're gonna use this domain, okay, so once we have that we can take a look or preview. What pipeline settings would be before even we jump start that so on this page, we're going to give you more detailed. What this template is, who is maintaining that.

B

But for now we can ignore that, let's click on.

A

B

Pipeline, so behind the scenes, what is going to happen is it is creating dr to your guitar people and all all you need to do. The user is to do and just uh merge it. Let's take a look what's inside this pr, the pr is adding a screwdriver.yaml file to a ripple with the following changes. You might not know or familiar with what those syntax are. You can take a look at here.

B

What's screwdriver, we.

A

Hover over one creating pipeline- and we also have this.

B

Quick start guide, we developed to help you, but nevertheless we will land you on this page, we'll have a detailed normal configuration.

A

And now, let's project.

B

After merging the pr and now we're back to this pipeline page- and we can see, give you some information we'll commit message bill status committers. When did they start? How long did last to result in this current status? Now we can call that repository has been onboarded to screwdriver.

B

Having a single pipeline is fun, so let's have more fun by creating nasty pipelines. So in here I've created two pipelines and let's take a look at their screwdriver.yama file.

B

So the first pipeline is very simple: just have a main job. The second pipeline, it's interesting, because now it's a single job that depends on the first pipeline's main job.

B

So let's take a look that in action, so I have two pipelines set up already here and that's how I get this pipeline number 5868 put into the second pipeline 5868 and the job name.

B

So if I trigger here by click, start from the first pipeline once that finished.

B

It will trigger this job dependent job for the second pipeline.

B

It may take a little while so that has finished. Let's take a look at here.

B

As you can see, I actually trigger the second pipeline's dependent job, which depends on the first pipeline's job name main.

B

For developer experience, we have also created this high trigger and show trigger toggle button. You can click here to see what downstream pipeline this job will trigger. Let's take a look at screwdriver dashboard again. On top banner, we have collection, a list of tools, quick pipeline search, a list of documents, a list of help and and personal setting. On the left side, we have a list of collections I have created and I'm going to show you how to create a collection, for example. Right now I have created collection, name demo and this for demo.

B

When I click on save and we have demo now- we can add the one I just created.

B

And now we have a collection that only have the pipeline that you want it to be.

B

You can also remove or add more you get the idea.

B

Next, let's dive into one pipeline from screwdriver collection, screwdriver ui pipeline right here, you'll see the latest commit branch of this pipeline and duration start date and will give you last 20 days of the matrix right here as preview.

B

In this page, let's take a look at the matrix by default. We show you one week, let's take a look month and if we only want to see success, which means successful builds.

B

We can do that in here and now we're seeing aggregated over last 30 days.

B

We have events, breakdown, build time, breakdown per events. We can also break down by a job. Let's see, publish job, we can also filter by steps.

B

This matrix will help you to look over your pipeline, see how your pipeline is behaving from 6 hours to 6 months.

B

On the right side of the pipeline, we can see two tabs events and pull requests under the pull requests is showing the number of pr's that are currently open under events. We're showing a list of events.

B

You can see the history of events in here. You can also share the event url by clicking on that. The url will update it and you can share that with your co-workers. Let's take a look at other tabs. We have events, secrets, option and matrix.

B

Let's click on the secret tab.

B

In the secret tab, we can see a list of secrets that you defined in your screwdriver diamond. Take an example. Here you can see, there are docker trigger git key github token ks token. Those are also and must be added to your screwdriver.yaml file in order to use it.

B

Let's take another screwdriver pipelines, example for secrets like here. I have defined a secret called this token, and this is screwdriver. That indicates this token and this, how I use that as an environment variable I'm updating the secret for key this token, and let's toggle this, I button to show and hide the value I just type default token, and once I click on update, it has changed. Let's go back to events.

B

And let's trigger the pipeline to see what's the value of that secret, you probably should not do that during your production, but this is a demo. It shows you the capability of how to do that.

B

Once this pipeline job has finished, we can see it outputs. The token value I just typed, which is default token.

B

Let's take a look at options. Tab in this options tab we're allowing user to change the pipeline that points to the git repository. You can change your git repository and you can change to the branch name of the git repository.

B

You can also enable or disable the jobs by disable the jobs that also disable the scheduled time for this job, for example, freeze window and periodical jobs.

B

And you'll also have to give a reason.

B

You can refresh the web hooks by click on this button or pull requests or pipelines.

B

Caching is another key point that, unfortunately, for a time, duration, which would not be covered in this session, but feel free to ask me during the q a session.

B

You can also remove this pipeline by clicking on the trash icon.

B

Let's click on the top of the event on this page, we're showing you a workflow graph. If you click here, if you click here, you will see vertical, which shows a list of jobs in here it will show the job name. You can sort them in here. You will see the last five jobs, duration, start time, coverage and action. You can start and stop and restart the job. You can also click on refresh to see the current status. If your pipeline is running.

B

You can also navigate to see you can also navigate by clicking on the dot.

B

Next, I'm going to show you how to preview your screen driver yamo in this validator tool under two tabs, so you can make sure you pass the syntax check before you commit to scm.

B

In here, I'm pasting: a scrap yaml configuration by using a special annotation called screwdriver.cd, built periodically to run schedule jobs. This is a crown notation that ensures this job will run on 12 p.m. Every sunday you can look at the syntax in this page and see the implementation details on this blog.

B

Conversely, we have another annotation called freeze window to ensure your job will not be wrong in given window time. You can see the implementation detail on this blog.

B

Let's take a look at this pipeline that has a freeze window setup already.

B

Here is a screwdriver diamond on line number seven we specify the freeze window.

B

It means it will block. The code from running from 3 am to 0, am in 21 hours.

B

And here the result you can see this no sign. That indicates it's a freeze window. However, if we want to run some, let's say: half axis or one time, ad hoc changes. How do we achieve that? We can click on the test and run star pipeline from here.

B

We have a magic string called force push and we give some reason.

B

And that will tell screwdriver to run this code, even even during freeze window.

B

And now we get a result back.

B

Let's take a closer look at this pipeline's job.

B

There are two steps for this main job and by clicking each step we can see the logs of given command. We can also click on the artifacts tab to see what artifacts this job generate.

B

And we can click on the files to preview before we click on download.

B

Let's take a look at the parameter drop down use case, comparing two pipelines ui pipeline and drop down pipeline in this ui pipeline. You will now see a drop down well in this drop down. Pipeline you'll see a drop down when you click on that it will give you a option. You can select free form. Another select.

B

You have predefined configuration that you like to be default and given option to select when you manually kick off another build.

B

Let's take this in.

B

B

As you can see, the configuration you have selected shows up here and you can see how that being used by clicking on the jobs- and here is the screwdriver yamo how to define parameters.

B

Now, let's talk about more tools.

B

We have just talked about validator, and now we're going to talk about templates and commands templates and commands are crucial part of screwdriver ecosystem.

B

We have various of templates that are snip of predefined code. That can help you to kick-start your framework by clicking on trusted. You can see a list of templates that are handpicked by screwdriver team.

B

It's either created by a screwdriver team or the people who trusted by the screwdriver team that are highly maintained. You can see the name of the package description namespace.

B

Let's click on one of the packages by clicking the fork button, it will redirect you to the template's github page and in this template detail page, you can see the usage of this template of this version. There are four jobs are using this version and there are three builds run. On top of this version.

B

You can see the content and supported images of this template and the steps that have been used and annotations if it were defined.

B

You can also see the environment variables it will define. You can also see the versions.

B

In here we'll talk about commands, what are commands, one can simply think commands are smaller components of templates to reduce redundancy of scripts. Similar to templates commands can also be marked, as trusted we'll skip here.

B

Let's take an example of docker trigger command.

B

By clicking the fork button will redirect you to the commands github page, you can see the version number the description and who is maintaining that when the last update time and how to use this in this session, I have covered how to leverage screwdriver ui to use the following features, such as user onboarding, screwdriver, yamo, syntax trigger other pipelines, pipeline collection, pipeline matrix job list, view sharing of permalink event, schedule, jobs and freeze window preview, artifacts, build parameters and use force, start to push during freeze window and trusted templates and commands.

B

I want to highlight these following links as our closing remarks and I sincerely thank you for staying with me for the whole time. Screwdriver is an open sourced project and we cannot do this without your support. Please consider joining the slack channel for questions and supports. Now we're going to the qe session. See you there.

C

B

Yeah, so if you have any questions, please type in the chat.

C

B

First question is: uh how are those secret stores are they encrypted? Yes, they are encrypted um and uh we use a library called uh error. Npm error.

A

um Two scrolling backs um what else.

B

Yeah so uh our configuration we have uh two different uh setups and the one I showed you is uh we host for open source. uh Those are in many: um it's not managed service from aws. We just host our nodes inside aws and manage our kubernetes ourselves and we have another one on-prem, it's all using kubernetes and managed by ourselves.

B

Yeah, so we once we set up the open source environment on aws, it will deploy and install new packages on itself, so it a close loop of self updating and installing and deployment.

C

B

We do use screwdriver in brazil media and it is not a personal project and we are hoping everybody are interested in joining and contribute to the open source project of screwdriver.

B

And the new thing is this whole project, it's written in node.js in javascript, so uh that should have a lower entrance of barrier so strongly.

B

Encouraged uh there's there are so many ways that uh we faced.

B

um So the question is: what's the pain that you think drivers saw better than other cd system, I'm not trying to compare to any system, but our system is written in node.js and it's very, um very cheap to scale up compared to other platforms, but I sometimes the cost is not uh it's not the only factor.

B

The second is we have been and building the whole ecosystems.

B

We have more control in a way that and that's one reason we choose to open source so people later on, they can plug in and play and have more options. What's the platform you want um yeah, we also have a link that you can host your screwdriver on your locally.

B

um If you only run very like two or three jobs, for whatever reason for your personal projects, and uh um I will have a link there before session closes.

A

B

Me check yeah. I think uh that's it's about my time.

C

C

Yeah so that um yeah.

B

Yeah, so the python two to python three. um We really haven't thought about that because still most people, they use python 2.0 x and we want them. Have you know they don't have to install python 3 if they want to kick off and I believe those code are compatible.

B

We have various of you know: domain knowledge, experts. They are very specialized in python node.js. They also help out if you are running, to incompatibility, follow a ticket and we'll get it fixed.

C

Thank you. Thank you marty. Thank you. Thank you. Thank you.

B

And I really hope everybody you know in this spectrum city. um You know get together and have more meetings like uh citycon, so we can share some ideas and I also see other sections are very interesting and uh you know when we finish our watch. Others yeah.

C