Kubernetes Kubernetes Contributor Summit - NA (San Diego) 2019, 27 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Contributor Summit NA 2019: Setting Up and Running Prow on Your Dev Machine Part 1

Description

Learn about the Kubernetes project by testing your contribution locally using Prow before you submit a PR.

HOW TO PREPARE BEFORE THE WORKSHOP

In order to make the best use of our limited time there is some preparation that you can carry out.

Background Reading
https://github.com/kubernetes/test-infra/blob/master/prow/getting_started_deploy.md
https://github.com/kubernetes/test-infra/blob/master/prow/life_of_a_prow_job.md

The above references are scripted up in the following repo which is almost complete.

https://github.com/RobertKielty/personal-prow/

I will be working on the code in the run up to the presentation and I welcome feedback via issues on the repo and of course pull requests are welcome also.

Runtime Environment

You should have a Linux laptop capable of building and running basic Kubernetes binaries, with the following software installed:
Go 1.13 (This is a change from the originally advertised Go 1.12)
Docker
git
make
bazel
kubernetes/kubernetes GitHub repo
kubernetes/test-infra
ngrok
kind
kapp
kwt

In addition to tools listed above run the following before attending the workshop.

Clone test-infra and kind:
git clone git@github.com:kubernetes/test-infra.git
git clone git@github.com:kubernetes-sigs/kind.git

Install GVM:
https://github.com/moovweb/gvm

Make sure to test the gvm command on your machine first as missing dependencies are called our when you use gvm and *not* during installation. On my machine I had to install bison from the package manager.

Event link: https://events19.linuxfoundation.org/events/kubernetes-contributor-summit-north-america-2019/
Session link: https://sched.co/Vv6Z

A

A

Thanks for coming to this workshop today., so this,.

A

What this workshop is about, is,, um,.

A

Deploying your own personal copy of prow, and, um,.

A

How it came about that I wanted to do this and,.

A

And come up for the idea of this workshop was in barcelona earlier on this year.

A

It was a kubecon and at the, at the,. At this event, in,.

A

In kubecon, in barcelona., um,.

A

I was looking for work to do on kubernetes's and uh, uh,.

A

Tim hawkins said to me, well,. We have a lot of bash scripts and there's some work that maybe you could do.

A

Around that. and I said, okay, cool. excellent.

A

I'll- have a look at the batch scripts.. He said, we've too many bus scripts and certain things happen with them. and,.

A

And casual passers by sometimes go in and make a change to the scripts, because.

A

They,, it's so easy. To, you know, bash is a useful tool,.

A

But it can be a sharp one. um,.

A

So I had a look at the preponderance of batch scripts in the,.

A

In the hack, folder and I, um, happened, upon, um,.

A

Uh,, a small issue, um, relating to shell check.

A

So shell check is a tool that, uh, that is, uh,.

A

Good for linting and checking the quality of,.

A

Of your bash scripting and uh, it's, it's,, it's used on the project and it's very good. um, there was,.

A

Uh,, an error that was turned off for shell check and that error was related. To,.

A

Um,, when shell check lintz your bash,.

A

If you include a file, uh,, a few source,.

A

Another bus script from your parent bash script.

A

And if the path to that bash script is in any way, dynamic,.

A

A shell check, cannot, uh,.

A

Follow the path and it'll hit and miss a warning saying, I can't,.

A

I can't check this, uh, sourced file. so, um,.

A

That warning was switched off., the,, the, the,.

A

The work around for shell check is that if you can, you could,.

A

You could annotate the script and say, oh, a shell check, um,.

A

This particular fire that I'm trying to source,, even though it's dynamic,.

A

I know it's here., so you can stick that into a comment and shell check will then check that for.

A

You, you know,, so it'll check the parent's script on a check that the source scripts.

A

So, um,, so I thought who would be a good idea to go through 80 or so batch files, and,.

A

Um, fix this problem, now. um,, and what I learned from that,.

A

There was a few things I learned from that. Is, is when people tell you, uh,.

A

Pick a small thing to work on, on a, on an opensource project,.

A

This kind of reasons for that. and, and my rationale and thinking was,.

A

I knew that this wasn't a small change,, but I was only adding a comment.

A

Strictly speaking,, it wasn't a functional change and it was smaller than I thought..

A

It was understandable,, but it was across 80 files, and,.

A

And the un, ultimately, when I submitted the pull request, um,.

A

A few jobs did fail. and, um,.

A

It wasn't clear to me whether they were flaky tests or whether it was related to.

A

My change., but I generated a lot of project charter on this pull requests,.

A

But I could,, I could send some feel that the,.

A

The overall thing was: who are you,? Why are you doing this?? What's the benefit?.

A

You know, there was hard, good technical review on it, both really from the point of,.

A

From my point of view of my journey to become a kubernetes's contributor,,.

A

I wanted to be able to, to run the tests.. I wanted to see tests fail,.

A

But I didn't necessarily want to generate all of this and project chatter and.

A

Waste people's time., so I happened upon the idea of saying, okay, well, let's,.

A

Let's see if I can do this at home., let's see if I can deploy this,.

A

Deploy prowl on a, on a local machine., so, um,.

A

So cut to a couple of months later,, I submitted the idea for the talk for this workshop and then I forgot about it.

A

And then I was accepted., so I've been working on this now for the past four or five weeks.

A

And essentially, what I've been doing is working off and the getting started,.

A

Uh,, the getting started, um, with pro, uh,.

A

Fire, intestine for, and I've been working through that to make it work for scripted manual.

A

Deployment. so in terms of my source, material,.

A

Um, in pers, this would be some of the, uh,.

A

Some of the links that I would have, uh, referenced in, in getting the setup is so specifically with deployment, the, the,.

A

The initial document for, for, for wrangling and,.

A

And getting prior deployed is this,, this guy here. um,.

A

So in this, in this read me here, uh, or in this, in this,.

A

Uh, markup file,. What we have is, we have a,.

A

A fairly detailed description of how to deploy pro- and it's done from the point.

A

Of view of serving the needs of the kubernetes project. um, and,.

A

And on the main pro instance, that's,, that's run running in gcp.

A

Um,- and this goes through, uh,, you know,.

A

Issues relating to running on gcp. um,.

A

And there is some tooling to help you deploy pro. um,.

A

But what I've done here today and what I've pushed to my repo is manually,.

A

Uh, effectively it's a scripted manual deployment of pro., okay., so, um,.

A

So I suppose, in terms of how this works and this,.

A

This is the script, so, um,.

A

Let's go down to the bottom, I suppose. um,.

A

So, so, and in terms of, uh,, uh,, kubernetes,,.

A

He starting point in the kubernetes journey and learning it, um, learning, communities, uh, deploying pro is, is, is, is kind of a useful thing to do. and so far as that, um,.

A

Basically, I would've learned a lot about. Kubernetes is by, by trying to deploy this locally. um, uh, pro itself, is,.

A

Is it's kind of a microservices based application and there's several components.

A

In pro. um, and I suppose I'll just,, maybe if we just go through how,.

A

Uh, I said about, uh, deploying it. um, it,.

A

Um,, that's what I do. Now. so, um,.

A

Is that legible and make a little bit bigger. so,.

A

Um,, the first thing that I do is, um,, I'm using kind to,, to,, to,.

A

To set up the, the, the cluster on this laptop. um,.

A

And the first thing I do, I check to see if kind of,, if we already have a cluster and if it is,, I just tear it down and start over.

A

Um, so, um, starting,.

A

So starting up the cluster is fairly straight. Forward.

A

A

Um, and, and there we have a kind, we're just doing a kind create cluster.

A

I give it the,, I give it the cluster name and um,.

A

A way for that to come up and kubectl cluster info and then, uh,.

A

Uh, sets the context for the rest of the script. um, so that the following, uh,.

A

Kubectl commands, uh, operation, work. um,.

A

One of the things that I would like to do with this code, is, uh,.

A

Is to reference back to the parts of the documents. um, that instructed me on,.

A

On, on each of the steps, um, eh, after.

A

Um, so,, so the three kubectl commands, uh, at the,.

A

At the end of the, uh,, kind, sort of bootstrap,.

A

A

Yeah,, so you can see the instructions here in terms of how I worked through this. so,.

A

Um, so moving, so, so,.

A

So here for example, we're setting up config maps, um,, and, uh,.

A

Uh,, as per instructions are in that document. okay. so, um,.

A

Moving on, um, once the, the kind cluster is up,.

A

It's time to, um, deployed the,, the, uh,.

A

The starter yammel., so in testing for it, we have, uh,.

A

A starter yammel file that allows you to, to,, to bring it, bring up the,.

A

Uh, to bring a pro. and that is,.

A

A

That's a, this here. so this,. This comes out. Of, um,, test, for,, uh,,.

A

Pro cluster. um, there is,.

A

There's a change of heart to make to that file in order to get this to work. um,.

A

And I will submit that as a pr after, after this talk. um,.

A

Uh, so that, um,.

A

Boom, boom, boom, boom,, boom,, boom,, boom. so, and on my journey in terms of eh, eh, eh, deploying this,.

A

I've, I've, I've taken a few, um, I've taken a few,. I've used a lot of tooling,.

A

Um, to try and make, try and automate this as best I can. um,, so one of the,.

A

One of the tools I looked at and used successfully at home was carp and carp is,.

A

Uh, is a, is a tool that allows you to, uh,.

A

Deploy applications onto kubernetes, clusters., um, it's,, it's a,, uh,.

A

Like the tool from the point of view that is, um,.

A

It gives you a am before it makes the changes,.

A

It lists out how it's going to change the cluster and, uh,.

A

How your cluster will be modified by, by applying some configuration to that cluster. um,.

A

I'm not actually using it at the moment because for some reason that I couldn't.

A

Chase down and it wouldn't work in the laptop today. and so, uh,.

A

I quickly refactored, um,, these two functions to be, uh, to have, uh,.

A

A kubectl, deploy prowl, and a cup deploy prior., so a kubectl works., so, uh,.

A

That's,, that's why I was amusing, that now, but essentially, um,.

A

Uh, that applies that., then. The second thing that I need to do, then, is: I need to apply some.

A

Configuration that pertains to my instance of pro here and for that,.

A

Um, one of the instructions and it's one of the instructions that I have to follow,.

A

Um, from here. no,, I didn't miss it that.

A

Mmm., it was to basically, um,.

A

Pick out, uh, and find, uh,.

A

These tags for these containers that, that the plank users, um,.

A

The, so, so does it, does a command there. You can see the kubectl get pod,.

A

Eh, eh, that this basically will return.

A

Mmm. yeah. the first one for you., so I think I can actually run that.

A

A

Now, what I,, what I do with this is, um, and I, I've, uh,.

A

What I do with this,, I actually run this in the script,.

A

Didn't work there, both. um, yeah,.

A

This is part of the tricky business of,, of,, of, uh,, doing the stuff from,.

A

From bosh and that without doing a deep dive on it,, this kind of feels like a,.

A

An escapement problem in terms of all of the characters there.

A

But I do have it up. Here.

A

A

A

So, so here, um,. What I do is I get this,.

A

I run that command and get that tag. Back. and, um,.

A

I use, uh, another of the tools that sits alongside cup and calls a ytt,.

A

Which, I presume is young, old text, templating. and,, um,.

A

Uh, there's, I've only scratched the surface of what that tool can do,.

A

But essentially we have, um,. If I then.

A

Configure it yammel, um,.

A

Here we have code, uh, inside in the amo.

A

It's a, uh,, a pythonesque type code,, I think called skylark. um,, but this code here,.

A

Um, that annotated comment, introduces, uh,.

A

A piece of code that can be evaluated by ytt and essentially we're concatenating.

A

Data to failures, our pro image, um,, and, and concatenating, that with the,.

A

That leading string, um, in order to use ytt in order to use that.

A

Um,, I need to have that comment at the top, that hash out comment.

A

And because ytt is relying on annotated, comments, um,.

A

Uh, that the normal tech space comments that you,.

A

That you have to introduce require, um,, an exclamation mark after the comments.

A

So it knows to ignore that., so it's, uh,, a common for a comment within a comment for why gt. um,.

A

So I've a few issues with this in terms of,, um,, uh,,.

A

Right now, in order to get that image,, I have to stand up the application.

A

So that's something I'll work on in the future. maybe, um, uh,, but the,.

A

The natural results of this from, uh, from the scripting point of view, is that,.

A

A

One portable bomb, bomb. slash: why don't you take,.

A

So from, from, from the scripting point of view, um,.

A

It's not,, not beautiful composition on the disfunction boss. um,.

A

That y g t minus f space, dat, um,.

A

Basically, processes all of the ammo within that, in the current directory. um,.

A

And, but then I'm left with the, the,, the problem of all of where do I put the results?

A

yandell, so I just drop it into a dot,.

A

Dot, dot, slash configured, folder, there. um,.

A

And then I run check config,, which is an application that exists and testing for it to check your, um,.

A

Your plugins and your config dot yammer for, for pro. um,.

A

So then f, so we may, as well,.

A

Um, have a look and see how this runs.

A

Any questions so far or a crack on?.

A

Okay. so, um,, I, um,.

A

I have an instance up and running now,.

A

Uh, but I can, uh,, I can kick it off. Again.

A

A

so step, one in terms of deployment is to run this. um,.

A

So you see there that that's the outpost of a check, config, um,.

A

Uh, indicating I think, there's one morning there and then indicating that, well,.

A

We can, we can push through this. um,, uh, and this,.

A

Eh,, the script is now bringing up kind and,.

A

Um,, it takes a time., it's not a very powerful laptop.

A

Like there's loads of memory,, but,, uh, sort, of,, uh,.

A

Two little cores that can in the cpu. um,.

A

So the,, the ultimate goal that I wanted to achieve was to be able to run, uh,.

A

The kubernetes jobs. um, and I've not got to that stage. Yet.

A

So maybe the next time I give this talk,, I will get those jobs configured and up and running.

A

Um,, although my fear is, is that, um,, the tests are too heavy, um, to run on a,.

A

On a sort of a laptop like this., so I need to do a survey of those jobs and figure out what can be brought up.

A

Okay. so, so the, the application is actually open running. Even though there's not invalid value.

A

There. um,, I can speak to that.. I haven't done the deep dive, on, on, on,.

A

On why this is., I have some references and on what that's about. um,.

A

But it's still runs.

A

A

so,. So here we have the, the pods that are, uh, that are,.

A

That are brought up from the deployment. um,.

A

So when that's open running,, you should be able to,.

A

The way I manage ingress is to use another of the ytt cup tools that can,.

A

I think that the k 18 tools, um, and that is here.

A

Let's make that a little bit smaller to get this.

A

I won't stop and start this because, um,. This does take your time to start up.

A

A

I'll leave that there.- probably you see, but the,- the commands to, to kick this off. uh,.

A

Let's see. yeah,, it is key. wt net and a service. I think,.

A

Am I need to check that that's in the,, it's in the rebate. read me on the, uh,.

A

On the report for this. yeah. yeah., it's not started yet.

A

So when I, uh, it's,.

A

I can only give a hand wavy explanation of this,, but essentially this exposes,.

A

Um,, the following: url, um, uh,.

A

Out of the cluster,, which is, should be dec service, local.

A

Or it should do.

A

Not quite sure why that's not working. uh,.

A

I was going to come back here.

A

Yeah. not quite sure why that's not working, but, um,. I should expect to see it.

A

Sometimes., it can take a time to come. Up.

A

And what I might do is just actually bounce pence,. This guy.

A

That's probably a good to do that, because that gives some indication of what.

A

A

So when that networking part comes up,, my expectation would be that that deck should display in the browser,.

A

Uh, locally. now,, while we're waiting for that,.

A

Hmm,, I didn't mean to do that. I'll leave that run. and while we're waiting for that,.

A

The, the, the,, the final piece, um,, the final piece, uh, to,.

A

To kind of talk about in terms of what a vaulter needed is that, when a pro,.

A

One of the, sort of the main,, the core features of pro is that allows you to run commands on, uh,.

A

Pull requests and have, uh, uh, uh, get homes a get hub,.

A

Hook based interaction, uh, between pro and, and get hub.

A

So, for example,, you know,.

A

If you run slash test all in,.

A

Or if you enters test all as a common sense,, your pull request and that will run all of all of the tests and pro,.

A

Um, and does it, does a little bit of, um,.

A

A credential setup required to get that done. um,.

A

But I have used some tooling from test infer to,.

A

Uh, get that working.. I want to just show that now.

A

Yeah, no, please. okay.

A

So, so the ad hook, ad hook, code, um, uh,.

A

What I do here, is, uh, at the core of it- is the anthill command,.

A

Which comes from test infra. and what that does is it takes the,.

A

It takes the public, url. um, and I've kinda s. Well, I suppose maybe I should handle how I do that first.

A

So the once I guess, the, the local instance on the laptop running,.

A

Um,, what I then do is set up: a, a reverse ssh tunnel,.

A

Uh, over at the teach, um, uh, back to that from it using an iraq., so, and grok. um,.

A

Let me just see if that's going, up. come on up. okay.

A

Um,, so when I run this,.

A

I'm just short running.

A

So what's happening here, is, is that,.

A

Um, eh, comparable,.

A

What's happening here is that, uh,. If I go to,.

A

Okay,, well I go to the, uh, the repo and the, that I've set up in a,.

A

In an organization that I've set up,, I think we may have possibly.

A

A

Yeah, yeah, that's unfortunate. but,, but I,. I can talk through it.

A

So basically, on a report, you,, you would have settings. uh, there. We go.

A

Um, we'll,, we will have settings where we can set up, um,.

A

Uh, web hooks for, uh,, the repo. um,.

A

If I just go to them, here, that's web hooks. There.

A

And what that script does and you can see as having run a few times. um,.

A

So an grok, uh,. Basically I just have a look at that.

A

That'll, probably 500 dash.

A

A

The, uh, email address ages me a little bit,. I think.

A

A

He'll be okay.

A

I can do it as this section.

A

I'm just going to go back and see if we have got this up.

A

A

Okay. so,, so this is, this is the web hook. On, um,.

A

Uh, on this repo and it's,, it's fairly straightforward to set it up manually and you just put in the.

A

Payload url. and, um,, so in, in my case, here,.

A

This will be something that's available on the public internet,, but we'll tunnel back to, uh,, a nonpublic part of the internet, where our,.

A

Uh, pro in station is., and what we need to have done here is we need the application type to be jason.

A

And then there are permissions, um, uh,.

A

That can be, um,. I think there's permissions here when you set this up, uh,.

A

That, that are required to be set., so you can see here, this, um,,.

A

This was an attempt to have a chat with that url and that has failed because,.

A

Um, that has failed because the, this guy isn't just,.

A

This guy's not coming back to me unfortunately. um, but,, but when,.

A

When, uh,, when that, um,.

A

Networking part comes up and when it manages dns, so that does dec.

A

Dot. service dot., local, uh,, address will, uh,.

A

Would point into the application running on the cluster.

A

and that worrying up does work and it's a reasonably straightforward to do.

A

So that's in terms of, um,. I know this has been more talky than,.

A

Than workshoppy. um,, but that is pretty much yet. I think is,.

A

Does anyone have any questions?.

A

It's reasonably straightforward, to, in terms of, in terms of the,.

A

In terms of the work itself. um, uh,, you know,.

A

The code is a little bit, might be there's,. There's uh, does uh,.

A

Does a couple of the things that could be done to improve the code., but the way I see, the way I see that script,.

A

That scripts sat alongside the, uh, documentation,.

A

Uh,, both of those things together would give you a good map of how you want to.

A

Create your own prep pro deployment. um,.

A

In terms of other projects that are using it or the projects have written about.

A

Their deployment journey as well., I think sto- would be an example. um,, but yeah,.

A

If the,, if there's no other questions, um,,.

A

I think we could maybe wrap it up. There.

A

A

Um, yeah, yeah, yeah. so,. So as I repeat, the question for it, yeah.

A

So this is the,. This is the question,. Is, is what,.

A

What are the benefits of using prow, eh, and get up actions, as opposed to maybe say something else like travis? um,.

A

There's a couple of [inaudible] in terms of my cia comparison that I would make,.

A

Um,, based on my experience,, I would compare pro to jenkins as follows.

A

Um,, the un,, and we would have people here who might be able to validate this or, or,.

A

Or add to this., but my understanding is, is that from, uh,.

A

From the, from the needs of the kubernetes project and pro was built, uh,.

A

In order to handle the scale of contributions and the scale of testing that goes.

A

Into kubernetes's and from the point of view of the skillsets and the on the,.

A

The teams on the team, that there was a desire to kind of,.

A

I suppose in a way eat your own dog food.. You know, both from,.

A

From a scalability point of view. um,, I believe the pro is a lot more scalable than an easy,. It's not,.

A

The jenkins isn't scalable,, but it requires a different set of skills in order to manage scaling out.

A

Overseas. you know, um, from the point of view of workflow, um,.

A

In my opinion, uh, and I, uh, in my opinion, the,.

A

The, the real benefits of prow and the,.

A

The ghetto workflow that's used in kubernetes's is it's transparent..

A

It's open, it's,, it's easy to understand as a developer and an end user and,.

A

Um, uh, and the whole workflow that, you know,.

A

That pro supports in with its interaction with web hooks, um,.

A

It's not difficult to learn,, you know, and I think, um,.

A

I think that sometimes you will see people from other communities and other open.

A

Source projects- say, oh, kubernetes,: how do you do it? and you know,.

A

There are talks on youtube where you'll see people from, from the,.

A

From the kernel community saying,, we don't know how you do it., we get hope both.

A

Once you understand how prayer works,, you understand how a kubernetes's as a project works., I think, you know, so,,.

A

Um,, I don't know a lot about travis but,, but what I know from, from looking at the pull requests that,, that I'm looking at,, it's a fairly easy end user interaction., it's a fairly easy, um,.

A

User experience. you know,.

B

We'll take that.

A

Um,, can we turn this into a business?? Is that what you're asking?.

A

Yeah. separate, separate question. yeah.

B

What's your name?.

A

I don't know., do we have, does any,.

A

There's a little bit, travis is really about. What's going to look at pushes and pull requests.

C

And pull request.

C

C

Whether it's somebody commenting on something more,.

C

Which is essentially what.

B

C

Lets you do those things simply because.

A

Well, my understanding is, and sadly, I didn't,.

A

I didn't get into job configuration, um,, for this talk. um,.

A

My understanding will be this, this,. You could orchestrate,.

A

You should be able to talk to jobs in parallel,, but it's a bit,.

A

It's a bit hard to review for me now,. I wouldn't have that experience. um, both.

A

Um, but he,, I think,, I think, that's something that that can be done. yeah.

A

Yeah, yeah. okay. any other questions?.

A

Okay. wrap up there then. thanks. thanks for come along to the talk. thanks very much.

A