Add a meeting Rate this page

A

That's.

A

Hello, hello,.

B

How's everyone.

C

Hello, can you hear me yep? Okay, it was so quiet. I thought, like my earphones, doesn't work.

C

So let me put the agenda to chat.

C

Okay, hey.

D

Everybody.

C

Hi tracy, so while we wait more people to join as usual, if you can type in your name on the agenda.

C

This week we have three to fight agenda as well.

C

Mainly about what's next kind of discussion, followed by presentation from andrew on lighthouse.

C

We can start slowly by looking at the only action item from last week, so this action item is still open. Yeah. We asked jackie to update the invitation, so we have the link to hack md rather than the google docs for the agenda, but the issue is still open.

C

I know how many times one could you know think but yeah we'll wait until that is done, and that was only x nikon and I am wondering if we can update google calendar invitations. I don't know if I have permission or not. If I had I could have update, but I know.

E

Let me share my screen. First.

C

So you can see the agenda now, yeah.

C

Okay, so that was donation item, and the next agenda item is about publicizing the work we are doing on this. I added this topic based on uh the discussion we had like two weeks ago or earlier meeting. As you know, we pushed or stored the roadmap in our repo, it's kind of released the first early version of our roadmap, and I think there are quite interesting things like the vocabulary work. We have been doing followed by this work stream setup. We agreed to establish and the first works work stream. We had.

C

There was events in ci cd, which is running and in addition to that, there are some other items like from ci to cd. What are the integration points or what are the challenges and so on, and one of the ideas was to write an article to publicize the roadmap and call people for an action.

C

But while I was thinking and seeing some other communities what they are doing, I start thinking about. If we should instead perhaps start working on some kind of white paper to at least list the work, we have been doing again, the vocabulary work or the knowledge sharing with these presentations. We are having from different communities followed by the work stream setup, because we identified some focus areas and trends like events and reusable libraries and perhaps some use cases from user communities and some updates from projects relevant to interoperability area, followed by call for an action.

C

So I'm wondering what people think if we start working on some kind of white paper or whatever the name it is, and we publish it with the help from cdf.

A

Hey fatty yeah, I think it's a good idea. um I think it matters how we like frame it or scope it, because I know sometimes I'll talk to folks about interoperability and and they'll be like. Is it interoperability for the sake of interoperability?

A

Or can we talk about it as a benefit for an end user like someone who's trying to stitch together system, so maybe tackling it from a specific point of view of benefits to to the wider ecosystem or to project adopters would would make sense.

C

Yeah I seen the uh article published yesterday on devopscom tracy. You were part of that article and it's exactly like how you describe what cdf is trying to and if we think in six contexts we can expand what we have been talking about during your podcast or within those articles, and perhaps working in interoperability in cicd area and how it could help users and what the projects are doing to address those things.

C

So not just like. Okay, here is the sake we are doing this, but rather than have it broader and actually highlighting the benefits of this potential benefits of this work. We are doing and it doesn't have to be limited to just this sig or cdf, but cross community type of you know highlighting the word going wherever.

A

Yeah I agree and like even in that article in other places I'll talk about the the work on the rosetta stone and the common terminology, and that always gets a lot of interest. But I guess in many ways it's it's kind of a github page hidden in a repository and not something like. Perhaps if it was in a better format like a white paper and more prominent, that would lend itself better to be shared and help others step in and contribute and drive. The conversation forward.

A

Any thoughts from anyone else.

F

I think making it more public is a great idea.

C

Okay,.

C

So then, again, since we, I think it is important to get you know. Android end users talk about the challenges their facing, I'm wondering if we have anybody who could contribute from their organizations with regards to what they are doing to address those things within their organizations based on the projects or tools. They are using and kind of highlight the fact that yeah, we all have some internal fixes to get those tools work in a way that solves our issue.

C

So if, if you can find one or two organizations, end user organizations to you know talk about their use cases and challenges in education that, if you can like tecton or jenkins sex or one or two communities talking about what they are doing in their communities to address or potentially address those challenges, that would make it even more attractive. So we don't simply talk about some dreams, but actually talk about some real problems.

C

In addition to the problems, some possible solutions worked by the communities, so I don't want to put anyone under the spot, but like do we have any one who wants to take part in this type of use case or end user story or project story?.

G

I can ask I can help from the chenker's ex point of view. I'm sure uh broke being a couple more people that are yeah on the corner as well, that are involved in jenkins, x, um yeah.

G

Sorry,.

F

I'd be happy to contribute from an end user perspective.

H

Great really happy to help on the chingan zak side and in general, on the.

C

Paper and they've can help from the point of view.

B

Anyone.

C

Else.

B

Well, I don't know if you saw uh feti, but uh I've put in my name for uh a presentation in a couple of weeks. Yeah, yes,.

C

Yeah, I I've seen the notification and I was like whoo finally.

B

So yeah, finally, my god, you know that guy got off his ass. Sorry pardon my french! uh I do speak french, no, I actually I don't um but um yeah.

B

So I've got a uh I'm going to give you guys a presentation of how we are incorporating tecton tecton cd within the ebay, tooling, the cd tooling, um and it will open up the way for us to you know, go into full cloud native development and also open up open up options uh for our developers to uh who developing groups who want who want to use jenkins x to basically.

B

Use the the tecton cd framework we're putting in underneath the covers, so they have that as an option as well, since jenkins x is also building on top of uh tecton for its orchestration framework, which again once again, tracy and james awesome.

B

The right thing to do so. uh Yeah I'll give you guys uh kind of an overview, I'm putting it through ebay, legal right now that we have some legal requirements and whenever we expose any sort of uh internal documents to the outside world. So.

C

Okay, so that that would be great like I'm, uh maybe we we could talk about this later on, but would you be interested to again take part in this white paper or whatever? We will call it as an end user, since you will already present this within two weeks or within four weeks? Maybe it might be easier for you to turn that into like two three paragraphs of text as well.

B

uh Yeah, I'm I kind of missed the beginning part of this. What is this white paper about.

C

That's simply like publicizing to work. We are doing you know. We have been talking about things last six months in january and we have been sharing knowledge between projects or as users. We are talking about our challenges and then we publish this vocabulary followed by the roadmap. It's simply documenting the work. We are doing a bit more in a bit more marketing friendly format.

C

You know everything is in github, but it's not always easy for, like non-developers, especially to find that information, and if we could write some kind of white paper and get help from continuous video foundation to publish that we could get more interest from end users and projects or companies or whoever is having similar challenges or working on similar areas. Let's basically call for an action, look guys. We are doing something here. We have some progress. If you join us, we could do things better and you can contribute to this work from your organization.

C

uh Yes,.

B

Sure I'll I mean I'll, contribute to it. uh Where I can there's like, I said: there's efforts going on inside the ebay that involves uh cdf projects.

C

So I am just typing some names here. These are not like. Okay, you have it now. You have to fix it. It's just capturing who said they could help with it sure.

A

Yeah, in fact, you can put me down to just help, tie it all together and sync, with the rest of cdf to get any resources we need to to work on it. I don't know whatever graphics or stuff like that.

F

Doesn't say: do we have a set of tooling to work collaboratively on a document like this.

A

uh It's probably google docs. Yes,.

C

Yeah I have a docx, I put some very high level structure. I can put the link there quickly. It's basically what I was talking about.

C

Let me put the link there.

C

Like what are we discussing with ci cd key concentrations, constraints, concerns trends, case studies, future conclusions like some very random headings, so we could start on google docs, and once we have some, you know, structure in place and some uh something on the document. We can get help from cdf and get their feedback on. If this is like a white paper, it's like something we work in github like because the work the wording should be. You know different based on the audience yeah.

C

We will probably need to address more than developer community, like decision makers or whoever yeah so but tracy. Thanks for that link to the.

E

To outline.

C

Just yeah, I I will include this in the meeting minnesota or perhaps we can send a separate mail with this with an obvious subject saying that here is the link to white paper. Please join and then yeah.

A

Yeah the other thing um I hadn't seen the outline, but I think that's really good, and we have a couple of meetings next week where we have a regular cdf ambassador, call it's a and that's a pretty active group. So just taking something like this and getting it in front of them and saying who wants to help? I think it's, it's just a nice way to bring it up again and get folks to engage in a in a more tangible way. So I can take on that.

A

Action.

C

Okay, thanks tracy, so any other thoughts comments on this, because we did some work and it is important. We highlight the work with it. It's not the best thing that could have been done perhaps, but given the progress we have, it is always good to you know, talk about what we are doing to get even more people. Take part in this. I think cdf in general,.

G

Yeah I'd wonder if, um as part of that, we could try and give some kind of areas where that we're looking for more end of user information in the you know, thinking about interoperabilities, we want to know about problems, it doesn't necessarily for one project to come, go and solve, but as a group here, we could look at problems that we can kind of work together and finding finding solutions for which.

G

So I was just wondering if, when kind of, if this goes out, is really to try and set some scope or something of what we're really looking for from user use. End user point of view: does that make sense.

G

Like use cases common problems, I don't know processes, I don't know just trying to get people thinking about what some some of the feedbacks and the information we're just trying to which it would be useful for us to then go and take away and then work together. As a wider group on looking at whether there's standardizations we can make we can put together, we can work on various things.

C

Yeah, I think that makes sense, and we all we have some things in the road map like I'm. I think andreas and emil and some others put some bullet points under standardized frameworks. So this could be like starting points. We could we can say here are the things we have been able to identify based on the current engagement.

C

If you have more things to, you know bring in the table to work with the rest and standardize or whatever join us, and I think white paper. The point of white paper is call people for action. We could even put wrong things there to just get people say: oh you are thinking it wrong. We actually think this way and we solve that problem already. Just let people speak or no kind of but yeah james, that's like it could do.

C

If you could list some use case or some problems we allow or enable others to come and speak up. Hopefully, that's.

C

Great.

C

Okay, so that was the first item in the agenda and the second item is about the topic: we've been discussing last few meetings, reusable libraries and I think, kara tracy- you both talked about this item like one of them, was like go icm, the other one was like tech tone or jenkins sex hand. Chart. I am wondering like. Is there any progress there like? Will we go and create a work stream for it or we need more time?

C

So I just want.

A

Yeah, I think um the gosmo will be interesting. I just saw that I don't know folks other the harness news uh where they've acquired drone and looking for.

H

Ravi.

A

On the call and he's not here because we were gonna, assign him tell him to go, get us the drone folks, but on the other one on the tecton helm chart. I think that is like a just low hanging fruit. It's on my list to do, and maybe what I'll suggest is. I could set up a working session and just get a couple of folks together. Anyone who wants to help and we'll just kind of just work at it, bash it out in real time and uh figure out what that means.

A

So I will go schedule that, maybe sometime next week and just invite whoever wants to and again toss it out to this group, the ambassadors and just do a a kind of byob we're gonna sort, a text on helm chart in cdf, oh and I see christie's on the line.

F

Is drone based on tecton, I I didn't know anything about it until yesterday, when my harness rep was like hey, did you see we bought these people.

A

uh No uh joan and tacton are not related, but there's a common library that uh jenkins x has used from the drone project. Goa cm and tacton also uses it. I would like to use it and we wanted to get it into a place where all the projects could could leverage it or at least get the changes. Jenkins access made pushed upstream.

H

Tecton is currently using it we're using the the jenkins x fork of it.

A

Cool and chris, did you hear uh the plans to just stick a checked on helm chart in the cdf repository? Oh, I didn't. I didn't uh catch that, but that sounds sounds good good. As long as it's got your blessing, it's good.

C

Okay, then you have it under control. Tracy.

A

uh Yeah I'll show that an email and just pick a pick a date a time and see who wants to come, join.

A

Okay,.

C

So before we move to white house, presentation from andrew do have any other topic. Anyone wants to bring up.

C

Okay, so andrew you are with us already yeah, I stopped sharing and.

E

Tried.

C

Okay, thanks for uploading the slice already, you are the.

D

Pro, it was a good way to force me to say they're done uh so. Here's a hopefully relatively quick introduction to lighthouse uh I'm andrew baer, I'm an engineer at cloudbees. I've worked on jenkins, jacob zach's. Lately I've been working primarily on lighthouse uh and yeah. So what is lighthouse uh at its base? It's chat, ops for pull requests. It's the idea of centralizing your uh communication and reporting and approval mechanisms for pull requests on the pull requests.

D

So you it handles responding to web hooks to trigger on pr's or pushes to master, etc. It tracks the results and records those on the sem provider, including links to logs, if appropriate. It manages approvals via chat, ops commands and an owner's file and automatically merges pull requests which meet pipeline uh and approval requirements. um I felt like I needed to do a slide here, there's just what is chat ups for pull requests, so I'm going to move on past that.

D

uh Why does lighthouse exist? So you may have heard of prow. uh Prow is the what lighthouse is very much based on prow it comes from and is heavily used by kubernetes test infra makes it easier for them to organize to find and run tests on prs.

D

The approval management started there in large part because they wanted to be able to give people the ability to say yes, this should get merged and have it get merged without giving them push rights to the repositories, because, eventually, you end up in problems when you give people push rights to repositories, it's a suite of components deployed as kubernetes applications and it's been adopted by a number of other oss projects for their own testing, for including tecton, uh so prow and jacob's, x, uh prow's, workflow uh and triggering mechanisms and chat, ops mentality fit really well into jenkins x and with this incorporated into jackets x, initially in late 2018, if I'm remembering correctly uh and then all in over the next few months after that, uh as well as it being used by uh jenkins x by the tickets x project itself, it's also part of the jenkins x product.

D

It's part of the distribution um so well. At least it was until recently, uh so there were some limitations with this other right yeah. That was right side. uh There are some limitations with prow uh from jackets x's perspective. Prow only supports github.com, it's very tightly integrated with github apis. It's got some really impressive, optimizations around caching and api rate limits to because at the scale of pull requests that the kubernetes test and for org has to manage.

D

You really need to do a whole lot of caching and rate limit management or you're going to fall over uh jenkins x. Wants wanted to be able to support other scm providers besides github.com, and that meant we had to figure out what to do so. The first question was: should we add support for these other providers to prow, and we ended up coming to the conclusion that that was not the right direction to go. Prow exists first and foremost to serve the needs of kubernetes test infra.

D

um Others are completely welcome to use it and, to you know, uh propose changes, but the team that actually owns and maintains prow is focused first and foremost on kubernetes own usage, uh the kubernetes projects usage, uh so they're, pretty conservative when it comes to changes- and I want to make it very clear- I think that's a good thing. I think that they should be uh prow inadvertently ended up being something that was useful for other projects.

D

But it's when you've got the its main reason is to make sure kubernetes prs, get built and merged and that issues get managed etc, uh and we came to believe that making the kind of changes we'd need would be fairly substantial because of how tightly integrated uh the github stuff is in prow.

D

The architecture, just it would have probably been significantly more than the proud team, would have been comfortable, bringing in which again completely understandable so uh a little over a year. About a year ago, uh we started what we're calling what we've been calling lighthouse started as a fork of relevant, proud code.

D

First, uh the web hooks uh handling and plugins for things like slash, approve, slash, lgtm, meow, the most important one um and then added uh the what was called what's called the proud tide, what we call in lighthouse keeper, uh the service that handles determining whether a pull request is mergeable or not, and if so, actually merges it.

D

We use goscm, as previously mentioned, uh for the abstraction over the scm provider, interaction uh so that we don't have to have completely separate implementations for each provider that we interact with. We can just abstract that away and deal with, and let go scm be where those implementations live.

D

Lighthouse has full compatibility with the existing pro configuration. So in most cases it's literally drag and drop. uh You know, and a few months ago, a couple months ago, three months ago, uh we switched uh jenkins x to using lighthouse by default rather than prow.

D

uh So it's been in production use on all modern uh jenkins x clusters for a few months now, a little bit about goscm.

D

So as mentioned, it was originally a drone.io project and there still is a dronego scm project on github. We needed to iterate it on it probably too rapidly to uh be able to get those changes back in in any reasonable time frame. um So we ended up forking.

D

That library is now used by lighthouse techton and I believe some others and, as mentioned we'd love to merge our changes back in and get to there being one goscm library a little bit about lighthouse's current capabilities, most of prows capabilities, uh pretty much everything in terms of the chat, ops, uh so lgtm approve test.

D

This override assign uh approval management via owner's file in the project, repository merging of uh approved, successful pull, requests, triggering of builds in response to pull requests and in response to pushes to master post, uh not included currently from capabilities that prow.

H

Has.

D

uh Batch merges because I kind of want to rework that I'm not a huge fan of uh the batching logic in pro and right now we don't support uh running pipelines uh in jenkins or as kubernetes pods, but those will be coming fairly soon.

D

We also don't have a ui pro has a fairly simple ui called deck, and at this point we do not have something like that. uh But what else do we have well build engines? The they can actually go run. The pipelines uh jenkins x is where it started. uh Jenga's x has its own, while jacob's x lives on top of tacton for its execution. It has its own interface for how we trigger pipelines there using a kind of meta pipeline.

D

That looks at jenkins at the configuration for the specific repository and generates a pipeline dynamically, um and then we monitor pipeline activity, as crd in jkinsex, to see what the results are.

D

Recently, we added support for straight vanilla: tecton pipelines uh with an inline pipeline, run spec configured for your jobs, and there is some level of extensibility for addition of further engines already, but I, as I mentioned again later, plan to work on that and make it more plugable and easier to uh configure with pro you have to actually define every potential engine's configuration in uh the one set of config uh structs and that can get kind of unwieldy.

D

So I want to make that more plugable uh scm providers, so, as I mentioned, uh prow only supports github.com. It doesn't even support github enterprise uh lighthouse supports github, github enterprise, gitlab, hosted or getlab.com, and bitbucketserver7.x and greater.

D

The experiences aren't necessarily as great on all of them on gitlab. You have to do commands a little differently in some cases, because uh gitlab has quick actions which hijack slash, approve, for example, as a comment, um and so we have to have some tweaks there and bitbucket server doesn't have labels, which we made some interesting workarounds to deal with, that, since lighthouse is very heavily dependent on labels for keeping track of things like approval, um whether something should be tested, etc.

D

So uh so so perfect, but we do have fully functional support for all of those providers.

D

uh Then there's an external plug-in architecture pro already had this concept, where every web hook that comes in can be relayed to a separate pod to handle outside of proproper lighthouse is gone in a slightly different direction with that, our external plug-in approach sends the goscm abstraction serialization of the webhook, rather than the github or bitbucket server etc web hook, so that the external plugin doesn't have to worry about translating between the different formats, and we also relay build events like start finish stage: completion to the external plugins as well as web hooks, so that external plugins could respond to build succeeding uh or send messages to slack or something like that, depending on build results, which uh I think is since it's information that we've got coming into the lighthouse and makes sense to be able to relay that to our external plugins.

D

Now, for the part that I'm always afraid of, let's do a demo, so this demo is lighthouse using tecton pipelines on gitlab, because I wanted to show something that prow can't do so hold on one sec move over here bar. I want to see better there. We go all right. So we'll start by taking a look at the config for uh lighthouse. It is literally the same as proud to the point where it actually still has proud names in it, because we wanted to have that complete compatibility.

D

I'll mention a bit later, how I'd like to created a an additional configuration layer uh that uh we can parse to from the prow configuration, uh but that doesn't stick with us needing to have all of the exact layout and terminology that proud does uh because we're not proud we're a descendant of pro, and so here, I've configured to say: don't merge uh my demo, repo that I'll be creating a pr in momentarily unless it's passed its pr build context- and I said before you know, when a pr comes in to that repository, let's create a tekton pipeline, uh then we're calling it pr build we're using a test pipeline that I've already defined in the cluster.

D

uh It's just simple: it just uses the git clone task from the tecton task, catalog to fetch uh the pull request, source and then just runs. A script, looks for a script.sh and repo and runs it just so. I can have a really simple example, and it defines the workspace that it'll reuse.

D

We then have uh the definition of how whether we use merge, rebase, etc for merging the repository labels uh that uh in this case, if there we won't be able to automatically merge apr to that repository. If it doesn't have the approved label, and if it has any of these labels, we also will not be able to merge it. Even if it's approved.

D

So that's the config.yaml, there's also plugins.yaml the main thing there is showing the various plugins that we have enabled on our repository, and we also have the config updater, which takes these two yaml files and turns them into config maps, uh enabled on the uh lighthouse, config repair.

D

So let's go look at a pull request, so we've got an owner's file here in the repository showing who can approve changes. My bot and me um not very interesting uh and we've got a script.sh that will get run uh as part of the pull request. That can be literally anything in the text on pull request. I just used this because it's simple um and it has a sleep in there, so we can actually see status. So, let's, let's go make a change.

D

Let's make this fail yeah. Why are you not letting me hit enter there? We go.

D

ah Typing is hard.

D

So now this pull request.

D

Will we'll open this pull request? This is just standard gitlab pull request. It feels very cluttered to me compared to github, but I guess that's just what I'm used to so now that this is run hopefully or kick off. We can see that it's reporting a pipeline right now. It's reporting the pr build. So if we click on that, we go over to the techdon dashboard that I have configured in the lighthouse setup to link to, and we can see that right now. It has not really started yet the pod is starting.

D

So, let's get back from that, uh we also can see that uh now the that keeper, the which monitors all the open, pull requests on uh the configured repositories, has seen that the pull request exists and is reporting on it and saying oh yeah. This isn't mergeable yet because it doesn't have slash approved, um oh, and we can see that my pipeline failed. Who would have expected that it failed? Oh there's a merge conflict.

D

Well, that's even funnier!

D

uh So yeah. We can see that information here in github it's in the status. uh You know the commit status at the bottom of the pull request. Let's see what's wrong with my uh merge commit.

D

What did I do wrong there?

D

Oh, I added that there yeah, let me close this sucker and make a new one. Sorry, this is how it goes. This is what I get for doing. A live demo.

G

It's good to see when what the effect is of when something goes wrong. You wanted to get that merge failure, yeah right, yeah, it's all good.

D

Let's see if this one won't fail after.

D

That and we'll cut the sleep down to 30.. Don't need to wait as long. ah That's why I was doing the ide, because I couldn't figure out how to do a new branch thing here. Okay,.

D

Yeah.

A

All.

D

Right, try that.

D

Yeah, do that again.

D

Wait until it hears back from lighthouse about the existence of a pipeline, though you can see it already responded by saying: hey here's the size of the oh go away the size of the pr, a message saying that it's not approved and that someone from the approvers list needs to approve it before it will, you know, get merged um and it failed again.

D

Why did you do that? Well, I don't know I probably screwed something up in something else. Oh I'm running too old a version of lighthouse to support that so we'll ignore the fact that it's failed, because that actually lets me show something else. But while it's going on, let's show my favorite plugin.

D

uh So if we wait a moment, oh and we can see, we can get. Why do you keep doing that? uh The links to the logs also will show up in for failed test contexts and we'll show up here, and you can retest them with slash retest and oh look a cat now, let's slash approve that will not work right actually because we are on gitlab, and so we need to do lh dash approve either one will work on non-git lab, but on gitlab you have to do that. Prefix.

D

I have a ticket open with uh gitlab to uh get them to send webhook events for those quick actions, at which point we can actually respond to them correctly. But so I'll do that and then in a moment it should update to say that the pr is approved, but it won't auto merge because the pipelines failed.

D

uh So there's something we could do about that in the case where we know that the pipeline failing doesn't actually matter, we can do slash override this. I think it's this hold on.

D

Let's find out, this will mark the uh pr build. uh That's right, it is override pr build.

D

And yes, you can see that because I put the wrong input in the bot uh lighthouse responded by saying: no, that's not what you meant to what you needed to do, and so here we did override the context we said treat it as if it passed so that now, when we go here, it shows up as past and now shortly uh lighthouse uh will uh see this update and will end up merging the pr. I will give it a yeah it's in the process of being merged.

D

uh So all that happened without me ever hitting merge. I could have retested it if it was the failure that I actually could do something about, or that was you know flakiness. I can add cat pictures to the pr I can assign other users uh etc. So that's my demo.

D

uh Let me go back to presenting.

D

And drag that bar.

H

Back out of the way again.

D

All right so uh future of lighthouse, uh more functionality uh so, like I said I we recently added the tekton pipeline support uh like a few weeks ago, uh and we want to add the remaining other ways that you can trigger builds in whether you can execute your builds in pro with kubernetes, pods and jenkins uh with jenkins. The goal is to initially just emulate: prow's behavior, which requires there to be an existing job already. That is set up with the right parameters to be able to uh get past what ref it needs to clone.

D

But what I'd like to do?

D

What I need to spend some more time investigating is ways to integrate with jenkins multi branch pipelines where jobs are automatically created for every uh pr and run for every change in the pr I'd like to integrate with that to count those as contexts that need to be passed in order for merging to happen and to enable uh slash test context, name to automatic, to be able to rerun failed tests from uh the pull request, rather than having to go to jenkins and click rebuild, and I I'd really like to see more engines get added.

D

These are just the ones that come to my mind, having worked on jake and zach's and jenkins and on techton. Those ones are all obvious to me and kubernetes pods are the base of what most people are using with pro, so those ones are obvious, but perhaps spinnaker could fit in. It could take advantage of this.

D

Perhaps there's other engines that can take advantage of this and, like I said, I'd like to work on making the support for adding engines more plugable and flexible um rather than just kind of wedging them in, like pro, has always done so there's some re-architecting to do there, uh reporting, ui and metrics um prow has a bunch of observability and metric stuff that we're not really doing anything with in lighthouse, because I don't actually know much about that area and I'd really like to get more in there and something equivalent to the deck.

D

Ui may actually make sense uh to add that could be useful to people, um but I am not a ui person, so it's not an area that I can really talk about much uh the installation and initial setup could be better. We have a helm chart, but it is tuned for jenkins. Ecstasy's case may not be the best for all scenarios.

D

uh Those config maps that we looked at earlier, the config.yam on plugins.aml jenkins x, generates those automatically based on other configuration within jenkins x, for about which repositories it's paying attention to, etc.

D

So, historically, lighthouse has not had to have a base copy of those which is not the most user friendly, but for its original use cases it wasn't a problem, so we do need to come up with uh an out of the box default that people can edit, rather than requiring them to find something they have to copy paste in. To start with, uh I mentioned that the configuration I'd like to rework it uh retaining the compatibility with the pro config has been key.

D

I think uh and needs to remain this the case for a while, but that doesn't mean that we can't have a lighthouse config that is more specific to lighthouse's needs and integration with other uh tools uh that can be translated to from the proconfig.

D

One thing that particularly is interesting is the idea of putting the pipeline run spec for a pipeline. That's using tecton moving that from being inside the one central config.yaml to being in the project's repository uh like a jenkins file like a travis file like a jenkins axiaml, so that you can define your pipeline.

D

uh Your pipeline run in your repository without having to worry about oh well, I need to go make a change to the lighthouse config repo to get that change into the config map, etc. So, there's definitely some interesting areas there.

D

uh The next thing to talk about in the future is a long-term home, so, while lighthouse very much started as a jenkins x tool, part of the jenkins x suite it has grown beyond that it. It still very much is part of the jenkins x suite, but it's not.

D

It doesn't need to be limited to just serving jacket sex. uh It now has tekton support. It will have kubernetes, pod and jenkins support within the next month or so, uh and we could also add more scm providers, though uh there's limitations with the note with like bitbucket cloud that I could go into but won't bother at, I feel like there are other tools that and users that would like to be able to get the proud behavior.

D

uh The get the pull request, chat, ops without having to be on github.com and using a specific workflow that prow requires lighthouse is a lower, lower level piece of the puzzle than jenkins x. It has value on its own and for integration with other tools and systems.

G

One thing I'd say probably say on that is probably jenkins expo map, which will also be extending to all the these other things as well. So it'd be interesting when we look at those things and see what those capabilities are, um because the chat ups have been something very. You know really that we want to be building up that experience for jenkins x but yeah. I agree it'd be interesting to see what those other integrations might look like, but so certainly jenkins, x's roadmap as well. It's going to be continue to expand as well.

H

And look.

G

For further integrations and plugable ways as well.

D

uh But yeah so lighthouse will need to build a community uh if it wants to succeed as a project on its own or otherwise decoupled from being a sub project of jenkins x, it's not ready to be a top-level project of cdf or anywhere else. At this point, there's not enough of a developer community for it to really thrive. It needs a community of contributors, uh and I am looking for a home in which to build interest usage and a developer community hi techton.

D

Questions.

B

Yeah, so uh if I may start um awesome demo by the way, thank you excellent.

B

um What I know you already mentioned it's, so you answered some of my questions by the last couple of slides, but uh what does it take to get this thing installed um on a you know, on-prem and uh what it? What is its um dependency on currently on prow and jenkins acts? What do I bring in parts of prior or jenkins x when I install white house, or do I have to have them installed? First, that kind of thing.

D

So it is completely separate from prow. uh There is from the whole point of it from the beginning was that it was the replacement for prow in jenkins x. uh It until recently had jenkins x code in its repo uh check, itsex related code, but I was able to decouple that into a separate controller in a separate repository, so you do not need to install jenkins anything jks are anything pro uh for it to work.

D

um The demo I just showed you I installed, via installing nginx, installing tecton, using helm, to install uh lighthouse with values.yaml. I set up with an hmac and oauth token for gitlab uh and the url for my dashboard, and I did have to add the config.yml in plugins.aml config maps by hand. Originally, that's like. I said what I need to streamline, that's something I need to streamline, but other than that it was not.

D

There were no other dependencies. I needed nginx for the ingress for the hook, uh because gitlab needed to be able to call somewhere uh to deliver the web hooks uh I'm hoping to get a uh quick and dirty install it on your own dock. uh Up in the next week,.

B

That would be excellent. um In fact, uh you for your, for your hook into uh you know, triggering things uh techton triggers is is awesome. Yeah.

D

I need to I need to spend more time looking in detect on triggers for some reason. Despite the time I spent tech on that one, and I was just ships passing in the night and I'm not sure yet exactly how much of an overlap uh or how much space there is for integration between the two versus overlap of how they're handling things that, because tecton triggers is aiming to trigger pipelines in response to events.

B

That's the really cool part about it is that yeah triggers I I I had an aha moment when I saw that it actually is able to trigger any sort of uh kubernetes api.

B

That to me was like wow. We could be triggering all kinds of things not just pipeline pipeline runs and other you know any sort. If it's a if it's installed as a crd, it can be triggered.

D

Yeah lighthouse does do a lot of things that don't result in a crd. It is where I'm not sure how what what the what's going to end up making sense in terms of gluing the two together, because, like the slash, approve or slash me out, there's no crd, it just go responding directly to a web hook and having to create a crd to have something else.

D

Do the responding would be more heavyweight than necessary, so it doesn't make sense to completely replace uh uh lighthouse's web hook, handling with uh tekton triggers, but there it feels like there's, probably something somewhere in between like at a minimum, reworking the tecton controller I wrote to rather than creating a pipeline run, it may make sense for it to send a cloud event to tecton triggers. I need to investigate this more, um but it's very much something I intend to.

D

I had a slide about mentioning tecton triggers in here, but then I'm like yeah, but really all I'm going to say, is I'm not sure yet, but I know I need to look into it, and so I decided to delete the slide.

D

Well, it's it's.

B

It's I I think, you'll find it.

D

Yeah, oh yeah! No, it's great! I just I'm not sure how its flow fits with lighthouse's flow, yet just not sure yet.

B

Well, we we at ebay may actually, if this is the case once you have, um this is an awesome tool for where we are trying to uh reduce the number of places where people developers go away from our github enterprise pages, especially the pr page to centralize around that the conversation around that and the workflows around the pr page is anything that helps us do that is that that's it? That is the thing that we need to use.

B

So if more sooners you have something together that allows someone like myself to just go and install the thing without having to bug you guys to no end. um That would be the place I I would actually get someone on it uh into an internal team to actually try it out and see how it goes.

D

Yeah I'll I'll get something up in the net uh by by week, from today, at the absolute latest, um yeah and and I can be reached in kubernetes slack in the jaguars x channels or anywhere on tekton slack, so, okay, cool. Thank you again. No problem.

C

So I have something: it's not a question, but people of you may remember. We had some discussion around this ci robot work group or something few meetings ago by max corby and he actually refers to lighthouse and chat hops. So I just want to connect and review with him because he he is looking for people to talk about these things, work with these things and he even have a proposal for a work stream for the sikh. So I just yeah.

D

I would love to be involved with that. Yeah.

C

Yeah, I put the link to the chat and I will ping you on that issue as well, so you get in touch with him, and perhaps you know work on these things together. That.

D

Would be wonderful.

C

All right.

H

Any.

D

Other christmas.

C

Yeah.

D

All right: well, I put oh yeah, thank you and here's. Some links uh to lighthouse, prow and jacob's x cause you know, jenkins, zack's baby uh and thank you all very much.

C

Thanks for joining and presenting andrew.

D

You're welcome.

C

So, okay, uh anyone wants to add last minute topic or wants to bring something up.

C

Okay,.

A

If not patchy, just a quick, just quick couple of things, I managed to ping jacqueline so she's, making the changes to the calendar right now. She had some access issue because she hadn't originally created it, but that should be sorted and for the the working session on a tecton helm chart. I stuck something in the calendar next week at this time, so anyone who wants to just join in uh just let me know I've invited a few folks, but everyone's welcome.

C

Thanks tracy for getting the calendar fixed, so our next meeting will be on 20 of 20th of august and if ramen gets approval for his slides internally, we could perhaps have that presentation. If not, we continue with perhaps white paper discussion and some other topics. If people want to have discussion on the anything else.

C

So uh thank you for joining today and talk to you in two weeks a nice day.

B

All right, thank you. Yes, thanks.

H

Bye.

H

You.
youtube image
From YouTube: CDF SIG Interoperability Meeting 2020-08-06

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).