Jenkins Cloud Native SIG, 22 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2021.1.22 Cloud Native SIG: Discussion on Cloud Events Plugin proposal and Tekton Client Plugin

Description

GSoC 2021 CloudEvents plugin proposal by Vibhav Bobade https://github.com/jenkins-infra/jenkins.io/pull/4079.

A

So um there's uh one thing I would like to discuss is uh how uh like, in what all places uh can we use the cloud events and kind of like start uh thinking over just how the plugin can be used.

A

And what all stuff needs to be done so.

A

I feel the draft isn't complete in a way because there needs to be a pro. There needs to be a repo, and uh I saw your review with the newbie friendly issues like without a repo and some understanding. There can't be any issues like even with some code I mean so.

B

You mean linking to the open, newbie friendly issues um yoga I in the jenkins project. uh The issues aren't are linked in the in the jira, so not as much github issues, but that's that's the way um they are handled within the jenkins project.

A

So uh these newbie friendly issues would be related to the plug-in only or any newbie friendly.

B

No, those are just uh jenkins, newbie friendly issues, so I I couldn't think of I didn't have a top of the mind, a newbie friendly issue that would be helpful for someone trying to get to grips with um cloud events for jenkins.

B

um But we can, we can look for some, although that might be a little bit harder to find you're. Can I can I ask you about your idea of having um of looking into what the cloud events would. It would contain like what metadata they would contain? That's pretty interesting. Are you?

B

Are you following um sort of the definitions and proposals that are coming out of either the cncfs cloud, events, work or or that's being done within um the cd foundation, uh with the interoperability sake and cloud events like that's becoming its own.

A

Check now I kind of missed yesterday's meeting because yeah quite late in the evening- and uh it was it was uh it was much earlier, so I I haven't been able to follow that, but my uh the idea I was having about the metadata with the jenkins was around how techton kind of uh handles them.

A

So my entire idea was like how techno does it so once once someone subscribes to like cloud events for jenkins, they would get to know uh if a job has been started if it's failed and like and things things like that. So probably uh this would mean that uh the uh when, when someone subscribes, we should be able to give the data for the uh what all jobs they have subscribed to, or even if uh like this this is, these are the nitty gritties. We need to kind of discuss and figure out.

A

So this is the discussion I want to have like. How does uh does the user subscribe to a particular job, or do they subscribe to uh yeah, basically a particular job, and once they subscribe to it? uh What all stuff goes through uh to the other end of the cloud events uh pipeline uh this? This is what I was thinking about, so uh we probably need to make a table of uh the what the metadata looks like. So this could be an action item I could.

A

uh I could take and make a drafter table for this one, something like io.jenkins.uh job, probably jobname, dot, started or true or false. Some something like that. So.

A

We could have this and.

A

So just thinking this makes sense right. What do you think.

C

Yeah, I think that would help to kind of um sort of fleshed out a little bit more.

C

Yeah I've got a few like action items on my side I need to. I need to go and like be a sign up to the sig um for cloud events and do a bit more research into it um before I can provide any sort of yeah value in my review.

A

Really, um probably probably, next week uh what we can do is we can actually do action items for like a bit of research and then we can uh kind of brainstorm over what all stuff can go through. So we could do that and.

A

So this this would be helpful when creating like uh event, listeners and techton, then uh like hooking up the jenkins uh cloud events and cloud events like easily in jenkins itself, so something happens in the jenkins job, and probably all of that is that stuff is propagated to a tecton of tecton task run. So I'm thinking along those lines.

A

If we can kind of have a chain of uh things running at some point in time, but.

B

A

So let uh let me just get the dock.

B

You just said that um a jenkins dog job will have a corollary of some types with um a tecton task is that is that which it was I understand correctly, and then you want to compare the metadata between them.

A

Oh, so, uh no not a corollary, I'm not sure.

D

What I was thinking.

A

Was uh the cloud events that uh the that jenkins will create that cloud events could be listened on by tecton and that configuration could be done through the tecton client plugin, uh so that you know that tekton taskrun listens for that cloud event for of jenkins. So this is this. I was think this is what I was thinking. Okay, so I'm just thinking out loud, um but uh but we should uh before we go there.

A

We should just uh focus on what cloud events uh can do and kind of figure out the user stories around them. So.

A

Once that is done uh so initially, what we'd have to do is uh we'll have to play a little bit with the cloud events sdk make something simple and then uh figure out how that works with jenkins, and if we maybe enable cloud events for a jenkins job, there should be probably be an endpoint uh like endpoint uh one, something that can be subscribed to, and uh this and whatever whatever other tool is that they should be able to like listen for events to that end point, so this could be like a first prototype that that could go about so that we understand like how cloud events work and then for the gsoc part itself.

A

uh I want to ask like how do we uh I've never done gsoc before or like been a mentor? I've seen like how it goes about, so um I'm just. I just want to ask like how how do we? How do we guide students to do uh particular things like? How do we figure out what they can do and what we should like help them with, and basically just how do we guide them and give them things that they can learn.

B

I am going to share um your I'm going to share in the chat right here the jenkins guidelines for for mentors, so it just gives you a lot of information that you know has been written over the last couple years um in terms of guiding students.

B

It is, it does get pretty, I would think pretty individualized in terms of the projects in terms of the mentor's background and then, most importantly, in terms of the students background and what knowledge they need to gain in order to engage with the project idea.

B

So yeah that leaves some room for creativity for the mentors.

A

Okay, that's good okay, so I'm thinking uh so for the cloud events plugin. Should we bootstrap the project before uh starting it off or do we make them uh start from scratch?

A

Bootstrapping would could help uh quite a bit in helping the student to get started.

B

C

Think that's a. I would say that I think that's. I think it's probably a pretty good idea um just going on my experience of getting repos created and set up and things like that and like a basic, build on day, one um that can take quite some time, so it might make sense to do that. Yeah.

B

I think the students would really appreciate it and, in addition, this summer's gtalk schedule has been altered from how it was traditionally done, and it's a shorter coding period. So you know, if you everything we can do to help the students like hit the ground running um for their coding phase would would be really I'm sure, they'd appreciate it, because then that will enable them to do more substantial work for the time that they have.

A

Yeah, I I remember I was uh I did ask a friend about it, who did it before like it was three months and now it's a month and a half right.

B

Yeah, it's basically been cut in half, so it's a lot shorter in practice, it's more because the students start engaging now learning about jenkins in general and thinking about what they want to make a proposal on. Then they formulate the proposal that takes some work and engagement and then, if they're chosen there's a community bonding phase, which is um a number of weeks so at which time they're kind of like getting ramped up, um but nonetheless that shortened full-time coding phase is is, is a limitation.

B

We have to deal with this summer, um so everything we can do to help the students is great.

A

Yeah, actually uh for to start from scratch would have been nice if it was three months, but something like tecton client plug-in, which is a little bit in still poc phase and there's still a lot of stuff. They can do.

A

uh You know that, that's so that that makes a lot of sense. Yeah then we'd have to bootstrap the project, then, because one and a half month is quite, I can't imagine doing anything enough in.

D

One and a half month.

A

Yeah, it's quite a short time, I'd say uh so yeah, I think uh so. When we bootstrap till what extent should we bootstrap it? Should we uh should we kind of boost up the cloud events sdk and the metadata stuff.

C

I guess I would have said I think, if it's a plug-in that we're creating, I think, if we can have a kind of like a skeleton of a plug-in that does yeah, doesn't have to have too much in there. But as long as it has a you know, jenkins file, that's doing a build and some kind of tests, and we have the ability to create a you know, an alpha data release or something like that.

C

Like an early release and people are able to at least I suppose, install the plug-in, even with a few manual bits and bobs but like there is a bit of a build and a bit of everything else going. But I think that's good.

C

um I don't think we need to get too much into the internals of how the plugin will work. Yet.

E

C

It would it would be nice to be able to you know, clone the project, build it locally and easily push up a pr and have valid automated feedback on that pr, for, when they're ready for when the sort of students start on.

C

This especially if we're going to go a month and a half.

A

Yeah yeah that'll be good. Actually, I think I think then, over the next week, this would be like our main agenda to figure out how to kind of fill the extent of the bootstrap probably play around with cloud sdk cloud events sdk, okay, now yeah. That makes sense.

A

Okay, um I I don't know, uh I don't know anything else to discuss about the cloud events plug-in, maybe maybe or something in on my on your mind, gareth. Oh.

C

Nothing um from my point of view, um I I need to go and do a bit more reading about this, and I need to look at the proposal in detail. um Yeah we're going to schedule some time probably later on today to do.

B

That similar, I think this looks good. I think we have a number of action items that I've put in our um meeting notes for us all to look into and and consider so hopefully, we'll have more on these uh next week to circle back and discuss it's pretty exciting hello sagar did you have any questions.

E

No hello, everyone, uh no I'm just I listening.

E

I'm just trying to understand um so that I can explore this one also and I'm little bit um curious in docker and kubernetes and cloud events, and this project is related to that. So that's why I'm just oh yeah! That's all.

B

Do you have um since we're all here on this call? We could we could uh find this for you. Do you have a good idea how to get started in doing the research that you need to do for understanding more about cloud events.

E

um For now um um last time, um a little bit, he said that first try to understand, docker and kubernetes so, um but for understanding your current kubernetes, I need to know the rest apis and back and stuff also so this week I learned about mongodb and express and node and all the stuff. Now I will try to um now. I will um integrate docker with that and then further I will move on. Hopefully, then, what problem? Actually we are solving yeah.

A

All right, that's good yeah sounds very good. I'm glad you're having a good time with it.

A

A

So, if so about the tecton plant plug-in, I I am still thinking so I didn't end up getting to the draft, because I was still thinking like uh what will uh what can be even done about uh the client plug-in like it's. It's not it's. It's in a phase, it's a it's, uh the poc is done, and uh there are. There is a lot of stuff the students can actually do uh if uh considering the timeline. That is there right now. Initially, I was thinking.

A

uh Maybe three months uh would be a little uh too much and maybe the uh maybe the project might get over prematurely.

A

So I was just thinking if it is like how do we handle that, but I think in the scenario that we have right now, it makes the the tecton plant plugin uh kind of fits very well into what can be done so so what I'll do is? I will create a draft for it and then share it share it with you guys on on the gso channel and also uh on guitar channel.

A

So I'll do that and anything like in particular gareth like uh you would like to, or uh you have any ideas for the client plugin like. Maybe there is that one feature we would love to see after that, one and a half month of coding, probably catalog, support or something that would be very cool or like dynamic uh tasks that, like like picking up our tasks like dynamically from the drop down menus for a task run stuff like that. uh That is on your mind, anything.

C

um Yes, so I've been on your recommendation from last week, I've been looking at the tech honors code um sort of is it text on his code or company and proposal, um and this is sort of a few of them out there in a few different pocks from what I can tell, but that's the kind of I suppose, that's the kind of thing that I was envisaging so not so much the ui side of it or the ux, but more of the um yeah like a like a dot on folder or something where you can automatically pick stuff up from within your git repo and have it um so.

C

However, that client would work. It would need to pick up the particular one and schedule the pipeline run with the right parameters.

C

um It's it's a yeah, it's a bit hard to say because I think at the moment, they're just proposals. So I don't think there are any sort of like concrete implementations of how that's going to work anyway. But.

C

um That might be a good.

A

I'm just thinking so would would jenkins be like an intermediary between tecton when we do this.

C

Yeah, I mean that's the that's kind of how I was thinking um we have. We have users that have you know large jenkins installations and lots of builds, and they want to want to go all out on techton from day one. So providing like.

C

You know whether it's the ability for them to view all their jobs in the same place and have a you know, a git repo that is listed and looks. That seems to be configured in the same way. And but when the job is run, it's actually kicked off as a text on pipeline, and then that could be. You need a jenkins file and a text on folder inside the same kit. Reaper to be able to do right, just have have.

A

A method uh thinking like it's it makes like I was thinking if like if we could do something like have a jenkins file only but make it run like technon dsl, so yeah that that scene, because if we, if we do like a dot, tecton the tecton as a code developers they've already kind of that same monopolized on the dock, not take down part. So I don't know, maybe maybe.

C

I mean the doctor. The doctor is quite nice because it fits with the you know, the lighthouse for jenkins x and dot github, and all that for well. If I forget how um it does do it, maybe it is just an integration between um the tecton client plug-in and the tech dollars code like have the ability to to do that kick off.

C

um Maybe maybe that well, that's just kind of simple as it needs to be to begin with.

A

um How do you so? How do you uh see the tectonics code or integration with tecton's land as a plug-in? I'm.

C

Not I I'm not sure yet.

A

I'm I'm thinking it's a it's a good idea. It's a good idea. Yeah. It probably needs so because the scope of the client plug-in and the tecton is a code. I'm just thinking how that would work. Yeah.

B

Can um can you share the text, honest code, like a link to the tectonics code in our chat and I'll put it in the.

A

A

One second.

A

C

I'll try and follow from this github issue.

A

That is the tectonic code, repo.

B

A

Awesome so this one is uh being created by one of my colleagues and he is uh bootstrapping it as well as making it production ready, and it's it's doing actually quite well. The only uh thing are waiting for is to get this merged into tecton like the tecton cd.

A

So but it works very well actually and it uses the tecton dashboard or to see all the tecton tasks running. It's quite good.

B

A

So probably what we could do is we could read the tecton as a code code and then understand like how it works and probably recreate the uh experience.

C

Yeah, because this is python, isn't it I believe yeah.

A

A

So this would be recreating the experience in java.

A

B

So when you say recreating the experience in java, do you mean recreating taxonomist code doing a java version of this, and then integrating? That with is. Is that what you're referring to did? I understand that correctly.

A

Like yeah, what I did for the tecton client plugin was kind of similar. I read a lot of the technon cli code and I basically just understood how a lot of the stuff worked there and then I just replicated that entire thing into a jenkins plugin so including uh like how uh getting the logs for the tech tasks and all like simple stuff. Also like it. It worked out pretty well nice, so.

B

A

So we could probably do that.

A

And uh we just need to figure out if it makes sense to do it or if it if it is even possible. That would take some reading. But uh probably we could keep this for next to next week.

A

So because this this week seems fully uh fully of full of cloud events, so probably you will set it for next uh next next week,.

B

Okay, so so for them for our next meeting, we'll focus our research on finding doing doing these action items on cloud events following week, we'll focus a little bit more on um the saxon client plugin. I think.

C

B

Probably a good way to break that up.

A

Yeah, that sounds good actually, so we could uh next week, as you said, we will discuss cloud events and uh the extent of the prototype, as well as the task students can do for cloud events plug-in and then post that we can discuss the tecton client plug-in and what needs to be done for that, and maybe a few issues that uh students can start picking up for the client, because there are actually quite a lot of issues that uh students can just start picking and you know playing around with it.

A

uh Probably we can just uh so I'll, create a draft and share it today uh after this meeting so.

C

I've just reached out to the james's on the jenkins x, dev list to see if um to see what they're doing with the lighthouse, so when they they have something that is very, very similar to the tectonics client clicked on this code, um piece where it reads everything from the from a lighthouse directory and the reason it's called.lighthouse is just because of that's the name of the component inside jx.

C

That does the work, but essentially it's just a it's a dot folder that has um a list of tasks, gamma files or tasks and pipelines or whatever, and they get applied um and run what I'm asking them is. Are they doing any manipulation or rewriting, or you know, enhancements of of those files before applying them um inside jx?

C

um Because I think that would be. I think it's quite. I think it's very similar what we're looking at doing we're pretty much looking at a yeah java implementation of that piece, so it can run from inside um jenkins.

A

So yeah that would be, that is, that is actually what we're looking at. What would be uh very interesting is to have existing uh jenkins users just run the jenkins file, with some simple dsl for techton and like they can just you know that they can just like they won't even have to think of new semantics, because I think there is this kind of schizophrenia involved with you know, working with new files. Oh my god. It's a new file.

A

I don't want to deal with you files and like it's like if they could just like use jenkins files and then uh probably just uh write some simple dsl like create tasks, colon and then put like task name and give whatever.

C

Yeah, I I I think you've got you've got people that will they'll want to do or recreate certain parts of their pipeline as techton, but there'll be other bits or other plugins that they'll be using that just make absolutely no sense to convert so like if you're talking about lacking, like um maybe archiving junit results, visualizing them in the ui. Maybe you you have the ability to run the in the jenkins file. You can run the protect on part of the pipeline first and then the end stash this information somewhere um store.

C

This render this and run another plugin. On top of that, that may make sense.

A

Yeah- and this would integrate like very properly with their old, uh like the old pipelines that they have.

A

The idea of like completely repurposing your uh ci for a new one like your pipeline for a new one, is: it is taxing in a way from users, perspective yep, so actually, next week we could actually uh think more about how this could be done.

A

Should we focus on our dsl uh for techton and jenkins file and also look at like dot tecton.lighthouse, probably we can have like a session and understand like which one you know would work best for uh what we need. So I think that would be good. You could do that cool.

B

I think that would be really useful should we aim to have this the session during this meeting, um I'm just thinking it would be kind of fun if you have time to try and pull in one of the changes or something I don't know what their schedule is.

C

um Either james strachan or james rawlings or.

A

James and paul james or james yeah, okay,.

C

I can, but I can bully one of them to pretend.

C

I'll bribe change james rawlings with some.

A

Wine; okay, uh that that sounds nice. Actually, we couldn't, we can uh take, uh we can ask them, maybe it would be nice actually they if they could provide the perspectives. It'll be it'll, be pretty good.

B

Great okay, um very nice, any any ques, any other questions or action items or issues to discuss for cloud native in general or or for gsac, but but any other clan. It doesn't have to be g-sucks.

A

Not from my side, I just I just wanna start attending the cd foundation meetings a lot more often yesterday. Yesterday, when I missed them, I was just like no not again after the timing change.

A

So not not not anything. In particular, I think we've discussed enough and have a lot of action items actually.

B

Yeah yeah, we do um great, I am so happy we are. We are moving this work and parceling it out uh for g-shock students, because I think this is a really exciting area for them so because it.

C

Is is that the events in city work stream in town.

B

Yes, um but that is that is, is it something called a workstream? It's really on the cost of becoming its own sake. I think it's just just.

C

Yeah, that's what it's in the cdf! That's what it's in the cdf calendar at the moment as that, but I think the next one there isn't one next week lots of things. I think it's every two weeks, so next yeah first of february.

B

Yes, let me pull that up and add it.

B

I have so many uh event calendars. It's a bit much actually.

B

I'll put it uh in the chat notes for the the cdf event calendar, so you all have that and um I'll put it in the notes from the meaning as well.

B

A

A

Guys, where are you located? Are you working.

B

B

I am based um in london.

D

And I'm in northern ireland.

A

Okay, that's nice! uh I'm based off of bangalore.

B

So you're yeah you're totally what time is it right now for you just so, I know it's better.

A

It's it's eight seven.

B

Okay. Okay, so that's not it's not it's not great, but it's not terrible. I could see how yeah two hours later it was much worse. Okay, I'm glad we moved this meeting good um excellent. I've just put the link for the cdf at the top of the the meeting notes, the calendar, it's kind of a funny-looking name, actually cool good. All right. I think we have covered a lot of ground um today's meeting. Thank you all for being here.

B

We have all of us a lot of action items to do, especially if you have thank you for leaving these two g-stock proposals. They're both super exciting and I'm really glad we are moving this cloud native work um and making it available for students to engage with so excellent. Thank you all for being here. Do you have any last minute questions sagar, anything that we can help you with.

E

Actually, no, I mean I'm just trying to.

B

Yes, good good, we all are excellent all right. Thank you all very much. We will see you at the next meeting.

A

Thank you guys. Bye have a good weekend bye. You.

D

Too, cheers you too.

D