Continuous Delivery Foundation Interoperability Special Interest Group, 4 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CDF - SIG Interoperability Meeting 2021-03-04

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

B

C

Good afternoon, good morning, wherever you are thanks good morning to.

A

You so let me share my screen.

D

B

A

Hello, how are you good uh trying to figure this daylight saving time thing I? I am even more confused now now, when I try to compare.

E

Yeah, it's a very awkward week.

A

Maybe we should cancel these meetings that fall between you know no man's land or whatever it's called.

E

I I think, that's a very good policy, just no meetings for the week.

E

I mean how I've seen it done in in other meetups is that it just rolls forward, and so the people who have shifted time zone if they can make it they can bang it and some same for the people who haven't yet gone through daylight savings.

E

um Otherwise it is very hard just because some people have this fall uh and in a new time during their day, and they might have other things. Yeah.

A

Because like if, if you remember kara when I asked like this, 16 utc was shown as 16 for us or something like that. But when I checked this time is website, it still shows 5 p.m. For us for 14th or 18th of march. But yeah we'll talk about that. When we come to the.

E

That that is the case, something weird like that happens in the uk, because london time isn't sometimes it's utc and sometimes it's not england might be even weirder.

F

Let's yeah, we have pst in summer british summer time, which is utc plus one.

A

Yeah we have topic for that, so we can start. Then thanks everyone for joining and if you haven't done it, please uh record your participation and here is the agenda. So, as usual, we will start with action item review. We have few of those and I think we will be able to close at least half of them, which is always good, yeah.

E

Are are you recording, um I'm I'm being told that I need to ask permission of the host and it might be because I'm still using uh the jenkins zoom, so it may not recognize.

A

Me I see blinking red dot here, I'm not host.

A

I can't do anything, but I think it is automatically recorded. Isn't it.

E

A

Okay, just wanted to make sure you see. You see that as well, yeah, somewhere clouds record something: okay,.

E

Perfect, okay,.

G

It looks like it's being recorded.

A

B

A

Okay, so uh once we are done with action item review, we will talk about daylight saving time changes because, like us switches chain makes the change 14th of march and europe next change on 28th of march, and then the next meeting falls in between which messes the calendars up and then we will give a heads up on upcoming discussion with tina portman, which will take place on march 18th, and then we will look at police during cicd documents, because I've seen traceragan added some notes about deploy hub and deploy, help pro and or tell us.

A

I also uh put some notes about ebay, ericsson and zul's take of policy based on the meeting recording, and then we will visit standardized metadata. This work seems to have stalled a bit, so it would be good to you know, bump the topic up and then we'll have a presentation from oliver on project piper, and then uh I don't know who edit the topic but cdcon.

A

So yeah. That's a good reminder. uh Maybe we can take that first before before the presentation, because it may be difficult to come to that topic. Based on how long the presentation is so, let's start with action items uh so dave sudhir. I don't see him in the meeting and, as we all may remember, he had some changes in his employment, so he may not be able to contribute as much until those things are sorted out and steve. You uh volunteered to help with the artifacts metadata thing.

A

So I'm wondering if we should keep the section item open and you just you know.

B

Yeah, I'm behind on that. So just keep me on it. Okay, so I move that to you then still.

A

Okay, so I'll tell you uh steve and trace to help bring all the standardizement of the example in liverpool. I think we can add this to the previous section item because you already volunteered for artifact metadata, so you do it yeah.

G

That's fine like, like you said we did update some of the one of the documents to reference ortulius and deploy hub. uh I can't remember which document it was, but we did start on some of that yeah. It's policy. I think.

A

Oh, that's right. That was.

G

A

Policy side yeah: I should put a lot of good information there, which will uh quickly look at it thanks, steve. uh The next action item is on tracy miranda and she already got kara and me in touch with tina graves portland about the discussion on four keys project and how we can collaborate on uh common things, because four keys project is about measuring uh delivery performance, and that requires some way of collecting all these metrics, which brings suspect to metadata and perhaps range topics back.

A

So this uh discussion and potential collaboration could be valuable for both cdf six and projects and four keys project, and she will join to our meeting on march 18th. So we have a topic about that already. So we closed action item.

A

About policy on the document, from order to use and deploy hub so closing that one, the next action item is on you and andrea and christie includes some information in the policy document. So we can close the section item if you think she captured the conditionals you mentioned, or we can keep it open up to you.

F

um I've not looked at it. uh Sorry about that, so this christy added it. I think he should be fine.

A

Okay, I close it well, I I looked at what she added and she didn't use the word uh conditionals that that's why I asked this. So maybe we closed this action item and you look at it if, like conditionals, is something used by community, it would make sense to add that keyword in the document. So we can, you know, translate the things between different projects.

A

Okay and the next action item on uh next site to me is about providing codes for cdf events announcement and I already shared the document link to document with andrea and cara. I don't know if you took this into account when you had a discussion with.

A

I don't remember the name of the lady, so that document is there.

A

You can close this action item.

A

I was supposed to add user project input first account based on the discussions which I did, which will locate it when the topic comes or closing this one and then uh project piper to the agenda that was added as well and we'll have the presentation today.

A

So we closed almost all the action items, which is always great like this is amazing, but thanks everyone for doing all these things.

A

Let's move to the tricky topic, daylight, saving, time changes, so, as I mentioned at the beginning, us makes the change on march 14th. I think it is next weekend and then europe makes the change on march 28th and our meeting on march 18th falls in between and that confuses people at least I am very confused like what time we should have the meeting.

A

So, according to our usual time, 16 utc, it is 9 a.m for pacific time and 5 p.m. For central european time. I'm not sure if this is correct.

A

So what should we do for the next meeting.

H

Those who are in the meeting I can follow the link I pasted to see what their time zone will be, what that time will be on the 18th of march. I guess that confusion. Could it just be lifted? Okay, for.

A

Us it will be 4 p.m. In central european time.

H

My fourth previous will be 5 pm in sweden. Oh sorry, 5pm.

A

Yes, you're right, yeah.

A

I mean you are in pacific time zone.

A

C

A

And what time this link shows for you.

C

Well, let me go.

B

C

See what it does so, I can write the times.

C

It says: 4 p.m. On thursday march 18th on the the next line, uh the line- okay, uh 9.

B

C

G

Okay and for me, it's 10 a.m, because I'm in mountain, so that time works for me.

A

10 am for my time and jeremy. I think you are in mountain or.

I

Central, so it's just 10 a.m.

A

I

Four years but eastern america is here.

I

East coast, you mean.

A

Yeah, and is it like 10am or 11am, for you.

I

um Right, no! Well, I really go by utc, so uh it's a little after 11 local time here. Right now,.

A

Okay, so somewhere, let me put that and then for a cara. It will be 3 p.m. Then, for you.

E

Would be fine for me, so it doesn't it doesn't matter, but I'm I'm actually saying it will be four, but in any case I'm flexible on it. It doesn't not matter for me.

A

Then I put this like this. I hope this is right. So if this is wrong, please update correct, because I will send a separate reminder saying that daylight saving time happens and uh it is 4 pm utc as usual, but check your time on this link. Let me put this link thanks emma for that.

G

Yeah and the the 11 a.m should be 12 8 a.m. This.

A

G

Yeah, okay 12 noon instead of morning.

A

A

How many people will join, because why? I am making this a big deal, because we asked dina to join our next meeting and I want to ensure she joins the meeting on the right time rather than joining the meeting at wrong time and finding no one.

A

So that is uh sort out them cdcon, uh who edit this topic.

E

I put that in there um the hard deadline is this friday, the 5th of march, so it would be great to have lots of topics and talks on interoperability with all of your projects, but in general I mean you know, it is very good for you to submit talks to cdcon, but it should be excellent and there's a couple really interesting tracks. So do have a look at the website. Thank you for putting that in there that's and um get your talk. Submissions then.

A

So end of day, tomorrow,.

C

Website in chat, please.

A

Yes, so it's middle of the night like uh tomorrow's pacific time, 12 p.m or 11 59 p.m, or something so we have a little more than one day to get the proposals in thanks for reminder. Kara. I forgot about that, because I already sent my proposal for early bird and I failed. I could have made it early bird. That's why I remember okay, so the next topic is uh the meeting with dean apartment, so portman from google.

A

uh Let me open this project, so she is working with the four keys project and, as you read the uh project description background, uh they are working on uh identifying four key matrix metrics to indicate performance of software development team so- and it says, focus- allows you to collect data from your development environment such as github gitlab and compiles it into a dashboard displaying this key metric metrics, as we have been discussing within our group's meetings like this, is one of the things we are also talking about how to collect these metrics and when tracy miranda suggested that we should have a discussion tina to find opportunities to cultivate in this area.

A

It was pretty good to hear because this topic is also end user aspect. There is a topic in end user console about measuring devops success, so it's like cdf end user console is looking at that thing, and four keys project is looking at similar thing and we are also looking at how we can establish some kind of interpreting domain to make these things easier. So this discussion is to find talk about those things and find opportunities to collaborate further.

A

So please join to the next meeting and she will present four keys in more details like 10 to 20 minutes presentation, maybe followed up by discussion, and then we see where we end up based on that discussion.

A

Perhaps we should send a separate email for this as well to talk mail list, ml, ops, maybe and other mainly so people can join to this meeting because it is more than intervality. To be honest, so this is just a heads up.

A

E

I think that's a good point that we should be spreading um encouraging people from other cigs and groups to join I've encouraged the jenkins cloud native's sake to attend and there's been a lot of interest so in your own um projects and other groups and meetings that you go to, you might wanna tell people about it. It's a very interesting project. The four keys.

A

Yeah, so thanks for doing that, uh maybe andrea you are involved in uh takedown community yeah, I'm not sure if, like uh it would make sense to get people attention to this discussion because her talk was great and now we have a chance to have a conversation with her. So.

F

Yeah I had a need to chat with the rest of the governing board and not much happened from it. Yet I guess it's, but it's definitely an interesting area. So um I think one of the points in dinner presentation last year was the faculty of trading different events from tecton, because there is no kind of obstruction there about what pipeline being deployment or specific item there.

F

um So I think that the work that we're doing in the event sig might help there and I will try to keep a close loop between the work that we're doing in that group and tactile to make sure that everything we defined. There is available object on site.

F

Thanks, andrea.

A

Okay, so mails will be sent to these like uh mail lists.

A

The next topic is policy driven ci cd. Let me open that document.

A

So now we have tracy dragon does no still, you want to summarize what tracy contribute to policy document or.

G

Yeah, so let me just get this right, so some of the things that we um were looking at uh at a high level were around licensed policy enforcement, uh cve enforcement and uh configuration uh policy enforcement. So the first few are pretty obvious. You know: um are there any um licenses or cves that should prevent us from moving forward with the deployment of a particular version of an application and then like on the configuration side.

G

It is more around the configuration of where we're going to uh matching up with what is needed, so a needs versus what a needs versus have uh policy um are some of the things that we're looking at on that front so like in the example, um is what we're requesting coming forward on a configuration basis.

G

You know: are there enough nodes or backing the pods or you know, do we need more compared to what is being asked for so that's kind of the the thought process around on the artelia side and then on the deploy hub pro side, we'd be looking at hooking into authorization policy who can do what and where, as part of that process and then in deploy hub, we do have a a concept of approvals and approval gates um and that would be around who can approve who can use an approved version of a component or an application.

G

So this has come about where you look at um teams that are organizations that have um a group like an open source group that they'll go through and and say that this uh is a good, let's say: mongodb. This specific version is a good version to use, and that goes on their approved list uh as part of that process.

G

So that'd be something that we would be looking at as another policy and then again, the licensed cve policy and the configuration policy are carried forward from the ortilla side and then one of the things that we put out there. We don't know how easy it's going to be do to do from a policy perspective but trying to rev the duplicate component policy, which we would try to enforce.

G

Reuse instead of copying um of code as part of that that level and then finally, the sll sli, which would hook into more of the uh telemetry that we're getting from the uh clusters themselves, the runtime environment and how we should react to what's happening uh at that level. uh So that's kind of a high level of what we saw from a policy standpoint for both artelias and deploy hub.

A

Thanks steve, I have a question. Actually you know when we were talking about police topic during previous meetings. Few policy, engines or police frameworks were mentioned, like open policy agent and open network automation platform uh policy framework. Do you use something like that? An external compound to enforce policies or like write policies as codes or something like that because, like spinnaker uses open policy agent and what ortelius uses.

G

Right now, we do not have uh any policy hooks, so we're currently not using any policy agent per se, but this is what we would be envisioning if we were to hook in something like open policy agent um in the process. So that's one of the things that would look at as a next step is to figure out how we are able to enforce these policies that we're looking to do.

G

A

Any questions to steve about or to use our depo policy approach.

C

What is the link to this document you're? Looking at I'm on the cd foundation, github org and I see the I see a doc there, but it's all tbd.

A

Yeah, I I haven't pushed the latest version. Let me put the link to chat. I mean sorry for that. That's okay,.

A

So uh techton andrea, this is the part christie included so yeah. You can take a look at it to see if you have anything to add there yeah and then my action item was about bringing the discussion from last meeting to this document.

A

So I wrote rule pieces here and jeremy if you can take a look at it to see if I described what you summarized or intimidating captures or if I was able to capture what you described, which document is that again uh it is uh you don't use zoom, so you can't see the link, but I uh I sent the link to seek intervality mail list as well this market right now you can yeah.

I

A

Pull that up thanks, because I probably couldn't capture everything you described there. I listened a few times, but I failed to reflect the notes.

I

That's fine! That's uh I'm happy to double check that.

A

Yeah so uh before we move to users like uh cara jenkins, I, as I mentioned too like few weeks ago, jenkins, has a plug-in, I think, cloud-based plug-in it's not an open source plugin, uh and I didn't include that explicitly because I didn't want to you know misrepresent what plugin does. So, if you could like help putting jenkins information here or ask your colleagues.

E

Yes, can I also we're interested in what the jenkins open source project has and, in addition, we're it would be good to put the information for the cloudbee's product.

A

Yeah, because when I googled jenkins policy.

E

No, no I'm I'm just asking from our from our point of view where we're discussing and and I can see why we would but we're discussing both the open source and the paid products yeah. I.

A

Think both should be good to both would be good to have so we see like how the communities approach this, how the offerings, like you know, uh enterprise versions, yeah, okay,.

E

Okay, I I will take that action. Yeah.

A

Let me add that action item before.

A

A

Okay, so spinnaker uh someone already put that was it christy. Last week, spinnaker uses our armory flavor of spinnaker supports opa and uh she put that link there. So I will uh add some text here. Based on the content, I will reach out to cameron to check if what we put the reflex reality into each.

A

Okay and then, uh when it comes to users, so I mean I again similar to what I did with rule. I tried to extract what you summarized during the previous meeting. I included it in the document, so please correct what I put there. I include like badges and central system handing out badges and that actually enforces which applications could go to which.

A

Environments like uh staging clear production and products, so if you could verify what I put, that would be good.

C

uh Sure I'll take a look yeah.

A

And same thing, for you emil, I tried to put confidence levels from ericsson there. So please check that and you know update as you see fit.

B

Here um just fyi the.

C

The approach the approach we've taken is to decouple policy enforcement away from uh continuous delivery workflow engine so that the cd engine does what it does. It's going to still go to the next step, go to the next phase, and only if, if things fail in between it gets signaled, it itself does not have uh any sort of a you know. Policy has been not been written into the cd engine, it's been, it's been it's been enforced by other things and which then can signal the cd uh workflow to stop.

A

So so you can see this diagram here I am showing. I think this is something like what you described like you. Have this cicd pipeline technology like tools, but you have an external system who actually drives policy enforcement stuff? Yes, okay, so it is pretty similar to our use case like, as I mentioned during the previous meeting. We use events and events actually contain the metadata about, like certain confidence levels or badges. In your context, I think we have some common uh think uh surfacing here.

A

Sprinkler uses open policy agent as an external system and behaves accordingly, rather than keeping the policy built into the pipeline technology itself.

A

Okay, so uh I mean: can you add that sentence while you are on the documents in ebay chapter on the document, because it's a critical point to highlight external system, uh doing that.

C

I'm sorry: what was that.

A

Like what you just summarized, if you can add that to the policy document.

C

A

Yeah, I said central system, but yeah you described much better than me like this decoupling.

I

A

Yeah, okay uh and then I started adding some terminologies things like ebay calls stuff, badges and policies are enforced based on badges and excellence, confidence levels again they are used to you know, enforce policies like if you can deploy a certain application with a certain confidence level to this environment or that environment, and then we can continue mapping uh the terms from the projects like tecton, uh spinnaker, jenkins, zul and so on, and that that gives us at least some way to tie these efforts together, saying everybody is doing something similar, and these are what they are called in those projects- and this is their approach either built into the pipeline technology itself or handled by external systems.

A

So any other opinion about the policy.

A

A

Okay, please feel free to update the document. It is out there and I will push this version to the repo as well, so people can find it there too. I think we have something here because, like everybody seems to be doing similar things calling themselves differently, calling those things differently and if you can find a way to bring these things together. That would help us in future, as well.

A

So going to the next topic standardized metadata. So this is the hack md document and, as I mentioned, when we were talking about agenda.

A

This work has not been progressing because of perhaps dave's uh changes in circumstances steve. Thank you for adding commit uh thing here. I think that is good enough information to start with and if you could get the artifact in place as well, that would help, and then we can talk with dina as well, because they are looking at git information from git and they are probably looking at information from artifact repositories.

A

So we can at least point her to this work, saying that this is what we started. Does it make sense to continue with this? What else we should look and so on, to have the conversation.

A

Okay, that was a reminder about the situation with standardized metadata work. Anyone wants to say anything about.

A

Topic so now uh we come to oliver, I stop sharing and you can take over.

F

Okay, just sharing.

D

Let me know when you can see the screen.

A

I can't see it.

D

Yeah and then let me introduce you to to piper, which is an open source project since I believe end of 2017, where we open sourced it. We started as an inner source project within sap. Today, it's essentially both. We have very sap specific aspects which are still maintained as an inner source project, but the majority of aspects is part of the open source project and we are probably the biggest user also of this open source project and within our development organizations, um just some fun facts for sap.

D

um If you like, you can read through it in past meetings, there was a question about the scope and in the end, we talked with an sap about more than 30 000 developers, I'm working on many many many different projects, different acquisitions, a lot of heterogeneity.

D

Now what is piper, I tend to say it's shared code in knowledge or shared codified knowledge, which is ready to use for a development team on the top. We essentially provide what we call ready-made pipelines. That's a continuous delivery pipeline or even continuous deployment timeline pipeline from code commit to production deployment.

D

This consists of several steps which come out of the piper step library, the steps they are responsible for interacting with the various systems you want to have in the flow it starts with versioning an artifact we'll look into that in a minute, I'm building um publishing to some some artifact repository, performing security scanning, open source scanning performance tests and so on, and those system connections.

D

This knowledge about how to integrate these with these kind of systems is basically put into these fighter step. Libraries underneath there's a common configuration layers, we'll see that in a minute, as well, essentially yaml file containing configuration for the individual steps and also for the complete pipeline.

D

Now, if you want to get started, um it's essentially four lines of code which you need to put into your repository. Two files um main focus is jenkins, but we'll come to that also a bit later: um teams using jenkins and there's a jenkins library which is included into a jenkins file, and then the pipeline is executed. Therefore, this jenkins file is just can be just copied over sap internally.

D

We by now even have an onboarding service onboarding portal, which asks some questions guides you through what type of project and so on, and it will create a repository and put these kind of files plus the configuration in there simplest way. You just define the build tool.

D

What you want to use you see here in the screenshot above the various stages, where a complete pipeline can run through there's pull request, voting, that's as simple as getting started. Building unit testing integration, testing acceptance, security performance until we reach a promotion now very important. We do have a separation of the pipeline logic and the configuration the pipeline script is. Then I showed it's just. Essentially these two lines of code are provided centrally. They are provided within project piper.

D

That means we allow for central central innovation. People can consume those things without doing anything since most of the teams just use the master branch of the project and with that they just consume. What comes in and by abstracting these things away, we dramatically reduce cognitive load and inside on the other side. There's this team owned configuration.

D

I said you have various configuration layers um you can configure on a step level here you see the example of the versioning. Now you can configure on a stage layer or even for the complete pipeline. Certain things what's possible, for teams as well is that they can break out of the predefined stages, extend them replace them.

D

People have something before the stage after the stage or completely overwrite the stage, that's in the hand of the teams.

D

Now, let's have a quick look at the configuration it's a layered configuration. We have defaults, which come with type with a with piper with the library itself, um with step configuration, detail, defaults, configuration tutorials for basically, overall generally everything you run comes in very handy. If you have like me in the company certain instances of a scanning tool with a dedicated url, we can just put that into the company default and no team needs to take care of it anymore. Any longer, then teams can also provide custom defaults right.

D

That means, let's share defaults across various repositories across various projects where they can prepare, centrally and typically by one central team in this area to have certain defaults on top of the company defaults and then oops from direction.

D

The teams have in their hands what they configure they can again override. What's coming in and the last resort is you can just directly parameterize the steps which we typically do not recommend, because then a pipeline script will not really be reusable.

D

Since you have configuration in your pipeline screen, let's quickly look into the automatic versioning, I think that's a that's a topic which is often of interest.

D

We have one step, doing the automatic, versioning and essentially hiding away the uh the different aspects of of the build tools like working for npm versioning for maven version for go and so on. Right and the only thing a team needs to do is tell the fill tool and maybe add some specifics for the versioning.

D

The default for the versioning is is a versioning pattern with major minor patch timestamp and a full commit id.

D

If your team doesn't want to have this, you can go with a controlled commit id, but there's also an option to say we have a versioning type, which we call library and then we just have a december version, major minor pension.

D

That's all the team needs to do um not caring about the nitty-gritty details of how to read a version out of a pom-xml file. Writing it back and so on now I I mentioned. We essentially started with jenkins um we're a pretty big jenkins shop. I guess within sap, but the world is evolving, continuous delivery foundation and other and other things right. um We see github actions taking traction, there is tactile and there is various various tools I mentioned at sap, we're pretty heterogeneous.

D

That means you find various systems within the company, banking's being still the majority, I would say, but other team, other things really getting traction now. How do we address that? We call it agnostic, continuous delivery.

D

We we do have this configuration file. I gave you not give you now an example again with the build tool. Npm um some vault configuration for credentials since we want to have also credentials in a place which can be reused not in the jenkins credential store, but somewhere where it can be reused.

D

um We still have some jenkins stuff in there for people who are on jenkins to essentially access this world instance now, looking at jenkins, if you're familiar with jenkins um scripted pipeline in this case, in order to do a versioning of an artifact which you check out, it's it's. Basically, these lines of powered radio empty your workspace just to save, check out the repository and do the versioning how the version versioning is done. If it's major minor patch, what have you it's just written in the config file?

D

Therefore, you can also across teams in a certain area just move that over. If you wish, if you now see the need to move to another orchestrator system- or maybe you want to first test it locally, you can just do it via shell. You can call piper, you can set the wall credentials, you can clone the repository and run piper artifact prepare version and it executes on the same configuration file.

D

Does the same thing: if we now go look at github actions, there's a type of github action available, same thing: checkout use the action with the command, artifact prepare version, and it does the same version. Therefore, you can really switch pick and choose orchestrator systems depending on your demand, also and also support migration parts tech time not yet done to come. um Maybe we can also have a chat afterwards for those colleagues being involved in tact on how we could best achieve that um and then to close.

D

um What's next, I showed you essentially the agnostic pipeline agnostic, step library that we can run the library steps on on various systems even locally.

D

The thing we want to go to is having the agnostic, ready-made pipelines like we have on jenkins, really an orchestrator independent pipeline throughout allow teams to move.

D

Prerequisites we see is we want to have centrally pro essentially provided pipeline flow and also provide centrally those kind of changes and do not miss out on on the autonomy of the teams and allow them to extend replace certain stages in certain areas of the pipeline.

D

On the other side, a team should not have a mandatory action in order to be able to consume something for sure. In many cases some configuration requirement will will appear, um but for the typical flow logic of the pipeline, a team shouldn't do anything and to give you some numbers um internally, we we have about 520 enterprise, github organizations, using it um with more than 3000 pipelines and having. I need to look up the number it's so high, 390 000 pipeline runs active pipeline runs per month.

D

That was the number of february for these agnostic readymades. We don't really have a solution. We looked into jenkins x, jenkins, x, 2.0 dsl, which is from my perspective, unfortunately deprecated, because that would would have allowed us to do something like that.

D

We were able to extend, replace certain stages and provide a central flow in an easy, easy to consume way. One could think of proprietary new dsl.

D

Maybe some people would say it's not really a use case. The agnostic library is sufficient at the moment. We don't really think so, and maybe it's something which could be a newer cdf project, new cdx into our project.

D

Having said that, um further information there, any questions from.

C

Here I asked the question in chat I'll just.

D

C

Repeat it are you using uh does piper use jenkins pipelines as its underlying uh workflow mechanism workflow engine? Yes,.

D

For for at least for these ready-made pipelines right for the centrally provided pipeline, it does.

D

But if a team says we want to execute on github actions, for example, um then there's no jenkins involved right, but then we don't. We cannot provide today ready-mades these. We call it. um We can only provide the individual steps and also in the in the open source project itself, we're using our own binary to to do certain things using it with github actions underneath, but jenkins is still the majority cases so.

C

So to get the get them get the most benefit from piper meaning centrally centrally authored pipelines from I'm going to please correct me: I'm making centrally centrally authored pipelines, but then with local customizations to get to get that benefit. A team would have to be using jenkins as their today. Yes,.

D

Okay, we're looking into how we can provide this for other orchestrators, ideally in a in a orchestrator, neutral format. Right, we don't need to have all the nitty-gritty details, a a an orchestrator has, but in the end we would need to have some kind of flow definition um and allowing to extending extending certain stages, replacing stages and so on. And we don't have this yet.

C

And you would need to have something that interpolates something that's generic. You know generic definition to an actual. You know definition, sort of a uh adapter or.

D

Plug-In kind of a mod yeah right and there we were looking into the jenkins exteriors. Now we have a prototype running where we basically use the jenkins xdsl um with an adapter, creating then a um creating a jenkins pipeline again or creating a github actions, workflow and so on.

E

So as as you've indicated, the um jenkins x, 2.0 dsl, has essentially been deprecated in favor of sexton, so the idea being that um tech town is, you know, they're going behind the idea that tecton's the best cloud native way uh standardized way to represent your pipelines and tasks for continuous delivery um and that they want that to be the primary. They think that should be the primary dsl for developers and tools. That's an opinion where the jingle x project is at right now, but it.

I

E

Be helpful for you they've all um the james's have also done a really interesting blog post on interoperability with um jing and zach son jenkins, and that may have ideas that are useful for you, since piper is predominantly a um works, works with jenkins but you're looking for, I, I guess a more cloud native dsl, um so that might be interesting. I'll put the links in the chat, but but having said all that actually would be really interesting to have. You um have a collaboration with the jenkins project I feel like this should be.

E

We could we could easily do like a blog post or something something there should be more. We should. We should be doing something to promote this project, because it is it's definitely very interesting. I I will have to ping you.

E

I will follow up with you.

D

That's good, then we we we had a talk at the devops world, uh the last devops world about this topic so perfect good. I proposed the agnostic talk also for silicon, which didn't get accepted for the early word.

E

Yeah, but you know all the talks that were um proposed for early birds are still in the applicant pool. It's not. You know.

A

About the bullet point, our new cds interrupt project. It's it's not interrupt project, but cdf project, like I don't know like, because I am checking the website and the github project. I think it's like there are lots of contributors. It looks like pretty. You know, attractive project. So what do you mean with that? Like you, because, like you donate project to cdf, I think if, in order for it to become a cdf project, yeah, do you mean that or something else.

D

First of all, what I meant- maybe there is a it's a possibility for, if, if people believe that such a neutral dsl makes sense um that that could be something to pick up right, I know the jenkins talking to to the jenkins x project as cara. You just said right.

D

um The guys are looking at tecton.

D

The jenkins x approach is a little different. Teams need to create need to update their local repository in in case central changes take place.

D

um We would love to avoid this because um having we have teams with hundreds more than 100 pipelines, that means having a change, wouldn't have the impact for them to change hundred pipelines right and look into 100 pull requests or even more and update those things.

A

The reason why I'm asking this because, like when we first proposed sick interoperability like a year ago or so like one of the things uh people highlighted under pull request, was like exactly dsl thing. You know, and now, when you talk about dsl and the idea you present like it makes sense, if you look at from that perspective, just you know remember that discussion from pull requests. That's why I asked and your point they're making a cdf project, but I don't know how things work to make a cdf. We have trace dragon here.

A

Maybe you can. You know, say a few give few hints, because you went through that process for ortelius recently right.

D

Yeah, maybe tracy before you go um from our end, it's it would also be a possibility to for sure, be a possibility to donate right um for sure we need to clarify, but um that can be an option, but that wasn't the option I mentioned mine. I meant on that on this line. Okay,.

C

Yeah just to provide a little uh little background context uh on on the uh generalization of a dsl for for cd, this this one and christie. If christie was here, uh she could probably provide much more color on this, uh but I recall the uh the early days of of tecton.

C

This subject was heavily debated in that uh in that group. In fact, uh there was a proposal to not have uh tecton's dsl be described as a crd, so it would be described as something very generic um and then it could be interpolated into a crd, which is you know totally kubernetes uh specific, because you know kubernetes. Yes, is the main uh cloud name, it's a very dominant cloud native uh framework, but there are believe it or not. There are other ones like uh nomad from hashicorp.

C

Does this similar thing as kubernetes does, but does it very differently so anyway? But the point is, it was heavily debated, but the end of the day the um the consensus of the community was that uh it doesn't add anything to if we're gonna go. You know full-on kubernetes cloud native. Let's just create the dsl as a crd and be done with it and not have to worry about generalizing it. So that's just um something that was.

D

Yeah, I guess that's, that's absolutely valid approach, if you, if, if you take kubernetes as granted, exactly if you look at it from an angle that you're that you have various various orchestrator systems, maybe also get directions, then the dsls are so different. Yeah.

F

So so, if I can add something, but we're still trying to do, um we try to keep any as much as possible. Kubernetes specific dates out of the api, and also we started an effort to have a conformance document where we specify the api um for tecton, which the only implementation we have today for is the kubernetes crd.

F

But it does not preclude the fact that there might be more a different back-end implementation for that. So we we kind of have that still.

D

And maybe in our mind- and maybe that's that's also a possibility right- I looked into tactum a bit every once in a while, not in too much detail, I'm not sure if it's possible today to to have these kind of extensions on on a repository side right that you have a centrally defined pipeline and a team says I don't like this specific stage of of the centrally provided pipeline, but I want to add a add a step to it add a task to it or just completely replace it.

F

Yeah, that's a topic we're talking about a lot, it's not uh straightforward to do. Today I mean there are some approaches you could do. You could use uh certain templating engines on top to generate a pipeline, but to do it dynamically. One of the things that you can do today is to basically break down your pipeline in multiple pieces and use events to connect them. So you could have a core pipeline and then use events to trigger stuff.