Continuous Delivery Foundation Events Special Interest Group, 30 Jun 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CDEvents WG - Connecting Events Edition - June 30, 2023

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

A

B

Hello, hello, can you hear.

C

A

Yeah I hear you.

C

Cool cool cool um yeah, so thank you um for for sending this meeting up. Andre I really appreciate it. Thank you, Emil, Andrea and also Victor. um You know the more people that we have in here, uh the better um so yeah uh so give me a website. Actually that's a good idea. Let me open my video as well: um okay, cool cool, um so I think. uh Let me share my screen.

C

B

C

Okay share my screen: real, quick and I'm, going to open up PR once I can get it in markdown view all right, so share screen. uh So many things open all right here we go. Okay.

C

um Can you guys see this.

B

A

Small but I guess I can increase. On my end.

C

Oh, that's why? Okay cool cool all right uh I can also enlarge it here in a sec, all right, how's that look yeah, that's good, okay, cool cool! um Let me also see if I can remove your eyes faces here. Okay, um so I wanted to kind of go over. uh Some of the comments that you guys had um mostly to uh you know we're gonna have this back and forth and uh I want to kind of get this knocked out.

C

So that's kind of the attention here um and then I will update this as well, because I saw also Dan had some uh questions as well, mostly about like the examples and whatnot so um so yeah uh where.

B

Did the comment go did.

A

C

There they are for some reason: they disappeared. Okay, um oh interesting I didn't know when he put it in markdown. It actually removes the comments. I thought. Oh.

B

That sounds like.

C

A bug because when you go back it doesn't it doesn't have to repopulate the comments, Okay cool, so we'll uh we'll start here. um So it says here the event: ID is unique for each vent. uh I, don't think this could be used. So oh yeah yeah. This is this is absolutely true. I was confused uh to which uh was the correct one, but it looks like subject: ID is pertaining to the subject that makes makes more sense um this one. This is the one that I was a little bit confused about uh Andrea.

C

So um so, like we have a test start and test run. Basically right, um like those two uh and he said themselves might not begin or end, but the activity they describe you know might um so I guess. My question is: what are you asking here like? What is it called because you you just kind of illustrate this point, but you don't ask any questions or anything of that nature.

D

I I guess was um a comment on the I I. Wasn't sure I understood what you meant with the event. We don't know what an event begins or finish finishes.

D

So I was trying to to see if that was related to these, um so that was kind of I wanted to mention what what happened beginning and end, um but yeah.

C

Okay, um yeah yeah that definitely uh makes makes sense, um yeah so uh yeah, one sec's dying here, um yeah yeah, so uh that that is uh partially uh true.

C

um So uh were um we so when we, uh when we do backtracking, um it's when when I say events here, I'm not talking about so I need to use I think this needs to be spanned um event needs to be.

B

There, let me see event.

C

Needs to be spanned, so what I'm talking about here specifically, is the the whole uh event span, um meaning like uh you know when, uh let's say uh when the whole CI CD flow has started and when it has all finished right, we don't know when that begins or starts. We know when the individual things begin and start, but we don't necessarily know when the individual uh like pipeline has started or whatnot Without Really, inspecting um the uh all the predicates themselves right.

C

um So that's kind of like what I was getting at here. Okay, yeah I'll end game that, because that's yeah definitely yeah because event is being used very, very vaguely or Ambiguously here. Well,.

D

Then then, um to add to it, then, since you're referring to a span, what I would say is that uh it may be a matter of perspective of the viewer, also what the span exactly is so depending on what you're looking for, if it makes sense, so maybe for someone the span is begins with a full request and finishes with a artifact being published to a repository right for someone else. It starts with a feature definition and it ends with a code in production for someone it starts with a deployment and finishes with some monitoring.

D

Events so depends absolutely.

C

And, what's great and that's why I have this section down here about the ad links, um because customers or users to actually Define what their um event uh life cycle or event span life cycle is um they'll need to go a little bit lower level, but we can. We can address that. You know once we establish the spec, we can figure out how to make that easy for customers, but they can absolutely do this with with this SDK.

C

So uh that's that's the goal, at least so. If that helps uh Andrea.

B

C

Okay, cool um so I'll also Define, that which I think I do Define it up here or I Define it lower I call it CD event span, but I failed to uh I'll link, yeah I'll link it down here as well. Okay, um so okay, I, see I, definitely saw Emil's Point here um uh due to the limitation of the scope, um but the reason why I made it uh this definition, so limiting in in scope, is because I thought the original, so maybe I'm wrong here.

C

So this could be my my complete misunderstood or what your guys's scope was for. Cv events, um but I thought: CD events was interoperability between CI CD systems, the life cycle.

A

Before I think it's not really a well-defined start activity for a cicd system, uh often I guess you refer to a pull request being either created or pushed.

B

A

But it could as well be some feature being defined and later the pull request for it is pushed. Oh.

C

Yeah, yeah, okay, and so that goes back to yeah what Andrea was saying and where the SDK.

A

Would allow yeah yeah okay.

C

Okay, so, oh since there's some confusion, so since you guys both had that confusion, I will make an action item here, um uh specifically specifically address confusion around uh uh how users can Define their own custom. Cd events.

B

C

Life cycles: okay: well, will that will this satisfy both uh you and Andrea's confusion, or do you think that will help future users reading this? uh If uh this, this one's particularly.

A

Added, okay, everyday does, and it says artifacts there as well in the same sentence and I'm not sure what the artifact in that yeah means really yeah.

C

I wanted to keep that ambiguous.

A

Since we use artifacts already in this pack, I think it in the paper can be confused.

C

Okay, fair enough fair enough.

A

um I guess it could be a storage change that is an artifact as well.

C

Yeah yeah um artifact.

C

Or figure out a better way of phrasing, uh how a how a CI, CV or CD event span.

C

It starts okay, cool that should be good, then, um because yeah I could definitely see the confusion, but I wanted to leave that ambiguous, because I didn't want to limit, because, like you said it's already, quite limiting I didn't want to limit it too much so, but yeah, okay. That.

A

That's definitely a question I think when we talked to um Steve, Pereira and BSM about this, didn't he mention a term like uh system of work or something no, not system.

D

Well, he he was using entities and systems, I. Think.

A

um Or something like that and yes, it's an entity could maybe not be an alternative. If you're, if you're with a very broad thing there then.

C

Yeah yeah, it is good um I mean entities, is, is yeah, it's definitely broader than in um uh than artifacts yeah, maybe uh entity entity for this particular one yeah.

D

Okay, so um I think regarding your question. uh Ben uh about the the mission of City events, I, think uh it's interoperability in the CI CD space um or in the city space, but not necessarily only between cicd systems as well. I guess it depends what the CI system gives you, but um like we, we include things like software Management Systems, which is not strictly speaking, error. Let's see ICD system or uh artifact Registries um deployment system so that the whole set of tools that you know make your CD from source to or even now.

D

Yes, maybe considering from idea definition feature definition to production and so forth.

C

Okay, yeah uh I had yeah, so that was that kind of aligns to what I was thinking. But I wasn't sure if that was the issue here, but it seems like it wasn't. It seems like it was just this use of artifacts and also this this confusion, um but but I just wanted to see if that that was a potential issue here, but it looks like not oh, so this is what uh Dan Han, also um So Okay cool.

C

So now we're kind of getting more into the meat of things um so uh and yeah yeah. So this kind of goes back to uh you know allowing users to kind of Define.

C

um What a start point is a starting CD events, man um and ending a CD effect span, which will create this Global ID right, um but you're saying you see all possible. You see an issue with Upstream Downstream events in the flow like um oh yeah. This is kind of related to that. uh Okay. So this this should still be solved uh with with that uh earlier comment, um this this is Ox. This is actually not potentially true, so.

C

um Generally, they will.

B

But they could have multiple or different global global IDs.

C

So um but family span I think what I'm going to call it I'm going to rename Global ID to maybe see the event span. Id. Do you think that will be a better name.

A

I don't know if we should use the term City events within this spec itself.

C

Oh right right, maybe just span ID. Then what do you think.

A

Yeah, if we should go with it in this way, we have comments later on logout, open, Telemetry, related terms and such as well. So we can maybe come back to this okay.

C

Okay, yeah fair enough um Okay cool, so uh yeah so including the whole parent. um This won't grow indefinitely Andrea. This is always the immediate parent uh or parents, depending on if it was a fan out, fan in situation.

C

um So uh so that's kind of the the reasoning for that and I kind of explained this to Emil uh back well, one of those things like why it's like this um and it's more for nine times out of ten uh uh people, are going to care about what their parent did, and this isn't the immediate parent. This is The Logical group grouping of parents.

C

So what I mean by that is um so when you have a CI uh Source change right that gets pulled into CI that runs a bunch of tests and that uploads the artifact right that that might be considered, one uh that might be considered a logical grouping uh parent and so that parent uh would be past only that um Source change that that thing that started it would be passed uh to the um uh to the uh the the newer fee or the new Fields.

C

If you will um so that's kind of the idea there, it's still kind of a little bit shaky, but um I'll I'll, still kind of Define uh Define that and the reason so that what's good about that, though, is users can again still Define what these logical groupings are.

C

um It all depends on this like link context, um but my goal is that the CD events sdks, would kind of handle a lot of this for you, and if you have some special use case, you can, you know, remove those hooks and add your own, so it's kind of my thinking there.

C

um Let me go back out here um so with that said, uh this wouldn't grow indefinitely. This would be the the root parent um uh that kind of like started the the whole Crux of things um uh that and that would get passed along, and what I try to do with this image?

C

um Oh well, this image actually doesn't have it. Oh actually, I'll add an image um to kind of better illustrate what I'm talking about action on um uh image better illustrates.

B

C

Right, okay, yeah! Actually it does make sense so actually at first I thought it was a little bit shaky, but it's actually not because tests are usually a sub thing that happens in the um CI system right, um so that's all contained in in the CI uh system. So like what I guess I'm getting at here. So CI uh kind of happens like this. So we got a source change uh which gets changed which triggers pests, which then triggers an oh.

C

If tests pass right and so let's assume it passes and if it passes uh uh it, uploads artifact right.

C

um So that's kind of like the the idea here so it'll be these three things and then uh the CD system would then uh let's say: consume, consume, uh upload, artifact, uh consume, I'm, gonna, say: consume upload, artifact, um CD event, which then would have would have or city would.

B

City would assume, which would, which would have be.

C

Source change um in there uh Source change as the parent does, that does that make sense.

A

um Well, in some cases it might, but in many cases you can have like hundreds of silver changes in one artifact, you can build like a composition of a lot of things within one artifact, so so I don't think we can contain all of that, always within one artifact published event or something like that.

C

Right no wait! Let me think about that because you would have let's say hundreds of artifacts, but no. No. The source change, though, would be like GitHub, though right like this upload artifacts would just be the regular CDO event. However,.

A

You integrate multiple things into one large effect, and then you publish that.

C

Oh then you publish that. Let me let me think so. If you.

A

You continuous after that I guess when you have published the artifacts yeah.

C

So that would just be the normal Source change initial CD event right.

A

Level 12 could be multiple More Store changes included in the same artifacts.

C

Yeah but wouldn't that be a fan out fan in situation.

A

Not to turn out but the fan in so you know, multiple I mean One Source.

B

A

Initial build but that build brings in lots of other solar changes as well from other places, libraries and stuff. Maybe.

D

I think it it it. It depends what what kind of semantic we want to to express in in.

B

D

Like you, you have in I feel different type of semantics. So if it's, what immediately triggers a certain artifact being created- and then you know, but if you have like an artifact that depends on 100 other artifacts, it depends you could you could express all the dependencies in there.

D

But then maybe uh the event is: is not the right spot for doing that, because it could lead to many of them, but it could be the specific or, alternatively, could be the specific event that trigger your build, which could be the fact that one of the dependency had a new version yeah that maybe someone created a ticket or an issue saying yeah. We need to actually update this dependency or it could be that there was a CBE that was the input of the process.

D

A

D

Guess, oh sorry, I.

A

D

I was going to say, I mean for for certain type of links. I think we still. We will still need to rely on things like having a standardized IDs like the the package, URL format that we were adopting. So if you want to be able to okay, let's say if you, if you let's say that you have a cve and you decide that you want to uh as this new artifact build this new artifact um by pulling in certain versions of other artifacts.

D

And if you want to have an extended view of your span, if you will or workflow um you could use the package IDs of the different packages to see their span of how they they were created. So you could have then like the tree starting or it's not a tree, but you could have like you could see like from Source change for each of them and they would then converge into.

C

Right right, okay, so maybe the idea here then okay, so we have the source change, but this would be a source change. Parent object, like maybe a new object um and and I was telling a meal like. If you need more information, you can query the links API to retrieve more information. So maybe maybe that's something we could do because a lot of times they just care about a few things from this Source change parent. So maybe we we just include what is probably most cared about. Only.

A

But yeah, but that's one thing that I'm a bit concerned about, because I mean the producer of the data shouldn't, really make a lot of assumptions on what the consumers would need in any special location. Yeah.

C

That's why I wanted to include everything, but.

A

Yeah, exactly so I mean either we should include everything or we should include just a reference, so it can be no I. Think I don't want to give some well-known, of course, Community data, but we will never succeed in having a complete pictures. I guess.

C

Yeah yeah, but but I think, but we can, though, make an assumption that some things are necessary and definitely some things are not right, because there are probably cases where someone might not care a word. Everyone might not care about a particular thing right, um which I think uh you know to say that you know is, is very um um you know.

C

uh I guess kind of you know like, like kind of you know hopeful, but um but I think we can start with a subset potentially, um and if people are asking to include things, we can either a include them or figure out new ways or maybe what we do is um include a subset of what a lot of people need and if they need more look at if this is a growing number of requests, that people want add that as a new field or tell them.

C

If they need that information, they will need to query the links API. Maybe we can go with a hybrid approach there. What do you? What do you guys think.

A

It's gonna make sense, yeah.

D

Something that I I was thinking about is that so when a producer produces an event, I think it has some extra context that might not be elsewhere, which is maybe I, don't know the decision why that activity is needed for a certain relationship between a certain input if there was an input event to the activity that is now being done and and that specific context is only available, might only be available to the producer.

D

So, going back to the example that I was saying below, maybe the the producer is evaluating a certain policy that is written in your system. That I don't know if a certain CD is discovered, you automatically need to regenerate the build and do something and um that that link um between the fact that a policy was evaluated, that the cve was discovered and that the build was triggered.

D

It's not found anywhere else other than the the producer business logic, so I think that kind of Link we we should request to to have it expressed in in the CD event, and then you know, as I said we we can have more information if the producer wants to pull in more information and add more links, but also other links, it's something that the consumer then might be able using the links API using some other Mains to to then pull that other information.

D

But I think that the information about the I guess the the consequence of certain inputs that generated the event that is being sent, that that specific relationship I think it's maybe unique to the producer, and so the the producer should have the responsibility of including that if.

C

It makes sense, yeah, okay, um okay, let me okay, so I think I need to think about this. A little bit more so I'm going to write the issues here and, if I miss anything, uh you guys can just update it, and I will come back to this, um but I. Think for now we should just include everything um until I could come up with a good way of dealing with this, because I don't want to create custom types and the reason for that is I. Think managing that you know we're already managing a spec.

C

Now we're managing what um you know parent! You know, parent links are going to be, um you know which could be open to change as well. So that's that's. Why I'm a little bit hesitant there, but if we need to do it, we need to do it, but um but yeah we may need a subset.

B

Of the parent and the whole thing: okay,.

C

So and then what was there any other issues here, just other than it, the parent being potentially really large.

D

um Noah, thank you. You clarified that also it would be just immediate parents and okay, yeah, so I think just one other comment: um uh we we have I, guess the benefit as a as opposed to maybe things like open, Telemetry or general tracing to have a specific context, which is the ICD, so I think it's probably very valuable to have those kind of flows like you.

D

You wrote here in the comments um as as part of our documentation and to make example of how to use what we are discussing in those specific contexts and then, if someone else comes with a new different use case and say: oh, how does that apply to to the spec? And if it doesn't apply you we haven't thought about it. Then we can add new things. Yes,.

B

C

Okay, cool um okay, so this is a confusion. Yeah I saw this comment. This is a confusion of I met, logical, grouped parent, so that that's the issue here um yeah. So if you guys didn't see this this, he was talking about test. He was like I. Don't he didn't understand that the parent? Why wouldn't it be test suit finished? Because you know you don't typically, as a CV system, you don't really care about this.

C

You care more about, what's being changed submerged, so um so yeah, that's just confusion on on the spec itself, so definitely need that image definitely need to clarify um and I did see uh your comments here, Emil um so I understand it's not been a major showstopper for for you.

C

um You know not having global IDs, but it's gonna be a major showstopper for for us. Unfortunately, um just because our our system, they're they're, just really large, like and I, was looking at some of some benchmarks from 2022 I can actually share the study. uh I'll find it and link it in here um that graph, like graph DBS and then just like querying like I I, saw that you mentioned too, and they actually do uh use as a graph DB as well.

C

So they do exactly what you're talking about down here um and and it's dramatically more slow than just using a global like a global or some sort of a primary key, um and because of that that that would be that slowness is, is going to be an issue for us like there have been so many things since being here, so I've been at Apple for for two years and since being here, we've dropped so many Technologies and how to replace them for others, because they didn't scale as well as they they advertised.

C

So this is more to prevent that um that transition for for for us, um because we've already seen it happen, like I've, already seen it in my two years here three three times like I three three times where we had to move from one system to another system, because it didn't scale well.

C

um So that's kind of like why uh so I hope that that's a good enough reason for you. um I know it's a little bit anecdotal and but it really comes down to um you know what we've already done. uh You know what we've already experienced in the past and This Global ID is the quickest way.

C

um Well, it's the quickest way. I could think of of getting all the events. So so is that okay with you or do I need a word. It's a better question. Do I need a word this better.

B

A

I, don't know I mean if, if you have a very very large I mean we have a large system, I mean we have probably thousands of events in one chain and we don't have major problems with this, and we have millions of events in the database. Obviously yeah, but.

C

We're talking about.

B

A

So so I mean if we would use the graph database, we would have even less problems. Yep I agree. We don't get melee or microsecond responses to our queries. We don't, but we probably don't need that either at the moment, but yeah I agree. It will be faster if we have an indexed ID. That could be quicker to look at I.

A

Yes, struck me right now, actually that that Global idea what you should call it could actually be be created on the if we call them that the links API side or the the store the database site itself, I mean.

C

If you oh you're right yeah, if.

A

You send events which have some kind of relation in between them. Just a Serial relationship or like yeah funny find out and those things, then the the database itself could actually identify that hey this event actually corresponds or is connected to this previous event. So I add another Global ID. On top of these events, which are already related, so the database could actually be introduced that if we want to yeah.

C

Yeah, that's actually a good idea. um I'm gonna look into this I have to think about this, some because it isn't a new idea and I'm, not the type of person that just says oh that'll work, unless it's immediately obvious to me, but things like this, where I have to think about it. Some because there's a lot of moving pieces, um I'll I'll, think about this and then I'll get back to you. If that's okay with you yeah.

A

C

Cool cool um and then yeah you're absolutely right hear about this, not being true, uh because there is a study where they do use ID. So I'll I'll word this better um to say uh limits to limits it to only graph DBS, with the caveat that you can use any SQL database as graph BB, but it's not as intended right um if, if that would be a better wording, there.

D

I I have a question on the on the global lady variant. Words, no worries what I'm trying to to figure out is that is that going to be one per event, or is it going to be an array per event? So does it is it possible? Do we consider the possibility for in event, to be part of multiple spans, so to say your flows or whatever we call them or.

B

D

Do we specifically restrict the one to only.

C

One, um and, and if and the reason for that is because um you could have so basically, you would have one right, but if, if you need multiple, that is probably saying you know, you just need to make that one include the multiple right like just you know um like and or so there's two options there. You have three.

C

Let's say you have three separate um uh spans: if you will, you can either have them as three separate spans or make them One Singular span again, that's kind of up to you, um but but I would be if there's any sort of relation to span two to span one. That should be one single span right, because it has to do like that. That means that life cycle isn't over yet right and and based off the definition. It's that whole life cycle does that make sense. Andrea.

D

Yeah I guess it. It makes sense if, if the the what the life cycle is, is defined in advanced, so when you're producing the events, but if you want to add the possibility um of having larger or different kind of flows later on depending on the viewer perspective, it's not necessarily I, don't know it's, but I! Guess you. You could have then different queries.

D

B

Select different parts and.

D

Yeah having the ability of connecting them yeah.

C

Yeah, that can definitely be done with the links API today, if, if needed so.

D

So is it the way that you, you must configure than one of the systems in your tool chain to say, you're, the one that is producing the global ID or.

C

uh Maybe so Emil brought up a good point right here: um I I'm gonna, first look into this um okay, so I'm gonna. Add to this. How Global ID is actually generated.

C

In system I think this is a big question that it looks like that's being asked here: um okay, so I'll look into this and I'm first, first I'm gonna look into uh meals um question, because if this can be done, that simplifies everything so I'm going to first look into this, but, like I, said it's a little while too many moving pieces for me to kind of figure it out right now um and then, if this works, I will update it. You know update it with this question.

C

You know how this Global ID is generate in the spec. If this doesn't work, then I'll continue thinking about it and that might um bring in you guys again to kind of like talk about some ideas that I might have and then kind of figure out what the best uh next step is for for the global ID Generation.

C

Um Does that work for you for now Andre, basically I'm saying I, don't have enough information right now based off of this first comment here or this comedy that meal had.

D

Yeah sure sure yeah I mean we can. We can follow.

C

Up, okay, cool cool, um because mostly the spec was to outline um the the schema right, not not necessarily how to define like how the system, like that's more of the technical details. But if we need to go that deep, we can. But I I wanted to make this more of a proposal rather than a technical document. If that makes sense right.

D

Yeah yeah, I, I, understand and I agree and only the fact that how the so what is the the logic, I guess or the behind generating the global, ID um I guess as an impact on whether it makes sense to have the global ID or mix S2 F1 or make sense to have many so I. That's just because there is this consequence, but I agree. We shouldn't you know, go and if you find here the details of the algorithm used to generate a globality or things like that that that's not.

C

Right so yes, so I guess with that said, um I'll leave the algorithm I'll leave the the uh ideas out here, but I will add. um I will first do some research into this think about this sum and then, if this works out, I will highlight that the global ID. That um is a simple way or it could be simply uh you know, generated by the link service itself. We don't have to worry about it. You know anywhere else, basically, um so that that is one uh thing.

C

I could first look into, but if that's not the case, then I will have an open-ended question. That I will say. A follow-up document is needed, so there'll be a proposal document which is this document and then a technical document which will Define how to hook and start or create all these moving pieces. If, if you guys are okay with that.

B

C

Makes sense yeah all right um so yeah like I, said action I'm here, absolutely not true, but uh but but it needs to be reworded or simply removed.

C

Okay, so that makes sense okay, so that's basically the gist of it. So you guys brought up a lot of good questions. I definitely have some big, some bigger action items. I'll look into this. um There.

A

Was also there's no comments on the general PR as well itself, nothing, nothing file, but in the pr itself there's some comments. Oh okay,.

C

Okay, let me see: let's go back here, okay, so this is that wait. One second did I expand, okay, so no.

A

C

That there's that there's that okay, so yeah wait. This is also partner. Spec, oh yeah, yeah. That's also true. That's true.

B

Also there, okay.

A

Here we go okay. That was a recent comment for.

C

Me yeah oh yeah, yeah for the open Telemetry, so so yeah yeah, so I I understand what you're talking about here and and again uh I think so. This is actually basically using uh open. Telemetry like it's using it's a very, very, very simplified form of open, telemetries Trace context. Basically, um that's the idea that, but that's again more of the technical document.

C

um But again we don't want to use open Telemetry because they have a lot more stuff than what we need. So it's kind of like what I was kind of going at um so I guess my.

A

Question is uh I, don't know if they agree actually, because I think we we could. We could have actually used open. Telemetry I mean there I think there is an extension or at least proposed extension to Cloud events, which would include open, Telemetry data.

A

So you can declare parents and and parent and child relationships and also spans in the cloud event on the cloudant level itself, but we could inherit it into the City Event if we like, but then I think we should actually, if, if we think that open Telemetry way of relating events to each other is a good one and we can make use of it for both.

E

Of them relationships.

A

But also for the Corsa relationships, maybe between these spans that we talked about.

C

Before open Telemetry, like we already use open, Telemetry like here at Apple and so like what you suggested would be a very easy way for us to use open Telemetry with with CV events, but even making that easier for us, I'm, saying no and and the reason why is using open Telemetry for this is like using a gun to kill an ant like it like open, Telemetry does a lot. It's and one thing you can do if you're really set on this, we can add so open.

C

Telemetry has this thing called a plug-in system that you can add for The Collector, which is basically what we're gonna have as well, um which would be the service if you will the link service and that plug-in could just basically be our um our CD event spec. So we can do that and make it compatible, but that's the thing we should make it compatible um or allow for a plug-in to be added to open Telemetry where it could be used and people find to use that you know, then they can go.

C

We can go ahead and always use that, um but, like I said it's it it's doing, open Telemetry is not it's also, not small, so you're bringing in a huge like I, said a huge Library into um into just doing uh to just doing these. Basically just these three things, um so that's kind of my concern there.

A

I I don't have much experience or no experience at all using an Open Championship myself, so I might be out on very deep Waters here, but I was just thinking that we, if we use the same not just the same Concepts, exactly but also use actually the use open set MSG in the cloud event or the City events part. We could actually reuse any different consumers all that open to them with your data. So we can.

A

We don't need to fill those, maybe even the link service ourselves, because open Telemetry can provide much of it for us but I. That was my thinking at least right.

C

Right and I'm all for doing less work, but again it is overkill and.

B

C

Think yeah, yeah and- um and if that is you know the case right like why, why invent a new service, particularly for this use case right, but no it? This is overkill like I, don't know if you like I said like you might not have experience. I can I can list the reasons why it's over I think I did at one point um somewhere.

A

I think we had a separate issue on that. Actually I think it would be very interesting to to elaborate a bit on open Telemetry relationship with citizens. So if you have experience, I think you can contribute there, it would be great.

C

Okay, cool yeah, so I'll definitely follow up here uh with with open Telemetry um because, like I said like this is where I started. um So when I was originally writing the proposal I actually started here, but then I was like okay. This is the open. Telemetry is way too overkill for this um and then moved to using the Eiffel lengths with with the own link service, um so, like I, said, I started here, but ended up after doing, War digging was like. ah This is probably not a good idea.

C

um It looks good on like when you first think about it, but once you start digging into like how big the dependencies are, what all open Telemetry does um it makes it it's a little um um yeah like I, said it's a little. It's a little. It does a lot. It does a lot more than um than just uh these. These three things um and it's definitely a lot more granular as well.

D

It does less as well, I, guess and well. It depends I mean for open television, specifically they distributed tracing part I. Guess it relies on the w3c specification that I built in the so the trace context, and that is pretty contained. It's pretty small I mean it. Does it's not like the whole open Telemetry specification is just like the the small beat that tells.

C

You how to propagate.

D

B

Oh sorry, but then.

D

What um but, but that's just just a tree right, it's just fun out yeah yeah, and so, if you want to have this concept of having multiple parents, like you, I believe it was an array of parents that you can have yeah. It's not something you can do with this yeah.

C

Yeah you couldn't, you would have to query that and actually get every single like and you would have to possibly use tags so that that yeah so you're, absolutely right. There Andrea, but also another thing. This is self-contained, but I don't know if you've ever pulled in a tracing Library open Telemetry is one Speck among so many. So when you pull in a tracing Library, it's going to include open Telemetry. It's going to include uh uh what is. It called uh starts with a J, uh so there's another, and then there's.

B

No yeah Jaeger.

C

So yeah there's this Jaeger spec, there's a Zipkin spec, there's a br there's uh open Telemetry. So when you pull in a single Library, it's including all three and they all are slightly different, and so how does? How does open Telemetry handle this with their collector? They have plugins that convert the Zipkin to an open, Telemetry uh standard or to the open Telemetry standard. So that's why I'm saying it's Overkill, because it includes all this other stuff.

C

um So yeah, that's that's the concern, but but with that said, Emil I think you're right I need to elaborate because I I have it I. Think I posted a a slack message on why it's not you know and I I gave all these reasons, um but I need to capture that here in this PR, because it's kind of like you know not, it doesn't really. It only talks about the proposal. The introduction, but not why not using this right so um so I can definitely do. That is. Would that yeah?

C

Would that satisfy uh your concern here? Yeah.

A

For the first section yeah- and the second section is something different there, but in the second section, is what I wrote before you're, actually explaining what the parent was. I thought that the parent was just the immediate events, but now you say that it's actually a full chain of events, but so so maybe we cannot make it as simple as I stated there. Oh, maybe we can anyway, actually yeah.

C

We could yeah yeah because.

A

Yeah, you can open up that example. Just there and a question yeah do what it looks like, because there is an example of one one event with some different links. Then I call them links, but you call them parents, that's the same thing, yeah yeah.

A

So if you go down to the second gray box there you have some every links object there yeah with two different two different parents, I'll say in two different things: one being that the coastal event that we that caused this thing to happen on the other one being the artifact in in question here and we could learn also at a storage change. Maybe here it's type store change. So the reason to have the type there is that we want to distinguish before we actually go into the actual event there.

A

What is this type of relation and then I use I should actually reuse. The City events terms there in the Target yeah yeah I see that there so so what is most important is the ID which is on the actual event. I did a globally unique ID and then all the other things from the subject.

A

Subject there could be optional because they they provide convenience for the consumers, so they don't need to look it up through a link service or something you can have all details there from the event itself or you can leave it out and then it's up to the consumer to to bring in that information from the links API or the yeah.

B

C

No so you're absolutely right uh with um uh with with what you just explained there or like. Oh well, sorry I understand what you're explaining there but yeah I. Think with um with explaining like what the parent is. uh We still should be able to do this and include all this necessary information, but the biggest question like I said since we've already talked about it so much. The biggest question is: if how large can this get right um so once I answer that? Oh, that was the other thing.

C

I have to look into and think about something um so yeah. So I'll look into that um I'll be sure to answer this question so I'm gonna put a. Let me see, can I put yeah I'll put this here just to remind me to look at this second part um and re-answer this as well as uh uh expand on or add I'm gonna. Add a section saying: why didn't we go with open Telemetry.

A

Yeah that would have.

C

Been ready but I think yeah, okay, yeah, okay, cool um and I'll. Explain that okay cool! So um so it looks like we had so we looks like we went through all the comments so with that said, I'm going to tell you um what the plans are for for the next uh kind of week, two weeks, probably two weeks.

C

um The plan is to address all those action items once so. It's the dock is starting to get uh it's starting to kind of like solidify I'm, going to start updating the Json schemas to what we know are going to be uh that are more likely to um not change and then the ones that are still kind of up in the air, like parent and so on, and and all those uh then I'll I'll, probably just kind of keep that as uh just uh uh an object that hasn't been filled out yet with any details.

C

But I'll update the specs I'll plan this for uh two weeks from now, I'll have this updated um and then I'll also uh update it to include this versioning policy um and then I'm, not too I'll have to look into this. But I'll do these within the next two weeks as well, so I'll get those checked off um so yeah I just want to kind of give you the the idea and the plan so within the next two weeks. uh This will be.

C

um This will be ironed out. um If you guys have any more questions, we might have have to have another meeting if there's still some things up in the air. But if the questions look like, there's not going to be a lot of back and forth, then I'll just leave it in the pr. Does that sound good.

B

C

Okay, yeah cool um yeah, so this has been super helpful, like I said Thank you for taking some time to you know, out of your day, to you know, to meet with me and and discuss this um I feel I always feel like these are uh a lot more productive than just trying to figure out what people are trying to say uh in the GitHub comments. What I might misunderstand so it's good to have the one-on-one back and forth. um So with that said, uh I think we're pretty much done now.

C

So, unless you guys have uh any other questions, um I think I'm good.

D

Yeah, no thanks a lot I think it's really helped them as well, and just just one one more thing, um not necessarily for for this next iteration but I. Think um when we introduce the the link service, we might want to kind of give more context about what kind of environment we're considering I guess, at least in my mind, I imagine an organization, and they have all these tools, so they set up their link service and that link service stops to the different tools within the organization.

D

We have use cases where we go actually beyond the binary, the the boundaries of a single organization um um and that's something that we might tackle in future. I guess so, but I think it's good, maybe to to give a little bit of context on where this feature. uh What what.

B

C

Context that we yeah, where.

D

It lies kind of yeah um but yeah. So that's just one yeah.

C

So that all like I said so uh once this proposal is approved, I'll start immediately working on the text, uh the technical document which we'll address those questions. So all these deeper technical questions which I think well, they need to be answered.

C

um I I will do a technical document um and then we'll address all these questions of how is global, ID generated that algorithm? Also, um where does this link service or how and where potential areas can this uh link Service uh exist and then kind of go from there? So um so yeah yeah.

A

We also have this sorry. We also have this issue about uh what is it now evidence store right, Andrea, yes,.

D

A

Quite related, I think to the link service. It's more or less I mean to me it could be another name for the same thing but yeah. So so please look into that issue and then comment there as well on this audience on the link search there.

C

Okay, perfect yeah, because I think I think they kind of some of that overlaps, like you had just said a meal, um so yeah I can definitely uh uh you know, maybe not in this proposal, but maybe in the tech spec uh start addressing those questions, um but at least with the proposal. We have something that's defined and then the tech spec is now going to be defining how we implement it and then we can start going ahead and implementing I.

C

Don't know if you guys do it that way, because I I didn't I, don't know if I saw technical specs uh in your guys's um uh repo there, but uh I'm a big fan of documents, so I I like to have things kind of defined um and then uh I feel like it makes uh implementation a little bit easier. But if you guys are not comfortable doing that, maybe we could just do a smaller text spec to answer the bigger questions and then go ahead and start implementation, so I'm fine with either.

C

You know like I said this. Is you know uh your guys's organization I'm, just kind of like trying to help out so whatever your guys's processes? Just let me know, and I can do my best to um to fit in that process.

A

Yeah I guess the spec could specifies some kind of requirements on such a link service, but students maybe specify exactly what the link service looks like in itself and their surroundings, but maybe some some requirements that they could be referenced. Okay, yeah.

C

That sounds good, then um so yeah, it looks like we're pretty much.

E

Done if that's uh can I ask here, yeah.

C

Sorry, Victor yeah yeah.

E

I'm, a I'm, a database guy I'm just interested in a lot of this stuff and I'm at a really high level, uh so um I just I'm, looking through the um the City events, look like there's some like road maps. Is there like a more um higher level like business level? What what is going to help? Overall, uh you know business value. What does the CBE event provide? Is there such a document or link.

D

Well, we have a white paper which might be what.

E

If everybody wants to have that information.

D

um Yeah, so if you.

C

So, just to kind of elaborate on your question: are you asking the use cases or what CD events is trying to solve or like what? What are you asking.

E

Exactly what what yeah exactly so? What is it try to solve what we see the event what it brings to the to the overall value right, I, look at the a lot of the documentation looks like the City events is uh good for, of course, Automation, and it's also mapping to the the daughter of Taurus right. What's the value of the dollar reports the Google uh brings, there are also other things that other groups are working on, for example, the the uh not directly related the the chaos group is also working on other Matrix.

E

That's, uh basically how to measure the success of Open, Source Community right. So there's a lot of good efforts. That's going on all together, uh so yeah I'm just curious, like the City events, what kind of role that they play! uh You know overall, a bigger picture. That's what.

C

I'm trying to figure out uh so Andre I know you probably can enter this best, but I think can I start because I want to give him like why Apple wants to use it, which I think might give him a good idea on why you know he might be interested in this. um um If that works, for you, Andrea yeah.

B

C

Cool yeah yeah, so uh so I'm, I'm Ben, uh so I work at Apple and we're very interested in in CV events and and the reason is so. uh I I talked a little bit about um changing between services and features because it didn't scale well right. But what? What does that have? What happens when we do that? Well, we now have to update everything we have to update the apis that we call we have to update.

C

um You know all the code that talks to all these different things so that that's the biggest thing, the tech debt uh that is required when making a change is massive. But if we had a system or or a standard or a protocol, that spoke the same language and the same apis, then we can swap things in and out add things as needed. We can also add you know just like, um like uh like a new, a new database that just came out or sorry not a new artifact store that just came out.

C

That still is in question, but we could just kind of add that to the big tool of chains and just see how well it performs right. So that's why we're interested is that it makes Plug and Play that much easier um and since we're constantly uh trying to scale you know uh at our level. This is a huge win for us. um So that's kind of why we're interested in it.

E

Very interesting yeah, so I'm a database guy traditional database guys so scalability is a big thing. Another thing that's going on in the in the in a cncs world is about a platform right. So what sounds like you're? Basically just testing different components, your final components, not scalable. You switch another one like by the by standardized CD events. They allow you to do all the testing. All the integration streamline that process yeah. That seems to be a huge huge use for this uh yeah I, see that yeah. Thank you.

D

Yeah and um so I think that that's great uh that recently it's very much. Maybe what I would say then so, that's that's great um I think what I've seen uh as well as like um within the same organization, different teams.

D

They have knowledge about different tools and they want to stick with them, uh and so that also is a similar scenario where you have different tools: uh they're, maybe talking different languages, and so it makes it harder when you then want to bring a new tool in and then it if you're working like at platform level like you were saying later, and you have some new tools that you bring in and then it has to talk with a tool that team a uses with a tool that Team B uses and the tool within C uses.

D

Then you have again the same problem. You need to start creating integration between all of them, and if there is a single language, then it's yeah, it's relying on interoperability, so you just drop it in and it should work. Yeah.

E

Maybe it can even expand it further along so I'd be interested in the uh there's, a lot of high level things and Security in particular, um it sounds like free download, there's a lot of uh like open source activities, trying to figure out how to handle the like the the CI CD pipeline for uh to make it secure right. There's a lot. A lot of effort on supply chain security, uh this this effort, the CD event- could also help there, because um they are, for example, the the hardware industry.

E

They are also trying to make it their supply chain security, but they're not doing as much as the software industry right so I mean this could be also adapt. The the same idea to a different use case.

D

Yeah yeah definitely I mean we. We started talking with the open ssf um because they they have similar kind of uh problems of or like questions of interoperability, because they want to I guess their mission is to make it Easy by default, to do your bills and of your cicd system, your your software production, factory, secure um and so through. City events, proceed events or similar type of standardization could be helpful for them to.

D

um You know, specify what what kind of information the different solutions exchange between each other so that you have the right context to do things securely if it makes sense. So we are starting with this conversation with them, and one of the use cases also that some of uh and and our end users brought up um as things like audit Trail.

D

So if you have all your tools in your tool chain that are producing these events and you collect them in this, um what they call evidence store, which could be the links database or whatever. If you can trust the content of these events, then you can basically build an audit trail of everything that is running in your production system. You can go look up the artifact IDs and then you can collect the entire series of events that led to to that state.

D

E

Cool. Thank you. That's interesting.

C

Well, yeah, it's good to see you uh Victor. It's always nice to see new faces um so yeah with that said, uh if we don't have any more questions, I'll go ahead and address those action items um so I've added on uh my Sprints. Actually so CD events I have allocated time to Now work on the proposal as well as anything else.

C

So if we need to do anything, just let me know, and then I I have that time to do that um as before I had kind of like fight for it, but now I have allocated time so yeah.

B

With that yeah yeah so.

C

With that said, um I think we should be good here and then yeah. If you have any questions, just update the pr and then we can, we can go from there sound good.

D

um Yeah yeah yeah sounds good. um Maybe I'll think you offline just just one um thing that I wanted to mention about the schemas yeah.

C

All right, awesome, I'll talk to you guys later then right all.

A