Open Telemetry Uncategorized, 23 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2022-11-23 meeting

Description

Instrumentation: Messaging

A

B

Joe, how are you I'm good for.

A

A long time yes, yeah I've been gone for a while, yes, I'm good I do.

B

Good to see you back.

A

A

Yes, hopefully I am back now, let's see got caught up in some internal stuff.

B

A

I got busy with some internal stuff.

B

Something interesting.

A

Yeah I think so yes, working on on metrics for quite some time now,.

B

I know nothing about metrics.

A

Okay, yeah it it's a bit confusing. Yes, sometimes but I'm, also not an expert, but we've been working on it for a while.

B

So nice, when I see a be all about metrics I know that I can ignore it. I don't know.

A

Nice yeah, okay, I do that for some kind of PRS as well guilty, or we have to filter some stuff right. Otherwise, yeah you got overwhelmed.

A

All right do you think anybody will come today.

B

um Yes, with Mila said children would join all right, hey Johannes,.

A

Because tomorrow is a holiday, I guess for them right there.

B

B

If not, we can agree on everything we will have nowhere. Resistance.

A

I've been looking at at low dimitas VR now I started today. uh uh Halfway through it I think no.

B

Amazing, how does Dwayne.

B

C

A

Hard-Working people, no no Thanksgiving holidays for you.

C

I'm in Canada we had our Thanksgiving a month.

A

And a half- oh yeah, that's true! It's true! Yes, you have you had before. Yes, yeah.

B

A

Confused with that.

B

It's just United States, yes, Thanksgiving.

C

uh Yeah at this time of year, I think so and it's a much bigger holiday in the states than us in Canada, because their Thanksgiving is a Thursday and Friday is basically a holiday and uh in Canada it's just a Monday and it's just kind of a long weekend. But it's a big deal in the states. It's almost as big as Christmas. It seems it.

A

It everybody goes crazy to the store right tomorrow. Yeah.

C

A

C

It keeps pushing back earlier and earlier right like it used to be Black Friday, and then it was like it starts Thursday night. Now it pretty much starts Black. Friday week like it started, Monday.

A

C

It is true I.

A

Remember when I, when I lived there, I lived a bit far from college and and uh a friend of mine took me to it was was uh like Friday and it was like. Oh, we have we're gonna Queue at Walmart, because I want to buy some stuff and I was like what yeah.

C

People stand outside in the freezing cold for, like.

A

Yeah, it was silly me from third world country. Brazil I had no idea about it and I was there freezing in the cold and then when it opened up Jesus Christ it was they they was like Maniac. They went crazy on on on buying things. You.

C

Got to wear protective gear, you can't you got to wear.

A

C

A

I was a bit shocked uh with it. To be honest, oh well,.

A

And I didn't buy anything so I I didn't abide to social pressure.

B

D

You think Lord Mila is.

B

D

Yes, she so first good morning, everybody good evening, Mac Miller said she will be joining so, let's maybe let's wait two more minutes uh and then let's get started.

A

Oh there she is.

D

Since she's here also.

E

E

D

So, first thanks everybody for being flexible and switching days, so great that we can have a meeting this week because we have some stuff on the agenda. So please add your name to the list if you haven't already I put something so on here. Maybe let's start with uh with this PR here, because it ties in with our it. Somehow ties somewhat ties in this all our attribute discussions and uh I would be specially interested in Luke Miller's opinion here, because what the?

D

uh What are you already answered with Miller? Okay, so I I, didn't see that yet so foreign.

D

Related system for setting message, dot ID because in Kafka message doesn't have an ID. The message is just defined, uniquely defined by the partition and by the message offset, and so they were asking here whether they should be the message offset into the message ready or whether it should be a separate attribute.

D

But uh let's see below you already elaborated on that here,.

D

D

Just out of curiosity.

A

The other can come out a bit later.

D

We can, they can do later, I mean about the body seat or that's in that's what I was asking Azure messaging systems. They have a message ID uh and the sequence number and an offset so therefore Azure systems message ID, is different from from offset.

D

So we then probably could be seen as a Kafka, specific and also sqs and SNS support. Both sequence number message: IDs foreign.

D

Mixing message ready and offset in uh and putting it in the same attribute so does having a separate attribute.

D

For that sounds good to me,.

C

Yeah I I think thinking about this a little bit more. It uh I think that makes sense like that.

C

The message ID, perhaps should be defined to be an application layer to how an application is identifying a message to whoever it's sending it to and for any messaging system that, however, it chooses to identify its messages, uniquely that that's under messaging dot system, Dot message: you know if whether it's offset and partition or whether it's just whatever term that system uses because mapping each system into something generic, like you know whether these numbers or strings whatever is most natural, it it's going to vary and and I think I think just yeah keeping them the messaging.message ID as something the application uses to identify.

C

The message is probably the right call.

D

I agree with the other thing coming to mind here. Is that I think when you see something like message.id? You assume that this uniquely identifies identifies a message, but if you put the offset in there in Kafka, it actually doesn't because you could have different messages with the same offset in different partitions secretary for Kafka. For this to be uh to uh to uh uniquely identify methods, you both need the offset, and you also need the partition.

D

You need to know the partition so also that might be confusing by having this by putting the offset message ready, then, because it actually uh does not uniquely identify the message when you here as a separate, offset attribute it's kind of key okay, you have to combine this with the partition or the partition key, and then you get like the unique combination.

D

Sorry, Dwayne I kind of yeah.

C

I I just quickly looked at the definition of JMS message: ID and.

C

It it says that it um supposed to uniquely identify a message sent by each provider, but I thought the maybe I misremembered I thought the application had was able to set it. But me I have to look that up whether it's sent by provider or set by application. The the different IDs have rules for who sets them.

D

C

Also, it will actually.

D

Linked like okay same as connector here.

D

Yeah but I don't exactly know what uh it maps to some.

D

It maps to a field, but they are not sure whether that is kind of part of the Kafka data model is just in a GMS specific.

D

D

But we have a specific real name.

D

But whatever maybe we can come back later to this back-end and- and we like can summarize but uh I I kind of agree with having uh it makes intuitive sense to me to have offset as a separate attribute and that kind of to put the offset into the to the message ready, but maybe a little bit later then can elaborate board on that. We can come back to that.

D

Then we have the attributes PR. Let's.

D

And there has been some offline discussion on that this week. That was great and I. Think there's still some.

D

There are still some open comments, so uh earphone, maybe let's go from the back on that Bob Miller proposed here, was a dedicated or for the patch name, space and basically renaming batch.size to batch message count because size could be interpreted as the actual batch sizing, bytes I.

D

uh That sounds good to me too. I can give a thumbs up here, because I quickly looked in the in the in other semantic combinations and usually for those uh for for those kind of attributes. Count is used and not size. Size usually refers to like the size in bytes, and if you actually count the number of something, then a count is used, or you know semantic convention, so I think would be love. What you proposes here makes sense using message message count instead of just size.

A

I am halfway through through reviewing the pr as well so I'll post my review, I guess either today or tomorrow.

A

D

Have a question.

A

Since we are here, um I was thinking so I was looking at the yaml file and I was thinking. I. Don't know correct me if my my question is stupid, but I saw that, for example, there is a cluster of attributes that are together the message attributes there together under this prefix ID there and then I noticed that some attributes like, for example, the destination I, don't remember now, destination kind or something they are defined, for example inside the producer, uh but they are then referred in the consumer as well.

A

So I was thinking if, like yeah, exactly like this, so there's messaging, Dot message. So I was thinking if, if it would be possible to, for example, have a uh have the the producer and and source and and destination attributes clustered in the in in, like all together, for example, you can I, guess I guess this works, for example, defining an ID messaging dot destination and then a prefix also message messaging dot destination, and then you have only the in this case.

A

You have only, for example, name and don't have to do destination.name, um because I'm reviewing now and then I have to go. Sometimes I have to go back and forth uh to find out where it is actually defined.

E

A

E

Yeah, it can be done um at the same time like. Let's do it uh no objections, uh but this that will not affect markdown at all, so uh maybe we can stage it right. Maybe let's agree on the stuff on there.

A

E

A semantical part and then I think I can refactor yaml before like before. You merge this PR okay.

A

That's good I I just wanted to ask if there is a a particular reason but okay yeah. That's that's perfect, then.

E

Yeah, the reason was that I wanted to reference. Some attributes that are specific to message and the only way to reference them from two different places is to put them on the third place right.

A

D

Yeah, let me understand, because you're still kind of you're shoving attributes around from one thing to the other, so uh I think it will be easy on you to kind of clean up the yaml. At the end, when we kind of agreed there.

A

It was just a thought when I was reading- maybe maybe some other folks will also uh uh say this: uh can they be together because I guess it would make reviewing easier because you just go through them once and then you just say: ref all over.

E

Yeah and also like I, have a PR for build tools um to improve a little bit. This attribute definition, so I will need to get it back after that PRS merged and to make some cosmetic changes in this yamo anyway. So if it happens before the pr this PR is merged, then we can add all of the stuff there. Otherwise we can do it a lot.

A

D

Thank you so, but the way we still it seems there's no position for renaming patch.size to batch to message count so.

C

D

It stands to everybody I think most people, okay, if it starts up for the next one here, that I think was the longer discussion that started about the started. All that was from our last week's discussion. Whether attributes like partition and routing key should be in the message namespace or in the destination source namespace and.

D

The I I think that that was our point of the summary from our last week's meeting. The Russia competition can either be interpreted as part of the message name based namespace, but I think I'm uh Amir brought it up can also be part of the destination source and similar for.

D

I think for rounding key. They said something similar. Yes, oh also, this routing is specific, specifically mentioned this message related interredmq documentation. So basically, then another thought came to me after uh writing. It up with that. Basically, this is the question of the granular granularity of the destination, because, uh uh if you see then look at the destination namespace as a whole kind of, if we have partition in there, you have a much more final grained way of kind of specifying a destination which has its uh prose.

D

Definitely uh because you can you can you can filter for uh on the partition level, then, just by using this destination, namespace and uh yeah there's some discussion, maybe I think I don't need to go into every.

D

We don't need to rehash all of it, but in the end, uh Luke Miller uh introduced uh consumer namespace and I. Think you added the partition to the consumer name space, the consumer, namespace and the partition will be part of source or destination. Yeah.

D

So that makes sense to me: I mean you can all again have a look at the pr look at the attributes, see if that makes sense for you, I had one point that I raised is that uh uh data set okay when the partition is now part of destination of source, then technically uh you would. uh You would see different like the same topic and different conditions of the same topic.

D

You would see that as different sources, which might be confusing, but then we'll build up yeah, it's just it's it's in the freedom of the user to pick which attribute attributes or which subset of destination attributes are used to uniquely Define a destination, so it it makes sense to me.

B

Say what it is, a consumer name spaces.

E

And so we have two properties that are specific to Consumers of our bonus consumer groups: it's not the property of source, it's it describes just the consumer um and then there is a consumer ID and uh I, like maybe just as a cosmetic change. I thought that maybe consumer namespace makes sense, and if we have more things about consumer in particular, we can put them there.

B

But I think that consumer, a consumer group is not a consumer property. It's a state of the that is managed by the vocal right.

E

If you look at the link, I have there, it explains how it works and the only way broker participates. There is helping increase of Kafka is helping elect the coordinator and after that, consumers just talk to each other without even broker doesn't even know.

B

Okay, okay I will read it.

D

Thank you I also, please add your thoughts to this thread here. So I think that's a pretty pretty important part in uh kind of figuring out what goes into Source destination message and also consumer. Now.

D

And let's see if there's any other.

D

I think that should probably be.

E

D

E

One is addressed, at least as far as I couldn't see,.

D

D

And also Trust, so those are the only kind of yeah the three open comments this is addressed. There is like some that's also to ask, there's some ongoing discussions, and then here we basically agreed to rename this to message count. So please again, I I know we all reviewed this PR already several times in different forms. Okay, I'll have another look so uh I.

A

D

We are approaching a really uh uh a really good State here, so thanks everybody for being patient with this work, and especially thanks to Mila for kind of uh uh iterating again and again, and improving this.

E

No thank you all for the review. I really appreciate all your feedback super important.

D

Okay, so that are all the open comments on this one and uh and now I'm not sure that Miller, if you, if you heard when we are kind, if we were listening when we were talking about this message, ready for Kafka now,.

E

I had my probably some prayer monsters, audio, so I didn't yeah what happened there.

D

uh We just quickly, we talked about this issue and we did. We try to understand, like your your stance here, but I think your your main, uh the the conclusion is that uh that that the user tests, you are not using not putting the offset the message ready but having a dedicated attribute for Kafka for the offset, and that makes sense to me and to think to others. Here too.

E

Cool. Thank you for the breakup.

D

The only thing that we were also with to end were wondering about. Was this this JMS message, ID I, think this message already doesn't go like as Kafka doesn't have a message. I think I think it's just a a specific attribute that somehow gets added to a Kafka message. In that case,.

E

Yeah I tried to find like how what exactly it's mapped to, but I, don't think it's in the documentation. I didn't look on the code to check it, uh but my point was that at least in some cases you can have message ID, which, whatever we don't know what it is some random stuff.

D

Yes, oh I, see. The point is so that you could populate message ready here and then this would conflict with the offset.

E

D

E

Assuming we have GMS Kafka instrumentation.

D

Makes sense so I will also add the comment here that can be from the uh from the work group kind of agree with what you wrote there and that putting the offset the message.d is not not a desired.

E

Yeah, maybe you can approve the program, pull request from a couch, because I think this is his main goal to to have the change.

D

Oh this one here, yes, yes, I can do that.

E

Perfect. Thank you.

D

After the meeting okay, then uh Amir brought that up in the in the in Slack. So we have the attribute work now that it's in a pretty good State and pushed that's pushed forward. That's basically open for public review, so I'm your product discussing next steps and.

B

Sections about links, and then we decided that it's it's too complicated to having one a big PR. So we said we will uh um focus on the attributes and then we will have a follow-up PR that will uh handle links. So I want to suggest to to work on this. Pr and I I have a draft of like it's just uh ideas. It's not something! It's not like. It doesn't look like a PR, but I want to uh to a talk with you about it and make sure that we have agreement. I.

B

Think a lot of it is what we already discussed in the previous semitics and yes, so, like basically, uh the goals. If you go to the beginning and I.

C

Think we can have like.

B

We can split it into multiple bios, but the first one that I want to suggest my goals are that giving an orbital array spell I want like a stable algorithm that uh to be able to enumerate to extract all the messages that are involved in this messaging operation, whether it's a batch or a non-batched, and for each message I want to be able to um to get all the attributes that are related to this message. So this is the goal. I have the next phasement.

D

To to shortly cut you off here, let's go into that, but I just saw that this was added here. So maybe it is last point. So maybe let's just quickly talk about that, because I think that can be wrapped up in in various.

A

D

Spend the rest of the time so talking about messaging links, uh I I, guess, did you add little Miller I.

A

D

A

I didn't put my name on it. It was me yeah.

D

A

um I, don't know if you know, but there is this group now um there is this new group for semantic conventions. I think it's I, think no. It's led by Josh um and I I joined a few meetings because they were discussing a lot of things related to metrics in metric Semitic conventions in as something that I was I am working on it.

A

That's why I I've been also away because I was was swamped with metrics metrics related work um anyway, so I I joined this this group for them and they were discussing a lot about breaking changes and and specifically on on metrics and what is the what is defined as a breaking change or not, for example, adding new attributes rename metric and so on uh in in one of the meetings, it was discussed uh things like that they want to decouple the somatic conventions from the spec version, so, for example, people could introduce breaking changes and bump the major version on the semantic conventions document without having to wait for a spec release or having to bump the whole spec version, and also about this schema schema your schema file.

A

The schema transformation was also discussed there. I, don't know if you know about this, but there is this yeah there's the schema file that is used to put breaking changes or things that happen, and it was also agreed more or less that we don't have the power to work on this now.

A

So, um even if we have the schema file where we can't enforce it or no back-ends do anything with it today so and there's not even an implementation in The Collector that does anything um so I think the the goal will be to to use this versioning so versioning the the semantic conventions, uh so we can introduce breaking changes and the reason I brought the series that I remember when I was joining.

A

We were concerned about introducing Auto braking changes um to the document to our document and I think we even consider creating a V2 document uh in keeping the old one, let's say as is, but what happened last week or something there was this PR uh that was a kind of a big deal breaking change.

A

There was a there is a status, uh there's a uh there's, a there's, a semantic convention uh for for a metric that I don't remember what it was, but it's a HTTP related and one of the attributes was the status code, and that was a string for metrics uh and for trades the same attribute exists, but it's it's a integer and the pi was basically changing the metric convention from string to integer uh to the value of the attribute, uh and it was quite surprising to me that it was merged rather quickly because it is a breaking change and and uh like if people are using back-ends that don't support, integer or anything other than string Dimension values for metrics, then they will basically lose all this now.

A

So I started this thread there and on this group and and the response that I got yesterday, was that um it's okay to introduce breaking changes uh because the document is marked as X as experimental and uh that's pretty much it um so I I, yeah, I I just wanted to bring this up because uh I don't know if we still have this concern, but uh apparently it's okay, uh that we introduced breaking changes and and now having this uh somewhat official thing.

A

I guess if somebody says something we have a counter argument now, because that was merged and nobody complained about it. Yes,.

D

A

Think so, Justice right just yeah exactly yes, just just this FYI to you um yeah, and this one was quite quite problematic because the schema transformation thing as well, it can't handle uh today changing attribute types uh like like it like they did there. um So there's not even an entry. Now in this PR there was no entry in the transformation file, schema transformation file. So.

A

Yes, so I don't know, I, don't know if you had the same idea, but my my feeling was that uh the convention that we have today are already used by people and we should be careful uh and then also with this new initiative. With this semantic conventions, group now I saw that some PRS got scaled for example, or got or got a lot of comments uh being careful what what is accepted, but apparently it's it's. It's fine. uh The the document just specified that it's experimental, so breaking change, can happen and should be fine.

D

Okay, thanks for all I think that is uh good to know also been going moving forward with the attribute PR, because that introduces a bunch of breaking changes, but they are good if it's uh clarified there also in writing in Slack yeah. So we can kind of uh refer to that. If anybody pushes back.

A

E

Yeah I also had this experience previously with HTTP stock, making a bunch of breaking changes so yeah. If we can push it through and every time it takes some effort to pursue.

D

Yes, but we have some additional ammunition or arguments here now to move forward, so yeah.

A

It should be fine, you know.

D

Okay, so thank sorry Army for Amir for cutting you off, but uh let's continue talking about about links now.

B

Yes, so so below, uh I want to suggest uh um like things that that I want to say about links, and my justification uh is like my goal. My current goal is to be able to tell if I have like a arbitrary message. A messaging span. I want to be able to enumerate all the messages that it contains like this operation, this messaging operation so like the algorithm that I plan to implement is listed below, and so first of all, the classify like classify Spencer Aspen is a messaging span.

B

If it contains any messaging span attributes. Oh, we talked, maybe if it contains messaging system, so first of all, everything below is only relevant to spends that are of messaging category. So this is like the first statement and then what I want to suggest is that every messaging spell will be classified into a one of three categories: it's either a batch. If, if it contains a batch attribute, that's been attribute, so it's either a messaging batch or messaging system batch. So if an attribute of this name exists, then it's a batch spell.

B

If there is a messaging Dot message attribute, then it's a single message. I spell, and if none of them, then it's a adult spin. So, for example, uh um I bought two examples from sqs, but I'm sure there are a lot of them like create queue. So it's a messaging span because it has like a messaging destination and messaging attributes, but it doesn't involve any message right.

B

It's like it's, not the message, the operation, it's like it's a messaging operation, but not message operation um and then, if you scroll it down a bit and then I want to suggest that we classify links like currently, the links are just one. Big array and I want to um I want us to agree on how we filter from a list of links how we filter those links that describe a single message, so I think the most straightforward way to do it is by by the attribute.

B

So one suggestion could be if they, if a link contains a attribute. That is a from the message namespace then it's just then we know that this link represents a message and if not, then it represents something else. We can filter it out, um so these links can either contain a context or not, as we talked in the past, because maybe we don't have the the context, but we do want to represent the messages link um so that the important part is that we have the attributes right.

B

This is what describes like a message, at least at least for enumeration of messages.

B

um So so, if we look at the analbital list of links, then we can classify those that represents a message and those that are not. And then, if you scroll down a bit.

D

I have a question here of classifying uh uh of this single message, link and uh other links, uh I wonder when whether it all or but it also makes sense here to look at the span kinds of those bands that are linked together, because when I have a link from uh from a producer span to a consumer span, then also that gives me some kind of information about uh about the about the link. Even if maybe link attributes are not present or are not supported by whatever backhand is looking at the link.

D

I wonder if that, if that is part of this section here, if that is, if you see that as a as a different criteria for classification.

B

So my suggestion is, uh it doesn't matter which span it is what's the kind what's like we don't we didn't yet agree or um or like um created, the PLS for the for the um the different kind of messaging spans and how they relate to each other. But what I want to suggest is agnostic to this uh classification like we can decide on it in the future and and change it and do whatever decisions we want, but this algorithm is stable because it doesn't um concern the kind of or the operation or something like this.

B

uh This is at least my suggestion and I'm open to discuss it.

B

D

Makes sense- let's, let's, uh let's, uh let's continue so that we get a complete picture.

B

Okay, so um so now to enumerate messages on an arbitrary messaging span. So if this messaging span is uh is other, so it's not a veteran, it's not a single message. So the the messages are just an empty array because it doesn't involve any messages and if it's a batch, then we go to the links and we filter those that represents a single message link and then for each of these links and merge the link attributes with the span attributes and we get the message attributes right.

B

So it's it's uh I, think it's a straightforward right for batches and it's very easy to to know that we have a missing messages. For example, if we see that the message a messaging batch message count is a 10, but we have five links. Then we know that we, we are missing five messages, so we don't have a lot to do with it. But at least we know it's, not uh it's not a mystery, but so this is the algorithm for batch and for a single message here. It's get gets a bit complicated.

B

So um for a single message. We can either have a link right and if we have a link, then we're expecting only a single, a link that represents a message right and then we take the slim cattle mutes and merge them with the spin attributes and- and basically we know that, there's a single message involved and we know its attributes and and if not then then just the spin attribute so presents the message out of this right. So so for a single message.

B

It can either be a link or not right, and so it's it's very straightforward. It's not a something surprising, but then now below I have changes that I want to bring to the specification to make this possible.

B

So, first of all, we need to agree that the links are used for two purposes. uh We talked about it a lot in the past. I. Just think that if we open a Otep or a spec, we should status that the links are either like a silo, a storage for a um for a like, representing a message, single messages on a batch uh or they can be used because we need to. We want to store the context and therefore like uh different, spends and build like a complete story for, uh for our message.

B

uh Travels in the system right and the second part that there might be controversial is that to introduce a link with the without context. So we talked about it. A lot in the past, I'm, not sure what the status is, but we will need to suggest doing this and hope that there will be no uh okay.

B

People will like it um okay, um so we need to I think to make this algorithm possible. We need to decide on the convention of how we, when we get, we look at a link. How do we know if this link represents a message or not a single message or not, because we could have a mix of links that presents a message and other links?

B

So it's something that we need to agree on, probably best to do it by attributes, and the other thing that we need to agree on is that we limit that each message can only be represented as a link once right, because if we have like a batch of two messages and then we have like three links then or four links, then it's not uh deterministic.

B

We, we cannot deterministically derive the the message list out of the links, because there could be many permutations right or maybe, if we know that each message can be in the links are they more than once, and we know that some messages may not be in the link array. Then, if we see like a batch of two messages and two Linux, we cannot for sure say that those two links represents two different messages. But if we say that each link, each message has exactly one link, then then everything becomes very easy um yeah.

B

So so we talked in the past. It makes sense that instrumentation does not have to create a link for each message right because there's a limits on links, so maybe we throw away some links because there's too much- or maybe it's just too expensive, so it is possible that some messages will not be represented as links and it's okay and just uh we need to State this.

B

um So regarding the link attributes, so I want to suggest that for link attributes, all the attributes are valid, except for batch attributes. So it doesn't make sense to to have like um to to say something about the batch on a link.

B

That represents a single message, but all other attributes are valid and it is recommended to uh to put the um the attributes that that are for a single message, so those that are messaging Dot message, dot, something or for the system, um and it is valid like maybe we have a link that points to somewhere and some of the attributes they repeat themselves like. If you follow the link to the other side, you will see those attributes again right.

B

So so what I want to say is that it's okay, like sometimes the the link May point to somewhere, but you you can't follow the link. It's missing the the link Target is missing and then all you have to do to to make sense of this uh scenario is: is the link attributes, so it is okay to to put the things that are useful there for this case.

B

um It's a it's not that long. It's going to end soon, um so links on a batch and single message. So if there's a, if we have a batch, then we know that we must represent the messages. As links I, don't know other options. We can do it on the spanner reviews for a single message. The link is optional right.

B

um So for a single message, the link is optional. So if it, if it's there, then then it is used. But if it's not there, then we have to visit the the spend parent to get like the spend parent. If it's uh represent a single message, then it behaves like a link like so yeah, I I, hope it's clear.

B

um So so there can be two options right, either to create a link or either to visit the parents to to see the context of the of the this spam um and the the last thing message. ah So the the link Target. So the link Target as I said before it can be missing right um or it can be um so so maybe like we don't have the the link Target.

B

We just create a link with empty context like with the with the invalid context, or maybe we have the context, but we can't follow the link. We we search for it in the database and and we can't find it because it's not sampled or because it's it still didn't arrive. Maybe there was some error when it was dropped, maybe it was sent to another back-end uh like someone sent a message to the queue and someone some other entity reads it and they don't share the same backend.

B

um So a link Target can point to a like. If, if you follow the link to the like resolve it, then you will get on the other side. You get a Spam so like the best case is if this span also represents a single message like if it's a create spam, so we we see like um a single message spent on the other side, but it can also point to a batch on the other side right.

B

So if we follow the link and on the other side we see a batch, then it is uh still possible. It's less useful, but it is something that can happen. So um this is my proposal and.

B

A

It matter I got the part, the part a little bit above the if missing this visit parent, um not sure. If, if you can uh explain that this case more was a bit lost there.

B

So um this is the current state of the spec. So if you have like a boss, spin and- and it presents a single message like an auto batch- then we don't create a link for it right then. What we expect to do is we expect someone wants to to understand the source.

A

Of the message like.

B

Who sent it? He needs to urge um to to look at the parent, and if this parent is a producer, then we know that this is the suspended, sent the message.

A

Right but I thought the current conventions, kind of I mean it was I, guess installed anymore, but it was implicit that the the some context would also travel with the message. No, even if it's single.

B

Yeah so we talked in the past uh the context via the messaging system, and then the receiver uh set the parent as the producer. Then it doesn't create a link in this case right. So if you get like a consumer spend- and you want to know who produces it, then you have two options: either you have a link. This is the the um like uh one option and then you follow the link.

B

But if there is no no link, then you have to look at the parent span and you have to like jump to the parent span and see if it's a producer and then, if it's a producer, then you know things about it.

A

But the but the parents plan could be just, for example, if the broker is instrumented, that that could be the broker the spend from the broker right there. That comes in into my application.

B

Yeah, yes, that's why I think that uh it will make everything very, very easy if.

C

B

Create a link for where, in.

A

B

But I know there was some resistance, like I.

A

Could always create a link and then use the context from the message you mean right to to for the link.

B

Yes, yes, okay, so because otherwise, if you just get a consumer spend and you, then you have to follow the parent. But if it's a batch, then you need to use the links. So it's a different behavior and if there's always a link, then you have a consistent way of processing those that the yeah.

D

A

I think what we talked to was that that it was a single message. We wouldn't create a link, uh but but I think it also makes sense to to have the link, even if it's just one yeah.

B

D

I think there's a discussion going on about the weather uh parents, uh parent spans for Consumer producer links should be possible in uh in singing method scenarios that is actually going on on uh on this PR here for span structure for messaging scenarios like to Midler brought this up, but for going back to this document now.

D

So thanks a lot Amir I think that gives basically a uh kind of a backhand, A, View From, The backend side on how to kind of work with uh spans in messaging scenarios and I'm not sure yet how to best proceed here, because there's uh there's this Otep for span structure that this kind of this draft Otep and uh I am I'm, currently not sure how the best kind of we are combining these works, like the works, the work that you did here and this um let's see about it, open the the draft here with the span structure.

D

Do you think this this one one can be merged into the other? Or do you see this kind of a different? Do you see this targeting a different uh work stream.

B

So I think it uh like all all the things that I describe here. They are um um agnostic to dispense structure so giving a span, no matter what he describes. If, if it's an operation that describes the single message or a batch of messages, then then he just describes how to uh to store links and to extract the messages from this pencil. I, don't see how it will contradict.

B

In any way, and also if this algorithm is if this, um if the the structure is changed in the future or if someone is not a compliant to the structure, then this algorithm will still work for any case, because it doesn't assume anything about the structure.

E

I think there is, there are two fundamental agreements we need to make with communities for both of uh these things. It's that the fact that we populate links no matter what, if there is a context if there is no context right in the second part, there's links after start, but you don't have to fight very hard on the second one, because we have workarounds, eventually it will get through. We don't have to be in this battle, but we need to be in the first battle.

E

We need to say: okay, we want to populate things, even if there is no context on the message um and if we make just one change after the attributes goes through that batch scenarios like in best scenarios, instrumentations, must populate links instead of me and must populate one link per message, no matter what like, unless explicitly disabled right, then uh I think that this will be a good base for both of the changes.

B

Over three minutes, do you think this should be a Otep? Can it go directly to the spec? Maybe someone is has time and wants to to do it together.

E

I would like to review the document and read more because I didn't catch all of it, but I think the part that we can populate links without an empty context. I don't feel it should be an auto type. It's just a small change foreign.

C

E

Anything in this pack.

D

I mean there's, there's something that comes to my mind, not here when we talk about you're, having a link with an empty context, uh I mean there's even the possibility that, when you put like a context, information on the message that uh that you put context information on there from a span that is actually not sampled, so you could kind of put a transparent on a message with the sample flag set to zero and there you then have kind of a similar situation.

D

You kind of also, basically, you receive the trace parent on the consumer side and then, basically, you create the link to this band. That is actually not sampled. I think that is just another variation of the empty link use case and I think it's actually interesting uh interesting to think whether whether that is something that should be mentioned or kind of recommended into spec that if you have an unsampled kind of context, should you still put that information on the message or should you or should you not.

B

I I strongly suggest to do it, because, even if it's not sampled, if you have multiple links that have the same context and you don't have the target of it, but you will see that all of those links have this point to the same place. Then you can correlate them. You can know that they refer to the same message just by uh comparing the context so I want to suggest to do it.

E

Yeah I agree on the sampling. Motivation was oh or nothing right. So if you sample it in you better record it all, it has.

D

So uh there's there's one minute left I. Think for for this. For this document I'm, your uh I think I recommend everybody here in this meeting to have a look at the document. Maybe you can put a link on the slack Channel and people can review and add comments and what I will also do what I will add to the slack Channel?

D

There are already several issues that uh where this, where this topic comes up with links and Link, attributes and types for links and also the possibility of having a empty links, particular links not pointing to any uh to any Target, and maybe then it would be a good starting point to kind of continue discussion on some or one of those of those issues and just see what other people uh other people in the community outside of this messaging group here say to this proposal and then, uh when we kind of test the orders uh in that way, we can go from there.

D

So I will I I I know I I need to look it up, but there were several uh issues where resource discussed or the Lord Miller I think was involved in some. So I will try to dig that up. Add that to the channel and then maybe onions, The Next Step, based on this document. We can continue like discussion in one of those issues and see kind of what uh what other people say to this to this proposed approach, and then we see, maybe maybe an old. Maybe it's something object is needed.

D

Maybe just this can go into the spec in a different way and we can we can. Then uh we can then continue from. There sounds.

B

Good okay, so I'll see you next week on Thursday.

D

Awesome thanks. Everybody thanks for uh being flexible this week and uh uh thanks to Mila for working on the attributes, PR and saying things I'm here for getting this link, work started and see you next week.

E