Open Telemetry Uncategorized, 27 Sep 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2022-09-27 meeting

Description

cncf-opentelemetry@cncf.io's Personal Meeting Room

A

A

Bye guys morning.

B

C

D

Foreign, hey everybody. Do we start? Let's start in one minute, see that scale in the meantime. Please add yourself to the Channel. Please.

D

Okay, let's start that's okay, thank you! So much for joining. Let's go over teaching the items. First, one uh metrics same compatible generation from German. Please review it again. Your Rams.

B

Yes, I'm here yeah, so this is a PR in the build tools um that uh James opened. So we we have been lucky for a while now that our our metric symmetric convinces are not being done via the tool. So it's just a markdown file. uh So James worked on on on adding this uh adding support for this in the in the semantic convection generator, and it had approvals for from from some folks. But there was more discussions and he did some. Let's say major changes.

B

The way the the result is uh and I have, reviewed and and and I think it looks good, uh but uh yeah. We need the eyes off people that reviewed before his changes to make sure that everything is all right uh yeah so that that's basically it I think this is important for the uh instrumentation uh initiative that is going on now, because this will drastically improve the experience for generating semantic Rich for metrics.

A

We should probably dismiss the reviews that were done previously if there were major changes to the pr.

B

Yeah yeah I I, don't know if I, probably I can't do that, but um yeah I agree that that should be the case. Yes, oh.

A

A

I will do that. I have the right. Okay, perfect! They can't do that. Yeah I will do that.

B

D

Sweet thank you so much for that. Okay, um and this is an important project for for the for instrumentation stability, so yeah especially important for this.

B

Yes, there's two issues in in that that this this VR will will fix so.

D

Like one one more besides, this.

B

You know the uh if you scroll below there is a list of issues uh that this usually tracks.

D

B

The semantic convection generator will solve both of them about metrics not being generated by the tool. Okay, perfect.

D

Sweet thank you so much for that. Okay, uh next one tigran uh decided it's original and lower camel case. Json Keys should be supported. Please.

A

Yeah, so this is a an issue with uh Json encoding. You know TLP the. By default the product Buffs Json implementations. They support uh two ways of naming the fields. One is uh to use the original names as they are declared in the product of files in the product files and the other is to use the equivalent lower KML case field names. So the names are converted automatically and the the exporters are allowing to use both of the names and the parsers are required to accept either of these names in the payload.

A

So this is the default behavior of product buff implementations, uh the the the the defaults that it shows chosen is the lower camel case but optionally. The exporters can also use the originals if they want to uh so this. This issue is about making a decision whether in otlp, we want to support this Behavior.

A

Whether we want to make both of these naming conventions acceptable for otlp specification, or we want to say that there is no need to provide a choice here and we want to choose one and stick only with one and exporters should export in one way and receive receivers.

A

Only need to support that that one way of field naming uh initially there was a very brief discussion, I think only uh Daniel, I think on the expressed opinion that we should support both, but now that there is a PR that attempts to uh make sure that this is explicitly listed in the specification.

A

I think they're not concerned that that is unnecessary complication. There is no need to support both naming conventions, so this is the question I have to you. If you have any opinion about this uh I'd like to hear it, do you think we should support both? Do you think we should support one uh any any arguments in favor one or the other approach.

E

But just to clarify, in my opinion, I I I, don't have a strong opinion one way or the other I was just wondering. If there's any, is there any benefit in restricting this? If, if, if the behavior of all the existing uh implementations is one thing, then why would we specify anything different than that like? If, if there's no advantage to it, then I don't see a reason to do it? uh If, if there's even one single Advantage, then fine I don't have a strong opinion at all.

A

One small Advantage is when, when you're re-implementing it from scratch, like we did in the collector in The Collector, we don't use the the the protoboss implementation. We have our own implementation for Json or PLP, and there you have to support both, which is I, mean it's fairly simple. To do no big deal, but it's a bit kind of there is. There is a possibility that you can have some bugs there like if you have typos or something like that right.

A

So I guess that small Advantage is there that if you are doing the implementation that then uh kind of nice things to do right, less chance to have bugs there.

F

I'm not really sure that we get a choice in this matter, though, because the proto-spec says that producers can choose either, and so since we know that there may be producers that are producing either form. We need to accept either form.

E

Yeah I was just going to say the same thing that Anthony did if uh there's some other and we we can always put it in our spec and and say you must do this, but uh drifting too far from the protobuf spec just makes it more annoying for any potential producers. You know if I, if I have a database that wants to export otlp for self-monitoring or something like that, um you know I think staying as close to you, the expected protobuf spec and uh precedent.

E

That is an advantage there.

A

So you're saying what we mean in the simplification of receiver implementation, we lose in the complication from for from the exporter's perspective, because now they maybe can't use the built-in photograph marshaller, they are forced to re-implement their own, maybe because what we require is impossible. Some people.

F

Yeah I don't know if it's additional complication, because the protocol specs says that producers May provide an option to use the original name. So I I think that anybody who has that enabled would have the option to disable it potentially if it doesn't conflict with other uses that they have that require that to be enabled for some reason, but I think it would be not non-obvious Behavior right. It's it's going to lead to systems that don't work for reasons that the user won't immediately be able to understand.

E

F

E

So if I, if I have that feature enabled and I just don't see any uh you know traces in my my collector, my back end or whatever, then you can always read the documentation and figure out why that is. But it's just one thing that you have to do.

A

I mean it's our respect right, our protocol, our implementations.

A

How is it that anybody is using? Are we talking about current implementations or someone who does the implementation of otop exporter in the future? Oh.

F

Tlps are are spec for a protobuf based data set, but it's not a spec for protobufactory Json margin, that's not ours that exists elsewhere and is already defined.

A

It's not already defined in the sense that it's not stable, yet right, that's what we're doing now.

E

Is I guess the the question at the core is: is otlp a protobuf uh uh product like protobuf-based protocol, or is it a subset of protobuf.

A

We already deviate in a couple places from product yeah and we had to so. We already did that. So if there is a reason that we need to deviate here as well, then we can do that. I, don't see a problem with that, uh except if there is no reason, I agree with you. Why would we do that right?

A

So that's I, guess what we need to decide. Is there a strong enough reason for one more deviation from the protocol spec or we say it's pointless? Let's just do what the protograph says we should do which I tend to agree with.

F

A

F

Where have we deviated right? We deviated on Trace, ID and span ID, representing.

A

That, yes, yeah, probably yeah the the the deviations are listed in the stack. If you look where I think it's a couple small things, it's not major deviations, but it's there.

E

Also I think some of the the enum handling we restricted beyond the default. um I would say: I I, myself, I think I've been fairly consistent about arguing against deviations at each each step along the way and from what I remember I think Anthony has as well.

A

Yeah I agree with that position: I, don't think we should deviate unless there is a strong need to do that. So in this particular case, I also tend to agree with you guys. That is probably no such strong reason.

A

So I don't know, maybe maybe you can also comment on the issue. uh I know you already did Daniel, maybe anything you can do as well and.

F

A

F

Come into the new PR, um okay, but I, think Yuri's not here, but there also doesn't seem to be anybody else. Stepping up in this meeting saying you should deviate, which is probably a signal.

A

Yeah anyone, maybe someone wants to speak up.

E

I would be interesting interested in hearing from anyone, uh potentially at a vendor who has implemented a custom receiver and not used an existing Library, where this would be a problem.

E

Jack in the chat for for the record said he supports not deviating from private buff.

A

Yeah, not a vendor, but, as I said in the collector, we have custom implementation and I mean it was simple to support both because of the big deal.

A

A

Who has arguments in favor or against one of the approaches? Please comment on the pr uh Jake I think it will be good if you do as well yeah I've just approved. It looks good to me.

A

D

Think we're good. Thank you perfect. Thank you. So much for raising that. Okay. Next item uh add changes to semantic conventions block for how long do.

G

You want to comment about.

D

G

Yeah, that's me Andre here, hi uh yeah, so I raised a couple of uh issues uh in uh in the certification. Repository, sorry and I got the feedback from Riley. That I should probably pause for now. If you scroll down to the comments yeah, this one is from probably from the private account, but the other ones are from uh from right. Young and it's related to this thing and I wonder: does it mean we cannot introduce any changes to semantic conventions for metrics or whatever else or and for how long?

G

If that's the case, what does it mean.

H

First, uh the feedback we provided is not from Individual.

H

We have a quick meeting, so it's a group D theorem from the church number two is uh the current instrumentation is a little bit that it has been running for more than two years um we haven't shipped anything stable, so there's some fundamental issue we want to address and before that we're saying taking more PR's, that's increasing the current experimental side if nothing to hold so instead we're taking a systematic approach to look at how we can ever get the first thing stable and then leverage the success we have.

H

There have other work streams to follow it before that. My personal feeling is, these PRS are not going anywhere like we. We can continue to spend the energy from the community having people review and got them merge as experimental. But later we have to revisit everything and there's another work student talking about how elastic common schema can be aligned with open Telemetry. So all these things is very likely to reset whatever we have, especially for Matrix. Even if you look at Matrix today, it's not following the way how how to listen. Romantic Community is working.

H

We don't have a formal way of eating yamas, we generate the size. So my suggestion is we. We save the energy here and solve the fundamental issue before we move forward. Otherwise, it's it's more like a wasting energy and redo 90 of the work here. But meanwhile, I I think some of the discussion, like the the priority mentioned something about the operating system or the process.

H

Level thing is really helpful, so I would suggest uh Mark those prns Drive continue to get feedback from others, but don't merge that for now, as long as we have Clarity, we think this is the right model. We can decide whether like we, we should write that in the Yama format after we clean up the thing or we we should get like can't even see another thing, because if yes already had that definition but to just take from UCS.

G

So it means that we are blocked and we don't know for how long this is that right.

H

So there is the summary from bursary like yes,.

I

Sorry I'm a little late yeah. Let's, let's talk about what's likely to break right. So that's that's! What we've been trying to do with this is to give guidance on these PRS of, what's likely to break so you don't spend efforts on things that are likely to break or that blocked component.

I

So, what's likely to break, is um choosing what resource to attach your Telemetry to we're, having discussions about like what that means and what that looks like um and then any kind of attribute that you define that could be shared with a metric signal right, there's, there's event-based data and then there's aggregated based data and for like time, series, databases, time, series, databases, struggle with high cardinality, and so we need to kind of limit ourselves.

I

If an attribute is designed to be in an event store, it can have high cardinality if it's designed to be in a Time series database. We have to be more careful about that cardinality and so calling out. The attributes that you expect to share are important. We have some actual bugs from like the Java instrumentation, where um some very high cardinality attributes and traces were shared with metrics and actually caused problems and and storage issues and query time performance bugs.

I

So those are like the things that are likely to change that I would be very careful of now. The rest of it will be about what uh allowable changes can come in the future.

I

So it's more about the evolution of telemetry, but those first, two things are: if your PR touches any of those two things right like what resource to attach a metric to what resource to attach a trace to or if it touches um any kind of label that you would expect to be shared between a metric and something else where the cardinality is not explicitly kind of, you know reigned in then you should expect some to have to come back and revisit it.

A

So I think the original question is still valid right. What Angie is asking is: are we blocked for an unknown period of time or months years? I think that's that's a valid question right and what what you said, Josh I, think we need to have specific guidance about what we do accept, Now versus what we think should not be accepted because it's subject to changes in the future right. So maybe we should do that right. Let's, let's have that guidance and apply that guidance on this PRS right. So maybe some of the PRS are fine.

A

They can be accepted and some of those because they are highly likely to to to change as a result of what we decide in semantic conventions. Work group, uh that's that. For that reason we we block the PRS I I do agree that we should not be blocking all of the evolution of semantic conventions that open Telemetry. It's probably not the right thing to do unless we know clearly that it will be unblocked in let's say two weeks three weeks. That would be fine, but we don't right. So we can just block it indefinitely.

A

I agree with this with that um Josh can.

C

We make a distinction between semantic, conventional metric names and semiconventional, attribute names and semantic conventional resource attributes, because I feel here that Andrea is proposing sorry for your name. Wrong is proposing metrics, not new, cardinality, literally missing metrics, so I think system. Uptime is really important, I think process, uptime is really important. I know, ECS has a a different schematic convention and we can support both we're just writing conventions for what these names mean. When they're used as metrics and I feel like we're falling behind process memory, utilization 2 seems really uncontroversial.

C

It's not going to increase it's not going to increase cardinality um I, just think all these things are reasonable and they shouldn't be stuck in this other debate about inflationary cardinality, which is about metric labels and and for the most part, I believe our resource conventions are about adding attributes to resources which really should not. They want to stop repeating this. We should not be experiencing cardinality explosion because of resource attributes in Prometheus, which is like the golden model that we have right now.

C

Those resource attributes are things you join against by by taking your your existing cardinality find your target info using instance and job. There are your resource attributes. You can join them using a group left expression in prom ql. There's no cardinality from new attributes just want to stop repeating that, and then we can move forward with what Josh and rally were talking about for that attribute names.

I

Yeah, so, okay, sorry go ahead in this specific issue. In this specific issue, I agree that I think these metrics are okay to add. There is one meta concern that, like Tigard and I, were discussing in the pending design doc around how to handle identified resource attributes, which is where do you attach the metric?

I

Should a process metric be attached to a process resource, or should it be attached to a host resource? Today, all of our process, metrics, are attached to a host resource and I'd suggest I. Don't expect that to change, we actually consider them kind of orthogonal to each other and that that is just a decision we've made.

I

But what we're trying to do is provide guidance for why we made that decision, so we can make similar decisions in the future. That's the only concern that I would say affects these from like what we've raised so far in this kind of discussion.

C

Could you clarify that concern I didn't quite understand so um yeah.

A

Process ID can be an attribute on the resource or can be an attribute on the metric data.

C

Point why would it ever be an attribute on the metric data point that makes no sense.

A

What the process ID so.

I

If I'm, if I'm getting process, metrics and I'm, asking the host I'm running like the PS command and pulling process, metrics right do I attach those metrics to a host resource and process. Id is a label on a particular metric stream or do I attach them to a process resource where the process ID is considered. Part of that resource, identity and I. Don't have process ID as a label on the metric, but.

A

There's that's equivalent data, but.

I

It's just a different modeling decision, they're exactly the same.

C

I

C

Is really a Semitic modeling debate, an automatic conventions, debate they're the.

I

Same yeah, I, don't think so, but in any case, what we're trying to do is be consistent and come up with a consistent set of rules for everyone to follow. If you're, following the existing Convention of all of these metrics are attached to a post-resource and that process ID is part of the time series as a label, then I think everything's fine here, because that's the approach, we're taking with the initial proposal.

I

However, I do expect us to bike shed that specific Nuance over and over and around in circles for a very long time and I. Don't want you to be blocked by it. So I think because Samantha like because in practice you could model it either way and they're transformable between each other kind of doesn't matter. So just stick with what we have now we'll describe. Why and we can deal we can deal with the Fallout later is basically what I'm suggesting. As long as you know, there could be breaking changes coming.

A

J

A

You can we have that that maybe one page guidance what you just said right Josh. uh Can you I, don't know, maybe you're you're in the best position to write that point where.

I

Where do you want? It is the question where do I put that that's that's the I, don't know where to write it. If I could write it somewhere, I would have written it already, um but I'm not sure where to put.

C

It can I say an example of the problem that you're describing just to make sure we all agree on that. So you know you could have an SDK a go SDK I'm thinking of and it's able to read flash proc and Linux and identify its own process. Cpu usage, its own host Pro parameters like how much memory is on this host and stuff, and it could report its own metrics on that stuff.

C

We could come up with a totally different set of semantic conventions for how a host metrics receiver on The Collector running on the same node is able to report exactly the same information using different resources and different metrics right. It and I don't see that as a real problem, because I might want to run an environment with no collector in my SDK and Report, some metrics and I might want to run an environment with a with a collector and no SDK support and both should be valid and both should have meaning I.

C

Think that's what I mean. That's my position here.

E

C

Is not a semantic conventions problem? It's a modeling question. First time, we've really talked about it. I think in this meeting.

I

Interesting so yeah, but the ques, the semantic conventions are a model right. How do I report these kind of metrics in this scenario, so I don't want to dive into it. Now, that's what we want that working group to be defined as and I think I'm going to schedule that first meeting I. Unfortunately, I was going to do it this week, but um I had a bunch of personal stuff, come up so I'm going to schedule it for next week um for the first meeting. But what like this is.

I

This is going to be that initial topic of discussion. We have a design, doc, ticker and I are working on which we want to give time for people to review um before we actually have that first working group meeting, where we'll talk through I, think the most important first things to discuss. But basically the goal I think that that we've come down on is if something is an identity.

I

Sorry, if a resource, if we Define Resources with good identity semantics, it actually shouldn't matter in practice, whether I attach to a process Identity or an SDK, Identity or whatever, because we can transform them in back, ends easily and figure out that kind of identity transformation. We're just picking a default for how we want to do semantic conventions modeling right. So there will be a default for how we model in otel, but in practice.

I

If you want to transform that- and we want to provide collector Transformations great, it doesn't matter because they effectively mean the same thing.

A

So what we're not going to be able to design it in this call right? It's it's a large topic. What I suggest that we do is we agree on one thing: we cannot block the evolution of semantic conventions indefinitely.

A

uh We we need to produce some sort of guidance which says what sort of semantic conventions we agree to to accept for now and what sort we please agree, because they are very likely to to be changed in the future, and we need that guidance quickly right, not in three months. We need it in a couple weeks.

A

Let's say right and in the meantime, it's okay to continue working on the PRS, which we believe are likely to fit into the guidance of what is acceptable, but we're not going to measure them until that guidance is published. Are we okay with.

H

That yeah my girlfriend is next week. Drug storage is going to schedule. That meeting like we'll have a meeting in in two weeks or three weeks, and we should use that meeting to address this issue and meanwhile I'm wondering like we still need the guidance, because I I I feel this is a very urgent thing that we need to address as soon as possible. So why do we produce some intermediate guidance? The Virgins have a meeting in two three weeks.

A

No, that's fine, I think you're right. We we can't produce that that guidance as as a result of the next week's meeting right. That's that's fine! Yes,.

H

Maybe I was there shouldn't, be a guidance for people who work on semantic conventions. They should just go and join the meeting with Josh. That's.

A

Fine, yes, yeah! That's that's a good point as well. If you want to have some sort of semantic conventions added to to the spec, and you have interesting generally helping with this notion, please do join that meeting as well and help us coming up with that guidance right.

H

So this is exactly what we post during the church, we're saying hey. It looks like you, have five PR's they're super interested in this area and they have some fundamental issues. So if you have an energy, you should just join us and discuss this fundamental issue.

A

Yeah the point that I'm trying to make is that we we don't want to block people indefinitely I, think that's not the right thing to do. Yeah! That's! That's all I wanted to communicate that we want to unblock you guys, one way or another. Either we say it's acceptable or we say it's unacceptable, but we do that reasonably quickly in a matter of let's say a couple weeks: not not vlogging. Definitely that's! That's all I wanted to say yeah.

A

Does that answer your question Angie.

G

Yes, thank you very much tigrant. Thank you. Jay MCD and this all sounds good uh I'm, not sure what meeting you are talking about with Joshua. It's not scheduled.

A

G

A

The meeting for the semantic conventions work group, the first one, that we want to commence and.

I

It'll it'll show up on the calendar and we'll advertise it in slack. So if you're on slack on the specification working group we'll advertise the the time there, um it's either going to be on a Monday or on a Friday. What time zone are you by the way? Because there is a uh doodle you can fill out still? Okay,.

G

Yeah I think I see the uh I'm in Europe in Warsaw right, so it's uh Central, Europe uh UTC.

I

G

I

Okay, so we'll prefer one of the earlier attempts, then yeah.

H

I thought for for Josh, I, I I feel there's a topic we we should discuss in the very first or maybe the second meeting, there's elastic uh common schema and we're looking for. If we we could align some effort here so for things that are very common, like system uptime or this cooperation. If you look at ECS, they already have that defined for many years and it's being used. So so there's a metal question like.

H

Are we going to encourage people to define the similar thing in open times and later we converge, or we should just stop them and and hurt people to use the ECS as much as possible. So we have lesson to converge in the future.

I

Yes, that's a great question: yeah um and I I will I'll. Add that to the docket.

A

Yeah yeah: let's try to have an answer for that as well right, at least how we see this. How do we see this merging whatever the merging means? It's not clear, but let's try to come up with an answer to that as well. Thanks.

G

Okay thanks: everyone I thought it. Maybe it's not a good question for a meeting. I thought, maybe it's better if it's a slack, but this conversation I think was quite brutal. So thanks everyone.

D

And by the way, out of curiosity is according to the discussion we have. Maybe this one is not blog correct, but anyway just some idea, because this is attached to the to the entire system, but anyway, let's continue discussing offline okay. Next one Bruno possibility to really suspect version. 131, yeah, Bruno.

J

um So um let me relay the request from the eclipse microphile folks, so we as we submitted, requests to to add the the property that would enabled and disable the SDK. That's that was uh requirement that that group came came to and this took off a while to merge. Now it's merge and it's solved, but um the spec was released recently and I. Don't force it to be released before the the upcoming micro file release, which will happen around October November.

J

So this is uh to ask you guys if we can release uh 1.13.1 with whatever we have up until now, or something like that.

D

So in theory, we will have a release next month, like monthly. We have like very bad luck in the past weeks because if you know summer was slow, but in theory we should have a release uh in October, which means next week.

D

um So that's a plan. If there was something to delay that, then we could consider doing uh one thirteen one I think.

A

D

A

This is a new feature, though right. It's not a patch released. Why? Why are we doing the patch release for this.

J

um This is to align with the requirements of uh the release on the other side,.

A

um No I understand that, but this is the you're suggesting a patch release. Patch releases are for bug fixes. This is not a bug. This is new environment. Variable right, it's it's a new feature and.

J

A

Think what we should aim for is just make a regular release. It seems like it fits your timelines. If we do it early in October, you should be okay right, yeah, 114.,.

J

Yeah I think if it happens during the the first week of October, it should be totally fine.

A

Okay, so maybe let's try to do that yep and we accelerate the October release a bit. Try to make it early October rather than late October. Is that okay? Yes,.

D

Yes, thanks very much. Okay, in that case, I can probably prepare the release on Monday next Monday and let's see how that goes. So, basically, we will have like a live review next uh Tuesday in the call, hopefully that that works sounds good, sweet, okay, uh two more from Josh Joshua McDonald's. Please all.

C

Right so um uh this first one is something I'd like to get into the next release. Frankly, so I want to bring attention to it. um There's So Meta comment on this uh PR here we're talking about uh it's been reopened. This 28.38 and um the observation I have is that I think this review strung on for too long. It made it so that reviewers lost context and couldn't really see what they were reviewing I I. This happened to me as well.

C

The original PR listed some options that we I believe discussed in this meeting two months ago um by the time I got to like really scrutinizing this PR I had forgotten that there were options and I was only looking at it from the frame of mind of. Is this a correct change to the specification which looked good but I kept seeing, but I was, and I was confused by the options and I'll say that just admit it, but what I? What I saw when I came to catch up with this PR?

C

Is that at least two people, one of them on the TC, seemed to continue to oppose it, and I was confused about that. Until now, so what I see now is the pr in front of us David has written reopened it without the review history so that we can stop talking about options. This is one option here.

C

um It does have some approvals. I think people are approving this because they don't care and they want to get it done, but there are still options and I'd like us to take the best one um when I finally got to understanding what I just said, I've started to feel in line with the other two commenters that the other option might be better. They also don't want to make work for David.

C

But honestly, this PR is so close to the other option that it doesn't take much to change and I want to discuss whether everyone else experienced that, like me, and they aren't quite sure what the other option was, and if so, we should hopefully ask David to do it, one more time or I. Even I looked at forking his PR, you know copying it myself and drafting it, because it didn't look too hard.

C

The idea is, and if you now scroll down to the bottom of this um Carlos um I made some some ASCII art there. um The point is that the the top diagram is what's in David's PR currently, and the bottom diagram is how I see the alternative.

C

um Bogan was on record saying he liked the alternative and Aaron Claussen uh go. Maintainer is also on record as saying that and I started. Finally understanding. It I actually think that the second option or the alternative option is actually pretty good. It leaves this complication about the the bridge out of the main SDK and the meter provider is not talking about bridges.

C

It does create a little bit of a usability issue if you're going to be registering a lot of bridges and a lot of readers, because you have to register every producer for every reader instead of doing it once per SDK.

C

That's how I see it and still I think it leaves us with a cleaner spec and it doesn't make a big change for David I just want to call that out. I think we should review the alternative and I also think we should move quickly, because this is this is blocking a bunch of stuff um with Hotel go and The Collector. um Would anyone like to speak on that I.

K

I have a question just to make sure we understand each other on the alternative you, you are saying that metric reader has an interface of uh where, where it exports, this export data.

C

Okay, usually it looks like a method named produce or sorry collect. The collective method returns a batch of data in some SDK specific way, yeah.

K

So I think in the and you think that is the same interface or any metric producer. So essentially metric reader becomes a producer or you think you.

C

Would call it a processor if it was in The Collector I want to call it an append, only processor, because I don't want it to muck around with data that I've already exposed. I just wanted to say here you can add Scopes, as you wish and I would even be willing to say it's, not the responsibility of anyone to validate that. There's no conflicts, whether let the recipient deal with that I would I would be happier I.

C

Think we're trying to simplify our specification right now and the reason why option one, which is the pr in front of us, is a little complicated more so than I. Think it has to be. Is that we're requiring error handling and and it is going to create a harder implementation? Basically, um this seems to be the slightly easier path. uh Okay, but let's.

I

Talk: let's talk the user experience here, like two I mean like come back to what this is targeted at, which is say getting rid of open census or or like a deprecation path for open census users right if you're, requiring every single metric reader to be overridden that that means like and there's not a great automatic path to do that right. That's like really ugly to set up or impractical in some cases for auto instrumentation, for example, um to actually do that in practice. Secondarily, the notion of resource being shared is an important aspect.

I

Resource is not exposed today in an easy way there. It's it's somewhat hard to get access to Resource. So that means some sort of weird interdependency between this overriding metric reader, because we don't give metric reader resource right. Are you going to change that so this thing can have access to it. I think the notion of sharing resource is the important part of this bridge.

I

C

Exposed resource, like you, see the resource when you're a metric reader like you just get one of them so I I mean that was in my draft of that.

I

Again, if you're hiding the metric reader interface right, does that mean I collect metrics from the original media provider? Look at the resource in it and use that then to generate the additional metrics.

C

When you append Scopes, you don't even get to say the resource you just give me more Scopes and I will and the re like. Okay, there's some enforcement mechanism, you're saying like if the the raw data is passing through, do they have an opportunity to to change the resource? I would still expect a single resource to come through to the exporter. The exporter does not expect to see multiple resources.

C

um I feel like we're we're describing an unexpected Corner case. Essentially I'm.

I

Suggesting that whatever interface you're talking about for metric reader is not the same one I know of in the spec. So if I'm looking at what's in the spell.

C

I think for metric reader.

I

And the bare minimum right, if you were to hijack that thing and insert like this metric producer or insert like a.

C

Bridge the same API and and now you've got a chain of readers. One can collect from the next.

I

Yes, how do I get access to Resource to make sure that this thing, literally, your.

C

Bridge does not get to change the resource. Your Bridge gets to produce scope, objects with metric data in them and not a not a resource object. I I mean in in practice you're going to have to restrict the API somewhat.

I

How are you enforcing that with the existing API? That's not how the readers specified like.

E

You're saying yes, you can only append.

I

Scope but like what I'm suggesting is that's not how metric reader works today.

C

To me, a metric reader is a is a um an implementation. The the implementation of metric reader is the detail here. It's every SDK can do it differently. The SDK the reader was provided as a configuration mechanism, and we added a single method called collect to get the data um I. Don't see why you can't chain them to to add data without it.

I

We haven't specified this to function correctly and the implementations vary based on language, and so we we actually it will require more specification work to put it into reader to make sure that certain things are upheld and the most important thing is I. Think the user experience of someone trying to use one of these Bridges.

C

I want to talk about user experience separately from the question about whether metric reader is is the wrong interface, because Java created a loophole that we've discussed in the prior PR. That makes absolutely no sense to me and I: don't want to go complicating our specification because Java exposed resource to the reader in the wrong way. So, but but let's pause that conversation for a moment because I'm I'm, okay, letting Java do whatever it wants as long as it meets classification, so I want to write specification to correct what Java did.

C

The usability issue could be corrected with like a helper method, which is something like saying this SDK setup will have the following metric producer: I want to call them processors since that's effectively. What they are here are some processors. They will be added to every pipeline, which means every reader will have automatically have processors attached that inject data as a metric producer would um that's. The preference I have.

K

But why? Why is not metric reader a producer itself and we have a set of producers and we allow to do a fan fan in equivalent. So.

C

Essentiator is a producer in the in the in the imaginary setting so.

K

C

It's just the first one and its implementation is going to be different and that again.

K

Yeah but but we shouldn't, we should do offending model, even if we should just be like it's much easier to understand. If you have 10 producers like an array of producers and you just merge them versus chain them anyway,.

C

That's fair implementation detail as far as I'm concerned. The point is that there's a bunch of people that are adding Scopes after the metric reader from the SDK, the from the meter producer but easy.

K

Is it after so so so you are requiring them to add it after and don't have I I mean independent really come with the newest report, so essentially, if, if we have have, if we have so you are you try to limit the fact that a producer can produce a resource as well.

C

I I literally have limited that in my implementation, so I'm I'm, you hope.

I

That that's not specified! You haven't done that in the specification like if, if you're going to make that proposal, I.

C

Actually pointed to where the resource SDK specification says: You must have one entity with one set of attributes, which is enough for me to at that point. Yeah you're.

E

C

It wrong, if you allow more than one resource out of an exporter and I.

I

C

I

It in the specification I'm not talking about what I'm trying to talk about is: how does this hook get access to the resource? It? Doesn't it again.

C

It just keeps producing Scopes the resources come coming from the original metric reader. The way I see it so.

F

What is the data structure? That's handed to the exporter, then shouldn't the exporter be given a protometrics or an analogous object that starts with a resource. The stop level is the resource right.

C

I

I'm saying it so Josh we've never specified this type, the the way that metrics are collected for traces. We specified here's. What a read write span looks like here's. What a span looks like for metrics, we never specified any type in the specification. It is up to the implementations, do whatever the hell they want. So if we wanted to go with that route like that means, we need to fully specify what these types look like in some fashion or be a little bit more prescriptive here to to answer that in the spec right.

C

I see I, guess, I see what you're saying I didn't think it was missing from the spec and um uh I I mean I, guess, I agree this points to a weaknesses in our resource specification.

I

No I'm, specifically talking about the metric specification metrics, do not specify like interfaces for in-flight metrics. So when.

C

It got the resource specifications. The resource specification makes requirements that are have to be met, it doesn't say how to meet them. I, don't see why we should.

I

uh That that's fine! What so, if you, if we go, look at the implementations of metric reader and Metric producer and the types that are passed around okay, maybe go allows passing resource separately from the data. But that's not true in every language, and we have stable releases of this because we didn't specify that that they has to be passed separately or.

C

We did specify that one resource equals one entity and, if you're, allowing more than one resource out of your SDK I feel like you've broken. Some other requirement is my point.

I

So what I'm asking is, if we add this metric reader thing, how do we enforce that.

C

It's up to every SDK as far as I can tell that's that's where I would put it and and in Java you can create a bridge like a producer I'm, going to call it processor to break through the past the processor interface that doesn't allow you to give a resource, and it just gets stapled onto the current export, which comes with a resource from the meter provider.

E

uh So I his conversation is a little tough because I feel like somehow we're both like people are talking past each other um and I'm really confused at this point uh as to kind of like the I.

G

Think I see the high levels.

E

I'm also really interested Josh sureth like how are metrics handled down a pipeline like did. Does the data type that you use in Java's implementation? Look anything like the um otlp uh data model yeah. It's.

I

It's a so what the way Java looks and I guess. Jack can correct me if I'm wrong here, but there's a um it's a flattening of the data model into a single kind of entity. So there's this there's this like resource metric that has a resource, a scope and a metric attached to it that you get a collection of these things.

E

Okay, yeah I mean that. So that's why I'm a little confused that that is very similar to what um we do in go as well, and so I I wonder if, if I just I worry that this conversation we're talking past each other into like what the structure is going to look like in the.

I

End well, what I'm suggesting is so so like in terms of let's talk about requirements for what this thing needs to do right. So the the goal is like, let's say: I have an open census. Metric system right requirement: number one is all of those metrics that come from it should attach to the same Resource as open, telemetry.

I

Okay goal number two: is we want this to kind of be um easy to configure and um an addendum to what open, Telemetry metrics provide the idea being if we want to deprecate open census and move to open, Telemetry I, add this bridge right and then I can slowly remove my instrumentation of open census, metrics and replace the dope and Telemetry without losing metrics, so I can go through piecemeal and change.

I

My instrumentation right um and everything should work now to actually work in the specification with like metric producer meter provider, all that kind of stuff. The question is what we tried to what we proposed for the open senses bridge for tracing and what already was approved. Was this notion of I? First, add the open Telemetry bridge to my application, which means I now use the open, Telemetry exporters with open census data, and we have this thing where context is actually shared between the two libraries and traces.

I

You know from one go to open, Telemetry processing right for open census. We can't do that for a variety of reasons. Basically, the view specification and the instrument specification is significantly different to the point where it's impossible. So what we're suggesting here is that we get the best possible benefit of okay. We use all of your open Telemetry configuration for exporting it. We just have to use the actual aggregate metric store for open census because of that breakage in the model right, but the idea of being from a user standpoint. I add this bridge.

I

All of my configuration for open, Telemetry doesn't change, but I might see some breaking changes going from the old, open census, exporters into the open, Telemetry exporters because of some changes in functionality and some changes in resource, and that's a one-time breaking change that I make and then post that I start slowly re-instrumenting my metrics right- and this is the migration path. This is what we're looking for from this particular feature like not to lose sight of like I want to.

I

You know this isn't just about open census, but it is primarily about it and and that path right. So how do I take this other storage of memory and attach it to open Telemetry such that the export path, Remains the Same, and so this whole, like, oh just you know, add it in a thousand bajillion places, I'm very much against, because it goes against that open census. Migration philosophy that we proposed right and I don't see the benefit to users either.

E

So Josh I, I uh I would be surprised if anyone on the call kind of disagreed with a lot a lot of those points you just made um I think fundamentally, I think everyone agrees that all the the data from open business needs to show up under this resource. I think that the user story of of having that come in in a migration path that, like is forward-facing, is also agreed upon.

E

um I saw this PR as the ability to do that, but if, if you, if you see something that is contrary to that, I definitely want to see. This is what I'm saying I, don't think I understand the the contrary, nature to this PR, to being able to produce that so I want to understand that a little better.

I

I'm, a fan of this PR I'm, not a fan of the Alternatives being proposed. Okay,.

E

C

Sorry go ahead, Tower no.

E

No I I was, it was just a thought. Pattern go.

C

Ahead, I was trying to help get this resolved and I I've stated my opinions um I'm glad to hear a strong opposition and and I think. At this point we actually still have a debate even if I stepped back bugged and had opinions about option two um I'm interested in getting this solved. I I will approve the current PR. 2838 is fine with me. I just think it's a little harder to implement Josh I do hear the usability issue. I did say something like with metric processor that gets stapled onto every export pipeline. I.

C

Think that solves the usability issue. um I want to step back and let others speak, but we should resolve this.

K

So I I can I am the one who opposes a bit. So the judge. Can you explain to me or maybe I'm, not understanding, so you are okay with this current proposal which, uh which you said, um does it not make a weird name metric producer, but you are not able to produce the entire metrics. You are just I mean you are producing only not just the scope and the metric is not the resource.

I

Okay I mean, if that's the, if that's the complaint, I would maybe call it the metric Bridge producer right, but like from the standpoint of what this looks like architecturally. No, that doesn't bother me. Okay,.

K

So so then, then, in your opinion than any SDK meter is a producer.

I

K

That's I think.

I

Yeah, what what you're trying to go for is an SDK meter provider is the same thing as a metric producer. Is that yeah.

K

That's what I that's what makes sense to me, but but we we are making like the SDK meter, be the equivalent of a provider because that's what I'm hearing, which is, it owns the scope and the meters, so the metrics associated with that score, but it doesn't own the resource.

I

So so Bogdan I would have agreed with you, except the direction we took with metrics is there is a odd relationship like inverse relationship between reader and provider, where reader can determine default, aggregation with provider and therefore a this metric producer thing this? What I would call a bridge metric producer cannot have that interaction or- or it's not obvious, whether that interaction is.

F

I

F

I

Yeah and that's why I think they're slightly different.

K

Okay but I think I. Think then the reader is a producer in this case, so so Provider Plus reader is a producer.

C

That's actually how I implemented it. In my prototype I, the each meter produces a set of scope, a single scope and a list of metrics, and the sdk's job is to call all the meters get one scope for each and my original concept for this metric producer was like this PR in front of us is just now called producers and get one more scope from each of them.

C

um So and I I feel that that is okay, I.

I

I see what you're saying, but if we focus on how you configure an SDK now things get weird if we take Josh's proposal, where there's this thing that you can attach to every single metric reader like a processor- maybe that's, okay, because again, but like the idea being I want to add this bridge in one spot and you reuse my export config from the SDK, but the problem is reader and provider are Blended and when I, when I configure one of them I'm kind of configuring a little bit of both of them in a weird way.

I

So, where do I hook this this additional set of metrics that will attach to every metric reader from that SDK I have to do it either at the meter provider. You know you know point or something that knows when all metric readers are registered, which is still meter provider right. So no matter what I'm going to have a call to the meter provider. That will be aware of this thing: okay, even if I register it in every single metric reader.

I

That call still looks the same right like we could implement this as a as a processor in metric reader in an SDK. As long as that call is like at the SDK level, you.

K

See what I'm saying? Okay, okay, I, see what you are seeing you you want to achieve another thing, which is if, if I register, multiple metric readers I uh to the same provider, you want all of them to see the the the the bridge metrics without having users. To think that there is a bridge. I mean you want to decouple decouple the registration of the readers from from the bridge and I.

I

Yeah, this is that this is that story for open census, migration of like okay. First, you take this bridge hit where all of you you're gonna, have a breaking change where suddenly you're using Hotel resources and exporters right, but both metrics show up then I slowly remove my instrumentation of open senses and get open, Telemetry metrics, and so my metrics slowly shift from one scope to another across the bridge right. But the idea is I'm reusing I'm using the otel exporter specification now and no open census things whatsoever for Xbox. So.

K

Then then, then, then, if we do this model, that David proposed uh I I start now to understand the motivations and kind of like it, with the only one exception, I think the metric producer, it's it's kind of a wrong name and I think I think the reason why this is ex so so what we are seeing is the data that are coming from. This is a scope plus the metrics associated with this scope. Correct.

K

That's that's the output of this, uh this uh interface, which for me, looks very similar with the with the meter SDK like with the meter so I think. If we come up with the with an interface I think the SDK meter should implement the same interface and from and from the perspective of the provider. We should have just a list of these interfaces, which some of them are meter. Sdk. Some of them are bridges, but all of them are treated the same way.

K

Does it make sense like just for me to make mental sense of this thing.

I

I I can see that as a direction to go with implementation yeah from from the standpoint of what a user interacts with again I think that would be an SDK level interaction but from a user standpoint I, don't think a user needs to interact with that I think the user basically registers the bridge with the SDK and then all the details of that are hidden behind.

K

Them right the reason. The reason now now correct up to a certain point: there were a version and I, don't know exactly what is the status of that do? Do open Telemetry views take effect for of this metrics. So essentially, do we affect this meter uh like the metrics produced by this meter and I?

K

Think here is where, where, where probably I have a shortcut in my mind, which is we are making these two look like a meter provider, a like a meter producer, but not all the things are applying to this as they are applying to the other. uh That's why, for me, is like I want to see it from different way, because I don't want to connect them with the views at all. Views are part of this metric reader combination with with the providers. That's why I said.

K

Oh, if I see this as a producer, this combination of those two and I have a way to to see this as a producer, then completely decouple the views from these. Now, if we are attaching to the provider, my problem is now I have a shortcut in my mind. When do a view apply or not to apply to these metrics, what I can configure? What I cannot configure to this, and this is where I, where I have troubles understanding? Does it make sense to you.

I

Yeah yeah, but the thing is in our specification: we've coupled the two right, so I can't I can't produce a metric export pipeline without attaching it to a meter provider. So if you give me a solution to that, then I agree with you, but unfortunately we have coupled those two they're the same thing. Maybe we decouple. This is maybe then propose something that decouples them. I haven't seen a proposal that effectively decouples them in a way that I think actually is any different than what's being brought by David here. So like yeah.

K

The way how I see it is, if we make the metric reader.

A

K

This producer interface, then, then the way how we connect an exporter to the reader is via this exporter interface. Sorry, this producer interface, maybe that's an option. I, don't know I I'll, think more about, but but anyway, I think we.

I

Discussed but when we think about that think about it in the context of how I configure the metrics SDK right and what it looks like to a user. If you give me a syntax of like here's, what it would look like and go Java JavaScript that you know effectively achieves the thing we want of I. Add the bridge right and all of my exporters are shared.

I

You know with with this new thing: great, that's fine, but I haven't seen a proposal that does that. Okay.

K

uh But you are not worried about the confusion that views are not applying to these metrics, even though we add these metrics in the in the provider.

I

K

I

Because I think with a I think we can clearly document it. B I think that's a one-time, confusion and C. It helps motivate deprecating like we want to get people off of Open census right, but we want to give them a very easy path to do so and I. You know fundamentally they're going to be controlling their views in open census, while they're.

K

I

K

But but but I was thinking that these these mechanism can be used for other metrics libraries. So I was thinking that if we want to plug in other metric libraries, which are not the one time only maybe yeah.

I

Well, let's talk: let's talk about that because, like this bridge should only be used for metric libraries, where you cannot tackle the events, if you can tackle the events at source, you should come through a meter and come through our instruments because.

C

You're going to get the best quarterly to that, if all of your metrics are asynchronous, you could also instrument using callbacks and yes, and that is a a nice way to do it, because you get your View support and if we would just say bridges are restricted, accumulative gauge metrics, we might be able to do it that way.

I

Well, yeah, but that's not possible because of the metrics. We have to support.

C

Don't you output entirely cumulative from open census.

I

You have you have choices, yeah yeah, you can get, you can get. You can get. The rare.

K

The Jewish.

I

I think there's a way to get Delta metrics.

C

We're way over time, you guys this was a okay. In any case, more complicated conversation.

I

You can get summaries as the main problem.

K

S Delta summaries; no, no, no, not Delta. There is no Delta an opposite, so there is no Delta. You get only summaries or he survives. Everything is cumulative. Yeah.

C

I I actually think that provides me. A third alternative, which is all bridges, must be implemented as callbacks and and therefore there's no producer. There's just Integrations that that provide way.

I

To do distributions today with that, unfortunately,.

C

Right and I think we could solve that and I'm sort of waiting for 1.0, because I want us to have a gauge distribution and I. Don't think we can get an asynchronous histogram without having a good Instagram as well to.

I

Confuse okay, we went back and forth in a whole bunch the point being uh summaries that there are things the only time you're going to use. This interface is, if you can't it get access to events or the appropriate callback. If you can't use our raw instruments, which is the case for open census today.

I

So that's why I'm not as worried about those confusing users, because I think in all cases, there's a technical limitation where you're going to have to use the in-process aggregation defined by the underlying system until that instrumentation is Rewritten and in almost all cases, I can imagine that underlying system owns the aggregation. So, okay, anyway,.

I

All right, we'll see.