Open Telemetry Uncategorized, 5 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2022-08-05 meeting

Description

Instrumentation: Messaging

A

Hello: everyone, hello, hello,.

B

A

I can drive today.

A

Mix it up today and uh go from the end of last week, seven days ago, instead of starting from the first page. Just because.

A

All right first issue: I think the answer is no.

A

I don't think we have a remote right receiver.

B

I think so I think we might have a proposal.

C

I think maybe we can export with a remote right but correct.

A

There's definitely a remote one.

A

A

I don't remember seeing an open.

A

Component, hopefully there is.

A

Maybe it's accepted.

B

Just linked to what I believe, the proposal in the chat it doesn't look like it has.

C

B

Labels on it burning, accepted or not.

A

This was discussed recently all right, I'm just going to link to this.

A

uh Should we label this, I'm hoping that we can just close this.

C

I mean it seems pretty clear to just be a duplicate of.

A

C

Just found, I'm not gonna do.

A

Anything them cool good, find all right next, one waiting for author. We might not need to do anything with this. Probably don't because.

A

Oh no they've responded, so the tag's not appropriate anymore.

C

I think we talked about this one last week. Oh did we.

A

A

A

Yeah did talk about this one we could throw on some. uh This was just like they, they just weren't able to get to their back and let's see what they added.

A

I can just throw an elastic exporter tag on here and move on probably.

C

Yeah, I think, last week we thought it would be just like a simple misconfiguration, but it seems like it might yeah. Maybe it would actually be a problem, so yeah.

A

C

A

A

A

Looks like maybe they've got a real bug. Oh, it's also already got priorities on it, so we go over this one too. I think we must have cool got all those all right back to the first page.

A

I did that one loki exporter.

A

Oh drossy already he opened this himself.

A

Let him decide the priority since he seems to have an idea of what he needs to do.

A

A

This looks like a feature request.

A

But oh, they do have a specific exporter in mind.

A

C

Yeah, it looks like the uh the stack trace attribute value is being truncated.

C

Okay, I don't know if that's done by the exporter or by gcp.

A

Dashboard can add labels, so maybe priority needed, but sometimes we do priority threes for these, I'm okay with that as well. They can switch it.

C

A

I think the issue is that.

C

We just have like a stack trace attribute, so they just.

A

If you look at.

C

The top one it just shows these are just the attributes or whatever it, but they have a particular format where, if you put it in their format, it will display it as a stack trace. So they're.

A

Asking for some.

C

Sort of transformation to be done.

A

Stack trace ends up in like an error event on a span. Doesn't it it has. I'm not.

C

Familiar with the way that google cloud works, what do you mean.

A

Telemetry yeah open telemetry protocol.

C

um I think stack trace ends up on uh good question. Actually, I think it's a span event. Let's see how fast I can get to the produce.

C

Should be recorded as an evidence.

A

Exception. Thank you. That's what it is yeah. Oh.

A

I thought it was on there.

C

No, it's not a specific part of the product.

A

Oh, it's just it's just an event. Yeah. It's.

C

A span event with a name exception: um okay and a particular attribute values which do show.

A

Those backgrounds show up okay and.

C

Stack trace is.

A

Just getting thrown in the attributes, okay, yeah.

C

So the google cloud protocol- I guess- has a specific place where, if you put a stack trace.

A

C

Be shown cool given.

A

C

That is a nice.

A

Little view there sweet seems like a good thing.

A

All right now we're doing that. One looks like it's already got as much needed tag so skip it.

A

I think this person might work at splunk.

A

Nope, I might just use splunk, then.

A

Traces and metrics python use log content length, config key. I don't I'm not familiar with the exporter, but maybe it's fixed.

C

All right yeah, it looks like it. It's the same person yep.

C

Cool those are the best yeah best issues, yep.

A

What do you think dan on this issue? Do you think it's a priority one, or is it just a two for us.

B

It doesn't seem like it's urgent, but it does seem like it's moving quickly. Well,.

A

B

A

I don't know what kind of urgency it has, but we do like this. We do say that when the one comes out we drop the other one. So like I think it's going to be something that gets dropped in 5, 8 and 58. I mean.

B

A

So maybe p1, I don't think that.

B

It matters the time because alex is driving. This yeah.

A

He's already doing it I'll just leave that yeah. It will.

C

Be done anyway,.

A

So yeah it's getting done cool! I'm just going to put a he's following up on that, I'm just going to put priority on it.

A

I thought we did, I mean I'm on an arm. 64 computer.

C

Yeah, I think it's just not in the ci, it's not tested. Oh so it's like it works, but it's not a build target. We got the same request in uh js gotcha.

A

Yeah, so we at the after we do all the unit tests. We check cross compilation.

C

A

C

That's checking that it builds.

A

Yeah, but not that the unit tests right pass on it yeah. That seems like an interesting discussion. Come back to that one really that one open.

C

I mean I assume that they would, but you know, I'm.

A

Very interested in a weird opponent, yeah. Definitely I actually just had a unit test fail on something in a gcp library where, if it was using a float and a unit test and the flow was supposed to be like 6.0 and I got like 6.0002 or whatever it was, which I think is.

C

Why yeah you're.

A

Not supposed to use floats from the unit test, I'm excited about this component. This is a big one. I think this will be good. Multi-Tenant stuff is going to be a big deal and we don't have a good solution for it, yet that one's a priority on that one's got it.

A

This one does not have a priority, but it probably could.

C

I mean logs are experimental. I wouldn't think that this would be a particularly high priority.

A

Yeah, the uh I think it I don't know if it's fully supported of metrics yet, but it would be nice if the telemetry gen um could do everything, because the collector can do everything so like being able to generate logs would be nice just like we can generate the other stuff.

A

A duplicate, that's probably a duplicate issue, but that's okay.

B

What was I doing? We have a way to generate logs in the uh stanza packages um like in uh it's like an input operator and just based on the configuration um operator. Input generate.

A

Generate, oh nice, maybe you should move this into the into telemetry, gen or duplicate it or make it use it or something that could be cool. I suggested it a.

B

Long time ago- and it wasn't well received, but uh I think that might have been before telemetry gen was as far along.

A

Yeah, I bet I bet now would be no problem.

A

uh Where are we loki.

A

There's been some permanent error work in core.

A

A

Is loki not using the exporter helper, because this is a pr open right now in.

A

Core that expands on what was already just expanded, um not that so all of these this is exporter helper, which I hope loki's using and all of these codes aren't retried.

A

But I guess 400. Well, I don't know it's 400, one of them yeah bad request. So maybe they're not on the latest version of the collector.

C

Well, bad request has probably been there for a while right, yeah.

A

Let me go look at: let's look at the change log real fast make sure it wasn't just released.

A

Actually yeah did all the releases yeah.

A

A

Just have to read them, I guess.

A

All right we'll do this another way.

A

A

A

Oh, it did it's, not a change log. Well I'll, just link to this then yeah I mean it's already yeah. They didn't add that one.

C

Yeah, I'm I'm looking into loki exporter to see if they do anything weird like I, even if you're using the exporter helper, don't you have to check if it's a permanent error or not, maybe they're not doing that check? Well, I think the exporter.

A

Helper, uh this file this this thing here, uh I think it's just used in the generic uh export method.

C

Where is it oh so like it won't even make it to the exporter.

A

Yeah like as long as the loki exporter is using this export method behind the scenes then like it should be getting. This is permanent client failure function uh yeah, I don't know. I guess that needs some looking into I'll, say well. Jurassic already knows that he doesn't need me to tell him so I'll. Just add a label on here.

A

Interesting, I don't know why that would be like that.

A

All right, prometheus receiver. We have a lot of previous issues.

A

I do not have an answer for that one. I don't do almost anything with prometheus.

B

All right, where are we where, where is.

A

Did yeah he added that, so we just need to add a priority.

A

Pablo's been doing a good job of going through issues and adding labels, but he hasn't been adding priority.

A

I think I'll reach out to him and ask him to to add that, since he's been doing like a lot of work to like get the right labels onto things, he can just even if he just throws a priority needed on there. That's helpful.

A

C

Link his pr already, I almost wonder if their input data is bad like are they do they have the attributes? They think they do.

A

Yeah and well, and also there's some linked pull requests open for this. So it looks like they know, what's happening, so I'm going to make them.

C

A

C

It's the host metrics receiver, so it's not external data.

A

It looks like it just might be how datadog exporters is doing something that would be my guess, not knowing much about dad tug's proprietary format.

A

Processor filter.

A

I think I added the label. No, I didn't, I did add a label to this one, but I didn't read it. I did what pablo did.

B

This can be a priority too. Okay, there was some functionality for filtering on severity text and uh I think number would actually be more useful. So I should make this happen here.

C

Yeah minimum severity seems like an obvious filter for logs yeah. That's such a good idea. Yeah. It's probably going to be just about the most used. One.

A

Cool, should we throw a help wanted on this one, or did someone that he already submitted his own pr? It looks like he did submit a pr for it, cool, so no help when it needed.

A

You've got uh context on this one dan yeah.

B

um Yeah I was just looking at this this morning. um I mean I just throw p2 on this. I think all right cool, it's being worked on.

A

I didn't really do a good job of thinking about help monitor good first issue labels today.

C

So this one there's a duplicate uh right above it. If you look at the way, this is.

A

A duplicate is it the exact same.

C

So they opened the same issue in this and the collector contrib repo or in the column. So.

A

It also looks like this is the exact same issue.

C

Yeah, it's the exact same issue, bogdan just transferred it to the oh.

A

He transferred it over here. Okay, I'm gonna close one of them done use this one.

A

All right, so this one needs.

A

Aws x-ray exporter.

A

I don't actually know aws x-ray, okay, it's probably just them.

A

It looks like their back end's, not there.

C

Yeah I mean it looks almost like I mean the url looks good to me, but.

C

Client timeout, while awaiting headers.

C

I mean maybe they're, maybe it's a networking config thing in aws: maybe they don't have permission to access x-ray from the host that they're.

C

A

C

Not an issue with the collector to me, it seems like, uh but yeah.

C

I wish there was a way to better uh discourage people from opening duplicate issues across multiple repos, something that we get in js all the time.

A

A

I should have done that here we are okay,.

A

Three minutes: let's do it. uh Yes, I told someone to open this issue.

C

Yeah I mean I created the dynatrace exporter and I gotta say it was. It was pretty difficult to figure out like what the basic required stuff was. That was needed at the time, maybe improved now, but.

A

Yeah yeah, I think the when they're talking this person in the cncs slack like they wanted to be able to like have like just like a skeleton right like either something that just lives there, maybe like a mate command, would probably make more sense. But I think the issue that they link to that pablo linked to is probably the best one. Yeah.

C

Would be cool to have like a code generator for, like you know, build the basic skeleton required for an exporter or.

A

B

Yeah, definitely in the in the past, I've seen that that done and what you want to do is have it run as part of unit tests too like it should. I should always be validating or generating a valid exporter, even if it does enough.

A

We generate some other code, so it's not unfeasible to do something like that. Like p, data is a lot of generated code.

B

Yeah because the problem is, if you don't, if you don't validate it constantly, then.

A

Yeah and you start generating components that don't work.

C

Yeah, if you generate, if, if a new user tries to say oh I'm, making a new exporter and runs the new exporter command and then that doesn't work, that's endlessly.

A

C

Divide by zero, this person works with slunk.

A

Yeah- and it's already got a priority, so I'm just going to close this say, which is this one shot this one cannot assign requested address. That title seems good sarah's, not a bug. Okay, that's good!.

C

Oh yeah, this looks like a configuration issue with how they've got their kubernetes set up.

A

Hey he's just a cluster.

C

Ip the same as their node port, both in 90 90.. So if you look right now,.

A

You you should go comment that, because I have not found it yet. Oh this here, 90 90., yeah yeah. You should add that comment. That's a good find! Do you want the link.

B

uh I'll find it, let's see.

C

12 975., thank you.

A

Cool, uh I think this is. We have a good tag for stuff like this danish there's something that's like user error should make a.

C

User error tag is there like an advice or discussion label or something along the way? There's.

A

A discussion label yeah not even necessarily.

C

User error he's just looking for advice.

C

This gush discussion needed.

A

Dang, that's not! I don't want either of those I think, there's a question label yeah just like so.

B

That when you go.

A

And we'll take bug off there. We go all right last, one uh the same time, another prometheus receiver and there's some people on it.

C

Can't scrape histogram and summary at the same time, that's why.

A

Summer is going away right.

C

A

Yeah, I'm just gonna put a priority needed and call it done. Sweet.

C

It looks like they have the same name yeah it I'm not familiar enough with prometheus to say whether that should be valid or not either yep or.

A

David said.

C

I think it's probably just a misconfigured yeah like I agree. Let's do that because if you have, if you.

B

Have a histogram.

C

And a summary they could potentially not agree with each other very easily, which would be really frustrating.

A

C

A

C

A

That was, I think, we're back at the top of the list.

C

Might be the first time since I started joining.

A

Those meetings.

C

A

For 33 seconds ago, I'm not I'm just gonna you're too late. He needed to open that before eleven. It's.

C

A

We should check okay, single node mode without making crashes off fails.

C

If a single exporter fails well, the receiver.

A

Will crash that doesn't always mean the collector crashes.

C

Oh yeah index out of range, so it's probably just trying to open.

A

C

Authentication method that returns an array and they're just not checking on it. This would crash this would mean.

A

The whole collector dies.

B

A

Were trying to spike.

B

He's been upgrading his components.

C

A crash is probably a piece of.

A

One right I'll put a pc or a p1 on this, because it's a component.

A

I already won cool, I'm going to assign this to him because he said he's working on it, cool all right, thanks, everyone all right now, we're done now we're.

C

A