Open Telemetry Uncategorized, 21 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2022-11-21 meeting

Description

cncf-opentelemetry@cncf.io's Personal Meeting Room

A

A

Hello, hello, that's awesome, seeing you twice.

B

Change that back there we go. um That's that's what I'll do in meetings where I'd run to go. Do something really quickly. I'll just switch the background so that so that I'm never gone.

A

Yeah, so people assume that you're still there right. So that's awesome like like the the glasses from humor Simpsons, where he had like the eyes on the glasses yeah, something.

C

Like that awesome.

A

I I'm not sure if anybody else is joining me and let's see.

C

um I'll win them.

B

Sounds good to me: oh God I got a new keyboard with layers and all that kind of stuff. It's the Ergo, docs and I've never used one of those keyboards before so. My typing is but.

B

Yeah, um let's see I don't feel like. We really should need to wait. No there's several anonymous creatures in here. Maybe we could wait like a few more short minutes. Yeah.

C

C

Maybe putting this Zoom link to.

C

New channel yeah.

D

A

So my zoom is not able to update right now. That's really weird looks.

C

Like maybe it's because you're using WebEx I don't know, maybe this stop us from that.

A

Maybe it's just people having having this document open all the time. I don't know so.

C

B

I have no idea, I haven't.

C

B

They don't seem to be active in the document, so.

D

Yeah, exactly okay, yeah.

A

Then I don't know, let's make it quick, yeah.

A

B

Linting stuff, or do you want to.

A

Wait yeah I mean I. I was curious to also get some some feedback from Patrice uh I. Don't know. Have you have you looked into it? Have you played around with it so.

B

um Yeah we we used Veil tried to use it briefly for the honeycomb docs, and that was um deeply problematic uh but like that was also it might have been misconfigured or it was too aggressively configured or something like that. But it was, it would have been a significant lift to get our docs to be like happy with or get Veil to be happy with her Docs um like granted. uh I actually do enjoy tools like this.

B

um We used one at Microsoft that it it didn't run on CI checks uh for some reason, but you could run it in vs code and.

A

B

It did like, uh like analyze text, you know, oh, you used the word it here but like it may not be referred like it may not be clear that this is referring to what you were previously talking about. Please consider blah blah um and then it would generate a score and the bar was that for any Microsoft writer. So like not external contributors, but any Microsoft writer.

B

If you want to contribute to the docs, you need to get a score of 80 or higher and 80 is like really really easy to meet uh and then from there like perfectionists will be like oh well. You know it noticed all these problems I'm just going to go ahead and fix those, because whatever.

A

um Yeah yeah a.

B

Similar kind of like a lightweight, feel to it that we could do where, like those of us who care a lot about the quality of the docs, can like pretty cheaply, go in and modify things, but then it doesn't like block contributors in the sense that they're, like oh now, I have to make like 50 changes to this document. For it to be happy.

B

um That's really I think like where my.

A

Bar yeah, no I, I I mean I, really played a lot around with it more than I should have probably uh and then at the end, I came to a similar conclusion that, while it probably is, is a complete, Overkill um and I also find it a little bit clunky to configure, because you have this any file, and then you have a folder and a lot of files in it and what I did at the end is and I think it should be there already in a very tiny way.

A

Instead, I added I think text lint, it is called. uh Let me see if I have it still here and the good thing is it's: it's JavaScript, so you can just um Let Me Maybe share my screen. Real quick.

C

A

Yeah so so here you can put it into the package Json directly and the thing I mainly cared about initially is things like hey make sure that people write open Telemetry like that or uh that Kate is recommended to write it that way. This is what I did I looked up the so the cncf has a very, very lightwide recommendation on that. So there's basically two things.

A

The one thing is write, open, source and Cloud native without hyphen technically, but but who is doing that with Cloud native I never saw it that way, but yeah, but but technically that that's that's their other uh recommendation and then they say like hey. Each each cncf project has their way how they write their their project.

A

Please make sure that you use it that way and I think the the most critical one are arcades and grpc, because they're a little bit special and cncf like that, um but but I didn't do more than that, and the cool thing is now um so a little bit of that I figured out at the end.

A

It's even also covered by sea spell, um but but the cool thing here is you can even do things like that here, so providing some kind of regular expression and then saying like hey check for something funky like that, um and the cool thing is, we could add additional rules over time. So if we say like hey uh I, don't know we don't like uh I, don't know, or we want our titles to be written in title case or sentence case or whatever uh like here.

A

We have a pixel mix already that we could just drop in a rule and do it like that and I said it's it's not getting into the way of uh contributor that that's doing something sometimes like the markdown limb right I mean I turn on markdown land and I open a document and I get all those those issues that annoy me, um but but I mean the the people that contribute from time to time. I think it should not get in their way. So yeah, yeah and I said I like that.

A

I can just put it into package.json like uh like we do it with uh Mark Donald, and now it was prettier as well, so yeah, not yet another folder and files, and then whatever so um yeah foreign.

B

I would definitely be in favor of adding heading I mean it feels like it's pretty pretty.

A

Light it's in there already. So if you do the most recent npm install, you should get it I guess uh and then I said it. It should do things like I, don't know! Hopefully it works right now. So if you have open source something so you see like here and then it's doing something like that and the cool thing about and that's something you cannot do with visual- is that it can do something like uh yeah, Auto fixing, so so the integration here is a little bit better. So that's enough yeah yeah! Exactly so!

A

That's that! That's that's! Really the only thing I I just added there and said we can grow from there. So.

D

Okay, uh yeah, oh well, should all not be there. Let me stop sharing my screen.

D

I hate that anyways.

A

um The other one was around uh the vendor pages, I, don't know if you saw that one yeah I think you commented on it on it as well. So there was initially the suggestion to have this table with all the languages and uh and and whatnot and I said, like I'm, not really Keen to to manage that. No, um but definitely we we could do better right and then we could do so. Just I mean you saw my suggestion like saying like hey: maybe we can do Native, Native, otlp, endpoint and then here's an exporter.

A

He has some distribution if they like, they can point to it, but or no they have to point to it right. So so we would play play it back to the vendors so I'm not keen to to do this research myself. So just giving this back to them and saying, like hey um yeah, please take care of that right.

B

And it might also be helpful to within otlp support to be able to say, like traces, metrics and logs um independently I know that so that's come up a lot in talking with prospects for honeycomb, where, like a lot of them like a couple months ago, they were like. Oh it's really important to us that you support open Telemetry logs.

B

Like do you support that and then we're like well not yet, but, like you know, logs only just stabilized, and so that's why it hasn't been a priority right now um but like I, imagine all the other vendors are kind of having similar sorts of conversations, um but like ideally, they shouldn't have had to like ask that question at all like they could have just or there could have been like a canonical Source like here.

B

You can see, or you know, when you're in the business of like okay, we got it standing up to collector we're now going to bake off between like these four vendors, um you just go here and be like okay, like these. These These are what, like all the vendors claim to support here, um and then you don't have to like you know, dig in through, especially for like a large Enterprise weight. You know, however long it takes to for that question, to like bubble up to the person who actually knows the answer and then.

D

A

No, no I I mean totally I mean having something like hey, we I don't know so we, for example, I mean um technically what one of our backends right now only do does tracing and exactly everybody's saying like hey, why are you not doing metrics and it's like yeah it's complicated, but if it would be on that page, exactly I mean people would see like okay, that's that's not supported, so yeah um makes sense, yeah, yeah yeah.

A

At the end, we need to need to bring it back to to the to our repo from the community reprint and maybe take it there from there. I mean yeah I. Think I talked with it with Patrice like two weeks ago. Something like that. The this vendor page needs an update, but I never thought it would be important, but I mean yeah. Since people brought it back so yeah. Let's do it, then: okay, awesome yeah, that's it um I mean I saw a lot of activity.

A

I mean you did a bunch of things for Ruby right so yeah. um But beyond that I.

B

A

B

Or um yeah I think this week, I plan on doing another pass at Ruby, um getting the probably the exporters side of things in place. um I'm also not happy with some of the existing stuff that was in manual instrumentation, um so like there might.

D

B

Like smaller tweaks, there like, for example, the span events section I actually want to change that, but then, like it kind of, gets the point across and so I was like. Okay I'm just not going to change it. I'll just leave it as in well we'll get that merged.

A

Good good good good enough yeah yeah.

B

Yeah so, like probably do some of that for Ruby um and then I really want to finish off the metrics stuff, but like it's hard, um I I, so I I, don't I might I'm, probably just not holding it correctly, which would indicate that, like this is a problem. This needs to be.

D

B

um So, like my understanding is that counters?

B

So this is for the synchronous stuff, and this is how it works for the synchronous stuff like counters, you can add a positive integer value um or I mean okay, like if you specify so, for example, if you're in C sharp and you parameterize the type to be of type like double, then you can add a floating point value but, like you know, you add a positive value, it increases monotonically um and then, if you want it to also decrease- and you want the the cumulative like you want the sum to go down, then you call you have up down counter.

B

um If you try to pass a negative value to a counter, it just throws an error. It's like no there's, no negative values allowed here. um So, like request counters, they like requests, don't go down, they go up, um but other things like that and then and then Gage is intended in theory. Of course, to be the one like it's non-cumulative.

C

B

The value is being recorded at the time, um then there's the observable versions of these or asynchronous versions of these they're called async in the in the spec, but they're not called async in the implementations. They're called observable in the implementations and then the spec also says that these follow neither the async nor the observable patterns which I don't know what that means.

B

But this is kind of confusing, um but then in JavaScript I, don't know how to do it because so like when I use the synchronous stuff, they work the way that I expect them to work. When I do the observable counter and observable up down counter. The only difference appears to be that if you pass in negative values to observable counter, it just reports the value as zero, um and so the way that works.

B

It's like you, register a callback and then you call this function called observe, and then you want to observe the results like you want to observe a result of some computation that you've done and so I guess. The intent is like if you want to extract like an integer value from something that went up or down like. Maybe you had to call into some subsystem somewhere, and you want to do that.

B

Asynchronously, like you, don't want to you, don't want to do that on the main thread for like a request or something like that: okay I get it, um but then you've set up an exporter. It's like this periodic. It's called like periodic reporting at periodic exporting metric reader. I. Think that's it's the name. It's just by default, every 60 seconds, whatever the hell was like reported, it just send it just fires off an event. That's like hey! This is this is what the report is for for the stuff that you've accumulated over time so like.

B

If you have 20 requests and you had a request counter, then the value after 60 seconds would just have 20 on it, um and so that thing that's reported from observing a value. um It only goes to Z. Like you. First of all, you can observe negative values in an observable counter which feels unintuitive me, because there's also observative observable up down counter that one allows you to observe negative values.

B

Neither of them appear to accumulate anything, and so, if I, just swap the instrument to be a gauge instead of one of these counters, it's the exact same behavior and so I, don't like I can create code samples that show how to do this.

B

But it's not gonna like do like I, don't I, don't know what commentary to put there and so I don't know like I should probably talk with the JavaScript folks about it, um they're examples and then in like in code, documentation is also very poor and doesn't like have anything that could like Hint it why this is the behavior, um but it's it's definitely it's hard to document and so I'm like okay.

B

Do we take a cut at like only doing the synchronous stuff right now, because that sort of worked like everything works, the way that you would expect it to work? um That's pretty easy to to like I, already have the docs written. For this. That's easy to get into a core request or like try to have this conversation and get the observable stuff in there as well.

B

um I will say that, like the difficulty Factor ramps up significantly when you are using observable counters, as opposed to the synchronous, um sorry observable instruments as opposed to the synchronous instruments and then also there are a lot of nouns and verbs with metrics, and that makes it hard yeah.

D

A

Okay, I mean to to answer that one question I mean then, let's put in the simple stuff first and then figure out the rest later yeah.

B

Yeah I'm, leaning.

A

Towards that the the other question is can can, can we take some some knowledge from from any of the other languages, I mean, don't we have a little bit of python or a.net around metrics already so.

B

Yes, um I was doing some of that, um it's a little. So if you read in the spec um here, oh there's a choice quote here that makes this challenging. um Okay. Here we go open. Telemetry API authors may decide what is the idiomatic approach to handling these observable things? Here are some examples, return a list or a tuple generator or numerator Etc of measurements or use an observable result argument to allow individual measurements to be recorded or be reported so.

A

Okay, so so we so we so we just provide us back that oh I hate that I think we do this way too often that that we give too much Flex I mean I I, understand where it is coming from, because those things are building consensus right, so yeah Java people are saying like no. It has to be that way and tighten people like this, and then they put a May there, but I mean I mean we wanted to have open Telemetry to not worry about that stuff. So yeah.

B

And and then the the there's also within the spec itself, it's funny there's some code samples and you're, like oh cool, like I'll, try to work off of that and they're in Python and C sharp um the C sharp one uses made up stuff that doesn't actually exist um like they have this thing called anatomic clock, dot, connect um that doesn't that that's not a thing in.net like this is made of class.

D

B

So and then the python one is like returning these like lists of tuples, but apparently python lets you do both where you can have a callback that either returns a tuple or you have. This results accumulator object that you can just sort of observe things in arbitrary amount of times, and so like.

B

It's also unclear like okay, when, when I want to observe a result in a callback like, should that just be one observation or should be multiple apps or should I be able to do multiple objects like clearly I can do as many observations as I want. It's just. This object that I can call observe on as many times, but like is that again, none of this is specified of like what the like, what what you should do with it.

B

A

Yeah but but but but then I would also lean towards, maybe not even having that in the docs right, I mean at the end, our core audience right now are people that are fairly new to all of that, and I mean at the end. They probably don't even care right, I.

B

Mean right, you know it is probably so like counter up down counter and histogram. They are synchronous instruments that, like basically just work, the way that you expect it to work. um The one thing that sucks is gauges are asynchronous observable period.

B

There's no such thing as a synchronous gauge: um okay, I, don't know why, but like that means that there's no way for you to accumulate pain, or rather there's no way for you to to report a non-aggregated value like today, without calling one of these kind of confusotron things that differs from language to language, um so I, don't like I I mean I can just align that for now, but that'll definitely feel like a gap. That's missing, I! Think for some people, because Gage is a pretty common.

A

um Yeah yeah definitely I mean think about something like I. Don't know, consumption saturation, something like that right, I mean you need it there right I mean yeah. You could do it, probably with a funky up down counter.

D

A

A

Yeah, but but at the end I mean if I mean if I would I have no idea about metrics that that's a good, so open, Telemetry, metrics right now, so that's a good part. So if I would be stupid and go there, all I wanted to do is exactly that. Right. I want to say, like oh I, want to count my requests, yeah and then I would need a synchronous counter and that's it and I don't care about a lot more than that.

A

So so this is and then maybe I come back later and say, like hey. I also want to have I, don't know my my arrows going up in like my areas per minute, I mean that's yeah, there's also a counter. What's an up-down counter, I, don't know, let's think about something.

A

um Active users is just something I could do like. If someone locks on locks off that I add add one subtract one or something like that. Yeah.

B

Yeah yeah active users is definitely one and.

A

Then and yeah and then from there I mean of course, at some point. I will get to the point that I said like hey: I need something observable asynchronous whatever yeah, um but but then maybe yeah I think we. We can cover the basics first and then later figure out how to how to do the rest. So.

B

Yeah I I love this. The metrics API May support an interface allowing the use of multiple instruments from a single registered callback.

B

It is recommended that API authors use one of the following: two forms of the Callback function, a list or Tuple Etc, so lots of things um uh returned by the callback or the observable and so python does both JavaScript only does one of those.

B

um This is. This is fun um concurrency requirements. This is the also. uh This is also the fun one. So I don't um okay, so uh because I worked on.net at Microsoft.

B

um uh We, you learn very quickly when you work on on on the stuff and that basic technology that the term thread safe is garbage uselessness. That doesn't mean anything at all, um and we have that proliferated here for all the callbacks they they all say like you know it should be a thread safe callback and it's like, but then examples in the spec are about like getting measurements from a CPU. I'm like this is not. This is no no. No, no like this is gonna lead to pain.

B

For so many people, um yeah I, don't know exactly how to approach I mean so okay. So this that's kind of what I was thinking was: go the just the synchronous instruments. Things first get that documented because, like I think I think you're right, probably the majority use cases are going to be.

D

B

For synchronous stuff in the first place and.

D

Then, like just.

B

Because their synchronous doesn't mean that, like the whole thing like the the Gathering of those metrics and then reporting that all happens, asynchronously anyways and so like the fact that they're called synchronous instruments is, might actually be a confusing point for some people, because getting the metrics out of your system is not synchronous at all. It's just literally the like Gathering the data itself like like the the API that you call to put the data. You may have even ex like gotten that out.

B

Asynchronously um uh because, like you, can imagine the.net API, where it's like an async request. Handler, um they have the await pattern, and so then, like that'll, that'll get deferred and it'll go pick up. Another request the same time, so the thread isn't waiting on that, like once it has the result back just calls it like that that tiny tiny little bit is the thing. That's synchronous. Nothing else is um so that's another sub problem to solve here that I'm trying to knock out but anyways.

B

That's why I've been working on Metric stocks for a hot minute, because.

A

Yeah it it sounds like fun and that's why people like automatic instrumentation right and they don't have to worry about it. They're, like yeah I, just throw in some stuff, and then it falls out for me and I. Don't have to worry about that so anyways.

B

um But yeah that some Ruby stuff and then there's also some like C plus things that I might contribute some one of our Solutions Architects did is like trying to add some C, plus plus stocks to the honeycomb docs and like so C plus, is not an officially supported Language by honeycomb, and so we're like whoa, okay, let's hold off here, but then we're like well. This is useful, though so, like let's I mean we should like add this to the Upstream stuff. So I might do that as well. um This.

A

Week, yeah, that's cool yeah, yeah, yeah I also try to push people sometimes to say like hey: let's do this internally just push it out into that's also how I try to do this serverless stuff, so I have a few a few folks now looking into different serverless things and I said like hey, maybe you can can put it into the hotel docs directly, uh because all the questions you have probably someone else- has them as well as the um Misty Force floss and the the.

A

How are they called layers and whatnot I mean uh yeah and and then maybe we can bring some some people from the the hotel Lambda stick or, however, they are called or project uh to to answer a few of those questions, then so yeah yeah.

A

Awesome anything else, I don't know so.

B

Yeah, that's that's about it. um I mean I would expect we're, probably not going to see a whole lot of activity this week since it's Thanksgiving week.

A

Yeah yeah and but we will meet them anyways in December 5th, or something like that. So so, if I see correctly, there's like two more meetings this year then and then probably the one. uh uh This.

D

A

Yeah exactly and then it's already the New Year. Probably we can skip that. One I don't know if anybody is working the second day of the year so yeah, let's see yeah awesome.

A

Then, let's talk next week, no in two weeks from now right exactly.

D

A

D

Awesome talk to you then yeah.