Open Telemetry Uncategorized, 22 Feb 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2023-02-22 meeting

Description

OpenTelemetry PHP SIG Meeting

A

A

How are you good thank you and you excellent.

B

C

B

A

B

I'm very tired.

A

B

A

Much as it's very late, you're right.

B

Yeah yeah, actually I just got home from the concert like five minutes ago. So.

C

B

Yes uh might just be you and me today, because uh rock city was working late yesterday and probably wouldn't pick it.

A

Yes, I've seen that.

B

B

I haven't started, I haven't tried to debug it yet, but I am I added the I o auto instrumentation library, to one of my applications uh today and I I got a handful of um f opens and F rights just as their own independent traces that were sort of detached from the touch from the um the regular traits. So I'm, not sure where, where they came from whether they've come from um composer sort of the completion of a composer.

A

Yes, I think that I think that they may come from a composer or something else before the main request is is coming.

C

A

I've experiencing something similar for the video calls when I was.

A

Auto-Instrumented Drupal, so there were a few uh traces and some of them uh contained only only um one PDO called for, for instance,.

B

A

And yeah and I think that that works correctly, but in fact I haven't investigated that deeply. So we can look into into that yeah.

B

Radio um all right, it's three past I'll, um just.

B

A

B

B

Can you see me yeah.

A

I said we have to add another uh meeting.

C

Step down all right.

A

B

uh Okay, this is new. um This is just one for.

B

A

Created that today, yes, I tried to test that with uh laravel.

C

A

Application and of course there were, there was a conflict because this example application required a newer version uh at least 10 and higher and I have 952 here and it didn't work. It I couldn't fix it, because there was, as you know, conflict in this version. So.

C

A

I relaxed this uh dependencies um yeah.

B

I actually did notice that when I was uh when I was doing a review, that I think I didn't mention it or it had already been, but I'll probably figure it.

C

Out soon enough.

B

uh That's fine.

B

Yeah that maybe that may end up being too relaxed because it may not work all the way back to laravel. You know 1.0, but okay, so, but we can. We can figure that out another time. That's okay,.

A

Okay, if you will have any other, you know suggestion, or how can we do that better, just uh open pull request or let me know, yeah.

B

Yeah it'll take a little bit of digging through laravel's code just to work out. You know when, when the methods that we want to hook started existing and and I suppose, the signature needs to be the same as well.

B

um So maybe if we can figure out how far back it does go, we can just do a greater than or equal to whatever the position is where available, six or or maybe we look only look back as far as what is an actively supported version of laravel, but for a moment, I'm happy with that. Okay,.

C

B

A

Yes and for auto instrumentation here, I opened a new pull requests with uh this new two new packages that we recently added this Symphony and a lot of it. So I added this to them.

A

um You know I'm using for the install of Auto instrumentation packages, but there is one thing that I would like to discuss regarding this, because we can get this list directly from packages.org just by using their API.

A

But but then we have to be sure, because um we have to find correct, Auto, instrumentation packages, and for that we can use this prefix Auto open, telemarket.

C

A

But then we have to have a guarantee that this will be always true. So so we have to you know.

C

A

Should be a kind of convention, because there is no other way to find that packages yeah.

B

I mean we've been very good at following that convention and nobody, except does can put things in this organization, so yeah I think we can do that.

A

B

Then I can, if we do just want to install absolutely everything.

A

Yeah but the yeah, so so this list is used in advanced mode. It's always asking you if you want to install the specific package, but the the code is taking the packages from this list. So then we can. We can ask you, as I said we can just ask a packages or dot org for this packages and just to get rid of this list. Hard-Coded in in this. You know in this install PHP script.

B

Yes, I, agree, I, think that's a good idea and.

C

I think we can safely.

B

We can be very confident that that nothing else is going to use that open Telemetry Auto, uh maybe convention, okay, okay, because we're in control you and I and Bob. That's.

A

True, that's true.

A

Then we will not need you know, preparing uh or creating such pull requests as I created today, because.

B

It will just work. Yes,.

A

um I think we don't have new bugs um no.

B

No, this one's been around for a week or no.

A

Yes, but we had it last week, I think right or yes,.

B

Yes, we did and I, but they'll allowed to come back to us or they won't I can't help anymore. That I have because.

A

B

Have a power PC? Yes,.

A

It seems very Niche I also don't have uh such a environment.

B

uh Yeah I mean potentially virtualization, but I'm yeah. It's not the most important thing in my life at the moment to look into that.

B

uh Did you have anything you wanted to add to the agenda yeah.

A

So I have these two things that I mentioned on on slack channel, so I think that we can, as I mentioned uh hook into or implement. This area need PHP area, init function in C extension and air shutdown function, and then we can open um a call. The code that will create spawn the root span there and end in the air shutdown function. So.

C

A

Think that I checked that and it will work in my opinion for any and any incoming requests, but it will not work for asynchronous one I I.

C

A

I mean by that that for us in Chronos calls we have to uh follow the current approach that we have. So, as we have this a async library, httpsync library hooks for for this Library. So um we have to follow the same approach here.

A

So this this C extension approach will work, in my opinion, only for normal uh requests- yes synchronous ones, yeah, and then we we're in um we can- and you know we will- uh um we will not need to prepare specific hook, specific Library, Auto library for specific Frameworks, because we can handle that in generic way. But of course we can prepare hooks for specific Frameworks as a as a kind of extension to that, but um whenever we we will not have, uh we will not handle a specific framework.

A

So let's say that there will be some kind of new framing. Currently we are handling a slim, laravel and Symphony, and for laravel and Symphony we are handing only one function. In fact, the the root root.

C

A

Request root spawn and whenever we will have any new library, this code in C extension would uh create uh root span for us always so.

C

A

We will not need to to you know to prepare specific Cooks.

B

Yeah, so that's useful. um How can we know when we uh I'm trying to say how can we know when we're using a asynchronous runtime so that that.

A

B

A

What I figured out that, when um we can get, we can get URI from from this function when, when this function is called, we can get URI, but for asynchronous calls we we do not have this URI, so that's kind. That's a way to distinguish these two calls. For instance, yeah.

B

We could probably do the.

B

um I'm trying to say sorry, um we could probably do the asynchronous runtimes as their own, whatever instrumentations I, think, because I would think they would have a method that we could look I. Think so.

A

Yes, but you mean about preparing specific hooks for for that right, yeah.

B

Yeah so that doesn't really help and unless, unless the sea Extinction itself was aware of the you know the a handful of different run times, but because they're all different, they all are going to be like bespoke code. Just for that, runtime I.

A

Think yeah, but generally, my idea was to support um this Frameworks that we do not hook that we do not have a hooks for so yet because it seems probably that users will be most interesting in the fact that there were just some some requests and they are probably not very much interested in the in the child's pants that we are creating as a result of instrumenting framework I. Think yeah, and this is the main, the the most important thing from the user perspective. Probably is this root root span.

B

Yep yeah, that would that makes a lot of sense, um um so I think provided that we could order heaven configure it on or off by a say, environment variable or something like that. Then.

B

B

Yeah, that should be location.

C

B

So, yes, a song is, as.

A

B

I can either turn it off if I don't want it or turn it on. If I do want it. Yes, um then that seems that seems fine.

A

Yes, and also, if we don't in, if users don't want to, uh doesn't want to have a specific framework instrumentation, he doesn't need to install specific Auto, instrumentation library right. So then, and then he will have only this, for instance, this root span that will be created from the call from the C extension.

B

Yes, uh you would also need, when you're, creating that brood span to consider Trace propagation.

A

Yes, that's true so so generally I was thinking about preparing some kind of helper function in the PHP um and called that from uh from the sea, but also, as you said, I need also this Trace propagation. So I have to pass some information to this function.

C

A

You know that just idea, I I.

C

A

Try to implement that and we can. We will see if that will work for all the or all cases that we have currently.

B

Yeah yeah I think it's a good idea. Let me just try and write down what I think we'll just talk about here.

B

um That's that does that cover enough about what we just said. Yes,.

A

C

B

A

Another thing to talk about yes and then another topic that I had in my mind, was uh that I was thinking about getting great um relying on these external objects and to store, for instance, in request span. That's something that we were discussing some time ago and yes,.

B

I remember yeah.

A

So I think that we can create. We can use the same similar approach as um for this uh creative roots, Panic extension, so we can in this air init function. We can. Whenever a new request will come, we can generate a new request, ID uh and store it into a php's uses. These Global objects as a requests.

C

A

Request and dollar post, and something like that, so we can store this request ID there inside this, this arrays and then whenever we will call call a function from a PHP.

A

We have to also have additional function added into extension. That will give us this request, ID. That will give us this request ID. So we can call this function from the C extension in the PHP code, and this call will return us. This request, ID and this request ID- can be used to as a key for the spans that we will store in in the context in.

C

A

Kind of map, so that's generally my my idea and I think it will work.

B

Okay, um I'm not sure that I correctly captured that um restoring a root span or a restoring yeah.

A

So so I will start.

B

From unique identifier for her.

A

Yes, so um the thing is that inside this PHP area function, which is uh which is called whenever a new request is coming. This is a c function from the C extension in this function.

A

We should create a new request: ID, we will maintain um a list or maybe count some kind of counter, because counter is enough in my opinion, and whenever new request will come, we can just increase the Discounter and then the the value of the counter will be our current request, ID and then we we have to expose one additional function inside uh in this C extension, which will be about returning uh this request.

A

Id and then this function can be used from uh from a PHP code, normally as you as we uh calling any other function that are, that is from extension, and then we can use this function just to get this request. Id and- and we can store then um any spawns that we will create in the PHP code. We can store this spans in a map in the context and map as a key, we'll use request. Id.

A

B

A

That's the whole idea: I can describe this in in more I. Don't know General way, maybe uh in some kind of document, for instance, if needed, and but also I, can try to implement that and check. If that will work.

B

Yeah yeah, my main concern again, is with um I. I can say that this I suspect that this would work nicely, but um you know more PHP and fcgi, but when we're using asynchronous runtime so particularly once we start using um fibers and have like multiple requests going on. Yes, you know within it within a within a loop within the one, within the one process that this might get very hard to manage. Yeah.

A

That's true, that's true: it will not work in uh foreign.

B

Or nginx they can just turn it on or or it's on by default, and they will get they'll get these. These two features for free without having to do anything.

B

um But if we were storing, if we now have two different ways to store the root span, I.E.

B

One one mechanism to do it within asynchronous runtime and a different method. If it's a synchronous runtime does that.

B

Yeah does that work with things that need to access the root span.

A

I, don't know um yeah, but maybe that's not a good idea if we will not resolve the problem of passing Chronos, because because the American is that I would like to introduce eventually uh should be should be General. Otherwise we will have a problems. I think yeah.

B

I still think and I I'm not sure that it can be done, but the the simplest way would be.

B

That tried to get a root span would just Traverse up the current context until it finds us like the earliest span, video um and- and that is the root span, because we we know that context can sort of fork, and so, if you're running it in an asynchronous um uh environment, then you can have multiple contexts going on in each one's relevant only to the the sort of request that's being processed, um and so, if you can so if get root span was simply able to Traverse up the the context and find that topmost span, then that is the root span.

A

B

But that's good in theory, but every that I've not been able to do it yet and um and Tobias seem to think that it was either a bad idea or respected about like another level. Why? But, um yes, coming from other APM solutions that I've been used to wear, you know a root span is or transactions. Sometimes that's all it is is, is an important component that you can. Just access directly did make I did make a few things easier.

C

B

So I, don't I, don't know if there's an answer in there or just more questions at the moment. Yes,.

A

Right but we can continue our investigations, so maybe we will find some better approach to to this problem.

C

B

Instead of problem, just to spell it out, is that that we would like a consistent way to be able to get the root span and easily store it, because sometimes we want to be able to change the root span after it's been created and.

C

B

Just for updating the name, because that's the only yes as far as I recall, the only thing that you actually the API allows you to change about the span after it has been started, because it's is its name.

A

Yes, I was thinking if, if that's something that we really needed, maybe we could, because uh this was introduced due to the fact that we wanted to reduce this cardinality of URI right yes and I was thinking if we can do that. Maybe in other way, maybe not. uh Maybe in the call that, where the root where the span is created.

A

B

Don't know yeah that really it really depends on the framework, usually not.

B

Yeah, so if we just if we just use something very, very generic like HTTP, get HTTP post, then that's that's good cardinality, but maybe too good, um and it can be nice as a as you use it to be able to see something more meaningful like what was this request doing, uh which is where a route?

B

Yes, it can be very, very helpful for things that use reality, but but we know that at least for most of the PHP Frameworks that we've looked at were asking happens. Foreign.

A

Yes, so in this case, we we have to have a way to find this road span.

B

Yes, we do, we do um yeah I.

C

B

A PSR server request is a like a good standard and it does allow for attributes. So it should be pretty good cross compatibility but Frameworks that do support PSR, but but they don't all such as WordPress.

B

C

B

B

Yeah all right: well, let's keep thinking about it and and um do some more do some more research.

A

B

um Yes, but I mean I I do see that that what you're proposing would probably work well for synchrons.

A

B

You know for the for the realistically probably the two most popular runtimes, that there are, which is not PHP and fast CGI, where you know, one request is one process and it does have an initialize and a shutdown, and you know, doesn't share anything um so I think that would work um but I'm not sure that it simplifies the problem for us where we also want to support asynchronous runtimes and even more so in the future, because um you know PHP is evolving and with fibers and things you know, I I suspect that PHP is gonna start looking more like node is now you know with a central event, Loop and uh you know doing lots and lots of things at once.

B

Instead of one thing in one request that, then you know self-destructs.

A

But I I haven't investigated that very deeply, but uh um is a PHP uh have some kind of uh support for this asynchronous calls in in the engine or how? How that's? Because node.js, you know that the engine is is built in this way, that everything is there that is needed for handling disaster. Synchronous calls, but.

C

A

In the engine, but I don't know how it's in the PHP.

B

So I think the with fibers comes that capability, so they're still user land code. That now required to you know create that um that event, Loop and dual game manage all of the asynchronous code, but all of the I think all of the constructs are. There now has a probably PHP 8.1 I think.

B

Okay, okay, yeah.

A

B

Yes, um it's it's there, but five is a quite a low level thing. So I, don't think that anybody expects the average user to be able to, or indeed to understand how Farmers work. um But there are a couple of good event loops that already exists: um I'm thinking, amp and react and I. Think Revolt is the the newest standard for event loops, which is those two, a collaboration between react and amp um yeah. So look I. Think it I think that it's that it's coming.

B

um And- and we certainly wouldn't want to do anything that would stop those from being.

C

Able to work with open Telemetry in the future.

A

Okay, I'm just looking into fibers, it's uh as I see it. It was introduced in PHP, 8.1.

C

A

And so it's that's, uh probably very internal mechanism that was built into the PHP engine.

C

A

B

Yeah, and so that's yeah I mean by all means, read some more about it, but my understanding is that fibers are the mechanism by which, in the future, PHP can be a lot more concurrent and do you know co-processing and have multiple things happening and that at the same time, in the way that that node does.

A

Yeah, okay, okay, I, understand.

A

B

Do you have anything else that you want to talk about today? No.

A

That's all uh on last meeting I said that I will presented this tool for um for generating these stacked races and then generate hooks for us. Yes from that, but I'm not ready yet because I was um focused on on different things this week, but maybe next week I will uh finish that tool and then maybe we can discuss that.

B

Oh yeah, no rush, no rush at all.

B

uh Excellent all right, um just looking back um uh I, did get on comment on this all request about some of the the velocity or levels of minority traces, but that hasn't really not much has happened since you and I commented on it last week.

B

uh I don't still kind of thinking about this I think I mentioned last week that.

A

C

B

A really good implementation.

B

Which Tobias has put together in his own area um yeah, so I'm actually kind of busy at work at the moment, but I'm I'm, trying to think about whether and how we can integrate that into our main batch processor, um rather than have to seem a little bit different batch processes: okay, okay, yes, so anyway, I still, don't know what to do with that, um and it's probably not a problem that I need to solve right now.

B

um I, don't think it's a thing that we require for a ga release.

A

A

So this is something that the bias was working before.

B

uh Yeah yeah yeah, it does work quite well. I I was able to test it out um um with an asynchronous runtime and it it does work it uses.

B

um It uses an event Loop that uses fibers, so it only works with nhp 8.1 Plus, um but but it really can do sort of background sending on a timer of of traces, which is like a really great thing: um yeah. It's just I think it has a few more dependencies than we'd like, and it also doesn't work with all of the PHP versions that we want to support the open telemetry, but but maybe in the future.

B

You know in two years when, when we're no longer supporting PHP 8.0, um maybe then it can become the sort of the primary or Main.

B

You know why it working mm-hmm, yes, but I'm, secretly, hoping that to buy a swing to shop and fix that because he's very, very smart and um yeah. That's a very good solution.

A

B

uh All right, well, I, don't have anything else tonight or today do you have anything else to talk about.

A

I think that's all I can only show you um that spans from Drupal that I had that I have.

B

Yeah, do you want to screen share to do.

A

That yeah I was getting yourself for a moment, so sure.

A

um Do you see it I do yeah. So, as I said, there are several spawns, there are several traces that contain only one spot and the as you can see. There are PDO operations here, oh and also, for instance, HTTP post. When we have this root span, so then there is, uh we have a red spawn and under this root span we have a um about 100 different.

C

A

That's how currently and Trace from Drupal looks like from this uh Drupal example app, so yep. That's, probably, is something that you also was using.

C

A

And now so now, the question would be why we have this um this traces uh with PDR that are not connected to the root span, but I think that this might be something expected due to the fact that probably some operations, I I, think that some operations are. You know invoked before this Main requests.

B

I think it's run. Some queries before you've started the The Roots band Yes.

C

B

That yeah, that's what I think, but it's my guess.

A

My too, but probably we have to check that.

C

A

Why why we have these these traces so.

B

Yes, so here I, so I I think I had a similar problem in WordPress and I solved it in a slightly unusual way, uh but dribble uses composer and yes.

A

B

Yes, that should be okay,.

A

um Yeah, this requires further investigation, probably yeah yeah, just to be sure what is the root cause of that.

B

Yes, um and so, is that using so Drupal is based on Symphony? Isn't it yes.

A

B

So as this is in the symphony, Auto instrumentation, or is this something specific to Drupal.

A

No, this is uh the the symphony instrumentation.

A

That we have this in this Auto instrumentation, so for another package.

B

Okay, so I do remember that, because I I was also playing with Drupal, so I was probably two months ago now, but um but it used.

A

Yeah you mentioned that you I it used the one.

B

Symphony interface, yes,.

A

B

Run like multiple times.

A

Yes, yes, so with this interface, the trace looks a little bit different, because we have uh more this uh post request.

C

A

So the you know the tree is a little bit different. We don't have this PDO operation under one call, but but there are also additional requests here so so.

B

That that looks like it's very like subrequests.

A

Yeah sub-requests, so that looks very similar uh what we have currently in Slim. Yes,.

C

B

Yes, so you only yeah, you need to do something to set up those sub requests at least create a span or something.

A

And and for now, I I just decided uh to hook into this concrete implementation, which is HTTP kernel, handle a method and not this interface.

B

Yep I think that I found the same thing.

A

But generally you know we can always change that if we will decide that this is maybe better solution so that that's open, open question. In fact,.

C

A

Okay, so I will stop sharing okay.

B

um Anyway, that's looking good you've been very, very busy this week, that's been exciting to see so many Auto instrumentations um available.

A

B

A

Have uh for now at least, we have initial versions for this laravel and Symphony, so we can extend that further.

A

Yes and I believe I believe that this uh Symphony and laravel I think that they are the most important from a user perspective. I I got also some questions from uh users on this slack Channel about laravel, for instance, right.

C

B

uh Yeah I mean they probably are the two biggest most popular Frameworks these days. Aren't they yeah.

B

C

B

Hopefully, their popularity means that someone will come along and make some improvements, yes to the instrumentations.

A

Yeah, so you know my my plan would be to next may be to write some kind of a blog post about Auto instrumentation, just maybe to to get more people into to to introduce more people into uh into our work.

B

uh Yes, that would be correct. um Yes, and maybe we need to reach out to a couple of the developed communities such as WordPress and Symphony and laravel just to say: hey we built this. What do you think? Does anyone want to test it out or requests, welcome and and just encourage um some collaboration.

A

Yeah, that would that would be great.

B

That would be great because we can only do we can only do so much and we're not um experts in in most of those Frameworks by any means. Yes,.

A

Yes- and you know the funny things that yesterday I I built this hook for a lot of her, it took me uh I, don't know maybe 20 minutes and it worked. But.

C

A

Had I I spent uh the whole day just trying to write a test at least one test. Yes,.

B

Oh dear yes, yes, I will appreciate the uh the pain there and there's certainly some of them such as WordPress, where I didn't even try to write a test. It was just. It was too hard.

A

Yeah, because for writing this, you at least need to know uh some internals of how framework works works, and that is very hard.

C

B

Yes, it is, it is um I was able to do some work for sleep because I understand it quite well, because that's the that's the framework I work yes every day, um but yes much much harder. It's it. Symphony and laravel. I think that.

A

It's hard to learn these Frameworks. In one day, you at least spend more time.

B

A

B

B

All right, well, I, think we might be done for today to go to bed yeah.

A

Thank you. It's. It was quite long for meeting.

B

It was, but it was productive, I think it was a good talk. Yeah.

A

B

Without with it more peer to um make us go faster, we just chat for a very long time, uh but it was good. Thank you, I appreciate it uh appreciate all of the discussion that we've had today. Yes, all right, we're done I will see you online during the week. Thank you all right. Take care bye-bye bye.