Kubernetes SIG Instrumentation, 8 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Instrumentation 20221208

Description

SIG Instrumentation Bi-Weekly Dec 8th 2022

A

Hello, everybody and welcome to today's edition of Stig instrumentation. It is Thursday December, 8th, 2022, probably our last recorded meeting in 2022.. What do we have on our agenda today?.

B

uh I don't see in a updated agenda doc.

A

Okay, I see one I.

B

Oh next agenda I see it I, never mind, I, see yeah yeah, so, okay, yeah first, our Deep dive video for kubecon, is out. uh The link is in our agenda doc. uh We have a couple things on our agenda.

B

The first item, I, guess, is for me um maintaining sub projects, so we have a plethora of sub-projects and we have not enough contributors to contribute to them, um and it is the onus is on us, since these are our sigs sub projects to either find contributors for them or think about releasing sub-projects from the Sig fold, because we can no longer maintain them and we should only maintain the ones that we actually have the bandwidth to maintain um and I.

B

Think we've been doing a better job lately like this is a talk that we had at qcon and a little bit after kubecon before kubecon. We've been talking about this for a little while, um like the two primary projects, which are well the primary project which is in danger at the moment, it looks like Cube State measure is starting to get in better shape. There looks like there's more maintainers now um is metric server.

B

I am quite concerned about metric server um and in fact the next agenda item is about the release plan for your metric server. But.

C

B

What's the state of.

C

Each sub project, if you're interested since I mean building most of them, so Cube statement trick, is like the most active one out of all of them. um We have like three active approvals uh and now three active reviewers, yeah uh and yeah.

C

A ton of uh first-time contributors like contributors and Keepsake Matrix tend to come at their future and then uh like leave, that's how it's happened usually, but like we have a lot of contribution, uh as I mentioned, like we add um a little bit of lack of uh approvals or reviewers like people actually looking at the PRS and the efforts um under metric server side, we have uh two active reviewers of like one attribute viewer, the other one is me, and then there is uh there.

D

C

Marek, that is approver and Marik is the only approver at the moment so kind of in a bad State, because, like neither Mike nor myself have really time to invest in electric civil League, we are making the minimum. uh That needs to be done, but that's about it and the two other one are like custom metric, CPA server and committee adapter I am the only over and.

B

I think we should we might want to disband them. We might want to disband them.

C

uh No, no, not just we can't. uh This has been customer CPA server. uh There is no way because, like that's the library that implements The, Matrix, API and so like since the Matrix API are part of our scope. We cannot disband it um for primitive, selector yeah. uh We might want to uh archive the project at something but like systematic CPA, so we need to work on it.

B

Yeah yeah I was actually telling you that has been a Prometheus adapter.

C

Yeah, and so like one of the idea, also is to like move uh the API implementation that is in metric server, currently to custom, metrics, API server and just rename the project to something more generic, um but that's the current state of it. And now there is Olivia that helps me with that. But before there.

D

Was no one but.

C

We still need like at least one more contributor on that side and I guess uh at least one on matric server side. Also, and that's the scope I know of maybe like klog I'm.

A

Trying to take notes real quick.

B

A

Damian, do you know how many people we have actively working on custom, metrics, API, just Damien, okay,.

C

A

Well, no, no there's.

B

C

Me like there is only Olivia at the moment uh and I'm, the only approver for sure, uh but I don't have time to do anything on it, uh like I'm, barely able to keep up with their POS that Olivia is sending. But now we are two like since last week uh same with Prometheus adapter, um like I'm, trying to put it in a good State just before typing the project. That's my ID because, like we, we won't have the bandwidth and like, as I, discussed with someone and Davey that you can like.

C

There is a good replacement for Primitives adapter today. That is part of cncf, which is scada, um and it has a like way, bigger scope than just what promises adapter is doing so I in every way like it seems to be a better replacement for it. So I would rather up.

A

The scene, can you give me a link to that? Well, what do you do? Yeah.

C

um And I drizzle like at this great push for the people who use this project rather than permission because, like it's.

A

30 years CF project I think it'll be fine. I think we get pushback of archiving it I mean it's been.

B

A

Hold that in but.

B

In the chat thank.

A

You that's awesome. um We had like some pushback from I think contribex about we basically like re-homed, Prometheus metrics adapter after it had been archived once and then we brought it back and they were like you already archived it like. Don't so I. If we archive it again, I think that's gonna be our last shot. We are not bringing it back into the fold so.

C

I mean yeah, like the the problem that I always permit is that they like full right here. Maybe it has only been 19 by Reddit in general um and, like we've done a very bad job at like bringing new people in and like even like the project needs a whole rewrite like it's impossible to embody. New contributors like it's super hard to comprehend. Like the code base, um the ux like even using the project, is difficult um like answering.

C

The issue is nearly impossible because, like debugging it with how complex the config is, is like super hard and super tedious um so like we are losing a lot of time by actually like keeping it around.

A

C

A

Yeah I think that's all reasonable.

A

This is actually really good timing, because uh all of the sigs need to report in with annual reports uh like, basically, you know, I think that process kicks off in January. So um having all this information about subproject health is something that we're gonna have to gather anyways uh so like knowing this going in.

A

uh Essentially, we can just start filling that stuff out and consider like what action do we want to take based on those reports, I think as long as we have like Alternatives and like maybe, rather than focusing on getting Prometheus metrics adapter into a good State honestly, it might be like a better plan to focus on, like here's, a migration guy to cada or whatever, and then that way like we're, not wasting our time on like improving a thing we don't want, people use another.

B

Thing we should do is we should uh right now? What we're doing is an inventory of the sub projects. We should have a subsequent meeting in the new year, uh ask for people to come to our Sig meeting and then ask for volunteers to become maintainers of sub-projects. We can add them as owners, and they can just like you know, start owning these sub-projects, because that's what we need and we're not going to be able to survive unless we onboard some people to actually own these sub-projects.

C

Yeah, or even like we can like that, goes in line with like.

D

C

In general, that we were trying to put in place um like being given the responsibility of being a maintainer straight away is like fully huge for some for some.

B

People I mean some of these. Some of these repos are a lot smaller than others like.

A

Some of the maintainers are experienced already they're, like.

C

Yeah for sure, but the idea is to bring like new people, yes um and like I, wouldn't want to push the burden of like maintaining a whole, her victim to someone yeah.

B

I mean we can see, what's appropriate like like I'm, not suggesting they all become a previous immediately or something, but.

D

I'm suggesting they get added.

B

To honors files and uh become an owner.

C

Yeah, definitely we need to to rethink uh all like at least fill the racks that are empty as much as possible or, like I at least know, what's the minimum that the project should have to be healthy uh because I think like most of them, you have at least two approvals uh and yeah. Three reviewers sounds like a bare minimum.

A

Yeah I think it's totally fine I mean honestly, like a Reviewer is more of a burden than a privilege. Right like it, doesn't give you any.

D

C

A

Over like being an org member, because you could already yell GTM, it's just basically like getting flagged to do work so like I, have no objections to adding people like anybody who says they want to be a reviewer to a sub project. That's the only way, they're gonna get the training in order to like become an approver, so there shouldn't be much of a bar like in the way of that.

A

If people aren't org members yet, then we can talk about how we help them become org members but like I, have no concerns about adding people's reviewers like when they ask.

C

Yeah and that's what I was discussing like before this meeting with someone from my team internally, um like giving being given the rights to become a reviewer, gives you kind of a sentiment of ownership on the project where you have like things to do, and you feel part of the project whereas like if you don't have like this role.

C

um Well, you don't feel part of the project at all like you, you may contribute every now and then, but you won't look at the project really often whereas, like. If you have this role, it now enables you to you're.

B

C

Is not into the project anymore I mean you you're technically.

B

You're in an honors file you're an owner of the project.

C

Yeah definitely so so you feel like the need to actually look at it.

B

uh We should bundle in the next point with this one.

A

B

Yeah I was just about.

A

To time check because we've got 12 minutes.

B

um We should bundle in the next one with this one. uh Do you know when we're cutting the next release of metric server Damien?

B

C

You did it okay, so this is done um so yeah yeah, so essentially like uh I saw with Marek married, didn't have time so I uh got the permission to cut the release because.

B

C

Going forward uh I need to find someone else to do it because, like last week, I think I was responsible for two releases. Well, no I I did three.

B

You could put me in the honors file. Show me how to do it and I'll do.

C

It all right, it's it's super annoying is.

B

It super nice, like.

C

Because, like oh at least this time it was annoying, uh it's like normally, it's super easy but like since it has. It had been like six or eight months uh between, like the two last releases, there were so many CVS and, like you, can only upgrade the dependencies uh like to a maximum like you cannot upgrade between vinyls without uh while keeping it stable.

C

um So it was pretty difficult.

B

So we we need to kind of release. More frequently is what you're saying.

C

Yeah um I, at least for like this one, was overdue for four months because, like there was no one to really cut it, like yes, Marek didn't have time, so uh there was no one besides him to do it. uh Essentially, that's I.

B

Mean I'm happy I'm happy! If you show me how to cut a release, I'm happy to do it. It.

C

Shouldn't be that often okay yeah every.

B

A

Month or two, it's super simple should be fine, maybe.

B

Catherine Catherine: do you wanna, do you want to be an owner for metric server.

D

Let me release manager for Metro server.

A

Yeah, why not? Okay.

D

Do we have a documentation and follow so.

B

You you can no Damian Damian has volunteered to show us yeah, okay,.

E

C

E

Have two volunteers.

B

See it's already working on.

C

Yeah Matrix seven doesn't have a release guidance. That's frequent, like we don't have to release it. That often.

B

But like we should release it every day. We should release it every Go version. We should release it with every Go version that that is what we should do. That.

C

Should be our yeah I mean like we do a release Whenever. There is a new CV because, like otherwise like there is no need to.

B

Yeah but Go versions go out of support, so we need to we need to. We need.

C

B

We need to release with every new Go version. That's what we need to do. That should be our release, philosophy.

C

Yeah you go version new release, yeah that wouldn't be a problem in general.

A

Okay, we got a plan, anything else on Metric server.

E

I try to contribute to to metric server. In addition to uh the two, uh um especially because we're migrating the the resource metrics from API from Matrix server to a.

A

Custom native API.

E

Server excellent.

C

So I'll try to help oh great yeah, but but there is not much to change in metric surrounding or like the code like, we made a lot of changes a year ago, but now it's pretty stable up to date with, like the latest thing that we can do in the ecosystem and it's pretty stable so like there is not much to release. We might have a couple of bug fix that are awaiting, but I'm, not even sure um so yeah. There is not that much to contribute.

C

It's mostly like tedious work of like uh updating dependencies uh answering like false, positive, CVS and stuff, like that in general.

B

B

Okay, we have eight minutes left, let's get to Elena's last yeah.

A

Okay, so um I just wanted to give a quick update on this, because we talked about creating this new sub project in October and then uh essentially, cubecon happened. I I had submitted this repo request, uh but nobody from Contreras got back to me. So finally, I like pinged Bob Bob, was like oh I totally dropped the ball and that's a very sorry. It's like. Don't worry, it's not a big rush, so we have a repo now I put in a description for the repo right now.

A

What I am working on is trying to get approval to contribute the code uh so we're getting everything cleaned up internally. It got a little bit pushed on our priority list, um so I'm hoping that we can get it out before the end end of the year. uh But if we can't do that, then um like early next year for sure just because we're kind of running out of year before everybody starts disappearing on, uh you know holiday Vacations. So um uh what do we have going on there so essentially like?

A

Hopefully, we will have a bunch of code that we will be pushing out soon. uh Then I need to help like get prow set up to do, builds and whatnot. So I am volunteering for that.

A

I know that the folks at uh I think both Shopify and Google, uh so I'm, coordinating some of the folks on the Apple side uh are like very interested in seeing this code, I've been contributing to it internally uh in sort of anticipation for it to go public, so um I'm very excited I think it will bring in a whole new set of like sub-project contributors who hopefully then might be able to also contribute to some of our other sub projects or code.

A

um It's like a very different sort of Niche than a lot of our existing projects, so so um super excited to see that it's like yet another thing that I think will help support like some of the scraping sea advisor, doesn't necessarily scale sort of utilization, metrics issue that like metric server kind of addresses, but not in this- it's not as High Fidelity as these metrics. So um hopefully people will be excited once that code gets pushed out I'm. Sorry, it's not done yet.

A

uh You know things things always move more slowly than you want them to, but I promised uh people at Shopify and uh that we had at least one person interested from Google who's, not normally a regular contributor to the Sig asking what's going on with this, so that's! What's going on with it um that's my update. I should write a note in the agenda saying waiting for her approval to push code externally, hopefully by end of year at latest January, it's.

C

Kind of funny because, like I, started working again on, like optimizing, the metrics.

A

C

Where, on which? We had an issue? Because, like there.

D

Is yet another.

A

C

That has issue with.

A

That yeah, but.

C

You know it's concentric and like they're, starting to pile up, there is like one every year or so now um so I started again to look into optimizing the client. That's such a lot of yeah.

A

Yeah well, I can say that, like certainly uh the folks from my team are interested in maintaining this long long term.

C

A

C

Given even though you're running it I I, don't see advisor uh implementation being changed uh at least yeah like even even if I create the library that optimized the metrics like no one will have the confidence to like merge into exchange and see advisor. So maybe at some point when we switch to CLI starts.

B

C

That's when we will be able to use directly the metrics from cubelet but I, don't know.

A

Yeah uh so one thing that I added here um I said like maybe once this is public, we can do a sort of like walk the code update or demo uh so like to give folks an idea of sort of what's going on what are the various components that kind of thing um so yeah? uh Sorry, we don't have it yet, but soon and I think that's my update. We got four minutes. What else would we like to cover.

C

Please do plan to present it at the next coupon- maybe oh, it won't be ready by by then.

B

A

So I don't know, I won't be there. I.

C

Mean because it's gonna be in.

A

Europe right.

C

A

Yeah I, don't think that anybody has I mean the thing. Is you can't uh you can't submit a kubecon proposal for something that is not yet public, um so I think the code will be public before the end of the cfp. I don't know if we have any plans to do that, uh but I can ask my team, uh because certainly it would be great to pull in like I know. Katrina from Shopify, uh like uh is interested in this code, uh was previously at Apple. Had previously worked on it.

A

So um yeah uh like there's, there's some exciting stuff. There kubecon has just gotten so uh like selective in terms of proposals and like it could be like kind of inaccessible uh to folks, so like I want to just ensure that like, if we can't, you know, get the the visibility of kubecon that we're at least doing something within the Sig.

E

A

And we can totally like you know, consider that.

F

Like certainly.

C

A

Yeah, certainly it should go into our state updates, I. Think um but uh I don't know if we'll be able to do a full-fledged presentation or like a talk. um Paige you've got a question. Yeah.

D

Not sure if it weren't uh a meeting note, but one I was curious. If this sig was involved in um getting Hotel tracing for the system, components that was a um something I got really excited about and made me want to uh come kind of contribute. So I was curious if I'm in the right Sig for this.

F

D

F

Sure I'd love to talk about it, um so I'm, David I also actually work in the open, Telemetry community so um and I've been driving uh work on kubernetes system component tracing so.

D

Right now, oh my God. Thank you David. Thank you. Happy.

F

To happy to have you here and happy to have contributions right now, the main efforts are in the cubelet and in the API server uh I would expect at least the API server to go beta next release. There isn't a ton to be done there um in terms of actual work, but they're.

F

It's always help someone to try it out and to provide feedback for cubelet tracing, there's, probably quite a bit of instrumentation work. To do and um I can point you in the direction I think there was actually someone who wrote a blog post on cubelet tracing recently.

D

F

um I forget their last name, um but uh that person is probably going to write an update to a cup which is a kubernetes enhancement proposal for what they want to do in the cubelet. And um that will.

A

Likely uh was that Sasha gruner, yes,.

F

I I wasn't sure how to pronounce it.

A

uh Okay, yeah no I just wanted to chat Sasha.

F

A

On my old team, at red, hat and I know, Sally uh I think was working on like spearheading the cubelet tracing.

F

So that there's a bunch of people active there and there's probably going to be a bunch of changes that come with the next release, I'm, not sure if it'll move to Beta or not, but it very well may it seems like there are quite a few people.

A

Oh so it's an alpha.

F

It's currently an alpha and there's the cubelet it turns out, has a lot of interfaces and plugins and stuff that all need to be instrumented. So there's yeah a wide variety of things to do. There.

D

Perfect: okay, we're at time and I've got to go, but I will dig into that blog post. I also am a big hotel fan, so um lots of good overlap and open source collab Happening Here, I love. It feel.

F

Free to Ping On slack.

A

Thanks everyone for attending we're at time, see you in the new year, if not at our next triage, bye-bye see.

E