Kubernetes SIG Instrumentation, 17 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Instrumentation 20200917

Description

Sept 17th 2020 recording of SIG Instrumentation community meeting.

A

Welcome everyone to today's edition of the sig instrumentation regular bi-weekly meeting. It is thursday september 17th uh and let me just get a copy of the agenda which I will paste in the chat. If I can find it thanks everybody for joining us uh for those who are unaware or for those who will be watching the recording. We now have a new passcode as part of the zoom kubernetes policies, uh so everybody is using a passcode of five sevens.

A

That is seven seven, seven, seven seven, uh our invites have all been updated and links and whatnot. But uh if you have trouble, that is probably why uh first item on the agenda today, uh I had something on here about talking about caps targeted for 120..

A

um I don't know if we want to talk about other things before that, because that might be kind of long, and I don't want that to take up the whole meeting but uh we're in a new release cycle. We need to know what caps we're planning on doing or con for this release cycle. So.

A

Other folks feel free to jump in I'm not the only one with a voice here.

B

I I'm not sure we necessarily need to um discuss all of them here. Maybe um I I think I saw david. You were planning on working on some things. Could you make sure um to re-ping people? If you need another review on anything, I I wasn't sure what state those were in um yeah. Maybe.

C

They were completely.

B

Done already, I just saw a couple of comments.

D

I need to reach out to daniel again and get him to look at the tracing cup, um I'm actually hoping to put together a little mini demo for them. um So I've been working on that a little bit, yeah I'll, poke them and make sure that moves forward.

B

A

B

Those are the only ones that I've seen so far. uh We've.

A

Got like seven caps on the backlog, uh I put a link in the uh the agenda.

A

I would also throw it in the chat here um and, like I think, basically, all of them other than maybe tracing uh I have like looked at recently, so uh I mean some of them are definitely like not ready to go ahead um like there was at least one that was just not complete the last time I reviewed it, uh but I'm curious like because you know the uh the release team is gonna start chasing us down in terms of what we want to include in the upcoming release, uh especially given that this release is supposed to be like focused on stability uh more so than usual, because there were a lot of test problems with the last release.

A

um Like I there's like seven things here, I don't know basically the status of any of them so.

B

I would say: let's go ahead with the other agenda items and then um take whatever time we have left to iterate over these.

A

Okay, great, let's do that. um I had a thing on here to follow up on the uh end-to-end test, flakiness and the state of the logging add-ons. uh Some folks have been assigned things there. Do you have any report back on that.

D

Yes, so I fixed the flake um in the metrics grabber test. It turns out. It was an interesting race condition where we were looking for metrics from the cube controller manager and cube scheduler.

D

But the problem was that the cubelet was the master. Cubelet was being slow in creating the mirror pods. So even though the cluster was fully functional and up and running and those containers were running, we weren't able to find their pod objects because the mirror pod hadn't been created. Yet so I just added in a retry, the europod should be registered fairly quickly.

D

Also, it seemed to be because our test was being run at the very beginning of the test suite. So we were the first ones to try and um test our functionality, and so we hit that race condition, but it hasn't flaked in the past five days since the fix was merged. So I closed the issue.

A

That's great uh and then I think there was at least one assigned to merrick. Do you have any updates on that one uh like stackdriver stuff, I think.

E

um I look at that. One for stat driver, um so it looks like the test is complaining about. Some of the uh pause is missing. Someone help comments on the parts maybe like rename from fluent gcp to phone dash gke. uh I haven't been able to bring up a text cluster and see see it. Maybe it should be just a rename, but I need to figure out how to run the test myself.

A

Okay makes sense uh great, and then I think there was one more and I'm worried that we assigned it to han and han is like not in right now, because he's out on paternity leave. Let me double check that one yeah.

C

That one's assigned.

A

To han so we have one uh similar test failing for elasticsearch as opposed to stackdriver. Does anybody want to pick that one up.

B

Did you confirm that this was assigned to han because I feel like? Maybe it was something else elasticsearch related, but we did assign something to uh yeah.

A

I pulled up the issue. uh The issue says it's assigned to han, so the test flake.

B

Then I was thinking of something else.

A

Don't volunteer once.

B

I I unfortunately have no clue about those tests at all, so if anybody has more than zero clue, they'd be better.

A

They also have zero clue. I I assume that anybody who's gonna be looking into this will go from zero clue to like maybe half of a clue.

B

I mean you can assign it to me for now. um If I don't find enough time for it, then.

A

Okay, then, you can.

B

A

To someone else sounds good.

A

A

Sorry, it is hard to try to run the agenda and also take notes at the same time if people want to go ahead and like throw notes in uh uh the the one that I just pasted in there frederick, that's the one I'm gonna be assigned to you and then uh uh you, if you wanna res or if you wanna, put notes on the one that you've worked on, uh that would be great uh lily. uh Next agenda item is uh what's going on with kate's prometheus adapter uh frederick.

A

I reminded him of this and we totally filed an issue to migrate it. So I don't know where that's at yet, but uh it's happening.

C

Yeah um serge is now working on it, uh sergio's he's working on it to migrate and to finish the migration. um So far, it's only him and the owner's file as the project was inactive. So he added me temporarily, as I have the most knowledge, at least in our team. But if anyone else is interested, it would be great to get more contributions on it, um but we're definitely looking into investing more uh time into it, but yeah.

B

I'm happy to be there for so again.

C

Sorry, it's a lot of kubernetes bureaucracy right now, so that's the where the stage is at yeah go ahead.

B

I was just going to say if you need reviews or something I'm.

C

I'll call you up, I'm.

B

Happy to be part of the owners file, if that makes reviews easier. I know.

A

B

Pretty well so I'm happy to do that.

C

Yeah, the pr is linked from dsu. So if you can just comment on there.

A

uh I guess just as uh possibly a silly question, but uh do we want sally on the owner's list, given that yeah.

C

Yeah we added him um as well he's not interested from last from what I understood um too fully, but he is definitely added in the oh, I'm going to mispronounce this in the past owners. What is it called.

A

C

You, yes, yes, um I always read it, but never pronounce it um so he's added on there definitely.

A

Okay uh and uh thanks for offering uh to help out with that one frederick, uh I don't think we have anything else on that one so uh lily has another topic: cube state, metrics, 2.0, alpha, that's exciting! Yes,.

C

Pre-Release though um so we finally managed to um merge all the braking changes and it has a lot of braking changes uh so make sure to have a look at the change log. It should be complete enough, but if there's anything missing, um let us know either on slack or on github, but it would be great if people can try it out and yeah just report back if there's anything um broken or wrong or just in general or good really. um So, if it works, that's a good indicator as well.

C

The main thing was a bunch of metrics we deprecated and the name spaces flag and allow list. The black white list was renamed and that's about it. I think but yeah the fool is in the changelog, so yeah, please try it out.

A

Great I'm sure it's excellent and everybody will say how awesome it is. Yes,.

A

uh Okay, uh do we have any other business before we circle back on caps,.

A

Okay, let's circle back on caps, um maybe I would be helpful if I share my screen with, like all of the backlog of the cap issues.

A

uh People are nodding. Yes, okay, I'm gonna pop this out and share my screen and pray that it works. Oh, let's see what even is the button to do that share screen.

A

Okay, uh can everyone see my screen? Is it big enough?

A

Okay? uh So this is kind of why I wanted to go over this we've got, I think, like seven open issues for caps right now and a bunch of them are like tracked yes from like they're kind of fallen over to this milestone, but it's not clear if they're, actually in progress or not, and so like. I just want to get sort of a better idea of where all of these are at what we actually want to target for 120.

A

um All of these five have the actual 120 milestone on them. These ones don't, uh but there seems to be a lot of interest in getting them done so.

B

I only see three of them with the 120 milestone. Those are the.

E

Ones that they've already had a look.

B

At um yeah, which.

A

I assume means if it was 119, it's gonna fall over to 120.. I don't know. If it's, I don't think it's done so.

B

We weren't really involved in this one, even though it's technically our area of responsibility.

A

This was the one that we got like a presentation on, like the last sig meeting right.

A

Yes, yeah, I guess let's uh come up to date on this one.

A

Did it graduate in 119 or will there be work in 120, oh great, kristen's, checking in on this one, I'm wondering what to do with this one.

B

uh There's a so let's track it for 120, so we just need to add it to 120.

A

Okay, so I will, I guess, update the milestone.

F

Yeah because it's.

A

A

Hopefully that works, I don't trust these bot commands.

A

Yes, oh now it has 119 too remove milestone. I don't know how this works. I will just let somebody else worry about that. um Okay, so uh david, what's going on with tracing.

D

um I need to ping daniel again. I think it would help if I implemented part of it just to show them that it's the code changes are pretty minimal, um but I yeah. I hope this isn't too controversial at last release. He just didn't have time to go through it, so I'm going to put some effort in in the next week and make sure that this gets his attention that I can address his questions.

A

It looks like for this one: we've got, uh we need to fix up the cap and make sure that there are like test plans and whatnot before the enhancement freeze. So.

D

That's october yeah, that's october, 6th, okay,.

A

Great uh structured logging, the ongoing adventure.

A

So this targeted yeah.

C

A

For alpha, it's got lots of nice check boxes here, which is great, uh so is the plan to put this in beta for 120.

G

So what would need to be done? Firstly, to scope? The work like there were some ideas for beta, but there weren't enough details to say like what should be exactly done so far like first, including just go beta, and I don't know if I will have time to finish before like because definitely I will not have time to manage the implementation.

G

This quarter.

A

Okay, yeah, so we're not sure if this one uh we're planning on getting this done before the enhancement phase.

G

Yeah, I would definitely need some help with with the.

G

Yeah scoping the work, and maybe someone like I could have implementation, but I could definitely not don't have time to fully organize it like. In previously, we had over, like 10 or 15 contributors with a lot of new contributors, and that was really like. It required a lot of work. It was, it was great, but it's pretty like doing it again, even on like four bites should be even more work and then it's.

G

I cannot commit this like this. For this quarter,.

A

Okay, uh that sounds good to me. uh I can maybe help with the scoping, but probably not the implementation.

A

So I think this one's a maybe.

E

I I can help with some some implementation merrick. If you need.

A

A

uh New unique logging metadata into key kubernetes logs. What is that all about? I don't think I've seen this one.

A

C

A

Tracking cap, this one, I think, is not yet close to being like implementable, so I'm not sure why we're tracking this one for 120., I think 121, might make more sense.

D

Yeah, this is one that resulted from a very, very long slack threat between me and this person. It's sort of the planned follow-up to api server tracing, but we want to propose or we're going to he's, going to propose it at the same time, but it does have a dependency on the current tracing cap.

A

I see yeah I well I mean we've still got like what uh a few weeks a couple of weeks before uh the enhancements freeze, so it's possible that this will get into a better state before then.

B

Is it really that big of a problem if it if the internet is missing.

A

Like a lot of implementation, details like it's just it's, the the cap was not done the last I looked at it, so we can't implement a cap that is like not implementable.

B

Yeah, okay, I'm just saying: is it that big of a deal to say that it's targeted for 121 for now and if we don't manage to then we move it up to 122.

A

I don't know what the owner is planning on this one. I think I gave that feedback like yeah a week ago and there haven't been any updates since then, uh and the release teams also said the same thing. um So you know if folks want to go and do all the work like I'm not going to stop them from doing the work. I'd be happy to see it in 120, uh but you know it it's it's either going to make it or it's not so.

D

A

Okay, uh I'm gonna suggest uh like assuming that the cap is actually completed and approved we'll target 120, otherwise we're gonna go for one.

A

A

Okay, uh so I guess we've got three more to take a look at here. um I reviewed this one recently.

A

A

I'm going to wait for clayton to respond on this one uh because I think there's a lot of feedback. That's happened on that uh the pr for the cap. I think it looks relatively good, um but it's just a matter of uh getting everything together to do that implementation. So I'm going to leave that with clayton.

B

Yeah, I think I think the intention is sound and largely the strategy is sound. I think it just needs to be consistent in itself.

A

Yeah, like it was just missing like it was like small sort of nitpicky things I think uh so, like you know, it just needs to go through uh the review process and.

C

A

Like work out all the kinks there, and then I think this one should be good to go. Certainly the premise of what is trying to solve and I think the general high-level implementation is very good, so yeah.

A

uh Okay, so uh that one is not tracked and we are good with that. uh Do we know what's going on with log sanitization and the static analysis.

F

Yeah so so I guess I can say a few words about those two. So there was no no um like significant progress on both of those recently, but those are definitely something that that we would like to.

F

Implement in a 120, so so bo and those are kind of interconnected, uh and we would like to have both in one twenty.

A

Okay, uh so I know that this one, I think, has already all been merged and whatnot, um so I can set uh this one: the milestone to 120.

A

But um for the uh for this one, uh I took a look at this, so this cap is currently in progress. It has not been merged or approved. Yet uh I think this one there was still like uh it needs more reviews and potentially needs. Like some uh some updates said yeah. It looks like folks have taken a look at this recently so other than me.

A

I don't know if all of us has been addressed yet, but I would say that this one potentially could target uh 120, but I think the cap needs to be like merged as like provisional before we can do.

A

A

Okay, then we got through them all and we've got five minutes left so see that wasn't so painful. I'll just add this comment to this.

A

E

I don't know: what's the.

E

A

Okay, I will stop sharing thanks everyone for bearing with me on that one.

A

I hope nobody has any like secret caps that they want to like sneak in.

B

A

Know like last second catching us all by surprise. I mean if you have a secret cap, I mean that's cool, uh I like secrets, but uh I will not be able to approve it so.

A

Does anyone have any other business for today.

A

Lots of headshaking no uh does anyone want to say hi.

A

No, no hello, why not don't be antisocial.

A

Well, uh given that we have no other business, we've done our kep review. uh It's great to see everybody, I'm gonna, stop the recording and thanks for joining us today, I'll see some of you hopefully in the triage meeting next week, and uh if not the sig meeting in two weeks thanks, everyone.

B

Thanks everyone.