Kubernetes SIG Release, 5 Mar 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes 1.10 Release burndown meeting 20080305

Description

http://bit.ly/k8s110-release

A

Welcome everybody: it is Monday March, 5th 2018. This is the coronaries 110 release. Meeting I am Jay singer, tomorrow's the release late for 1:10 and happy to lead us through today's meeting. If you're, looking after the fact, I want to see the agenda and things we talked about, it's available at bitly, slash, kids, 110, burned down without further ado timeline. For this week we need to close out Doc's PRS and get dachshund in good shape. Generally speaking, that includes release notes.

A

Hopefully we have a draft that we can start reviewing soon, there's a lot of stuff that we still need figure out there. We're gonna have a meeting, hopefully soon to talk about what is actually supposed to be tagged in there. As far as the release notes label goes and that's still TBD.

A

Obviously we have a couple weeks to sort it out. It's not the most urgent thing in the world and there's not new. There shouldn't be too many new things going in for abilities, notes, so we'll figure it out, but again it's in process.

A

We need to figure out release health really sprint health, so basically sorting out what flakes are real and from what are or what our flakes and what are real and basically figure that out and make sure that nothing has gone into the release branch. It's going to destabilize it and we continue that trend. Ford Marketing is proceeding, we're reviewing the blog and making sure that it syncs up with, what's actually being delivered.

A

There was a change in terms of delivery on TLS bootstrapping, so that's kind of depressing that that's not going GA this time around, but, alas, 111 will see a TLS bid stripping I'm sure next week. Next week is when the stuff gets real.

A

We start the Monday through Friday, burned down meetings and we see the end of code freeze and do the final branch fresh words and all that good stuff. So basically we're gonna release the floodgates assuming that we're in a good position to do so in the past. We have delayed that for various reasons and if we need to this time well, basically, after that point once the branch reopens, then we do, we basically are in the business of cherry-picking specific things. So hopefully it's a low-volume.

A

The reason that we do fast-forwards during this code freeze period is because it's just kind of a pain to do cherry picks so having the having the release frozen, allows us to do fast forwards, because it shouldn't be getting anything to master that isn't specifically for the release next week. We'll probably also do our first prediction on go/no-go that's important, because we're gonna have to make our best guess so that we can time marketing activities and all the stuff that goes along with the release.

A

Hopefully we'll be able to get some consensus on those things of contract or not, and the next week we're gonna probably be all focusing on docks in one way or another, because that seems to be out rolls and cleaning up tests, cleaning up all the stuff to make sure that we have good signal so moving along any questions or concerns with the timeline stuff. There's things in chat: no, no, no I.

A

Can you know if that's better.

B

A

Okay, so I just did a monologue. There yeah this week, Doc's stuff next week, crunch time, cetera that was a teal here, metrics review, we've got 27 open issues in the milestone.

A

14 of these are in a state that could get dropped by the, but there's 15 failing test issues, it's quite high, but that's a good thing and yeah that's pretty much. It Josh I'll go into probably more detail. If he's here yeah here so.

A

Let's go ahead and move into that stuff. Do you want to give it a run down? Josh yeah.

C

We haven't finished updating the issue tracking, yet this morning, from the weekend, what most seasons have happened is we've gotten a whole bunch of new test, bugs they're, actually only sort of new, as in there was a major test failure and it's been broken out into a whole bunch of individual test failures because they belong to different SIG's I'll. Let signal comical bit more about that, because he's the one who opened all all thumb.

C

So, numerically, it's a bunch more issues. It's not actually a new failure, it's just being more specific about a failure. It happened last week. In the meantime, a bunch of issues, I think had been closed. I haven't gotten around to updating which issues have been closed yet so I don't details on that, but it looks like in terms of stuff. That's still open we're, mostly looking at as you'd expect at the stage test failures. I, don't I, don't see a pattern necessarily in the in the test failures.

C

There's a bunch of upgrade tests that are failing, but then there's a bunch that are passing as far as I can tell the the issues. Special concern remain. The same as they were on Friday, namely the steel weeds problem in was the other one.

C

Sorry, it still reads problem turn the other one else. Oh.

C

Right the memory increase, so we had a major issue with a potentially huge memory increased by the API server memory usage increased by the API server analyzing. That was being deterred by a skill. Do the test flake a scalability flake is supposedly no fixed, so I'm. Now waiting for an update on whether or not the memory usage increases real.

C

If it is real. That will be another issue of a particular concern, because we can't really double the memory usage of the API server without a good reason.

C

The no new issues of special concern that I've seen there is one thing: that's actually come up in terms of tests and issues against test, which is that we had one issue against the conformance test that turns out to be apparently caused by a bad update, push the conformance test, and this really is the whole question of how we're treating the conformance test for this release, because a lot of the conformance tests are relatively new and still kind of works in progress, which makes me ask whether or not we're can whether or not on four new conformance tests.

C

We are considering failures and those conformance tests to be blockers.

A

Okay, so on that note, this is this is sort of a new thing that we have to look at this. So it's an interesting new flavor of test flake or test issue and I guess you know that the conversation basically said that this is new in this release.

A

My feeling on it is it whatever we're using to get good signal on in terms of release, health or release, quality, ÿû conformance or otherwise, even if it's new, that doesn't necessarily mean it, the that it should be disregarded, but we definitely need to have an integration period where we study what it's actually telling us about the release, and maybe some specialist input on sure in terms of if it's actually a flake or not, and if the assumptions around the conformance are are correct.

A

So we need to figure that out I'm, not sure what the best way to do that is I know that Tim Sinclair has been pretty involved in that Jordan. Have you had much to do with that? Is that kind of in your zone at all, or is that.

B

The compartment lift with the upgrade test they.

A

Conform, its specifically.

B

Only to the degree that it touched the signal, then I would say that if we have a conformance test that doesn't pass against the release we cut, it should be untag to the conformance test, because it's these are the standards by which distributions are measured, and so it doesn't matter whether we ran it or not or deemed it blocking or not. If his tag is conformance and we don't pass it, then that's bad right.

A

Yeah and then, of course, the underlying assumption is that it's a valid test.

B

Sure yeah so ones that I guess have never passed and only got noticed because I got moved into a blocking bucket. We need to determine if the test is valid. Iii did want to mention something about the upgrade test.

B

There are at least three of the issues that I'm aware of where the failure is that a19 test does not work when run against a 110 master, and it is because deprecated function was removed in 110 and so I didn't think that we ran tests from a previous branch against a 110 master just because that requires you to go back and change previously least code when you remove a deprecated feature which seems wrong.

B

So at least three of the issues are that there's there's just not a version check in the EDD test in 1/9, which shouldn't surprise anyone. But that's that's where those failures are coming from yeah.

D

Speaking so, the design the DVD test was upgrade from when we can see another version, and there were twist week when run the other and the other one ran the test. So, for example, when we upgrade from 191 10, we want to make sure both one nine and one 10 tests are working. If, if they're individual tests, that is not supposed to run, we can maybe just at a select tinkle skip with the feature tag into the that specificity, huh mmm so that it does not run that specificity test. So.

B

Like for the example as a proxy, the proxy e to e, so it that tests about 50, different paths and seven of them were have been deprecated since one two and got removed in one ten. And so we definitely still want to run the other 42 paths. But.

D

B

The seven that were deprecated and removed, but the only way to fix that is to go back and edit.

A

B

A new one, nine or push something into the one. Nine branch which again, like I said, seems backwards.

C

E

Nine branch is still active, still support I.

B

Mean it so I mean that looks like an active maintenance branch and strange to push changes to that for 1:10 development and actually right now, it's frozen because of for stabilizing prior to an upcoming release. So yeah.

E

I mean I think it would be more weird had like if the tests weren't bundled so like tightly with the code, that's under test, but I. Think in this case. These kind of weirdness is just going to happen because, like there's, not a an external repository that contains these end-to-end tests or these like acceptance, level tasks.

B

Yeah I guess I would expect the conformance test changes to it to be reviewed for correctness, and so the 110 tests like we should be removing compromise test unless there's agreement that the behavior shouldn't be tested anymore. And if you do that, then the 110 test should suffice.

A

A

Is there something there we need to sort of come to some decision on is this? Is this getting into the policy realm like suggest, possibly.

B

Part of it part of it, is practical. Like there like I said, there are at least three issues that won't turn them won't be able to be closed until we change how we run upgrade tests or make changes instead of at least one nine branch. So.

B

A

Alright, you want to go through your stuff. I mean you kind of already started that you put a lot of yeah.

D

Sure so the major testing like has been failing clusters or when there's the cereal sweet, mmm Dara I opened five, so I started to use the like job Steve real sweet failure, as they upgrade I, say, umbrella issue and I open five more like issues for individual suspects. hmm Hopefully each sick will start to scratch that and become green and for a pretty sweet. So after the dimmest at six, the upgrade is like six, it's actually successful now, so it's actually upgrade from 191 ten.

D

Now and now they are like revealed more actual test failures, a store dimension. Maybe some of them need to be further. It's cursed, mmm-hmm, but I also open a list of actual test failures. mmm The other concern that maybe actually concerned is the dungaree test from master 209 is currently failing, mmm I assigned to stick just like psycho, and hopefully they can charge that.

D

That, in terms of the stability sweet, there is a stick, storage fix, went in and according to sham, that made the test looks better, but but he need to follow up to monetary. If there is any performance, degradation seems Talib. You have a comment there.

E

Just looking at the out the log output from the tests, it's like the same two pods ones like metric is bordered and other ones I Prometheus pod. They don't seem to be running I, don't know if it's a scheduler specific problem or if there's some underlying infrastructure set up mmm thing going on. Okay, can you comment back.

D

On the issue and yeah.

D

Last thing on my list is the cubic cubed. Even Swede is still failing. Garrison band took some testing imperfect system, but still it's failing like cannot find Q comfy, but I saw a previous comment mmm that people are saying they want their lists across the lifecycle want to get rid of Q bad mean like get rid of communities anywhere, I'm, not sure. If that test need to sleep inside master, blocking.

A

And everybody here someone's.

D

A

Enough insight on that to be able to tell us if the plan is to keep it there.

A

And I'm sorry I'm missing, like at least one-third of all the words, because my stupid Internet is really bad here.

D

Yeah, it's just. If you're clicking to the issue comment link there sig is saying they are trying to they're getting rid of the commands anywhere. So I'm not sure if we want to keep job inside the master box and dashboard.

A

Okay, I guess we'll just have to sort that out then.

D

Yeah I guess I can send a send it into optimistic and see what they got.

D

In best also pings across the lifecycle about the upgrade test issue, Jordan brought up and see if we can get a better design or what's the action line item would be 49 failures on the one I to one pan of grade test.

D

A

Okay, thanks for docs updates here, if anybody's got those I.

F

Will add notes to the to the doc? Okay, so I Jase, it's Jennifer and I'm here, I, do it late on things are looking much better than they were on Friday people have been super responsive to getting poked and, and um my biggest concern right now is a small handful of alpha features where I'm still not sure what's going on, but it also looks as though some of those issues have been updated. Since I checked this morning, I haven't had a chance to check again, so things are moving faster.

F

Looking good and sick Doc's generally is doing a great job of getting PRS merged. So my biggest concern is getting the few straggling PRS in at all. Getting them in getting them merged by Friday, provided we get them in George should not be a problem.

A

Okay, you broke up a lot of me there. Hopefully what you're saying is. This sounds like it's on track. Yes,.

D

E

But I guess for Jace, so you said Jennifer you said you were gonna put the link to the document release, notes, draft in the meeting minutes and.

F

I'm doing Doc's not release, notes, I, don't see Nick on this call, but I can ping him and ask him to do it. Okay,.

E

F

A I would like to circle around with you, though, because it looks as though there's still a small handful of features, not just um not just the stuff folks have mentioned up to the up to now. That might not make it in and so I want to be sure. I'm understanding state properly I want to spend my energy bothering people who need to be bothered and not people who don't yep.

G

So I've checked all the all the features did you Doc's request and I'm, not just that some of them are really not making it to release, including pls bootstrap was actually one of the major features that was planned for this rule is great yeah sue I also dated.

G

Spreadsheet with them, so if you I'm going I'm going to double check all of them, because I'm working closely I'm actively work, you know marketing material for Natasha and from for the marketing team to to push it forward. But if you'll find anything else that I could not notice in the features report on the spreadsheet, please, like okay,.

F

I added a note: I chose a one line: release no description, column, I added a note about two of them on that looked as though they might not make it in depth.

G

F

I think you you, um you use, you did a strikethrough on one more and then I mean the other thing folks might be interested in knowing it looks like there's a whole clutch of storage related features that may be covered in existing docks, pr's and I'm. Just trying to find a right person to confirm this with.

F

It looks, like you know, a bit of a tangle from my perspective, so I'm gonna track that one down to I hope today, speaking.

G

About just search features suddenly from Google is the primary person responsible for abilities at community. So if, if nobody else is responding to you in the general communication, like an officious repo in the mail and tracks, I can suggest you to in okay.

B

G

Hope he can, he can confirm ever ever seen this happen with storage, okay, cool. Thank you.

A

A

Okay, let's see what have we missed here.

A

That looks like at the end here, somebody wrote: the milestone command had a bug, but it's working now. Is that that's awesome, yeah, that's! That's it cool, fantastic, okay! Anything else, one just one bit of part on my side is I'm gonna, be I'm at the open source Leadership Summit this week. So it's a little tricky for me having time and obviously crappy Wi-Fi. So nick has volunteered to lead the meeting on Wednesday. If I can't do it, which I probably cannot so we'll see how that goes.

A

Any other things that people want to talk about before we adjourn.

G

The selection mentioned at and will be it instead, open-source leadership's on it as well this week, so one be able to join on Wednesday. Some kinda asked if you community median on Friday so like for me to junior I'm, always with you and down channel I.

E

Do like the idea of a are always being with us: I'm not going to follow. I'm gonna deviate from the playbook and not fast forward the branch until things calm down on master just because we I've introduced some instability on the release branch by fast forwarding. Then, unless anyone has any objections, I guess mostly, you send.

E

Cool well, wolf. Carry on with that. Then witches do nothing! So that's easy! Yes,.

A

Please keep doing the nothing you're doing. We definitely need more money. That's all right cool! You thing else before we call it.

A

Okay, great everybody. Thank you so much again and I'll, hopefully catch up with you soon and have a good meeting on Wednesday. If I'm, not there thanks, everybody.

E

Thank you see ya. Jason thanks. Everybody.