Kubernetes Community, 26 Oct 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Community Meeting 20171026

Description

It's the Kubernetes community meeting! Demos! Discussions! Beards! Something for everyone.

A

Be using the API, and it also requires a pretty detailed amount of information about the API as you are using and you have to represent all of those accesses inside the roles that you develop and it's not particularly complex, but it is pretty fiddly, especially if you're trying to build a role for a user or a component that you don't know the details of right.

A

You know that you want to use this application, but you don't actually know all the individual API calls it makes, and so what we have done historically is kind of run, an application see where it fails by looking through logs or the applications logs and then piecemeal iteratively build these these roles by hand and that's very tedious and as any good programmer, I'm very lazy, so I wrote a tool called audit to our back, which takes advantage of the audit log functionality that was added in 1, 7 and 1/8 to generate our back policies based on the API calls that a user or a component actually makes so.

A

The name was shamelessly stolen from audit to allow for SELinux. But you take a look at what I have here: I have a local cluster running with a really simple audit policy, which basically says I want to log metadata about requests from these users and I. Don't actually care about seeing the request body response body I just want to know like what what API calls did they make and I can look at the audit log here and see these audit events.

A

So I can see that the system admin user tried to list pods and you can see that that's live. So if I list nodes, then I can auto event for system admin listing nodes. So it's great so I'm able to see the API calls that are being made now one of the new things you can do. If you are a privileged user, you can actually impersonate other users, so I'm going to run a similar call as a user named Bob and when Bob tries to get pods.

A

Bob is forbidden because he doesn't have a role that lets them do that, but I can see an audit event indicating that Bob tried to list pods, and so this gives the audit to our back tool the information it needs to be able to see what Bob tried to do and craft a policy that would allow that, and so, if I switch over to this tab,.

A

The inputs that the tool takes you have to give it the audit log and you have to tell it what user you want it to generate a role for so if I run that feeding it my audit log and telling it to generate a role for Bob, you can see that it creates a role inside the default namespace, which is where I was trying to list bugs that allows getting listing and watching pods, even though I only listed bugs best practices.

A

If you are letting someone list generally, you also let them watch and get individual things that were already included in that list. So there's some heuristics to kind of craft typical roles and and these things would be optional, whether you wanted to do that, expansion or not, and then it also sets up a roll binding to give that role to the user. Bob inside that namespace. Now one of the neat things I can do and I can actually pipe this to keep control, apply and load.

A

Those and now, if I, go back and try to do the same thing, I just did it's allowed now there actually aren't any resources there, because I accidentally did it in the default namespace instead of the namespace I meant to do it in. So let me just jump over and look at it in the other namespace and oh I'm still forbidden, because the role that was crafted was a pretty tightly bounded role.

A

It didn't give me permission to look at pause anywhere in the system, just in the namespace, where I actually tried to do it so now that I've tried to do it in some other namespace.

A

That's that's still not allowed, but if I rerun the audits to our vac tool, you'll notice that instead of granting permissions to get pods in the default namespace and in the cube system namespace, it has promoted my permissions up to closer scoped, the idea being, if you are constrained within a single namespace, typically what you wanted to limit them to that particular ting space, but if you're operating across namespaces frequently it's it's extrapolating that to say you should have permission in any namespace and again this extrapolation is optional.

A

You could you could generate tightly bonded roles in lots of individual namespaces if you wanted to. But in this case this is what we want so now, if I load this.

A

This would let.

A

Get less than watch pods in any namespace, and indeed you can see that I can now listen. This other namespace now where this gets fun is if we set up loops to do these things so I'm actually going to I'm, actually going to start this tool running over and over again, so that it will, as I am using. The system continuously generate this role and then I will set up another loop, which will show me the differences in what's being generated.

A

And and finally, a third loop which will apply them so if I sit here watching the roles that are getting generated and now just use the system as Bob. So if I try to get secrets, you can see that.

A

The secret permission gets added in and if I try to do that a moment later, because this is getting continuously applied in the background that's allowed- and this applies to all kinds of things right, I can look at create permissions, see those come in and then now that it's been applied, that operation is loud same thing for delete permissions. So this is basically let's a program: do the work of figuring out what the exact resource name and the exact API group and everything is required to allow those things you're trying to do all right.

A

So that's that's what it looks like to do this with interactive users? What about programs applications things running on the system? A frequent thing that we have to do is build roles for things running in pods, that we don't actually have detailed knowledge of, and so put together an example job.

A

So here's a job that is just going to try to make a bunch of cube control calls inside a pod, so get some config label it modify it get some pods get some nodes run a couple sub jobs and I've written this so that if any of these fail, it'll just keep retrying that until it succeeds and then proceed so I'm going to start this job running and.

A

Wait till the pod is running: okay, great and now I'm going follow the logs, and you can see that it's stuck on the first thing it was trying to do and it's getting forbidden and so I'm going to set up those same loops.

A

Only this time, I am generating a role for the service account. So in namespace one service account one that that job is running as and I'm going to setup the same loop to show me differences and, what's being generated and the same loop to apply those, and so now we just get to sit back and let this application run and watch the the role be generated to let it do what it needs to do. So it's trying to delete a config map is trying to create a config map.

A

It's trying to get and then patch an update which it needs to do in order to label. But you don't need to know that you just need to know these are the API calls that it requires it's getting a listing. Pods now I've got enlisted pods across a couple names faces, so it's got a cluster role.

A

Doing the same thing for nodes and by the way this is, this is the kind of thing that would take me like two hours to build a roll by hand, so it this makes me so happy that I don't have to do this anymore. Creating jobs created a couple different jobs, so now it's no longer restricted by name this gets fun.

A

So it's actually following logs of those and this gets into how you represent sub resources in our back rolls again things that are fiddly things you could figure out if you wanted to, but now you don't have to and it's making progress and eventually it completes, and you can see the the roll that covers all of the API operations that needed to be done.

A

So that's that's the demo if you'd like to follow the repo or download the latest releases, I have more improvements that I'd like to make, but it provides a great starting point for developing your own roles or building roles for applications that aren't yet distributing their own all right. Any questions.

B

Just so, this is a huge time-saver and it's an incredibly creative solution to a tough problem. Historically, so perfectly.

A

Cool there was one question about the retrying, so there are a couple ways you can develop. This one is to run the application in the development environment and just give it full permissions. So let it do whatever it wants to do in an environment that you don't care about.

A

Security, wise and the audit logs will still record the API calls it made, and so that lets it do whatever it needs to do, and then you can generate a bounded role from the operations, apply that remove the broad permissions and then rerun it and test it under that. So that there's several different ways you could could do this.

A

All right, no, no other questions head back.

C

Jordan, thank you so much for a great demo. There, that's fantastic and I echo the the fact that this is tremendously useful for an hour back implementation, which is notoriously thorny to do so. Thank you very much for that work. We're gonna, move on to release updates Anthony, unfortunately, could not join us, but I'm. The secondary release lead so I'll run through this real quick feature.

C

Freeze is tomorrow, which means that if you are trying to put some big functionality in 1.9, that should have a reflected issue in the features repo there's a link in the agenda to that. If you want to add that there is an exception process after the freeze date, but we highly encourage you to not have to use that, but please take a look there and make sure that the work your sake is delivering is represented less than four weeks until code freeze, which is on November 22nd.

C

This is a short cycle for this release because of the holidays and good cron, and the release team is is going to be pretty busy with a lot of other things. So please make sure that you're getting stuff done ahead of code freeze again, there is an exception process after that, but we definitely don't want to be getting involved in that because of the timing for good con coming up.

C

So please make sure that you that you meet that deadline, if at all possible, Anthony put in there there's still plenty of time before the 110 code. Freeze, though so yeah, if you missed the one 9 bus, don't fear when 10:00 won't be rolling out as per usual in March of next year, so 1.8 dot 2 is out. The ETA for 1.83 is November. 6Th Joe Betts is the patch manager.

C

So if you have something that needs to be cherry, picked back, make sure that you ping Joe as github ID is JP Betts to let him know that the that is necessary. Also, if you have something that you want to sure that gets into one a day, make sure it's in the 1.8 milestone that it is labeled as a cherry-pick candidate and it doesn't hurt to ping Joe directly. One does something that line is out.

C

We don't have an ETA on 117, yet they'll definitely update the community when we do and one 612 was quietly released to Mead a four-week cadence objective, I'm, not sure exactly what that means, but it means something so apparently Anthony was really thinking that there are no more bugs left. Let's hope. That's true, but I consider that highly skeptical.

C

So um any questions on release updates release stuff, 1.9, your burning questions, fire away. Anything.

C

All right great so moving along Sega updates sake, contributor experience, Garrett is not confirmed, but let's hope thicker is around I.

C

Would take that as a no Garrett, we will reschedule that I can give a brief update, which is. This contributes, is working on finalizing our objectives for this egg for next quarter. There's a list of objectives in the document. If you go to the bit dolly slash sake, contra Beck's document, it has a link to the agenda and the backlog of things that were working on four one: nine in one ten, if you're curious about that, hopefully we'll get a full update from Garrett moving forward. Until that time, my Rob Hirschfeld forsake cluster ops.

D

So cluster ops was pretty idle over the summer and then in our last meeting we had a lot of people show up an express interest in a lot of different topics, so we have agenda for at least the next six meetings, which is going to be focused around going back through demo in various deployment, different deployment, options, technologies and approaches.

D

The next meeting, though, is actually about the cluster API, so this life cycle, cluster lifecycle sig, is promoting some cluster API work to try and create some standard practice, not the best person to describe it. What I do know is that we're looking for operators to come together next week and discuss the API.

D

So if you were operating a cluster or writing deployment technologies and management technologies around kubernetes next week we are going to be discussing common tasks across the boyars, so not the player specific, but cluster management and operations specific next week, so please join in the battle, probably become a repeating topic, and then, after that we are going into basically a survey of cluster operation deployment. It won't be the only thing we discussed, but we're looking to.

D

We have at least three different demos on that, starting with cracking k2, mesas kubernetes work, coop spray and node emission. So it's for all different ways to get clusters up and running. Our demos are long because their discussion focus, so we want to see a demo and then we want to discuss the speeds and feeds and the whys. If you have a approach to deploying and managing a kubernetes cluster, please come get us get on the list.

D

We will give you objective feedback, talk about pros and cons and sort of work things through that's what this SIG is for and then another background topic for us is that, because sig ops, this really cluster office is really a cross-cutting sig. We're looking at transitioning to a working group, which means also reviewing and reach, are during what the sig does potentially finding or changing leadership.

D

So there's there's a lot on our plate coming up and really exciting stuff right. Kubernetes has gotten to a point where cluster operations is a key concern and you want to keep that front of mind and ensure that it's not a vendor specific thing. A community concern and Jase don't know if Jason Jason has proven himself so capable with the other roles. I, don't know if you're gonna pick pick up the mantle again but we'd love to have you back in meetings. No.

C

Thanks yeah, it's a it's I have more more things that I can do than time to do them. I'm sure we all have that so yeah I just want a second everything. You said that how important Cygnus drops and if it eventually is a working group that this is critical information to get back into the product, so so that we can make communities a better experience for everybody. So thank you for that update. Let's move on to announcements Erin. Would you like to hop out of the clown car and give the steering committee update.

E

So hi everybody welcome from the steering committee.

E

We're still here making progress so I wanted to just update us on some industry. Yeah. Sorry I'm. Looking around find the bullet points that I wrote for myself in the meeting it's so you can all follow along. If you look at the meeting notes, so we have had two meetings thus far and we sort of have put together what we consider to be minimum viable process.

E

So basically we have a mailing list. You can mail steering at kubernetes that I do and it will contact everything steering committee. It is this Google Groups that I have lengths of the steering Engelberg anybody opposed to it. Anybody in the public, you can read the.

A

E

Is we want to be doing as much of our work out in as you possibly can? So that's where you get to see, for example, if I can share my screen.

E

So we can click through and we can see how I asked if it was cool. If I gave an update the community today and some consensus on that basically saying bullet points and we're.

A

Like yeah, maybe.

E

It's not a good idea to try suggesting things 40 minutes before the meeting. Sorry I kind of forgot about this until the last night. You can also see how, for example, Joe is hopefully put together sort of a contributing MD doc in our kubernetes steering repo.

E

So hang on sorry interests trying like that.

C

E

You can see that, basically, if you come to the repository, we're asking that the main method of engaging with the steering committee is that you email.

A

E

List so that we can have the discussion out in the open and we find that it's easier to track stuff. That mailing list, as opposed to spread across myriad github issues and pull requests and.

A

E

That many high traffic individuals can't necessarily follow on a day to day, let alone week by week basis and we're also sort of trying to spell out the rules of engagement, the steering committee members ourselves so as much as possible. We intend, to you, know, try and populate this repository with the documents, decisions and use pull requests to do so.

E

Alright, stopping my share, so you know give us give us some time. We sort of just came to consensus on what minute enviable process is yesterday, so we're taking some time to clean up the toxins and recoat to describe everything, I'm just sort of staying out loud right now, the first two meetings we had were privately attended and not recorded at all. Our meetings going forward will be held every two weeks on Wednesdays, so starting from last Wednesday. The next meeting will be two weeks for them.

E

It's going to be attended just by the bootstrap 280 people, but our intention is to publicly to record the meeting and then post it publicly and do that for every meeting going forward. We will also plan on posting the meeting notes publicly so give us some time and we'll get the meeting notes from the prior two meetings, where we maybe for folks to take a look at let's see.

E

So. The very first thing that we are working on, as was sort of described during the election process, was that we yeah Thank You Jenna. The meeting is our next meetings eliminating so the first thing we plan on doing is ratifying our charter, so this was the you know, initial bootstrap charter that was circulated throughout the community and everybody got a chance to comment on.

E

The very first item on our agenda is to make sure that that looks good enough, that we have addressed any of the major controversial concerns and that we have put in place an amendment process. The the thinking here is that we really want to sort of make ourselves real and whole and gratifying. The bootstrap charter does that we recognize that it is not perfect. We really do want to improve it and work on that. However, we feel that there are many other pressing concerns that folks are looking to this steering committees to help address.

E

So to that end, we we sort of have chosen our top three priorities that we think are worth addressing. The most important thing that everybody largely agreed on was the incubation process and by an incubation process, I mean what does incubation mean? What is permitting into incubation mean? What do you get? What are you when you are incubated? Where do you live? What does coming out of incubation mean? What is graduation from incubation mean? What happens if you don't graduate from incubation?

E

Is incubation even necessary, try to imagine in 13 people having 13 different opinions on this topic and so understand that? Well, you recognize it's incredibly important. It's probably going to take us a little while to take this apart into discrete actionable items, but, like I said, we have that public mailing list where we would encourage contributions and discussion and take that into effect, and you can see us our discussions on this particular topic.

E

The next most important thing we feel is clarifying the code of conduct policy and committee, so we sort of have something loosey-goosey in place that more or less works. What we'd like to add a little more structure to this, so look for us to figure out how to refine what the exact process is and how to put more accountable decision-makers in place rather than having to back-channel everything. We recognize that code of conduct is an extremely important piece of any healthy open-source community.

E

You take this very seriously and if you have any strong opinions on this, we welcome feedback at the steering nailing list. The final thing again: it gets kind of difficult to break this up into small. Discrete chunks is clarifying city charters and by that I mean what gives us sake. What are the roles and the sake? What sort of powers or whatnot was that to note you? What does it mean for a sake to own a particular piece of code or a particular process, or some other abstract part of the project?

E

What are the mechanisms that we use to identify which pieces of the project are not yet over by States? How do you say, coordinate and communicate with each other? What's the difference between a cig and a working group, you know, what's all of the infrastructure and administrivia necessary for sakes to do what they do so again.

E

We recognize that there's a lot of need to fill in these gaps and sort of clarify some things, but we feel that clarifying at that level is less important than making sure the code of conduct committee and process is nice and tight and addresses everybody's needs is less important than making sure we clarify what exactly what exactly incubation is and does anybody from the steering committee think I missed anything or have anything to add there I.

F

Think one thing- and this was part of the original, Charter and I just want to emphasize it- also is that we do want to find ways to get. You know, after sort of the first sort of chunk of businesses, dealt with to get the steering committee out of the way and distribute a lot of this decision-making. A lot of this discussion across the the wider community, and so some of the sort of important stuff that that is in this list is really just point in time at steady-state.

F

We're really hoping that the the steering committee won't do that much it'll. Take us a long time to get there, though,.

E

Yes, that's that's absolutely correct, in fact, in framing even just those three problems right, we're always looking at how can we best identify pieces of these all-encompassing problems that can actually be decided today by the existing sakes and leadership structure that we have? There are only thirteen of us we're all very busy doing other things, so we really want to find ways to make sure that the decisions can be accomplished and spread out in parallel as much as possible and really only escalated to Lhasa in strictly necessary, so yeah.

E

We really want to decide our way out of the problem.

E

Any other questions from the broader community.

C

Okay, all right there um I'd one more me: are there meeting minutes or anything like that? Do.

E

You have a meeting and it's from the past a meeting place I'd just give us a little bit of time and we'll get those posted we'll get those posted. So even though we don't have recordings from the past two meetings, you do have meeting minutes from that and we'll get those shared with the broader community shortly and then anticipate that for every meeting going forward that will be shared with the community. In addition to recordings of those meetings.

E

Okay, awesome just found. Like a personal note, I can't tell.

A

E

Excited I am to be working with everybody on the steering committee. um It's a bunch of really incredible people and even though we all have very strong opinions, some of which go in different directions. We all are very interested in this project, succeed in the Highland Corps to us, helping better empower the community to drive this project.

E

So just a big thanks to everybody who voted and participated and putting this together and looking forward to serve you all.

C

Thank you so much and I really appreciate it and thanks to the steering committee for your service, this is not an easy thing to make time for, and it is a huge import to the community. So thank you. Thank you. Everybody here, I, don't see any more announcements. Is there anything anybody else wants to add before we wrap up Paris, doing anything about the contributor summit or anything or anybody else um sure.

G

Really briefly, members of standing that received lottery forms. This is your last call to put that in for confirmation that will go out tomorrow, close a business Pacific time. That's it.

F

Great okay, okay, so one of the other things in it's um it's it impacts a lot of folks across across the community. Is that we've been having some really interesting discussions in save architecture, and some of this is is prioritizing sort of some of the low level work that is impacting a lot of people and and how do we? Actually, you know, get to the point where the architecture the project supports, the type of velocity is type of growth and the type of interactions that we all want.

F

Brian I, think it Ryan. Are you online I? Don't know if you're still around you, you said that you were interested in maybe talking a little bit about sort of you know keeping some of this stuff top of mind and just sort of beating the drum a little bit. I, don't know. Yeah thanks.

H

I'm here on time with you ready, yeah I can talk about this I'm, going to send an email to kubernetes dev as well, and we've been talking for a long time about the things we need to do to improve the health of the codebase and the project and principally a lot of it involved. It's making things more modular making it.

H

So the changes don't have to go into kubernetes kubernetes master, so we're trying to unblock that so I'm going to spend some time collecting the the various efforts underway and finding out where they are so I'm, probably going to reach out and ask people to provide status on that. So we can kind of create a dashboard for where we are and it's multiple efforts.

H

There are extension efforts that are underway there, efforts to move code out of the kubernetes kubernetes repository, there's effort to enable feature branches, so I'm going to try to collect the things that are ongoing, and then we can check status here at this meeting on a regular basis. But definitely it is the priority that the health of the codebase and main effort, what some people call core, is top priority for the project at this point, and it will be prioritized over other things at the moment, one it just as an example.

H

There are questions about what is the plan for external cloud provider and what we're going to do there so I think we're going to need to align some of these efforts and put some time rough time frames in place is a stake in the ground, so we can provide some clarity and maybe urgency towards chipping away at some of at some of these issues at some time next year in the first half of next year.

H

I would like it to be the case that only whitelisted development activities happen directly and against kubernetes, Cabernets master and other changes happen either in other repositories or in future branches and in particular cloud provider. Specific things I would like to all be out of the tree, except for things which are in api's, which we cannot yet remove, so so that anyway, that's the the current thinking reviewing that here.

H

If people haven't taken a look at the new dev stats, please do I'm also going to try to get some metrics in the dashboard so that we can actually measure progress on some of these efforts and actually see whether we are making things better or worse. In the short term, almost certainly things will get slowed down in the short term, as we make some painful changes, but the hope is that overall development will be unblocked in the future.

H

Is that where you're helping poor Jo yeah.

F

Yeah I think that the TLDR for a lot of folks is that is that sick architecture is gonna, be doing some prioritization and some shining the light on stuff. That is really just sort of like paying down some debt in and eating the broccoli and doing the stuff that we know we need to do, and so there's gonna be an effort to try and sort of focus and shine a light on that.

E

No new cloud providers because we wanted to prioritize getting cloud providers out of the tree rather than fire, so I would anticipate we're. Gonna see things of that nature. Yeah.

H

So just an example: we've drawn a line on new volume sources, and now we have to make flux volumes, work and the successor to that.

I

Just talking about the subjects and projects that give out key belt stores, I don't think we can take that policy until we have a home working extension right. So in the case of volumes, it's probably okay, because flex volume is there and it exists when people can integrate with it. But in the cases like the key key management stuff, there is no plug in right now, so it seems it seems like not in the spirit of the project to say. Well, you can't add anything new.

E

That the argument was, if you wish to add a new key management thing. You are less incentivized to help us develop that plug-in based architecture, yeah.

H

So I don't want to route home this specific issue here, but yeah I'm digging into that either we will rip out the camus stuff that has been added or we will come up with a plan where we take a certain number of cloud provider. They CMS's up to a certain release, and after that we stop taking new ones and at some point they all have to move to. The plugin mechanism like I agreed.

I

You can say: hey, you need to invest in the plug-in, and you know, as the price of getting your kms provider in is a you know, gentlemen's agreement that you're going to invest development resources in developing the plug-in, but until the plug-in model exists, I, don't think we can prevent people from this.

F

We talked about this very thing during Sega, Architecture, Brian, Brennan and and I I think everybody understands and here's what you're saying and agrees that that we can't just sort of arbitrarily cut things off. Yes, we.

H

Will come up with something fair and workable, but I think having some kind of rough plan or timeline for what we're doing will help people would be able to buy into whatever the proposed approaches yeah.

H

But yes, we would welcome to help on the plugin mechanism. Brendon yeah.

I

H

I

Think I mean I think one other thing we should do is we should whatever plugin that can definitely build for both cloud providers and everything else's. Why not just wrap up the existing cloud providers into the plugin? That's the easiest solution in my mind, is well.

H

And yeah I don't want to get too deep in yeah. I, don't want to get too deep into those details, but there are a bunch of issues there, like some of the things are called from the nodes and some are called from a cluster control plane and it's kind of a mess right now. So.

D

H

D

F

Refactor eases off is always easier in the whiteboard I.

I

Already on my way, but if it's done ship it I can type in PowerPoint yeah.

H

So the the kms stuff is mostly being discussed in sagathia an API machinery to the rest, our degree so get involved there. If for people who are interested in that specific plugin mechanism,.

C

Great thanks already for the discussion on this. It will be continued in the Sagarika texture meetings. I posted the link to those videos. There will be a video posted shortly of today's meeting and I will make sure that the sake architecture agenda is updated. At that link, quick, we have a dev stats demo for context.

C

E

Hey me again, I just think that this is something Garrett wanted to show and Brian mentioning dev stats reminded me: I, don't know how many folks in community have seen this lately, so I wanted to share quickly three graphs that I found of interest in dev stats. So here's that stat stat stockades thought I oh it's driven from github events just make sure okay yeah. Yes, that's is incredibly amazing. This is something we have been asking for and begging for for a while huge incredible thanks to the CNC F for funding.

E

This huge incredible thanks to Lucas I forget his full name Lucas for actually developing this Brian for help pushing forward the definition that many metrics, the repo is github.com CN, CF, /, that's issues and pull requests are welcome there. It's the sort of thing you should be capable of running on Europe. There are many different dashboards here. What do they all mean? Kind of unclear? Many of them have little things that tell you, roughly speaking what they are I wanted to show you three of interest to me.

A

E

First, one is that reviewers dashboard I'm going to be focusing just on the kubernetes repo just kubernetes to Manetti. So I'll tell you why in a second here, so what I'm looking at here is the unique number of reviewers who I get added in lgt an or an approved comment to you a full request. This shows the growth of that number over the past two years. You can see we went roughly from fifty to be roughly two hundred, so that means we doubled the number of reviewers in the project year over year.

E

That is awesome, good job everybody. Thank you. So much next I'm going to take a look at approvers, and this is why I was looking solely at kubernetes kubernetes, because.

H

I'm sorry I just want to quickly interject in one of my hopes is with these. We can assess the impact of some of the tooling changes we make so for.

A

H

In January we rolled out the owners mechanism and, as part of that, we added we, we did literally add double the number of reviewers as part of that process, and we could see the impact after that change, so yeah we want to be able to. If we make a change, we want to be able to tell did it make things better to make things worse. Did it matter, so the hope is. We can start to assess some of that by with these graphs 100% like.

E

Nothing has me more bonkers than making decisions and not really knowing what the definition and the of successes. So this is why I'm just trying to show what state stays so Brian mentioned owners owners is where we sort of broke up reviews in kubernetes communities into a two-tier system. People who are reviewers people who are approved approvers have the final say over class cannot get merged intelligence, both in lgt MDI, reviewer and approved by an approver. This is I.

E

Think one source of bottlenecks in the kubernetes kubernetes repo is that the number of approvers has not really significantly grown a month or a month or over the lifetime of the project. Another way of phrasing. This is the people who have ownership of pieces of the codebase are not doing an effective job of growing and delegating that ownership. This is something that we, as a community, should figure out how to address. The contributor experience is discussing this on sort of project wide level.

E

I would encourage individual SIG's to take a look at who they have, in their sake, that's an approver and what they could do to sort of reduce the burden on their approvers me and say: testing are also discussing the selection of approvers that are assigned to certain PRS because it seems like we still end up signing from the same set of really high traffic individuals, because we don't have owners files in all of the code base areas and so the algorithm it ends up selecting route people more than it does leaf people I'm just trying to sort of see discussion ideas we can continue to have in slack I.

E

Don't necessarily want to hash this out here. Yeah.

H

But that that's a really great point do actually have a provisional process for adding approvers. If you look at the community membership page and the community repo, they want to give it. If you want to nominate yourself to become an approver for some part of code, there is a process for that and I'm sure their details tiring out, but we need people to actually try the process torrent to identified those issues. Are you gonna show the time metrics graph as well.

E

Let's make sure this one I can show time metrics afterwards, if you'd like, if you want to be today, so this one I just wanted to show as time first non author activity, I believe this is across all issues and all pull requests now, I'm just going to make up a story here, I have no idea if this lines up with reality, but what you're looking at is the amount of time before somebody does not be author of an issue. Oracle request does something to it.

E

You know applies a label comments on it closes it merges it whatever um this 15th percentile graph down here. This is like the insiders club right. This is people who get their coal request or issue look back in less than an hour. This is cool. This is absolutely okay. These are highly engaged individuals on the project. The median case is anywhere from 4 to 16 hours again. This is probably pretty reasonable for an open source project. The 85th percentile case. That's a stream line up here, which is anywhere from three days to three weeks.

E

This is where the impression of kubernetes being unfriendly to new contributors comes from is that some people have their issue or their poll requests. It here completely stagnant for quite a while. It could be somebody who's just trying to contribute a bug fix. It could be somebody who's trying to scratch an itch. They may not necessarily be interested in climbing the ladder wholesale. They just want to do the greater good, like they're, used to pull other open source projects and then bouncing right off the atmosphere of kubernetes process.

E

This is where we can have people who don't necessarily know a ton about the codebase we're super internet involved. We have, we can have people do first line treat others first line defense and really help us bring this green line down to something that's more in the day. Rather than be control, okay and then Brian, you said the time metrics traffic. You want us now.

H

I totally believe your story by linear, okay,.

E

So I actually haven't spent much time at a time metrics craft. Can you switch to a monthly short.

H

Okay yeah, so here you can see the blue I believe is the approver time. The overall time to merge did not change radically positively or negatively I think there may be a way to also change. This is last one year: yeah I think you can click on that and select more time.

H

Thanks last two years yeah, so you can see there that the time to get stuff merge did not change radically positively or negatively. When we added approvers, which I guess is not bad at least I was hoping that adding more reviewers would actually reduce. Latency did not seem to appreciate reduce latency, but it did not appear to make things radically worse. Either. I mean part of the intent of adding approvers was to enable us to be a lot more open, with adding reviewers to get more people involved in the process and more people.

H

Reading the code and ramping up to the point where they could become approver. So yeah I think that mentoring process is where we need a lot more deliberate focus, so we can get those blue bars down and if the blue bar is high either the approver is not being responsive or it's requiring a lot more iteration after the initial review, it'd be good to get more data on that. So we'll be looking to figure out how we can get more data yeah.

E

Roughly speaking, I think the blue bars now represent the amount of time it takes. Our CI infrastructure improved to emerge means the pull request, has all the right labels applied. It should be ready to go in, but for whatever reason it hasn't yet made it there. So this could be assignment as flaky I, think I think I heard somebody else wanting to share something.

B

H

Yeah, but so that, so that's where you know it's important for everybody to look at these graphs and figure out what can we learn from these graphs? What can we not learn? How do we get the data that we need to understand where our bottlenecks are like? How many times do tests have to be restarted? On a PR after it's been lgt and/or? How many times do people have to rebase things like that?

H

We tried to get data on some of those things in the past, but sometimes it's a little bit tricky like most PRS, don't have to rebase, for example, but the ones that do have to rebase a lot of times right so figuring out exactly what the right metric is and how to present it sometimes takes a few iterations. These graphs took several iterations to get something that what you know at least seemed like they might be useful.

H

So this is intended to be an evolving thing, but we definitely wanted to be able to make it a sort of a one-stop shop for or like the two stop shop, at least for get it figuring out. What's going on and where we need to improve things, absolutely.

E

So there's this sort of thing interests: I'm gonna put my Garrett hat on. If this sort of thing interests, you please come by the state contributor experience meeting, which happens every Wednesday, sorry every other Wednesday at 9:30 a.m. Pacific time. This is a key part of our one, nine kosis figuring out, which of these graphs do you care about, and how can we make them move in the correct direction and yeah I, hopefully didn't steal carrots Leonard's, like I, said I think he was playing that demo and a couple of this.

E

So this is an incredibly awesome site, thanks again for everybody putting it down.

C

All right thanks so much so with that we've got six minutes left anything that people want to cover before you adjourn.

C

I'm posting before you break here, I, just posted two links in the chat. One is for the confirm, X meeting minutes, which also has link to recordings, etc and also the recording of today's Sagarika texture. If you wanted to hear about the conversation about cloud providers and why not so alright, everybody? Thank you. So much have a great rest of your Thursday and I'll look forward to seeing you in a week happy Thursday, Thanks, bye, everybody.