Kubernetes SIG Architecture, 12 Jul 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Architecture 20180712

Description

http://bit.ly/sig-architecture
11:12:21 From Joe Beda : "Mostly" is so absolutely on brand for Brian
11:12:51 From Joe Beda : Jaice is an expert cat herder.
11:13:06 From Timothy St. Clair : Here-here! Thx jaice!
11:13:43 From Jaice Singer DuMars : meow!
11:13:48 From Jaice Singer DuMars : thanks
11:16:50 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : Is that an Emacs flag behind you David Eads?
11:17:05 From David Eads : someone else's
11:17:10 From David Eads : vim for life!
11:25:07 From dwat : +1
11:31:34 From Clayton Coleman : some examples
11:31:35 From Clayton Coleman : https://godoc.org/k8s.io/kubernetes/pkg/master?importers
11:31:39 From Clayton Coleman : 218 people are vendoring this package
11:31:46 From Clayton Coleman : some of these are forks obivously
11:32:17 From Clayton Coleman : we really need better data on what the community is doing
11:35:26 From Clayton Coleman : shame GitHub blocked general file search
11:35:38 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : Clayton, insight beyond the import is something I hope we can discover with the APISnoop KEP
11:35:42 From Matt Farina : https://github.com/kubernetes/kubernetes-docs-zh
11:36:20 From Christoph Blecker : 🎉
11:36:46 From Clayton Coleman : haha
11:36:47 From Robert Bailey : Conscription
11:36:51 From Clayton Coleman : that look was great Ken
11:37:28 From Bob Wise (AWS) : That was a great look!
11:39:05 From Clayton Coleman : raises hand
11:39:22 From Robert Bailey : higher level tools want to just vendor the config types for the system components, e.g. kubeadm
11:42:39 From Robert Bailey : Every team I’ve talked to about adopting component config has been waiting for the kubelet implementation to pave the way
11:45:25 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : Deep Insight into what the community is doing: https://github.com/kubernetes/community/pull/2363
11:45:37 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : https://github.com/ii/k8s-community/blob/650700c2ddb1b90788cea894ba86937de2a137c4/keps/sig-architecture/0016-20180601-identifying-api-usage-patterns.md#summary
11:48:25 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : https://github.com/ii/k8s-community/blob/650700c2ddb1b90788cea894ba86937de2a137c4/keps/sig-architecture/0016-20180601-identifying-api-usage-patterns.md#api-interaction-identity-who-are-you
11:48:52 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : https://github.com/ii/k8s-community/blob/650700c2ddb1b90788cea894ba86937de2a137c4/keps/sig-architecture/0016-20180601-identifying-api-usage-patterns.md#api-interaction-purpose-why-are-you-here
11:49:43 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : https://github.com/ii/k8s-community/blob/650700c2ddb1b90788cea894ba86937de2a137c4/keps/sig-architecture/0016-20180601-identifying-api-usage-patterns.md#how-do-we-communicate-these-larger-concepts-of-identity-and-purpose
11:50:31 From aFTaOrdmj/54qAkTN5Q4OME1RU31+rUfvfga7zW5KfwSkPL3gmc2DLDZnuBNOKpfJoch7ru9tL9TPE1LaiaPvw== : https://github.com/ii/k8s-community/blob/650700c2ddb1b90788cea894ba86937de2a137c4/keps/sig-architecture/0016-20180601-identifying-api-usage-patterns.md#tying-it-all-together-how-do-i-turn-this-on
11:51:32 From Clayton Coleman : this sounds a lot like distributed tracing - wouldn’t that be a better pattern for us to be investing in? what’s the difference?
11:51:56 From Clayton Coleman : also what’s the difference between gathering stack traces from components (like perf tools)?
11:51:57 From Jordan Liggitt : suggestion from audit/sig-auth was to submit an audit id with the request
11:52:04 From Jordan Liggitt : and correlate that with client side logging
11:57:23 From Jaice Singer DuMars : we have ~2 minutes remaining
11:57:30 From Jaice Singer DuMars : we probably need to table this
11:59:14 From Matt Farina : yay Tim
11:59:17 From Matt Farina : enjoy your time off

A

All right welcome everybody. It is Thursday July, 12 2018, you are in the kubernetes sig architecture, meeting I and my name is Jay: singer qumar's, not sick architecture, as it may say, in the thing there so I am your host I will take Google and I will be hosting today's meeting. So the agenda, if you want to follow along, is available via bit links. So--But got Lee / sake.

A

Architecture is where you'll find that and we're going to go ahead and dig right in so one of the things I have been working on is to try and create a way to track all of our work in a manner that is traceable and also prioritized. So people understand that not only do we have an agenda, but we also have ways to to prioritize things on the agenda and track them over time. So the three main concerns that were we're tracking right now, I'm gonna go ahead and share my screen here.

A

So everybody knows what I'm talking about.

A

So can people see that definitely, okay cool? So let me pull up here so this in the the agenda I'll see a sinkhole kept tracking review. Essentially what this does is. It provides us a way to differentiate the tracking of keps from an administrative standpoint from the work to actually get the caps done themselves. So one of the problems we have across the project is that we have commingling of the project management, part of things and also the actual technical discussions and content.

A

So what I'm trying to do project-wide is to create a standard by which we can separate those two things out and actually have project management in kubernetes, which is desperately needed. So I did this first for the steering committee backlog and it seems like that's going to be very effective and next is to this was some of the things we're doing so they're, essentially three core projects here, that we will be tracking so API reviews and that will be tying into the API review process.

A

The kept tracking, which is tied to the kept process and then conformance test review, is simply conformance tests that we probably have a stake in and should understand the the impact of those and, if their architectural, a consistent, because conformance is one of our core concerns, so so just moving along here. I'm we're not going to go through this today, but in the future, I would like to quote unquote: walk the board.

A

If any of you've ever worked in the agile shop that had Kanban, you essentially just go through things they're in and process, and and then look at how how they're doing and what's in process so, for example, it caps. If we're, if we're actually going to review one in Sagarika texture, somebody would hopefully drag that into column three or which does need API or architectural review. So in the future, we will look.

B

Those over Joe, so one of the things that I'd love to to figure out here and I think this is so like one of the ideas at least I, don't know if it's proven out. If we're doing it, you know, is that check in early with the cap so that we can actually make the technical discussions be isolated and we don't end up with these, like issues that won't load gets locked in right.

B

How does that mix with the idea that so much of the tracking stuff and github is actually tracked by issues right by issues and and by PRS right, because to some degree we want to track caps in terms of what state they're in, but that state is not represented by I appear now we may actually say: okay, you know if you want to get a review, submit a cap to change the state from from you know, from what is it pending to implementable or whatever we decided on, which ends up being the sort of approval bit to some degree?

B

Maybe that's the point where we review it or maybe there's just a placeholder PR. That's some of the stuff that I'm wondering how you're thinking how that works. Yeah.

A

I had imagined that if somebody makes a material change to a cap that essentially goes into the the kept tracking backlog as a new, it's a new issue, although these these issues should essentially live for the life of the cap, so we can do it either way we can just move like. So if something is over in progress and it gets stalled, we could potentially drag something back over to review or any of those things. So if we want to.

B

Move like okay, okay, so these are. These are essentially project tracking, yes, Hugh's, not the PRS for actually getting the things in correct.

A

This is a totally. This is a completely project manager board. This is somebody who has no course in the race in terms of what the cap is doing, we're simply managing that. If things need to be review, somebody's got to do this and say hey wow. This.

B

Thing needs to be looked at, so these are kind of the parent issues for the stuff. That's actually happening in the repo, as actually gets turned and discussed right.

A

Exactly yeah, so when somebody adds something new we should be, you know, reviewing those and then moving them where their current status. So these these should live as long as the thing is merged or or is being forked, and they appear even before they're they're really vetted. So it's one of those things where there may be a little bit of duplication of effort, but you know either they have a little more cruft and we can start through that. Then totally miss things. No.

B

It sounds good, sounds good and by the way, I see that, like there's two for teens there I like our flying time on a weekend at some point. If we can get some like and get like a test and excellent like more than four there's I know, I know it's it's five, six yeah. Some of this is because we don't check in early, so these things hang around and then, when they do get checked in nobody wants to actually sort of risk their thing by chain. Turning it. What different issue yep.

C

B

A

Funny thing is during your probably working on kap-40, all of them so yeah, so here we yeah. We can definitely easily see where there's duplication, I did put dupe after the ones that are duplicated, so that helps us figure that out, okay, so then yeah and so my my two grand vision, it may be totally aspirational or delusional.

A

You can pick your adjective is to have all of our all of our big level, projects that, hopefully, at the sink levels tracking in a similar manner, so that at any at any moment you can look at the the major work being done by a sake and understand where your thing is at in that as a saying or as an individual, so that that's something that I'm hoping will come to fruition over time and by the way.

A

This is something that should be applied to features as well, because features are long lived and have States and all that stuff. So I'm gonna push for this to be part of the the release, slash feature process as well, so we stopped having spreadsheets out there that people don't know about into an update or are out of sync with github. So let's make github the record of note for everything that this project does and is important. So if people can actually look at it, okay going back to the agenda.

A

Oh one last thing: this draft document here take a look, give it a last once-over, there's a few things that they need to address in it. But essentially this is the API review process that will feed into that project board for API reviews, so we're hitting attraction got mostly LG TMS on on the core stakeholders, which are the actual people who do the approvals and there's a lot of commentary in here just to read over.

A

But if you have something at it, I would request that if you have major sweeping changes, please don't because this is really our first crack at this, and you know if there's things you want to bring up as aspirations for future process stuff. There is a section at the bottom called aspirations, so I would love to have anything that you see missing added here, as opposed to trying to knit the other sections so I just.

D

Want to say, Jace thank you for pulling all this stuff together, both the review backlog boards and and this API process. Thank you. Yeah.

A

Glad to do it and honestly it's selfish, because it makes quick, easy best, sort of work, lazy, sysadmin, hat work; okay, so let's go back to the engine, so we have. Our first item is brought to us by the esteemed Matt Farina. You want to take the helm there buddy. Let me know what this is sure. So thank.

E

You thank you. So when kubernetes 1.11 came out, they there was generated code that was no longer checked into kubernetes kubernetes source and folks who went and vendored kubernetes kubernetes to get two packages say in the package directory their stuff started breaking and that caused problems for a handful of applications shortly after as soon as they started to try to upgrade for 1.11, and so the question became. How should we handle this situation now?

E

No, in the short term, we are working on taking that checked in code and bringing it back into 1.11, hopefully that one, it does have a do, not merge on it right now, but it is being.

D

I approved it this morning, I don't know what the Doudna merge is about. I managed.

C

By the patch release branch manager, okay, they've got they're just directing traffic on the branch.

F

E

Perfect I asked about it, and I haven't looked back. Okay, so so there's that for that that, but this does bring in the long-term implications that are probably worth us starting to discuss, is kubernetes kubernetes meant to be vendored, or should everything that is venerable? Basically, all the packages be moved elsewhere and we've started to do that with staging.

E

This is going to have an impact on things like this, so like Vigo is moving forward and Vigo assumes that everything will always hold to semantic versioning and the go API and nothing should ever break from it. You know you can never set things like a max range like don't go higher than 1.11 or 1.12 when you're importing. It doesn't work for that because it assumes you will always never break semantic versioning, and we do that with our go API in every minor release.

E

We don't follow semantic versioning in the go API, which means importing it with the direction of Vigo is even going to lead into future problems. So I wanted to raise the question of how do we start handling these situations? Do we say not generate a code in kubernetes kubernetes but everything's out staging elsewhere and we keep the generated code there so that way imports work.

E

You know how do we just approach this situation and, if we're breaking from the NGO community norms than to be explicit about that, so that people who consume things can kind of have an idea of how they should do it.

E

So anybody with opinions out there have thoughts.

D

So we've never said that kubernetes kubernetes was supposed to be ven durable. We know it's not go gettable right.

E

Go is a language states out of the box, though, that everything go and its public API is always dog edible. So if we.

G

Break from that convention.

E

We should at least communicate to people we're breaking from that convention, so they know it right that.

C

Hasn't been here since one seven, actually one seven was when we stopped checking in everything that you needed to build and you had to run make and make would generate that.

H

Yeah, although I don't think I, don't think that we should intentionally work in such a way that you can't work with us with the lengths normal language stack that if it's hard and if the API changes and if your code breaks you know fine, some things are hard but making it intentionally making it impossible to do because, and we don't think you should do that- doesn't seem very community friendly.

D

So I don't think we're doing it for philosophical reasons, I think we're doing it for technical reasons and those technical reasons are rooted in at least historical truth, which was checking in generating code, was an enormous pain point for conflicts and for correctness, people who would run their tests against non regenerated code and get passes and then upload it and it would fail again see I write so.

H

A fairly frequent contributor I have found that the conflicts that I get are largely caused by nasal. They are not caused by by most the other files. They occasionally happen. You occasionally have to regenerate, but it's it's not the problem that once was it. So you know I'm looking at it saying it's a solution that doesn't really help me much on the development side, and it doesn't really help me much. It wasn't me at all on the consumption side, I'm a consumer as well right worked for Red Hat. We have OpenShift.

H

So it's a technical solution to a problem. I don't seem to have.

D

Yep and we're sort of rehashing the email thread right. It's certainly not as dire as it once was, and so the question is: should we rethink the strategy for the last year's? The general directional strategy has been to move generating code out of the checked-in set.

D

Do we need to reevaluate that so.

E

I would just like to jump in that the go standard is that generated code is checked or unless I shifted a little bit to automatically call and go generate and doing it that way, if it's something we can reevaluate, should we look at following along with what go intends and what go developers expect I.

D

Saw Matt's hand up was that the same thing Matt? That's what I.

H

Was the one who just.

D

E

I saw Jordans.

C

D

Jordan's hand I.

C

Was just gonna say to your earlier point tenet that some of the stuff we have moved out of the repo like documentation stuff doesn't generate any more. No.

A

C

It generates out some stuff we stopped generating all together, so we switch them like you, gorgey.

D

That was, though, probably the single biggest change. We made yeah.

C

David I think there are two files.

E

C

Are not being checked in one? Is the open, API file I think if we started taking that one in again, that would quickly become the biggest conflicts, but I said to me seems like a problem with that specific file like the whole repo combines into.

H

One arrival yeah.

C

H

C

Files, so I I think we could probably address those two files and make them not be conflict targets, and if that would make life better for people getting the repo and building it with standard, go tooling. That doesn't seem bad. So.

B

Are we talking about the swagger file there.

C

Are two files I'm aware of one? Is the vendetta and one is the open API generated spec like this mega spec file.

H

Right, no, it's a go file that gets generated and it really does look like one where you would want to split it into pieces, one per API group and then work off of that it yeah.

D

I mean it was that way, actually at some point it used to be that way, and then it was moved into a single file and I forget the reasons for it. I.

H

Don't have a clear memory.

A

So I feel like this is one of those issues that there's kind of opinions on both side, and it seems a little bit hard to reconcile. How do we, how do we actually get to a point where we can make this decision I like.

C

Like Tim said, I think the original reason for moving away from it was pragmatic. It was these things were rebased magnets and it was slowing down project velocity, and so, if it really is just a couple of aisles and we can make those workable and not reintroduce those rebase problems, it seems like that would be worthwhile.

C

Another alternative could be that once we cut a release branch, we check in the generated files, because at that point velocity slows tremendously.

C

That's not it's not exactly the best solution in my mind, but it would certainly solve the I'm bumping to the next version of cube and I want to bender it. If someone wants to vendor for master I feel less bad about them. Having to run development file processes, but I, don't know what do people think I mean.

D

Another option would be to say you know: can we break out anything that people are likely to vendor in two separate repositories that are go gettable this stage evolution and leave the generated stuff? As the like? You really don't want to vendor this I, don't know if that's really practical or not, there's a lot of generators so.

A

I

The the I mean if we were toward it, we could definitely make package Venerable like we could make it go gettable, and if we spend the effort to do that, what's the win, what's the benefit I mean? Is that something that we really want people to be vending, like, generally, isn't the idea that you'd want people to use a client generated off of the open, API, spec or the vendor to go client in order to interface with the system as opposed to venturing out individual components? Well,.

E

That you know I'm gonna jump in here. That really depends on what you're trying to do. Those aren't the only bits that people want to use or are you in practice, and so we shouldn't assume which parts of the package people want to use. I'm, sorry I think I jumped on David.

H

Yeah yeah, so again, you know: I depend on a lot of pieces inside of cube I think there is a difference between you saying: here's, the thing that you should use. We will maintain compatibility on this or we will. We will feel bad if it is really hard for you to use something like client go and we won't feel bad if it is a little hard or the API keeps changing.

H

If you vendor something from cube cube package, there's a difference between saying that and saying you know what we're flat-out, making it impossible for you to vendor this without you going through and then changing the code running a build on our local system, generating the files and then carrying shims that you can't use with standard been during tools like glide or death and removing the generated code says the latter not former. So.

I

Right it does. The question is: if we put in the effort in order to again make it venerable what news cases are we supporting?

I

So if you vendor out of controllers, for instance, or you vendor out of couplet, and you try to build a stable product on that you're, probably going to be in a world of hurt, no matter what.

H

Built a stable product on it, ran it for years.

I

H

It's not zero energy.

I

Yeah, there's a Loa there, no matter what you're gonna do unless you shouldn't do it I just think that we should be clear on the use cases that we want to support before investing effort in doing it, and then we'll put some fire out.

C

Jordan you want to take it instead of me. I was just going to say that checking in artifacts, so you get a reproducible build without, depending on a lot of pride. Specific build infrastructure, I think has value. So if you can get the go source and run go build and get they build. I think that has value we got Matt and then Tim well.

E

I was just gonna say you know we are having this conversation, because things that aren't inclined generated are already being used by people in their applications. It's not whether they they should or shouldn't it's that the state of the world is today, people are doing it and they're dealing with the upgrade of kubernetes versions to keep this stuff going, that's the state of the world and they found useful things in there and they found processes to make it work.

E

That's kind of the state of the world today and the question is, is if we're not going to support them, we're saying we're not supporting part of the community who's doing this, and we should say maybe we're gonna change the support and you need to use a different route to do those same things but I mean part of the community is doing this today. Just just as is that's reason. This whole conversation came to up when some generated files went out everyone, my stuff broke, we're having issues.

E

I need to get this done, and so it highlighted from a real-world concern. So, let's think about how we move forward from where we're at rather than what people should or shouldn't do, because that gets philosophical I'd like to maybe keep it practical to solutioning for people and.

H

It's also not the case that we are we're trying to add things to make it easier. It's the case that people are actively spending energy to make it harder and.

D

Tim well they're, not spending energy to make it harder, making it harder as a side effect of spending energy on other goals, so I actually agree sort of, fundamentally with what Jordan said before. If we can have a reproducible bill, then why wouldn't historically, the problems I think were twofold, at least as I perceived them. One was the constant need for me basis if that has not gone away as Jeff and I have Jeff's on the call.

D

Jeff did some analysis on the thread and showed that some of the top, the top couple of rebased targets, are still Zee. She generated not stuff, though there the numbers are much smaller than they used to be so there was the problem of the rebase --is. Maybe we can solve it with respect to open api. Somebody needs to go figure that out and bendata.

D

The other part of the problem was that people would forget to regenerate it and they would push their code up and then they would get reminded that they have to push their stuff and they would invalidate all their testing. We've seen I've seen that, from from the reviews that I've been doing, I've seen that get better since running, make automatically regenerates. People don't need to remember to run pack update generated star, and so that's been better.

D

So honestly, if we move more of the generators, there's I think four or five left them, but then aren't in that same mode, move them to a place that automatically rebuilds when you, when you type make or when you type basil, right, Jeff and leaves those artifacts in your trees, so that when you run git commit a you see, hey man, I've got these changed files.

D

That might be enough for me honestly.

A

Well so so, whatever we decide here, we're gonna have to fund with people actually doing this, because the the sad part is it Matt's concerned about this. Actually giving us a black guy in the community is whether people are doing the right thing or the wrong thing for their projects is, is something that we kind of need to address so I'm a big fan of putting something in place. That's a little more immediate and less effort to at least kind of address. This.

D

We reverted the bin data change, and is anybody really complained about the open, API change like I said: that's been that way since one seven so.

A

Yeah, so what I would say is um if we can summarize what we want to do in the the meeting agenda notes. There's a bolded thing there that says decision under this topic. If you can, if you are passionate about actually doing the work and not just talking about the work, whoever that is in this assemblage of people put something down there and let's, let's figure out how to actually make that happen and looks like Ken and Jeff all three strands, the same time: Jeff they go.

F

First, okay, I just want to I guess, pilot I. Think there's like two subtly different issues here that we should probably make sure we tackle both one is. Do we want people like how much do we want to support vendor incrimination kubernetes? It sounds like we want to continue editing, coordinator communities, and so we need to make that great, clear and you know there's parts we don't.

F

Maybe we should be using internal or make it very clear which pieces are not supposed to be, and the other question which is related but slightly different is what do we want to do about generated code and so those I think are kind of different things. We should really decide them slightly differently, they're related, but not exactly the same thing. Okay,.

A

I'm breaking those separate things, I think.

I

There's a third thing that we should also think about so taking one example directly apply so apply is something that was really popular either to vendor directly out of kubernetes. I believe helm was doing that for a while and strategic merge patches. Another thing that's very popular, rather than make that more easily venerable, we decided to move it to server side functionality right, so we identified something inside of a community that wouldn't that people really wanted to use and try to move it server side.

I

If there are things that as we try to make things venerable, if we're going to break this down and say these are things that we really want a vendor and we're gonna take other things and make them internal and say you really shouldn't keep entering those. You should probably try to look across the community and see what people are actually been during and make a decision about whether those are things that we might want to move server side as well. That's.

D

A great point, I I, think we should, if we're gonna, go this route and I'm I'm. Okay with it I say that as somebody with the PR that deletes all the generated code from all the other generators I'm. Okay, with the idea, we should discourage people from vendor and kubernetes communities. We should figure out what it is that their vendor and either like Ken, said moving server site or just move them to other staging repos and let people vendor them from top level.

D

Repos like that, which is a little bit of a segue into a later topic on today's agenda, which is the creation of new staging repos for components and component configs, which I'm a fan of I. Don't think we should go out of our way to make any guarantee he's about stability with incriminating if somebody's ven during package cubelet something something something and I move that to staging cubelet I, think that's a legit change and if their vendor in it I don't care if I broke them.

D

I do care if we actually made it impossible for them to work correctly. Matt.

E

I just wanted to add two things here. One is you know. Many tools like kubernetes version, support multiple versions of kubernetes, so even when we move apply server-side for a version before somebody can truly count on that and some of their tools, they might have to wait for a couple more releases of kubernetes and compatibility across before they can even drop handling it internally.

E

If they're trying to do that, multi version support which, which many tools actually try to do, and so it's not as simple as just saying when we moved its server-side now some of these cases immediately handled, because there's that transition period right and then I just also want to say I like the idea of moving the stuff out to smaller bite-sized repos vendor in kubernetes, kubernetes is incredibly painful and I wish. We didn't have to do then any of the tools I ever touched.

E

In fact, we try to use the the staging repos like client go and otherwise every place we can just so we can see when we could finally someday have everything out of it, and we don't need it anymore because it would be really nice and so I'm in favor of moving things into smaller, more bite-sized repos and making that much simpler. So.

D

I'll put out a plea for all the people who are vendor in kubernetes kubernetes or dealing with tools that do vendor kubernetes communities, help us figure out what the smaller bite-sized repos would actually look like. We've got a few already: we've got packaged, utils got moved out to communities utils and we've been moving out. You know a lot of the API machinery and stuff has already moved out and again it's a segue into the later topics, I'm proposing that we move out all the major components into separate staging repos that doesn't leave a ton.

D

A

Ok, so I'm gonna go back to our decisions, which there are two in the agenda. Please, if you're willing to sign up to actually help this either III would actually recommend that we don't try and tackle this Mallory both immediately simply because that's gonna be a tremendous amount of work and overhead. If we can do it, some other way to start at least in kids, interaction they'll be good. Well,.

D

We're rolling back the major pain point right, so we're back to the same level of discomfort that we were at before 1:11 right.

H

And I would be fine just keeping it there. Okay.

A

Tim and then, let's move on to the next agenda item after this, my.

J

Only request is like a requirement if we're going to make any of these types of changes.

J

To add things to staging is that somebody managed the dag for the dependency graph between these things, because if you touch anything inside of KK, namely anything related to testing, it's literally like a buckyball, you pull one strand, you tip, we touch everything else, so the if it's clear to define the day, if somebody's going to do this work, it should be a requirement to like say like here, are all the dependent libraries that will be associated with said sub-element, and usually that will go into the separate repo to have the glide lock file or whatever.

A

Okay, thank you for raising that. That's a super important okay, moving along Matt yeah! This is the next one. Is yours if we can try and keep this short, that would be great, hopefully,.

E

This can be short. The process says that any new repositories need to go through sig architecture, approval, there's a link to the issue right there for anybody who wants to do it, and this is a shared process thing they stay under the kubernetes or we've already created for the doc site. The Chinese translation one and now.

E

Wherever it's at there's a Chinese translation, repo somewhere and so now, if I could remember, the name of it yeah kubernetes Doc's does eh is how it is here is the existing one for Chinese and now the docs team is requesting the same thing for Korean and Japanese, who already have internationalization teams who want to work on this stuff and are already doing it just under the kubernetes, or any objections speak now or forever shut the hell up.

A

Approved crate, putting it in the the notes.

A

A

A

Alright, thank you for bringing that up at and let's go ahead and go on to the next thing. Tim.

D

So there's a proposal from Lucas I, don't think he's here, I think he's actually joining the Army this week, but.

D

He's proposing to move the various component, config definitions out from the sort of buried places that they're in each place around the code base today they're kind of a mess into a single, more accessible place. That is more easily vendors, see earlier topic.

D

If the initial proposal was to create a new component, config repo discussion on the doc led to the idea that maybe we should create per component repos under staging and start the process of actually encouraging. Cubelet cube proxy cube, cuddle everything else to actually exist in staging repos. That could be split out into bite-sized chunks. The same way that ATM machinery and other things are I like the idea and I would like to encourage it so I.

D

Second, the motion to create staging cubelet, cube proxy, etc and put the config files under there for now. Config might be the only thing under there, but it gives us a way to start testing and we're moving towards separate repos for these components, David and then Jordan. What.

H

Is the distinction between these AP is that we are going to serialize on disk and the kubernetes api that we serialize both on disk and into SED? Why would they not live to gather the.

D

Distinction to me is the kubernetes api s are independent of any particular implementation. It is at least hypothetically possible that somebody could implement kubernetes entirely from scratch. Using those API definitions and they'll be fine. The cubelet component config is really related to this particular implementation of cubelet. It's not part of the user facing API. So to me, it's different.

H

But if I were to create my own cubelet and accept a compatible version of the config to help people transition, that seems like a perfectly legal thing, sure.

D

And you my vendor, cubelet you vendor kate's, that io cubelet config, as opposed to ven during all of Kate's, that IO API is.

A

Its peers take.

D

Not a satisfying answer, I see Tim with his hand up.

A

Yeah actually Jordan had the floor first and then I.

C

Just wanted to braise something that was discussed in the doc as far as having a cubelet repo with config, underneath it as opposed to a cubelet repo and a cubelet config repo, and I think one of the the idea was to make the external config format. So the qiblah consumes able to be vendored and dealt with by people who are wanting to produce config to feed into a cubelet or whatever component, and if I understand it correctly.

C

Some of the vending tools support ven during just a sub package, and others do not do we know what Vigo and DEP would do with something like that. Would you be able to vendor just be config subdirectory of a cubelet and not pull in all the dependencies of the capelet.

D

Who knows Vigo best man.

C

E

C

Point in the chat was fine, yeah.

E

So DEP is only gonna walk the packaged tree that is currently called in use to look for dependencies. It's not gonna grab anything, that's not being called in your code. The last time I looked at Vigo I thought it looked at all the dependencies of a module. Not just the package tree that was called I could be wrong on it and I hope somebody would go investigate it, but I thought it was at the module level it would pull into the dependencies. So.

C

We conceived ibly declare the config portion of it as a module correct, because a single repo can contain multiple modules nested at different levels. Okay,.

D

That's my understanding, but I haven't played LIGO since the early days. Yes,.

E

I haven't looked at it in a while either so I could be very wrong in this I feel like go.

A

Into the investigation part of this, and actually find out just to be safe, okay.

C

Do we agree that one of the goals is for people to be able to use.

J

C

Vendor the config bits so I agree with that. Absolutely that's!.

J

The primary use case so.

C

Let's make sure that nesting it this way would work at least with depth and Vigo if it required us to do a sub module or declare a nested module for just a convicted. That's good.

D

I'll, throw that comment in the doc I.

J

Love this effort, but I'd also want to stress that for a number of releases, both Lucas myself and a number of other people on the cluster lifecycle have been trying to push the component config to be a more supported item across the project, to no avail. If the goal is to treat this as an external consultant consumable as if it were almost an API type, can we at least get some sort of buy-in from the project to push this towards more maturity?

J

So it can be consumed by folks in the community, because there has been a long-standing desire to get these component configs to a state of maturity that we can consume for doing automated provisioning, I.

D

Don't think anybody disagrees that it's valuable I think it's just not high enough priority on anybody's agenda to finish some of the things that mostly just work and aren't really actual pain points a.

C

Great Robert mentioned the cubelet, so tequila config did actually go to beta, so it is out of alpha. It was paving the way in some ways serving as a warning and other ways. Scheduler is probably the next closest. It has an alpha level. Config file today, I think getting these in the right place, both so that the go types can be vendored the way we want them to be and then maturing the Alpha, the beta beta the GA is, is.

D

The road banner also like I, never touched you proxy, because I wanted to wait for cubelet to actually like plow the field. First yeah.

F

So on the separate verses together, so we're probably going to want to make sure we have relatively reasonable API review processes around these, not quite the same level of the cute ones. But these are API as they have to.

D

Deal with backwards compatibility that still affords compatibility they have to fund. They should reasonably follow a set of conventions that are may slightly diverge from the core api's. But for a couple of those reasons, focusing attention can often have better outcomes, but the same reason that we put all the cube api's together, there's potentially reasons that everything that cube admin might want, for instance, should be reasonably consistent, so there's some there's some level of human wise, concentrating these versus separating them might have to bear dividends.

D

I see they were called the API review doc as it stands actually includes these explicitly yeah.

A

That's correct one.

C

Of the nice things about the config api is, is that you don't have the same round-tripping problems as you do, where you have to let arbitrary clients, write them and other arbitrary clients, read them and write them and read them and write them consumed by a single.

C

The target is a single component, which is nice, yeah.

D

I just meant that they're subject to the same process, not necessarily the same standards, yep cool.

A

So, let's go ahead unless there's anything else here that people want to talk about by the way the actually I'm here too I enter the decision. If you can daddle them or descriptive language to the decision, that would be super helpful. All right, opening your favorite so well know great and then actually you're up next and then we're going to move on to ap Acker for ap I, see.

D

Okay, I'm worried that this could take up the rest of the time and I need to punt this to next session. If we have to okay, let's go through API.

A

Snoop real quick if the Acker and then we'll we'll go to the last one. If we have time at least start the conversation.

G

Thanks for that, I dropped a link to the to the PR in the chat, and it's also available and the docs rather than share my screen. I'll drop a link to cap it off in the summary and kind of walk through it a little bit. One of the reasons we're doing this from the conformance working group perspective is to really understand what we is being used by the community. So we can prioritize what needs to be tested, as well as getting some confirmation around.

G

What we test is actually what is being used so I'll, just the like, just a shout out to New Zealand and the Maori here, the culture of the me he and the FACA papa, as far as their really core concepts of identity and kind of influence. This right now, when our applications talk to one another, they just say: hey I'm this little here's, my user agent string, I'm Mozilla I, built at this time right so um when they get together. There's a protocol and the protocol says hey.

G

My people arrived here on this particular canoe, and here is my ancestry. All the way back to here- and this is why I'm here today, this conversation allows identifying the relationships within the community, because we know the the there's reference. There's things to correlate. So if we look through at what I would like to do is enable any application really simply they just answer the question. Why are you here? Why do you think you're here and what are you wanting wanting from me?

G

As far as from the API servers perspective, I go through a bit of there's some overarching API snoop goals which are not part of the cap, but they're. Definitely the the reason why the cap exists.

G

I can go through that. One thing is I'll: go back to the goals for the AP AP a snoop afterwards, but we'd like to be able to do this cluster wide for every binary that uses any of our official libraries and it.

G

What that looks like if we go on to our the proposal itself is really three things. What does it mean to ask the question? Who are you and why are you here? What is your identity and purpose? What does it codified look like and how do we suggest the the creation of that at the time of the API interaction and then, thirdly, how do we collect this information and aggregate it for the entire cluster and, as we do it across our community, to create a large data set for machine learning and whatnot?

G

What's the the message for that aggregation under API aggregation, API identity I want to drop a link there? These are the more technical bits right now. We we say: here's our application and version and architecture and the commit which is great, but things like the Linux kernel included the timestamp and the who built it. So we have a little more information about that particular binary and it's and where it came from, if we go a little bit further on through the to the purpose.

G

This is more in line with the the papa and then the me he's the me he is actually transmission of that identity and this link of a of the call stack and and submitting it with each api interaction and would provide some really interesting data down to each line of code in every piece of every application within our community and be able to cross-reference that, with the response and and the request response and and looking for patterns there and that's, it seems pretty straightforward all of our our languages, that we support, support, introspection and I'm just suggesting we find a way to enable communicating that essentially this last one is: how do we communicate it?

G

User agent has previously been limited to various sizes. We actually got a PR through to create and record that in audit logging- and we initially it didn't, have a limit and I was able to create the data set that I'm going to link to right now.

G

But we we put that at 1k, because it's a it's a traditional limit but I wanted to suggest we look at other cultures may have longer definitions of identity, or we may find a better way and protocol to to gather this information and that's why I wanted to put this forward as a as an enhancement proposal.

G

No and the last little bit is how do we tie this all together and how do we turn this on? My suggestion is that we make modifications to client go and support a variable that says: hey do something different. Tell us a little more about yourself. Tell me who you are and why you're here and this could be enabled for all of the kubernetes binaries as they're deployed and also with an admission or initialization controller for all the applications within the cluster and gathering that could be via audit logging, the dynamic log.

G

What link would be great but we're currently doing it statically, and we could aggregate that I'm similar to how sonobuoy has a URL says: hey? Would you like to audit your cluster and here's, the URL and here's your report so that eventually have a community-wide large aggregated data set to really have some deep, deep insight into what our community is actually doing?

G

I'm looking for feedback to see, is this a good approach or not I'm? Looking for sponsorship, I need a sig sig architecture or sig testing sound like appropriate ones, but them I'm gonna open it up now to discussion and feedback.

A

Tracing and basically try to differentiate that, and and since.

B

A

E

A

Art there and and why not, if you want to just discuss overlaps differentiations between this and say, open tracing or whatever so I.

J

Can I can speak to open tracing because Wojtek and I went down this road in this? It's a epic failure because of the watch mechanics of how kubernetes works. We you can lose tracing insights across the watches, so we actually punted completely on it, both Merrick myself and white- to expand to releases train to make this work and came up to the point where distributed tracing phones over across the watch semantics with inside of it's not a micro service based system yeah, but this wouldn't apply to watches really I mean they would like.

A

I'm watching to fill a cache, but all the other calls would.

D

Be PC style interactions, I, guess I'm asking so just to ask my question: this is a somewhat. This looks like a somewhat novel way of passing contextual information along with API requests, there's already a lot of patterns for that. Why is this? Why is this not more structure like that? If we did a request, ID for requests and it came from clients and their client identifiers? What's the difference and it's.

G

We have turned to make it simple for everyone that has existing clusters to be able to enable it. So we can see what real world information looks like in the same way that we can run quickly to enable so nobly and sonobuoy provides a conformance test to say what is this cluster kubernetes, conformant or not? I want something that simple that anyone in our community can go click on a URL and say I would like to contribute this clusters data to API, snoop, I click on the URL, the good cuddle apply, and it's there.

G

I didn't find an easy path with indifferent tracing methods that would not create a level of complexity that would work as simply as this approach.

C

Yeah I'll be honest: I, don't really I, don't really want to collect and be responsible for this level of data coming from clients, including stack traces from who knows what, with paths and like data through to a server and then out to another collection. Point that to me that both from the information exposure perspective and from a risk of this becoming a bad API.

C

If you, if you want an API server to react differently or respond differently, if it sees a request that has a specific application in the sec trace or a specific version that there was some discussion of trying to enable use of user agents to do things like this years ago. There were this was something with a particular version of queue, control and wanting to adapt the API server to be able to react to user agents in a particular way, and to me that that sounds like an unstructured API that I don't want to support.

A

Yeah I have I have my own concerns about opt in on this, because it feels like a lot of groundwork. That's gonna be applied for something that we don't have a real understanding about what the opt-in would look like, nor nor the opt-out, but also a little worried about expanding the size of YouTube user agent for a number of reasons, scale and other concerns that might have you know deleterious effects across large clusters, and these are just things that have to be ferreted out over time.

A

But I feel like there's there's some relatively significant concerns about that I.

C

G

C

Think the idea of improving visibility to the source of API calls is interesting and something that would probably be useful in our client libraries I'm curious how far we could get with enabling that tracing locally and then submitting like an audit idea or a request ID that you could then correlate. Yeah I.

G

Did look into two ways to try to augment the cluster. You know cluster wide and trying to gather all of those logs and bring them together, and it seemed like a lot of a lot of overhead on only on the other side, the I thought was to try to. If we went to simplify, we could try to make it smaller and have it be a hash, and that was if you go down and the maybe it's an alternatives or risks and mitigation.

G

So that we we can I it's it's we'd have to find a way to automatically create these hashes later to compare them so that we'd have them available. If you were to, for example, for each of the lines inside of the call trace, it's, you could put those down to a hash. You could identify this sequence of the call stack and be able to map that later, so that it was a smaller piece of information. As far as the size yeah, it stay. Yeah.

A

I think I think we're gonna. Do it a disservice by having a truncated discussion on this? What I would say is: let's put this on the mailing list for Sagarika texture and have a more thorough discussion. There I think the the provisional decision that I'm hearing is that this I think people are cautiously curious, but I don't feel like we're in alignment about this being something that the sake should necessarily rally around to just sponsor. Is there anybody who's on the call right now who feels differently than what I just said.

A

All right yeah so that send it to the mailing list. I appreciate one just personally I appreciate the level of diligence you've applied to these these caps. It's much higher and I, also like the fact that you're bringing in cultural and other concerns these things, because I think that's something. Sometimes it gets lost or missing. So I think that's a really interesting and valuable thing to add to the discussion. So thank you for that. Thanks chase, okay wow! This has been a dino mite meeting.

A

Thank you, everybody for attending and your patience and just patient I will see you in one week and have a great week.

A

Tim you're on mute, I, don't know if you're saying something to us, but okay, wait! Sorry.

D

I I want to talk about the my the next topic of pluralization I will be out next week and the week after and the week after that. So so maybe why don't I start a thread on the mailing list instead of putting on the next week's agenda that sounds great I'm gonna be gone next week as.

A

Well so we'll figure it out and hopefully you're going and doing some cool things down, we'll see all right. Well, hopefully, all right take care. Everybody have a good one too bye.