Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 22 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: KEPs: Evolving Kubernetes at the Speed of Trust - Caleb Miles, Google

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

KEPs: Evolving Kubernetes at the Speed of Trust - Caleb Miles, Google

What is Kubernetes? Every now and again we try to enumerate the scope of the Kubernetes Project; but this only represents a snapshot of our best understanding of the value that Kubernetes provides to our community. With the Kubernetes Enhancement Proposal (KEP) Process we have been working on a mechanism to evolve Kubernetes at the Speed of Trust™. Learn what the KEP process is, how to use it, how it connects to the API Review Process and Enhancement Tracking processes; and how we are employing KEPs to solve our own ship of Theseus problem. ¡Todos a bordo!

https://sched.co/MPZJ

A

Loie all gonna be talking about evolving kubernetes at the speed of trust, so this is gonna, be talk about caps. So again, if you are here to listen about a cube flow or some other talk, you are in the wrong place. There must minute mixup with the edge so sorry about that.

A

So these before, were you I'll, say: Who I am and will mostly spend time talking about what the kept process is at least what I would like the kept process to be a little over Eva, where we are today where to go with the process and ways to get involved, and you know acknowledgments for people who helped us get this far.

A

So so Caleb point us I, don't usually do this kind of slide, but this is about me I've done stuff here, mostly in sick release and sick p.m.

A

Yeah network at pivotal working on kubernetes, so let's write into it. What is the the kept process? Well, it's short for kubernetes enhancement proposal, we've modeled this on the rust RFC process and the Python pet processes. The rust RFC be more to be a lightweight process that takes advantage of the fact that github exists and is a available worldwide, more or less, and then you know people have opinions about it. What is that kept process beyond based on these other processes?

A

Some people? You know this is an evolution of the design proposal process that you may have remembered from the early days of Cooper Nettie's. If y'all were here, you know thanks to Erik tune for creating that process.

A

You know there are, if you've, you know, been involved with the criminal release process. Recently, you will have noticed that caps are becoming required to release your enhancement, so in that sense that they also serve as a kind of launch coordination checklist which would be familiar. You know to you if you have a job at a big co, but you know for some people any kind of process at all is just as mr., via since I helped create this process. You may imagine I, don't feel necessarily that way so I mean really.

A

This is me trying to sell you on what we saw, how I think about the kept process, so I think of kept as kind of a precip and we'll get into that in a bit really I think it's a way that we as a community can solve the problem. The dynamic problem of what is kubernetes. So that's in as a thought experiment is chip. Apiece is kind of that metaphysical kind of pattern, and you know it's a way of keeping everyone in the loop, because you know as humans.

A

We all like to be consulted before things change. It's super human nature, so preset I'd say you know, definitions on the slide there general rule, but really it's a which I think about it is a way of thinking. What are the questions? You should be asking yourself when you're about to or you're thinking about shipping a large enhancement to kubernetes it's a kind of a help by a guideline to help you surface common problems, we've experienced in the community. You know we certainly are not I, guess trotting brand new ground.

A

We stand on the shoulders of giants. You know other large communities, the OpenStack foundation have given us a lot of information about kind of what works and what does not what you should be concerned about when you know you're proposing a feature or if you are an operator when a new kubernetes release coming out kind of? What are you concerned about and we want to in an ideal world surface a lot of this stuff in the open, the open source project itself.

A

So it's available to anyone and it's not trapped as a you know in a silo someplace it. You know a large a large corporation, because we raise a community project and about y'all here we you know, wouldn't be able to do it so I really, like you know, I went to a wonderful talk in San Francisco from a former DARPA program manager. She was talking about her work in synthetic biology and she opened her talk with a point to the Kyle Meyer activism.

A

And hopefully you can read that in the back there, but this is kind of least in the early days of DARPA and if you're not familiar with DARPA, it's a military research agency in the United States that guess some of the things that famous for the lasers that try to uh you know intercept missiles, the internet, kind of DARPA, greatest failures and great sense.

A

But what I really like about is like you of the first point, and this speaks to you know: Russ Cox is blog post towards go to when you'll be talking about foreign open-source project. You know as a maintainer. You may not be familiar with someone's use case and it's really important to help motivate. You know. Why are you trying to make a change? So we can all understand and a you know agree that that's within scope or not so that's kind of like what is kubernetes. This is a question.

A

We have to ask ourselves all the time, because you know it's changing and evolving all the time. So this is this is this is something I really love.

A

Now to the shipper pieces, it's it's a thought experiment! It's you know. The basic premise is, if you replace all the planks and a ship, is that the same ship more or less right? So you can imagine for a software project if you change all the components or after the 10 years. Are we still talking about kubernetes?

A

Don't know, but you know, cap is a way of trying to find that out, because you know we try every now and again to enumerate the full scope of what kubernetes is, and you know one that takes a long time, because what could be days is depends on how you're using it now some people are using kubernetes in their retail stores. Some people are using kubernetes out on oil derricks some people using kubernetes. You know at cern and now those were my favorite talks. It's actually here.

A

You know CERN engineers talk about the data science they're doing with both communities and OpenStack. It's amazing how many different use cases there are, and so, if you ask you know ten different people, you'll get ten different use cases or what's important to you, and so this you know this. You have to forget other sit figure all this out as a community.

A

And you know it goes to a basic.

A

You know basic need for humans to feel like they are involved in decisions which affect them, we're a very large project and if you try to keep up with all the goings-on, I'm sure you've felt that there you know it there's just a lot going on all the time, and you know you want to have as many opportunities to get as many eyes on a proposed enhancement as possible. You know the bugs are only shallow if people are able to see them, it's the important part there.

A

So that's basic you meanie that we're trying to do at scale there well over a thousand contributors on our regular basis to kubernetes, and you know it is very hard to get give everyone aligned and moving together.

A

So kind of what is the state of hips today, people are writing caps. That's awesome! This person who worked on this quite a bit. It's really exciting, to see people, you know using something you helped come up with we've extracted last year, cap content from kubernetes community, which, if you subscribe to that repository, you know it's quite noisy, and so this is an attempt to help get more better signal-to-noise ratio, people who are just interested on and enhancement so they're now at kubernetes enhancements.

A

Unfortunately, there's you know there are some limitations of our original design, a single flat file, especially if you look at some of the cue ADM caps. You see what, if you're trying to include like diagrams there's, it's it's really challenged and there's no easy convention to do that, and it ends up making things like searching and sorting much more challenging, especially because metadata is embedded in that single flat file and if you ever tried to extract structured text from an unstructured, larger mass I'm sure you can agree.

A

It's a challenging because of all this one because of the size of the community and for how many different audiences need to look at caps. You know there's the cig, that is sponsoring a cap, there's SiC architecture, which has you know, purview over large API, really any API changes to kubernetes there's the release team, which has to understand what risk is being introduced in any individual release both for consuming and, as you know, new lease goes out the door. This is the number one.

A

Question least yessir ease that I've had the pleasure and privilege of working with or wanna know. What's in the release, how do I know if it's working I don't know if it's not and what is append on and what depends on it? And these are the questions that I'm sure everyone who operates any software of any kind wants to have answered, and there need to be a place for that, but, unfortunately, because of the various different audiences, it means that the template itself is rather long and it's right now.

A

You know: we've added also a actual release checklist and it's really kind of one. We lead to work with them so that because there's basically no tooling it's you know it's it difficult to get this stuff sponsored and out the door, but you know I've been working with some others on a kept kuddle prototype that is under development. You know, maybe say more about that, but you know talking about where we want to go, and this is really about tooling. So we've got the process and that's worked well enough.

A

There's definitely pain points, there's definitely growing pains. But if you look at you know, we agreed on a second cap that defines where we want to go. So it's kept one a that has us moving from a single flat file to a directory structure. It has more structured metadata, that's separate from that flat file, so you can actually operate on it safely with you know, tooling, and consume caps and I made an automated fashion.

A

So you know we have the community site, which is a kugo generated, static site and it'd be great to be able to surface like new caps or caps that are being opened for you know a month without any comments. One of the things I think the Russ project mentions as being important for helping keep development active and the project moving forward is having a final common period and there's not an easy way of doing that.

A

If you can't easily ascertain when a cap was created or when was last updated, so just a lot of a lot of life cycle and you know Good Housekeeping kind of stuff.

A

If you spend time in cig architecture, you will know- or you know, follow their mailing lists- that reviewer bandwidth is really at a premium and it is every release of a release.

A

There are, unfortunately, people who just get blocked and aren't able to get their enhancement in because there was no time to review it and so making that process easier and allowing that review process to begin earlier and also trying to surface kind of the workload of maintained errs is another important design goal for the next stage of, because now people are burning out, and that does not make a sample project by definition, but also, if anyone has any questions, I mean feel free to stop me with looking be plenty of time to ask.

A

You know all the questions you have about what caps are or we're going what's what's going on, also getting the automated integration with the anthem attractant process. So if you try to ship an enhancement in either 115 or 114, you may have found your confirmation that you're doing a bunch of copy pasting between your cap and B enhancements, tracking repository and so there's not any good reason for that. I can't tell you that you know you just love it.

A

Everyone loves copying and pasting, they don't and so picking making reducing that kind of friction is another important goal for the project.

A

And you don't wait to get involved so today, one of the real challenges. If you look at the cap contents, there is not always enough attention paid to explaining in prose why enhancement is important. Who is it for, as this gets back to the the haislmaier calculation? That part is super important way before you get to you know the architectural design you need done. You need to get agreement as to why something is important, and this is a great opportunity for people who maybe aren't involved directly with the development of a proposed enhancement to contribute.

A

You know in other ways not all contributions are code and without all the non code, contributions again would not have a project. Certainly, if there's no documentation, there is no actual release.

A

If you are a kept author, one thing that's really helpful is if you can clear out the templates sections that you're not using so particularly the release checklist. If you're not releasing your kept immediately, you know that's really cycle. You should probably just remove it because it makes it difficult for people who are following along to know what is active or not, because you have to already know what the template looks like to remove it in your mind and just more friction for people. So it's like keeping your caps kind of you know.

A

Lint-Free in that way, would help a lot. If you know once we are moving to a directory structure, this you know we're getting a lot of help in making this content available and easily consumable by consumers. So we use a hugo and many other parts of the project to do this kind of work, and certainly I have not had the time to explore how that is going to change when we move from flat files.

A

We have today to directory structure so yeah, there's that's a super Help Wanted, general kugo experience or if you've got you know your favorite that excite generator you know and container is generally fine, and you know if you download the slides here, you can get a link there, but there's a preview version of the cap cuddle tool, total prototype does not.

A

Does not work, it's not fit to be released yet, but that is you know where I've been a lot of time, trying to sketch out a boat like a prototype that we can get in front of people get in front of maintainer x' to start playing around with I've gotten a little bit of feedback from what I've shown like you know develop thus far, and if you were at KU con Copenhagen, you may have seen a demo from the JSON I showing that work at that point after some additional feedback, we start to make a bunch of changes that kind of reduce the handoffs between a cap, author and I, can't maintain so trying to also introduce more productivity enhancements like automatically having the pull request file for you, that's kind of what we're trying to go.

A

You know in this next generation. So that's taken a bit of time, yeah trying to coordinate both get and github operations kind of tricky. Some.

A

You know it or it has taken quite a lot of hard work across the project to get to even where we are today, and you know, I would like to opportunity before we get into questions to say. Thanks to the you know, a lot of people who have made this possible. So first I would like to say thanks to Jase Jase I, don't know if you are fortunate enough to know him, but he has been project since the early days. He came in from base for the Microsoft acquisition and he is an amazing person.

A

He's he's one of the most thoughtful people. I personally know he thinks so much about how planning should work and how to deliver value to so you think about all parts of that from testing. Well from how do you describe a change to? How do you document a design proposal to how do you get that any write code that implements that? And how do you put that cut under test how'd?

A

You get that code in the hands of other people, just the whole software delivery pipeline he's a is fun I like to thank my old boss, Brandon Phillips, who sponsored this a lot of this work.

A

You know it's, it is not easy and certainly not when you know we were a small scrappy startup to have a person working full-time on project. Health initiatives is like a strategic project improvement. You know, work stream. That does not have any. You know immediate commercial relevance to us as a start-up and to have that kind of foresight and vision. You know it's it's you thank. You also did Joe who's, my co-author on the the cap Rafa's itself, even Augustus. He did all of the legwork really in getting the capital 1a.

A

You know alignment and approval to get us to even starting to work on addressing some of the problems with the first version of cat Philip would Rock another good friend of mine provided a lot of books. Some helpful and engineering review the first prototype that we demoed at Copenhagen, Aaron Quicken burger, was instrumental. There was his decision to start making EPS a required part of the release process, and so you know really grateful for the work in to adopt and to really extend the process that you know still still in its infancy.

A

So he did a lot of work on that, so Brian grant and I had a bunch of conversations about trying to establish kind of provenance for any enhancement. This is you know again. You know. Every operator that I have ever met would like to know what the why and a particular enhancement landed, because, if you're looking at this after the fact, do we hard to piece this up together, especially over you know, hundreds of commits and pull requests that go into each into every release.

A

Tim Hawken also gave a pretty useful feedback about trying to further improve kind of the the productivity from a developer standpoint to doing things like making those pull, requests early and minimizing the handoff between a cap author and a cap reviewer. So that was really helpful feedback. Eric toon also provided a lot of feedback on cap 1a and as the original author of the design proposal process, his input was invaluable and trying to iterate on that for the first version of cat and the whole Cooper community who's had to put up with it.

A

You know us rolling out a new process as we go.

A

Awesome so okay, I guess we are just are out time. So I've got this and producer.

A

A

Her release, it varies it's in the dozens yeah, so the anthems ever release good summer between mmm 10 and 20, ish Blanding.

A

It well when, as caps have been becoming requirement for releasing it's gone up. So that's one of the one of the texts there.

A

You, the question was any question or any integration plan with dev stats. I mean yes, as a community's repository there's, you know the ability to do that. I haven't thought about it. That's pacifically, so I don't have any plans yet, but it is a possibility to get more stats on contributors there.

A

um One thing that would be really nice and one thing we're trying to do is they have the ability to associate individual arrests against, like Tribune a disturber Nettie's or, like some other repository, put the cap itself to try and kind of roll up all these like disparate contributions into one larger kind of epic, or something if you're familiar with kind of the agile terminology.

A

A

Yes, so um cigs have generally or seem to have generally adopted either the kept process wholesale or have a some modified version of it. So cluster lifecycle for cluster API they're writing a bunch of caps. They store those outside of the centralized kubernetes enhancements directory some other tags. You know they're free to do that as well. It's not a top-down. You have to use this process. We want to make it useful to the cig.

A

So cig architecture is one of the I guess larger proponents of it Sega release it's mandatory to ship now, so you are going to I. Guess it's going to spread virally like that.

A

A

It depends per sig, imeem I, you know, I've talked with some maintainer. She want to see. Basically, if it's anything larger than like a two line, pull request to see a cat. It really depends for you know for something: that's you can API almost. Certainly you will need a cap. It really depends on your sick and SIG's are free to decide their own criteria. It's not yeah, it's a again a pretty decentralized or federated model.

A

So the question was: what is the minimum bar to have a cap versus normal PR and it depends on your sig, but yeah I feel that cigs are moving towards more and more using hat.

A

A

So the question is: how long does a cap live and it's supposed to live throughout the entire conception and either to retirement or even general availability? So the caps are supposed to be a living iterative document that you update, frequently as changes come in and one of the real challenges we've had as a community is coming up with good graduation criteria, so moving api's or new enhancements from alpha to beta to general availability and the old days now used to be.

A

If you were just in one release as alpha more or less the next release you could be in as beta, but now the project has grown so large. We need more. You need to be much easier for contributors or just and interesting people in the wider ecosystem or community to be able to do their due diligence what's being used. How are people being? Are people successful with an enhancement before trying to graduate it through general availability, because there's a lot of em I?

A

Guess it is so if you have been through the TPR too Ciardi conversion I guess trauma that is kind of what I'm talking about. So if you bet on an API and you're not tethered to its usage before you rev it, that can be really painful for your users and so trying to avoid that in the same way that go has a bunch of experience reports doing that same thing on an enhancement by enhancement basis. So you can actually have good.

A

You should base criteria like it would I really struggled to believe that something you go from alpha to beta, without at least you know, a few documented uses, but people being successful. But how would you know have you gotten feedback and it's important at least to me, to get this information out of the silos where they usually work and make it available for everyone, because that should not be a competitive advantage? How much you've used kubernetes, listen.

A

That is it go to the keynotes take off coming.