Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2018 (Seattle), 16 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Intro: AWS SIG - Nishi Davidson, Amazon Web Services & Justin Santa Barbara, Google

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Intro: AWS SIG - Nishi Davidson, Amazon Web Services & Justin Santa Barbara, Google

SIG AWS has been a user forum thus far. With K8s 1.13 we are doubling down on AWS’s contribution roadmap to maintain and release code, binaries, features, documentation and tests. We have 6 subprojects actively being developed with the community. We would like to discuss these projects, their release status, give an update on our 1.13 release features and our investment wrt testing and scale testing.

To learn more: https://sched.co/Hgru

A

Okay, great, my name is Nietzsche and you all known, Justin, SB, he's very famous, so we're gonna talk about sagaie WS and give you a status update. I'm gonna run very quickly through it, because we have a couple of demos and we're hoping that we can run through them fast. So, as a summary, we did update the cicada Bleus charter. The link is available there. You can go and see our charter as a summary.

A

What we want to do, in the sake, is make sure that we are doing maintaining all the interfaces, libraries and tools that are necessary to integrate kubernetes tightly with AWS services. We've also worked on enabling prowl, which is a CI tool for kubernetes upstream and get it integrated with both cops, some of the work that Justin did and on eks data plane. It's actually providing see a signal for most of the cig AWS sub projects.

A

Right now in the future, we want to work on scale testing and there is a perfect afford, where a lot of numbers for GCE and gka are published today for kubernetes, we want to expand that to AWS.

A

Sig AWS will continue to be a user group where we will provide support for issues and feature requests on an ongoing basis and there's more support from Amazon now so, hopefully, the pace at which we are resolving issues where users goes up in also documentation. So right now we actually host five sub projects. We will talk through each of those sub projects today as fast as possible. We have also formalized caps now previously, we do not have any caps and for all of these five sub projects.

A

Now we have caps and for every new sub projects that would we would create. We want to have the process of actually providing these caps and making sure they're well main maintained chairs.

A

You already know: Justin SBI just joined two quarters back and, of course, this Chris Nova should be joining us towards the end of the talk so quickly. This version, 1.1 3 release, was exciting. For us, we launched three out of three features. So, let's go through each of those features and I'll show you some demos on it as well, so we release there being dressed controller I'll talk to you very quickly about what ingress actually means. Sometimes there are so many API objects and kubernetes.

A

It confuses me, so it helps to actually talk about the concept. We did an alpha release for the ingress controller. We, the initial design, was provided to us by core OS, but we had to refactor a ton of the code. We did some optimizations like using the controller runtime library which is provided by the cig, API machinery and also adding some AWS SDK level. Caching, that allows the code to be more optimized. We added support for not just note board, which is also called traffic mode type instance, but pot IB.

A

What that means is you can route the alb traffic directly to the pod if the cluster has support for CNI kubernetes, so I'll talk through that in more detail, but that's one of the features that we stabilized we've added, CI signal and I'll show you the actually I can go click. It does make me a little proud previously. We didn't have a cigarette blue s bubble. We were resting under cig, Google or Google, because there was nothing being done for AWS.

A

So now we have a bubble for cig, AWS and test grid, and if you actually look at Al being rest controller, there are many other sub projects. Then there's like basic lint and unit testing available. Also gear for from my team is working on enabling and eks deployer interface that integrates with brow and test intra. That will allow us to actually run more extensive integration, testing and advance the feature from alpha to beta.

A

So going back to the slide, we also have docs and we've done some blogs in case. You want to run through the process yourself of doing a demo. We have had feature requests from customers whereby they don't want to build one ingress and associated with one alb. Essentially they want one alb across the entire cluster, so that ingresses can be grouped in and they can be multi-home down to a single ILP. So it's a feature that we are working on and hopefully we can make that available in beta.

A

The main person to contact on this is young. He's sitting right. There he's called moonfish. He doesn't talk much so I have to give him credit, also Craig from Ticketmaster, who were deeply involved in making this work possible. We're also thinking of actually moving the ingress controller into the core out of tree cloud controller manager as when it actually gets into beta, because we feel that that might be one place where all the cloud provider related code can sit so yeah quickly. What does ingress actually mean for a person like me?

A

I'd really want it as dumb as possible. Ingress allows you to enable external, HTTP or HTTPS traffic to enter your cluster and actually reach your services. Your services are allowing l4 traffic to reach the back-end set of pods right and so services install IP tables, and that allows cluster IB to route the traffic back to the backend pods, and so what interesection does? Is it just acts as an entry point? It's a sort of in smart router that is routing traffic to multiple endpoints in the back.

A

There are multiple versions available in the market, so we're using AWS alb when you create any ingress onto a cops cluster or an e KS cluster. That's the basic concept. This is another detailed diagram, but basically the ingress controller that we launched runs on the worker nodes. If you see the bottom of the cluster, you can see that the ingress controller essentially is allowed to look at the API server.

A

There is our back enabled, and so it can watch as ingresses are created and the ale being dressed controller or the alb which is created whenever an ingress is issued, allows the traffic to enter the service. So, as I explained, there are two traffic modes instance mode and IP mode instance mode enables your normal node port, which has a hop because the alb can route the traffic to any node, and then the node will use IP tables to find the part where to route the traffic for part.

A

Ibc and I just directly move the traffic to the pod and there is no hop available there so quickly, I'll jump into the demo, I recorded it because I'd be highly stressed doing this live. So please forgive me, but here we're going into the repo that I already cloned I am running up cops cluster here, so that cluster is called cube. Con cops, kubernetes, dot loco there are thereafter I, actually create an iamb policy, and this policy allows the nodes on the cops cluster to talk to the AWS services.

A

So I first created the policy and then I attached the policy to the nodes once I'm done with that, I have to then download an hour back and that are back. Roll is for the ingress controller to be able to talk to the API server. So I associate that our back for the ingress controller, there is llamo for the ale being risk controller already available. So we download that and we modify the cluster name in the ingress controller, to refer to the cops cluster that we are working with.

A

So we'll quickly do that and then we will just deploy the ingress controller on the worker nodes fairly simple, hopefully you're, not sleeping and with me.

A

So we do a cue card apply awesome the deployment went through and now we can check the logs you'll see that the controller started the workers have started. Everything looks good so now, obviously, I have to have a service before I put an ingress in place, so I will first create a namespace and and I'm going to run a 2-0 for 8 game. So in that namespace I am deploying a deployment and then obviously a lot a service.

A

So until then, everything is fine. Let's just check everything game up properly, because I have to be extra, careful, okay, so the service looks good and we did it with traffic type node, but not for part IB. Just for simplicity. You can change it by using annotations when you're working with it. The deployment is there and we just check for the namespace, which should be there, given the deployment came up but and then now I'll just deploy.

A

Now that everything is set up, we'll just deploy the ingress which again allows HTTP traffic, so that's the 0, 4, 8, ingress and I will get the ingress and happen fast. Okay, so that's the endpoint, oh okay, and just to prove that I am right and not lying to you. Let's just oh I'm, not able to open it! Why? Why sure.

B

Sure, yes thanks it's my laptop yeah.

A

It's your laptop, oh no,.

B

Miss, oh because.

A

We only have what happened.

A

B

I think we can trust you that it would have worked.

A

Okay, breeze on.

B

Your laptop, we have a little HDCP protection.

A

It does work, I promise you so hopefully that made sense and in terms of how it works and what we do so just walk through it. The docs are amazing. It'll make sense. So next is the alpha for the EBS CSI driver that we did again. We pegged it to 1.13 and it's compatible with CSI 0.3 spec, and if you are following the release cycle, Sarge's did a one-dog spec release.

A

So hopefully, with with our beta version, which comes out in q1, we will be compatible with the one dot spec, so we're actively trying to reach parity with the in volume provisioner. So the cloud controller manager or the tube controller manager does allow you to directly attach EBS EFS volumes. Our goal with the cloud control a cloud provider team Walter is sitting right here and I hope, Chris, and everyone is here.

A

Our goal is to make sure that we strip out the entry volume provisioner and move all the logic of the storage components into CSI, so we can isolate it there and leave the cloud controller manager to tackle all other cloud controllers that are specific to the cloud provider. So that work is being done, we have added support for different storage class types. There is like the file system type, which includes ext2 ext3.

A

Involuntary types which are very specific to AWS and also encrypted volume and kms key IDs in this particular feature. This particular feature does not have support for snapshots yet, but by the time we get to beta, we will had support for snapshots again because the eks deployer, which integrates with brow, is not fully complete. We have to.

A

We haven't done the complete suite of integration testing, but there is still some lint and unit testing that you can see in the CI signal upstream also, we are working on a project that allows migration of customers who are using entry volume provisioner to CSI seamlessly, because we don't want customers to be bothered with the total process of migration, so we're working on certain libraries with Google in order to make that happen, and so hopefully by q3 or so that work should be done.

A

Cheng is involved in doing most of the work, but we're also working with folks from Red, Hat and, of course, Saad and his team. In order to make this happen quickly again. For me, it's very important to understand what it really does. There are two components in the CSI interface: one is the the controller plug-in, which is also called Co plugin. It's a G RPC endpoint, which serves CSIR pcs and it can run anywhere so basically on the master or the book and also can run in both places.

A

The node plugin must run on the node and it actually is responsible for the storage providers published volume. So it's very simple from that standpoint and there are RPC calls being implemented between the two. So the main RPC is our identity service, a controller service and node service. So you can think of identity service as allowing access to the other two services. The controller service is basically responsible for creation and deletion of volumes and the node service allows staging or unstageable volumes. So that's the basic concept of CSI.

A

This is generic diagram of how the deployment from AWS aside looks like we did a headless deployment and both the plugins are deployed deployed on the worker nodes. There are two other modes in which you can deploy it. So if you want more details, you can go to the CSI spec, but you'll notice here that we have a driver registrar. We have an external provisional for CSI. We also have an external CSI attached ER and then there is a CSI node.

A

So these are the four components that comes into picture when you're, actually working with the CSI driver. So again, because I would be too stressed to do the demo I had to record it so we'll run through it. um So this is the I already cloned it. This is the EBS CSI driver. You guys can go and take a look at it again. I'm running a cop's cluster, it's called CSI. Now all the pods are up and there are certain flags that need to be modified and the cube API server and the tube lit.

A

So those flags are, you have to allow privileged equal to true, and you also have to provide feature gates for CSI, node and CSI driver registry to be true, once that's done in both cube, API server and Kuebler, then you can actually run CSI effectively. So after that is done. Please believe me that the cubelet was done because it would be too much to log into the machine and show you so after that is done. We have to deploy a CR D and that's how the CSI info is recognized and distributed across the cluster.

A

So this is how the CR D looks like it's called CSI node info, you can do a cube. Cut' will get CR D to verify that so the CR D is running, the flags are changed and I'll show you that if you go into the repository there is a section on the deploy kubernetes, where we have multiple versions of manifests that you could use in order to play with the CSI driver, so I'm trying version 1.12.

A

There is a secret here, so the secret is in the format of essentially your key ID and access key ID, which is identity in any ec2 node. Obviously, in the future, we will do something more interesting like adding a pod identity or cube. Do I am way of adding these features instead of manually typing it into the file, but for now I'm copying the secret I will apply the secret and then we will deploy the CSI attacher, the CSI provisioner and the CSI notes.

A

So now that I've applied the secret you'll see that the are back for each of those components have been applied. There's a service account created and a daemon set. That's running again. This allows each of those components to talk to the API server and to actually register when volumes are being created onto the cluster. After that there is a sample app and the sample app essentially has a storage class with the CSI driver.

A

It also has a persistent volume and claim, and we will slowly see that if you do a you can get for the PV you can get, you can see that the PVC is attached to claim one and the claim one is bound.

A

Essentially I'll just do a cat on the board yamo. Just so you know, that's actually called claim one. So you can see that the claim one is the claim. Name is claim one and the part is called the app. So we can look at quickly. Look at the storage class. Previously the storage class would have been a direct EBS reference. Now it's a provisional called EBS dot, CSI dot AWS.

A

So what you do is then just apply see if you can find the app. So if the app came up properly, then you should be able to see app is properly deployed and it's running so hopefully that makes sense. I ran through it very quickly, but the components are exactly the same.

A

Going back to the third project that we worked on, this is the external cloud provider. Aws I was just referring to it. We did an alpha for it. We are working with Google to make sure Google and digitalocean and all our friends in OpenStack to make sure that a bunch of interfaces and dependencies for the cloud provider code is being moved into staging so that we can cleanly cut out the cloud provider, controller manager from code kubernetes. But in the meantime, we've also done testing for the cloud control of a binary per-say.

A

For this to work, you have to change certain flags. Obviously this will run the master. It can run as a diamond set, or it can run as pretty much any sort of API object. That kubernetes has you just have to make sure the leader election is properly taken care of the cube, API server and cube controller manager as a 1.13 does not will not run the cloud provider flag. When you run the out of tree cloud controller manager, but for the cubelet you will have to call cloud provide equal to externals.

A

So once you enable those feature flags in each of the manifest, all you have to do is run the cloud controller manager binary, which will be available as a image as a daemon set or whatever other object in kubernetes that you want to run. It adds again. This is alpha, it's not yet completely stabilized, so we're still working on some of the docs, and you know some of the dockerfile and modifying and updating stuff. But you should take a look at it towards end of the week and give us some feedback. Mike is more.

A

Is there quite involved in this work, but also other folks like Yahoo and young will be involved, because we will be adding new controllers into it and, of course, Justin does a large reviews, which is awesome. So this is the AWS kubernetes tester, as I said, for proud to integrate with eks. We added this deploy interface into test intra. We worked with Ben elder and San Liu from Google, and it was. We had a great time trying to add this over and today our test coverage is around 3%, but hopefully going forward.

A

We will expand that to more, and that will help us provide better CI signal to all the projects that we are working on. With that I'll hand it over to Justin Thank.

B

You Missy I think that's that's a great update. I yeah you've really laid down the gauntlet there. I I'm thinking back to all the sort of previous take up dates. I have given, where I have sort of probably glossed over a lot of these things. So, yes, I will I will aspire to that level of Hoops. Where we go sorry, I will aspire to that level of depth and future. So yeah I'm going to cover some of the last couple of projects we have going. The first one is the AWS encryption provider.

B

This is a support spin around for a while. That's going alpha in or when alpha and chimera is 110 requires sd-3 and a bunch of other things, including most notably API server changes. So this is something that probably you will have to get baked into your upstream tool, whether it's cups or eks or any other tool you're using the I, do have a demo on this one. The attack it's sort of preventing or the problem. It's solving is that in the past, or up until now, the data in at CD has not been encrypted.

B

The volumes underlying it can be encrypted, but let's see, if I do I think I have a config map here, for example, and I should have, which is called example, and if I were to get that value from at CD. That was a lucky autocomplete and dump it out. You should see nope, that's the wrong one, never mind that. Don't pay it no attention, let's don't name them both the same there we go all right.

B

You can see that the actual values there are at the end there in text, tip config maps are not not for secrets and if I were to do, coop got a coop cuddle, get config, for example, yeah.

B

Well, you can see that is actually would be pretty much a one-to-one representation, that's encoding using using protobuf, but they are not encrypted in any way, and so what the ATS AWS encryption provider does is it uses the kms service of of AWS and kms is a sort of secure store of encryption keys and is effectively a key management and encryption service as a service, and if we look at what a secret looks like in this world, so, although through the API server its secrets through the API, so you're able to see the rule values, as you always were before,.

B

Me there basics before encoded, again really not encrypted. If we were to look at that in EPS CD, you can see that they are encrypted. There is here a header which says this is an encrypted value using the AWS kms provider and then there's a whole bunch of junk, which is not just basically for encoded.

B

This is actually genuine encrypted, and so the idea is that if you were to gain access to @cd in some way as an attacker, that does not give you access to everything in the system you can configure which API objects you want to be encrypted or not encrypted at the moment, I believe just secrets are encrypted. If you were more paranoid, you can encrypt everything, but that's a very quick demo of that project.

B

I, don't think Seth is here, but Seth Pollak is the person that wrote that project and has been doing some great work on that yeah.

B

This I said, gets a bit a bit of background here. This was a something that was originally built, I think for GCE kms and was originally built into the kubernetes api server, and then it was extracted and made. There was a real interface created and the GCE implementation was kicked out of tree and it now runs as a separate pod and the that means that any cloud provider or in theory, a bare-metal provider, can also plug into that same interface and implement it.

B

And so this is something we're seeing again and again with all these projects like we're able to develop outside of kubernetes kubernetes in the community SIG's repo. We have good management around that now and and great testing thanks to the work you've been doing, thank you and yes, that's the encryption provider. The next project is the eight of us. I am Authenticator which I can't remember who originally started this I guess.

C

Matt Landis.

A

B

Yeah no anyway, thank you, that's it. They happy originally started this. This allows use of I of AWS credentials to authenticate to the kubernetes api server, which is a really nice integration. What yes, as this is that is a dubious Aria Authenticator, it's it's available, and it it is. It is very handy both for end-users and for I think they could use it for easy like if you had an ec2 instance. You could also use it for authentication there good progress on that as well, and the cluster API AWS implement.

B

Chuck is here I. Thank you. So Chuck has been doing great work on that.

B

This is essentially the the submerge of seed cluster life cycle is producing an API called the cluster API, whose big addition is the machines API, which gives programmatic access to infrastructure in a way that we haven't really had before, specifically to spin up machines through the kubernetes api there's potential to integrate with, for example, the autoscaler is sort of the the most obvious like killer use case.

B

So we we have a otis scalar, kubernetes autoscaler integration with a diverse, auto scaling groups today, but it's a little bit flaky, it sort of uses, there's a little bit fragile. It uses tags, and this is a we make the we essentially surface, ec2 instances in the case of AWS as first-class instances in kubernetes, and on top of that we build objects, called machine sets and machine deployments which are similar to deployments and replica sets for pods.

B

So we essentially treat machines as first-class objects in kubernetes and were able to with providers through this well-known interface, which chuck has been doing great work on. Thank you. We're able to you know, create different your launch instances using different a.m. eyes and configure the control plane, the control plane but and the nodes using cube, ATM and yeah. It's it's great and it's if you're looking for a perfect to get involved and I would definitely encourage that one is a great one to get involved in.

B

For me, that's the most exciting personally I find it the most exciting project in kubernetes right now. So that is a great one and I encourage you to take a look at that, but it is. It is only alpha at the moment in terms of usage, I wouldn't run your podcasters on it. Just yet is that fair, okay, there yeah, but please do try it out ich es cuddled. You wanna talk about this. One err: yeah, okay,.

C

A

One is not an existing sub project, but it allows us to create ETS clusters with great ease. There has been a discussion about making it a sub-project, it's still up in the air, and we have to go through a formal review with the steering committee. But we didn't want to talk about it because it's effort put in by V books and also Elia.

A

We call them error developer and that's why I just wanted to give a status update. But obviously this will be up for debate within the sig and then would probably ask the steering committee if they can approve the project. Yeah.

B

I think I think our sig will be talked to in the meeting. Our sig is generally by way of background. Our sig is generally supportive of the idea. I think it's an unusual one and it's only usable with a commercial offering yeah, and so the question is: does that? Where does that fit in in the universe yeah of projects, but yes, cut all yeah great tool, I.

A

Suppose that's just EDD status. Yes,.

D

A

Wanted to call out gear ho who's been doing great work with us, especially in the CI signal integration. I can see Erin sitting there he's been beating us up to make sure we have better CI signal, so we have done some work with him on the ETS tester, but apart from that, obviously yet CD got donated and we did a future blog with Google on the roadmap that will maintain for its city. So I just wanted to give a shout out to kyoho for his great work. Yeah.

B

And I'm excited by some of the other things coming like I, think the non-voting members will really make it much easier to manage that CD, but it was perhaps a different thing: yes, Wow, yes, and so I I don't know, there's anything you want to call out on this side. I think I would call on this, like just that. There is a lot happening in the sig.

B

It obviously is a lot to keep track of, but there is a plan to get a lot of stuff into into alpha and beta or through past, alpha to beta nga, how people move their existing clusters that or the existing volumes I guess to the future world of CSI yeah, and some of these dates are perhaps a little tentative or aggressive.

B

Maybe but there's there is a lot happening on many paths, but initiate job of imposing more order on the sig, and so hopefully, you're able to you'll be able to better track that through caps and those sort of efforts, yeah.

A

And I did want to call out that a lot of people have been talking about NLB, not being baited, and people are concerned about that. But we will get NLB to a beta stage by q1 and then move it forward into a ga State, but other than that. The existing sub projects will be moved from alpha to beta to GA and of course we will just try to improve the testing coverage on it. So that's a summary of everything that we had to talk today.

B

D

B

We open for I don't want any questions, wow that was fast. We have mics because I don't fits easier to do the mic, sort of repeat it, but I will run down to you with a mic. If you would like.

E

One of the core issues in running gcpd RCP on a device is that the alb supports h-2b to up to the actual, a or b, but basically negates it or joseph back to 1.1. Does the alb ingest support HB 2 all the way back to the pod.

B

There we go, we have an.

C

A

B

That is, that is a lifesaver.

A

C

Terminate so HTTP of traffic as a LP in the plan, so the back end doesn't receive a HTTP to traffic. So I think we were supportive. Wise they'll be yourself supported, but we need to discuss with the LPT.

E

B

Okay, you and then come to your flat. Okay,.

F

So, where does route 53 fit in all of this, like, obviously, we still at the DNS controller in cops, it feels like it needs to be pulled into it, something with the ingress controllers that you're building I mean.

B

I can I can talk that we have the external DNS project, which is which is going to, which is intended to a bunch of there were a bunch of DNS controllers. That sort of came together into the external Dina's project and cops has its DNS controller still in cops tree, and we need to get it out of cups, nine and start adopting external DNS. But xform CNSE is configuration of DNS outside the cluster, which is why it's called external DNS and it includes route 53, but also any has a pretty about a dozen providers.

B

It supports. It supports Google, Cloud DMS, it supports about flair, I, think it supports, go Daddy and like these sort of you like a bunch of different things, but that's sort of the path for it and it doesn't necessarily need particularly tight integration with the ingress controller, because it's able to observe ingress objects and I think service objects and their support for observing direct DNS record specifications as well. So I, don't think in the design of communities.

B

I think I think allows it not to be tightly integrated and cops in particular needs to get on to use using it. But if you wanted to configure DNS records, I think external DNS is, you know, is in a good state to do that today.

B

B

D

Thank you, um I'm, not entirely sure. This is the right venue for this question, but is there any concern or information about cops sort of lagging behind mainline in terms of the versions that it supports and deploys.

A

B

So, yes, the cops is not yet released, one in 111. We will do so shortly. There is one more PR we want to land in. We want to land upstream cops, does not do our own cherry-picks as it were, so we sort of are waiting on that are that that particular bug is actually fixed in 112. So we will do that. The the feedback we've got from cops users and I- don't want to spend to him but yeah. The feedback we've got from this is supposed to get us. Someone gets the fever.

B

We've got on, tops users that we, the cops users like that the dot zero release actually is basically prod ready and doesn't go out with known issues. The way we are going to square the circle as it were is I have promised to do alpha releases of 112 and 113 and I think 114 imminently. So you will have the option to run.

B

Should you decide that you in fact are ready to run 111 or 112 or 113 or 114 that you could do so you're, not gated on us doing so and then, when cops, zero lands? That is sort of the indicator that we believe it to be more ready.

D

Okay, so it's something that's coming in a format that we can run it for ourselves, but a matter of when is sort of fuzzy well.

B

I mean so, you will be able to run the dot that the alphas whenever you want and then the dot 0 will be indication that we as a project, which is a cops project, believe it to be not just the communities but the ecosystem of things that we manage to be to be ready. Okay,.

D

Great. Thank you by.

A

The way all the demos that I did was on cops using version 1.12.

A

So unless we have I think you guys should test and maybe throw more issues out and see if what goes wrong with one dot well and once the alpha release comes out, we'll probably have a more more stability around the cops tool. So.

G

This is going to be a slightly different question. I was one of the steering committee members who reviewed the sig AWS Charter and I wanted to commend sig AWS for putting language in the Charter to specifically make diversity and inclusion, one of their top priorities and I just wanted to call that out. In the context of this room, I can't see the front of the room but I'm looking around in the back and I count.

G

Maybe two people who are not white, dudes and I'm curious, like what specific things you're thinking you might do to help improve diversity and inclusion. I'm, a big fan of whatever we can do to make this place seem like a more welcoming community and I'm just struck by the makeup of the room right now,.

A

So, first of all, you have me- and you have Chris Nova, which makes it better and Justin has to tolerate us, which is always awesome. So the other thing that I will give credit to Bob, but he did give me the privilege of working with two ADA interns. One of them is here: Kirsten and karena is not here, but they've been involved with ETS cattle and I've also stressed them through working on cops and also they've, been building tools for us to better track AWS issues as they pile up.

A

So we can resolve these issues and provide some form of SLA to the community upstream.

A

Also, one of the other interns Corinna has helped me actually measure the contributions which were really bad before and now they're averaging at 120 per 120 commits per month, which is quite decent given where we are at so we've at least started by having ADA interns, both of them converted their full time of full-time roles at AWS. There are more interns in a schwarz team he's right there. He runs the care service team I mean we're doing what we can for diversity.

A

I'm sure Amazon is interested in improving that, but, as of now, I mean we're all involved in the mentoring sessions as well baby steps and maybe I- will help address out because she's overwhelmed and she needs help and more mentoring, and that might be the way to move things forward. Thank.

B

You and I wanted to give a shout out a talk to an issue for actually doing that cap, which I had certainly been not. They, yes, are their Charter, which I'd been sitting on for a while and not not actually making progress on, and yes, obviously, I need to make more efforts in that department as well, because whenever here I thank you earlier where or did it? Oh yeah, sorry, what's different perspective, also.

H

Sort of related to the same cops question on terms of lag between what's available for its 110 now might am just told me, 111 by the end of the month, three KS. So how does your stuff feed into that release schedule? But it's because one tenth or at the end of life, okay, 111, 110,.

B

So when that is not end-of-life well, it's just the last support it with oldest supported, release right now um and I. Think yeah.

H

I, don't know: if are you feeding into that, so when can we expect one always gonna be revs behind on AWS? uh Well,.

B

I mean so in Kherson. Cups are separate tools and we have separate release cycles in the EPS managers right. There's.

I

Alright, so 111 is very imminent yeah, so so the reason why we like we went out of 110 when we meant GA it's taken as four months to get to 111 right. So the reason why we delayed 111 is so that we can actually get get it alongside with the upgrades ability to upgrade stick clusters from one ten to one eleven right. So that's the reason right.

I

So as we move forward you'll see, is releasing it at a at a much prettier burst pace, because because now that we have upgrades right so so we're planning on one one twelve two months, which is an improvement from four months right so and then our goal next year is to be is to be, is to make new versions available in eks within one month of the community links right. So so that's our aspiration and.

A

We need help with cops versions, so if you want to help you're welcome yeah.

B

I mean cuffs, we're gonna, we're gonna push to make really has more automated and have the alphas being basically available and it's sort of on your own head. If you want to try it, but you have at least you will have the option to try it yeah I, just.

G

Wanted to add some some color to that a little bit, one of a WS does main competitors in one of the founders of kubernetes didn't actually roll 111 out and they're hosted offering until like a month ago right.

G

It feels an awful lot like the downstream providers of kubernetes offerings are taking a while to actually keep up with the upstream, because they're experienced and actually rolling this out doesn't seem to be all that fantastic and, as somebody who watches the upstream dashboards that describe how do we automatically upgrade from an older version of kubernetes to a new version of connect? You burn, Eddie's and see those be perpetually read. I am not surprised that providers are having a difficult time making it really easy to upgrade.

G

Kubernetes I suspect that if you would be willing to contribute resources, be that other people or time or money or descriptions of how you are upgrading and what you are using to upgrade. We collectively as a community could probably overwhelm the efforts of any single cloud provider or downstream provider to make upgrades kind of a solved story.

G

I feel like the sick cluster lifecycles sega's may be one of the places that's trying to do this via cube ATM, but I'm, also kind of looking for a canonical open source story for launching a kubernetes cluster and upgrading it that is staffed and well maintained by community members. The problem is today on on the majority of the CI: we run for the project.

G

That is a couple thousand lines of bash that the community as a whole kind of refuses to maintain, and it falls on a few select individuals and the company that I in Justin work in there's also so cops. But again, I don't feel, like there's been a phenomenal level of support behind cops.

G

So I'm really interested to see what happens with the cluster API project, because that seems to be where we are trying to once again redefine the concept of a cluster description that would be portable across providers and providers that are kind of completely open-source as well. To stand those clusters up and I think that would be the appropriate place for the community to help make it easier for them to get releases that are well tested and available sooner. So hopefully, that provides some color and.

B

I have a talk tomorrow about how we're suggesting to use operators for add-ons which is part of that story of having an approach which could be used across multiple providers. Well, we run them up to Bob.

J

Hi I'm way in the back of the room, I wanted to add to something Erin said and just a bit of perspective in terms of where it would be good to put more effort.

J

This came up at the small but intense at CD panel that we had this week and I think that when you're doing an upgrade, probably they were like one of the riskiest parts of doing an upgrade is actually the NCD part of the upgrade and right now it's kind of this all-or-nothing thing where you tend to upgrade everything in one: go and being able to have matrix of testing between different versions of a CD and different versions of the control plane.

J

Bits is probably one of the areas we need to do a lot better in so that we're able to not have upgrades, be this big bang thing and, of course, we're working a lot in the NCD community, I guess I guess kyoho is not here, but he certainly our leader in that spot. So anyway, I just say as a place to work. That's probably one of them. I, don't know if you agree with that Aaron or not, but all right great. Thank you. I reckon.

I

Aaron with know, Bob is 1/6 scale chair as well as the GM for I'm, so any kiss they.

B

Were some more question at wells.

K

I have two questions for you. So since we're already all over the cops topic here, okay, other projects will often tag low-hanging fruit for issues for new contributors coming in. What's the best way as a new contributor that I can help with cops so that we can start getting releases coming out faster.

B

Yes, yeah, we just.

K

Loaded, obviously, I don't.

B

Know if those were your two questions, but they do feel like two questions. I think one of them was how do I get involved as a new contributor, which I needs to do better on identifying what issues those are and then there was.

B

How can I help get releases out faster, which is not a new contributor issue that is actually mostly just automating the build scripts and dealing with infrastructure type things which is probably a frustrating place to start so I would not recommend that unless you really want to you want to that, we should talk about it afterwards, but, yes, the I I think in across all the projects. We should do a better job of tagging issues there Jen you're, genuinely good first starter help-wanted.

B

They do tend to rot. So if I am bad at removing that I have I've seen issues where I have not removed the label until someone has tried- and it is not an easy issue- and that is sort of why it lingers for like three months. So it is a great idea to comment first to say: I am interested in doing this and here's how I am proposing to do it and then I can really maybe dig in and really think about it, because sometimes as a bizarre Mis applied.

B

So we need to do a better job of applying them to in more places. So.

A

I have a we started doing something around this and according to exactly what you're saying but I stole the idea from Andrew Kim say cloud provider. They do a very good job of tagging, every issue as a feature or a bug, and then they also tag it as easy, medium or difficult. So we're beginning to do that with all the issues that we are collecting through Kirsten's tool. So you should participate in seg, AWS, cuz, I'll start reporting on all that and then you know we can get to a better place.

A

Just like Justin mentioned, and maybe you can start with the easy shoes and see if it that works for you and then graduate up will helping with lust APA. Thank.

K

B

It was when I'm down your reaction yeah, but we can. We should talk later than yeah I hope I can find them yeah. Thank you.

E

L

There any plans to bring the a to produce VP CCNA into the sig and second question: are there any plans for a sig, Blessed workload, identity, things, I'm and I cubed to I am okay. I am yes.

A

So if I'm a baaad identity is a proposal from Micah, he will be presenting kept very soon and that proposal is out there. We just need a cap and a formal review on it. He can talk more about it or you can talk to him about it. It's yeah go ahead, my guys we wanna. We.

I

Be told that you're 10 minutes overtime, Oh.

D

Breakout and yeah just come talk to me afterwards. Yeah.

A

And the see and I will not be as a project, yet it'll stay where it is, but that's how it's done. Thank you so much.

D