Kubernetes SIG Cluster Lifecycle, 10 Dec 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cluster Addons meeting: 2019-12-10

Description

Meeting Notes: https://docs.google.com/document/d/10_tl_SXcFGb-2109QpcFVrdrfnVEuQ05MBrXtasB0vk/edit#heading=h.u7mt91ljn6o5

A

Hey folks, we are recording cutting the cluster add-ons agenda and sorry is we're getting a little feedback anyway. This is the cluster add-ons meeting where a sub project at a sequester lifecycle and we've got a little bit of an agenda to talk, though so. The first thing we wanted to cover is just any action items from last time and I.

A

Think if we let's see I did reach out to Evan Evan. Are you here right now.

B

A

And well, he told me that he asked somebody else to attend the cig, apps weekly call and is that person, maybe here, to give an update at all.

B

As far as I know, so we will definitely get to the next one. Okay,.

A

Cool sounds good, I'll just move these action items up to the next.

A

So here's that.

A

Move to next year, all right and then there were some other things that we're starting to happen.

A

So aside from just talking to see gaps, I know that the redhack folks wanted to build a prototype and then perhaps do a demo of this app discovery and like observability portion, so the fir'aun's in particular and then I think in our agenda. We have a demo if that person is available.

B

That's me: I. Can their quota tickets to be super work yeah.

A

um Let me you go ahead and rescind the presentation are you able to present now let.

B

B

Will try to put this on directly? Can you folks see this all right? Where does it need to be increased in size.

C

Could you increase the font a little bit.

B

C

I could do it Thanks yeah.

B

So effectively, we just made a little controller to implement the customer research submission when you talked about last week, which was essentially just an aggregation of references to resources with conditions invested when they were found. So we have something doing exactly that. The link is in the in the agenda agenda and effectively every time. This controller sees something with this specific label prefix, followed by the name of the add-on that we want to talk about it aggregates it into the status as an enhanced reference.

B

So this is, we can't see the whole picture, because we're kind of large I.

B

Have a few add-ons gonna go downs that I have here. One of them is corniness, so I have a kind of cluster with Cordy and that's running on it. I will get that same one, let's extend it down and you can see. This is like a selection of the resources that I'm able to get some deployment. Some pods names basic services.

B

This starts to get really useful when you have things that are spread across an entire cluster, so this is like Cordy and that's as a few resources, but we can also do something like :. So here I have a set.

B

Clem resources with a customized configuration to add a label on to it that names it as an add-on over.

B

Apply that to the cluster.

B

So whenever this operator sees that label applied to a resource that will actually generate a one of these add-on resources here and you can see that it pretty quickly picked up all the different resources that existed there.

B

And even picks up problems with our customer resource definitions, because it's again extracting those those conditions and showing them appear as part of the references um so like I said this is super basic right now it is dynamically watching discovery to find new things to add. So you don't have to do any sort of like entertaining CR DS to get the controller to actually pick up your resources.

B

It's pretty straightforward, pretty simple! So that's about it! For now. Do you have any questions about this demo as it stands,.

C

Things looks really cool, I'm curious if and I like the I, like the approach with just like having this piece, be discoverable I, think you know one when we started there really early out on work. We're kind of like okay, well, we'll just use an owner Refs, and we can kind of traverse those and figure that out. But I don't know if that that doesn't work for things that cross namespace objects so having having this be a little more abstract to things. Awesome I'm curious.

C

If you see any sort of way to like aggregate status beyond the individual objects or any sort of fun with like way to describe all up health, maybe that's not something you can solve generically, maybe that's domain-specific, but I'm curious. If, if you see that working with this model.

A

You may be on you if you're talking, I am.

B

Thank you so yeah, like I, said right now. This is using unstructured to basically unmarshal the conditions from the broad JSON into something that we expect.

B

There is some stuff in the proposal that is also linked on this repository that talks about potentially having some sort of JSON path, selector over a set of fields in a resource that we could then map back to a condition which I think would be a pretty cool thing to do.

B

The only the only thing that I'd be concerned about there is structural schema and what that looks like, if that, like would violate structural schema.

B

C

Concerns few sentences that make sense yeah like the aggregating of conditions, I, think that's pretty cool, you know I think. Maybe we can think of think of ways. On top of that, like look at this as a collector and then think of ways to look at these conditions and infer some kind of status or something.

A

Maybe I missed it, but just could you show like a coop cuddle output for the kind.

A

Then you'll be just get better. The kind.

B

It's that's efficient. You want this ready, I'm.

A

Just like kind of oh there's only one of these in the cluster right now right, oh.

B

B

There's multiple actually I described yet on discovery as an add-on as well.

A

Very cool, okay and then you're installing OLM link this right, maybe I'm as well. Well.

B

Not really installing, but it's just like adding a customized layer on top of that applies common labels. Everything right.

A

Okay, cool, oh and OLM is installed with customized. Oh.

B

It was installed as a set of manifest I just used customize to get the way blank I see.

A

B

But that was something that we'd like to do.

A

That is that's really cool. Cuz I know that the motivating kind of like force behind this right is like. How do we make things more usable for people right? So then the tech enables us to kind of build these things around collections of objects that are grouped in some generic way. We can use that generic mechanism to surface statuses and similar things about you know how an add-on, dis, packaged and then yeah as new resources get created by the add an operator they could follow.

A

That label schema, that's pretty cool so and then you would even just be able to bundle this custom resource with it as an option. Maybe huh yeah.

B

Like as I was kind of alluding to before, like I think that there could be a potential future where they, in addition to the label on a resource, let me get one.

B

You can have essentially an annotation that has the same prefix, but now that I know which operator or operators it belongs to, then the value could be like the stuff to fill in with.

A

This get like really crazy, unmanageable for, like a daemon set on a thousand notes.

B

Admin set on the thousand nodes I think you need to be careful about aggregating stuff like this, because there's the one megabyte limit in, like the fault, cubed CD for the resources I do think. However, it's better to have a place to see all of this stuff together and extract it than it is to have to go to all those resources and look at them yourself.

A

And then it should perform fine yeah yeah.

B

It absolutely needs to reform plan like what I didn't get into before was now that we have this mechanism for adding things to a resource.

B

We need some way of like protecting that so like the first thought that comes to mind is okay, well, I'll, just put in a mission webhook between all potential resources that could ever be, and this and then have some way for this controller, to sign and accept based on user identity and then later on, ensure that you know this label that says that it that this is a component of this add-on was put there by somebody at some point who is authorized to do so.

B

So that's a huge thing that could be careful. Okay, if we end up going preserve this I.

A

Really like this, for an enterprise like management, cluster I think, like the more layers that you can give somebody to understand. What's running, like people will install those things, particularly when you are in a large cluster and I'm always concerned and like keeping in mind those users, the edge users right who want yeah, minimal number of things running and just keeping in mind what the overhead of running in control a like. This would be, which would probably not be too much, but maybe unacceptable for a single node. So.

B

Yeah, we probably need to do some trick. Stuff I just followed the acute garbage collection logic for dynamically, adding Informer's to this, but we probably need to do some trick stuff or like we're preventing things from being cached unless there are resources on the cluster of that kind that have label. You know it's tough football and.

A

Then you mention that this is collecting on label selector, but does it also follow on a references or is that just left up that is left.

B

Out for now, so we can reference obscenity problems. It's.

A

Really impressive, the turnaround time that you, you folks just play here, because this is it's not a small achievement. This is really cool. Thank you. The demo comp.

B

Thank you very much.

A

Cool well back to sharing sweet thanks so much for the demo. The next item is mine, which is just a quick notification that the cubed am PR, has an upgrade implementation now and I'm working on a rebase for an upstream refactor of some component config things. So the PR is not green at the moment it does contain both the init and upgrade functionality. Now, as of the most recent commitment, also thanks to Daniel for pushing some Doc's updates and then yeah, that's it on that update.

A

Does anyone have any questions regarding add-on, installer and koopa DM and where that's kind of going.

A

Cool this is Geoff Keighley.

C

This may be one to punt. We, the github issue for Daniel started, did like how to write an operator sort of a while back, and this is kind of one those are. We was early days, we kind of started something, but it wasn't totally ready, but now I think we're getting to the point where we certainly understand things better. Uh-Huh.

B

C

I'm curious what needs finished here before. We think that we have like something that someone reasonably familiar what the project could follow, but it honestly might just be better to wait for when Daniel comes back to talk about that. Oh.

A

Right, okay, um yeah, so there's the COO builder patterns that we started putting together and did they've got like a good baseline feature set. But I don't recall if we finished porting that pattern to be usable with v2 the.

C

Pattern yeah that works with v2, with with controller runtime v2 mm-hmm.

A

C

We are working on getting customized support in the operator itself, yeah.

A

You had some initial patching logic that you're playing with and then you're looking at, maybe a little more customizing stuff, yeah.

C

Yeah it looked like we were just going to continue to read but I. Think having a surface for patching is nice on the CRD. But there is just tons of you know: there's an expanding feature set of customize. That's just nice for all has to have God.

A

Said yeah um I would be very open to like factoring the add-on installer library in a way where the templating and patching portion of it is separate from the API and the application of those manifests. So just like food for thought in the future, so we can have good interoperability and keep things maintained and working in a uniform fashion.

A

Yeah that make sense they don't they don't have to be the same right now like we can build them separately and then factor together. Yeah.

C

I think, as long as we're like as long as we, we feel like customizes the right path forward as long as we're pushing the mutations so to speak, to customize.

B

C

That means you're doing cube, CTL apply Kay or, in our case we're like creating an in-memory file system based on files. We load sticking it in there customize run yeah.

A

C

We aren't duplicating that transform type stuff, yeah.

A

And you you can use similar basis, you can yeah have API. Is that have the same inputs? That kind of thing so yeah I, agree yeah.

C

I was worried that we was going down the path when I added patch as a goof. Are we just gonna? Add every customized feature here and I mean that's not good. Yeah.

A

A

Yeah, oh yeah, just the I do think customizes a right path. I, don't want to leave our cig apps books. You know in a different building and helm. Three is very mature and has massive community usage, so I'm always willing to revisit that conversation. I think others probably are too, but yeah would be add-on. Installer API the configuration component config, the the customized draft, is called out right specifically with the technology in the field thing, so we may be able to extend upon that later on.

A

If that's a thing that makes sense.

C

Yeah, that's good to have the flexibility.

A

Cool well did anybody else. Have questions concerns things they wanted to bring up stuff that they didn't add to the agenda or anything they wanted to circle back.

A

A

Thanks so much Nick, it's for the demo.

B

A

And for doing that, work and then yeah good discussion think we'll close up appreciate everyone's time and again as a reminder, this will be the last meaning for this year, so we will not meet on what is it this 17th and the 24th and the or whatever we will meet on the 24th, so our next, our next meeting will be on what is it 6:00? Something.

A

7Th January 7th, 2020, yep cool thanks, everyone and well we'll see you all later have.

C

A great break, everybody.