Kubernetes SIG Cluster Lifecycle, 4 Dec 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 20191204 - Cluster API Office Hours

Description

20191204 - Cluster API Office Hours

A

Hi today is Wednesday December 4th 2019. This is the cluster API office hours. Meeting cluster API is a sub project of state cluster lifecycle. We do have meeting etiquette if you are interested in discussing something. Please add your topic to this agenda document and please use the raise hand feature in zoom if you'd like to talk and I will do my best to call on you.

A

This meeting is also being recorded and if you haven't already, please add your name to the attending list and we will go through the agenda, so I don't see any PSAs or demos or POC. So we can move on to the discussion topics. There's one here about hardware, configuration validation, I'm, not sure if the person who added this is on are you here by any chance.

A

All right, maybe we'll come back to that if they show up so Michael, you have the next three items.

B

Yes, hello, so we've been discussing downstream about some power options and node maintenance, and this is kind of related remediation as well. The first thing up is rebooting machines, so we have some users, particularly their metal folks, that are really interested as part of the remediation story being able to reboot a particular machine and I've had some back-and-forth about where that functionality should live, but I feel like the actual implementation of handling.

B

The reboot should probably be centered around the Machine, and then these avi, the machine, the infrastructure provider and so basically we're providing just an interface there into the mechanism to be able to do that. Rather than say. We have this thing called the bare metal operator, rather than writing our remediation system to interact with that thing directly and there's a couple different, like weird edge cases when it comes to like rebooting like.

B

Maybe not all providers might support it and things like that, but but, generally speaking, I would think it's probably broadly applicable, and so this just talks about what I think looks like a decent model.

B

Having the machine object have a couple new fields, maybe an annotation or something in the status field, and basically something requests a reboot. It could be anything and in the way that it does, that updates that particular feel on the machine and it puts a timestamp of when it requested the reboot.

B

So the timestamp should be basically the time of the request, and then the Machine looks at the requested reboot timestamp, and it compares that to the last boot time of the machine. If the timestamp of the request is newer than the boot time, it reboots the machine.

B

Otherwise it does nothing and that gives us some functionality to let other things kind of cooperate. Let's say that I have two things that might mean to reboot a node for one purpose or another and I: don't really want them. Racing on reboot idea that we have well-behaved clients to understand this logic, and we can export some methods that they can just run this and say yes, I want to request a reboot or not.

B

They look at that time stamp and say: oh hey, this thing's already been requested, since the machine is rebooted I'm going to do nothing and then I'm, just gonna kind of wait has obviously reboots about to happen, and it just gives us some mechanism to coordinate some of the logic without having to build a really complex state machine, either in the machine controller itself or external to the machine controller.

B

That's pretty much it for this one. Okay,.

A

Does anybody have any feedback for Michael for right now and if not I would suggest, following up in the github issue,.

A

Anybody feel strongly opposed to the idea of adding, in some support for rebooting machines.

C

Does this and do any explicit mutation of the machines, or is that literally just reboot exactly the same way say once before.

B

Yep, just to reboot the machine strictly like a power cycle operation like if you go on the AWS GUI and you restart the machine, that's pretty much. It.

D

That's fine. You me.

A

You Daniel I think your hands up yeah.

E

This reminds me I think that there's a kind of rollout restart deployment functionality that was added I think earlier this year and it's I, don't I, don't know exactly the implementation, but I think it uses kind of the same idea or was timestamp for annotation. So Michael is that I.

B

Actually, don't know anything about that, but cool if two different people came up on the same idea and it to me it it just makes sense about it for a while.

A

Would you mind adding a comment to the github issue and then sounds like it's, maybe some good prior art to take a look at all right, I'm, gonna, move on to the next one for Michael's. So if you all have any additional feedback, please add it to the github issue. Next up we have stopping machines right. Yes,.

B

Okay, so kind of similar to what we were just talking about power, state of machines generally reboots, one use case and that's most closely tied to automated remediation, at least from our perspective, stopping machines I think we could build a lot of different, interesting use cases on this, the first one my thinking is or what I've been told us, especially for our bare metal folks is they want to have the ability to shut down a host and perform some kind of maintenance on?

B

Maybe that's swapping a drive or a dim what-have-you and in their particular use case.

B

Reaper visiting a host entirely from scratch is not desirable, especially if you're talking about like a converged storage type thing where there be a lot of replication, etc, and so that's one use case here. Another interesting use case that I thought about was. It would be really nice to be able to take like a snapshot of, say, your master for disaster recovery purposes, and it would be really cool and I have some tooling that can build. On top of this, maybe automate that effort. You might need to do that.

B

Also for data retention requirements for your particular environment what-have-you, so this is basically defining a new field as part of the machine speck called desired state and it should be one of three things: running stopped, forced off and the fourth off one being like optional I guess the first two seems obvious, and if the field is not provided or otherwise blink we're gonna assume that means running, and so that's the idea there.

B

Unless it's some challenges, you know that you're gonna have like instances where something is going to want to.

B

Maybe do some other thing to machine, and this you know there could be multiple things that want to use this thing at once, but I think what's nice about this particular mechanism, especially doing it on the machine, is a you know: well-behaved things such as machine hell, checker talking about deleting machines and stuff. So when it says, oh hey, this note is unhealthy.

B

Before I just go off and delete this machine, I'm gonna check the running state because if it's request to be stopped, obviously it's going to be unhealthy right, because this is obviously something that somebody has directed the machine to do so. It serves as a point of managing the instance power state, but also informing those other abstractions that are trying to do things with the machine.

B

And so that brings up the kind of next feature under challenges is like some things. You can't just stop or start in instance our machine as you can see there, like. You, see two docks like spot instances, you can't stop a spot and since that's not a valid operation, and maybe there's some other deployment models, we're stopping this. You know just not implement it for one reason or another.

B

So the next thing would be features, but let's talk about this before we move on to that I guess: yeah.

A

So Dan had a comment in cat. It would be good to have an addition to the spec that will disable these actions for white-box servers that have no out-of-box management.

F

A

Gain you want to clarify on that sure.

G

I was just thinking for the things like HP cloud line servers or the the super cheap servers that have no remote control.

G

If you accidentally give a capability of powering off large amounts of servers in a remote date center and with no way of powering them back on again remotely it's somebody's gonna need to fly out there and send them all back on again again so for servers that we know I, don't have these capabilities it'd be good to have a safety gate that we could at least specify which stops these actions being performed on them.

G

B

I think with that at least our particular bare metal use case we're doing everything out of bands, so all the power operations would be out of band, so I'm, not envisioning, like a demon or something that runs a shutdown. You know H on the machine. Rather this this would be out of band operation, and that would this would all be implemented by the particular provider and so be up to the provider to determine like the infrastructure provider, how to actually go about implementing how to start and stop these things. So.

A

Basically, if it doesn't have a remote management interface, then there's no need to disable it, because it's not there in the first place, yeah.

B

If there's no remote power management, then you probably wouldn't implement the stop part at all. I wouldn't think that that obviously would be bad so, but we can talk about that in the features thing on the next part, but.

A

Are there any other comments or questions for you? It seems, got his hand up, but before we go on Dan to that address, your comment.

A

Now we can't hear ya: I know, you've been having some audio issues, feel free to follow up in chat and we'll go over to you. Seen.

H

So yeah, just an FYI we've been working on the side of sick cloud provider, signal than six storage on a way to handle notes that are shut down, or at least in a failure mode. So we like we might want to like the instant quit sicker provider and signaled, especially if something happens, improving in these communities I'm linking a I kept that I'm working on with Clayton and Jim. So like this might. If we implement this into cluster API plus into kubernetes Kuban Aires, you might run into some like multiple.

H

We tries to do the same things.

B

B

So that sounds more like remediation. This is mostly just a mechanism to enable those things. Are you talking about actually stopping the instances via the cloud provider, so.

H

Yeah in the cloud in the cloud interface, we have right now called that says if an instance is shut down or not so, depending on the instance lifecycle adopted by the provider. Some providers model like a failed stage as as a real as the real state in the state machine, so, depending on the call how the code is implemented, is shut down by provider. Id would return true if the instance is stopped or failed, so it can handle both technically.

B

Okay, I'd say that like requesting, what it currently is makes sense, but is it attempting to set those states.

H

I'm not sure I understood but like what we have right now is. This is the call at the cloud interface level, so we basically where, whenever there is a note that is shut down or in a failed stage, we add an change into the not object, but we do not act yet on it, because there are a lot of concerns around race conditions and the way we did handle volumes and persistent volumes.

B

hmm I'm not sure whether the race condition would be coming from. It sounds like this is mostly a read operation, so.

H

Well, the race, the race condition, can happen. If the note, if the note comes back again, the cubelet region will register under start containers, so you might be detaching volumes while containers are writing to disk.

B

Well, I think that's tricky. It's probably something that we need to account for mmm I'm, not very familiar with this particular Kip. So I'll give it a review. Thanks.

A

All right any other comments on the.

A

Machine states before we go on to the features issue, all right, I, don't see.

I

A

So let me go over two features.

A

All right, Michael back to you.

B

So this one kind of directly related to the previous two: they talked about some infrastructure providers. It might not be desirable to reboot a particular host for one reason or another, or the functionality just isn't implemented, so something that would be nice is so when the actual infrastructure object is being created and we're sending things in the status field.

B

On that object, we can have like a feature section and that's like a section that will be synced between the infrastructure object and the Machine object, and that can serve as an informer to well-behaved clients or to end users to say, like hey. These things are supported or are not supported, and so two of those things are like stop and reboot. Obviously, the ones that were talked about here and another useful thing would be.

B

We could potentially build a web hook around this to prevent users from trying to do things themselves outside of automated systems.

B

I know to check these fields, and so the web hook could say: hey you're, trying to set the desired state to stopped I, don't have a feature field that says you can do that so I'm going to reject this request and I think it would really help to add a really polished user experience and there could potentially be other features that are, you know, needed or add it in the future, and you know we can put that here as a way of communicating some of these high-level things.

A

All right any comments on this one.

A

A

I'm not seeing any hands up my take on the three of these together is that I think they seem reasonable and I think that it's definitely something that you or somebody could prototype by adjusting the API and changing the controllers to work together and see if see how this plays out- and you know, do PRS get stuff merged in and I'll reiterate stuff that we've said for a while now that we're still alpha and we're still evolving so I think anything that makes us even slightly better is a good thing to be working towards so Michael is this: are these things that you would be interested in developing, or are you at the point right now, where you're just looking for kind of general approval and we'll get around to it eventually, so.

B

Where I'm at I would be happy to develop some of these things, but where we're at internally is we need some of this functionality, and rather than do this on our own and in cluster API ends up doing something completely different. I'd like to you know, try to do it here in. We can all kind of try it out and make sure that this makes sense as a model. At least you know for now, and that way you are not continually doing our own thing and I think that would be better for everybody.

A

Yeah I totally agree, so we don't have an official tilt file for all of our repos and if you're not familiar with tilt it's a tool that makes it really easy to make changes and get them deployed fairly quickly without having to go through the hassle of building the images by yourself and getting them deployed. So this I think would be a great candidate for a tilt type of environment, where somebody could make the API changes and the controller changes in core cluster API plus at least one infrastructure probe.

A

And then you could deploy everything together and and try it out and you could. You could have a series of complementary pull requests that when you put them all together, add this functionality, so I think you know, will I think anybody who's gotten feedback should definitely check out these three issues and add comments and I. Think it's it's totally worth trying to prototype and and do those PRS.

A

You all right, I, don't see any other feedback yet so I'm gonna, move on to your scenes topic can't be support for b1l three. So.

H

Yeah I added this item just to discuss infrastructure providers, support for v1 of the tree, especially the ones that do not have built-in solutions for load balancers, so I've seen that the control plane proposal requires a stable, API endpoint to be provided so for Kathy. What we were the one is that if you do not bring your own control plane endpoint, we will fall back into the first control plane IP.

H

So, given that V 1 alpha 3 is moving to having an API endpoint or the edge is already hit there. I wanted to discuss if there's any other solution to still support users that do not bring load. Balancers.

A

Well, when so you seen when I talked a couple weeks ago in San, Diego and I asked I asked you do you think I asked this to mush as well? Is there a situation where people would not have any sort of load balancer with a stable end point name, and would that be something we'd want to continue to support for providers like can't be and I thought we had mutually said?

A

No, that that you need to have a stable end point and that would be some sort of load balancer, whether it's engine xrh a proxy or NSX or whatever, but it that would be a requirement. So.

H

What like um there, there are environments where users won't have any load balancers, especially it were given the these first queue that you're using or you might run into some environments that do not provide load. Balancing support. So I know that we at some point we were relying. We were fall back in into if, like. If the end point wasn't set, we were fall back in into one of the IPS, so what I wanted to know is if we could implement any fallback mechanism if the API disabled, API, endpoint wasn't set so.

A

I would prefer that that could be removed and we come up with a basic reference load balancer that you could potentially deploy on the management cluster to use as the stable end point rather than trying to work with the IP addresses of the VMS themselves.

H

So yeah this would I agree that this would make it feasible, but it's still like it's still gonna be managing I, guess, multiple machines with some either proxy or load balancing solution that we would deploy out of the box from CAD V.

A

Yes, it would be some sort of minimal load, balancer, potentially running on the management cluster. That would balance through to the machines and I see a couple. People have their hands up, so I do want to get to them as well. Daniel e were first and then Justin I think.

E

I'm, just acting exactly what you're saying idea, which is I was gonna mention keep alive D, which is just a specific implementation, I think of what you were describing right, where you have a it's a VIP, it's something that the VIP is something that you know before you create or something that you can define reserved before you create the cluster. You could provide that I. Think that's one of the one of the options when there's not an external load, balancer or some kind of load, balancer service available.

A

Yeah something like that. You also could have a kubernetes service that and manage the endpoints like we can have the a machine load, balancer implementation that manages the endpoints we could. We could run H a proxy nginx envoy, whatever on the management cluster and manage its configuration based on the IP addresses of the machines. I mean there are options and I think we could come up with one at least one reference, implementation that is deployable on the management cluster and, hopefully would work.

A

I know we had talked about like pivot and dealing with making sure that all of the certs are still good. If you, if you're starting on a bootstrap cluster and then you pivot over and how does the stable endpoint work and I, don't know that we have that solved, but I definitely think we can come up with some sort of reference implementation to do this and get rid of the code. That's machine IP, based inside a cat B as a fallback.

H

Okay, I'll try to come up with at least a POC to see how we can handle this, and if it's generic enough, maybe propose it for other providers that do not have built in load. Balancers solution.

A

Yeah and I'm happy to sync up with you offline too, as well. Justin I see that you end up still yeah.

J

I was also just going to I, also played around with something similar on metal. I got my hands on some machines, and I was using like gratuitous are so if it, and that seemed to work fairly well and it it looked like we could use kubernetes as the coordinator, because this is a dovish.

J

It spends exactly how you do it, but the way I brought it up. I had a cluster running before I needed that stable IP. So we don't even even need to do any of this keep alive stuff. We can induce like leader election or something, and it would be relatively agnostic other than having to support arc yeah or whatever. The correct networking word is.

A

I'm gonna try and pronounce your name correctly. Is it mile exactly.

F

We just wanted to point out the current range of the exact same problem in metal cube project for the little set up and we were select looking into having. We are interested in participating in to the getting a reference implementation together like, for example, a typical F, G and H epoxy on the node, because we will anyways need to pivot from our bootstrap cluster. That is disappearing, so we're working on this epsilon outside. So we would like to calibrate.

A

Great so I think I know, don't believe he's here now, but mosh has been working with you. Seen on some of this, so I think the my best recommendation would be to sync up with motion you seen and anybody else who's interested in working on this and Scott. Some spare cycles generally. What we are potentially proposing for the future beyond the one alpha three is that there could be a new machine load, balancer API that is very similar to how we have the split between between machines and infrastructure machines.

A

So like there's a machine and there's an AWS machine or a GCP machine or a beast, your machine, and so we could do something similar with the machine load balancer, which would be generic and then sort of, and then some sort of specific provider like an AWS ELB or anything else that you can think of nginx and so on and that, ideally, we could have this as a first class API element that anybody could take advantage of, and it would allow us to separate the machine infrastructure and cluster infrastructure from load balancing.

A

So if you were on AWS, for example, and you're using CAPA the AWS provider right now, you're forced to use an AWS elastic load balancer, it's provision for you there's no way to turn it off. There's no way to change it. And if we have machine load balancer as a first-class element, then you could use that out of the box or maybe, if there's something else you want to use.

A

You could switch so I think it's a nice path forward and it really will help unlock or unblock anybody that doesn't have a built in load balancer. So anything that runs on Prem or not in the cloud, including bare metal, would be able to use something like.

A

So there is a a Google Doc that I can go, dig up and I know as part of the what I mentioned with motion you've seen before they're working on doing a prototype for cat B specifically, and if things look like they're working fairly well, then, maybe for b1 alpha 4 or whatever comes after alpha 3. We could consider trying to promote that as a core API in cluster API.

A

Anybody have any other comments on this load-balancing topic.

A

Right over to you, Warren for the IDI test framework.

C

D

I've been recently working on some webhook related stories and I thought of writing like an end-to-end test to essentially ensure that everything was hooked up correctly and I thought it occurred to me is if I could use the end-to-end test framework that Chuck initially gonna proposed within cluster API now I know the intention of the a to at ETV test framework was meant for, like the providers, but I was wondering if there were any reasons or issues for using it within capi. One thing I noticed also.

D

There was quite a bit of firm like duplic, some some duplication and helper functions and stuff like that in capi, so I just kind of wanted to get understanding from a community is this something that would be useful to you know have a set of end-to-end tests within capi that uses the test framework or you know, should I just go about creating get up issue and have a discussion there before I create a PR for it, um or is it like an anti goal? You know to just even using into an test framework.

D

We think happy Jesus.

K

So I think in general I. Definitely like the idea of leveraging the EDT test framework within capi itself, where we can I think the one challenge that we might find is, if anything actually requires some type of a provider. We would probably have to also add in, like a deployment of the example provider that we used to use in v1 alpha I.

K

Don't think we really use it in any of the v1 out for 3 testing so far just to provide out that kind of missing functionality, but that outside of that, hopefully would be able to reuse we'd, be able to leverage it in in capi and then also kind of leverage, the same framework outside and the individual providers as well.

D

Cool yeah yeah for now I think might only use cases or the test cases rather I had in my head were for just plus rating API, slowly I'm, not necessarily any external providers but I I'll. Keep that in mind. I do hope. So, what's the as an actual item like do I go about creating a like I sort of get a you can explain this or do I just create a PR or and then have a discussion on the PR itself.

A

If you want to create an issue that it'd be nice to be able to do integration, /and and testing of web hooks, that's fair. We also have an umbrella issue for web hooks, which I haven't looked at recently, but it may kind of check box for wanting to do and end testing I.

A

Don't I, don't think is really controversial. So the issue, it would mainly just be a placeholder, so you can say I'm fixing it with this PR but yeah I. Think if you wanted to do some testing or as some testing that can run the web hooks that'd be great.

D

Well, Andy Chindi an umbrella issue for the webhook stuff. Do you know what number that is by any chance.

A

Let me come back and I'll I'll find it for you, assuming it's still around.

A

All righty I don't see any other topics and before I go on to backlog grooming, something I should have done in the beginning. Just looking at attendees today, I see some new names, so no pressure, but if you're new and you're interested in saying hi and introducing yourself I'll, give you all a minute or two to do so and again no pressure. So if you're not interested in saying hi, that's totally fine.

L

I will say: hi I'm listed as a Miko on the list there. My name is Michael McCune I work at Red, Hat and I'm just kind of curious. What this group is doing so I showed up today to listen in and see how things are going. Hey.

A

Thanks for joining welcome.

M

To Malta from Red, Hat and I, just kind of didn't fly on the wall' mode.

A

Welcome anybody else want to say hi.

I

Hi hi Dianna Pidgeotto from classics I'm first time that I joined this meeting, just curious to see what you are doing. Thanks.

A

Great. Thank you.

N

Our vendor from AT&T also my first time joining this particular meeting but I've been following the path really I work for a while.

O

Hi I'm Robert Pham, where he's from pivotal, had a chance to meet several people at a cube con been following along mostly the vSphere cluster API stuff, so first time attending this meeting, also in fly-on-the-wall mode.

H

Hi, it seemed I'm working on cap, V, good Rob provider and same here. First time is as this into this call.

A

Alright, thanks everybody just an overview. I know: I didn't really.

A

We didn't have any PSAs or demos, but in the future, if there's anything that you want to show off, we certainly welcome and would love to see that and it's a very open discussion as I. Imagine you probably seen so every time we lead up to one of our weekly meetings, we'll have will have an entry in the agenda document and feel free to add discussion topics beforehand and we'll go through.

M

A

Alrighty I'm gonna move on to backlog grooming, so this is the part of the meeting where we take a look at every open issue in cluster API. That does not have a milestone associated with it, and we currently tend to use three milestones beyond the blank one. So our current release of close to API. That is what we tend to call D 1 alpha 2.

A

Is this V 0 to X milestone, and we use this for anything where we know that we need to backcourt a bug-fix, so we'll create an issue for 0.24 b 1 alpha 2, but we generally do not. We don't do new feature work in the 0.2 release stream. We have V 0 2 3.0 for our upcoming release, so everything that is being developed on the master right now ultimately will make its way into 0.3, and then next means that we have triage the issue.

A

We've gone over it in this call or the author of it or one of the maintainer x' has reviewed it asynchronously and decided. This is definitely not in scope for the next upcoming minor release, and so next basically means we've looked at it and we've talked about it or someone has made the decision that will deal with it in the future and at the end of every release cycle, slash the beginning of the upcoming one.

A

We do try to do a backlog grooming, where we look at everything all of the open issues and try and go over them to make sure that we don't lose track of anything with that. That said, I'm going to start at the bottom and go with the oldest open one. First and I know Chuck is not here today, but this was about trying to include files differently, based on the the choice of the file content for the convinient bootstrapper. This was something that had been moved from the old heavy K repo I.

A

Don't see us doing this for the next release, given the amount of work that we have so I'm going to put it in the next milestone and we can revisit this at the next cycle, and this is an open thing like I'm gonna tend to talk a lot because I'm just going through these, but if you feel strongly that I assign the wrong milestone or you want to change the priority, please feel free to raise your hand and speak up. I. Don't want this to just be me deciding everything alright.

A

Next up, we have cluster API deletion, lifecycle, improvement, and this was a proposal to.

A

Go as part of the cluster deletion reach into the workload cluster and issue deletes against everything in there before terminating all of the instances and tearing down all the infrastructure for a cluster, and this was partly in response to some logic that we added to Kappa to go, find any services or basically go find any elastic load, balancers that were associated with load, balancer services in the workload cluster and delete them, because otherwise we end up with a dependency violation error and were unable to delete the AWS resources that we need, and so this proposal was let's have cap capi go in and just tell the workload cluster go delete everything to try to avoid having special code in the various providers to deal with that.

A

So I don't see this happening in v1, alpha 3, so I want to put this in next Jason I was.

K

Just gonna say one thing I'd worry about here is you know if we do try to go and just clean up all resources underneath the cluster that might have unintuitive unintended side effects for the people who are managing those clusters.

A

Yes, although I think generally, if you're trying to delete a cluster like you would expect that any resources associated with it would probably go away right. Well,.

K

I think we're assuming we're trying to assume user intent here and I think almost any way that we go with this. Is it's not going to seem intuitive somebody yeah.

A

Michael you've got your handout.

B

Yeah I left the same comment in this, but my concern is: if the cluster is you know down, and so it might not be able to respond to these requests and would probably need that clean up logic and in the cluster API anyway. So I don't really see a reason to try to duplicate that effort, because we don't want to leak resources. If the clusters down.

A

Yeah, that's fair, well, I put it in next and stuck it on the backlog. Priority wise, so I think everybody has had valid comments, so we can revisit this next cycle and close it or defer it again. Hard work on it we'll see.

A

Already we have a whole I think we have a bunch of bootstrap provider ones, although this was this is a request to be able to customize the cubm call specifically now some of this. Some of these were things like I think that some of the flags are fields in the cube idiom configuration types, although it might I think we might need B, 1, beta, 2 or newer to get to some of them.

A

I actually was hoping to be able to customize the log level, which is not something that we can do today, because the cube idiom call is hard-coded, so I think this would be a nice to have and I think I. Don't remember specifically what was in the back and forth here, but in terms of milestone, I.

A

Would be okay, trying to put it into 0.3, but market is a long term important or important long term and if we get around to it, create and if not we'll, just defer.

A

Michael I see your hand is up. Is that from before, or for this?

A

Okay thanks, alright I'm gonna put this in the milestone and we'll go long-term.

A

Okay, I guess my proposal to move the Machine bootstrap data and not have it as an inline string and make it as a reference to a secret because we store it as a string on the cube, ATM config for the cube, ATM bootstrapper, and then it gets copied over the machine and given that it may have some sensitive data, it should be in a secret instead, and this one would be a breaking change from an API perspective. I don't know that it would really be convertible between alpha 2 and alpha 3.

A

If we were to implement this change because we're talking about moving data from an inline field to an external object. So I don't know if this is too disruptive for alpha 3 and we need some more time to plan it. But I definitely would like to do this. Jason I saw your hand briefly go up.

K

I was just going to agree. I think we might have some options to potentially help with the conversion here, but it would definitely be tricky, but I do think it's valuable to try to get in for the upcoming release. Yeah.

A

I'd like to take some time and do some brainstorming and write some comments on the issue around. What would it look like if somebody's got an existing V 1 alpha 2 management cluster with existing workload clusters, and then they upgraded the V 1 alpha 3? What a machine set continue to function? Would the control plane well cube alien control, planes new, but or you know what? What sort of issues what we have so I'm gonna stick this in the milestone.

A

I would like to have this soon, but if it doesn't get in because we can't get through all the edge conditions appropriately, I think it's okay. If it does slip, Michael.

B

Yeah I mean for me what it sounds like we should do is add a new field and deprecated this field, and then so they can upgrade the API level and then before they go to the one alpha four. You know they're in charge of migrating all that stuff.

A

That's a good idea, or something like that.

A

Thank you, okay, all right, so this is the milestone. We have support for additional certs.

N

A

A

Basically, a way to get additional certs into the trust store on the.

A

A

Has anybody run into a need to do this I'm curious.

P

Yeah yeah, so, where you've got in, say you own AWS and you have a transparent proxy. You would want to add the certificate authority for that intercepting proxy to your machines, and you can do that either by baking into the image but or by passing it into cloud in it.

P

A

I, don't know that our team will necessarily have time to do this for alpha 3, so definitely want to put health want it on this one and priority wise go as long term.

A

Maybe I'll touch base with Andrew and see if he has some timelines in mind or you seen have you talked to him about this.

H

Not specifically about this one, maybe we should sync with Andrew: okay,.

A

I'm gonna put it in next for now and come back and sync up with him offline.

A

All right we have cluster cuddle, create command, feels fails with the newer version of kind and I know that.

A

This is really only an issue for cluster cuddle for b1 alpha 2, because the new cluster cuddle that we're working on isn't going to be creating a bootstrap cluster. So, let's see.

A

Although I was using, cluster cuddle create cluster against master today with Kathy and Kappa, and it worked fine, although I was pretty creating the big drop cluster. So that's different.

A

I don't know Jason. Do you have any thoughts on trying to deal with this for altitude, like I'm, I'm kind of tempted to just say, use kind, 0.5, dot, 1 and don't upgrade to 0.6 for right now, as a short-term solution.

K

So I know we had a recent pull request that I think fix some of the error output handling around how we were wrapping kind in a cluster cuddle. I was.

A

K

A

K

Yeah so I think that should at least you know it won't fully fix the issue, that's being brought up with the deprecation, but it should you know, prevent the immediate issue that was being faced by these.

A

Okay, this may have been fixed by.

A

All right, so at the very least, this is gonna go into the 0 to X, milestone.

F

All right we have about 4.

A

Minutes left we're not going to get through all of these ok cluster cuddle, v2 contract for providers.

A

I think we need to do this as part of the Alpha three work.

A

A

A

We talked about rebooting hosts, we have a dead links issue that I think had a PR that was I, know Jason. You and Vince have been looking at this one right.

A

A

All right, I'm going to put this one.

A

Put this in the milestone.

A

Next up is supporting private registries with bootstrap kind clusters. I feel like. We just need to close this because we're not going to be creating.

A

Clusters or we're not creating kind clusters from cluster cuddle and master going forward.

A

If there was enough need, we could maybe do this in just the release. 0.2 branch, but I'd rather not and recommend going to alpha 3 once it's ready.

A

I'll come back in with the comments on this one.

A

I had a documentation, one on policy guidelines on core dependencies and tooling versions, and we we already do have some documentation saying that we need to be careful when upgrading core runtime dependencies, like controller runtime and client go so I think this is probably not a huge amount of work, especially given that, at least in master, we now have customized coming from the go module, and so we can pin that. So this is one that I was going to work on so I'm just gonna. Stick this in the milestone.

A

And we probably have time for one more, oh, maybe a couple, so we have conversion web hooks which Noah is working on. You know she's working on this right now, and this is a milestone.

A

And last one or two here and subs does not installed on Mac OS. By default, we had a PR to do something with this I thought.

A

Maybe not, we will put this in the milestone and give it a soon priority and documentation, and we have one more, which was Nadir's documentation, internationalization.

A

Any idea when you'd like to see this mid-year.

P

So yes I, despite if anyone was at their breakfast in San Diego last with the other week, and then you know death. Some people from cyber agent in Japan to the streaming service produced a Japanese language book which Jason is modeling right now, so they are willing to boyd the japanese translation, all our documentation, so yeah this just an issued. You track how we actually do that and jason suggested separate subdirectories. It looks like the way it should go.

P

Haven't got timelines yet they're gonna have a stab at it. Over the next few weeks, I'm gonna meet with them again yeah. So it's it's important long term. I. Think okay,.

A

I'll, stick in the milestone. We can always punt it for later if needed, and that's all the issues we're at the top of the hour. So thanks everybody for attending and next week we're gonna try and do a discussion on the proposal process. The release process see, if there's anything that we can improve upon and I'll see you all next week thanks.

A