Cloud Foundry CF on K8s Forum, 1 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CF on K8s Working Group Forum Call Recording Nov 1st 2022

Description

2022-11-02

[DB] Upgrade test on CI
[RI] What is our path to a 1.0 release? What are the main features/capabilities that Korifi is expected to have that it does not have right now?
[GC] Validations
[PC] ReplaceableNetworkerProposal
[KB] yq in makefiles (again)
yq installed in Korifi subsystem bin like the other binaries we use
…

A

B

All right we got the recording going so welcome everyone, I'm posting the document in the chat. So if you have topics you would like to add, please do so. If you cannot add them, please drop and we'll chat and I can do so for you.

B

So if we look at the document, it looks like we've got currently one topic, but you can give people a couple moments to add them.

B

But while we wait for that, let's get started so we.

A

Have it, this is just a reminder from our previous conversation previous meeting, that we should probably have some sort of checks, whether the upgrade between releases work. So just to let you know that uh I think on Monday, not sure, but it doesn't matter. We introduce State change in our CI that, after every build, it would uninstall query and install latest release latest official release, which is currently zero, four zero and then on subsequent run. It is going to deploy the current commit on top of it.

A

So we hope that if that fails, this means that the change that's being tested somehow breaks the upgrade scenario, and it's looking into that's it. Yes,.

B

um That makes sense, we should definitely look at making sure that we are have a upgrade path and let me know what will break any breaking changes Etc and all that, so yes definitely should be looking at that.

B

Unless someone's disagreeing with that notion, I think we all feel like that might be worth pursuing.

B

Cool all right, I see another topic, which is our path to a 1.0 release. One of the main features capabilities that creepy is expected to have that it does not have right now.

C

Yeah this question. Basically some comes from a lot of folks on the quote: unquote marketing side who keep asking us when we can expect like a major Milestone release and what are some of the features that you know be um will expect to have and um what it is. That will actually take us over the line um for a 1.0 release or like a major um push. So yeah I was just curious in terms of what the roadmap is and I.

C

Remember, folks, um having like a major milestone for like the first releases, um which was you know, obviously getting into work and then getting the auth and other stuff to work and I know recently, like the install process moved to like a help based uh one and so are there other?

C

You know Milestones or things like that and that we can keep track of or that we can communicate to uh people who ask us this question about what is like the um feature roadmap for the project.

D

I would say: uh I, don't know exactly what's missing, but like one thing that is definitely missing is like being stable, like being like what I would like to play. I would, until we are confident that this thing can actually run in prod. Just like CF does I I wouldn't call it 1o, that's what I'm saying so like how do we get there? It's an interesting question but like to me, 1o would have to be like stable enough that, like to me, wanna, would communicate the message. Okay, you can actually run this in prod.

D

Just like you do with the other CF uh we can think of how to get there. I think it's.

D

The feature. Surface is interesting because of course, we'll never cover 100, so we could con. We could think of which features we think are like in terms of like surface. What surface do we want to cover before we can call ourselves 1-0, but then, on top of that, it's like all of this surface needs to be like. Actually, it needs to work well enough that we will be confident to recommend people to deploy this in prod and give it to the development teams. And you know.

A

One of the questions that I'm often being asked by colleagues who are looking at Griffey are basically always to first one can scale for large deployments, the second one. What about multi-clusters? Well, the multi-cluster support is probably a very, very long shot.

A

I think that we should come up with some performance metrics, but honestly I, don't know how to measure and buy the book. Every time. You're running some performance test. You need to have a Target I, don't know what sort of Target we can set.

D

Yeah and the two things are probably linked like the whole scaling.

D

Like the answer to the scaling question is probably multi-cluster just because we know that at least for now, people seem to have experienced problems, scaling a single cluster to the sizes that CF can reach with a single installation, but yeah, who knows I mean maybe maybe kubernetes.

A

D

Don't know much.

A

Clusters is not only about performance, it's also about security, for customers who are quite um but don't think about isolating their data from the rest of the.

D

Yeah I mean I'm, not talking too so much about performance, but like scale like how big can you get before it falls apart? Basically like we know, there are CF instances that are like huge. They have many many many many Diego sales and many many many many workloads running uh I, don't know how much a kubernetes cluster can like how many nodes will a kubernetes cluster handle before it falls apart and how many pods like it looks like at the moment.

D

People do multi-cluster mostly because they can't fit all their stuff into one cluster, but I could change um in the future, like maybe kubernetes is going to get better at handling more nodes like more sales or whatever they're called, and more pods and Etc.

E

I wonder how like, if this is something we need to test? How are we going to test it, because it's this pretty expensive thing to be doing as well? Isn't it yeah? It feels like something you want to partner with. If somebody wants to use karifi and at scale, we can work with them to make it work. If we borrow some of their resources to see what that scale looks like I,.

D

Also wonder if this, if this changes by provider like I, don't know, maybe I, don't know, if you do your cluster yourself, you can push it further. I, don't know versus having the classic uh like Cloud clusters from the main vendors I, don't know, but yeah. That is definitely one question because like if we tell people- okay, it's one or, but we only do endnodes. We only run on one single cluster I think it would be a fair thing to say, but.

D

I, don't know how many people would find it useful to run Curry on a single cluster. If you see what I mean like how many users would need to run Griffin prod, but on a single cluster, how realistic is it.

E

Stats here on what Kate's thinks is big for a cluster, no more than 110 pods per node, no more than 5 000 nodes, no more than 150 000 total pods, no more than 300 000 total containers. Large.

D

Yeah, that sounds pretty big.

E

But I think I've seen people say take it stuff falls apart at about third of that level.

D

ah Again, like yeah.

F

Yeah you can't go to the the outer edges of all of those envelopes, um necessarily I. Think I've seen some presentations about like scale limits that you hit in the API um that are a result of different sets of resources, kind of compounding as well. So there's a there's, a more complex polytope um that describes the actual practical operational limits that I'm sure is changing all the time, but yeah I I, think that you know on the multi-cluster issue.

F

I've definitely had the impression that, even before you hit those scale limits, people are interested in this as a way to get harder tenancy between different organizational groups at the cluster level, instead of just at the namespace level.

F

So I would expect those uh those concerns would be things we'd want to represent in any kind of multi-cluster proposal or where they want to be able to say, like different Works get different Hardware or something like that. You know.

D

Maybe is this something that maps to isolation segments.

F

Yeah potentially I mean you, could you know I think we've been trying to find ways to map like um you know the those existing org space and isolation segment, Concepts onto different kubernetes concepts, or to figure out how flexible that needs to be.

D

That could be interesting, so remember. Isolation, segments in CF are like very much a deployed concern like when you deploy you kind of tag, the node, the Diego nodes that you want to be isolated.

F

uh Yeah, if it's, if it's just an infrastructure, isolation issue, then you could probably solve it with um different node, selectors and taints and mapping those onto the workloads so that they're, segregated onto different physical or, like virtual machine nodes.

D

But but that will be inside the single cluster still right, yeah.

F

But again you might say like no I, for whatever reason I want these API instances to be separate, maybe because I, don't trust the case API or it's not hard enough tenancy um but I mean I'll just say that still looks like a complicated and um subtle track of work to do at some point.

G

And I think to kind of trying to come back to the original request around like what does a 1-0 look like and what like, when we get people asking about it like? What's our response, ultimately I think we're on where you were coming from like to me. It feels like our big thing is stability, but we can't get stability without engagement from people who are using it.

G

So I would say if somebody's asking for 1-0, like please redirect them to to reach out and kind of interface with the team would be my thought, um because I think we want to see and work with them understanding their their needs and know what what that means that may also bring out more features that they care about, that we want to build into before we do a 1-0 I, don't know if that's really the answer you're looking for ROM, but that's where it sounds like we're kind of going towards I.

F

Think there might be a distinction there between end users, who might be asking about that and um people who are marketing this are covering it and they don't they're, not planning on being end users, but they kind of want to understand what the trajectory is so I think like for anyone, who's a potential end user, just having more direct engagement with them about what they would expect to see in that that sounds that sounds productive, I think with 1.0 yeah in general.

F

We, you know I, think we need to hear what is important from the community for that level of stability and their expectations. So I don't think we can give a super precise answer right now about like 1.0 needs like this this and this and then done.

D

And every time I think about it, I think of so many things that are missing and like okay, we need so many we have so many potential API features to add and uh the all these, like big questions about, multi-class, that there is the services story. There is the buildbox story, there's like so many I think big things that are still missing, that I just yeah it's hard to to to estimate when it just this just tells me: okay, oneo is far right because we.

A

D

Have to take all this big question all these big questions, so it's something that I maybe will think one like I feel like when we are ready. We are approaching wanna, we'll we'll feel it because we it's going to be harder to find like big things that are missing big gaps but like we have definitely far from that. So it doesn't really concern me too much to think when it's gonna is going to be one hour, we're mostly thinking about okay. What's the next big thing, we can the next big gap we can.

D

We can close basically.

C

Yeah, thanks for all those inputs, um Phil I, think to your point. uh Eric's answer was very uh precise in that there is two very distinct audiences to whom we need to address questions, um I'll, absolutely have engineering teams or like engineering, orgs or technical folks.

C

You know, engage more with the community and try things out and let us know what's really missing, but there's also just you know, analysts and PR folks, like the public relations PR I have folks that are um you know interested in coverage interested in learning what the positioning is and my answers have typically been. uh You know we don't have the services story in yet um we've not figured out um certain things about um how you would push an existing team that had like a bunch of Docker containers to use corifi.

C

So we don't do that kind of deployment, and so those are some of the gaps that I do highlight and obviously I show them coverage of um so we've thought of auth at first. That's done we thought of like installation processes. Second, that's done, and so we're moving on to these other stories.

C

um Obviously, there's no like definitive answer that I've given them either, um which seems to be the case, but I thought it'd just be good to know like what are like. The big features that you know folks are thinking about: uh building into um Curry fee as.

G

C

You know doesn't even have to be like one point to release. A 1.0 is just like a like a thing to put in a question. So thanks for all the inputs, so I really appreciate that yeah.

F

Thanks for asking Rob.

A

B

F

B

Good discussion there I think it's gonna be born out by all what we said. Getting people engaged and figuring out what the shape of that is as far as like scale I think we should do a little bit of preliminary testing uh just for to see if we have any major shortcomings. But yes, we if we want to actually check scale, we need a major partner um so anyway, uh cool thanks. All uh next topic is validations.

D

Yeah so yesterday, I was like okay. I'm I've got some spare hours, uh I'm done with the backlog work. Let's pick up like an easy story to just you know, get rid of very quickly and call the day with something done: ticked in the backlog and I start working on annotations and labels and I realized. Okay, uh the this this type actually has validation.

D

So let me check the validations and I look at the validations and we have these like API site validations and I realized that I was like okay, so I should test that the validations are actually applied and there's no tests for any of the validations that we apply to all our API types.

D

So I started thinking about how to write those tests and talk to the team, and the team was like: why do we need these API site validations in the first place because we're validating stuff?

D

And then this date has been sent to the kubernetes API, where it either needs to be validated again or otherwise? People will bypass the validations just by going in by a cube cattle, for example, um so also some of these validations, for example, for labels I think they kind of try to anticipate they try to catch problems. That will things that will break on the kubernetes side up front, but sometimes they do it not exactly like kubernetes would do so. There is some sort of like inconsistencies and stuff.

D

um On the other hand, there are other places where, like we don't validate on the API side, we only validate on the crd side, but uh it looks like it's a bit tricky to get the error from the hook. That validates like the the admission Web book that validates the resource and produces an error for that thing.

D

Fails but like producing the proper error message in the API from that failure seems to be tricky like we take the arrow and we kind of serialize it and stick it in the reason so that it gets deserialized on the API side and then presented to the user. But it also doesn't feel great because the reason is not for Json serialized, Json stuff.

D

It's for Plain language, like it's from plain English reason of the failure, uh so I don't know exactly how to do it, but I will be really tempted to just get rid of all the API site, validation and just try to find a way to do everything in the hooks and make the API just present errors coming from the hook somehow, uh so that we avoid the duplication and uh yeah. We we just delete all that stuff and we don't have to bother about testing it. If we want to keep it, we should.

D

We should start testing it, because the pretty much no tests about how we configure those validations on which fields and, for example, for metadata, we have two types: one is I, think supposed to be used for rights and one is supposed to be useful reads, but there's no test that will fail if you swap them out so, for example, nowadays we only value, there's a bunch of places where we only validate patching of these labels and annotations, but we don't validate when they get sent the first time for the creation of the resource.

D

So you can create stuff with invalid labels, I guess. But then, if you try to patch them, then the validation will fail, because the creation is not validated, but the patch is validated and I think this all comes from like just not having good enough tests.

D

um So we, if we want to keep them I, guess we should test them, but also it feels too much work to have these things done twice, because it's also a problem that some things are only validated on the API side right because they can be bypassed.

H

Yeah um well, first I fully agree. It probably should have been test. I think it might be worth having like surfacing some of this stuff in slack, because I'm I'm not sure the people who worked necessarily on that are on this call. They might have more detail there. One one hunch I have for why patch might be tested differently than create or or not tested, but implemented differently is because um Cloud controller has some special behavior for patching labels and annotations that doesn't exist in the kubernetes API.

H

So the way the remove one is like you like send like null in that request for a specific key and that will remove it and I. Don't think I think that on um kubernetes API you just delete the key.

H

So there's like there's some subtle differences there, so I imagine they had to to do some API specific things for that part, but other than that, like the only reason I could imagine, duplicate validation was for air Message like ease so that I don't know if there's a way to get just get that from the rejection from the kubernetes API. That sounds reasonable. We.

D

Do it in some places, but it just doesn't I'm, not very happy about it. The like we, we basically come back with a Serial given we need. We need struck like a semi-structured object to come back, but we can only send back a string. We just serialized the object inside that string, but that means that someone who's using the API just by who's just using the crds they'll, get an error with reason, a bunch of Json that is unreadable but yeah I, don't know I.

D

Guess uh it's also a compromise that we need to strike somehow because yeah. Otherwise there might not be perfect a perfect solution.

D

But like duplicating the logic, it just feels wrong, and maybe we should just you know, get rid of everything and.

H

Yeah, it also might be a case like some places, it's very important to have the same errors, especially for older API, endpoints and stuff, but I.

C

H

Feeling this is a place where, having the exact same error, messages, Cloud controller doesn't actually matter like the the CLI hasn't even fully implemented um this full feature set. They only support labels right now, like I I, doubt anything is keying too hard on the the validation errors here.

D

um Yeah, it's just I want people to like do something and get a failure that doesn't really. It doesn't allow them to figure out what they did wrong. Yeah, because.

H

That's just right right, I guess, I'm saying is: if we can bubble up something that tells them what they did wrong, but it looks different from.

D

G

H

That's probably good enough. Mm-Hmm.

D

Yeah but I guess we can take a deeper look at this and uh just uh because to me if we could just eradicate the whole validation layer from the API. That would be amazing, because then we it means we have one place and one place only where the validations occur, and that needs to be on the kubernetes API side, because that's our ultimate API, like that, that's our innermost surface in a way uh putting everything on the API is not feasible because people can just bypass it.

D

So that would be great, but.

H

D

Me I'll just post something on slack and see what people think I.

H

I do remember there being some conversations and some like some like trade-off conversations that may have led to this I I. Don't fully remember what they were, though so posting on slack might surface. Some of that.

B

Yeah it's more than just bypass the API. It's then you have indeterminate Behavior, so that gets kind of bad yeah.

D

Like labels are special because, like they're validated, given they map so closely one to one kubernetes, uh we don't even need to validate them on the kubernetes side, because kubernetes does that already uh but like we still try to validate them at the API set, trying to imitate what kubernetes would do, but just not exactly so. That means some things get blocked at one level, some things get blocked at the other level, some things just slip through also on the API level.

D

We try to reject API labels that have cloudfoundry.org in the key, um because I guess the idea is to reserve those keys for us, but that again can be bypassed anyway. So.

H

Yeah we're only doing that in the API. That's not sufficient. um Yeah I.

D

Also I don't know, I would just stumble that if people want to shoot the Samsung shoot themselves in the foot and mess with our own labels and annotations stuff will probably break yeah.

H

I think I think that question becomes more interesting in the future, epic that we should will probably come sooner than later around propagating some of these things down to like deeper workloads like like the like the staple set or the Pod or the.

C

H

Whatever because then, you can actually have real, probably bad consequences, if you like, steal stuff, but I, think we could solve that in our controllers, with some type of allow list of what is allowed to be propagated, the what's in the API right now, it's like not sufficient and probably not actually helping a ton yeah, but.

D

Yeah so I will, before I, carry on with, like the whole annotations and labels thing uh I will will probably I'll take a look at this thing, see what we can do to just to just get rid of it, and that kind of also eliminates the need for testing them and and stuff like that, because it's lots of code as well. It's just like there's lots of validations that we do.

D

Which I don't think it all it all just goes away? It's probably a lot of it probably needs to be moved into an admission. Webwork does.

H

Yeah, maybe it's the case of the kubernetes API error is not rich enough. I'm, just wondering why we can't just rely on that. Failing.

D

I think the problem is mostly like returning the error in a sensible way, because I think it just fails with an error that might be like I think it's a 400 or 400, and what is it for four or three, maybe like it, may look like you're just not authorized it. Just doesn't try to give you lots of information.

H

What led to this a little bit then like they just weren't, getting good errors out of that yeah.

D

And the problem is, if you want you can, if you want to bypass that again, you are just sticking serialized Json in places where it doesn't belong, but maybe it's you know the the least bad thing to do uh kind of, because in the end you we still have to validate at that level of abstraction and like duplicating every validation.

D

It just feels like too much work.

D

But yeah, maybe I'll do an explore or something and let the team take a deeper look at it and see. If we can what solution we come up with yeah.

H

I mean even before that, like I, think just being on slack just like. Why did.

D

We yeah yeah yeah, okay, I'll, definitely raise it on slack I.

H

I think some some folks might be able to actually just give a quick answer that could inform that Explorer or like cut down on what we need to poke at.

D

Yeah because maybe there are good reasons not to get rid of the thing and then it gets more complicated yeah than what I'm thinking.

B

Yeah, that makes sense to me it is probably people's knowledge and they probably are like yep. We did it this way because of some weird thing, but it's still worth uh pursuing uh pushing as much as we can to the kubernetes layer.

B

Cool um just a time check, we still have a bit of time. um We have two more topics. Replaceable networker proposal is a document link that everyone, I believe should have access to. If you don't, we can probably figure out how to get you access, yeah.

G

Yeah and I just put this on here, because I think it came out. I think it came out last week and I know we had people at kubecon and I just want to make sure everybody had a chance to take a look at it and go talk about. It is really it.

D

Yeah I think all the discussions on the document itself are resolved, so I guess what we're looking for now is just an approval, so that I can basically just IPM the stories and get them into the backlog, because I didn't want to do that before the proposal was officially approved by someone else, some someone other than me basically, and once we have that I'll just schedule the stories it's actually little work I think um it's not too much.

B

um Yeah I can definitely read through it and take a look at a bunch of the rest of us can as well might be worth this uh making sure everyone in the creepy, Dev Channel I, see there's already a link uh from Monday but just sort of a Last Call on it and say hey everyone's kind of on board with this. If you have any comments, make them right now, because.

H

D

Otherwise, again yeah.

H

I I didn't have a chance to really look at until yesterday. um I added one comment this morning for Westworth. It doesn't change the anything too deep. But it's just more of a concern about the number of separate reconciler separate, like images and components were splitting out versus rethinking some of that and maybe just making them conditionally added to the controller manager at runtime.

D

That's interesting so I make one big image anyway and then just dynamically at startup leave some reconcilers out yeah. It could be a way we do it.

H

The reason I bring that up is just um we've. We've started to see just that when you run these on larger environments with other stuff running um they they start to bloat like the when they're all running as one process, they can share an Informer cache and everything, and you can just give that single deployment a ton of memory.

H

But if you have like a bunch of controllers running, then operators need to know like I need to give this one four gigs, this one's six, this one's okay with two and it like the design, looks really good like from like a diagramming and just like a discussion standpoint but I think like practically. It might be better if the default ones can still run as the the main karifi controllers.

H

D

Interesting because I wonder how that impacts, the helm chart as well, I guess, because if we move to one image, that means we don't have the sub charts. I mean anymore, potentially.

H

Potentially I'm.

D

Not I'm not even necessarily saying we.

G

H

What we did for the the others right now like the X workload ones, because those are like the way they are and we could consider consolidating them again and it's more of just like CF route is like yet another and then like what, if we find another and it's like I, don't I'm worried that we'll eventually end up with like a separate component for each controller, which doesn't.

A

H

Seem good, so that was that's my only concern with with this everything else, like makes sense to me,.

D

Yeah I mean the way we do it is I would separate the component to its own image for now. But then, if we think this is a problem, then we could think of a solution for all components like because otherwise it would be weird to have some to be out and some to be in uh yeah I.

I

Guess I feel like if we're gonna do that we should probably fast power away with consolidating them, because it's it's kind of getting unwieldy, yeah, I, think and I think that concern about memory is real. I was just looking this morning at declining a cash code, for other reasons, for the controller runtime and it's incredibly.

C

I

It's literally just a nap with absolutely no logic to throw anything out from the map, so I.

D

See so it just goes forever, and this.

I

Is this is the metadata cache I'm, not sure if I've found the right caching code for for all circumstances, but when the controller runtime plan is just stashing metadata in a map.

D

I'm, honestly struggling to think of a separate of another component to extract this was like the last one that logically was missing. Like we've got, builds running, we've got tasks running we've got networking what else could come out? So maybe we'll you know we'll never have to think about yet another component, but yeah the.

A

D

We have they're going to become API controllers, the two runners, the Builder and the router. So it's six. We.

E

Could also be more sensible about what we're caching I think I think as soon as you do a get or something in a controller runtime code, whatever you've got, starts being cached the whole lot of it so go and get a pod or something and all the pods on a system end up in there. Even if we don't really care about watching them or anything, uh I think you'd be explicit about what you want to cash.

E

So we want to go and cash all the things that are going to trigger our control lists and maybe exclude everything else. That's.

D

Interesting I, don't know you could do. That is.

E

There a way to do that: yeah there's some options when you create the manager or something you can control the cache, you can pass some cash options, I think.

I

There must be a different cache yeah.

H

I think that's the important former cash versus the client cash. Oh.

F

Right: okay, well,.

I

I guess they're all client.

H

Caches but I think there's there's different client cash just but.

D

But if, if we have a memory consumption problem, maybe we need a memory. Consumption investigation which looks at how do we reduce memory? Consumption on the individual controllers is that the way to grow is consolidating the way to go, and then we can I guess reason about pros and cons of each one of these Solutions like it looks like the real problem. Is we end up taking too much memory right.

H

Sure I mean that that's just one example I think um yeah, there's just.

B

G

More like philosophically.

B

We want to try to figure out whether we want fewer images that are a little bit larger, that might have some unnecessary things, bundled into them or like more discreet, smaller images, but more of them, because they all have like pros and cons right.

H

Exactly like I'm, even just thinking of like an operator who might be installing this on an air gap thing they now have like much like you could have one like thick image. That's like 50 megabytes or something like we all use like like thin base layers but or you could have like 10 like 30, gigabyte or 30 megabyte images like it. It just starts to add up as you're tracking, more versions of them and stuff over time and.

B

If you need to move them from like one registry to another or like people of the memory clusters or you.

C

Know there's like.

B

Lots of things rate limiting. We saw that at kubecon, like everyone got rate limited immediately by Docker, Hub, so I think it's worth probably some just philosophical discussion about which direction we want to go, maybe put down all the things we can think about brainstorm like Pros cons for each approach and.

D

Yeah, because I don't really have a preference for any of those people, but like I, can only see two ways either is either we have one in one image for each thing or we have one image for everything or potentially, as they were suggesting like the API being its own thing, because it's basically just a shame and then the control is being a big image with some controllers being. You know,.

D

I

D

Don't hate any of the two things? So if one comes with some benefits, so again, the surface of the ham chart would change. I, guess some I don't know the impact. What the impact would be, but.

I

Probably not huge right, it would simplify.

D

Yeah I guess yeah I! Guess it's not that bad, because in the end I think you can keep the values shape the same. Even if you don't use subtracts anymore. Maybe it just simplifies it and that's it.

I

Yeah I just feel like with these controller managers like the amount of actual controller code, turning on or off or Packaging is pretty small compared with the surrounding ecosystem, you need to run a controller manager right.

D

So there's lots of overheads.

I

Typically, yeah.

D

Even code, just looking at the codes, there is lots of boilerplate, basically.

I

Yeah and now we're stamping on six copies of the same boilerplate uh yeah supporting libraries which for what's effectively like six different records. Oops.

C

I

It well so yeah, that's that's kind of my opinion. I guess! That's fine! I.

E

Just I found the code it's at the top level when you create a manager. You just set client disable cash flow to a list of objects. You don't want cached for that manager, so it is a top level thing with standard Cash. That's.

H

Probably something we want to do regardless, but yeah.

E

So, no because we saw that in Irene, when somebody did a um load test on CF or Kate's um forget who I was, but we got a whole load of emails, saying Irene was crashing because it was using too much memory and I think we looked into it then, and we just saw this cache was populated with a whole lot of stuff. We didn't expect in the and.

I

D

I

E

Everything You Touch gets Cash for Life and not just the thing you touch the entire list of it.

B

Cool so sounds like we need to have some more in-depth discussion about many.

H

B

Aspects of this yeah.

H

I'm I'm happy to start a doc, and then we can start throwing stuff on it.

B

Thank you, Tim. That would be fantastic.

B

um Oh okay, hey we're just fun that I'll spin up spun out of the replaceable Network proposal discussion.

B

um We have about 15 minutes left. uh There is another topic which is something in make files again is that.

E

Okay yeah, so it's the other day when we're doing the um Helm file stuff we found we need to do some um tweaks to what control the Gen was generating. So we can reproduce. What customized was doing so we're using said, and then we found out that said on Mac is the BSD version by default and it has slightly different syntax.

E

um So we introduced yq and then.

G

E

It was a problem with our workstation setup, between the between Europe and America, so I think America you've got the other version of yq installed and used in Scripts.

E

So anyway, we went back to some solution where everyone had to install G said if they're using a Mac and we've had our first real user go and raise a bug about it. So we considered thinking about yq again so I mean I, I, think I suggested at the time we should just install the version of yq for the make file to use inside the make file like we do with everything else.

E

um If Matt was unhappy because he thought that that would you know, break the rest of the system, but to say today we installed it in like API bin controller bin, the way the place where controller gen gets installed. It's not going to be any path, it's not going to interfere with anything, and that means we can use yq to do the yaml manipulation now and it's loads better than using said it's very clear.

E

What it's doing it goes lacks on the tree, rather than just looking for text and putting stuff in at the right indentation. uh So yeah just heads up about that. It's gone into main. Don't expect any problems with people running stuff on a Mac, we've verified that it does generate the right stuff, still um so yeah. It's there just be aware, but I think it's fine.

B

Cool yeah, it's local uh like some of the other binaries we use, then I, don't foresee any major problems. um It's nice that this is sort of uh the common platform. First approach, which I know most people are using mags, but the actual where things run in CI is Linux. So probably.

E

Can use the smoke we don't over here by the.

B

Way most people in the US tend to be using Macs I'm, one of the holdouts that uses Ubuntu.

E

I use a Max or on a VM with Linux on it. Unfortunately, just so I can have email on the same machine. You've.

B

Got everything um I get the fun of blowing up my machine every once in a while, but you know it's: it's cool I told the people at canonical and they said, oh that shouldn't happen, but um cool yeah.

B

uh Thank you for the the heads up on yq I think that's a reasonable approach to make it like that contains but useful tool and we'll see if there are any issues that arise but we'll probably back home, so cool I see an anonymous frog is putting something in then there's an anonymous dinosaur, damn so many anonymous creatures.

D

They're crashing our.

E

Especially in karifi subsystem bin, it's a API controllers, cool.

B

Sounds good to me um if anyone else has a other opinion, please let us know I've been known to be wrong frequently, so you know.

H

Yeah I was just writing that down in case Matt saw it I think he was only concerned with it being installed globally on his workstation. We talked about it later and he was happy with it. Yeah.

I

If it was him, he went in the end, he went and switched to it, but I mean other people who it's just that it takes different command line options. So if you're used to one.

B

Then I'm generally aware of installing anything globally for anyone for any reason they should have to like make sure they want to do that. So.

I

This feels like the right answer to me: cool.

B

All right uh last chance for new topics, cool all right. If there's anything else, then I'll, let you all go a few minutes early. Thank you all for uh attending the bi-weekly Club Thunder and kubernetes working Group, Forum meeting.

B

That's a little bit of a mouthful, but anyway um I'll see you all in two weeks. Oh thank you. Bye.

C