Cloud Foundry CF on K8s Forum, 28 Jun 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud Foundry On Kubernetes Working Group Forum 28th June 2023

Description

[JH] Staging disk size setting
https://github.com/cloudfoundry/korifi/blob/main/helm/korifi/values.schema.json#L182-L185
It appears that we set this in every CFBuild object, but then we don’t use it for anything?
Julian to add a story. We should mark the CFBuild CR property as unused also, since we never used it.
Staging resources settings are under an `api` key. Could we move them to a more global location?
Julian to add a story to refactor. Make sure we don’t leave the old properties references in the docs for any properties we move. We should make sure to get proper review for the PR also.
[JH] korifi/tools unit tests: are we running them in CI? Should we be?
Yes we should. JH to add a chore. Also add it to the test-everything script?
[TD - can’t attend] What is left for 0.8?
Missing anything in v0.8?
https://github.com/cloudfoundry/korifi/issues/2604
https://github.com/cloudfoundry/korifi/issues/2605
https://github.com/cloudfoundry/korifi/issues/2645 - bug about cf push not retriggering a build when the buildpacks change after a failure. (It just gets stuck) Probably we should not block on this issue though.

A

B

Do we do we already talk about the Future Past topics or.

C

No, they wasn't sure helping.

B

Me there's one from Tim I, don't know that one that one might be a new one, I, don't know.

C

D

Around those notes from the last tricks meeting.

A

My last one was the 14th see those must be the new ones. Oh okay,.

B

I think we're good.

B

All right well I'm digging to him. Actually you got the first couple.

C

Yeah so um I was adding a new setting um this past week to our uh helm properties, and so the new setting is essentially enabling us to declare how big the persistent disk should be that uh kpac allocates for caching build artifacts from one build to the next, uh because basically we had a customer complain that they're ginormous python app didn't fit in that cache and therefore it was like re downloading. All the PIP things.

C

D

C

Built and it was super slow, um but in the process of adding that I noticed a couple things that seemed a little odd to me.

C

um One was we already have a disk size setting in that in the helm chart and we pass that to the API and the API sets it in the CF build object, and then we ignore it everywhere. We don't do anything with it, um so that seems I, don't know a little silly I'm wondering. Could we get rid of it uh or should we get rid of it?.

C

C

E

If it's unused I vote for yes.

E

It's really it's always about potential impacts. Breaking changes. Etc is.

D

E

Going to would it result into anything breaking.

E

Like what's the risk, is there if someone, maybe reuses an old values file, something like that I, don't think it's likely to break right.

C

Yeah uh and we're not 1-0, we don't make any promises that you can reuse values, files, the.

E

Other thing is what: why is it there in the first place? Is it just a mistake that we ignore it and it was actually meant to go somewhere that we would like it to go to.

C

Well, I mean I can understand why, in theory you would want such a thing, so I mean I think Tim's take on it was that the original intent of this was that it would set the size of the ephemeral disk for the build pod, which sounds useful right, like maybe you need a bigger ephemeral disk in order to get your build done, but it doesn't appear that kpac supports that, like we did, I didn't find anything suggesting that that was something you could set, um so it might have just intended that setting and it just.

B

Was impossible.

A

C

Like you know something we wanted early on and.

E

How does it differ from your change, so your change is the cash.

C

So it's a it's a different disc right! There's a there's! The ephemeral disk that the build pod has but then there's a persistent disk.

E

Yeah, which is the cache.

C

I see one per app that we use to cash uh artifacts, so yeah.

E

It's one per Kpop image right, yeah,.

C

E

You got satoshed I guess to each uh build pod right.

C

We're like reusing the image um we actually get a real benefit from.

D

C

And if you're re-push now.

E

Yeah, let's just kill it that much yeah.

C

I mean I, guess, I think it's rid.

E

Of it, then that's it.

C

The all right I can make it sure uh it also exists in the CF, builds.

D

C

Which is again like we probably can't take it out of there, but we could mark it as like unused or something it hesitate to call it deprecated because it never worked. But.

C

D

A

C

C

So the other question was um all of these settings for staging resources live under an API key in that Helm chart, which seems like in the case of the existing ones, that's kind of an implementation detail, because we consume them from the API process, but this isn't really about the API. This is like a you.

D

Know could just as easily.

C

Have been injected by the controller um so I'm wondering if those and then.

A

The new one I added.

C

Is ingested by the controller and I put it into the API key just so it could be with its Buddies um I'm.

D

C

They should move into a more Global location that isn't under API, but I wanted to check to see if anybody had any any feelings about that I mean once again it would be a breaking change in terms of the values file, but uh either we care about that or we don't yeah.

A

There's a bit of history there, because we used to have five different Helm.

C

A

But now we've got one Helm jar, so it's a bit arbitrary, where those things go right, stuff in the API tends to end up in the API config, but yeah. It technically doesn't really matter at all.

D

C

I can make another another different story to review that we might want to just kind of rationalize it. I mean there's some stuff. That really is genuinely API uh related like the Ingress domain or stuff like that, then there's other stuff like this. That's just sort of like I think it's there for historical reasons.

E

We could review the whole structure of the values file if you want to like. If we don't care about breaking it, it's structure I mean probably something that we. If we want to do it, it's probably better to do it as soon as possible. Right yeah.

C

So uh yeah I, I, agree um and I I. Think that makes sense, like maybe we'll just take a pass through and try and rationalize that structure. I.

D

Don't know, haven't we already deprecated, maybe some Properties or something in the helm, chart or not I, don't remember there.

E

Might be, there might have been cases of us keeping old things, just ignoring them and maybe marking them as deprecated or something so it might also be an opportunity to get rid of them if.

D

We could do this for share this, but not shorts.

E

um Yeah crds, we do that okay, yeah I mean it's probably going to be a matter of personal preferences as well like the structure, but we can I guess uh start with a proposal: slash draft PR and internet until we reach agreement, and then we merge it in. Why not yeah.

C

I think that makes sense.

C

B

uh What I'm hearing.

C

At any rate is like nobody's, standing up and saying no, the way it's it is currently is like.

E

Yeah exactly it's just grown up the way it is and when we add stuff, I, guess yeah, we try to put it where it feels like it. It belongs, but I don't think we ever looked at it in a kind of holistic way as a whole, fire and reasoned about how it should look like so we're gonna. Do this surely, would be useful. Okay,.

A

The tricky thing is uh if to change the docs when you modify the uh Helm chart.

D

A

That's the eye scripting yeah.

E

We need to be very careful with that.

C

E

Need to change the docs like share scripting I guess will break.

C

E

We will notice, but we will not notice if we break the docks.

E

So we should be careful about that and then also there is a schema file right from which we generate the like this markdown file documenting each and every field.

A

E

We also want to make sure that we change the scheme.

C

That would be wrong right, there's like because most of the docs are just generated from the schema file, but then I guess in the install docs. We have an.

E

Example, yeah exactly and these toolbox in general, we tell people yeah, which values make sense for which Fields Etc.

A

And there's multiple installs such as AWS flavor of it and possibly kind yeah.

C

A

Might be more than one place sure: okay,.

E

Yeah, it's very easy to forget those, because even when we release- and we we do the like an example in store for before releasing we usually pick one of the three right. We either do it on kind or we don't do the three of them, because it would take.

D

E

Which means it's easy to miss stuff. uh It's true that I think this. The platform specific readme's try to delegate to the main readme as much as or the main install.md as much as they can, but they might have their own references to those values so yeah, just let's just do yeah, maybe a grab of each feel that we move to.

C

E

Find all places it's used even outside of the code just to be sure that would be a good idea, probably.

C

I mean to me compared with some of the refactors we've done in the past. It still seems pretty small but yeah.

E

Yeah, it's just it's. There.

C

Are a lot of places yeah.

E

It's a change you, you know it's a change that doesn't allow you to lean as much on the compiler and stuff and the tests. That's why it's tricky like when we change code we've got. We have the tests, we have the compiler, but when we change the docs or this these kind of things, it's so easy to slip in mistakes, we've I think the first few releases we made such a mistake. Every single release, you're like oh yeah.

E

We forgot that thing: okay, go back, fix it and it's uh yeah it's hard to go back and fix the released, turbo and stuff.

B

Nice um last topic is from Tim who couldn't make it I just want to check in what's left for 0.8.

C

B

C

Haven't done the middle topic yet? Oh.

B

We haven't, oh I'm, sorry I'm. Sorry, sorry.

C

um Yeah, which is it should be quick, I mean I, just noticed like when I run, make tests locally. It was failing and I was failing on some pretty basic failures that weren't flakes or anything uh in uh tools suite and but I, don't see, CI failing, which makes me think we're not running those tests in CI. For some reason,.

D

I'm not even sure what these tools are. They, like.

C

C

I'm not sure either.

E

But yeah if we have tests, why not run them in CI like it's about integrating the whole code base, so why skip and they're, probably not there, because of because we forgot or something and not not on purpose for sure yeah I know these are these are not Standalone tools either right? They are just Utilities in the code.

C

E

So it's definitely a mistake if you're not running the test, there's stuff like the image client, which we use to communicate to image registries, there is a Kate patching client, which is the code we use every time we do patches.

A

E

In the patching reconciler and stuff yeah yeah, which has its own tests uh yeah the patching reconciling is actually here right with its own test. So yeah there's, there's really no reason not to test this stuff in CI. It must have been.

E

So that's another tool. We need I, guess yeah.

C

It should be pretty simple, I think.

D

E

You you have to run each Suite separately right, yeah, so.

A

If you didn't make a test, it does include it and it runs all the it's probably from when the make files were split up into uh ones per component and CI, just picks up controllers and API and ete. Doesn't it and things like that, and this is just slipped in at the top level, so it's not classed as a component.

A

Okay, yeah, it's just being skipped. Isn't it.

A

And it sounds like people aren't running mate tests, so we've got a scripture. You run called test. Everything that also misses tests misses tools.

E

So it's not really everything. Okay,.

B

Yeah, it's most things.

E

Yeah, let's make a chart to rename the script and.

C

B

All right, yeah, sorry for skipping that.

C

um Oh no worries all right so carry.

B

On I know, you're good yeah, so this last one is just from Tim uh who can't I get uh it's what's left for. 0.8 I threw a link into them for the Milestone, but I think the other two issues I linked on here I think we also want 0.8.

E

Yeah I think that's what I heard from team as well, so we might as well just um add them to the milestone.

E

Also, the Milestone has this story about uh globally configuring. The kpac image cache size, which is the thing we discussed earlier, which isn't to do but I assume you already have a merge, PR, so I assume I can move it to reviewer approved and it can be accepted either by me or someone else and I'm going to add those other two stories to the Milestone to zero. Nine so that we don't I know they already. Are the zero eight zero? Eight?

E

Ah That's the thing they were. Oh, that's: zero! Nine, because I added all of them to zero to zero nine. But here we're talking about zero. Eight yeah, yeah, yeah, yeah I.

B

E

We want these two and zero eight I.

B

E

The rest can go in zero, nine if we want to yeah okay, okay,.

B

Is there anything else that we're missing other people on the call trying to remember if there's any other energy.

E

There's the other jobs if we want to be to do it to be complete, but the bug that I found today is also a bit Insidious I wonder if we want to fix it before yeah I mean if we have time I'm gonna, pull it top of backlog, because it feels like something that is a bit dodgy hey it's. It's probably been there for a while. So I don't think it is that urgent, but.

B

It probably makes sense.

E

Yeah, maybe I, don't know, I, don't think you should block the release necessarily like if you manage to fix it by the time we're done with the other two things or three fine. Otherwise, it can also wait for zero, nine, maybe or we can release zero eight one. If we think it's the thing is I've I've reproduced it I, don't know how they are likely. It is to occur. I, don't know how bad it is.

E

What I mean like I, don't know if it's a symptom of something much worse or if it's just something that we don't really care too much about. um Yeah I need a pair to take a look at it. Okay,.

B

I'm, sorry, if you mentioned earlier, I lost it like what's like, what's the high level on it, does it basically.

E

I was accepting um the bill packs the the story about specifying build packs when you push and I was pushing manifests with a list of build packs and I was taking Bill packs in and out of the list and.

B

E

Found a specific sequence where, like pushing a node.js app with both Java and node.js, build packs succeeds right, then I take out the node.js will pack, so only the Java one is in push fails. Telling me you know, there's no candidate, Bill pack for this app, which is correct, because it's an OJs app then I I put the node.js bill pack back and push, and that gets stuck. So it doesn't succeed. It doesn't.

D

E

Just gets stuck, which should and looking at canines I, don't see a K-pop build being generated, which means I, somehow managed to do something that doesn't result in a Kpop build being generated.

E

I, don't know why, like I, know that, since we've moved from one image per build to one image per amp, we had to introduce a bunch of like checks because sometimes builds can be skipped and corrective in that case kind of forcefully generates a build manually, kind of and I do that. I do see that sometimes, while doing these kind of experiments, but with this sequence that wasn't happening so maybe there's a bug in there. In that logic, maybe it's not even that sequencing like I, tried many sequences.

E

This seems to be to to be pretty consistent, uh but maybe it's um it's more like it's more random, actually like. Maybe it's just about an arrays or whatever. Maybe my machine- and this particular moment is generating it more often, I don't know, has to be investigated.

C

So if you restage, when you're in the stock state, does it unstick.

E

I, don't try I, didn't.

D

A

I expect it wouldn't yeah.

E

Exactly it's like because, like basically, what we do is we kind of we change the image and that triggers a build, but sometimes we'd the push results in uh no in no changes to the image. So we have to kind of manually generate a Kpop build or something.

A

um Or if we do something that k-pak decides that no, this would never succeed, so I'm not going to create a build. So we have to work around that.

E

Yeah something like that, so I don't know. Maybe it's because I'm pushing a certain like I'm, pushing a manifest a then I'm, changing it to B, pushing that fails and then I'm changing it back to a and maybe somehow it remembers.

E

I'm pushing a state that it's already been in, so it doesn't trigger a build or something like that it might. It might as well be a bug in k-park or I. Don't know it's weird, but I just didn't have time to investigate further, but we should pick it up and take a deeper look.

B

Yeah yeah that.

E

Makes sense, um I don't know if we want to block zero eight for a time because zero, it is, is so long overdue, honestly that I would block it, because if this is a bug it's been around for a while anyway, so it doesn't feel like it's urgent, so I would release zero eight as soon as we can. If we managed to slip this fix in good, otherwise we'll do either zero eight one or zero nine with the fix, because it's probably unlikely to to occur anyway.

E

Most people, don't even I, guess mess with the build packs list. So so much as I did.

B

Sounds good any other topics, anything else you want to talk about over here. Let's.

A

Just say Tim says it's a way of getting out of this apart from deletes in the app. So one way is just modify the source code and re-push, so it is simple to get out of it. You can work around it.

E

Yeah again, I haven't tried any any particular workaround.

E

But yeah that another useful thing might be for people to try my steps and see if it happens to them too, just to rule out that it's just a weird condition: I randomly got into uh and then yeah and then once you're there you can try. If you have any ideas about how to work around it.

D

B

Covers our topics, anything else.

B

All righty well! Thank you sorry! So, yes, all.

D

B