Cloud Native Buildpacks Working Group, 12 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Working Group: 2020-11-12

Description

* Stackpacks

A

A

I think you just need to make sure that when you join the link you're signed in to your zoom, because I thought I didn't, but I was wrong and ben was right and uh I just need to.

B

A

Signed into sso in zoom before opening the link.

B

Just sso auth to zoom to add a thing to a calendar, so I thought I was almost certainly added, but maybe it's a different something and my local client doesn't have a cookie.

C

Okay, uh are we ready good, recording, um okay, so a couple things I went through on this section, which is where we said we wanted some examples, and so I came up with several with the help of jesse and josh and some other folks um and I ran into one that I think illustrates what might be a problem. So I wanted to talk about that, but I'll go over these first, um oh and real quick. The other thing that I updated in here is the stack platform directory.

C

um As I was writing that up, it seemed pretty straightforward, but I'll come back to it, just in case okay. So for these examples the first was the annie mixon build pack essentially like the aft build pack, this one's pretty straightforward. uh It passes if one of a mix-in is required by the build pack- and I should probably say not provided by the stack or a mixing- is required by an app and not provided by the stack, and then it will be skipped if there's no mix-ins required.

C

Oh and I say that down here- a mix-in required by buildpak, but the stack provides it same thing for an app for uh the next example was uh again, this is kind of fictitious, but it's a postgres uh stackpack where it can provide limp pq and also like a custom db cert.

C

um So in this case you might have like a ruby app that requires the lib pq mix-in and the db cert, in which case it would pass. But then a java app that doesn't need libq would just require the cert and it would still pass, but it wouldn't install the lib pq um package mixin and then oh yeah. This is just an. I don't know why it's python, but it's an example where the stack already provides lib pq and it's required, but it doesn't get installed.

C

It would not pass if an app requires the pq mix-in, but not the cert, which seems okay, it would just skip and then not install anything and then, of course, neither of them now. Here's where it gets interesting uh josh. I don't know if he's on the call, but he had a good idea for a chrome. Chrome driver stack pack where it would install both uh google chrome, stable, mix-in and then chrome driver as like a non-mixing just a regular dependency, and so this would pass if the app requires uh chrome and the chrome driver.

C

So both it would pass if it only requires chrome driver and it would pass if it requires chrome, driver and and chrome, but chrome was provided by the stack it'd skip when uh it requires chrome, but not the chrome driver, and then it would skip when neither are required.

C

So what's interesting- and I think you may even already have this question in your head- is this case right here because there's a situation, and I call this down on calls out on line 204, where you can't have at least I don't see it yet. The you can't have the chrome driver dependency depend on the mixin, and so it makes it very possible or likely. I guess that you'd end up in this state where detector pass the build, might even pass, but then at run time, you'd be missing.

C

Chrome, because you didn't include it in your mixins and that feels out of whack to me because depending on chrome driver should imply that um so that's the one that I wanted to talk about.

B

um Can the build pack so the that the mix-in was required by another? Build pack does not preclude the chrome driver build pack from installing the mixer right that makes.

C

Sense right, but but then it does, but then how does different trade if the stack already should like is the like? Is the solution to just always install it, even if the stack already has it.

B

It's something required just to repeat the case, so there's a non-mixin chrome driver and a mix in google chrome, stable, that's required by chrome driver and you want to make sure that anytime chrome driver gets installed. Google chrome stable is installed because it's required by chrome driver, so you could have things that request. The google chrome, stable mix and without chrome driver by itself- or you can't do that- is that.

C

Right not with this build pack, yeah.

D

C

I maybe that's why this is just too fictitious. Maybe those are just two different build packs. You know.

A

Even if they I'm sorry, it was going to be like. Is this an example of something installing chrome driver that you would actually want to do with the stack rack right? Is there any reason.

C

A

Build pack can't install chrome driver.

C

Yes, I believe so because uh I'm not again this maybe jesse knows that or josh knows, but like we've had trouble with that in v2. I don't know if you guys have anything to add to that.

D

Yeah, I don't remember if chrome driver needs to be installed, I mean chrome driver has to know a bit about how google chrome is installed, but I think, if we're doing google chrome stable, then chrome driver could be installed by a normal, build pack, but I'm not sure, but even if it could, you could have two stack packs to your point earlier right, joe. If you had two distinct stack packs, then it could rely on google chrome stable, which would then resolve as we expect. I believe.

A

I guess this doesn't get into.

A

The questions we're trying to answer here as much which is about the usage, but is this also like the kind of thing that is abi compatible that you could change on a rebase right.

C

uh Yeah a good question again, maybe maybe this is like not a good example, and maybe the problem is not that you can't do this, but that this is not a good use case for for a single stack pack. uh So to your point like: is it um api compatible? Is it should this simply be decomposed into two stack packs.

B

I want to get right to the like problem. You pointed out, which is the case where nothing requires something requires chrome driver and nothing requires google chrome stable. Why can't the stack pack, in that case just install google chrome, stable anyways?

B

Is it that we don't have a method of pushing forward to get the metadata to say that the mixing was installed and it'd be missing that metadata on the image, because that that could yeah? I.

C

Think yeah, I think the problem is that it could right, so you can totally install it. Even if it's not requested. I think the problem I'm worried about, or maybe it's not a problem is that something else already installed it or the stack provided it and then you either end up with you know, just bloat from it being installed twice or potentially the wrong one on the path.

D

Not in the build plan right, it's not in the build plan, which means that you can now have branching logic on whether to install it. It's not only if it's in the build plan as a mixed in now. It's- and also I don't see it.

B

To me, all of this is the reason we haven't exposed. The mixins inside of the container, the mixins that are already installed inside of the container, is because it's we're, assuming it's possible for the stack pack to tell when a mixing is already installed or that if you run apt-get install and it's already installed, it will say yep it's installed sounds good um yeah, so you can just check the packages and see if it's there or just install it. It won't install twice right, it'll, just be a no-op change.

D

But wouldn't your default, if you were writing your own like app to build pack at first just be to spin through the build plan that you were given and then I guess at some point: you'd realize that maybe there was some introduction of someone not requiring it or you know I don't know I could see where it would evolve into. You writing a build pack one way and then now you suddenly have to like rewrite that build plan based logic.

B

Many many times a mixin is going to be. In most cases a mix-in is going to be required and a lot more than that. Mix-In is going to be installed right because you'll ask for uh image. Magic and image. Magic depends on 30 packages, so app get install image, magic will install. You know all of that right, and so we we definitely need a way for the build pack to report that those mixins were installed. But I thought that was in there through the reporting interface or you know,.

B

I forget what that interface looks like, but there's a way to do that for normal build plan entries. um Have we. Maybe this is sort of a separate question from? Is it appropriate for the build pack to do that, but is there a way to report the mixins that actually got installed forward.

A

I feel like we do need that right eventually,.

B

I'm not I'm not totally sure if they have to be reported as mix-ins, because once the build finishes the mix-in list itself, it actually doesn't have version numbers. It's not actually useful right like we could just say there is no mix and list for the application.

B

um As long as there's a way for the you know, whole dpkg thing to be dumped by the app build pack to say this is you know like as long as there's a place to capture that metadata so like normally in the new version of the build bill of materials thing? What does that look like it's you put stuff in launch tunnel and build tommle.

B

I forget what that rsvp.

A

Or, like your outgoing build materials entries, you put them in the launchpad or build tom all, depending on whether it was uh launcher, build dependency.

B

And does that work for stack packs? I assume it does for normal entries. So does that work for mix and entries.

A

I'm exactly sure what's in here, but I think we should do something a little bit special for mixing entries or, like maybe the output looks the same, but I would like for us to gather all those up into the mixing label at the end of export. If that makes.

A

A

Like if I use a stack pack to install a mix in on the run image and then export an image on top of that and then later I want to rebase and the image I'm rebasing with already has the mix in. I feel like that should just work transparently without having to rerun the stack pack.

A

So I'd like to take stack, packed, installed, mix-ins and then end up putting them somewhere put them with the rest of the mix-ins right, because that final, all app image has a label that says what makes sense are there and it should be complete, and we could use that to our advantage later. If we keep it complete.

B

We should we support rebasing with different base mixins, because then you're performing a different operation on the rebase than you were the time you built the image.

A

Like we already let the mixins change so long as none of them have been removed right, which is why I want sort of the metadata for mix-ins and non-mixing things that were installed by stack packs on the app image to be distinguishable from each other, because if the only thing that was installed is a mix in and we have a definition of a mix in, then we shouldn't have to reinstall it. If it's already there on the run image every base.

B

Should you be able to remove the mixing for the base image if you have the apple pack, because when you rebase it'll now install that vixen on the rebase instead.

B

A

Think that's the right right thing to do right. Otherwise, we're not trusting our mix-ins to really express the contract. We thought they were yeah.

B

Maybe it's gonna make.

B

It's gonna make it hard to write an app build pack, because you have to figure out how to get a list of all the packages that you installed, not all the packages that are there right, but just just some number of packages that were installed in the previous run and extract that and put it somewhere.

B

I guess we can.

C

B

All just get the.

C

List first and then run app, install and then get the list again and calculate the div.

B

I I mean like base: image, will have some packages, you'll get some requires for mixins and then you'll install, maybe twice as many mix-ins as what the requires are because of dependent. But.

C

Before you do anything, you get the list of packages installed, then you do your build pack thing and install 20 packages, and then you get the list again and you calculate the diff between the. Before and after.

B

Yeah, it's just it's a what was a simple build pack.

C

No, I don't think apple pack was ever simple, like my example is simple, but it's not really what we'd ship that I I feel like that's just par for the course you have to.

B

C

B

Stuff like that, what happens if you don't do that? What if you just failed to output the mixins you installed? In addition,.

C

um I don't think anything, bad happens, it's just emily doesn't get her data and then she's sad.

A

Well, if we build the rebase to check whether mixons need to be installed using that data, then something bad would happen on rebates, because you wouldn't reinstall them right, but I think that's probably the right thing to do like it just makes sense coherently,.

C

Why wouldn't you reinstall them.

A

Because my proposal is at the basically there's a implicit mix-in requirement on rebase time, and that requirement is the diff between the app image and the run image.

C

So, even if you have let's say you, you depend on mix and foo and it depends on mix and bar and then in the a future version of foo I mean maybe yeah, maybe that's part of the compatibility it no longer has that dependency.

C

You would still install it.

A

Yeah, because we don't know.

B

Like you'd have to do.

A

A rebuild to get something with fewer dependencies.

B

Apt right app differentiates between explicitly installed things and not explicitly installed things right. So if, on the first build, it gets foo and it doesn't get borrowed because bar just happens to be a dependency of food that nobody requested specifically and then on a subsequent build. It gets foo and bar because food and bar were both installed. That that strikes me as risky app is going to do slightly different things.

B

It may install if there's a suggested or there's some packages, some dependent packages that are only installed if you explicitly install something that are not installed. If you don't explicitly install something so I'd want to I'd, want to see.

A

We don't need bar to be in that list. Then right, it's like did you need foo or didn't you.

B

If you don't need buyer to be in that list, does that mean you shouldn't have to report that you installed bar.

A

Yes, because if you're missing foo and you have to go, install it you'll get bar and if you have foo, you have bar.

B

So then we don't need to do the subtracting of the lists right or like we don't we don't need to report every mix that got installed. We just need to report the explicit ones yeah. I.

A

Think that makes.

B

This easier because that's you know those ahead of time: the buildback doesn't actually import those yeah.

D

Yeah, that's already in the build plan right we're already reporting the explicitly required ones, except.

B

For in the case, where you're dynamically.

D

Installing one on your own, I guess.

B

So we don't end up with a full list of all the packages installed on the mix and list of the generated app image.

D

But it does mean any stackpack could install any number of app packages that are technically mixed ends, but it didn't if it didn't request them as mix-ins from like an app one. Then it can just do whatever it wants to the image right, which is kind of that's kind of what we're coming out with the chrome one right. If chrome driver stack pack was responsible for installing chrome, stable itself, then there's no way for it to report that that happened for downstream build packs.

B

Maybe that's, okay, I think that's.

D

D

Because if you do need the reportability, then it would be up to the stack author to break those into two different build packs. And then you would have reportability by requiring it explicitly.

B

Also that it should still be possible to generate that dpkg list of everything that got installed. If you need it right without going to the mixing interface, because you need things like versions also separately.

A

I think most people who want a dpkg list are gonna want a dpkg list. They're not gonna, want to like translate our mixing label into a dpkg list and take it on face value.

C

Okay, so step back for a second like this seems like an important discussion, but one of the things I'm taking away is that we have given this example of chrome and chrome driver. I think we have enough knobs and levers that this shouldn't be a problem, because this is probably a poor build pack design, um I'm thinking at a minimum. Maybe these are separate, build packs or stack packs. Maybe it's one build pack, one stack pack.

C

um If we're. If we agree on that, then I I'll re-tool the example and uh not worry about it, not worry about this case or really this case down here then we can go on with the rest of the discussion.

B

It sounds good to me. I don't even think this is necessarily poor design or, like. I think it's okay for us to assume that packages may get installed in the background that aren't during the build process, not before the build process, but yeah right.

C

B

Yeah, that's what I mean by.

C

Yeah, that's what I mean by knobs and levers like. If that's the way the stack pack wants to do its thing. That's fine yeah,.

B

Okay, because we already said that we're gonna sorry so one last one last justification for this. We've already said that, through the build plan, um stack packs can make non-mix and changes to the base image as long as they're abi compatible right. That's exactly what installing google chrome stable is when it doesn't report it as a mix-it. It's just.

B

It falls into that category of other abi compatible, but non-mixin contract changes, even though yes, it could be, could also be reported as a mix there's just no way for a build pack to express its dependency on it in that sense, but it doesn't matter after you know, once the build process has started.

A

Oh, that makes sense. I also didn't mean to apply as a bad design. I was just legitimately asking questions. Yeah.

B

That's possible.

A

With stackdriver.

C

Okay, um before we get back into the discussion, we're having is this related to the open unresolved question about mix-ins, changing on the stack.

D

I was about to ask that actually, like I didn't, realize you're gonna go down here, but I was just thinking like our discussion earlier like if someone installed jq now and you used to install it like, should we be during rebase making changes to the build pack plan that we store for rebase, or should we just pass it to the stack pack and then let it know up itself like essentially in that case, when it detects the jq's already installed,.

B

I would keep the logic that says that you can't lose mix ends on a base image um on rebase for now and then maybe decide to relax it later. um I agree with emily that it it does seem like if we're going to trust the contract right. We should we should let you rebase with less as long as you have that star, right or or as long as a stackpack could provide it.

B

But are we going to be able to guarantee that this backpack actually does provide it during the build process or fail right to me.

D

That's, I think what you said earlier steven that it could do something else. That's also abi compatible means that we have to run all stack, packs that ran previously just because it could have installed. Jq didn't mean it also did something else, so I think we kind of I don't even think we could ever optimize that away. Yeah.

B

Actually, I think I I think that makes sense. I think I grew that.

A

I don't know, but.

A

Yeah, I feel like if the only thing it provided was a mix in which, and the mixing is present when we release we shouldn't have to rerun it.

D

That'd be up to the platform. Who knows the stack packs to be able to like get rid of the build plan and then not have to pass that in because, in this case, you'd have to know that your apt build pack is safe to be removed from the build plan that had only app packages in it. That are now part of the stack.

B

What if we allow you to change the mix ends as long as the build could as long as a stack pack claims that it, because the mix-in provides are static as long as the stack that claims that it can provide it, we allow, regardless, if it's more or less, we allow the build to proceed, but we always provide the exact same set of mixins to the stack pack, as happened on the original build, even if they're present in the base image.

B

That way, if there are additional things that would get installed, when the stack pack installs the package, it would still run, um and it would just be a no op on installing some of those other packages.

B

I would I would almost like to go the other direction and say actually, when you can only add mixins in an incoming new rebaseable base image, if there's no stackpack that could install for safety, I think that would be the safest thing to do and then say you could never remove.

B

But I don't think we have to figure this out here, like I think this could be.

B

These decisions could go into this back pr.

C

uh I'm gonna: we can go back if we want, but I'm gonna move up to this new section and, let's make sure it's what we discussed. So the detect and build phases will be provided with a new input, the stack specific platform directory uh instead of a user space platform directory. So it's not actually a new input. It's just a replacing another input.

C

So in this way the lifecycle can ensure that no environment, variables or other platform-specific extensions are coming from other build packs or other sources that may impact the stackpack's ability to successfully rebase in the future.

C

The new uh stack, specific platform directory will require new options for it'll, require that new flags essentially are added to the lifecycle, detect and build interfaces, uh but the detect and build executables will continue to keep the same interface and just simply get different platform doors so down in the specs section changes I've added a stack platform to the detect, the build the rebase and the extender, and the extender does not have a normal platform. It just has stack platform.

B

So for environment variables, having special having a separate set of environment variables for stack packs, if you need them right, you explicitly say hey: this is a stackpack one. It's gonna have to you know it could vary on rebase and therefore we have to call it separate. That makes sense to me: do we feel the same about services and their services are still in platform right?

B

This isn't a question.

C

B

C

Bindings exactly yeah yeah.

B

This isn't a question about whether we spec it as having a separate platform or it's a question about from like the pax cli ux right. What would we refer from from the recommendation for a platform? Would we mount the same bindings in um at into both pla separate platform directories?

B

Do we consider that an unsafe operation for ebays.

A

I think something like pack is already gonna have to distinguish between stack and regular environment variables. um Also, right now, pac doesn't know where binding should be mounted. I'm pretty sure you have to do it yourself with the volume so you'd still have to do it yourself.

B

That makes sense, I think I guess, regardless of the ux the recommendation, I was gonna. I think what I was getting to is, I think it's okay, if bindings vary or if, like we use the same sorry, we use the same set of bindings between stack packs and regular, build packs because they represent connections to real services.

B

There's just like there's no way we can replay.

B

You know a previous set of bindings right.

C

Are we then saying in this case that it's just up to the platform to provide them in both directories.

B

Yeah like if, if a service is a service that provides you with app packages right or like that, assists you in installing you know or provides, or is you know, a license key for installing some software? No that's the way we've used services before we're binding this before then.

B

I don't think we need to change any of this. I just want to call.

A

Is it okay to make it the same for both and not the other way like? How do you know what things you need at rebase? I don't know why it's any different from the environment variables.

B

And then, okay in the environment variables case, there are environment variables that are intended for regular, build packs right that would get exposed to the stackpack environment that if they changed between build and rebase, could cause problems for the stackpack. So we separated the environment variables out into a different list where you know hey. If I, if I change these, I really could break rebasing right and so that makes the accordions those environment variables office safe for stack packs for bindings, because bindings don't represent a really configuration that could change in the application they represent.

A

I don't know if I agree with that.

B

A

Think it's that different, it's something that could change! You could definitely mount in different values in different places. Right.

B

But they represent an external service that, even if they don't change, could still change.

A

E

Feel, like sorry, go ahead.

A

Well, that's true, like which service you're connecting to matters, and I feel like we and that could change and that's a different, more fundamental type of change right and I think you want them separate, so you know which ones you actually need to bind on rebase and like know which build packs are consuming, which things also, I don't think we in the way we've done bindings at this point.

A

I've said these always represent an external service. That's generally how they're used, but I'm not sure if that's like a hard and fast rule, maybe could it could.

D

It just be, they may be the same like isn't this really a platform complete decision like I'm, not sure.

B

E

D

Do think there is some risk there, but I don't think the guidance should be very strong here, but.

E

I I agree, I think pac, you know to some extent, could allow right for both you to specify what build time bindings you want versus what um I guess stack pack things you want as far as binding goes, and then I would maybe even go and lean on the other side where the bindings could be applied to the build right, but they wouldn't be applied or available to stack packs and therefore it wouldn't impact the rebase operation.

E

But I think that would be up to the platform and what those use cases would be.

B

Seems like we need some language somewhere that describes. They need to be able to be different for sure, and we need some language somewhere. That describes the implication of changing the ones on the uh rebase side, but we probably don't need in the rfc right like it doesn't seem like there's a anything that would lead to a change in this from.

B

That that seem.

A

A

Are we dead sure that for the first pass we don't just want to say no platform.

B

Well, you still might need environment variables right.

E

Yeah, so you need them for at least that part of it. I think we talked about that one. It's whether bind mounting should be allowed and, like I said at least for pack, I would say that initially we wouldn't want to provide that just because of the the hairiness or the implications behind it until we have maybe a better mechanism to to at least help or guide that use.

E

Case in more controlled environments- maybe that's different, but they might want it right, like uh google, if everything happens on the google cloud platform, then they could obviously do it and it shouldn't be a violation if they know that they're attaching the exact same thing for.

B

B

Does that mean we can we think? As long as we say, the platformer has to be separate. We can defer on what exactly that platform. Dirt looks like in the stackpack case until later, whether it has bindings things like that yeah. I think platforms platforms could even choose not to allow stack environment variables yeah if they really wanted to right to be empty.

B

D

They really want to share with bill packs. They can do them too.

B

I think I'm satisfied on the platform.

B

C

Okay, um I think I'm at a loss for what to do next, which might be a good thing.

B

So there was yeah, there was a discussion we had uh going back to the build plan, mixing combinations. We remember, we deferred one of the first. The first thing we deferred was.

B

There were some combinations of mixins not being present and build plan entries not being present and the stat the extend versus build phase that we we were undecided on. Originally, I don't know if we ever got back to that that discussion.

B

um I think that was the last thing that they I can remember that there might be still more significant, open questions about now that we've resolved the platform thing it's like. Should you run the I forget it was like? Should you run the build and that and not the extent sometimes if they have different sets yeah it's like.

D

Stage specific identifiers right, if you only need you know the stack pack for the run phase, can you you know? Should you skip.

B

D

In the build phase, always right um and I think yeah- I'm less concerned about the build phase. But if you only if you skipped the because you.

B

Definitely want to skip run.

D

B

D

Don't need anything for run you just don't you definitely don't want to pay that cost, and that was part of the challenge, though, but it's a lot easier to understand. If, like you submit something that needs to be done and then you just blindly run been build for both faces, but I think that's just too simplistic.

C

Is it I I was hoping that's where we would land that like if it like, if it passes it's gonna run for both it just might no op in one of the cases, my hope being that in the future we could make a non-breaking change where it becomes smart enough to just hey. If nothing, if there's nothing, that's needed for extend, we don't don't even run the extend phase, but maybe it's optimistic.

B

I think once the once, the build pack runs as root and can make arbitrary changes. It's going to be very expensive to tell if it didn't do anything.

C

Not if it didn't do anything it's to tell if it needs to run right.

D

Yeah right but yeah, our mix-ins, I think what makes it different are just mix-ins in general. Just because if the run image doesn't have jq but the build image does then the build image could theoretically skip my build pack. But then I might have more work to do that wasn't related to installing that mix-in. If my only job was to install that mix-in, then it makes sense to. But I don't know I don't know if there's a way that we need to specify that this. My only job is to install mix-ins.

C

Yeah another example in the inverse example might be lib pq dev, where the run image has lib pq, and so you just need to install it and build.

C

Well, no, I guess maybe that's not a good example, because you have one.

C

B

So if a normal build pack makes a require, that's a regular require from a stack pack right, um there's no like. Currently we have and like for potato, we have run equals and build equals, and uh um you know require entry, that's read by the build pack, and then the build pack can decide if the layer generates is going to be run time or build time um for those normal entries.

B

Here, the entry is going to get sent to both sides of the stack pack right and the stack pack is always going to have to process that entry. So as soon as as soon as you introduce regular build plan entries into stack packs, I think from what we've currently defined, we force both the build and the run to always run right.

B

Are we comfortable.

D

There's no stage specific prefixes is that right from normal yeah.

B

Are we comfortable with that.

C

I I am because I think it can be the responsibility of the stackpack author to account for that. uh That doesn't mean that, like my hope of a like, changing it backwards, compatible might not be true because some people might like. I think this is what you're driving at make changes.

C

Even if there's no uh build plans stuff but yeah I mean, I think I think it's simpler for the stackpack author to think about, even though it is something they have to think about, rather than trying to like figure out when it's going to run and when it's not going to run.

B

So if a stackpack provides a thing, it's not a mix-in, then you immediately know that it's going to run in both stages.

B

Oh oh, but that is sorry. Those provides are dynamic right, yes, during the detect process. So if a stack pack.

B

Provides a runtime mix-in only and another thing, then the stackpack will still know that it's going to run in both phases and there's no situation where on a rebase it runs in one phase and not the other phase right.

B

I think this all works.

B

But I thought there was some combination. We had last time that we ran into a problem.

D

Well, yeah, I think I think it's still if you, if the application requests jq on the run image and only on the run image, are we saying we're going to run, build now always like on the build phase.

B

I think we're saying no right how do.

D

You know to skip that. Is there any way for a build pack to say I require this mix-in to do my job, but my job is not installing mix-ins.

D

My job is to do some other abi compatible thing to this stack image.

D

Like in addition to the mix-in right, or is that just too far-fetched, I don't know.

B

Sorry, I didn't understand, I think I didn't understand, which kind of build pack.

D

Yes, I guess it's kind of like the chrome driver, one yeah right where someone requests chrome driver, it doesn't exist on the run stack, but it exists on the build stack.

D

But in addition to doing the google chrome stable, I installed the chrome driver like. I still need to do that at build time. Potentially.

D

Even without a requires like.

D

It's hard to think about, but I think there's something there.

C

Maybe, but I also feel like, like kind of like with the chrome driver example, I feel like there might be enough knobs and levers and that maybe we're talking about a case where that's just not the right architecture of your stack pack built back and whatever like it like. Is this another case where splitting those into two different build packs or like a stack pack and build pack solves the problem?

C

I agree with you. It's like, I feel, like there's a thing, a thread to pull on there, but I also feel, like the mechanisms are in place, to solve 90 of those kind of problems.

C

I think that's really what we're trying to figure out right like we're not trying to catalog every use case of stack packs here. I think we're trying to determine if the abstraction we've created is sufficient and capable of solving generically solving problems, and I think we're confident it is for the 80, the app the app packages and the ca certs and a few other things. I didn't even put the cred helper, one that you described jessie because I'm like it just seems like so straightforward, yeah,.

B

I'm actually I'm not worried about use cases, we're missing, I'm worried about edge cases, we're not thinking of with the having the two phases running simultaneously with different sets of mixins and then rerunning them later. That's that's the thing I'm like you know.

B

A

B

Thinking about it, if we miss things, we can add them later right that doesn't that doesn't bother me, but it's like, like I'm a little tempted to say well, should it be stack for now? Should it be stack, packs can either provide mix-ins or regular entries right and force people to.

E

Split them until.

B

E

B

We understand all the edge cases for a stack pack that could do both, but but if that's the case like is it, is it terrible like if there's some horrible edge case in what we do now? You know?

B

Is it that bad? If we have to come up with something later to fix it? Maybe it is.

D

Yeah, I think that's the question: can you skip if no mixins are required? Because can you make that decision confidently for all stack packs like I said you know, and if we can do, we just want to go ahead and like solidify that in the specs and we're right we're kind of like a clearing. If you want to be like mixing only somewhere in here.

B

Can you skip it? No mixes are required and there are no regular build entries right, yeah, exactly yeah.

B

I think so I think otherwise the app build pack is pretty bad. You wouldn't want to put it somewhere right, because it would hurt performance. Is that is that what the text says right now joe?

B

I didn't follow that uh if you, if a build pack requires, run the bq and just that and you have the app build pack, does the build stage run.

C

um I don't know if we capture this, I know we've talked about it.

D

Yeah, I'm not sure we said we said they can run concurrently and I think they say they get the. I don't know if it actually.

B

B

See details below.

C

B

During the extend phase, stack belt packs that pass detection.

B

I guess it's a question of: did they pass detection, though.

B

I think it might have actually been back up sorry.

B

B

The platform and compare the list dimensions that are statically required by all build packs uh with the static list of mixins provided by the stack, build packs and failed to build.

B

Well, I see that's just that's just pre-validation, oh and then during the I'm sorry.

C

I I kind of think I'd know where, wherever I might want to go, but go go for it go for it. I'm sure you're right.

D

Passes detection, it's very vague. It doesn't say anything about phase specific that I see.

C

This is what I thought had it, but I don't think.

C

C

I think if we do talk about.

C

D

Yeah: okay: there it.

C

Is yeah? This implies here that it's going to run both and it just won't appear right.

B

C

I don't know if I'm saying that's, I'm saying that's what the text says, I'm not saying that's what we have to decide on.

B

Yeah yeah, I just mean I don't know if saying that doesn't show up in the list, says what happens when nothing shows up in the list. If that makes sense yeah like do we have a language, any language that says pass right. If a stack build pack provides a mix, it's not required. This tackle pack may pass.

B

Detection for each phase, if a stack pack.

B

For each phase.

B

Up there at the top on 133, right and.

D

None of the mixes are required. It must be good. Yeah 135 is the one that.

B

D

That's right so.

B

I think you would skip this technically. It says for each phase there um you know, there's a different pass and implies that there's a different pass and a fail scenario for build versus, extend and therefore would be cut in the build one.

D

Yeah this this, maybe isn't. Quite I mean it just just talks about make sense. It doesn't talk about other requirements, but I guess those are just implied.

B

It doesn't say it must be skipped right, yeah and.

A

B

A

I feel like we only skip things right now that are listed as optional. True, do we have a stack pack that just decides whether to pass or fail without putting things in the build plan? I feel like in some ways. That's.

D

It's optional, the the gear we need to kind of make this make more sense.

A

If the order looks like the other order, it would be assumed that there's optional for each build pack in a group.

B

Aren't stack packs, always optional equal true. If a stack pack hard fails exit one during detect, do we actually fail the whole build.

C

B

So then it's just it's optional. True is the only state right and then yeah they're extra weird, because the mixins can actually be yeah. You know partially satisfied. Sorry.

C

Yeah, no, that's all correct.

C

um If a so, I think I think the case we're worried about is that a stackpack does something put something in the image that it needs that isn't captured with either dependencies or or mix-ins.

C

And it's, and so I think, we're worried, we're worried about two cases. One is where it it's not supposed to run, but it runs in one of the stages in either extend or build, and it does something with a side effect that it wasn't supposed to do or it kind of needs to run and it, um but it doesn't run.

C

uh Right, that's sorry, I lost myself there anyways, I was gonna say: is it possible that the solution is the the the where the where a build pack provides and requires the same thing to sort of ensure that it always runs, but in this case, can they but they can't require mixins, but can they require a regular? No.

D

Because the order is enforced, right, yeah, that's right! I think I thought or wait did we. We may have changed that actually.

B

They can require things from themselves from other stack.

D

Packs and for themselves, I guess yeah.

B

So that something can something a stack pack can definitely enforce that. It always runs what about a stack pick that doesn't have any mix-ins or.

B

Require or provide anything that always runs.

C

uh Wait, no that wouldn't always run it doesn't if it doesn't provide anything at all, it still passes may pass yeah it may it may pass, and so it could pass based on um something it sees in the app something it sees in the image the stack yeah.

D

Yeah, but if it does pass, then it runs in both that's the important thing right like passing out detection on the bill means you will: you will run during rebates.

B

The things that will cause a stack pack not to run are a regular build plan. Entry is not matched that it provides, is not matched a regular build plan. Entry that it requires is not matched or it provides some mix-ins and no mix-ins are.

B

Yeah- or it says it can provide some mix-ins and no mix-ins are required.

B

Right, this is getting a lot.

A

It has to detect fail in that case,.

A

Because we're not putting the provides in the build plan or it detects past, and then how do you know the difference between that and what? What? If I know, no one.

D

A

This idea, what if we just did the mix-ins on the first pass, it's much simpler.

B

If we, what if we approve as is, but we put a line at the bottom that says we're going to implement the mix in thing first, without the regular build plan entry and then ship it yeah. I will approve this as the.

A

Vision but I'd love to get a spec pr. That only does the like. We could do the detect uh I'm comfortable with.

B

That yeah, yes, I'm.

A

B

With that, I would love that sorry, yeah.

C

uh Because yeah, so I think that's fine. That makes sense like this is not a spec. This is a vision, I think that's a really good way to frame it.

C

I think we can do that.

D

So real quick joe, um can you search for requiring mix-ins in here because we do have a section? That's.

C

D

uh Requiring that's what it says.

D

Yeah, it says a stack. Build pack may not require any entries in the build plan.

C

You didn't make some other non-mixins who wrote this uh anyways. I think that's fine. That was like.

C

I was just considering it as a mechanism I'm definitely leaning towards um if, if it passes detection it runs in both phases, but, like I I I kind of don't like this idea of it passes detection and then it maybe runs in one of those and in it's like almost it's, not non-deterministic, but it's like hard to figure out which phase or phases it's going to run in, and I think, as a stack pack author, that's harder to think about.

C

Then it's going to run and I need to account for that, potentially, with the forthcoming stack type environment. Variable.

B

I I don't like the idea of if you make an apt build pack and something says I need this package in the runtime image that a layer gets generated or needs this package in a build time, image that a runtime layer gets generated because then it goes sorry, maybe I misunderstood, I mean yeah. It would not so.

C

C

There's two cases: one is that the stack already provides it. Let's say: let's say this: let's say: um let's say the run stack has lib pq and the build stack does not, and there is a build pack that requires lib pq, that app package, the apt stack packet stack pack is going to pass detection and then, in the build phase, it's going to run because there's no mixing and in the run phase it's going to run, but it has to be smart enough not to reinstall it. I guess.

B

But but if you run the run phase, if you run that you mean the extend phase right, if you run.

C

This yeah the extend phase, but no it's not going to get it. Is it's not going to get it in the build plan? So it's not going to install it so.

B

It won't, but but the build pack will still execute in the extend phase, which means that we have. We don't know what else could have happened. We have to snapshot right layer right and then on rebase. We have to replay the build pack.

C

But that layer should be empty if the build pack, if the stack pack is written correctly, um that tarball should be empty and we can just chuck it right.

A

C

We currently don't.

A

Throw out empty layers.

A

What I would like, which is similar, but maybe slightly different, because I've wanted this in other cases, for regular, build packs as a way for a build pack to return and exit code.

A

That means I'm I'm not doing anything.

D

I think the could we improve, like we just said where it runs in both places to simplify things and then come back around with an option to be able to like have build, packs, have more granular control on whether they run in both phases and like potentially exiting with the status code or predefining that it needs to.

B

Skip if we just start with nixons in the implementation right, then it's safe to skip the phases if they don't have any mix ends, and so I I recommend we leave it as an unresolved question for once, we've implemented the build plan entries, but initially we implement it so that it's it only. The only things that run are the phases where the mix-ins are needed.

B

If that makes sense, that would that would make me comfortable and then then we can decide later if we need to run in both phases and chalk empty layers, which scares the crap out of me for reasons, but over.

B

B

um With that, I'm I'm ready to approve.

B

Does anybody feel uncomfortable about saying this is good enough to proceed to spec pr and implementation.

B

Maybe maybe we're there almost.

C

Believe it when I see it, yeah.

A

I feel good about it. Let's shift it yeah. I feel much better about the idea that we'll do the mixing stuff first, which makes me fine approving other things, I'm not sure about, because I think, as we do that and get this framework for running stack, packs it'll be easier to think about all these other questions that we're laying on top.

B

Really good job thanks, everybody.

C

See you bye, everybody take a look at this one last time and vote. If you can.