Paketo Buildpacks Working Group, 23 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Working Group: 2020-06-23

Description

* Paketo Community Charter: https://github.com/paketo-buildpacks/rfcs/pull/11
* Branch Name Changes
* ACR Mirror: https://github.com/paketo-buildpacks/stacks/issues/21
* zlib1g: https://github.com/paketo-buildpacks/stacks/pull/23
* Buildpack Ordering: https://github.com/paketo-buildpacks/builder/pull/23
* Opening Working Group Meeting
* NodeJS Re-architecture: https://github.com/paketo-buildpacks/nodejs/blob/architecture-rfc/rfcs/0001-buildpacks-architecture.md

A

A

You doing I beg.

B

A

C

C

D

D

Comes up trying to talk.

E

E

E

E

E

I can ping Steve and see if he's coming, because it is that who were primarily waiting, yeah, okay,.

E

D

Hey sorry, I'm late I still waiting for anybody. I should kick things off, but I can't.

A

Make it today something cannot get the show on the road.

D

The list first thing on list is review outstanding, RFC's I'm, going to share my screen and we can look through them. If that sounds good everybody, it looks like there's just one and it's a draft around the Community Charter I'm just going to share links, since we only have one.

D

Put it in the working group doc: oh, it's actually a working group doc there. There is I think for us to open this. One is for us here today. Thank you.

D

Any updates to this, or anybody want to talk about a rat I, see I've got some approvals.

D

Oh looks like been approved, but it's still in draft mode. So will they prefer for us to take it out of draft mode before we send it through.

D

Anything else on the community RFC any questions put that.

D

Cool next thing: the list is branch name changes.

A

I, don't know if you guys have talked about this yet, but on the CMV project, we've renamed all of our default branches from master to main in an effort to have used more inclusive, race-neutral language. It seems like it's something we should do on the Paquito project as well. I.

D

Think Ben is already started doing this for his repo zanpakuto. Is that right, yeah.

A

That's right so I guess the ask is, let's just light the bulb and get it over with for everything. So we don't have to think about it again. I.

D

Think there would be a really large effort for everything for the Cloud Foundry build packs to convert across because of just didn't kind of sprawl we have, but maybe a good place to start would be the repos that are in the picado argue that are very public. Maybe dan, that's something you could talk a forest about kind of prioritizing.

C

D

Cool anything else on that change.

A

There's a script under guess: I've seen this that Dave's I wrote that, like repoint, PRS and stuff, which is pretty convenient to help with this migration I don't have the link and you but I can so I. Could you guys, after this.

D

Spring project decided to make this change unilaterally across all their repos too. So there's a lot of lot of tooling, for it.

D

Right any questions on that.

D

Let's move on to the next thing: ACR mir-21.

A

Getting good performance on Azure, which is important to a lot of people using the poquito build packs, would be markedly improved by having a run image mirror on Asher I think been opened. An issue was checking the status of that I heard.

D

That higher uses separate domains for registries, and so it would not not have any impact. There's a thread on the a C and B slack about that. I think been started this morning. mmm Yeah.

A

I think we can, um on the C and B side, improve our mirror selection logic. To make this work better, but I think getting a mirror in place would be a good starting point. I.

D

Don't think cross we put blob mounting works across those registries, so I don't know if there's actually a difference between having a mirror on Azure versus like an ad one domain on a shark versus and a you know, having them in GC are are.

A

They, like I'm, sorry I'm behind on the starter they region specific domains. Are they more granular than that they're.

D

Like ECR user, specific domains, mm-hmm from what I haven't tried it, it's just what people said in the thread, so we were talking about on the capac side, creating a smart stack that would relocate towards new user specific. You know the run image toward user specific domain to do a rebase to get around it, but I, don't know! If there's a helpful thing, you can do. Okay, don't.

D

And it's commenting on the thread right now.

E

It's in generally related to that and I think Emily brought up. We ran into a huge number of issues this morning, because the Builder is now include a docker hub, run, image mirror and the default. If you name your app when you do like a pack build ends up being like a indexed docker, IO library, /. Whatever you happen to call your name thing.

E

The output of that from PAC build is incredibly unclear that that is actually what's happening and like that you're now, switching to a different mirror that you maybe didn't expect so like going along with that as your stuff. If you're going to be doing something like that, that portion of the codebase, maybe in kpac and maybe also impact, probably means a lot more work in order to be very clear about what's actually happening, we found the behavior, or this morning, I found the behavior to be like incredibly surprising and really difficult to debug.

E

What was even happening was.

D

The behavior that was surprising that had changed mirrors when you targeted you're in for a different location, yeah.

E

So I mean we've been using the GCR builders and, in fact, I even handed a builder that was a GCR builder. It was like this builder is on GCR, but then for the run image it was like. Oh I, see that you've called your app like my app and internally I refer to my app is like this docker hub registry kind of fake name. So that means you probably want to grab the run image from docker hub, but that's not really what I wanted it to do.

E

That behavior is kind of strange, so I don't know if you're gonna do something. That's like even more complicated than that like auto, relocate images for folks thing, I would say this feedback is like. It should be very clear to the user that that is what's happening. It should state it like I'm doing this. I'm doing this I'm doing this because, like trying to I I, basically just went to the codebase and read the code base to figure out what was happening and that's like not the best user experience.

E

C

A

Logs, like we could talk about moving that from debug up a log level, but I thought at least at one point in time. It did tell you what mirror was selecting in from what configuration when you're running packing for boss 1. It does tell you what you mirror. It's selecting doesn't.

E

Tell you why it's like didn't.

D

Underline things people don't realize that my app really means the official my app on docker hub with that assumed globally assumed prefix. Whatever.

A

Go ahead, I think say wonder if the weird part of this is actually the that it still does this behavior when it's in the demon configuration mode, alright, so I think if you were running with publish, you really would be going to docker hub, and it would make sense that it's using that run it would even be pulling it. It would just be generating an image on top of the layers that happen to be co-located conveniently for you and we made the decision a while ago in the demon mode.

A

If you need to pull the run image to do the same logic so that the difference is between daemon and publish are minimal, and it's not confusing to people that running, publish in some edge cases would get you a different image than running daemon, but I wonder if, because you don't the mirrors, don't really matter in the daemon case, if we should just be pulling down the default one. In that case,.

E

Probably, or at the very least, allow me to do something like set default, run image and set it globally. Instead of the current. Only option is on every invocation of PAC bill to tell you which front image you want in word override it. So it's really, it would have been very painful for us to have to do because this is baked into basically every one of our integration tests across the entire potato organization. So it'd been like very expensive for us to fix that specific problem. You.

D

Can cite global run image mirrors in the PAC CLI I? Don't think you can set it global, like I'd.

E

Love to sit with.

D

E

And figure out how that works, I tried for probably two and a half hours this morning could not get it to do what I wanted it to do.

D

Happy they're happy to sync up for sure.

D

I, don't think, there's actually there's logic we would change like it definitely could be more transparent, I. Don't think we need to change logic to make ACR work because I don't think it's possible for ACR to work because there's no, you can't cross new blob down across the Registry's anyways, so I don't think it will get worse, at least, but that definitely a good idea to make it more obvious, what's happening in those cases and I'm happy to sync up on getting real engineers to work. Something that's be helpful for you. I.

A

Wonder if there could be some small gains from doing the mirrors, even if the cross ripple blob mounting, doesn't work honestly, because if you have to stream it through the exporter and back up if you're say like running kpac in Azure, like is probably closer and the pull and push times are faster. Even if you can't cross your goal blob mount. So if people are choosing to export to Azure, they might be doing that in situation when they have a better connection to Azure anyways you're.

D

Gonna you're still gonna spend the time downloading the whole thing from Android, then re uploading it to Azure and so I. Don't know if there's any guarantee that I mean like put it on a list, but it doesn't seem like there's an Inc guarantee that it would necessarily be faster for any particular subset of users. I do.

A

Think in general, once you're running in one of the cloud providers, clouds using their services are faster right, like I find PC are slower for me to use normally for my workstation that, but it seems you're running a task in concourse. Near concourse is in juicy are all of a sudden. It's by far the fastest option.

D

Make sense I, don't think we should not do it I just don't think it's as beneficial is it. You know the normal run. Image mirror work would be.

D

So maybe action airs don't close the issue, then Ryland can take a look at it along with their list of priorities.

E

Yep sounds good. We talked about this today, so we're ready to start taking a look at it probably this week and we'll keep that issue updated with the progress, and we can also link like a tracker story to not sure we have one yet, but we can get on them.

E

That makes sense, sounds good.

D

The like, if it's a lot of work, then yeah I, don't think the impact is as much as you know. Adding out there run image mirrors so like no cash should think about prioritization.

D

Alright anything else, nice yeah I'm here.

D

Add Z Lib 1g to tiny it's.

A

Necessary to do the job native gravi, M stuff on tiny other libraries, it needs to be statically compiled in, but not this one and there's my green to me that it's a slippery slope and we shouldn't add more things to tiny, but I think this one does make sense, could be a big win for Java and the sets of things that needs are pretty stable, so I don't think we're gonna follow this up with more requests afterwards.

D

It's just a tiny run image. What's the impact on size.

A

And what's the one playing around with this.

D

E

Can't quite hear you there, it.

D

Is the next-door backlog? Look at their prison awesome definitely be good to look at the effect it has on the size. The image I think making tiny Java apps that use that natively compiled using crawl seems pretty awesome. Hairlike definitely expands these kids for tightening out to many many many more actual production users that we have right now so, like very supportive of making these case work I'm a little bit worried that this is. Do you know what girl uses zealand for like?

D

Is it unpacked jars that are zipped or like something that we could do ahead of time during build, but.

A

I, don't have the detailed knowledge of this dancer that question I. Don't.

D

Question whether it should work I question whether it could be done in the build pack, for instance whether it really needs to be something that is installed on the tiny base image like I. Don't every go app to it. Could it be a.

A

Layer yeah the build back pulls in itself and installs. Maybe what.

E

About adding would this be something that could be possibly added like, even as like a mixin to Java build parks or no like if it's only useful in the Java cases. If.

D

You're talking about the like dynamic nixon's, then you can't do that on tiny coz, tiny can install packages. That's right.

F

D

You're talking about static mix-ins like like the stuff we have right now, you could make a different stack image for java, like a java, tiny to go tiny, but at that point I, don't know if it's worth it.

E

Yeah I remember us discussing like potentially doing separate Java stack images before and we decided that absolutely was not worth it, because maintaining just that many different stacks would I, don't think, would pay off. I will.

A

Say getting this working smoothly both? Are we bringing in the build pack or put it on the run? Image is going to you think it's a pretty cool story because there's a lot of hype around girl via and it's a little bit hard to get set up right now, and the fact that bill pipes could just do it and with the maven integration for Bill packs I feel like could bring a lot of people to come. Try out the kettlebell packs.

E

D

I'm leaving a comment for Ben about whether it makes sense due to launch later, instead.

D

But I'm: ok, adding it to the stack of me unless it's huge or something.

D

Cool anything else on tiny and Sela bungee any questions about what that means. What all is any of that.

D

All right so there's you know, Java has like a runtime component called a JVM right. It compiles into Java bytecode, which is not you know. Machine code is not not doesn't compile into assembler compiles until like a another language that the jet that's sort of more efficiently described, that the JVM runs right and all the all the languages in the JVM use this.

D

So growl is a compiler for Java code that takes the Java code and compiles it into it's like it makes java a little bit more like go it compiles it into native code, you know, and then you know it puts a runtime next to it, that you know executes as the native code executes with a garbage collector, and so it's like. So it looks a lot like go basically.

D

There are some speed advantages and then there are some speed disadvantages, because the JVM is highly highly optimized for running Java code, but it definitely results in in smaller binaries and a lot of the use cases that kind of spring penis looked at it's it's worth. It.

D

Okay, thank you.

D

All right anything else on that.

D

Next thing in the list is building builder ordering RFC right.

B

Yeah, so this was me, so we saw some people on the bill tax lap, slack saying that they couldn't deploy their apps on the because the older ordering change which put nginx and httpd before the PHP build pack making it impossible to detect true on the PHP build pack, because if you had an engine, ex-con file or httpd.conf file, which you need for the edge and X builds back sorry, what you need for the PHP builds back. It would detect either true on either the httpd or nginx back.

B

If that makes sense, so we wrote up an RFC two to come up with an ordering that we would stick to and so yeah it's pretty simple. We just kind of looked at the for drink in Cloud, Foundry and basically stuck with that.

B

So you kind of click on the RFC and see it's just basically a list of which we tried to do something fancier before in PR number 22. But it turns out that- and it's like hard to like just run some sort of algorithm to tell if the ordering is gonna make sense or not.

B

Based on our experience in cloud foundry yeah, that's it! You can check it out and approve it er we're not any feedback would be appreciated.

D

It's not exactly the same as the Cloud Foundry ordering Josh and I work together to come up with an order that, like tweaks it for like Java, has a really wide. You know, detection criteria now with all the things, and you can kind of combined some of them together now definitely interested in feedback from the java build pack side on kind of Java's placement. There wanna make sure that's. Okay,.

A

I'll take a look later.

E

So a question for this one I agree with the motivation and stuff I had one question on so there. It said that there could be a tool to validate this ordering is correct. Is there is that same tool or like a separate tool? Could that be made to create the ordering itself? Well.

B

We kind of just, but that was in the original request. Does that idea, but we kind of threw that out, because after talking to Stephen and with our previous experience in the cloud foundry build packs, it was just like, um like sub build packs, just need to come before others just to keep like good developer experiences, and it's hard to like tell just by looking at to build facts. What I should be I guess like an example is like.

B

Like a lot of dog net, core apps have packaged uh chase ons in them, and so we just put dotnet core before nodejs, because otherwise it'll detect truest nodejs app. If we put no J's first, that's hard to that's hard to like write code to like figure out. What's what's going on there, like you kind of, have to know, I think.

E

In that case, though, how would the validation work then.

B

D

You could run through all the sample like that kind of ideal sample apps for it and make sure they all get the right selection to build packs. If you wanted to write a test, you.

B

Could write tests.

D

Any other questions on that RFC.

D

Next thing on the list is opening working group meeting, isn't sharing on Paquito slack I.

F

Just threw this on here again, I know and like some internal meetings, we've talked about this a little bit. Are we like in a state of doing this, and maybe like next week, sharing this out to more folks, I.

D

Like that idea, where I think there's a little bit of risk and putting the zoom link somewhere public, but the we've done that in the cnv project- and it hasn't been problem so far-.

D

Idiocy should be as open as possible to anybody from any other company, or you know, contributors who just want to contribute to the project for fun to people enjoyed it and even work to you know, work their way through the governance structure of the project. If they won't do.

E

Currently, do we have any visibility on like current outside contributors? Do we have any consistent ones for Poquette? Oh.

D

We have some people in slack who ask questions to use the projects, um but I haven't seen very many PRS I, don't know if people who maintain errs are different. You posts have any experience with community people so far.

E

Yeah we've talked about ways it towards the beginning of all this to like lower barrier to entry to, like you, know, contribute and become part of this project, and that was like it's a lot of the feedback in the first hacker news post as well, where people are like it seems like I'm, a target demo for this, but I don't wash, should I care about this or like it's really confusing, so that some changes were addressed.

E

Like I remember on the website, some more documentation was released, but is there anything else that we could do to lower that barrier to entry to contribution? Or is that something we're? Even you know, interested in I? Think.

D

Making sure that we're engaging with the community, we have right now be really helpful, so like in a doctor, Nick opened a bunch of issues, and you know few other folks have just making sure that we're pretty responsive and that people feel like they're there's, not a core team of people that you know tied behind their backs all the time. That kind of thing be helpful, but that's a long process and we're coming from you know, process that doesn't look like that right. So it'll take some time. Yeah.

E

D

Think there is a you know: cash up and I think Xavier and some folks we're working on getting build packs, I, okayed, Ohio kind of like make sense and have a good story together. I don't know if there any updates on that.

F

No, not really. We have designer presence now on our side, silver like getting Sam to just help out with the overall design of the picado website and proving that improving, like the sort of limited Docs ecosystem that we have today and also like just working with the CMV project, a little bit more. So what Xavier has in mind is come more, a better concept of like an ecosystem.

F

If you can imagine like going to build packs, IO is the one place, that's gonna hold say references of everyone in the ecosystem so like Heroku and Google, and the Cato build packs and anyone else and build packs, io being the one place for like concept, definition and stuff, so we're still pretty early stages on that I think the CMV project has made some good effort on that so far, like they've, like improved their landing page a little bit, and things like that next steps here, just around this ecosystem that so Sam starts working on that a little more we'll start, bringing some of that under the team.

D

Awesome I think there's a once. We kind of have everything in order and feel like we're comfortable with the state of a lot of you know. Well, we've put together, we could do now marketing push for sure.

C

Quick question about this word: does the CMV project currently like have a zoom link where's that visible in.

D

The community repo I think is that this this zoom link nope. That's the CNBC link, Oh.

E

For the meeting Thursday's, what is it Wednesdays and Thursdays yeah too weak for that one.

D

All right anything else on that sounds like a good thing to do. No chancery architecture, oh.

C

Yeah, just this Ryoga tech chure got some feedback and we've kind of gone in and made some changes are going to provide a no they're built pack with this, like tiny binary, to manage all the processes for node and yarn. Apps might be useful outside of this, but just want people to take a second look any of the time.

D

um Can you still use it without the tiny binary.

C

Can you still use like write an app? That's, not gonna use it yeah.

D

C

D

Can you not use that build pack like you just have the build packs run, NPM start I see it requires tiny, that's what I'm that was MPM start requires tithing, and that seems interesting. Yeah.

C

So we could do that. I mean it's like such a small tool and it solves this problem of sending all signals to the node process that gets started by a yarn. Er, no, so seems like we could definitely add groups, and that would do this, but it just means you get some weird behavior around signal handling.

D

Like you could write, basically, you could swap the NPM started. Yarn start build packs for node start build packs triple pack. That didn't need tiny. Is that right.

C

Yeah I mean the other option is like have some user configuration to determine whether or not time tiny is actually gonna be used? But I, don't I feel like Dimmick is a recent issue that made it seem like. This was something that we always wanted to fix right, which is a few stock. Your apps you can get like. You will never have the opportunity to gracefully shut down an NPM or yarn app right now make.

D

Sense, I see a lot of node containers that don't use tiny or a process manager. I worry a little bit about like making the assumption that every every user wants it for every app. But it's not there like I trust, your judgment, I, don't think we have to do it. It's just curious.

D

If we is there's a feature where you can provide multiple alternative, build plans, and so, like NPM start could say no to NPM a node modules, tiny or node NPM node modules and then switch switch off without having to you know, but it would add some complexity and if you're convinced that it needs to be there for every app then like do it do its right. I just thought it was interesting leader, quiet.

C

Oh gosh I can't.

F

Hear you, oh sorry, is the problem around tiny, just specific to like the nodejs language family, or is there value and adding this to any of our other language families? Don't really know this.

C

Is like the first case where we definitely know that signals don't get passed down from the npm process to anything below it, I'm sure that there are other store commands. We write to do this. We haven't really investigated what those are, though,.

F

D

um Php we had the old PHP little peg has a process manager that was written in Python for it. So that might be one that could benefit argument could be simpler or maybe right now you know. If we haven't tested the signaling might be worth something to look at to.

D

D

Any other questions or anything else, on Oh Gerry architecture.

D

All right, I think that's the last one on our list. I wanted to mention, there's a lot of discussion of what we're called Route build packs, but maybe called stack packs or stack, build packs or, however, you want to describe the functionality of dynamically install an operating system packages at build time or before build time on existing stacks in the CNB project.

D

I might kind of schedule something to people want context on that kind of cash people out by the conversation, because it's pretty hard to understand going through all the CMB RFC's and it's very impactful, like you know, I mean still packs, can request. Packages means stacks. You know the meaning of mix-ins on stacks will change a bit. It's a big change to the API that enables like a major feature. People have been asking about for a long time, so I want to make sure everybody feels comfortable at the direction that's going or at least understands.

D

Why we're doing that.

E

Yeah I definitely like to be included in the conversation awesome.

D

All right anything else, any other topics for today.

D

Thanks for buddy.

E

D

E

That setup was really nice I'm jealous. Thank you, I. Don't.

B

Know why you must realize I should sit out here in my dark room upstairs.

B

This is in background.

C

B