Cloud Native Buildpacks Implementation Sync, 17 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Implementation Sync: 2021-06-17

Description

Meeting notes: https://bit.ly/38pal2Z

A

uh Do we have anyone willing to take notes today.

A

I can do it. Thank you, anthony, hey all right. uh I guess. Let's get started with status updates.

A

um Oh jesse is also he's having he's he's busy, so he may also join, but anyway uh I can give a a quick update. uh We are kind of closing in on the issues that are open for life cycle. Oh 12-0, um there's one sort of big question that we need to answer about: how we're going to architect some code to solve a tiny problem that it should be tiny, but it's uh it's challenging to solve.

A

So once we figure that out, um I put that on the agenda, but we should be able to proceed with uh shipping our release candidate, and I have some more thoughts about how that might go that I put on the agenda.

A

It's all for me.

B

um I had a question because I was thinking about this- is sort of related to this tiny issue, to sort of let the schema change in stack, tumble right, I was wondering if we thought about how this affects builders like right now stack tumble is baked into builders.

B

um Does that mean that no one will be able to use the new api without you know, pack changing the builder logic to create this file, um and if pac does change that logic, does that mean you'll be able to use new builders for the older apis? I think this is getting into one of our problems of api overlap that I hadn't really thought about until this moment that the stack tamil schema changed sort of uh changes like right now you can use any builder with any platform api, and that might not be true. Now.

A

I don't think anybody thought of that until now, so we should talk about it yeah. I.

B

Don't know like I don't have a great idea right now. It like dawned on me recently. This was a problem, so I I wonder, if like we might all need to think about it and then come back together with ideas.

A

Anybody else have updates to share.

A

I thought we could uh we could move on to release planning uh which we kind of talked about. um I could share my screen if we want to just take a quick look at the milestone. I don't think much has changed since last week, but just go.

A

A

A

So these are all the open issues in the milestone.

A

This is, this is just a. We need to open a question actually jason. Maybe we can ask you. uh We were gonna, ask a question and ggcr um I'll I'll, throw it on the agenda so that we can circle back to it, but I think, for the purposes of release, planning uh doesn't hopefully doesn't cause us additional work um and I'm I'm working on this chore as we speak. So it's all in motion. I think the big one that requires work is this well, which we touched on.

A

um There are changes to the schema for stack tomml and the new platform api, which has led to a cascade of changes in our code and may require additional work on the platform site as well or somewhere to handle this builder problem. um So let's we'll get to that.

A

um Any other comments on the milestone before we proceed.

A

Okay, all right, so uh I guess let's dive into these discussion, I think that might be circling back to we're. Gonna ignore the stackpack ones for now um so so convenient that you're. Here I had an action item from the last of these meetings to kind of reach out, um because we have some folks who are trying to use build packs and if they do not provide a trailing slash when they configure their secret, it all like just blows up because of the way we're doing the matching and I kind of dug into the code.

A

uh If I follow the trail right, I think you know this trailing. Slash is sort of a hard requirement and um I believe, there's code in docker as well. So we were wondering if there's anything that could be done to make this a friendlier experience for users, because they they get very confused as to why they're not able to connect to their registry with a secret that looks correct.

C

Right without completely knowing the full history of this, uh I'm sure that we could do a better thing in ggcr about this, and even something as simple as like. If there is no treading slash, add a trailing slash and keep going. uh I will talk to john who I think knows more about the the spec requirements, or at least behavioral. You know de facto spec requirements, but uh this off the top of my head, it seems like this should be easy to fix in ggcr and pick up um so yeah.

C

Thank you thank you for bringing it to our attention and for not uh doing what I think a lot of people would do, which is to paper over it before uh contributing it upstream. So definitely thank you for letting us know, and we will take a look and let you know actually I'm going to go. I'm going to uh file an issue on ggcr now to talk about this lcc you natalie or anyone else who wants uh and and reference this issue.

A

A

I guess with that uh that companion concludes the ones that we had for needs discussion, um although I think there's quite a few uh discussion items which maybe we could jump to, we usually leave some time to kind of just say what are the open rfcs that relate to implementation. I think all of these are actively being talked about. uh We just had they were all mentioned in some form, so I guess just encourage everyone to keep an eye on the discussion. That's happening, yeah.

D

Sorry, the there's a the core one I opened this morning. Oh.

A

Yeah, we don't have generally.

D

We said only, we said only one label per it's like a camping orient implementation. So you don't see it here, um but that's a really.

E

D

Change for you.

E

D

Folks, here it's like mostly life cycle work. uh If we do that, so just there's no point. I have no idea what.

B

If I switch it over to implementation, actually yeah.

D

I'm thinking that that's nothing, but I.

B

Think we usually just leave core for like governance and other things and implementation, usually for the big spec changes, although obviously core team owns the spec, so it's not clear why we do that, but I think might be more useful.

B

A

That's this, this rfc, which is like.

A

As you mentioned, quite a sweeping change, uh so yes, we should all weigh in on that on this one.

A

um I guess with that I think we can move on to our uh agenda. I know micah added this here, but jason. I believe you opened this issue. Do you want to speak to it.

C

Yeah, uh so I don't exactly remember how I started looking into all of this, uh but I noticed there was a there's, a long-standing life cycle issue for supporting arm, and I just sort of got curious about it and I think it's, I think, it's dangerously close to being possible to uh support arm and life cycle.

C

uh It's even easier, I think, to support arm and pack and possibly other things that might want to support other platforms um but sort of that that spiraled and ballooned into taking a look at how the image is actually built in the first place, and it seems like uh it's.

C

uh A lot of the logic is in the make file. The makefile builds one linux, amd64 uh binary, and then one windows, binary uh windows, amd64 and then does sort of parallel logic to assemble those into um to images and then to stitch them together in the release github action into a manifest list. um I have quite a lot of experience. Doing that sort of thing, that's exactly what code does? Oh basically is, is sort of like build packs except hyper focused on go only, uh and then it also has this arm where it does.

C

Some animal stuff- I don't really like that part, um but uh the idea for co is that if you have a go, binary or you know, go source and want to build it into an image and you don't depend on sego, uh we can do that for you very efficiently and we can do that with multi-arch images, multi-platform images uh very easily because it goes cross compile support. So um the idea is that either buildtext could build.

C

Next lifecycle could use code literally use code to build the images uh or just standardize on something like what code does um and do cross compilation. I think the image is actually only built on a windows machine right now.

C

Which might complicate your release process, it should be in theory. It should be possible to build a multi-arch image from whatever platform, because it's just a go binary um asterisk. You currently use sigo, and I don't think you have to uh I forget which issue that is, that uh is to use 6 30 life cycle, 6 30.. um If you do that, then you, then you can much more easily take advantage of those cross-compilation support and, I think, really at least straight uh streaming streamline slimmed down. I tried to say both of those words.

C

At the same time, um you release process into sort of a an easily reproducible stamp outable process across all architectures and os's.

C

That would make arm support easier, but that would also like, incidentally, make uh you know: s390x support easier and power pc easier. I don't know if that's all interesting to you but um yeah, so I I think micah uh mentioned that. The whole reason I'm here is that mike mentioned this meeting was happening, and this would be a good sort of set of people to talk to about this. um I don't know. uh Does anyone have any thoughts? Does anyone think this is a good idea, a bad idea?

C

Math idea needs more investigation.

D

I think there's a lot of complexity with the set uid stuff that led to the inclusion of seco that has to do with bugs and glib c versus lit versus uh muscle versus. I think emily, probably.

B

B

There's a reason we did this, you see go instead of just using this is called library. I don't 100, remember what the reason was, but I think I can filter through the history and find it out, but we definitely didn't want to use ceco. You know it would be fun to not use seagull, but I think there was a a hard blocker on just using the go standard live what we were trying to do.

C

Interesting, okay, that is that's good to know. I I uh suspected that was the case, because what kind of monster would you see go when they did.

B

Like just for fun yeah but yeah,.

C

uh But I thought just in case I mean if it's uh if we can root cause that to some issue in like this is called packaging go 111 that is now fixed in 116. That would be. That would be ideal all right. Maybe we didn't, we don't need to use it anymore, um but, uh oh, maybe that was that it this uh this set uid uh multi-threading issue, I'm not sure. uh uh Certainly I think we all agree. It would be.

C

It would be super nice not to use sego and then, if we do, uh I would like to uh propose that we we streamline the release process so that it can just be like four arch in these arches for os in linux windows.

C

You know build and stitch together an image. I think that would be awesome, uh but obviously you all know way more about this than I do so I I'm at your mercy.

B

I'm happy to hear that this issue has been solved and this does look like the issue we had originally because we could strip out ceo and like right. Now we have to compile and then alpine something.

C

Yeah, with muscle versus.

B

Glibc just using go like everything would be so much easier.

C

Yeah I saw uh uh I saw that in the release process and I was like oh no. What what? Why is this? No, but I assume there was a reason uh if this is the reason that's great, because that means we can not have it anymore. um If there is any sort of uh uh like test case that I can run to to prove that building with with uh says paul instead of sigo works, uh that would be ideal.

C

I couldn't I lost the thread on why that change was originally introduced, so emily, maybe maybe that would be helpful to have, but.

B

Yeah, I'm like it's slowly coming back to me, I think when we use the um go implementation, uh the dropping privileges only worked for a single thread. So if you ran one of these bills that has to like start as root and then drop privileges, some threads would end up with the wrong privileges.

B

So I don't think we had like an automated test for this, but if you just run enough builds in a row like you'll, eventually see it or you won't so you'll never know for sure. It's not there. It's statistically likely that it's not there if it succeeded.

C

Yeah yeah yeah, uh my favorite kind of bug- one you can never completely prove, is next uh uh great. So so um I will attempt to not reproduce this bug with the syscall change and then, uh if I think that change is sort of nice, whether or not we end up re rebuilding the release process, um but this being out of the way, would make it possible for the release process to be a lot more sort of standard across os's and platforms.

C

I I had originally proposed this or phrased this proposal in terms of literally adopt the tool co and I think, as I've thought about it over the last week. I think it's probably not a good fit exactly because you also do like simulink stuff to make the uh life cycle binary available as detector and analyzer or whatever. um I don't. I don't feel strongly that you need to literally exactly use code, but I think.

B

The reason we can't use code literally is the same reason that we really can't use build packs literally to build this, which is there's this elaborate file system based api that needs to happen in inside this image, so you like really have to lay things out. Very specifically, it's not you know it's not just a go up. That runs it's right, yeah.

C

uh I am here I mean this is purely curiosity and and not uh it doesn't need even to be answered. I'm curious why it seemed like you have this: this life cycle uh single life cycle entry, point binary that you sim link to many locations and then, when it runs it says, am I being run as reflector, and you know analyzer or detector or whatever? Is there a reason you chose to do that, instead of to just literally build eight, whatever binaries, that.

B

Makes the image smaller.

C

Yeah: okay, oh.

B

The only reason.

D

Yeah there's also the creator is like a way to execute all of them at the same time as one binary, and so it made it feel a little more cohesive.

D

There's one that'll string out all the you know like different parts of life cycle together as in a single execution, so you can do like a pack build in a single container. So I think that made it make more.

E

C

Yeah, no there's no uh uh there's no judgment there. I just thought it was a interesting and I'm sure, motivated by something image, size definitely makes sense. uh uh Aesthetic streamlining also makes sense. um Okay, so so I will try to see if the syscall change is good and if it is, we can get it in and then, if that goes in, we can see about trying to streamline the release process. uh At least streamlining it, for the two you have today would be great.

C

If streamlining it makes it much more easy to add two or three or four other options that would that would be even better um full disclosure. I intend to use this in tecton. We also use. uh We have a lot of users who want multi-arch support and uh increasingly annoyingly windows support in tecton, and so the more buildback supports this. The better tech time will be and the better build packs in these. So that's that's one thing.

D

Is that when you say, windows, support and tech time is that tech ton scheduling task containers on toe windows, kids, nodes, yeah, yeah, interesting.

C

So thanks, uh I don't really have anything else, but uh I will happily seed to the next discussion topic. If there is one.

A

Thank you for looking into this.

A

This would be cool, um so we have two. We have three more topics. I think we should maybe just like this is this. This one is like. I want to look at some code. I want people to tell me what to do with this problem. It's probably a longer discussion, so maybe we could get the first two, the last two out of the way um I I added them all um so the into the next life cycle we're planning to ship a release candidate.

A

We want to hopefully avoid some of the the uh the the mistakes of the past and have like some sanity checks on our stuff in in a variety of environments. That's that's the dream right uh before we ship the final one. um When I floated this with the picano team, they said we'll happily consume a release candidate, but please don't tag it with a senver tag, at least sorry.

A

The life cycle image that we publish, please don't tag that one with a semford tag, because we'll mistake it for a real one and pull it into our our production pipeline. um They they've since fixed that issue right. So they can now properly tell if it's you know, there's like a dash rc or whatever that they won't treat it as a fully released one, um but it kind of got me thinking like who else right?

A

Who else out there is, uh you know, has automated pipelines that might trigger on a new life cycle image, and you know.

A

Like if we just all of a sudden start shipping, these images, these release candidate images, you know- are we gonna, create problems for other people that we don't know about right? um So.

A

I don't know how valid is that concern, but when I created the github action to actually create the release candidate, uh it will create a release candidate on github right, so you'll get a. You know that github release page with the tar balls are all you know, versioned with an rc, but it won't actually retag the lifecycle image. So it you know, you'd have to know the commit shot and go pull it.

A

You know deliberately, which you know I don't I I don't know how that workflow is, um and I was also thinking that you know even a release candidate on github could confuse some people right. You never know what logic people have to pull in these artifacts, so um just wanted to get everyone's thoughts.

B

There's a part of me that says we shouldn't worry about it too much like. I think. The thing that we shouldn't do is, I know like right now, when we publish semver life cycle images, when we publish a new one, we also move the latest tag. I definitely don't want to move the latest tag to point at the rc um other than that I feel like people can deal with it like.

B

I know, we've had situations where we're consuming automated dependencies and you know other types of automated dependencies in a bunch of situations, and maybe someone doesn't get the logic correctly right for how someone starts publishing an rc. But hopefully you know a pair of someone is looking at things before things get shipped out to users, and I don't.

B

I think this is a standard enough way to do things that I don't think we're really inconveniencing people not saying that nothing will break for anyone, but I think it's it's reasonable.

D

Is the potato team doing something besides, looking at the latest tag like.

A

Yeah, so actually emily your comment, it got me thinking, uh I don't know if we currently move the latest tag every time we because we publish a new image for every new, commit to maine and the release branches, and so we might actually be updating the latest tag already. Let me just see where do I get latest.

D

It's usually how people pull for you know. New images fortunately see.

A

These are all pushed you know three days ago. If I can, I just do what's latest oh last pushed a month ago, so we must be. That must be done in our github action that uh that that re-tags.

B

A

B

Sounds insane right yeah, a month ago, yeah.

A

Yeah, if I look for.

D

I mean that there is a way to get tags back and then you know you could some verb, parse the tags and then try to figure out the latest thing. People are doing that that'd be really interesting to hear about, but I I don't think I've seen anybody doing things. I'm.

B

Not so you know, I worked on the java stuff and it sort of has its own automation, it's different from the other buildbacks, but I don't want how a lot of the image automation works on the java side is. Basically, you know using crane ls to get a bunch of tags and then sorting them and then picking the latest one. So it's not just using latest all the time.

B

A

So maybe we could tag the life cycle image with. You know oh 12 0 rc.1, but we don't update the latest tag and you know some people like the kettle team has already fixed this. But if somebody didn't update their logic to exclude release candidates, it will find out.

D

I think if you're gonna list all the tags and then sort them and then pick the greatest version, then the onus is definitely on you not to pick up rc's there that you know it seems like that's.

D

If you're already implementing the custom logic.

A

I mean we can do our best to publicize this right to let people know that this is happening.

A

A message in the general channel or something.

A

Okay, having a tagged with the version of the livestream that'll, make other things simpler and I'll probably make it easier to actually use that rc image. So right now I was putting a little note in the release. Notes like go go here to find the image associated with this binary. But that's that's not a good experience.

A

Okay, uh I'm glad we have direction on that um and then the other thing that I wanted to discuss. This is just kind of like conversation. That's been happening among various people, but do we need to have this meeting every week.

D

uh On the cadence, I'm trying to drop my platform maintainership, because I don't do very much on that side, but I still open rfcs and kind of occasionally do things around the lifecycle. So I'm trying to show up to more of these or, like wanna, pick the sub team and show up to it. If that makes sense, um so bi-weekly cadence would definitely be better for me, because my calendar is terrible if you're looking for votes, but I think emily's feedback would probably be more important.

A

Well, uh we'll we'll leave this open, I would can bring it up again next time. I I. I also think that it's good that we touch base. You know in person every so often like, don't want to lose that, but I'm not sure that a weekly cadence is really necessary and you know we spend a lot of time on the updates and it's like you know it doesn't change a lot week to week so anyway, uh we'll bring it up again um in the time that we have left. I would love to.

A

I know: anthony's had a chance to look at this code, but I'd love to kind of just talk about this problem that we're facing, which is um you know right now, uh when you think about the tomal files that we're using, for you know, platform communication. A lot of those are just represented, I mean all of them. Right now are just like bare struts that we marshal into and out of, or you know we might construct um on the fly and um we're currently making the first non-additive change.

A

You know in a while uh for the sector in between, so if I pull it up, no packs back and I'll just look at the platform side, here's stack tomml.

A

As it was right and then if I go- and I look at this same one on the new branch.

A

Right this is a this. Is a non-additive change. Oh, I don't know, okay, so here that you can see both of them right. What happened? Did I totally misunderstand what was going on? Oh, no! No, sorry! No! No! No! No! It's analyzed.

A

Oh, oh then, emily's builder concern goes away too right. I I was wrong, but actually, I think, that's still a concern. But let's talk about the let's talk about this next analyze: okay, yeah! So here's how here's, how it was, and here how is how it's going to be in the future- and so you know having a bear struck is like it's no longer.

A

You know simple in the code um and and we've talked about various ways forward in the past, we've said you know what, if we had one struct that held all the different ways that you could represent the data and then you just have some sort of like validator per api, that you know checks that the data came in in the format that you're expecting it. That's how we went in a previous pr. We had like an encoder decoder, which I think it actually worked out pretty.

A

Well, so maybe that's one way, but then, when you need to get the data out right when I need to ask okay, you know give me give me the image that was analyzed. You have to know how to refer to it. Right you're going to either do dot image for the old api, or you do previous image for the new apis. It's like kind of still messy um and so the I I I spent some time.

A

I was like all right how how how do I want to solve this, and I made a pr um which has many many changed files, because I also had to move some stuff around, and so I just want to go back and get the interesting ones.

A

So if you look at the let's go to platform, what did I do so and I also recognized I tried to show this to emily in about 15 minutes a couple days ago, and she was like I, you know this is a lot right, so I don't expect someone who hasn't read the code to kind of know what's going on, but I made an interface to front the struct.

A

I put it in this common place in the platform package, and I made this. We can ignore this for now some complicated interface to allow you to build up the struct while keeping it platform agnostic, um and then I moved all of the bear strucks right because I didn't want to have to like I mean this, you could like make infinite interfaces right. I wanted to be able to return a pointer to a concrete thing without having to have that also be an interface.

A

So I just moved all of the bearish trucks into this common place as well, um and then what I did is for the individual platforms.

A

So many of them for the individual platforms right, you have this like builder stuff. For now you have this.

A

Here's the here's, this the bears, whatever here's the struck, that holds the data, and then I have this like you know it implements the interface and you know for the fields that it doesn't care about. It just returns empty stuff, right, um previous image and previous metadata being the stuff that the earlier platform apis care about right and then it's similarly for the oh gosh, the newer platform.

A

Right is going to have you know it's going to implement the interface and it's going to return more stuff because it holds more stuff and that's that's the essence of the change. You know, however, it's like if you ignore, I mean if you ignore the 54 change files, which was like me moving stuff around. um It's a it's just a lot of like boiler boilerplate. I know anthony had some thoughts about this builder pattern.

A

I I just didn't know uh because you have a couple places where the analyzer like it needs to build up the struct, and then it calls right tamil on that struct and it's like.

A

It you don't know it's gotta work for all the platforms, so I needed some some mechanism to like keep that struck private, but also so it's just like there's a lot of code. I didn't enjoy having to write all this code. I don't want to have to write all this code for every single file. You know if we end up changing like I don't know, group tommle or something in the future like this is just a lot of like boilerplate. It feels like at the same time it works and it makes sense.

A

You know I'm like I I you know when you read the code. It's like. Oh, is this it's very clear to me: if I'm not in the platform package, it's like very clear what's happening, so this is by my ask for just like please give feedback because in some sense we're setting a pattern for what we want to do going forward, and you know I I want to get it right because it's just been pain been a pain for a while.

D

What's the purpose of the builder pattern and for you know these particular structs.

A

It might have been me just getting really excited because I just learned about the builder pattern, but it's like the the individual structs. Let me see where it's used. It's like this is private right. I I I want to.

A

I didn't want to like uh when the analyzer is building up the struct, so that it can write it into a tamil file later I didn't want to have to have like, if platform api less than blah. You know, write this struct and then write this tomale right like it.

A

I needed a way to construct.

A

This thing outside of the package and have it be platform agnostic.

A

So it could be, you could just have like a function that takes a bunch of inputs. Right ignores some of them. That could also work.

D

The like a key thing about that. The builder pattern where I've seen it used before is the kind of lazy evaluation of everything so like. If you look at like ggcr, is one of my great examples of a large large go library, it's kind of close in the space that you know does some similar things to what we do.

D

I think it uses builder pattern a lot for like you're, going to write a remote image, or you know, like kinds of actions, look really different depending on the type of image, but they all fit the v1 image interface right, which is kind of similar to what you're doing with the like. You want to create an abstraction across different versions of the api, that's similar, but I think they use the builder pattern more.

D

You know, because those actions, you know, don't really make sense to occur individually. It's better. If you configure your object, you know using that pattern and then apply all the operations at once. You know with custom logic that pertains to the you know if it's remote versus, if it's local versus, if it's like you know in memory or something like that, if you're just setting struct fields it you know and builders, the building pattern is like adding complexity to the kind of design that you you don't like.

D

You know, I wonder if the moving to the interface pattern, so you reduce the complexity of having to deal with the different versions at the same time and then flattening the builder stuff out. Maybe it seems like yeah, you go if that makes sense, but you know it's a like. Without it's been a long time since I've I've looked at, you know, what's good. Like uh last last time I looked at lifecycle code, there weren't multiple versions of things in it, so uh the you know I mean that could be terrible advice.

A

No, I'm I I I'd be happy to get. I mean it was fun to write this code. I'd be happy to eliminate a bunch of it in favor of like a constructor that just takes like six things and ignore some of them. If it's on the older platform for my p.I, I think that would be just fine.

D

And the reason that's like so I'm thinking about the problem, you have these files and they look different for different versions and the builder pattern is used to create structs out of the files, and you is that right and you don't want to um you- can't really marshal, because the structs could look different.

D

Is there a way you could say like like an interface I can imagine, for this is like you know, b2 or because you know the api version ahead of time right, you're, not you're, not you're, not trying to guess what the files api version is. Is that right um like? Could you have like v2 package dot? You know create, and then you pass the file name. Then it returns the struct for that package.

D

You know as a generic interface.

A

Oh yeah, it's that I think we have. They have that somewhere else, uh because there's like different ways like there's different ways that we we make the structs like. Sometimes we read the information from a file. Sometimes it's like.

A

We know the information and we're assembling the struct so that we can then write it to the file later. um So, like I mean this, this could all this could all go away right in favor of like one function that you know takes a bunch of stuff and returns a you know, returns the interface.

D

Yes, do you need a function, or do you just just need a struct and then because all the methods on the outside are just going to set fields in the struct? Is there a reason you want the abstraction for creating the struct from scratch versus just having the consumer? I guess the struct isn't exported. Is that the problem? Could you export destruct.

A

I could export this struct, but I don't want to like an if else in the like say in the lifecycle package. That's like you know if, if platform api, you know this then make this kind of struct and if platform api, this make this kind of struct. I just want, like the struct itself, to be sort of invisible and just like, I want a data object that has these things and the platform's responsible for like for putting it into the struct.

A

That makes sense for that platform and then later this would be better if we were looking at the code but then later in the lifecycle package, it's just like okay, write, autumnal file that contains the contents of the the thing I gave you right like, and I I don't have to care about the the schema for that file.

A

D

You have a shouldn't. This might be a terrible idea, but could you have a generic struct that takes all of the fields right or that takes the format that you want to create and then have all of the individual version packages consume that struct and then turn that struct into the version specific things that way you don't have to use an interface. You don't have to use like a pattern that requires code outside of the package, if that makes sense it from outside of the package. It just looks like I do normal go things.

D

I build a struct out of fields that I understand and then that struct is then translated into version. Specific structs.

C

E

You're saying casting right essentially cast it into things of different versions.

D

Yeah yeah, I mean like the cast, would be you know manually implemented in the package right. It's not not like a like a direct type cast or something like that. But but yeah like like have a struct. That is the generic version that then can turn into a version specific type, and you probably you probably wouldn't create a persistent instance of it. It would be like you call, you know, vo0 dot, new analyze metadata, and then you pass analyze metadata config, but that config doesn't live in the package.

D

The version package, the config lives in a general package and then.

C

D

Is the versioned uh instance of the struct.

A

Then, how do I get stuff out of that truck? So if I want to like later like introspect on well, what was the previous image right? I don't want to have to know to that it. You know in one version of the platform it lives in this field, on the struct and in another version of the platform it leaves in this field like I just want to. I just want the previous image. I don't care how it's stored.

A

D

And maybe I either don't understand or I'm not explaining? Well, I think if that makes sense, maybe the so stepping back your your goal is to create a version specific struct right and you want you don't want a bunch of conditionals when you're constructing the version. Specific struct, um like you, don't want to say, like.

D

This better way to explain this.

D

Describe how okay, sorry, when you're using the builder right, let's talk about what it looks like now right um on the outside, you know what version you're trying to create, and so you instantiate a builder for that version, and then what are you switching on like? What are your? What what are your conditionals on? In that case.

A

Oh, no, that's fine! I don't mind instantiating, because I can just pass in the platform api to some factory and get back a builder. That is for my platform, api right.

D

That's okay, um if you didn't do that, what's what's the data that you would be, you know using um excessive conditionals in order to uh have to deal with.

D

Creating the struct that you need.

A

So if we get, if we have a builder that gives us back a struct with like everything right, some generic, like okay, if I'm understanding quickly like I, the builder- is giving me back a struct right and it's the struct that is sort of generic to hold things for every platform right and then I want to pull information out of that struct.

A

I have to know which field holds the information I care about, and therefore I need some sort of like platform, specific logic in my consumer, which is the lifecycle package right.

D

Got it I'm not talking about a way of constructing the struct? I'm not I'm not talking about returning a generic struct. Let me talk about generic strike as input, because I thought I thought the problem was that it's like hard to create the version specific struct with the interface in front of it without the builder pattern, because uh you have to you know, switch on conditionals based on the version to construct the version specific struct, but instead, if you had a generic input, struct that resulted in version specific output, struct does that.

D

Is it a simpler way.

A

Maybe it's just easier to look at because I just pulled my id yeah just get the the what's this one.

A

Okay, let's just get started this one! Okay: uh where do we actually consume this? So we just say like with previous image, so that's like in the analyzer right, so the analyzer.

A

Needs to return it, it needs to like eventually, this analyze metadata is going to be written to automo file right, so it needs to return a thing that has these properties right and in this package I don't want to care what field holds the previous image. I just want a thing that has that data stored in the appropriate format.

D

Okay, so a dot platform is a version or a dot platform is a version specific platform. Is that right, but the builder that get er, but that conforms to an interface that allows new analyze meta with new analyze metadata builder, so that you can get different versions of the struck back in different versions. The underlying structure structure built when you run those commands right yeah. So, instead of instead of using, like the maybe a little more heavyweight builder pattern that requires you know, the chain function, calls and interfaces and boilerplate there.

D

You could have one generic struct instead, that just has fields has fields that apply to all of them like previous and previous image, metadata or whatever, and then you construct that here and then you do a dot platform dot you know create or something and you pass the generic struct to it and create is implemented to consume the generic struct and transform it into the version specific struct. That way, you don't need the builder boiler plate and you can kind of simplify your input thing, and maybe that that gives you enough.

D

You know removal of boilerplate to switch to the interfaces on the other end also.

A

Yeah, no, I I totally I'm not married to the builder pattern. I was just like. Oh this will work. You know, I think uh something simpler. I'm totally up for that in the for the creation of the struct.

D

um That's all I meant is like it's like there's an alternative to builder. If you wanted to keep ness there, but he didn't want to do the heavyweight builder part as well. Sorry,.

A

Yeah, so that's that's fair! I I uh yeah. I I think that that that will go that will go part of the way toward uh removing um removing uh the. I guess the volume of the changes that were required. A lot of this is just renamed stuff right, so, like this part, would condense down into you know.

A

A few lines of code right- and so it's really like- are we okay with this boilerplate, because that will never go away right.

D

um There's a way to reduce that boiler plate sort of using embedded structs, so you can come up with an embedded struct that returns all nil values or sometimes, instead of mill values. In this case, I'd like to use, I use a return value and an okay and just make people explicitly check that like because, if you're saying it's not supported by that version, you may want that information separate from the field wasn't empty about that, but separately from that a trick. I've used in this case is create a like a thing called.

D

You know: empty, empty, analyzed, metadata or empty in the name of the interface that always returns all the empty values and then embed that struct inside of all of your substructs, and they inherit those methods, including.

B

Through an interface.

D

And you don't have to re-implement them over and over again with empty values. You just have to implement the ones that.

B

D

It's a little clutchy sort of, but it works really well.

E

No, I I dig it it's just you know. I think the problem here is about when it gets used right like when you're using a specific field right like what does the code look like is this? Is it gonna be like if platform, six? Oh six, do this? If platform o7 do that and then how many times does that show up right.

D

um You wouldn't need that because the like.

A

Like this, this is.

D

A

D

Is just a trick for reducing the boilerplate, that's highlighted there, it doesn't doesn't change anything on the outside. It's consumed exactly the same way. This is, if that makes sense.

E

A

Don't I give it a try with with the suggestions that you made given, is, I think, and then we can kind of revisit it and we're getting a long time as well? But I I don't know I I like the direction this is going if we could just reduce some of the the amount of stuff right that makes.

D

That makes sense I want to make sure I understand anthony's comment, though too.

E

Well, some fields get, let's say a field gets renamed right, like it's literally getting renamed and then the actual field name changes right like it used to be in.

E

I don't know bill pack string and now it's in bp string whatever right. What would the code look like from someone who's consuming the struct.

D

I think that would just change at the level of this this file, that's highlighted right because the field name and the json could change, and maybe you wouldn't have to change the name of the struct. You probably want to the name of the field in the struct. You'd probably want to change the name of the field of destruct too just to match, but then it would only have to change in this this file that encapsulates that logic. It wouldn't have to change for all consumers of the general.

B

D

Of data, because the interface would stay the same unless I'm misunderstanding terrible, I'm sorry.

D

A

I think we would have that problem anyway, right, no matter which way we did if it if the field, if the the name changed, we would have to make those changes anyway.

D

But you don't have to change the interface right, and so these the consumers.

E

D

Aren't affected by it.

E

I think I'm just not communicating uh correctly, but I appreciate the discussion, though I think plenty of good ideas here. I hope natalie you got what you want, though, out of it.

A

I got I mean I think I have. I have more direction than I had before. uh Well I'll keep bringing this up. I think this is. Sadly this is something that's going to hold the next. We have to figure this out. You know before we can ship the next milestone, but uh I'm gonna. Oh, you know I'll keep prodding uh for input from various. You know people and hopefully we can get enough consensus that we can proceed, um but I I already with with the suggestions that have been made here.

A

I feel that uh you know I can make it better. So.

D

That's good. My the two things I mentioned are just random ideas like don't don't feel like you need to do that for any reason uh you know emily might have. I really have a different thing and I don't know what patterns you're already using so also like just you know, create a grain of salt or whatever thanks.

A

Thank you. Thank you.