VMware Cartographer Office Hours, 14 Feb 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cartographer Office Hours - Feb 14th, 2022

Description

00:00 Intro
01:03 CRD Shortnames feature request
03:14 New RFC: add params object to MatchField selector
12:50 Incoming RFC: resources in workload status
16:48 New RFC: add MatchFields to top level of blueprints
21:51 New board to track RFC status: https://github.com/vmware-tanzu/cartographer/projects/2
26:58 Tanzu level best practices for OSS
29:22 Multipath discussion

The purpose of this meeting is to discuss architecture-changing ideas (in the form of RFCs) and provide in-depth support to the community of Cartographer contributors.
You can continue the conversation by adding comments to the RFCs PR: https://github.com/vmware-tanzu/cartographer/labels/rfc

Agenda: https://docs.google.com/document/d/1ImIh7qBrOLOvGMCzY6AURhE-a68IE9_EbCf0g5s18vc/edit?usp=sharing

A

Right folks welcome everyone to the photographer office hours today is the day after super bowl and also is monday cartographer office hours great. Let me share my screen real, quick.

A

uh There you go. I would like to give a warm welcome to to all the maintainers team, to the the people behind the project and also to mr scott rosenberg great to have you here, especially because we'll discuss an idea that you provided. So that's great, welcome all right um yeah who would like to take notes for this station.

B

I can take notes.

A

A

Put you in there in a while: um okay there, you go so first topic here, crd shore names, uh so I don't know if caught rosenberg, if you're available to just provide any comment or intro to the idea.

C

uh Yeah, um so in general, I think that um you know, as I've been dealing more and more with cartographer. uh Having short names would make it much easier to run cube cuddle commands uh just writing out. The entire long names of the crds would be is very tedious, um and so the idea was just to add um some short names there. I did run a check through, like cancer application platform and other environments.

C

I ran some tests. uh You know on tanza application platform as well as in other kubernetes clusters, to see if there would be any overlapping of names and I didn't find any either um so that was you know. In general, I put a list of the ones that exist like intense application platform and things like that to make sure we wouldn't overlap them, but that was the general idea just to make it easier to run commands against the different crds.

D

Awesome yeah thanks for doing that. um That's very, very helpful. I think everyone is pretty much on board that we should definitely do this.

D

We just wanted to like take a minute to uh like let everyone know that we're going to do this and then, if anyone had any objections to any of the short names, they could voice them now. Otherwise, we'd just move forward with it.

D

So yeah thanks scott for putting that together.

A

Yeah, thank you. Thank you to team for moving forward with, with this request, awesome: okay, new rfc um joshua. I think this is your show from this point on. As far as match people selected there you go. Thank you.

D

Everything else is me right um cool. Do you mind clicking on the readable one yeah great.

D

So this rc is about adding a uh top level params object to the match, fields, selector, um so that we don't have to do crazy things like that top yellow block um where we have to like pick a param off. I I actually don't even know if that'll work, I kind of just assumed that something like that would work based on some of our other selectors, but we have to like pick a param by name and then you know pick up its value off. That object.

D

And the other thing is right: now we can based on the match field selectors.

D

We could pick params off of a workload, but we can't pick params off of a supply chain, so you couldn't do any sort of conditional branching based on something that's only defined in a supply chain and the example I provide here is you know? Maybe you have some sort of configuration um that determines whether or not you want to push to get ups uh get repository or a reg ops, like registry.

D

And so, by giving access to params we're basically enabling that workflow.

B

I think this is super simple to do, since this is what we do today for the stamper.

B

We go ahead and we basically do our param hierarchy and then send params down to the stamper as part of the context and what we've just implemented for options this pr out, for it um also takes a context, and so I think we could just pass params as part of that context.

B

E

Support the fault values specified in the template.

D

No, it would that's kind of that's, so that's the gotcha right um you, you can't take the entire um param hierarchy, because at the option level right, we don't have. We haven't matched on a template yet so we wouldn't be able to take the params from a template.

D

The other interesting gotcha is that it it's not part of this issue, but we we oftentimes talk about that.

D

A selector for a template should probably work kind of the same way as the top level supply chain selector um and if we were to add the params and match fields at the supply chain level, you wouldn't be able to use supply chain params at that point either, because, if you're selecting also, if you're, trying to select a supply chain, all you have access to at that point is the workload I don't know if I made that clear, but if you're like, if you wanted to select at the top level on a supply chain, you could only reference the workload params.

D

If you want to select a template within a supply chain, you can use the workload params or the supply chain params, but you could never use the uh the template params because you haven't made that decision yet.

B

I'm wondering if, instead of like a match fields, this becomes like match params or something outside of, because right, right now match fields is simply on the workload.

B

I don't know I'm just curious of like breaking it out gives us any more flexibility or like doesn't tie us to the ways we've done things before.

F

I think you're actually touching, I'm. I think you I'll just describe this, but I'm just not fully making sure I understand um when I see on the what it is when in the match fields, when you say prams.promotion, for example, is that doing?

F

Is that automatically doing that hierarchy filling in or like a replace on whether it came from workload or came from the cluster supply chain or like? How am I saying which set of params I'm getting in that match field.

B

Yeah we have a hierarchy specified somewhere in the docs right like it's the same, I assume it'd be like the same hierarchy that we have today, where I'm not even going to pretend to know the actual.

A

F

B

It's it it's in the code right.

F

There's some magic helper that when I say prams in match field it'll like resolve to one or the other, and if I wanted to say I don't want the magic helper and I just want to explicitly pick from somewhere. Is that that's not a thing.

D

Then you would probably have to reference it right off the workload. um The david sorry, if you scroll up.

D

Yeah, sorry, that that top block, like in theory, that very first yaml block workload.spec params, some like you're you're picking the one with the name of something and then you're picking the value from that gotcha. So.

F

In theory, you could do pick it directly off workload or do the magic hierarchy that you could never say. That's sort of just like an accidental, because we have these paths.

F

D

D

Match fields match params. I don't have a huge opinion. There.

G

Is the idea here that you know preams? Is this thing that's merged together?

G

You know in a lot of cases from different sources, and we want to make params kind of like a top level thing like workload, but some kind of magic object.

G

Then I think match fields, kind of works for me because it's like it's doing the same thing on something else that doesn't actually resource in the cluster. But you could treat like a resource in a cluster. It seems pretty reasonable.

E

I mean the whole point of this. It would seem.

E

Is that we're able to put the faults in our supply chain if we needed them and also just make it a little easier to like it's tidier, just to say params, and it is workload where you are explicitly jumping the uh the params uh hierarchy right?

E

Is it the best way to put it josh, yeah.

D

I think the other thing the other call out is like right now, if you only if you had a param that was only specified in the supply chain, you couldn't do anything with it.

E

Yeah yeah, which you wouldn't want to if it was specified hard, but if it had a default, you would right.

E

There is no point, uh switching on uh declared value at the top of the supply chain, because that should always resolve one way and editing the supply chain is required to change it or do you perceive people wanting to like I've got to pass through the supply chain, but the only way you'll ever get the other one is when I do the edit to the supply chain as like a constant if def or something like that, which is what it would feel like if you used value, whereas if you use default value, that would make sense to me.

D

Yeah, the only.

D

The only way I could see you may be wanting to do it based on actual concrete value in the supply chain is if it was templated out.

D

And you wanted to be able to make that you wanted to still manage one supply chain, but you wanted to template it at installation time, but that's a pretty contrived example so, which.

E

Is like, if death right, yeah kind.

D

Of yeah it is yeah yeah, yeah.

E

Yeah, that makes sense I mean I get that, but.

D

Yeah true yeah.

E

D

E

And you're, saying like when using ytt, to generate your supply chains as you deploy them. That.

B

E

Sense too, okay, yeah and we're only supposed to introduce these, so I won't go into any further all the questions. I could possibly ask.

D

I think one call out that you want to make before we move on, though, is that I don't think this breaks our rule um of that we we can statically analyze these things right, because we have all the information at the time. So this doesn't break our rule. That's.

E

Why I asked about templates, but you're right, you couldn't even hope to so. Yes, that's great, no dynamicism, no special prerogative to set a param in the middle of a template generation, either right.

A

Right any other comment regarding this rst: no, okay, you can move to a new one up. Would you like to share your screen joshua, so you can read that or not no problem.

D

I I can try. If I share my screen, my upload usually dies and then my audio gets really bad.

D

I think you're doing a great job, damn.

D

So this next one is uh scott, is planning on submitting uh this as a separate rlc. So this is a comment that he made on the rfc 18 artifact tracing rfc, and he proposed an alternate status in the workload which takes a resource-centric approach instead of an artifact-centric approach and to me anyway, it seems like long-term. We could. We probably want to support some version of both um right and so scott's, going to submit this as a separate rfc that we can contemplate in addition to rfc 18.

G

The separate rfc doesn't aim to solve provenance. It just aims to allow representation of the graph without having to go around 20 different resources and correlate everything which is still extremely important thing to do, but I just want to make sure that's. I have the right interpretation.

G

Yep awesome that sounds great. Does.

E

This rfc tackle the fact that we now have optional outputs and that we only need to know like does it say we only need to know the ones selected and um uh the others are not going to be used for this workload, because this is workload level status right.

D

E

D

Idea is that your resource list here would only represent the objects that exist in the cluster right. So if there's a path not taken um it shouldn't be represented here. If there was a path taken at one point and then that path changed because of something right uh in theory, it should kind of only show the the latest and greatest the new representation, but I think we've kind of talked about gcn those resources when.

E

You change your path in a different.

D

Issue so right.

E

Yeah so there's two there's two sort of like prereqs there gc and then ultimately we can do even better, but we'll have to make a really strong play to um eliminate being able to switch tvks and templates all right, and then we can actually clean them up. But that's quite a ways down makes sense.

D

Yeah, but this rfc is really just gonna, be about servicing that information right surfacing the information of what's uh what's stamped out for a given workload,.

A

Wait um any other comment here: if not, we can move to the update to this particular rfc.

D

So this is the snippets rfc and we've talked about this a whole bunch.

D

I aggregated all of the different syntaxes um other than the one proposed in the rc and I've created a gist for all the different syntaxes. So we have them all in one place, so we can just go through and compare and contrast and talk about which one we like the most, because it would be very nice to move one of either this or multipath alone.

D

So please take a look at that and add any comments.

A

Yeah, that will be great awesome.

A

Thank you. Here we go, and now there is a new rfc.

D

So this is another new rfc I put together. This is pretty straightforward, at least on the surface. um All this does is, it proposes. We add the match fields from the template level, selector to the top level supply chain, selector, um which I think we have an implicit goal of like keeping these two selectors like fairly similar.

D

um The big gotcha here is that this isn't. This is not a backwards compatible change, uh so it gets complicated.

E

Why is it not backwards.

B

Well, that's, I think I mean I was going to say so, would we add, match labels and that's the piece that makes it not backwards compatible? I mean it could be backwards compatible if we don't add, match labels, but that's really not great syntax, like I don't want to do that right.

E

Oh as in making the match labels that we have now consistent with right.

B

It's missing one key.

E

At the moment, selector is there's something bespoke yeah.

A

E

Forgotten about that.

B

You could make.

E

It we're having.

B

Sorry, I think we have a few breaking changes coming up, so I don't know if we like support, match labels and this weird top level no match labels thing for a little bit.

D

And also like, if we do have a bunch of breaking things that we want to push through, we could look at adding a conversion web hook and version our supply chain.

D

I actually called that out in the rfc down at the bottom.

G

I want to point out: I don't think we should do this, but you could make it a mostly not breaking change by respecting labels that aren't matched labels or mesh fields at the top level of the thing, I would not recommend that path forward, though I think a conversion might pose. If we're you know, we're gonna need that infrastructure eventually right.

G

I think that's probably the right way to go.

E

I think um there's just a like there's something cautionary here, to make sure that we look for um look look for idioms as much as we can before. We accept our rfcs on design, especially things that yeah. You know, you know what I'm saying this this this existed long before this. uh For some reason, we use the bespoke way to select.

F

Do we have somewhere that we're aggregating, I'm wondering if just like, we could try and push the like if it makes sense to aggregate all the breaking changes into sort of a single release? It's gonna be painful, but at least it's only one painful release. We have a list of them right now.

E

Maybe that could also be a ramp up to a 1.0.

G

I think longer term we need to have like this kind of change is easy to do relatively seamlessly from the user's perspective right. They can still use the old version of the resource. We can keep that supported for as long as we want, I think, longer term. We need maybe some larger tap level policies around. You know how many versions of things do we want to keep supported. You know what does deprecation look like right.

G

I don't think we want to keep those old things around forever, um we'll probably set just in very brief conversations with other programs about this. I've heard like maybe we keep things around for six months or a year, or something right um so as you're thinking about those conversion web hooks and when you want to time releases that collect breaking changes. Imagine that you'll keep the old version supported around for significant enough.

G

The time that you know, customers really have time to move away from the old resource versions, um but also that we won't have to keep you know. It's not a goal to keep everything around forever right, but there may be changes that they need to make over. You know a longer period of time.

G

Folks, on the secure tool, side are thinking about the same problems right now and they have a bunch of breaking changes. They want to make.

A

Right anything else regarding new rfc, not uh we can move to the next one. It's um last week we had a suggestion from mr kartik looncat and uh on how to have a board with different columns to track status. Variabilities, and I believe it was rashid who first started.

A

You know organizing this board, so um maybe I was to use this to keep track of the status and and also spend some time during office hours to actually review status and okay right now the columns are widely inaccurate, so yeah. Maybe the proposal would be to start checking with where to put the you know, each rfc in the appropriate column right fresh.

E

Yeah that'd be great, I mean if it's to work, we have to use it right. I think I think it's helpful right. um I've certainly not been able to work out what the rfcs are doing and what we need. There's um there's. Definitely the uh I feel like the in review is maybe hopefully the important thing to draw the bead for our uh our maintainers that are not able to be.

E

You know with the team all the times. People like uh stephen um and I forget, is it scott who's rather like not every.

H

E

Or james anyway, one of them, uh you know the they look at the interview and they they realize that you know we may be looking for their their help there to land work all right, and I hope that's how it could be helpful as well um yeah.

E

I we should get everything that I mean all of the things we saw today, I think, are now already it sounded like we all liked them. So maybe it's time to slam them all in a review column, all right.

E

Thank you, yeah.

E

I think you know we can ignore draft, but pending there was intro rsc more option selectors. Although washington is not here to talk to it, it might be better to wait until he's available. If we, if it can wait, uh gives us a sense of what's going on, we can also just mention you know: hey folks. We are currently working on rfc-009 emily's done a lot of work in that area.

E

Already, there's already a pr on that. One.

E

And if no one likes this, let me know I just it seemed like a good suggestion from another team. That's similar.

B

Needs I just: how can we get these calm, these things up to date? Should we just um take five minutes after words rash and try to.

E

Yeah, if you think anything's inaccurate, please please do. I think I think it's close to right now, but um I think my biggest concern was for some reason, some didn't show up also by the way david. You see, the ad add cards, one new and the top. That means that person hasn't added it to the project, but it says they've added as a project, but it's satin.

E

Oh, that was me. It's a triage that.

F

E

That doesn't belong there at all. It's just a placeholder, so I'll, take the rfc flag off of that and it won't appear. So that's where we look, though, for new incoming um okay, a new incoming stuff that needs triage, especially from external contributors, because they won't be able to assign anything to it.

F

You say it's inaccurate, but you just mean like the cards in the columns are inaccurate or that you want to change. Okay,.

E

Yeah yeah yeah, I did my best, but there was no guarantee that I scanned it properly.

F

Okay, okay, so it's probably really close.

E

So if you feel like you've got an rfc missing, go, go sort it out all right: do you think it's got the wrong status applied to it, go sort that out.

A

Yeah, thank you raj. This would be really interesting to get everything in a single board.

E

Sorry I wanted to also mention one other thing if you've got an issue and you think it needs an rfc, throw rfc on it and put it into the proposal need column, and then we've got a chance of talking over new rfcs that need to be produced.

A

Great. Thank you right.

A

Okay, uh next, one time to level best practices in oss.

E

Yeah so stephen brought it up, so I thought I might as well throw it on here. You talk about um like the the tanzu open source organization, having some ideas of what's best for maintenance, um uh um you know maintenance versions and and support, and I think that's great.

E

I also know that internally, we have uh ideas about best practices for logging, and I just like things like that. I'd like to push the the vmware tanza organization has a place for these. That's sort of like a best best best, as you can do shared great ideas to use across the open source, org.

G

Yeah, that makes sense even earlier I was talking. I think I I maybe said a little bit too much towards the you know: commercial side right, um you're, totally right that we need or if we have guidelines for conversion web hooks across our open source projects like cartographer and kpac or whatever, that vmware open source. Those guidelines should be. You know, public and somewhere, where we're saying you know for these projects, you know here's a centralized place where we're trying to present.

G

You know things. We've come to consensus on for how you know we we achieve things like that. I will totally bring that up um with some folks to see if we can get something together. That's more centralized for those projects.

E

I mean I can still still be completely open source right, it's still somewhere where our contributors can go over there and say well what about this? All right, because.

G

We have something like this, for you know, on the internal side, it'd be great to have something you know get the same things out. You know publicly for the open source projects.

A

All right yeah that will be awesome. Okay, great um anything else. You would like to bring up here to discuss.

F

Yeah, I guess then I quickly I mean we did a bunch of intros to some rfcs, but we've had some in flight. Is there any discussion we need to have with this group now to to advance any of the ones that are in review or are those already happening.

G

Is it worth looking into the snippets of multipath? There's a big. You know interesting thing right now in joshua you showed a bunch of different options. You know all those things is it worth going through those in more detail? I'm especially curious, like uh you know, zach, I'm not sure. If you've seen some of those designs and it'd be really great to have your feedback. Maybe you know I have some opinions.

G

D

I'm happy to talk through them. I'm happy to talk to him. I didn't want to go through them in too much detail to leave time for everything else, but uh yeah. If we have extra time.

E

I was going to say: could I throw down one one criteria that we're looking for, which is we would like to have a solution for multipaths soon, um and none of these are exclusionary. So um I don't know keep that in mind, for which one we actually choose to implement first and whether it's acceptable.

E

I guess my question is multipath acceptable enough, or would it feel horrible to choose to use it uh choose to use snippets as well and then have these two ways you could do things in the in the long run, because multipath is very easy to implement comparatively or at least as far as I'm aware.

E

We could learn something from investigating all the options that josh is pointing out that there's a quick and easy way to do snippets as well.

G

I I think, like both options, add different kinds of complexity. Right, like multi-path, adds complexity.

G

You know in a way that I'm worried about at the top level right snippets adds complexity in a way that you know can create indirection or create you know too much nesting or maybe makes you repeat stuff right, and so I don't want to. I want to focus the conversation away from at least right now, for what josh is going to present from like making a decision about doing x or y, and just look at all the options get as many eyes on it right like see.

G

If we can kind of generate a broader consensus for which we're going to implement first right, that you know where we think it we're really doing what the right thing for the users is. If that makes sense,.

E

Yeah, that's all right! That's.

F

Second part is really interesting to me, because I'm definitely when I, when I scroll through these one thing I struggle with, is making sure I map it back to like what is the specific outcome, I'm going to be able to do as a user at the end of this. So maybe, when we look at these, someone can sort of help me digest like that piece of that.

G

I think the the really high level goal here is like right now we can say a resource is a or b right. One particular resource in the graph can be. You know conditional can be one resource or another resource or another resource right based on some information, but sometimes we want to say actually I either want to do this one resource, or I want to do these two resources connected to each other right.

G

Does that make sense? Let's stop there.

F

Sorry so we did the no. I I need a little more. I think we got the so we did the selector thing right and we're working on that and that lets me pick one of template a b or c.

G

I'll be really concrete right, like say, say, based on something in the workload, you can either pick a tact on task that does a conoco build or you could pick a kpac image resource right resource one resource. Two right detector would be one one runnable resource, but it's it's. You know a or b but say you wanna do something where you either want to do a dockerfile build using conoco or you want to do a kpec image build and then you want to run a tekton task after the kpac image build right.

G

You either want to take one of two pads, but the paths aren't just one resource or another resource. It's one resource or an alternative tree of resources.

G

Right I see, and then maybe you want to come back to the same point right and there's two proposals for implementing this and they both have different kinds of complexity. I have you know, opinions about. What's what's the better kind of complexity right- and I think you know rash- has some other. You know, I think, very pertinent right opinions about other other kind of complexity and then I'm sure lots of people have different opinions about it. So it'd be great to get your perspective.

D

So if so the example we're going to run through is the same in all the different cases. It's the example taken from rash's multi-path rfc it's in this somewhere. I think it's at the top.

D

Yeah, so this is the example: oh yeah, sorry david. uh This spec produces this graph under what is it that yaml block right? There is what we're going to talk through in all the different use cases.

D

So what's interesting. What's interesting about this use case is that this allows you to take a pre-built image, or this lets you take source code, build an image, optionally, run tests um and then proceed with the rest of your supply chain. I think there's a configuration, a deploy in there or something like that. The interesting bit is right is that it can take a pre-built image or it can build an image in a series of different steps.

D

So I won't get into the details of this rfc, particularly because I think we've covered it a whole bunch, but if you could open up all those gists david that were here, let me send a link.

F

D

Right, yeah, all these ones, so this is the first proposed alternate syntax to the snippets rfc. um Instead of having a cluster supply chain, snippet type, this actually proposes that we give all supply chains a type. So this would introduce a cluster image supply chain, cluster source supply chain, cluster config supply chain and clutch just that it would keep the generic collects to the supply chain as well.

D

And the idea is that each type, each supply chain that has a type inherits the same syntax as a template. So it lets you expose. um So sorry, david. If you don't mind scrolling up a bit.

D

So if we take, for example, um if you could go to file number zero one or something like that uh yeah, so this is the this is a cluster image supply chain.

D

um So if you look at line 10 right, this exposes the same uh syntax as like the cluster image template where you can say my image path that gets promoted along comes from here and if you notice the very first segment of the path is the actual resource name.

D

So it's yeah. So it's grabbing the image path from the build image resource and then it's looking at its status delayed latest image and then.

D

Leading up to this last resource, there's there's a whole bunch of other steps that you can define in here.

D

And then the main idea is that uh sorry, david, if you scroll up to the very top to the first file.

D

Is if you look at line 15, you can drop a type supply chain in place of any other template.

D

Which works because.

G

It has the same.

D

G

And so for for zach, if you haven't seen some of the snippet proposals before the idea is that you still only select one or the other right, but those different options can be entire encapsulated trees and there's a version of this that, where you could inline the supply chain inside of there to be able to see the whole tree, you know, although it might get a little bit nested.

F

Gotcha, okay, so snippets. This is like this is a composability feature, so I can compose supply chains of other supply chains, but that have specifically typed outputs. um So I can keep growing these together.

G

Exactly- and I think another alternative here is instead of having a whole new set of supply chain types and template types, you could overload them so the template could be either resources or an actual. You know template and treat that as an abstraction right and then maybe the whole catalog just becomes you know a mixture of templates and supply chain. Snippets you can reuse right is that.

E

One of your variations josh.

D

Yeah, that is the third one down.

E

D

So maybe we can talk through that one, real, quick.

G

I want to make sure we have time to talk about the multi-path, the rashes, your proposal, also so exactly a sense of a wider alternative. If that makes sense, because I think it's worth all right, you know, I definitely think it's worth talking about.

D

Cool uh yeah, so the the main difference with this uh and the previous one I just spoke through is this proposal actually keeps all of the existing templates, so it doesn't introduce any new supply chain types but david. If you scroll down to the first file that sorry, the second file, the third file, maybe yeah this one right here.

D

um So what this proposal introduces is the idea to add resources into the templates themselves, so uh line number nine.

D

So this lets, you add, multiple steps to a template and not to make this about my opinions. But the reason I don't like this one is that um it changes what a template is. It really blurs the line between a supply chain and a template, because now like to me, it used to be that a template had one like it. Templated a crd and the supply chain described a graph of all the interconnections between those templates um by introducing resources into a template. It really blurs that line of like what's responsible for what.

E

Yeah my argument against that is you're you're already creating something other than a supply chain or a template anyway, and that it's not that's not strong enough. Not not that you're, not right, but it's just like that's not strong enough to me. If you've got a if you're going to have to have this, I think I would feel as a developer. I'd just be like. Well, that's a natural progression.

E

This this object, that is a is a template, can be a template or a sub-supply chain. They both give me this option to, and even better if I can inline a template in any of them, because sometimes I don't want to have to go and create that as the unique thing. Maybe the unique thing is this: like you've got it right here. Maybe the unique thing is this source template. That is actually some other template that we reuse and then this thing that you don't this is where it's going to be.

E

I like this one a lot.

G

I I think, uh along those lines, something I hear from people in the ecosystem is a common anti-pattern is creating too many resources right like where they don't either, where you could use a config map instead or where you're creating a slight different, slightly different version of something that has basically the same purpose right, and so you know, I worry about having all these type templates that behave the same way in a lot of contexts and they create a whole other set of typed things that behave the same way when they're referenced in similar context, right, they're, both they're all consumed the same like a cluster image, template and a cluster and a cluster image supply chain, are all used in the same context.

G

Do you really need to you know different crds to define those things.

D

Scott was just saying to me this one is confusing from a supply chain. Author perspective.

C

Yeah, I think I mean, from my perspective, it kind of just messes up the level of the template in the supply chain and just seems very confusing to understand, from a user's perspective, um just the nesting within a template, which is the idea of it being a single resource, changing that around and turning it into a supply chain of itself.

C

I think that having it as a type of a supply chain object makes it more clear that a supply chain is a chain and a template is a resource.

C

um This is turning a template into a supply chain, which seems a bit odd from the you know, from someone who's riding a supply chain, it seems a bit weird.

F

Yeah, I sort of plus on that I think my gut is that I'd be surprised that a template is sometimes a template, sometimes a supply chain. I think I want like if I, if I were to say like without knowing any of the thing and like there's a lot of good reasons we have typing and stuff. Like my like naive. First thought would be like if I want a sub supply chain, I'll just create a supply chain, and I like that, the like a cluster image supply chain and a cluster build spike.

F

Those start to get weird to me, but it's like. Can I just create another supply chain and plug it in and obviously there's the nuance to like we care about that typed output. But if I really thought about like what felt natural, that would be like my first gut response.

D

Yeah, the other, the other drawback in my mind. To pursuing this option is that templates aren't runnable right like I can't just submit a template to a cluster and have it actually run. So, if I'm the team that just wants to care about the just the build process, if we have type supply chains, then I can just submit that and actually just run it in the cluster and test it out.

B

The only thing, the only thing I get nervous about is for having like these optional outputs on the cluster supply chains is like. We've, we've already made the choice that we care about certain types of output and we've drawn those lines, and now we're saying: oh now we have these optional outputs and it's like okay. Well, we could, we could go back to day one and have optional outputs on just one type of template right.

E

B

Made the choice that we want typed outputs, so that's the only I like the idea of oh. Let me just create a sub supply chain. Basically, but then I feel like we're we're going back on what our original stance was of the typed outputs.

F

I'm definitely not arguing strongly for that or at any point yet I'm just sort of like I would, if you were just to ask me what I wanted and no.

B

I, like it, also zack right.

F

B

Of interesting agreeing with you but.

F

I'm a little hesitant, I'm.

B

Also wondering like.

F

And I also like what I really like is that we're walking through uh a real world example like this is a real supply chain that we know we want to build that people want to build. So like I really dig that I am also trying to think a little bit about like future arbitrary complexity and like how much we care about that um and to that end I'm wondering like it's like, but in the the image I love the link by the way.

F

I don't know if everyone saw this, but the giant pile of link uh that cereal gave us for the graph here, um but um like I'm, trying to imagine a world where um yeah so in sort of the third column over you see, so you can sort of see the trees right.

F

Source tester is relevant when the source is source code, and then I want to build an image so, like that's, a really valuable sub tree right and then there's the other one that is like oh well, source is already an image, so my sub tree is only one long right and, and that makes sense and then maybe there's you know what does it look like if there's a third row here?

F

That also wants to do source testing but doesn't want to. I don't know, I don't know what a real example is, but has some other sort of thing there? Do you see what I'm saying where the branch would sort of like go around um and so, like I said I like us being based on concrete examples, but do any of our approaches. Support like that level of configurability.

E

F

This one does the.

E

F

E

The one show on which is based off of multipath yeah, designed to take.

H

E

Graph, because it is just a graph- not this one here on the screen, but the multi-path one.

F

E

F

E

Too yeah, but this was designed specifically for that.

H

I'm just saying, because I know the designer.

G

You can do you can add any kind of complexity like that into the snippets one by encapsulating. You can have like.

D

Three options.

G

Per you know thing that, oh.

C

Right right, right.

G

That's right so.

C

F

G

Gets verbose, but we should totally let's, let's show I want to show rash multipath example, because I think it solves this without as much rubosity, but it also creates like I. I think it creates a different kind of complexity at the top level, and I'm not saying I'm right, um you know about favoring one or the other, but like I, I definitely want to sarah zach. I want to see what you think about that is someone coming in to this fresh.

E

Can I also just make a request that if someone has spare time to give us an example of some deep complexity and branching, that's still based on reality, because I know that there are discussions if they could give us an explanation of that situation, so that we can actually run them through these rfcs. That would be great. I haven't taken the time out because I've got other things to work on, but I would love to see all of these.

E

F

E

The best of both of these tested out.

F

I think we can do that and I think we can actually do that really easily by just the like um the source here. So here's an example of the source, one. If I look at the source and I dig into its guts and I see a docker file, I would like to just do a docker build. If I don't see a docker file, I would like to do a kpak image build and that would give us that actual third row complexity.

H

That's versus an image input, that's different again! That's where, and this.

E

Is something that's where yeah that's dynamicism and we would like to answer dynamicism with something that's.

F

E

At dealing with it as a so.

F

We don't have to make that a dynamicism, as this is interesting. Obviously you all talked about that and I think I'm assuming I understand what that means, uh but I don't know if you have to introduce that like. I think you can use that use case without introducing it by saying that there's some magic step that knows how to like, we can put a black box on there. That says it's able to determine the difference for you right and then the logic tree stops this. I I think.

G

A problem is that that commit that has the docker file or build pack in it is going to flow through the graph, and so different parts of the graph could be at different states. At the same time now, if you said you're going to mark your workload cr to say this is a docker file app or this is a buildpack app and you're not going to drive it from the source code itself. Then it's fine, then it's a static problem, but when it's a dynamic problem, this whole thing becomes okay, well,.

F

That's different: maybe let's break that down into the second problem of let's try and come up with some good concrete use cases.

E

Or your image builder produces your image builder knows how to see that and produces an image, because the.

B

E

No type of input or output has changed, so the sole for that one is simply. The image builder is better at handling source, turning it into an image one way or another, which is where I'm saying we push it down into resources where they have to be capable of that sort of thing. Otherwise we will never be.

E

We will never be able to be anything other okay, this this is hyperbole, but this is my feeling we'll end up being another pipeline tool and not a choreographer. If we don't stay with determinism.

E

It's possibly hyperbole, but I'd love to have that argument. One day.

G

um Could we move over to talking about the kind of multi-path thing I don't know rash? Do you want to talk about multi-path a little bit give zach an idea? What that looks like or I could do it? I don't want to push on spot.

G

E

Oh yeah david, if you could stop sharing I'll I'll, bring it up wherever I can find it.

E

Because it doesn't have a number.

E

I can't find it.

D

Oh, it's linked in the chat.

E

Cheers I wonder why I can't find it that's curious.

E

E

Yeah, okay, so in this spec it's are you familiar with the current rfc to have um options, be able to select on different fields, um yeah exactly yeah. So the extension point here is that options can also select on inputs so to create this kind of behavior, where um let's start with this simple one here, no actually this one's simpler. Let's start with configure configure, can choose to uh to configure a something: that's an image. That's been provided by the source, the source path or it can choose to provide uh to configure an image.

E

That's been provided straight out of a registry, and so just looking at the name of it configure so configure has two sources, and basically the rule is the first of these that will be selected for so we traverse the graph. We look at it and we say um whatever selectors bunch up, so this one will bunch up, because this actually looks for a field that says uh um spec.get versus spec.source.get spec.source.url.

E

So it will definitely select on this side and not on this side, or this one says I will select from spec.image and so we'll choose the path that provides an input that will provide an input. We don't necessarily stamp them out yet, but we can deterministically work out which one is going to be selected for, and so we and and this should be an error, but we have a problem where it can't necessarily be an error if both of them would select.

E

uh But what we do is we select for the first one that occurs because the scenario where it could be an error is this situation here, where the image builder selects on the input either from the source, tester or the source provider. This one's always provided right, because without this you can't get a source tester. So this output is always going to be provided.

E

So this one can't select on it can't error that it gets both of them. We simply have to choose to pick the first one that selects, which would be the source tester, and so in the case of the image builder, we say from the source tester or from the source provider, and this graph is literally generated from that spec.

E

So if you wanted to add another configure step um based on some other input that you've set up a partial chain for you just draw a line from the configure to that and say that's also a possible input. That's the designers that made sense.

F

I think it does um and I think it's I think it's interesting. I think, like the thing I'm trying to like some of this subtree stuff to me feels like the right word, but it feels like we're: building mini pipelines and sticking them together right and I sort of think like we talk about choreographer as this like in my head. The way I think of it is it's sort of this like pub sub system based on kate's statuses, and it's it's apis right, and so it's sort of like.

F

Can I just sub to a stat like? Why do I have to care whether I'm associated to some special subtree or some special thing uh in terms of like a subcluster or whatever, and why? Why aren't I just subscribing to the output of some some source, and when I see it, I do my job right and we sort of get that that choreographed thing, and this feels closer to that spirit, not not to say that that's our guiding light or anything.

F

I just like when I think about what I've heard choreographer talked about it, cortis sort of stands closer to that. So I think that's at least interesting.

G

I mean this: it lets you drive the decisions based on the inputs right so like you can have conditionals at one level. That say this is a resource or this is another resource, um but then later down in the graph right, you can have two options that don't have a conditional that are driven by whether one input higher up in the graph or another was selected, and then from that you can have more.

G

You know like things that are driven by decisions further up the graph and I think it makes for it's like a very clean, like you, don't have to have a lot of nesting. You don't you know, um you know it doesn't encapsulate complexity, but it also brings that complete. It makes the graph many graphs at the same time, and I worry that that's not very concrete right, sorry, rash.

E

No, no! No! It's good. I just the the one thing that I was thinking when I tried this approach was simply: how can I make sure that all I'm really the only approach I'm taking is? How can I turn conditionals into declarations right? That's the thing that was in my head, which is like like?

E

uh Can you prefer polymorphism, basically right the pattern right, so the pattern in my head was simply: how can we make the like the imperative if this, then that something that is declarative and what was important and valuable to me was that at the configure, for example, or at the image builder, I was just saying what matters to me is that you've got me a source that I can configure like an image that I can configure.

E

Where have I got inputs for that are here and here so most of the time sorry I'll go back to sharing most of the time. uh So in the case of like configure here and even an image builder here, you're always expecting the same type right and the decision you're making is. Where can I possibly get that from? And I don't have to think beyond that, hopefully right, um but there is another choice you make, which is well this one's the field's choice. Sorry, this is not the inputs choice. That's right!

E

I was going to argue for for the rsc already accepted, because I I parallel invented that so that I wasn't I just. I came up with all the things I wanted and it was a parallel invention of what oshuma had already suggested so yeah. I don't need to argue for for why? That's the other thing that you can do with options because we already implemented options. I came up with options because it solved both of those problems and now it seems to me it's consistent to keep solving these problems using options.

G

I know we're out of time. I think one thing I keep coming back to is like I can imagine an interface for dragging and dropping boxes and connecting them together. That lets you encapsulate supply chains into boxes, but I have a hard time imagining that interface, and I know you have a good graph right but like as an end user right, I'm going to like create multiple options at a point, but I they're not associated with the conditional they're associated with multiple paths that could happen up to the graph right.

G

It seems like it'd, be hard right that, like I keep coming back to like uh you know, it's really clean like when you do a nesting example. It gets pretty far out right, but you know I'm having a hard time getting past that, like I, don't know how a user gets. One view of it what's going to happen right um like like, can cleanly understand right the possible options right if all the graphs are laid on top of each other, they have to somehow configure those when they're dragging and dropping those boxes together.

G

Right thinking about, like you, how we could build in the future.

E

Yeah and my argument is that you, you don't.

D

Know how to talk about an.

E

Edge, sorry, go, go josh.

D

Yeah, I would I don't know how it would work with uh drag and drop, but multipath definitely lends itself to auto complete, really well right, and so, when you're actually authoring the resource right, you're sitting there figuring out like what input I want. um It's it'd be super easy to do an auto-complete of like you, have two options pick one and to me: that's a pretty good user experience.

H

G

Got a live editor to show you that if you want it, but would somebody really understand like if they're looking, even in that graph, you show there right like it? It's not it's not clear to me if it's a fan in fan out or if there's optional things, it's like, you have a conditional further in the graph that can drive multiple options that are overlaid on top of the same graph. That can drive multiple options that are overlaid on top of the same graph right like.

E

G

E

I understand what you're talking about, but I thought about these things. As I went. One of the things you can't see here is that I modified this into something that supports field selectors right.

E

So you can't see that at the moment, because I wasn't showing field selectors here, I'm only visualizing label selectors, but there are field selectors here and what I think I could show you very quickly is that as I hover over one of these, I can show you the multiple paths that they could land on and for each of those paths highlight the selectors that were required to get them. And then, if I pop a workload down I we can do this with all of them.

E

We can do this with all of them, but- and I think that's what we should do is we should visualize editing for people, which is why I'm working hard on making an editor, because I don't think it.

E

I again that's why I would like to see the more complicated example because I feel like I would happily do mock-ups or even a live editor now that I've got the live, editor working that showed you that, if you dropped in a workload, this is what would select and you can know by highlighting it makes sense I gotta drop. Another call. This is great.

G

A

Okay, thank you all for joining, see you next week and hope we can keep conversation. I think thank you.