VMware Cartographer Office Hours, 29 Nov 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cartographer Office Hours #2 - Nov 29th, 2021

Description

00:00 Overrridable vs default params
03:13 RFC 014 - what information is passed on user notifications
13:43 RFC 010 and RFC 015 move to Accepted
15:20 Open questions from RFC 016

The purpose of this meeting is to discuss architecture-changing ideas (in the form of RFCs) and provide in-depth support to the community of Cartographer contributors.
You can continue the conversation by adding comments to the RFCs or creating a new one:
https://github.com/vmware-tanzu/cartographer/pulls?q=is%3Apr+is%3Aopen+rfc

Agenda: https://docs.google.com/document/d/1ImIh7qBrOLOvGMCzY6AURhE-a68IE9_EbCf0g5s18vc/edit?usp=sharing
#opensource #softwaresupplychain

A

A

There you go. Thank you.

B

um Okay, so yeah, just to recap right from rfc 10. um We had talked about these two different approaches. One was the idea of having a value with an overrideable, true false flag in the supply chain. The other approach was having a default value, semantic that got applied to all the different places where params are used, where default means that you can override something value means that you can't override it anymore.

B

So what we did is we originally implemented it with overrideable flag per the initial recommendation, but then we found prior art um in the pod spec template uh that comment right there, the one that david's sharing right now, and so we submitted a new pr that added this behavior um and it got merged. So we went ahead with default value.

C

B

C

uh Like an example of what that looks like somewhere, like I don't know from.

B

C

I found 387, but I I don't see like something I can look at to understand what the interface is.

D

C

An example somewhere in there, so it looks like it's name and default.

C

In the supply chain, and but the supply chain is the only place with default- is that right or is default? Also the thing in the template, so template says default.

B

A supply chain has default and value, and the workload only has value.

C

Interesting is there a reason template has it's like defaults are always overrideable and values are never overrideable. Is that the idea yeah exactly.

E

And template has had default since it's like the original cartographer setup, it's from like issue nine and continue from way back when.

C

That makes sense, I feel like it's a little undecided about if I like template having default, but I don't think I have a strong enough opinion to say either way. So it sounds good. I just wanted to understand where, where I went, I can update the kind of large internal dock also to cover that awesome.

A

Great okay, um yeah. Let me add a note here.

A

If there's not anything else, discuss.

A

That will be it for the week.

C

uh Actually one one quick thing on from last week, I'm just looking through stuff that was talked about on rfc 14, which only suggested adding information about the resource that it passed through. um Is there a reason we need that versus just the name of the like there's a separate name that identifies the component that it links to? Is there a reason to include both the oh, the resource information and the name of the component? Are you worried like what happens in the supply chain changes or something like that.

E

It's less, it says when the supply chain changes. uh I think in the in that note, I had pointed to an issue um where essentially the the reasoning from that issue. I think applies equally here uh and josh can speak to that. I believe that that 378.

E

um uh one, I know one motivation, and maybe the only one is that uh workload authors I know the you know they may have access to the objects, but they don't always have access to the supply chain, and so when they see uh xyz xy.

E

This was about uh you know the messages. Oh we're waiting on something and if they didn't have the supply chain there, the component name wouldn't be very useful to them to understand. What's going on. uh Similarly, uh if we're, if we're saying like here's, this artifact- and it comes from this component name, but they can't look at the supply chain, then they don't really know what what object that comes off of.

C

I'm curious about it's like the. The proposed association creates a kind api version name and namespace association with the artifact or with the reference to the other artifact right. So it's like it's not associated with the artifact itself. It's just when an artifact comes from another artifact. You you say came from that artifact through this other resource. Is that right, but then we still have this past list that describes the stages that have been through. Do we need the past list anymore if we are keeping track of this?

C

How what resource the other thing like should? Does this conflict, or is this duplicate information with past? Do we need to reconsider that interface.

B

I think I think the problem is that past reference is a concept in the supply chain right, whereas if you're looking at the workload, you're, probably also looking you're you're associated with resources within your namespace right, and so we need like.

B

So we need to make the connection between the underlying item that would have been in that past block. So, let's like we're talking about if the supply chain tax criteria is like an image builder right, we need to actually make a reference to the underlying resource.

E

Yeah- and I think the this muddies uh like whether this information should be in from along with that id, I think, is a I'm not sure that it should, like those. Those are really different concepts, um but I yeah there should be some uh some term that allows you to specify like here's. The original object that this was read off of.

C

So, like you have source code and it passes through a few stages unchanged right does that source code say it came from source code from itself from itself from itself with different references, or should all this really be in the past list. The past list says this resource passed through sorry. This artifact passed through this these resources, I'm pretty sure it should be the latter right, and so, unless you disagree, we could flip the model so that it, you know it creates chain of links to itself.

C

But I think, because we're trying to form an artifact like say artifacts turn into other artifacts right. It might make more sense to rep to represent the artifacts, the top level and then use pass to represent the resources that they pass through. So like moving the information that you proposed from there into past instead and then having separate resource name versus component name in there might be one way to solve it.

E

Yeah, I I don't have any objections to that. um I wonder if there should be some- and this may be tangential or orthogonal uh some way to note like the resource that it wasn't just passed from, but that it originated from and maybe if passed is just an ordered list, then we're like yeah, it's the first one in the list.

C

The resource that the artifact originated from not the artifact, the artifact, originated from that's what it meant right. Yes, so the first one in past should be the thing that generated it. Yeah yeah, I totally agree just making sure on the same page cool. I can leave a comment after your comment on that to move it around. I just just wanted to make sure we got the nitty-gritty right, cool.

A

B

Kate's events have they have an owner reference right. The event object, it's just like any other crd, so I'm assuming it has an owner reference.

B

Which is like actually, multiple owner references, you can have multiple right.

D

C

D

Objects like you can link one to the other, I'm not sure if it has an owner reference and like in terms of gc. You know.

B

Right sure involved objects, yeah, which is probably more accurate in this case anyway, I'm just wondering if we should try and model these relationships as events instead of having them in status.

D

One thing about events is that like they might be collected at any point and I think there's some aggregation that can happen as well behind the scenes, but I'm super into considering it. I think it makes sense.

D

I don't know enough about it.

B

Even if, like maybe all the historic, like all the historical artifacts, get captured in events, and then we have a concept of like where the like, I don't know, the most recent, like the current accurate representation as per the workflow spec, um can be.

C

Kept in the status, we don't want historical things to be kept as events, because how many do you keep an scd right? It's the like. What's the right amount to keep on the cluster? In that case, I figured the historical relationships should go into some kind of more permanent storage, but it's like could you represent?

C

Maybe could you represent the artifactry and status, but then represent the passing of artifacts through through the resources as events instead and then have something that goes back goes between those two things, but at that point, what's the use of having separate event objects.

D

I think we should have events anyway, like as a way of providing a nice experience for folks to be able to tell like. Oh this thing happened. That thing happened agree. It has a storage, medium yeah, I'm not sure.

B

But it would, in theory, be representing kind of the same information that we're trying to represent its status here right. It's like an artifact did something.

D

Yeah, like I think the semantics uh like makes sense to make the the relationship between events and what we're trying to do, but I think the way they're treated and uh uh is expected to be used in kubernetes. I think there might be a mismatch there, uh but again, I'm not super familiar with events. Really, I think it's worth putting a note, though,.

C

I think it's going to be helpful to have at least the current state of the world tracked in one participle place that we can easily return back to the user and show in a a graph right without having to parse through events. But I I agree. I like the idea of and that's being made available when things happen, to improve the user experience.

C

Our goal is some prior art for, like keeping track of you know this type of information right.

C

B

I I think they they overflow to a different storage mechanism too right. I believe, I believe.

C

Do that or yeah? We could do that, though right you, you can even set the registry as a backend right for uh the the artifact tree and then, as artifacts drop off they're no longer part of the tree. They you know stay in the registry based storage and then you don't need any external other kinds of storage and your cli can you know parse understand how to go from the crd to the registry that it's writing to and automatically give you historical information. Then you know magically on vk in any place.

C

You did deploy cartographer, you get infinite history, but I haven't sold everybody up me. Everybody should use the registry for every idea.

C

Blob storage may not be the right thing for this too.

A

Great okay, thank you. um Anything else got an rc, 14 or any other.

E

Yeah, so uh we I just kind of dropped in rfc 10 we've got an implementation, that's merged. To main. I assume that we should go ahead and finalize that rfc with the current default value. That can be some of the documentation on how that and how that works, and then I think that that's accepted we can mark it accepted. We can drop it into an accepted folder.

C

Sounds good to me.

B

Rfc 15 as well right same thing.

E

um And then rfc 16 um had an action item to update it with some concrete examples which I've done.

E

There are some open questions there. I don't know what.

E

Maybe I'll just have those been answered, shall we poke him again next week.

C

Is there a link to rfc 16 issues.

C

Oh there, it is okay.

C

Are there comments here? I just see a draft with nothing on it.

E

I think all the comments came just in our meeting last week. I.

A

E

So there, so there are a couple of open questions in the in the notes. um What does validation mean for config, which is a large block? Vml? Do we need another template type for config known to be case objects, so I I have preceded my understanding has always been that cluster config templates are meant to hold a a case, object and I've written it like that currently. But I think that we, you know we had some questions, some open questions at the last meeting.

E

Are they being used in other ways right now and do you expect them to be used in other ways and that uh I think that in some ways that's a little bit of a validation question? But it's also just a what template types do we need question.

F

Okay, I I threw that spanner in the works from what I'd heard in the past.

C

Pod templates back they're used to like convey chunks of information about that would go into a deployment or another resource. Pretty often in examples we have so I don't think a config is a cat's resource. I think it's. It could be many kids resources. It could be a snippet of the inside of a kid's resource, but I'd say it's probably valid, saying that it's valid yaml, it's probably good enough, and that also buys you json as a subset of yaml, which is nice, um and so maybe just validating it with a yaml.

C

Parser would be a good first step.

F

So we never have to worry about strings people trying to do string information.

C

It feels weird that I don't think if people are using config and they're just passing a string through and then interpolating it somewhere else, like maybe maybe they're they're thinking that it's going to be ammo. That ends up there and it's not right so yeah, maybe better to be strict.

F

It's just yeah. It was just that because I was coming out. I was coming at that particular validation from having heard people say. Well, it's just any old string right. I think they meant it's just any old string. That represents a bunch of documents all right. It can be one or more that's what I think they meant, but because they said it could be in the old string. I wanted to double check that yeah.

C

Document or subset of document right, yeah.

E

Here's a question: isn't a string valid yaml? What what's not valid now.

F

With yaml has has to be key key valued it a string won't pass in the yaml person.

E

So I'm fine with this saying that it has to be key value pairs. um I don't know how how how canonical yq is, but if you echo high to yq it'll it'll be like yeah. This is a string hi.

F

It's not canonical.

F

That's developed snippet of yammer.

C

I guess when we say we want to allow us part of a resource right, does that part of a resource include a string field of a like a value? That's a string field of a key right, and then I think the answer is probably no. It's got to be.

C

Value pairs, yeah, yeah, okay,.

F

Object: yeah yeah, yeah.

D

Three of objects.

F

Or comments or.

C

It has to be a valid yml document by itself, but it doesn't have to be a valid case resource. Maybe it's a way to put it.

F

And it can be multi-yaml docs, right, yeah.

C

For sure uh multi-document it could be multiple resources, I guess yeah sure it should be able to be what happens if you yeah? No, I think that's right. It should be able to be multiple documents because.

F

I'm pretty sure that's one of the things that the intents do doing, they're coming up with a bunch of documents. You might want to apply right right, yeah, hopefully.

C

What about images should we be validating the images or really it only digests, because you shouldn't be promoting a tag really, should we translate tags into digest as soon as we see them,.

E

uh I would not.

D

I would need to have the credentials right if, if it's going from private.

C

I don't like the translating I agree, but I think I think maybe some that came. That was an idea somebody had early on. I wonder if just enforcing that they are digest, though, and not tags would be a good first step, so you can't accidentally promote a tag that gets overwritten.

E

Right now it says digest it does not have a call out for tags. I'm happy to stick one in there.

E

E

Okay, and presumably with those changes, rfc 16, is ready to go.

C

Sounds good to me all right.

C

um You might want to upgrade it from a draft before I get approval.

C

Or I can't hit the approve button to signify my approval until you move it from a draft into the. I think.

A

It sounds cool yeah. Those are all the helping questions from last meeting um and yeah. Thank you for adding the action items there. So um anything else.

A

Nope, okay, so thank you so much for your time see you next week.