Add a meeting Rate this page

A

Get all the gadgets in place from the weekend good day, folks.

A

There we go. It's always interesting is everything on monday morning. So thank you.

A

Justin, unless I just saw your uh pr feedback that was awesome. Thank you.

A

And let's see.

A

I'm hesitant to paste the link into zoom until we get a couple more people, because I think we've said that if, when you join, you don't see the previous posts in chat. Is that true.

B

Yeah, I think it is.

A

All right all right well paste it twice. Then so people can jump.

A

In while we're waiting, if peop, if peeps could sign yourselves in that'd, be.

A

Great.

A

And we'll give folks another minute or.

A

Two.

A

We'll see how time goes, but um she just did the pr on the feedback for the jwt format, which I've just been digesting myself this morning,.

A

One of the interesting things they'll come up in the distribution api proposal because it actually has an influence on it, especially after reading the quay. Folks post-mortem was an interesting conversation.

A

And I think it was in every request: they were doing uh a tar, uh decompress or decompression. Would you call it a.

A

Detour.

A

And let me actually just ping the zoom to see if he's going to be able to make.

A

It.

A

Okay, so why don't we start with the first one?

A

I can manage my windows here.

A

All.

A

Right.

A

Okay museum says he's running late, so we'll go there all right. I'm gonna check this time, make sure everybody can see my screen before I.

A

Start said yes, thumbs up all right cool all right, so the first one is really just some cleanup. uh Well, it's two things. Obviously it's a cleanup uh moving things to the root readme, because the more people that were trying to get up to speed on what we're doing with notre dame tracking what we're doing with notary v2. I realized we started slipping into the tribal knowledge thing of a bunch of stuff, that's assumed, but not actually clearly documented outside of our notes.

A

So what you'll see is there's actually two changes here. Let me see if I figure um one was uh we had this format that was kind of nice for a powerpoint that justin I used for the um uh kubecon thing, which is, I think, next week or two weeks, not sure is it next week or two weeks I.

B

Think it is next week. Yes,.

A

Okay, I.

B

Was confused? I thought it was two weeks and then someone said next week. I was.

A

Like we do have to be online, so it'll be good to get feedback on it. So.

C

Anyway,.

A

There's one we had this, which was an interesting hierarchical one, it kind of grouped it, but I struggled a little bit with it.

A

So I tried a more bulleted approach um so and the reason I kind of pushed on this one is you'll see uh in the other pr, when I was trying to talk about the various options for distribution apis, I'm trying to refer to requirements as to why we took a certain approach, and I realized we just didn't- have these written down in a way that really made sense and ralph also committed um before he left on vacation for most of the month, which is great. He should do that.

A

Don't get me wrong uh that we were gonna talk about the phases of which we're doing these. So I left these here as a starting point. uh So by all means, let's uh feedback is wanted and so forth um any comments here before I go to the next one.

A

So this is two things. One is just to move a movement from what was in the scenarios document up to the root readme and then it's just a specific call out. Without going into deep deep details for each one of.

A

These.

A

There's a phone number clicking.

A

Okay, so with that the next item- let's see, how do I do this? The next item on the list was the requirements um we separated out scenarios from requirements and, honestly, I'm struggling with this one a little bit, because I, like the high level requirements that we're calling out some specifics.

A

But again we wanted to defer details into things like the key management scenarios, which actually has that part as well.

A

um The two things is that I kind of thought well, there's three, because I realized when we started talking about formal clients, so one we wanted to make sure that we didn't embed key management itself, the underlying keys into their registry there's a little bit of a conversation has been uh starting to tee up on.

A

Can you get the key through the registry apis, but it isn't necessarily the place they're stored because we want companies are going to be comfortable with their key management solution, whether key vault and azure, or aws's, p-vault or hashicorp vault, if they're using on-prem or something else.

A

So all of those are, uh I don't know the person's got the phone number um your mic, please I'm sorry.

D

Thank you uh I I don't know, so I don't know that this is written in a way that um I don't know what to make of the way this is written because, like when, I think about something like a requirement requirement is, um is more actionable, and here um I feel like this is more saying we're going to punt this part of the problem space which is uh punt but interface with, and I I don't know that, um like sort of from a building a system standpoint, I think that totally makes sense, but from a security standpoint you all of a sudden run into all the corner cases and limitations of whatever different key management system you have, along with the difficulties of making it integrate and play with what you're you're designing and so um like.

D

I think the fact you didn't use the word has to securely do any of this um is is somewhat telling, because I think doing this securely is not this. This isn't just like a box that you that you plug in and everything sort of works.

D

um So I would, I would argue that um talking about this with respect to the security outcomes you want is, is really key, because they're not just using these boxes, just to use these boxes they're using them, because they want a security outcome from them.

A

I I try to because there's two parts: one remember this is we're not writing the spec. Yet this is the iterative phase to make sure we're pulling out the right pieces.

A

So I'm trying uh what we're saying here is the clouds and companies have key management, pure key management solutions they're already using and those are, I would position secure because that's their purpose is to be a key management in the end and workflow.

A

All we're saying is we want to integrate with whatever the key management's, the actual key part management solution, the discovery and how those keys are fetched and retrieved is something that I'm again deferring to the key management group. All I'm saying here is and and maybe cormac if you could help me here, because I'm trying to relate to what we talked about in the weight, docker distribution or docker notary v1 was done.

A

Is we want to make sure it's not embedded into the registry or even the the notary service, because there was no good integration with you know the key vote that a company is trying to manage all their keys with.

E

I think there's two requirements being uh joined into one here. One is the support for the vendor cloud. Key management solutions, which is the second sentence here, which says notary v2, must provide extensibility enabling key management by any one of many solutions, and then the second requirement is that registries should not be required to play a role in uh signature, verification or any any any part in that process, which is what allows us to move containers across multiple registries right.

E

I think those are two separate requirements, and if we separate those two, uh then at least we have those goals to shoot for.

B

I I I just think that these things just seem to me a bit weird because having a separate.

B

Why are these not in goals or scenarios? Why are these called out separately.

A

Yeah, it's fair. I was struggling with a couple of things that I felt were core requirements that the scenarios support um and again. This is why it's a pr for discussion, but I really until I did this it was. We didn't actually have these things specifically called out anywhere. Immutable tags does reflect in the scenarios, but we want, but I was trying to call it a more deliberate thing, including the key management solution. So is the scenarios support the requirements and the requirements support the scenarios?

A

I'm not suggesting. This is the final way to represent them. It was just getting something written down to have exactly this conversation and I'm up for any proposal on alternatives. I.

B

Mean I think, it's a kind of weirdly ad hoc list of things that.

B

That, maybe I mean maybe.

B

um

B

That I don't think fit in one document. I think maybe that maybe you should open them individually as prs against the places where they make most sense. If you think.

A

They're missing, as you feel or in the key management scenarios,.

B

Well, I think they might be in different places for different things. I just I I found it an odd set of things to have a separate dock for because, especially because some especially all added at once, there's a list of things because they're very they're, just a random set of.

B

It's a it's a it's a set of things that are not particularly related to each other and don't and they're not complete.

B

So it's just it makes it for really weird dark on its own yeah.

B

So.

A

As.

C

It says.

B

It's a set of requirements, it's not a a complete, it's neither a complete set nor a um so just it's just weird to me as a separate document.

A

So when I, when we write other design, approaches like, for instance, when we get to the distribution apis, I was trying to refer to something that was not necessarily just a scenario but kind of some core requirements. That we've said was difference between v1 and v2. um So I can certainly merge this into the scenarios. Doc and just say you know, in support of scenario 1.2 or whatever we you know call it. um I guess I was trying.

A

I thought that there was some of them that were just more core one-liners and I guess that's the cons. I guess that's the feedback, because it really isn't that simple.

D

No because I think the way you want like you, want a security outcome for these, and you want actions that people can take and so on. I think some of this is covered, and some of this isn't- and some of it isn't really clear some of it you're describing like a feature rather than sort of like a way in which somebody uses something like a like a workflow scenario, justification- and I think that's that's also part of what I'm trying to push back on here.

F

I'm not sure I follow the workflow.

A

Part like they're.

D

So in in general, you know the way that the way I think you you know the scenarios and things are set up is- is that this is like. You have somebody whose goal it is to transmit.

D

You know a container securely, despite the fact that the registry might have been compromised and have another party be able to verify it and that other party may have no state initially.

D

So you have a party that has no state or very old stale state and has to be able to pull things images through a repository that may have been compromised, and so that's sort of a scenario um like basically enable you know describing an outcome you want to have, and then um you know, but if you look in three, for instance, the second sentence on the initial poll when the client is discovered to need initialization a secured process is initiated like I don't. I don't really know what that means.

D

I think it's very vague- um and I also don't know like I- I'm not trying to say that um it's wrong, but if someone came and had a way of doing this, that didn't have a secure process of doing this.

D

At the time of the you know on the initial pull, it isn't necessarily that that solution is wrong or doesn't work doesn't meet the actual need of the system, but the goal is, is to have that transmission and have things happen securely, despite a potentially hacked registry, and whether this happened on the initial poll or whether some other way of this happening happened. You know like something else occurred to make. That feasible is, I think, irrelevant to what your, what your like, the way in which you would judge as a security solution.

A

I mean part of it is they are intentionally vague in some ways, because I wanted to stir the conversation, get the experts in the areas to come up with. You know what should the specs say so to speak, like, for instance, the thermal clients came out of this conversation around the verification concepts of the update semantics we've been talking about because we're saying that these clients are ephemeral. So how do we support that? It's not implying that.

B

That is a security requirement that this must be secure for ephemeral, clients, as well as non-ephemeral clients.

A

Right so would you say that put it into the scenarios then yeah, one of the seniors we talked about hey a client is spun up. It's brand new client because it's given as a whatever you know,.

F

Yeah.

D

Exactly.

A

Okay,.

D

You know it can have state, that's old, which is you know, because you you're likely to have to tell it something for initialization. But it's not expected to have up-to-date.

D

It's not expected to have information, that's let's say less than something like a month or a year old or something somewhere in that range, something that that is that totally makes sense, um but but then, like part of the yeah. But what I'm pushing back is. Is that you don't you want to be careful not to add design in to requirements or to scenarios.

A

So.

B

I think the question would be which ones.

A

Did you want to cover? Maybe that's what you're saying I just saw you light up. Is this one which ones you want to cover as part of the key management scenarios as opposed to the? How will we separate the name of the other scenarios.

E

um I think there are some of these like the revocation and key management that goes into that, but I, I would actually also kind of say that, um rather than calling these requirements calling these goals, because we did agree on some changes that we wanted to make and address as part of notary v2 right uh like enabling key management across uh different solutions like hsn's and cloud providers, we took that as a goal when we started this project, so there were a set of goals that we agreed on, that notary v2 would address, and I and I think that documenting them makes sense.

E

um I think I'd agree that this is probably uh by saying must provide extensibility. um I think we may be going a little too strong on the language here, but at least the goals and objectives that we addressed. I think documenting them. uh Whenever I've been working through some of the scenarios and use cases for key management, um we have had to come back and say these are the things that we want to address right, and so I don't think like I don't think, there's harm in covering them as a goal.

E

As long as we we understand what the goal is and why we're headed for that.

A

So I think what I'm, what I'm hearing is goals, so we should you know iterate on these. You know we'll just provide feedback if we've captured the ones in a distilled format, then we have the scenarios that uh somehow I miss all right I'll have to check this.

A

This is the roommate he's supposed to have the scenarios listed here uh which are basically here, um but the question that I have is: how do we reconcile so we, let's get rid of this, or let's take this as teasers for making sure that they're in the goals or scenarios, the thing that I'm trying to figure out is we we said we wanted to have a separate set of things called things for a second for key management.

A

um Do those just should those fold into a combination of goals in the scenarios, or should there be a separate key management scenarios, or are we just doing a separate key management scenario, doc for the sake of a pr feedback and we'll merge them in here.

E

um I think that's a question for organizations like if we want to eventually merge them in. I think it makes sense right now. I think keeping the docs separate makes it easy to iterate on those and cleaner pull requests, um um and I can't also see why we can't link one to the other as well right if we just kind of call out in this overall sizing scenario, where sort of like the uh key management scenarios or documents at least we're not losing track, and the new reader at least knows to go.

E

Read this other additional scenario. Talk.

A

Right, okay, so what I'm hearing is well sorry, I should let others add feedback.

B

Yeah, I mean, I think, conceptually they're part of the same doc, but it's fine and everyone always raises on them. So probably as well says.

A

So effectively we have goals, spelled properly in notepad scenarios and temporarily, let me say, temp uh key management as a separate doc to be merged with the scenarios as they.

A

Complete their pr.

A

No separate requirements.

A

As reminders- and I actually probably just paste this in the uh hack md, we said well in a second using some reminders for scenarios and goals.

A

Is that what I'm.

E

Hearing.

E

I think that works for me. Okay,.

A

All right, good!

A

um Okay, so I didn't really hear that these that we're not we don't want these, it's just a and it was intentional to should we put some requirements, but let's make sure we get these put in the right place because they were not some of them weren't captured in in its detail, and I wanted to make sure we brought those in um the next. What was this? Oh, this was the.

A

This is the thing I just popped. Oh no, this is the distribution thing. Was that the next one? Actually, let me do this jwt one. um uh No, I'm gonna! Do it the way I put it in the list? Actually, um okay, so the next one on the list, as we talked about a couple weeks ago, is now that we're getting some structure around what a signature object could look like um and we're iterating, including the gwt format, as opposed to a json formatting which we'll come to in a moment.

A

We've been wanted to write down. How are we thinking about persisting it into a registry um and then, if, once it's persisted in a registry? Well, how does it get in and how does it get out? You know the push and the pull and what is the discovery of a collection of signatures?

A

uh Api look like, and it's teased out a couple of interesting conversation, points that I wanted to surface.

A

So, as you see, what I was trying to do was here was highlight some of the requirements um in in here, and I pulled them out and I'll. You know we'll fix this, to be able to link them back to a specific scenario and so forth, but basically the original digest and the collection of associated tags shouldn't change.

A

This way, a cust, a user that has deployed something or is referencing something as part of their deployment scripts that doesn't have to change just to be able to get a signature verification we want to say the signature. Verification verifies the thing that I'm referencing multiple signatures per artifact. We did push a better picture of this.

A

uh Where did I push that hold on and get a uh distribution? No, not distribution, second might have been the right one requirements. So in the scenarios this now has the updated picture that talks about this walkthrough. I updated. The text also refer to this, so the the wabit networks builds. This network monitor image and their registry doesn't matter where they have them. They've got a registry whether it's public or not, is irrelevant.

A

They push it to something like docker, hub or docker up where the wabit network signature is available there. Somebody may or may not trust it yet, but docker can say this is certified content and they trust docker. So now, there's a second key on it. This reload right here says the distrib, the and um digest can't change, because we're just adding another signature as it gets pulled into the acme rockets environment, they can put their own signature as they verify. This content works for their environment and then their policy management says.

A

I only trust things that the acme rocket team has certified and it's often deployed in life is good. So, um and this has been, you know, updated with the text to refer to that. We've been talking about this for a while. I just I hadn't gone back and updated the picture in the text to reflect the the feedback that we had previously.

A

So where was I um sorry? So that's the multiple signatures that we've been talking about uh native persistence. This is the I don't know what the star is doing there we'll fix that um that we don't want to have a side car another service sitting alongside the registry, because things get out of sync very quickly, depending on where you pull the tag from. uh We want to make sure that we've got that uh included. Let me just pop this out.

A

uh Oh okay, great these are recorded, so um that's helping with people in different time zones and different schedules. uh So that's the native distribution native persistence. We wanted to support moving right. We talked about and multi-tenant registries making sure that um sites like docker hub, that has multiple organizations or uh the github and google model where it's a single domain, but they have multiple uh customers in that same domain or the aws and azure version where it's different domain uh for each cut each registry.

A

But within that we have different groups that need to be able to support. There's lots of ways people hash out how they do multi-tenancy. We want to make sure that we can support the aggregation of that content uh private registries, so they move from public to private as the picture showed um and then, of course, within those private registries, new content is created and, of course, the air gap environments.

A

So to support this um we've been discussing, and we haven't really had a chance to review it in detail that we want to use the oci artifacts approach, because it allows us to leverage all of the infrastructure that a registry already supports everything from garbage collection to object, listing tag, repo delete management and so forth. So we're trying to not have yet another end-to-end model for dealing with something it's. It is a special type of object, artifact, but we're hoping we can make it just one of those.

A

The today in ocr effects we deal with manifests. We say that, based on the media type, we can determine that it is a a particular artifact type. So in this case it's cncf notary v2. Let me see if I have an example of that in here.

A

I think I do down below.

A

All right, not not in this one but basically use the artifact. Oh here we go uh notary, v2, config and we'll see how this will switch over to a jwt and the other pr and then inside of it it references the digest, but this is in the config object. uh The question is: should we be parsing? Well sorry, this is what would work today, uh because we can stick the signature in the config object uh and the config and the artifact itself can be defined as a signature.

A

The pros and cons I've tried to call out on this is that the pros with this is it works with artifacts. Today, um wait! Wait! Oh here we go.

A

So we can and we can use the rs client, whether it's a library or not. uh The problem is that you would need to actually add a new. I'm sorry, I'm trying to read my own notes. uh I guess I I finished this friday night and I was debating and presenting this, and I realized- I don't have it all done here. So the problem with this approach is you actually have to parse the config to figure out the linkage? Oh, I know why there's a separate conversation of how we do linkage.

A

That's why I left that out. So the at this point there's a reference. The other approaches in an index an index already has the ability to reference other manifest, including manifest lists, and that's what registries are the ones that are actively adding this. In fact, we'll have a conversation on wednesday around the use of index, um but the design is there that it index already knows how to reference other manifests, and that was the original proposal that I had.

A

The problem is that it also requires the index to support a config object, which we said we wanted to do as part of the artifacts conversation, but we haven't actually gotten to it yet so this and cnab- and I think, there's one other- might be the impetus to you know kind of say: maybe it's time to go, do that, but that would actually be yet another change. We'd have to do the so then we get into how to link them. This is where I separated out so in the current current.

A

Whatever current means in the json version of the signature object, um there's a digest. That's already in here. In fact, we've made this a full descriptor by putting the media type in here as well of what it is that you're signing the digest of this is separate information for verification, but this digest right here for that same repo, says hey. I can link this but again. You'd have to parse the config object to pull this out. Is that the right approach?

A

So um the idea is where am I with this? uh Oh, so then what we're trying to figure out is: how should we do this? Should we be parsing, the json object. uh Sorry, the sig. The config object to do that, and or is there a separate api that we use for linking, so you would still upload it using the standard apis for uploading to a distribution uh right to a registry and then there's a separate api for signature.

A

Then you basically add a signatures object to it, and all this says is take this digest and link it to this one. And if you don't call this api, then the artifact is up there, but it's not linked one of the pros with this is. We can actually put our back on a specific api in if we only.

A

Sorry, uh if we do it as part of the upload and parsing the config object, then you're saying hey, somebody might have push rights to the registry, but because they're pushing a signature, they also need signer rights and that felt a little weird to be sticking it into that api in the middle of that as all of a sudden, the push is successful, but it fails because the type of object you're pushing with this approach on the upload. Sorry as a linkage, you can actually put the rights check the roll check on the api.

A

um The downside, of course, is there's a separate upload and then there's the linkage api. The way distribution works. There's a bunch of these separate ones anyway, blobs get uploaded and then eventually a manifest gets uploaded. So it's not the end of the world, but it's just called out.

A

The other is there's a unique signature. Upload api. Since the signatures have no layers at this point, they are literally just uh amount of a layerless uh layerless, manifest uh or zero layer manifest with a config object. Then you know. Maybe this is another approach, because here what we can do is we can do the roll check on the upload and it's a little more cleaner. The the downside is, we are treating it on the front end a little different than other objects that get uploaded.

A

You could argue it's a special artifact, it's still better than having a separate notary server. um That's part of the conversation we get into signature discovery and we go into another set of conversations. So I'm going to pause there and and kind of ask for feedback, and I again I apologize. I meant to get this done last wednesday, so there's feedback on the uh to read it first. So I I don't expect all feedback here. I just will start the conversation.

C

I don't see any benefits to to using oci artifacts in here. All we really want is a mime type and a blob either way. The registry must have a special support for creating links and listing links either way. Every client must have a special support for downloading or uploading signatures together with the image.

C

So all we end up with if we are using the oci. Our defect approach is for single protocol to have two http requests per signature.

C

uh If we introduce a specialized api, this can be more efficient and it doesn't even limit neither the clients nor the the servers from using any storage format they want. So if some registry wanted to internally represent signatures as special linked oci artifacts, it could.

A

Yeah, it's interesting. The distribution spec doesn't completely drill into the details over how we all store these things as blobs and so forth. There is some discovery, not just whether it's discovery there is pull apis that we're also trying to be consistent with so once you have a manifest of a signature or standard artifact poll would just work, um but we we have been debating the upload one and we're also trying to make sure.

A

Well, while each registry has slightly differences in how we persist the objects, there is a mechanism there and we're trying to make sure that we don't make major changes to how uh an operator could would have to store them. But to your point, if somebody wants to go off and store them in a different way, because they can retrieve them differently, there's nothing that stops that.

A

So I think what I'm hearing from you is treat them as a as a separate api, as your preferred.

C

Yeah.

A

An.

C

Api that allows ideally getting all of the signatures. You know single http request or something like that. You know.

A

I'm deferring the retrieval yet I wanted to get the push part first I'll get into the uh signature discovery and.

C

um Single request to add a single signature, or something like that. Probably.

C

um Either way, this is going to run into some difficulties against the original design of the docker distribution server. I think because there was designed to be content, addressable and fairly state-less.

C

So the idea of the registering, maintaining and synchronizing a list of signatures that are relevant to a single image is not something that comes very easily, but it it promises.

A

What the index api does, though, like an index, does keep track by design of the manifested reference.

C

uh Yeah, but the thing is that you upload the index as a single unit and it either is there or isn't there, and if you change the index, you just upload a new one uh here there would be a stateful object that contains the list of all the signatures that are relevant for an image.

C

That's.

A

No, I think we're saying something different um and maybe I I wasn't clear in in buck so.

C

Maybe I just shouldn't speak about the implementation design of the registry, because I don't really know where.

A

No, no, it's the index part that you're referring, and I think that's, maybe where I wasn't clear um when you push a signature, the idea is that, instead of using manifest- which you would have this new thing in the config object to have to know the linkage or the separate api to do the language so to your points to the idea is: do we use index which already has a means for tracking linkage between a list and other manifests and a list? A list can technically port to another list which is totally valid.

A

So the registries are already rephrase that registries that support index, which is growing that not all registries did. I was surprised actually so we've been starting to see more and more registries support index now and the idea is that we're trying not to introduce yet another way to do tracking. So, if you would support index, you already have to support the idea that an index references other manifests the manifest have to get uploaded first and then an index gets uploaded.

A

It says here's the things that I point to and if you delete the amount of you should you should not be able to delete a manifest that is referenced by an index.

A

If you delete an index that doesn't necessarily have to delete the manifest right, there's a a rough check, ref, counting uh system, trying to say the word game, because it's some it's it's kind of like this guacamole game anyway, there's a rough checking, ref, counting implementation that should already be put in place. I'm trying to leverage not not create another one, but there is no single index that has all of the signatures, because that would make them. You know that would make it updated and we're trying to your original point, content, addressable and immutable.

A

Every signature is a new object. It's uploaded. We don't expect to have hundreds of signatures per specific digest. um Even you know the one or two that we talked about, but the idea is that I keep on adding more signatures and nothing else has to change. So I don't have to worry about updating any semantics in place.

A

Did that answer your question.

C

Not directly, but I just realized that I really don't understand the storage model of the registry and I'm just going to channel.

A

Please don't now you have some great questions. It's uh the underlying pinnings is. Is the sausage factory we're trying to avoid to your point, but the interaction is the part that I'm trying to make sure it gets handled cormac. You were starting to say something.

B

um uh Give me a minute: okay,.

A

Let me paste this into our chat session.

A

So this one is referring to this picture kind of shows two separate signatures got uploaded.

A

One was signed by the wabbit networks and one was signed by acme rockets, so the original signature for the net monitor image came from wabit networks and then, when it got moved, I didn't show the third for docker registry just because it was getting to be too much. I was trying to show the beginning and end, um but here acting rockets said hey. I also attest to this contents for my environment, but they're separate completely separate object, because this one got copied in from the original registry through docker hub to the wabit networks registry.

A

Sorry, the acme registry and then here a new signature got added to support it for that specific environment. So we want them to be separate objects all together, whether it's a manifesto.

A

Index.

A

Should I go on to the discovery apis and just give people a chance to read and and give feedback throughout the week.

B

um Why wouldn't you use a manifest? Sorry, that's what my question.

A

Why wouldn't you use the manifest to reference another manifest.

B

No, why wouldn't the signature.

A

Itself be a manifest.

B

Yeah.

A

So the idea that a signature so usually manifest reference layers, like I don't know, of a place that a manifest ever references another manifest.

B

I'm sorry! Yes, sorry, oh yeah, okay! So, okay, why wouldn't it be a manifest list.

A

Agreed so the reason so I was my original proposal was to use a manifest list. The problem and I realized we call it index. We could talk about that.

B

Sorry, I thought the terminology is really confusing.

A

I know I just and it's also an image index, which also adds a confusion, because it's not just an image anymore, so I I'm taking liberties and how I'm preferring to try to keep it as neutral as possible. Anyway.

A

The the only the reason it can't be an index today, which none of these things can be done today to be fair, is there's no config object on an index yet so we don't have a place to stick the signature yet like in manifests.

A

We stick the config, the signature in a config object, but there's no way to reference another manifest an index has a way to reference another manifest, but it doesn't have the index yet. Oh sorry, it doesn't have the config object yet and that's a problem for two reasons. It's a problem here for a notary because we want to be able to store the signature in a config object and it's an also a problem is that an index doesn't know how to say it is something other than a collection of arbitrary, manifest.

A

There's, no way to say that this signature, sorry this index is a cnab. This index is a signature object, so if we can add config to the index format, which we said we wanted to do for a while, then we unlock the ability to get both persistence and linkage through standard mechanisms. That registry should be supporting today. Well as they support index, they would support today.

B

Because they're, I think, yeah. I think that would make more sense to me because there are times when you want to refer to the signed object by its hash, as well as cases where you don't. And if it's not a that's much easier. If it's a normal object.

A

I agree: what do you mean by a normal logic? Just so I I.

B

Mean just a kind of normal registry object that you can.

B

That you potentially.

A

Like your index,.

B

Is.

A

Already a normal object that has the concept of linking other things.

B

Yeah yeah.

A

So it's just a matter of us. Look we're adding in we're going to have to add something like no we're not going to get notary v2 for free, like we know, we don't want it to be as big as a notary server, but we recognize there'll be some apis that have to change we're, trying being a registry operator we're trying to make sure that we don't have to redo our whole storage and and all the infrastructure there, and we feel the same pain for other registrars, especially because we've all done optimizations around our own clouds.

A

So we want to minimize that pain.

A

So the theory, if you remember back originally, we said, look adding another api, isn't the end of the world because look we're coders. We write apis all the time, but persistence is like a major change, so the thought process is rev the index because we have multiple reasons to do it and all the rest of the infrastructure comes in place.

A

So I'll leave that for some digestion, obviously reading staring giving feedback is the conversation we want to have, but there's a lot of a lot to digest here. um Let me go on to the discovery apis, because this one also had some interesting conversations that came out. I don't think I've done a really good job.

A

Writing them up here. Yet so I'm going to discuss it and make sure the conversations come out. Then I need some feedback on how I write it honestly.

A

So we said for the discovery: api notice, I'm skipping the poll api. For now the pull api is assumed to just be another artifact, but we'll come back to that. The discovery. One is the interesting one, because we want to be able to say that I want a list of signatures. I want a specific artifact type. I want a list of signatures for a given uh digest, so it doesn't matter whether it's an oci image, whether it's an oci image index, an s-bom, a helm, chart a uh marky mark document. It doesn't matter right.

A

Anything in a registry should be able to be signed and for artifacts that don't have a signature model, they can adopt this and it will just work for them and for ones that have special ones. Then that's great. They can persist those as well the so you regard this to almost realize. First question originally was a registry. Is this thing that you can use a tag to get a digest, but once you have a digest, it's a content, addressable storage. Using that digest. I want a list of keys.

A

The list of keys could is a collection. We don't expect it to be thousands. But of course we don't know- and I don't know if I want to say the list should never be more than 5 or 124 or 128, or something right, some arbitrary thing. So what we're saying is we want to do a paged api as all rest list? Api should be the interesting problem or question that we have is. There is a I don't know how to title. Let me put it this way start with this way in the distribution.

A

Spec there's a way of getting next and last. You know the last one that I got. There's a paging api. That's in the distribution spec, it is not as consistent. It is not consistent with whatever somebody would call the standard paging apis that people put on the registry put on and rest api.

A

So my question is: I don't want to reference just google docs of google apis, because google didn't invent the way to do rest apis. They just have a good doc to reference. It there's other ways of do. You know that also talks to I don't know how to refer to the non-distribution spec way of getting standard paging through a rest api, so I need some help there, but, more importantly, we need to make a decision.

A

Do we want to be consistent with the rest of the distribution spec for tag listing which, because we deprecated catalog as a spec driven api, is tag listing the only thing that's paged or is the list of layers? No, the list of layers are in a manifest.

A

So, actually, if I think about it, the tag listing api is the only one that actually has paging in it.

A

So do we stay consistent with it or do we stay consistent with what people consider to be the standard way of doing rest, api.

A

Pagings.

B

Sorry, what do you think the standard way is.

A

I don't know how to describe it, so, um in fact this is where uh it got to be dinner time on friday, and apparently this is where I left off. um I look. I haven't written these in a while. It's been a couple years since I've written production rest apis, so uh there's apparently a different way of doing it, and I need to find the doc that referred to it. It was actually in the pr feedback. I think it was sam, provided the pr feedback um to what was originally the signature.

A

uh Pr, let me see if I could find it, because it's the right conversation.

A

It was in this one because this original prototype had the distribution docs in it as well, and that was somewhat of a mistake. We well we evolved and decided we wanted to make it later. Let me just find it here here: it.

A

Is.

A

Excuse.

A

Me sorry, I was keeping my notes here.

A

So again, maybe we don't have to have the full conversation here, but I definitely wanted to tease this conversation out because I haven't finished. Writing it very clearly, but this was there. What should be? I think we agree that there should be a paging api.

A

We agree that the signature request, the signature collection api also needs to say. I only want the signatures of rabbit networks. I only want the signatures from docker hub. I only want the acme rocket rocket signatures and I want to filter the rest, so there's there's that part of it, but regardless of what parametering parameterizing we put on it, what should the standard paging apis be? Is the question.

B

um Yeah- let's not discuss that here. I think it's probably the courage answers to that question.

A

So that's the place of looking for some feedback and then the the last one and again this is where I was editing some stuff um so uh overall gave some great details here, which obviously I'm still digesting, because this is where it goes into the sausage factory. uh But the signature pull should just basically be any other artifact. You just pull once you know through the uh requesting the the signatures api.

A

Just like you do for any other artifact right, you get a manifest, you say hey, I know what I want now now I'm going to go request it. The idea here is hey. I would get to figure out what the digest is of the signature object and then just going to do a standard signature pull because we don't. We don't think that pulling a signature, a verification object of an artifact should be any.

A

I can't imagine registries would want to put any more rights on a pull of a verification object than they do for the artifact itself. In fact, I could see lots of holes where hey, I can pull the artifact, but I can't pull the signature to verify. That's the right! Artifact, so that feels wrong, so we're trying to say- and I obviously don't remember what I wrote here- but this should align with any other artifact pope.

A

So that tees up the content that I had in the conversation today, the only other one that I just got this morning and I'm just going to queue this up. Also, when I put it in the slack conversation is we took the feedback on doing a jwt format for the signature um that she was. He actually didn't make the pr all the way up. Yet it's on his uh registry so or his git repo will push this up as well to the notary v2 project.

A

But the idea here is this- and I was scanning this myself this morning, but basically the signature is no longer a json object that we have to have all these conversations around. How do you encode it in white space and not and so forth? um The interesting challenge, though, is once it becomes this encoded string to the conversation justin and I were just having- is we don't want to have to parse the config object, to figure out the linkage and then to milo's leosov's question? We don't want to have to have two separate apis.

A

So what I'm? To summarize what I'm hearing is we like the index object even with the additions we have to make to it, because it gives us standard tracing for linkage. The upload api should be a separate api, not a normal push api.

A

So this way we can put role management on it, but the parsing of that would be a standard index object, and then we don't have to read into the signature itself to understand which digest, because you don't want to have to parse this encoded object, which is a performance hit, that we want that. We don't want to take on every upload.

B

um The role management pieces potentially need to be brought into scenarios, because I think that um is.

B

Yeah, it's.

B

Having a separate.

B

Api, because you want separate role, management is not in the end of design that scales for any kind of object you want in ici. So I think it's a more it's a more general question. That's worth asking of oci as to how you would do role management for different types of things. I mean you could imagine a registry in which you want to give people different roles to be able to upload under helm, charts and container images, but doing that by having a separate api doesn't scale.

A

Agreed, I agree and I.

B

Thought conversation.

A

That I was thinking of is and that's why I wrote this our signature's special enough like if we think of the notary server we have today. That is a completely separate server. Of course,.

B

Yeah but that's kind of incidental, but I yeah my view would be the.

B

Designing for a world where signatures are totally separate, it makes them less likely to be implemented.

A

So what I think, what I'm hearing you say is just building on the conversation we just had is index is good, but do we need a separate upload for signature indexes as opposed to no just use the index the upload? It happens to be an index and there's a roll check that is implied based on the artifact type and that yeah those customers.

B

Who have scalable choices? The registry always has the ability to have role checks based on artifact types, and things like that. I mean the fact that no registry does right now is suggests to me. There isn't much demand for that, but.

A

Well, there's only been one artifact type until recently, so.

B

Well, yeah, but um I mean, given those ignitions are independent objects that just have linkage. You could perfectly well push them into a totally different repo with different permissions.

A

uh Well then, you get prop indexes, don't know how to reference things outside of their repo, which is, you could say, not really.

B

They do, they do know how to access things outside their repo. Don't.

B

So, let's save that conversation.

A

Oh.

B

Yeah, okay, that's.

A

I have to jump out of the call, so um I think we got the right conversations teased up by all means. Slack and pr feedback is our friend. It gives everybody a chance um wednesday. We have a conversation on index specifically darren. uh I got shafer stuck in my head. That's not his last name um darren's going to talk about his experience with index and we'll do that on the oci weekly call. So with that I'll see folks on slack and thank you very much.

A

Now, let's get out of.

A

Here.

A

You.
youtube image
From YouTube: CNCF Notary Meeting 2020-08-10

Description

CNCF Notary Meeting 2020-08-10