OCI Weekly Discussion, 16 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: OCI Weekly Discussion - 2021-06-16

Description

Recording from the OCI weekly developer's call; agenda/notes here: https://hackmd.io/El8Dd2xrTlCaCG59ns5cwg?view#June-16-2021

A

Let's see, let's grab and grab a link to the hack md, just in case somebody doesn't have it.

A

A

I think we have probably about as many as we're going to get did you want to go ahead and uh finish up your discussion, uh we're on automatic content, discovery.

B

uh I mean so, I had gone ahead and updated the pr um and.

B

Made some changes based on some discoveries from docker distribution and provided a reference implementation, which is what lasker suggested as the next step for getting it merged into the spec.

B

And I wanted to see if um people had anything more to say or you know whatever.

A

Oh, do you want to, I guess, go on a presentation and give it a quick summary and make sure everybody understands.

B

uh Sure I mean I don't think that presenting the thing is necessarily the most um best idea, uh just because it's a lot of text, um but I guess restate. um The original issue was described as a mechanism to deduplicate uploads in a registry between repositories, because what we found is there are many common layers between our repositories that end up getting duplicate uploads when we do something in ubuntu upgrade, let's say.

C

B

And this literally results in you know um like a non-negligible number of redundant uploads um in order to potentially avoid the situation, uh one idea might be to um go ahead and allow for arbitrary deduplication or- and we kind of had this capability already in the mount api, um and there were some ideas. If, if we could do this in situation, we we mean we, everyone like the oci community uh adrian- is that uh so the.

B

In, like speaking, very specifically as a representative of netflix, I am speaking about our production registries.

B

um Our very specific use case is that we have one registry that people develop out of um and then a bunch of registries people will go ahead and deploy into so with the cross mount. What happens is docker doesn't know or whatever the development tool that we use uh doesn't know that those other repositories have been pushed to all those registries that are the production registries, so it can't. It doesn't know that it can do a crossbow here and in this situation we were like.

B

Is there a way to do this safely in a way that's secure uh where there is an access model that has well any kind of access model right, any kind of access restrictions between repositories, especially without disclosure, right of security information.

B

So we attenuated the initial issue to instead of doing that for repositories that do not care about information leakage about knowledge of blobs, allowing the mount api to perform automatic discovery of blobs, as opposed to requiring the user know which repository to mount the blob from and all that really required was dropping the from in the mount api.

B

And this is a really nice backwards. Compatibility effect, because in the specification it says, if from fails or amount fails, just continue to do a normal upload um and uh yeah. That's a rough summary of where we're at with the pr.

A

John, you had done a review on this derek. Do you have any any thoughts, how's.

D

It doing the permission acl stuff I saw saying about like oh if the registry or the repositories allows for kind of sharing that information but how's that defined register. So.

B

The oci spec talks nothing at all about access control models and I don't bel. I don't. I don't think that I should write about security, um but I believe that not.

A

B

In the pr it is fine and they commit to uh strongly suggest that users do not do this in a security model where they are trying to keep the knowledge of blobs confidential uh between different repositories or.

A

Authentication or.

B

Yeah yeah, exactly if I mean you can do this with any offset authentication authorization model in theory um that I know of, but it's just really ugly to do it and maintain 100 confidentiality.

E

um A use case that we have- and I I like this pr because we already implemented- um is that they're, like gcr, has a very small set of like well-known maintained public projects.

E

So we know that those are always accessible, and so we can look those up and we don't care to leak, the information that they do or do not exist because we're only looking in these public spaces, and so this this lets us short circuit like lots and lots of traffic. That would otherwise have to be uploaded for a thing, that's known public and is easily accessible. um So I think that's a kind of similar application. You don't have to think about the overall registry auth model. If you have any concept of like public things,.

A

Right, okay, so we're not we're not going to boil the ocean, but yeah. I think I think we've got a similar fix in container d right there.

A

So you have some trusted registries, for example,.

D

Not sure well from the I mean, what's the impact from the client perspective here, if any, that a client is expected to handle an auth code that possibly if it went down a different code path, I guess.

E

D

I think if there's.

F

Anything in office.

E

You you would get a 202 response and the client's expected to just do the upload itself like this is a um like optimistic.

E

uh You know mount if, if I can do it otherwise, you'll give me a 202 and I'll continue.

B

I think that there might also be a small bug, it's unrelated to this, um where, if the blob exists, it doesn't give a 201 um but um that's unrelated to this, but it should give a 201 if it already exists from in. In that specific repository.

A

I don't see jason. It would probably be a good idea to to see if, if you had a proof of concept, if we could check out the test cases and see if this you know what loosening this would actually work. Okay, uh you know the test cases.

B

So I mean there's test cases in the uh attached reference implementation- um it's not plumbed through to the outside configuration of docker distribution, which I can do if you want, but.

A

Okay, fair enough yeah, I was trying to figure out how we could, if, if we modify the distribution specification in this way, would we have to modify the test cases that we use currently to verify that the registry? Oh.

B

I mean we'd certainly want so it seems like there's two this I don't know if the conformance test is actually very good on this. We would want to add this to the conformance tests, um but we shouldn't need to like we should. We should add the does. The blob already exist to the conformance tests, but I think that is a unrelated point to the uh pr or to get the vr.

A

Right: okay, fair enough.

A

Okay, thanks see what was next on the agenda, or did anybody have any more anybody, have any questions or want to carry on this conversation.

G

Sounds like everyone's okay, with this merging.

C

I I'm okay with emerging, I'm I'm curious for the implementation on your site. Are you running your own client to implement this, or is this just the first step you're going to start going now upstream and start trying to get the builders to implement it.

B

Yeah I mean the first step would be to get this merged into the distribution spec. uh Getting this into docker distribution should be relatively easy. I mean other than the fact there's like one one person reviewing pr as it seems uh and there's a massive backup of yours um but um the clients. You know.

B

If you go ahead and merge this uh into the spec, the client should be able to adopt it like tomorrow, um and the fallback behavior based on the specification should always uh had like be okay with registries that don't implement the new capability.

E

Yeah, but my only concern with rollout there is like I expect some registry returns of 4xx in this case. Really you didn't supply from um so I'm going to bail, so a client may need some. You know ideally right. Everyone adopts this method right because we can collapse the an existence check and like an opportunistic deduplication and the you know, I'm going to start uploading requests all into one. So we get rid of all these round trips, which is great. um But for that to happen, registries must return the correct thing.

E

Otherwise, we're going to add an additional like okay, head request then try to mount, even though I don't know where it's from that fails. Okay, now I'll start an upload, that's exactly that's!.

A

Sort of why I was saying maybe we need a test case here.

E

Yeah it's hard because you know registries and clients. We need to do this sequencing of things, but yeah.

B

But I mean that registry would be non-conforming right, like the specification specifically says- and I think you wrote this right that says quite.

H

Possible, if it.

B

Fails to upload, you know 202, but, uh yes, the clients might have to handle bad registries and it turns out. I tried to implement this in a different registry. They did not correctly implement the fallback behavior for the front mount uh so hence I went to distribution instead.

A

E

It might be nice to write up- maybe not as part of the pr but um like how we expect the sequencing of rollout to work.

A

I think it's fair to in the in the details of the pr to talk about this sort of thing and then maybe in the specification you know just it could reference the pr itself, and you know for background information for why this change went in.

B

Commit git commit please, uh because github is not super great.

A

Fair enough, okay, so phil invents. She had the.

F

Floor yeah vincent had um image, spec issue pr grooming, so.

I

And I'm in and not a good place for that now I've had.

F

I

Come up and, like literally, was in the car trying to download zoom and uh get it on my phone again, um so I might have to put that put that off to an actual meeting like I said I would last week, um but I got I got a couple of folks. You know asked if I wanted to do any kind of q a on the throwing the flag email that I did and then it might just roll into the uh kind of grooming piece.

I

But um I try my best to be clear sometimes, but if the email was confusing to anybody, I'm welcome to take any questions right now, but hopefully the biggest thing is that finding a way forward for all the folks that are involved and people who consider themselves lurking or otherwise bystanders that we're in agreement that there there are some changes that are needed.

I

There are like existing things that we defend and the you know have defended in the distribution and image spec, uh and otherwise that we're working towards kind of a common goal of having ways to work with all these different artifacts and that everybody's on the same page. um But.

H

I

I'll open it up for a minute, if anybody has any particular questions to ask now I'll do my best to answer and anybody else, that's obviously and chime.

I

G

I don't see dan on the meeting.

G

So I guess I.

I

Don't know I don't know, steve is going to be able to join either so.

F

I

F

Yeah he's out of the office for a couple days, so please.

G

It seems to me that um well so I I'm not speaking for dan, but it seems to me that the purpose that what he was trying to accomplish with his uh proposal, which is to add references, was a minimal change uh to the spec.

G

But I did not understand from your email since whether that qualifies as a minimal change.

I

um It's a good question, so the the focus would be- and I think that's where, like there are aspects of references in some of the threads that are happening and even like branches that are happening in the artifacts repo to try and tease out what that would look like and possibly even proof of concepts in different. You know, forks of the distribution registry.

B

I

Code, um but they they're they're large, and I think one of the big things that we need to all the maintainers to be on the same page about is kind of decomposing, those and breaking them down into incremental pieces so that it doesn't look like a substantially different kind of spec, and so uh it is my fault that we're all on the same page as that, some of these, like incremental pieces, where we can focus on maybe the references aspect of it or making certain things more, generic or not, really generic.

I

um It's still incremental pieces, like the references conversation such that they can work their way back into the existing spec and effectively have you know some amount of like not breaking existing clients, but having like forward compatibility as well, and that those kind of things just take a level of review and sometimes even a review of a proof of concept that um effectively they're small enough or even this is. This is something that you know. Historically, even we got into nitpicky like. Would this be a v a v one dot o dot?

I

You know next, or would it be like a v 1.1 um where you could? You know, like semantic? Versioning, really doesn't work well in the spec sense, uh but if it's like small and incremental enough, but it basically bolts on to what we already have in here's it board, then I think that's fine again I the spec is not frozen. You know I can speak with whatever authority being just one of the maintainers, but the spec is not frozen.

I

It just has to have enough review of the folks that have um you know kind of like maintainership of it and otherwise they're involved to say yeah, that's an incremental change and we can like slowly add out this feature set without breaking what we already have in production, and it also enables us for future use cases.

I

So yes, it very well might be small enough incremental enough.

G

Okay, thank you.

G

G

Oh, I have actually, I have a follow-up. I forgot to ah sorry um so so vincent uh does that do those conversations happen in these meetings or is there going to be a separate forum for it.

I

Yeah, that's you know we. We almost have too many ways to communicate. There's the slack there's the mailing list, there's the weekly chats and then a lot of discussion happens. You know diving into all the different issues and prs.

I

Sometimes it does help to have just kind of a working doc, even sometimes those happen in hack and d or otherwise.

I

uh I think the thing that we all should you know work towards is maybe not brevity, but just at least clarity of like how here's, how the work's evolving, if it's not all in one issue, on a github issue like if it ends up having to track out the different conversations and they're related, maybe it's time for a doc or something else if they can be one-off conversations in a and uh you know just working something through through asynchronous stuff, like um email or slack. That's fine.

I

um I love in person conversations like there's there's times when these kind of chats are necessary just to work through specific topics uh or at least give time to something that isn't working in you know github prs and issues, um and historically we found that you know like sometimes things were just happening completely on github issues, and that was fine. Sometimes they bubbled up into the general call, and sometimes we just had to set aside times for like this is kind of like an image that call, or you know, run time.

I

Spec call you know back in the day, I was like. No other maintainers really need to worry about joining. If you don't want to it's, not a general call but anybody's invited to sit and participate if they're interested in that particular topic, um and sometimes those are very, very useful. Just like uh really work on this. You know certain agenda so really whatever works best.

A

Yeah, I prefer if we could get together, maybe maybe that can't happen yeah, but uh that would be nice. You know one thing we could do vince, I don't know we think we could. We could split this call up um into, and you know every other week is specifically targeting dot one extensions in this kind of area or some other scheduled. You know, pre pre-decided, you know plan we another. Let's use this call next week to talk about.

A

You know what are the different options that we could use right, modifying image, spec, adding a new manifest, adding a reference.

A

You know wherever the various options are in this are are and then get a and get enough maintainers in that call, as you say, you know that on the distribution spec that own the artifacts work, that's going on and the artifacts repo and the imagery pro, I think if we get a collection of you know maintainers there, we should be able to make some quicker decisions about what would need to go into a two or you know, or a v2, and what.

I

A

Because I do what I do, I like.

I

That's that's one thing that I you know like. Surely, for one of the big reasons that I'd say, the spec is not frozen.

H

Is very much because, like.

I

If we do, you know, determine like this is fundamentally different. It needs to be a v2 that doesn't mean that, like like, I don't want to increase the burden on folks like that is keep things as incremental as possible, and that just takes um actually some amount of consensus and sometimes like going back and reworking something um reworking.

H

I

Threads that we have is, it was interesting. This is a side topic, but among some of the image spec maintainers, we were chatting last night.

I

It didn't happen on the open channel, even though I just said you know, but um that, but it was kind of interesting because, like steven day was piping in um he's not always very active, but at least like go brainstorms and stuff um and he was citing some of the pr's from like back in 2014 2015 era of some of the same stuff that we've like iterated on and like the same kind of proposals that had different different threads and we almost reached consensus before other things happen, um and it's it's so it's like.

I

None of these are new ideas, and I think we've got enough lessons learned. We just actually have to uh get down to actually the technical, technical details. You know implications of it and just work through it. Incrementally I think that's that's the best way forward.

A

With the right people, so yeah yeah, getting stephen day in here would be amazing right, a lot of history on it cool.

A

So so what so? What do you think? Just all maintainers did do some kind of a a doodle or something, and we get try to get a call to go. Maybe on a different. You know different time schedule.

I

Oh I like kind of a maintainer cadence that would be.

I

Sure I, like I like doing those things, add hot, but I do think it needs to happen. So that's I. I want that to happen for the image, but sooner than later, um okay.

I

I'm just saying, as far as like not setting up the page.

A

Yeah I mean you need you need an initial meeting to decide the cages. You know meeting for a meeting right, but I think that's what people are asking for. I think I wasn't the only person what I think that might be what niche, what what's that next step? How how is it going to happen? Where is going to happen some somehow that has to happen right, decide where we're going to be discussing this.

A

In the image spec is.

G

I

Make sure you're starting to ask another question. Sorry.

G

Oh well, I was going to ask if this maintainers meeting is open to other folks who are interested, but it sounds like it's.

A

No, no, no, no nisha! You have to be there. Please you've been asking for this for years we have to, we have to let you in on yeah- and I did I didn't mean I meant at least the maintainer, so so that there's.

I

Sometimes sometimes the inverse happens where we have really good discussion and not enough uh maintainers on the call and that's frustrating.

A

Yeah, that's that's what I meant.

G

Thank you for that acknowledgement, mike.

A

I think the the federal government has acknowledged your emphasis on this area.

G

A

And I think that's why we're all here misha, you were right.

G

Thank you united.

A

G

Federal government.

I

Cool any other questions on on on that.

J

um Yeah, I'm kind of curious like how does this uh change the sort of working group model that we've been talking about, because that that seems like a solution that was also designed to address that same thing. It's.

I

Almost in the same vein of it, because I think some of the some of the pieces that we're trying to reconcile is that uh even like with the artifacts working group and some of these pr's that become kind of proposals proposal level like what to what extent is you know a proposal affecting multiple specs and or is it a breaking change, or what not like ensuring that something has that kind of level of review was part of the problem?

I

We recently had a t.o.b call about putting in a working group model, so it's all in the same vein. It's just, I think part of this is even going to be kind of like retroactively, saying like this actually should have had better scoping or review or whatnot. What was the success and all this kind of stuff, um so that's kind of a growth pain that we'll have to just go back and kind of figure that out what would have been success, criteria and otherwise and making sure that all the things were reconciled.

I

um The working group- I just chatted with uh amy, scored apparent this afternoon um when I, when you talk when asking about the subject line, email anyhow, she said that there is they from the tob call we had, they had sent off and gotten some like working group. You know, wording put together and there's a few like feedback that we have on that. um She's got some updates to post. We should have something official about the like working group model in general.

I

um All this is really just saying that, like we're looking to adhere to something that will kind of guide these conversations on an ongoing basis, so it's very much in the same thing.

J

Great yeah, well I mean one of the.

J

I don't know if other people feel this way, but one of the things that I have been struggling with finding a place to contribute where it seems like we have several prs that have been proposed, uh but then there's no real place for the respective owners of those prs and other people who are interested to actually iterate on anything. uh And so instead, we just kind of uh make comments and you know have dm conversations and then cause you all to have to send emails and be nice to get a place where we can.

J

uh You know iterate on the on the proposal that allows us all to merge onto something that doesn't necessarily mean that we have like made anything final. Yet.

A

Right, like a sub branch.

J

If you will, you could call it a branch yeah.

F

Yeah I mean I yeah. I definitely think part of working groups is, is establishing a more formal way for that to happen, um but I think you know, even in the discussion we were just having about meeting who needs to be there like. I think the interesting twist is that what's being worked on today is not like a single change to a single thing.

F

It's, like you know, you've got. People are working with registries that potentially need modifications and to really try and end.

F

You know, attempt to like validate a certain proposal is is crossing like multiple repos, so you know there's solutions for that, but hopefully part of what working group gives us is a way to like set that up in a way that there's one spot to go mess around. You know create branches.

F

You know create scripts, to pull all the right things and build it and test it, but yeah, I think that's one of the one of the complexities of like not making a ton of progress here is, like you said, michael, like it's not having that kind of centralized like hey. This is our playground. While we like figure out, if these proposals make sense or work together or break.

H

F

Registry or your registry, or this tool or that tool.

F

F

Yeah, like like vincent said, like you know, not not to throw lawyers under the bus, but, like you know, we want to bring that to reality as soon as we can it's, but you know we've vincent and I both talked to amy, and you know we need the next respin of that language. So hopefully we can finalize it and vote on it and it's obviously taken longer than what we had hoped uh to bring it about.

F

But I think you know I think we can start the wheels turning on some of this without officially having that, I think, is what we're trying to.

A

Say yeah just a couple phone calls to get get some stuff kicked off and that'll help too to create the work group because we'll know who's the most interested.

A

I

All right, good questions, everybody! um I don't have anything else to add right now, so I'm not.

I

If you do have any questions, you can think me on the general channel or otherwise. So I'm happy to do literally whatever it takes to keep things rolling, and it really is a reminder that, like it's it's funny, even when oci was getting formed and back in the days of like cncs and like people wanting to ensure that there was like a governance model or in place or otherwise and having opinions in both directions.

I

That one of the things that oci was literally founded on was a very a very open governance model and ensuring that no one company or whatever, have too much of a sway. And you know it's in times been cumbersome, and it's also meant that decisions really did require more of a consensus than ever before. And um I think I think we see that in some sometimes people say it's like hard to come to a decision or whatever in oci um but to the other side.

I

Other token, like it was also put that way to protect that there would be particular players get in and swing it one way or the other and we've seen that and it's like been the benefit of it. So um um you know the lawyers under the bus they've helped us with good wording and we've actually been able to hold up pretty true to that. I think the community is the better for it. So I'm glad you're all here and asking questions and we'll continue carrying it all forward.

F

Thanks for phoning in from somewhere vincent.

I

All right, let's say people can get 20 minutes back to the day.

B

uh I guess derek you want to stay on the line and maybe hash this out.

D

Sure yeah we can, can you discuss that yeah, I'm just going through um so there's a this change also should affect the detail.md um and I'm just trying to think of. If we were going to add an faq, I think it might be worthy of it. Considering it's like a point: release change.

B

Okay, yeah all right, I did not take into account the detail on md. I will edit the detail.md.

B

um Do we have an faq.

D

There is an faq and it has it. It covers topics that I think would be similar to this. So I I think it would be.

B

I agree with you, I don't know.

D

Yeah, we don't need to mention the security model. We should just mention what the behavior is expected to be from a client or registry perspective. Well, I think I I I.

B

Think it's um okay to add to the faq and be like: don't do this if you're doing security and you care all about confidentiality um and here's. Why.

D

Yeah I mean yeah you're right. Those are two scary words to put in there but yeah. I think that the faq is kind of designed to be to help implementers a little bit so kind of like an implementer's guide, so yeah. We don't need to put that in the official specification, which should just be very. We can be very literal about what what it does and doesn't do, including it.

B

But I I guess the faq would be like. Why is this optional uh for registries to implement and then the answer would be well.

B

There is problems with the disclosures in security models with registries here and explain that and then we don't need to use the word security, but we could talk about confidentiality and explain the security problem and then let people draw their own conclusions.

D

Yeah, that all seems fine, okay, yeah. I think just it's it's more about just highlighting it's. It's very subtle. If it's just like that one line like if it's not found like if it's found it's it's, it's pretty subtle to the readers. I think so. I think.

B

I mean the commit talks about it, but I think it makes sense to add it's fine dad today.

H

You read commit messages.

B

No, I mean, like other code base I develop on. It, turns out that they don't have github so um yeah commit messages, are the go-to, but um the I think adding to the faq is fine. I I and then adding to the I didn't even know about the detail md, but I can also add add to that. Is that generated, or is that written.

D

It was generated at some point, but it's no longer it's it's hand and.

D

A

D

I mean it's, it's basically, the similar wording, just I think, calling that out as uh optional like I don't know how much I don't think more. Details necessarily need to be out of there.

B

I mean I actually think that the this is wrong, there's probably stuff in there. That's.

D

B

It doesn't talk about how like on failure.

B

It involved name or digest; uh no, it doesn't. It says like on failure. It should give a 400 bad request, not 202, created details.

D

Yeah, that's part of the generated that was definitely um because of how they were generated before but yeah. We can clean that.

B

Up too blame john here right, he was the one that added the unfailure started, an upload.

D

If you want to add something you have to fix it too,.

B

uh I I I can fix this one and then add to the faq. um I will make the faq commit a discrete, commit um and explain in the commit message. Why not add it to the thing, because it turns out that capturing all the nuances is fairly difficult in this? Maybe a detail- or this may be details that are only interesting to implementers and then I think I'll make a different commit to fix the detail first and then in this commit we can actually get the detail with the new terminology.

B

So, does that sound like a good set of next steps for yeah.

D

Sounds good to.

B

B

E

I'm excited to see this get implemented.

B

Right, I will give everyone time back.

A