Sigstore Sigstore Community Talks, 28 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Life of a Sigstore Signature - Jed Salazar & Zack Newman, Chainguard

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Life of a Sigstore Signature - Jed Salazar & Zack Newman, Chainguard

Recently, Kubernetes SIG-release announced that the official Kubernetes container images have adopted Sigstore code signing to protect the supply chain of millions of downstream users. Sigstore, an open-source project aiming to be the LetsEncrypt of code signing, allows Kubernetes users to validate that their images came from the simple, free, and trusted official supply chain. But how does Sigstore actually work? What happens behind the scenes when I sign an image? Why should you even trust it? This talk follows the life of a Sigstore signature for your container image. On this journey, you’ll encounter keyless code signing, certificate authorities, and transparency logs. You’ll also configure an admission controller to create a signing security policy for your clusters.Our request hits every Sigstore component and you’ll stop to learn how they work, from the cryptographic and architectural levels, and discover how Sigstore mitigates supply chain attacks.

A

um So I don't think we need to give much of an introduction to what is Sig store here, um but I think this talk is really. You know, principled around the idea that maybe six door is new to you. So for those of you in the room who are probably already you know, familiar or invested in the Sig store, um this you know is probably was probably your journey at some point, but I think we're targeting.

A

um You know folks, who are look interested in learning what six door is and how things actually work, and that was really the you know.

A

The the kind of premise of what led me to you know talk to Zach about you know, building out this talk um so generally, when you ask what is Sig store, somebody gives you something like this right like this is a high level diagram um it it's throwing around a lot of jargon, um and you know for the most part, it's not the most helpful resource to actually Rock exactly what's happening when you sign something with Sig store.

A

um So our goal is to basically dive into each of these components, and you know break down each step so that we can really kind of go under the hood as to what happens when you sign something with Sig store.

A

So this is really kind of like preface by my personal experience when you know early on in my it career I, you know was learning networking and I. You know kind of read about the you know, seven layer, burrito or the OSI stack or whatever, and you know tried to really grok what was going on by looking at layers and things like that.

A

But really at the end of the day, like you know, I didn't truly understand what was happening until this magical moment when I popped, open, TCP, Dom and you know, started capturing packets and- and you know, with my hands on something real I could actually understand what was going on in networking right, like I felt like I was.

A

You know that was my kind of uh Neo Kung Fu moment when I truly understood something- um and that's that's really like the idea here right is like we'd like to actually give you something that you can touch and interface with directly, as opposed to just kind of talking about higher level, Concepts and kind of maybe remove some of the abstractions that six door makes um you know for developers to make the tool easy and actually kind of dive into how things work under the hood.

A

So the way that we did this is we basically, um you know Zach added a few lines of code um and you know created a uh basically a fork of cosine that, instead of actually having these things, just exist in in memory uh when you run, you know a cosine signature to actually dump them out to disk right. So like we actually have these artifacts around, we can inspect them.

A

We can truly understand everything that's going on um just because they're, you know locally there in the file system, and we made this uh available as well and there'll be a blog post that will kind of walk you through this. If you want to recreate this yourself.

A

So this is, you know. Basically, you know what we did and in this case we actually just signed this. This blob of you know hello, sigster con um and you know. Fortunately, we don't need this uh cosine experimental flag anymore, um due to the cosine being GA, which is great, but you know again. The key uh piece here is that you know when you run through these steps that are that are recreatable.

A

Is that you'll end up with these artifacts on disk and we can actually kind of walk through each of these artifacts and and find out what's going on during the signature operation, so first off, like let's kind of lay out what we're talking about so we're talking about like keyless signing, um obviously, six store, you know, provides multiple ways to sign, but you know the the keyless signing is, um you know, probably the most interesting, particularly from a human perspective, right like if you're, like a machine like you're, probably not going to be using this, but as a human like there's.

A

A lot of uh just intrinsic benefits to this right.

A

First off is like we're going to start here with oidc and like what we call the oidc dance um and a little bit of like a plug here is like the whole concept of what we're trying to do is we're actually trying to assign you know your identity to the artifacts that you produce right and in the old world of, like you know, using things like you know, gpg Etc, like that identity is basically just a blob of you know uh crypto like on a file system right and that's not the best way to assert your identity like in most environments, like you know, SSO um and oidc is kind of the the way that actually helps you.

A

um You know assign what that identity is. um So there's there's a lot of kind of benefits into into oidc right like um if someone leaves like you can you know revoke their uh their access. You don't have to worry about a key leaking.

A

um There's a bunch of you know baked in benefits here, but in essence, what we're going to describe is kind of how this openid connect flow, Works. um So, first off, when I signed something, um you know we're going to be interfacing with public or the public GA.

A

um You know, fulsio instance are certificate, Authority in sing store and just like any other certificate. Authority like we basically need to convince that authority. To kind of you know, give me a certificate that says this email or this identity is the thing. That's actually assigning.

A

um You know the the artifact in this case just the blog, so we're going to walk through the authentication piece like how do I actually prove that I control, my identity and as I've been kind of discussing like oidc's kind of prime for this right, like as opposed to um keys with oidc, you can enforce things like you know, uh non-fishable, two-factor or strong, Factor authentication.

A

um You can set up policies around it. You can, you know, trust certain identity providers and things like that. But at the end of the day that oidc is just going to come back with. You know a jot um that you know is basically going to help us take that identity and actually do something with it. So when we get back that oidc token, what we get back is basically a you know: base64 encoded, um you know Json document, um and that includes a header, a payload and a signature, we'll kind of walk through these.

A

So you know, we've talked to the identity provider. We've you know done the sign in with Google flow, we've convinced them that you know we actually own our identity and when we get that job back. When we look at the header there's a couple things going on right and the question that we're asking ourselves here is: like you know: how should the verifier, which in this case is fulsio the certificate Authority actually check the jot that actually gets assigned to us or token?

A

So, if we take a look at the header, we can see that first off. It needs to know that the algorithm to verify this is you know, RSA 256, there's also a key ID, which is helpful for us to kind of locate which public key we're actually going to be verifying against. This is in case, like you know, Google or whatever my identity provider is decides to actually rotate.

A

Those keys and then we get into the payload right- and this is definitely the most interesting piece right so inside of this jot, we have a number of interesting attributes like the first one.

A

Here is this issuer attribute, and this basically tells us that um you know the issuer is oauth 2.64.dev, but in this case it's actually referring to uh to Dex right I just mentioned that you're using the Google kind of sign in flow as one example, but we actually use Dex for reasons- um and you know we can discuss those if you're interested definitely feel free to ask questions. But um beyond that we have a couple of really interesting attributes about jobs.

A

One is this audience component, and this defines that you know the audience is Sig store, so when we, when we give you this jaw, that means that you can't reuse this to log back into Google or take your Google credential and go authenticate to any old service right like the idea is that we we scope that audience down to only be valid for Sig store, which is an important certain security attribute there. There do be dragons here right, like there's some components with uh certain.

A

You know, implementations of job that um you know might not parse this correctly, but there are additional protections here right. um One is that we have this expiration date and if we actually kind of zoom into this, we can see that this jot is only valid for one minute. So the risk of this leaking- and you know, causing a very bad day is, is you know diminished by that that short time of validity?

A

And then, of course, we actually have the identity, um which is you know in this case it's Zach's email, um so you know, feel free to send Zach an email here, um sorry for the slight docs there Zach and uh finally, we have this um connector ID and the reason that we have this is because you know we kind of talked about how or Federated claims, and then you know, connect your ID um because we use Dex right and like the reason behind that is, like you know, we kind of need to like Smooth out.

A

You know different implementations for different, a different identity providers um and in this case you can kind of see that, like the Federated claim actually does, you know, come from accounts.google.com in this case right so like we know that this person uh who claims to be or who actually was proven to be uh Zach at chain guard, um went through Google to actually sign and get that into that piece of identity from this jot and then finally, um you know if you look at the the final bit, we have a redacted signature um and the reason that it's redacted is, you know not because we like to wear 10 full hats, but because, as we mentioned like there are some implementations that um you know, don't do a perfect job of kind of verifying those.

A

um You know those values and we just didn't really want it to leak so cool. So so far, we've talked to the oidc provider, we've validated, who we are so we you know. Google has told us that you know this is Zach. um We got back a jot and that jot, you know, was basically scoped down so that we could only do six door things with it. Next, what we're going to do is we're actually going to generate a Certificate request right.

A

So you've probably done this at some point where you basically have a you know: signature request to a certificate Authority if you've ever registered a domain and got a certificate for it, it's very similar, but we'll actually kind of break this down.

A

So in the Certificate request, we have a a freshly generated public key right, so we generate a public private key um and we, you know, publish that public key um in the Certificate request.

A

We also have a signed email address, so we have a signature over the email from the oidc token and the way that we basically do this is we take that private key that we have and then we sign that um you know email address and that kind of proves that we're the true owner of this public key um and then, finally, you know there's kind of a reference here to you know we we see the CSR kind of being null.

A

um This is because you know we uh six door doesn't use the standard kind of pkcs number 10. Sorry for throwing um those you know bits of jargonet if you're not familiar, but that's just kind of the standard certificate.

B

And then now you've sent your Certificate request. Hopefully, when you request a certificate, you get a certificate and that's what we're going to see next um or or skipping over the CT log for the sake of time, but happy to talk to anyone afterwards.

B

um So this is a long document um and what we're doing is we're printing it out using a tool called Step, which is a very handy way to look at a certificate, uh and so we're just going to run through the highlights so very similar in many ways to the job that we saw before the issuer here is sigstore.dev and we're actually coming through uh Sig store intermediate node in in the middle like the jot. If you look at the validity, uh it's only valid for a very short period of time.

B

This is one of the key security guarantees we give you with Sig store is that the risk of a private key actually leaking is is mitigated because the certificates that we're issuing are only around for for 10 minutes. So after 10 minutes, such a key is going to be pretty useless and then there's there's usual stuff. That's in every certificate. The public key and so on.

B

If we go to the next page, you can see you can see some fun stuff. So there's my email again, so Sig store basically took my email from the jot and it stuck it in as the subject alternative name, which is sort of uh for convoluted reasons. The way you specify the name of the party uh that has been granted the certificate. We also have all these fun numbers right below it. Those are an x509 extension registered to the Sig store project.

B

That indicates what the identity provider used as the source of the identity that wanted that wound up in the sand is so in in when you're going to verify this very typically, you want to know who's signing, but also who says that that person controls this this public key, so zjn at chain guard.dev as a as a tested to by accounts.google.com, much much more convincing than zjan chain guard.dev as a tested tube by Neopets um the. uh So the next thing we're gonna we're gonna highlight is the SCT.

B

This stands for assigned certificate, timestamp again we're going to gloss over many of the details of certificate. Transparency here, but this is basically a receipt from the certificate transparency log that says uh I the certificate transparency log have seen this corresponding cert and it's gotten a company in signature um and then, finally, at the very end of the certificate, you're going to see a signature- and this is actually a signature from that full Co intermediate node.

B

So you can take this certificate and there's going to be a certificate chain that gets you all the way back up to the full see a root key which you, the verifier, will have gotten through tough.

B

uh We saw a great talk earlier today that went through many of those details which is great because I didn't put them in in this deck, but yeah and so, and so this basically uh with this certificate, you can sort of Trace all the way back up to full Co and know that the identity mentioned in the certificate as long as you trust fulsio, that any signature that validates against the certificate is legit for that identity.

B

um Great. So now we have our certificate: let's go ahead and sign something, and typically what you're doing in six door: uh you're not uh just signing uh the the naked bytes of the image or the. uh In our case, the data.txt file, you sort of have to format that in a little bit of an envelope- and that's that's going in the API- something that's referred to as a proposed entry.

B

um So there are a bunch of different types here, we're looking at the default type for assigning an artifact in in cosine, which is called a hashed record. So a lot of base64 nonsense that I don't expect you to be able to read but I I'm illustrating for you what what these things are. So this envelope is going to mention the hash of our artifacts. So if you see that that value right there at the top, if you run shot 256 some on that data.txt file, we created earlier that said, hello, six doorcon!

B

That's what you're going to get! Finally, you're going to get a signature over that data, the data in the hash that validates against the public key that we just generated, and that goes in our x509 certificate, so remember the x509 cert uh had a had a you know, public key in there. uh That's that if you take that public key, you can use that to validate the signature over that data and then finally, we actually stick that entire certificate in here, under the slightly misleadingly named public key dot content field.

B

uh This is actually a base64 encoded version of that certificate that we just saw so then record is going to take that proposed entry. It's going to make sure that the signature checks out and then it's going to return to use something called assigned entry time stamp, and so this this is, uh we started in log entry.json.

B

This is relatively uh simple document. The complicated part is in verification which we'll see on the next slide, but for now the body is the most interesting thing. That's actually a base64 encoded version of the file that you just saw on the last Slide.

B

The astute Observer might note that there are like triple or quadruple base 64 encoded things in there by this point. That's you know, that's just how life goes. Sometimes, there's also an integrated time. So remember we we made a big deal earlier about how the public key was only going to be valid for 10 minutes. Does that mean you can no longer validate an artifact after 10 minutes? That would be very, very annoying and make this a pretty useful system. So no, you actually can validate things after after 10 minutes after the certificate has expired.

B

What you do is you check that the signature was made during the lifetime of the certificate, and so that's this integrated time here uh right now record is typically the source of this time. There's some some fun ongoing work to allow you to plug in a timestamp Authority following a protocol like RFC, 3161 or rough time, and use that instead or in addition, um yeah and then finally we're going to look at this verification stuff. This is information that you can use to cryptographically check that that log entry that we we submitted actually wound up.

B

Inside of recore, uh and uh unfortunately, don't have time to to go into uh what a Merkle tree is how a transparency log works, but this is sufficient, basically cryptographic information that you can check. Okay, all this stuff wound up inside of record and then finally, we have the signed entry timestamp, which uh for the for the lazy or for the offline. If you don't uh want to go and do that check all the way back up to recore, you can just take recourse word for it.

B

So if you have recorded public key you can this sign. Entry. Timestamp will be a signature over a bunch of the rest of the data in this log entry.json, and so that should be enough to convince you again that this other stuff, the body and the timestamp are, are legitimate.

B

um Yeah. So uh now, you've seen everything step. Four is just publishing, which is going to vary a lot based on your use case, uh but could look as simple as like FTP.

B

um You know it's just about getting uh sort of the data that certificate that signed entry time stamp and then the signature over the data um getting all of that. To the end user, who's going to verify it and there's some some ongoing work to turn those four things into one thing: we're going to call a bundle uh shout out to Frederick who's been working really hard on a on a specification for that and.

A

For the kubernetes folks in the room, like I, think it's worth calling out that typically, the the location that we store. These values is in an oci registry.

B

A

Don't have to think too hard about where you actually store this, because it sits alongside the container image, if you're using containers and kubernetes, which you know, makes it that whole. You know experience like just super easy in plug-in.

B

Yeah yeah and so for those who don't have the luxury if you're uh working on the integration with a package manager, um I see a couple of folks in the room who are doing that right now, uh you're going to have to unfortunately take care of this step forward yourself, but all that is is pushing bits around. There's no fancy cryptography. The fancy cryptography happens before uh and at the end, where, where folks are verifying, um speaking of uh this is kind of the reverse of that process.

B

This is what a verifier has to check and it seems kind of complicated right. Six steps is a large number of steps, uh but but you'll actually see these correspond pretty closely to the steps we took when creating all of this so you're going to check that the signature is from who you um think it's from the way you do, that is by looking at the certificate check the subject and the issuer of the certificate.

B

So if you trust uh universally artifacts for me- which maybe you should maybe you shouldn't- um but but you can, you can sort of check for a subject- alt name that says dj.changard.dev in an issue or accounts.google.com you're, going to check that that certificate is valid, that it validates against falsio's public key that fulsio public key again. You'll have gotten via tough, so refer back to the to the tough talk earlier in the day, um you're going to check that the signature on the log entry.

B

So that's where you get that time stamp from is valid for the um sorry, the signature in the log entry. So that's the signature on the actual data itself is valid for the public key in the certificate you're going to check that the log entry time stamp is signed by recore you're, going to check that the timestamp was during the window in which the certificate was valid. And finally, this is actually really important and really easy thing to forget to do.

B

You're going to check that the data that was signed is actually the data that you care about. So it doesn't. Do me any good to check all of these things and then do not say oh by the way. You know I I validated this over here and I'm, going to use this totally unvalidated blob and run that you've got to actually match up the thing you're running or the thing you're downloading to the to the log entry um yeah.

B

So at this point, uh hopefully this has uh demystified and I use that word very deliberately uh a little bit about how sex, storing cosine work. um uh It's not magic right. All, that's just bits and bytes uh and uh I, see see Bob in the audience here. um I would I would encourage you if you want to learn more there's going to be a blog post version of this, which is cool.

B

If you want to go even deeper, uh there is a link which I know you cannot click, but if you just Google it uh it'll it'll pop up Sig store the bashway, which basically takes you through this very like bits and bytes approach to how Sig store actually works, but totally on the command line. You compose all these Json documents yourself and like any good cryptography security project.

B

If you screw up, it just fails and it doesn't tell you why it failed, but but it's a really instructive exercise, I think to go to through sort of the next level of of take it apart to understand how it works. um Yeah we hand waved over a bunch today uh happy to take questions offline about uh sort of decks and uh why there's a transparency log and how that works and so on, um but hopefully this this sort of took you from having this abstract understanding of how this all all works.

B

It's a complicated system to sort of understanding a little bit. Oh yeah, okay, that's that's how the data flows through the system. That's where this entry comes from! That's what this means anything else. You wanna that no, no cool! All right thanks! Everybody.

B

Cool, uh how are we doing on time and do we have time for a question or two.

A

B

Anyone have questions yeah, oh sure, sure I will hopefully be able to get that back.

B

A

B

B

So the 10 minute window is here in uh the validity here in the x509. Sir. Oh sorry,.

A

Oh, the uh yeah yeah.

B

ah Yeah yeah yeah, so the 10 minute window is going to start when fulsio creates that x509 insert um and it's going to go for the next 10 minutes. So you better um get uh that proposed entry in the next step to recore within those 10 minutes, otherwise uh that time stamp is going to be outside of that window. uh Fortunately, the typical time that this takes is like a couple of seconds, so so 10 minutes is a pretty big grace period.

B

Sorry, okay, yeah.

A

B

Yeah yeah, so so the 10 minutes is a somewhat arbitrary number. uh It's it should be pointed out. uh We chose it as sort of like very, very worst case scenario. uh This is all gonna happen. Even.

A

B

For ecore right now record is actually quite Speedy um due to uh a bunch of things, including the the sharding work, uh which is which has sort of sped things up quite a bit, um and so we're not worried about that. But you know in a the long long term we might worry about okay, maybe recore is eventually consistent. It takes a while for things to get integrated, um so we want that time stamp to happen. You know we we think 10 minutes is kind of a good middle ground between you know.

B

ah It doesn't feel like very long at all. It feels like very unlikely that your key would leak and an attacker will get it and be able to sign stuff during that window, but uh uh still long enough that you're going to be able to do what you need to do.

B

Yeah yeah yeah. That's that's a really really good question. uh The the sorry oh yeah Greg. That's a really good question which I'm going to repeat right now. uh The question was: uh when you're verifying? How do you know what the subject that you're supposed to be checking is, and in particular, how do you know what the issuer is for that subject?

B

um So this is uh the sort of really unhelpful answer I could give. You is that this is out of scope for Sig store um uh and it's a really unhelpful answer and that's that's sort of not where we want to go in the long term, uh but but basically sigster handles the mapping the identity to the artifact leg of things. We would strongly encourage you to look at something like tough for doing the delegation for that for that artifact. So, in.

A

B

Closed ecosystem in your own, like in an organization, that's your own company, you might have one party that's in charge, you know one service or one individual or one team, that's in charge of signing all the artifacts. In that case, you can sort of hard code at the verification site. uh The a verification policy that says every artifact I run must be signed by a security team at mycompany.com, um I'll I'll pick on pick on William, because I've seen the audience here.

B

If you're going to do uh this at the package repository level, then basically the package repository, something like the python package index is going to basically need to say for package.

B

X maintainer Y is trusted to sign that and the reason I've I've been sort of so pedantic about also always putting the issuer there is that I think you should consider an identity to be a tuple of sort of you know an email or a username, comma, the issuer of that identity, so zjan chain guard.dev, as attested to by accounts, as opposed to yeah or Neopets. If, if that's, who you really really want to base base the root of Trust?

B

On um that answer, the question yeah it punted a little bit but I'm happy to discuss with you offline some, some cool uh tricks, okay,.

A

B

Macbook is telling me we're done so. Thank you very much.