Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2022, 11 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SBOM X-Ray Superpowers: Making Better SBOMs, Using SBOMs - Brandon Lum& Chris Phillips

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

SBOM X-Ray Superpowers: Making Better SBOMs, Using SBOMs - Brandon Lum, Google & Chris Phillips, Anchore

Creating SBOMs (Software Bill of Materials) for our software artifacts is very important in understanding our software and responding to security attacks/vulnerabilities. However, creating SBOMs is challenging. To be effective, SBOMs must be as accurate and complete as possible, but at the same time be usable. Today, Software Composition Analysis (SCA) based SBOM generation tools strike a great balance in this regard. There are several great SCA-based SBOM generator tools today, but all of them have blind spots, such as finding an executable file that has no metadata associated with it. What if there was a way for SBOM tools to reliably fill in these gaps in order to produce a more complete SBOM? Enter the SBOM X-ray vision! In this talk, we demonstrate a novel way to peek into these opaque files through SBOM discovery and look-up. Through the use of the Rekor transparency log and In-toto attestations, we’ll show how easy it is for existing projects to share SBOM information with other projects using native CI integrations. We will then show our new superpowers in action through the Syft tool to generate more complete SBOMs!

A

So again, um we're going to tell you about. You know s-bomb, actually, superpowers and more specifically, how do we make s-spons better with more s forms? Of course um we love reference.

A

um Just a quick thing to note before we we get started. A lot of this work was also contributed by Marco Davis, who was a internet Google.

A

So if you haven't been on Twitter for the past few years, you've probably already heard of s-bombs or kind of heard of software about materials. But if you haven't uh software bill of materials is basically you can imagine it's an ingredient list of of your software application.

A

So, what's in your software as as to what's in your food- and this is really being driven by, um especially in the US, the EO executive order from violence Administration, as well as the OMB memo that was recently released, um saying that here's a deadline, we need a federal agency vendors to produce s-bombs, um but as bombs really is in compliance with compliance sake, and there are actual use cases for s-bombs, such as vulnerability component analysis figuring out, uh you know: do you have that log4j or now the text for Shell vulnerability uh in the software that you create or are provided by a vendor and also other use cases such as licensing?

A

So, let's double down on this food recipe, because you know we every s-bomb is the ingredient list. You know the first question is: how do we get this ingredient this? And so, let's take this scenario here: uh I I go to this really nice, restaurant and out of this bowl of ramen and in this hypothetical scenario, I'm allergic to peanuts. So what I want to do is ask myself this question.

A

You know I'm really hungry, but I need to know whether um that's anything I'm allergic to in this, and so one way that I can do. That is you know by visual inspection. I can take a look at the bowl. I'm, like you know, I, don't see any visible peanuts.

B

I I don't see any peanuts.

A

Yeah I think you're good, that's good right and then because I'm super careful, I let Chris like have have a bite and he's like no peanuts. Yeah we're good, couldn't, taste it yeah. We're fine should be good. All right, then I I I I eat the bowl of ramen, obviously right, um however, so the second option here is um asking the chef right. So I can go back to the kitchen and be like. Oh, what do you actually put in this I'm allergic to peanuts? And in this case the chef comes back and say yeah.

A

You know this is a secret ingredient. We actually bought the broth with peanuts, so it's more fragrant and obviously I'm a security practice practitioner and so I'm a little bit more paranoid and therefore I go with option two.

A

um But how does this all tie back the software so option one? When you talk about software, we're talking about observing analyzing, something that we have so this can be a bundle, application, a container, a binary and we use software composition analysis. We look at binary symbols. We look at package metadata, uh heuristics and fingerprinting, so that's option one, and then we have option two, which is like ask the chef.

A

But what does that mean uh in the context of software- and this means usually I'm going to talk to the vendors I'm going to go to the folks that are building my software and ask them what's in this, um but the tricky thing about that is in modern software. There is a lot of complexity in that it's not just one step.

A

You're asking it's a whole network of different chefs and these chefs can be in different countries in different organizations, may not want to tell you what their secret ingredient is and so on, um and what happens now is that these are the complexity, makes option two of asking the chef very difficult for s-bomb generation very much like, um and so what happens now is a lot of folks today are still generating s-bomb through option one through her sexual analysis and that's a good reason for this.

A

It's very usable and it provides a um a baseline of security that we can work with, and this ties back to the kind of ho question of balancing usability and security.

A

So in this case, what we want to do in this talk is to show you uh how we can maintain use of usability by the same time, kind of turn, the knob on security.

B

Nice, so my name is Chris. Phillips I'm also worked a little bit on this and what we're trying to drive home with the idea of software composition, analysis it's no longer on the consumer themselves. There's been this idea of like oh. If we shift left and all the developers scan these things that are on their computers and they do. Oh, the binary is fine, great, no vulnerabilities, they post a report, they upload it and it gets shipped out to production.

B

But what actually happens is if you have like a static asset, that's sitting on a container or on your machine. You have so much less context than if you did the build time of itself right, the the producer themselves. You can get manifest from the DB. You can go through all the other metadata of the build system of like the Jenkins pipeline. You can even query and look at every single bit or part of what went into building that binary before it gets tripped out and over. So the analogy here is just like the top.

B

This is Superman as the producer and by the time it gets to on my laptop I'm, just like staring at a rock at a rock trying to decode it. Compilation is obviously lossy in itself and it's always a losing battle. If I'm trying to do that as an individual developer.

B

Here's a good example: we just ran a package manager tool from open source against one which was a file, and we found 121 packages in that go binary. But then, when we stepped up a level and we scanned a directory as our context, we found 140 packages, so there were 19 or so packages at that directory scan level that might have been used to go into the build process.

B

So imagine you have like an intern and we're looking at the dependency list for the go module and someone who goes yeah. Is this everything that's you know been introduced in the pull request, um I just scanned what was downloaded so that seems good, and then you get the lgtm and you get to go home, everything's deployed to production, you're, fine, but there's always a an issue, there's more. If we actually look at the build logs for when we scan the directory, we found.

B

Oh, there was going excess at this commit Shaw and also this commits uh in a temporary, and there was one of this commit show. Did we scan those? Did we do any analysis against any of those?

B

It's in production? Isn't it and you have to show up on Monday, and you have realized that you're in a lot of trouble, because you didn't go deeper on the graph of analyzing how it was being produced.

B

So what we're going to show in this demo and kind of what the tools that we put together is we want to make metadata discoverable as producers of software rather than consumers, putting the owners on consumers to generate their own proof of things being vulnerable or not or secure, so producers and people who publish software get ready.

A

Awesome so um just to to set the stage for the example and the demo we're going through. uh What we have is a build of a rust binary.

A

Here we have our developer our Chef here, compiling something we call um a vulnerable demo binary, which shouldn't be vulnerable right, um and we do the thing that most of us want: Once Upon, a Time while we were learning Docker do, which is create a Docker file from Alpine or something run call into a binary and ship it right it's in production, um and so so what we see this has stayed up today right. So if we take uh two like shift and scan, this contain an image. It's going to tell us a few things.

A

It's going to say: cool you're using Alpine here are the list of Alpine packages that you're, using and by the way I found. This thing called vulnerable demo, have no clue what it is, but yeah I should be fine right.

A

um So I think the the question that we're trying to say here is like you know what, if I could go to the source and ask you know what's in this thing, um and so what we have here and uh alluding to what Chris uh mentioned about having producers um kind of share the metadata. And now we have the chef here on top of creating and compiling the binary.

A

uh They also run shift to then generate the s-bomb, and then we add the step of attesting the s-bomb and putting it in a discoverable stall and in this case, for a demo we're using six star and, more specifically, the recall, transparency lock. So it's taking the s-bomb, it's testing it, putting it in the store and now what shift then does is now when it sees the the binary is okay. Let me go check out.

A

If there's any metadata available, it's going to take the hash of the the binary and take the shot of the 56 and as recall, any metadata any as bombs that I can use. And in this case, because there is one recall returns with the s-bomb and then uh what safe now produces in the s-bomb uh includes and encapsulates the additional metadata right.

B

So we can put all these like nice fancy graphs on the screen, but everyone comes to keep calm because they want to see the code of how do we actually get to this final step right here uh now we're going to choose aside we've gone back to the old uh GIF, so everyone will get to see live step by step, how we would do this as a producer.

B

So the first part we talked about in that graph, build the artifact itself right, just a nightly build release and we get our binary tailed as time you could use make you could use, go build anything the next bit we do. Is we go in, and we say: okay I'm going to take this show posts I'm going to take this new thing that I build and I'm going to double check and see yeah thumbs up no vulnerabilities right now we're the consumer.

B

We are going through this whole process and we're saying my code is not vulnerable I'm going to ship this right now we're good right and then finally, we're going to generate an image and if you're, following along from the graph that Brandon put together, this is where we are where we build and we package that binary in the image we scan that as well, and then this gets uploaded to jfrog or Docker Hub and everyone's extremely happy because we're zero vulnerabilities. We could report that and we get a nice little pat on the back.

B

We have solved supply chain security.

B

That's incorrect, though, right. If we want to go we're going to go a bit deeper, let's try this with more Fidelity, so we're going to do the directory scan actually get the full contents that go into this rust binary that we're building we're going to get the shaw 256 of our binary, so we're going to go into that release, we're going to fingerprint it and then we're also going to fingerprint our s-bomb. So if you think about it, you say this artifact that I have right here and this s-bomb. These two fingerprints are linked together.

B

What that means is that when we go and check the Retro transparency log, if the pointer to whatever the blob store, is of that extra metadata document that you're uploading that s-bomb that vulnerability report, if that Shaw does not match the fingerprint of the thing you're attesting to bailout, you cannot verifiably use it.

B

For any reason, someone is trying to man in the middle you and there's no way for you to understand how or why that's correct data here is the code and I made sure that this one didn't play it for a little bit, so you can watch it go all the way through, but there's a binary that s uh called s, mom attached that Brandon wrote and what this does is this interacts correctly with Sig store, we build the predicate type which is in this case Google s bomb.

B

You can that's how we know when we go out to pull down the extra metadata that it is the type that we want to decode and scan for vulnerabilities. We include the subjects of the binary itself, so that's us committing to the transparency log that this file, this exact fingerprint exists for this metadata and and then we upload it and then sign it with whatever method that we want keyless wise, so in this case I'm using my oidc provider for GitHub, but you could think of all different trust policies.

B

This way where a local uh I would say like the Golden Goose keys of your company is signing it. So you only trust things in Sig store that were signed by those keys right, not just a random developer, Chris Phillips at GitHub right, the levels of trust and policy that can do on this are kind of like nested, where you have that that trust policy problem and then we're if you're, following along in the the diagram again, you are here so now we're uploading this additional metadata.

B

So now we go back and we say: did this work? Do we have a better image now, with this fingerprint uploaded the six door when we go to catalog and when we go to actually look at everything? That's in this image we will be greeted with in this case this is human readable, but you can also see it as an annotation machine, readable, uid, and this says: hey I fingerprinted- that file that previously I had no idea what it was and some person. That's in your config that you trust, says thumbs up.

B

That thing has additional metadata. They have generated as a producer. They have put up there and uploaded for you for vulnerability analysis. So if we pull that down, we use our new superpowers.

B

We're going up to record with this command we're using the uid, we're finding the attestation we're pulling the predicate and then we're scanning it right here, and we can say that, because the producer has been responsible because the producer has uploaded a document, that's more complete based on all of the metadata and all the information that they had in the build system. We can alert the consumer that there might be vulnerabilities that there might be something that otherwise was just a black rock of a compiled file before and you think about this.

B

Imagine if you went and built a system where something hadn't been fingerprinted and you're just blindly scanning no vulnerabilities forever and then all of a sudden you go, and you start making these correct attestations to the thing that the producer is putting out there right all of a sudden, because it's in recore- and you say you trust it- your vulnerability reports can start growing where you can see that new metadata, that's being populated by the producers that are, you, know, building correct s-bombs in this case.

B

This is, in contrast to obviously the very sad and very localized example that we did earlier, where we as a developer, were just trying our best to analyze the thing there and.

A

This is basically like running the exact same command with different results. Yeah.

B

Yeah, and so everyone loves memes, they said there's no way, you can see those vulnerabilities in that compiled file you're right, but I can trust the person. That's uploading, the data that knows what's in there from their build system, with all the additional details and then get the vulnerabilities and like the world is your oyster too. You can pull license data from that as well. Any kind of compliance check that you think that you can assert data on from those build systems. You can upload and run it into an extra tool.

B

They're. Obviously, caveats with great power comes great responsibility, so Brandon, please yeah.

A

We did a mixing with the Spiderman now um so one thing that we're using here for the discovery of the metadata, we're actually using the recall, experimental, Association search function.

A

We are building another project called guac that we'll be talking about tomorrow, that we really focus on cataloging all this metadata, so that it's searchable, so you can search it by contact addressable hash um and we're having a talk about this tomorrow, 11 A.M in the same track, the security track. So if you're interested find out more about that there, um another quick note is Association format. We're using is a little bit custom, not exactly the same as the Intel spdx1.

A

This is to keep it small enough, so that recall was installed at the stations and finally, as Chris pointed out, you know there is a lot of things you can do with trust policy. You know it's a list of people that I trust for certain artifacts. Are there like organizations uh like Google, Microsoft or Intel that trust for the binaries that they produced?

A

So the couple open issues still in discussion um and I believe this is all Upstream in Maine, so you can just play with this today. If you just run Swift on your containers,.

B

Cool, oh yeah, and the last part too that I did want to mention is that um we do want to start thinking about and getting ideas from the community about how the parts like multiple entries from third and first parties and how to do that kind of decisioning against who you like. Who is the first order of like which one do you take uh when you're evaluating right?

B

So if the producer themselves is always favored or if you know that, let's say red, hat or Google are doing independent analysis of things right and you want to take theirs instead of the producer, because you've had you know some run-ins or some more vulnerable things that have been LED on that you can just lift that in rather than the first party producer themselves.

A

Yeah, so in conclusion, um I think we, this is kind of a shout out to be like you know, everyone should be producing software at uh metadata and then testing it and making it discoverable.

A

um We see that you know it doesn't um we can kind of work at this problem from both angles, both from um asking more about metadata, but also from the producer side, and making this basically transparent and seamless and maintain usability of s-bomb generation, but having better s-bombs.

B

Check out our tools and check out all the awesome open source tools that exist that produce these documents as well. um It's it's awesome, just there's so much going on in this ecosystem right now and the more that we get everything kind of working together rather than just granular insular and one single project, uh the better it will be for both the consumer of software and hopefully, producers uh in that guard. So thank you.

C

Okay, we have time for questions and I see one return.

C

Hey this is really uh really interesting. Stuff um I I just have a question about, uh though, so you have a bunch of problems with the permission model and stuff like this, and and also I'm curious, why you use the attestation format in in Toto, because link formats in in Toto, actually in in the layout formats, actually handle all those permission. Issues for you and things like this, so I'm curious.

C

What uh why not just using total links so.

A

So we are using um that I think the um that is like the expressibility in the the locator of the metadata. So um we had this restriction where uh recall, doesn't store, attestations above I can't remember a couple of kilobytes or something um so we do need a way to start that we are using the in total attestation we're just having a custom predicate for that.

C

Okay, but but the link, uh because there's different formats for attestation versus Link format, did you look at the link format also or not.

C

It's okay: we can take.

A

C

I'm not I'm not trying to nail you to the wall here or anything right.

C

uh Maybe I didn't really get it. What kind of format do you use for your response? So oh.

B

Yeah, that's a great question.

A

um Yeah so um and I think in this demo we use the spdx Json format, but essentially um you could use basically anything because you basically say here's an additional Melody that I can go download it. You can take a look at it.

A

um This does lead to another thing that we kind of hinted at over here, but didn't talk about explicitly, which is S1 composability So within individual ecosystems. It's generally pretty simple, as we saw like with the golden one, you can have a good idea on what everything looks like within your ecosystem, but then, when you pull for other ecosystems like for the rust and the continuous one, this one you kind of need to put them together.

A

So in this case the metadata we can gather for any kind of response or any kind of format with the second DX or SPX. There is a aspect of composability which, for example, in spdx. What we do is we use the describe relationship to say that this binary is described by this particular s-bomb, which is in a separate document. So that is like a little bit of nuance in there, but in general, any format. Any type of metadata should be able to be handled as long as you. You know what you're expecting.

B

C

Any more questions.

C

Okay, so um there's there's a couple ways to go about generating something like an s-bomb and one of them is scanning and another way is to pre-populate this. Somehow, basically, you can initially do a scan and then have this be like the set of rules for the things that are supposed to come in and I I've heard a lot of discussion about how scanning as a whole has a bunch of problems with especially malicious actors. If there's a malicious actor anywhere in your system, then scanning is just not gonna.

C

Save you and I'm wondering if you can just comment on that and where you think this might go to move to a model, that's where your work might go to move to a model, that's secure against malicious actors, not just you know, Oddities of scanning tools, not picking things up. Do you want to take this off.

B

Yeah, it depends on what the malicious actor is doing. So in the case of like if they're modifying your APK database right, they're looking at different entries- and you take the sum of that and you're using those digests and fingerprinting as much as you can and you're diffing against the different outputs of s-bombs. Hopefully, the ecosystem evolves so that we can see those different diffs between those documents and say: oh, the fingerprint here changed from APK to APK. Those are not the same bomb out.

B

We need to investigate why this wasn't the same for our quote: reproducible build if the that's the that's the main use case. I've looked at as far as like a malicious actor changing data on the system to represent that some version is not there or that some like, let's say, you're, doing a scanning tool and you're not actually reading the file you're pulling a bunch of package management info or metadata.json and you're, not asserting that that thing actually exists there.

B

You can go a step further and compare that metadata that package.json that package lock to the files read the file itself. Does that match up exactly with what exists there if that's been changed bomb out a certain show to the user that something different has happened at this level, and you can take that localized example and you can go even further up.

B

So if we're talking about you know uploading these, uh like s-bombs to some blob store that could be discovered via stick store, then when you go and you pull that down, and the company has said this is reproducible, this is the exact same and you diff it against a previous version that you've pulled. Then you say: hey. You said this was reproducible. You said this is exactly the same. Why am I seeing like a git diff of a red minus and a green plus here, where it wouldn't have existed? So if there are other?

B

But what like, whether we can talk more offline about other militias acting kind of concepts of what they could do, because that is that's awesome to think in that space, yeah.

A

And also to add on to that right, I think that um s-bomb is, you know, concentrating on figuring out what and like the process of trying to menderize it I think there are other aspects that we want to tie into so like having the provincial S1, but also having that of you, know the South Side associations and kind of figuring out. You know whether there's a bad actor there's a bad identity which documents are affected and which s-bomb generation processes will affected very much with like a regular supply chain, attack and I.

A

Think, like um that's kind of like what we're trying to solve, with like the block project as well right is that now we have the S1 information. Let's let the S1 people like worry about their problem. We will solve other aspects of this with salsa with developer insights. With you know, tracking, like oh, was this a malicious developer. The social malicious comment propagating that through the transitive graph and then doing analysis of that.

C

All right wave at me in the back, if you have more questions.

B

One more shout out for the Glock talk coming up as well.

B

Okay, if no more questions give a big round of applause.