Cloud Native Computing Foundation CloudNativeSecurityCon 2023 - Seattle, 2 Feb 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: Fighting The Next War - Future Threats to OSS and Software Supply Chain... Brian Behlendorf

Description

Don't miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from 18 - 21 April, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Keynote: Fighting The Next War - Future Threats to OSS and Software Supply Chain Security - Brian Behlendorf, Managing Director, Open Source Security Foundation

Buffer overflows, typo-squatting, leaked credentials - many of the biggest problems in securing software today are the same greatest-hits since the 1990s. More or less once a year we see a novel kind of security attack, taking advantage of some new centralized service, a weakness we incorrectly assumed could not be exploited, or a new IT advancement that changes everything. As a keynote speech given at a 2023 Q1 conference, we are now legally required to mention ChatGPT, but ignoring the hype, the prospect of AI enabling uncanny spearfishing or automating mass pull requests with backdoors seems much less sci-fi today than it would have a year ago. What other new kinds of attacks could emerge, and what should OSS projects do to prepare?

A

Thank you very much.

A

uh It's so exciting to be here, uh there's so much gratitude uh and and and and really debt that we owe to the cloud native Community uh at the open source, security, Foundation uh and uh just really happy to be talking with you about a little bit about what we're doing, but but I actually wanted to start out a bit with um the kinds of things that we want to be thinking about next, because so often we're uh fixing security rules today that that reflect uh attacks that uh have been kind of pervasive on the internet since since uh time immemorial, uh but but I want to try to set a context for how do we prepare for things that are coming up right?

A

uh And so pardon me if this seems like a little bit of a late night brainstorm using chat, GPT and Google image search, but uh hopefully you'll. Forgive me. So um you know in the early days- and you can tell this- is a slide about the past, because there's a CRT on the on the uh the counter there. uh You know we you know.

A

Does anyone remember telnet I, don't think anyone born after 1980 uh has ever typed the letters telnet unless you were like doing something really weird um but like I wrote a whole like chat system where telnet was the client, it was awesome right um and you could, you know, go from one machine to another. You could pretend to be a mail server.

A

You know that kind of thing, but um we kind of just took for granted that the systems administrators and the network admins and others weren't going to be sniffing our packets because, like why bother well there's nothing interesting going over uh this traffic anyways uh SMTP uh was not secure by default when it first came out right uh all these different protocols, kind of uh we we kind of, took for granted that we could trust a certain layer of the internet or the people that ran certain layers below us, so that we could move more quickly so that we could like build the kinds of things we wanted to build, but those kinds of decisions left uh uh to to Future, Generations uh kind of patching up those assumptions and defaults and biases right.

A

It was great that we were able to move quickly, but I I, the internet was kind of like didn't, have security by default uh for a long time, and arguably it still doesn't either right, but it took us us a while now to our to in our defense, one of the things that kept us from pervasively, adding uh public key cryptography to all of our network connections, and the like was the fact that public key cryptography was considered a munition by the US government.

A

It was considered a weapon, a dual use technology that was regulated and the only way that you could transport that you could send a a software package that talked uh public key cryptography protocols that used more than 40 bits of entropy uh was to do that was to limit it to 40 bits or 128. If you got special probes or whatever that meant you couldn't ship source code, you could not, uh because source code meant you could use arbitrary length Keys.

A

Now, in 99 there was a court case that was decided called The Bernstein case that the Apache software Foundation submitted an amicus brief to that actually stipulated that software development is a uh as a function of speech rather than a function of Commerce and that researchers at the very least needed the ability to talk about Crypt photography publicly. It also helped that people took RSA and reduced it to like four lines of Pearl and printed it on the back of t-shirts and wore them as they crossed borders.

A

That was kind of fun, but we made this case and we won, and we won the argument that uh uh that basically public key cryptography should be widely available to help uh help us bootstrap and and get moved Beyond. This default bias uh that you know we we couldn't have cryptography on the the lower levels of the internet, um uh and that was that was pretty useful.

A

That was a pretty nice thing to be able to assume, uh but uh that only uh Stripped Away one layer of the onion and we realized there were a whole lot more places where we had some default assumptions that that would later come to bite. Us in the ass now this is Dan Kaminsky uh uh who passed away. Sadly, a few years ago, Dan um is probably the only person that I would ever credit with having saved the internet. Just like unquestionably Dan discovered a bug in bind the now.

A

It was more of a protocol bug in the DNS protocols. uh Frankly, that allowed for something called DNS cache poisoning, which meant preemptively an attacker could tell a DNS server. You know, google.com doesn't actually go to one IP address. It goes to another, and that meant that you know, even with all the other work that we did yeah.

A

uh If you could uh corruptively, you know corrupt all these DNS caches out there, you could direct traffic wherever you wanted uh and and use that to steal credentials, use that to steal other kinds of traffic and the problem was.

A

It was such a trivial trivially, easy bug to exploit that if now I'm- a big believer in security through transparency in trying to be public about about flaws, and that kind of thing so that as a means to get them fixed faster, but he kind of realized as a bunch of people did when he started to talk with him that the more this was would be known before it was fixed, the more likely that the entire internet would really come down. I mean if you bring down DNS. uh Where do you else?

A

Do you bootstrap from I mean we would have had to use Etsy host files uh with IP addresses shared over Billboards, or something like that in order to bootstrap back up and get to a usable internet again, and so uh he worked very stealthily, but but very determinantly with a whole set of the main DNS admins, the back, uh the uh the the the Upstream kind of a root, DNS server, sorry uh and others to get patches to software to get protocol changes where necessary to basically get to the point where DNS responses were verifiable uh and eliminated.

A

That bug and it had to be quiet, but it got out there and it got updated, and so we, you know the transport layer, security, DNS, layer, security, there's all these places where there's this technical debt that we accumulate as uh as developers and as protocol designers and as people out there in the world um building upon it. Now, let's, let's give ourselves a little bit of credit here. Every project bootstraps by taking a few things for granted. Otherwise I mean you can't boil the ocean.

A

You can't solve The World's problems, you have to start somewhere. You have to to decide where who you're going to trust right? You have to decide uh how you're going to to get to rough consensus and running code quickly enough before your boss cancels your project or your grant. Money runs out or other things before you're able to prove hey, we've got something, but um but those decisions, those biases, create technical debt and often those assumptions and biases are never well documented. They're, not issues in an issue.

A

Tracker they're not left, as as things to do, they're left as things for people to take up and and try to fix after the fact, an example of this is there's a company called trellix uh that I announced last week two weeks ago. It was the January 23rd that they had automatically submitted, pull requests against 61 000, vulnerable, open source projects that had adopted a c python, actually module called tar file that had a cve from 2007 in it that was not fixed because the tar file maintainer is actually the C python.

A

Maintainers contend that to fix fix it in a way that solved. This problem would break posix compliance okay, so this debate from like 2007, but because it was unresolved because the the the the issue on it.

A

You know the comments go back 10 years, it's hilarious uh uh because these weren't 15 years- sorry- because this wasn't fixed thousands of projects picked this up and have used it in an unsafe way without validating inputs uh to make sure that expanding a tar file didn't overwrite Etsy password, uh and uh because of that, uh it was easy to scan easy to find these projects that were using in an unsafe way, easy even to come up with fixes for that and submit pull requests that fix it now, trellis I think has done those 61 000 projects, a favor but I think they're going to be lucky if 15 of those projects even respond based on the work that folks, like Jonathan Leach, who gave a talk about automated pull, requests and and fixing bugs at scale at Defcon last year.

A

uh His experiences and doing the same kinds of things identifying patterns identifying old, bugs that need to go, go back and be fixed. It is extraordinarily hard and, frankly, for good reason: maintainers hate it. When you show up and say my scanning tool found a bunch of like worry some things in your code right, so we've got to figure out how to get better at doing this and and how to go back and fix some of these low-level assumptions that are being made now.

A

Thank you, uh Priyanka, for mentioning Sig store, thank you to all of you who are using Sig store, who are helping develop it, uh who are are pushing it forward and getting and helping establish it as a standard for signing artifacts in the software supply chain. But, frankly, signatures on on objects, as they flow through the chain and into our dependencies, should have been a problem we solved in 2005, 2007 and, and we solved it somewhat.

A

Most people saw it at The, Last Mile with you, know, signatures, and you know apt-get and that kind of thing, but further up the chain. We kind of took for granted that if something was on an https website a module there claiming to be a build of source code over here at this repo, then we could trust it right and those trust that again that default assumption that default trust is really dangerous to have now what are some of the bad patterns? We're still persisting today.

A

I know some of you just want to see the world burn, but when you tell your users, you know curl, you know URL pipe to pseudo sh, you're kind of doing the same thing, I mean your argument might be well. Apt-Get is just the same thing. You can run a post, install script that that you know compromises systems and everything should be in containers anyways right, but that's cynicism about uh you know being able to build better resilient.

A

More resilient systems is kind of out there, I mean even a dark pattern like choosing which package to use based on the number of stars on GitHub uh is. Is worrisome right like like? There should be other data you use to figure out uh which packages to use and which to build, because there's this thing called time that works against us in so many ways um time- and these are just some general principles again two in the morning- brainstorm apologize for that.

A

But there is really no such thing as true: zero trust architecture, there's defaults assumptions biases and things we build on top of for better and For Worse that we have to take for granted right. You can take for granted that you know if you took the subway here or or a taxi here that it's you know the the wheels don't come off at the wrong time. You know- and it might once in a billion times, but but you're able to take for granted that that system works right.

A

So it's not a flaw that we take things for granted, but we just have to continually interrogate the the attack surface on the trust decisions that we make right attacks we think are hard to exploit also get easier over time. This is kind of the chat, GPT thing as well right. This is the hey. It used to be hard to write a lot of meaningless texts that looked good uh kind of it's mansplaining as a service.

A

Is the term I like to use, but um it's it's it's it, but now it's actually trivially easy to do that and even to script that into processes. My other favorite example of this is side.

A

Channel attacks like the fact that you could try to Divine a private key based on the timing of a CPU, that's generating that like what the that's magic um but uh but the second principle, and the second way the time works against us is that things that we think are norms, things that we think people would be embarrassed by like who would possibly tell that to Port 25 and send mail from president whitehouse.gov while I did that as a college freshman.

A

Somebody will eventually do that and if your theory of stability rests upon social norms, that's fragile! That's super fragile! When you get to the number of users Beyond, you know, dunbar's number and personalities stop being a useful check on on security, and then finally, software deserves to have should have an expiration date. It really you know there should we should assume software falls apart, that software is suffers from bit rod in unusual ways um that you know you know how depreciation works.

A

You get to write off a computer asset over three years or five years, whatever well I think you should write off a software. That's been written and past three or five years take it from like a positive asset to a liability right that you try to get off your books, because we have no other great way to measure technical debt except hey this. This stuff is really all and I don't care if it still works, it's still, it's still a risk. So putting these things together and I swear chat.

A

Gpt did not require me to put this a slide in here. It's I I came up with this all on my own, um but let's think about ways that new tools like Chad gbt and AI Tools in general, might generate new kinds of attacks on these. These things, these these things that we take for granted I mean first off automating spear phishing attacks on open source projects themselves using AI generated.

A

You know uh uh pull requests or comments and threads, or that kind of thing has gone a lot from being like that's sci-fi, that's Corey, doctoral territory, two! No! No! This is going to happen this year, I'm sure, absolutely along similar lines. I I I'll come back to the corrupted AI models in a bit AI generated, pull requests that are just right and just wrong enough to DDOS teams, Upstream teams to slow or stop the reaction when a real zero day is reported would be a really big risk.

A

Ai generated contributors who make automated like Dock, cleanups and things like that and then build up a repute station in your project. Maybe you even give them admin proves right. uh That's that's that that is a risk as well, uh and then we know that AI models can be corrupted.

A

This is a picture of a stop sign that some researchers were able to convince an AI to recognize as a speed limit, 30 sign, uh which is a very different thing than intended and think about ways that you might corrupt things like copilot to make its recommendations lead to insecure software.

A

um A second attack service that I think is emergent, is an identity and centrality we've not solved the internet identity problem this by the way is the other Brian bellendorf he's a maintainer on opencfs.

A

He gets our name and the Linux kernel change logs he's pretty cool uh he's, uh we've met, we've had pizza uh he's a much better programmer than I ever have been or will be, but you know, if you see my name, Brian bellendorf I know because I get a lot of support, queries for open ZFS that I forward on to the other Brian bellendorf, and it kind of freaks people out. um But uh uh this kind of thing is real.

A

uh You know the prospect of somebody showing up pretending to be somebody else else and being very convincing at it. I think would be, would be a real big factor, there's a bunch of others, but the the one I want to focus. That's kind of you know related to this is there's often these these things that we take for granted like the accessibility of an npm package uh repository or apache.org and resolving dtds uh these kinds of things that we have to assume in this world.

A

One in a couple of you know orders of magnitude times uh those systems will be gone. We won't be able to access it. Your DNS will fail, uh and do you really want to limit your production push to production, because some non-profits website is down I kind of don't think so. The third one I'd worry about are free writers and Regulation, and this is kind of more nuanced than I. Think I can get out in the 20 seconds.

A

I have left, but um as maintainers we kind of take for granted that users are on their own and can be self-sufficient, and often we try to think about them. The better ones of us do you think about how to pull people in and cross that bridge, but users also free ride on the assumption that as maintainers, we take security issues for granted and and the number of reported security vulnerabilities that don't get responded to by maintainers in a reasonable amount of time.

A

I mean nobody's formally tracking this and frankly, we probably should and I think in between the two lies some risk and that's risk that that can be exploited by grifters and scammers uh uh and and the like that I'm not quite sure exactly what those attacks are going to be, but they're going to happen. The second category of this that I really want to highlight is there are moves afoot at a regulatory level to respond to some of the free writer problems.

A

Some of these issues by putting more liability upon all of you as developers, the European Union's cyber resiliency act, has a whole bunch of worrisome kind of requirements that might fall on you if your software happens to be unlucky enough to be used in a European nuclear plant or or by a transportation agency. This is where a cement it could get us back to the bad old days of having to remember what country you're in to depend upon whether I can ship code to you or not.

A

um The second wrong answer is to make developers personally liable for the use of their software for illegal purposes, uh read up more on the tornado developer and what's happening there. These are not ways to solve the cyber security problem, but some of the tools will make the problems inherent in this so much more severe. So at openssf, we've got a lot of activities going on we're kind of a circus.

A

It's really great lots of different ways to tackle this, but we've organized ourselves around working groups in a thematic way that uh we think will prepare us to think holistically about many of these challenges to think about vulnerability, disclosure in a long-term way like what not just how do we? How do we write a tool to solve one problem or come up with a spec to solve another but think holistically about the whole supply chain challenges? How do we think about a digital identity and uh supply chain Integrity?

A

How do we think about uh uh about all these different kinds of new forms of attack? We don't have a working group on AI, yet maybe chat gbt will start one for us, but I I. It's it's I, think we're in a position to be well prepared to be thinking about this next wave of attacks and I just want to thank all of you again.

A

uh Open ssf happened because, because the cloud native Community said, hey, there's a bunch of tools, we should be building and uh and brought those to the party and shipped code and shipping code is really all that matters, uh and and for that we have a debt of gratitude to many of you in this room and, frankly, to all of you who are using these tools now to validate your work. So thank you very much and have a great conference.