GitHub GitHub Satellite 2020 - Work, 7 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Securing the software supply chain together - GitHub Satellite 2020

Description

Presented by Maya Kaczorowski, Product Manager, Software Supply Chain Security, GitHub

Writing secure code is hard in its own right, but understanding what vulnerabilities exist in your code— and how to keep up to date with the latest patches—is daunting for even the most sophisticated software teams. In this session, you'll learn how GitHub is making it easier to secure your software supply chain, and how to get started in protecting your code and its dependencies.

GitHub Satellite: A community connected by code

On May 6th, we threw a free virtual event featuring developers working together on the world’s software, announcements from the GitHub team, and inspiring performances by artists who code.

More information: https://githubsatellite.com
Schedule: https://githubsatellite.com/schedule/

A

And I love the way like because when you talk about devops and in a source, they often both come down to like just as much or even more so the cultural shift as opposed to a technological shift. So I think she like absolutely narrowed it on the head and just agrees.

B

A

I have an image now of like all these, like it, companies are like little people walking around like flags like revolution,.

B

Well, this would give up something like this.

A

We leave revolutions at github satellite, but enough about leading.

B

A

We've got some really cool things happening for the rest of, like I think we've got maybe a couple hours left. Oh my gosh, it's going so quick. I.

B

Know so, would you like to.

A

Give us four more sessions. Would you like to give us a quick taster of what those four sessions are going to look like.

B

And just a little bit more yes, after our next session, we actually have becca zanstein coming on to talk about collaboration beyond the code and after that, we're having brian douglas always known as b dougie, to give top 10 tips for project maintainers. So we're not done we're not done it's still going.

A

I'm pretty excited so much more coming all the exciting stuff.

B

Yes, so, and I want to make sure everybody remembers- uh go to getupsatelite.com discussions, that's where you can ask questions with denise right now for the next 30 minutes and for all of our speakers that are coming up, but speaking of coming up mitch. Do you mind if I introduce our next speaker.

A

uh You can do the honors go for it. Thank you. So very much.

B

I love doing this. Our next speaker has a background in cryptography and game theory and has this magical skill to bake without following a recipe, and it's still doing well. This person is a product manager here at github, and the name of her talk is securing the software supply chain together. Please help me introduce maya.

B

C

Awesome, thank you so much so today um welcome to gift up satellite, first of all so happy that you could join us virtually. um My name is maya khacharovsky, as we just introduced, and I'm a product manager working on software supply chain. Security at github today we'll be talking about what we can do together to secure the software supply chain.

C

In our short time, we'll cover what a software supply chain is and why you should care about securing it. How github helps you secure your software supply chain and what you can do as a developer, maintainer or researcher, and give you a quick summary of what you can do, starting today, so first off, what's a supply chain? Well, it's anything that you depend on to deliver your end product for a pharmaceutical company.

C

It's the provenance and the verification of ingredients that go into each drug to enjoy high quality for a chocolate bar that you buy at the store, it's the list of ingredients and some information on nutritional content, maybe organic ingredients and maybe production facilities. So then, what's a software supply chain? Well, it's everything that goes into your software and where it comes from who wrote it when it was contributed, how it's been reviewed for security issues, any known vulnerabilities, supported versions, license information, if applicable, everything, everything and anything.

C

It's everything that you could need or could want to know about. The software that you're running to help you determine your risk profile. So a software supply chain, then, is anything that goes into or affects your code from development through your ci cd pipeline until it gets deployed into production.

C

Why does a software supply chain matter as a concept? Well, software dependencies are pervasive. It's normal for your projects to use hundreds or even thousands, of open source dependencies that you didn't write yourself.

C

Industry data suggests that anywhere from 85 to 97 percent of enterprise code bases come from open source, but those are things that you depend on in your software and if one of those dependencies has a vulnerability, then chances are that you have a vulnerability as well being able to leverage the work of thousands of open source developers means that thousands of strangers effectively have commit access to your production code. So an innocent mistake or a malicious attack to your supply chain can affect you deeply.

C

Now, when we talk about your dependencies, what's scary, isn't as much your internal dependencies that you develop, because, hopefully you have controls in place to say prevent developers from pushing code from their bathtub or are you restricted to a super admin rights? What is scary, though, are your third party dependencies externally, like open source code, that your developers pull in, because although open source is easy for developers, that means that it's also easy for attackers, and I don't want to scare you. I want you to keep using open source code.

C

I want you to keep contributing to open source code. I just want you to do it thoughtfully from a security point of view, so what kinds of software supply chain vulnerabilities? Are there and do attacks actually happen? Let's start with the attacks first, because that always comes up so unfortunately supply chain compromises are real. We've seen quite a few attacks in recent years across many different ecosystems.

C

There are several different methods to attack your supply chain, inserting malicious code directly into dependency, taking over a maintainer account or a project compromising a build tool, signing keys or distribution tool or even typo spotting, which is when an attacker creates a malicious package with a very similar name to a known package, hoping that users accidentally download it all of these supply chain. Compromises are occurring in the wild to highlight a few examples that you might be familiar with.

C

First gen 2 on june 28, 2018 maintainers of the gen 2 linux distribution were notified, that they had been removed from gen 2's github organization. An attacker had used credential stuffing, which is reusing passwords from another site somewhere else to gain access to an administrator's account then created a dummy admin account and removed. Other admins they had also added or attempted to add malicious code to several repos to rmrf, so that users running the code would delete their data accidentally.

C

Thankfully, this was noticed by an admin within a matter of minutes due to github email alerts and within an hour, gentoo was working with github support on the issue. The attacker had tried to make six commits in a span of four hours, but these were all reverted and it's not suspected that any end users ran the compromised code.

C

Another example event stream that you're probably familiar with in the fall of 2018, the user volunteered to take over the event stream project on github, though it has more than two million downloads like open source projects, it could be better supported, and so the author welcomed the extra help and even gave the newcomer publishing rights. I want to emphasize that the author is not to blame here, because we should have better ways to support these developers and also verify trust of our ecosystem.

C

So what happened next well on september, 9th 2018, the new user added a new dependency to event stream flat map stream and then rewrote the code about a week later to actually no longer require that dependency.

C

That removal was made in the code base, but it wasn't pushed through to where the library is hosted on npm, then, on october 5th, another user added malware to flatmap stream. So then anyone who built or used event stream and pulled in the latest version of flatmap stream would receive this malware. The malware was very specific: it targeted developers of a specific cryptocurrency wallet, the dash copay wallet to try to insert malicious code in the wallet itself.

C

This was discovered almost two months later and unfortunately, the malware did make it into a handful of releases of dash copay before being remediated, and before you ask no, I have no idea what was going on the summer of 2018.. I don't know why there were so many attacks there, but almost all of the time. The aims of these attacks is the same. Like other security attacks, it's almost always about money, that's usually malware to mine.

C

Cryptocurrency say or as part of about as part of a botnet, but it might also be a backdoor to allow for future malware or data exfiltration. If the package ends up in a particularly you know interesting juicy place in some cases, an attacker could also be aiming just to disrupt the service and take it offline.

C

Now I often see security vendors trying to sell their products and viewpoints with fear the sky is falling and you need this new security tool. That's really not what we're trying to do here. I want to talk about attacks, because I know that I'm going to get asked about attacks, but I do not want to be fear-mongering we're talking about the attacks to level set and educate these attacks happen, but they are rare. We are working to help make the whole supply chain more secure.

C

So, although we see these compromises occur in the wild and we do need to address them, the reality is that most of our concerns with the software supply chain is just forgetting to patch your code.

C

The issue in supply chain security is unpatched software. It's not open source use and it's not software supply chain compromises. Think about it. This way, if a component in your code base is open source- and it's likely that quite a bit is as we talked about earlier, then if any of those dependencies have vulnerabilities, you are potentially affected.

C

How can you address that issue? Thankfully, maintainers are pretty good at well. You know maintaining it's estimated that 85 of vulnerabilities in open source are disclosed with a patch already available. It's your job to deploy that patch. What you want to be really really good at is knowing when there's a new vulnerability and patching it.

C

At the same time, developers and security teams today find this hard, and I would go even further. 52 percent of developers find it painful to update vulnerable components. So that's what I want to focus on today. This is something that should be easy. This is automatable and it's something that we can make easy so now that we understand where the issues are making it easy for you to address known vulnerabilities and also dealing with those rare compromises.

C

Let me paint the picture for you of what github is trying to do here. Our goal is for github to lead and enable developers, maintainers enterprises and researchers in securing the world software.

C

We want to make consumption of open source software, something that all developers can do with confidence so that they have more faith in the security of open source code than in their own code. We want to make security of open source software as automated as possible so that the community can scale to secure all software.

C

That means a couple of things. First, it means that ensuring that the open source community has the tools they need to securely develop and maintain open source projects. That means automation. It means not needing maintainers to invest time, which they don't have into patch management, vulnerability, reporting cve issuance. We want to help you second, it meeters enterprises and developers to provide and utilize information to decide which dependencies they want to use in their supply chain and how they want to consume these in order to minimize risk.

C

That means knowing what you are consuming and having automated processes to use this information.

C

Reading the label and then acting on it and thirdly, it means empowering enterprises to understand and control what code binaries and packages they use based on their security, privacy, licensing and audit requirements, letting you make decisions that work for your business and with your requirements and that's where we're headed. That's how we're going to tackle the issues that we see in software supply chain today and we're just a couple of steps along that journey right now, so in delivering on that vision, we need to make sure that our tools work for all users.

C

That means making sure github software supply chain. Security tools are great for a few specific sets of people developers, and that includes enterprise, security teams, maintainers and security researchers.

C

Let's talk about developers first, because well, github puts developers first as a developer. You want to know what dependencies you use know about vulnerabilities in those dependencies, patch them and then get back to work. When you hear about a new volume on hacker news or twitter you're like damn, I didn't want to be doing that today, I'm busy!

C

I have to go feed my sourdough starter later, so how does github help you well first to know what dependencies you have use dependency graph dependency graph identifies all upstream dependencies and public downstream dependents of a repository or package. You can see your project's dependencies and some other properties like vulnerability. Information to generate this information dependency graph needs read-only access to a repository to access information on explicit declared dependencies, which is a standard format for each ecosystem.

C

For example, package.json files for npm, we parse all known package, manifest files in a user's repository and then use this to construct a graph with known dependency, names and versions. Items are added to dependency graph. Basically, when you add a new dependency, that is when you push a change to the default branch that changes the manifest file for that repo dependency graph is on for public repos by default, but must be enabled for private repos. If you enable it for private repos, then the owner of any of your dependencies doesn't see that info.

C

It's one-way visibility and, if you're using github enterprise edition, you also have access to dependency insights, where your organization can view the summary of their dependencies and vulnerabilities across repositories.

C

So now that you have information about your dependencies, you have time to feed your sourdough starter, that's great, but tonight was a big night. I was going to cut my hair, so how do I know if I actually need to apply a patch?

C

Well, once you know about your dependencies, you want to know when they have new vulnerabilities. You can do this with security alerts for vulnerable dependencies you'll be alerted when your repo has a newly discovered vulnerability to do. This, github compares the information in the dependency graph to the information in a github's advisory database which I'll talk about in a few minutes.

C

A security alert can either be sent to uh sorry. It can either be sent when any push happens to the default branch that contains a manifest file. That is, you added a new dependency, and so we check any vulnerabilities in that dependency or when a new vulnerability record is added to the database. That is a new vulnerability. Is discovered so we alert any repositories that are vulnerable.

C

The alert is sent to repo owners by default. This takes into account versions. So if you're running a newer version of a dependency, that's already been patched, then we won't bother you.

C

Similarly, security alerts are on by default for public repos, but you need to opt in and enable these for private repos, and with that you know about your dependencies and you know that they're vulnerable. So you can take a break tonight and cut your hair, but wait you still need to actually apply the patch I mean as a developer. I actually have to code. This is terrible. I was going to watch tiger king tonight.

C

Thankfully we made this easy for you too. The pentabot will send you a pull request to update a dependency to the minimum version that resolves a known vulnerability. This is done automatically based on known, vulnerabilities and known patched versions. Dependable security updates create a pull request to update package dependencies in a lock file by proposing a change to the lock file.

C

When I say the minimum change, I mean that it's the first version with the patch, because a small change is easier for you to review and easier to merge, and we want you to keep moving forward. In fact, once you turn on dependency graph and security alerts, then dependable security updates are automatically enabled easy, and that really is so easy. You can review it.

C

You know review the pull request, check the test results or your cicd results that you have for that pull request and then make the fix and then get back to tiger king, you, cool cats and kittens and we've seen a lot of you use this since dependable security updates were integrated into github natively we've generated and seen. You merge a ton of pr's over 750 000 pr's, in fact, and over 4 million total, including dependable preview, we're glad that we could make patching easier for you.

C

We've also seen anecdotally when we had an error in generating dependable prs, which thankfully is now fixed that when you receive a pr you're, almost twice as likely to patch it within a day and one and a half times more likely to patch within a week than when you don't receive a pr, and that's super critical in fact, for repos that have dependable security updates, enabled since ga the mean time to repair mttr is only 40 days now.

C

That sounds you know good or bad. How do I know? Well, fortunately- or unfortunately, I can't compare this to repos without these features enabled on github, because we respect your privacy and if you don't turn the feature on, then we don't know about your vulnerabilities. So I have to compare to industry metrics. The industry says that this currently takes months. Sonotype says that mttr for time from von disclosure to patch in a project is 180 days and sneak finds this for open source projects to be more than two years.

C

Getting your repo down to just 40 days is a huge improvement. Unfortunately, today we can't always generate a pr for you and we're working on this. We're continuing to invest here in the ecosystem coverage for alerts and prs and making it easier to review test and merge these pr's and since acquiring the pen about a year ago, we're continuing to add functionality into the core github product.

C

Look out soon for dependable version updates, which will send you a pull request to update a dependency to the latest stable version you're already familiar with this functionality, if you're a dependable user prior to the github acquisition. But this will be new to github native depend upon, and this helps improve your security as well right when it's really critical bug hits you need to be able to quickly patch in general.

C

Patches will only come out for the latest, supported versions of a dependency and so regularly updating your dependencies means that a change will be relatively minor, and so it's less likely to break all right. So we talked about developers. Now, let's talk about maintainers, you also have a critical role in keeping the software supply chain secure and that's properly disclosing.

C

When you find a new vulnerability in your project and patching it, when you find a new vulnerability in your project, you can create a security advisory to develop and publish the fix in a private draft advisory and in a private fork. You can discuss and fix your issue and then publish information when you're ready to go public like once. You've developed a fix, if applicable, maintainers, can also request a cve directly from github for their vulnerability.

C

Vulnerabilities are published to github's advisory database, which is an open source database of vulnerabilities for several ecosystems, supported on github. It's licensed under creative commons 4.0, so it's free to use with attribution, because we think that this data should be free and easily accessible so that anyone can use it to verify and patch their code. The github advisory database is what powers our features like security alerts for vulnerable dependencies and the dependability security updates.

C

How do you even know if you have a vulnerability, though? Well, you might find it using a tool like code scanning which was talked about in the keynote this morning and, if you're lucky a researcher or community member reports, issues directly to you now. Let me tell you the number one pain point of security researchers. Today, it's not knowing who to go to with a vulnerability. It's trying to do the right thing and having no way to do it.

C

So many researchers that I know who chose to go public with a zero day, which is a vulnerability without a patch available. It's because they tried to contact maintainers and got no response in the 2019 state of open source security report. From sneak they found that if you had a, if you had a disclosure policy, 73 of incoming reports would be private, and if you didn't only 21, would that's people bashing their heads against a wall trying to help you and trying to do the right thing.

C

So what do you do? It's actually surprisingly, straightforward lay out your security disclosure and reporting policy in a security.md markdown file for your repo. This describes everything. Researchers and users need to report a potential vulnerability. For example, you might include a security ad email account. You can create per project policies or automatically apply one security policy to every repository in an organization if I could wave a magic wand and make all maintainers do something to improve their security.

C

Well, this is actually not it. It's 2fa, but this is a very close second, so please consider adding a security markdown file to state your disclosure policy and, if you're a researcher- and you find a vulnerability in open source repo back me up here, go check out that security md file and see what you find to make this even easier when you're filing an issue in a repo say about a vulnerability.

C

If the repo has a security policy, it surfaced to you right in that reporting flow click through to check out the security.md file and the recommended way to report an issue. Even if not, please try to report the vulnerability privately to other contacts that you find online and soon we'll make it easier for you to get recognition for the work that you do as part of security. Advisories maintainers can credit anyone who helped discover the issue or develop a fix as part right as part of the advisory.

C

These will show up as part of the vulnerabilities published in the advisory database. Thank you for the ongoing work that you do to support the security of our open source code.

C

So let's do a demo, we'll recap what we talked about today and what developers can do? We will create a new private repo, we'll enable dependency graph security alerts for vulnerable dependencies and depend about security updates. Then we'll commit a package manifest file and watch the magic happen, we'll be able to see our dependencies be notified of vulnerable dependencies and receive prs to fix those dependencies and then patch and you'll see uh too bad. There aren't commercial breaks anymore because you could do this all in commercial break for tiger king.

C

So first I'm in my organization here and I am going to create a new repo and my satellite demo great and I'm going to create a private repo because, like I said, these features are enabled for public reboot by default. So you specifically want to see what it looks like on private repos.

C

I create a new private repo, and now that I have that going. The first thing I'm going to do is commit a manifest file on my repo, so I'm using java, so I'm creating a project object.

C

uh Manifest file, which I always thought was a funny word to say, have a bunch of dependencies specified in that I'm a great developer, I'm committing straight to master. Please don't be like me: okay, let's go over the security tab in the security tab, you'll see the new way of enabling a bunch of features that we talked about earlier today in grey's talk and one of these items is dependency alerts, so we're going to go ahead and click on that and first I'll, allow github to perform read-only access.

C

This is to access the actual contents of the package manifest file and then I'll enable dependency graph and security alerts, and those little green check marks means that I'm good to go and, as I mentioned earlier, once you enable dependency graph and security alerts, then dependable security updates are also enabled by default.

C

So, let's go over into insights and I'll click on dependency graph, and I should see all my dependencies right here. So what's in that package, manifest file includes xmp, core android, svg, jackson, databind et cetera, and so I understand what all my dependencies are and you'll see. You know just in that time that I committed and enabled that functionality, it's already discovered some known security vulnerabilities and some of those you know aren't looking aren't looking great. So, let's go. Let's go see those security alerts that I have so my security alerts.

C

It tells me that I have um five open security alerts and wow one of them is critical. Let's click on that one, so this tells me that my security alert is in jackson. Data bind. uh It tells me how to remediate my uh the vulnerability that I have here. I need to update to a to a later version, and it tells me which cves this is related to, and there are a lot of them a lot of moderates and criticals and highs.

C

This is not looking good, and so I need to update this and actually apply the fix, and thankfully you'll see right at the top here. Dependable has actually already opened a pull request, so let's click on that dependibot has opened a pull request to upgrade my vulnerability, so I upgrade my dependency to a uh no longer vulnerable version.

C

um It says that it fixes the security vulnerability and in the package manifest file, it moves it from 2.8.11 to 2.9.10.4. So if I had some cicd checks, they might run here, I'm good to go. I'm going to merge that thank you to pentabot for fixing the issues that I have there and you'll see that security vulnerability is gone because we fixed it great so to quickly recap that what do we do? We created a new repo.

C

We enabled some features, and then you saw within a matter of minutes that we actually had a pull request to fix issues in your project minutes that was awesome.

C

Okay, looking across how vulnerabilities are discovered and remediated, then for security researchers. You know what is gonna do for security researchers once you identify vulnerability, please report it check the security.md file. If the repo has one to understand their vulnerability. Reporting policy for maintainer make sure that you have a security md file with your policy. There's one thing you can do right away: it's this well and enable 2fa.

C

Then, if or when you receive information about a new vulnerability in your project, you can privately discuss and create a fix and then publish a security advisory detailing information about the vulnerability and soon give credit where credit is due to those that helped discover the vulnerability or develop the fix for developers.

C

You want to know what dependencies you use you know want to know about the vulnerabilities of those dependencies patch them and then get back to work dependency graph, helps you understand your project's dependencies security alerts for vulnerable dependencies.

C

Let you know when one of those contains a vulnerability and dependable security updates, sends you a pull request to automatically suggest an update to a fixed version and soon you'll be able to use dependable version updates to keep your dependencies up to date on an ongoing basis, and if there's one thing that you do today is enable these features for your private repos they're already on for your public repos by default.

C

But please go into your settings and enable them for any private repos that you manage, and there are lots of other capabilities that github offers in the security space that we didn't talk about here, primarily to help you discover. New vulnerabilities in code code scanning uses code, ql, a semantic code analysis engine to query, code for potential vulnerability patterns and secret scanning revokes secrets that you've committed to public repos and alerts your secrets and code in your private repos.

C

So all together, you can use a suite of github security features to help secure your code throughout your development lifecycle, to learn more check out github's overview of security features at the link below.

C

Thank you. Looking forward to q a.

A

Thank you so much that wonderful presentation.

A

C

Love the way you.

A

Just like dropped in all the things that people are doing in isolation like feeding their sourdough starter, cutting their own hair and watching taiyaki, I'm just like that is just gonna resonate with so many people. Now you don't need to cut your hair jared you're, fine, just get them.

B

Are you kidding me, she.

A

Told me to cut my hair.

C

This weekend, or something for this presentation, like you're, seeing all my colleagues that have fresh new.

A

Haircuts I just like sit at myself. Like you know, while my security vulnerabilities are being fixed, automatically love it so good. Now we have a couple of questions coming through from the discussions. I probably have time for maybe one or two questions jared. What have I got.

B

Yeah and I just need to say, like I'm, not cutting my hair. If I cut my hair, it looked like I jumped into a lawnmower, so I'm gonna leave that to you or more skill, but going to the questions we have uh from g.a beach. Are there plans to expose an api endpoint to check the vulnerability alerts, are enabled on a repo.

C

Oh to check, I see that just whether or not it's enabled yeah, that's really something that we're looking into. I know that um being able to to manage all of this information via api endpoints and web hooks is something that we want to make really easy for you, because I know a lot of you have built custom tooling on top of this, um so that's certainly part of what we like to do.

B

That's awesome, I think they're going to be super excited for that. So I think we got time for a couple more. I think so. So next question: okay: what does it do? What is this yeah? What does the dependency graph use to determine my dependencies.

C

The dependency graph looks at a couple of different things. It primarily looks at your lock files to determine your dependencies and it also takes into account transitive dependencies. So if you're, um if your, if your dependencies have other dependencies, then it looks at those as well.

C

So we we look at the the versions that you specifically include in in your lock file, packed manifest files um to determine what you're using.

B

Outstanding, okay, I think we have time for one more and then maybe we can get your last tip maya all right last question: if I find a vulnerability in a github project, but I'm not the maintainer, can I publish a security advisory.

C

You cannot today, um we want to make those we want. We would like you to talk to the maintainers directly and figure out how to how to create the fix and the advisory. Although we hear you, we get this request a lot like a lot a lot, um and so it's something we're definitely thinking about doing in the future.

B

A

Now, apart from cutting your own hair, watching dieting and feeding your sourdough starter, what is your number one top takeaway tip for this maya.

C

um Not even related to my presentation but enable 2fa like so many swatching compromises come from what account taken. If you all unable to fa, I might be out of a job.

A

B

Okay, so every single person out.

C

A

Watching right now go and able to fa, um I mean look, we love you maya, so you're not going to be out of a job. I don't think you know we'll find some cool things. You can just bake cool things for us and kind of frontier would be great, so I think it'd be fine. Man enable 2fa and then thank you so so much for joining us and offering up all your advice next days. My is going to be over in the discussions for the next half an hour. Thanks maya.

B

B