GitLab Fuzzing 101, 31 Jul 2020

Previous Meeting

⏯

youtube image

►

From YouTube: High-level workflow of coverage guided fuzz testing

Description

Fuzzing documentation: https://docs.gitlab.com/ee/user/application_security/coverage_fuzzing/#coverage-guided-fuzz-testing-ultimate

Fuzz testing direction page: https://about.gitlab.com/direction/secure/fuzz-testing/fuzz-testing/

A

Hi there I'm sam kerr, I'm a principal product manager here at gitlab, and today I want to talk to you about coverage, guided fuzz testing and what the high level workflow of using coverage guided. Fuzz testing is like so that you can find bugs and security vulnerabilities in your programs and apps that traditional qa processes might miss. So, let's dive right in so coverage guide, fuzz testing always is going to start with your application.

A

This is really the reason you would want to start looking at buzz testing so that you can find bugs inside of your application, and so, if we look at what an app is composed of there's a lot to it, there's many different libraries, many different interfaces and ways that interacts with your customers and your users.

A

So one primary unit that applications are built of are functions. These are individual building blocks of code building blocks of logic. That physically do something in your application to allow it to provide the experience and the capabilities that it gives your end users.

A

An important part of any application is testing it to make sure that all these functions in the app work, as expected make sure that they don't have any edge cases that cause behavior, that a user wouldn't expect or that could potentially harm you or your business. So we're looking at an application right now that has three functions. Let's take a look at one of these and see how we would use coverage guide, fuzz testing to look into it.

A

So the first thing that we've done we've gone ahead and picked a function that we want to do. Coverage guide, fuzz testing on the specific function that we're going to look at. Let's say that it's all about loading, a pdf!

A

So, let's zoom in on it a little bit more to see what it has so part of this function. Its inputs are going to be a pdf file that we want to do the loading of, and let's say that after running the function, it's going to output a jpeg for each page in that pdf.

A

Maybe you would use these for copying into a presentation for sending it to emails or for archival purposes. So if you want to look at this function and model it visually, this is what it might look like. It's taking a pdf on the top left. It processes it in the body of the function and then it outputs a number of jpeg files at the bottom, and this is the normal way that this function would work.

A

It's going to work this way in the happy case where nothing goes wrong, and this is really going to be the way that you tell your users about this function and how it's used.

A

So one very important thing to note here, though, is that, because this loading pdf function is something your end users could use. This is a great place that bugs could be lurking in your program that you would want to find. This is why it's a great candidate for fuzz testing, because any function that a user can interact with directly means that not only could errors happen in the function from malformed inputs, but it's also the place that attackers are going to start looking at your applications as well trying to find those security vulnerabilities.

A

So now that we've talked about this function and how it works, let's talk a little bit about how coverage guided, fuzz testing would approach fuzzing and testing this function.

A

We've talked about how this function takes one pdf as an input, but the way that the fuzz tester is going to look at it is: let's try many many different different pdf documents, all with slight changes to them, which we call mutations in fuzzing to see if we can cause the load, pdf function to error or behave oddly in some sort of way.

A

I've got four pdfs pictured on the top left here, but don't let that fool you? What coverage guide fuzz system is going to be doing is going to be trying thousands, possibly even millions of inputs to this function in an attempt to get it to behave strangely or crash.

A

It will continue to generate random pdf documents to try and exercise the load pdf function until eventually one of them produces a crash. As soon as the application crashes, the fuzz tester immediately stops and realizes that hey, I found an edge case here, that's not being properly handled. This is most likely a bug or a security vulnerability as soon as that happens, the fuzz tester reports that hey this is the specific pdf document that caused an issue.

A

I don't know why it happened, but if you rerun this pdf document through the program, it will again crash, and this is going to be repeatable over and over and over again.

A

So at this time we now have a function which is accepting an input. We have a known input which causes a crash. This is exactly what we need to start debugging, the application to understand more about it, so that we can ultimately fix it.

A

So now that we have that pdf document, a developer or an engineer would take that they would create a test harness which has most likely already been created for this pdf function, and they know that there's a bug in there somewhere, because the application has crashed. This is not normal behavior that should be happening. So the developer can then take a debugger. They can have print statements.

A

What have you to ultimately go and find where that bug is in the function and do whatever is needed to fix it as soon as that fix has been created that can be committed back to your source code repository and your function can continue to be used in the application, and you know that you won't have that bug in your program anymore.

A

So then, as we zoom back out, this function has now been tested by fuzz testing. You know that is working better because that bug doesn't exist and you can continue this process looking at every single function, every piece of logic in your application, with fuzz testing to find any of those bugs that might be lurking, and this was a very high level, quick walkthrough of what coverage guided, fuzz testing is like and how you might use it in your own development processes.

A

So I wanted to share a few other places where you could get some more information. The gitlab product documentation is a great place to start. I put the link here in the slide. If you want to hand type it, but I'll also put it in the description below.

A

You can also look at the get gitlab fuzz testing direction page at gitlab. One of our core values is transparency, and so our direction page is a great way to understand how we're thinking about fuzz testing, what sort of problems we want to solve with it, what type of use cases and industries we want to target with our fuzz testing offerings, as well as our short term and longer term roadmaps about where we're going and where we plan to be in the future.

A

And finally, everything at gitlab is public. Again we work asynchronously and definitely feel free. If you want to create an issue and talk directly with us, I said before my name is sam kerr. You can see my handle for gitlab at the bottom right of the slide, I'm st kerr. I would love to talk to you more about fuzz testing, any of the problems you're having with your traditional qa processes and really just help you get using gitlab and finding more value in fuzz testing with that. Thank you so much for your time really appreciate.

A

It have a good day.