GitLab Fuzzing, 7 Jul 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Weekly Secure::Fuzz Testing Group Meeting 2020-07-07

Description

Weekly meeting of fuzz testing group

A

They're great yes, I had the the first one. I saw a notification that we were asked to make an issue about file size for funds. Testing did that get created, I'm, not sure, I sell notification for it, if not I'm, happy to create it and CC everyone on it. I.

B

Saw that as well, I did not create an issue to be honest with you, I'm, not even sure, where that whole thing stands because I think they were implementing asset size restrictions and then they probably need a estimated size for us. That's been sitting there for a while, so I haven't moved on that and and I don't think they were enforcing restrictions at least initially right.

A

Yeah, that's in part tied to the whole like understanding more for customers about their storage, what they're using so they can do better predictability around their resource and since it's not created I'll go ahead and take the action to create that and I'll see, see you all. We can start the discussion there. Asynchronously perfect.

B

A

James over to you.

C

Alright, this was just kind of a question about the open, merge request that I have on github runner to use the fuzzing features. I could expand on the mr2 fuzz and some of the other targets that I had mentions like the file names within zip files or there's one other one that seemed like it would be interesting to fuzz.

C

But regardless of the targets, its fuzzing I I was thinking about it and we could either leave it open and then every vamp it or update it to use the features when they land or we could just close it and consider it more of just a proof-of-concept. So what what do you guys think I.

B

Don't have a strong opinion one way or the other yeah.

C

I guess I should say um part of the the reason why it doesn't see that clear to me is because it is not the best mmm it's not as simple or ideally suited for fuzzing as say an image, parsing library right and that's the reason why I'm a little on the fence about it. It will use the features and it will add some extra degree of testing. But it's not Mike. It's not an ideal situation for fuzzing.

A

Yeah I'm still looking through the merge request, so bear with me, but so this is using the the coverage guided fuzzing that we've been working on up to this point and fuzzing the gitlab runners implementation. Is that correct? Yes, yeah.

C

And currently it's just fuzzing how it parses the yeah mo, which isn't that useful that new Marv was mostly about going through the process of wiring everything up just doing the dogfooding. There are a few other targets within github runner. That would be more interesting to fuzz. But even then it's not an ideal situation for closing I I.

B

Would definitely do I I would not merge it. Okay, because you know just looking at the implementation and how this is written. It doesn't follow all the things that we have in the documentation that you Vienna is merging in so I wouldn't want to be using things on gitlab, proper projects that doesn't follow our own documentation. Oh oh yeah,.

C

And that is part of it is if we did want to get it merged in I would stay a work in progress until all the features landed, and then we update it to use what's in the documentation and then we merge it in yeah, I, guess I. My description wasn't that clear, but.

B

But I think this does get to a point which is we should we should get to a point where we have a couple of these merged in to get lab projects. You.

C

Know it might be interesting to yeah.

B

Because I think from a sales perspective or a product story perspective, it's like well here. You've got this great fuzzing technique, technology, but you're not using it, and it's not merged into any of your your projects. So we do need to figure out how to get some of these merge into our projects. Okay, yeah.

D

I think we can can merge it, but uh even even if it's not the ideal situation, just just for you know, internal testing, I think it's like you already wrote the first target and it works. I mean why why not go probably find more more bugs that we have, and we can, you know, catch things early on as well.

D

If we introduce somebody like new bugs in the in the future, but definitely wait until we, we merge the the mvc2 to master, so we can use like update the mr2 use all the documented, the like style and integration yeah.

C

Yep, that makes sense, so that's about it. I didn't have any other mm-hmm questions about it. So I guess I have the next one as well so API dogfooding I, didn't really work on it that much last week, but it's going pretty quickly thanks to the documentation that Mike provided yeah. So it is it's not as simple as just including the CI mo I do have to jump through more steps, but it is pretty straightforward about what I have to do and related to that. Though, I created the miscellaneous group under fuzzing backend Seth.

C

If you don't like it feel free to delete it, but I ran into this problem and on the vulnerability research team, creating top level projects within fuzzing back-end. There's permissions wise makes it easier for all the members of the team to be able to move projects around and manage them if they're within subgroups of the main group and the epic that I linked kind of talks about that and yeah, so I figured I'd bring it up. Since it's a pretty fuzzy back-end is a pretty fresh group.

C

We can kind of think about how we might want to structure it.

B

Yeah, this is a good question, because we have we have a lot of utilities that are not necessarily in the analyzers, the analyzers folder. The we have things in there is those generate doctor images and for customers that are using offline installs. They just get analyzer slash whatever, and so anything that has a docker image, so projects that are not generating docker images, don't necessarily need to live in analyzers, and so we do need a good home for that I haven't, given it much thought.

B

I'm gonna talk to some of the other people and secure to figure out how they're, organizing things to figure out. If there's a good pattern that we should follow because.

C

It may be just the way, vulnerability, research or the vulnerability research group was set up, but mmm it causes problems having the top-level projects, because, even though all of us on the vulnerability research team are maintained, errs of that group, we couldn't move projects, and so, even if we wanted to organize it, we couldn't if they were all in subgroups to begin with, we'd, be able to move them around. That we'll know.

B

Yeah so I think that makes sense to have a subgroup if that gets around the limitation and then yeah, for example, this dogfooding isn't a customer facing project, so we can throw that in there all day. Long I think the organization question I have is, if we're not creating a docker image, but it's just a command-line utility.

B

Where do we put that and that's just something? We have to kind of figure out what our different options are. Yeah.

C

That's it for me. Thank you're upset, perfect.

B

Yes, so I just wanted to link over I, don't know if everyone has access to this document, but I can add anyone that needs it. This is just the okay yeah, so all I do. This is just the integration document that mostly Mike and I have been working through of the peach architecture and the one of the lengths is over cuz Sam. You had mentioned a question about one of the diagrams and just responded to your question.

B

This document has probably the most up-to-date discussion and then the goal is that what we're doing is we're going to start taking these and creating issues as as this discussion is fairly mature at some point, we'll have more than discussion in issues, but this document, as you can see, is like 20 pages, so there's a lot more efficient just to have it right. There make all the updates, as opposed to going through the whole issue, workflow so yeah this.

B

This has pretty much what we're doing for the a couple different milestones for peach and the architecture.

A

Is this document linked on those different issues about the the fuzzing architecture? Already it.

B

Hasn't been no and so I think yeah I, don't see why not.

B

The only thing is that what we're not trying to expose them the concept of milestones to the public, that's all, and then the the we may be better off closing out some of those public issues and just creating new issues coming from that Google, but coming from the Google Doc, we'll just need to work through this Google Doc and figure out the best way to like it out. I think this. This document right now is at a point where most of this discussion can move over to issues. Like anything add on that I.

E

Did stick the outline in there.

B

A

So this is ready to be converted issues. Would that be a different one for all the mouse on a architecture, tasks that the architecture, tasks and then see, or how do you see this breaking out into new issues? Yeah.

B

B

I would not call like milestone a milestone, be what I would do is say: okay, we want to, and we've already started this right. We we know that in order to get a lot of these, we need schema updates. So there's a schema, merge request, that's open now and we'll need to do that for for subsequent.

B

So what I would say is we'd open up individual issues for the particular things that need to get done and then perhaps those would fit into an epoch and that epoch might be milestone a witch or milestone B, which might be instead of milestone. B, it would be more, you know, customer can run API fuzzing job and then that would have the particular issues that need to get resolved.

E

B

E

Outlined section into the dock, that's gonna, hopefully translate into the tickets that we're gonna create.

A

Okay, cool yeah, no that'd be that'd, be great. I'd love us to be an issues but yeah 20 pages. That would be a lot to write as issues from the start.

A

Probably as we split this out, these would I mean they look like they would mostly fit into our existing maturity epics as sub X under there or even under some of the feature epics. So yeah I'd love to see the the outline and the proposed issues, and we can get things transferred out of here and onto on to get lab in that will read the document since I have not gone through this one. Since I've been back.

B

Perfect and then I think James you've got number five yep.

C

So this is kind of a status update on the fuzzing report schema. It is it's starting to get some good feedback from other tech leads and I've during the discussions. I've realized that it's kind of doing two things at once in the mr and it may make it harder to get emerged then so I'm thinking of splitting it into solely adding the fuzzy and schema versus the refactoring, yeah and I, can have the refactoring based on the fuzzing schema branch. Just so it includes that as well.

C

So at one merge request will depend on the other one, but yeah.

E

C

Getting good feedback and I say Sam. You had asked I had a note in here. We go from one of the other meetings asking about the schema being in the MVC I, when I made that I didn't realize that I could make a bookmark in the document and so I added the comment just so I could link to it. That's all and.

E

F

C

Yeah, so that's what that's about and actually is that still the case that the schema is not required for the MVC I.

A

Mean I think we're gonna have to have a schema that we implement against ourselves for MVC. It's not a requirement that it's considered stable for customers to be interfacing with directly, though oh I see I, don't think we're gonna get it right from v1, so we don't want to publish it out to everyone say this is what you need to use going forward. Yet, okay, all.

C

Right, gotcha, okay, cool and I've got the next ones so.

B

When you're outputting data from the fuzzing tool, are you using the schema that James is proposing, or where does that stand? Oh.

C

D

Yeah sorry I think I said yeah I hope so, but it like now no I I looked at the EMR again so I see that something's changed but I can I mean I can change the output pretty pretty easily after the schema will stay stabilized and.

C

That's that's exactly why I was asking if the schema needs to be an NBC because that they they don't currently match and I didn't know if they needed to match for the MVC. If the schema wasn't required, you know, then we could just update it after the fact. After it's stabilized.

A

Yeah, you know I think it's on all of our best interest to iterate more quickly and not make this a stable format right now, because we don't expect integrators to start using this until after you know, we've got our own formats and things in place. Yeah.

D

I think to change likely output will be pretty easily because, like essentially, this is only up currently at least only up to me, because this is what I write now so I can check. I know where the code is and I just can't change it yeah, so so it should, it should be. It should be pretty easy to play with it and and the schema it doesn't impact write any any other scanners right.

C

The current mr does um it should be backwards compatible, but with the refactoring and pulling common definitions out, it does have an impact on the other report formats. It basically adds extra optional fields yeah. So that's part of the reason why I'm thinking I should break the mr into two pieces, one for the refactoring one for updating to add fuzzing.

B

Yeah so I guess it doesn't make too much of a difference whether it gets finalized for the MVC once that gets finalized. Then that's when we can start building more work on the dashboard and I think that's probably where the the stability is really important is because then the dashboard will start reading particular fields, but right now we're not doing anything on the dashboard for fuzzing. So.

C

I actually have a I can run this down later, but I just wanted to note that it's not clear to me exactly who uses the schema, because right now it seems like it's only used for reference by people who want to integrate with get lab and that we're not using it in unit tests for the analyzers or anything.

C

B

The idea with the schema is whether it's get lab. That's producing that from one of our fuzzing tools or a third party that we produce consistent, looking data and then the schema. So there's two things that should happen. One is the tool should be validating its output against the schema, so the go fuzz get lab code. Fuzz should generate a JSON file and then validate that against the schema to make sure that it's compliant and then most importantly, is that data.

B

The dashboard should be validating the JSON file that it gets against the schema, and if about it's against the schema, then the dashboard should be able to read that data and just spit it out on the dashboard, and that's that why that scheme is so important is so that if a third party is sending us data, we just validate it it's valid, then we can show it on the dashboard. Okay, all right cool, but there's right now, there's nothing!

B

That's reading that file other than the dashboard which is reading that just some of the high level fields. It's not reading any of the new editions, yeah.

C

Yeah, okay, that makes a lot of sense. Thank you. So, unless there's more on that, I've got the next point to sock into Todd. So my mm right now I'm kind of splitting my time on other work and working on helping figure things out. That's fuzzing and dogfooding Todd mentioned that I should transition off of that being something I split. My time on around the end of July, so just kind of an FYI I current understanding is that I will not be permanently attached to this group involves, probably but not splitting my time, 50/50.

A

And that's it for me. So will you be primarily on VR stuff at that point, then James that.

C

Is my understanding? Yes, he's always changed.

D

Okay, I think I think I'm next one right, so the pipeline yeah pipeline dashboard, merge request, was was merged, I think last week and it already deployed to state staging or pre pre-production I think by yesterday. So that's good.

D

We have an initial documentation that request, which I think most of you reviewed, reviewed, reviewed it already, but you can take another loop and we are waiting to the Technical. Writer I also have to answer one of your comments that I saw it so Allah answer it. It's.

C

D

Really good by the way, Thanks and a there is another large request which is actually part of the first one. I just had to split it, which is like parents. The coverage guided, fuzzing Gamal, which we, which we users will essentially include using the template keyword.

D

So essentially, we are waiting for the both of those mr2 to have the demo that I showed marriage into master, so I think like both of them are waiting for yeah. The last one see is now waiting for final review by the maintainer and I think those are also also the missing pieces for for the milestone, B and C.

D

B

One another piece and I believe this might be a front end issue is the the download button on the pipeline page yeah. Do you know if there's a merge request for that or where that fits in yeah.

F

So I'm working on that it's in progress, so I'm hoping to have that knocked out this week. You know unit testing the request up so I. Don't.

B

F

Now, I, don't anticipate it slipping okay,.

B

Perfect and that's just a front-end issue, yeah.

F

It's a friend an issue only I'll put in the dock here in a second, but there is no back-end components that yeah I'll be using an existing API endpoint to get data. I need for that perfect good.

D

And- and this is also like part of the last part of the front end like download all the corpus is also part of the milestone. A yeah I, don't know stuff. Your youth.

B

So I'm sure the security dashboard piece no.

D

I mean secure, no I mean I meant. You said there is another missing piece, but you meant like messages.

E

D

For the MVC or for the milestone, a I wasn't sure.

B

I'm I'd have to look at figure out where we've drawn the line. Yeah.

D

Okay got it so Andy is the security dashboard which is I. Currently we show the vulnerabilities that the coverage guided fuzzing found in the pipeline dashboard and I'm working on the M R, which is relatively vegan. So it's good that we split it into two and Mars like pipeline dashboard and security.

B

D

Some I'm working now on the security corner, vulnerabilities.

E

D

Be saved into the database and showed on the security dashboard, so yeah I'm. This is working. Progress. I still have some issues there. Something is not still not working on on my local machine. I might choke fine, it's fine. Sometimes you talk to the other back-end devs to try and figure out what. Why are why everything is not working yet, and hopefully I'll finish this by either end of this week or early next week.

D

You, because essentially, because essentially, we are like um I'm kind of following the same principles. All the other scanners followed. So it should work kind of like not out of the box, I need to add more more classes and more scanners, but it's kind of a copy pasting, the same the same code and just changing the the schema a bit. So it should. It should work without any really any new. You have a functionality, so hopefully we'll get to this either by the end of this week or early next week, perfect.

F

My next point is I was just curious. So, right now in terms of the front-end, we have I guess where I'm working on the download, artifacts I just type lines page. Do you guys know what would be next in terms of front-end work? That's up and coming just like just broad, not! This is really getting into like specific issues or anything like that.

A

Yeah so fanatically, the next couple of things were going to want to focus on are going to be corpus, support so being able to reuse those between jobs if users want to download them or interact with them. We're gonna be want to focus on that, as well as being able to show fuzzing results in the security dashboard.

A

What some of the different pieces of information that we're gonna want to show for fuzzing and specifically compared to some of the other scanners I, can put some links in the document to the different design issues that camellia is working on for those as well. Okay,.

F

I might want to follow up then, because I think on that issue, I linked there there was a I had taken a screenshot of the right now on the security's tab. We actually download the button.

E

F

Looks like the running the fuzzing job does generate and item in that table, but it looks like this I'm missing data there or maybe I, don't know I'll follow up with you X on what should go there or if that's eventually can be populated, because if you click on the issue, the two one, seven one five one and then look at comment in the last screenshot. It looks like okay, Jemima.

D

Share my screen for a second.

F

Report- okay, pull this over here. Can you guys do this? So we have the way. So we have the security tab down buttons going to go over here, but then it looks like this vulnerability or this result was generated from that that job run. But you know, index out of range, I think severity, unknown I, don't know how we would generate a severity level for that, and then identifier is blank, so I don't know right now. It's particularly useful!

F

If, for this version you know this issues only pertains to generating the I mean creating the download button when there's an artifact, this I don't know if you want to follow up on in a follow up issue to address that from the UI.

D

So this is, this is already implemented like this. This is already ended in the UI. Ask you.

F

D

Not it's not a mock-up I just created from from the UI, it's also already merged into master or maybe I didn't get the question. Although.

F

The question is right: now: is this business? Look alright, because right now, in that kind of range, is specific value for this column, saying you know and then unknown like I just want to make sure this is not like I know we had generated actually a new report type in this view, but I don't know if the data that's here is currently valid, as is or if it's even useful to the user.

F

There's my question and if we just want to leave it as is right now, yeah.

B

So this data, so because coverage guide of fuzzing is kicking out a schema that is mostly compatible with the existing schemas. What you're seeing on screen is what naturally shows up this is you're bringing up the issue, which is that the fuzzing results are not necessarily vulnerabilities, so the fault that was found was index out of range and that's the fault but like this design is for you know, the title is vulnerability and it may not be the same. So we're shoving the data in there for now and I think this is.

B

This is a very good design question of like do. We want to have this table that is really for vulnerabilities show the Falls. If we don't, then we need another layout. But then, if you go down that path, then you have this issue of, like you can't view all severity x' in all scanners, right sure.

F

I mean I didn't mean to go off track here. I think my so I guess. The outcome for me is and I brought this up in the issues like. Maybe we should have a follow-up issue to kind of mitigate those, but for now I will add the download button. As for the mocks, which I think this is what it should look like over here, so I'll do that for now yep and then I'll guess, I'll follow up, see X on the concerns. I just brought up now, yeah.

B

And in in in the near term, what we're trying to do is get all this data into the schemas so that it lines up with all the infrastructure that we have, which means there's there shouldn't be a much front-end work or back-end work other than getting it into that schema, and that's why I think understanding the priority of you axe is very important, because corpus is, and all that is separate. Then this type of work all.

F

Right no worries thanks for clarifying yeah.

A

I mean I would look at it as a this is a iteration that is better than what we had yesterday, but by no means is yeah to your point. Users might be confused by it. We can put more information there and that's really what those design pieces of work around. What is the way we want to actually present these results. That's gonna be what it focuses on it.

F

No worries I just wanted to point it out just to surfaces so.

A

We're part of on the identifier column. Would it be possible for us to just put something static there, like just fuzz testing result, I mean even if it's not a specific ID, just it would be better than a blind column. In my opinion, I.

F

Could because I think I know what that particular row is then yeah I mean I can do that, but I I guess I just want to run it by you. Just I. Try not I mean what do you think yeah.

B

I we should just have that conversation in the issue. I would be more inclined not to have anything hard-coded on the front end and if we hard code it we should hard-coded in the back end so that later, when it gets updated, we can change the code and you on the front end, don't have to change anything yeah.

D

Yeah yeah I think it's like a bog in the Corinth version or like a feature in the next one. That's a way that we like it should like. This field should be connected to some output that we we control on the on the back end and that it should just yeah, be whatever we decide or either some useful information like identifier of the of the crash, for example, yeah I added.

C

A note or notes on the meeting on the schema, merge request. I did mention that we could use the existing identifier field and the different crash types should map directly to cwe's it. It's not something. I would push for for the NBC, but like a an access violation, trying to read data or if you're fuzzing, with an address, sanitizer you'll know it's an index out of bounds or Stack Overflow. All of those vulnerability types do map to cwe's, that's something we could add later yeah.

D

B

And then Mike I think you get the last item right now. Yeah.

E

Just why they toss in a link to the schema changes, we're proposing for API fuzzer.

B

River again just a quick background, we're trying to extend the ghast schema to support the api fuzzer and then there's a bunch of comments on there. That Cameron has put together so I think the next step is to address those and maybe break this up into a separate, merge request as appropriate.

A

Does this, mr only have one file on the disk file, not the source file.

E

Right now, I've just done the schema file changes for dest I'm just to get the conversing.

C

You'll want to update the source file. Everything in the disk over is generated from the files in the source file. I, see, okay, yeah, there's I think the readme mentions there's the scripts directory their scripts. Slash, distribute SH, that's what regenerates everything.

B

Excellent, any anything else.

A

Nothing for me nothing.

D

For me, maybe like another question like the currently the documentation review, so we are waiting for the technical writer. So is it the same process like any other review? You should like answer in the day or so or is it like the longer process, yeah.

B

So you can assign that over to Russell, Russell Dickinson and usually the that review goes a lot faster because it's you know it's grammar and maybe some slight clarity. So he should probably get back to you within a day or tip.

D

Got it cool just want them to understand if the process is kind of the same, for like technical writers and for developers, yeah.

B

There's AI once he reviews if he can merge it in, but it if he's happy with it cool.

A

Isn't Mick our technical writing counterpart not Russell or does Russell cover something different.

B

I thought Russell was taking over.

A

Okay, maybe that's I'm. Looking at the categories page and Nick's name is still in there. Okay,.

B

A

I think that's a question for them. Yeah.

B

It's a good question: I I, don't know Russell's been following he's been doing the last couple issue so as long as he keeps long as he keeps doing the work.

A

Maybe get a technical writing review yeah right.

B

I don't care who it is. If they keep doing it, then I'll keep sending it to.

B

Sounds great well, I think that's that's wrap great.

A

All right thanks folks,.

B