GitLab Secure Stage, 6 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Dependency Scanning - Workaround for monorepo

Description

This demonstrates a possible workaround to configure Dependency Scanning for Java monorepository. This approach can be replicated for Scala and Python projects for which Dependency Scanning have a similar behavior.

Related issue for feature improvement: https://gitlab.com/gitlab-org/gitlab/-/issues/393078

A

Hello, if you want to scan uh with the pencil skinning a project that uses another repository structure that won't be possible out of the box. We have an open issue to expand the support and allow to scan multiple Java and python files in the same project. But currently, if you look at the documentation, it's highlight the fact that we only execute one build uh in the first directory that we detect.

A

So if you're, using a multi-module project that works out of the box, because you have a parent, promix email file that is capable of building all the other sub projects. But if you have independent projects, then it's not possible and we will just scan the first one that we're finding and similarly for python. So uh in the meantime, until we achieve that, Improvement here is a quick workaround for you to make it work. This is a sample project.

A

We have three subfolders with simple public, simple file there, just with some dependencies for the sake of the demonstration, the searches work you're on here is leveraging the parametric keyword to automatically generate multiple jobs that do exactly the same thing, but with a variable component here.

A

The variable component is the target directory, um so we are including the template for the pencil scanning, which includes a bunch of jobs, including the uh the gym even depends in scanning one, and here we are overriding that jobs uh with uh an additional before script that will actually remove all the other subfolders except the one that we want to scan and looping through the metrics.

A

Here you can expect that we would generate three jobs and for each of them we will delete all the other folders except the one that is currently assigned to the Target tier variable.

A

So, looking at the pipeline for that, you can see quickly what this means. We have all these jobs grouped into the same part, because this is the how the UI react to the parallel Matrix configuration. So they all have the same prefix, but they have then a different suffix, which is the variable name here: project a b and c. As you can see, uh it works successfully scan uh all these three different projects generating 144 vulnerabilities and since this round on the default Branch.

A

This is already fitting the gravity report page uh with the cinema of dependencies of gravity. Sorry, as you can see, we have links for uh the project, a project and uh later yeah. We have for projects here also too. Similarly, the dependency list is fed with uh where it dependencies for the various sub project. Here we have Jackson data buying uh for project a and here for predict, C and uh license complaints. Page uh will work through learning too, so this is working for simple projects.

A

If you have a lot of different repository to configure and a lot of projects within the same repositories, it can be a bit tedious. So here is another approach using Dynamic configuration, which is relying on Dynamic type Pipelines.

A

In that configuration we have a specific job that will generate the config on the Fly.

A

um This is outputting a very similar configuration to what we had in the previous example, except that the list of Target D here is dynamically generated without small script, which is basically running in the root folder and listing all the folders by with the Commerce operator list, and then this is fed to that array here and this uh this content is outputted into a file which is called Dynamic config here, and this is saved as an artifact.

A

As you can see on that line, then we have a trigger the trigger dependency scanning job, which requires the first job to run. You can also use different stages if you prefer, but uh here we just need to ensure that this one runs after this one has been completed. This was also specific. This fact that we need to have access to that artifact and it will generate a new type pipeline using that specific artifact.

A

So, looking at the pipeline view now you will see what I just explained. We have the job to generate the config. Then we have the trigger job, and then we have the type pipeline that has been automatically created as a downstream pipeline, and then you retrieve the exact same approach with uh the parallel Matrix with the three sub jobs here and then the security tab fit with 144 damages in the same fashion and the reality reports also here, because so, unfortunately, the gravity report the verbs management system in just uh reports for from Child pipelines.

A

So this is working the same way, even if the security scans are running into a different pipeline. The dependencies also work with that and the license compliance also work with that. uh Please uh be aware, though, that this is a work around and not all our features are meant to work uh with a mono repository structure. So, for instance, there won't be a lot of filtering capabilities in the relative report of the dependencies list that will allow you to focus on the single sub projects.

A

uh You will have all of them listed in the same place, and you will basically are likely to have to manage them all at the same time, uh but at least you have the dip and skating feature working uh for you on the repository project. Thank you for watching. If you have any further questions, feel free to reach out. Thank you.