GitLab Secure Stage, 16 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2020.03.16 - Secure Brainstorming

Description

License Compliance in air-gapped networks

A

There are you, my name is Judah perfectly I'm, a distinguished engineer for secure and they found- and this is the brainstorming session on the license- compliance topic in their gaps. Environments. The idea here today is to see if we can find some new ideas to get license from clients more target compliant, which is not the case as of today.

A

We have one engineer with us more, who spent a significant amount of time already benchmarking and investigating, mostly in the current through that, were using license finder, and there are some findings that are not indestructible at that right after I finish this intro.

A

Well, basically, we have a few places where rice and compliance is not doing to work in our gut environments, mostly because in the way we are preparing the environment and also in the way that we are fetching the dependencies and matching or extracting the license information from the dependencies that we just installed on the difference, entry that we just created, I.

A

Created on Friday a.

A

Very short day to explained where we stand with a mother, I think well you've seen this stick Friday already remember well, I read the issue: is that the same as it's exactly the same, just a visual representation of the issue and so which I must ring with especially this slide.

A

So it's nothing that has been documented so far or named or anything, but we need to start putting names on different phases at different stages of what license compliance is doing, because we have this false feeling that license findings just be one thing, whereas under the hood it actually does maybe four different steps, maybe more than that. Not so sure about that. The kind of things that we should discuss today and, of course these names are completely work in progress- feel free to use any other game, but we we need to start somewhere.

A

So, with the sake of iteration in mind, let's start with analyzing the load scan matching report and see what we can do to improve that situations. Yeah I guess we already have. You cannot radiate. You want to speak to your point.

B

Yes, thinking back about this issue with private registry to download the dependencies, this issue actually applies to other categories like a pants, skinny incest, because as soon as we need to do to build the project in some technologies, we need to download the dependencies.

B

B

For I think for Java and some other like support the language is incest and everything that is supported on drugs. You need to compile the project because you're analyzing the charros files, so you will need to download the dependencies unless the customer is customizing a job so that it provided appearances beforehand to the job itself, but I think we. We focused a lot on these four license complaints, but this is actually something that we would have to support for all other category features.

A

Yeah, that's basically the idea behind creating the stages so that we can have a better overview of what we can share between the different categories that we have insecure, there's. Obviously, some overlap with dependency scanning, where we do exactly the same fork is the two first stages where we analyze in the node. So it's the first stage that we I could patent on a single stage.

B

Yeah my my pay that I agree with this when it comes to the logic to to build the dependency tree. This is clearly focusing on nice and compliance and come on with dependency scanning, but the ability to support that in a negative environment is something staged way to me.

B

It is not clear. I mean the distinction between the two things.

B

B

Okay I, try to refrain that. So when you want to generate the dependence three you might need to download dependencies, but it's not obvious always the case. You might just rely on some manifest file which is available locally, but in case you need to download them and the problem can be to hold. Are you able to reach to the registries or not and the case when you're not capable because you're in it in a cave environment? This is something this is a problem that is shared with the other categories.

B

It's not just license management, let's escape I'm, sorry and dependency scanning. This is also true for SAS, for example, yeah.

A

Okay, so I didn't, and we obviously won't agree on that. That's why I'm saying we are. There is some overlap between license compliance and different scanning, a problem such as you said when we, where we have to download the differences. So to echo my comments on the issue, my let us commence because I've seen what Moe did in the past weeks wrong vehicle that offline registries.

C

A

C

This problem a different way because, like okay, so if we like level set where we're currently at the current assumption, is that, in order to determine the dependencies and versions of those dependencies and then also any software licenses associated those dependencies says we have to install packages. That is the current direction in the current assumption, whether it's pulling it from public registries or from private registries. That's the baseline assumption, so I'm like okay.

C

Well, if I'm in a truly air-gap mode, what happens in a world where I can't connect outbound network connections, so I started with a different starting point just for fun as an experiment. I'm like can I actually build an offline index of software licenses for every package, slash every package version and if I could? What would that look like how much disk space would it take.

B

Why would I want to do that.

C

So if I'm in a like, you know I'm gonna, say era gap because I think they're realizing it means different things to different people, but if I'm in an in a in a world where I can't make out connections if I can provide you with an environment, in this case a docker image where, in that environment, in already knows of every software license associated with every public package, then at least when I do a scan of that project. I can I can tell okay if I can figure out what dependencies are.

C

What versions are in that project, then at least I can look in that offline index.

C

That's on the in the image and say: oh, this is the licenses associated with that version, and that name now, there's going to be a lot of cases where I can't tell you, because of like the private registry scenarios, where you publish a package to a private registry with a internal software, license that we don't know about ahead of time, but at least there's a good chunk of information that I can pre-compute in the offline index instead of having it to do at runtime at runtime today.

C

What we're doing is we're installing the packages and then from the package manager tool. We ask it. Can you tell me what packages are installed and what versions of those tools and that's great, because the package managers know how to parse the manifest file? They know how to like download the packages they can as their schema changes. We we don't have to deal with every schema change. The native package manager can do that across different major versions of the tooling. Where we fall apart is like we still have to make that outbound connection.

C

So the starting point that I came up with.

A

C

D

C

The way I was looking at this problem was I was gonna, do a code to walk through the places finder, but maybe I'm gonna rewind here so I'm wanting to see okay, here's here's my starting point can I just use. This is sort of like my whiteboard for now. Okay, so assuming that I can determine the I, can parse a lock file right I mean we've got Jim file, lock, we've got pip file block and we've got Hamed XML.

C

We've got some lock files that are difficult like the Gradle ones, I, don't know what to do there exactly, but we have locked files and for the most part we know how to parse, lock files and those lock files gives us a couple pieces of information. They give us a list of dependencies, their name and their version right and in order to get the name of the version for the most part that tells us what dependencies a project has.

C

What it doesn't tell us is like what is the software license associated with that license with that version of that package, name and version? And so do you get that information? We ask the package manager. Can you tell us, in the case of gem files out lock in the actual gemspec, there is a license attribute and if it's defined in the actual license attribute, then we can get that directly from disk. If it's not, then we have to do an API, lookup rubygems.org and there's like a JSON endpoint.

C

That will return some JSON representation of that, which I think is in most cases very similar to the spec file or the actual spec file. So in fact we can even skip this step so.

E

Much about it, it exists in the in the in the gem. If the authors put it in the gem, but it would exist on rubygems if they, when they upload the gem, then they filled it out like manually. Is that.

C

Why, in that particular case, it is from the gem spec. There is an additional step, so the API would match the gem spec itself, but the the thought that came to me is like I'm having to determine this every single time that I install rails right once I've installed rails once rails, 50.0 and I determine its licenses.

C

Mit do can I, take that information and store it so that I can reuse that knowledge for future scans rather than having to recompute it every time, meaning and reinstalling trails, re parsing, the rails, gem spec file and re, like figuring out that it's MIT the way I looked at it's like okay, yeah. This knowledge exists. What if I were to build an offline index so I hadn't index, which, let's just represent as a CSV file, and in this case it's just the package name, the version 5.0 and the software license associated with it.

C

If in that's you know, index I could represent every single version of rails, I, don't know 3.20 and the software licenses. So do you see associated with that because the software license can change depending on versions? Then there's precomputed information I can use it scan time.

C

I don't have to go reparse the gemspec file, because I've already done that once and if I can represent this information in a small enough index format that first of all has to be optimized for humans, because there's going to be errors in that data, so humans need to be able to contribute, and then second optimize for machine. So we can actually transport it. So it's small enough and we can read into it and get the information that we need as quickly as possible.

C

Then I save myself having to do the actual package install step plus the parsing of the gem specs plus determining the software license from those gem, specs and I can enhance on like today, there's actually multiple sources of information for this license. It isn't just the gem spec, but we can also actually search the project files for a license file and then from the license file determine what the license is. So if I could actually take that parsing and that knowledge, instead of doing it at scan time, do an offline ahead of time.

C

In this index. I can actually improve the efficacy of that index. I can improve the scan time because I don't have to download packages. I don't have to parse those packages. I just take the data that I have, which in so case it's just the gem file, lock name version, go look it up in the index. I'm like okay. This is a crazy idea mo. Why would you want keep like, like this is going to be billions of bytes of data right to store this much information?

C

So I started with my tracer bullet was spandex, so, let's just take. If you go gem install spandex today, he will need a ruby, and this is my tracer bullet. It is not beautiful code, but it is free and it's opening up there and one of the things I want to do is I want to be able to track all the network calls so to do that. I have all Network calls going through a single HTTP client and that client allows me to log all traffic to a log file that I specify so I'm.

C

Just gonna. Give you an example of what a lookup would look like when I'm allowing outbound network cults I'm going to scan. Let's do that.

C

Chimp file dialog so I'm just going to parse the gem file, dot, lock and I'm, also just going to redirect all the logs to a local disk here and, let's see what happens, please don't know.

E

Can you just describe what spandex does so.

C

Spandex is at this point: it's going to go parse that lock file. It is going to take each version of each like it's going to determine each gem. That's in that lock file take the name and version it's going to check in offline index, which I haven't shown you yet which is stored on disk and see if I have an own license for that offline index. If it doesn't have an own license in the offline index, it then makes a call to the rubygems API to look up the license for that gem at scan time.

C

So it's using its going to pull what it comes from like the local offline cache and then fall back to the network to actually determine the information it couldn't find locally. So if we look at the HTTP log we can see this is every Network request that went out to do this scan in this case most of the gems or MIT licenses. Why did I choose JSON I? Don't know that was the existing report. Format and I thought I'll just play along in hindsight.

C

I, probably would have done a line-by-line report, which is easier parts, but that's that's again. This is just an experiment. This was not. This is not production. Ready, I, don't think, but you can see here are the licenses that we were able to determine. So in many cases it was MIT some Apache thrift we weren't able to determine. So this is something that we'd have to go. Look at the project source code and see. Is there a license file? Why didn't they specify it in the gemspec?

C

What is it and we can add that data to the offline index? The second piece is like. Ok here is all the network calls that were made so so.

E

Did thrift not have a license even on rubygems no.

C

If we looked through the log here, we'd be able to see the API call for thrift and it we didn't, we weren't able to determine the software license for it, so that one is someone one that needs additional information now I'm going to do the same scam but I'm just going to add a dash a to the end- and this is gonna- be my air gap node and.

E

C

E

The thrift would fail in today's implementation right. It would just not have a license. Yeah.

C

So if we take a look at the actual air gap file now we'll see, these are all the network requests that were made. Cache misses.

C

So these are all the ones that weren't in my local cache in terms of like the offline index, but you can also see that there were quite a few that were detected MIT, Ruby, MIT Apache, so I was actually able to provide some information for known licenses that I was able to compute offline, but there was also quite a few that I don't have, because these were required network access to determine that.

C

So now, if we were to take those two reports and we can actually do a diff- let's do it diff of error, yeah JSON and, let's just see the level of difference between the air gap version versus the non air-gap version, and let me close this- we don't need that anymore.

C

And now I'm gonna do a diff, so you're gonna, see like we're. Gonna I expect to see that the live that JSON is going to have a more comprehensive list of software licenses because they was able to make Network calls in the places where I couldn't find the license in the offline index, and but let's see what the difference is and and that difference is going to be as good as my offline index as as complete as that offline index can be so now. If we do them do.

C

Please tell me to stop talking if I just took too much of air time here. Okay, so first one ace, rails, AP pin see on the right hand, side we were able to determine MIT, left hand, side, nothing--, Acme client both hit hit right. This was in the index, yay right hand, side, Apollo, upload server. We have it on the if we were able to make outbound calls, but we didn't in the air-gap note, but we did have a hit on the Santa. We had a mess on a batch loader.

C

We had a hit on be good, so you can see that like it's, it's not as complete as the live mode, but it has some level of completeness that we couldn't have achieved if I had to install packages in a scenario where I have true air gap like you have no outbound connections, so this solution sort of works to some extent in the live mode.

C

It actually works better because it's going to pull the data that it has local first, if it doesn't have it, then it goes out to the internet to get the missing information and I would love to take that missing information and feed it back into the index by some route. I haven't figured a power okay. So let's look at this completely unbiased index.

C

B

Allocation with a state statement that this would be a constantly moving target because we are constantly raising new version, the labor software. So we constantly have to keep track of those new versions to add them to the offline index, which means we need to have a high frequency update of this offline index to provide random data for customers. Absolutely.

C

So right now, my high frequency of offline index is one day now that high frequency, like you, get a better value, meaning you get more cache hits the more frequent you update that index, however, for any cache misses, you can still go get that missing information by making outbound Network calls. So the level of accuracy in the report doesn't increase or decrease that much for live mode when you're when you're, you know, you're not updating the index as often now with this scenario, where I'm using Postgres here.

C

So what I found is like on rubygems, they have nightly backups of the rubygems database, and so what this does is on a cron. It pulls down the latest nightly backup it. You know, attaches a Postgres connection to that backup and Infectious out all the ruby gems from the ruby gems table and builds an offline index. That is looks like this.

C

It's at the moment it's just a bunch of CSV files, I partition it into 256 different files, because I wanted to be able to sort of balance them as much as I could I couldn't find a good partition key in this case, because I don't know what like, if, if we overuse the letter e as a prefix versus under using letters ed, so I just chose an arbitrary, a partition key, which is the name of the gym.

C

I, take the name of the JAMA sha-256, so that take the first two characters, and this gives me approximately 256 data files that I can put the data in. So this is it that's I mean this isn't optimized for human consumption, not for machines, and if we actually check to see how much space on disk does this take. So far, it's not a complete index. So it's not going to be representative of all of rubygems. So in this case it's five megabytes and the number of items in the index is not comprehensive.

C

So we've got approximately 140 mm so because I'm only doing nightly, diffs and I've only been doing this for a few weeks. That's the number of records that I have in the database, but what I came to the realization is I, don't actually need to index every single version of every single gem and this actually works better, because this is only indexing. The latest versions of published gems, because the mission of our team is to keep you up to date right. We want to give you like tools to say.

C

This is why you should update to the latest version of this gem, because these are the vulnerabilities associated with it. So if you're on version, 0.10 of hibernate, you've probably got so many vulnerabilities that you know me telling you, the software license. Yeah it's going to be more expensive because, like not having that in the index, is fine because I in an ideal world, we don't have many people using older versions. So I don't know if this actually ever needs to be a complete index. Okay mo that's great for Ruby.

C

What about what about Java I, don't know about Java, but I can tell you like parsing pom files that, on that XML is a another solved problem and maven central does provide like an offline archive that you can download when I last looked at it. It was something like 600 megabytes and then, when you expand that it was like a couple of gigs so being able to like unpack that and just take the software licenses.

C

You need and produce like a very focused specialized index that can be optimized for just deploying in and off in in the docker image. I think is realistic is actually doable and CSV files we just saw was like five megabytes.

C

If we actually take that and drop it even further message, pack or even just ship, a sequel like database or even I mean like I, didn't even compress those files, and you saw how many times it said, MIT and there's tons of duplication in those text files that a good compression algorithm would be able to highly compress those data directories, but I didn't focus on that for now. So what I did focus on is copying what you call it copying the ruby advisory database spandex index update.

C

So this is just gonna pull the latest index via get so in essence, when you're building the docker image. If this were a solution like an offline in offline index, if we'd be like it's gonna, take a lot of work to actually build the index and keep it up-to-date, improve the efficacy of it and then figure out what an optimized machine version is for the scanners are. If, if we did choose to build those offline index and try to manage it, it's it's a bit of work right.

C

So each package manager is going to be different and maybe that's not going to be a comprehensive list. Maybe you don't do this for all package managers? Maybe you just do this for specific ones like the top two or three that we care about it would at least get you. Some depends on like licenses for those truly air-gap scenario, but then it also improves as your scan time, because you've got some of that information already pre computed and allows you to fall back to making Network calls for the missing information.

C

That was the experiment and that's oh I, see Anna, yes and I can't scan a comm file. Let me find one.

C

But this is not. This doesn't have an offline index. I didn't build one for maven or anything else. I am doing one for nougat at the moment. The new index mode.

E

This also relies on us scanning, like you're, saying ruby, gems or certain repositories right. So if people are linking to you know in in Ruby or in rails, you can just pull a random repository off github or any ghetto castaway and.

C

That's better to do ahead of time, not at scan time so like for, like so in the case of the new get one I'm finding like the original pages. There's so many misses because they changed the respect that I'm just recording it in a new get that unknown file, and so this files like these are all the ones that I need to go like figure out by pulling down the source code, doing an offline analysis of the license file once I figured that out once put it into the index.

C

I don't want to have to scan that license. File to you know do a match based on whatever the current licenses we have in SPD X at every scan time. I just want to do that like once ahead of time- and that's you know, this is what I'm pedaling my dead letter, Q of things to two files later I think.

A

What, since was mentioning, is that you can refer to your dependency. That is not officially vicodin, but the journal could be just a DT arrived somewhere.

E

So that's one so yeah I mean I. Think the offline index idea is great. My question is how many of those I mean of our customers packages is that gonna cover? You know like.

C

It'll be the public ones, the private.

E

C

You have to give them an opportunity to insert those into the index themselves. Yeah.

E

And in the truth of matter, is I, don't know how they're dealing with the private ones anyway, because presumably their application has to go out to the internet to get those well.

C

For private ones, they'd be hosted on their private, like artifactory registries right.

E

C

I guess that's that's.

E

The next piece is, you know, would we put our license? Finder be written to go against their local repository, I. Think.

C

That's the current goal. Correct me: if I'm wrong, that is the current goal. The current goal is to see if we can use the existing license finder code, let's go look at like this finder.

C

Okay, so here was an example of the Gradle package, manager' class in license finder. You can see it's using to download licenses tasks which has to be installed as part of your Gradle build, and so the catch here is like where this command is happening. We have to convince it to get that information from a private source rather than you know the public sources right.

B

So I think this.

C

Is the immediate direction and honestly makes sense to me like if we can convince that, to you know, go to an internal source and I don't have to maintain an offline index. You.

E

Know if that's written to have like multiple layers, so the primary source fails. It goes to secondary, so on and so forth. I'm.

C

Not a Java person but I. Okay. Let's look at the Gradle example engine tier so in cradle you can specify repositories in your build.gradle great and so there's some default ones like maven, central J, Center Isis, and you can put more than one in here so I've seen that in the documentation, so I think the next thing is just saying: okay, private, you know whatever that repo location is and then a hijacking or removing these for the private scenarios, so I believe so.

C

Okay, I, don't know for sure that needs investigation and it's also going to be dependent on each package managers ability to, but so far what I've seen like Ruby, we already know you just specify the source. Let me change that. You know the Gradle one here you can, it looks like you can specify the default ones. Maven central J, Center and I haven't checked to see how to do the private ones palm. Let's go back to you.

B

So you will actually have to duplicate the same behaviour at the pack in major when it comes to the logic about where to fetch. The information very.

C

Interesting, no but yeah like I'm parsing, lock files and so yeah you're right. The.

B

Example from says it is really pointing at this like, if you are setting multiple registry, for example, with fallbacks this. This logic is including to the package manager, and we will have to replicate that one on our side in the license scanner in the way to find which registry to point at find the license. Information.

B

Oh I, see we're saying, and this is joining my next command and the dark, which is the alternate option, is we already know that we will need to download the dependencies to build a project anyway, unless the project is built outside of gitlab, which I think is maybe a use case. We might agree to not support from now, but if we need to build a project within gitlab, we need to download the dependencies. So we are.

B

We have those dependencies in the in the local file system, how we looked at the different package, manager and language and we're supporting today and do we know if any of them doesn't contain the license metadata in the local phases there and still requires to have the data being fed to an external API I thought I.

A

Would like to extend that if we are able to download and install the dependencies, that means we have access inside is our dev environment to account for its history yeah. It's so even.

B

A

Don't have that locally that would be available directly on the network. So that's the only drawback that I see in this offline index. I think that it is great I, just wonder if we, if we will be in a position where we never had access to develop, I mean I, understand that there's a performance kit. By doing this, this recurrence registry is.

B

Bradley one different.

A

B

Can have some replicate of extra of to be created three with some local features like, for example, the Kitab packages features. They could replicate the API to download the dependencies when it comes to building the project, but it might not replicate the may be a custom API that provides license informations yeah. This is something that we pointed buy more and so as you're saying more, when you are installing Python dependencies. What you install locally doesn't include license information. You have to always reach out to the Python registered. You get the license. Data, oh I, please.

F

B

C

G

To try and avoid to call out a customer name, but there's one Python customer we have who one of their complaints they had with us frequently was that they pre compiled everything before they put it on their get lab instance, and so there could be situations where we're not using a on local area network registry because we're getting precompiled stuff. But in that case like are we gonna just choke on it anyway, because it doesn't have the expected little text files and things. We need well.

C

We'd have to ship that as part of the artifact that we give them in their installation, so I don't completely know how we give them. The docker images today, I assume it's part of the omnibus. No oh they're, downloading, I I. Think.

H

They have to go fetch him yeah. Maybe we.

C

Could provide like the image as part of the omnibus then that image would already have the offline index in it. I guess I, there's another assumption. I just made and uncovered I I thought we were shipping as part of the omnibus that.

B

Might be in the next iteration I think.

G

Okay, Matt's.

B

Already suggested that, but not part of the MVC I mean.

G

On the other hand, if somebody is choosing not to use the scanners if we keep by default having them all in the default package, our default package gets bigger and blow tier for things that people aren't using so having some kind of option to say, I want all of the secure docker images versus the base ones versus whatever would make some customers happy, because then they could save there. How much they're, downloading and scanning that's.

C

Why I'm so excited about the comment on the package proxy and being able to host rpms or Deb files? Because if we can host no Deb files specifically on get map comm, then, as we ship it as part of like the omnibus at install time, we don't have to build their baked everything into the image.

C

We can actually just install the Deb files with you know these indexes or the tools as part of the scan step without having to reach out, because my assumption there is like yes, you can, but you still need to be able to. You know, touch your local self installation of gate. Lab I mean you'd, be able to access the any packages that we ship as part of the get left. I'm.

G

Pretty sure we can write that one down as an assumption and be like. So my current assumption is within your local area network for limited access scanning or you know, limited internet connection scanning. You have to be able to connect with your runners. Your get lab instance.

G

You know like so there's a bunch of what I believe to be very reasonable assumptions that we should list out so that somebody could tell us if they're they've got some other scenario, but I think that's perfectly reasonable. To say, like your runners, your you know get lab instance, your code etc. You it whether or not they're in that same spot, you need to be able to call them within your cluster.

C

With I can't tell me, we have and.

A

Then, and if you don't there's always these possibility of fading reserve tendrils of namespace like slash secure, where we have a project that the customers would be able to fork and that we'd have scheduled pipelines, unloading the files on a regular basis, I'm saying that, because if you thought you can assign some very specific run errors that would be able to fetch something outside of I.

G

Don't think we can rely on cloning or forking, though, because sometimes these customers need to be able to do a download command on one machine and then drag those files and contents over to the git lab limited connectivity instance can before they hit their. So could they do that on just a regular computer like request a fork if.

F

They're yeah because.

A

The project is itself doesn't contain anything any package. The project would be just a way to download the packages, so actually the older they are lifting is done by not the project itself or negative instance.

G

To do is run a command, it would give me the files. I could have those files scan by security and then that's.

A

A pipeline, yes, it's getting python, and so they come define and they can choose. I.

G

Might not be able to have a pipeline reach out to the interwebs. I'm saying I have to be able to do this on, like my computer, in a dmz area, what.

A

You can instantly rather recording on your computer, okay,.

G

But like I can do a single command on my computer. Have it bring all.

A

G

Files down and then copy/paste those files into the instance. Yes,.

A

There's a point.

G

A

Then do that and to put that somewhere in the instance, one second sorry Lucas I, think I.

D

That's the script that we're talking about that doesn't essentially like a doctor, save a CP docker load into the environment. So that's what we've been talking about for the air-gap thing, so that's we have an established process for doing that. We just need to figure out how to recommend something like that to customers.

B

You have something that that are in place because they need at least one base image to do any build job right. So they need to find a way to put that base image right.

G

They may have all cooked up their own personal method, so us being able to say here's our recommended method at least could get everyone following one process, instead of whatever homegrown separate processes, which is harder to support yeah.

B

We can but I think it's it's the same thing. It's like what is your company's way to install and dependencies, and we provides the dependencies and it's its regular doctor images, whatever they way to install the other Deker images in their company. They can't just do the same way for our Deker images.

B

You just get the conversation with that.

A

So okay Morgan see yeah. You can see the chat. If you know the voiceover, can you stop sharing your screen.

B

So the high index looks great when it comes with huge madness costs and a lot of work to have full coverage for every languages. This is what I kind of wrap it. Okay, I can wrap up this.

C

Yes- and it may not be necessary now that I now that we had that conversation about the intranet- and you know my assumption of air-gapped met one thing versus now: I see some value, but it's more like an optimization. If we have internet access, but.

B

Then it could be a fullback calling because we are a treasure to find one solution for all languages, and this might not be the best approach. Maybe we could have dedicated solution for a I mean, for example, Python. If, if we have no local licensed meta data for Python, maybe we could bring an index for Python dependencies, so we could just do this result.

B

This lookup on the local index when we are dealing with Python projects, but for all the others we can rely on the private registry. You can to be contacted by licensed finder and that will be all fine, yep I. Think.

A

That's the main concern I love. This idea of you know friend indexes, but we are making a lot of assumptions on the network. What we want to see that running, yeah I think the next step really would be to gather all the customers, requirements and information so that we have a better value of what is really available inside this kind of network if we have really or even justify conserver so.

G

A

G

Respond better when you make a proposal and they can respond, whether you guessed correctly or incorrectly, for whatever reason: I that just seems to go well, so I think. If we wrote out you know, we believe that it is probable. This is the configuration we have access to a package registry, bla bla bla like if we wrote that out as a unit and I could have the Tam's and stuff share that with the customers. I believe we would get a higher percentage of feedback than if we just asked them about hey.

G

Could you tell us if you use X, Y or Z I have no idea why that just seems to get better results. So if we can make a little statement where y'all could tell me like this, is our preferred way that we'd like to see them set up and then I can make a statement saying we currently believe that and then I could get that kicked out. Then we can get feedback saying yay, nay and I could like finish it with.

G

If that's not currently the way that your setup would you be willing to share, you know, are using XYZ or something else.

C

Could you reduce it down to would you prefer your updates to come, buy a DVD or curl.

G

Include examples: it's.

E

Like choosing between your Netflix subscription.

G

We all see what happened to the DVD models: Netflix, that.

G

On the other hand, they were not blockbuster so.

A

What would be the form of these issues.

C

Sorry miss the end part of that question. What would be the that's.

A

The question for me kind of where our or what do you want to see that and what's the form of this so.

G

I think in the investigation issue we have for trying to figure out a solution. We might want to put a comment with that as well as maybe make a call out to it up in the description and just say: hey we've had internal discussions. We believe that it's reasonable. You know, based on talking to people, that the environments are set up like XYZ and that updates could be gotten through a or B.

G

You know, we'd love for you to let your Tam's know or let us know through whatever means, if this is an incorrect assumption, because we're gonna be prioritizing currently x and y. Until you tell us otherwise and then I can send that comment around to everybody and then, if we want to like draft kick that around in like a Google Doc, so we can all tweak it to make sure it's clear before it gets pasted in there just to save shenanigans. We can always do that.

A

With the gyri at least, I can do that for them. Unless no, you want to.

F

Do so I can do.

C

All right, I think I did a poor job of communicating via text previously, and so this is an opportunity to get better, but I also feel more comfortable with. Oh, no.

A

That's right because we we have no idea who that is talking to everyone, and we can clearly see, though they're integers and maybe the deserving pages right of this POC. But that's great.

A

Okay, similary.

B

A

Under maintenance, I'm.

B

Writing up some conclusions to make sure we agree on what has been said here and I've, a quick summary of what we're aiming to the world, to what they're going to do sorry. So our index has high I meant discussed but might still be useful for language and package management that can get the license in a tie. We look at registries our downloaded dependencies, but the primary way to address I get support would be tall or licensed finder to be configured to, although calling local registry is instead of public once we agree with this. Yes,.

G

B

Any other things that we want.

G

The next thing we might want to say is: we believe that getting the updates via curl on an internet, accessible machine and copying those files to the air gap is an acceptable method of receiving updates.

G

See that's what I said if we just say like we believe this is acceptable and then we like phrase it together and be like after brainstorming so easy. They will. Let us know if we are incorrect. We just need to say we believe X based on our discussions and I'm sure we will hear otherwise. If otherwise is the case, but.

A

Just to be sure, when you say these facts and when he is, writing updates we're talking about D of Y and indexes. No, he.

G

Changes the docker images for the scanners themselves.

A

Okay, so that's for me that's way beyond discourse.

G

License compliance but I mean that's right. That's what we're saying right now for SAS tests and everything is yet yeah. You got to be able to get the docker image somewhere.

G

Please, and thank you.

B

This is where every issue is looking to other issues, because we all come back to what is a negative environment. How do you deal with it.

G

So I'm hoping that Mr I booped David this morning and Sam I'm, hoping that Mr gets put public. So we can just point to the user documentation because I have a public definition of that once that gets merged and so I am hoping it gets merged soon. So we can all just point they're constantly, instead of reiterating like one single source of truth, but it doesn't mean to us and we can fixing that Mr or making newmars against it. But I want to have one single source of truth. Could.

A

You put that in there in the doc somewhere on the chat. It's.

G

In the it's in the group conversation this morning in response to okay, yeah I can find it again if they need to know.

C

Until we get some answers to these questions, is it okay, if I assume intranet like so that I can I'm trying to figure out like what's the next step for myself in terms of license I.

G

Assume that there okay, is a for the primary language. We're looking at that. There's gonna be a registry that they're maintaining that you can get to. With the caveat it probably is gonna have some CA username/password nonsense going on so I, don't know if you want to make that like a second issue to follow up on, but that is highly probable that that's gonna be the case.

C

Okay and primary language again is Java right, I.

G

Made a comment in that issue. Is that what the comment and the issue said, because that's the latest data I have I can go find it if I need to.

B

Square dance: yes, yes, report. We just rolled out that for all the other security features so I, oh, maybe definitely some inspiration to find there pythons.

G

Number one job is number two Java scripts number three C C++ is number four. A five is number one. I. Are you sorry, but that's.

H

B

G

You Johnson.

B

G

You wanna solve for Java. First, like that's in the top two: that's okay so solve for that one first and then no.

C

I I think like am I doing the hardest one first.

B

Which pythons because python a multiple way to play with dependencies different outcome? So.

G

If you can phrase that in a way, I can ask Pitts.

C

File that lock is a requirements. Txt is another, you know less known package manager or lock.

G

File and package manager.

H

A

G

Will go ask that, however, this customer has a lot of projects, and so the answer may be all of them.

C

Okay, I I'm trying to break this problem down to something that I could.

H

Demo like this week, if.

G

You want to start with Java. It is one of the top to you. If Java, it's easier for you to wrap around and start with, and then what I will do is I will go. Ask that Python question now and attempt to get that answer as soon as possible. I'm.

C

Good I would like to focus on maven I. Think.

C

Yes, if I could demo a maven, we I need to get a private artifactory or something do we have access to licenses for artifactory, or is that available and visit I'll figure this out? Stop? Okay! That's.

A

So we're saying if python is the priority and we need something up and running really quick I, remember that one of the customers replaced our license. Finder and I reserved like this, which doesn't seem to require a network connection or not one in the person show that it's worth trying and the good part. Is this too, as a that is compatible with licensed finder, so we would be able to get rid the results directly into into the passers. The only issue that we have is we don't have a way to tell licensed, find our lessons.

A

Finder analyzer, don't start if there is a Python file, yet so both analyzers, where you try to run at the same time.

G

Before I could comment, is this an accurate question? Would you be able to find out for me which types of lock file in which package managers specifically are in use for python and which java package manager, maven or jay, frog or artifactory, is in use.

C

For the maven one it'll be maven or Gradle J, frog I think it's like endpoint.

C

Artifactory would be like an example of a package that.

G

Might be the tool they're using ok, so just saying.

C

G

Or Gradle will get us what we need. Yeah.

C

I think so for the Java scenario and then for Python. Could you just repeat the Python part again.

G

Which types of lock files with which package managers are in use for Python I, just.

B

Added it to doing the duck for you really cool. Thank.

G

C

C

And one last question: is there a demo every Friday.

G

There is a demo every week, I haven't been invited to it. Yet so I don't know when it is. It covers all of the scanners, so the first one, for example, could be fast.

G

You know and then the next one could be us, etc, etc. I need to verify things are not clear.

I

As long as we're brainstorming, I'd like to upload Philippe's comment regarding having a repo and then updating from the repo like having the index and everything because I think good lab, we offer this working functionality where you can mirror a fork. So if we're okay with getting stuff from the internet temporarily like intermittently, then if we, if we give the client a fork of our index, then what can happen is we can create a CI pipeline on their side that they can intermittently pull the fork and then create a Mars?

I

And then that's a lot more visible right. That's like rather than merging, docker registries, which are like opaque, like we well, we trust, obviously good laugh, but but they can just see the mr.right and they can just review like here's.

B

The index and here's a few more licenses and.

I

I think that might be a little more transparent, so just upload.

B

Them Philippe's.

I

C

Is there any chance we can do it like a demo of what that looks like the pipe they're mirroring of the repo and then the pipeline. It sounds to hit the points that are concerning, but I haven't used. It maybe I should just go. Read the manual I.

I

Could see if I can.

B

All right should we stop there.

F

May your grace perfect timing.

A

All right that was great. Thank you, everyone, so next step I'm going to issue, and we will ask the customer see if it's a good fit or not. I will probably start with the Google Docs, as as you advice, Nico, so that we can share before recommending all right thanks everyone, the good rest of your day.

A