OpenSSF Alpha-Omega, 2 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Alpha Omega Project Public Meeting (November 2, 2022)

Description

https://github.com/ossf/alpha-omega

A

Awesome everybody well welcome to the November uh 2022 Alpha Omega public community meeting. uh This is a forum for you all to ask whatever questions.

B

Learn about stuff.

A

Here from us directly.

C

A

Us uh appointed questions whatever you want um our families here, so so welcome. um If you would um like to give an update on npm you're welcome, you don't have to, but there's a forum for anybody to chat about anything related to uh AO, current Future Past um and all that I do not have. This is my only slide, uh so we.

B

Are but it's a good slide, I.

A

I, like the slide I like that, the logo is uh top notch, so uh is there.

D

A

Anybody would like to talk about.

C

uh I can start um I would like just to give an update in the node.js slide uh this this week. Actually, past week, we we opened two important pull requests that uh includes the the threat model for an OJs and also a document that we call uh node.js Secrets best practices. It's pretty similar that the one that I think Alpha, Omega or.

D

C

Okay, yeah, he is creating. So basically, we explain that some threats that might affect node.js is not considered a vulnerability node.js and what you need to do to to to guard the Glen against those threats. uh Those are very important uh documents and they they were created through the secret working group.

C

It was very cool and.

B

Well, that's really fantastic news! Rafael! It's it's a significant like it seems like a small thing, but it's actually a really big deal and I'm really happy about it.

C

Yeah actually I have a presentation: uh yeah where's, yours, okay, let me start sharing with you.

B

You have to have a slide, at least as cool as Michael's only slide. Okay,.

C

I I don't know if I have but I'll try.

E

I I will quickly add if you have a link to it, I'd love to slip that into the notes so that people later on, who just you know? Oh.

C

E

C

The link in the chat too and I will.

F

E

C

E

I got it. Thank you so much.

C

So basically, uh I have presented these those slides for the node.js collab Summit. It's a summit that we that week it's a collaborative from the node.js members we discussed important stuff and what is the future of node.js? What's the current threats, what what's the the current features? uh Print bugs and so on? Basically, it's pretty simple I have showed uh the current uh secret written group participants and the current initiatives.

C

I have showed it I, don't know three weeks ago or one month ago, more or less and basically the active members are Facundo, Michael, uh olisis and and Thomas and well one. uh The first initiative that we have concluded is one called node.js vulnerability, assessment, workflow I know this is a long name, but we don't have a better sorry for that.

C

Basically, one of the problems with no jail, not not node.js, but any any two that depends on other um uh dependence. For instance, node.js contains a bunch of dependencies, for instance, any g832, open, SSL review, V and so on, and when those um those dependencies send um expose or disclosure cve, uh we get not not set by the users, or at least in you know, with some delay.

C

This workflow has the intention to to basically run every night and check if there are any uh disclosure CV so in case, let's say open without just bump it uh critical uh secret release. Yesterday we are operating today by the way and we received an issue telling okay. This is a new CV.

C

We need to predict because those those versions is affected by node.js and also it's a good way to to perform assessments because, uh for instance, that's that's what uh a real vulnerability from zillid but does not affect node.js and users, were the users were complying complaining about the these vulnerability and that's a good, a good repository that they can see if that affects their their apps or not.

C

The second one is the threat model not too much to share about it, because it's pretty concise, uh basically most uh almost uh 40 percent of or hacker one reports were closer closed as a non-applicable or informative.

C

This is a clear indicator that security research does not know, uh uh don't know how node.js works because, for instance, let's say you install a package and this package perform malicious requests. It's not consider an old yet vulnerability, but if the user installs it they can be infected by it. So we need to be clear in all the things that node.js, trust and or not node.js does not trust. That's the intention of this threat document and it the pull, requests were created or was created and we are discussing it with the TSC probably next week.

C

We have something ready to to that to the public Raphael.

B

This is can I interrupt. This is really interesting and I'm curious. Like and general question to the group here. Is this a is there a standard practice of this system here? If you're going to report a vulnerability here, here's our threat model, please express your vulnerability in terms of this threat. Was that a common practice and you guys are getting there or yeah.

C

uh Honestly, I don't have a strong background on on that field to say, but I did some research before and basically all the runtimes like python or uh Java uh PHP. They have their own threat model and some vulnerabilities. Some threads can be applied to some uh to those run times, but in different ways. For instance, supply chain attacks in node.js is very common, but that's the way the node.js works.

C

If you want to to to to guard against it, you need to mitigate it somehow, but it's not node.js responsibility and that's the the point of this document. We need to make it clear that we need to express how they can guard against it and no Jazz can't.

E

If I make jumping real quick, there are I've seen some terminological differences uh in in my mind, you know as long as you get to where you're going I, don't care, which words you use um some people, you know refer this as the threat model. Some people call these security requirements. I tend to call them security requirements, but whatever you call that having some idea of what you're trying to achieve and not achieve is pretty important.

C

Yeah yeah totally so uh yeah I mean the idea of this document is to be a live document. uh I am pretty sure that we, we are missing a bunch of uh statements in the in the initial version of this document, but it will be built in collaboration with secret researches, because once we receive hacker one uh reports, we we say: okay, it's missing in the threat model. Then let's play that but yeah. Currently the name is threat model, but it might be like what no just trust org is not addressed.

C

Yeah, uh okay, just moving on moving on uh the idea when, when we started that that initiative, it was just threat model, but in the middle of that document we were adding a lot of explanation. That does not that the target was not clear and uh the intention of uh threat model.

C

The target of threat model is secret researchers, but we also uh wants to provide a kind of explanation for end users, how they can secure the application, how they they can understand how node.js Works under the hood, without going deeper in the code, and then we call, we created the node.js secret best practice document the we have a request.

C

I need to play this Slide by the way we have a podcast for both and they are, they are they're being analyzed and that's important to say that most of our hacker one reports is uh who are our attacks. That is a fact, a fact not just developer, but not just users. This is important to say that uh developers are more susceptible to vulnerabilities, because some of them just follow a hand on tutorial on their internet so to to install uh a malicious like a package with a typo that performs something very weird.

C

I can tell one history left: let's keep this one, but let let me tell this history this this. This was solved in some in Prior versions of node.js.

C

uh So, let's assume that you are on Windows, okay and you have uh opens itself installed and that's pretty common, because let's say if you are on Windows and you install a game, you have open SSL, that's all, then you have node.js and you are following a a tutorial in the internet that tells you to install fetify with e in the end with a typo, because it is a different one, and then this package basically contains a providers.dll inside of that package.

C

Okay and the content of this dll is pretty simple, like it's the most dangerous thing that you can do on Windows. Is they open the calculator? Okay.

C

And then, basically, every time that require a module that requires open. The cell opens the cell gets uh loaded and by default it loads its search for providers.dll in the current working directory, which means that when, if my malicious package specify contains a script, posting Style just to check the npm version, that internally calls require crypto, it will load the providers.dll from my current working director with maze. It will load my malicious dll instead of the the expected one from the open, SSL, and then the user will see the calculator so that that's that's.

C

That was a real vulnerable, real node.js. It was, he said hopefully uh but yeah. That's that's the intention of uh how how this thing works and after this um well, we are uh basically saying the threat model that, if, um if a package or if dll, is not specifying the documentation that node.js will try to load. This is a vulnerability and my and needs to be reported to node.js. This will. This will increase their collaboration between uh node.js developers and secret researchers and also I'm working on a test.

C

Is that basically I asked race that will will check what node.js is loading on the startup and then we document it and every time that we perform a pull request. We check if that changes or not just prevent vulnerabilities, but you know that that was a crazy one, but that that's all I have another presentation about the permission system, but I'm still uh I'm still developing it, uh and once it's done, I I can share it here. Okay, foreign.

A

This is awesome. Thank you. um Thanks, I.

B

A

Two two questions and feel free if this is outside of the bounds of what you want to talk about, um just tell me to be quiet. um First, one was on on um on open SSL.

A

Specifically, it does no do you guys feel like you, have the right level of Engagement for like Advanced notification of, and it's not about open, SSL specifically but like for the for the major tools that you include in node, um your your as a platform, um you know, I've put you guys on par with Ubuntu or anybody else in that you know. Essentially, did you get Advanced notification of openssl and what and if the answer is no, would exploring that further be be helpful.

B

I know you guys.

A

You out and the the fix that in a.

C

Couple days and.

A

I know that um it, you know it went from critical to high, but so we kind of dodged the whole repair, but.

C

um We do uh actually we just uh subscribe to their newsletter uh as a thing, and we receive notification when uh upcoming secret release on upcoming secret release. The only thing that we are we need to work on is to receive the the assessment before they release, because once they they release the the the secret release itself, we basically assess it on node.js to check if that affects node.js or not, and then we we discuss internally.

C

When should we do the secret release, because normally critical or high uh vulnerability should be updated as soon as possible, node.js Right, but can.

E

I jump in on this real, quick um I think the open SSL one is a little unusual in that a lot of Open Source projects. The way you find out is the up. Here's, the patch open SSL in this particular case basically set gave a heads up to everybody publicly we're going to have a release in a few days at this. Well at this particular date, there's some vulnerabilities for fixing get ready.

E

um So I, that's a little unusual frankly um in the in the grand world, but I think it can be helpful, so Raphael I guess you know. Let me put you on the spot a little bit. Was it helpful for open SSL to have told you that at a time- and if so, is this- something that we should incur projects to do in general is to give heads up that we're going to make a release on this date. Yeah.

C

E

C

Absolutely uh well uh we received, and we normally when, when we see the the assessment from the the open SSL, we start the workflow to perform security release nowadays. Security release on node.js is a tough operation. It requires at least four releasers, because it's a 42 steps to perform a security release because we have a well-defined workflow. We have some companies depending on on it. That needs to be get notified, and if we receive a notification like today, we can't do it because it requires at least one week of effort.

C

So, for instance, if we receive if uh silly disclosure as uh height vulnerability that affect node.js today or tomorrow without prior Nazi, certainly it will affect node.js in a bad way.

E

C

E

Sorry go ahead, Michael.

A

Well, as you can say so what it sounds to me- and you know obviously like go back and discuss this on your end, but like if you believe so there were vendors that received an advanced copy of the patch from openssl like there were embargoes.

A

This is you know how things are done in in those cases and- and those are you know this relatively small list, um but if, if, if node thinks that getting access to that in advance of public would be helpful in getting a release out at the same time as when it goes public which is kind of how like Ubuntu did it um and Canuck on some others like it, I would be happy to at least attempt to broker that conversation, um so just put it out there.

A

You know, if you think, would be useful um yeah a couple days, and in this case it doesn't really matter, but in the future you know.

C

Yeah, uh the Decades of no jazz is light different from uh from the vendors that use open, SSL directly, we don't use. The main openness is how we use a fork version, because we use the quick protocol for the 16 and 18 uh version and we basically know open Cell release their fixed. We wait for the fork from the chromium, I guess.

C

Is the name let me send here?

C

We use that fork uh because we we have uh initiatives using uh quick and probably in the 832.

C

So yeah or use cases like different because we have someone in the middle okay.

A

Makes sense so it's really quick TLS that would you know in theory, need the the to.

C

A

As possible, but.

C

Yeah but usually they are very fast yeah, for instance, um it tends to to to to be updated in one day maximum yeah, so they are good cool awesome.

E

Yeah, uh sorry to get so you so it sounds like you do think that it's helpful to have at least uh the advanced warning without the patch.

C

E

It might be Michael's been asked about the one with the patch um okay, because that's that might Michael. This might be useful feedback to the vulnerability disclosures working group, because right now, I. Don't think that that's you know giving an advance hey we're going to have one I, don't remember! If that's in their process, I don't think it is, uh but at least something to consider.

C

um Yeah totally I mean, if all the dependents do it would be awesome because we need we need planning. We need uh most of the the books working on nodejas. It's not they don't work full time on it, so they need to schedule. They have a schedule to work girl, no jazz, so yeah.

E

Has node considered using Vex or something else to report when a dependency has a vulnerability, but you can determine yeah.

B

E

It's not, it doesn't apply in your case.

C

We didn't uh we have some internal discussions that are on that, but I'm, not the part.

C

So yeah, probably we'll have something pretty soon, but I I have no idea cool.

A

um And then the the um and one other question when you were showing the GitHub action that you know spits out. You know.

C

A

Sealed is that just like depend about out of the box kind of stuff, or is that something custom or oh.

C

uh Basically, it's a a pretty simple uh python script that look toward uh dependence 3 and check if they are open, uh vulnerability, open cve. That uh node.js is not uh it's not a war, but.

A

Okay, so so it's against the vulnerability, the the it's against siaries and and Z live and the ones that you know you're using already, it's not doing a full like oh.

C

We have uh like a hash map or something like that. Okay, uh I was I, just sat in the chat there. Oh.

D

C

D

A

Anything else on this topic.

D

C

A

Thank you very much, really appreciate your uh your collaboration.

B

And assistance and keeping the world.

C

And everything.

G

Really appreciate it so.

C

Yeah pretty soon I think I will have something to share related to the permission system. Yes,.

A

C

Will be, that would be good yeah yeah.

A

A

Anybody else other things anyone would like to talk about I'm working.

A

I do have something I'd like to talk about, but I'll mine will be the lowest priority in the in the stack.

A

All right well so the topic here so I posted a message in um Alpha Omega um last week. Maybe uh and there's a there's, an open issue about it in the in the repub. Essentially, I wanted to get everybody's thoughts on attestations like I'm, not sure if it's an attestation or an assertion I think it's actually an assertion, but it doesn't really matter um so. Here's a scenario um as part of the Omega analysis. So we find an open source project. We run a bunch of tools against it. We get some output.

A

That output can either be probably one in one of three categories: um the tools ran. Nothing interesting was found, looks good.

A

um The tools ran something interesting was found, but it turned out to be an interesting after triage or it found something interesting and it was a vulnerability and we reported it and it got fixed um I guess maybe didn't get fixed, but it's basically those kind of three categories.

A

um Now we could do that ourselves and just keep that information. You know private on which, which ones we looked at and what the results were, and you know eventually a component will get we'll, have a vulnerability fix and everybody will benefit and all that, but as a consumer and I mean as a consumer developer or you know, a compliance department or whatever, um knowing that that particular component was examined and found to be in one of those categories, um allows you to make different decisions. That gives you knowledge that you didn't have before.

A

So the question is: if Omega were to generate- and let's just keep this at like an abstract level and an assertion saying that we ran this tool against this component um and these were the results um aggregated. So you know we found there were no issues, there was an issue and it hasn't been triaged. Yet you know whatever things like that, um we we try to reproduce it and we couldn't or we could. We checked um here are the scorecard values.

A

As of like that that moment in time, um uh I don't know you can think of other things other than static analysis too, but essentially we did some Assurance work against a thing.

A

Here's the information you can trust it because we said so, and you know that's through the magic of signatures and went out. Would that be valuable? Would that be something that one could consume uh only technically I mean. Would that be useful for anyone to consume.

E

Well, I, oh just for me, but uh you know basically I think it is useful to know that hey a tool was run. It didn't find anything while that doesn't guarantee things. Guarantees are hard and I I. Think a lot of organizations are looking for some information I'm. Looking at this package is there you know any indications of either problems or not problems.

E

Now I think you have to be a little careful if you do find a problem I'm a little hesitant to make an indicator yet uh for a variety of reasons, including the fact that for a lot of tools, false positives are absolutely a thing. So I I think us I. Think attestations of I ran a tool under some configuration and it didn't find anything or didn't find anything at a critical or a high level. Is useful, I'm really hesitant to help the attackers too much I I saw your note about hey the attack.

E

They can run the tools public tools anyway, which is absolutely true, but it is a little extra work to run a tool as opposed to just download somebody else's work. So I I I hesitate to help the attackers too much look right here, um but as soon as you get to the point where you can make an attestation that you don't find anything or don't find anything, a certain level I would say, go for it. I, don't think you need to say: hey didn't, find SQL injections, uh I mean I.

E

I, think that that level of detail is probably not worth it because it really just depends on the tool anyway. So you know the the tool each of the tools. You know the tool may not find SQL injections, or we only find very specialized cases of them.

E

So I'm not sure how strong, if you don't know the details about the tool, I'm, not sure that that gives you much information and it sure will be a whole lot of extra work for probably a little gain, but yeah two Lux didn't find something or didn't find something at certain levels. I would certainly like to know that of this version. Obviously version.

C

E

Version of of um you know from a version of tool, but more importantly, version of software, because obviously you know 2.0 may have zero vulnerabilities of 1.0. Maybe a disaster.

A

Yep yeah and then what I'm imagining is you know the the consuming end of it is a a reference implementation of a you know. Your your policy is no critical vulnerabilities and I don't care if it's reproducible or not, let's say so that that's expressed in some policy format and then use a consumer. You just say uh you know, check left pad policy, one two three and it says pass um so from from that perspective, it would be something that yeah you'd it'd be like a you know. Pre-Install about you know, uh test pass.

A

You know, Test passing check.

D

So the producer will have to be like similar to a certificate. Authority right I mean that is producing attestations. What is preventing from uh like an authority spoofing like somebody else, maliciously producing some attestations. uh That is wrong. Those are wrong. So so what.

A

I'm thinking there is um the so each of these assertions would be signed in in some way. The signer would either explicitly be trusted in the way that, like Linux, like ppas, are or delegated trust based on lab, the ca system works or something else.

A

um I I, don't want to say totally like left as an exercise to the reader, but um I I think that there's I think we would do better to have this system a b distributed so that you know like we would run one, but you know others could run their own private one.

A

So you could like point at a registry when you're doing the policy evaluation, um but then, more than that you can say I only trust, a b and c or I only trust people that trust that are trusted by a, and you know that kind of thing, so you should be able to um ignore the you know. If, if you know just let's just say, random people are providing assertions of whatever.

C

You could ignore them.

A

And you say: I only trust the ones from openssf and my company, um or only my company or you know, whatever.

E

Yeah, if I could, um what do you think about the idea of signing them with like six door.

A

I yeah I yeah I want to go too into the. What, with the right implementation, be right.

E

A

Sense, like whatever yeah I.

E

Yeah, it's just that that is, is designed to make it easier to verify, because signing is Trivial as long as you don't need to verify it later. But.

B

If you don't verify it, what's the point so so,.

E

Yeah, so that's the uh the I mean it doesn't have to be six store necessarily, but if.

A

We want I mean it would be. It would be uh as I'm learning more about this, so it would be cosine either off of an oidc thing from us wherever or I could have a hardware token or anything else that reacts back the thing so um and.

E

Then right I mean six door can do either of those things. It's the it's. The oidc that's uh different and unusual right, uh but I mean it can do the uh tokens as well. But the point is that they don't have to have your public key to be able to verify that it was. You know uh you know at org or whatever yep yep.

A

Yeah and I mean we would want to make it from like at minimum like Alpha Omega, not right. Okay,.

E

A

For that, we would, you know, generate it.

E

I, don't think we've generated that workflow. So if you want to do that, um we probably ought to I I think that's a great idea. We just need to work out how we make sure that that can actually happen. Yeah.

A

E

It might be as.

A

Simple, as there are small number of signers, they all sign with a hardware token that we provide them that you know yeah, we'll figure that out.

E

Yeah yeah, it's totally doable. We just have to figure out what's what we, what to do.

A

um And the other nice thing about this is: it does provide us a way to generate manual assertions. So I took a look at left, pad I reviewed the code. I think it's awesome, no issues found and I was taking my reputation on it by credit by you know. Creating this assertion and saying you know: Michael thinks this is fine um and and putting this out there so similar to these Security reviews um uh open ssf, you know repo, um you know there should be just a another.

C

Consumer and you may decide.

A

Actually, as a consumer I, don't trust manual assertions because you know people make mistakes and machines never do so. um Just blanket provide no value to the um you know. Don't look for manual assertions.

E

Wow, those are that's a whole lot of assertion about uh automated systems. Never making mistakes, I.

A

Hope a lot I've, never known the computers.

E

Too often to not believe a word of that.

E

Yeah but but in in fact, I think what you would like to see is multiple assertions from multiple different sources. uh You know this tools run. This manual analysis was run, hopefully by work people organizations you actually have a reasonably are reasonable, and then that gives uh with multiple signals like that and give you really strong uh justification that well I mean mistakes can happen, but there's been some real effort here and.

F

E

Saw Rob Underwood pop up: are you about to ask a question? No.

F

I'm just saying hi hi, we had a few of uh my colleagues I think on the phone earlier at least until 12 30., so David you're uh you're in Brian's yeoman's effort is, is showing some results. Thank you.

E

Oh well, thank you and uh well for Alpha Omega, the two Michaels one of which has I think had to leave early. Sadly, but uh Michael scavetta, Michael Windsor, um you know they've been you know very much polling on this for a while. So um and you know, we've got our our new awesome person. Citibank so I mean we do there's a number of folks who are pulling this particular rope and I'm grateful to everybody.

F

Well, we're we're hoping to get more people from our organization involved in openssf generally, but Alpha Omega in particular, and I. Think now that we've piqued interests it's great. So thank you again.

A

Awesome um anything.

E

Else on the topic.

D

Of assertions contestations.

E

uh uh No, not myself but Aaron whitsling just put posted in the chat and I think it's so important. I'd like to highlight it. um You know coming back to the hash. What about posting? You know find no vulnerabilities, um I I, like the summaries, uh so I'm, just gonna post it. This feels like no failing tests when you're picking up a new repo, yeah, obviously no failing tests when you're looking at a repo might mean hey.

E

They haven't written any tests yet or they only have one, um but nevertheless there is a stronger feeling of wait a minute at least there's some there's at least some signal that uh there's uh things are kind of on the ball from that and I. Think that that's a really important point.

E

Aaron wanted to clarify on that.

G

No I think I think that was right on it's also yeah and the idea is that you do like you said you don't get this feeling of a guarantee from it. It's just another signal, another thing and I think we have that sort of in the testing testing space pretty well yeah.

D

A

Cool um what else do I have um so on the also on the Omega side, um so you know release the tool chain. The docker Docker may go back. uh I am I, haven't gotten a ton of feedback on it, which is fine because it's mostly for our use, but really that that's what the primary you know, user of it is.

B

um But if anybody else.

A

Has picked it up and use it and has feedback, and you think other tools should be added or you think it's it's you know whatever um uh it is. It should just it should just you know, Docker build blah and it should just just build, um and um you know we have users, you know to find things um I I do I, do have some plans on you know, adding better support for scanning things like you know or like existing jar files.

A

So you can just kind of point it at a thing and you know have it go, um but uh if there, if there are features that you think if it had it would make the lives of security researchers easier and the the intent of this is always it's a security researcher Tool. uh It's not really meant for um someone to plumb into their own. You know their own repo, um although I guess it could. But that's not really the purpose um and please open an issue or you know, get in contact.

G

Are there any particular communities Michael that you would like to get feedback from you talk about? Obviously, security researchers is pretty squarely aimed there, but is there any.

A

I, probably so, I don't know right now, I I think I might know in like two months. So we have a uh software engineer. That's going to be starting soon, uh so I'd imagine part of their job. You know, will be to kind of push on this on this tool. A little bit further and I. Think it'll show like where the edges are um I, know I, think I think the Java ecosystem is probably its least mature support.

A

um I think for the you know, and and.

D

A

In general, the the the architecture is start with a package, as opposed to start with the repo. So we probably need to think about that a bit too, because you know you can do some analysis on you know a python or node or whatever npm package, um but a jar file is going to be compressed class files unless you're like decompiling and then trying to recompile or doing something weird um you're only going to get value if you go back to the source and then kind of start from there, um so we that support.

A

Doesn't it isn't too good at this point? How.

G

Does that compare.

C

With DOT net sorry sorry.

G

I'm starting to drop there I.

E

Just got it it's hard for us to know, go why don't you go ahead? I've already talked.

G

You you're speaking of uh you're speaking of uh source code and so forth, between java.net. How does that compare Michael.

A

um The same thing nougat or existing nuget packages are going to be um I, believe I do do I'll spy against them um so that some of the things will work but I, don't believe code. Ql does a very good job with um uh decompiled.

A

um Semcraft might do better and.

G

Which is why one of.

A

The the it is kind of kitchen sinky, where, like everything, goes into a directory, and then the tools are just like go explore. Tell me what you see um so I have a little bit more rigor on that would be certificate.

E

Yeah so you're saying many of the tools want the source code. It doesn't do a lot with uh just the compiled versions, uh but I I know there are tools that work on compiled programs. I mean you know, PMD and spot bugs, for example, are specifically Java tools that uh look at I. Think yeah. My apologies I see too many tools, but I think both of them at least one of them works on the source versus the uh on the bike goes as other source.

A

Code I think you're right, yeah, I I, don't remember which one but I remember coming across under that. So say that would be a great great example of like hey you shouldn't. You should use PMD or spot bugs, or you know whatever. Instead yeah.

E

I mean I mean what what you might want would be something where I mean really. What you want to do is analyze both you want to analyze the here's. What actually is going to be running and some tools are designed for that and then be. You know, so do the analysis of the compiled package and then, ideally, it can automatically figure out from that. What's the expect what is supposed to be at source code?

E

What I would call The Fugitive source code and then go grab that and analyze that also and and for extra credit you can see if it rebuilds yep, that's reproducible, build yeah, I mean I'm sure this extra credit is just if, if the package doesn't rebuild, it may not be a problem, but it may be yeah.

D

um So our guys have done a test drive on on uh on the analysis, tool chain and and uh yeah they're right now, Conjuring the results, so I I will have some feedback when ready, I'll, send it directly to you and uh what there are some minor issues like there's a python version, a python version thing where python version two versus three uh mismatch for which it was not building.

D

So it took some anion, so maybe some clarity Etc needed there uh as far as and also the the fact that the package needs to be provided that that was kind of a uh like. A stumbling block on like just to wondering about like okay for us random project from GitHub like what would what needs to be provided, uh there's kind of like at the starting point, but yeah I mean these are feedback that already have been discussed. Specific feedbacks when radial send it to uh you yeah that'd,.

A

Be great you you and not to go too far down in the weeds, but you should GitHub um tags I believe are a valid package URL in the way that that I I use them so I think it's like like package, colon GitHub slash, you know, Foo slash bar at TAG, name, I! Think it's something about it! So if you um I I, we we can connect on it, but but you should be able to do that if they're tagged.

D

So that's why I I can't really, like you know, say like exactly what the issues were, but there were some like minor needling, but I mean they could start in, like so a few hours or a couple of hours, and a majority of that was actually because of the docker build which was taking I. Think about three four hours to finish. We.

C

D

One other thing was which was not peers and we were not sure of like where how to because of the uh licensing of code ql and red R. uh How could this be uh like? What are the implications of like running it on some other projects? So what we did was we actually created a bill since these two projects, uh code, ql and radar and and then I mean we could run it on whatever we want just for trial. uh So is that what what would be the recommended way of doing things I mean?

D

Maybe that that specific guidance of like what there is no to the legal uh thing about like what can be run? uh uh What can this be run on Etc, but maybe some clarity on that would be good yeah, that that.

A

That one's far in so that one is so code ql and I'm, not a lawyer. So this is the official disclaimer of like I. Don't really know what I'm talking about, but um my understanding of the code. Ql licensing terms are that you can run it on I believe anything with an open source license.

A

With Source on GitHub I believe that's the depth thing and and the the link to the coql license should explain like that. Now.

A

If, if those are true, which is true in the vast majority of cases, um then you as a consumer should be fine and you can run the run the tool chain now, if you wanted to run the Omega tool chain on your own proprietary source code, um that may also be allowed through the code ql terms, but maybe not um I, and- and there are a couple areas where you definitely cannot like are not like allowed to do by the license.

A

um So you would have to you know, comply with those.

B

C

D

A

We're I didn't see where Daria I thought redari was just like lgpl yeah.

D

We just didn't want to like Mark with like these two ad licensing issues that reflect so, let's take them out and it actually helped the bill. So we could clearly took about three four hours to build without it took like a few minutes so yeah that.

A

Actually helped even and yeah and the reason I I pre-built code ql in the docker container, is it speeds up later execution significantly. Otherwise it needs to recompile the rules each time it runs, and it's it'd be ephemeral at that point. So you don't need that um the other one is uh snip code, uh I, believe I I pushed an update to it to use snake code, um but only if you provide the license key.

A

So um you know we have a license key, so we can do that um and it's it should just be silent and and fail fail. Happily yeah.

D

We didn't get anything from sneak or didn't hear anything about it. Mostly. It was FPS from like same grip. That's that's! What we're guessing they're getting more most of of them! Yeah! That's.

B

D

We were running specifically on JavaScript and uh and PHP uh projects just to see like what what sort of results are being found. In these cases, nice very cool.

A

um So that actually segues nicely into the into the last thing that I'd like to just throw out there.

A

So we have results um so so we've as part of Omega we've collected the results from lgtm.com, which is like 350 something thousand projects. um We have the infrastructure. Now we can scan. You know entire ecosystems.

A

um The problem there is, then we get a giant list of you, know millions and millions of findings and the ability to efficiently look through that I don't mean like how do you grab, but um you know we we started on a portal, uh I mean going back, probably six or eight months to kind of bring in all these findings and show them in a per project or a per finding type. So you can do horizontal or vertical kind of analysis, and then you can triage it and say that you know Yep.

A

This is this is a real thing: I'm gonna, you know it needs to be. You need to go out to the to the.

G

A

Or this false positive or all these are false, positive or this tool is garbage or whatever um and I'm. Thinking that there might be value I mean the attempt was always to open source that when it gets to a point that we're not completely embarrassed by it, you're not there yet.

A

um But you know I'm wondering if that would be of value to the larger Community.

D

F

D

A

A as a you know, tool I mean you can run it locally. I guess.

E

I'm one of those people that says you know you know as long as you're, making it clear like putting his version zero point. Something, and maybe you know maybe clearly stating you're going to read me you're, better off releasing earlier than waiting until it's all wonderful and perfect yeah. uh You know just you know uh now, if you're, if there's a licensing, question, hey I, don't know if it's legal for me to release something, that's a different matter! Yeah go! Go!

E

Make sure that that there's nothing that you're going to accidentally release that will not that you don't have authority to do so, but yeah as soon as you've made sure that it's legal to do just post it. Even if you don't get feedback you can. You can always go back and said. Yes, you have that opportunity.

F

Hey Michael just to check for understanding this is the and I'm sorry I'm I'm new to this. But this is the my understanding is Alpha sort of the identification of the critical few on the Pareto projects and then Omega sort of the longer tale of projects is. Is that right and and on that, do you have any Viewpoint or any? Can the data tell us like what are the most used projects? Then? What does the long tail look like by industry.

A

So there are, you know: David David has a great answer for this. For the.

E

Yeah, okay, so so I I, think that was a dump dump to me. uh So um uh so I'll make a first that thumbs up all the hard Parts on Michael uh uh people have very much worked on that. uh In fact, a number of years ago I led an effort to analyze, to do quantitative analysis, to try to identify uh critical projects and in order to scope it down. We specifically looked at the Debian uh packages.

E

uh Just to kind of you know: hey, let's, let's, let's focus on a specific scope, because at the time you know there was just a hey. We there were some let's hurry and get at least a reasonable analysis out. That's actually public called census. One much more recently. Harvard did an analysis using SCA vendor data um when they basically average their Z scores, and we can talk about that which is I.

E

Thought was a very interesting and useful way to handle that, uh and they came up with a sorted list of what were the top dependencies as far as both direct and indirect dependencies go within um uh language level packages, as opposed to the system level, packages of say Debian by the way, although we've analyzed Debian most the other Linux distributions pull in a lot of the same packages, heck even a lot of the other distributions, like the bsds, often end up using some of the same packages, because you know you want to use software.

E

That's you know reasonably high quality does particular kinds of functions, and so, while it's certainly not all the same, that's not true. uh There's a lot of overlap. uh Similarly, for the ecosystems within each of the ecosystems, certain packages showed up over and over and over again I. Don't think. You're you'd be terribly surprised by that, um although the timing was not where we would have liked. For example, log4j was very high on that list uh would have been.

E

You know, it would have been nice to know that before, uh but you know that's, uh you know you wanted to do everything at once. uh So census, 1 and census 2 are publicly available. You can go, take a look at them right now. uh You can complain about analyzes, as with any analysis, I'm happy to tell you all the words and everything because you always have limited data sets. You know, there's always challenges any kind of analysis, but we do have some of this.

E

There is a separate group called the critical projects working group which has been who actually they were the working group that oversaw the Harvard work and um basically they've been trying to take the various data sources and from that try to identify some sort of critical list. They do have a draft list now they're in the process of trying to make things better, uh but I'll tell you someone who did census one and the some of the challenges showed up in census. 2 also um there's a lot of context.

E

You miss with the purely automated metrics-based analysis.

E

um I'll I'll give you, for instance, in census, one the mail transfer, the the mail transfer agents. Mtas were really high because they're directly exposed to the internet. uh They process data in all sorts of crazy formats, but the reality is that very very few people run fully open mtas directly, so even though it had all the earmarks of man, this is important when you start asking yes, but it's more complicated.

E

So um it's it's my position that we really need to do all sorts of analyzes and so on, but then add the human factor to add that information that otherwise you know the quantitative analysis really misses so.

F

Let me just editorialize to say I think- and this may be an intersection point between openssf stuff and finno stuff, but there's a lot of people that that care a lot about the answer to the question. What are the top used open source projects in Investment, Banking and banking Capital markets, and that answer may be industry. The dependent like I would.

E

Hypothesize I would hypothesize that.

F

Our use of numpy and pandas is higher. I would also probably suggest that our use of O camel as a language is probably higher than other Industries just to pick one right, and so so some of the and every industry is unique so that that would just be and then in the longer tail. Looking at some of the projects that are in the longer tail so that industry context it just would be helpful for us and that's why I was curious. So.

A

So for 2023, like for Alpha Omega, we are specifically looking at different verticals and how do we apply the not you know not.

B

One size fits all, but.

A

It's you know for for whether it's medical or Automotive, or financial, or whatever, like essentially what are your top and and let us think about how to how to manage that, and some questions are going there on on funding. You know how do we, you know, you know, uh put that focus on those areas, um but uh the intact yeah, like obviously there are different. uh The curves are different in different. You know areas.

E

Yeah and indeed the critical that this is actually an area that came up um earlier this year, I mean the critical projects working group, but it was kicked off. The theory was, you know, hey we're going to make a list, you know just looking broadly and I still think that's appropriate, because when you start merging the verticals, you still find some things that are really really common, that it's worth everybody, but you're right.

E

They particular verticals or particular organizations, um are going to have a different set and so I think they're actually they've been talking about not just making a list, and you know working out their process and making a list, but coming up with a process that you that can be franchised. Basically, you know hey your organization's different. Your vertical is different. Here's a process that we use, but you might want to reapply to your particular vertical, because your answers are almost certainly going to be different.

E

I haven't talked about this as much in the finno setting, but I've I've talked with some government folks, particularly in the U.S federal government, where they have exactly the same concerns. Their lists are not going to be the same as a generic General list and I think that's right and So, eventually, I think it's incumbent on organizations and verticals to try to help identify the things that are unique. uh I would be unsurprised just from undercuts I've heard that there's a non-trivial number of Finos- it's not just python but python 2..

E

And what do you do there? So you can talk to that Rob. But but you know that would be. You know there are interesting challenges and I don't know, I have the full answers, but I do think that that makes sense.

A

And and frankly, that that's why you know applying like.

C

Larger shovels.

A

And construction equipment on the alpha side to larger challenges is kind of one area, even at the per industry, you know basis and then Omega to just kind of absorb the yes give us a list of another thousand projects, we'll, like you know, put in the backlog and just turn through them, as we as we go.

A

um I think probably make sense as an as an operating model. The theme for 2023 is going to be like let's operationalize, like what we've been doing and write it down and have something that's kind of repeatable that we can. We can share.

F

um Just related to that, a bit of just meta feedback or thoughts for what it's worth is that in terms of engaging with our industry, I'd say too that you know I think you know I know some of our other organizations from Capital markets have been taking um significant roles in openssf. I would just say that um I I don't want to make a sweeping generalization, but I would share that. It's probably a reasonably safe generalization to say that um uh there you'll have a higher rate of success.

F

If you want some one of our developers to take something on, if you can suggest hey, there's like one or two things, we really need help on. Would someone like to help on XYZ, rather than just put like a long list of issues in a GitHub thing, um the chances that someone's going to go and just like? Take and I'm? Not this isn't to say that we're the only busy industry everyone's busy, but it's just it's just kind of a thing so totally.

B

G

D

One thing I do one thing that is also related just one minute. uh One minute comment, so Michael you and I. When we were reviewing, for example, the Pi Pi results, uh you made a comment or like we discussed about like the content of a file, not considering them as uh as stain sources Etc, but I've had clients from specifically like female Industries, where these are also considered actually highly valuable, so it's dependent on on a particular vertical. It's also the the kind of threats they are very, very different. Yeah.

A

I, don't agree, yeah if you told me that one of these things we're going into a pacemaker or space shuttle or another kind of Can't Fail system. The bar is going to be way higher than like. This is this is part of part of the calendar widget on a startup website, um but yeah we should. We should think about that. We are over. Thank you all very much for the conversation. This was amazing, um stay tuned and see you next month and please engage on slack or um or just get in touch and.

F

I just I would say a lot of us can't use slack. So that's another thing, uh but we'll another.

E

Industry, wrinkles, yes got it, yes, send email you're, not the only one. So.

F

Yeah we we we, we yeah, there's, there's some focus on on us using channels right now, so yeah got it all right. Thanks all right, hi.