OpenSSF Scorecards Biweekly, 13 Jul 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Scorecards Biweekly Sync (July 13, 2023)

Description

Meeting notes: https://docs.google.com/document/d/1b6d3CVJLsl7YnTE7ZaZQHdkdYIvuOQ8rzAmvVdypOWM/

https://github.com/ossf/scorecard#connect-with-the-scorecards-community

A

B

C

Foreign, a few more minutes for people to trickle in can everyone see the notes shared on my screen.

D

Yep I can see it.

C

And just in case, uh the new document has people typing suggestions. um If you want to fix it, you can join the same scorecards Dev Google group, and it should give you editor access. But if suggestions work then uh carry on and we'll accept them when we can.

C

It looks like everyone's doing it already, but as you file in, if you can put your name on the list in the doc, uh if you don't have it I can post the link in the chat. uh It should be at least viewable by everyone. These days.

C

Yeah, so that should be the docs, those that don't have the link.

C

All right, uh it's about three after so I will go ahead and get started for after uh hi everyone, I'm Spencer, one of the scorecard maintainers and I'm the facilitator for today's meeting.

C

um So I will be protecting this window uh with the um the notes as we go through everything, um and we have a relatively small agenda for now, we'll see if anything gets added later.

C

um But usually we start our agenda by inviting anyone. That's new to the sync to introduce yourself say a few words um who you are what you're hoping to get out of the sink Etc. um This is optional, but if you want to now go ahead and mute and introduce yourself.

D

I will- and this is how long I can stay so I'm just going to introduce myself and then run because I am here um and you're getting to know the rest of the open, ssf Team. My name is Adrian Markham I have just joined uh LF within the past three weeks as a technical project manager.

D

My first big assignment is to help get the scorecard project up and running, so um some of you have already poked and I'll continue to poke around and just try to see what we can do to get um get the progress happening a little bit faster. If we can, um let me know how I can help and what needs to be done.

C

All right, thanks, Adrian.

E

I can go. My name is Jonathan Howard work for Lockheed Martin uh I'm on the hopper project and I did some work for uh well on a hopper plug-in for the open, ssf scorecard I'm here. Just for uh you know, awareness and exposure.

C

Cool thanks, Jonathan good to have you.

C

All right with that, we can jump into things, um so it doesn't look like we have any announcements. I did miss last sync um I, don't see it um scorecard. Vae 11 came out a little bit ago, um which you know had some things in the patch notes, but leads me into the first agenda item um and that was sort of a change in how two of the checks work, where sometime between V10 dot I think it was four and v11.

C

There was a change in a check that for gitlab made no sense um so gitlab didn't support the notion of GitHub workloads, understandably, and how GitHub token permissions work. So there was this PR that basically changed it to say that if you don't have any GitHub workflow files, then we're going to return a minus one score um for repositories that were on GitHub. This meant that the score now dropped from a 10 to a negative one.

C

um So not every repo has a GitHub or profiles and there's some discussion in the issue about you know: does it make sense to have a negative want?

C

Does it make sense to have a 10, um so I thought Drogo made a good point um about how it impacts the aggregate score the overall score once you combine all the checks when some of the checks are minus one, um so I think this issue is sort of simple enough that we don't have to come to a conclusion today, but just sort of drawing attention to the issue if people feel like weighing in and then I'll be pinging, some of the maintainers um I'm not sure how many others are on the call right now.

C

um Yeah so David in chat says that we say the score is zero to ten, so negative one doesn't make sense. Traditionally, we've used it for when, like we either don't have enough information to score a particular check or if there's some sort of error that prevents us from doing the analysis. So this might be.

C

You know, um for whatever reason the token you're using doesn't have permissions, there's different tokens, and sometimes some of them disagree with certain checks, or you know we're trying to score you on code review, but the only PRS that we see are from you know depend about so yeah I mean negative one's unfortunate, but it's usually reserved for situations where we can't make a judgment um and I think even with workflows gone, we can make a judgment, um but yeah.

A

C

That's the issue. So if you have some comments, feel free to mention something either in the issue or during the meeting, but that's pretty much it for the item.

B

um What are we trying to measure here, I'm just thinking so one of the projects I work on is k-native and k-native uses both GitHub actions and prow, which is an external system running on kubernetes, um if you're doing a security assessment and you're just looking at the GitHub actions, piece you're missing a whole bunch of other stuff, that's running in order to produce a release, and so you might spuriously give a high score when actually there's some other stuff going on that you don't see, that's worse and so I'm just trying to figure out what are we trying to do with this score?.

C

Yeah so traditionally scorecard only supported GitHub repos and it was common for GitHub repos to have these gotchas where token permissions are a little unintuitive and the permissions are, you know, overly broad or it's possible for um basically untrusted injection, so the check was originally interested in.

C

Do your GitHub tokens or GitHub workflows? Have these common GitHub problems um so.

F

That's part of the reason.

C

Why for the gitlab stuff it returns minus one um and I I, just don't think it's been expanded or a parallel version has been introduced through GitHub.

B

Well, but also you could be using something other than GitHub actions for your CI, and it seems like what we're trying to assess is how trustworthy is your CI.

C

A

C

Also so like historically when it was just GitHub, it was how trustworthy is your GitHub actions, because, like people still used Travis or Circle Ci, or something like that and I, don't think we ever like. That was to my understanding, not the part of the check because, like I I'd have to get input of someone, that's been on the project a little bit longer, but I'm proceeding under the basis that it was just to look at GitHub actions.

A

G

Support you there Spencer, like the the point, is to check to see if you're doing something wrong, not to check if you're doing everything right. So it's it's really just for the the things that scorecard knows about. Are you doing anything wrong, so yeah somebody's using Travis if somebody's using whatever CI it, wouldn't give you a bad score, but it also wouldn't give you you know and, and that's just the the extent of scorecard's knowledge um and then it would.

G

You know, as scorecard becomes more and more developed, it would maybe learn more and more about other CIS and be able to detect uh improper usage there as well. Does that make sense, Evan foreign.

B

It does I just wonder if hey we couldn't detect any workflows, um whether we should be doing this as a negative points from a hundred system or like a positive points from zero type system.

B

And so we can't give you any points for your CI, because we can't tell what it is.

G

Yeah, that's an app that's a great question. I think that's the topic of this discussion like what do we do? Okay, yeah.

C

Yeah and the way scorecard scores right now, there's not sort of a great answer of what happens because, like the aggregate score is based on all of the checks and if we continue to say you have no GitHub workflows, so everything is a 10.. You know that sort of vacuously, true that will sort of boost up your score, but, as jogo mentioned in the issue thread, if you say you know, you have no workflows. I can't really give you a score based on this, then that inflates the weight of all the other checks as well.

C

So it's kind of pros and cons of both approaches.

H

Yeah I was immediate reacting to the hey. We tell everybody that we give score everything zero through ten, so it should be at the very least. We should fulfill our promises and score it 0 through 10.. um As far as the hey. What do you do when you don't know I mean that's always been a challenge, um and uh you know in in one additional Quirk I should add. Is you know some systems may use multiples. I mean I will point out the best practices badge. We use GitHub actions for a couple things.

H

Most of our checks are on Circle CI, um so I'm not even sure what scorecard does in that case. So.

H

But I think we very least need to have zero through ten and for not giving them tens because we don't know I mean zero, because I, don't I can't see that you have a CI pipeline.

H

um I think it's defensible, but then we need to say: hey we didn't detect it. um You know you might have one. We just don't see. One see I see a CI pipeline, but you have a problem. I can see it. You know a certain score at that level um and more generally, I think we always want to have the higher level description in the scorecard, the more General description and then within the scoring in more details. Hey you've got a pipeline. It has these kinds of problems the score, because you know hey.

H

We support one pipeline now, we'll support others later. um It should be possible for us to figure out approximately where our value is going to end up. Just as we add more and more of these cases, thanks.

C

Yeah um I may have to find a different room in a second, but this check isn't necessarily concerned with. Do you have a CI pipeline? That's more with the CI tests check looks at this is if you're using GitHub actions. Is it configured appropriately.

I

Yeah and just one comment: uh David on the zero to ten thing, just in case uh there's a bit, there might be a misunderstanding: the minus one isn't actually a minus one.

A

H

That's right exactly.

J

H

Okay, yeah I, don't object to that. It's the if you were actually averaging it in that would be a problem, but as a sentinel, that's fine.

J

um I I can uh uh so I I'm uh put some comments in the issue as well, but it's not inconceivable that in the future a scorecard would support um other CIA methods um and it's definitely like open for Community contributions uh like we I, don't think all of us are experts in circle CI.

J

But if an expert in circle CI wants to come in, say, here's what a dangerous workflow looks like at Circle, Ci or I think some of the gitlab people who have more experience with that would be like this is what a dangerous workflow looks. Dangerous pipeline looks like in gitlab. That's certainly with a scope of the scorecard, and we we should um do that and uh the the reason I believe that uh a inconclusive result is uh suitable in the case where we don't find workflows.

J

Is that we're we're just uh it's almost like scorecard, failed to do something or scorecard just not programmed to detect a particular kind of workflow, so it's just like scorecard itself is inconclusive and just can't say anything one way or another about whether the repo is falling good, uh good practices, yep.

C

Yeah I think that argument is a lot easier. Sorry Pedro, real quick um if we had structured results now sort of thing where, if we could say this is the GitHub sort of check, then returning minus one uh makes sense. I I think you know you bring up a good point that right now the check is dangerous workflow and we can't evaluate it in the sense of if this is a git lab or a GitHub project um and I. Think.

C

For that reason, returning minus one for a gitlab repo makes sense, but for a GitHub workflow, if it doesn't have any workflows, I think you can still score it on zero to ten, where none of the workflows are dangerous because there aren't any workflows.

H

Yeah it sounds like you're giving them a bonus, though, for not having your workflow, which doesn't that sounds right. It doesn't sound right.

C

I, don't know if it's a bonus so much as this portion of the score has always been towards workflows, and if you don't happen to have any, then you can't be marked down for it.

C

I

uh Yeah I know, and but uh just like giveaway, like raghav mentioned of like we might include dangerous, CI CD on other platforms, uh the minus one becomes uh more explainable becomes more defensible. Shall we say in in that regard, because then it's a minus one. Now and then we add information on Travis uh in tomorrow and then the minus one becomes a real score.

I

um That is more natural. Shall we say than you get a 10 out of 10 and then uh we add Travis tomorrow and your 10 out of 10 becomes a zero uh that becomes a bit that's a bit. I might feel a bit weirder. Shall we say in terms of hey. If you couldn't judge my dangerous workflow, maybe you shouldn't have been giving you the points all along, so I, don't think anyone's going to be currently complaining about getting a 10 out of 10.

C

C

So I mean it sounds like there's some um votes both for and against I'd be curious to get um laurent's opinion. As you know, we work towards structured results on if we just maintain the status quo for now on tens and then, if structured results, are you know six months out? Does that change sort of the discussion?

C

um But yes, uh you know it. There's some good discussion about.

C

If we're only checking for GitHub workflows is that.

F

C

Know a point boost and do certain repos get Point boosts for either choosing to or choosing not to, um but yeah. Thank you for those that left comments in the issues. um I I think it's hard to make I guess a conclusive decision with uh only two maintainers, but thank you. Everyone for comments, um so I'm happy to move on to the next issue. uh Pedro.

I

Yeah, so this issue here is the fact that the dependency update tool check uh when it's looking at the for when it it can detect, depend about by looking at PR's, uh and they can all just check like the history of PR's in the project looking for depend about PRS, and if there are any, then it currently just assumes okay, this project has depend about and, however, a project might have turned off and depend about because many projects get annoyed and depend about, and it happens relatively often that'll project will just turn it off um and it's hard to detect when looking at PRS.

I

If that has happened, um one solution is to instead of looking at the PRS or as well as looking at the PRS is uh looking. If the project has the dependabot yaml file the settings file for the pendabot, but that file isn't strictly required for the pendabot to run. You only need that uh the depend about yaml file. If the project wants the consistent, frequent dependabot version bump PRS.

I

However, a project might not want that, and only opt-in for what they call the dependabot security updates, where dependabot will send a PR to fix dependencies uh having known vulnerability, uh and if, if you have a product, only opts in for security updates, it doesn't need a depend about the ammo file. So the question that ends up happening here in this issue, especially further further down, is, uh should we should it be like?

I

Should we perhaps change or depending the dependency update tool, when looking at the pendabot uh to look for the file or not basically like should depend about only count if the project has opted in for version.

A

I

Or only for secure or is the fact that the project has accepted security updates sufficient, but then also if that is sufficient, which means no file we can look at.

I

um Is there a way that we could possibly determine that better than looking at PRS or something I, don't know yeah.

H

I mean I, I, guess I'm going to drop back the first principles. The the goal isn't depend about.

H

The goal is that if there is a security vulnerability, if there's a vulnerability that they updated, prompto right so um I mean so if they have that file, that would suggest that maybe they're doing it um one other approach is looking at past uh I'm looking at what what are the simple indicators, um obviously, if they have depend about uh uh commits that have been merged in that's a pretty good indicator, um you know worse comes to worse, and this would be suddenly a whole bunch of analysis.

H

You know if they haven't had any updates based on aversion. We can look and see. Are there any known vulnerabilities backwards in that should have been merged in and not. The challenge is, of course is that suddenly is a lot more work, even if you're, only looking at lock files.

I

um Yeah, the the current setup for dependency update tool is the second one of looking at DRS and how mistaken it only looks for like. Are you receiving the pendabot update, dependent Bots, for example, or renovate bot? Whatever? Are you receiving uh those PRS I'm, not sure if it looks at whether they're merged or not, but just like? Is it? uh Do you have it?

I

The issue of looking at PRS is that if you don't have a file to know that the panda bot is running right now, sometimes it'll detect dependabot because of a depend about PR from three years ago, and then the issue is: has the last depend about PR, been because this project doesn't have any other vulnerabilities that or hasn't had other vulnerabilities that depend about where they're fixed or have they turned it off?

I

um So we don't have that signal. Unfortunately, as far as I can tell, and so the file would give us a cheap way of figuring out that of skipping that whole thought process, but it also is forcing people to accept version updates instead of just security updates, which is I think probably closer to the spirit of scorecard.

C

I I don't think it has to be an either or kind of thing where a project can just sign up for depend about security updates like they're doing now, and we do attempt to look for it in recent PRS. It's just if a project isn't happy that you know hey I have depend about installed, they can go out and create I'm curious if even like an empty dependabot.yaml Works, where you know you don't sign up for anything but I, I. Think in the thread um I can change my tab back to it.

C

I was sort of suggesting, like a repository, can add a dependabot.yaml to say yes, I am actually using this sort of thing, um so I I, don't know if too much is required to be changed on the scorecard side other for other than the issue. You mentioned that this project three years ago used depend about once and then turned it off sort of thing.

C

um So, at least to me what makes sense is some sort of look back with a window which will hopefully get rid of those false positives, but may introduce false negatives, in which case the solution would be potentially adding an extra file, which kind of sucks and scorecard has this problem every now and then, where we don't detect everything um and we're working towards user specified remediations or like saying no, no! No! This is why that's not accurate, um but we're not there yet so in the meantime, I I think are you just seeing.

C

I guess I'm I'm saying: is it acceptable for a repository to include one if they want to get rid of a false, negative and you're asking if it should be required or.

I

Yeah, like I mean like I'm, asking like uh should yeah like, should scorecard change to require the Dependable uh yaml. Another thought that crossed my mind is. It could be like uh five points for having the ammo five points for PRS uh for having at least one PR uh or something I. Don't know.

I

um Yeah I mean this year, like I came with a question. I didn't come here with an answer of what the scorecard position should be just raising this issue.

C

I think that's why I opened the check doc? Assuming you know, the docs are accurate, which they're not I, guess the purpose of the check is to see if you use a dependency update tool and If part of this says out of date. Dependencies make you vulnerable to flaws and dependabot is giving you security updates, then, to me, as written, requiring a file, that's not necessary for what the tool is trying to check wouldn't be necessary.

C

um If we want to add like a recommended like hey, if you're using this but you're, not we're not picking it up, here's something you can do as a hint I think that works, um but I'd be curious. If someone feels strongly, if saying you know the point of a dependency update tool, isn't just for security updates, but also blah blah blah.

H

I I, let me um I certainly encourage updating in general, because if you don't update, it gets harder later on when there is a security update but I'm a little hesitant to press the. You know, please keep your everything updated at all times, uh because there are cases where you can't update um I'm thinking, particularly of a case where uh there's some software components that we're licensed to a non-open source software component.

H

There's a new version. Oh wait!

H

So there may be good reasons: you're, not using the current version having nothing to do with security, foreign.

C

I, don't think any of this is looking to see if the PR's actually merged, um because, like on Dependable, you can say ignore this version. Ignore this like ignore this dependency. Okay,.

H

All right, so the question really does does become. Do we want to just encourage updating all versions, period or focus on the security updates.

H

Two minds on this I do think in general. It's wise to update I also do think that you want to prioritize the important ones whatever that means for the bigger projects. It's a little over Dependable could be a little overwhelming if you actually enable it for all versions, I'm pretty sure we only enable it for uh uh the security vulnerabilities, for example, so best practices.

K

Yeah David I agree with you that there's definitely good reasons for not merging stuff in and and kind of where you're going earlier, the ideal state would be.

K

Could we detect if that caused you to start taking on vulnerabilities, because you weren't updating, that's kind of more of where it starts to become a concern for security, because I think what you just said is very common, where you at least don't Auto merge things that are beyond that patch version um and you may choose to delay even manual merges of those minor and major updates for very good reasons, but those Also may become unsupported, and then you do have a legitimate security concern at some point.

H

H

I guess we're on firm ground on asking people to you know, make sure you've been automated. You have automated enabling of security vulnerabilities and you know encouraging. You know, push the button and update the question here is: should we broaden that to just you're out of date period as a warning.

C

I think some of this would be helped if GitHub had a way of querying. If someone had depend about security updates enabled, but it's just not something at least I. Don't think that you can query with a normal token and I'm, not sure if you can query it with an admin one either, but.

K

Yeah, the idea of encouraging it because I feel like a lot of projects, wouldn't go, do that by default, and it really is a huge Boon towards your security to not have that maintenance burden of constantly watching for 50 dependencies. You have so I I really do think. It's a good rule in general.

H

To enable an automated review.

K

Yeah, to enable an automated, automated uh dependency update, that is merging things like I, think that is a big security help for a lot of projects.

H

Absolutely absolutely yeah, so the question is, it comes back to it. Do we want to emphasize that only for the security vulnerabilities or more generally,.

H

Be nice if we had some announce like somebody had done some analysis of the impact of doing it either way.

H

I don't know if it's, uh if there's any such research.

H

I'm doing a quick, Google Search right now. This sounds like the sort of question that somebody might have answered: yeah I'm, just picking up you know, GitHub information about how to set the configuration.

H

ah Research gate on the use of dependabot security, pull-up requests interesting all right. Well,.

C

All right, I'm, just trying to capture some of our discussion in text um to sort of summarize.

C

So it's like they're sort of two choices where um either we require it or keep it, as is which is a signal, but not necessarily required, um and then I. Don't think this was part of Pedro's question, but are we trying to get I guess a decision on?

C

Should we be disqualifying the current implementation that looks back indefinitely for a defend about commit which I I think the answer is? Yes, it just becomes very hard to say how far back do we look or what are we willing to accept sort of thing, but.

A

H

Yeah sorry I'm just trying to think through the pros and cons here.

I

I

Like a clunky way of doing it would be to not make a decision and instead say hey uh if you've got to depend about PR over the last year you get a 10 out of 10.. If the last one was two years ago, three.

G

I

Ago and you don't have the dependent body ammo file, we don't know if this thing's still running, but maybe it is so we'll give you a couple of points we kind of give the. If they don't have the file we give them fewer points as time goes on, um as we lose confidence that they actually have depend about uh turned on, like it's a clunky.

H

It's actually it's clunky, but it's not as crazy as it sounds, giving them less points for something we have less confidence in um it's a little clunky, but it has.

A

H

uh You know because I guess the the easy solution is, if they add this font the depend about file, then we know for sure yeah right, that's the one well I guess they could turn it off and have that yaml file in there couldn't they, but.

I

I, don't think so. Actually, if you have the yaml file like you can't turn it off, I I understand it just kind of keeps going.

C

I think the question is what happens with an empty file or a file without an ecosystems because, like in the file, you say this is an ecosystem for some of my packages and I guess what happens when you don't have that.

I

That I yeah I have no idea, but I wouldn't be surprised if what it means is that it runs as like. It runs the the scan, which means nothing, but if you have the, but if you have it active for version updates, which is what you would with the Dependable with the yaml file, you also have it active for security uh updates. So it actually might be that, having like a blank file, um also automatically puts you in for uh security updates as well, but haven't texted. It so can't be sure foreign.

H

Yeah so I mean really, it sounds like if we have, if you have a file we should check, but our current under estimate is that that probably means it runs. You know probably should go. Do that research and then come back, but let's assume for the moment it does. It seems likely.

H

So in that point the only real question is: what do we do if not and hey, if there's depend about or some more results and we keep saying depend about, but obviously any other tool that does dependency analysis and helps them keep up to date is what matters so yeah either either we see evidence of it in the in uh proposed changes or request, merge requests, or we see some configuration that suggests that in fact it's it's going on.

I

Yeah so I mean the things I depend about is a specific case because with renovate and the other ones they are like config as code like renovate, you can usually identify, and actually you can't actually I'm I'm I'm wrong right depend about it. Renovate's actually activated on the account, not on the uh that's true, never mind. We.

C

Do look for the um renovate config files? That's the only way we have evidence of using renovate right now for scorecard. Okay,.

H

Okay, but you're still looking for it and again you're you're, just looking for evidence right. If you have evidence hooray, you get some points so we're just looking I mean this is just no different from the other Checkers. It's you're. Looking for evidence, I mean same for gymnasium on the get lab side right, I, don't know if that's being looked for today, but you could certainly look for it at some point.

C

Yeah as I understand it Evan, this is a year using dependabot, but for whatever reason we can't detect it because.

C

B

Know it's not there's no evidence that it's doing something yeah.

H

You don't have it configured, there's no evidence of a config file and there's no evidence of any people request. Merge requests for from it at that point at that. At that point it does make sense to give it a zero score, because we have.

J

B

H

That something's going on.

B

I mean I'm, just thinking about um can I have a depend about yaml, but actually have gone and slid the slider to off.

H

Well, it in general that we're not really assuming maliciousness yeah um I mean there's actually a much simpler way of disabling depend about, and that's simply using a language that it doesn't support that.

B

H

I can write about you know. I can write a million lines in a language that isn't supported and yeah. It doesn't matter what what I do.

B

Or just use a dependency tool that it doesn't know about.

H

Right, which I mean that's a general problem right, yeah.

C

Yeah I mean there's all sorts of corner cases like if you're writing a library that has no dependencies too, like what do you do um so before we get too deep into a can of worms, I'm going to realize back a little bit, but it sounds like the biggest question is: how does this behave with an empty depend about emo?

C

um You know: we've brought that up a few times um so I think a good Next Step would be someone um either Pedro or I could try to find time between or before next meeting, just sort of run, a little experiment see what happens um and then once we have a little more information, it might be easier to sort of make a decision.

H

I think the the coming bound to the brass tacks, though the whole point is if there is a vulnerable dependency, we want there to be automated reporting and, ideally, automated um repair and the only indicators we have are pull request, merge requests and the pre and a yaml file which may have to be more than empty. Depending on the results of your tests. uh The empty dependency case is actually easy put in the ammo file.

H

It will detect that there's nothing to depend on and never report anything you're golden and as silly as that sound and you get a 10 and as silly as that sounds a lot of projects eventually managed to get themselves a dependency.

B

Well, it also shows that you've thought about it.

H

That's right, that's right, you've thought about it. You don't have any dependencies today, that's great! You.

B

Create the yaml, you put a comment in here: I'm doing this. For this reason, yep walk away.

H

H

And if you truly don't have any dependencies, it's a one-time commit.

C

I

uh Spencer, in the previous uh point on dangerous workflow token permissions, uh you said you wanted to get a Long's opinion. He is in the in the call now at least I believe he is at least someone called so far.

C

Yeah um Lawrence, currently muted, but if you're listening Laura for context, we were talking about sort of structured results earlier in the call where there's this notion of how do we judge a repository um or like a repo on dangerous, workflow and token permissions when either so like there's two scenarios, one where it's on an ecosystem that doesn't support GitHub tokens and workflows, because it's not GitHub or it does your GitHub repo.

C

But you've done your CI CD on another system sort of thing, um and it would be easy if we had structured results because we could, you know, say not applicable or something like that. But in the current framework like does it make a difference where, like a gitlab, repo gets a minus one, a GitHub repo gets a 10. But what, if you have your CI in a different system?.

L

So what's the question, does it matter right now.

C

um Are you uh did you get a chance, probably not um to sort of see the issue.

L

No I haven't I understand the high level problem that ins in on some platform. It doesn't make sense to say that it passed because it doesn't use workloads, for example, on GitHub.

C

Yeah so I I think on gitlab no one's arguing that you know if you have a git Library po, a score of minus one sort of makes sense. The question is like on GitHub repos without workflows.

C

L

C

One still makes sense, or you know, because you have no repos, you score a 10 because there's no dangerous or workflows, there's no Dangerous Ones.

C

It's just annoying because having the negative ones does change the scoring.

L

The score yeah I think even for gitlab I'm, not sure that minus one makes sense because minus one means inconclusive, which, which means I, think we we don't know, I, think what we're looking for is like a new one that says not applicable, which is basically a yeah. So I guess I. Guess that's my first reaction about the minus one. Maybe it's not a problem in practice when we compute the score, I haven't thought enough about it and on GitHub yeah.

L

If you don't have, if you don't have workflow I would say it's the same: it's not applicable, but it's not applicable in the sense that you pass, whereas in the on the gitlab, it's not applicable, because it makes no sense. So I guess this this differentiation.

L

Ideally that would be able to capture because otherwise, someone on git on GitHub that doesn't yeah no I, guess those. This is what's going through my mind as I uh address here. So I think we need to consolidate the return values that we have they're, not good enough. I! Guess! Yes,.

C

Yeah I I think you've left some holes in your enumerations for structured results to sort of input more um yeah.

L

Yeah absolutely yeah for that reason that I actually didn't know what we needed so.

A

L

Think it should, we should make sure that it works with structured results. I hope we can.

L

Is there an action item that for me, or is it just something for me to keep in mind for as I send PRS for the structured results.

C

um I I I think it's something you know before. We move structured results to a release, kind of thing we should say and I I think we'll see this as we migrate more more checks, um I I think the only action item for right now um is sort of getting a maintainer majority on If. Part of the scoring change should be reverted if this is just something that we're going to deal with until we have structured results, um I don't think that has to be right in this meeting.

C

But if, at some point you can respond to this issue, I think that's sort of the action item and I'll. You know I'll tag Naveen as well, but I think uh Markov and I have weighed in.

L

um Yeah sounds good, so just at a high level, I, don't think waiting for structured results is necessary, because anything that structured result does is really just reusing. It's reformatting the output, but it's the scoring is happening independently of the structured results, so anything that any scoring we would change I think wouldn't affect the structured result. It's just that the scoring will happen on the structured result instead of happening on the Raw results.

L

Today, as it's passed that if that makes sense for those who are more familiar with the code, we can take that offline offline Spencer. If you want to discuss.

C

Yeah I mean I think there's some interesting like major version. This is how we change scorings, like does it make sense to change How We Do scoring significantly in the middle of E4 like right. Now we have zero to ten with a minus one if we're not applicable. So, at least in my mind, as long as we're on V4, that's sort of what we're sticking to.

L

Do that make sense to me.

C

C

um So Brian I saw that you added that if you want to talk about it, you can otherwise I will just wait a few seconds and then read it.

F

Yeah I thought I'd just take an update that Steven put in slack. It's also generally my update building off of the road mapping, um I, guess presentation or what I I talked about in our last meeting, for just what at least at Google, we were committing to um and the feedback from the group to build a broader Community road map, um I'm gonna be uh chatting with Stephen and um Adrian Markham, who, uh if you haven't met yet is a is a new TPM for the openssf.

F

So if you remember from the previous meeting, Stephen volunteered to kind of walk through how some other projects uh built up, Community road maps with me- and so we delayed that a little bit- uh and it just happens- that um patreon is- is looking at a very similar thing across a lot of different projects. So the three of us are gonna, just have a discussion about what that can look like and then, hopefully for next meeting.

F

We can bring back um not just kind of a summary of what we discussed, but also really start that process of of building up a broader Community road map for the project.

C

Cool thanks, Brian yeah, really interesting to see.

C

Lauren, did you want to talk about this I, unfortunately had to miss last meeting so I think you got a little more insight.

L

Yes, so I think, last time we met, um we discussed the uh the time zone and whether we can alternate between a EU friendly and then a Pacific.

A

L

Time zone I think someone said they would share a doodle or or something equivalent. I was I just wanted to to follow up because I know this thing just slip through the cracks and then we're going to forget. uh But just I was just curious. If anyone has updates on whether something was shared on slack um or anything,.

L

And if not I, don't I, don't remember who was who said they would do something.

C

I think we can um tag um I'm. Looking at the thing from last uh Dan, we can ask him. I would assume that as the one that brought the issue he would at least uh it said.

C

um Okay I think I'm getting confused, but yeah. Let's just ask Dan about. If there was a doodle Zone in slack and.

F

C

Can follow up on it yeah thanks for catching that.

C

With that we are at the end of the agenda, so, barring me last minute, questions um all that's left is to pick a facilitator for the next sync, which is currently July 27th.

C

um So if anyone wants to feel free to volunteer.

C

C

They are otherwise I'm sure we can force someone to do it next week. One of the maintainers.

A

C

Right uh thanks everyone for coming um discussing. uh Please take a look at the issues and leave a comment. If anything you want to say wasn't represented the meeting or added to the notes, um but otherwise see folks in two weeks and thanks, everyone.