CHAOSS Common Metrics Working Group, 11 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CHAOSS.Common.June.11.2020

Description

CHAOSS.Common.June.11.2020

A

So we're now recording welcome everyone to our chaos. Common working group on June 11 I, already shared the minutes document, so please, head over there. I'm just gonna share my screen. While we go through the action items from last week or two weeks ago, do you see the minutes I'm not always sure I'm sharing the right screen.

A

Can someone confirm that you're seeing the common metrics well I.

B

Am seeing the common.

A

Metrics page awesome. Thank you. So first agenda item is to review action, items and notes from previous meeting.

A

We have here the first one is Don Foster who is not on today and I, didn't see anything come through the issues or pull requests, so I'll just copy paste that up the next one is Shawn yeah.

A

You wanted to look at language distribution within a project. Do you have an update, yeah.

B

So my update is: there's not a metric ready for that yet, but there will be by the time we do release. So in two weeks: yeah, that's I! Guess it's a little bit was it two weeks from tomorrow? Is that.

C

B

A

Weeks from yesterday, okay.

C

Yeah yeah 13 days, then 13.

A

C

A

And this is implementation in augur that you're talking about not.

B

Actual new metric, okay, it's theirs! If there is an implementation in Agra right we'll be you know, new metric.

A

Okay, do you already have a draft you can share with us no.

B

Don't okay, this one practically writes itself so.

A

Alright, we have.

A

The concept or location metric that we do want to look at later, so I'm not going to drop anything in we have Daniel is not on the call so I'm going to just carry this forward.

A

Also make that comment that dance is not moving forth this week and then we have given recent call coming to the list which was done perfect and then.

A

There was another one for Kevin to move the metric onto the website and this is the time to first response and it is on the website.

A

So thank you. Kevin.

B

And we have an implementation of that in order to that I can share.

A

A

Yeah we can update the metric and add new visualization to it. Mm-Hmm just.

D

Make sure that you add that note to the release comments issue the comments.

A

A

A

Can I add this as a new action item for you, Sean yep.

A

Perfect, so next on, our list is to review pull requests and issues and by the way, if anyone has other items, they want to talk about other agenda points. Please do let me know I'm happy to be flexible.

A

We have seven issues and zero pull requests. We have the metric and a time to close, which is is should this be done since it's now on the website.

A

Kevin I'm, looking at you.

D

Oh sorry, I lost my lost. My controls, I had.

A

That happens yes,.

D

That should that should be done if it's on the website. Okay,.

A

I like this, because now we only have six common metrics release, notes, that is, for the upcoming release. So as we make changes, we can add in here time to release.

A

A

A

To beat them just so Kevin because you said you wanted this in here we have repository authors, equals commit, asked authors as a question.

A

And Shawn, this is a question from you.

A

Can you help us understand what does this, and but this is still an issue.

B

Yeah, that's still an issue. I, don't know that's what it means.

A

Can you help us understand so.

B

The constructive or what posit Ori author exists inside of the common working group, but it's not. It wasn't clear to the evolution working group a few months ago and still I think isn't if that is the same thing as commit authors or if repository authors expand to include things like issues, comments and other phenomena. So we're just looking for a clarification of what a repository author is or a definition, because it's a construct that this working group created but which doesn't exist anywhere else on the chaos project.

A

So I'll just say it's the same thing as a committer, because.

B

A

Don't think they have discussed anything else, okay and I. Guess.

B

I would ask that it changed to repository commit author instead of his deposit or author, because I think the term repository author is misleading and confusing.

A

That's a really good point, so repository commit author boom.

B

A

B

A

D

So so it this is just four commits. This is not for issue authors. This is not for any a lot.

B

D

Authors, nothing like it's burning.

B

Men I look if I look at it in the context of the other things that are listed there. I think that is the intent, but I didn't create it. So that's why the question got asked so.

D

From from reading it, it kind of looks like this is a high-level any author in a repository, so it could be a commitment or it could be someone who authored an issue.

D

It could be someone who authored something on a mailing list, apparently because that is a that is a filter down there. However,.

B

I, don't see commits anywhere else in there I.

B

Also, don't see.

A

Pull requests quote review is our quote: work alright, pull request.

B

Yeah, that's so confusing, but yes you're right. That is, that is I thought it was just I. Think it's just review. Participants I, don't think it's code, review participants, I, think I! Think we call reviews pull requests, we don't call code reviews, pull requests because reviews and and code reviews are actually a different thing.

B

So and you can get from github I know I'm, not sure if gitlab gives you the data for code reviews, but you can get code review data from github. So when someone does a pull request, it can, for example, require reviews by the either the code owner the person originally committed it or a collection of people who have connect. Prep, writes on the repo I. Imagine get lab, has something similar, so code reviews are are different. Things.

A

So how about we say: review participants for example, and requests yep, that's more clear and yet they're.

B

Kevin's question was: is there an umbrella term for any kind of these contributions.

D

: right yeah, we can't call it repository, commit authors because we're not just talking about commits this. Is it's an umbrella, all type of authors, so perhaps repository authors does make more sense. Although I'm not a big fan of that name either is there? Is there a different kind of high-level name that we could use for all contribution? Authors well and.

B

If you don't use, you have to have commits in here somewhere too, if you're doing contributors then, which is why I I guess you do you remember the repository authors being an umbrella term, Kevin.

D

Where was the, where was deep, it was well that wasn't the issue. Wasn't it yeah.

B

If a repository author is the same as a commit author, that was my issue because we were trying in the evolution working group so.

D

The answer is, is maybe okay, so sometimes sometimes repository. Authors are the same thing as can get out there or sometimes authors are.

E

Closeted, we all turn on that.

B

Yes, yeah cuz it. It depends in the case of both interpretations of the term repository author. Yes, if it's a repository author covering an umbrella of all the things you can do, then obviously a lot of people can do open issues and do things like that. So if it's just repository author and that repository commit author, then then yes and then, if it's repository, commit author anytime, my pull request is accepted. Then I'd be also become a commit author inside that repository as well. Even if I'm, not a committer.

A

So my just I understand that there are many different authors, and so this metric is about who is contributing to our project, and these filters are just examples, and so, of course, we know that commits is not everything. Issues is not everything. Reviews is not everything and so I propose to just use commit authors. If that is in line with what evolution has been using as one example for contributor, you.

F

A

There are other other contributors across our project across our repositories, so.

B

I think we're actually using code contributions or something like.

D

That a check so I've always I've, always taken author to repository out here. The person who initiated the issue or the pull request right so notify everyone. That makes a comment or not everyone that like, if someone provides a review in in a pull request that wouldn't be an author, they're, just they're commenting or reviewing someone else's work, the the author would be the the person who initiated the pull request or who initiated the issue. So if you you.

B

Could you can get the code review data out of github, so code reviews like authoring a code review can be a contribution and it's actually github calls code, reviews, code, reviews and reviews, pull requests mmm so large. They are two different things and I. Think your initial point Kevin was that repository authors might be the umbrella for no filter right, like I. Think what I hear yes and I think I think the question is repository.

B

Author would exclude IRC, blog and mailing list, because those are not repository, grounded phenomena like there's no platform that ties those things to a repository writings I exist outside of a repository. Yes,.

D

I I, don't I, don't like repository in there.

B

D

But in the in the title, but I don't think we should just focus on commit either is what I'm saying I think it needs to be a higher level of it needs to be a higher level than just commits.

B

Tory as well so I think I think, like all contributors would be basically the application of no filters and the specification of what data is included so, but you can run with augur everything on this list except I, assume event. Participants is actually probably external events, so then participants, mailing lists, authors, IRC, authors and blog authors. You can't I, don't think with any tool.

B

You can make a claim that you've gathered all of that, because most repositories don't have a canonical list of all of that, especially when it comes to events and IRC kinds of work and whoa.

A

Kevin I hear your point that there may be a collection work here. Do you think we need to capture this in this list of example, filters?

A

Well, it's true. Let this be even something that someone who is interested in seeing commits issues and code reviews. They just apply that filter, because it's logic and I'm trying I'm trying to strike a balance between the clarity of the metric and the usefulness and using it.

D

Yeah I think that's fair.

A

And I will create a pull request and we can talk about whether.

A

This is a great pull request. We can move on so.

A

This is the change as its proposed right.

E

A

Story booth commit and then clarified the code review to be a review participant and as a reminder, these are pull requests yeah. If someone else is on the call who has commit rights, please feel free to merge. This PR I think.

B

I can do that.

A

B

I'll keep track.

B

Request doctor, you can closed.

A

And this results 57.

A

Excellent, then, that should go babe.

A

In a moment, and let me have Numa.

D

A

Awesome. Thank you. We are down to five the other metric idea. They have another metric idea, discussion of responsiveness, metrics and management of organization, filiation email. Those are all ongoing conversations that I don't recommend we get into today any opposing views. Any other suggestions. I have.

B

A question about the release metric issue: actually Kevin are we are we? Are we having one issue for all of the metrics released.

D

So so, each metric has its own issue where people can make comments. Okay, and then we have one issue that tracks the release: notes for all of the metrics that you release. Okay, so the that one that's their 60 is basically it's a coordinating issue, and if you, if you open it up, you'll see that we reference all of the issues yeah. Yes, so the new method, types of contributions, if we scroll down a little bit you'll, see that there there's another new metric time to close.

D

So this is this issue: should capture kind of the entire discussion of the release or or any edits that you may make to an existing metric that has been published, mm-hmm.

E

D

Each each metric that is released needs to have its own comment issue as well. You'll notice that in there I linked out to what that issue was okay,.

B

Okay, but I don't see any of the common issues tagged as metric release. Candidates like I, have in the past I guess that's the origin of my question. The.

D

Little you mean in the labels yeah.

A

That is because the the last one we did went through the continuous metric release so.

B

A

Already closed the comment period on that I thought yeah do we have.

D

No comment or disclose.

B

Those are so there are no new metrics that have not already been through the common period for the comment group right now but like when I finish, the draft of the metric for code distribution, then that would get I'd create an issue for that. Following the standard language, correct.

D

And take the tag it with the metrics candidate release tags got it okay. Now.

B

I'm following we don't have anything that's in that state right now, but we will in 13 days and this release notes. One is gonna be in every repo yeah.

D

Every every working group should have one release, notes, issue mm-hmm.

C

D

And I will I'll close that when we do our regular release and I'll open a new one, no.

B

I'm full now I'm following them and remembering the full process, or it take up your time now, I get it.

A

Okay, the next item was progress on continuous release for time to close metric metric. Was it Chu website no metrics currently in the release cycle, and we have a conversation about cross-referencing metrics and we have I I. Don't want to do this right now. I want to focus on the metric release actually totally.

B

Cool I think I do think at some point. As I span a number of working groups. There are, for example, three working groups right now, looking at the Forks metric, so three working groups risk evolution and on another one value, maybe all consider Forks to be a measure that they are interested in for the purposes of those three working groups and obviously the metric will be created in one working group. But it seems that that metric will be shared by three working groups.

B

So one could argue that I should bring and metric from those three groups that I participate in over into common, because it is of interest to several working groups and I. Guess I would ask the opinions of the folks on the call, at least if that makes sense or not.

D

Are they all looking at forking in the same way, I think.

B

So is it I think the four keys of working, of course, of course,.

D

Okay, well I mean and wouldn't when some people talk about Forks, they talk about Forks as creating a fork to.

D

Work on the software separately or to create a new piece of software, whereas in our you know in the normal development process, and you get up, for example, you you fork something with the idea of merging it back right and I. Think.

B

It has I think what we can tell from any platform is how many times a repository has been forked, but the repository actually loses track of that fork.

B

If you disconnect your fork from the home repository and just start off on your own, like what would what's commonly or sometimes called a hostile fork where yeah you create a fork and then you detach it from the original repo and create your own, it's it's up to I, don't know, I, don't know that we have a way that we can easily track those kinds of things and I, don't know if they're super common, so my interpretation of all the working groups is they're really just looking at the fork count, and maybe some details around how many of those Forks maybe have done, pull requests, or maybe something like that which can be derived from a platform but I, don't think they mean hostile Forks, which is the only other type.

D

Of network not really the so not the purpose of the forks or it's just it's literally just a count of working within a repository yep.

B

Okay, I think I think that's that's. My interpretation of those three working groups cause I interpreted the scheme because I and all these three working groups yeah not, is also a in the same we're here. Yeah you have it assigned in one of them. I can ask yes but I'm wondering if it should be brought over to come and given the shared interest. So, instead of doing the mapping exercise, that's on the agenda. Keurig I'm wondering if the metric with so much shared interest should just be. We should just put that in common too.

B

A

I make a suggestion. Yeah, as any of these working groups already started, work on describing the metric.

E

To do this upcoming.

B

Risk meeting so like just like code distribution, it's a very it's a metric that kind of writes itself. It's not super complicated and we all know what it is. So you know it's been assigned. It I think in two of those three working groups, and- and so so the question really is like. Can we just I?

B

Think I would just like to ask permission to have Vinod do that metric in common and then the other working groups can point to it as a because I think that seems like one of the use cases that common was created to address I agree.

A

So if you look at our our table, then where would be at forks? Is this a people contribution of time or space, its contribution.

B

A

Think it's a contribution.

B

Because the fork is a signal of at least the intent to contribute it mean. I could make an argument that it's about people but I. Think it's more about the code and so I put contributions under the code. And you could put that as in progress and maybe make a note that there are three working groups with interest in this metric, which is why it's being moving over the common risk, evolution and value.

A

Okay, yeah, we know, do you already have a document that we can link to here? Yes,.

B

Sir I think there is one in risk. It may be maybe a template, but I think one exists for in risk or evolution. Maybe both have created documents.

E

Let me look at it I'll post the link to this thank.

A

E

Thanks for that yeah, so.

A

Time to close, that is, it is released right, think so. Yeah.

A

I, don't I, don't know if Matt marks them as released yep, they just.

B

A

B

A

B

It's I forget Kevin. Is we mark them as release it on our spreadsheet and then make sure that they're noted in the in the review in the issue yeah yeah and the issue for release notes, but.

D

Generally, when it goes to the website, I make the note of it in the original. So we should we leave it as under review so that you know, or no in the in the feel free to call it released in our spreadsheet yep. Okay,.

A

Okay- and this was the when metric time to first response, I just want update the link, let us in the spreadsheet here, because this is Google.

B

Doc excellent, exactly yeah there.

A

Now we have the actual markdown and then I.

B

Keep trying to click on shared spreadsheets and zoom I'd yeah.

A

I think if you have time to close, that's already done if you get Forks and language distribution that that will be great and when we have one more metric that we've been working on, that is the contributor location metric and so for the last 19 minutes that we have I, invite you all to review this metric, and maybe we can push it over the finish line and actually add it to the next release as well.

A

What do you think.

B

I'm sure the Lincoln- let's see, how are you yeah.

A

I'll stop sharing my screen.

A

So there's some cleanup that needs to happen. That's easy enough to do it's. The old template.

A

So in that goal we say to determine the primary location of the contributor. Are we looking at the primary location? Is that too specific? Should we be asking about where they're contributing from or where they're so leave it open? Since this is a common measure.

D

A

It to just location and then have different filters as in location of residency, relocation of contributing from time zone.

B

Yes, I mean most of the work that I've seen around identifying.

B

This metric is that they, like is done notes in her comment. Time zones are as a proxy for location.

B

And I guess you could yeah primary doesn't need to be in there.

G

A nice good, quick question mm-hmm as far as getting someone's IP address are there any privacy concerns or anything like that that we should worry about weak.

B

I mean I, don't think the platform's give you that.

E

Well, get them, you have the IP and you can access it and locate it. But there are people and get up. You have an option to not to disclose. Then you will get 0/0 and latitude and longitude, and you just point out to the center.

A

So the ethical concerns and legal concerns around any of these metrics and data is something that you've you're discussing time and time again, and we still have to document our position on this and the metrics themselves are.

A

There are many metrics that touch on personal identifiable information that is protected, and so far we have taken the position that we will not include disclaimers inside the metrics. This is something that we should do at a global level, because it applies to almost every metric.

B

So the when you say global level, Georg I think I, don't understand exactly what you mean, but like I do understand, you mean that as a Kaos project or a community, the question of resolving privacy concerns is it's something that will be handled at the community level and it doesn't really apply in the creation of metrics. It's really something that the software that's pulling. Data has to attend to right.

B

E

How about we air that disclaimer to this like, if the data is publicly available, you can use this something.

B

Along that line, I don't think we want to add disclaimers at the metric level, because then, then, you've got a lot more pain when it changes right, I think I think it's a global issue, and but I guess. My point is that, for my question is I think it's not something that in the creation of a metric, you might have a lot of detail about the people, for example, if you're in a tech company and they work for you and that might that might be okay, if you're not making that data publicly available.

B

There are redistributing it.

E

We existing github data, most of the cases and open-source communities like and then on. Github you have this publicly openly available. People have given the concentration of their IDs yeah, but.

B

I think I think you have to I, don't think it becomes an issue until it's implemented is my point: yeah I think David. So, in the definition of the metric we can't make ethical calls I think as a community. We probably need to make some kind of statement about privacy and personal data and our respect for it, which I think is Elizabeth's main point right and so.

F

B

Probably something to make sure is on the agenda at the next cast meeting, because I think at the next community call the first of July first Tuesday in July, because it's something we've kind of punted kind of kicked that can down the road a few times and it's probably time to have. Can land somewhere and I. Think we all kind of are an agreement about it being a kind of a choice of the person implementing it. And if we're not making the data public again, then you know I, think we're.

B

Okay, I, don't think we can get in the business of other enterprises that have people working for them.

C

B

C

Oh god, no you go ahead. I was.

G

Just gonna say it just: it would be a shame if, if a potential contributor decided not to contribute to a project because of the way that someone was implementing our metrics and like we didn't give any way to let people opt out or any guidance around ethical usage. Or you know what I mean like I'm I wouldn't feel really bad. If, if we're, if a project is losing contributors because of the way that data is handled or something that you know we touched on, but.

D

I, don't really understand.

G

That it's really not our job to to monitor how people are implementing, but I think that yeah, maybe some kind of, but it is.

B

It would be more than a shame, though, if using metrics the caused people do not contribute right. There.

G

A

B

Of our intent right, yes,.

A

That would be so I think so for June 16 in the chaos weekly meetings. I put it on the agenda to talk about chaos, statement about privacy and ethics around metrics and analytics. So we can talk about on Tuesday yeah.

E

A

That's a good idea, yeah and then today we can talk about the contributor location metric and see if we can push it over the finish line in the next ten minutes.

A

So aggregators is something be had and the old template we don't have it in the new template. I can show my screen. So you all know what I'm talking about.

A

Does anyone remember how we are dealing with this aggregators, how we converted those I think.

E

They were, they were in the fridge.

A

I think this is from a different different metric total number of contributions. Doing a given time period sounds like it's from a different metric somewhere.

A

Parameters, do we still use parameters anywhere? What do we just get rid of this.

A

I think this Rosen here.

A

Yeah I think that's filter.

E

I think this fits into the type of contributor the repository owner issue, authors.

A

Like that, I just had the same idea.

E

Yes, I'm sorry I'm Finn. We have covered timeframe of activity in the period of time.

A

A

Have a question for y'all: we have in implementation information that we could use information from profile that we can use IP address or timestamp, and then we have a section down below about data collection strategies.

A

Should we merge those two together I.

A

Feel like moving this part, that is under implementation down into the data collection strategies.

A

D

Just do this and.

A

If anyone disagrees that we know.

A

So what do you all think I after looking through this metric together, I.

A

Propose that we create a pull request and added to the upcoming metric and have a open discussion on it with everyone sounds good.

G

Do we need to mention anything about the fact that people can set their own profile locations like you, can literally put anything you want so yeah.

B

G

B

Think that's a good point like mine.

G

Where that belongs, but maybe just to acknowledge the fact that you know there could be like random.

A

Here, how about this a little bit note that custom profile information may contain false or nonsensical information yeah.

G

I think just to acknowledge the fact that you know okay.

A

Okay, so note that contributors may enter in their profile information, false or nonsensical location information, for example, earth right.

B

Like yeah well, I think a lot of people actually use the earth or.

A

Internet yeah I've seen that too another classic yeah.

A

Okay, that was a good addition. Thank you, then I will take this as a action item to create a pull request.

A

Any that that completes our agenda for today.

A

Yay with two minutes to spare wow we're killing it.

B

Excellent well done here on top of things, mm-hmm I wanted to put you in charge of the meeting. Key organ and release pushes her it on through.

A

Oh, it only works because you're all such good collaborators, there we go, we go. Is there any.

F

A

Anyone's mind for the last minute, nope.

A

Okay, before we open another can of firms of talking about with the and thinking of metrics, or anything like that I'm going to close the meeting. Thank you.