CHAOSS Common Metrics Working Group, 31 Oct 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CHAOSS.Common.October.31.2019

Description

CHAOSS.Common.October.31.2019

A

Alright, so we're this is the chaos common working group meeting for October 31st, as I said I would take minutes. We did right, ok, just making sure the Merrill got it so there's the document.

A

Ok, so the first I thought I'd pull up. I have the notes here. So the first was the couple action items from last week.

A

So Shawn you were gonna, provide write the API, endpoints and I think the like, not just for common, but for all of them I'm trying to really get people. When we talk about tools that are used tools, providing the metric mm-hmm to not just say, hey, here's a link to like were more lab or here's a link to our Gurr, but being a little bit more specific in terms of yep.

A

Everything like where it's actually provided.

B

So so I'll give you a repo link. We every one of our end phones has both their people and retail group summary level. So you can just assume that's implicit.

A

So that's code changes. You.

B

Know- and that includes the time stamps with time zone for each commit okay, which is what was displayed in the discussion last time. Okay,.

B

So the way the way yeah we then I look at what was displayed. There's that's just what we saw was just decoration on coach. You just yeah.

A

B

A

Is time as related to code changes but obviously could be related to a variety of different things: Yeah right, yeah, anytime.

B

Time is the thing yeah.

B

A

So yeah, so these would be easy to each other, so this is. This is an example. Yeah.

B

A

So these are the kind I think is you're like on some of the metrics that we're gonna release like for version 2, the ones that were released the first time, yeah I I, might be pinging. You want a few of those that would be providing kind of exactly this link that you just provided me yeah and we're going back through all.

B

Of our work, I mean we're basically at the point where we're generating new endpoints right now so yeah.

C

B

Another influence over the next six weeks leading up to the release, so yeah I think you guys know that's gonna co-evolved with the metrics that we define from what we have so yeah.

A

If there's a, if there's a concrete example from augur, that's what I want to put in as a post, just like sure you could do it in augur, but we don't have done. Oh yeah, okay, that's good thanks!

A

Okay, so I had I was supposed to put together a whole request. Your you're all you're, both gonna yell at me, but I, mean I put together a pull request towards the German.

B

A

So well, I did it I mean I took like combined it with my restructuring of the so there's one giant pull request. Well, that's just good leverage of your time. Well, it's it's actually not a terribly complex pull request. It took a while, but when you look at it, it's not crazy.

A

Weird! So we'll talk about that here in a little bit and I've been.

B

A

Longer pull requests like last month, so don't merge this one quite yet, because there might be a request to make it into one. Is the repo redesigned and then a second pull request? Is the metric itself.

A

So we'll talk about that in a second John is not here so that back so grimore lab so Shawn. There was an action item for you that you were gonna, create an issue and a document with respect to the grimore lab elk.

A

Get metrics, so the idea was here's the link, then I went through and pulled out some that I thought were applicable. Okay, but from this link are there things that were more lab has kind of identified in a schema that would be candidates for common metrics.

B

A

For example, in this schema list, they they list, you know date and time and we've already identified those common metrics. Yes,.

C

A

Wholesome this list and once did I had identified. So this is just basically the JSON dump file from the commit.

B

That good more lab generates right here. Am I right about that?

B

Sorry? Can you ask that again is this? This looks like the dump of a gift from that log. That's probably Jason the schema yeah.

A

There's a few things I think the.

C

Scheme of CSP is manually maintained because we have right.

B

So the scheme is no metal. Oh no I'm, just thinking I went to concrete data so in terms of the source of the concrete data behind this I think it's the commit log.

B

Forget yes, I mean for this I guess this schema. Yes right! Yeah you get that CSV yep! So.

A

What is the question map, so the question is on this list of 52 rows. They have here. Are there when you read through this list? Are there things that kind of stand out as potential candidates for common metrics, and so, for example, date and time obviously shows up a lot in this? This list.

B

And so pretty commonly we do the same by breaking out author and.

A

Computer days yep so date and time, that's something that we've taken care of in common rain, yeah. Well,.

B

I mean so so common comments, interest in date and geared you guys, correct me if I'm wrong, the Commons interest in date is using it to understand when people are committing and then what time zone they are so date is late, is sort of a filter. That's spread across almost every metric. That's why it's common! Yes, sorry, okay, so put spicy music, but it's it! It's that I guess. My question is and I'm want to call it.

B

Fun is about missing the previous conversation somewhere, but date is a component inside of basically every metric, but a date itself is not a measure. It's a it's! A field inside of the metric. Well,.

A

Just we have a date/time metric, just it's a common, it's a it be a common filter that you would apply. I mean it's something that you have to pull kind of like preamble, but.

B

I mean so, if you say, there's a date time metric. What is the metric well.

C

One of the reasons we start at the common metrics working group is because we had metrics that didn't really quite fit all the other working groups agreed. We have some date related metrics like we're, not people contributing that is not clearly evolution, not clearly any other metrics or another working group. So that's five in common.

B

So I agree, but what is the I guess? Maybe I'm missing something? Is there a metric so I'm, distinguishing between a metric and healed? Okay.

A

So what we're going to find the chaos? Maybe here's the here's, the document that is the date time.

B

Yeah, okay, I guess I didn't so. When we talked about this last week, I didn't realize it was called date time metric, but we've been talking about it for yeah, so I. So, what's always resonated with me about this. Is that it's the time zone? That's really important that that it's not actually a date time metric. It's a it's a time zone, identification metric like activity, dates and times, I guess.

B

But I think that I guess I would say, like we have aggregations listed by UTC time and by local time, which is true. It's.

A

Just different ways that you didn't think about tomba yeah.

B

This is a really sophisticated.

B

D

B

Trying to be argumentative yeah.

A

I mean, if you want to get rid of this metric, we can I, don't know, I, think. If the community agrees it's a metric, we can make it a metric. I think you don't seem to agree with this, so I mean.

B

And I'm not disagreeing with it, because I have any particular problem with it. I'm just reflecting on the conversations that- and this is maybe, where being being involved in discussing things of evolution with faces for so long I may be into color like prior thinking about us, but we distinguished- and maybe here you can speak though acacias maybe might be here, but we distinguished between filters and metrics and time was a filter.

B

So what's interesting here, though, is that this is an important metric when you talk about the difference between UTC time and local time, so we're saying that the UTC time aggregations and the local time aggregations are filters in this case, but I actually think that those are essential data components and the metric itself. In other words, the they're not really filters here. They're they're, essential pieces of the metric, because.

A

Well, could I ask you to modify this metric yeah accordingly, I mean I, we're.

B

Stepping backwards I'm, sorry and that's, but for not really rocking what this was called.

A

Because we've talked about this for a while yeah, yeah and I think and I guess the description is accurate, but the naming are you proposing. I also hear you're proposing that common should be called like common filters, no common metrics, no, no.

B

I'm sorry, I I think when I look at what this measure is I think.

B

The name is probably okay, I'm processing, this little bit more than me, I think just hearing that name activity date, sometimes through a little bit but reading it again. This is this is about localization the principle questions that this asks. The goal question metric framework, the question that this answers is what time, what is the local time when contributor is doing their work?

B

That's that's the common metrics groups, interest in this. So that's that's the question and I.

A

Don't know we do you want to change the question.

B

Yeah because I think it's more about what are the what what our local dates times danceable contribute activities occur. Can there be more than one question? No.

A

B

So implicit in that question, though, is it's being used as I understand through the previous discussions to estimate or guess at two to four people to make guesses about where on the planet someone's doing their work, like that? That is the underlying reason that this metric came about in the common group, and here you can color that in if you want, but for the discussions I've been in with common. That is the thing that people are trying to understand through this I.

C

Have nothing to out there.

B

You mean: is that consistent with other conversations, you have her, or is this not something you've heard before it not.

C

Matches: okay, what I'm, what I'm doing right now is the I'm reviewing Matt's request and I'm fixing a few things, but then I think they're ready to merchants.

C

So I would not open this metric up for discussion right now. That's my opinion. Okay,.

B

So the metrics implemented this way, I think that's I'm. Sorry, no.

A

I'm just trying to get up what you're trying I mean it sounds like you're asking for some fundamental changes and I. Don't think comment, functions, I, don't think so. I think there.

B

Is an assumption, so the origins of why this metric is this? Is it is fundamentally trying to understand you where on earth people are making their contributions, there's an understanding that people are funny hours and their local times their local times may not align.

A

Clearly also meant to understand, I think there's also dates in here yeah, so to see. If there's it's not just time zones so you're right, but so that they distinguish.

B

The presence of time zone data is I think regarded is a more you know for people who are doing things on UTC on their servers, which I do sometimes, but for people who are actually making local commits in the time zone that they live in, because computers tend to sync that automatically it's regarded I think, but, as group is a more robust indicator of where on the planet, people are making their contributions from, and that's that's the underlying reason that this metric exist and I think there's been a nearly every discussion about it and I.

B

Don't think it has to be I just don't want to lose that. But I, don't think. Ask me right now to hear its point. We can merge this so.

A

This metric actually came from where my employees right exactly I, mean that was it, that's it exactly and Shawn was right. It's not in the metric.

C

Right now, but.

A

That was, we decided. So as part of this conversation, we decided that from an organizational perspective that was too hard to do that it was just. There were just limitations on trying to figure out where employees are when my employees are contributing, just because of weird servers and kind of how things function on the internet.

B

So this is this: is this data will be used to probabilistically estimate where people are and there's going to be a recognition that it's not perfect, but that's it. You know that people want this yeah and that's just not reflected, and that's probably fine, because all the people who are going to lead the charge building it understand that. However, it might be helpful for consumers of cast metrics to know that that context exists, and maybe maybe that's something- the Edit to the description after this post is merged. Ok,.

A

So can I give you that action item? Will you do that are y'all okay,.

A

Well, the pearly crest also claims. The pull request also contains a repo update right, yep.

C

I read all that: ok.

A

B

You think, what's the reason: ok.

A

Yeah, if there's and I think if you, if you do that, merge if there's any little details that you can my.

B

Question was give I'm just curious. Why we're squashing the commit.

C

Because there are so many small tiny commits in this one, ok, and if you squash it, then you get one clean, commit with everything. It looks like Matt used the github interface to create I, see, okay, those also other granular locally. There's no way to.

D

Combine it and get the get up in the face, because that's washing it I'm a I'm, a granular person, I, never squashed, but I get it. That's cool I just wanted to understand thanks yeah.

C

But update readme update, readme update readme update readme update, read yeah, you know, that's.

B

Kind of it's an annoying series of of commits and that does it yeah. Okay, that's I, get it it's cool I, just I didn't I have like a philosophy of not squashing, but I get why you would squash this so go for it.

A

Ok, so Sean now that that metric is in there if you want to propose changes, let's just do it.

A

B

Sorry, I'm really sorry about that diversion I I think it ends value ultimately, but it was a long conversation that didn't need to happen so yeah.

A

And so what the the that took us away from kind of the original question. Sorry on that list was from grimore lab that list of 52 roads yeah. So these are the.

B

Get timestamps, that's I mean that's. What's in it get log like it's almost defined by get yep.

A

So in there there are things like date/time, which we just merged. One of the. So are there other things in this list that would be considered, perhaps common filters or common metrics that could be applied with pull requests, issues whatever it might be, and the clearly committer name yep. So, author, author I, think there's.

B

One author name, you know I, think I, guess you're storing author email as a privacy practice gearing author you're starting off the domain instead of email as a to address privacy considerations.

C

What I think is just the ending on the email right.

B

But there's not a full, so we use the full email to resolve identities across github, get lab a bit bucket and the git repositories. So the email is kind of a key that we use to resolve common human beings across different platforms. Yeah.

C

B

Can certainly understand why one would not store that, given that bit, turkey is headquartered in Europe. However I you know yet, yet, as a general rule does have a written documented exception to the GPR, so I'm late.

C

I think I think it's just a matter that the identity itself, this managed in sorting hat. So that's.

A

Okay, so yeah I can I ring this in just a little bit yeah, so just I mean like message on you can't come to these calls anymore. Just what I'm trying to do is just like the havoc level. Things in this list and I will give you my exam I'll, give you what I wrote down. So author is one of them author. Yes, you might that's a author.

C

A

Author or whatever, for the time being well authoring, 200 are two different people: okay, yes, okay, so the other. The other thing that I had the other things that I had on here was kind of the construction of the project, and so one of the things that is in these rows is lines files, repository branch and project, which I think is organization in here, so because there could be a pull request.

A

That looks you know that could be filtered by files that could be filtered by lines. There could be an issue that is particular to a branch, I think.

B

The commit hash is another one, I agree with all those I would add or hash as well, because that's used to identify a specific commit, which is very useful for a lot of things. So.

A

Gay, or did you see any others on there, so for your thoughts on what I had said and what Shawn mentioned some.

C

Of those do you said our structural metrics, structurally or because of a sensitive.

B

C

To me, for example, right or lines added that's nice to have metric, but how does that reveal crashing.

A

Yep, so I was just thinking kind of building up to the growth of the project, so that lines can lines are the lowest most granular level thing that can be modified, I think.

B

The granularity on this is files, though well, that's a number. It's not the actual file right.

A

It's a number of files, yeah.

C

A

Then there are files there, files within the repository. The repository has branches that I'm just I mean. Maybe they don't work, but when I look at what we had talked about last, we had this call about the focus areas mm-hmm.

A

They were who so the first, the new focus area is who, which is people and organizations, typically as I, think the way we were talking about him? What so? What the contribution is.

B

That was what what was so. What is what so this doesn't include the what it includes the number of files, but not the files, no.

A

It doesn't think this is not about what yet when which is the square meter, what we had talked about who? What when and then where, so, where the I think it's where the activity is occurring. So, if I look, if you scroll down when I guess, where is a little bit different, so it'd be this would be a pull this up put it, you see the bottom of the minutes. There.

A

See I did the who? What, when, where do.

C

So this is what came up last week, so maybe it is what metric, what are they doing? Changing this number of lights? Changing this number of lines.

A

So maybe files or repo lines- maybe these don't fit as common the things at all so I.

B

Think I think counting well I think they do I. Think counting the files is a meta statistic that gives you an idea of the scope, without bothering you, with an enumeration of every file in a commit I. Think there are I guess where would the? What of what are the files in the commit exist?.

A

So my thought was: let's say you issue a pull request right and we're we're looking at say time to merge on a pull request right.

A

Would it be also useful to know so time is obviously a filter they're. Also the number of lines that are being changed.

B

Right request does is that in the CSV they missed that no.

A

No again I'm just talking about these common metrics so would see, would lines be an issue would an issue? Would it be interesting to know that an issue is pointed at a particular branch.

B

Issues would generally not be pointed at a branch that.

A

I have an issue with what's going on in the dev branch, oh I'm,.

B

Gonna, let her open an issue like that, but I suppose you could.

C

It's possible but I, don't think it's a common enough use case. No.

B

I think it's it's! So when it comes to the question of lines, I think I think just having lines in a commit is hard to understand. Unless you know like knowing the number of files, if you're just listing the number of files, then that is I think as far as you can go for usefully describing the scope of a commit I.

B

Think if you go to lines, you really need to understand which pony lines are changed in which files like giving squashing the there now to two layers of abstraction away from the scope of a commit. So, for example, the number of lines in a javascript file is going to be it's going to be larger in general than the number lines in a python file or c file for a similar scope of change.

B

And if you abstract away the number of lines from the specific file, I think you're taking away information, that's useful that renders that line count not valuable.

C

And so grimore lab has a different index, the areas of code CSP and the schema document where it does store specific file, name file path and what changes have been done to those files. So it's just in a different index because it separates out from the commit and then it creates for each commit one entry, I think in the areas of code index.

C

So I just want to say maybe to your point: Shawn, maybe shouldn't be limited for looking at the commit level the get sees me, but also all the information we pull. In addition to that.

B

Well, get the get, log does include the files changed and you can derive than alright and you can count the number of lines in each file from the git log. So like the get definition that you have there for elk is squashing information. That's in the get log out of the get lon and it sounds like you've just addressed that in a separate file which is called what again the areas.

C

Of code I put in the okay.

B

So I mean this is program or lab designs. I mean y'all can do whatever you.

D

Think is best I. Think.

C

Point is simply to say that information, since we are looking at the grow-op schema this. The information is there just in a different file. Yeah.

B

So it's gonna filter people consuming grimmer, labs and know the areas of code is derived from the git log.

B

A

Coming back to common metrics, so, yes, we think about if we think about date and time, yeah eight in time is something that is applicable across a number of metrics in a variety of different working groups. It's important for risk. It's important for evolution, it's important for value.

B

A

Story: I agree. Okay, so are there any? Are there any of these metrics? When you look at this list, are there any? Is there anything else that, like understanding lines or repository or organization, are there any of those that could be useful, supplement or supplementary metrics or filters for evolution and for risk? And for that's what Commons trying to do? Yeah I mean I, think so, like the CII badge in risk? That's not really a common metric because it's highly localized to risk and there could have some play in in other areas.

A

If you have subtract it out, but are there any other metrics in this list? Like date time I.

B

Mean I think files and lines of code are doing they're, not looking they're, not they're, not in this list, but they're useful to all the working version. So I don't know that we have to drive the discussion from D CSV files, but I think they're useful for identifying.

B

The points that could be common so I would say the file a listed files, not just the counted than the newer files and then the lines of code in a file.

B

A

And those have a.

B

Hierarchical relationship so I think it's helpful to represent the hierarchy which we do in the augur schema.

C

So the way that I see this I put it down below, as we have lines, add a change removed and files edit it as metrics as numbers, and then we have names of changed files, directory of changed files and file, type change and people name, and those are all filters that can help us to.

C

Segment, the data.

A

So would these bike there's in the common on these.

B

Common metrics, so I mean I, think like files changed and lines of code change are very similar. I think Matt's point there there conceptually identical to the timezone there the activity date time question things would be in the act. These are specifics of the. What the people I think will be interested in.

A

So here did you hear that question so.

C

What what I am hearing Shawn says? If we have the names of all the tiles change, then we can calculate the number of files well,.

B

And you can enumerate the number of files and then you can show the lines of code changed. Then the lines added and removed on each file, which I think is really I- mean people kind of expect that from a get long analysis, so.

C

But the name of the file is a filter that you can apply to look at that level or you can omit the filter to get you aggregate sure, and the edited is the actual.

B

Right so just say that just lines of code is meaningless unless you know this file that the lines of code comes from so lines of code or the commits level is not particularly helpful in most, unless there's only one file being edited so.

C

So we agreed that number of lines changed at it is metric, but we need the filter by filename for it to be meaningful.

B

Names of changed files, yeah and and I I think it's yes, yes, but the file name includes the path as well, which others, like you separated well yeah meet me Ken, because in many cases there's the thing filing multiple directories in a project like readme is a good example. Yeah.

C

But sometimes you can also just look at the file path as a way to look at modules within or different areas within the projects.

B

Yes, yes, you can look at bike paths, sub directories and filter by that as well. Exactly.

A

Okay, so what I'm hearing with respect to the? What focus area from the two of you is that those two number of lines a number of files, basically how or they're titled- would be the two metrics.

B

Yeah they're two metrics I think we named a few others as well previously, but I think you captured those would.

A

It be useful to just combine them the way that date/time was combined.

A

You know that you would just have all metric, that's called files and or lines and files files and lines I think so. Yeah.

B

Yeah, that sounds like a really useful working like the activity dating time activity, files and lines of code.

A

Yeah: okay, okay,.

A

Why don't you I'm gonna.

A

Give myself this.

A

Let's start this: okay.

A

Based on what Georg is what we've talked about and what garriga's chatted down mm-hm and then I'll bring that forward in two weeks, yeah.

D

A

C

And then we have the organizational metric for the who that's right, correct.

A

Yeah, actually, if you look at the repo now it is, the organizational diversity is what's currently in there, yeah and so I mean the other. The other kind of seems like obvious metric to me, would be author and make committer.

B

Author and pewter I think are important. Yes, those are the two who's.

A

Yeah but maybe we just don't- have specified.

B

C

Yeah I think we stay specify them. I would combine them because that might be generalized across other collaboration backbones as well, and then the contributor can be an author on a commitment.

A

Yeah and that can be taken care of in the business. The description yeah.

C

Holster, what teacher I don't know if that's the thing.

A

C

A

C

Someone on Twitter, which means yeah yeah,.

A

It's a tweeter, so does anybody on this call wanna take a shot at just using our new metrics template to describe it to describe the.

B

The aunt committer and yeah author I mean I can do that. We already have that kind of defined. Another metric.

A

B

And this certainly.

A

Doesn't have to be done, it's mostly just like setting it up with the new template and I can take that to do kind of writing a few sentences on the description and the objectives. So we.

B

Wanna make committer and a third part of the same metric I think we do that's.

A

What yeah Gary had suggested, calling it contributor, yeah I, think that's that's what we call it another.

B

Working rich, so that's consistent, so.

A

Here we would have contributor.

C

Okay, then I think as filters. We can employ the demographic information like.

D

C

Where their organization anything.

A

That we can get almost.

A

Okay sounds good, alright, so at this point we are, this is good. I. Think honestly, just moving forward on these two metrics would be a great great thing.

A

So we'd have to moving forward, one is lines and files and then the other is contributor and then put lines and files. That's a what metric.

B

Is that right? What's.

C

B

C

B

A lines and files live files and lines, and they call it and I can take that one too no.

A

I got it: okay,.

B

I'll start that one.

A

Again, I mean so kind of getting these things into a Word document, just starting these things out, it might take like 10 minutes. It's not meant to take a terribly long time, all right anything else.

D

No that's good.

A

I'm at meeting we got some things merged in and yeah we're all common def we're good to go. Okay, we have a couple action. I'm scared, you're, leaving without any action items, is that okay, I'm.

C

A

Imagine you are all right thanks everybody all right, thanks man.