CHAOSS Evolution Working Group, 21 Jul 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CHAOSS Evolution Working Group Meeting 7-21-21

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

That was initiated.

B

I started recording the meeting. This is the yeah, so.

A

And so um being being part of the open source program office at alejandro, um we are interested very much in just well we're at the beginning phase of everything yeah all our projects. I think most of them are monitored through lfx insights, but we're not doing much with it um and so in general, I'd like to get a handle on uh on metrics, and I see chaos as the main source of information about that and actually pushing the boundaries pushing forward great I'd like to join there. Yeah.

B

Great um so the yeah, the evolution working group, as you probably know, since you're here we focus on growth, maturity decline. You know just understanding. I think the what I would call the fundamental metrics that have long been monitored in open source uh yeah, the activity.

C

B

Activity level, kinds of things and engagement and participation and stuff like that and so kind of the way that chaos thinks about metrics is we have these working groups and you were in the meeting yesterday, so you know we use the working groups to define metrics, but often it's some bricolage of metrics from multiple working groups. That are what people use to understand: the health and sustainability of different projects and, and so how people use metrics is.

B

I would maybe the word orthogonal um to how we actually do the work of defining the metrics um in a sense because we use like I said we use them in collections, that cross working group boundaries quite frequently, and- and we have a couple technologies, auger and gremore lab that are part of the project that kind of put these together and make them visible to people.

B

So, um okay, I wanna just ask uh kevin and matthew uh and elizabeth is in terms of uh where we're at and I'll share my screen, because sometimes it's helpful just to have the screen shared, so we're kind of on the same page.

B

uh This contribution attribution is ready for review and we kind of this is sort of taken care of right like uh not sent the email out, and this is all under review. So.

D

I don't know if he sent the email out or not okay, but I'm because it's not live on the website. Yet I'm not terribly concerned about that email. Okay, all right! So I think when, uh when the website is live, I have a to-do list of metrics that I need to pull in yeah and after I do that I will probably reach out to elizabeth and and ask if she can uh kind of do an announcement to let everyone know. They're.

B

There so- and that's actually later today, that we are moving the website so that it's uh on a new hosting provider and the reason so the reason that it's not released is is that.

D

Yeah and technically it is yeah technically it is, it has been released into our continuous contribution cycle. It just doesn't have a presence on the website currently, and the website is that is that main point of uh presentation. However, uh the issue has been created, so we can, we can collect comments on it uh and and the process has begun so. Okay, excellent.

E

Yeah, I don't think that you, I don't think there was an email set out about this specifically.

D

As I kind of hinted to earlier, uh even though we do have a continuous contribution cycle, the uh the active review period, it really usually occurs 30 days before the the official release, so the metrics are. The metrics are open for review at that time, but we uh we do not get very many comments until we get into that 30-day review cycle.

B

Yeah, I think that's that's a fair characterization, of course, um so with with that, uh what we usually do with these meetings, then, is um sort of uh look at the next metrics that are up for release, and it looks like there's two that we're working on right now: reaction, filter pr and it could be a filter on a number of metrics, including burstiness, and these are the correct me if I am misguided here. But I believe these are the things like the emoticons.

B

When people have a reaction to a comment, emotion, you know like a thumbs up or uh 100 or whatever the available things are in in a on a platform, so they're different, of course, on github and kit lab, but they they share common properties of happiness, sadness, surprise, shock, love, those sorts of things.

D

um Yes, yeah, and I we talked about this last week and and we actually, I think- we've talked about it prior to I've kind of been I'm kind of in favor of slowing this one down, okay, uh because I'm not exactly sure uh how helpful it is so because, because because emoticons and the uh these, these reaction emojis have different meanings.

D

Just counting counting the number of emojis that may occur in a comment doesn't tell us anything. So the and the conversation in the conversation that you and I had last week- we were talking about uh uh you know. We know what the thumbs up means. So the number of the number of thumbs up signs counting that may mean something, but counting all of the uh emojis together.

D

uh uh We don't really know what that means, and then, additionally, these emojis have different uh contexts different uh projects, so I would just uh I would I would vote on slowing this one down and giving it more thought.

D

Okay, maybe because it's not a it's not as straightforward as it seemed initially.

B

And and so let's just take a minute to to give it more thought, the the metrics that were modified were change, requests, change, requests, accepted decline, active issues and new issues. Those were the the five metrics to which emoticons were or reactions were added.

D

What oh, this, isn't uh this isn't the metric one? This is the this is an edit to the uh template. I'm sorry, I was confused. We also have a, I believe, there's also discussion of creating a metric to handle. uh Yes,.

F

D

And that that's that's a different issue.

F

That was discussed in the last meeting when we were going through this pr. That emojis needs a separate metric rather than adding as a filter that that was the discussion. I recalled from the last time. Okay, so.

D

Not, rather than not, rather than a filter, I think it could be. It could.

C

D

It could still be a filter, but it needs some sort of definition uh to go along with it.

B

And we are, you see, is: are we suggesting a definition to each emoji or a definition of well.

D

That's that's kind of that's kind of why I'd like to slow down on defining the uh the metric, uh because I I don't. I don't think it's appropriate to define each emoji, but I also am not sure it's appropriate to just have a metric that is emojis.

E

No, no, I'm not a community manager, but isn't one of the things that community managers tend to measure is engagement. It seems like.

C

C

And then I think it's up to them if they want to take that a little further and try to measure sentiment, but definitely engagement, I think, is a thing that they care about.

E

The example that I probably have mentioned before that comes to mind is just like you know. Sometimes there's these random, like github comments that, like end up on reddit or hacker news and suddenly there's 10 000 hearts for something.

E

You know that kind of thing it just if the part of the intent of this is just that they want to find out where people are paying attention and they somehow weren't tracking that one issue or something I could see how somehow I'm having a metric to say, wow this one has 500 times more engagement, but I don't I don't know if that's the intent, I might be. You know interpolating, but um that's the example that keeps coming to mind with this one. For me, I think that's.

D

uh That's a really good point and uh I would uh revise the way I think about it. Based on that.

C

D

If we're looking at uh emojis as a as a measure of engagement, I think it's it's fair to. It would be fair to have that metric as a just.

D

We, we couldn't treat it as just one metric without splitting them out.

F

D

F

Do we have an engagement metric in any of the working groups?

F

That's a really good question.

C

F

That is the case, then, that this emoji can be a filter in that metric, because we are gauging the engagement, not the emojis, and the engagement is gauged through emojis, but not like. uh We are not looking in depth to the emoji.

D

That might uh an engagement, emoji might be that might be, or I'm sorry, not emoji. An engagement metric might actually be too high level, uh because we're kind of sorry everything that's done, you know in a project that we can count is is kind of engagement right. So uh how many prs that tells how much engagement there is. How many comments that tells how much engagement there is so engagement is, is more. Is this high level?

D

It's it's closer to a focus area than it is to a uh to a metric.

B

So in the in the met, this is um for for um nico. This is nico right.

A

B

Yeah, sorry, I I when I and I'm sharing, I can't actually see people's names and I'm going on memory. I just wanted to make sure I wasn't remembering wrong so searching the um this is a metric spreadsheet. We have for all five of the active working groups and the word engagement isn't present in any existing metric. Nor is the word reaction so to kevin's question. I there is not presently anything that I was that addresses it.

A

Yeah sorry I was, I was looking at the same spreadsheet um and, of course, there's the the number of issues. um Just yesterday I opened I'm not sure which working group- sorry, I'm not uh something about page views, that's also a metric. You can see on github at the moment, but uh that's not really.

A

um That must be a new one, yeah yeah yeah, um but that will be um that's not really engagement in the sense I mean that's sort of passive. You just open the page. Look at it, that's it, whereas actually putting an emoji. Is you insert information in so I I guess it's. It's sounds like it's a wording and we need to have a more specific term for it.

D

I, like the I, like the term, the davish uses laura davish. uh She refers to them as signals of attention, okay, uh which I'm sure it's there's several comments throughout this discussion about them being referred to as signals of attention, and if we do, if we think of them as singles signals of attention, then it very much is a a measure of engagement right and it's also a measure of uh those signals of attention can be uh used to inform uh how attentive the community is to uh to contributions in general. Right.

E

I would think something like this is rather tied to the technical implementation for measurement. So I it makes me wonder, sean you probably know the most like does. Gramor lab have other types of ways where they integrate this sort of signal into various.

B

Dashboards- I don't I don't know about dashboards. I know that both grimoire lab and auger do capture um these reactions, and these it looks like these are the ones available on github there's, eight and they're captured in the form of events. So each pull request issue.

B

Each pull request, issue and comment has an opportunity to get one of these reactions and that that shows up as a list of events associated with each issue, pull request or comment uh on both platforms, and so we have the data.

B

I think the question is you know what do we do with it in terms of representing it as a chaos metric?

B

We we don't right now with auger and I don't know about grimore lab. I I because I I haven't looked at it recently. I don't know if that's incorporated in their dashboards right now or not, but because both of us gather the data incorporating into the dashboard would be a say, a trivial exercise.

B

I think I think what we both struggle with is: what does that representation? What is that representation? That's useful to people? What is the metric that we can show.

E

I just wanted to say I'm sorry, I misspoke, I meant auger.

B

Yeah, that's okay, I mean I'm I'm. Obviously we are familiar with each other's stuff. Yeah I mean to me, it looks like like it would be hard for me to distinguish the or interpret thumbs up, love rocket and party and big smile like pot like almost these are all positive thumbs down and frown are negative.

B

I've always taken that rocket to mean let's fast track. This.

D

Okay, so it's not actually uh like a not a, not a positive. Well done thing, it's more of a put this on the fast track.

E

Okay, that's not how my company uses it. How do you use it? We all we mean like hey, we launched a rocket like this was awesome.

E

Like going to space as a billionaire for.

B

E

Well, yeah, but.

C

E

That we internally use called like strike or strap a rocket to it.

D

Yes, I guess to the to the point I made earlier: the the context is different for emojis based on project based on organization. uh So, but I.

E

Think it's wise to try and interpret those.

D

Yeah, I think we, I think we need to avoid making value judgments, but I, but I do think we can uh in in retrospect after having this conversation, I do think we can move forward on this metric and I think we, uh I think, you're correct. I think this is a it's. It's a measure of engagement.

B

And is the is the interpretation of an emoji really any different than any other chaos metric and so far as what we always say is that we're defining a metric- and this gives us a standard definition so that when people are comparing metrics across projects using a chaos, metric they're using the same defined metric- and it might not be right in every way that a person wants it to be right. But it is consistent, and so it enables comparison. And is this any different than that?

B

Really that you know, I don't think so, just a name comparison.

D

I think the language in that pull request was uh it kind of treated them all as like, plus ones. If I remember correctly, I can go back to that pull request.

B

F

B

Right to me, too, uh reactions number of thumbs up for other reactions.

B

And then the lang oddly, the the language, is all the same, regardless of it's all on issues, even though some are yeah.

D

Maybe maybe we make that maybe we make that a little more agnostic and uh say something to be the the number of emojis that that signal attention?

D

Okay or something yeah, so reactions the the number of emojis that are inserted into a comment that that signal attention.

A

Yeah I like the word attention as well and there's one thing with with context. Apart from the the emojis having different contexts, I've sometimes upvoted out of frustration, because you see somebody making an issue or progress. It doesn't come true, but you want to support it and and so that the thumbs up is actually. It comes down to the project, governance.

B

Oh, I see so so so sometimes sometimes the thumbs up is a way of signaling support when it perhaps the project is not, the maintainers are not necessarily motivated to do it, so it has a negative.

A

Connotation to the project so yeah, the gentleman attention makes.

A

D

But yes, I think that adjustment that adjustment to the pr would fix it in my mind and then I would I would say we could probably merge it so for the adjustment to the pr. Could you could you repeat the language so we're calling those reactions currently yeah right, so it's it would be uh signals of attention or reaction and then the number, the number of emojis that signal attention to an issue.

D

Something like that.

B

And is is: um is there anything we want to say about balancing of the emoji so that some emojis, emojis or emoji reactions would be eva? The valence of the reaction is interpreted by the project and may not be the same in each context.

D

uh I think we saved that language for the metric if we define if we define uh if we define these reactions as a metric. I think that's where we we use that language in the metric and I and I and I do think it would be appropriate to move forward with creating reaction as a metric.

D

But as a filter, I think it's just. uh I don't think we want to get too complicated with uh descriptions of filters in in metrics documents themselves. Those if the. If the filter needs a description, then it should exist within a metric document, its own metric document. Sorry, uh the goal is to keep the the metrics documents as simple as possible and focused on the the actual metric, we're defining okay. So a new.

B

Metric uh what we call right now in the spreadsheet there is a metric listed called reactions and emoji. um Would we change the name of that to be to include the word signal or how would what would be the name of this proposed.

D

Metric, so if the, if the filter is called reactions, I'm uncomfortable with the name reactions. However, you know you know when we, when we start defining this, we we do edit and have discussions about the names as we're going so that could change. uh I think, as I said earlier, I like signals of attention. However, if we used signals of attention in the description, I think reactions, reactions is fine, so just a reaction is a a reaction. Is a is an emoji that signals attention.

B

Right, okay, so just reactions or I I suggest, I'm suggesting reactions and other attention signals. um Obviously this is emoji, I don't know. Maybe we want to keep emoji in it. um Yeah.

D

Maybe to be to be as descriptive as possible, uh uh we may want to uh yeah. We may want to include emoji in the in the title. I don't know what that would look like. Maybe it's just emoji reactions.

A

Go ahead, how does that relate to the couple of rows above review comments.

B

So I think I think, review comments are, is a tracking of the comments made on a pull request, review or a change request review? So that's descriptive text and whereas an emoji is just a click, I guess that's the difference, but making.

D

uh So uh that does bring up the question of uh so in in in our other name in our with our naming convention we are, we are actually kind of pulling out these are. These are change request, comments versus issue comments.

B

D

B

This is a metric that kind of spans both code, development and issues right like it's.

D

So perhaps it belongs in the code development activity focus area rather than the code development. I can see that.

B

Process, quality- and it's um I mean it's, I guess the challenge with this metric is it's actually it spans both code, development and issue resolution.

F

B

F

Talking to the nikos comment is like how it is different from the review comments and the emoji. I was thinking that term came into mind that, like the picture speaks thousands words like uh thousand more than thousands of words, yeah.

B

F

In that sense, I feel both are saying my.

B

High school english teacher did not count that as a thousand words in my essays, I tried, but I hear what you're saying and and.

D

Per the conversation we were having earlier, it may be a thousand different words to different people, so.

B

I I agree: it doesn't necessarily belong in the class of code development process quality.

B

The only generic category we have right now is community growth. I don't think it's necessarily that either.

D

You don't think it fits in that top one. The.

B

My I guess my only development activity.

B

uh My my only my only reservation is that it can be applied to issues and issues are not necessarily code development. That's why we know that. So we do have a ratio.

D

Of issues to change requests metric, that is uh in there precedent, yeah, okay and, I think, they're, I think, code development activity. I think there's a lot the I think there's room to look at.

B

All code, development activity, right issues, issues and change requests all right. Well, let's see if I know how to cut and paste.

D

um Nico does that resolve your issue, or do we still need to dig in deeper to uh the difference between an emoji reaction and a comment.

A

No, I I there was more as a sort of a challenge for clarification, uh because I don't know what what's in the description of the comment, but okay sounds good yeah. I.

D

Will say for so for the metrics we have here, we are being we're being very explicit with uh a lot a lot of the evolution. Metrics are things that that we can count that git lab has provided us that github has counted. That's provided us so there's these metrics are very explicit right, so change requests comments. Is it's very specifically looking at the number of of comments in a in a pull request right, so that would be that. Would that wouldn't include comments in an issue.

B

B

All right, so I think I think so that's resolved. Do we want to go back then and suggest this language and reopen that pull request and ask that it be changed like are we? I think I heard two things I think I heard we want to develop a metric, and I think I heard we want to uh have the language and the pull requests that we'd closed last week changed to this, and then we would be willing to merge it. Did I hear that right.

D

That's what I said uh I don't know if everyone else is in agreement.

B

That seems completely reasonable to me to me too. Okay. So let me go back to this pull request and I guess this will be like uh upon further.

B

B

Separate metric called emoji reaction, but just remember.

B

B

Yeah and then um because these.

B

A

Also, there was the remark I think kevin made about the naming. Everything was named an issue even though it wasn't a different yeah.

B

Yeah and I think the with the language reaction number of emoji and that emoji that signal attention to communication or contributions within a project, so that would, if they use, if they, if here, were to use this exact language that would resolve this issue.

B

uh Yeah. That's a oops, not geeky girl,.

B

E

There is an entire atlantic article about whether or not the word emoji has a plural. That's emoji or emojis.

B

I yeah I I um I got. I confessed, I just sort of punted and it felt like emoji was right, but I don't.

D

Know it's fine, but in the past, when I've written it it's always been emoji. But I know when I, when I talk I say emojis, so I treat it differently interesting.

B

Well, we need a linguist.

E

It's gonna be official, you know, do it right.

B

Reactions all right, okay, so uh the pr is reopened with.

B

Reflecting this discussion boom.

D

So we may, we may want to move this metric forward now, while the conversation is uh still fairly fresh.

B

Sir sure um yeah we've gone off and on with this one for a while okay um before we do that, I just want to ask nico since he's new, were there particular metrics that you were interested in seeing developed uh in the in you know, participating.

B

I certainly want to give attention to any anything that might be on your mind as well.

A

No okay, all right, like I mentioned in the beginning, I'm I'm I'm mainly here to learn and uh I'd like to contribute to the discussion, but otherwise I would have created both yes, I guess so. No.

B

Okay, all right, um so our usual process is um kevin. I think you're suggesting that we would bump emoji reactions metric up ahead of change request commits as a working item in this meeting yeah. I just worry that if we revisit this in two months, we're going to have the same conversations we'll spend 20 minutes reconstructing the same conversation right. I tend to agree with you based on history, so the first kevin do you have uh the link to the template handy.

D

B

D

B

There actually there.

D

B

Be it might be at the top here.

D

It is so there is a link to it in one of the uh yeah it's at the top of the here yeah. I do have it handy.

B

Oh good, it's not in the evolution working group, though, is it.

D

Yeah curtis.

B

I assume you're putting.

D

It in chat, then the link to the drive folder is in here as well.

B

See right, okay, so this is the this is the template?

B

Is it me, is it me or is github doing something different with markdown right now, like all this usually marked in the past, I've seen markdown just displayed, and now I'm actually seeing the markdown code, which.

D

I'm sorry I forget which uh which focus area are we we decided that we would put the oops sorry. Oh this isn't actually I'm sorry nevermind. This isn't uh hang on. That might be.

B

Not a code block: where did I I'm sorry? I.

A

See this is it called block.

B

Oh there we go. Okay, ah okay! Is that okay? That's that's why I guess it makes it easier to copy and paste it.

D

And I just created the github uh doc.

B

Okay- and here it is, are we doing this.

C

Yeah josh may have added some some instructions around that template at the bottom, which is why it might look different because I don't remember seeing all that stuff before. Maybe it was there, I just never scrolled down, but now there's guidance on how to like refer to images and things like that.

B

All the right: okay, yep, that's.

D

Exactly what that is.

B

Yeah all right, okay, this is that makes sense, and so kevin has created the google doc and I guess I've, I'm gonna paste. So this is the.

B

Oops I'll, let you do that kevin.

B

And while you're doing that, I will go back to our minutes and just create a link onto this document that you made.

D

And then the uh the preferred uh the preferred way of working with these is to use the the suggesting, rather than the editing.

B

Yeah that lets us see, and I don't know you know be aware of- what's been edited as we go through things all right, so I now have the correct link in the.

B

B

D

B

B

Is this what I typed under objectives, really, the description.

D

uh The objectives should be example, use cases so uh more.

B

Of a description, yep.

B

Words are hard.

D

Yeah so get rid of the uh get rid of the bit. That says, the aim of this metric is and and just say, uh the number of emoji reactions on any get platform object with a.

B

B

Then obviously a filter could be a type of emoji emoji platform platform would occur yeah. I agree, I think- and I think that's important, because um it's funny that I don't know if uh any of you have uh looked. There was a study by the university of minnesota about how different emoji looked different on android and apple platforms and actually elicit different different reactions from people who see an emoji depending on the platform and how that platform uh sort of presents that emoji.

E

Yeah and so yeah go ahead. Yeah there's a ton of interesting research on that, and even like gendering of those emojis that the like there's a default mail in certain instances and some of them switched to try and be not default, and then that created other issues and then so anyway. Yeah.

B

E

Of a lot of fascinating research on that.

B

Yeah and in the united states, there's of course the I I really struggle with what there's like five or eight different skin colors that my thumbs up can be, and I I just like: okay, I'm a white guy. But what is that like the whitest thing, or is it like the more actual skin color thing um you know? Am I appropriating culture if I use a darker emoji skin color that I'm supposed to who decides that so yeah emoji certainly is a active topic of active.

B

B

I think it's also um this. This might be interesting, um I'll share it in chat, there's an actual style guide that google has put out there regarding words to use or not use.

B

um Which I I find interesting and I was actually made aware of it, because I turned on there's a new github service called prosbot that will flag text that you use in in different things, and I've turned it on for augur, and so every time we make a change or pull request the prose bot scans it and makes me aware of words that are uh sort of indicated to be avoided in the um in the google style guide and, like um I think, then it came up in the context of discussing the word grandfathered, um where something was grandfathered in which is a common term in the us for okay going forward everything's going to be this way, but if you've already got something you're grandfathered in and that in the us, that's actually a term that can trace its origins back to our slave history.

B

And so there are words like that: people, don't think of as having any problematic nature of it, at least not uh organically. But in fact, uh yeah. There's a lot of a lot of u.s language that is problematic in different ways.

B

Sorry that was, I didn't know about the bulk. That's interesting, yeah yeah pros pros bot. I believe it's.

E

B

I'll put a link in the chat to prospect as well. I just did okay. Oh you, okay yeah! I.

A

B

A

Oh yeah, you did yeah yeah.

B

Yeah when it was first, uh you know I was looking for plug-ins that I thought would uh be helpful for augurs auto. You know automating different parts of the work that we're doing. um I ran across it and I thought well. This is useful because we are updating a lot of documentation.

B

And so I think, uh usually we end these minutes in about. Oh, I like thank you for putting that in there. It would probably be good to have the I'll try to grab the get lab version of that real quick before our meeting time is up in a few.

B

A

I guess the the visualization is more about visualizing, metrics or wow, just for reference.

B

Yeah, um so git lab actually enabled.

A

That's good documentation, at least also yeah.

B

Yeah get gitlab, I'm looking at their emoji right now. They literally have the entire universe of emoji that you would find on your phone. um They have thumbs up and thumbs down listed, but uh I'll take a screenshot of this and put it in.

B

B

B

So this is like you can you can I won't. We obviously can't place them all, but you can see that.

B

B

um And I think uh we are at time, but.

B

I think this is really good progress, so thank you. Everyone.

B

I think it's my job as facilitator to say that we are officially out time and.

B

Thank you all very much for participating today and uh your active engagement and we even got started on a new metric and got a pull request reopened and gave g-org some work to do so. Any meeting that results in more work for g-org is a win um nice to keep that one moving forward. Yeah yeah! Thank you! Everybody! um I'm gonna, stop the recording and uh we meet every two weeks, we'll be back here. uh Whatever's two weeks from today, it's gonna be in august, so it'll be july, 35th, so august, 4th yeah!

B

That's uh that's not my calendar, math or math works, because I had when it's a 20 when it's a uh when a day of the week is a total of american football touchdowns. It's much easier for me to calculate what how far forward 14 is.

B

C

B

Into the mind,.

C

B

C

I just look at the calendar.

B

Oh, that's one other way to go. Yeah all right. Thanks, everybody we'll see you.

C

Hopefully, next.

B