CHAOSS Value Working Group, 8 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CHAOSS.Value.WG.Oct.8.2020

Description

CHAOSS.Value.WG.Oct.8.2020

A

Yeah welcome to the value working group on whatever day it is and whatever year it is. Oh october, 8th 2020., um we are jumping in we've, got pretty common crew here. So there's no need for introductions want to see what are on people's minds when it comes to agenda items. Otherwise we can get to the recurring ones that were coming up the last couple weeks or last couple meetings.

A

I think the the biggest one being on my end that I think I worked through first draft of project popularity that I'd love to get uh some eyes on and discussion around and then there's the alternate.

A

The dev focus popularity as opposed to the reputation based popularity that is still outstanding, but um we can dig into that uh so matt added a couple items. Anyone else have things on their mind: they want to bring up or follow up on.

B

That was a game. Oh go ahead.

C

Yeah I added two one is we are going to have a new zoom link starting november 1st, so just letting everyone know to make sure that we are not in the old zoom room at that point,.

A

Okay and are, are you on point for kind of updating that on the calendar and the agenda.

C

I don't know who's going to do it yet.

D

Okay, we'll get a calendar update yeah I mean I think, probably somebody could go in now and just update the links.

B

For all four.

D

B

Yeah, there are a variety of things like even on the meeting minutes, we'll have to get that on the participate page. Get that so we'll we'll do this clean sweep yeah.

A

Great yeah yeah, it sounds like multiple eyes are on it. I just want to make sure I knew who had the keys, because I don't think I could unlock this meeting, but I think we have enough people with it, but I wasn't too worried.

C

The meetings in the future will be unlocked again.

A

Okay, okay, great um scm or anything else on zoom people have questions on good to go all right, so fcms discussion uh yeah. I just finished watching last week's conversation. um If people are out of the loop on that, it was very revealing about the the conver conversation about sms so what's evolved. Since then,.

B

um I don't know which there were a couple videos one. I think the biggest thing is um we have to change the name.

B

Just we got legal advice to from the lf that, if, um if they are hoping to use scms as a name for their business, the trademark- and they want to trademark that name trademark- can just get real thorny, because if you can't prove the origin of a name, you actually can't get trademark or you can lose trademark right, and so, if, if they want to use the name- and we have it in the chaos project, it just doesn't it doesn't line up.

B

We can't do that, and so, and I do think um benia and dylan had had at least mentioned. The trademark was something that they were maybe pursuing or was maybe on their mind like down the road. So the recommendation was just to change the name in the chaos project, and I think we had come up with a name that was social listening.

B

Was that right.

A

That sounds uh heavenly on my ears honestly, because, like it's just, I know exactly what you're talking about when you say social listening and uh it's nice generic, so we won't run into uh yeah. So that is my overwhelming plus one.

B

Okay right on so I can, I can make those changes. I think at this point um it's been pretty much talked about enough, and so I.

D

Can make those.

B

Changes in the repo just renaming social currency metric system to social listening, this.

A

Shouldn't be too hard, yeah that'll be um it'll, be really fun to share, like, I think pretty much. Every it's a bit of an aside, but every every devrel team or kind of community builder I hang out with on the internet, is, is trying to figure out how to measure their community and um I've wanted to point them towards scms for a while.

A

Now I just I think it's been, the the language has taken quite a bit of explanation and I haven't had the time to like shepherd them through it, let alone knowing whether I knew what how to use it myself, but um with some retooling and renaming.

A

I think this could be like rolled out to a lot of teams, so it's very encouraging anyway, if you need a if you need a review on a pr or anything just tag me on github awesome, not during your week off, though god, no I'm gonna uh as much as I do, love github activities, the the new interface does make it seem like yet another inbox. I have to manage, so I will be turning that off. Yeah.

D

Yeah, I don't want.

A

You to be doing that. No, I appreciate it. It's cool! We are in a a fortnight cadence here, so that'll work out perfectly yeah fortnight. Is that what two weeks is that with that? uh Technically.

D

Wow yep, I uh see how I reverse.

C

Engineered that.

D

A

Outside of academia, we call that context.

D

We call that inside academia too, but with a lot more words around it.

A

uh I love it okay, um so the thing that's been around for a good bit. uh I think we could it'd be worthwhile to dive into project popularity. uh At this point, and let's just kind of recap where we're at.

C

Before before we move on on the social listening metric, we have discussed at some point of breaking out into individual metrics and all that yeah. We were discussing that. Maybe that's a lot of work and yeah. Our time is better spent elsewhere, because we have the social listening, metric defined sure it's maybe a little much in one metric, but it's enough to get people's ideas going. Yeah yeah and I think we achieved that without doing much more work on it. For now, we can focus on other things.

A

Anyone else have thoughts or on that.

E

uh I was just gonna say unless, unless.

D

I like to look at the spreadsheet and see where we're at I mean if there's project popularity seems like something worth working on.

C

And then we also have the conversation with unicef, digital public goods, social value, elizabeth had a point. I think she was trying to make on sorry.

E

I'm totally fine. I was just going to say unless back on the scms social listening stuff, unless someone comes along and really wants to champion it and shepherd it and and own it and drive the those metrics, then um then that's fine, but for now yeah. I just want to add that.

D

Is is the scms working group that that is that a working group? Now I see meetings there's just one meeting and there is no working group- that's going to proceed with the certain number of metrics that we identified that are part of that.

D

B

I don't think so if to elizabeth's point, if somebody would like to do that, I think value this working group would be a good place to do that, um but keeping meetings off the calendar is cool by me. Yeah.

D

I mean the question I mean yeah. Maybe this has been deliberated in a context I wasn't in so I should watch them. I should watch that meeting. No, it's okay! uh You can bring up your points here.

A

Though, that's complete.

D

I mean there were some useful metrics. I thought that emerged from it, but um it sounds like I guess I don't I'm a little confused about where that's going then or not.

B

So the current social currency metric system.

D

B

That is part of the metric release is going to be renamed social listening, yeah.

D

That part of caught is just gonna. That's just gonna exist as like this mega metric with.

B

Yes, so unless we want people to so we had a couple options here, so one is to break it out fully into the 10 metrics that you identified and build those metrics out, but we don't seem to have a champion to do that and the amount of effort to do that is pretty high. So I mean if, if you are offering to to do that, then that's cool.

D

But other than that, it's a spectrum issue for me, I'm not sure I can leave that mega metric out there without building the details.

B

The other option was to have scms become a program. You know like dna badging, but again we really didn't have the capacity to the bandwidth to shepherd that as a like, a service that we offered in the chaos project. So that was also a challenge, so we're trying to balance what is the most sensible way to still capture this information and make it available.

A

To people totally get it, so is this still related to uh the comments sean you you have from issue 75 and just pop that in and I'll add that to the.

D

A

D

A

A moment to review that and see if that's captured the the next steps that are on your mind or we're kind of um you know a bit of a gap between our open issues.

D

Yeah, oh okay, yeah. This is exactly what I was looking for. I thought.

A

I did a pull request that put out a bunch of.

D

A

Totally you did um so that pull request is built into um the metric.

D

A

Com yeah the release metric under communal, so okay focus area, communal value, scms, okay, um but, like I, I think, we've kind of I personally have lost the thread on the discrete you know what we were calling atomic metrics that you pointed out yep. I know I know them because I spent you know an afternoon.

A

So sean I would volunteer to spend some time with you to just write those down and see if it is something that we feel like we could batch produce into metrics like because you're totally right that there's a ton of context there. That might be interesting in in the discrete state.

A

I just think we generated a lot of there's a lot of really good work there, and um but um do you think.

D

That would be yeah. Let me read through this, I mean, let me read you you're taking a week off. Let me put this metric on my.

A

D

Today, no, but I mean it's thursday, so.

A

Right, yeah uh yeah, so I'm I'm around next week, I'm out the week after, oh okay, so yeah I'll miss this meeting in two weeks: okay, all right yeah! So let me read it over. Let me put it on my trello board. Yeah take a moment to metric again, so look through it. If 75 isn't the right place to follow up on it. Let's, let's open a new issue to do so and I'll. I would team up with you to go through that at least totally that was totally.

D

The one place where I put that work yeah it's also on the pull request, but I don't think there's we can move on from this. No well.

B

Could I make one suggestion on the work? Maybe when you're going through those? Is it five to ten atomic metrics? I think I had eleven when I was also eleven. Okay, um maybe try to place them in the spreadsheet as well. I mean we could start kind of capturing them in the spreadsheet under a focus area, and the remark could be. This is pulled out from the social listening method. I think if we just go that far right now, that.

D

That's that's useful, but let me read the issue again. It's been a few months sounds good.

A

Yeah all right and I'm just going to update that issue. So it's not stale.

A

Okay, all right great question your gun following up there. Thank you um anything else on.

A

Sms: let's talk about popularity.

A

I can almost briefly share my screen just to make sure everyone's seen the same thing.

A

Okay, so from the question of popularity, um we had the initial conversation that separated out this idea of, uh like the reputation of the project versus the market share of the project.

B

I'm sharing your screen map. Oh, am I I don't know no. You are met, but you also have a couple other windows open that if you don't want those shown in the recording.

A

B

Will cut them out? Oh no, okay, that's fine! I could yeah I'm.

A

I'm like it's just hiding.

D

I know how computers work, it's hiding the screenshot from me, or did you stop screen sharing? I did this this moment.

B

D

Okay, okay, yeah, I'm sorry, my zoom gets a little wonky, sometimes.

B

With multiple moments now I only see that window. Okay, do you want me to.

A

B

In the video and.

A

Oh no, it's like it's. Everything on my screen is public. Okay, I just want to make sure thanks. I can still see a little bit about my personal to do this, but we.

D

A

That we know that.

D

He's really andy sandberg, and this is uh he's doing.

A

D

A future character.

A

Yeah for the follow-up to the social network.

A

Such a funny uh that's a great thread. um I can definitely think of worse comparisons. uh So, anyway, uh speaking of comparisons, let's talk about popularity that was a weak transition, we'll take it. uh We yeah. We basically got to the point of saying: there are two discrete methods to discussing popularity.

A

One has to do more with usage adoption development type popularity that there's some comments on in progress here and some updates from sean, but I don't know exactly where we left off there. um What I wanted to draw your attention to the project popularity reputation.

A

um I think I got this down to something where I'm ready for this group of people to kind of do the really detailed review of it to see if it's ready to open into a pr um most notably, I would really like some other visualizations. I grabbed this one from devstats from the kubernetes.

D

A

It looks exactly like a dubsteps visualization.

D

A

That would start down to the color.

D

A

No, not at all, no, I mean actually with.

D

A

Yeah yeah um um so.

D

It's recognizable.

A

Very distinct, without a doubt, I'm just playing um so tools. I listed territory, agar devstats. I guess I'm sure there are more, but those are the ones that come to mind. That matter to me, uh and I have a little bit of a description of data collection strategies, I I will need somebody else's opinion on whether that's helpful or insufficient, but is is that a draft document is that link in the media minutes or in the yes yeah I'll stop sharing, and uh it should be linked to you when you see.

D

A

D

I wasn't expecting the.

A

Fancy fancy kind of linking all right. I love that's probably my favorite addition to google docs that, like you, can one click yeah change from the url right into the title, that is nice. Okay, it is fancy.

B

So, do you want to matt? Do you want to take some just some time and we can just yeah.

A

I'd like to go with you guys, why don't we take five or so minutes and people can read through kind of bring up discussions yep help yeah, I'm with you york, sorry! Well! I was tight.

B

Oh yeah, sorry.

A

B

Right, what's up.

A

Which is james.

B

Message we should.

A

Record this, because this is the this- is the meat of it. Oh did we not record? Are we not ready.

A

Are we recording again yeah quick recap for the recording we're now having a discussion about project popularity and whether it's framed appropriately?

A

I wrote it with the intention of framing it toward the scenario when you're in a business context and you're, making the decision on whether to use one project or another um the I. I really had this like idea of implicit bias in my head, as I was writing it of like when people don't have clear about like understanding of the value they need to derive from something we kind of default to this lens of popularity, without discussing it openly and that's why I'm so fascinated by it.

A

So in that way, I've seen people make huge like many million dollar investments, just based on like well that one's got more stars you're like how do I capture that level of just immediate value so that people know that while stars are vanity metrics by their very definition that vanity has real outcomes?

A

um So that's the story. I was telling myself um but your story, elizabeth of, like I'm an individual, do I want to contribute to this version of a container runtime or that version of a container in time, because there's more popularity over here, that's it's the same assessment but um different context.

E

And also like matt, said, matt g said job opportunities. So if I have to learn a framework which one am I going to learn, you know for the most opportunities. Oh maybe I'll, learn the one that has the most stars and seems to be the most popular. Because then I know that that's a more valuable skill to know this framework and be familiar with it and maybe even contribute to it than something else that nobody's ever heard of that doesn't have any stars.

A

Yeah, so this this reminds me of um a challenge that I see as times of like I guess in the description we could maybe talk about the interconnectedness to other metrics um or, like I think you know this probably isn't the right venue to do so. But I I have been wondering if there's a pitch for like in the default template.

A

There is an interdependent or interdependency type section uh to a metric of like this also relates to how we think about x and also could depend on y, and we start linking across working groups or within working groups. If the context calls for it.

B

So I think that's a good idea um to date. We just kind of had it those relationships built as links in the text. You know so, if you're referring to something we would just say so, for example, um the metric that is called change, requests which are basically like pull requests.

B

We have a a metric called change requests and then we have a series of companion metrics, like um you know, number of open change, requests, number of closed change requests, and so in that case the latter two would reference the first one because we're yeah, um but having an explicit section of like what other metrics. Are you talking about or you're pulling from? That's a nice idea.

D

So it doesn't get buried in the text yeah. If the if this metric exists in a network in your head might get a rough picture of that network would be a novel way to think about this.

A

Could be fun yeah? It's um definitely come to my mind a few times I'm happy to open, maybe uh in the in our chaos metrics or I can just put a placeholder issue in there to further discuss that yeah cool um and this one elizabeth. I really like your point. I I added in the in our document to links it's to the individual value, job opportunities and individual and individual value, organizational projects, skill demand, uh if you take a peek at those and if you think they fit into what you were saying like.

A

Maybe we add a discussion point in there in the in the description to make sure we capture like another lens on popularity, is.

A

E

Yeah, I think so too,.

A

If anyone's feeling their creative writing and want to add the the.

D

Well so so I was actually, I took your document link and I started project popularity, development and usage, so I edited some up there, which I realized now is in a different location. You guys are working on.

A

D

This stuff- oh, that that's.

A

Fine uh yeah, I I have been interested in in getting that one further too. I know dwayne tentatively uh offered to follow up on it, but um we haven't reconnected so you're already.

D

A

D

All the project popularity metrics from this perspective of developer, making a choice or is it reputation? No it's this.

A

Oh and yeah, oh there, there is no predefined context. I I added a context to project popularity reputation because it was just the lens I was thinking through. um I don't think there was. We didn't agree to one ahead of time when it came to project popularity, development and usage. Well, that's what I added in what you're saying so it.

D

May not be right for what we mean, but I wouldn't mind if these are networked in some way.

D

I guess just develop is development and usage that is really not about corporate consumption necessarily than in your in your head.

A

Yeah, I I guess, um because I I think.

D

A

D

Mm-Hmm, do you think you're right, you.

A

D

I mean we're: actually reputation is it's. The actual specific metrics are the same for both of those metrics right now. It's fork stars pull requests downloads.

D

It looks like two parallel metrics.

A

Yeah, it's it's almost like we're just trying to build two different interpretations of why you would measure it, but when really there is no. I can't find a difference between oh.

D

A

um Kevin added this when we had the initial discussion of engagement, how conversations take place around issues and pull requests clones? How many followers do people can yeah? Oh, how many followers do the people contributing to pull requests and issues have, but that's relevant in both contacts?

A

So I think we're really trying to split something that based.

D

A

D

Yeah, so I mean what is the difference between development and usage and reputation? I like I'm, trying to understand if there's really a distinction.

C

So one one thing we have done with other metrics is to say: hey. We are looking at this idea of looking at the project like in this case, project popularity and then, if we have different lenses in the objectives we can have, we sometimes have like bullet lists. Of these are the different things we can use this metric for and then in the data collection. If that's the same, then it makes sense to have those lenses just be listed in the objectives.

A

Yeah, that's great so reviewing the meta above the notes at the beginning. It really hinges on. There is a conversation around is popularity about reputation or is popularity about market share. I think that's the most concise way of saying it like who's actually used more. Is you don't measure github stars to find out who's used more you measure, you know, I guess downloads, but that's yeah uh you're, trying to I guess, you're. What you're attempting to calculate with your measurement is different.

D

um I I think, you're getting it yeah at the very beginning of a project. It's all. I think reputation, because there is no market share, like the reputation of the people or the project or the organization would lead me to get involved in something very early and then farther down the line and really, as an open source contributor.

D

That would be I'd, be looking for reputation to decide where I would make a contribution or invest my time as a sort of freelance open source contributor um setting aside my own interests or her needs.

D

That's that's how I might make a decision, but if I'm approaching it from a corporate perspective, then popularity is related to my ability to get a job somewhere else. If I don't like where I work, and so then you're getting into the market share things like what are the people at the devops meetings or the the local meetups really talking about um that's when I was a developer, that's the lens that I saw the choices about what technologies to pursue. That's how I, how many decisions in my own head was?

D

What are we talking about with the people who are on the bleeding edge at these bleeding edge companies? You know and you get to know in a local market if you go to meetups pretty quickly.

D

What's what's hot two years from now, and that's the I mean that's about like that's different than market share,.

A

D

No you're right.

A

Yeah, it's you're right, it's pre pre-market share more just like the doesn't have the energy like. We have all these weird metaphors to apply to it as opposed to mind, share yeah, exactly all the words are listed out at the beginning of that bottom. One um mind: share influence, interest, excitement, momentum, um so so sean are you saying like there?

A

There is a way in which we should separate and have separate metrics.

D

I don't I am I'm approaching it from the lens of whose perspective I'm taking that's, that's the that's the how I'm looking at this and I think that corporate individual making technology choice is probably more likely to be looking at current market share or employment opportunity kinds of metrics.

D

If so that they, you know, they don't want to pick a language, that's not popular. They don't want to pick a platform that there aren't lots of people using and then reputation is really it's. I think more. It may be developer.

D

You may be able to take that from a developer perspective as well. I mean it kind of depends on your profile, so I think it's this phases of a project life cycle like at the very beginning. The reputation of the people involved in the project is how you decide whether or not to invest early, but so I don't if they are different. I think I I think I think there's different.

D

I mean I wouldn't so I wouldn't measure if I'm thinking about it from a reputation perspective, I'm interested in the followers of the individuals involved in it a lot more than I am in pull requests, forks, repo stars or issues. um I might be interested in blog posts and- and I think the the followers of the individual like at the reputation level, I'm interested in how many followers the individual developers involved in this project have totally. um That is more. What reputation is.

E

I might also make a connection to bring this conversation. Full circle bring connection to these social listening, because a lot of that reputation, stuff, I think, is buried in in those metrics.

E

um I think there is a connection or some I don't know how we want to explicitly call that out, but um things like trust and utility, and um you know the other three that I can't remember right now, um but there is some. I think reputation does touch on some of those issues.

E

A

Yeah man every time I feel like we're, I'm like very happy that we're like really trying to gnarl through all these details, but at the same time I feel like every time I feel like we start to pull on a thread. We just find that it's all one big clump following the one thread well.

E

I thought yeah, I agree, but also, I think I wonder if there is some in those discrete metrics that sean was also digging into is. Are there any in there that we want to pull out and use for reputation.

D

I think I think it's the it is the followers stuff um uh I'm pretty good comment. Then I actually did a paper about this, but open source developers absolutely follow individual developers with a lot of followers from project to project yeah, um and so, if you're talking about reputation, I think it it's the organizations that are involved and the individuals that are involved that that so that's it's. The stars, maybe are the forks on existing projects inside that github, org or gitlab.org, and it's the followers that the individual developers have like.

D

I think those are reputation, kinds of scores for early stage projects and then the stars, forks issues pull requests. Those are involved in the like. Once a project is popular all, then you can see the stars and you can see the forks and you can see the pull requests once the project is popular.

D

Does that make sense as a distinction.

A

I'm I'm hearing like leading indicators versus lagging indicators when it comes to kind of nascent projects and and.

D

A

That yeah, the leading indicator being like are people of great influence involved, yeah full stop and then so. There's like a. How do we calculate their follower account and on what platforms are we pulling? That follower account like? I think we could.

A

You know there are people out there that are interested in this, that don't know that, like twitter is influential in open source right and like that, you can pull the following account from twitter, pretty quickly that might be worth documenting. Yeah.

D

When you have people following you on github.

A

As well yeah exactly yeah, so what are other channels.

D

I mean github. Followers is a pretty good indicator from from what we saw four years ago, um like I have 88 followers. I have no idea why people choose to follow me or what that means.

D

It means you you show up in, like I yeah like that number like I don't it's a very it's like a medium number, it's not any like. We had people with thousands of followers in the study that we did yeah.

A

That is fun to look at like who am I following it? Why.

D

Yeah, I don't remember following anybody, but I'm following 77.

D

D

I'm sorry I feel like I made it more confusing and no no.

E

No, I think that's my job.

A

Okay, so I think it is definitely a more confusing state, but I don't think you made it more confusing like we're. We're discovering confusion, we're not creating it. Okay,.

D

I'm just trying to think of, I think your notion of leading and following indicators is a good frame.

D

A

Update today's date, what are we thinking? It's like.

D

A leading indication, reputation is a leading indicator. um Development and usage is a a legging or a parallel indicator. Probably lagging would be a good way to draw a.

E

E

Do we care about uh people who are giving talks at large conferences or popular conferences like does that fit in here somewhere? I feel like.

D

E

I don't know how you would collect that data easily, though.

D

It's reputation, I think so.

A

D

A

Not not automated, but it is easy. I'd say like if you know the list of events in your space, you can scroll.

D

A

And tally: uh it's not graceful but you're. I totally did that uh when I was looking for who's influential in like the go programming language in 2015.

D

um Who's participating, if you picked up on go in 2015, you were following leading indicators.

A

Totally yeah absolutely yeah. I didn't learn it well enough to do it as a like to be good at it, but yeah. I was trying to pay attention um yeah, okay, so I uh we're coming up on time, so I don't know I I don't feel like. I have like a a very clear conclusion on what to do. I think we should probably I'm gonna, keep thinking about it and see if these do separate into these leading and lagging indicators.

A

Should we start.

B

Well, while all of you were talking, I was hanging out down on page whatever, okay in prague and in the project popularity reputation, so I I've just been kind of doing editing and trying to think about things, but based on your conversation, this may change a little bit, which is okay.

C

A

Yeah no, but your edits are appreciated. um Yeah, I think where sean was going and and elizabeth got us started down that path of it will make the whole thing make more sense um and kind of reminds us that there could be two separate concepts happening at once here.

B

This is the leading and lagging indicators. I like that from what I.

E

B

It makes a lot of.

A

Sense, it's kind of a fun way to say it to yeah, like popularity as a leading indicator. Popularity is lagging indicator.

A

And yeah, I think so. The capture.

B

The question would be: do you want a metric? I know we're at time, but do you want a metric for leading lagging or do you just say?

B

Maybe you talked about this already, which is like project popularity reputation and then within that metric you say: there's like leading components and lat and lagging components right so.

A

A

D

A

D

No you're good.

A

I don't have an opinion on that, yet if people come up with one um I'm just going to update some of the notes in the open issue for popularity, um I welcome dialogue and github issues or violent the mailing list. Let's uh be respectful of people's calendars. I think I need to put put a pin in the conversation for today sound good. Thank you. Thank you man for me, yeah. I made some.

D

Additional edits, so I will see you next time.