CHAOSS CHAOSScon EU 2020, 23 Feb 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Measuring Culture | Sharan Foga | CHAOSScon EU 2020

Description

Measuring Culture

They say you can’t be certain of anything unless it can be measured – so what about culture? We know it exists and can be passed on to others, so there must be a way to measure it. Using Apache Kibble we attempt to look for cultural indicators at the Apache Software Foundation, set a cultural benchmark then match real Apache projects against it. In theory we would expect a perfect match but as the results show, culture isn’t that simple.

Slides: https://chaoss.github.io/website/CHAOSScon/2020EU/slides/Measuring-Culture.pdf

-----
Sharan Foga

VP Apache Community Development - The Apache Software Foundation

Sharan Foga has over 20 years in the IT industry working on the delivery and implementation of IT projects. She is currently based in Stockholm, Sweden and works for Sandvik as an Agile Delivery Manager. She has been involved with the Apache Software Foundation (ASF) since 2008 and has presented at several conferences about ""The Apache Way"" of engaging and empowering communities. She enjoys working on open source community management and is VP Apache Community Development. She is a Committer, ASF Member and is involved in several ASF projects including Apache Community Development, Apache OFBiz, Apache Kibble, Apache Training (incubating) and Apache Incubator.

-----
About CHAOSScon

Learn about open source project health metrics and tools used by open source projects, communities, and engineering teams to track and analyze their community work. This conference will provide a venue for discussing open source project health, CHAOSS updates, use cases, and hands-on workshops for developers, community managers, project managers, and anyone interested in measuring open source project health. We will also share insights from the CHAOSS working groups on Diversity and Inclusion, Evolution, Risk, Value, and Common Metrics.

https://chaoss.community/chaosscon-2020-eu/

A

Here at to the Apache Software Foundation I'm involved in a few projects, including to the community bank development and one of the projects that I'm involved in is magic pebble. Now, a few years ago, I actually went back to study and one of the things that we needed to do as part of the study. This program was to come up with some topics for the paper, but to.

B

A

Up with some topics for to write about in that as part of the sort of the course evaluation and I've been involved with a patch stuff like foundation for quite a while and I realized, there was not a lot of information or research or whatever was done around the foundation, so I thought well: okay, I'll choose a topic around the foundation and one of the key things that they promote a lot in the Apache Software Foundation is the culture the Apache way, so I thought well.

A

Why don't I have a look at culture and see if I can measure it, because if you think about culture, it's about what we believe in, but it's also about what we. What we do so there's behavioral aspects to it and you can do something then there's an impact. So therefore there's a chance that you might be able to to measure it and the belief within the foundation is that that Apache way is so ingrained in the foundation.

A

That I thought that there should be some essence of it in how we interact in how we communicate in what we do. So therefore, I look for some indicators that would help me, because there are some aspects of the Apache Software Foundation culture such as meritocracy welcoming and things like this I thought. Well, actually, we can maybe look and sleep what we could find that could indicate this. So one of the things I came up with was five indicators that I was going to look at. Oh I got to mention the Apache cable.

A

So for those of you who don't know what a Patrick cable is it's a like an aggregator for community stats. So a while ago the foundation was using, a software called earth salute to find out to gain metrics on various projects, and this plot this that this company don't donated the software to Apache, and that became Apache cable service.

A

We have a source code for all the the metrics that were that we were running previously around the community, so it aggregates information about the contributors, it a brigade them just sentience and how sentiment analysis it does contributed their details. It pulls in detail from your issue, tracker, etc.

A

So anyway, one that advertise you an advocate eval on just want to talk about what I used it for so um I found five indicators or chose five indicators from from compatible wonder was that the pony factor and I think you may be refer to it as a bus factor, and this is the sort of like the sort of minimum number of people that it would eat that would take sort of kill off to make the project die effectively. So if you have a fetus, you know four or five key people in your project.

A

If they were no longer there, then your project will decline and probably die, and we we call it the pony factor. There seems to be something around ponies in a factory of pony males. We have protein factor, but this is what it is and the sort of calculation is sort of like you know, what's that, how many people is it that make up say 50% of your codebase, so those people that are putting in the big contributions so and that's the number that we use? We also can look at the contributor experience.

A

You know when you look at community is an evaluation. If evolution, you know how long have people been involved with the community? Are there new people coming in? Are they actually sort of evolving or or evolve evolution moving into sort of longer term contributors? We look at sentiment, analysis. Sorry, you know. What's the mood of communication there? Are there a lot of discussion and alchemist, or is it generally positive? We can look at that over time and one of the other things that we we have. We have there is a kefir keyphrase extraction.

A

So when you look at a communication, you can sort of look at key phrases that community is using, and these are the important thing. So, if there's any cultural language, then you can use the keyphrase extraction to pull out or highlight this cultural language. Okay. So so, let's see so before, I start right. One of the things I wanted to show you what I said: HTTP server. So what do you think of a patchy Apache was formed based on this one project right, so the culture of Apache comes from this one project.

A

Everything the parties today came from this this project, so what I did I thought? Well, if I look at the webserver and now mine creates a baseline of what the webserver looks like well, then, surely all the other projects that have something evolves from the apartment adopted the Apache way. Culture should look pretty similar right wrong anyway, so I see well so anyway, so um one other things I did was I said right.

A

Let's look at the pony factor for the webserver over time and I looked at this I thought this looks really really weird and there's three sort of tracks on the bottom that we have on cable for the possible for the pony factor. One is the pony factor of the contributors, so the number of contributors that are making up both over 50% of the codebase.

A

Then we look at the pony factor for the committers, because we can't differentiate between the two. So then you can see whether or not it's just the committers that are contributing rather than new people coming in, and this is important because you know part of it is about being welcoming and being trapped. Putting new people to come in and be attributed in a significant way if this isn't happening. This is really important for you to sort of look at and look at and the third one which is in it's around.

A

We call the megaphone factor, and this is around trying to measure the different types of organizations that are involved in in the in the project. So the pony factor that we have. We have been over the code. We have it over the email and have it over the issue chapter as well, but I've just extracted the one for the codebase and I looked at it.

A

I thought this looks really really weird, because it's like the pony factor for the code, but for the contributors, it's exactly the same as icon as the Pony factor for the commissars. So that's a telling me hold on you know: there's no new contributors coming in, but if I looked a little like that, we'll hold on HTTP server is an old project and it was actually done on SVN and we for kibble. Well, you think it to pull in if we're doing the Apache mirroring into the git repository is to pull in the detail.

A

So actually looking at the data. Now, it's telling us one thing, but then, when you look at it, when you think about it, you think well, okay, if it was done with SVN, then only the only the commissars could merge it anyway. So therefore, this is a bit like a false reading. You see so this is. This is something so then what I did as well was Amazon to compare?

A

No, so here's the old original project and let's have a look at some new projects, so we have incubator incubator, where projects that want to join the foundation come in, and this is where we teach them about the governance. We haven't the foundation, we teach them about the Apache way and what it means. So, therefore, what we want to do really or what I was looking for, was the same route of care. He was a piece of projects that came into Apache.

A

They don't know anything about our culture and now throughout incubation they're, going to maybe sort of move or demonstrate some attributes of the culture that we're trying to embed in them and when you look at and I so I sort of selected at random, some six projects they've been incubating in the foundation for less than 24 months, and so, let's just highlight it here, they're sort of sort of timeframe that we're looking at so this is that they exist in on they have their own git repositories beforehand.

A

But this is around the area time this time for core that they actually came into up due to incubation I need to see you know, look at the blue behind is the poly factor the contributors, so you can see now doesn't have a stolen, and maybe this could be one of the reasons why they decided to come to Apache, because sometimes projects come to Apache to build their communities, and you can see also as well that they're, the perpetual committed is the blue. The red one is actually going down.

A

So what that means is that the committers were being less and less active on the project. So this is another may be a reason why you know thought they'd come to a party. We can actually build the community buildings and third one was the metal Pony factor which is about organizations but I'm, not so sure that that is as accurate as we want you to be so I put that side, so I'll just focus on the mischief and the contributions, and you think.

B

That there's some.

A

Sort of like ER that point at it here so here you see that it's tracking and this difference, and one of the things that I wanted to look for, was ask the contributions.

A

Go up that the actual commission ship goes up as well, and it's sort of because it effectively what this means is that, but if it's only in parallel, but some of these contributors are actually being converted into commissars, it could be that the commissars are the commissars from here that were here with were that were acting up here were actually less active, became more active and more likely scenario is that the committed the contributors are being converted into inter commissars and to put that in in the party terms, is a cognition of merit.

A

So if you do something that is good for the community, then being made the committer web history code to doing event, management or documentation, that's recognizing that merit the next one I wanted to look at was around to contribute to experience. So we can have a look at this is HTTP server. And when you look at this, you can see like okay. There may be nineteen percent over seven 19 percent of the people have been in have been there for less than a year.

A

There's some people interpret there's some people that most of the people in that project have been on that project for more than five years. All right what you don't see here, you don't see any there's no progression at one to two years. So this is some perceive up hold on you know is there? Is there a reason why people are not progressing between the years and also as well? Maybe this could maybe due to link into the what was the growth maturity by decline, the sort of evolution?

A

You can see that the you know the the actual experience of the people is going up, but now it's starting started to slip a little. So it's! This is maybe another talk from another time but anyway, so this is. This is reactive people, the people that are being retained, some of the people that are left and some of the people that come back, so we can track with the coming with the communication some of the the this this traffic.

A

So once again, comparing that to some of the incubating projects, and once again you look at the contribute to experience how long they've been involved with the project, and you expect this- it's a new project. You know the bulk of the people are being in there for less than a year.

A

You know and believe the lowest number of people have been there for more than five years, I mean some of the people have been involved in the project before it came to apache, but we can still track that and you can see the evolution of less than a year, one to two years, two to five years, more than five you. So this seems to be a nice sort of like flow going through the contributors moving moving from one to the other, and once again you can track this as well over time.

A

So here you've seen that the active people is going up, which is really good, because this is what you want. So is this an indication of the community is welcoming and happily accepts new contributors. So this is something to look at as well.

A

So one of the other things that I did was at the mood and the sentiment analysis, so I ran it over the HTTP server and one of the things that's work. It's, although the overall mood was positive. One of the things I feel.

B

A

Look, this is quite significant.

B

This is negativity.

A

Hold on, why would negativity be that high and I think? Maybe this is my business, my board, you know it's a mature project and also as well. You know people who can be quite strong in their in the way they think the community cares, but I think there's a night. So maybe the negativity is caught with that, but we have the mood over time and we can actually track all the different moods there being.

A

Yeah, this is a pretty low. I mean actually have a date around about when he happened, so we actually drill down into the mailing list for these clinics project and see what were they talking about around here? What was it? Let's cause? They stick. Similarly, here this a peak of sadness, something happened around this time, not not a big one. The food something happened on many lists around at this time that indicate this sadness sentiment. So this wasn't a nice. This is a nice tool. We have a habit sort of my next I think.

A

My next thing would be to sort of like jump into this okay right. So let's look into the mood at the time of the the incubating project, and you can say- and this is unexpected as well- is that it started the negativity is a bit lower. It started.

B

A very, very positive, but.

A

It's actually going down. So what what is happening is this. That is this as a result of the projects dealing with the governance dealing with the Apache. Are you dealing with something dealing with with releases, but once again you can go into the mainly mix and have a look and see what it's causing these sentiments. So this was quite interesting.

A

So then, the last piece that I looked at once around Keith they raise extraction and one of the things that we have that we use a lot in Apache is what we call a consensus indicator and consensus indicator. Is this thing up here? I? Don't you can see if it's a plus one, so normally what that means is. If somebody has an idea- and they say- hey I- think this isn't be a I like to do this. The response, knobblies plus one yet I, agree.

A

I am happy or minus one or you know, NIT zero Mitchell, but normally, if you want your greedy or you want to help it's plus one- and this came and okay, you see when you see some people's names in there as well. But you know this: these are the key phrases that are coming out for HTTP server and this consensus indicator was created there. So you expect to see if it this is the key phrase: extraction for the incubating projects. I say less than two years, so normally I'm guessing but I.

A

Think I suspect that you know not everybody in the understandings don't use it discuss one, but one of the things that I did find in these projects is and at a lower a lower level. But here it is again the consensus indicator so to me and showing them at least from a cultural language. Effectives something is being transmitted through the incubation process. That's allowing them to decide to use that language as part of their everyday and communication on the mailing list and we are being able to pull it all out.

A

Okay, we have people's names as well or whatever, but I want to saw that when I did my paper, I was really really excited because I wasn't I, didn't know what I was going to get and to see that was really really quite exciting, and that was a key thing that I put in putting my paper. So um so what did we find?

A

So in summary, we found that nothing really matches completely, because you know I expected to create a baseline from hey the web server and then you know match people match the projects on top of it, but that didn't happen. Actually I found some more questions to ask some more investigations to do and looking at these five indicators I mean, maybe they like them anymore, but these ones seem to be. You know the bus factor.

A

Quite interesting ones to be able to use to at least maybe sort of try and identify some of the culture that you're trying to.

A

Today and if you have any question and I've also included a link to two papers, that I wrote in the presentation, so one is around just cultural analysis, where I did an analysis of projects that have been within apache under five years and over five years and looked at those compared with the web server. The second paper was around a similar sort of thing, but I want you to look at things of projects that come from a different culture.

A

So we have quite a few Chinese initiated projects in Apache now, so I did an analysis of the Chinese initiated projects that came through incubation. Some of the projects that didn't go through incubation because we do have some and plus the web service I didn't mismatched or love this match, but a match of those as well, and that had some different conclusions, especially around the Moodle and the sentiment analysis.

A

The mood analysis was very sort of subdued, but anyway, so if anybody wants to have a look at those papers, I've included the links for those in the presentation as well. Okay, so any questions.

B

Yeah the ability to okay.

A

We don't we don't currently have it in in cable, but if we can define what it will look like, then we're quite happy to incorporate it. So if you maybe we can have a chat afterwards and we can maybe sort of put something there. So yeah yeah, look at that. Okay,.

C

Thank you very much for showing this Kimball is always a good partner project and I loved the collaboration that we have between chaos and kidney. Thank you very much for the talk.