CHAOSS CHAOSScon EU 2020, 23 Feb 2020

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Application of Health metrics on a Cross-sector software ecosystem | Johan Linåker | CHAOSSconEU2020

Description

Application of Health metrics on a Cross-sector software ecosystem

Business and software ecosystem health can be considered as a key performance indicator of an ecosystem's potential to create opportunities for its members.

In this talk Johan will present the practical application of ecosystem health on JobTech Dev, a software ecosystem of job-matching actors in Sweden. The ecosystem is lead by the Swedish Public Employment Service and is underpinned by open data and open source software that is used and co-developed by the whole ecosystem.

Elicited metrics cover the ecosystem's productivity in development and maintenance, robustness to withstand change and disruptions, and openness for new business applications, use cases and external contributions.

As the metrics has just been introduced at the Employment Service, Johan will share some initial lessons learned and discuss the roadmap of connecting the health metrics to impact metrics relating to the ecosystem's common vision of improved job-matching on the Swedish labor market.

Slides: https://chaoss.github.io/website/CHAOSScon/2020EU/slides/application_health_metrics.pdf

-----
Johan Linåker
Postdoctoral Researcher - Lund University
@johanlinaker

Johan is a postdoctoral researcher focusing on how the public sector can create platforms with open data and software on which ecosystems of actors can innovate through cross-sector collaborations. In his Ph.D., he focused specifically on helping companies make contributions and engage with communities in alignment with business goals. He sits on the advisory board for Hack for Sweden, the Swedish governments gov-tech initiative for enabling open and data-driven innovation. He is also the founder of OpenHack, a non-profit tech-community solving real-world challenges connected to the global goals in Agenda 2030.

-----
About CHAOSScon

Learn about open source project health metrics and tools used by open source projects, communities, and engineering teams to track and analyze their community work. This conference will provide a venue for discussing open source project health, CHAOSS updates, use cases, and hands-on workshops for developers, community managers, project managers, and anyone interested in measuring open source project health. We will also share insights from the CHAOSS working groups on Diversity and Inclusion, Evolution, Risk, Value, and Common Metrics.

https://chaoss.community/chaosscon-2020-eu/

A

Or I've learned a lot today about new nuances and things to take home about metrics. I will learn about what the meaning break a leg means so been a good day for lots of learnings. So I'm going to talk about the application of health metrics from the ecosystem, part of literature and on a cross-sectoral software ecosystem so and this software ecosystem, I'm talking about it, has access both from the private and public sector.

A

It's initiated and facilitated by a big swedish agency called the swedish public employment service, so they are responsible for enabling job seekers and employers to to meet each other and hence trying to reduce the unemployment rate in sweden. So, with this ecosystem, they want to release, create a platform with open day, open government data, open source software and enable actors within the job tech space to create better services and products that can help or improve the digital matchmaking.

A

On the swedish labor market, a quick bio I did I've done a phd finished this autumn, focused on what companies to cherish open source, how to engage in communities in ways that align with their business goals. So since october, I've been involved in this project, it's a two-year project together with the employment service.

A

So my role is doing something called action research, so both helping trying to helping them adopt best practices in terms of developing open source and and community adopting these practices, but also trying to find a model or lessons learned for other organizations within the public sector to to adopt, in their own digital transformations. Adopting this platform ecosystem concepts usually applied within the private sector, but first, what do I mean by ecosystem?

A

So it's a metaphor taken from biology introduced to explain business, ecosystems or software ecosystems, as I'm talking about so I I thought: I'd use uh the android ecosystem as a referral here uh when explaining so a software ecosystem is a set of actors.

A

um That's functioning working as a unit and they're interacting uh through a shared market of software and services and their relationships among them, and then this these ecosystems are underpinned by some kind of technological platform. So this platform could be one or a series of technologies, products or services.

A

That brings everyone together solving a common problem uniting everyone under a common vision and in the android case, this technology platform would be the android or operating system and software development kits that allows developers to to develop apps that they then can sell on the common market the play store and this ecosystem. It's governed by google, which we can refer to as the platform leader who basically steers the the roadmap and the development of the platform, and also sets the rules for the market and so on.

A

So in our case, uh this ecosystem is called jobtechdev. So jobtech is the space for all of the actors trying to use uh digitalization and digital services to improve the way you're working.

A

So, as I said in the beginning, the the the vision here by the employment service is to create this platform to enable third parties to create better services and products, hence getting more a better matchmaking digital matchmaking on the swedish labor market, uh quick review of the of the platform, it's basically free, open government data pools and and a bigger open source project. So the first one taxonomy it's it's a taxonomy of common wording and labels for skills, job titles and relationships among them, making sure everyone speaks the same language.

A

It's a job, a pool of job ads, so they're, basically for scraping and for partnerships, they're um aggregating, all of the job ads on the swedish labor market and then making them available to all of the ecosystem, companies or organizations, and then job take my data. That's basically your cv data, that's built! uh It's a solution, build under my data principles. So basically it's you who own the data so or the cv data.

A

So you can log into our website, say, update your cv and then say that okay, this should be reflected on linkedin and this site on this site. But then, but then you can revoke the data from linkedin, for example, and they can no longer read your data. So it's a way of giving you ownership of the data and then on top there's a search engine solution built on elasticsearch.

A

So all of the companies don't have to build their own search engine which a lot of a lot of them have done.

A

So currently there hasn't been that much most of it. Most of this platform has been developed by the ages and themselves.

A

There has been some contributions both in terms of of the data sets enriched and new ones, and software, as well as in widgets example, applications and so on, but I hope it's too to get this growing, this ecosystem. Looking at the actors, it's it's a lot of actors from a lot of different niches or areas. So it's not just those working with recruitment and matchmakings, also insurance companies, uh some cooperating with career guides or education, guidance, local, regional and national government trade unions.

A

So there's a lot of different interests, a lot of different use cases so so now not all of them are interested in all all parts of the platform.

A

So, let's move on to the main topic so, along with the concept of ecosystem, comes the concept of ecosystem health. So we can use it as a as a key performance indicator, basically for the ecosystem's ability to to provide sustainable growth over a longer period of time, but also to to create opportunities for the different actors within the ecosystem and also building on on the ecosystem, health, literature from from biology and so on.

A

It can be further characterized through the three dimensions of productivity, robustness and niche creation, so with productivity, I'm referring to the ecosystem's ability to create new new and improved functionality based on the the input of material.

A

So in our case that would be the ability of matchmaking companies or insurance companies or governments being able to create new and improved services or, in other ways improve the operations based on the data.

A

For example, that's provided looking at robustness looking at the ecosystem's ability to withstand or survive shocks and disruptions, so in in our case, that could be one or two of the of the key actors within the ecosystem, with a lot of dependencies on them, dropping out or drawing bad pr to the ecosystem, or maybe the public employment service reorganizing, because it's and this is high risk, because the public employment service is under a lot of heat from both from media and politicians, so they're being ping-pong back and forth about how they should look like and how they should operate.

A

So that's a big threat here and then these creations, which refers to the ecosystem's ability to attract a diversity of actors, focused on different niches in terms of their value creation. So that would would, in this case, for example, be the matchmaking companies, those working with recruitment, insurance companies and so on.

A

So as we've seen all through the day, you should always start with a goal, so the main goal of the employment service. They want to reduce the employment rate.

A

They want more job seekers and employees to meet each other and match so and and through this, they want to improve their digital, matchmaking and guidance by enabling actors within the ecosystem to with the optic space, to improve their services, and here comes the concept of the ecosystem looking at the ecosystem, so they want to improve the ecosystem health, hence improving that its productivity, robustness and niche creation.

A

So I'll return to this uh later on. So what we did through interviews, uh we tried to elicit different kinds of metrics within these three categories and so they've we collected them and uh and they've undergone internal scrutiny. So the idea is to have some kind of baseline for how the ecosystem develops, how its activity develops and so on and looking at productivity we're looking at metrics that can exemplify some kind of growth from from a period in time.

A

So this could, for example, be visitors to maybe not the main website, but to more technical parts like the documentation pages for the apis, for example, or the number of checked out api keys and api's keys that are being used.

A

We we are also looking at the activity on ticket support and community communication channels. Discourse and slack are, are the ones used mainly and also physical meetings, because they do a lot of one-on-one meetings and then contributions here we do differentiate between three kinds of contributions, so knowledge contributions which could be answers to questions reports.

A

Data contribution newer, enriched data sets, which there has been some of and then the more technical ones, which can be more characterized on the open source space like code commits test cases, technical documentation or design inputs and prioritization inputs and so on, and then metrics, where we look at how new applications get of the data comes up which doesn't have to be aligned with the number of actors or users of the ecosystem or the platform looking at the robustness.

A

So here we're more looking at the size of the ecosystem, so a bigger ecosystem, more actors, hopefully means that it will be more stable and more can withstand these shocks and disruptions in a better in a better way. So here we're looking at the actors, for example, what size are they and how mature are they looking, for example, in at their turnover or years of years being around and how active they are in in the rest of the cases?

A

Most of the metrics are usually the same as in regards to productivity, but here we're looking at the absolute numbers. The total number of contributions, for example, of the data sets.

A

Looking at niche creation, we're looking at more at the diversity of the actors within the ecosystem, more of the of the the solutions here. So what kind of areas of operations? How many? How many actors do we have that are working with matchmaking? How many do we have that's working with recruitment? How many do you have that's uh insurance firms or national government agencies?

A

What kind of solutions or use cases do they have so it's kind of tracking how new ones get added and how many ones are are active in the in the different niches here.

A

So lessons learned.

A

A lot of these metrics requires a lot of upfront investment. We need to have a lot of good knowledge about the community, about the different actors about, for example, in what field they're in. Are they working with recruitment or are there an insurance company? What kind of technical applications are there? Are they doing as so? This requires a lot of research, so this has not been done from from start, but they're now they're doing this research trying to categorize and get more info and knowledge about uh about the actors. So.

A

And it's this thing you, the metrics, can only provide you with an input and you you can't focus on one metric and if you have a ton, tons of them you'll run drones. So you need some kind of balance here, getting the right amount and trying trying to use them to draw up a picture, and you should use that picture as an input to your qualitative knowledge about the ecosystem and about the different actors.

A

So that's why it's important here to have a good stakeholder analysis and knowledge about everyone in the ecosystem and also trying to to find the right abstraction level. So, right now, most of the metrics are focused on the ecosystem level. So covering uh actors or companies organizations on a general basis. Are they contributing at all to any of the different parts?

A

How are they interacting with the developers on a general basis on a network basis, but as this progresses, it may make more sense to also look at each individual project like the search engine for example, or as new software projects get added on each data pool to look at these sub-communities. You could call it so more on project level, but then a big challenge is knowing okay, so this ecosystem health concept.

A

So we can know that we have a healthy ecosystem where everyone is thriving and so on, but does that actually lead to improved digital services and actually having an effect on on the labor market? I mean these things are very important, especially in a political in a in a public organization where you have politicians that are very keen on growing money from you if you don't perform, and uh so we're we're trying to identify and investigate trying to create these impact. Metrics that can explain if these ecosystem metrics have has an impact.

A

So uh we're we're starting our collaboration with stockholm school of economics, where we're gonna look at the different actors, trying to identify performance objectives among the different actors and see and then trying to elicit kpis. So we can link the actual impact with these ecosystem methods. That's describing the more operational context, um yeah a way forward, I'll try to follow up and revise these ecosystem metrics. So this is a two year project, I'm helping them, I'm advising them.

A

I want them to to succeed, but we need to have some kind of baseline so that, while I'm documenting how I'm helping them or how they are improving or adapting their practices, we need to know where or in what direction. They're growing yeah thanks.

A

For have 60 seconds stress one question or two one question or two crystal clear:.

A

Cool any of the work being done that is going to be open sourced as well. So all of the what do you mean the kpis yeah, obviously yeah. Of course the hope is to have full.

A

Transparency, cool.

A