CHAOSS Metric Models Working Group, 29 Aug 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: CHAOSS Metrics Models Working Group Meeting August 29, 2023

Description

Meeting minutes are here: https://docs.google.com/document/d/1qE2duE73NbEOHzRRAbqgPSUSAfdH-AmvLruOu7uCIYg/edit#heading=h.ourjpk1nrito

Meeting summary is here: https://chaoss.discourse.group/t/metrics-model-working-group-summary-august-29-2023/255

A

Hi everybody morning evening afternoon, everybody I think this is truly our most Global meeting that we have in terms of like encompassing the globe. So um so it's nice to have everybody here minutes are in the chat. Here is the metrics model meeting, so I'll share my screen so.

B

A

um Today, um one if Sean can't join we'll just we'll kind of defer to a little bit later but um you're the first. Do you want to tell us how the compass lab released, went? I, think that was earlier this week is that right.

B

Actually it's the last week, it's also last week we officially released the compass lab and we make a live table uh on on on Thursday or obviously are quite quite remember that, but anyway, it's it's released. Last week.

B

Actually, the week before the last week, I have made a very short demo in our meeting and using the Prototype environment, and today, I I would like to quickly go through the whole live functions and to let you know how is it going on and and by the way, we also provide a Blog written for, introduce this Compass live written by by John Joe, our Compass Community manager.

C

I love, the uses of emojis is punctuation.

A

So was there something you wanted to share as well Yahoo did you want to share your screen, or maybe.

B

I can quickly go through yeah.

D

A

Thought I was looking at Yahoo screen, so no, this is my screen, so all right hold on just a second. Are you, you should be able to share your screen. Yeah.

B

I I can quickly share my screen um and.

D

A

We can see it.

C

B

And here's the live, and actually someone has made a metric uh a light before light. uh Also, my colleague made a demo for starter project chaos and that's the magic created by yourself, yup.

B

And we have several entry points you can use this create model or create a model now to create a new model, and you can. You can enter a model name here to select selecting into one of the dimension. Maybe here is a test kill test, and this is a general General model or it's a specific for one in Industrials or Fields.

B

You can choose that so the default one is a common model, but whatever because some some Metric model only available are suitable for for one specific uh technical Fields, uh you can, you can click here and and the choose public or non-public means.

B

If you would like to share the the creation of the uh the models progress with everyone to to share it under the slab, then everyone could could see the pro how it's going on and everyone could join this discussion or you want to create it by yourself and collaborate with your members uh thing, except for yourself and and your collaborators and the other ones could could not say the creation of the models.

B

And then um you can. You can pick up any of the models under you can pick up the the data site. You would like to create them the models on several data sites. You, if you would like to choose, for example, the big data we have several ones on the blockchain, whatever.

B

And confirm that, and it will tell you: okay, you you pick up the two uh technical Fields, why it's big one is for B Big Data binds for digital currency and which, um in each of single Fields it can includes two uh divides. I mean the two, uh the different repositories as you are this site, and then you can add your your matrix model and and we have to refile different uh categories.

B

Gate related, easy related peer pull request, related and reforceries, and contributory related from some of the the metrics, because if we have a very clear definitions and it's powered by chaos, we will add in light oh yeah and some nothing happened. I haven't been added to to chaos. Then we will not showing that, but we did not even on time so yeah so anyway, so uh I hope, I, hope. Finally, we've can add all the metrics marked with the powered by chaos.

B

Finally, so after you pick up the how the metrics and this slides, you can Define the weights and stretch hold for the each of single metric, so here the default of it for the each of single metric is a it's a just average. The weight of these things.

C

B

Them yeah yeah, but you can you can adjust that as you wish also, you can use some different other algorithm to to to to to calculate the weight as we wish and.

C

The threshold does the threshold imply a point of uh have like a like a safe range like a range you're happy within with for that metric. Basically,.

B

Basically, it's a it's a threshold coming from the like more than 20 to 30 solvent repositories. Okay, already we we have in compass and we observed value range between those repositories. We think we give a something like a reasonable values. Yeah most.

C

B

The the metric, but, of course, if you think the threshold or Maxwell values, is too high or too too too low. You can tell us you can we can. We can draw that, but anyway you can as before. You can use the default one, but you currently you can. You can cite as you.

C

So so those are, those would be literal thresholds like um they're literal numbers. It's not a scale um yeah yeah yeah.

B

Yeah yeah, after after the weight on struts hold the definition done and we pick up the algorithm selections, which means we use the all the metrics to calculate the final result and I I show the the algorithm we use it's a derived from the drop pack and he he before he's retired.

B

He working in at Google for more than 20 years and before he working in Google working in Bell libraries for many years, so they invent base algorithm and we also uh announce that it's an initially employed by LSS, gradientical, score and yeah, and we make some slide of improvement for this algorithm to make them more uh time range suitable. Because currently it's just a one shot point in time. Point time. Yes,.

C

B

So here we explained the whole algorithm and each of single parameters. Many uh yep us means Matrix values. T means threshold, Alpha means width. So this is not the default algorithm, but yeah. If you have any more ideas about algorithm you can. You can just tell us. What's your ideas, we can input the new ones for for everyone yeah after after we have understand the thermal views and we click the C1 yeah.

B

So in this one we'll show up here, uh chaos test and uh and this the it has a default version number which, which is zero there one and if you wanna click on one more version for that it's new versions, uh the default name will not change uh the. If it's a public or not public, we are not changed, but you can. You can give it's a specific new version number and select the different data size and different uh metric for this new version.

B

Anyway, and upon this new, um the new metric model, you can trigger the analysis after the triggers analysis. Yeah, the last time triggers unsubmitted yeah I can confirm that so it's started analyzing and after you you, after it's automatic some time anyway, so we can track the one we have completed the analysis. We can see the view report and you can see that we have already the several uh data sites for this starter project.

B

Here's matrix model you can, you can click up what anyone animal thing for, for example, I, click uh and and next this one yeah and and if you notice that we have discussion here and you want to discuss around this matrix model or you want to discuss one of this each of single metric. You can click here discuss and it will showing here you can. You can share your your ideas about the how to create this matrix model. Add your comments.

C

Yes, is the intention to be adding comments about this metric or metric model for the repository I'm, looking at or more generally, from a design perspective for.

B

The design perspective it's more like we may create a new matrix model definition or matrix definition on the Google Doc yeah. You can give comments, uh of course, yeah, excellent and and yeah here my models, uh if you notice here um members I just mentioned I'm the creator of this new matrix model and but I would like to uh collaborate with other guys, for example, from you, since I I would like to invite you to collaborate with me and and I will give you some access right.

B

The default one is just a blockable, no matter I, public or non-public. If I invite you as a collaborator, you accept that you you, can. You can see the this model under your your models this this here? Okay and you can also give you the accessory to trigger the the analysis. So whenever you add a new metric or whenever you change another new data site, you can change you. Can you can trigger this matrix model or you can modify so matrix model?

B

Which means you can either either one more or delete some Metric from this matrix model, or you can yeah. Do some other configurations, yeah.

C

As the underlying intention, then that the data sets exist and people can access them and I can create a metrics model in any data set.

B

Yeah exactly because we have more than 20 solvent data sites here, but so the the. How do you invite the correlator you just email, the multiple email address, if you would like, with commas separately,.

B

Yep yeah: that's that that's the generally, the uh the the quick demo for for all the uh for odd functions. As far as I can see and- and my colleague blanjo just gave us a Blog- we prepare to post it on the uh on on on Compass, of course, but I would like to um you know, make posts on our case our chaos Community.

B

Also. So, if you think, uh there's something you need to modify or update you can you can help us to change that if you like yeah and who can who can use this Compass lab, as I mentioned in the last meeting, everyone from our chaos Community are welcomed to use this life, because that's the my initial intention to help our members from chaos to quickly click a new matrix model and and create this collaboration process with everyone in the community yep that basic demo for for Campus Life can stop sharing.

B

C

Pretty amazing.

A

Yeah I, I, think I. Think I have a few comments, so every time I think it can get cooler. You do another presentation and I'm kind of blown away, so um yeah, it's pretty amazing. So, thanks for sharing that you Hui, um so uh the blog post I can connect with Elizabeth I. Don't I think you know posting that on KS isn't isn't an issue at all, so um so I'll take a look at that: um okay yeah, so it like raises a couple questions for me.

A

um So this is it's interesting because we have metric model definitions that we are publishing as part of chaos, and this is metric models that are just kind of emergent that people would like to to see. So is there any?

A

um Is there any thought about kind of trying to see the things that people put together regularly like it may not be worth articulating a single metric model that somebody just randomly puts together, but it might be interesting to know if 20 different people are kind of putting together the same metric model over and over. You know, there's there's overlap between, say like what Don put together and what I would put together and what Sean would put together.

B

A

As something people continue to ask,.

B

At this phase, I haven't made too much concern or or worry about these points, because I pray for more people to make more matrix model. You know coming from our chaos Community to let people realize that it's helpful for them. So later, maybe, like you mentioned that some kids happened. So many people made it similar Matrix models, then in chaos we should make decisions which one is better. We recommend to to to describe the similar use case that that's the thing our chaos should do that in Commerce life. We're just doing the engineering thing to.

A

B

People getting together.

A

Yeah for sure, but this is it's an interesting way to, because normally our metrics or metrics models are always produced kind of um not always but they're, often produced via these types of meetings by people bringing ideas together, and this would just be a different way to kind of identify what those those metric models could be. So that's super interesting.

B

um Yeah yeah, and also we are thinking that we do provide some some convenient way to help people to to create the new metrics model. But there are still some group of people like people from University from research centers.

B

They would like to use the data directly. So that's also what I'm thinking to provide through the rest API from from compass. So except for this matrix model and the Matrix we already have I mean the data behind this Matrix and metrics model. We also have many other enriched data, so we would like to prove what this rest API to all the people from chaos. So if you want doing more data, precise analysis- maybe you know for for auger to process some data to generalize the new visualizations that you perform.

B

We can prove one but just API list and help people to do so.

A

Yeah we have actually, there is a question too from Anita, but we have we've talked about that as well, about providing data um to academics in particular. Who may want to take a look at data because you're right A lot of them, don't just want to use the UI that you were showing, but they want access to the data, so they can manipulate it.

A

um We probably I, don't know how academics are in China, but at least in the US academics Can. It can be kind of a one-sided relationship where academics will simply take the data without much intention of contributing back and if there's a problem with the data. They'll just push the problem to you, so we might want to think just about what that relationship, but I agree. Providing the data is a cool step, but we might want to think about how we do that yeah.

B

uh You know you know we already provide some data to some universities in China, of course, without adding data priority concerned because of AIO the anonymous some technical thing to remove all the personal information yeah it provides such like that to to the university. They are ready. You know to publish some paper, uh some meetings so I think that's. Why he's working okay.

A

Okay, cool um Anita had asked: is there a possibility to add metrics? Subsequently, assuming a team wants a particular metric that is not already listed.

B

I mean I mean all the metrics first should have a definition in chaos. That's the preferred one and you can nominate that from like from Common working group. That I would like to say like through the chaos or through the auger whatever, and you can tell us, we can implement this Matrix quickly for you, it's because actually I.

B

As far as I can see, all the data I mean all kinds of different kind of data, for the I mean exists on the GitHub and and any other code, many management platform like Getty and all data we already have. So you just tell us if this metric is unrelated to some data on GitHub. We can quickly provide this one yeah.

A

Does that help Anita.

D

Yes, that helps thank you.

A

You bet um I'm curious, like done.

A

Zoo I'm just kind of your impressions of what you're, seeing today.

D

uh Kind of echo what you think every time I see it I think it's really pretty pretty amazing. So I'm really I'm really happy to see some of this customization and Encompass now, yeah I think just think. It's really good.

A

Right on um right, usually.

D

A

Was the what was the reaction um when you were doing the introduction uh last week? Do you have a sense of kind of what people's reception is when you were presenting it? I assume you were in China when you were presenting it, that that was where the audience was and actually.

B

We made a live demo and everybody is watching it through um uh uh some different channels. In China, I can see some feedback that I would like to. You know take a try on some specific technical ideas. That's forced to call me bike, and also they also mentioned that if, if I can size the data directly through the API okay.

A

B

Yeah that make me more flexibility, create a new metrics model, yeah, okay,.

A

Did you get a sense of companies that were interested in in this interface.

B

Some companies already asked us to provide such uh capabilities of compass Vibe for, for the I mean as a as a law of also, and also there are some companies to do some software security tools, so they would like to add such matrix model. As a part of this, a message report I mean the security analysis report to as a complimentary information yeah.

A

Just so like a security report that would be supplemented by the data come as pumping out of compass: okay, yeah, okay, um great uh okay, so uh this is, of course super interesting and um but like I personally, thanks for kind of you know, chaos, metrics and metrics models forward. uh I think.

D

That's just great from a visibility perspective.

A

As well, okay, cool anything else for you.

B

A

Okay, Sean I, don't know.

C

If you had no I, don't know, but yeah I mean I would like to play with it for sure on several levels.

B

If you want access to wrestling bi, just to tell me I can I can give you directly.

B

You don't have to. You know, enter that pattern to access the rest API. We.

C

B

C

Yes, yeah I'm, not we're not really front-end people over here, but that's awesome.

A

All right cool uh all right, well we'll kind of go back here. um The next thing on the agenda was the uh OSS EU chaos meeting. Do we have? Maybe this is right back to you yui, because I think that yeah.

B

I, just love on the track with you guys, especially from either space uh yeah. Do we have any schedule for that because they could I I'm pretty sure they can provide the two hours Let's sessions, meeting room uh on September 21st, and so how many people from our chaos could draw and what kind of topics we can we can discuss to how such such gender? Already no.

E

um Brought it up in a community meeting and I think we have maybe three three folks from chaos: that'll, be there, so just kind of threw it out there that we have the space available and.

B

E

Were a few ideas, but nothing confirmed so it's still whatever we want to do and.

D

I think there there are more people, probably from the gas community that might come that aren't in the weekly meeting. So I was I got a stock message from uh Daniel luciardo I said they thought he would come.

B

D

So I think maybe if we look down through, like the speakers, I think that we'll see some other names of sort of chaos. Books, like my Prime profit, for example, has a talk there.

B

Do we have some onion onion uh topics already you know decide to to to to to talk or discuss, I think that that will be great uh because I I I have um I need to feedback, give some feedback to to my company to say. Okay, we have uh some people to join this session and several topics we we plan to do, but but I mean the details of topics is, is whatever we want to do. It's totally open.

E

Don I didn't know if you wanted to take that time to um talk about how people are using chaos, metrics and kind of your survey in person.

E

um I, don't know if that's the kind of crowd that would come to a session like that or if that's what you want to use some of the time for, but it might be available to you.

D

Yeah I'm not sure I, think um Daniel was talking about maybe having sort of a workshop on open source health as part of that room but I'm. He said he kind of I think he talked to you about it or got a message from you. So.

B

I think we did.

D

We could do something like that, for example, I'm, not sure how many let me we might want to recruit people to attend.

E

Will this go on the uh osscu schedule like publish schedule on their website, or is this just kind of blocked off for us on the side.

B

uh They do publish their scalios for the three days sessions.

E

So we would also have to let osseu folks know what we're talking about, so they can put that on their schedule.

B

I I think it's not needed actually, okay, because this room is available for for the uh government sponsors like open ruler, so they can decide who would they use this meeting rule so as that means mentioned, so we have two hours sessions available for for chaos. We can use to discuss and revise anyone from from chaos or from outside of chaos, whatever they are. Welcome.

D

But I think if it's not on the schedule, if it's just a meeting room, people might not know about what we're planning to do there. Foreign.

A

Do we know when it is.

B

Up to the end of 21st, okay,.

A

B

2 P.M to 4 P.M uh to 4 P.M yep.

A

Okay and it's the 21st a day of the September. No but like, is it a day of the conference or is it a day before I? Don't remember like.

B

It's at the last day of the conference. Okay,.

D

It's a pretty light day. I was looking at the uh schedule for osscu and there's not a ton of stuff on Thursday afternoon.

A

Okay, so the the event is still going on on the 21st. Okay.

A

um So I mean given two hours we do have a lot of. We do have a lot of success with respect to kind of asking people to work together and report back. We've done that quite a bit, so um I think what has worked well in the past is we have somebody give just kind of a like a 15 minute talk or a 10 minute talk or a five minute talk like just to kind of here's the topic.

A

Let me talk a little bit about it, just to kind of help, locate people um and then ask people to kind of speak about whatever that issue is in their own particular context and report back.

A

um We usually do get really good feedback in terms of things that we might want to be paying attention to in the chaos project. So that would be really helpful, I think to me and to Elizabeth um one of the things we could do uh with Don being there is something around you know: data science. What are the really the complex questions that you're trying to answer? Not that that would necessarily be everything that we would do with data science, but it might give us some orientation on how we want to think about it.

A

um Just one thought another would be like software accessibility like what do we need from a software perspective, so this is also kind of lined up a little bit with your survey done not that you would give that talk but like what are the needs that people have around software and what are the challenges that they're facing at the moment?

A

um That could be another kind of prompt that could could help people. So those are just a few ideas that I have, but the workshop approach does does work pretty well, at least for me,.

D

Yeah I mean I I have to admit since I'm since I'm out starting on Friday and I'm, not all next week and then I'm back on the 12th, um because I'll be on holiday. I, don't like I, don't know that I have time to prepare to help facilitate it. Maybe we need to make sure that we have someone who's ready.

B

D

Willing to facilitate that degree, just somebody who can spend some more time on it over the next couple of weeks. Sure.

A

D

Don't know is Georg gonna, be there.

A

I don't know I'll I can ping him and ask because he's usually pretty amenable to Leading sessions like that.

D

Yeah he's pretty good at it. Yeah.

A

E

Or or if you said, Daniel already had some ideas about doing just open source Health.

C

um What do you think Daniel.

E

Would be willing to.

C

Daniel certainly has a lot in his wheelhouse. He could bring so e.

A

E

You all know if he ever is, he I know he's in our slack. But do you know if he actively checks it because I can message him too, but I.

D

E

I didn't know if he ever had time to even check Slack.

D

He does the challenges that he's also on vacation right now, I think he's coming back in a few days, maybe or next week.

B

C

Far as I know, he comes back if you're going out.

D

C

um We could also.

A

Ask you know you know more um specific questions. These are all pretty broad if we.

B

A

Can find a facilitator and kind of put together an agenda and I'm happy to work on that, um but they could be more specific, like.

A

A

You know like how do you support a global, open source Community like what are the things that you think about as.

D

A

Example, like it's a little bit more specific than just kind of software challenges or community health challenges, um or um let's see what else can go.

D

And do we think that we'll also have a bunch of people from open open your who aren't necessarily involved in the chaos project at this meeting as well.

B

Yeah I think some people would would like to show some best practice getting from the open Europe about their Community I mean the health evaluation things like they. Actually, they have almost 100 sick teams.

B

I mean the special interest in groups in this in these communities, so they they are using our chaos communities to to modify all the sick status for for them, and they have a global dashboard to show up all the contributions from the different organizations and and and and individual contributions, and you concluded, found who made contribution in the last six months or one year. I think they would like to share that one sent from the you know a community by best practice.

A

That would be great one of the do. You think there would be an opportunity to speak to um there's a couple questions that I hear consistently, which is like how do you?

A

um How do you effectively manage, and maybe it manages a good word but um like effectively managed dashboards, so I hear a lot about people saying that they just have information overload from dashboards so like? How do you as a community like think through that, so that the dashboards are meaningful to you without overwhelming you I'd really love to hear about that, and then assuming you can get past. That issue like what do you do with the data that you understand from the dashboard.

B

Like, where does that go.

A

Like does it go into that person simply taking actions? Does it go into say like what you're talking like a security report that gets pushed up somewhere else in an organization? I would love to know a little bit about the directionality of where that information goes, and anybody who could speak to that would be really that'd. Be really interesting.

C

A

C

I mean yeah I'd be interested in that too I mean I'm, I, yeah I, just I'm very interested in all of it. It's like too much at the buffet.

A

But I'm thinking, if folks from the open, Euler Community, could talk about the global dashboards that they do have yeah and speak a little, because if they are, it is their their room, and it's very nice of them to to invite us to be part of that conversation that we could talk about, maybe the dash or they could talk about- maybe the dashboard. This could be the prompt that they could talk about the dashboards that are available to them or the dashboards that they use.

A

um They could speak to kind of the design of these dashboards and how they are important for them without being too much information, but being just the right amount of information. Kind of that balance that they're able to strike um whoops.

A

Amount of information and then, where how do they, you know how do they use.

A

That dashboard data.

A

Like where does it go next other than just you know in in your brain like? Where do you take that data to to make data-driven decisions and how do those decisions get made.

A

um Because I think that would be an interesting prompt because I think there are there there would be potential and we could then, um as we say, like kind of announce this or you know, recruit people to attend, so I think there would be a number of people in the audience who could speak to this as well yeah for sure yeah. So, like Sean, you know like the folks.

C

That you want to be there. Yeah no I know the people.

A

There, like Brian I, know what red hat is probably dealt with a lot of these issues as well.

C

A

An example: oh.

C

Yeah, it's a good one. Yeah.

A

And so it might be an interesting conversation because open Euler folks from Oakman Mueller could kind of present this originally and just talk about their own particular context. But I also think it would be really like, probably very clear points of connection for people from the audience to also speak to these particular issues as well. Yeah.

A

So what about what about that? If we did kind of this one as the.

A

As the dash or dashboard- oh, my goodness, as.

D

A

Agenda all right thumbs up. How does that work for you.

B

I mean except the open ruler. Community. Do we have any other communities to make to share their experiences about the community health evaluation.

A

Well, I'm guessing the people in the audience will all be representatives of a whole variety of companies and communities who will be able to speak. You typically in these, like I, think I've talked to Don about this, like this is a very chatty group of people generally at OSS, EU or ossna they're, usually very happy to to talk and contribute, and um so my guess is. Is that that if there was a presentation from the open, Euler Community for maybe 20 minutes,.

D

Or something like.

A

That and then just kind of an open discussion following that it would probably go a pretty long way. We would need somebody like, like Georg, to kind of um facilitate yeah facilitate that conversation so that it wasn't just you know we like, following the open Euler presentation, we could have small breakout sessions where people kind of thought about these in their own local context and reported back.

A

um The other option is we try to you know we ask Brian if he would like to kind of speak to this as well from a red hat perspective and he's also usually very amenable to to speak about things as well.

A

um So you know we could. We could propose to have open Euler speak to this, for you know, 15 minutes and then have Brian speak to this from his context for about 15 minutes and then um and then have people kind of break out into sessions and then kind of report back, that's usually pretty successful, but again we'd need a facilitator and I. Think it's okay too.

A

If we have the room for two hours that we don't like go right to the end of the two hours, sometimes having kind of just people mingle at the end of that two hours is pretty nice as well, just to kind of have open discussions as people are walking around.

B

You will need to make a posters for this for this chaos meeting, because I I can ask people my friends to create a new one, I print it from China to to EU yeah.

A

I mean if we have like even just a like a digital poster or a real post. Real poster would be good too, but a digital one, because then we could. If it was around this um and again, we can ask I, can ask Brian, um but if it's around this or somebody um it would be a little bit easier for us to put it out on our channels as well that there's a an open event that you're invited to.

B

A

B

To talk about these.

A

Things, foreign.

D

Yeah all right.

A

Does that help you.

B

A

B

That would be great okay. Thank you. Yeah.

A

um So maybe there might be a few there's an action item to create a poster.

A

Wire for this baseball flyer line, how do you spell flyer? Is it f-l-I-e-r? That's.

C

How I was going to spell it so we're both wrong? If that's wrong, it's not flyer. Maybe there is no e.

E

I would have put a Y2 I think just yeah, don't listen to Sean yeah.

A

It was really weird um or the event um AI, maybe reach.

D

A

Another person.

A

A

You know yahuwi from who from open Euler would maybe do that presentation.

B

And like I, can't find someone okay, okay,.

A

They could do like a 15 minute presentation, it's just to give the ideas of what we want to talk about. It would be about this.

A

A

D

We will work on.

A

That all right, we have three minutes to go through all of these things, so.

A

No, this is great I appreciate this conversation because actually osseu is coming up relatively.

C

Very quickly, yeah, it comes on fast.

A

um All right, well I, maybe the one thing that we'll say is uh business. Readiness of an open source project has been released, so you can click on that and it is up on the website.

A

um I was gonna, put some some more some comments into this Persona model because I'd like to get that out. Oh.

B

That would be great. Thank you. Yep.

A

um I, just these are my only comment was as I was going through. This like these are metric models. These three, the domain profile model role, profile model, they're, they're,.

B

D

A

Than our other metric models, just in terms of um just like how I think about them and so I, here's a little bit different, that's all.

B

Yeah, it's a it's a let's be the difference from other. You know the definitions now I'm thinking that, because I'm thinking and they we may need some base model like this contributor personal model, so depend on this model. We can create more other things, so I don't know how to call that. Maybe we can call it the metric on on a Model. Yes,.

A

Exactly okay, so you under you're we're in the same spot like the naming of it is it just seems a little bit different than what we had been referring to as a metric model, particularly.

B

A

The things you were showing today like they're a little bit different than that so I'll Think through that as well. uh So- and we can talk about that next time uh and then next time we're still here- uh okay, uh well we're at the end of time. So thank you that.

C

Was fast, it goes faster.

A

C

Cool stuff, yeah.

A

C

Show me new toys every time, no.

A

Yeah all right: well, it's good to see everybody and thanks for the feedback and I think maybe what we'll be working on like kind of immediately.

D

A

Immediately is this with OSS EU all right thanks.

D

Everybody it's.

B

Good to see you.

C

Thank you, bye.

D

C