Kubernetes Usability Special Interest Group, 1 Oct 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 20191001 SIG Usability Meeting

Description

[Tasha] Kubecon Update / Planning
Register for the contributor summit! https://events.linuxfoundation.org/events/kubernetes-contributor-summit-north-america-2019/ #contributor-summit for conversation
https://github.com/kubernetes/sig-release/tree/master/releases/release-1.17
[Rael] Slides for the KubeCon SIG Talks
https://github.com/kubernetes-sigs/sig-usability/issues/3
[alejandrox1] Quick hello from the release team
[Gaby] Update on usability study questions
What would the community like to learn?
What questions would we propose asking to different personas?
[Pamel/Boaz] Present on personas used at VMware to get feedback on creating upstream personas
Developer on Kubernetes
Operations of Kubernetes
Developer of Kubernetes
[Spencer Sugarman] Go over the Dashboard UI survey https://github.com/kubernetes/dashboard/issues/3256
Open mic

A

Okay, hi everybody. This is our regular cig usability meeting and we have a pretty packed agenda today. So I'm just gonna pull it up and you can, let's take a look at it together, real fast, so I'm gonna do a quick, coupe con update and let you all know what we're planning for coupe con. If anybody wants to come to the contributor summit, that is a really awesome event.

A

If you haven't been before it's the Monday before coupe call, and there is a contributor track, so current contributor track also, if you've never done it before, there's a new contributor workshop, there's both a 101 and a 201 that you can sign up for, there's currently a waiting list and then I also have a list of people that for who are underrepresented in kubernetes.

A

So if anyone wants to do 101 201, you can either join the regular waiting list or ping me if you'd like to request a prioritized spot as an underrepresented person in the kubernetes community. So that's all available and then we just have the regular current contributor track as well as part of this. We are going to have the sig leads staff a table and talk to people about what we're doing as part of sig usability. I.

A

Don't think we have a talk at contributor summits specifically to us, but there will be an open forum that people can suggest new topics to. So if anybody wants to go there and suggest a usability topic, they'll be sort of an attract just for for things that everybody agreed like yeah. Let's talk about this today, so that's ready to go file.

B

It under the sig but I did submit I talked to the contributor summit for better defaulting for kubernetes security.

C

And community settings so.

B

That's something that goes pretty adjacent, I, think yeah.

A

Yeah I think I saw that one it got accepted right. Yes, it did yeah cool I. Think the final schedule for all of those talks is going to get published on the contributor site pretty soon, and so everyone can check that out. Let's see right Rio, you were going to talk about slides for the cig talks, but basically both the Valerie and I submitted talks to the regular coupe con schedule.

A

So every cig and working group has the opportunity to do both an intro talk at a deep dive, talk and so I'll be doing an intro talk at coupe. Con and Valerie will be doing a deep dive talk for the usability sig at coupe cond. The goal of the intro talk is mostly to raise awareness of our sig. Let people know what sort of work we're doing where they can learn more, how they can get involved. So it's just I, don't know how many of you have been to coupe cons.

A

I would guess most of the people on this call, but usually there are a lot of people at coupe. Con who've never interacted with the open-source community before so. It's a good opportunity to kind of just explain like this is what a sick is. This is how you can get involved. This is some work, we're doing that you might be interested in and then Valerie do you want to say I can give a couple words about what you were thinking for the deep dive.

B

For the deep dive I thought would be interesting to connect more with Gabby to get a bit of information about any survey work as well as focus in on defaulting, so take some of the stuff. That's right now, I'm talking with sig networker on the logistics of like if we make network policies here by default versus the other way around so kind of focus in on that some of the stuff that you've raised with the multi-tenancy working group around like how do we deal with the conformist logistics and so on.

A

Awesome that'll be really cool yeah. Let me know if you want to talk any more about the profile work that we're doing in the multi-tenancy group, yeah cool, so that's kind of where we are right now rail did you want to talk about this thing? You put up four slides for the coop contacts.

A

D

Yeah, it was like having a place to do a bit of brainstorming, so everyone can like add information that they think we should like present on the talks, because it I think it's it's. A new sig is a first talk, so all the people will be like interested in coming and knowing more than for for this music, so try to cover everything that we want to work on the next year, so new fault can join into the sick.

D

My dear, for example, we working with Rossi about documentation and how people that it's new to cover notice what kind of challenges they face they are able to like go forward, keep them using the official documentation or if they need to go outside on this kind of thing, will be a good thing also to cover during this or the intercession or the deep dive, but just try to help with with the slides, because they sort of work for providing our thoughts.

D

So if you need some have, we can like work together on the slides and just I didn't know, don't work on the slide and I do too to get the final deck together.

A

Okay, yeah I think my approach to doing the intro talk is going to be just putting together a simple slide deck using the. So they publish a slide template that you can use that for each cube, cons so I'm, just gonna use that slide. Template yeah.

D

In fact, in the issue is already created, this deck, with the official templates and I put like just the templates and the titles for both tags in case you want to use that particular that, or so in. This is already linked to the world of slides and the group of securities are at a theater, so I don't know if you can already edit them.

A

Cool yeah yeah thanks for kicking that off that's cool yep, looks good.

D

And you just other bleep permission that is already in the website and some of my ideas to add to the so we can like start working together. Instead of doing all the work you yeah.

A

Yeah I think so, if anybody has ideas for things that they want to have presented, definitely just throw them either on this issue, or you know, obviously, Valerie and I will need to do our slides so that we know what we're gonna say, but I think that's a good idea, just kind of if anyone wants to brainstorm and think about things like just kind of putting them into the buckets of intro is really just kind of high level of what might people be interested in like kind of sketching out both how to engage?

A

Because a lot of this is like for new contributors and then also projects they might be interested in and where we're going and then the deep dive Valerie Valerie is going to kind of pick what she wants to do more of a deep dive in and then potentially work with Gaby to have some of Gaby's work highlighted as well. Yeah.

B

I wanted to focus on the defaulting thing, though, because that's a fairly complex topic in terms of carbonates, mechanics and I think that's a good topic to engage like just think your tribute rissalah's yeah.

A

Yeah and the cool thing too, is picking one thing to kind of do a deep dive on this time and then next cube con. We can pick something else, you know and also like these get. You know obviously videoed and then we can share them with other people and it can provide like a helpful starting point for a longer deeper conversation.

D

A

So, let's see Alejandro I see that you were gonna. Do a quick hello from the release team, yeah.

E

So, first off hi everyone, my name is for my name- is Jorge Gurgel I'm, one of the shadows for the release, a further release, a further release Timothy- and this is actually our first official week in the 117, the ones left in Belize cycle. This cycle is going to be a little bit shorter than YouTube because of the vast amount of holidays conferences and also you know we just wanted to go around with introducing ourselves with all the six giving a heads up. They enhance enhancements.

E

Freeze deadlines is it's gonna be in about two weeks total for things. You know, if any of you how many caps it makes sure to get together to get a PR and create an issue in a and Smith's repo him other than that yeah just we. We want to make ourselves available. If any of you have any questions related to the products or anything really.

A

Awesome. Thank you so much.

A

Cool Gaby, would you like to talk about your usability study I'll, stop sharing. If you want to share stuff, yeah sure.

C

I can pull up the question stuck.

C

Okay, um so just for a little bit of background last week, we talked about for the survey initiative, kicking off a duck just to gather questions that we might have as a community that we would want to either just ask at a high level. So under general questions and then also potential survey questions so I'll go through a few of them. Thanks again to those that contributed.

C

So in general questions we got what is the journey of new users that they go through to learn what they need to do to get the tasks done, that they need to get done, adoption funnel for career Nettie's and where are we losing users and then who are our kubernetes personas and then in survey questions we got some about what you're trying to achieve with kubernetes expertise.

C

Have you deployed to production, recruiting questions you be willing to speak to as a researcher involvement in open source as well, and then we got some category level questions about just what was your experience, learning Corinna DS using the documentation as well? Did you have a background in containers and what was it like? Jumping into the ecosystem.

C

Is the current IDs documentation site? How is it to use issues with using the site? Then we had the language category.

C

You know if you're consuming this information in English are there other languages that that would benefit from seeing documentation or even like messaging, within to the command line, experience in other languages, which is pretty cool and then the last category that we had was community questions, so you know, are they involved in local communities or online communities just trying to like measure that level of involvement so yeah? So that is um that's the overview. Does anybody have any questions about that or any comments kind.

F

Of question Gabby I think you mentioned an email that you reviewed some existing CNCs surveys, yeah.

C

F

You find answers to some of these I'm curious. What you've gone from the existing research.

C

Yeah um so I think that in the CN CF survey you know we find it. We find answers to maybe like something like the questions of how to use communities before having made open source contributions, and we see a few kind of like profiling questions about like experience with containers and stuff, so so I definitely think that we can use some of those surveys to just bring yeah like kind of go about answering those questions.

A

Yeah I thought this was a good start, some of them I kind of wanted to like tease apart a little bit because, like and I did, I did totally do this a little bit but, like some of these questions, I think we're too layered. So, for example, trying to figure out like. Would you like error messages in your own language and then trying to say? But what? If, if like?

A

Basically that meant that the error messages sometimes are out of date or incorrect because they're not as many contributors like I kind of felt like that needed to be teased apart a little bit more to say, like I, think everyone would prefer an error message in their own language, like I, almost feel like that's a strange question to ask, but then, when you kind of get into it like how worried are you that if an error message is out of date or like incorrect, because it's in your preferred language I feel like really what you want to dial into there is.

A

Can you understand an error message in English right like and if they can't read it, then it's not useful. If they can read it and it's more likely to be accurate, like that's kind of where, like the balancing act, kind of comes in and then I was also wondering just as a group like what of this, could we potentially automate so like if we're automating be like? If we know what all our error messages are and then what languages they need to be translated into like how much of that automation can we say?

A

Oh this changed. This is everything that needs to be updated and then what's the mechanism for getting that prioritized on whoever's, doing the translation work so.

C

Yet another great comment.

A

For sure I guess the question is well, you added those were you did so what did you have in mind like? What did you want to get out of asking these questions a.

D

Problem that is happening right now with the documentation is that we have a small teams working on localization, and sometimes they cannot keep the paste of the English team because they are like 10 versus 100 people. So there is drifting between the your own language documentation. So how can we tackle this kind of problem of consuming out of the domination? Should we have a link or a mechanisms to be able to detect when something changes from the original version?

D

Okay, we can this as like a warning or some automation to be able to spot when a page is not up-to-date. The same thing, as you said with there, are messages. How big can add this kind of feedback to the consumers that say, okay, this page is in your language, but maybe be out of date. You can also check this out.

D

The kind of rest resolve in English, for example, being able of putting some tuition to take this kind of drift, because it's important and having the feedback of if a user can support this kind of written and, like I, start consuming out of the documentation and then move back to the English documentation, but with more context because he has like has been able to start with his own language and then move to the Inglot orientation.

D

So it's easier, like you say, starting with the English documentation, that's from the good augmentation side and about could cut all these already. For example, some effort, Catholicism Italian, anything in Japanese German and a couple languages, but the process is a bit out of date. This seems deprecated the last language edition anything was a year ago. So the pulling is he is there? The mechanism to implement localization is there, but for some reason, is not being used at all right now, so also make a use as to the culture uses.

D

If is useful, to invest some time in time to recover this effort of localization for Catalan trying to make like a process or some kind of automation to to easier to work on localization of dueling? Yes,.

A

So I would suggest that this isn't something we actually need to do research. It sounds more like we need to describe what the user experiences that we want to have right, and so it's pretty obvious that having out-of-date error messages out-of-date documentation, that's inaccurate is a usability problem. So I, don't think you need to ask a user whether or not they liked it or whether or not that's good.

A

What we need to do is describe what a better flow would be, especially if we're relying on contributors sort of in their free time to keep it up-to-date in their language in their preferred language. So it sounds like what we need is. We need some way to say. Okay, we had all of this translation work done. Then we cut a new release. This is now out of date.

A

Are there pieces that we know are wrong or we're like somehow tied to a feature that doesn't exist or like did we introduce new features that now don't have documentation? How do we highlight those? So we know what's out-of-date versus what's okay and then, if a user is using something and we don't have documentation in their language of choice, for that version, do we basically show them a message that says hey?

A

We don't have this translated yet in your language of choice, here's the English information that they could maybe copy and paste into Google Translate if they can't read English. You know like I feel like this is more like a journey map like thinking about like how could we improve this rather than asking if it needs to be improved, because what from what you're describing we already know, it needs to be improved.

D

Ok, that's ok, because actually that is like some tooling in documentation of steam to spot this kind of briefs. So maybe we can I can go back until we've with holes from six dogs and try to gather all the information that the the tooling that we already have and start working on that to make like official documentation of ways to to try to fix this and improve these and add some because, for example, the Korean thing I think is working with releases at the same time with inner steam.

D

So they are rather some effort to try to keep this kind of recycle with documentation. So I think we only need to get everything together and tie to to find the best way to all the teams can go the same way. Okay,.

A

A

Cool so Gabi. What do you think next steps now that we've started brainstorming over here.

C

So I think we got a lot of really questions that we can probably answer with a little bit of time. um I did notice. I did notice that some of the higher-level questions it's like what are you trying to achieve with kubernetes and then kind of like these, like high-level ones here, um I the reason that I sent out the email earlier or last night I guess was because it's for, like four jobs to be done, approach, I, think kind of tries to tackle those questions and I'm.

C

However, you're like what people's familiarity with jobs to be known was in the community and that's kind of like. Why isn't done that? You know I love. Looking.

A

Into jobs to be done, I think that that might really help I think. Maybe personas are too advanced right and like if we focus on jobs to be done instead, like that, can turn into a persona project. Yeah.

C

um So I think what I like about jobs to Eden is that usually it's kind of like a hybrid qualitative and quantitative approach. So you kind of get the minute you're not talking to like. Maybe like I, don't know. 15 is a lot for qualitative, but even that compared to quantitative is very little. But I think that we could try and do something like that, and it would be a little bit of a bigger, a bigger project. But that's why I was kind of like testing out the waters now I'm totally down and.

A

Yeah would anybody else on the call like to help out on a jobs to be done, project I.

F

Might be able to contribute something when we talked about the personas in the next agenda item, it includes some potential seed seeds for a jobs to be done questionnaire and- and so, if it's of interest, I didn't get a chance to copy and paste it into this document. Yet I wanted to like see if people are interested in it and then yeah I'm happy to copy that into here and go from there but I think it's a good idea.

C

Yeah, that would be awesome.

C

um So I don't know kind of like what um so that was obviously you know like one-one next step that I saw I know that there's a lot of great questions in here. There are a little bit more specific and I know that jobs to be done will certainly is kind of like a targeting trying to target the higher-level questions. So if anybody has ideas for four things that we can do with the other questions, definitely open to them, but I think that that's kind of like what I have right now.

C

And then just a question about the people on the call it has anybody ever gone through like a full like tweet done, press it's because I've read it kind of like all the literature on it, but um and her done like bits and pieces of it, sir, not like a full one same for me. Cool so would be fun to get to do it then.

C

Okay, well, I think that that's it for that's it. For me,.

C

Isn't you know, questions and then just kind of flesh out I think our approach to be done? Would it be what would be that that's way of blowing about collaborating on that.

A

So it kind of depends uh if you want to kick off a dock like we can collaborate on that, we can hop on. You know impromptu meetings, to kind of push it forward and then present it when it's in a more final state at the next Sigma ting, or you know two sig meetings or however much time we need so and then I think we should chat about it in the channel, the slack channel. So people can kind of offer to jump in if they're, interested, mm-hmm.

C

Yeah cuz I think maybe something like um like Paul's just like here and there in between the sig meetings, would be helpful, just especially for like for scoping and for hashing out kind of um like a little bit of like the of the details. Yeah.

A

I'm and I'm totally down to do that and kind of help push the ball forward faster than our every three weeks. Meeting cadence would allow sounds good.

C

Okay, cool, so I will follow up with I'll, follow up on the side channel about next steps. For that then okay sounds.

A

Good okay, Boaz and Pamela you're next up, you're gonna present on personas used at VMware to talk about potentially creating upstream personas.

F

All right so I'm go as I'm research around the central design team, mint VMware and last year. I did some work to align our personas across different business units. We have sort of compute storage networking business units, so it kind of gave me some exposure to different roles, people play and for when Tasha and Pamela brought me in to this meeting.

F

I was thinking you know, should we bring some of that directly in and actually you know one of my learnings from doing that process was, you know, maybe, instead of just having a few stereotypes of the developer and the operator, what if we were to orient more towards the diversity of the kubernetes audience through something that we could call perhaps user profiles. So let me share my screen and and talk about what it means in more detail.

F

F

Again, it's my first time of the open-source things I didn't know exactly how to contribute yet, but I drafted a list of questions that we could imagine and some sort of user profiling question here. You can imagine this for a user panel, so you know if we got kubernetes users, but on the developer and operator side they could fill out this survey and we could contact them for feedback on relevant features and then we could also do some aggregate statistics.

F

So you know you have your basic contact info so that you can get so that the purchase you know how to reach out to the participant. But for the most part, what I tried to capture here is what are the dimensions on which users potentially differ, so they differ both at the individual level, but also in the orga level. You know what we oftentimes see that infrastructure and operations is not just an individual activity.

F

It's a team activity, the combination of the team and their technology, so oftentimes, the or type of organization they're in makes a difference. So you know, are you at an application team, or are you in central IT then across the different aspects of the technology stack, you know, do you primarily work on on-premise infrastructure, or are you on a team, that's responsible for various types of DevOps tools?

F

Some of the team structure that you're embedded in is an indicator of the types of tools that you would want integrations with. It's an indicator types of knowledge that you bring into your work and then there's a variety of other things like are you a service provider where you're, you know consulting a bunch of clients or are you actually are you not in a service provider you actually like the end customer who's using kubernetes, and so what there's there's a whole bunch of these but they're at the team level and at the individual level?

F

But the idea here is whenever somebody is designing a feature to to give them ideas of what are dimensions of difference that they should be thinking about. You know, take org size, it's like! Oh, what, if I'm, assuming that this is a large enterprise but and I considered a small organization or vice versa? You know what if I considered you know, have I considered somebody who's a consultant to many different companies, because I know that those people tend to automate more and you know have some other things that are unique about them.

F

So the idea here is to think about these dimensions of difference and use that as the way of thinking about users as as opposed to just showing developer and operator- and you know, I- can just give a brief scroll through this list. There's some team level information, there's some environment level information and, for example, like do you work on premise or in public cloud?

F

How much do automated versus manual work if you're at the user? Are you on the development or in the infrastructure operations side? Are you a junior person or more of an architect? We see really different responsibilities based on level of seniority. Are you a people manager or an individual contributor? There's, you know you have certain accessibility interests and then getting into the jobs to be done. Aspect of how often do you manage various types of infrastructure?

F

This is more on the operator side because you had access to more of the questions I put together on the operation, I'm infrastructure operations, side. You know how much do you manage storage? How how much? How often do you manage networking you know: do you? How often do you and operating systems?

F

How often are you you know resizing workloads?

F

How often are you you know doing application performance, monitoring on the operations side, so there's and then a few contributor oriented activities and potentially some tools, oriented activities, but generally the sections here are: you know some things about the organization, the user in their environment and then some things around the the jobs to be done, which oftentimes is sort of tied to the technology that they're administering.

F

So now, that's a lot to take in I'm I'm happy to copy this into the same questionnaire or you can where we can. You know create a new document for this, and but before we get to the logistics of it and the story, some people's overall impressions of like user profiles as a way of describing users and as opposed to just one or two or instead of just a few personas.

C

Yeah I I, like it cuz I, think that it's it's very rich, so I kind of like that way of presenting it. So you can see kind of like multiple dimensions in this new place. I.

A

Guess so I would wonder if this is too too much to ask people to do like if they would fade out before they got to the end or like. Would we want to try and get this out of different efforts?

A

I mean and that's just all kind of, and then also the other piece I wondered is like this piece in the middle, like it almost feels to me, like it's very infrastructure, focused and I, wonder if we want to kind of skip to the next section, which is more like focused on like what tools and services are you using that are more kubernetes focused. But that's just like my initial, like those are just kind of questions. I would ask before we would do this like kind of what are we?

A

Would we what what are kind of our top questions, and then should we focus on those really our top questions? First, because we're worried, we can't get answers to every question and then maybe, as people identify like yes, I'd love to be involved in further surveys and like this is kind of my realm of interests. Then we like target them with another set of questions totally.

A

Do people on the call feel like this is information that would be valuable to them, as they are trying to better understand, kubernetes users, what's kind of the feeling in the room for like getting this kind of level of data on, like our end users, basically I.

G

Definitely think it'd be wonderful to have which don't feel like this right now we seem to be asking very generic questions like experience level and but I, like the idea of perhaps like you suggesting Tasha of asking using some initial questions to target to then ask more targeted questions. Those go! Oh as if you have you have you used a lot of these questions with any existing communities, users, so I.

F

Haven't done this in a survey form to kubernetes users? We did this in a I. Did a survey to VMware infrastructure users with with the jobs to be done section here.

G

And, and that was pretty successful.

F

It was I would say it was it was sort of a mix. It helped us in. We had some key questions about which products to integrate, and so seeing that the people who do who administer this technology also, like you know, most of them, also administer this other technology. That was helpful.

F

Initially, we thought this was initially inspired by the way that Salesforce does Chris Nona's quantitatively, and they have some some interesting medium posts about that, but basically that they they do this jobs to be done, type I mean, might be a stretch. The jobs be done. People wouldn't necessarily call this a jobs to be done thing, but they do they. They listed about.

F

You know 30 to 50 tasks, and then they do a cluster analysis to mathematically kind of put their personas and I tried that on this initially and it wasn't particularly useful, it seemed like the qualitative way of doing personas just work better. It was easier to make sense of it. It resonated more with what stakeholders have been hearing so using so so I I mean, if somebody comes, you know, from the sales force is better. You know insight into making the collector cluster analysis work better.

F

You know interested to hear that oops, but I wouldn't expect based on that experience. I wouldn't expect this to tell us what the personas are, but it's anything I think I'm, it's more interesting to just know some of the key questions one at a time like those are.

F

Those are the things that I feel like I reference more like this question that comes up a lot here is: do the same people administer kubernetes on prim and public cloud like do you want one kubernetes tool that governs all of your environments or do you have separate teams? And so there's some there's some surveys out there, like flex era, has a state of cloud survey which you know has some data points on that I feel like the answers to some of these particular questions, especially in the organizational structure.

F

Are things that I find myself referencing much more frequently.

C

Would it be worth capturing these in like a separate document to be able to yes like reference them in the case that we wanted to come up with, like maybe like some like cross-referencing between, like the CN CF, end user survey and kind of like the? What what that knows about the coronaries community and then maybe some of the dimensions that you can capture in this document? Yeah.

F

Definitely and I could imagine also doing two documents like one with more of the user profiling stuff and then one with the jobs to be done section and then kind of moved. Those forward independently.

A

Yeah, that sounds good Boaz if you could move into public Google Docs and then share the links with the mailing list. That would be awesome. Okay, yeah.

F

I'll follow up with you afterwards to figure out like what the folder is and then I'll copy that in today, okay.

A

A

So we had Spencer Sugarman on the agenda to go over the dashboard UI survey. Did he join the call, not sure I see him? Okay, maybe you know their type and then Boaz you're still sharing your screen.

F

You don't every might you know anything. I.

A

D

One thing it would be nice to use the github issues to kept track of like the personas and the job to be done is the DES story is more visibility of what we are doing. Yes,.

A

So what I would suggest is I suspect we have more people who are subscribed to the mailing list, then, who are regularly checking our github repo, so the way that I have used to github for other groups in kubernetes is I've created. A I created a tracker like a project plan within github that you can create issues and like track them and frankly nobody ever uses that and then I also created a readme and a dr. repo.

A

And so, as we start to have a more formalized understanding of some of our projects, we would link to those Doc's from the github repos. So we basically just have a list of like these are all of our current project plans and like work in flight. So people can find them because you're right, like the problem with Google Groups, is that there's no great way to just say, like here's, all our Docs and we're like sharing them as a group and so I think what we can use the github repo for is.

A

We can create a docks repo start to link our ducks in there, so people can easily find them, but for collaboration, late, putting links to docks in the Google Group is a better way to kind of blast them out to everybody and also dropping links into our slack group. So people in the chat channel can see them and then collaborate on them.

A

It tends to work pretty well sounds good, cool and I would definitely encourage everyone on the call to become a member of our github organization, and that way we can assign issues to each other and collaborate and github, and so what you need to do for that, if you haven't done it already, is on github requests becoming a member of kubernetes syncs organization from the kubernetes community, page and I'm happy.

A

If anyone wants to do that in a in order to be able to assign issues to themselves in github, just hit me up and it consume you, the link and instructions on how to do that.

A

Cool I think that's it. We now have kind of open mic. If anybody had anything else, they wanted to talk about today.

A

Cool yeah, it seemed like everyone kind of talked already: ok, well thanks everybody, a great meeting and will collaborate offline and then meet again in three weeks. Thanks.

D