GitLab Monitor:APM Team, 20 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Monitor:APM Weekly Meeting 2019-11-20

Description

- GraphQL
- Instance Administration Project
- Review issues moved to 12.6
- Moving Pod Log features to core

A

Okay welcome everyone. This is the monitor, APM weekly meeting for November 20th. So first thing on the agenda is from Miguel, but he is not gonna, be here, the middle of the night for him, but here he was asking about if we should start adopting the graph QL.

A

So he had one use case around data that we fetch I, don't know if anyone has more context on from the graph QL effort, that's going on I.

B

Think Shawn Arnold had open NMR to do to create an a graph QL API for error, tracking I think that was the start of graph QL in monitor stage. Okay,.

A

I know there's a and I think in this issue that Miguel has linked here. It links to I know there remember where it is, but there's a documentation update for graph QL, that's in progress right now as well, for you know, what's more background on it.

A

I, don't know, have you had a comment, I think.

C

A

Miguel camerton was just asking you about it. If we want to release api's.

D

To the community to the community, no I think he means API.

B

Is created using graphical are available, for you know API consumption as an it's, not just meant for front-end to call back in.

A

Okay, so that's a consideration that if we switch to that and we're exposing those API set, you're saying ribbon yeah.

B

Yeah so it's kind of like a benefit to the community as well, of course, which.

C

B

C

Which is a good thing I think about if we, if we expose them and then we'll probably need to document them and if something is breaking their behinds, we should also communicate it to them, possibly also measure. If someone is using that if it's possible but I mean that's details that we can handle and we can approach. But that's that's my point of view: I'm supportive.

B

So if we do create a draft, we live here for environments, you'll have to sort of create it alongside the existing API, because the existing API is used in a lot of cases and until we switch to using the graphic really.

E

There completely.

B

Be one will go to move the existing up here.

C

Yeah but I guess every new API that we will have will be based on the skewer and right I mean we won't have like a new. If we need to add a new way, a new API we will need to have them like we need to develop them.

B

Sorry I didn't get the last bit if.

C

We need to create a new API will it need to be duplicated as well, always only only for that.

C

That's the question you.

B

Mean if we create a new API without that.

C

You said that we need to maintain both versions- the god ql, and what we're using today and.

B

C

Is let's say that we develop now a new api needs to be.

B

C

Terms of main thing, I understand that we will need to maintain both of them if we are using. You know in some parts of the product category and some parts of the product. I understand that this is the way everyone works when moving to a new technology, yeah.

B

I wasn't providing sort of a negative point out, I'm for pleading with.

A

Okay, so there's more information in that issue, that's late, so people can continue discussion there if needed as well. Do.

F

You know sorry, just one last quick question: do you have a sense on how big of an effort that is for from a back-end perspective like? Are we talking a couple releases? One release, I'm not sure what that process would entail. Back-End wise, that's open to anyone, not in particular.

B

So I think it'll be a slightly large task, but I can't really say whether it's completable in one release or two, because if you look at Shawn's, mr it's it's pretty large and the environment API itself is quite large right. It does a lot of things, so yeah I think it'll be one of the larger sizable.

G

A

Yeah, that's good question thanks. Okay, all right! Moving on so we're getting close to being able to release this instant administration project feel like we've been working on it for months now, but the last step is planned for this milestone, which is to add a button. So a user can actually install the project, we're gonna wrong about. What's the project will get installed, but then it'll be mostly empty. That true yeah.

B

It'll be more simply but I think the task to create a dashboard I think it's to deal for the next milestone, I think so. Yeah.

A

Yep, so that's what I linked here then I think this is the same one.

A

So I wanted to start planning that issue, so we can figure out well, I! Think it's in this! It's.

C

In this milestone right so I wasn't there.

C

D

For this release,.

A

There's one that I linked in a agenda: that's for 12.7! That's the title: who can figure self monitoring project display instance, metrics, yeah,.

B

That's the same one yeah.

A

I think that's the one I was thinking, you know what there's another issue you were thinking.

C

C

Six and when I saw there the amount of.

A

So, whoever it lands, but what I wanted to bring up is that we should start thinking about what actual metrics we want to capture and put into this project. I, don't know if we need to there's if we want to work with the infrastructure team. If we just know ourselves what we want to do, I think.

G

A

G

Gonna say: would we start just pulling things from dashboards, I get lap net.

E

G

Maybe not everything is some ginormous page there, but video infrastructure, I think you're right infrastructure had some good input on that I think we've got common things towards the top and in dashboards. Oh.

C

C

To open this discussion with them to hey, we do whatever boat. You want us, because we will be like a never-ending task, so I think but you're in this scalability working group right. I think this is a valid point twice, though,.

A

And we've talked about it a little bit so I can bring it up in there again. That's right.

C

Maybe point and point them you're pointing to the issue. You point them to the issue, so they will add comments in the issue and will be visible to all of us. Okay, there's another discussion. There's another discussion about I. Think it's in 2007 about English, engineer dashboard that was asked by and I think it wasn't. A selection of thing take it you're in this discussion right.

C

G

About an ingress I know you not thought about a bit.

C

I can find I can try to find the issue.

C

Discussion, maybe maybe maybe we can combine.

G

Prometheus has already got the ingress matrix and we just had to make some graphs for.

C

Me Scott English I'm, not sure that we're using the English exporter.

C

Do we don't write? We don't need it because why, because exposed think I think it's okay.

G

I, don't I think that it's Prometheus has already scraping that thing. Yeah I mean it's anyway I'll. It's a bit. It's unclear to me why we would need the exporter for things that we're already capturing a bunch of metrics out of ingress. It seems so I, don't know what I don't know, what extra metrics the exporter of things but grabs, but but yeah we do have.

C

C

So the tree, the three metrics that we collect for upon genex, are elevate.

C

Those things and I think this is like this is a good and which see if we need there are critical metrics that we should add to them. But it's let's stop.

A

Just for some background, we, like Dovan I at one point I, think talked about this- that we want to be responsible kind of for the framework of how to add metrics. Maybe we can add the first couple ourselves and then the idea would be that infrastructure or some of these support teams or someone could go in and add other metrics charts that they find useful. But they don't have to rely on us to keep doing that.

A

C

That's a good point. Thank you for raising that, so I mean the long-term. We don't need to hand-holding the infrastructure team and this entire project of self monitoring eventually to land in the infrastructure team. So we need it. We need be maintained by us forever. What we would like to achieve eventually is to provide the team with enough tools for them to add on tricks to create their own alerts and and create their own dashboard once we'll get it into a into a minimal state.

C

Maybe even maybe minimum is enough, then we can hand it over to the infrastructure, so we don't need only to focus on how we can provide more value in terms of moment, it's a dashboard. Let's do the minimum, yes, but let's think about how we can provide them with the ability to do that by themselves. Then hang it over to the team.

A

A

So the next thing on the agenda that I had was I, was just looking over the retrospective for 12:5 and I noticed that I don't think anyone on our team has put anything so just a friendly reminder to think about that. Add things. If, if you have any items, so we can make sure we have some some valuable feedback from across the team. I.

H

A

Have some I want to make sure here that we get more more feedback, the more the better so.

C

The station of self monitoring without the dashboard is there any useful, creating the empty project or will it be behind the chocolate until we will add metrics to the dashboard.

B

It could put it the end of applying I wasn't planning to.

C

Reading this, without so I guess putting it behind feature flag until we'll get something.

C

It's a button that doesn't do anything beside create an empty project.

B

Okay, I'll put it behind a feature flag and we can enable it next release. When we add the custom.

A

Okay, so then hope you had the next issue about issues that are automatically removed. Yeah.

C

So it appears that in the next in the crimson is but the pills that all the issues that not closed on the 18 moved automatically to 12.6. At least they were labeled, like mist, automatically moved automatically system label based on 12.5. So we don't need to go and label them.

C

But the problem here that I don't have weights from those, so I'm, not sure which status they're at I assume that they want that I will review or probably already complete or nearly complete, whether some development and.

C

I didn't know how to handle the amount of issues that we have now in this coming version.

C

So what I guess the question is, what do you think we should do, and maybe how can we avoid it for the next days? Should we schedule less issues? I mean I need to need your feedback I.

E

Think perhaps scheduling less issues is an option like just because I'm, both health and a p.m. it still seems like there's a lot more issues scheduled in a p.m. than there are in health in a given milestone, like I. Think, there's, probably only like tennis, you scheduled for health and a given milestone. So it's like a lot smaller and I.

E

Think I, don't remember how many we were at I think it was more like 20 or 30 or something so, maybe maybe it's less I can't remember the exact number you ended up with, but it always feels like there's less scheduled. It's like more realistic in terms of actually gonna get done. Perhaps perhaps less ambitious, but I think that even with the smaller list of issues, they still have issues that don't get done so they but I guess they have pure.

C

Yeah I'll take this into consideration, as we are scheduling, 12.7 and also maybe to some additional welcome 2006 to reduce even more the issues that are scheduled in this.

A

A

C

Yeah, so we got approval, so if you know, but up until now, still Portland feature is in ultimate and we got the okay to move it to Cole and not sure if this team will be responsible for that. I would like to understand, therefore, the summation here, but right now the perfect team is scheduled to work on that it may take them like a milestone or two depends on the airfield I, don't mind waiting like another milestone.

C

That means two things so, first of all, I want to make sure that once will have elasticsearch in place for belonging use case. It will be in call, so we can wait with this move until we have elastic search when elastic search is there I want to make sure the clock isn't, and this also mean is that elastic this entire elastic search effort and the solution will also be involved.

E

Good comments, feedback is this: the the locks that are currently available off of operations environments, or is this I'm assuming so.

C

You know you to any other I guess.

E

I just meant, because we've just added the padlocks operation spot logs, which.

C

Are the same, I mean it's the same eventually redirect you to the same page, so yeah I mean both of them, which are same.

A

A

The on that topic.

G

I think that's a good step. I think it's a lot more people to our order. Devops.

C

Yeah, maybe I personally personally, I am strong supportive moving. There's many features as fast in the me going to cuoco.

A

We have analytics around the usage of this. Remember yes,.

C

So the usage around.

C

The usage around the pod log is really minimal. It's I think it's less than a percent. Last month it was like 0.7 percent of the users don't need to log in.

B

Only that users should increase with the links.

H

B

Operations, because this.

H

C

This is one conserve. The second thing is because its ultimate.

C

E

C

C

It's in periscope, and so you can see the metrics around the environment. You see, the environment is the most popular one. After what is the metric? The metric.

C

You know environment number, one, the matrix, number two and then population settings, yes, yeah.

C

This is by the way, one of the reason that I think the environment page is very critical and can be the point for users to interact with additional.

C

Solutions within within monitoring, so the infrastructure dashboard, is all about the environment page. So one of the reason is that, because so many people are using it in this great way for them to get exposed to our features.

A

A

Next I started update to our APM handbook. A number of teams in the ops group are going to be doing this, so I started the kind of our version of it at least to document what we do now I think there's probably some areas that we can improve upon, but I wanted to start with. This is this is kind of how we work today and then they'll be easy to open them ours to change that if we ever have ideas and we want to iterate on it, so anyone has any feedback.

A

I'd appreciate that we have to talk about it right now, but I just wanted to kind of point that out to play. If anyone has time to look it over ok, so we still have some time left. Do we want to spend some time looking at either new issues that have come in or bugs or anything like that.

C

I'm open for suggestions, we can all go over the new issues and want it I think. What will help me is to go over the issues that were moved from 2005 to 2006 to understand. Maybe we can add weights well, we can and then we can go over all the new issues of the box.

A

So, okay I think I'm sharing key people see this. Yes, this is just the planning board filtered by that mist 12/5. So these the ones that were automatically moved.

A

So maybe we can just kind of walk through there's seven in this column and one that's already in twelve seven, so we're gonna start with that and I think there might be one security, one that no, no that's something! Okay, so, let's see so taraji Ryan. Do you have updates on this one yeah.

I

Both the front end and back end and mark for that are in review, so it should be done pretty soon. Okay,.

A

This one has a weight of one.

A

Okay, so this next one.

G

Tomorrow, yeah I've got an M R out on that I set two way to zero, and you know in retrospect right now it might ought to be one just because of the there's. Some gonna be some front in work that is not gonna pass muster with whoever looks.

C

G

A back-end and front-end required front-end work to get it to. You know useful, that's all! In the same, mr, you saw in the same manner. Oh I made two other branches, I split it up, but I don't know if it's impossible to test it. You know without without having it all in one anymore. So maybe it should just live.

E

G

One anymore and and get multiple reviews, I mean I've, seen both things done so I asked the group: how do you? How would you rather review something? Would you rather review it all in one, mr or in front-end,.

I

G

Back-End split em, ours.

G

B

Well, usually, are the same sorry go ahead, better.

J

But sometimes it's just easier to do the back and get that in first and then the front end could be updated. But again, if there are too many function changes, maybe it's better to have it as its own, mr because that will also be easy to revert such changes.

J

G

Easy to split out so for if a reviewer says split these out, I don't want to see this I can so we'll start with it be together, and then.

C

I

C

Issue, if you scroll all the way down.

C

There is a front end issue.

C

No, so this okay doesn't matter this. This issue is linked to them all both for the front and the back. That's what right.

G

There's one anymore and I've got two branches that are there. If you look yeah and there's two branches that where I split it out but I haven't.

C

G

There was a there was a front-end issue, the pre predated this issue, but it could be, it could be.

C

You should live in it.

A

Okay, all right! So let's continue on here. So the next one.

A

So this one looks like it's in verification, mr that was merged yeah, so we'll just clarify that so the weight of one is probably correct.

A

Unless anyone has any other thoughts on this one all right, so this one doesn't have a weight yet.

G

We've talked about.

F

G

Think ripping out all this code is the wrong way to take this.

G

B

But one solution would could be to just remove change the ordering for the new proxy API. You know yeah.

G

B

Then you don't have to rip out the old code. I.

F

Think we decided we didn't need to rip anything out in order to like rearrange based on the yellow order. But oh this issues description seems like the.

B

Skull is a lot larger than.

F

Just that like this is maybe we need to either update the issue, description or turn it into an epic or something because this is a like. Actually, a really large right here, I mean.

G

Scroll all the way, the bottom- it essentially says we just need to take out the ordering ability. I mean this. This could easily be the ordering could be easily changed to do that too, by just changing all the the weights and priorities to the same number.

H

G

Then either default sorting. You know, there's.

K

No essentially, no sorting but I think the default. Sorting is just the order that it appears in the mo files. I'll.

G

Have to verify that? Yes, that's what it should be. Yeah.

B

I think our goal was that just order it according to how it's placed in the mo file and remove the requirement for priority, and they.

C

Will make sure to subscribe to update the description.

G

Give me a way to way tattoo on that.

A

Sure it might be even higher than that if we're yeah changing how we would think we're about, we wanted to go about it, even though most of it done so. Okay.

G

Sorry, it's it's hard for me to think of weights and percentage of done. Instead of you know, weight or you know so yeah as soon as you're done might be different.

A

G

I've haven't even started on this one, so this weight of ten okay.

A

Seems correct then, questions about this.

G

Point there's no point in me being assigned to that and just camping out on it.

A

There we go okay next one, oh.

A

Okay, is there more design know if there was any more UX that we needed to do on this.

C

E

Were I think a couple comments that came in yesterday? What are some tiny updates.

E

C

About about about this issue, and so, first of all, I believe it requires some bacon walk, especially when we check King. If configuration exists, because you got the description you go to the design to the walking in progress design. There are several scenarios, no data, timeout and connection failure and I guess some of the issues here only front end and some involve content and bacon. Maybe we can break this issue into tiny and you tell me it makes sense to boy getting tiny iterations and walk only on know that timeout or connection failure.

C

What it's the easiest one to accomplish at the moment just go further down.

C

Table so there is the level of the chart in the level of the group, so we can also say: okay what's more important and we can start I think the chart level is more important than the group and maybe also we can like, say: okay, let's walk on the child level first and then think about the group level. Oh, vice versa, if it saves.

G

Is there a way do we want to have the back end or is there a problem with the front end, just blindly grabbing everything when it kind of knows it's never gonna get any data from some of those things. There's some way. We need to have it back off or say these things are.

F

Yeah I mean it seems like an easy. First step is exactly that is just to basically throw up a no data thing on the group level and then kind of bring it I mean because if there is stuff like in the charts, we do at least have a no data state at the chart level. It's not detailed. We've discussed like adding more details to why there's no data in the chart level, but I.

G

Love that default graph that comes up there now, it's beautiful.

F

Yeah at the default, you know no data, oh well, yeah I mean we could do something similar for uh for the group level, basically, and just have it at least that way when they click on the arrow, you get a there's, no data to display, and then we can go into you know those reasons so yeah, don't think the ideas is good. Let's just you know, divide it up into smaller issues, and then it should be a quick win at least to get just a default one and into the group state. That way.

F

We eliminate this confusion because it keeps coming up. I mean it's keeps coming up.

C

There is another one that I was thinking while I was walking on this looking at to the issue, but it's published separate discussion, but it's kind of related. When you have no data on the gosh, you cannot, you know, set up an alert because they had more bottom. The more button is available only when you have a chance, I'm, not sure if it's by design or it's possible, just to add this and configure alerts, I'm, not sure how the backend is capturing, the metrics that are on the graph in a different way.

C

But basically the question is: can we alert.

G

Could be could be defined on.

C

G

Technically, did the data doesn't have to.

F

G

For the alert to be defined.

F

G

You know that's a.

F

That's a friend only fix thing because you're saying David, we already have support on the backend to do that as.

G

Far as I as far as I know, yes, okay,.

C

When you choose, when you create an alert, gives you the name of the metric value, it.

G

Well, I mean it had to know the name. The name of the metric is defined in the graph not defined in the data that comes back from the from Prometheus, so the name of the metric should be you.

E

Know there in the.

G

Legend of the graph, so yes, it's, the the definition is technically technically all there to create the alert.

G

Now, in the case of you know, as we add these four custom metrics, if there's no place that the user sees what that metric name would be right because you know it would be, you would have to. You- would see the name of it in the legend, so you may not be able to couldn't paste your your name for the custom metric into the for a custom metric.

F

Does that or maybe someone on the and maybe we need to investigate this a little bit too I can't remember exactly like. Are you saying then David that the response comes back with each individual metric and the data set there that we wouldn't know well.

G

You get you grab the de metric and all the metric ID and all that kind of stuff from the from the dashboard information that comes back over that defines all of the graphs and as far as I know that it's what's used for creating the Med, creating the alert in the backend and the so yeah. That's that's: what's there there's a there's, an ID there of the alert I mean if the metric and and just a mean operator and a value, it would be hard to know what value you wanted.

G

You know if there was no data there, if it should be a hundred, five or I, think it obtained or something.

F

Yeah I think what I'm getting from this, then is. This is like adding that more button is a separate issue. I think from even this will just need to create its own issue and then get it scheduled up and I think it should be front end only from the sound of it.

C

J

Also for this, it should have been looking at should be even have them this to a lock file enable because it was clearly not very for development in 12.5, and we did not miss the.

C

It's the same, it's design issue, it's I mean from that. We need to open support issues and schedule it for the for for the milestone we can. We can say it's completely. Okay, if we drag the terminal because from that I think we should open additional issues or maybe promoted to an epic I'm, I think I.

B

Think the missed label is added by a bot I think it's.

A

Okay, so we have actions on that one- and this is the last one. I was a little confused about when I saw this one.

A

Know if anyone else has more information about this, but oh that's right. This is the community contribution.

A

F

A

A

So do we need I, don't even know if this is.

A

Have a look at this one that close a so we might just need someone to help kind of push this one through and our side too I.

B

Don't think that's a community a mod at least the person assigned Sims, video or the author is the community.

A

Okay, so I think yeah, so maybe someone on the back-end team can I think this is bad, go back and work can just. Oh that's.

A

Okay, well maybe someone can help kind of Shepherd this through and help guide this community person, because it'd be great to get some more community contributions in here.

A

A

So I think those were all the issues that removed that are still in 12:6 this one. This other one was.

A

Was a missed 12.5, but it's already moved over to twelve seven yeah, so I don't know if there's much else, we need to do on this. One.

C

It's yeah. We have enough on our plate, then, okay,.

A

A

They were also gonna need thoughts about the mist, 12.5 I think yeah, never mind what I was gonna say.

A

All right, so then there are there any other issues that we want to look over, we're running short on time and a few minutes left so.

A

If not, then I'll call it there, I guess, and so thank you everyone and have a good yeah.

B

Do we want to go through newly open bugs yeah.

C

Wait: five minutes. The question is: do we have enough time? Can we can look at one sure, yeah.

A

It's let me pull it up here.

A

Okay, so I mean: let's see this right way: oh yeah, okay, it was started right and then I did it the wrong way? Okay, so we could look at the last few least once this refreshes are you me in the last week they were.

A

15 or so I don't know if there's any that we want to one security one in here that we won't talk about this one Raju open this one. We could just start and go through the first few on the list, I guess and we'll just kind of review those, and then we can I, don't have to be it. So this first line.

J

I think this regression so when, when you dashboard and you open, the date/time picker you'll see that the default 8 hours is selected and also the from and to extend person also filled, but that that's must not be the case, because I should be pretty straightforward. It's okay.

B

We'll probably have to make this video private, because title was in there. The title is pretty descriptive: yeah.

A

Okay, I can do that. That's fine.

A

This is, we talked about this one already.

J

C

There is an example project.

C

Viktor Pesta from this lecture now.

C

All right, okay,.

A

We'll just do like one or two more here and it'll be up on time. No chair, Association, okay,.

H

A

At this one, so just the documentation it's assigned to you, I.

A

Think this is a the documentation issue. If I it looks like there's already a mr for it. So that's good, then we'll skip the triage report and then we'll look at this last one last one that will be.

C

To cover this API I pushed it towards the day, maybe maybe we'll just start it. If they're thinking about the solution, okay,.

C

A

Because this is the problem that, if you have elasticsearch you're, only be able to see you last search logs, but if there's some problem yeah, you want to be able to get back to this thing. You know we took a.

A

That's a all right, I think we ought to call it there. So thank you. Everyone and yeah have a good rest of your day. Thanks.