GitLab Pair Programming - Frontend, 24 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Frontend pairing - Jest speed reporter (Part 1)

Description

In this session we start looking into a GitLab CI solution to generate a report for slow Jest tests.

Part 2: https://youtu.be/BgFLT4oOwa8
Result: https://gitlab.com/gitlab-org/frontend/playground/jest-speed-reporter/

A

uh Yeah, so ip is um walking through creating a new uh ci job for creating a report on the slow tests.

A

We're heavily inspired by how we're doing reporting for coverage.

A

Maybe like this, uh I don't know, if you can, can you do variables and I I wouldn't deviate from our coverage, the way we're doing it and the coverage one too much.

B

It's I mean the problem is that, like you know, we have an alternative here right, so we have the just base one. How does the just base one actually work?

B

I think it just runs into stuff.

B

A

Add it's. I don't want to touch this necessarily.

B

A

I would go back, I would go back and I I think, you're on to something by by doing what we're doing with coverage. But one thing we need to change is on 196, where we just reference a uh xml file. We want to reference the directory and then I think everything else should be fine. You're.

B

Just saying like like this, and we should be good, the problem is like uh reports junit, so ci reference.

B

The question is like: can you use a variable in that or not j coverage report keywords? So the only thing that I'm saying is like yeah. That makes sense when are artifacts created because other than we could just do like after script right and then we could just be like copy yep uh this one into junit that x, report.xml yep and then we are happy.

A

That sounds good to me. Let me see if we use that ci node index anywhere and a.

B

We use it definitely on gitlab.com.

B

uh And at www.gitlab.com for the about website. Oh.

A

B

A

Yeah, I don't see it anywhere in the gitlab project um other than.

C

A

Our script files, I'm trying to see is it in our yaml anywhere.

B

In our yammer yeah, the problem is because it's of this parallel thing we're definitely actually using it inside our test. Parallel ci sequencer there. You can see that we're using it.

B

Which is the thing that you know tells just.

A

Which tests to run and so on, but I meant I meant, can we use those environment variables in a.

B

Yeah definitely in an after script, we can use it, but I don't know if we can use it in uh in uh so the question here would be. We want to have jest right so give us two of those xmls. Those should be sufficient right.

B

Then we can oh, we defined it here right, so j unit front end, so we want to have junit front end. uh We just one want. You know to be honest, like every xml in that directory right locally, I'm just going to create that um and then I'm going to all right. Thank you.

B

This is like what I don't want to whole artifact shenanigans command s. Oh it's a web page! ah Okay, let's try! If we can download it like this yeah uh c d, j unit front end.

D

B

O minus l is that download yeah. It looks like it. Oh you rock, um and this one uh move junit. What what did we say? We just basically want to have like. I don't know something like this like ah junit, because we glob all of the xmls. It actually doesn't matter how the file is called.

B

Four minutes to go.

B

And we can be like whatever happens here. Report files map require report file, so we don't want to require the file, so we definitely need a const.

B

Read file sync equals require fs yeah and we can for every report file.

B

uh We can say, report file, uh file, sync read file, sync, and we obviously want to have an utf-8 so that we have an array of strings rather right, sure and.

B

All xmls equals report files right and then console log all xmls, and where is it node scripts? What are from folder um node scripts front and slow? No, how is the file called get slow?

B

Maybe I have some like you know. ah Great coverage map is not defined. We don't want that. uh Yeah that looks good um now. Npm junit is definitely a junit package.

B

Okay, I've used it so j unit parser there is a junit parser. This report node have.

A

B

Parser out of the box.

E

um uh I remember the name is a little bit ridiculous, but it was something called papa parser or something.

C

E

Yeah, I think so.

B

That's right, wait! Wait! What! What are you saying what.

F

You're saying node has something built in they can read xml. Is that what you're asking that's.

B

What I was asking um so that.

F

B

What do you folks think of this? One: hey! That's! That's perfect! That's more than stable.

C

E

A

That's a lot of zeros on the version number.

B

If it pauses to file.

B

So I know that there's a reporter to generate it right like this is like for creating it, but we don't want to create it. We want to like.

A

Yeah, that's a good point.

B

B

I mean xml2.js, isn't there like the what's his name? I forgot the.

G

Node.Js reading xml here we.

A

Go xml to json is a thing it looks like.

A

So how do we use this.

B

B

Xml to json pass our pass.

A

What um stack overflow suggested that I use xml to json, it's xml, the number two and then json.

F

That's literally what I just posted in this uh zoom chat.

B

Yeah see this is at least 0-12-0 right, yeah yeah, yeah yeah. It's.

F

Got quite a bit of downloads, weekly.

B

Yeah left pad also has.

B

Okay: let's do that xml to json, then we can just be like parser to jason right of this file. Yep.

A

Or we can, we can do another map, but yeah. I think that's probably the best thing to do.

B

B

Yeah and then we can, you know, do a flat here, because we probably also want to you know, extract the first thing, but yeah.

B

Oh and we had time right, yeah do you wanna? This looks like json, you know it does.

A

Gosh the glorious amount of, but.

B

It has time on there, so I.

A

Think we could check out.

B

Minus b uh j unit to json.

B

A

Cool that sounds awesome.

B

Hit add git commit oh, come on minus.

B

B

um Yeah you push the branch out for sure yeah we'll do I send you the link in the path channel.

A

Awesome all right man and he's out.

C

uh Does anybody wanna.

A

E

um I'm still processing, he was found.

F

Yeah he moved super fast. He was moving.

E

F

What I pretty much think he did was so he created a new ci job with um and downloaded some artifacts that we already have, and then he had a script on that job. That would read that data parse xml to json and we're going to try to find the slowest jobs out of there.

A

Yeah yep and I think all what I'm really thinking we should just do is just sort it by time and print test time. uh I don't know if we need to do the whole test file or if we want to do just an individual spec.

A

But I think yeah we're just trying to generate, I think some sort of output, but then maybe we can have something else pick up and read: um that's the the goal: um yeah.

F

So I'm trying to think of, like the usefulness of this, though like since we already have that data pretty much all we're doing is filtering it in a way that it's easier to identify, slow jobs.

A

Yeah, we're aggregating it too, because it's split up into four files right.

F

A

Just one through.

F

Four um to me it would be useful to have I mean yeah, that's nice, on the ci pipeline like our normal pipeline, but if we could do some sort of like notification system, like you know, yeah.

D

F

Dropping it in a channel or something like hey team. You know this file path just finished out at 30 seconds. You know I would. I would love that, um because I'm thinking it's going to be a pain to go into the actual pipeline every time you want to kind of see the test.

D

F

Then, like track down that ci job and look at the output like that's just so, we have.

A

um Yeah, so what do you? What do you think it would look like for us to um create some sort of slack bot that could potentially just post? You know here's the top 10 slowest, you know top the files with the slowest tests. That's the most that's! What I would think is the most important thing.

E

I I wouldn't like slack because of their retention policy and.

A

Yeah, let's see a point, I was kind of thinking like how we have master broken but like for.

D

Us on front.

A

End, it just gives us like okay, do we have issues for these things and.

F

This may be a good candidate, then, for a maybe a separate, separate domain uh that we could have bookmarked that you can regularly check for like the performance of the test. It may be quicker than you know. You know, if slack could potentially get too.

A

Noisy that's a good point and that's, I think what ip was suggesting too. Let's talk about like reading something from pages, oh, how he creates all those like those. You know small side projects that basically read these reports and just present it um and that's one way we could do it for sure um that would be nice.

E

I was thinking, maybe doing it the prometeous way uh what we could do uh in order to limit a little bit of cardinality and not to destroy your infrastructure, just getting the front-end specs.

E

What we could do is do an average on every single uh folder on the average uh amount of time that we spend running the test in that specific folder, because if we do individual files, we will cause a lot of this usage and kind of don't want that because primitives already kind of you know uses a lot of disk space and due to averages, we can do alerts and we can have this on the graphina, dashboard or whatever and on the ground dash. Where we can say, hey or goals are like.

E

I don't know for this folder. We want everything to run in less than one second, and as soon as we get up more than one second, we will have an alert and immediately we will know hey something's wrong.

A

Yeah, that's interesting! um So what was your? What was your motivation you're talking about? We don't want to do every file because that's going to make something really.

E

Prometheus has something called cardinality problems, so the more because what we will do in order to classify each one of our files is that we will create a label for each file problem. Is that the more labels that you use, the more resources primitives will take from your system? So in order to mitigate some of that cost, uh we could just create labels based on folders and send the data there, as averages.

F

I don't think, though my suggestion had anything to do with, like I don't think it's going to matter with the disk space, because we're like each pipeline is still going to. You know like generate those artifacts, no matter what we're just going to take advantage of those artifacts that are already generated.

A

That's true, I think you're talking about jose, though from like we could get it reporting in grafana like rafana yep. We want that data. If we do it too granular, it shuts everything down, because it's uh at that level. If we wanted to have some nice grafana reporting, we may have to do a higher level, exactly yeah. That.

C

Makes sense not sure.

A

um Okay- and I think I think those when we figure out the best way to so I do like I would even love to just get an email of this um fiddly you're a uh you're, a test expert. uh What would how would you like to see like how we got really slow tests that are, you know, flaky and breaking mastered?

A

What would be your ideal like notification stream, for you know making people aware of these specs or even like okay? I have some time to work on something I'm going to pick up. One of these slow specs, like.

D

um For me, it should be like we have a danger bot which reports um increased or decreased bundle size which will report instead um execution time of our test.

A

Yeah, that would be really nice. It's it's tough that it's also seems so flaky like. Sometimes they take 12 seconds, and sometimes they don't like. That's really tough, too.

D

Yeah well yeah um speaking about it like a test because we run them in um emulated environment. So in theory we I should not be flaky and we have to resolve it.

A

Yeah, what's I think, what's what's difficult is that the um the speed of these tests can be so flaky, not necessarily the results of it, um but just because you know, I guess we're all using shared runners, so it's dependent on the um the amount of load on the runner, but also sometimes it seems like you just feel sick some days, and I don't know um yeah, I I don't.

A

I don't really know what that what that um end result is gonna look like of how we are aware of this stuff, but just pulling it from the ci and having a way to query. It sounds like the best first step, um so I think ip got us like 75 percent of the way there does. Someone want to pick it up.

D

C

D

Can because I used our books today, so I'm awesome.

A

Yeah that'd be that'd, be great.

A

um Are you up for sharing your screen philly.

D

Yeah, I'm just trying to check out current master, but it looks like something happened.

F

Yeah, I just I have I'm shuffling four, mrs on another topic and all four pipelines just failed. So there has to be something wonky going on with master right now. Yeah the review builds cng job is.

E

H

Okay yeah, I was like there's no way. I got four out of four.

H

A

um The mr uh and the branch is on the um slack channel uh or the slack thread in front in pairs.

C

If I can find the branch, here's the branch.

A

I'm looking over uh the changes ip made.

A

Man these these uh xml files are really large.

A

I'm kind of not a huge fan of, let's read all the files into memory, convert it all to json and, like that's um a little.

E

A

Yep, I kind of wonder if maybe we should write a bash script that like is able to pipe all this.

E

um uh I mean the less memory we use the better we already using like eight gigabytes of memory on on our pipelines. Oh.

A

Yeah, that's not great, so all we need to do is just read these files and we can do that in bash.

D

You can just use x after query uh required attributes.

A

Is there like an xml query like command.

D

Well, there is a gqueue for json files. I believe there is should be a say.

A

D

A

F

Sense, um yeah, but I don't think this should exist on our pipeline. That's for sure, yeah.

D

F

I don't think it, I don't think it gives us any benefit really yeah I mean just because, like.

A

I mean okay, that's a good point. The data's already there.

F

A

Not like you're saying is, the data is already there we're just we're just manipulating the data right.

F

Yeah, okay, I mean I mean, does anyone else see it useful like to be able to go to that job and and see that data.

A

Well, I was hoping to have. I was I was hoping to have one job that like was just really readable, but maybe, if we're already needing to build a view like on top of it or something um we should just solve it from there and not introduce something that could take up a good amount of memory. Yeah yeah, that's just my that those are my thoughts. They could be.

C

Completely wrong, but.

A

I know, I think, that's a good thought.

A

So we could try instead of because, right now what we're building is yeah. This is a good point right now. What we're building is the uh um thing that's going to read the files and then kind of aggregate them. So let's just read the files from a url right yeah. I think that's a great idea. um Do you want to try that out, fiddly or I can.

D

Try it I'm trying, but but I think we have some incident ongoing, but because I can't fetch anything from gitlab right now.

E

Oh, no, can anyone fetch, let's, let's see, runs like forever for me here.

F

I did like right before the call.

A

All right can we get this information from graphql? Do we know if, like we can get I'll.

C

Go ahead and share my.

A

Screen and what I'm looking at.

F

I don't know if we can so are you talking about? Do we have an endpoint to hit that.

A

Yeah we like able to hit you know, what's the you know, master pipelines.

F

Oh yes, yes, you can go to query uh project, so I've got a project, yeah and then pipeline.

F

I think we have it here. This should be singular pipeline.

A

That's good, but I have to give an id.

A

Maybe yeah maybe should I look at like a branch on the pipelines? Oh, I can get the ref. Oh yeah, yeah pipelines, ref you. uh You got an ore right. There.

F

A

You're you write it. I do um okay.

A

All right, I'm looking for.

A

F

So the thing about this is.

F

I'll hush! No! No! No, please don't! um No! I I think I may be wrong, but uh but like I'm just so you're trying to write a query to pull that data. Is that right? Yes,.

A

Or at least at least give us some data that.

A

That we can then at least get a list, I'm imagining probably to get a list of urls that we could then hit. Oh okay, okay, yeah um yeah, so this is sorted by yeah grid. This is sorted by the latest pipeline on master right.

A

I don't have oh look. There's like a coverage thing see I'd love it. If there's just like a slowness prop.

F

I can talk to our engineers about, like our.

A

Background and our product manager is about yeah, yeah, um okay. So what I I want to get, though, what is the stage going to lead me to.

F

um It it probably will just lead you to what stage you can filter out the stages.

A

Before the pipeline master, oh gosh, and then here's the jobs all the way down there and then it's just the name of it.

A

So, like I want to query like I want to say, it has to have.

A

Okay, I guess, because that we only want the ones that actually have jets jobs right. We want the latest one that actually has a just job right, yeah.

F

Yeah, the latest pipeline rental master, always and only those four jobs yeah. um I don't know what after means.

A

F

I have used this query. I mean this part of this, but not this granular.

A

After test before test, I don't know what I'm doing: okay,.

A

See if I can just get page info and it's still not gonna, let me just.

A

I don't think it likes my query: okay, yeah after pre-test before post-test, uh I'm looking for just yeah. Okay, all right, I blew it up.

A

Is it gonna give me like all the stages, because that's not necessarily what I was wanting yeah. It should.

F

As well, okay, I don't know if it takes a param to grab a certain stage name.

A

Maybe I'm looking at this wrong way. Maybe I should be looking at jobs.

F

Yeah, if we can get all the jobs with the right name, we can that that makes sense.

A

Okay, let's check this out, though,.

A

A

And so then I can go to this list of groups, I guess and the jobs yeah. So I guess I can just get everything here um and I guess that's fine, but I wonder, is there a way for me to given a project instead of just looking at pipelines? Look at jobs.

A

F

F

I'm actually writing one on my side too, just to yeah.

A

F

That's helpful.

A

I'm not talking yeah, no, it seems like graphql might not be the way to do this. uh Maybe in the rest, api there's something.

C

F

I'll pull it up. Let me.

C

C

A

You I don't know what these edges are.

A

I don't know what edge is I'm not gonna think about it.

A

Okay stage stage group nodes, here's all the stages and then we're gonna. Oh yeah, we're gonna get the name of the stage and then we're gonna get jobs, and I guess I did this notes thing.

A

Again can't query: oh, is it called groups? Oh man, I gotta go even deeper: okay groups, nodes name, jobs, notes name, oh my goodness.

C

E

I think this is too much. Oh there you go.

A

I don't know what happened. I don't know what I'm looking at so ah the groups versus the jobs. Okay, so I'm not really interested in the jobs, uh because I really wanted to see hey. Do we have the just one in here? Oh can I can I look up a group by name. That's the part that really bothers me. Is I can't anyways okay, um okay, so we can get you know. I can use this to see all right. What is the the pipeline id.

A

Of the last completed one that has jess jobs, you can use that to download the artifacts.

A

A

Yeah you wanna, try it.

A

Out, I feel like I have that's a weird looking id id.

E

A

Think I mean like I id probably yeah.

E

Yeah, that's probably uh specific stuff.

C

E

Okay, there we go.

A

Cool uh yeah. How many of these I get? Do I just get 10? Am I just looking at the top 10? It's probably paging the yeah, the results. I wonder, if I can, I can filter by status. That would be really nice. um I lost my docs project.

A

Pipeline, oh none! No! No! Sorry! Oh! I lost it again, uh pipelines oh status. Oh I can yes! Yes, yes, yes, yes, yes, okay, success, who's, running we're! Talking we're looking for success! Oh no! It's canceled! Is it actually called? What's it called I'm moving too fast success? Yes,.

A

E

You probably need a.

A

Yeah wow, it's.

A

Smart cool, that's great, okay, so then I get the so then get the pipeline id all right and let's just say, I then use that to get I'm gonna look for the one that has uh um maybe I can keep using graphql. Let's see.

A

Oh man, no, this is gonna. Give me.

F

All right, I would like to say we do have a rest api endpoint directly for job artifacts for our project. Oh.

A

Wow, can you can you share this rest, api endpoint? That sounds actually on the sim chat. Oh okay, that sounds actually more.

F

Helpful, that was the pipelines, but I just the very latest one is the job artifacts rest endpoint.

A

Okay, so given a specific job id, then we can get the artifacts cool. um Do you want to try all right? So here, let's just say, I got the latest pipeline. I'm gonna open up another graphql explorer, um I'm just planning all this out. um I was gonna. Let's then see if I can find the specific job id and then I was gonna curl this and see if we can get to it. Yeah.

A

That sounds right all right um here, I'm querying something. I guess I'm going to project.

A

Getlab.Org gitlab pipeline and the iid is 7806.

A

And I'm gonna be doing. Oh, I lost it. Don't you need to add I I d. Oh you're right. Thank you. Thank you. Yes, I do. Oh and that's not it either there you go um project pipe.

A

Oh, no, it's not letting me search for it. That's weird pipeline. um I want to get stages. I gotta do this whole thing stages.

A

uh Nodes not nose.

A

C

A

Id name, oh, is it just my name? Is that the id? Oh? So I don't even need to oh wait. No, I need an id.

A

What's this. What's an example of the id yeah, it acts as an actual id uh ci group ci job. Let me go back ci job job, oh no, it has just the string. Is that helpful.

A

I guess it's not going to work.

B

Are you now doing it via the apis.

A

Yeah yeah uh peyton brought up a really good point that um so we looked at this, and these files are kind of humongous um and the data already exists and we want to present it anyways. So why not we just let whatever is presenting something: do the fetching the files aggregating or whatever, rather than you know, having our pipeline be burdened with reading these huge files into memory and all that stuff? That was the suggestion.

B

We could in that, in that case I would highly suggest to go similar to the webpack memory reports that we're doing uh and basically just create an external project that queries those few things from our api yeah.

A

That was the plan. Yep yeah doing something on front end: playground, that'll that'll, do that um some sort of dnl script um yeah, uh but this has been interesting. I am gonna, probably create that front-end project. If I get really bored today,.

F

Yeah, I might I was thinking about just for fun, spinning up one with the rest api just to see. You know.

G

Yeah, let's, let's create that project real fast, so that we can both be working on the same thing.

B

Right the rest api is literally like uh the thing that I've written for the failures yeah, because the.

F

Rest api has a job artifacts endpoint. So yes, that's the one you used, yep, okay,.

B

Yeah, it looks perfect. Let me let me just add my script and make it download the xml, and then we can go from. There also has the benefit that that we can, uh that we can add like a templating engine or whatever in that pipeline and just publish it to pages, because the problem is that when we um that that's usually also something that you don't want, uh it's also usually something that you don't want to add to our gitlab project right. If you want to use something like pug or whatever right right, right, yeah,.

F

So this script, you already have it written to query over or iterate over the four jess jobs. Yes,.

A

Okay, perfect is, is test speed reporter a good project name for you, all, maybe jest is, should be somewhere.

F

In the title, okay,.

C

Jazz speed reporter that's fine, okay,.

A

Cool uh well, this was this was fun and educational um and uh yeah. So this is oh, I created, I didn't, create it in the playground I just created under front end. Oh.

B

Gosh, it's fine! It's fine! It's.

E

B

Leave it leave it.

E

Leave it, but I.

A

Can't I can't keep it, I'm not a maintainer.

B

Well, I'm a maintainer, I'm an owner, give it share the link share the link, give the link. Okay, can you? Why aren't you a maintainer of front end.

A

Well, the front end's hard: um can you can you move it to the? uh No, I think when I became maintainers just like, because some people are maintainers of the gitlab org group and some people are.

A

I was one of the people that I'm just maintainer of the project, um but I think it's just a it's just an admin issue, but I don't think I have push access to this. That's.

B

That's what I'm saying is: if we, no, you should have push access as your developer, I mean well.

A

Not to master it, so I want to put on the playground so that we can all just everyone. That's everyone's a maintainer yeah, I'm moving it. Okay, thanks! It should be on the playground. um Sorry man, that's frustrating! Let's I wish it wouldn't. Let me create a project, and now I can't move it.

G

You could you could blacklist it. I I can't even rename it what's. What's the thing, it's called playground right.

F

Yeah front end dash playground or underscore. I.

A

Think it's front end. I think it's a sub group. I just needed a thing. Oh yeah.

B

Let me let me push my script, that downloads the stuff, and then we already have the stuff and someone else can deal with the xml stuff.

A

That'd be awesome should be really easy. um Yeah! That's awesome, hey thanks everybody for hopping on and helping out, and it was it's fun diving through these uh ci stuff. If.

B

You yeah, it should be there now if you reload the files perfect.

A

B

Great and let's maybe start with a main branch, first uh sure yeah.

A

Are you able to change the default branch? It's the first branch that you push.

B

Oh really, yep. Let me just push a main branch.

A

Oh you're right, oh well, okay! Well, everybody have a great rest of the day and thanks for hopping on- and this is, this- is fun uh so yeah I'll catch you all later, all.

A