GitLab Backend Pairing, 18 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: BE Pairing - 20220718 - Improving speed of elasticsearch specs

Description

In this session, we discuss ways to improve the speed of elasticsearch specs. We also discuss approaches to make a script more deterministic like the test environment

A

All right we're.

B

A

Ready to pair on the back ends.

C

Okay, but so um I have this dream to move all of the specs that deal with elasticsearch to um move away from creating the elastic search indexes and deleting them like every single time. It runs um I think it's around the context, and so anytime you have things at like lower levels. uh It gets really expensive, like deleting the indexes and recreating them.

C

um The search team tests used to be like the top three longest running, spec files it used to be ours, I, don't think it's ours anymore, but um about I, don't know how long ago this Mr was about a year ago. um This is just to show the before and after of these four specs, where this was originally introduced, and it made a really a significant impact on the times so I thought, maybe it would be fun to yeah that.

A

Sounds awesome.

C

To see if they will all work, I.

A

I love fast, specs and I cannot lie yeah.

C

Yeah, ours are bad, uh and this in particular, are the ones that were changed over. They were um testing permissions, like these permission, like almost like tables yeah,.

A

C

So it was just like constantly deleting the indexes and recreating them over and over and over again, but um I don't see any reason why yeah we have to do that. Yeah.

B

I've noticed that too I think like I was just looking at one. Yesterday, I was like you, don't have to test every combination of permissions for every single thing. You.

C

Can have oh yeah.

B

Lack of permission checks for the happy path.

C

Oh yeah, ours I didn't change what was being tested. What I changed was whether the indexes are being created or not. So what um I can show?

C

Let me show the state of the code right now: um yeah I.

B

Kind of think shared examples are an anti-pattern, I I, don't like them. It's so hard to debugged. You can't focus them. Yeah.

C

Oh yeah, it gets confusing right.

B

And they're easy and multiplicatively explode like that, without knowing it.

A

I feel like the parameterized test, Chad.

B

Or just write methods extract the behavior out the methods and call them anyway. So you.

C

So my computer's freezing hold on. Let me let it try to catch up okay, uh so we have this uh config. That's set up for.

A

Is it possible you're not sharing the right screen.

C

A

The right screen, I just see.

C

Obster, oh, no, that's not the one. It says it's sharing hold on. Let me see maybe a picture on one. That's very possible I could look at Oxford I, don't want to I'm.

A

Just looking at a bank, oh.

C

That one, it's very possible, that's really weird. It looked like it was sharing the right screens, but that looks right. There's code yeah.

A

C

Hopefully, now Ruby mine, oh.

A

Yeah: okay, the right code: okay,.

C

So we have this um config set up where it's called Uh colonelastic, that's the old one and around every example. It does a setup and A Tear Down the setup um kind of clears some tracking deletes. All the indexes creates an empty index for a few things.

C

um We have migrations, it marks all those is completed and then it um kind of refreshes elastic search and then the tear down deletes the indexes and clears the tracking, uh and this is cool but I. Think it's like it's adding a lot of time.

C

So we have um a second one that was created, which is just deletes all of the data in the index.

C

Instead of creating and deleting all the we have like I think five or six indexes right now and it's gonna get possibly more, as we add more data to elasticsearch or continue we used to have one we've been working on splitting it up into separate like that. Work will eventually be done, so we should have somewhere around like seven, but if we decide to add projects or groups or users to the index, those will now be separate indexes.

B

C

I'm, aware of not the way that I think databases have yeah.

B

Because that's how you get around this problem with it transactional, fixtures and rails like it, starts a transaction. Then it runs your example and then it rolls it back. So it's all just oh there we have so.

C

B

I like this, that are external and non-transactional, you have to do other things, yeah.

C

Yeah I, don't think it has the concept like that, but it is a good idea to maybe look into into that. um I feel like most of the place. I worked at, don't have a tough Suite as uh comprehensive as as GitHub, which is nice, and what I prefer I like tests, because I don't have to worry that my stuff's Gonna.

A

C

A

I can I throw some really naive questions your way. um Yeah. Do you know if the end-to-end QA specs run into time issues because of the last to search it all.

C

C

The ones that deal with ours I- um yes, they are yeah, so it was a similar. That's.

A

A similar problem- and we may not necessarily want to fix that one, because that one is more about not messing with the test setup and working on as real of an environment as possible. But this is just also for our feature specs or run into these.

C

Yeah, this is like for egration tests and the um unit tests and probably feature tests. So my.

A

My other thought, my other question was to confirm that this indexing and re-indexing is causing slowness. uh The question is really like. So what what have we done to confirm that that is a problem and that, like maybe removing delete index, actually makes things slow? It makes things faster like.

C

So um I did I remember how I timed this I think it just timed it locally, but about a year ago the times for these specs, like.

A

C

Is the before uh running the entire file, and this is the actor I think this was on my computer, so I mean take that with a grain of salt. I have one of the 2019s, but um I think the numbers were different enough. That I felt like it was going to improve in in the Pipelines.

B

There's a half that last one is a quarter of the.

C

Time yeah so I mean your question makes me think we probably there are still probably cases where we may want to clear the indexes like we have some tests around like creating the indexes and so having them not be deleted and not have like a fresh uh elastic search instance like might cause some problems, but when it's around just kind of mashing data in there and making sure that the search is working, the way it's supposed to work like it doesn't matter. If the indexes are already there or not, yeah.

A

So my my my really naive thinking was like my understanding of the last search. Is it supports very unstructured data.

C

um Yes, like it's not like a.

A

What do they call it relational database? No.

C

No, what we have like some of our data has parents joins set up, uh but we are moving away from that and we're trying to denormalize the data so that anything that has like a joined record, which should also be stored in elasticsearch, is gonna, be kind of have that data copied. That's part of the work that we're doing to like split all the data off of one index into multiples. So my thought my thought.

A

Was um so like coming from like working a little bit with like mongodb.

A

Inserting data the database does some sort of indexing every time you just put data in there like.

A

And my my thoughts was okay, we clear out the indexes, but we just insert all brand new data every time like our is this, are we really clearing? Are we even able to really clear things out just the way elasticsearch works and like uh I don't know so? I was I'm. Just surprised that this this would work I, think but because I also just don't know elasticsearch a lot at all. So I'm learning about a lesson search right now, so so.

C

Well, that's why I thought it would be fun to do this, because I really wish that we could get like more devs, at least more comfortable with like testing our Mrs, because it requires like you, have to have it set up in GDK, and then it has a whole other piece of elastic searches. A lot of people don't know, um but the delete. This is the Mr that was originally introduced, like you can just tell it to delete everything.

A

C

A

Index mean something different in Alaska search land than like it would and like the postgres land I.

C

Think so, yeah, okay, the index is like a collection of documents and mappings. His.

B

C

Are like almost like data Rose and um yes and I apologize.

A

It is kind of like it's schema, then I, guess or yeah.

C

Like it's, where it's almost like the index is like the table and not the feeder upper Garden for queries.

C

A

C

Are right about the data being, but um the you can tell elasticsearch to have like a strict and mappings are the way that you mappings is like the way that you have your schema defined, that's closer to like a schema. Our mappings are locked down. You can't just add new data to an index.

C

That's not already defined in like the mapping or the schema got it got it. I'm gonna, throw an error we have it set to like uh oh I, forget what the name the term is, but it's like it's locked down. You can't just say: oh I. Have this Json document I'm gonna. Add this new field like it doesn't work like that. You would have to recreate the whole index from scratch with the new mapping that has that field defined got.

A

It I've got one more question for you, then.

C

Yeah you can ask as many.

A

Questions what is um so? That's really really interesting. So just correct me if I'm wrong, but your this change is about we're just going to clear out the data.

C

A

Not Gonna remove these indexes right.

C

It's like yeah, so in the same thing with a I, don't know how the database works in the tests, but the way that elasticsearch is set up right now. It would be like if you completely deleted all the tables and recreated all every time you ran a test. Yeah, that's what's happening and I think that we for not all of them, maybe I feel like it has the potential to speed up any of the tests.

A

C

Have the colon elastic associated with them, but.

A

I thought the my thought, though, is with since these are deaf, since these indexes are like just an elasticsearch land different buckets second hand, somewhat unstructured data, I'm kind of like setting up indices for a relational database would be more costly, I.

B

This is just I.

C

A

And uh than just setting up unstructured data bucket and did what do you think is causing what do you think?

A

What do you think is the slowest operation that Alaska search is doing when it's creating all those indices so many times like.

B

Why why is this.

A

So significant, okay.

B

I, don't know, I, don't know it's interesting, yeah I, wonder I! Think it's the fact that they get deleted.

B

This change is not deleting the indexes, so it is probably the creation of the indexes that is, uh that slows it down, whereas this is, if you look at the two approaches, there's like clear, tracking delete indexes, create empty indexes, create migrations index and all of that stuff is not happening.

B

So to me this sounds like a great idea and a perfectly reasonable Applause.

A

B

If you only if there's a test that actually cares about, the is affected by the previous state of the index is not being cleared, and if that is the case, you can still flag those as elastic yeah, I, think my we did flag as elastic. Why did those need it like? What are they doing different than the ones that use the delete by query? Oh.

C

The delete by query we only created and applied to these four because we got Pigs by the engineering, productivity team or the quality team, or some team was like you need to fix these. They are the top slowest I'll see if you can find an issue. I see. Okay, so, like.

B

Theoretically, the elastic ones can all be changed. Also.

C

Theoretically but.

B

C

My thing was well, let me just change it and see what breaks and a bunch of stuff broke and so I'm, like. Maybe that's not the case.

A

Hence we're all here right now talking about.

C

Yeah so I was like, oh maybe yeah. I didn't expect this to happen. I mean it's not that many tests, but quite a few broke.

A

These are unit tests, so does this even affect the unit? Tests am I missing.

C

Yeah there are all the unit tests broke. I also didn't see, I guess what I don't understand is like. Is there an easy way to to compare the runtime of specs I? Don't know I.

A

Mean you can look at pipeline times, but.

B

There's so much in.

A

B

Too yeah and locally is fine. Yeah do two do like two in a row of each one to make sure your cashes are warm and whatever is running locally with.

C

B

C

Yeah, oh wait without GDK running well.

B

No, like don't stop your GDK in between oh.

C

B

Okay, Mario do like two runs and take the second one. So your.

C

Cash I want to see what all these failed, because I think it's all the same things and I'm wondering if that's oh, no! This is different. So.

A

We actually have a um oh I've seen this uh so long ago. We actually have like some profiling thing built into.

B

Ensure elasticsearch index like it's.

C

uh Are you looking at um one of.

B

One of the ones that had to have elastic in in the Mr- oh okay, hold.

C

B

um I mean I'm wondering if, like the index, isn't getting set up in the first place. But maybe you could just do only that in this case, but not all of the tearing down and it would and then they would still all work.

A

C

This is, um uh let me look at this in the code. Something will be easier to trace refresh index refresh index I think it calls. This is like an elasticsearch term, where it runs um like when things get indexed to elasticsearch they're, not immediately available for search, but it has this like refresh um process that runs I, think it's like usually every 30 seconds that will like flush all of the changes and make them available for search, but in tests.

C

We call it manually a lot of the time um to make sure that any new new documents like are visible in searches. Otherwise the tests would like fail, and it's not really a failure. It's just like the way that elasticsearch is working.

A

C

There's a uh oh yeah see that it's called there's two methods that we've used refresh index and then uh the ensure elastic search Index. This one is um to make sure that anything that gets cued up for indexing and redis gets processed by the workers that are called by the two Services bookkeeping and initial bookkeeping service, and then it does the same refresh that says: okay, anything that was indexed then now needs to be available to search. Can.

A

I ask a really crazy. You can throw a really crazy idea out there. No.

C

Yeah, this is good I, like these crazy ideas, because I'm going to take notes. Okay, so it.

A

Does like a last search has a level of like non-determinism, and this is what it sounds like one of the reasons why we manually call this method of saying, like you, gotta flush, your stuff, so our tests can actually work does allows to search, have any like like testing oriented server like a server that conforms to a last search, API yeah.

A

C

Like a like, a like a like a uh like a mode like a test mode or something I guess.

A

So yeah something like that, so it's.

C

A

Interacted the same way in our tests, but it's it's not needing to meet the same requirements that it does at production time, because our main goal is to get the interfaces right. Are we integrating with this correctly and so.

C

This is like could.

A

We replace this service with a test. Double have something like that on the box would be really interesting.

C

C

Yeah I, don't know, that's a good question. It'll probably be faster and.

A

I definitely wouldn't do that with like the end-to-end specs. But that's then, like hey, that full integration we get covered with the high high level specs, but then now we have a lot more Comfort writing fast unit tests uh and that's that's easy.

A

C

A

I I've I, don't know, I, don't know what other people do.

C

A

I, like my previous.

C

Jobs like people just didn't write tests, um that's the fastest test. You know that is awful. I was at somebody tell me that they.

A

Asked development fast Pipelines.

C

After I left the job they were like, oh these tests keep failing that, you wrote, so we just keep deleting them and I was just like.

A

C

I was very nice to not say anything, oh man, but it made me very sad.

A

C

Sad, oh yeah, okay, the profile thing, that's good I might do that and then I think I could probably um run it twice just to see where the slowness is and what the impact is.

C

um Should we question about elasticsearch being almost either as a test double or maybe some sort of like mode where we wouldn't have to do this refresh all the time. I don't know like.

B

An auto flush mode, yeah yeah I doubt that there's like a completely different cached version, but there might be a final flesh mode, but that might make it slower actually yeah I'm, just doing it on demand. Oh yeah, the uh I'm curious to see some of these failures. Like I mean we can look.

C

At some of them, I feel.

C

Like try to stay so I say a little bit, um I think what I was wondering and what you you all are asking me questions about it. It's like what should we do when we're testing things that should be creating indexes? A lot of these specs are written with the understanding that it's a new instance of elastic search um and so I can open up this spectrum just so we can look at one of these really quick.

C

The other one I know had failures was the rake task which is used to create indexes it's pretty much used to like, create and manage the elasticsearch indexes outside of things um so yeah. This is. This is one of the failures which is basically creates an index if it's not found um I'm wondering if stuff like this might need to have the old route like the old way. I, don't know.

B

Yeah, like anything that directly looks at the indexes, probably needs the old way, but everything else that is just caring about is this data found and used the new way.

C

I think that's a good distinction. I really want to rename both of the helper or whatever these things are called yeah.

B

More specifically, and also like it looks like it's not just deleting and recreating the indexes. The old one does a lot of other stuff, and it would be interesting is like which of those things is like the one that's actually needed and which of them are the slowest yeah and have you used a benchmark in Ruby.

C

um Yes, but I always have to look up how to do it right so.

B

That's a good way to say to see like which of those individual steps is taking the longest wrap it in a benchmark block. So.

C

Just for this setup really and see.

B

Yeah, like each of those things in the setup like just wrap a benchmark around it.

C

B

And see how long each one is taking and you can even like have you know, Global, like dollar sign, like literal global variables that you collect them in so then run the whole test suite and like okay aggregate, which of these steps is taking the most time like add The Benchmark times to them.

C

Yeah, we would probably need it here. It's almost like.

B

Yeah like around each one of those lines, basically and in the teardown too,.

C

A

A good idea, and then maybe iteratively so that we're not we're not making so large of a change which is causing maybe unrelated errors to the actual speed boost. We're trying to get that's interesting.

B

Yeah so like the the true lines individually and clear tracking too, oh.

A

B

Like the yeah and like the.

C

So I feel like.

C

You're aware, the time is being spent I just know that as soon as we don't do all these things around it, it's much faster yeah, um but I yeah I think it was a good idea to maybe look at renaming if I'm going to make this change renaming elastic to maybe like um slowly like an empty elastic instant like like almost like an empty elastic surge. You know, instance or empty cluster, or something like that and then like uh like a preserved or I hate me making too much like.

B

Maybe with like, with with full elastic tear down like.

C

B

That's that's good. The rails module naming convention with whatever yeah.

C

That's a good idea.

C

C

Naming my children naming my code been awful.

C

um This is cool, I might I, unfortunately, have to go because I'm in the working group meeting and I keep missing the uh Monday meetings because of the holidays. We've had in the friends uh family and friends day, um yeah.

A

C

I I really appreciate y'all walking through this of me, even though we didn't do any coding kind of talking through uh the benchmarking idea. I think is going to be super helpful um because this I feel like this is almost like a fun coding project. For me, I mean all opportunities like this is like kind of not so much something.

C

That's prioritized for my team at the moment, but um I don't know I kind of appreciate watching you Paul, like kind of just chunk away at things like and eventually they get finished, but it's not always like. Oh I'm working 40 hours on this.

A

Thing yeah: it's.

C

Something that you want to see like changed right so.

A

Yes, yeah just I have I I easily end up where I have like 10 Mrs open at once. I, don't forget about them for months and.

C

Then it's like.

A

C

A

C

A

Let's do this and then push it forward and.

C

Stuff but it gets done I.

A

Hope this gets done. I hope this gets done sooner than any of the things I've started, so yeah I.

B

A

This is thank. This is awesome. Thanks for bringing it up, I, I love, I mean Alaska search I, don't that's a that's something. I I, definitely don't know about, and I want to learn more about and so seeing the code that runs some of the stuff man.

C

A

So helpful for me to um get a more clearer understanding of what's what this must be doing so cool.

C

Thanks for sharing, I really appreciate it, but um if yeah, if I, think of anything fun on the Mrs I might tag both of you. But it's probably going to be a slow going. No.

A

Yeah, that's great because we just keep bringing it up here. That sounds good all right.

C

All right, thank you so much and then I'll see you later pairing bye.

A

Oh, hey Chad: uh do you wanna, do you wanna pair on something else sure okay, I got something cool. That's great!.

B

That's awesome, I, don't know if you have the answer to it, though. Okay.

A

It might not I have I have something too uh so, uh but.

C

B

A

Urgent-Ish, which is ironic actually because it's about query urgency, which we talked about earlier, what is yours about it's about The, Courier, urgency, stuff. It's not that it's! It's all right. Let's.

B

See you're saying.

A

We already talked about query urgency before.

B

um Yeah we're still recording, we are.

A

B

B

Sure there we go.

B

Get or the I'm going to use, get lab flavored and marked down uh snapshot. Examples like you want to have a across every run. You want to have a stable, like, instead of data, to run them against, like the same group, the same project with the same name. You have the same URL. So when you snapshot them, people run them on a different environment in a different machine. They're always going to be identical, like there's ways to ignore the changes, but I did.

B

Ideally we like them to have like the same IDs and and all of that 100.

A

B

B

It's really just what is the right way to do that, like I found it in there's tons of places where we like make seed data like here's, how you know what these are fixtures development.

B

And these aren't even what is supposed to be under a standard rails, fixture directory, there's something completely random. That is not what you know are, but um the the problem is like it wants to create a repo, and these need to be run in Dev mode, like you just run it from the command line, and it does its thing I, don't necessarily want it to have to use the test environment, um yeah.

A

Oh so it runs on the developer database.

B

Yeah I can just run it from the command line. I see.

A

B

A

B

Think the test environment, but.

A

B

Doesn't matter do.

A

You think it'd be helpful to not use models that are actually committed on the database, so just create.

B

The problem is, this: uh is.

B

This is a command line, script and speaking of test speed, the reason it's um I use to iterate faster unless I use, Fastback helper. So, for example um like how fast.

B

uh This one, for example, runs.

B

um You know it ran.

A

In 22. yeah yeah, I love that.

B

There's one test at the end here, which is the only thing that actually does the shell out to the back end and shell out to the front end and have them generate all of their stuff, which is the slowing like it's not time's slow, but it takes, you know, I, don't know 40 40 seconds or a minute, and so that's all like in a subshell which is this. You know this render static just called from a subshell.

B

And you could even um you know, Skip it if you want, but.

B

Yeah generate static, yeah.

A

B

Which just calls it at a in a sub show can.

A

You share an example of um one of the outputs that could vary across environments and runs and stuff.

B

um Anything that, like the last, let's see, yeah, there's not that many yet, but like here's sort of an example like on footnotes. I like hacked around this in a different way by like hard coding, the ID to 42, with the app.

A

B

Like the text of it, the classes, the name of whatever it's referring to anything that comes from the model and is included in the output HTML, which is lots of things for forget, live flavor markdown like.

A

Really like what like for uh the ref like a.

B

Yeah footnotes um like where's the example here in this one yeah like a a link to the wiki, which has the namespace the seven namespace, the the name of the file, the path file, the name of the file, all those yeah. So.

A

My here's- this is my um so you did sounds like you did something with the ID attribute already and my thoughts was I. May we could try tackling this of a making all of the data consistent across runs or we can sanitize our output.

B

I already have a way to do that, which is like there's this whole for things. That I can't do this for there's like this whole approach, where you have to Define regexes that pull out parts of the HTML output and has a you know, a numbered set of replacement regexes. But that's like Yay. It's a pain and you have to manually curate it so I want to do that for as few times as possible. Ideally I want to have the data be contrasted.

A

B

My thought was like.

A

What, if what, if line 58 inside of the ahrefs I actually just said, Wiki file path like it wasn't actually like? Is there a way for us to maybe just to stub out.

B

Yeah, I didn't even documented it. um If you look in here and you look at normalization like there's a whole approach that can these are all the sorts of things that can vary. Yeah.

A

B

You can um like the fine, regexes and reuse regexes to do parts of them and replace refs, but these need to be like curated on a per example basis right.

A

So I'm saying maybe if we could just do like a um like, allow method to receive this, is that thing so like doing this before we build it? We use these methods to build these paths. This.

B

Is in a rail stuff, shell, there's it's not a test environment, it's not a test. It's literally shelling out to rails. There's no, allow! There's, no stubs, there's no mock framework.

B

It's really yeah to really generate really call the API on the front end and the back end.

B

What is the point of it? You know it's a high level. You know characterization type test for the master type test, yeah yeah, you don't want to mock anything else. It's the real Behavior.

B

So I think this approach is fine. I know you like your crazy ideas, Paul, but oh.

A

I'm, just yeah I'm fine with getting the crazy idea shot down, but.

B

Yeah, my real problem is like this will work, but um foreign.

B

So many ways to.

B

Just like create a dummy group in a project and there's so many dependencies. A project has like it's got to have a repo and a path and like here, they're apparently like importing real ones and you gotta, sidekick and stub out all of this stuff. So, like my meta question was like, is there a standard way to I? Just want to create a dummy group in a project and where would I look to see anything.

A

In a database seed, so you want to seed a dummy group, a project.

B

I mean seed has a specific uh meaning in rails, which is like it's to see the development database, but I'm saying you know in a more General sense: I just want to create a group and a project and create them with the same data every time and if they already exist, don't create them. Well,.

A

If it's has more to do like to set a runtime I think it looks like there's this create service right. That seems to be the.

B

Yeah, which is the path I was going down to like stealing this code.

B

Which which I'll probably do? But um it's like.

B

Kind of unfortunate it's it's so coupled like you, can create a project in the UI right that doesn't have a repo and, as far as I know, these don't even need to have repos right. You can yeah.

A

B

But I'm like for that, and there used to be a um this.

B

Come on this skip storage validation, flag, which apparently stick the repo but I. Guess that like went away so.

A

Wait, the create service is expecting there to be a repo really.

B

It wants to create, like that's where it blows up like trying to validate that there's a repo and I just try to direct use. You know, rails uh active record credit commands to create it. The.

A

At what what line is it blowing up on? Well.

A

So it will 100 blow up if we're giving it like a project to import which that one file we were just looking at, looked like.

B

I'm gonna stash this stuff.

A

B

So if we run This falls back, you can see like I was just trying to naively say: hey find it by its name and if it doesn't create it and give it some stuff, and it blows up like repository cache, something something you'll see in a minute.

B

You see how slow the like, the actual, showing.

A

Oh, but you're you're using just create you, may not necessarily be using the create service. No.

B

I'm not right, so that's that's the path I was going down all right, let's use it that makes sense and I'm like. Is there like a standardized canonical way to do that? If I just want a dummy one and like those, those ones are what I found, but you can see here, it's like the the Italy client is trying to yeah your stuff and.

A

It's probably it's probably because there's a flag to opt out of the repository.

B

Exactly and that's what I was looking for and I couldn't find yeah.

A

B

Would be great yeah.

A

Yeah, that's a that's a good question. I really don't know.

B

And, like I said there used to be this environment, variable that looked like it did it and it just doesn't exist anymore and I tried to look into get history to see when it was deleted, and why and I couldn't find it.

B

I couldn't find where it got deleted, which was weird.

A

B

Looking across the project for.

A

Just things called we use this like create service thing all the time, so.

B

A

B

We use the service pattern like the controllers and then there's a service layer which yeah it's responsible for creating it. But I was trying to avoid having to do all that if I didn't need to. But it's it's possible.

A

Yeah, it seems possible that there's a lot of um there's a lot of baked in assumptions that you're going that route. You know.

B

B

So we go back down that route. Let's try it so like. What does this do? How does this feel right so yeah? This is what I was going to work on today. It's like well all right, I have my group up here, but if we went back and looked at um like what this was doing, this conditionally creates a group, and it's like.

C

This kind of does exactly.

B

What I need? Why can't I just extract this out? Yeah um I stole this whole thing. Maybe that's what I need.

D

um Hi um I've tried to to perhaps look into project Factory. We. uh We also uh tend to create repositories in factories as well, so I.

B

Think the environment. This is like the the live Dev environment shelling out to to rails Runner. So you don't have a uh yeah.

D

Yeah sure, but um as far as I understand, you are looking to create a repository right and uh of course you wanted to avoid the create service yeah.

B

You can always forget, you can use Factory girls outside of the test environment.

D

I just wanted to to uh point you at the possibility to look into a specific Factory and look what they are doing and.

B

D

They are calling project.create underscore repository if it's just about uh creating a repository without using the service. Well,.

B

Is that factory.

D

I think it's factories project.

A

I can send you the link to the uh I I just found it, but you may have already done.

D

Yeah, if you look search for custom, underscore repo custom, repo yeah, you can see online uh two to seven I think it is. It tries to look up the repository. If it's not does not create uh exist. It's just called create repository 227 yeah, it's a little bit hidden in the I suppose.

B

But theoretically, if I like require Factory bot, I should be able to just use.

D

This yeah, you can, you, can you can totally load Factory outside of tests, always.

B

Forget that let's do it I've done it before and I was like. You can't do that, and somebody told me yeah you can and then I forgot it again.

D

I tend to use factors a lot like when testing on my rice console, so it's uh it's already there, so it just use Factory bots on that. This.

A

Also implies that you could even get the um use the spying Behavior outside the test environment.

B

C

A

Javascript, because the test libraries and spy libraries are somewhat decoupled. So if you want to mock one of those methods or whatever.

A

It is possible to use that outside of tests.

D

Yeah yeah sure so so for aspect I think, like all the marketing functionality is a different repo from uh aspect core and I think you can use it outside of uh of aspect itself, so you can just load the library and it's very likely you need to include some magic module in order to have its like enabled globally. So you can do like stop or allow to allow stuff to so, but you can totally do it.

D

I'm not sure if, if this answers your question no.

B

It that's those are good ideas and uh what Paul and I are discussing before honestly. It may be better to force this to happen in the rails test environment, because then I can just blow away the database and because the other, the other thing like in addition to like these, are all the things that could be random and so I'm trying to avoid having to do the rejects.

A

B

For all of these things, so I've got a you know this list of things to do the other one is like: how do you control the IDS right, so you can like manually reset the auto increment ID to one you know in the database to ensure the IDS are consistent.

A

That that is a that is a good I think perspective. His do you do have a non-determinism requirement in the test. Environment has a non-determinism requirement, so it does sound like there's some sort of.

A

Potential for a healthy companionship.

B

Yeah I guess the reason I was avoiding. That is like I didn't.

B

So all of these dependencies, like the test database, is going to be there but like sidekick in the test environment, uh plastic search. Whatever else is needed during this. You know the course of these set up an API call. I think those are all started, probably in the tests. You know uh spec helper and not necessarily, or they just guaranteed to be there by the GDK running.

B

I, honestly, don't know you know Peter yeah, that's why I was avoiding the test. Environment. I was like I, don't know if all these opinions are going to be there, but I know they're they're in depth, so I'm just going to try to well. There is a difference between recording yeah that I want yeah.

A

As you know, like there's a difference between spec Helper and fast spec, Helper and even in our normal spec environment, Fastback helper makes the specs run pretty fast. So I know that it skips some sort of like at least giddly stuff and a whole bunch of back and.

B

Stuff, well, what fast spec helper does is it doesn't load the rails environment right, it doesn't load, ing.

A

Its lines of Ruby code right exactly and but you can still use like factories and stuff in the Fastback helper right.

B

A

Can directly Factory.

B

Box, let me chat. Okay, it's a whole different environment with a whole different configuration. It's here.

D

So so I I think you. You cannot use it by default, but you can try to at least try some of them to you so um because we.

C

D

Have a non non-active records factories, so you could, in theory and in practice you could use factories with a Fastback Helper, but for that you need to load manually, Factory Bots, but also the used models which again should not use any kind of rails code. So if it's just like some in-memory models, then you can use it.

B

A

B

What do you mean models.

D

um Wiki page, for example, I'm not sure wiki page has any kind of.

B

Yeah like whenever you try to use anything from rails to controller or a model in the fast helper, you end up just going down this path and Madness trying to require the world yeah.

D

Right right, but uh you can, you can try actually to to only require the pieces, you really need and it would be still faster than uh just uh requiring rates and what all comes with rails like or active support and all this stuff. So.

B

That's not really the approach this takes like all of this is being run via um rails, Runner.

D

B

A

B

Literally, invoking this this script through rails and there's no test environment or anything involved, it's just well run this.

D

Yeah, so sorry for being late. So what the task attendance was to there's a script which kind of creates snapshots.

B

D

B

Big picture here is this: is the um the get lab flavor markdown specification, and this is sort of a that serves many purposes, but the it's performance testing to say that our markdown is always rendered the way it's supposed to be on the right end, the back end, but also there's this uh characterization or Golden Master testing Behavior to it.

B

You may have seen some of these, but it's all of the um like for every possible markdown example and there's going to be like you know, between 500 and a thousand, it renders the uh static or back end and front in HTML, but the problem is the API calls, and especially even for different contexts like a single API call can be in a Wiki context. It could be in a like um if it's on here, no.

B

Yeah here, for example, for a single API call, depending on the context you pass and surrender differently, maybe for wikis Snippets groups and projects. So that's why, in this setup, I have to have you know at least one of each of those things for when the API calls it'll do it.

B

But ideally it's got always exactly the same path, exactly the same name, so the vhtml it renders won't be variable, because there is a way here that I can deal with variability, that's built in which is this normalizations approach, which is like for everything for every example. That does that you have to specify regex and what the replacement values are for that Red X, which is a lot of duplication, and you can dry it up by using yaml anchors. But ideally this is all manually created. We don't want to do any of that.

B

Ideally it would just be exactly the same record with exactly the same data every time, and we don't have to do any of this. So that's specific, compare.

D

Okay, so you want to kind of create.

B

Non-Violence to.

D

Say the same environment every time right: okay,.

B

So, just the same records right, the same records that I can refer to a given whatever in the whatever API call. It is we care about, but that record it refers to needs to be identical. Every time.

D

And um you you want to do this in in our spec Suite or like manually or.

B

Products the way it works is this: there's a there's, a command line, script; this which does it and for the most part it.

B

If, in order for it to run fast, all of the rendering is done by shelling out and calling for the back end, this rails, Runner, um which says hey, uh read a file: that's got all the markdown. They need to render and iterate over all of them.

B

However, many 7 800 we end up with rendered them out and dumped that file, and then the script picks it up and it's the same same approach for the uh front and two except we shall have to call yarn jest to for all fronted ones. Okay, so, interestingly front end, since just is really good at completely setting up an environment just like webpack would.

B

uh This is a lot easier problem on the front end, because it's Jess does everything for you, but on the back end, it's like well there's so many service dependencies and states mm-hmm.

D

Okay, so um basically you want to get rid of all the normalization by by being able to to.

B

I want to minimize the need for normalization. Okay,.

D

Can you show the list again which which one could we tackle immediately of the list, the the list from from the docs yeah? This.

B

One right I mean.

D

You want to minimize.

B

Something really it'll be most of these that we're getting rid of this one I think the only random value was uh the footnotes and I fixed that, with, like a an environment variable that I plug in to hard code, the name the value of the footnote. So it's really these which come from you know the group The namespace, the file names, the wiki, the wiki name and stuff, like that attributes of the records themselves.

D

D

Yeah I I I was wondering like I. Imagine you you already explored this like what happens? If, if you run the script twice, should it like create the data twice or just rely on existing data, meaning like if you create a project with a record identifier.

B

D

Recognize it's fine also means ID. So it's what's like potentially conflict with your existing local IDs app because, like usually, if you set up gtk, for example, you get like 20 project created right. Unless you are doing like more enable more project ratio, it takes an hour to trade or whatever. So.

B

Yeah, okay, to not recreate them, it's like a find or create, uh and as far as the IDS I have a note about that here. What I was gonna do. Is uh this all right, the primary key to an arbitrarily High number like just set it?

B

You know the next primary key is ten thousand and uh like the a previous project that I worked on, we we relied heavily on Rails fixtures and instead of you know like Factory bot creating over and over all the time we relied on Rails fixtures, which most people don't like to do, because they have to curate the ammo files. But what we did is use object mothers to generate the yaml files, and it was a a very similar problem in that you wanted them to always be consistent.

B

So the way we did is like uh here's, my sequel, but you can basically hard code what you want the next uh Auto increment for a primary key to B, so I'm, assuming there's got to be a way to do that in postgres. So that's what I was planning on doing to force the ID to be whatever okay.

D

So these very stable, but it was just like uh record IDs as the.

B

D

Yeah but but the other one like past name and so on, you could in theory or group, name or whatsoever. You could just Define right and.

B

Yeah, which is what I'm doing here like I, got yeah yeah the names for the group for the path. That's the plan, so I think that's a good idea. I think the what I came away with is like I, probably should use the test environment uh just to have a clean sleep. Then maybe I don't have to worry about setting the IDS and.

D

B

Look at using the factory bot to try to do this, and maybe, between those two things. That's all I need.

D

Yeah I would probably also verify that you are really on a clean slate, so like the very first ID should be one and not something or some some other number. You come up with right so uh just to.

B

Make sure that'll be very clear: all I got to do is run the script twice and if it comes right for different numbers,.

D

Yeah I didn't get it right. Yeah! That's true good point.

A

B

A

That's good conversation.

B

We'll see if the tests are.

A

Hopping on thanks for hopping on Peter sure, better late than than ever, I.

D

I want to come earlier about, like sometime life's life, does not allow someone to join earlier.

A

I I totally understand. Yes, there are um because many times when I'm on a meeting and I was like. Oh it's been five minutes no one's showing up so I I close it out. Then someone shows up like at 20 minutes later I.

B

Was like, where is everybody.

A

B

That the grass Railway I did I showed up with about 20. After that.

A

Did you really.

B

A

Waited for like this is really fun. I.

C

Wasn't just sorry.

B

A

Was being productive too? Don't don't tell my boss.

B

It's hard to make it. It sticks.

A

In the shot, those those meetings have gotten really thin. I need to just I need to sleep right here. I know we have other pairing meetings, so I need to just cancel those ones. Anyways thanks. Everyone for hopping on all right, see you around. Thank you.