GitLab Digital Experience, 8 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Pair programming Google Analytics utilities in GitLab codebase

Description

Miracle Banks and Tyler Williams talk through some changes to the Google Analytics utility code in the GitLab Enterprise Edition codebase. Covers the changes to make, and goes over some fronted test-driven development practices, and other tips and tricks for working with the GitLab product codebase for engineers on the Buyer Experience team

A

Cloud, hey cool, recording uh super impromptu, so you know um extremely uh uh unfiltered, but it's going to the unfiltered channel, uh so we're gonna talk about um uh so this is primarily aimed at like people on the buyer experience digital experience, team working um in the gitlab product code base, which is not sort of like our normal place that we hang out in uh so- and this is something that I've spent a good amount of my time getting up to speed on, and uh you know, there's just a lot, it's just a different context.

A

So this might um I basically I've got a thing. I'm gonna do and I'm gonna like have miracle like watch me. Do it and ask questions as I do and I'll sort of like talk out loud as I do it so we'll go like a little bit slowly here um before we get started uh and like when we get started I'll talk about what I'm doing. But before I do uh do you have any questions, miracle, cool uh and I'd- ask the audience but there's, but they can't uh ask me contemporaneously.

A

So uh let's share the screen um cool, all the all right cool um sweet. So uh this uh this issue is more confidential, but it's not actually um that was erroneously done. So don't worry about it. If you see that little thing there, um so I'm working on so we have uh so in the product we have places where we have google tag manager and google analytics running.

A

We also use uh snowplow for analytics in there too. So these are two different. They run into different contexts right um and the analytics team wants to be able to join uh some of the google analytics data with the snowplow analytics data uh and in order to join them, they need to share some attribute, and so um we think that, uh like a good attribute to do so um actually already lives in. uh We have this in uh snowplow uh standard context, so um I'll show.

A

So it's even here it's it's kind of it kind of hangs out everywhere. um You see this number one, three, four, six, five, three five three is the like:.

B

I don't think you're sharing the correct screen. I see you see what I see a really cute screensaver. Oh.

A

My gosh, uh well, you can.

B

All see my fiance.

A

And I uh in our engagement, photos uh desktop2 hello, do you see this or do you see my screen.

B

No, I see this okay. That makes way more sense because.

A

We're talking about.

B

Confidentiality- and I was like I guess he means his engagement photo nope.

A

Nope well, anyways uh uh yeah! That's that's that!

A

um So what I was talking about was this issue here um and again we're joining data between our google analytics and snowplow, um and there's snowplow already has this namespace id, and so we want to grab it.

A

So before I dig into the code, uh I have been like back and forth on this issue for a week now- and this is important because um developing in the um the gitlab product with the get lab developer kit um is a much slower feedback cycle than we have in like our next repository, because as it should be, it's a huge code base with you know thousands of contributors, um it just like there's more moving pieces right, um and so you know your mo when you're working in the in the product is mostly to shorten your feedback cycles uh and the best way to do that is to lay the best plans you possibly can.

A

So um I've done a little bit.

A

I've done a little bit of coding like exploratory stuff, but mostly I've been talking with people to like hone in on like what do we want to do because, like if you make a plan, that is not the right plan um like you're gonna you're gonna have a bad time like it's just like it's gonna, take a long time to find out that you've made that you're executing against the the wrong plan, because it's gonna take just a longer cycle, um so spend a lot of time planning and talking to people and like really digging in on, like what they're asking you to do, um especially if you're working with stakeholders in the buyer experience team because like uh because this isn't really our code base, the art of the possible is not so um uh not so readily available.

A

People don't really like know what we can and cannot do right, and so the requests are very high level and sometimes lack the clarity. You need to make the right choices. So I spent a lot of time talking about this, uh both in this call and also before I even sat down to do work. um But I now know what I'm doing, I'm like extremely clear on what it is, which is basically so we have um this.

A

uh We have this event that we we, when we, where are we transactions yeah um when people make transactions on the.

A

When people buy stuff here, um we fire this transaction um and we track it in google tag manager and now what we want to do is we want to add a key called dimension 36, and we want to give it some namespace id that matches snowplow here right.

A

That's it once that dimension 36 goes in and it's not an arbitrary name. It's like it's meaningful in google tag manager, um even though it looks somewhat arbitrary um once this once this starts getting pushed up to google tag manager, our enhanced e-commerce analytics can like join its data against what we have in snow plow.

A

So the two analytics teams, people working on google tag manager and google analytics and people working in snowplow can begin to like have an entirely shared context where they can say we're seeing this information like across both of our tools, um which is gonna, be super useful right.

A

So that's basically it um but there's a lot of steps to doing it so um I'll stop. There does the thing that I'm doing makes sense to you.

B

A

Cool, um so here are my tips for what I mean, I'm just gonna show you what I what I do when I work on here, um I usually grab just I just I love markdown. It's like my favorite thing, so um I just have like. I just call it scratch.md.

A

um I just have this in like and I have it. I have a repository, that's like outside of the gitlab thing. It's just like it's like notion. You know like, like it's useful, to write something down outside of the code. You're writing raise if you like, stop and start get interrupted or just like need to think things through. So um I've already done a lot of that thinking in the issue over here um and I've confirmed stuff with jay and dennis and like here.

A

We are we're going to use this value so then my scratch feels like okay, let's use whatever in in this value here um and so like the the actual first thing I might do is like sort of, like I say, like okay, like find where we set snowplow standard context. This is actually like. I would literally like write this down for myself and then um in my in my vs code. Like workspace, I would you know ctrl shift f, for let's just find it snowplow standard context.

A

um Assuming that I'll be able to find and hey there. It is there's a bunch of different files. I've actually already um sort of found some of this stuff. um So there's like there's already some spec files uh we use just to like write tests and stuff for the front end. So the nice thing is, I can sort of see like I'm looking for stuff, I can copy right like I can see patterns of like okay, here's how they're using just to test stuff about the snowplow standard context, because it lives in the front end.

A

It lives in the wind, the global window object, um so that makes sense. There's uh getting the standard contact spec, something about track, initializing, tracking um and really I'm sort of looking for like okay, like is there a namespace id? What is it namespace id not in this one namespace id? Not this one.

A

Maybe it's space id right. I think that, actually it lives. Oh it like it. Drills down, there's like we cr in this um haml file, is where they create the the window.gl object, and then they provide the standard context with, um and this is all haml.

A

So it's like embedded ruby here right where, like there's this standard context, ruby thing uh name space id hey here it is this is so basically it takes um like the the the two hash function here, um like the two context function, I think, uh like probably wraps this 2h um and it takes the namespace, finds the id and drops in this namespace id thing, which is what uh ends up in uh here right.

A

um So all that to say like this is just me sort of like it's exploratory right and that's kind of like what I I've already done, that here and I'm sort of like getting started, doing the actual work. So now that I've sort of like pulled on a thread and I've been like- I I like how this again, it's the art of the possible, like think about like like, what's already out there right. um What like? What can I use? What can I copy? What can I emulate? um What do I need that doesn't exist?

A

That's like that's always a big one to find like if, if this didn't exist, which is a lot of my work in the past couple days has been like finding it at all and like before. I knew that it existed. uh I was like oh jeez. This might take a lot of time because it doesn't, I don't know, I don't know if we can get this data we finally found it. uh Thank you, jay and dennis for helping me find that so um cool, so you know write it down.

A

Do the thing uh and I'll delete that, because it's not so useful here um so the first thing I do is: uh let's start with tests right um miracle. Are you familiar with the idea of red green refactor? No, oh okay! So um in test driven development, uh there's this idea of like red, green refactor, which is basically if you're doing tdd, and you don't have to do it very strictly. I mean there's there's a lot of value in like exploratory coding before you start writing tests. I just like.

A

I already have this in my head like if, if you're doing a thing, that's very well defined and you have a clear plan on doing test driven development uh is a little bit easier to do. It's still like kind of weird, but it's really useful and it helps you in your. It helps get your merge requests approved because uh we try and test everything in the gitlab codebase really or practically. We test the things that matter, um which is most things uh and so red green refactor, is the idea of the before you do anything.

A

The first thing you do is you write a test that fails right, um because what you want to do is you want to write a failing test? Then you want to write the code that makes the test pass and then you want to clean up that code and make sure your tests continue to pass and what that does. Is it stops you from getting false positives? uh It's it's very easy to write a test that accidentally passes, sometimes because you're not testing the right thing, and so that's actually what I've done here.

A

So um we have this. uh Google tag manager index file, which has a bunch of utilities for running google tag manager stuff in the gitlab product on the front end, and it has an associated underscore spec file and there's a bunch of tests that already exist here. um But I'm just going to focus here.

A

So I've made a new description block, which is you know for getting the namespace id from the snowplow standard context and there's going to be two contexts right, there's going to be when one exists and has a value and when one doesn't, uh because sometimes because I'm anticipating that it might not- and so I want to define both of that behavior. um So I'm gonna start with the with the happy path which is just when there is a value in window.

A

Gl, snowplaster context, data in the space id right, and so um so just runs in a node process because it runs in a node process. It doesn't have. It doesn't actually have access to like a browser. It doesn't have browser apis like window, so we actually like create a window variable um right, like it's, just a global window, variable as like a testing mechanism, because it should basically function as though this javascript code was querying.

A

It was like talking to a window object because it's just an object, um so I'm actually going to set this entire. uh I'm just going to like set this up to be like a fake namespace id of one two three and then I and you'll notice.

A

um I haven't even written this function and it's even red because it's like hey, I don't know what this is. This is my red on the red, green refactor um or you know part of it. Then I'm going to run a just spec down the terminal here, make this a little taller and you can you can target specific spec files if you run yarn jest you'll run them all, and it's a lot.

A

You don't want to do that and again back to the one of the first things I said is like you want to tighten up your feedback loops right, um so I don't even have gdk running uh the nice thing here is you can just run jest and they're isolated, they're unit tests, so um here I can run this test and you'll see if I already ran it before uh so it fails um and.

A

You know it runs. It takes a minute. Let's go, do something: hey there's my failure. um It'll sort of tell us something about like uh get namespace id is not defined right, um so let's define it. uh Export const get namespace id and again red green refactor so like let's keep it red, let's not make it work. Yet, let's uh return, let's just return null right um and then oh. I have to actually import this here.

A

I'll stop here is this: does that make sense like what I've just done is like cool test, fails right stuff to make it pass or write stuff to get past the first failure, um and then we should be failing based on not just on like the mechanics of like. Does this function exist, but now it should be um we're failing on the wrong value right. We expected one two three and we received no cool.

A

um So let's make a pass, uh you know we can do. We can do something super simple right. We can even literally just return one two three um just to get it just to get it green. To demonstrate- and again this like demonstrates that like hey, I could pass it um obvi. This is now we will get into the refactor part. So, like hey, it's not working.

A

Perfect, okay, this is actually this is actually great, so this is why I do red green refactor because, like I'm testing my assumptions- um and this was kind of silly- I thought that uh I thought that.

A

This existed and it doesn't so it's like failing on trying to add stuff to it. So uh do I have window.gl.

A

This is before each block that does stuff.

A

See if I can get this to pass.

A

Okay, cool so that pass so the snowplow standard context works. So then let's make a dot data and what did I want to do and I'll set it equal to an object and let's give it namespace id one. Two three.

A

A

Okay, pass now.

A

Oh you're, not happy with me. What's up, you know, set properties defined setting date. Oh interesting, okay, uh window.

A

uh Gl equals snowplow standard context, data and interesting cool. Are you happy with me now again? This was this is stopping me it's also stopping me from getting like false negatives. We're like like I'm gonna, like really hey there.

B

We go okay, cool.

A

I just needed to like mock this out a little bit better, um so yeah. Now we've set up this thing. The snowplow standard context didn't exist before now it does. It has a sub objective data with a key value of namespace id123, um but you'll notice. I it's you know now it's a false positive, we're hard coding to return one two three, um you know if I set this to three two one, we would get a failure and again this is just part of that red green reactor cycle.

A

It's like, let's make sure we're testing the right things, we're not just like asserting true like sometimes you can just be like assert that true equals. True and hey that's interest! Oh yeah cuz! I didn't change my assumptions, sorry, I'm coding and talking and I'm like thoroughly unprepared for this. So you know, mistakes are bound to happen. um Hey there. It is okay, so that failed so now um and you can actually yeah. That's fine, we'll just keep this hard-coded. um So, uh let's see return.

A

Interesting window.jl.snowplow standard context, stuff data dot, name space id. This is not how it's going to look at the end, but I want to see and again this is why we're going to test this be super fast, like you'll notice. Again, I'm not doing this in the browser, I'm not like constantly refreshing, I'm not like having the rails app reload, I'm like really testing like one thing and that that keeps those feedback cycles super tight. um So if I do this, does that work, uh let's find out.

A

Yeah, I think it should work it doesn't. Why not at least one assertion to be called but was not found. None cannot read cool okay and that's because this thing doesn't exist. I think my like autofill gave me something incorrect. There.

A

I think that was just some typo and I think this should pass hey. It passes right cool, so uh now we are so now. um Let's do the next thing. Well, what's uh you know? Let's see what happens if um this thing, if that value doesn't exist, which I think is a pi is, is like a thing that can happen um so we'll describe when it's undefined.

A

A

A

um What do we want to do so, let's see what happens if it returns, placeholder value.

A

So, let's expect get namespace id to equal, uh not available.

A

Just have a really explicit like null value here, so we're going to run the tests again.

A

I think I can. I wonder I I might be able to hey look at that. So um this tells me a couple things um one this before each block is getting set um and staying set because it's a global variable. So I think that, like there's a side effect here and that's fine, um we can do.

A

B

A

Let's just make.

A

An empty object.

A

Because the way it failed was it failed by giving me a three to one instead of not available. This is again really important. The red green refactor, I'm seeing like what's failing, and why and now here we go- I'm actually yeah. So now we're actually bailing out on um on the actual execution path right.

A

So the error here is actually on line 281 in the source file of index.js, where, because this no longer exists or the data no longer exists, because window.gl is actually a an empty object right and like if you access this, this ends up. So it's like uh window.gl exists is an empty object. Snowplow standard context will come back as undefined, so it's not an error.

A

Yet it is like a valid thing to have an undefined attribute and an object, um and then, when you try and access data on undefined undefined is like whoa, I don't know what dot data means um and that's what the error is there. So uh this is great.

A

This tells me that where this is my red and now it's time for my green and then I can sort of refactor, um so I think the thing to do here is just uh do a little um or operator and say not available right, and we might even do let's make a little constant up here. Well now I'll save that for refactor uh yeah, we'll yeah, we'll say that for a refactor, that's an internationalization thing and then see if we're green or not. Yet um what broke? Okay.

B

A

Is actually tossing, so let's do um okay.

A

Do a couple things.

A

Let's do uh const gl equals.

A

Window for an empty object, const snow class standard context equals a gl or an empty object. Const data equals snowplow standard context or an empty object. Const namespace id equals data or an empty object.

A

Yeah and then we'll return, the namespace id.

A

Are you mad at me? Oh camel id, uh that's fine! It needs that's just what it needs to be um cool. I think that'll do it so now we're like decomposing the object many times, and that should stop me from running into that problem. Hey it passed right, so we were red. Now we're green. Now we can refactor.

A

So do a couple things to refactor. First, is I'm going to make this string a constant at the top here, just to communicate like const or saying empty namespace id value is equal to uh not available.

A

And so this just communicates to people in the future that, like this, isn't really a mat. This is not really like a magic string or anything.

B

A

Just like whatever value we want it to be should be that it should be relatively constant. We can change it, um we shouldn't, because we want to keep the data pretty like happy, um and then I've got this little enter error, which is the camel case thing uh and we're gonna disable it for this line, because um this mat. Oh actually, I wonder if uh what's the thing when you rename a de-structured thing which comes first, is it?

A

Oh? It doesn't matter.

A

I think it's namespace id right. Are we happy now? Hey we're happy now, okay, cool, so we I don't even need to do that. um So again, this is the refactor. So what I did there is.

A

um I took the uh if you're destructuring an object, but you want to rename the key that you're destructing from you can use it as like a key value pair and you put the original key first as the key and then the value is your new name for that key uh and I always forget which order it comes in, but there's only two options. uh So it's pretty so you know it's quick to yeah. Why remember it um when it takes when it's so quick to do so now we did red green refactor, we refactored.

A

We should still be green. Oh- and I wonder if I can that's I was I'd- was half half form thought 15 minutes ago was. If we have it, set up to watch yeah. We sure do um awesome. So two things here: one we're still greed, even with the refactor two. There is the dash dash watch command, uh which will watch files for changes and run the stuff, and so uh let's make a change uh to an empty string and hey look, it runs automatically you don't have to.

A

You, can save yourself some keystrokes there and look we're we're red again.

A

Cool, um so that's testing my get namespace id, um which is only part of the puzzle. The next part of the puzzle is, uh I need to uh hop into so I'll. Stop there, because that's a really good, stopping point um questions.

B

No, I was following you: oh cool.

A

um Questions that you had in your mind that I that you answered yourself or that I answered by continuing to move on, we should in case there's other people watching this. um Were there moments where you're like oh like why or what.

B

Yeah, it was when you had the typo. I was wondering why you would change the name to global on the right side, and then I realized it was a typo yep.

A

Okay, cool um well, sweet. uh We can just keep on keeping on um so then up next is so. This is just a function to like get the namespace id. That's all, um and this will be useful because we'll actually want to use it.

A

I think I there's a bigger picture here about what we're doing the namespace id that's outside of the context of this issue, I'm working on, um so I'm going to leave it as an exported function because I'm going to use it elsewhere for other stuff, so this is sort of like, but this is it's important like this.

A

Is the iterative thing like as much as you can you want to do very small, merge requests in the product code base and anywhere, because it's easier for reviewers to reason about um people will not share contact all you know you will never get a reviewer from our team for merge requests into the product code base, which, which means that no one will know. No one knows what we're doing like like you're only going to get people who have no idea what our project looks like. So you want to make like really small.

A

Merge requests so that people can like reason about it as effectively as possible, right, that's kind of the goal here so um into that. To that to that end, if you've got like a big like, I have sort of like a weeks-long series of things we're doing to join this data together. um I'm gonna do this and I'm gonna make like one more change in my track. Purchase thing, so there will be basically two blocks of changes in this merge requests. However, I know that get namespace id is gonna.

A

Be super useful for like two three four merge requests down the line for me, um but I don't have to like bundle that together with that, because it will already exist so as much as you can- and this is there's advice in the development documentation about like sort of like unbundling changes and being like hey if you're working on something like you split it up to like back end changes front.

A

End changes like what can you do to make it smaller, but still, like you know, code that can be tested and deployed uh and, like sort of like built like you know, build iteratively. So that's that's one of the advantages to what I'm doing here is like this is like a small function and um and I'll use it later so um cool uh up next is so we're gonna end up using this in um here. Let's make our test, uh oh yeah, sure.

A

Yeah, let's make our tests fail again, so I need to get that namespace id into my track transaction block, and so I will because I don't think exported constants get hoisted like that. If they're arrow functions, I'm actually going to define it up here. uh This is a refactor, and so it's green. So that's good um and then let's do in the track transaction. This should break my tests. I think um uh so.

A

Const namespace id equals uh get namespace id. Then.

A

Oh, um this is silly, but I'll refactor it out later. I think this actually needs to go into the product block. It sure does so. Let's uh expand this.

A

And give it uh this is dimension. 36 is the key namespace id is the value. Why are we red? Oh because I don't have a closing curly brace cool. This should fail boom. Look at that it failed it's exactly what I wanted it to do, but I'm testing my assumptions. I assumed it would fail because I I wrote these tests you know months ago so, like I know what the tests do, um this test is testing a hard-coded uh like um uh what should we call it?

A

A hard-coded object, that's sort of like a golden object uh of like what it expects, the response to look like, and I just modified that object by pushing this dimension 36 key into it. Regardless of whether or not you know this is not available, even if this was pulling a number as I ex a number as a string as I expected to, I would expect this test to fail because it's it's new. It's not part of the the golden object that I had at testing against before.

A

If this test had passed, I'd be like that's weird, um so it's really good that this test failed and again we're back to the red green refactor cycle, so we're red uh and now I'll make it green in this case, the way to make it green is, I just have to go and update the the assertions that I have um in my.

A

In my tests- and so this is failing on uh when track transactions is invoked, and then I think we expect these things to look like this hello.

A

Products boom category, so this is this is not where it's going to end up um dimension, 36 not available.

A

Why are you mad at me, okay and then another one, when it's invoked that references the select plan when track transactions is invoked. It's an invalid plan object. What else do we have is that it there's two things it looks, pushed let's save it and see.

A

If it passes hey it passes, okay, cool, so um what I actually need to do next, uh but I'm gonna sort of cut it short here, because we've been going for a little while uh and I gotta like take a break, walk the dog, and I think this is sort of like the rest of this is just it's just more work for me to do right um and it's until it gets a little repetitive from there, but um then we get into the refactor stuff.

A

So, um instead of just like, I need to like set up tests that will test different contexts when there are and are not uh namespace ids available to sort of like thoroughly test all the execution paths on this and then I'm going to refactor it a little bit because, like I've, sort of made a mess of this, because I've just been live coding on uh zoom rather than like sitting and thinking quietly about like the thing like about style guides, so like what I'll end up doing is I'll go to like um get lab.

A

Javascript style guide right uh I'll, go here and I'll sort of like um you know I'll check this against myself and they say when you make a merge request. You know the best thing you can do is actually review. It be your first, your own first reviewer. So once I have the functionality working, uh the tests are passing I've refactored it I'm gonna, um go I'm actually gonna start up the gdk and do a local server, and I'm gonna like test it manually too.

A

Just to like make sure that, like this stuff translates to reality, um sometimes it doesn't, I'm gonna make sure it translates to reality and then I'm gonna go through I'll, probably make a merge request. So I can see all my changes run. Ci make sure I haven't broken any ci stuff. um If that's all passing before I assign it to a reviewer, I'm gonna do my own review for myself. I'm gonna, I'm gonna like go through the javascript style.

B

A

Like make sure I haven't like made any like uh clumsy errors that are like that are people are gonna like burn cycles. Telling me like: hey, don't destructure like this right that I expect them to want me to destructure differently um and like it's not worth like they've already someone's already written it down. It's like not worth our time to discuss it. If there's like a preferred method to do it, so I'm gonna go look through the style guide, find it and uh and do that.

A

So cool: um do you have any questions as we wrap up here.

B

No, but thank you for showing me the style guide. That'll help a lot.

A

Yeah, there's a ruby style guide as well. There's a scss style, there's a bunch of different style guides in the same set of documentation. So um yeah. I think that's uh that's about it.

A

So you know keep your feedback uh cycles short as much as you can uh plan as much as you can write tests as much as you can, but like also keep in mind that, like I, am showing you this after having like, sat and thought and talked for like two or three days so like it's, not all gonna come off the top of your head, like that, and.

B

A

Also working from a bunch of existing files uh where, if you're doing like net news stuff, you're gonna have to sort of like navigate through like where is the right place to put this and like what do I need to like bootstrap the file so that it actually knows what's going on?

A

Okay, so it'll come with time it'll, it will come with time and uh you're going to get knocked down, and you just pick up pick yourself up again. People are super nice, too they'll be they'll, be really gentle. um Like you know, the nice thing is the worst thing you can do is get feedback like like you know, you'll write some code, you'll, try some stuff, uh you'll push it up and it'll work or it won't work. It'll be like up to standards you're, not up to standards.

A

uh In your word very worst case scenario, someone's gonna be like hey, you need to make xyz changes, and then you will know the things that you need to change about it and then the next time you do. It will be easier. So uh you know it's like very low stakes for not knowing what you're doing here.

A

B

Well, that makes it.

A

More better yeah, well, I'm going to end recording and then we can sort of log off.

B

Sounds good, thank you. So much.

A

Yeah see ya, bye.