GitLab Handbook Help, 28 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: More Powerful Find-and-Replace in the GitLab Handbook

Description

Derek shares with Monica a high-level overview of what regular expressions are and why they are useful. This chat is geared for those less familiar with programming.

A

Thank you. It's just helpful to have in case I ever been sure to do this on my own.

B

Sure so, um and if and if, depending how uh successful this goes, I can we may just end this and then I'll record it once I'm actually kind of in the groove or something. um But I think at a high level, we should be able to solve this so um just to take a step back.

B

The goal of this is um to basically do a find and replace in a more efficient manner, as opposed to manually, going through all the links and selecting them and replacing them with the corresponding new destination link is that is that your expectation.

A

That is correct. Yes,.

B

Okay, cool so yeah, just again, the small amount of background of initially chatting with you monica I'm obviously talking to whoever's gonna, be watching this. I guess well, which is kind of weird, but it is what it is. So um basically what came to mind almost instantly when you initially shared this, I think in slack was basically we can use regular expressions, which is basically a um it's very cryptic, which is the lame part um to learn it, but it's really powerful.

B

Obviously, and all it's basically a way in all kinds of different languages is to use this kind of meta language. It's called regular expressions and all it specializes in is finding patterns, so this regular expression, language basically allows you to.

B

Create a um a essentially a pattern that you want to find within a huge amount of text or a small amount of text. That's why it's powerful too right as you can iterate over tons of files. It can be used over your entire operating system in some cases.

B

um So, for example, a really good use case would be say you have a uh this one might connect so say. For example, you have a digital camera and you and you you know, store the uh folder on your computer or something people might not do this anymore. But this is an example.

B

Maybe everyone uses the cloud now, um but you could technically do this on the cloud too, like it could do the same thing. You can basically say every single image that starts with ing underscore. You know those there's, these common patterns file names just get. You could replace that and also automatically um kind of do a batch renaming where you could say, remove the image dash underscore and just call it one and then two and three and four. So you can replace every single one which again might not be a big deal.

B

If you only have ten images, but if you had hundreds or thousands or you know, I'm sure that's basically where the power is um okay. So what we'll do here is, um I don't know? If we'll, I could have prepped a little bit better to be honest, um but we can give it a whirl. So basically, what we'll try to do is- and this might be, um what is it called regex or yeah sweet? So there are.

B

This is a a website and there are other ones like this that allow you to kind of in real time test this. So it's a lot quicker um which, as you might expect, when you get quick feedback, that's where the winds are so sure so up here we have what's known as the regular expression excuse me and down here. We just have some body of text okay.

B

So what would what- and this is actually what I've done on the static site. Editor, in fact is. I would take the um let's just find a uh yeah. We actually have the right file, so this is- and this is all in a single file correct all these links that.

A

Is correct? Yes,.

B

Okay, so that's just an interesting bonus to be honest, um so we can at least focus our efforts here. um Naturally, you could expect if you, um if your pattern, if your regular expression wasn't uh good enough or it wasn't tight enough, you could act and you ran it over a bunch of directories and not just one file. You could obviously the with with uh great power, comes great responsibility right. You could actually do that this uh find and replace on more than you'd expect.

B

So that's something you got to be kind of careful about, but since this is in one file, that's actually even better um what I'm gonna do and if you have any questions, feel free to.

A

Okay, yeah, that's creepy, to be able to like mess everything up that. I think. That's what that's that's! What streams me away from most of this stuff yeah.

B

A good sandbox and that's why exactly why, as developers, we have developer sandbox right, like you have your you: do it locally, so you can kind of mess around um and iterate towards a solution. Basically, so what I'm going to do right now is just copy these file contents.

B

You wouldn't want to do this if there was sensitive information here um on a public site like this, uh it's just something to be aware of, um and since this is just a public uh set of con like, for example, if you had passwords here, this site might actually record what people look at or like what people paste in here just so it can learn.

B

Does that make if that makes sense, but we don't have to worry.

C

B

That's just like a small thing, um so we.

A

Can see right now, good question, sorry, just not to interrupt you but quick question so since so the use case for this definitely is like getting them replaced from this page, but it's also like looking through the handbook to see like where are those old links linked, if that makes sense like where are they hyperlinked and then replacing them with that? Okay, I just wanted to make sure I I didn't know if I was like.

A

I was like reminding myself as I was looking through. It.

B

um Wait so let me uh confirm what you just said. So are you saying um we're not just we're replacing um all these links in just this file with the sorry with the corresponding replacing link over here, but you also want to make sure that any of these links on the left side in the current anchored link that are referenced anywhere in the handbook also want to be replaced.

A

B

A

And sorry, if I wasn't clear um on that, yes, so that's and that's the main use case really is like because um it makes sense the most of those links on that handbook page are actually going away, like a lot of that copy is just there right now as a band-aid, because there wasn't um there wasn't like any kind of redirect or find or replace, and so the main thing is like yes, once all of these links are replaced everywhere with the new link where it should be, then um you know, then we can just go ahead and delete all that copy on the homepage.

A

Now that has like most of those links, if that makes sense, and the reason we did that is because I know we talked about it. We couldn't we couldn't redirect an anchored link.

B

Was kind of I didn't remember that actually.

C

B

A

So I don't know if that probably just made it a million times more complicated. I apologize.

B

No, it didn't it's just going to take a little bit more time. Basically to what I just said a few minutes ago, like great power, comes great responsibility. uh We just want to I'm going to need a little bit more time. Just to com, you know be sure, is basically okay, um so we can still finish this recording no problem. uh We still have 20 minutes and I can at least convey the general idea about it, and then you can just imagine now we do this.

B

Instead of all that's going to change I'd do this locally and not in this little tool got it because I would have to iterate all the files right. So basically, one of the other bonuses about computers right is, you can just iterate through.

C

B

Directories and then for each directory. If there's a particular file that has a md markdown extension, we could only target those and we could run the same regular expression over all the files. So that's what I'll basically end up doing, but we can just focus on this one for now. Okay, so, as you can see, it's already matching some things. So basically, what and again this is where it's really cryptic- and I don't this took me a long time to like every time.

A

B

Then it'd be a pain in the butt and then it's like, ah but it's powerful, I'm gonna dabble in it. Oh, but it's so cryptic. So it's like it's a hard thing to get past um and there might even be it'd, be uh I'll. Just I'll just stay there yeah, basically it's kind of cryptic, but again it's really powerful.

B

So what it's doing right here is- um and I don't expect you or anyone else to remember this because again it does take time, but I can share a link with you that basically defines what all these patterns are and then you can kind of like if you wanted for yourself to like play with it to experiment. um That would be again.

B

You could use this tool in combo with what the actual rules are and you could probably just type regular expression, documentation and I'm sure there's lots of various tutorials and sites, and that kind of thing.

C

B

The one caveat I'll say with that is the regular expressions syntax like the certain. Basically, the special characters that it's going to represent can sometimes be a little bit different depending on the language. So right now, I'm doing it in javascript.

B

um You could also do pcre, apparently, which I'm not familiar with in different programming languages, sometimes have this. I have to use a little bit different characters, but the same concepts.

C

B

The same, and I believe that would only be because that language had already predetermined one of those characters to mean something special in its language got it. That makes sense, that's an ideal world. It would be the same everywhere, but that's probably one reason why that might not happen. Okay, that makes sense. But anyway, what this is doing is saying- um and you can see the little hover which gives you, which is awesome too, so a to z, uh any characters between a to z, um followed by.

B

A word character, matches any word character and then one or more of the preceding token. So it's basically, I think it's finding a capital, it's finding the capital letter, the capital, a through z, so there's no capital in any of these there's no capital. Here, oh, we found a capital letter. So that's where at least this is has potential, then, after that we're looking for any word character, which is just slash w again. This is very cryptic, but that matches to um any alphanumer alphabetical character. I think numerical oh yeah.

B

It says right here, alphanumeric and underscore so you can see that in the hover. So basically each of these has like a little bit of some extra characters that you might consider.

B

So since a l, e and s are all what's dubbed word characters, and we also added this one or more, which is what the plus is keep going, is basically what the what this pattern is saying, so it kind of keeps going, but then it runs into uh the parentheses and that's not part of this pattern.

B

Where we have this capital, then we say followed by some kind of word character, which again is uh alphanumeric or underscore one or more times, and it happened one two three four times, but then it knows how to it knows to automatically stop, because it ran into a new character. That we didn't say is what we want.

A

B

And then, and then these little parentheses say it's called the capturing group, and what this basically says is capture this for later, because I'm going to use it later, if you didn't care about that- and you were just going to like typically what you do is pretty much. What we're going to be doing is typically, you want to find patterns and replace certain portions of that string with something else specific that you know you want to switch it with. So that's where caption groups are really helpful.

B

So, for example, we have this caption group of just the s. Maybe you wanted to say so. Here's kind of a funny example that I think um facebook actually did uh a while ago for like pirate day or something is basically on your page. You could pirate piratify it or something and all it did was. Basically, you can almost do this without a regular expression, probably but um yes, but basically it found every r or maybe it wouldn't probably found every word that ends with r now turn that r into three r's.

B

So it's like you know, piratey r, like that.

C

God, that's obviously not a very practical thing, but it's a kind of.

B

A little thing, but so that's a interesting use case, so it captured that r that lower case r and then just replaced it with whatever you wanted, whatever they wanted in that case, because it was that funny, pirate theme they replaced with multiples.

A

B

That's why we're getting these matches just from again, all I did was paste in our handbook sales page source, and this was already there. They had already pre-defined this.

A

B

So what we're going to do is uh what do we got? We still got 15 minutes. So what we're going to do is we could basically say.

B

We can copy this just as an example.

B

And I'm gonna have to so. This is where it gets a little, because this slash command is a again. This is where the cryptic part comes in again. This is already.

C

B

Special special character, so there's um another pattern you have to add to escape it, which means I really mean this character: don't use it as a special regular expression, meaning use it as the literal meaning. I want.

A

B

Yeah so again, this is really cryptic um and we have no matches.

B

Let's just keep it simple for now all right I'd have to refresh on this, so we have 15 matches with https, for example,.

C

B

And so I so these would all be um kind of the beginning of absolute urls, whereas I think a lot in a lot of these cases. We might actually just have relative urls. I don't know like if it's just the anchor and not proceeding about doggylive.com.

B

So that's where I was saying like we'd, uh it's going to take me a little bit more time to uh basically um do this right, um but but at a high level, that's basically what it's doing is saying what we're going to end up doing is saying I'm going to create a regular expression where it has so. Another thing you can do is in regular expressions. You can do optional, so it's like it is there or it's not. So what we're going to?

B

What I'm going to do is say if it's https all the way through sales, but it's optional. It might also just be followed by the pound, sign, okay and then, after that, I'm gonna have basically a list of this is my expectations. uh I could be wrong, but I'm pretty confident this is gonna work, it's basically gonna, say https all the way up to sales, but it's optional.

B

It may not be there and that's okay as long as it's still followed by the pound sign, and then it's followed by one of these literal values which are going to be each of the actual links. So we'd have one two three four five, six of those and then we'll be comma separated, and then that will be the end um of the of the regular expression that I want to create and then again I'd make sure.

B

That's you know I'm getting next the results, I'm basically wanting in a tool like this before I really run it um and then, when I have that in code, I can actually do a replace of all those instances of these seven instances. Six instances with.

C

Just this value.

B

So similar to the example, I said before, where, instead of us just finding the lowercase r at the end of a word and replacing it with three r's, we're now going to say, find all these examples of that I just mentioned, and and then replace it with just this url.

C

B

Basically, what's gonna, it's gonna end up happening, and I haven't actually done this in um um in vs code, which is the tool I was initially gonna. Think of doing, I don't know if it's gonna have like string limits on this or not, um but if it doesn't, then I'm just gonna have to do this on the command line or something I have to figure out exactly how I'm gonna do it, but.

A

So you're, essentially finding you're like mass searching right through all of the handbook.

C

A

Time these things are listed and then you're, saying okay, now that you found them all, replace them all with this one thing.

B

Exactly so that's so I remember I had the note of um create one, mr, so I think I have I I think again. I took this note a month ago. I guess I think that was your preference. um We probably want to break them up into smaller, mrs just for the sake of review, because, as you can imagine, if that's something I got to think about a little bit, is we want?

B

It needs to be in smaller chunks so that you can easily review them, because if again, if you just do a mask, find and replace we're going to have a merge request, that has potentially touched hundreds of files and that's like extremely difficult to know. If there was an error in some place or not sure sure.

A

No, that makes sense, and- and just I mean I don't know for sure, but my hunch is that there's. I don't think that these are interlinked an insane amount. There might be a couple that are, but most of these yeah I mean I definitely agree. The 1mr thing was just like more of a note to like you know it doesn't have to be separated in any particular way. So, whatever you think is like easiest on the on the organization side,.

B

Cool so um so I guess we can just we still have nine minutes, so we can give this a whirl.

A

B

C

A

um I'll just do this for now.

B

So this is, this is kind of um if I was a regular expression guru. This would go a lot faster, um but this is kind of the idea we'll try this again. I I mean I don't even know if this is important for you now, but.

A

No, it's interesting. It's I'm always happy to learn.

B

um So I think colin might have a special character, so we have no matches. We saw earlier that if we remove this, we do get 15 matches. So, let's just scroll until we get one just to see. So we have youtube.

B

So that's the thing I think a lot of them are going to be relative, um which again you're familiar with that right.

A

I am familiar, but I've always wondered why, like I'm familiar, I do it when I'm referencing the handbook, but what's the reason.

B

That's a good question. Actually, so that's a really good question. I haven't thought about that. A long time.

B

I would uh if I were to guess I would have to.

B

Guess it was kind of originated out of a developer time kind of uh optimization or or like usefulness. I.

C

B

That's actually a very good question, um so the absolute url will always work assuming the um server actually like has the requested file at that location. Then you're then you're good to go so it might be weird. Then why would you ever not always have the absolute and again that's a great question. This answer could be wrong, but my guess is like um in so this is actually a good example so for um uh in like, like, we even have it right, we have, we actually have.

B

I think we have um review apps that are on gitlab.io, slash blah blah blah if they were all absolute, which actually happens in some cases like if like, if you were on, if I were to be running the handbook locally, which I do during development. Sometimes if the link, for example, this youtube link, it would always point to that exact location, whereas if you did the relative url, it would be relative to this file currently, where I'm being hosted. So if I'm on gitlab.com, it would point to that file.

B

But if I'm on gitlab.io in a review app, it would point to that file and in that review app. That file could also be part of what I'm testing it's something I changed. So that might be a better. Does that make sense.

A

Yeah yeah, that makes sense it's a little hard to wrap my head around, but but I get it, I get. What you're saying.

B

It's basically kind of a small way to make it a little more abstract so that we have a little bit more flexibility. I think it would be a kind of a way to think about it, but that's.

C

A really great question.

B

I haven't thought about that in a long time. I remember when I started doing web development like that was kind of a confusing thing as well. I just haven't thought about it in so long.

A

Yeah it's just like when I started like mike snow on my team, like he was my my onboarding buddy. He was like yeah, you just make the links relative and I was like sounds good, but I never really asked why and every time now I do it I'm like. I wonder why I'm doing this well.

C

It's worth it and not everyone does it.

A

C

A

Handbook pages I've gone and edited handbook pages. Where I see that the owner, like doesn't do that. So I'm just maybe curious.

B

So one so this might be another example of doing a regular expression if it was important to, for example, always have relative or always have absolute. We could do a regular expression. uh You could that'd be a little.

B

This example actually would be a little iffy because there's obviously other urls like youtube, for example, so you'd have to start matching all these url possibilities, while excluding youtube, for example or other ones. So look.

C

B

That gets that gets a little scary, but that could be a potential regular expression use case. Basically, it's to do a mass replacement of. We don't want to do absolutely anymore. We know everywhere where we have https gitlab.com blah blah blah, and we want it just to always be relative. Then you can remove just that first portion of the url, for example, across all the files. So that's just another example. I guess.

C

Interesting: okay,.

B

But um we have about four minutes and yeah, like I said I think I'm gonna have to refresh on this. I didn't really prepare as well. I'm actually a little confused. Why.

B

So that works yeah, I don't know, I'd- have to look at it again, but basically that is like I said conceptually was what's gonna happen. I'm gonna ensure that we have so these are all sales. So for this page maybe I'll do one, mr for just this page or something I'll have to look into how, as you say, you feel like there's not going to be a lot of instances where the slash sales is referenced from other files.

B

um I can do a quick test locally to kind of confirm or deny that, and if that's the case that can inform how many mrs might be a good idea or not yeah. So it's not a crazy burden during the review process.

A

No, that makes sense, yeah and, like I said, whatever is easy like one one is fine, it's one per page like is fine too, like whatever, whatever makes the most sense, um and of course just just let me know, for some reason it you know it becomes unattainable or it's just like too much, and we can figure out something else. But um I really, I really appreciate you helping automate this a little bit or at least make it more automated, because this would have been crazy to try to do.

B

That's kind of, like I said, that's kind of stuck out. It seemed very, uh very possible, so all um yeah, I probably what uh the 30th is. That right is that the due date.

A

Yeah this was we're working in like a a couple sprints. It's not we're not like married to that. So if it's not possible this week, don't worry about it. um Just in the next like week or two would be, would be nice and then I'm gonna just go through. Like I said once it's done, then I can go through and delete all of that old content and we can finally kind of finalize that new home homepage uh format. So just let me know what's realistic but yeah.

B

Definitely don't.

A

B

All I should be able to get so. I was just trying to get an idea, but I anticipated to this week and probably tomorrow, to be honest, um it should.

C

B

It should go quickly. uh I like I said I just have to refresh a little bit um so basically again just regexer.com if you're curious to like maybe just explore other little patterns, because again it's very useful to do this um and there might be other tools that make it even easier. So you don't have to learn the cryptic part I'd. Imagine that would exist um right, there's trade-offs, obviously there and that you might not get as granular as you might want, or something.

C

A

And a quick, quick question with like the two minutes we have left so like. I see how this helps you like, find things right so when you then go replace them. Is that done in like a different language or method or is.

B

A

Regular expression.

B

Exactly so, that's why the difference would be um I'm going to do this locally on my machine, because I actually have the capacity to run over the files right and do a replace so got it, and I was just in just for sheer transparency of how I was going to try to do. It was just um and I can share real quick. uh I think I shared this last time too.

B

um So if I did a find in folder over app we'll pretend like this is the um source handbook just pretended? That's that's what this directory is in here. I can do a find and replace so right now, I'm looking for this is just from some work.

B

I was doing previously this particular exact literal string, um but I could obviously do basically you can do a regular expression instead by clicking this button right here, use regular expression and then it's basically doing exactly what I did over in the other area, but it allows me to replace it. So this is what I was planning to do at least from a first test, just because it'd be really quick.

B

Otherwise, I'm probably just gonna have to go in the command line and and and do this there, but I'd like to do it here, if possible, just because it's easier easier.

A

Yeah, so the regex is the find, so it finds it across multiple files to your point right and then you just use this and like a simple like copy paste or like a paste with what you want to replace with.

B

Exactly so yeah, it's basically the replace, say we wanted to we're refactoring in this thing called description, template and now it's like new template odds, I'll see a terrible name. Well, you get the idea, and now it would say it would replace this with that and that's again kind of uh yeah anyway code editors have different kind of tools like this to help. You do this in a better way than this also, uh but this obviously has a a great use when you, when uh you want to replace certain unique patterns uh nice so.

A

Cool alrighty well awesome.

B

Yeah, so um I will so yeah, I guess we can kind of end the recording, I guess but um yeah the soul just ended. I guess.