GitHub Git Merge 2013, 5 Jun 2013

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Git Merge • bfg-repo-cleaner (Roberto Tyley)

Description

Subscribe: http://bit.ly/subgithub • Roberto Tyley demonstrates his BFG tool for cleaning up your repos. The demo includes a speed comparison between BFG running on a Raspberry Pi vs. git-filter-branch running on a regular PC. The BFG wins handily.

About Git Merge
Git Merge is a Git user conference, held to bring together Git users and Git developers. This May, they came together in Berlin to improve the world of Git for everyone.

A

Hi there,, I'm roberto tyley..

A

I work at the guardian in the uk and I wrote a tool called the bfg,.

A

Which is like a simpler, faster, alternative to git-filter-branch.

A

basically,, addressing the problem of you having unwanted crap in your git history, that you want to get rid of, that you can't get rid of with a simple rebase, because it sort of pervades your entire project.

A

yeah, so,. It's a faster and simpler alternative to git-filter-branch,, getting rid of unwanted things. so, for instance,, big files. That might make your repo really large to download.

A

Or passwords,, maybe you wanna share your repository on an external hosting provider and you don't really want everyone to see your old passwords,. Even though, obviously,, you should've changed them-. So are we going with vga? cool! old style.? This is why I cover think path because you know, they still have vga connectors.- is that gonna work?.

A

Ok, so who has seen the video of the bfg?

A

anyone?, we've got some people at the back, there. hey!, hi, there! ok,, well, seeing as not everyone put their hands up,, I'm gonna play you. The video.

A

So,, one of the main compelling points is that it is faster.

A

And so to demonstrate that [laughing].

A

I put it onto a raspberry pi,, which is like a 25 pound computer,.

A

Very cheap computer, I have one here.

A

You see that?, the bfg's command was substantially simpler and there's this little 700 megahertz device, compared.

A

With like a nice big quad, core mac,, which is, you know, a nice computer., and so they set off and look there,, the filter branch starts off with a major lead..

A

That's pretty good. and there's the bfg running the backgrounds,. That's a little screen showing what the raspberry pi's up to. ahh but,. The bfg is catching up.

A

Even though it's running on this.

A

yeah, and as we speed up even faster, we can see that- wow, the bfg's like totally taking the lead here.. It's like 41,000 commits and.

A

A

So that took 25 minutes on that bit of hardware, this bit of hardware.

A

and so the end results,.

A

The mac took 45 minutes.

A

bfg took 25 minutes on the raspberry pi.

A

and if you have a regular bit of hardware,, maybe not this., then, it'll take a minute or so.

A

and, you know,. If you have like a gigantic repository, you might have an overnight job on your hands,, especially if you're doing something complicated. so,, I'm just gonna, show you like a few things. The bfg makes easy.

A

Well, not that...

A

Anyway, so removing a file,.

A

That's how you do it with the bfg.

A

delete the right file. with git-filter-branch. That's pretty much a standard, command, like-! I mean,. I know that all of the options, it's a great chainsaw,, but for the.

A

Specific task that a lot of people want to use git-filter-branch for,, which is removing a lot of history, then,, probably stuff like that, is easier..

A

It's just gonna delete every copy of a file with that name that it can find, and you can use globs and regexes and stuff if you wanna.

A

now,, supposing you have massive files.

A

Clogging up your repo,, then that is how you do it.

A

And maybe git-filter-branch isn't the best tool for doing that,, I'm not sure, but I've never really found an answer that was shorter than massive python bash script or whatever.

A

and then,. If you wanna, remove a bunch of passwords, just put a bunch.

A

Of those passwords into your file and run it against the repo. or you could do sed in the git-filter-branch. so,, it's just a bit easier and you can use wildcards and substitutions if you want to.

A

Alright yeah, treats you like a reformed alcoholic in that it is not gonna play with your latest commit. alright?. That's like a principle that you're supposed to be clear,. Tidying up your current history., I'm just gonna tidy up- your current commit..

A

I'm gonna tidy your history, all this stuff in the past., and so, if you run the bfg and it turns out you're trying to delete passwords, it says, "oh,, there's actually still passwords like this in your current commit..

A

Maybe you should clean that out before we continue 'cause, I'm not gonna touch them.", so you're a reformed alcoholic,, someone who's made mistakes in the past, but now you've cleaned up your act.

A

and,, oh yeah,, and this is just a little nice tweak,. So your commit messages might reference other commits and the bfg will fix those commit messages.

A

If the commit id resolves to something that it knows, and it will just say, "ah,, this was formerly that commit." and so,. Hopefully it's a nice friendly way of cleaning out your history. we've used it a lot at the guardian,.

A

It's been used in uk governments, on some of their repos and various places.

A

so,. It's relatively reliable., it's built on jgits,, it's written in scala,, so you can script it if you want to in any jvm language that you like,, if you like any jvm language.

A

Thanks very much.