GitHub 30 Minutes to Merge, 21 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 30 Minutes to Merge: "Basics of metaprogramming in Ruby"

Description

#30minutestomerge
Our monthly 30 minutes training session, enabled by GitHub’s Implementation Engineers, hosted by @beardofedu, live-streamed via http://twitch.tv/github, and with a followup up Q&A.

In this edition, we’ll learn from GitHub's Principal Software Engineer John Crepezzi. "Basics of metaprogramming in Ruby".

https://twitter.com/seejohnrun
https://twitter.com/broadstreetads

A

All right, sweet, we're back uh hello, everybody. My name is matt desmond or beer to be to you. um Today's another amazing session of 30 minutes to merge, and today I get I have the pleasure of introducing my new guest. uh This is this is gonna, be john. He is, or at c john run. um As always, when I uh introduce people, I like to ask a couple of questions just so people get an idea of like who this person really is.

A

um So I just kind of want to go through this laundry list of awesome things that I learned about john before today's uh session uh and then kick it over to him so that he can uh rock and roll on his topic so really quickly.

A

The laundry list of hobbies that john's into is kind of daunting for me, uh he's huge into woodworking and metal working and, like the maker space um he's also uh working on a project called beer broadcast, uh which kind of helps people connect with new beer uh releases from like their favorite breweries. That also includes like a podcast.

A

He also has an airstream so he's constantly on on the run in that airstream uh and when he does find a moment to settle down uh with you know the five kids he uh he also likes to bicycle around town and he owns way too many bicycles. But you know he's always gonna look for a new one. uh He does have a puppy named oliver.

A

It's a brittany uh which I learned is no longer called a britney spaniel, but just to brittany, uh and as always, I like to ask our guests who what like a recent purchase was- uh and he made two that I thought made the list so one he bought a headlight for his bike.

A

So you know safety is, uh but then he also bought a uh propane forge for blacksmithing, uh because woodworking and basic metal work was not enough for john uh when it comes to open source he's involved in projects as a maintainer like ice cube and uh haste bin, uh and then I asked him for some feet. You know some input on. You know if you're starting it to like look or get into the open space world. You know what you should do is like a newcomer uh and he had some some advice.

A

So you know, if you're, maintaining a project, try to work with people uh which obviously comes with its own set of problems or issues uh or you know, work.

A

uh But you know kind of divvying up the work across a group of people makes it a little bit easier for you to maintain a project and then, if you're, trying to get into the open source world, maybe try to find smaller uh tasks or projects, as opposed to like jumping into like the big monoliths or the big mammoth projects uh and trying to make a big splash start small and work your way up until you become more comfortable and then.

A

Finally, as always, I like to ask for like a fun action uh that they would be interested in making uh and he thought it'd be cool to make some kind of like peloton leaderboard, that uh kind of created, like a small subset of group like friends. That would let you know that you know maybe john blasted, you on his recent ride uh and to kind of make you more interested in riding again or you know, at a better.

A

uh I guess visibility for for the young kids in the in the world that you know exercising staying in shape is cool, so I think I can ran down all of the cool things I learned about john john how's. It going.

B

It's going great yeah.

A

B

uh Thank you for having me by the way.

A

Yeah absolutely always excited to have great guests and um yeah, so I'm gonna, let you take it away and uh have fun.

B

Sure, um and if people are interested I can, I can always come back and talk about woodworking and metalwork, but today we're going to talk about meta programming in ruby, um so first just to overview kind of what is meta programming and what is ruby. So I will start with what is ruby, which ruby is a scripted programming language uh that lets you.

B

I guess one thing that's really nice about ruby. Is that it's very readable when you're writing it. Ruby also is the language that you use with ruby on rails, which is a web framework. That's pretty popular! It's also the web framework that we use for the monolith application at github, which is the core application that powers github.com.

B

When we talk about meta programming, what we're mostly talking about is writing code that either modifies or writes code itself, so the meta part is like when something is meta.

B

When I, when I think about when something is meta, I always think I used to work at genius, which was called was rap genius at the time, um and there was a uh um genius has different sections for different parts of the site, so they have, like a a rock part, a wrap part, a books part, um but they actually had one part that was internal only. uh It was basically the uh the internet so to speak of genius and it was called meta.genius.com.

B

So when you, when you're thinking about meta, it's really just um doing something about the thing itself. So meta programming is no different. We will be programming uh code that itself writes code.

B

Ruby particularly has a lot of functionality for meta programming and a lot of functionality for kind of dynamic introspection of code.

B

So we're going to see some of that today and the final kind of thing I want to say about meta programming before I jump into the actual examples here is that um you know some of the things I'm going to I'm going to show you're going to see and you're going to say wow, that's just really wild that you can do that, and no one should ever do that and partially agree with you, but partially also think that, in order to you know, understand systems, sometimes you need to understand what's possible.

B

So a lot of the meta programming stuff that I'm going to show you, while not directly, um you know, being something that you'll use will be something that helps. You understand how maybe the libraries that you're using uh use it because uh frameworks like rails or libraries uh like hashi or money or a bunch of other popular libraries, actually use meta programming under the hood to do the things they do uh also including testing frameworks like r-spec, okay, so, let's jump in, I am going to. Can we switch over to the terminal?

B

I don't know how to do it great okay. So, first we're going to start by calling a method and all I'm going to do. Is I'm going to output a string in reverse, so we just say puts hello.reverse so that takes obviously the string. Hello calls the reverse method on it and if I go over here, I can actually run that and you'll see that it outputs hello backwards.

B

Just like we expected now what's actually happening when you do that is you are doing a thing called message passing in ruby, so you can actually write the exact same statement as puts hello. Dot send reverse so when you actually call a method in ruby. What you're really doing is you're sending a symbol uh to them to that method, sending a message that it should run its reverse method: you're, not uh just calling the method. So you have this ability to to send something which actually gives you the ability to do something like this.

B

So probably a better name for that would have been like method name, but just to say that you can actually send uh you can send these symbols to different objects, and you can have them do something, so we can also show how that will work with uh with when the method takes arguments. Real quick would be something like puts hello g sub, h, okay, so what g sub does is it takes the thing finds all of the occurrences of the first part and replaces them with whatever?

B

The second argument is so if I run this, if it's going to change hello into jello and the method, passing version of the same thing would look like this: send g sub, h, j, so, basically anything after the first argument will be passed as arguments to the method itself.

B

Hopefully this is making sense so far, and it's going to become very clear in a minute why I'm even showing you this so just to extend that idea a little bit further. What would you imagine happens when you do something like this n equals two plus three now in most languages that do some kind of object-oriented behavior? uh They make an exception for things like primitives, which are you know, numbers like this, so while objects maybe exist, these numbers are just uh numbers that exist.

B

You know they're not part of the object-oriented nature, but ruby everything's an object. They all work the same way. So actually, when you're doing n equals two plus three, so we'll do puts two plus three, we'll output that and then we'll do, puts two dot send, plus to three so in ruby. Even addition uh can be represented, essentially as a method passing.

B

So in this case we are passing the method plus with the argument three to the object represented by the number two, so that should be kind of wild in itself, but we can use that uh to try to do some examples. So what I'm going to do to try to demonstrate how this works is I'm going to walk through the creation of a basic library that creates typed objects very similar to how the money gem itself works and don't worry? If you don't know how the money gem works? I will describe that as we go.

B

So the fundamental problem here is that you have uh you have something like two dollars and you want to be able to add uh five dollars to that. Now, that's pretty easy because you could represent that in the world as just two plus five, but when you write code like this you're making, unfortunately, an assumption about those numbers you're, assuming that everyone that works with those numbers knows that those are us dollars.

B

So we come into a problem when you do something like two dollars, plus uh maybe something like five euro or you do something like two dollars, plus um 50 cents. Basically, there's this idea inherent in these types that they're not just numbers, they're, actually number plus currency.

B

So what the money gem does is it allows you to create money objects instead, like this.

B

And you could add them to other money objects.

B

uh You know maybe the same type or maybe different types, and if you try to add types that it can't convert between, because maybe we don't have a way to convert them, uh maybe you're trying to add us dollars to gigabytes or something that this would raise an error because there's no way to make a conversion across them.

B

So we'll see right now how to use operator overloading, which is what it's, what it's called when you overload, the behavior of the plus sign itself to accomplish this behavior feel free to drop any kind of, as I'm going through this in the uh chat which I I'm pretty sure I see the chat, um but I don't yeah.

B

If, if there's something you want to say you can say it and I will try to respond okay, so the first thing we need to do is we need to be able to create a money object, so we'll create a class that class is going to have an initializer which will take in a value and a unit.

B

So when we create a money, object, we'll do something like this money dot new to usd and it will say uh you know, objects. One then maybe we'll do something like output puts object. One money.

B

Oh typing too fast.

B

Okay, so we run that we'll see that it outputs, uh basically, the object. Representation of the money object, we'll change that to be a little bit prettier.

B

I will take in these two things and we will make the 2s method defined, which will say basically value in it.

B

Okay, so now it says two usd instead, so the 2s method is the method that gets called when you just output an object. Basically, the puts method itself basically takes the object that comes in calls 2s on it and then outputs the result of the 2s operation.

B

So what we want to be able to do I'm going to write it is we want to be able to do money.new 5 usd we want to do, puts object, 1 plus object 2..

B

So let's see what that would look like if I run that you're gonna get an error and the error is pretty telling now what we know about operator overloading. Is that there's an undefined method plus for money? So what it's saying is that we haven't defined the plus method, but we're trying to call it.

B

So all we have to do is go to find the plus method, we'll do plus other and then the answer is basically create a new money object where the answer is the value of this object, plus value of the other object and the currency of this object.

B

So in order to do that, we need to create a ability to read the other one's value, so we run that and we'll see that the answer is 7 usd.

B

So that's pretty amazing already, but we haven't actually got the benefit yet because still what would happen you probably can see? The bug is if we change this to five euro and we run that it's still going to say 7usd. So since we own the behavior of plus now we can actually do something like this.

B

We can say if the other unit is not equal to this unit, raise an argument error that we need to add the outer reader for unit, and we run that, and you can see that we raise an argument error because we're trying to add two different types of objects that are not compatible uh together. We can even add uh incompatible objects here, we'll run that again see incompatible objects.

B

Okay, so now we want to be able to do something like add the usd to the eu. All we have to do inside of our money, and I'm not going to do this because we're a little uh you know short on time in the session, but all we have to do to do.

B

That would be to create a table that basically handles the conversions between one unit and another unit and then instead of raising just if the units are different, we could raise uh based on um whether or not there was a comparison available between the two types of objects.

B

Okay, so now we're going to kind of take what we have here, which is this foundation and the ability to define operator overloading, and I'm going to ask a quick question which you can kind of answer in your heads, which is, if I do something like this.

B

Dot new, I do object.name equals john, what's actually happening there, so we're creating a new open struct, which is just an empty object and then we're calling object.name equals john, and this might be pretty surprising, but what's actually happening in ruby, is we're doing object.send to a method that ends in an equal sign.

B

The argument john, so these two lines are roughly equivalent to each other when you call name you're, actually calling name equals with a first argument. That is the right side of the assignment operation.

B

So with that in hand we can actually go and we can create an object where, when you do an assignment on an individual property, it responds in a certain way. So let's write some version of that. Real quick, uh we'll do class something and in here we'll put name equals name and then we will do uh puts we tried to set the name to name and then over here, we'll just use something that new.

B

That name equals john and run that, and we can actually see that now we have hooked into not only addition in the last example we're also able to hook into property assignment.

B

So when we hook into property assignment, we are able to hook into an individual property. But what if we wanted to be able to do something like have multiple of these methods? We would have to write each individual property that we wanted to add all here right next to each other.

B

Now what we can do instead is. We can actually do something that is meta-programming, which is called method missing. So I will try to describe how method missing works and then we're going to go through a whole example using method missing to implement another library.

B

So here in method, missing, I'm going to start by doing, puts hello, dot, pig line, and when I do that- and I run it you're going to see this error, which is undefined method, pig, latin for hello string and you can look it up but actually what's happening here- is that when you call pig latin it's not found and when something's not found ruby will call a method with a particular name.

B

It's called method underscore missing, which exists on basic object, and if you actually look up the source code for a basic object in ruby, which you can find you don't know, you don't even have to clone it, you can go to rubydoc.org and look it up. You'll find that basically, the implementation of basic object is just essentially a class basic object, def method missing.

B

This takes a method name and a set of args, and all this really does is raise no method, error and then object, inherits from basic object and uh string inherits from object. So string looks for the method, pig latin. If it's not found it goes, hey object. Do you have the method? It's not found. It goes to basic object, it says: do you have the method method missing and it actually says?

B

No, I don't. I have the method. Sorry, I do have the method method missing, so run it inside of there. It does raise no method error, which is actually how we get. This no method error on the right hand, side.

B

So, using that we can actually we're not going to define pig latin in this example at any individual level. But what we can do instead is we can choose to define method missing at a higher level. So what I'm going to do here is I'm going to try to make a method missing object, no, a method that works with method missing, so we'll do class something again and in here we'll do method missing in the two arguments that method missing needs, which is method name and args.

B

We will output the method name. So we'll say you called method name and then we'll do something.new.pig, latin and we'll go run it and we run it. You'll see that instead of getting error that the pig latin doesn't exist, it's done the thing that I kind of hinted that it would do it outputs you called and then the name of the method. So our method missing has essentially given us the ability to override what happens when a method isn't defined.

B

So that will let us do some pretty neat things. So, let's test another thing, something that new. That name is john. Let's run that so you can see here, you called pig latin and you called name equals. So this is just kind of proving even more, that you got new dot name plus five.

B

Method plus for nil class, I say I did this wrong, something that new plus five, so you can see here that it's doing exactly what I said, which is method missing, is overriding all of these behaviors. So what can we do with that?

B

What I'm going to do now is I'm going to try to create a library that some of you may have used out in the world that library is called hashi, I'm going to try to create a basic implementation of the actually library using method missing in, I guess, five to seven minutes, which should be fun.

B

Okay, so the way that hashi works is uh and I'll just write out how we're going to use it and then we'll go to start implementing it. The way hashi works is you should be able to do hashi.new to create a hash and then, instead of using it like a normal hash, where you would do something like uh you know, property name equals value.

B

Instead, what you'll do is hash dot property name equals value, so you'll be able to do things like hash.name john kid counts five, and then you also should be able to read those things back out. So you should be able to say, puts.

B

Hash.Name as hash.count.

B

Our goal is to make that work. It's basically an open object where you can assign any properties that you need and just for fun. We should also be able to do hash.each do key value.

B

So in here we'll just put this: this is the kind of the contract that we're going to try to go for with the hashi object, using method missing we'll start in here by implementing the message method, missing method with the name star, args and all we'll do in here- is I'll output, the method name. So we can see what gets called. If I run that you're going to see first name equals gets called, which is lining up to here. Then kid count, then name then kid count. Then what is this?

B

The next is yeah you have the blank. Has blank kids, that's outputting from the put statement on line 18, and then you have each, which is called here note here that, because we haven't defined what each does, the inner part of the block never gets called. Okay, so we'll start with name name itself. So in our initializer for the hash initialize, we will just create a um inner store which will just be an empty hash for now and then in method missing. What we'll do is if the method name ends with equal sign.

B

Output setting property property name, we need to be able to get that property name which is going to be equal to the method name from zero, pushing off the last letter. I run that sorry and 2s.

B

We'll see here, setting property name and setting property kit count. So now we have access to the thing that needs to be actually set. So all we have to do is take the inner store and at the property name position in the inner store, we will place uh args.first.

B

The reason we want argstat first is because remember the first. The only argument passed to an assignment overridden method is the thing on the right side of the equal sign. So args.first is the thing that is passed to method missing.

B

So if you run this now we'll see the output just there and we should actually be able to write our else now too. So the else is the uh rita property.

B

This one is the right in our else. All we have to do is do inner store of method, name, sorry of property, name and similar to property. Name up here. This will be method name except this time. We don't want to take the last letter off. We just want to take the property name so uh 2s, because we need to convert it to a string so run that, and now we get the output.

B

John has five kits so, just with this little amount of code, we've created an object that can take in any property called on it, it can assign things and it can read them out. The only thing we have left to do now is we have to implement each for the last part of this example.

B

So all that each should do is it should be able to yield the key in value for each iteration. We could just use a delegate here, but what we'll do instead is we'll just write it out uh enter stored at each do key value.

B

We'll do yield key value, and if we run.

A

B

You'll see now the properties just like we promised, which is naming kid count.

B

Okay, so what's wrong with this implementation, there's one thing that I have to say is wrong with it before I, uh before I'm kind of done with the example here, and the thing that's wrong here is the um I'm going to show you I'll show you actually what's wrong, which is, if I go in here- and I I'll just do it over here- actually, output um hash, I'm going to say: does the hash respond to the name method and if I run that you're going to see false at the bottom, so the problem here is now that the hash doesn't think it responds to a method that it definitely does respond to.

B

That's not a huge problem, but it is a problem, so there is actually a way to fix it and the way you fix it is you define another method called respond to missing this. One takes also method name and arts, and it's meant to return a boolean about whether or not that method should act like it exists.

B

So you know the simplest version of this would just be to write true, because um that's a pretty accurate representation of what we're doing here, meaning like they're either something ends with an equal sign, in which case it exists or it doesn't, in which case it exists. So in this, in this naive implementation, true, is the way to go, but just to know that when you define method missing, you also need to define respond to missing.

B

So I think this is kind of like I enough to give you a peek into the power of what you can do uh with ruby meta programming. There's other things that you should look into. If you're interested in this, like define method instance, val class of l, uh ruby lets, you define methods on the fly, ruby lets you define methods on pre-existing objects, essentially reopening classes to define methods.

B

Ruby lets you list the methods that are available to call on different objects. You can eval code against other objects. You can call private methods from outside objects. uh Ruby is super wild. So if you are interested uh in somewhere, where you can kind of like you know, learn more about the structure of how um how code works and how it kind of interchanges with each other. um There are plenty of examples you can read about this in the wild and it's definitely something to look into for yourself. So yeah, that's kind of it.

A

And I'm back okay, so first off that was nuts, and I appreciate you kind of taking a deep breath like you did for us today. um I know that you had also kind of said that you were interested in um coming back, which is always fun for me to hear.

A

um But I know you had an idea for a talk, so I wanted to kind of give you the opportunity to introduce your idea and then hopefully, chat will come back and tell us that it's awesome or maybe we need to take it back to the drawing board and and throw it back out of them.

B

Yeah, um so the idea is that I'm a um I guess like we, I don't know if we said my role earlier, I'm a principal software engineer at github, which means I work on things that are kind of important across the entire organization. I've worked at a lot of other places. I've been an engineering manager in the past and as a result of kind of all these different roles.

B

I do end up seeing, like you know, reading a lot of resumes and seeing a lot of engineers kind of trajectory through different levels and what I'm particularly interested in is kind of sharing some things that I think I've learned about getting started as a software developer, what it takes to kind of you know to to move between levels and what it kind of means to be the different, the different levels, at least in my opinion, kind of like, as as you grow as a software engineer, what you can expect around how your role will change, how your life will change, how your interactions with others will change, because each engineering level is really um is really very different from the one before it.

A

Yeah so hit us up and chat. Let us know if you're interested, because I I'd love to have jon come back, um and maybe you just want him to dive into ruby again, uh always.

B

Down to do that too,.

A

Yeah, it's very apparent just based on today's session, uh where you just kind of like immediately deep dived into like here's, all the crazy stuff that you can do uh as a as a like, a a preview, so very cool. um As always, I appreciate uh you showing up hanging out with me and uh everyone else on the stream hanging out and checking out what we had to offer. uh Quick plug. Github universe is next week, uh which is gonna, have a really awesome set of uh announcements.

A

There's gonna, be people sharing all the cool crazy stuff that they do with github um and I'm super excited for it and then um yeah. I don't think I've got anything else that I've got to say uh from the handlers.

A

Not getting any like hey, don't forget, so that's always fun. It means I'm doing my job right, uh but yeah. So thanks again for joining us. I hope you enjoyed this session of 30 minutes to merge.

A

uh We'll have another one next month and uh just keep us uh just keep telling us what you want to hear and we'll keep getting great guests like john and others kind of share cool stuff, but uh yeah thanks again for joining it's a great time to be a hover, and I hope you have a great rest of your day.