GitHub Demo Days, 18 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Setting up (your dev environment) for success! #DemoDays

Description

Josh Abernathy talks through GitHub Codespaces and some of the ways that developing in the cloud has changed how the GitHub engineering team works.

0:00 - Start
2:45 - Lowering the barriers to entry
7:23 - Decreasing distraction
11:36 - Increased productivity
12:45 - GitHub Codespaces Demo
16:23 - "What didn't happen"
20:28 - Installing additional tools to a Codespace
23:50 - Making a change to the app and making it sharable/reviewable

https://github.com/features/codespaces

A

Hello, everyone welcome to another fun episode of demo days. Today. I've got a special guest with me, josh abernathy. He is a staff software engineer here over at github and today we're going to talk about how github uses code spaces and, in particular, uh how it has affected our ways of uh of developing software. In our engineers, josh.

B

A

B

All right, let's dive right in here, so we're talking about code, spaces and we're talking about uh and what we talk when we talk about code spaces, what we're talking about is moving developer environments into the cloud which uh but but like what does that mean, and why should we move developer environments into the cloud you know for most of us, we've been doing software engineering for years and years and years and we've done it all on our laptops or our desktops.

B

So why move it into the cloud.

B

A lot of what I'm going to talk about today is uh based around our own experience at github. So, a few months ago, before codespaces was was released.

B

We undertook this effort to move github's engineering team, specifically the folks who work on github.com, get them using code spaces to do their their normal day-to-day development work and through that process, we've we've really we've learned a lot and, and we've been able to make the codespaces product better as a result, but it's also helped us um really appreciate the ways in which uh this idea of moving developer environments into the cloud is uh is has huge potential and and is the way that we think development is going to be done in the future, which is a bold claim, but uh but our experiences have shown that there's a lot of potential here.

B

So so exactly what does that mean right? Why? Why do this? What did what did github learn from this experience of moving engineering into the cloud?

B

um There are two big pieces that I'm going to gonna try to drive home today, and the first is that doing this has lowered the barrier of entry to doing work on github.com, and the second is that it has allowed us to decrease the amount of distraction or busy work or just kind of like unnecessary chores that engineers have to do throughout the course of their day-to-day work.

B

So, let's, let's dive into this a little bit more lowering the barrier of entry. This one is uh this wasn't is is interesting because I think uh getting local development environments set up is is oftentimes a difficult task, and we often think about this in terms of like new hires or new contributors to a project right in order to get productive, they have to actually get the thing working in their development.

A

B

And- um and we experience that with with new hires folks who just joined the company, uh but we also experience it with um with some some other other groups of people that we'll talk about in a minute um and within github, we had invested an enormous amount of time and effort into getting this onboarding experience this. This startup experience as smooth as we possibly could and to a large extent we succeeded.

B

We had scripts in place that you could run and you could probably have a development environment working within a half a day's time to a couple days, depending on what you ran into, and that was that was pretty good. We felt pretty good about that and that that it took a lot of effort to get there, um uh but we could do better and that's one of the things that we learned with code spaces. So we with code spaces.

B

We can lower the barrier of entry for these new contributors, be they new, hires or folks, who are just onboarding onto github.com from another team.

B

One of the other interesting things that we discovered within github after doing this work is that it was also a really uh enabled people who are not in the code day-to-day. um These infrequent contributors.

B

These folks, like uh you, know maybe engineers on other teams or or product designers, product managers, engineering managers like all of these people, who are kind of like around the code base, but not working in it every day, because for them you know because they're not working in it every day the idea of spending half a day or a couple days, getting a local development environment running is just it's just absurd right, like that's. That's a lot of time that they don't have to make.

B

What's probably pretty small change in the first place, but once we.

A

B

Code spaces: what we found was that these folks were some of the very first people to really use uh code spaces to do github.com development because suddenly, what was you know? A half a day's work of setup was now minutes worth of clicking a button, uh and it's just been hugely enabling for these folks, and so it's kind of opened up this whole other audience by lowering the barrier of intrigue.

B

We've we've opened the doors to these folks who, otherwise uh you know it would just feel like a bridge too far for them, um and then the other group of people um that code spaces really helps out here by lowering the barrier of entry is new software engineers.

B

uh One of our our engineers within the company, john capezi, ran a software engineering boot camp for folks who were new to software engineering, maybe coming from other careers or underrepresented backgrounds, and, and he did.

A

B

Code spaces- and this was because, uh when you're just getting started in software engineering, the last thing that you want to do is like have to think about how to install the right rails version or battle with compiling nogogiri, or you know all of those things. They'll learn in time. You know everyone. Everyone must at some point do battle with uh nokogiri, but that shouldn't be a new software. Engineer's first experience with software right their first experience.

B

Writing software should be uh that they are dumped straight to an environment that just works where they can make a change and then see it reflected in the product, because that's that's how we for for a lot of us, that's where the magic of software engineering is so so so that this is one of the big.

B

The big things that we found within github is that using codespaces has allowed us to lower the barrier of entry, which has enabled all these different uh different people who may or may not work in the code base every day and then the second thing that it's allowed us to do is um is decrease distraction and that this distraction kind of it comes in a lot of forms right, um there's a lot of technical work that we do as engineers that is not working on the product that we're building- and this can be things like um you know- we clone a new project and install its dependencies and now my device, my environment, is broken or it could be.

B

You know I started working on a feature and then we changed direction, and now I have this weird feature still stuck in my development environment.

B

All these things, there's all these like little things that as software engineers cost us time and take us away from doing really the valuable feature work that that we should be doing that we want to be doing, um and so, if we can decrease distraction, that's a huge win and we do that uh in codespaces in a couple ways.

B

The first is isolation and really like both of these points are are things that are lessons that say, like the infrastructure world learned has been learning in the past 10 years around, uh like containerization um and uh orchestration all these principles, these ideas of isolation and reproducibility. These are all things that we've kind of accepted and are the norm. Now within production systems, we've accepted that our production systems need to be isolated. They need to be reproducible so that you know if a pod goes down, we just it's no big deal.

B

We spin up another pod. We know how to do that. It's all codified within the project itself, and what we're saying is that we can eliminate a lot of distraction and a lot of a pain.

A

And annoyance by.

B

Applying these same principles to local development of isolation of reproducibility, so with isolation, this means that code spaces are are isolated from each other. That means I can, for example, I could start a code space that has a dependency on python, 2 and another code space that has a dependency on python 3 and these things aren't going to conflict for whatever reason for me, it's always maybe it's just me in a complicated relationship with python, but it's always python. Two and three fighting on my machine but code spaces.

B

Isolation fixes this right because it says these two things coexist independently in.

A

B

And what happens in this code? Space doesn't affect this code space, that's great, so isolation. That's one thing! That's one thing that lets us, you know decrease the distraction of engineering and the second is reproducibility, and this is this idea that.

B

Once we have a development environment that works and we've put in that work, we should be able to reproduce that infinitely to any engineer. Anyone who wants to contribute to the project and again, like we tried really hard within github, to do this through scripting, but ultimately people's developer environments, their local machines are just this. This sea of you know chaos and dependencies and other software. That's running that has a mind of its own and try as we might.

B

We can't achieve reproducibility in our local development environments, um but co-spaces does that and code spaces does that by by codifying how the development environment for a uh for a project should be defined and we'll take a look at that in a few minutes here, but um but by taking these principles that again are are well known and well.

B

You know celebrated um already for how we deploy services of isolation reproducibility by taking those and applying them to the local development experience.

B

It gives us a lot of a lot of power and it eliminates a lot of busy work. So the combination of these things right is, if we lower the barrier of entry and we enable all these other people and at the same time, we decrease distraction for the folks who are already doing their work.

B

Then the net result of this should be increased productivity and that's what we've seen is that within github by eliminating these whole classes of concerns, um we we end up with increased productivity.

B

So that's what code spaces is uh now. This is demo days. This isn't josh talking at people days, which is thankfully not a thing. So all right, let's get into an actual demo of what code spaces looks like now. Unfortunately, uh I wish you know, I've been talking about all this in terms of our experience, getting github.com development working in codespaces and as lovely as it would be to show you all working in github.com.

B

um You know that is some people would probably be pretty unhappy with me if I, if I did, if I start showing github.com source code, so we're not going to look at github.com.

B

Instead, we are going to look at mybigdad repository here, so I am a dad. I have three kids and, um and they have chores and by merit of me being the dad and giving them chores it makes me mean dad. um Having three kids also means that I can't keep track of who's done, what chore and and when and the days blur together- and I don't remember things so. This is the little web app. That lets me keep track of the chores that still need to be done.

B

uh For the day, so I can mark them off it's not as complicated or as interesting as github.com. Again. I wish I could show it to you, but sadly no, um but this is what we're gonna we're gonna, take a look at what working in the code space uh from this mean. Dad repo looks like so I could come over here and I could create a new code space for my code tab, um and this would create a code space.

B

Then it would drop me into it here in the browser, and that would be that would be just fine. um I personally don't enjoy working in the browser coding in the browser. So instead I'm going to come over to vs code and I can make a new code.

A

B

From within vs code, so let's me mad. Yes, there's me and dad your new code space for maine and at this point uh it's creating a new code space. It's cloning, the repository down and then it's getting the whole development environment that we're going to be dropped into.

B

Defined up to date in the way in which we defined it, we'll take a look at what that means here in a minute, but but but this is this is this is part of the whole reproducibility piece right is that I don't have to do any local setup or configuration to make this work. I just create a code space and by the time my co-space exists.

B

It's it's done everything it needs to do it's it's ready to roll, and this is again super helpful for you know if, if other people wanted to contribute on that, if mean dad takes off and it becomes the future of torque uh chore tracking, among mean dads everywhere, um uh but it's also helpful for future me right, because uh I contribute to this infrequently and when I come along in three months and I'm like gosh yeah, how does this thing work? um We don't.

B

I don't have to rely on my memory anymore at that point right, because everything is already defined in the repository and it it just happens for me. So now we're in the code space. We can see my code over here. I've got my terminal running here, so just so that we can get a feel for like what what is what is mean dad. Let's go ahead and run the server.

A

B

Dev you can see down here, it told me: hey port 3000 is running here. We see port 3000. If I go over here to my ports, tab it'll also tell me: hey: port 3000 is running. It also tells me that I have postgres running, which is also handy to know, but for right.

A

Now, let's just go.

B

Look at the web app.

B

All right super simple: all it is is chores right. It's just a list of chores and a button to click. I can click the button done and you know now now it's marked as done um make your bed feed pippin. That's the dog who's right down. There um set the table, uh so that's the app that we're gonna be working on here in our code space.

B

Now one thing you might have noticed if you've worked on, uh you know any node project. Before is what didn't happen, so we we got into this code space and I immediately ran npm run dev.

B

At no point did I run npm install and that is kind of our first glimpse into this idea that everything the whole definition of what it takes to run. This project is codified and codified and happens automatically so that I don't have to do it. um So, let's take a look at how this is actually configured.

B

So we have over here this dev container directory and in this dev container directory we have a dev container json. So let's take a look at that and I'll collapse, some things, so we have more room.

B

This is where we start to configure exactly what our codespace developer environment looks like um we're going to skip down we're going to come back to these. This first docker compose line in a minute, but we'll skip down and we'll take a look at this first, so we're forwarding our port 3000, which is our our server. It's our web app, it's the port that it's running on we're forwarding.

A

B

Those out of our code space so that they're accessible to us and we're forwarding our postgres server and those uh show up you know over here, so that we can remember what you know. What in the world is running on 5432, it's postgres.

B

And then here here's our post create command, and this is where npm install happened.

B

This means that every time I create a code space after it's been created, it's going to run in pm, install for us automatically um and this. This is why, when we first, you know dropped into this code space.

B

I could just run npm round dev uh without having to do this to begin with, and then, lastly, down here, we have this set of extensions, and these are vs code extensions that are automatically installed when I create the code space- and this again is getting to the idea that, within this code space within this repository, we have defined all of the tools, all the environment, that we need to have a successful development environment and so for this project. That means installing eslint and it means installing the uh the postgres sql tool.

B

So it means that whoever comes along and wants to work on this automatically has these out of the box.

A

They're already set.

B

Up to to be productive in this environment without us having to do anything else right without them having to install them or without us having a document to read me: hey, look, you're gonna have a better time. If you install these, it's already done for us.

B

A

That's some of the configuration.

B

But we also have this docker compose file that we're referencing here.

B

And we're gonna get into the nitty gritty of docker compose, but one of the things that our web app needs is we have the the node server, but we also need to run postgres so that we can save the chores and save which ones are done um and one of the ways that we could do that is.

B

We could do that by using docker compose to have both our web and our database running side by side, and then we can tell uh code spaces that hey the one that I'm going to be working on the service, I'm working on is the web service, and so we end up having our database standing up without us having to do anything. It just happened for us and that's super cool, and then lastly, we have over here uh our docker file, and uh this is all just boilerplate.

A

B

um But if we wanted to to install some additional software here, we could um and again it would be installed in all of our developer environments.

A

And in fact, let's do that. Let's suppose that.

B

um You know the vs code, postgres extension is super handy, but I'm a command line person, and so I want to. uh I want to be able to just run the command line client for postgres, but it's not.

A

B

All right, I could manually, install it here, but I would rather install it so that everyone who works on this project and any other code spaces I create from this project, will have this out of the box.

B

So we can do that by modifying our docker file over here, we're going to say, um install postgresql.

A

B

Saved their docker file and now vs code has noticed that um this docker file, this docker file- that is kind of the basis for this container, that we're in right now uh has changed. So it wants to rebuild it and we're going to go ahead and do.

A

B

We're going to take that suggestion.

B

So I didn't talk about this when we were first creating the code space, but what's really happening here with codespaces. Is that we're creating uh docker we're creating a docker container, we're defining a docker container, that is our entire development environment?

B

And so uh when I modify that docker file to install postgresql client, um it means that whenever I create a code space, the container that it's going to end up dropping us into to do development.

B

Is that container and so all the all of the kind of knowledge that you already have from working with docker files and containerization. All that still applies here and we can use those those familiar tools like docker compose, like we already saw. We can use those things um all that that knowledge just transfers right in um now. While I was talking, we, we missed it there, but you can.

A

See right here.

B

Is where our postgres client is getting installed, so it's going to continue to pull all of our images. Now it's standing up our docker compose and now it's putting us back into our development container and once we're in here, we should be able to then run our psql command and actually connect to postgres.

B

So, let's give it a shot. Fingers crossed live demos for me, so uh we are going to connect to localhost we're going to connect to what was that port again. 5432.

B

Our user is admin and our databases work and there we go.

B

Psql is now installed and again it's not just assuming I commit and push this it's not just installed in this code space, it's going to be available in every code space that I spin up for this project, and so it gives me an incredible amount of leverage for for anyone else, who's going to work on this thing with me and for my future self, it's always nice to be kind to my future self, all right now, let's, uh let's make a change in our app suppose that, um let's see, let's.

A

B

App running again.

B

Let's dive and we'll just go make sure it looks good.

A

Yeah, it looks good.

B

All right, one of the things you'll note here, is that um we have these two make your bed entries and that's because there's two different kids, who really have this chore and actually these two other chores are assigned to other kids. So we want to show what the the kids name is you're supposed to be doing this thing we already have this in our sequel.

B

You can see we have a person here in our chores table and it's already assigned here. These aren't the names of my kids, we'll protect them is the name of my dog. We don't care about protecting him um so that uh that field was what person so we'll come over here and we'll just throw uh chores for assignee's name here.

B

And there's our change now we can see he's assigned all these things and that's great, that's lovely. They will love that um now. Let's suppose that uh that I want to show this change, you know I I'm gonna, I'm gonna make mean dad go big. It's I need to get some funding. It's gonna be the next big thing uh right now, I'm just I'm visiting it on. You know localhost port 3000, but actually I could share this.

B

I could go over here to my ports, tab and I could say on my my web port here, port 3000. I.

A

B

Say make this port public.

B

What you'll notice is that our address has changed it's now. This github preview url, if.

A

B

We should see the same thing. Oh no! Well, you know, demos live demos, refresh it and it's fine. It's software we're gonna do um but, but now we have this url.

A

That is, that is publicly.

B

Accessible, um this is publicly reachable, and uh those of you who are maybe faster uh or who are fast can grab that and go mark. Some chores done, I'm sure my kids won't mind, um but now I could share this url right and uh for chore for mean dad that that is handy, but we found this to be really useful within github itself, because um this means that working on github.com we can share changes with each other with other engineers, but also with again in designers, product managers.

B

All these folks who are kind of around the project. um We can just send these urls to them and we can verify hey. Does this like do what you wanted it to do? Does this you know behave how you think. What do you think of the design? The interaction do you find any bugs, um and this this is just something I can do with with a single click within a code space. This doesn't require any other dependencies.

B

You know I don't need to deploy it to azure or or set up nginx or anything, or um you know ingrock or anything like that, um and this is again a great example of the ways in which code spaces can can kind of eliminate these little little distractions that we would otherwise experience the whole question of like. I want to show this to someone with code spaces. You just make it public and you send them a url.

B

So that is, that is a brief overview of code spaces of what the experience of using codespaces looks like the ways in which it allows us to define our development environment change, our development environment, make changes to the product that we're working on and then share those changes with other people.

B

Let me switch back over here.

B

And that is the end of my demo. um Hopefully we've we've we've shown today. The ways that codespaces this this idea of putting developer environments into the cloud is, is immensely valuable and gives us a lot of power to make our development environments more more productive for all of us and enables whole new classes of people to do development.

B

Thank you very much if there's any uh questions or I'd be happy to to talk more to anything in particular. Well, thank you all very much for your patience and your attention. I hope this has been helpful.

B

We have lots of documentation on github.com around code spaces and all the different ways that you can set it up, configure your environment, and I hope that it is. It is productive and helpful to you and makes you a more productive engineer. Thank you very much.