Numenta NuPIC Tutorials, 17 Mar 2014

Previous Meeting

⏯

youtube image

►

From YouTube: NuPIC Development Tutorial

Description

Using git, github, and travis for NuPIC contributions.

A

Hello, new pic, uh I'm matt taylor, I'm the community flag bearer for the demento platform for intelligent computing, and I'm going to give you a quick uh tutorial on how to get code contributions into new pic using, for example, a new pick core. If you have something you'd like to change, so this will uh give an overview of our developer process and walk you through each step required step by step.

A

So I am starting off here on the new core repository on github and uh if you're starting from scratch, I'm I'm assuming that you have some basic get knowledge.

A

So the first thing we are going to want to do is fork this repository from the github web ui. There's a big fork button here on the right. If you click that it will bring up a dialog that asks you. uh Where do you want to fork it? If you're, a member of any organization still show?

A

My personal profile is dimmed out, because I already have this forked, but if you don't have it forked, your github id should show up here and you'll fork it in your own namespace.

A

So once clicking that it will show you a screen where it says it's currently forking, but it's because mine's already forked, it just brought me to my fork. So, as you can see now, I'm on github dot com, rhyolite, slash, snippet core, not numenta, slash mubicore, which is the uh main location of the code. So this is my own personal fork um to get a clone of this of my new fork locally. You need this clone url down here on the right menu, so you can just click this button to copy to your keyboard.

A

I'm going to switch over to my terminal.

A

I am just in an empty directory that I'm using here to clone it locally, you'll type, get clone and then paste that url that you got from the github page and it will start cloning the entire fork locally onto your local, hard disk, depending on the size of the repository.

A

This could take anywhere from a few seconds to several minutes. A new core isn't really that large, so it shouldn't take too long, but new pick, I believe, is pretty pretty large. So if you're cloning new pick, it would take you a few minutes.

A

So now, if I take a look here, I've got nuke core clone locally, I'm going to cd into it and, as you can see, this is the new core source code. So let's say I have a change that I would like to make. um I'm just going to, for simplicity's sake, update the readme um and instead of incomplete I'll say, is not complete, so this could be any change. It could be a significant change.

A

You could change, uh you could move things around or you could just make small changes, but essentially, by doing its status, it will show that I have modified the readme. This is a change that I want to now turn into a pull request.

A

However, I'm still on my own master branch on my fork, so a very good thing to do is to create a remote branch that is not master to do your changes for a certain feature uh so for this example, there's an easy, uh git command to do this, I could say: get checkout on a branch and I'll call it my test feature.

A

So this does two commands. It creates a new branch called my test feature and it checks it out. So it's essentially the same as typing git branch, my test feature and then get check out my test feature.

A

So, as you can see, it switched me to a new branch called my test feature, so um I do a git status, which is what just showed me on branch. My test feature I'm no longer on master um and it says I have a change. That's not staged for commit this readme file.

A

Now this is a good thing if I get checkout master at this point, my change will come along with me. So, as you see here now, I'm back on master, but my change to the readme has followed along because I have not committed it. Yet it's the act of committing that will actually associate your changes to the branch you're currently on. So I want to go back to my test feature.

A

Okay, I'm back on my branch, my test feature branch. I still have this readme change that I want to add so git commit.

A

Actually, I have to add it get add readme. So this will add it to a list of things that will be staged for commit. So at this point it says change is not staged for commit. As soon as I add that file it will now be staged, its status will show changes to be committed and the readme is right there. So this is ready to be committed. It is staged.

A

However, there is no commit and message associated with it. I have not actually created a commit, but I can do that easily by saying git commit this will bring up a default text editor for you to write a commit message if you did not provide one on the command line, so I'm just going to say updated readme for no reason now the for github sake. The first line will usually will be the uh the title of the commit and if you add lines underneath of it, it will be sort of an amplification to that.

A

So I always put a short title up front and then I'll and I'll add uh amplification details afterwards, just a dummy change for a tutorial, I'm doing okay. So I'm going to uh save that file, and now I do get status.

A

It tells me I'm on my test feature, there's nothing to commit, because I've already got a commit. I've already created that commit and my working directory is clean.

A

um So at this point you can push your changes for you can make as many commits as you want on your feature branch. It's a good idea to break your changes up into several commits, especially for doing a uh a detailed feature anytime. I'm I'm working through something- and I find the repo I'm working on in a usable state I'll, usually want to try and commit that, um because if you have a commit, you can easily roll back and forth between the shah's associated with those commits.

A

So but this example we're just going to have one commit um in order to push that, because this commit this branch, my test feature only exists on my local hard drive right now. I have not told git or github about it, so I need to push this branch, which is a local, remote local branch into a remote branch that exists within the overall git repo that I forked. So to do this, I'm going to say git push now before I do that, let me show you get remote.

A

This will show me all of my remote branches or my remote locations. Excuse me, so my origin is my fork. Okay, um so I only have one remote, so I can just say: git push. You can say origin to specify what remote you're pushing to to be explicit, and um it's called my test feature. So this command will create a new remote branch on my origin. Location called my test feature and it will push that change to the readme and it it tells me I've got a new branch.

A

Now it originated from my local branch called my test feature, and there is a new remote branch called my test feature. If you want, if you don't want to use the name that you originally created for your local branch, you can change it to. I can also say get push origin, my test feature and I'll call. It renamed test feature. So what this will do is create a another remote branch on um my remote origin location, but this time it's going to be called renamed test feature.

A

Oh it'll contain the exact same changes that are in my test feature, but you can name it whatever you want so uh so I only have two branches locally master in my test feature, but I think why does uh get branch r? I think this will show you all your remote branches as well. So this shows me on origin. I've got all these remote branches that I'm playing with and there's my test feature and there's rename test feature. So we know about those two remote branches.

A

Okay, so let's go take a look at what these look like on github now, since I've pushed it.

A

Let's refresh this page and you can change the branch you're looking at here and as this is the page still refreshing there we go so now. Github um already sees these two remote branches that I've pushed and it's giving me an option to compare them and create a pull request. So, anytime, you push a remote branch. If it's within a certain time period, github will try and help you out and say: oh you're, probably going to create a pull request from this remote branch. So here's what it looks like.

A

So, if you click this button, it brings up a comparison view. It shows you uh what files were changed and the diffs between them. You can also look at the and see the commits you might have a long list of commits here for this pull request and how many contributors.

A

So this is the pull request that I want to create. uh You have a chance to update the title of the pull request.

A

Github tries to guess for you, but if you have a lot of different commits here, I think it uses the latest one, so you might want to rename the pull request to kind of summarize what the pr is doing and put some details in here about what's going on, a good thing to do is, if you have an issue associated with it, there's an issue number, so you could put uh fixes number 444 or something like that or number I don't know.

A

Usually it will uh try and auto auto complete this, uh but you can put the issue number there and if you do that, it will automatically associate that with some ticket.

A

uh So in this case this is well it's a pr, but um just uh and then, when you merge this pull request that ticket will be closed if you specify fixes pound sign and the uh issue number, but we're not going to do that on this because there's this is just a dummy thing, so I'm going to send the pull request. uh There's no danger at any point in doing any of this stuff. This is this pull request, though, will then notify uh the people who are watching this repository.

A

The new pull request has been opened. So here is the pull request. It's pull request number 44.. uh If you look through it, you can see. There's one commit. There's one file changed and all I did was change the name of that. You can also see that initially github shows this thing. That says it can be merged automatically.

A

Don't don't trust this? It probably can be merged automatically, but we have our own tooling. If you refresh the page, you'll get a message that says waiting the travis ci build has not started so before one of our reviewers on the new pic development team will approve this pr and merge it. It's going to have to to pass the travis bill so right now the travis build hasn't started. So, if you click on this details, tab it will take you to the travis build page for numenta muppet core.

A

um It doesn't always kick off right off. The bat uh travis has sometimes takes a while to allocate a resource for you. So if you click on the pull request, tab you can see there, it is. This is my pull request. Travis has allocated it as build 40. However, it has not started yet so there's no telling how long this will actually take before it starts. It's usually just a few minutes.

A

If you're lucky, it will start right away. But if you keep this open, you can keep an eye on it. Otherwise you can always just refresh this page to see if the build is started or not so um I'll. I will uh cut back to this once travis has started running this build, so we can take a look at it.

A

Another thing that might be useful is, if you go to status.nementa.org, there is a status page and there is a new pick and a nuke core tab. Currently, the new core tab will, as soon as travis actually starts running. This build will display the status of your your pr here on the on the right here under er build history.

A

So as soon as it does actually start, we will take a quick look at that.

A

So I'm just going to wait around here until it starts there goes so it looks like it has started, um as you can see, there's a build matrix. There are two jobs associated with this: build that we've configured for new core one is for sealang here and one is for gcc.

A

um It looks like the sealang one has built and passed, and the gcc one has not quite started yet. So you can dive into these and see what the output is. uh So this one exited with zero. It ran the build and everything seems like it went fine.

A

um This one, however, is still being created, but if you look on the status board, you can see that you at least get got an indication here that the pr has started and if you hover over this you'll get more details about the message um and when, when it started and validate for finish because it hasn't actually finished yet and your this is also listed as the latest new core build. That's currently running so again, we'll have to sit here and wait if we go back to the um github page.

A

For this pull request and refresh it you'll see the the message here changed. It says the travis ci build is in progress now so now, if you click on the this details, link it'll take you directly to the build that is in progress and we're still waiting on the gcc build to get started.

A

And there it goes looks like the gcc. Build has now kicked off.

A

If you click this button to follow the logs, it will keep up with the latest stuff coming out of standard out, and it is now complete. The nippy core build simply builds at this point. It does not run tests yet in the process of doing that so now build 40 is green.

A

The pull request, tab should show green, and if we there we go, the the uh nuke status page should show that pr is green and back to the pr we refresh this page. You should have a big green merch button.

A

um If you are a if you have push access to the repository you'll see this big green merge button, if you don't, it won't be there. So currently only committers can push to the master branch.

A

So otherwise- and it is just standard policy for a project that someone who creates apr is not the person who merges the pr so generally you're going to ask someone else to review your change and press this merge button and that someone else is going to need to be a committer on the project.

A

So I am not going to push the merge button, because this really is a useless change, but that pretty much walks you through the process of getting a change submitted and reviewed for new all of our newpick projects. So this this process is uh the same for new pic with the core uh cerebro the website, the tooling server. All of our projects are pretty much the same, and that is about it uh once you have merged it um locally, you can get rid of that branch.

A

You'll have an option once you hit merge to delete your remote branch, and I suggest you delete it just uh just to keep your remote branches clean, I don't always remember to, but then uh once it is merged, you can just go back to master.

A

One thing we don't have, though, is a remote pointing to the original new pic which is new core here, so I'll run through this real quick for you. We want to add another remote. So if you remember, if we do get remote and v is for verbose, I only have one remote location called origin, and that is pointing to my fork of the new core repository, I'm going to add another remote and typically the standard is to call it upstream and then I'm pasting.

A

uh Actually this url from the numentadcore, the original uh repository, and so I'm basically telling my local get whoops get remote, add upstream paste there we go so now. uh If I look at my remote branches, I've got origin pointing to my fork and upstream pointing to the original port. So I can now now I can uh get my changes updated on my master branch that I've made by saying get pull up stream master. If I had pressed that merge button, this would give me those those changes that were merged into master.

A

But since I didn't everything is already up to date and it's all good another thing you can do. uh Let me check out my featured my test feature branch again. um Sometimes you'll get a merge conflict. uh Let's go back to this pull request.

A

Oh or was it it was an open pull request. So here it is read me for no reason. um Sometimes you won't be able to merge because there are merged conflicts and you'll have to manually resolve those merge conflicts. So what how to do that is from your test feature branch. This is your local branch uh you're going to want to try and pull the changes that need to be merged from upstream, so the quickest and easiest way to do this is, to just say, get call upstream master.

A

So this will pull any changes that have been made to the master branch upstream. So the most recent head of master development head into your local branch. Now I don't have any changes. So it's just going to tell me it's up to date, but this is where you would be displayed any merge conflicts that need to be resolved. You would manually resolve them turn that resolution into a separate commit push that commit up to your remote branch like I showed earlier, and the pr will update with any further commits.

A

So let me show you something uh something additional so sometimes when you're on this pull request, somebody might come in look at your file and say something like hey. This is a dumb change.

A

Why even do it because it is it don't change, but but it wouldn't be me, somebody else come in there and say that or they will just suggest that maybe you should do something different and you won't be able to merge until you take their advice into consideration or discuss it or something. So let's say that happened to me um and I want to go into the read me again and say um so: uh instead of not complete I'll just say is uh isn't done yet I don't know just something silly.

A

So I've made another change, so my git status will show me that uh this change is not staged for commit so get add. Readme, git, commit and I'll say, uh re worded, read me taking my own advice.

A

Okay, so then I will push this now to origin.

A

My test feature, so this now gets pushed to the remote branch. My test feature, which is linked directly to this, commit, as you can see it immediately said this page, is up, is out of date refresh to see the latest, and now uh you can see my first commit here and my second commit where I've said reworded, re-me and, as you can see, you can expand these to see some amplifying text about the change.

A

But once again, since I pushed this, it's the the status now is pending, it is waiting. So I now I have to go through the whole cycle again of waiting for travis, and if we look back here under pull requests and refresh this page, we will see that there is a new build or, for my latest, commit so back back to the pr every one of these shaws every one of these commits that you push will be validated by travis. If there's any update to this pr, uh travis will attempt to rerun everything against it.

A

So now that I've made a second change in response to someone who's critiqued my code or something like that after I've pushed, I have to sit here and wait for travis to run again because we're just not going to merge anything that does not pass in travis.

A

Another thing that popped up here is: um all of these statuses are posted into the nuke hackers irc channel. So, for example, I got an indicator from travis here that said that the build passed it will show me the change view in github. So it's so this, oh! This is actually some other build. That was running, but that's good um and uh details, so this wasn't actually associated with the change that I was working on because I have not merged it to master yet.

A

uh But if you do um lurk in this new big hackers area, you'll get all of the different travis status changes. Github status, changes post it here. If you want to keep an eye on it and be notified, so uh the build has still not started, but I think you get the gist of this as soon as that build runs and passes.

A

This will turn green and say it's okay to merge someone again, not the originator of the pull request, but someone ought someone on the review team will be able to press this merge pull request button once it has turned green and your change can uh get into uh the repository.

A

So that's that's! That's it. That's the quick tutorial for getting changes into a new pick project thanks for your time. Watching. If you have any questions, um please email, the list, probably um you can either email- uh should probably be more fitting and new pickpackers if you are attempting to get a code change into the repository so uh sign up for the nuka packers mailing list here um you can subscribe in this form and ask ask a question about this or and uh be happy to answer.

A