Node.js Node.js Live Events, 25 Jul 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Ashley Williams - Node.js Live London

Description

Ashley Williams provides tips and tricks around npm in this Node.js Live presentation. She also provides an overview of Node Together, which is an initiative to improve the diversity of the Node community by bringing people of underrepresented groups together to learn Node.js. If you are curious to learn more about how you can be involved as a student or a mentor, head here: http://www.nodetogether.org

A

Hey I'm, Ashley, Williams and I'm, going to tell you about all the things that you don't know about. Em p.m. however before I do that I want to share that. If you like these slides, you might consider checking them out. Additionally, if you're the type of person who sees a typo or something wrong in a presentation and gets like freaked out and just can't, handle it I'm on github just file an issue and I'll deal with it later so yeah.

A

You can see the slides here, I'm going to be telling you a lot of kind of tips and tricks, so you might be interested in just saving this now before I go into mpm stuff. I want to talk about something amazing that happened last night. This was over at makers Academy in London, and this is node together, which is a workshop that I teach to get people who are either new to note or new to programming into node, and in particular we focus on underrepresented groups and we got a tweet on our Twitter.

A

The other day said this was one of the most diverse pictures. People had ever seen of a tech, community and I was super happy.

A

We had a great time so recalled know together, and this came out of a problem that I had so we're having a node interactive in Amsterdam in a few months, but we had our first note interactive a little while ago and I showed up and I was super bummed, because everyone was a white dude and it was so much in fact that someone took a picture of the back of me and my friends and said: I've found the only women here, and so that was really quite a bummer and I was like all right.

A

Well, if we're going to get this to be a more inclusive, diverse environment, one of the best ways that at least I know of is education. So I created this initiative, node together and so we've been to LA, we've been to Bangalore India we've been to Paris, and then we were just in London now and I bring this up partially because right now we kind of need your help.

A

Part of this is kind of getting run on em PM's budget. Part of this is getting run out of my own pocket. So if you want to make the node community more inclusive, please reach out and help us.

A

This is our website no together org- and this is us on Twitter and I- would love to get more people involved, alright, so on to mpm on to mpm, alright, so every day- and this is kind of old stats here- around 1 million eight hundred thousand install events occur and on average each saw install is about 70 packages which, if you put that together, is 126 million packages. Alright, that's a lot of packages and I work at the company. That is serving you all of those packages, so you might know me from Twitter.

A

My name is a g-dubs. You can follow me at your own peril. um I, don't recommend it, but it, as it was said, is I'm the developer community and content manager at mpm, and you totally know what that means right. Yeah. Definitely not. These dogs are adorable right. So what I like to say is that my job is that if you don't understand how mpm works, that's my problem and so hopefully we'll be dealing a bit with that problem. Tonight.

A

My job is to help you make sense of how we're helping you handle your dependencies. So my boss, Lori tweeted this about four months ago.

A

It was our one of our biggest days that we had ever had and what I want to point out is that was a hundred and eighty eight million downloads, but it was more than 2,000 and 100 downloads per second, and this is the time that I'll remind you that we have ninety-nine percent up time on the registry, so we're doing some serious ops work, which is really cool but we're delivering more than 1.2 billion downloads a week.

A

So we have a huge huge ecosystem and if you want to know what that looks like it looks like this, and this is what VC people call a hockey stick curve I, don't know why it's just an exponential curve, but we're spiking up there and all of those dips generally christian holidays, very strange um anyways. So these are our numbers right now. I took this about an hour ago. The trick is that they're all bigger as of right now, but about an hour ago we had 300,000 packages in our registry. All right these are.

A

These are just huge. He huge numbers, all right and module accounts was shown by Michael. This is what it was today and, as you can see, we are just way above what everybody else is doing in their ecosystem, which is fascinating and again, as Michael said. This is because we're covering so many different types of programming, one of our biggest growth sectors, is people who are doing front-end code.

A

We're seeing people put lots of things that aren't even note or JavaScript into packages we're seeing CSS packages and it's super fascinating, but in the end the trick is that mpm is the largest package manager in the world, which is super awesome. How many people here use em p.m.

A

that's awesome super great all right. So this talk, as I told you, is going to be about all the stuff that you don't know cuz everybody knows MPM install right, but it turns out that there's a lot of stuff that npm can do for you that not everybody knows about so.

A

While you might know some of the tips and tricks in here, I hope that by the time I'm done, you've learned at least one thing: that's going to help improve your workflow, because fundamentally, the goal of mpm is to make you building your software faster, more maintainable and more enjoyable, and so sometimes we deliver on that goal. Sometimes we don't, but hopefully you'll, learn something in this presentation. That's going to help you do that so I'm going to start at the beginning.

A

So in order to create a package.json, you can actually type mpm in it, and so often people will do this, but people don't actually know how smart this command is. This smart. This command is able to tell your depths from an already existing node modules folder all right. It also can infer your repository from a git repo and it will always add keys. So, if you've forgotten to do something, you forgot to get in it before um p.m. in it it you can just do it again.

A

It'll always be additive, it's never going to undo something that it did now. This is something maybe a fair number of people know, but what a lot of people don't know is this command, which is NP a minute dash dash, yes, which, instead of asking you a bunch of questions, just gives you all the defaults that it assumes and so I'm always just like yeah mpm in it? Yes, yes, super good.

A

So beyond that, though, what you might not know is that using a dot mpm in it Jas file, you can customize the questions that mpm in it asks. So, if you're using NPM inside your company- and you want to enforce certain open-source licenses, if you want to enforce certain naming things, you can actually write questions that have logic that enforce that, and you can do that right in here, which is super super cool all right.

A

So another thing that I see a lot of people actually don't know is that you can use the NPM install command, which we call the interactive command to actually save things that you're interactively installing to your package.json you're able to save both normal dependencies and dev dependencies, and this is really neat, but what you might not know, and given that you're all developers, you want to type less right. You can just do that and save yourself a whole bunch of time, which I think is really great all right.

A

So maybe now it's time to discuss the elephant in the room that we have mpm three. How many people here are using NPM three right now how many people are happy about it all right, that's interesting, all right!

A

So with NPM three, we really changed how we manage dependencies and unfortunately we change them in a way, and we didn't really describe why we did it and additionally, the documentation was a little bit complicated, as somebody who writes the docs I know that we really needed to explain the big changes that we made and so um yeah. We know people are not super happy with NPM three, but here's how it works, and it actually is super awesome.

A

So the whole idea is that we're trying to flatten the dependency tree, how many people here work on Windows yeah? How many people have hit max path? How many people are a happy that we're going to get linux on windows now yeah, that's so cool super super excited, but part of the reason that we did. This is because one of the largest growing communities- that's using NPM, are people who use windows, and we really care about that, and so to do this, we've lowered we've made it so that we have a significantly more shallow tree.

A

What we've also done is started, including d duping, in how things get installed, which is extremely important, and that might not be completely obvious. When you see something like this now, an MPN we came out, particularly when it was released with node 5. A lot of people were surprised because they looked at the top level of their node modules, folder and they're like what is all of this stuff.

A

Alright, what they were used to is looking in their node modules, folder and seeing their primary dependencies, and it turns out that you can still do that and you can do that by using the NPM LS command. So NP MLS will show you your entire dependency tree, not as it is in your file system, but as it's understood by the mtm ecosystem.

A

And then, if you just want to see the primary depths, you can pass depth zero and that will give you the exact same view that you saw before when you were just looking at the node modules, folder and mpm to all right. So here's another thing that I get a lot from people. How many people wish that they could install offline?

A

Just a few people has anyone taken down like a delta airlines, flight Wi-Fi trying to do at NPM install because I have it's cool, but it could get kind of tricky if you're, using really large things like lots of dependencies, it can be complicated and so a lot of people, maybe don't realize that NPM actually has a global cache on your machine and that's in the dot. Mpm folder, usually in your route like user directory, but it'll depend on how you installed mpm, and so the modules are stored there.

A

But unfortunately you can't use this offline because when you run an NPM install, it's still going to go, try and check the network to see if there's a new version that fits your cember expression.

A

However, you can kind of trick this, and so you can use NPM install dash dash cash bin and you can pass this super big number or you can actually write the word infinity with a capital I and basically this will allow you to use that global cache. So if you're on an amtrak- or I guess that's a very American reference- I'm- not sure what it's called in London.

A

If you're on the train, I'm, you need the NPM install, um you could try and use this and you're actually going to end up like having probably a better time and we've always talked about the idea that we should alias this to NPM install offline one day we might FYI that does not work right now. Oh lots of people wish it did and we're definitely working on that.

A

But there's another way. You can do this to that. I think a lot of people don't understand, which is that mpm also offers you the ability to create a tarball out of your own package. So when you actually npm install or mpm publish you're turning your package into a tarball, that's then put in the registry and then, when you install it you're pulling it back down, and so we actually have an internal command and it's also available publicly as mpm pack and what you're able to do with NPM pack.

A

Is you take your package and it turns into a tarball, and then you can actually npm install directly from that. So if you have dependencies that you use all the time you can just pack them up and then just have them on your machine ready to go, no need to worry about Cashman or anything like that.

A

Now, there's one little tricky thing about this, though, which is that unfortunately, pack doesn't include your dependencies, but there's this thing called bundled dependencies, which will be how you can take dependencies that you list for your package and have that code, pull down and bundled into the package you're using. So a combination of mpm pack and bundle dependencies means that you can just have local copies of the packages that you need to use regularly in your project and particularly you'll be able to use them offline. Now, let's talk about shrink, wrap.

A

How many people in here know what I'm talking about when I say mpm shrink wrap alright, how many people just think this gift is funny like what is going on here? I have a no idea, someone actually sent mpm is Twitter. This and I was like I'm going to use this forever. um So the idea with mpm shrink wrap is that when you're setting something up into production, you don't want to have the dependencies that you have in production or staging or testing be a different version than the ones that you're using locally.

A

It's really not safe. You won't have exactly the same thing and that's a good assumption to have, however, because of how mpm deals with cember that can actually be kind of tricky.

A

So what you're able to do with mpm shrink wrap is you're able to lock down the exact version that you are using on your computer and by lock down every version, I mean every version of every dependency, all the way down the tree, lock that down to a specific version and then, whenever you NPM install, so you push whatever app you're writing up until your production server.

A

It's going to install the exact same version that you had locally so in case you're dealing with any package managers or owners, who maybe don't do some very quite right. You don't have to worry about that. You're able to know that you're using the exact same version in both places and what's great about this. If anyone has you shrink-wrapped before, unfortunately, was a little bit unusable in mpm too, because you always had to keep doing it. However, an MPN 3 if you use the dash dash, save or save dev commands.

A

This is going to update your shrink, wrap right away and you won't have to shrink wrap again. This is going to interact with that. If you have it present and that's a really awesome reason to update mpm three because shrink wrap works, much much better alright. So we get this thing a lot. How many people here have used something like Ruby, gems or bundler, all right, so you're used to a lock file. So your question is shrink wrap. Why is this opt-in?

A

Why is this not a default feature, and so this is because fundamentally mpm believes in cember.

A

We often say we are a bet made on cember, which is that, generally speaking, when we're bringing in versions that are slightly more bringing in a patch version or a future version for you we're getting you the bug, fixes and features that should not be breaking your app, and so we believe that this, generally speaking, is a much better option than having to lock things down, specifically, though, that's obviously how they contested, which is why we offer the shrink wrap function now, because we love stember.

A

So much we've actually tried to make it much easier for you, and so, if you're ever publishing something, you might be manually editing the version in your package.json. But you actually don't need to do that. In fact, you could just use the MPN version command and pass in major minor or patch, and it will immediately increment that up for you, which is super awesome.

A

Now I always like to take this moment to explain cember for anybody here who maybe doesn't understand it, which is the idea, is that cember is kind of a social contract which says for a major version. That's breaking changes, which means that if you update your app will probably break and then for minor and patch, it just means that your app might break actually. Minors feature patches, bug fix talks, but this is a little bit confusing.

A

One thing that people often get very upset about is that when you have to update documentation and things like that, you actually have to publish a new version, and this is something we stand by as a feature, because that we believe that Doc's are actually part of your product.

A

So when you are improving it by adding Doc's, who really should be updating the version all right, so another thing that I think is super awesome that I, unfortunately I, don't see enough people using and was mentioned in a very confusing way by Michael a second ago is mpm scripts. How many people here are using mpm scripts, awesome, great okay, so with NPM we saw like in the original kind of JavaScript ecosystem. There are a lot of like automation, tools, I grunt and gulp.

A

You saw broccoli if you think that I'm just saying random words you're, probably right, um but we thought that this was a really useful thing, and so we added it, and so we have mpm and you're able to add any sort of custom script that you would like.

A

So you can type mpm run and come up with a name and define it, and this is like a great way to be able to just write shell scripts and then automatically do that, but there's a lot of other really awesome things about mpm scripts, that people don't know. So what are these things? Is life cycle events all right? Basically, what we're able to do is we're able to have you define scripts and then those scripts are going to be context-aware, so one of the most interesting ones that I like is pre published.

A

So there's things like publish tests start you can do pre and post these events in the life cycle of your package, but pre publish, is very interesting because what it will do is it'll run something before you publish your package, but it will also run something when anybody uses your package by installing it, which is something that often can catch people by kind of off guard. But basically it's way if you have to do any sort of minification and you're sort of compilation.

A

You can do that during that and it's able to hit at both of those life cycles, which is actually pretty tricky, so I find that that's a very, very fun side note. I am currently writing like a whole bunch of docs, for this so be on the lookout. For that. The other thing that super neat is that run scripts are composable, so you're able to write a bunch of scripts to find them. Package.Json and then you can have them reference each other all right, and that can be really great.

A

So you don't have like a super long script like right now, like you, want to hit the 80 columns right, so you're able to compose them together, but the best part is how many people here haven't had read or read me, and it made you install something globally ever how many people have to work on a machine. We're installing something globally is maybe not okay, all right yeah.

A

So it turns out that if you want to use something that would need to be used in the command line, which is when you would probably install something globally to make your packages more user friendly, you can define mpm run scripts that use packages that you would otherwise installed globally, but install them as a dev dependency and those dev dependencies are automatically in path for your run scripts. So you don't have to install something globally at all.

A

Instead, just have it be a dev dependency of your project and call it in a script, and it's just right there, so it makes it significantly more accessible to a lot more people in addition to those dev dependencies being in path. You also get all of the keys in any sort of NPM, config or package.json as variables for free. So you really have the full context of your project available in those scripts, which I think is super fun.

A

So another couple of things that people don't know- and this is the part of the talk where I'm like mpm- is a company. Did you know about that? Not a lot of people do um so. One of the things that we've started doing is doing scoped packages, which is really just a fancy way of saying namespace packages and in particular, we're seeing a lot of ecosystems in the front end adopting this, because it helps increase the discovery of packages. How many people have used mpm search, how many people are frustrated by it? Yeah it's cool.

A

You should just keep your hands up, we can understand, but we're seeing tons of communities react is doing. This angular is doing this and they're making it so that it's easier to find those packages because they're within that ecosystem scope, and so the way you can do this is you just MPM install and you see an @ sign. You write the scope and you have a name all right now.

A

School packages are private by default, but you can make the public simply by doing this, which means that right now, if you have an MPN username, you actually already have a free scope that you can publish packages under so if you've ever really wanted a package name, but somebody else took it. You can have at your username / that package and that's already available for you for free, which is pretty cool now.

A

The thing that's not free is private modules and, while I'm a huge open source, Finn and I like everything to be public, sometimes it can't be like maybe you're at a company that has some strange off, and you want to package that up and be able to share it and repeat it with people. So that's something we offer now. Some people think that mpm private modules is weird right, because we're supposed to have packages and some modules versus packages. This was a total naming faux pas.

A

We thought it was cute because it's like mpm private modules is mpm p.m. him. It's like recursive. Yet it was a bad idea for changing it.

A

um Sorry about that y'all uh-huh, but the way you can do this since I to start your package with a scope, is you can use that MP emic in it command and you can just use dash dash scope and pass your name right through there, and the cool thing about this is: is that you're able to use these private packages right alongside the public pack just seamlessly, which I think is pretty neat? And so we had released this for a while, and it turned out that managing access to this stuff was super hard.

A

Much like pouring milk and cereal is extremely difficult. Clearly, so we came up with NPM organizations, and so we have this thing called mpm team. So the same way, you can have orgs and github you're able to use orgs on NPM and manage your collaborators and access to packages that way, which is super useful all right. So we have an enterprise product I'm not going to talk about this, that much it's pretty cool.

A

Basically, it means that you're able to have your own registry, that's behind a firewall and you're able to have just a certain amount. You can whitelist packages, so people could only use certain ones, but the cool thing about mpm enterprise is something that we just released, which is mpm Enterprise integrations and I gotta say like I'm, not a sales person but I'm like really amped about this and we're hoping to get this for the public registry really soon.

A

But the idea is that within NPM Enterprise we can automatically integrations that will check for all of the open-source licenses. Your package requires it'll, tell you if you have any obligations or if you have any dependencies that have a license that you shouldn't be using. We also have been working with the node security team, so they're automatically, like checking all the packages for security vulnerabilities, and then we also worked with bit hound to get package metrics on here and so I. Don't know, I find that this is really cool.

A

One of the neat things about NPM. Is there so many dependencies, but because there's so many, it's really hard to think thoughtfully about all of them and so we're trying to automate that away now. This is only an enterprise right now, so, if you don't want to pay a ton of money, I totally understand you want.

A

You want to be able to do this right now, though, like we know that the community wants more and the CLI in the registry are really community assets, and so what I want to end this with is kind of a call to action. First off it was kind of a quiet release, but we have a new repo called the NPM / registry, repo we're first off. If you run into any problems with the registry. Please file an issue here.

A

The registry team and me are like actively answering everyone's questions about this and its really neat, but the other cool thing that's happening here is that I am writing. I'm currently documenting the public registry api here, and so, if you want to work with this data, if you want to build integrations, that's possible, and we have the data here and the docs to tell you how one of the coolest things you can write is a registry follower which means that every event that changes the registry, you can follow and do something about.

A

So that could even give you all of the packages of the registry all of the changes and it this is a follower and it's 22 lines of node just 22 lines. It's extremely easy, and at this moment, then you suddenly get access to all of these public packages, and it's super super neat and I encourage everyone to both write. A follower. Do something cool about it and tweet at us about what you're doing, because we're excited for people to build these things?

A

This is built off of what we call our replication endpoint, which means that if you want to mirror the NPM registry, not only can you, but we encourage you to do it and we will help you do it. This is the end point. You can get every single change from the NPM registry and it's not only just public packages, but it's also all of the scoped public packages now, which is a big new change. So this is an awesome place to get just super-fun data all right as the last thing.

A

What I will say is we do have ninety-nine percent up time, but recently, we've started: we've created this bot called mpm status that checks if your mpm is running. So, if you're ever wondering, is it just me or is there something up with NPM we tweet from this now automatically with all of our status updates, so you should definitely go ahead and follow it. So that's my talk. I hope you learned something and please reach out. We want to see all the cool awesome things you build thanks. We love you.