npm npmCamp Presentations (2016), 30 Aug 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: npmCamp 2016 - Tonic by Wolf Rentzsch

Description

npmCamp 2016 - Tonic by Wolf Rentzsch

A

I think that was a great talk from Isaac. It's a great day too I guess I knew most of the background of a company and product. I love. I was also especially excited by the Bojack horseman reference. I, don't know how many of you got that I.

A

Almost they made the same reference, so I think we have like minds, I'm also incredibly honored to be the first non mpm speaker, not an MPN Inc speaker at the NPM conference.

A

So I am full French one, the few people whose names can fit into emoji and I work on this thing called tonic and you might remember the name tonic you might have heard of it, and that's probably because you go to this website a lot and over there in the sidebar of every single package on em pm's website is a link to our website where you get to try out note packages right in your browser.

A

So this is what the page looks like if you actually to click on it, and you can see I click on load dash here. So you have the load text, entry field here and we've even written the hardest part of it for you. They require for you.

A

So then the zoom in a little bit here and you can type some code and get hit the Run button. It stop happens. Amazing stuff happens, you see, there's a little result down there and so I. I typed in little bit of code hit the Run button and you get resolved. So the thing that I want to call attention to is the things that you didn't have to do. You didn't have to think of a name right, daming and cash in to be validation right to do hard.

A

Things like a I know about you by always pond anyway, I just always named everything foo that turns out to be a problem, so don't have to grade another directory called through. That doesn't mean anything, don't have to initialize a bogus package.json. That's because you want to try this one package out, don't have to save or anything like that, and you don't have to you. You know, then, a by the way. This is my I want to show other agnostic here. I use the dollar sign editor. So no one beats me up for anything.

A

You don't have to create another file right just require it right. Just remove all the friction, make it seamless it's linked right from the MQM package page. It's totally easy. You can start exploring package immediately.

A

So we like to think of tonic is where code starts so you're, like oh I, want to do this thing. Oh I bet there's a package for it because guess what just write a thousand of them you're, probably right and you just you- do some searches, click on it start typing. Some code start exploring.

A

So a couple things I want to call out about this page because lob, it isn't obvious. The first thing that is very non obvious is that we support a bunch of different nodes at the same time. So, if you want say oh this chunk of code, you, how does to run an older version nodes, that's what we have deployed or how does the future look? For me, it's very easy. It's pop menu had run again. You get the output, the other thing that I want to call attention to is the URL on top gear right?

A

So as neatly so you start playing with this package right. So it's a brand new package. You want to guess you want to explore it. Why and get some work done and it starts building up right. It's like! Oh, that's, actually, working I'm getting a feedback. This is something that actually makes sense to me so that URL, while you're typing we've saved all your work right and you click on link. It's now a notebook. So now it's ready for even more work and it's at a permanent URL that you can share with people.

A

So the idea here is that make it seamless right nothing to install the start. Using starts fluoride switch engine on the fly nvm. I love you, but a problem menu is even easier. We want to make this a great javascript velvet development experience, even if you're using an older node will use Babel and actually a custom superset of Babel. So you can use things like a sink await.

A

You can use we even when you're targeting no 10 0 10, so we want to make it easier because, like this is not, this is your top level, where you're playing with it we're playing with a new package, and you want to see how it works. That can be even if you have to use an older node to support all the package or something you still can be in the future, we're a website.

A

So we can do all sorts of cool stuff that terminals can't do like visuals visualizers, which I'll tell you about ira mentioned auto saving as you're typing, we're always saving that URL is shareable, pass it around. You no longer have this directory that has a package file on there, I guess maybe zoran Dropbox, or something like that. You showed someone else, it's already there and something that I won't get into now, but it's also reusable that notebooks can require other notebooks.

A

So I want to talk a little bit about how, because we're website, we can give you like a better experience of a council like a development experience live development experience.

A

So if you use the chrome or web or a safari web inspectors, you know how great they are I like showing you the little icons for different types of objects and drilling down and give you a great display that I love notes rebel, but you know we techno dance like this is good, pretty much you can get in a textual form, but we all know that the web development experience with inspectors is really fantastic.

A

It is now so we have visualizers like you'd have in the chrome, inspector or fire inspector, like all the ones on the left. You just assume that you would get you know, everyone should have them, but the ones on the right are something a little bit special that we will detect when things look like HTML and will render it for you. Well, if this, if you looks like you're more who you're getting data back, then it's you can say display this as a chart.

A

This is all this built in, don't have to download a package for this is a week. This is what our object inspectors can do buffers. This is really great when you're dealing with binary data, and you want to actually see some ASCII embedded in a stream of bits, URLs nice little hidden feature here- is that if this thing looks like a URL, will let's go get it for you and then well. You know it's real easy. You know you have to download a supper packet, Willis fetch it for you and show you the results.

A

Our regular expression object. Spectre is great for iterating on a regular expression. I, don't know about you, but I tend to be often like off by one or like. What's that special character or something so this is. This is not just me trying to put in a sneaky reference to a switch band.

A

This is me actually like showing you that what's the A's are highlighted, that we will show you visually, where your patterns match, and one of the nice things about tonic is that we also support geo JSON that we, when things look like coordinates, lie. Ladron longitude will put up a map and put some pins down there for you automatically.

A

This is an interesting little project at guy wrote where he there's a there's, a website that shows a bike bike share stations in Barcelona, and so he looks like hit that API got this list of G huge, a geo JSON, and we already knew about display it.

A

So I mentioned the best possible JavaScript. So, like I said, even if you're using an older note, if you have to go to a node, 0 10 node, 0 12, you still can use all your modern features that we have in node 6, including stuff that we don't even have yet a way to sync.

A

So I want to talk a little bit about correctness and I'm, glad that Isaac's everybody's, like brought up semantic versioning, there's, obviously love play there, but I want to talk about things we take for granted that don't necessarily map onto correct correctly functioning software. So the mental model here is philosophical that you can't step in the same river. Twice river keeps on moving so dip. Your toe in twice has a different River right. So imagine you npm install something you will you go go away for a day.

A

You nuke your node modules directory and then you install again question is you've run npm install twice are those business? Are those bits identical.

A

Barely so the answer is mi still might okay. The answer answer is, it depends, even if you're your own dependencies haven't been updated. If you have the same package.json and you do an update, even if they your semantic versioning, it allows a like a version upgrade for st. bug fixes even those didn't change. Even if yours package.json didn't change, maybe some other dependency, three levels down changed.

A

So one of the things we want to do is that we want to have reproducibility, so we want to have notebooks be able to be used for given to this version of node and this package. This is the output, and that is consistent for their end of time. Right. You want to allow people to use notebooks and rely on the results.

A

So what we do is that we internally timestamp all packages we've downloaded and then, when you asked for it we've written a file system, we call module of s that does the math and temporarily tracks things so that essentially it's we're running, shrink wrap for you all the time, so you get that reaches reproducibility for free.

A

So this is not an actual command, so I don't recommend you actually try to do this, but we have so. We want to make all of em PM's power all the communities mazing contributions. We want to make it instantly available, so we thought about a model like downloading on demand type stuff. We didn't want to have to wait to wonder you even milliseconds right. We just want to have everything there immediately all the time, so we did that we've downloaded olivine and gam ahead time.

A

That means yes, every package and every version of that package, and this is updating real time. So any guys pushing right here instead of listening to me, publishing modules is Sahn tonic right now.

A

So how do I do this? Well, npm is really great. They have that couch DB heritage right and they've always been very free with their data right and so one day n points is this: there's mpm has a single skim, skim DB right. It used to be like the use couch and they put not only thus the packages then now just like the meta data for the practice, but also the package data itself, like tar balls inside of couch.

A

That was a bunch of scaling issues that isaac mentioned there before they had to get away for with that. So for a long time they had the full fat database that had the packages and he had the skim database. That does had the metadata. Well now they had to shut down full fat because that was couldn't scale, so they just had skin. So skim is what we use, and so we have this couch feed that for council listen to, we get the changes in real time and we download the tarballs.

A

We extract them put in our database and instantly available. So in your notebook, so you can neatly go over type type in the NP MJS, the URL, with the package in a minute, click the tonic link and we're worried there were your latest version.

A

So I thought, since we've downloaded everything, I could give you some interesting statistics here we have. These are the numbers I got from last night, three over well over 300 thousand packages.

A

What people don't talk about, though, it's probably because it's like less easy to access is how many versions of those packages- and so I was hoping we'd break 2 million. So I could you know, put on my party hat here and we all could celebrate together. I guess we're going to celebrate together anyway. So it's not really a big deal, but we're dark clothes, because we've downloaded every single, every single tarball and we extracted them all. So they all could be readily instantly ready.

A

um I get to do a file count here so that 162 million files currently an NPM. That's a lot that ends up. That is a taking up three point, two terabytes of data. Currently, so we did npm install and got that all working, and so we have every single package ready right, but then like well we're doing this talk and like these stats are really interesting. It really amazes me, as we pull that off, but what crazy things greed to you for this talk.

A

So what we thought is that what would what happen if we just try to require everything- and I don't know- I think that almost looks like a japanese emoji thing like a bunny or maybe but uh but the idea is like we've got 300,000 packages.

A

What happens if we just iterate through all every inquire them and towards that end, we're like well what interesting six statistics can we have, but we have this cus of file system module fasts, like so instrument, is instrumented to admit some data, so we could like what what's going on that turns out to take three weeks: yeah, that's the typical while, but it all balls out his lovely charmed.

A

So I can give you some insights that no one else has because no one else crazy enough to try to do this stuff average NPM package when you require it, just tries to open 68 files, that's a little bit higher than expected, but is not outrageous, of course, is always outliers. The biggest was sub thousand files. 7800 there's one crazy package out there that really likes to stat the file system.

A

Does it twenty two thousand times, but probably the interesting thing is how what's you know how much data does actually get brand and that weighs in it? That's about a half, a megabyte that actually reads in oh and the maximum is seventy four megabytes I'm thinking, someone put their photo album and mbm or something I, don't know so I'm you know I was I was wondering whether Isaac would trot out this chart, but I thought I. Maybe if maybe we'll come later, but I I think is this is a fantastic chart and it's really important.

A

So this website is module, counts calm, and this is a growth of NPM compared to all other package managers, and you can see the trend across the mall is generally increasing. That's a great thing, I think it's a really healthy in terms of community health for developers to be sharing the work and it helps us all right. This is why we're all here we all know this, but NPM stands above and it is amazing.

A

Javascript is a very welcoming language and mpm is a very welcome package manager. This is a growth for of all time. You can see this yeah, that's the one, those exponential curves that freaks out people when you actually already on the implementation side of things, but as users, we really benefit from this.

A

So this is this is not a small thing right. Mpm is amazingly successful. The most successful thing out there and small modules I'm, so glad Isaac brought this up in his talk right. It's not yeah I, don't get me wrong. Npm has a vantage that there's the small modules type of philosophy pet and to helps bring these numbers up for EM p.m. but I think, even without that that philosophy, the numbers would be above the rest.

A

This is a not a small thing, because I think this is the future of programming like us all, breaking down the problems into composable chunks into finer and finer grain, ways that make it more composable so that we can spread the spread, the load as spread the wealth and that's why I'm so excited to be able to use NPM to build my projects to contribute the MPN to help other people solve the problems I've solved and with tonic, make it a little bit easier for us all to use the bounty of NPM. Thank you.