Node.js Node.js Interactive Europe 2016, 24 Sep 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Diving Deep Into Browserify - Yoshua Wuyts

Description

Diving Deep Into Browserify - Yoshua Wuyts

Browserify has been one of the staple tools for packaging javascript and shipping it to the browser. In this talk Yosh will explain browserify's internals, the current state of tooling and the path that's currently set for the future.

A

um Hey there, hello, hello, everyone, um my name is josh, I I usually don't talk with like a giant echo. So don't be afraid, I'm just a normal human being, um the thundering voice of me. um My name is josh, uh and today I will be talking about a thing called browserify um yep.

A

So I am a human who makes a bunch of npm packages um who's. Actually, I've lived in amsterdam for four years, so I guess I'm local, I'm a local speaker. I think there's like one more local speaker and that's like us, so yay, I'm from here um you might know me from a little framework called chew, which is a frontend thing. It's a little train. That's the logo. um I tweet a bunch yeah I've been in doing node for like a couple years now. um Yeah so today is about browser five um browser file.

A

Why browser file? Yes, browserify.

A

um I'll be uh I'll be going over some some just intro stuff, like yeah, what's browserify like go over the internals ecosystem, some other stuffs, um but um yeah. This is this browser file thing is um it's it's important to know what what is browser? I feel like there's a bit of a misconception there um is this le. This is less echo. Is it yes great?

A

um Oh no! It's not.

A

So um the way I view browser fi is um the same way. I view ls or like cd. It's just a unix command used to compile javascript, it's pretty stable. It takes javascript in it, does some stuff to it like combines it and then like out, comes javascript and well. That's it it's a low-level level tool.

A

I'm you know like. Why would I be talking about this because we will forever be compiling our code, but I feel this is my my personal opinion. It's a bit of a pipe dream to be like we will stop compiling our code. You probably want to be doing gzip. You probably want to be doing minification. You probably want to be doing like at the very least those things and probably like compatibility or whatever.

A

If you look at other languages, they're still compiling so saying, we will stop compiling our javascript is probably not going to happen, and if we're going to be compiling a javascript, we probably want to be using some sort of low-level tool. We can build other tools on top. So that's what this talk is about. How do you wield this compiler tool chain and shape it to like create the perfect thing you want it to do, which you will learn today, sort of hopefully whoa.

A

This is called node interactive. I don't think it was supposed to interact with, like all the um yay, so um back to the talk. uh Browserify is a modular um modular tool, which means it's composed out of a bunch of modules that together create a bigger pipeline, a bigger tool chain. um You can create your own browser file easily pretty much. um It uses uh extensions uh through transforms and plugins.

A

There's a mouse distinction there generally you're using transforms to transform your code from one state to another state, um or you know yeah um and it's it's pretty stable browser file is now version. Eight. I believe they take breaking changes super seriously. If, like an error, message changes they like uh create like a major version along those lines, you can pretty much always rely on browserify unless you make me patch a thing and then browser 5 breaks, which I did once a year ago, don't make me red patches, but everyone else.

A

You know it's it's stable um yeah. This is pretty much how it works. I was saying it's a unix tool. You say browserify your uh intro, like your index.js and you just pipe it out. Instead of like standard out you just pipe it to a file and there you've just compiled javascript all right. um I could demo this. I assume people would like already sort of understand this I'll show demos later yeah if you're doing it in javascript. It looks like this.

A

I really like the if statements are like in blue um highlighting is going a bit wrong, but that's fine. um So what you say is you import browserify as a thing you say browser by this file? Never do it like this, but you know for demo purposes, then you say: bundle the thing and the result of the bundle which is a node stream, you stream to process.standard out or fs.create right stream or your http server to create like a live, http uh piping thing which, which is very viable to do.

A

um It's yeah, it's like five lines of code. If you compare this this, it's it's all like stream based note streams, shell streams, but um yeah. This is basically the basics and if you want to like add transforms, um you add the dot, transform and names in it, and then you like have your whole pipeline. That's the basics of browserify yay, not very deep, yet, uh oh demo, sure.

A

So how do we do this? We say: browserify, uh zero, one vanilla.js and then ta-da. This is like how many lines.

A

Lines we just compiled 564 lines of javascript in less than a second. I don't even know how long it is uh yay browser, fi, whoa,.

A

This one, this one, this one yeah here: um okay, cool, so internally, uh browser file, uses streams pretty much everywhere. um For those of you who don't know node streams. They are this thing uh for asynchronous io. uh Then, instead of like doing a state machine where you say, hey, I'm ready to receive data, hey, I'm not ready to receive data, uh send data from here to there. If state is this, uh you create a little.

A

um You combine all these things you inherit, like constructors and stuff to create a nice flow of data with back pressure, and you use the dot pipe to like do all that stuff and propagate errors and whatnot. um So internally, browser file leverages this to hook in transforms, which are also streams which transform code at their own pace, and all these transforms together. They like transform code at their own pace, and it becomes this very efficient machinery of piping out code, which is low in memory usage and reasonably fast.

A

um It takes in a a it, creates an asd. It um then starts pumping out that asd throughout, like all the transforms it's like: here's chunks, here's chunks, here's chunks and then internally you're, like cool, I'm just going to get this code, I'm going to apply transforms, I'm going to walk the asd using something like falafel. The sd is created using esprima, no acorn.

A

Nowadays, packages packages, um you walk the ast and an ast stands for apps, abstract syntax tree, which is basically a way of saying a giant nested thing of objects like all the way down, and everything has like words on it. That says what it is like. This is the start of an if statement. This is the end of an if statement they go like all these words.

A

I always forget them um yeah, so you apply local transforms first, which are streams, so they apply an individual files and then over the whole combination of that thing um you apply global transforms global transforms, apply to dependencies. Also so say you were to import es6 stuff like const. You want to change all the cons in your whole code base to var you would create a global transform.

A

That applies over your whole code base and takes all the con statements and replaces them with vars, which you could probably write in like 10 or 20 lines, or something which I'm not gonna do yay, but you could, um and the thing already exists uh register require calls oh yeah, so plugins are the other thing I mentioned it.

A

uh Initially, you got your transforms, which are to transform code and your plugins, which are rarely mentioned, they're, like sort of secretly tucked into like the readme, because um substack the person that created browser5 kind of like doesn't want people to use plugins, they're kind of like abusing the thing, but you get to do like really cool stuff with it, um because it takes browserify itself as input, because browserify is just a bunch of like pipes, just a bunch of like streams, um a plug-in like hooks into like between one of those steps.

A

It's like a giant array of streams that eventually like get smushed together. You get to say uh before we like label every thing in the asd. We want to run a transform or like at the very end of the bundle we like want to do. A thing and- and you you get to do that- which is kind of cool uh internally code- would look sort of like this. I I took this from one of my my repos called extract, css, which, after a certain step um before the final output bundle what it does is.

A

It takes all the there's this one package that writes css to like a head element. We would take all of those uh smush them together and output, a separate file and remove all the statements.

A

That's what this code is taken from. But what you say is in the python yeah, it's readable in the in the in the pipeline.

A

You get the label statement, you say select this thing and before that add the extract stream, which takes out all those things and then the next step is applied after that there's, like eight eight steps, you can hook into um yeah, which is not a lot of work, but all of this is goes like very undocumented for good reasons, but it allows stuff like, um for example, watchify, which is a does partial builds, which is great for both cpu, because it doesn't use a lot of cpu.

A

It doesn't do the full rebuild all the time and it's extremely fast because it doesn't do the whole thing anytime every time, um but it needs to hook into this thing, so it can inject like new data, it watches the file system and it injects new data into like the stream and, like the rest of the thing, just continues.

A

I hope that statement made sense. It hooks into this thing and that's where plugins are useful for um yep internally. I explained this already ecosystem yeah. This is where I get to tell you like hey: this is the stuff. You probably should be using um there's a lot of stuff you could be using for browserify.

A

um Just like I don't know, 200 modules. I think I got internet, let's see if into live internet. That's like live demos but scary. There's like a lot of plugins. This is like half the list. I believe this list hasn't been updated in like months um but yeah.

A

If you want to optimize your your code, if you've got like a bunch of client-side code and you're like I want to optimize my code, um probably uglify is what you want to be using because it it just uglifies your co. What's it called minifies, it removes that code, simply um there's bundle collapser, which replaces long uh file names with tiny ones.

A

um There's this stuff called what was it not acorn the tree shaking thing roll up, which does a bunch of optimizations you can get like half the optimizations roll up does with some of these sort of it's a bit of a statement.

A

Actually, regarding rollup, you probably don't need that people might be wondering like hey, but you know I've heard we're talking about optimizations. I've heard like rollup optimizes a whole bunch of stuff. Someone recently wrote the blog post about it like hey. If you've got a thousand modules in your like front end system, then you know, roll up might have a boot time.

A

That's 0.5 seconds faster and it's like yeah, that's fair, but if you got like a thousand files, a thousand modules in your front end system, you probably have bigger problems than a 0.5 second delay in your startup time. I I don't know um um yeah. I don't think that is worth it like. These are the low hanging fruit and, if you ever like want to optimize beyond it, there's probably options. um Let's say: yeah bundle, collapser collapses, a bunch of names under certify removes all assert statements.

A

um Nasa has this coding guidelines, which is kind of funny they're like every function you ever write in c should have like three asserts at least, um and I was like. Oh that's funny. I'm gonna do that with my code, so I'm doing that with my code. I've been doing that for years now, and actually it's pretty nice, some people like typescript.

A

I literally just write up my types, I'm like assert that equal type of this thing should equal this thing, here's a custom, error message and you know it just works and no need to compile unless you optimize it and then you like strip out all the asserts- and you know javascript for me- is almost like statically typed.

A

I also got rsi. I don't know if those two are related, I'm not sure but yeah you can do that um and exorcist is the last optimization you can make. uh You can create uh source maps uh with the dash dash debug flag in browserify, and then exorcist takes those source maps and exercises them into a separate file. They're like no be gone here, separate file. um So that's that's useful.

A

If you like serve those too uh cool things, uh sheetify which uh hugh my friend hugh and I wrote which is css in javascript, so you can just be re almost like require this css file and then it's in there you say: uh cons sheetify, then sheetify, tachyons and poof. You got tachyons in your project, which is amazing uh brfs, which is uh um which is the file system. The browser file system, like kind of like that word, it's um it's file system, but for browserify, because you can't read files from the browser.

A

What this thing does is it reads files ahead of time and then includes the stringified content into like your bundle, so you're like I want to have an svg in my thing, just like compile time, it reads out the svg in lines it and there you have an svg in your bundle and if you like, want to optimize from there, you could like treat it like import the svg and then optimize export it and optimize it, which is the way we do css, which is similar to like how webpack does it sort of at least an idea.

A

There's bulkify, which is bulk imports you're, like I've, got 15 models in my slash models directory and we all import them the same way. It's a lot of boilerplate and we always forget to import stuff pokify. Just like. Does here's a bulk of imports, cool um watcher 5 for, like the reloading stuff, css extract to extract css from your browser.

A

Javascript bundle into like a separate css file. I'll come back to like this one and tachyon's sorry sheetify later then, there's like wrappers around browser browserify, because so so far I've been like mentioning like here's, a bunch of tools. Here's how you do plug-ins, here's how you do transforms like look at all these things. You can combine them together and my it might sound like a bit of an effort, uh and it is so people like built wrappers around all of this, so you can be like ah I'm just gonna.

A

You know, use a wrapper and not worry about all this stuff and hey. Now, it's a very nice dev experience. uh Budo is probably the one you want to be using the other. Two are a bit older, though they're still good, um optimizing, oh yeah bundle!

A

If I is the same as budo, but for like production builds, it applies like seven different optimizations to your code, so you don't have to which is nice uh and there's bangkai, which does all this stuff, but I'm gonna, I'm gonna, show bangkai is that the only I'm gonna show him more than that cool for the demo part. I present.

A

um What am I doing with time? I think I'm all right with time. uh So if we do bruto01 vanilla.js, let's show vanilla.js actually is this vanilla? Yes, this is vanilla, so in vanilla.js, what we're doing is we're saying import, html, uh import, yeah import bell, and then we make a tag, template string, which is just html, so we could like render that and we say append this these elements to the dom. It's like your hello world right and in line.

A

We just put a little style tag which says: make it a big font, so y'all can see it uh well, that's a fun, sound, um okay, cool um and out pops. This thing yay, um if I do bruto dash dash, live, live and we got like a live reload running uh yeah. So this doesn't do anything whoa, whoa where'd. My thing go: does this even work? Oh, it's a different thing. I think is it? Oh, my god do we have errors so dash dash live is supposed to like give.

A

You live reloading on this thing, but somehow it's not working great uh anyway. It's not buddha that I wanted to show off there's zero, two which is css.js.

A

So this is just um we got style tech in here, which is says 200, pixels and you're like yeah. That's not ideal!

A

uh So there's also this one which uses sheetify, which in lines a bunch of css into like your do. I need to like up the yeah it inlines. uh It creates you, you just say: hey here's, a here's, just real css. This is real css. Look, it kind of looks like real css um and you it gives you a class back a prefix.

A

It's a unique sorry, it's a unique prefix and you just append that to somewhere in your html and then all these things get like compiled together and then, uh if you run this, which we are already running this hello humans, oh god, why is it not working? This is the worst? No is it no wait? Wait, wait, wait, wait!

A

That's right! Budo was having hiccups because they've patched it recently and that is not great.

A

Bangkai bankai bunkai start entry this one, so I wrote a different version from budo I used to maintain voodoo. I no longer maintain budo because I've got different opinions. I was like you know, building stuff like having a live reload service cool, but I write my http service by hand. I use streams I kind of like want to aim for the 40k request. A second thing, so I like need to buffer everything.

A

I probably want to use a cdn like do all these optimizations and- and I I like- building my own stuff, so I build up bankai as a server library first as an alternative to budo, um so you can just create your own own server and then two weeks ago someone came along and was like here's the command line tool, and now we have a command line tool for bankai, which has a start and a build command. This is a start command. I think it live reloads, I'm not quite sure.

A

If we say hello, you does it reload, hey it works, see you just get to like update this probably have like other stuff soon, um but the coolest thing about uh bankai is you get to say, build uh which is bankai build whatever file. It is, I know, dash dash entry and dash dash. There is too much typing.

A

We need to change that to a nicer interface, but you say this command and if you look in dist, which we just created the little directory there, you get your bundle at css, bundle.js and index.html, and you get to pass an optimized flag. So all that stuff I was talking about, like optimizations extracting of source maps. Minification doing all that stuff um is built into this one tool. uh Bruto has doesn't have like a build step. um I believe webpack does um yeah.

A

uh Where was it going with this? Oh yeah, if you're like doing browserify and you like- want to use it in production, this might be useful thing to. Like start, try be like uh how can I compile a website um small commands and it's built on top of, like the modular browser, five core? I think a lot of people are like starting to build on top of that and we're like creating a nice little little ecosystem of higher level. Tooling, like it should be um yep, that's bankai and that's my talk.

A

Thank you very much.