GitHub Universe 2016 - Flight, 28 Sep 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Making Electron Development Simpler, More Pleasant, and More Productive - GitHub Universe 2016

Description

When you combine the best of web and native with a framework like Electron, you give your app superpowers – but, with great power comes great responsibility. Taking a hybrid approach to application development brings with it a unique set of challenges. Machisté Quintana, Software Engineer at Slack, will teach about the tools the Slack team has developed to help reduce friction in Electron development, letting you use the latest and greatest from the web community (like React, ES2016+, TypeScript, Less, and more) natively in Electron, from local development through production, all without breaking a sweat.

About GitHub Universe:
GitHub Universe is a two-day conference dedicated to the creativity and curiosity of the largest software community in the world. Sessions cover topics from team culture to open source software across industries and technologies.

For more information on GitHub Universe, check the website:
https://githubuniverse.com

A

All right thanks, everybody for coming so and thanks for coming right after lunch, I know what that can be like. So my name is machiste I work as an engineer on the desktop team at SLAC and I'm here to talk to you about making electron development simpler, more pleasant and more productive and, in particular, one of the tools we developed to help. You do that. So, as I said, I work at SLAC, where we're trying to make people's working lives simpler, more pleasant and more productive and I'm.

A

Here today, to tell you a bit of a story you can think of it almost in a sense, and this will make more sense as we go on. You can think of it as a superhero origin story. So I'm going to tell you the story of electron compiled so electron compiled is a tool that my team developed to help make working with the latest and greatest from the web community much easier. So it lets you use things like typescript, Babel, les stylus natively within electron without having to add a separate, build step or compile step.

A

Or what have you, which you know? That's a time you time you spend writing this. Is time you're spending not working on your app so so, to give you a little bit of background, because everyone has sort of varying levels of familiarity with electron, why? Why is a slack using electron so turns out that slack ships on a lot of platforms, so we support Windows, Mac and Linux and different builds for each platform, and this would be a bit of a nightmare.

A

If we had separate native clients for every platform, we would have to have separate team. Dev teams, separate QA team, separate design team even- and that would add a lot of overhead- and wouldn't it be great to if we could somehow reuse the amazing work that our web team does and so for this, the slack desktop app is built on electron.

A

But uh what is this electron thing again? I mean it you know may have heard of it may have not. Hopefully you went to the earlier talk, but even if you did in what is this electron thing, so electron is a platform for developing cross-platform desktop apps with web technologies. So that's like the the line that you often hear, but one of the ways I like to think about it is that electron gives web apps superpowers. uh You know something like that.

A

You know like you're traveling through a radiation storm in space, you come out with like building smashing power, it's kind of like that, so normally a web app. If you have worked with in the web before. As you know, a web app is just a web app and it's limited by what the browser exposes to it. So the browser is usually set up a really tight sandbox. So you can't do things like access files on a user system.

A

You can't, like hook into native, desktop api's on different operating systems, and that means that you're a bit limited when you're working on web apps you're limited to whatever the browser exposes as an API or whatever javascript exposes, or whatever the Dom exposes, with electron you kind of get the best of both worlds, the best of native desktop development and the best of web development.

A

Because, all of a sudden, you can write your UI as if it were a web. App and have access to these lower-level api's, so you can do things like pop up native notifications, for instance, something we rely on very heavily for the slack desktop app and access the filesystem and do all kinds of other really cool things. So out-of-the-box electron works with the things that chromium and ojs work with out of the box.

A

So electron is a runtime, that's built on both chromium and nodejs chromium being the non-google portions of Chrome and combines them together into a single runtime, and that means that if you're developing an electron app, you get access to all the things that chromium supports and all the things that node supports and out of the box. Both of these things support HTML, CSS and JavaScript, or HTML and CSS in the chromium part. But what, if you want to use some of these modern web languages that are making the rounds?

A

What if you want to use a really new fancy JavaScript before it's a support, is added officially in chromium? What, if you want to use typescript? So you can have optional typing, you know. Maybe you want to use react. You want to use JSX and that needs to be compiled. There's less and style is Jade for compiling to HTML.

A

There's been like an explosion of alternative web languages that compile to HTML CSS and JavaScript, and would it be great if you could use them in electron? Only thing is that with most projects with web projects, there's always some kind of build step, some kind of build pipeline involved.

A

Since these long just have to be compiled, you have to actually compile them before you can use them in the browser. So this can often be a source of frustration for users who are new to this kind of development, and you know you have to set up a custom build pipeline every time. There are great tools for this on the web. You could maybe set something up with web pack. You could set something up more custom with gulp and grunt and browserify some tools for writing. Web build pipelines, but electron isn't exactly a web app.

A

It's not exactly a desktop app either you get the best of both worlds. So what if there was some way we could make electron natively understand these languages without a build pipeline, no build step whatsoever. You can just like in a script tag, start writing type script and it'll just work. What is there somebody could do that I mean that would be pretty cool right. It would save you a lot of time.

A

You wouldn't have to spin up custom build pipelines every time you work on a new app, so there is this is electron compiled, so my team developed this to save ourselves from this exact problem and we have open sourced it to the world so hopefully that other electron developers can make use of it. So how do you how to use it? What is it even so?

A

An electron app is structured exactly like a standard nodejs package. If you've never worked with node.js, there is a package.json manifest file in the root of the repo, usually- and that's where you declare some things about your package so metadata.

A

What the entry point of your package is, and importantly, your dependencies, and that is exactly what you do with an electron app so first step for setting up any electron app would be to set up electron as dependency if you want to use electron, compile and get to have all this fancy native support for these modern web languages, you just install electron pre-built compile. So this is a version of electron, that's pre-configured, to work with electron, compile you don't have to do any setup whatsoever. You just install this one dependency, okay, great! Now.

A

How do you run it? So? The other thing that's very common for a node package. Is you want some kind of start script, just something you can run from the command line? It's quick and simple that lets. You start up the electron binary, so you can add, as you don't really need it, you could still do it without a starter script, but it's a common first step for any node package and also for an electron app. So this will just make it a lot easier to launch the electron binary.

A

That's it no step three! You did it well done. Okay, you're thinking like I didn't have to do anything but, like surely I might want to configure some of how it works. You can do that too. So, like maybe you want to pass some option say you're using Babel, because you want to use some of the latest es2015 and Beyond features like async await.

A

So maybe you want to pass some options to how it works like maybe you want to set some presets tell Babel how you want to like what it should be compiling what shouldn't be compiling? Maybe you want to include source maps, so you can debug much more easily so for that you can just add a dot, compile RC config file. So this up is just a simple config file.

A

It takes the same options that you would expect for each of the compilers that electronic compile supports and I'll go into a bit more detail on that in a bit. So maybe you want to do different things depending on whether your app is running in development or in production, and so how would that work? It's the same thing. You just add another entry to this config file, and then it will do different things depending on your environment, and that's it that's all you have to do so.

A

This is all well and good in the abstract, but might be wondering. Has this work in an actual electron app? So why don't we dive into one.

A

Ooh spaces are not working today.

A

There we go all right, so let's take a look at what we're building first, so I mentioned that one of the first steps is to add a start script, so this is so. It makes it very easy to launch. So all we have to do is run NPM start. So let's take a look at what we're building yeah. Okay, that doesn't seem too bad right.

A

Let's see, I wonder what happens. Maybe if you click on it, something will happen: yeah, okay, so it loads like an emoji wall from somewhere shouldn't be too bad. So, let's dive into how this app this very contrived demo app is structured.

A

So this is the package.json file. If you have used worked with node.js before this should look pretty familiar if not I'll run through it a little bit. So this just describes some metadata about your app and it's what nodejs uses under the hood to be able to launch your app and install dependencies. So we have a pretty simple one here: I have a few dependencies. We have react.

A

We have some babel presets, so we don't have to configure all that manually and we have electron pre-built, compile which is really doing the work here. So just to prove to you really quick that this is in fact built with web technologies. I'll do something that you can only really do in a browser, open, dev tools, so we can inspect the structure here and you can see it's just a bunch of nested divs, nothing too crazy, so this is indeed built with web technologies, but the more you dive in the more the plot thickens.

A

So let's, let's take a look here, so you can saw that the you I was HTML, so that might be a good place to start. So this is the literal HTML file, that's being loaded as part of the as part of this electron app. So it's a very simple there's not much going on here we have a title. Okay, this is the first thing here. That's kind of interesting, so we're loading some CSS directly from a less file. Less is a language that compiles to CSS and is not supported, natively in any browser.

A

So how are we doing that? huh Okay, we'll come back to that? We have an empty div with an ID. So maybe that's where the app is getting loaded and then we have a script here. Our main script, that's getting loaded. So probably the main script is where we want to go next, so let's see what's happening there. So this script is using es2015 modules, which is not supported in Chrome or any other browser that I can think of at the moment yet and yet we're using them directly in this HTML file.

A

We didn't set up a compilation, step of any kind, it's just in there and we have these very nice-looking import statements. So these are just loading dependencies, so we declared react, react Dom as dependencies in this package. Json file, so we're just loading those into this file and we're loading some component here it looks like called main and then we're just rendering it into the Dom.

A

So if you tried to do this in a standard web page, it would blow up and yell you you probably don't want do that, but it's not like you can see that we have a functioning app there weren't any console errors so interesting, wonder how why that's working?

A

Maybe we should take a look at the main component, that's being rendered. So this is the main UI component that you see and if you've never used react before this might look a little weird but bear with me. So we have this main component that has this render function, and this render function has some really weird syntax if you've never used react before. So what is this fake, HTML looking language, so this is JSX and react, uses it to put functions that bind UI to state.

A

So it looks really weird, and if you tried to do this in a browser directly with no compilation step, it would also blow up so we're using yet another language that needs to be compiled, natively an electron, and yet it's still just working. Somehow that's very interesting. You know I wonder if the plot thickens even more, if we look at the last component, so we take a look here, we're conditionally rendering another component and so I'm wondering what this emoji wall component looks like.

A

So you can see, we have some more yes, 2015 modules going on, but really interesting is that we're using some async await. So this is part of the es, 2016 I, believe spec and is not supported natively by any browsers yet, but we're using it here again with no compilation step, and it just seems to work so if you haven't worked with async await yet, basically, this is just a way of making asynchronous code flow much better, so you can basically everything happens in the order.

A

That's written, so you can treat asynchronous code where you might otherwise use callbacks or promises or something, and the way it's written is the order in which things are run. So this the set state here will not happen until these other things that we are waiting will happen.

A

So that's interesting. How are we able to use this when we haven't set up a build step? So that is the magic of electron compile we set up zero boilerplate, and yet we were able to write with all these modern web languages, and it just worked so I mentioned earlier- that you can configure electron, compile to do different things depending on if you want to pass options to any of the underlying compilers or if you want to do different things in production versus development.

A

So we do have the compiled our C file in here that we're using to configure this. So you can see we're doing slightly different things in development, in production and so for in development. We are adding sort source maps to our JavaScript file. So that way we can debug it much more easily since that'll map to your original source, instead of looking at like the compiled jumble. But we don't really want that in production because I'd add some overhead so and that's it. These options that you're, seeing here we're passing directly on to babble.

A

So these are options that babble supports and you can do that with any of the languages that electron compile, supports. Out-Of-The-Box.

A

A

If you're curious about.

A

Wait for this to come back here there we go if you're curious about how this works. You can visit this link. This will take you to the repo for this demo app and you can check out how it works with electronic compiled. So we took a bit of a deep dive into how this works in development, but huh you know how does it work in production?

A

So maybe we're like you know. Maybe you can take some guesses to how electronica pile is working behind the scenes, but maybe it's like so it's compiling files only as they're requested or something, and if that's the case, that wouldn't work in production, would it like you, don't want to add the overhead of compiling files on every request that would make your app very slow. So how? How would this work then, like maybe there's some kind of pre compilation step? Well, you want to use it in production.

A

The easiest way is to just package your app with electron, packager, compile so electron. Packager is a really amazing community project that makes it easy to distribute apps for any platform. So you can output a package for Linux for Mac for Windows and makes it a lot easier to do so with electron package or compile it's a version of electron packager that automatically pre compiles your source files for you. So you again don't have to add any extra build steps for this.

A

If you really wanted to, though you could also integrate this with a some kind of custom, build and release process because there's an API you can use, but you don't need to. If you don't want to and as for using in production, that's it you did it there's nothing else.

A

So, to sum up what you get with electron, compile you get something that just works out of the box. You only really need to configure it if you want to, but you don't have to another thing: that's important to note is that it's a declarative by default and imperative choice. So what I mean by that is in a similar vein like if you don't want to configure it you don't have to, but if you do want to configure it, it's a declarative interface.

A

All you have to do is like define some things in a config file and that's it, but there is a more advanced API should you want it, and all this combines to mean that it's really ideal for quick and easy prototyping, since you don't need to spend time setting up a build pipeline, and it's also really useful for prototyping web apps, not even just electron apps, because if you were just working on a web, app you'd also have to set up a build pipeline and all this boilerplate and you could just get working on it right away with electron, compile, even if it's eventual target will just be the web.

A

So now I don't know. Now you might be wondering how does this even work? This is weirdly magical like well. It turns out that there are features of the electron platform that make this possible, so wonder what electron powers make electron compile possible. So why don't we go through them and then you'll, hopefully get a better sense of how electronica pal works under the hood, so electron power number one? You can intercept protocols so protocol being something like a file. Colon slash, slash or HTTP HTTP.

A

These are all protocols that the browser supports so with electron. You can intercept them and return anything you want in response, so electron compile takes advantage of this by whenever a request comes through, we hook the file protocol and whenever a request for a file happens, we instead go and compile that file if it needs compiling and return the compiled result, which is really awesome and makes it is a big part of the magic.

A

So what else might what other powers might there be? That we're taking advantage of?

A

Another thing is that you have full access to node built-ins, including require, and this is really important, because file requests are not the only way to load a file in electron.

A

Since electron is chromium and node combined together, you can load files via a standard browser file request, or you can also require them, so requires just the way you load dependencies in a node module. So we also need some way to intercept, require and return compiled code, and so we can. It turns out that node provides a hook that we can use in order to return compiled code, and so that way we have all of our bases covered, any way that you request or include a file it'll just work.

A

A

Another thing that's very related to this, but also critically important- is that you have full access to the file system, which is something you do not get in a standard web app. So this is really important because you don't want to like. If we've compiled the file, we don't want to recompile it every time if it hasn't changed. So what we do is we create a write through cache that persists to disk, and if a file was already seen hasn't changed, then we don't need to compile it.

A

So we'll just return the cached version, and so that would not really. It would be very difficult to do in just a standard web app if possible, I don't even know if that would work at all.

A

But this is something that electron is a platform provides, and it's incredibly powerful for this exact kind of purpose, and also incredibly important for all of this is that everything I've been talking about is platform agnostic. So you don't have to worry about the differences between like NTFS and like Mac file system and Linux file systems. You don't have to, like you know, figure out where all the paths are for every operating system.

A

Instead, you can just rely on node and chromium to extract these things away from you, and you can just focus on exactly what you need to for your app and in our case we use that very heavily with electron, compile so that we don't have to worry about that as we're like writing things to disk as we're accessing underlying OSAP is so by these powers combined we get electron compiled.

A

So this is a very complicated looking diagram, but I'll try to walk you through it. How each piece fits together so we'll, let's start with just a regular request for a file. So we, as I mentioned, we intercept this file protocol and since we intercept it, we can do anything we want before we return a result. So what we're doing is we're passing it on to our cache and what we want to do is see. Is you know, hey have we compiled this already?

A

If we have and the file hasn't changed, then we probably don't need to do any work. So, let's not do extra work and let's return the cached version.

A

If something has changed so we use the file change cache to determine. If any files have change, then okay, maybe we have to recompile it. Maybe you don't like it's possible that the file changed, but the code is virtually the same or like functionally the same, so you don't really have to compile it.

A

Maybe you do and if you do, then that's when we go and run through any number of the languages that need compiling, so it could be for babble for typescript for less for Jade or pug, and then we go ahead and cache that and then return the compiled result to the requester. This it works. Basically, the exact same way for nodejs requires. We just happen to intercept something different, so this is all made possible by electron.

A

If you were writing a native app to do the same thing, you wouldn't have this sort of innate knowledge of web technologies. That platform like electron, gives you and also the power to do things like access the file system to be able to create caches, so might be. Thinking now, like hey the power to intercept like any arbitrary protocol, sounds really awesome sounds really fun. It also sounds really dangerous like what, if your app got like man in the middle, then couldn't they just like make your app do anything?

A

Well, you may have a point there. So, with great power comes great responsibility, so I'm going to run through a few best practices that my team has encountered from working on electron, apps and working on electron apps with electron compile. That should hopefully help make your lives easier. So continuing on from that, this is very important.

A

Don't embed remote content without a sandbox so as I was mentioning, if you so one of the most common things people come up with when they want to when they're like curious about electron is something like this? Like hey I already have a web app. Maybe I can just reuse it and embed it into an electron app give it some special powers and then boom I have a desktop app well I mean.

A

Yeah, it's a trap. So problem is that if you are loading you, if you are embedding remote content, then should you ever get man in the middle? The attackers will have complete unfettered access to your users machine, so they can like copy over all their files, like do pretty much. Anything wreak total havoc, but only if you're embedding remote content. So there are a few ways you can not do that.

A

Electron has a built-in sandbox element called webview and you could- and this works much the same way- that Chrome embeds flash, so chrome basically runs flash in a separate process and exposes like the bare minimum possible api that it needs in order to run, and so it's essentially the same idea with webview. If you really want to embed your remote web app, do it in a webview and limit the api to the bare minimum it needs to run so it won't have access to all of node.

A

It won't have access your users filesystem, and that is much more secure, but it also spins up another process. So if you're, you know it's, it can be a memory concern depending on how resource intensive your app is. The other option, which I think is a bit better, is to not load your remote web app and instead, if you want to share code between your desktop app and your web app, you can extract components and literally use those same components in both apps.

A

So that's a bit easier if you're, using some kind of view, library like react where, like every components encapsulate- and you literally- could pull it out into its own module and just like slap it into electron, because it's all web technology, but that is I, think a bit of a better way to avoid this security nightmare.

A

So another thing that is a good tip is that whenever electron gets native support for something you should use it so, for instance, I think recently chrome added native support for the spread operator that lets you like expand properties of an object or an array into another. One I could be wrong about that, but let's say it did.

A

Then you should use that native built-in version. Instead of having electronic compile, run Babel or something on it to compile it down, you will usually get better performance and you'll also get better reliability because it won't be like shimmed or anything you'll just be using like the actual real method that the browser supports.

A

Another thing, that's really important for performance is that you should make your requires lazy. So, as I mentioned earlier, if you haven't worked with nodejs too much, you can require dependencies or import with the new syntax import dependencies at the top of your files and problem with this is if your app gets big enough, and you have enough modules that are loading enough dependencies it'll take a little take a while it'll add a a, not insignificant amount of time to your apps startup, and if you value startup time, you probably don't want that to happen.

A

So, instead of eagerly importing all the things at the top of a file, you can instead only only require or import exactly when you need a dependency instead of all upfront, and that should save you a lot of startup time.

A

One more tip that may help you out is that maybe should consider being selective in what you compile. So this is really helpful, say if you are refactoring a codebase and you were originally writing in like vanilla, JavaScript es5 and you want to move to typescript, but you were also just using CSS before and now you want to use less or something too.

A

If you, it would be kind of a bummer if you had to like do bo3 factors at the exact same time, because that would be a lot of work and be much harder to review. So, instead, electronic compile lets you selectively disable compilation for certain files. So if you don't want to do less on your first run, then you don't have to you. If you don't want to compile types or you want, don't want to do JavaScript your first time through don't have to so.

A

This makes it a lot easier if you're refactoring, because you don't have to do it all at once. So some summarize again don't embed remote content without a sandbox use, a web view or use local assets, which I think is a little better and when electron gets native support for something use it ignore the little bit under there make your requires lazy. If you value startup time and consider being selective in what you compile, it will help you a lot when refactoring.

A

So this is where electron compile stands now, you've gotten to see it in action, you've gotten a chance to see how it works, but we're also looking at the future.

A

So one thing we really want to add and plan on adding is hot loading and file watching so you saw it would be really awesome if you didn't have to reload the app every time you made a change. You do right now and actually I'll show you really quick. What that looks like.

A

So I showed you have this demo app works, but let's say let's change something, really quick. So we had this text. That said hello, github universe, you know what maybe it's a planet now. Why not?

A

So we have the app running now and you can see that it looks exactly the same as it did before, but if we reload we have github planet. So this is how it works right now, every time after you make a change, you have to reload, which is a bit of a bummer and.

A

One of the things we want to do is make it so that whenever you make a change, it'll automatically load your changes in line without you having to reload, and it will only load exactly what changed. Instead of like all the Java Script are all the CSS and your app there. We go so hot loading and file watching.

A

That is the main thing we're working on next, but even more important is we want to add support for use cases from users like you.

A

So if you are working on an electron app if you're curious about working on an electron app, please try out electron, compile because the more you use it, the better it'll, get and really hope that it helps make your development lives simpler, more pleasant and more productive. So if you're curious about how to use this and want to take a look after the talk check out, this bitly link bit lead slash, electron, compile. This will take you to the repo with the readme. It will walk you through everything.

A

You need to do to set it up and use it if you're even more curious about how electron compile works under the hood check out. The slack engineering blog at slack engineering, which it turns out, is a real URL, oh, but definitely check that out. There's also a lot of great posts from other engineers in different areas of the company. So, if you're curious about how things work behind the scenes at slack check that out- and thank you so much for having me and we're also hiring.