GitHub GitHub Presents • Passion Projects LIVE (Full Length Talks), 5 May 2013

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Passion Projects (Live) 2: Heather Arthur (Machine Learning)

Description

Part 2 of our Passion Projects series, featuring a live talk with Heather Arthur. Heather is an engineer at Mozilla in San Francisco, where she works on Firefox developer tools. In her spare time, Heather creates and maintains her share of awesome open source projects.

As always, feel free to leave us a comment below and don't forget to subscribe: http://bit.ly/subgithub

Thanks!

Connect with us.
Facebook: http://fb.com/github
Twitter: http://twitter.com/github
Google+: http://google.com/+github
LinkedIn: http://linkedin.com/company/github

About GitHub
GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Millions of people use GitHub to build amazing things together. For more info, go to http://github.com

A

Heather arthur].

A

My name is julie,, I work at github and I'm also the creator and organizer of passion, projects.

A

I'm really excited to have all of you guys here and I'm incredibly proud to introduce heather arthur,, who is our speaker for tonight.

A

so,. If we can just give her a nice round of applause and welcome her to the stage,, that would be great.

B

Alright,, so just to give you a little background., I'm heather, and harthvader on twitter, harthur on github., and I am a developer at mozilla..

B

I work on firefox,, so that means I write open source code,, all day,, every day,.

B

Which is really awesome.

B

in particular, what I work on is the firefox developer, tools. so,. These are the tools that are built into the browser that help web developers debug and create their webpages.

B

So,, this is just a screenshot of the stuff that I work on, but I'm not gonna talk about this today..

B

I'm gonna talk about something else that I'm really interested in,, which is machine, learning., more particularly,, bringing machine learning to javascript.

B

so,. Let's talk about machine learning for a second., so I think machine learning, as a phrase sounds really intense..

B

It kinda sounds like a robot apocalypse to be honest.. I know,, I remember telling one of my friends that I was doing machine learning stuff and they were,. They sounded terrified, when I mentioned, and they were, like, "oh my gosh you're, making cylons or something?

B

so,. I think when a lot of people hear machine learning,, they think of robots and math formulas, and it's actually a little bit like that.

B

But not,. It's not all like that.

B

You're not always making robots.

B

And it's also, can be really useful, even just for everyday applications.

B

So,, I like to think of machine learning as a collection of algorithms, and these algorithms are used to solve problems that people have been really good solving,, but programs have been pretty crap at automating in the past.

B

So, examples of this are spam: filtering.

B

so, your email deciding whether or not something's spam. So you don't have to see it or not. face detection, recommendations like netflix and amazon, do.

B

character, recognition,, so digitizing books.

B

So there's some of the applications.

B

and if you sat down to write a program to solve any of these,, let's say, spam filtering.. If you sat down.

B

To try to solve that without machine learning,, what you might do, is, you might look at some emails.. You might be like, "alright,! Well, a lot of these spam emails are in all caps, and I don't see any legit emails in all caps." so,. You might make a rule that's,.

B

If the email is in all upper case,, then it's spam.

B

And then you might look at some more emails and you might be like, "alright, well,. A lot of these spam emails are also mentioning free ipads. and that's kinda sketchy and my other emails. Don't mention that.".

B

So you might make another rule, that says "or. If there are the words free ipad in it".

B

Then,, it's also spam..

B

So what you're gonna end up with is a bunch of painstakingly hand-picked "if", statements,, basically. and even then,.

B

You can't look at all the emails in the universe.

B

you're gonna miss a lot of things just doing this manually.

B

and what machine learning algorithms do. Is they find these "if" statements for you and they do this better than you can and with less error.

B

So essentially, machine learning, algorithms, are, like,, better programmers than you.

B

oh, just for these particular applications.

B

So,, to sum it up,, on the one hand, we have traditional programming,, where you just have,, you choose the conditionals and all the parameters.

B

and then, on the other hand, we have machine learning algorithms,, where the rules are learned from the data. and so,.

B

This is actually really powerful.

B

and there are some applications, in fact,.

B

Where a machine learning solution is the only really.

B

Manageable solution to that problem.

B

So yeah, basically, machine learning super important and super useful..

B

So now, let's talk about javascript for a second.

B

Javascript is a ubiquitous programming, language.

B

This is a screenshot of the top languages on github.. I believe this is the percents of repositories where the main language is javascript.

B

And some language, or some repositories are multiple languages,, but I think it's safe to say: javascript is squarely in first place.

B

And not only that,, but javascript is essentially the only language that runs in the browser and also runs in server with no js. so yeah,. It's a huge language.

B

So it makes sense that we see if we can put the two together,.

B

Machine learning and javascript., so let's do exactly that.. So what we're gonna do, is we're gonna solve a problem, with machine, learning, and javascript..

B

So this particular problem we're gonna, solve is,.

B

You get bombarded with a lot of content today on the internet, and sometimes you're wondering if you should bother,.

B

Checking something out..

B

I have this problem, a lot.

B

so,. You might ask yourself, "should. I bother clicking on this link?.

B

Should I bother checking this thing out?", so the answer to that questions lies in another question, which is, "does it have any cat pictures?".

B

So this problem is called cat detection.

B

and this actually,. This is not an original problem at all.

B

If you look this up,, if you search for this,, there are research papers and phd theses.

B

On this, going back years. so it's,, clearly an interesting problem.

B

So the exact problem we're gonna solve is given an image,. Does it contain a cat?.

B

We're gonna break down a little bit, more, something more manageable.. So, just given a section of image, we're gonna see, is that section a cat head or not?.

B

So what we would need to do this, we need something that will take an image and will be,. Yes, this is a cat, or no, it's not a cat.

B

So what we need is a classifier., so a classifier does exactly that.. It takes a piece of data and it tells you which class it's in.

B

For our case,, we have two classes: cat and not cat., and classifying algorithms are usually trained on data where you actually already know the class to it.

B

And some examples of classifiers, there's bayesian classifiers, for spam.

B

k-nearest, neighbors,, neural networks, support, vector machines.

B

These are all different classification algorithms used for different things., we're gonna use the support, vector machines.

B

So, to talk about the support, vector machine, algorithm.

B

so,, it's also another phrase that sounds pretty intense but we'll get to at least figure out the vector part in a second..

B

So these support machines,, just like I mentioned before,- are trained on data where we already know the answer..

B

So it's trained on labeled data.

B

and what I mean by data is, basically,, a bunch of vectors of numbers.

B

Arrays of numbers. and then, what I mean by labels is,, we're gonna, give it a one, or a negative one.

B

So in our case, we want to label a vector.

B

With a one,, if it's a cat, and negative one,, if it's not a cat.

B

And also, in our case,, what we have to do. We have our labels,. We know it's one, it's a cat, if it's negative one,, it's not.. But what is our data? Gonna look like?.

B

So we know it has to be an array of numbers., so we have to figure out how to take an image and get an array of numbers that represents it.

B

So,, what's our input gonna be?.

B

So one thing we can do, is just straight up: take the pixels from the image., so we could just take the [rgb] values.

B

And flatten it out into array, or,, we could just take the gray scale intensities of each pixel.

B

in both those cases. We're gonna end up with like a ton of information,, not all of it's relevant,.

B

There's gonna be a lot of noise., so we wanna help out the classifier a bit..

B

We wanna give it something it can chew on a bit more easily.

B

So here's another idea,, which is we could just look at the edges in the image.

B

So this is the edge of an image. edges in images are places where the image changes from dark to light, or vice versa., and here we've taken.

B

The gradient of each of the images above.

B

So each of these images on the bottom are the gradient of the image above it., so the gradient is basically, getting the edges from the image.

B

and you see, after we've done, this, we've taken out a lot of information, like the colors and whether it's dark or light and we've retained, basically,.

B

The only thing we've retained is the shape of the cat head. so,. That's really awesome., that's exactly what we want and we don't have a lot of extra information that we don't need.. So it's great.

B

So we're gonna do something similar, but a little bit more sophisticated., so we're gonna do, is get the histogram of oriented gradients of the image.

B

Another intense sounding word., so it's also,, I call it a hog descriptor.

B

I've never heard it said out, loud, I've only read it on the internet and in papers,.

B

So I hope that's how you really say: it.

B

Machine learning person,, but I call it hog descriptor, and this will capture the edges that we were looking for before, but also capture the direction of the edges. so,, we'll capture the angle of the cat, ear.

B

So, and it does it, even better.

B

and a hog descriptor is an array of numbers., so this is perfect,. This is exactly what we need to feed into our classifier., so just giving.

B

You an idea. the hog descriptor is gonna, be an array of about a thousand numbers.

B

Alright,, so now we have this,. We know what data we're gonna give the classifier..

B

So now we actually have to collect it..

B

So this is an important step,. Getting a lot of data to train our classifier with.

B

So what we need to do is we need to get a bunch of cat pictures and a bunch of non-cat pictures.

B

And we want a lot of both of them,, so the classifier knows about both classes.

B

So for the negatives,, those are the images without cats. for those, we're just gonna download a bunch of images from flicker and crop out random sections of them.

B

And then for the positives,, we need thousands of cat pictures and we also just need the cat head too,. We need to crop around the cat head., so they're actually, there happens to be this amazing data set of cat pictures.

B

Each one is annotated with the location of the ears and the eyes and the nose.

B

So this is very, convenient.. I can't believe.

B

This exists but it does. [laughing].

B

So that's very nice for us.. So all we have to do now is take each one of these cat. Pictures, rotate it so that the eyes are level and then we crop the picture so that we're.

B

Just framing the cat's head, like that.

B

And then also we want each of these pictures to be the same size., so we can feed it to our classifier.

B

ok,. So that's awesome.! So now we have you know, thousands of cats and non-cats.

B

And now, we can start training with that stuff.

B

So, training.: this is where javascript comes: in.

B

so,, I'm just gonna, go through this a bit.

B

So, first of all, there are a couple of packages that we're using from the npm package manager.

B

There's a hog descriptor package, to do the hog descriptor. and the svm package,, which will give us our support, vector, machine., so you're, creating a support, vector machine.

B

And then we go through each of these pictures that we have,.

B

And then we first extract the hog descriptor from it.

B

And then we give it a label. one, if it's a cat, and negative one,, if it's not.

B

and then finally, once we've gotten all these inputs and their respective labels,, we can call svm.train with this and put some labels.

B

Alright,, so that's it., so we've trained our support, vector machine and just to give you a better idea of what's happening when you do that, what training, does,.

B

Is it creates this state.?

B

So this is a json object,.

B

A json object that is the trained state of our support, vector machine..

B

I hope you can see. yeah.

B

So,, you know how I said earlier that machine learning algorithms find "if" statements for you?, so they actually, that's actually not what's happening.

B

In reality,, this algorithm is finding floating point values for you.

B

so,. The "if" statements are fixed and the floating point values change based on which data you trained it with.

B

so, the values of these numbers here.

B

Will determine, will be combined.

B

With your new data that you haven't seen before and will help you, will help the support, vector machine determine what class it's in.

B

So ok, so you've got it, our support, vector machine trained..

B

So now what we have to do is go back and start solving our problem,.

B

Which is, the first problem we're trying to solve is just given a section of image,.

B

Is it a cat head?? So what we're gonna do.

B

All we have to do here, is take this image.

B

That we're gonna try to classify,.

B

We extract the hog descriptor from it,, just like we did with the training image.

B

Then, we just call svm.predict with that descriptor.

B

And that result will be one or a negative one and that's just,. That's not a label,. That's actually just a prediction of what the support vector machine thinks the image is. so then,. If it's one,, we'll say,.

B

"True,, it is a cat." and if it's not, false,, it's not a cat.

B

Alright, so great, we've that will take a section and tell us if it's a cat head or not.

B

so now, we can go one level up.

B

And finally, answer our original question,, which is, given an image,, does it contain a cat?.

B

So what we do, is, we take this function that will classify individual sections and we run it on all the sections in the image.

B

So we're testing windows at all different locations and scales throughout the image.

B

And then, after that,, around cat heads we'll have a bunch of overlapping detections,, so we're gonna combine all those and also weed out, in the process of doing that,, we're gonna weed out any detections that, that dont have.

B

Enough detections. they're, probably false positives. so,, not really cats.,.

B

So yeah, and then that's it.

B

that will answer our original question.

B

So,, let's go to demo.

B

Ok,, so we have kittydar.

B

I actually did this about a year, ago,.

B

But this will run this algorithm that we just made.

B

So we're gonna, try it out. we're gonna, give it a cat, picture. ok,, and you see what it did,.

B

Was it did that thing where it detected at different windows, and then it combined those overlaps.

B

So bam! it got,, there's a cat. [applause].

B

We're gonna try another one. yay!.

B

Ok, we're gonna, try another one. you'll, see the limitations of it.

B

so,. You can see here that it got two cats,, but there are 4 cats. There.

B

It didn't get the two on the sides., it didn't get the one on the left.

B

yeah,. It's left for you. ok, didn't get on the left, because we didn't train the classifer to detect cats that are facing the other way..

B

We just trained it on cats that are facing the camera. Directly.

B

And then, the one on the right has an abnormal shape. [laughter].

B

So,, it's not, and it's also a little tilted, which will make it harder as well.

B

But still, it works pretty well.. So that's pretty cool.

B

And I have to mention one other thing that I did.

B

this is running in webpage.

B

But we have to run it on web worker,, so in a separate process., this stuff, we're doing is serious number crunching..

B

It's basically the equivalent of doing a while-true loop for a few seconds on your page,. So you really want to,.

B

You really wanna make sure you're not running that on the main thread, because it will block everything and it will be really bad.

B

So it's one more thing that you did,, that I did as well.

B

alright,. So what I think is really awesome about. This, is this is running in the browser.

B

I just think that's the coolest thing ever., it's all running on the client's side and actually, well,. We see the url harthur.github.com,.

B

So you know it's static, and it's really cool.

B

So finally, alright, so,.

B

We just saw that running in javascript., so a few years ago, or several years, ago,.

B

This just would not have been possible in javascript, and there are a few things.

B

That have made it possible.

B

so,. I guess the big one is just speed.

B

Javascript has gotten a lot faster. The past few years.

B

And this kind of number crunching just ran way too slowly in the past.

B

So,, that's one thing.. Another big thing is node.js,, which came out a few years. Ago.

B

You wouldn't have been able to do that collection and training stuff. We just did without node.js.

B

We really need a command line and access to the file system to do those things., but I also,.

B

I have to note that you don't have to do each of these steps in the same language. so,. You could do your collection or training in another language and then, export your train, state,, so json or something and then use it from javascript, too.

B

so,, that's another option. and actually, for the collection,. I think I used a [indistinct] script to get flicker images, cause the api.

B

Was easier to use there. so yeah,, you could still mix it up, but node.js makes it possible to do these steps in javascript. If you want to.

B

another thing: that's helped out. A bit is typed arrays.. So when javascript-.

B

Usually,, when you have an array,, you can put anything that you want into it.. You can mix types.. You can have indices where you don't put anything in it, and then some have strings, and some you put numbers in, and some random objects. so,. It's very lax.

B

And with typed arrays, you're telling javascript, this array will only contain,.

B

Floating point 64 bit numbers,.

B

And so what java can do is,? It can figure out the offset into memory a lot faster.

B

Using that information, because it knows the type of everything in the array.

B

so,, it makes it a lot faster. in fact,. It makes it about two times faster for training., so, instead of it taking.

B

Ten hours to train our support, vector, machine,.

B

It takes five hours,, which is a big deal.

B

If you are trying to iterate and figure stuff out.

B

So,, it's one thing.. Another thing is that workers really make this possible to run this stuff in the browser.

B

otherwise, it would,. There is no way you'd want to run it in the browser.. It would just block everything.. So there's some big things that helped out a lot.

B

And as you can see,, it's clearly possible.

B

To do it now., but there's still stuff that javascript is lacking.

B

Javascript is definitely not your typical language. You use to do machine learning.

B

Things like python, languages, like python, have a ton of really good building block libraries like mat, lab.

B

And stuff like that,, which javascript is lacking.

B

Matrix math, library, statistics, image processing, that kind of stuff.

B

Is just not quite there yet in javascript.

B

So that's one thing that still really needs to happen., but also in a, also it'd be nice if it were even faster,. So it's cool.

B

But yeah,, let's see here.

B

But things are still happening: though.

B

There's some libraries out there., there's neural networks,, there's support vector, machines, clustering, and the natural language processing library,, which I really like, called natural node,.

B

That also has a bayesing classifier, as well as, doing natural language processing,, which is, kinda similar to machine learning, in some cases.

B

So I wrote the neural network and the clustering a few years ago.

B

also andre [carpathy] has been contributing a lot of libraries, too.

B

Including the support vector machine implementation,, which we just used,, so that's cool.

B

and there's some applications in javascript, too, using machine learning.. My favorite one is this: real-time face detection..

B

So this is face, detection that runs really quickly.

B

So,, that means that you can do it on every frame of a video, like an html5 video.

B

So that is super cool. there's also a hand and eye detection library and the first machine learning I heard of in javascript. Was this ocr,.

B

Captcha solving algorithm,, which is pretty cool.

B

And that uses neural networks to craft captchas.

B

So it'll be really interesting to see.

B

Where machine learning will go in javascript., it's hard to tell right now,, but I hope that people will try it out, [sure].

B

And finally, I want to talk about if you guys are interested in machine learning., so, machine learning doesn't always look like this.

B

There's not always this classification, you're not always doing classifications.

B

Sometimes, you're predicting continuous value,.

B

Like temperature, stock price.

B

Sometimes, you're, not doing,, you don't have to do.

B

This training stuff at all., sometimes, you're, not using labeled data.

B

So, there's a huge variety of things.

B

That machine learning does.

B

so just be aware of that, too.. This is just kinda like a small example.

B

And also, there is a stanford,, a free, online,, stanford machine learning, class.

B

and that's actually starting on aprill 22nd,. So that's in a few weeks. and that's hosted on coursera.. I actually took it a year ago.

B

And it was really good and it really helped fill in a lot of holes for me.

B

so,, definitely check it. Out.

B

And finally,, what really, I think I learn.

B

The most from is just starting with a problem and going from there.

B

so,. When I was working on kittydar,, I just kind of, well, first thing I just searched for "cat detection" and then from there, found research papers,. I read them, when I didn't.

B

Understand something,, I looked it up on wikipedia and then they linked to other papers, and read those, and I actually ended up learning a ton of stuff just from doing that.

B

So much stuff,, I never thought I'd, know anything. About.

B

So there is, yeah, there's a lot of value in that.. I really encourage you to,. If you're facing a problem,, you think you can use machine learning, on, just go for it and you'll end up learning a lot..

B

I think that our brains are kind of like.

B

These machine learning algorithms,, where you can just kind of throw fancy words and research papers at it and eventually,. It will just figure things: out. [giggle,, laughter], that's what I found, at least., so,, yeah, anyways.,.

B

So try it out., it's my best. Suggestion.

B

On deviantart] ok,, thanks. [applause].

A

Heather arthur] hi guys, welcome back.. I hope you're enjoying all of the yummy snacks that my wonderful co-workers provided for us., pretty awesome.

A

That was not a plug, I swear., so heather just gave an amazing talk and it was probably a lot different from.

A

Rachel's talk last week,, which was more sort of like high-level and organizational,, whereas heather's was super technical.

A

And so I'm just kinda gonna jump in and ask you the question..

A

How did you get into programming?.

B

In high school.

B

There was this three week, seminar period class in high school, where we could take one of five classes.

B

Or something, and my friend alyssa and I decided to take this game programming class.

B

and it was just kind of random.. We were just like, "oh games, that sounds kinda cool.".

B

And so we signed up for it and just the first day of class,, you know,.

B

I saw what the code looked like and I saw what it involved. Finally, for the first time,, I had no idea what it entailed: --, the movie hackers, right?.

B

Your screen. no,, but I just saw the code and I was just like, "yeah,. This is definitely what I wanna do..

B

This is awesome." and then ever since then, I just, you know..

B

I basically started in school, and then I kept on taking classes, and then majored in it in college, and then, and so on. From there.

A

You have a really awesome story in that it's, also,. It's completely different from rachel's, who learned to program kind of after college.. You went to school for a cs degree at carnegie mellon and you continued straight out of college.

C

But I think it's really interesting.

A

Kind of the way that you found mozilla.

A

did you kinda wanna talk about that?.

B

And I've worked there ever since college.

B

and actually I found it,. I was at my career, fair.

B

At my school and I'd known that mozilla,.

B

I mean, I'd used firefox two years,. I knew about mozilla, but I didn't know that they actually hired people..

B

So when I saw their booth [laughing] at my college's career fair,, I was like, "oh, my gosh." they,, you know,. I went and talked to them.

B

And I, I probably, I dunno,.

B

I think I sounded like a total idiot., but I guess they liked me, alright.

B

And then I became an intern there, and so interned there, one summer, and then,. When I graduated,, I started working there because I loved it so much.

B

I've been there ever since. at an open source, company?.

A

I don't have anything.

B

To compare it, to. [laughing], it's really awesome,. I don't think I could go.

B

Back to proprietary, unless I had to.

B

It's really cool to be able to talk about what you're working on and not have to think, "oh,. Maybe I can't talk about that, or something.".

B

And I can release, yeah, everybody can see my code.

B

The best part is getting contributors, too.

B

You'll be working really hard with your team and there's these bugs that are falling by the way side, nobody's fixing them and a contributor will come along and fix.

B

This really critical paper cut bug and,, it's just like the most awesome thing.

A

So github obviously, has a lot of open source projects and we asked,. I got the privilege of talking to heather before her talks,, so I cheated a little bit and I got to hear her story a little bit and how she got into programming,, obviously, and also open source. But we talked a little bit about sort of the role github plays in open source and.

C

What it was like to sort of transition.

A

Into using a tool like github.,.

B

I actually, well,. I have been doing open source a few years. Now. I'd been doing it before github was around.

B

And I was hosting my code on google code and stuff like that.. I definitely wasn't getting any contributions.. I don't know if people saw it even., but definitely after github, when I started putting my stuff on github,.

B

I got so many pull requests and issues, and people saw my code.. It's just great.

A

Basically., I think, pull requests make the best christmas presents.

B

Has been awesome.

B

and also git, too., it's helped. absolutely.

B

Very essential.

A

I kinda planned to make this more into a discussion,, so if anyone has any questions, throw your hands, up. don't be shy.

A

There's a question. hi steven.

D

And statistics as part of your college track?.

B

Or is that something that you started learning afterwards in order.

E

To use that to solve problems?.

B

Major and I was a math minor.

B

But the math I was doing was this crazy stuff, ring theory and stuff, so it wasn't really very practical.

B

For instance,, we had the option to take.

B

A matrix, algebra, class, or linear algebra.

B

linear was really theoretical.

B

And I really could've used this matrix stuff, for the stuff I was doing here on the talk., but I didn't take that class.. I took the really theoretical linear algebra class that was really abstract.

B

so,. I really kinda skipped over that practical math and statistics stuff actually.

B

so,. I really learned that through this.

B

And another thing that I learned,- I don't know if you guys found this yourselves, but when I was in school, taking trigonometry classes, trigonometry was the subject where I was like, "ok,. This is like,, I'm never gonna use this.". You know, the sine angles.

B

And stuff., but for doing the cat detection,.

B

I had to figure out how to rotate the image the right way, and I was like, "wow,, I'm actually using this stuff." and I had to really re-learn that.

B

yeah, basically, a lot of it. I just learned from figuring out problems. yeah.

B

Even though I did take classes on related stuff.

A

And I'm always really curious to hear how you chose a language, or how you chose what you would work on: next, 'cause, you, obviously,, you code a lot in javascript and you also write.

B

Javascript programming for so long.

B

And I really, I mean,, I have to write javascript for my job.

B

firefox is written and the parts that I work on are written in javascript.

B

and sometimes actually platform stuff,, which is c++.. So right now,, it's really about what I have to use.. I can't just call out.

B

To ruby from firefox or something, or people would be very mad at me., but so,. That's part of it., but also,.

B

I do have a lot of side projects and they're mainly javascripts..

B

I really like doing them javascripts so that they're in the browser 'cause. I love it. When things are in the browser. but also, I really,, I love node.js too a lot..

B

It's been, made it really easy to make.

B

Command line scripts and one-off things, too.

A

Open source project, obviously, right?.

B

Kittydar was just a demo to explain machine learning., but I think my first,, I think the first.

B

Open source I did was working as an intern on mozilla stuff.

B

But I guess, the first one I did myself was a firefox add on that helped web developers pick colors on their webpage and save colors and tag them and stuff like that.

A

With machine learning,, basically?.

B

My first experience with machine learning, too.. I have this problem where I would be displaying these colors that [he] collected from this webpage,.

B

I also displayed the color value on top of this color., so if it was a light, color, I'd wanna display black text over it and if it was a dark color,, I don't wanna display light, white text over it.

B

and there are colors, like bright green, where it's not really immediately. Clear, we're like, "which color. Do you wanna, black or white, to display over it?"?

B

So I looked up some formulas for doing that and.

B

I found them to be pretty spotty.

B

so,. I just kind of, in the back of my head.

B

I had heard that the words "neural networks".

B

Before and I thought it sounded- super cool. [laughing] 'cause, they do sound, really cool.

B

and I figured out actually,. I might be able to use them for this., and so I wrote neural networks in javascript and it was a firefox add-on, so it had to be javascript.

B

and it actually work pretty well at determining which color to use, black or white over random color.

B

So that was my first foray.

A

Projects outside of what you work on at mozilla, or does it matter-- yeah, yeah, occasionally.

B

Whenever I have time. at this point,, I'm really just maintaining open source projects,, which is a lot of work in itself., I'm trying not to create too many new ones 'cause. Then I have to maintain them.

B

Which is also open, source.

A

Yeah, the excited guy.

E

And all those amazing libraries in python, and last week.

E

We had mozilla launching s and js,.

E

Which would enable python [inaudible].

E

Being used on the browser.

E

Do you think that,? What do you think will be [inaudible] this technology.

B

That I don't know yet.

B

Yeah,, I really I have no idea what effect that stuff will have and how many people will end up using s and js and.

B

Other languages compiling to javascript.

B

I can't-- [julie] predict the future. right now.

A

I can't predict that.

B

I'm not sure., it's not something. I would do right now., I wouldn't be like, "alright,. I'm gonna use this python and convert it to lvm.

B

And have it run unscripted, or whatever.

B

Yeah,, I don't know. [julie] the answer.. We don't know.

B

Interesting to see- and I hope people try it out for sure.- maybe I'll- try it out at some point.

A

A

Now fight. [more laughing].

C

Citizen science, projects like galaxy, zoo or planet hunters are using machine learning behind the scenes, because they're collecting so much data that could be used for people basically.

C

Tagging different photos and things like that on facebook?, [heather], ok,, so...

B

She's asking you about specific projects.

A

And whether they're using machine learning.

B

Like galaxy, zoo and space--,.

C

Planet, hunters and galaxy zoo.

C

And there's a few others out: there, like protein, folding, ones, and all kinds.

C

Like that, where they're, basically collecting a whole bunch of data from people who are entering information,.

C

About images, they're given. and you think, on the back end they've been collecting that information and developing machine learning to do it. [Inaudible]?

C

of at least one app.

A

That's actually a game and it's used to detect protein bindings, or something.

A

And so they're using, yeah,, so there's a lot of really cool things.. I mean that I think is a native ios app or something like that,. So it's probably running on what,, c, or something.. I dunno., but I don't know of any,.

A

Do you know of any others?.

B

But yeah, most likely they are.. I think, anytime, that you have a lot of data and you're using it to predict stuff,. It's probably using machine learning.

A

We had some really simple examples of machine learning and applications that we all use or sort of,. You wanna talk about those a little bit?.

B

Like spam filtering,, everybody benefits from that.

B

And also,, I think a lot of recommendation.

B

To recommend things for you, based on what people like you also use.,.

A

That horse mask, they're, obviously using some type of machine learning.

B

There's computer [ridge] that uses a lot of machine learning. so, face detection, self-driving, cars, they're, probably using it, too.

A

A question? she's hiding., I can call people out by name.

D

For people who are getting into programming?.

A

Do you have to people who are getting into programming now?.

B

I've been programming for so long that I wish I could empathize more.

B

It's, I think, for me,. I came from.

B

An academic background, and so I kind of.

B

So you could always do that,, which is just taking online classes, free online classes,, like the machine learning one.

B

I was talking about., even that one,, you don't have to know programming to take it..

B

The professor, andrew young, is really good.

B

At explaining things- and you don't need.

B

To know programming, but you might end up learning some in the meantime.

B

and also I know I've heard of code academy that does.

B

This online javascript learning.

B

and I've heard nothing but amazing things about that..

B

People say how easy it is and how easy is to understand things immediately from that..

B

So that's another thing: I've heard about., so there is, yeah that's kind of coming from the schooling kind of background., but also, I think, if you can think of any problem that you just wanna solve, kinda, what I was talking about with machine learning,, it really,.

B

Once you try to figure something out,, you end up learning so much. and then, before you know, it, you'll know a lot of programming.

B

just by trying to solve something.

D

The approach that you take when approaching a machine learning problem versus traditional programming problems?.

D

How do you make those logical leaps to find the algorithm you need?.

B

How do you approach machine learning problems differently from regular problems, maybe?.

B

So, I guess, machine learning,, well,.

B

I certainly end up doing a lot of research with the regular program, I'm kind of,.

B

I know what steps I'm gonna take, more or less., I'm like, "ok, well,, I'm gonna make this object and then it'll probably do this thing and it'll communicate this way.".

A

And best practices around regular.

A

Programming problems,, whereas machine learning is still very open,.

A

Depending on the problem and,, I mean, there can be an entirely different path. [Inaudible].

B

With machine learning,, it's more, you have to do a lot of research.

B

you're, "how, the crap. Do I do this?" and then you end up just searching for it.

B

and also another big thing,. Is he needs to figure out where to get your data from 'cause there's, always data.? That's a big difference, too.

E

You choose whether to use webworkers.

D

Or if you use a more traditional rpc asynchronous framework and what are the trade-offs that you're finding?.

B

Decide between web workers and rpc asynchronous.

D

At least naively, that they have a lot of the same benefits. that you can do: data computational tasks or computational heavy tasks without worrying about it. Using up your [data space.].

B

Haven't heard of the rpc asynchronous stuff.

B

I'm not exactly sure what you're talking about actually.

D

An rpc call to a server,, maybe I'm just using the wrong terminology.

D

Handle the processing and you send it back.

B

Doing it in web workers and the browser versus having it done on the server.

B

and calling out to, ok, cool.

B

So there's a big difference, there.

B

so one thing, I actually,. I like.

B

Doing my webpages all on the client side, if I can and having as little server side as possible., not that.

B

I'm saying that's the best thing to do or anything,, but I usually do that.

B

So I prefer using web workers to do something like this. and also it's just one less network request,, which is cool. and a lot less dependency on that.. So that's the main difference,.

B

I guess. it's just whether you want to do.

B

The computation on the client or the server and do an extra network request, probably.

A

Anna?, will you go first?.

D

What are you really passionate about?.

C

What makes you get up in the morning and be excited to go? Do your job?.

B

A

And what makes you get up in the morning??

A

What are you excited to work on most?.

B

Mainly, about I love my job,.

B

I'm very invested in making our firefox developer tools better..

B

I think it's really important that we help web developers out.

B

So, that drives me a lot., I'm like, "we need to get this feature done, so web developers can use it.".

B

So in that sense, a lot of my motivation is about just getting stuff done., getting particular things out there.

B

So people can use them., that's the main motivation. also, some,.

B

I mean, just programming in general is really fun, so that's a secondary motivation., it's just, "oh. I get to do fun, stuff now.". So that's cool..

A

Or one of the reasons we started passion projects originally, was we wanted to hear from women who really loved the companies they worked at, and I remember during our interview, I asked, you know,. What is your favorite thing about working at mozilla?

A

and I think you said it was the people.. Do you wanna talk about that?.

B

Again, I haven't worked at other companies, but at least at mozilla, everybody, there really cares. it's an open source company..

B

It's also wholly owned by a non-profit.

B

And we have a really clear mission and I think that a lot of people that work at mozilla are really passionate about that. so,. It makes for really awesome co-workers and they're also,.

B

You know, they're doing open source, but they're really used to helping out a lot of people and so they've been extremely helpful and they're. Also just really nice.

B

They're not, they're not.

B

Assholes basically,, which is awesome.

B

Everybody that works at mozilla.

B

I've never met somebody that works at mozilla that I didn't like a lot.

A

About when you're getting into programming. surround yourself with people who wanna help other people.

C

I know, I was really lucky.. I started.

A

Writing code, when I was at yammer- and I had just this incredible network of people around me- who really wanted to help me- learn really quickly. and I got just pointed in the right direction and I mean, if you're self-motivated and you have those types of people around you.. It's the perfect formula for success, business., other things.

D

[Heather] yeah. [julie], so people and such,.

A

And open source companies, I think, attract those kinds of people. people who wanna build things that make other people's workloads more efficient and make them better at what they do.

B

Open source companies are the best.

A

For letting us rack your brain and also for the amazing talk on machine learning., we record all our talks,, so we usually post them in the follow up blog post., so anyone who's watching from home and maybe missed the beginning or whichever, we'll catch you up and thank you so much, heather, for being a part of passion, projects.

B

Heather arthur].