Node.js Node.js Interactive Europe 2016, 24 Sep 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Text Mining with Node.js - Philipp Burckhardt, Carnegie Mellon University

Description

Text Mining with Node.js - Philipp Burckhardt, Carnegie Mellon University

Today, more data is accumulated than ever before. It has been estimated that over 80% of data collected by businesses is unstructured, mostly in the form of free text. The statistical community has developed many tools for analyzing textual data, both in the areas of exploratory data analysis (e.g. clustering methods) and predictive analytics. In this talk, Philipp Burckhardt will discuss tools and libraries that you can use today to perform text mining with Node.js. Creative strategies to overcome the limitations of the V8 engine in the areas of high-performance and memory-intensive computing will be discussed. You will be introduced to how you can use Node.js streams to analyze text in real-time, how to leverage native add-ons for performance-intensive code and how to build command-line interfaces to process text directly from the terminal.

A

Okay, hello. Everyone thanks for coming to my talk on text mining with no jazz.

A

My name is philip burkett, I'm a fourth year phd student at carnegie mellon university, um mostly working in computational statistics with applications in healthcare and education.

A

But what brought me here more is my alias plane, shifter who's, quite busy working on npm modules, for statistics, text, mining and other things.

A

Text, mining and node.js is not something you usually hear in the same sentence. I would say in fact one might raise the question: if not statically typed compiled language such as c plus or c, would be more appropriate because of the obvious performance benefits and in case of really large data sets where you have like hundreds of thousands of documents.

A

That's certainly a valid argument, but let me tell you from my domain, like in statistics, for example, there are many other considerations that people have besides speed. For example, the majority of statisticians use this programming, language called r to do the analysis, and this language is actually horribly slow, but r has the advantage that all the statistical methods people have implemented, basically in r. So and if a statistician wants to do tax money, she would probably ask himself okay why we invent the wheel.

A

I can just do it in r so, and I think, similarly, that there are some valid reasons why we might want to do uh text mining applications inside of node, and I can think of at least four that I would like to outline the first being that javascript is language of the web, so with webassembly now other languages might target the browser. But if you want want to run anything inside the browser, then certainly javascript is the way to go there.

A

um Also, because uh we can basically target any platform like with browser we find webpack. We can bundle our code for the browser. We can easily write and note command line applications or desktop applications nowadays with electron. So if we have some text mining that we want to do inside of our applications being able to do it right at once and then have it on, our platforms is certainly like a big benefit in my opinion.

A

Thirdly, the v8 engine of google, which powers node, is actually fast enough to handle like reasonably sized text mining applications and that's in stark contrast actually to python or r. So if you look at the popular text, mining packages in python r, they all basically call down to c or c plus, plus because the languages themselves are just not fast enough.

A

So if you want to avoid kind of that language switch or that context switch being able to just run everything in in one language, it's actually a pretty nice thing and you can do it with javascript inside note and fourth, I think the note streams package it's really great, to build like data pipelines. If you want to do uh handle text data streams and there we also don't run into the memory issues that you might get into.

A

If you do batch processing of large large documents collections, um I think one big drawback is that just the ecosystem is not really there. They are quite if there are a few packages like that, are popular and like, for example, natural by chris umble and others, but there's just not a big uh ecosystem of text, mining modules and so many things you would just have to code up on your own.

A

um Just to give you an example where I personally encountered that choice. We were confronted in as part of a research project with the task of de-identifying patient data, and this is required by u.s law. So I thought there must be like many open source software solutions which actually do that, but I just found some abandoned projects written in in java, python and other languages, but um they were not really supported anymore.

A

So the best way forward just seemed to build this tool on our own, and we did it with uh nwjs so kind of the predecessor of electron and so being able to basically build a tool that now anybody, even without like any technical knowledge, can use if he or she has to de-identify uh such confidential information and to make this a bit more concrete.

A

Here's an example like a little screenshot for like how you can process a single file in this app just drag the file on there and then the text mining aspects, basically that we have a model which detects our personal identifiers, for example, here john doe, this person on the left and we replace it by like an automatically generated uh fancy identifier, so here john doe becomes layla arise now that might be pretty neat or not.

A

um But when you delve into text mining, I think you will often face this this challenge, namely that text data is incredibly messy to begin with, especially if you compare it like to more structured data that usually stun a database- and I think there's some philosophical reasons for that. uh One being that words have different, meaning, which one calls policyme also words, often like their meaning is depends on context, is ambiguous, but even on a very basic level.

A

One confronts like just some tricky and cumbersome tasks like splitting uh with um strings into sentences where you you just, cannot just split by dots right because, like people nowadays use emojis and stuff like that um or tokenizing a sequence, a string into a different words where you need to account for punctuation, and so you again cannot just simply split on white space. So um the first thing you always got to do.

A

If you work on on any text, mining project, I would say, is clean, clean cleaning up your data in a sense and get it into a shape that you need for your type of problem, and I don't think there's that there's any general way of how to do this, but really depends on on your given problem that you want to solve.

A

But why do I think this is rewarding? um It is estimated that unstructured data, and that's mostly in the form of free text, accounts to roughly of all of 80 of all data that is collected.

A

So, uh given the example of the medical sector, so people who have been doing like analytics or data mining usually just looked at this tip of the iceberg and of the numeric values like lab values of patients, etc.

A

But now more and more people are aware that the wheel treasures might lie, be underneath the water surface, in that sense, where you have like doctors, nodes discharge, summaries, etc, and I'm pretty sure that many other industries, the situation, is kind of similar.

A

So now, instead of giving like a full overview of text mining techniques, uh it would kind of be infeasible. Also given the length of this talk, I would just present like two basic applications, just the two basic techniques, and we will look at how we can use those uh uh implement them in javascript.

A

So um first one is sentiment analysis, so pretty basic one. Basically, we want to analyze whether a text expresses so to speak, a like or dislike.

A

We could ask analyze film reviews and find out whether the new movie by woody ellen is looked favorably upon by film critics or not so much. um Similarly, we could analyze twitter feeds. Looking at whether uh the twitter population looks favorably upon like donald trump's, latest outbursts or not um so the other application that I want to highlight is kind of cluster analysis and topic modeling. So this is a problem which requires often kind of more sophisticated models and it's.

A

But let me first try to explain so imagine you have a collection of documents, for example like newspaper articles and in the newspaper you have various sections right, so you have like the sports section. You have like celebrity news like politics, etc, and basically um now imagine that instead of having a human who categorizes these texts into different categories, we want to have a computer algorithm doing that for us, and so the computer obviously doesn't understand anything about the difference between like a soccer game or like a paris hilton shopping tour. But what so?

A

How can we teach the computer in a way to to group these data into different categories in a plausible way, and what many of the statistical methods for this task do is kind of pretty simplistic?

A

They treat documents as a kind of back of words. So basically, what one does is one just tweets: a document as a collection of words get rid of like the whole order in that sense and then what what kind of a topic then is just like a co-occurrence of words, for example like in a sport like a sports topic, could be you we would have even for different sports, like tennis or soccer.

A

Some words will appear with high frequency like score, and someone has a like win points, losers, game, etc, and so these words kind of will link these documents together and now. um Basically, it's just like the algorithms work by exploiting that co-occurrence of words and trying to find like like an allocation of the documents which makes uh kind of this most probable now, but without uh enough of these preambles. So let's get a bit more practical. uh So the first example I want to give is of sentiment.

A

Analysis of state of the union addresses by president obama. So the state of the union address is the an annual speech by u.s presidents, which they reflect upon the state of the nation and in which they talk about their priorities and future plans.

A

So um here's a bit of code first, what we do here is we load. uh We load these speeches, so this this module stand lip. This is uh mother that I'm involved in uh and that's still very much uh work in progress. If you're interested you can find it on on my github page, uh you would need to check out the dev branch, though, um so what we then do here is we also get a word list.

A

This f in word list here, which is basically an array of uh an array of arrays, and each array element will be a word where we have like a valence score from minus five to five, depending on whether this is a positive or negative word, and then we also get this tokenizing function to just split split the text into into white space into into its individual tokens.

A

Now the first thing we then do is: let's here, create a hash table um just turning these words, so that we can have a fast lookup being able to just for any given word find the associated score here and that's one thing. I really learned the hard way when you do any of these type of tasks.

A

You really want to think about the types of data structures that you use and uh and the types of algorithms because imagine like, even if you have only a thousand documents and each a thousand words long, it's still like, then you have already a million uh things to to iterate to if you do that, for example, so so you really have to think about. Okay, you want to avoid searches like how do you efficiently implement things um and so okay, what do we do? Then?

A

Here we get these speechless of obama, and then we do like this pre-processing that I talked about. So that's like often the case so here these speeches are just basically uh an object. Array uh has some other attributes. We are only interested in the text right now. um So what do we do here? We just uh get the texts.

A

We then turn them out to the lowercase and we tokenize them- and I kind of didn't, follow my own advice here because, like in general, I would advise against using the array math prototype methods, because this, like again, if you have like really large arrays uh the function, context that it's created like this will eat up a lot of memory, and so you can okay, you will run into problems quite quite soon with like, as your data sets go, so I mean in this case we have only eight speeches, so uh it's not really a problem, but in general, like for loops, like the basic concept, would be much more feasible for that type of task.

A

So then we just simply here allocate an array. We want to calculate the sentiment for each document and then we just calculate the cumulative score for each document and that's so that's a really simple method, right, a really basic method. What do we get out of it? We basically get this array of of sentiments course, which we can look up here um from 2009 to 2016., and uh I think some things one can really uh rationalize here right. We see first 2009 2010.

A

We have still like really low, like a comparatively low sentiment score, um which kind of makes sense, given that uh it was kind of in the aftermath of the financial crisis, and so the recession still was hit hard, um and here we might have like this might also be interesting. Looking at the at the different words that are associated with a um like that that occur in in these speeches, like the positive and negative words, so we see the narrative ones mostly recession like the wars, the us is involved in and a good side.

A

We have like things like good etcetera, um so here one has to be aware that we cannot really with our simple method right. We cannot handle things like irony, negations and uh ambivalence so like, if like in in shakespeare's. uh You do caesar like when mark anthony, gives a speech about about. Brutus uh might get a high sentiment score because he refers to him like as an honorable man, but it's basically just a sarcastic. It's a total annihilation attack on brutus. So so we might not get that with our method here.

A

um So that's what I want to talk about with respect to sentiment, analysis, um the other thing topic modeling. So we thought about uh what was the goal?

A

The goal is we want to find documents which share kind of the same themes and identify those themes, be it like politics, business or sports, or whatever the domain is um and like one of the most popular methods for doing so is called leighton d share location, which is a probabilistic method that was developed by bly and others in 2004, and what it assumed like the basic assumption is that a document has is is represented as a distribution over over topics. So so that's what differentiates topic modeling from cluster clustering algorithms.

A

So you might have come across like k-means clustering right, for example, where the assumptions that any observation belongs to a single group like we just but here in the in these type of models.

A

The assumption is that any individual document is kind of a mixture has different proportions of different of the different groups, and while that makes sense, you can imagine that, for example, like an article about david beckham might be like both like a sports, but it also might be this celebrity likes paris, hilton uh topic, so um so, and then there's no clear-cut distinction of them and that's why these types of methods usually work much better uh on text data than like something like clustering, algorithms and this particular method was is- has become really popular because it's easily expandable and so many people have kind of developed variants of it for any given problem.

A

So now, let's look at how what we do here. So here's a simple example again where we use. uh I use this in javascript. uh We again use our corpus of speeches from presidential elections uh this time. I also get some like another word list here on top like these stop words in the second line. So that's, uh let's see, stop words in text. Mining denote like high frequency words that, like pronouns or conjunctions like the end etc.

A

So all these words, which do not have any any significant meaning on their own and uh in some contexts you won't just want to remove those, for example, in the sentiment analysis case, it didn't really matter because they might not get a score anyway, but here in this case it matters, because what we will do is we will represent a topic by its most probable words.

A

So in that sense, if there are some words which are so frequent that they appear in kind of an all topic, then this would kind of obfuscate our results a bit, and so we rather remove those at the beginning. um Besides that we load this lda latent, share location module here and and then we pro proceed. Similarly, as before, we tokenize our texts, and then we here now keep only the words which are not in the stop word list, and then we create this lda model.

A

This function here takes like two two parameters. First, is like an array of documents, and the second is the number of topics.

A

So in this type of model people, you actually have to specify like how many topics you want in that sense, and there are some some some criteria that you can use to choose like an appropriate number of topics, but what people often do is they just run the algorithm for different numbers and look for kind of what's most sensible, so it's considered kind of debatable what's best in that sense, um and then we just fit this algorithm. uh So here's some some uh parameters which uh basically decide how many iterations uh we won here.

A

So let me show some of the results of this. So um for the speed list from 1930 to 2010 we get, uh we said we wanted to have three topics so, and these are the high probability words for each of them, and so I think at least like the first and third topic. They have a clear uh theme. First seems to be mostly like military interventions, wars etc.

A

um Where and the third one seems to be mostly like government programs and and economic programs uh from from the government, and what now it's interesting to look at how the different speeches are like are made up of these different topics, and we can look at this over time right because each for each year we have one speech, uh and I found it's like the war. One is pretty reasonable. I think so this.

A

So we see that here right in the second world war that it spikes uh and then we would see also like korean war and then now in in the 2000s after 9 11, there's another like spike in that topic proportion, um but some of the other ones. I was kind of uh surprised in a sense, because I would have imagined that um mostly there would be like differences across the different political parties, so that, when democrats are republicans are holding office, that we would see differences.

A

But it seems like more like a time trend here, and the time seems to be that this uh topic of government programs and government interventions really in the 1930.

A

It was like almost like the whole speed like 90 percent here, and it goes down to less than 20 like 20 in the last uh years, and so that kind of really, I think, shows that, like after, like the worldwide recession in 1929 like there was like the new deals, lots of government programs to aid the poor- and that seems to be like have kind of disappeared in the uh to talk about that and on- and this coincides with the rise of the green line here, and this is more like a topic.

A

I think, where it's about like tax cuts and health, health care and balancing the budget and all these things. So so I I my guess would be that this kind of attribute to kind of the win of neoliberalism in the sense that now cutting expenses and that government efforts to aid the poor are not not that big of a discussion in the united states anymore.

A

So with that, I would like to close hope. I gave you some some ideas of what typical text. Mining applications are, I'm free for questions and I'm around also. So if you have any, if you want to discuss anything, please let me know thank you for your.

A