Numenta Live Streams, 6 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: AI / Neuroscience Chat - Cortical.IO & Semantic Folding

Description

Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_

A

B

Go back: okay, I'm gonna, shut this off.

A

And we're gonna talk about quarter, piyo and semantic folding, so just to be clear. I work for Numenta a company called Numenta and cortically o is a partner of Numenta, our first partner.

A

We have known them for a very long time. Francisco weber, founded cortically. Oh look.

C

At that neat graphic.

A

D

A

A

So we've known him he's been he's visited Namid so a long time ago, and so let's watch this video about what they, what they do, so they they don't actively use HTM in their current products, but how they.

D

A

Products and like what they do with them is based on the properties of SD ours that we're always talking about in HTM, and what they built is something that could definitely is already sort of positioned in a place to be used by HTM systems as sort of a I think of it as almost like a rosetta stone. You know you can you can use this. This is like a communication medium. The thing that they've set up here so that different systems can can learn and different.

A

Using different representations and communicate I feel like as a consensus reality I'm, always talking about the different three different types of reality: internal reality, consensus reality and external reality, I like to think of cortical I/o, and these retinas that they're creating as sort of a way that we can. We can use as a consensus reality for agents for intelligent agents that have their own internal realities, could use cortical I/o as a sort of consensus Claire to communicate concepts.

A

So so, let's think, let's talk about what this does so, yes, I'm gonna watch a youtube video with you.

A

B

Semantic fingerprints.

A

B

The brain as a model, the neocortex, is a two dimensional sheet. First.

A

Obviously, repeating okay, I'm gonna pause it and we're gonna keep going. The animator who worked on this is the same animator. That has helped me with things so, like the cortical column, animations were came from this. You know. I said I want to start from this brain model and the ones that I use in HTM, school and ones of uses some of our other marketing videos. We use the same animator, so just wanted to point that out I liked. What do you do here.

B

Neocortex is a two-dimensional sheet of continuously repeating modular Assemblies of neurons, so.

A

It's certainly more complicated than that. You know as we break it down into a sheet. This is showing one layer essentially, so this is simplifying it by making it look like there's just one layer in there, but there's there's not it's lots of layers. The.

B

Process all sorts of information like language visuals and sounds in a consistent. That's.

A

A that's, a good good representation of all those different. The same circuit, running.

B

Coleman spawns distributed representation.

B

Strs are very long binary vectors with only a few bits active at the same time, each active bit there's some semantic aspect of the overall message, which is distributed across the representation.

A

D

Is a great visualization, you know.

A

That, because it'd be great, if you could have in a bigger area here, but but that whole transition of.

B

Mice, knowing the distance across the representation a little.

A

Bit different, but you know, and when I do the scaler in HDL, when we talk about scaler encoding and the RDF see when you move it a little bit, that's what it looks like station.

B

A

Moving it a little because you're just you're, barely.

B

A

Through the cat spectrum, you know so as each.

B

Bit has a specific meaning. The activation of the same bit in two SDRs indicates that they are semantically similar in at least one aspect. So.

A

It's crucial to understand this if you're gonna understand how HTM works. So if anybody doesn't understand this type of representation and how you can semantically represent reality, using this type of representation, raise your hand and I would totally take questions. Explain, explain it stupid questions, any questions, because this is like the gateway drug to HTM is understanding.

A

Semantics, sparse, distributed representations, because this really is the way you're. The memory and your brain works is based off of this I'm. A hundred percent convinced this is the language of the brain or the memory storage form out of the brain super super important to get this.

B

Generating the semantics.

A

Oh, none of this stuff is biologically based and.

B

Since we cannot use it selected in order to define the semantic space to.

A

Create I mean when you, when you give a a word in a language to cortical iOS API, it gives you back a representation like this. A sparse district is representation and through the retina, and you can have many retinas, so you can have different languages for retinas or a different subject matter for the retina could only be trained on some certain subject matter in a certain language.

A

Excuse me, but that the process of creating that retina is still it's not representative of how you learn information, it's more of like they're sort of doing spatial pooling in a way well, yeah.

D

A

The biological counterparts for SDRs are our neurons. So if you think, if you think about an SDR every bit, we go back I'll.

B

Come back, and this is important- every.

A

One of these bits is a neuron.

A

And also okay, imagine if you're looking at one of these ignore ignore one so you're, just looking at one of these and you're.

C

A

Right, and maybe this is one of your dendritic segments- so that's also an SDR. Each of your dendritic segments is an SDR right and there's going to be a sparse activation along those segments. There's going to be sparse, activation along the the concatenation of all your segments, you know there's STRs everywhere, but all the bits all representing neural activity and when you have an SDR you've. Basically, that's like a receptive field in some way or of a neuron. That's like what the neuron observes.

A

So this you could also you could think of this as a receptive field or and also each one of these being a synapse to another neuron or being the representation other northern Iran at the end of that synapse. So these are.

C

All the neurons.

A

Yeah this bit how they do this. They wrote a paper about it right in awhile, but they're bit they're sort of doing pool in the posing.

B

A

Doing like a semantic war is.

B

Cleaned of all non-text elements, but.

A

Not in a temporal way at all right, that's the thing this is.

B

Not text is cut into meaning based slices called snippets.

B

The snippets are distributed over a two-dimensional grid in such a way that snippets with similar meaning are placed close to one another and dissimilar snippets.

A

B

A

B

A certain number of terms, the resulting grouping by topic, is called a semantic map. Each snippet is associated with a pair of coordinates that defines its position on the semantic map.

B

Generating semantic fingerprints for each word contained in the collection of snowpants, the system examines all snippets to find those that contain that word.

A

B

A

And gets, however, to find snippets, and now it has context for that work when.

B

A snippet contains the word. The coordinates of the snippet are set to one on an empty map when the positions of all snippets that contain the word are activated. The semantic fingerprint for that word has been created.

B

The semantic fingerprints of all words form the entries in the semantic fingerprint dictionary.

B

Let's look at a simplified example. When the term organ is converted to a semantic fingerprint snippets with similar meanings, activate bits that are close to one another different meanings of the term. Organ form clusters that are easily discernible.

A

See how spatial pooling is happening with their method they're doing this they're there they're doing it this snippet so they're their dental context, like the sub context of the different snippets you can see by the groupings in the STRs occurring.

B

When text that is longer than one word is converted to a semantic fingerprint, the system first converts each word of the text to an individual semantic fingerprint. These word fingerprints are then aggregated and Spotify see.

A

Ya there, the tip oral context was just completely removed, their.

B

Threshold, so that only 2% of the bits remain active. This is how the final text.

A

Can get a fingerprint of a whole sentence or a whole paragraph or a whole book.

B

A

Think not you see the meanings zoom.

B

In them, which is really cool.

A

Because you can.

B

Tell what it's about right.

A

Just by looking at the the clusters and the different.

B

Meanings look different.

B

To learn how cortical IO solves previously baffling issues related to natural language understanding visit us at cortical, don't.

A

C

B

A

So, let's look at the demos they have up here if that still works, yeah free tools. So if there's stuff you can do on the webpage and then there's iris. So let's first do extract keywords, so automatically extract keywords from a text or from a web page.

A

Okay. So let's go find a web page about hopfield networks.

A

C

Go to the Wikipedia page for hopfield.

A

Networks and see what it says, extract keywords, paste a text or URL URL submit so I assume it's doing what they said. They're gonna be doing, couldn't have done the same with unions. Well, the these are these already unions, like the SDR as you get back, is sort of like a union of all the snippets I.

C

Think it's still living.

D

A

There's also iris, which.

A

If this doesn't work, we'll use that it's giving of our sneaked photos.

C

Okay, it's closing some things close close close, oh.

A

Shoot this doesn't seem to be working. What if we just select all copy.

C

Paste it right in.

C

A

Says generate a fingerprint: the aggregation has weights the aggregation on those aren't weights. Those are just stacks they're, just stacks of binary activations, so they're, not weights. You just set a threshold to filter them out, so I.

C

Got a fingerprint.

A

So what do I do so there's a there. We go terms located here: random variables, convolution converges integrals, so the each one of these represents a what do they call them a snippet right and so now I can actually look through the fingerprint at these little groupings and tell what what those snippets are about approximation algorithms, iterative, random, integer digits.

A

So can I I wish there was a way to research point topology, differentiable,.

A

Contextual information, factual accuracy, cognitive interesting, do the. If the word organ is used in several files, the stack seems higher than the other words yeah, but it doesn't I. Don't think that matters that the stacks higher you still choose a place to cut it off and cut it off there. It could be just as high when you cut it off that you're losing information that doesn't matter I think.

A

Okay, so that worked and I don't know what look it highlighted some things in here in the input it highlighted. That's weird that must be I.

A

Wonder why those are highlighted are those like related terms. Those are probably terms located in the SDR, but why wouldn't hop feel the network and connections and all that's highlighted I, wonder so.

D

This is pretty cool. You can just go to this website.

A

And do this? Oh, oh! Those are the keywords keywords, so that's probably just based on these. Maybe these are the groupings here. Kachi was kachi. Protein map k, homology, homolog, histone, heterodimer, heterodimer amino activator protein, okay, so now where they think the really biological terms over here, nucleotides MN, RNA, mRNA.

A

Neuronal mutations I wish I could zoom in and see that but okay, so that was extract. Keywords it's to compare text.

A

Okay, let's take like. Let's take a example of an author, let's say.

A

Stephen King example pros I, don't know shake a stop. Writing style of Stephen King.

A

Alright, should just that's not what I should do, who should I I should just go, get like.

A

Let's just go to a scientific Wikipedia page like pyramidal pyramidal neuron, to go to that Wikipedia page and then go to another wicked peach, Wikipedia page for, like.

A

A

Cuz there is there a Wikipedia page for that: yeah, okay, chicken soup, so I don't know.

A

Let's just take the first paragraph of each and compare the text here. Chicken soup, it's text or we could just at the URL, didn't work last time so and then here we'll do the pyramidal so I've, no idea that this shouldn't have anything in common right, I, don't think what I have to take. But let's try it what's?

A

What's the common? What's common between pyramidal neurons and chicken soup, biological combined randomized, coronary drug dosing, chemotherapy overdose, addicts, bowels.

C

A

Oh, so there's only a 24 percent overlap, which is not much at all, so here's like one of them, competition, knockout competitions, so so perhaps, which is a pretty orthogonal term to chicken soup and in pyramidal neuron, but maybe there's some documents out there that have talked about competitions, soup, competitions and pyramidal neuron competitions manager, third, division, Westham, tottenham, hotspur relegation, Dundee, United, that's so weird, so these terms are so far apart from each other. The overlaps seem to be pretty random. So what if we cheat? What? If we? Let's?

A

Not, let's, not put okay, so this is really good soup that I recently had Tom Kha soup.

C

Tom Kha soup, Tom Yum.

A

Tom Yum, that's close enough!

A

I'm gonna see if this. If the URL one works here, because I'd rather put the euro.

A

And let it do the thinkin. Please work.

A

D

C

We got two soups.

A

And there's 74 percent overlap, ten minutes to.

A

And then so the combined terms here naval stationed what is what what is some stuff here? They must like there's a big group of overlap between chicken soup and Tom Yum Providers, ISPs, Internet, Explorer messaging, malware, I, don't know! What's going on, there's a bunch of software terms. What why are there chicken? That's! That's weird jpg blitz file, caption.

A

I'm trying to find like there should be some food. There should be food communities charitable there. It is Hardy, mustard, mushrooms, oven, onions, so here's all the food, the food grouping, all this other stuff is so strange. Ders up here, I just saw coriander, oh no, that was down there. I, don't.

C

A

It's I assume I mean this is coming out of the retina, so they should be stripping all the HTML out, but let's try it. Let's, let's take like the first, the whole first section here and just paste it in instead of this text,.

C

And then the whole.

A

Well, I, guess it's just this one.

A

It's a little smaller British in manor 74 overlap percent overlap to 51 percent overlap, and so the overlaps are all in these terms: food chefs beverages, okay, so this is like food industry terms, that's more. What I would expect- and here are like biological animals or foods, so this overlap. This makes much more sense. There must be something wrong with the way they're processing URLs, because that was odd. All this stuff there still is some.

A

So that's weird. So this means this is what used to be up here with all of those software terms.

A

D

Makes much more sense.

A

Water desalination.

A

Interesting okay, so we did compare text detect the language.

C

This is gonna, say English yeah,.

A

Detect the language detected language, English disambiguate terms, so it was a sucked example Apple I.

A

Think the orange boxes were the overlap and then the the other bits were not overlapped.

A

Okay, so this is for Apple, and so it gives us the context of it: software, fruit, hardware, recording tree or sales. So let's talk about the fruit, and that shows us the bits. I think that represent fruit yeah. This is the software. These are the hardware bits that are really close to the software bits. Aren't they think of these bits are overloaded by a lot the recording bits, the tree bits, the sales bits, let's put in.

A

Well kind of, let's put in dear.

A

So we've got dear-dear Lakes, which is a must be a place dear animals. Deer remains. Do your habitat, Deer, Park, I! Guess there's not that many takes on deer.

A

What about Mucha lucha set English.

A

Don't now, this is a verb, so I'm not really sure.

A

I'm not really sure dear I, see what you're saying Dear John would probably come up in some places, but I don't know if it's strong enough to be a context for the word: don't help don't park, don't data dump aircraft dump investors dump truck I was thinking truck. Why is dumping help.

C

A

That is interesting. Let's do.

A

Jaguar, this is a good one, because there was a car and a feline.

C

A

Chassis, a race species, software aircraft, temple, mercedes-benz,.

A

Ok, now we're going to filter, tweets and then we'll look at iris to play with that, a little icky I, don't know if this is working.

C

A

A

From I'm gonna do mine right light into your retina filter. How do I do that type of type hood term here politics, artificial intelligence,.

A

Tweak your retina filter twist still auto refresh well, if you'd sure why not oops narrator she, okay I, don't think it's working.

C

A

Oh yeah, then there's iris, that's where I just was so I downloaded this and I'm gonna run I don't have been running. This is the this tool was created by David Ray, it David. If you're watching this. This is a cool tool. So thanks trying to can it get to like your user page I, don't want to view all commits I just want to view you anyway.

A

That's neither here nor there.

A

D

So this should look familiar I.

A

Think see my tum yum soup in the background, so.

C

A

A word here, let's find well I guess we could enter Jaguar, then give us a finger pick for Jaguar and I think says: hold down L mouse button to show similar terms for positions. Aha,.

C

Oh, it's making a.

A

C

A

Somehow, entering L over here hold down the left mouse button. Oh I'm.

C

A

I was pressing, I was holding down L.

C

All right, Clarisse, all.

A

Right, Jaguar, that's interesting, penis and clitoris come up there.

C

Fiat, okay,.

A

So then, let's add another term: let's take out the car right and we want to say not car I think Jaguar, not car. So now the car terms are gone. The clitoris and PS are still there. So okay hunted hunted hunting, hawks nocturnal nests.

A

Sporty trim coops cars, but those should be gone right.

A

Cars do I have to say cars as well, not cars, if I didn't think they incorporated plurals air-cooled, Aero, Pistons, piston airline injection, so that I have to say also planes is there some type of so I have to say motorcycles and planes to.

C

Planes not planes, oh shoot.

A

The dreaded null pointer exception.

A

That's the worst, that's the worst, a second I work.

A

Maybe I can take.

C

It back planes.

C

That's I gotta locked up a little.

A

Well, that is a bummer I got locked up activity.

C

A

A

So ready hit you with that. That beats so hard. Just then.

A

All right we're back to the beats, so I can add two input. Windows, I'm still fit I'm, still messing with us, so you can like link these together, which is one of the interesting two more agents communicating through this.

A

To answer that question I'm gonna have to go to the white board yeah yeah.

A

I think take off the white board so.

A

Here's how it work so say: we have a red nuts off in the clouds somewhere. You've got cortical IO retina API whatever and.

A

It understands some language.

A

So then you have and we're going to keep this philosophical so so.

C

A

Do is you can send it a term, you can send it a term and some.

A

D

Let me think this through I.

A

Worked this I work this through once before and I'm now I'm. Starting to doubt my sanity.

A

A

All this does all this is going to be able to do is to take some context, some some term this this. Maybe we can call this up label. It's.

C

Not even a label, it's not it's not.

B

A

It's a it's data in some language. It's a it's a. What do they call it? Let's just call it a snippet, but.

D

It's but it's essentially.

A

A

You just say: okay, like it doesn't I, don't get it I sort of I had this worked out in my head at one point: how this would help agents to communicate like a like, create a consensus reality communication protocol for things.

A

But I think they would have to be able. The agents would have to know would have to have labels when you confer the snippet okay. Now, there's always a delay.

A

C

I, don't want to call this a were you're really sending so I'm going to use.

B

D

Example, like this cat.

A

So some some agent is off in the world and the tricky thing is like he sees: he wants to communicate, there's a cat to some other thing. This other agent and this this agent doesn't speak, that agents, language and and.

C

A

That's the thing that maybe this is breaking down, because the common language is so all this has to know. If these talk the same language.

D

A

Now I'm, starting to remember.

A

So it's okay, so one of the things that you it depends.

C

On how the the.

A

Revit is trained, but if we can train the retina.

A

C

A

To communicate cat to another agent who.

D

Has- and it has no idea what.

A

How the internal representation of this is, or what happens?

A

It could tell cortical I/o to communicate cap to it and it would send it some sparse, distributed representation of a cat along with the label cat or something something like that, so that this would now have a representation of what at least that thing thinks is a cat, but they both have to be pointing to the same thing. You know so.

B

A

Thing they sort.

B

A

To train each other.

A

Excuse me, as I work out this philosophical existential question in real-time, but I thought I had this figured out at one time, but I don't think.

D

I did so so say here is a cat.

A

How am I draw a cat cat? This is a real cat right here.

A

There we go agent, one sees a cat and.

A

It wants to tell that wants to communicate. You.

D

Know to hate it.

A

This orange agent over here that there that it sees a cat or something like that, but it doesn't they don't speak the same. It has an internal representation of this cat, like it.

D

C

An object, representation.

A

Of cat that it can't that only.

B

C

A

A

If it's shared, it's object, representation of cab with this agent- it's not like this agent could could look up cat and know that it's that agency's a cat there. So there has to be something to identify.

A

It has to say I know this is a cat in English right. It has to somehow and the part of its representation have have the label cat and send it to this in order for this Descendants representation of cat to this agent.

A

D

This would essentially be saying.

A

D

A

And it gets the representation from here. This agent is going to have its own representation of a cat. It's going to have an object, representation or maybe, if it hasn't learned it yet.

D

A

Say that it has learned it, it has its own representation of cat, and so there have also has to be some type of linkage of this object that it's getting from portable I/o to its representation of a cat.

A

It's almost like a pooling, so it's a it's like. um We say that in a in a cortical column, there's there's lateral, shearing going on and and what some of the layers that are representing the object. So so this representation means a certain thing like it classifies an object to this column and it's communicating what it thinks. This object is to nearby columns and they're.

D

Using that, as.

A

A part of their representation.

D

Like they, they use the neighbors.

A

That these are distal connections. You know that are part used to help predict and identify what object is is being represented. So these are all sort of intimately entwined at a cortical column in lovely.

A

But this you can't just send this representation, which is what I'm trying to do here to another agent. That's not that's. You know outside it's not in the same brain is so.

D

I'm trying to think it seems like there's a way.

A

That we could use for bio to be sort of this stable representation of. As long as you know, the language you can get a stable representation, semantic representation that you can start from to make links of your your object, representations to these consensus.

C

A

Representations then I.

D

Think that was the.

A

A

D

Creates STR labels.

A

Well, it's you wouldn't be able this agent would have to learn that this is a cat somehow, and it would have to learn to sign the label to its object representation. It's.

D

Not going to be able to send its representation.

A

To the court Clio and ask what it is, because agents are going to create their own internal representations, they're not going to get it from core Clio. This is sort of an extra layer of aside from these two realities. This is the consensus reality we have to think of it as I think, just just a just a way to pass as much information as possible through a label, because this thing knows a lot about whatever it learned about the retina. You know it knows.

B

About all the associations.

C

A

You could tell it a cat, it can give enough information about the to this agent so that it doesn't, it knows what a cat is. I could never, at least if it's learned how to enter our interface with this consensus layer. It knows what a cat is. Excuse me for.

A

I got a lunch date.

A

I'm, honestly, not sure if all this makes sense, maybe it's too if it's too.

D

A

Two hand: wavey alright I'm, going to wrap up the chitchat on Court of Ohio I. Think we've got done a pretty thorough pass over their capabilities. Now. Court of Ohio also has this contract at intelligence. You.

D

Know Enterprise Solution.

A

So if you're looking for NOP stuff or the way you can apply this they're working on that so contact them.

C

A

A

D

A

Am going to head out I'm gonna go, have some lunch and take care of a few things and work on blog posts, I'm, writing and editing some videos, the rest of my afternoon, so I will certainly see you Wednesday I, think pretty sure will they'll be a research meeting. Wednesday I can never be certain, but pretty sure that will be a research meeting Wednesday. So thanks for joining my morning this morning and have a great day buh-bye, you.