Numenta HTM Chats with Jeff Hawkins, 22 Mar 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Deeper into the Neuroscience (Part 2)

Description

We go even deeper into the neuroscience behind new theories of sensorimotor inference in the cortex and HTM.

You should have some background in HTM before understanding these videos. See http://numenta.org/htm-school/ for more videos explaining basic HTM theory.

Music: "Holy Roller" by YACHT
(used with permission from Free Music Archive)

A

Now, let's go back you a little bit more detail: we can yeah. So let's start with your slide, this is basically just a PowerPoint picture. What I was just drawing on the whiteboard here, and so you here you see, we have this piece poet input this after I've been puttin century. Please go into later for that's what terminates its classic neuroscience late for this in the green arrows letter for projection layer, two three neat art and they are.

B

Two three is an output layer that is literally.

A

The cells in layer, three project elsewhere in the cortex, so you have a two layer, input, layer and output layer. This is well known.

A

This is nothing special about this at all, also layer, two three projection along distances: we'll talk about that moment and rejects back to Leo I'm, not showing here the temple memory layer again, number I said earlier: I'm just going to talk about the new model, we're not going to show the temple ground so we're just showing you later for, and we see you later, three we're kind of ignoring lower leg, sweep 3b 3b, okay. So that's the basic model is a two layer in for the infant layer. It is an output layer.

A

This is well established, neuroscience. It turns out just to get between a little more complicated I'm, a bargain at slated this same motif. If you will appears twice in every region, the same input that goes to lay or goes to layer, six, a it's one of the same types of representations, spaceship or many columns- tell us stuff that projection to layer. Five as a five is an output layer and, and they five project long distances and they project back. So this this two layers, secret, we've come up with.

A

We understand pretty well, now appears twice and we're not going to get into much speculation. Why? That is what we're hearing there? We don't quite.

B

Know well, we have.

A

Some big clues, but we don't know right. We got some hypotheses. We have some a lot of evidence to sift through it's not like. Oh my gosh, we have no. I yeah it's just like well, this possible possible. So if you ask you to lend look of just sort of slide here, you see that the states repeated twice now with the new thing which we were just talking about. This is orange line. The.

B

Outer picture and I look a little.

A

Bit there locate allocentric that comes from someplace else and we'll talk about in a second, you ask the question: we've been there and you see. Actually it comes in and it initially projected layer six, that's what we want right, because we're going to model allocentric position in both.

B

A

Layer 6a and that's our go thinning down and then later a becomes the input to layer four, so the Alex actual outside you can put the layer for is coming to my success. Oh my goodness. What's going on and we're sir, we haven't doing that. Talk about that I'm, giving you circuitry and plumbing yeah okay. These are these are physical facts, okay, and what we've said is that the distal inputs on the input layer are going to be allocentric, location right and.

B

Some of this some of these ideas, like we don't know, that's why I'm sort of it works that we know what it's doing right.

A

Fella yeah, you have to be careful. We understand a lot that something to done. I well. I wouldn't normally want the circuitry understand this thule circuit really. Well, we don't mention it occupies twice. We don't stands back all the transformations on, so I'm just giving you the facts and then we're going to come back come on.

A

Ok, you reason you ask weirder the allocentric location come from yeah and the main thing here is it's: coming from elsewhere: it's not coming from this region itself, it's coming from where somebody or is that this go, which is another diagram here.

A

This I go. This has two pictures and now I've to introduce this idea. I have to show I talk about something else as well known winner signed is.

B

That, whenever you have a.

A

Region and sensory or century to like the region, get input from the eyes of region kippers from the skin from the ears, there's always a complementary region, this tool, collection and they're. Sometimes they go by the term. What, and where is what region? Where.

B

We put in the same space of so.

A

There and now two separate regions. Imagine we think about a hierarchy of you know the region, region region. The hierarchy was actually two parallel hierarchies region, region, region and one's called. This is first discovered a vision, so envision is the ventral and dorsal stream, but the term the term one where is sort of likes about it. So this entry in these two parallel vision, streams and there's two parallel auditory streams is to parallel somatosensory a touchscreen.

A

What and where? Okay and let's talk about vision, first, okay, because indigenous wrists first discovered they should want to look at this- is two separate visual systems going on. What's the difference between well, the reason they say, one is called what is because that seems to represent objects in the world, the what is out there and the.

B

A

To sort of be something they see say so represents: where is right, so it turns out, one is an egocentric and ones an allosteric representation. Okay. So let's take the allocentric representation, which would be classic v1 and v2, and before is what this is, the one. If I have I'm looking at an object and I take and I say: okay I'm going to damage the left, half way, I'm going to damage the two or something like that. So what do I know as a person? I would say: oh I see there's something there.

A

I know what it I know.

B

Where it is in reference to me so I.

A

Can go out and reach it, but I can't tell you what it is. The.

B

A

Gone right, this is.

B

A nice, it's not something almost like. You can't recall what you see.

A

If you just have no idea what it is, it's like it's something I can't tell you what it is that announced that, can you use it as your country? Well, that's! Well, it's not to get there because I. If my image is my vision right now, okay, but I can say well, I, don't know what there but I can't reach it and I grab it and you can do it really well, so it's like I have to provide it if you damage the other side, the where pathway but then I.

A

Look as though there's a coffee cup I know there's a pen and this whatever does that, but I can't reach it. I said I, don't I, don't know how to get to it. I'm looking at it I see it. I know what it is, but I can't actually get to it an active on again because the egocentric representations, blah I, don't know it is relative to me, but I can know what it is and the other times I know way to drop me, but I can't see what it is right.

A

So this is a trauma themeselves in your cortex. So.

B

We're think about this structure in Iowa. Ventricle patient reaches talking about one of those hierarchies just yeah one.

A

Week well, I'm so about what's your song here we developed.

B

A

About touch, but it would apply to vision or hearing as well. So when, when.

B

I'm showing right now to take.

A

A classic v1 or as one of these are the primary sensory, cortex or vision in touch and we're saying count of that work, and, if we're talking about that, that's a what pathway right I need to get an a licentious location. Now, where is the allostatic location coming from right all right? Let's go back and look at this slide here. Okay, this is some more Anatomy. That's basically known little sketchy untouchable literature, but there are because we have these two parallel pathways.

A

What we think is going on is any signals exist. We think that there's a communication between the what and the where it's basically providing this information between these two hundred, which a movement that hierarchy right. So that's our somatic sensory there's a motor, cortex m1 and if it's more complicated this, but the sages m1s wants or the motor cortex in the somatosensory cortex there are there like where and what and they connect to each other now.

A

Why would you want to do this because, when, if I'm about to move relative to something let's say, I, have my coffee cup I'm going to move my hand and my finger relative to it? Well, I'm just going to move my hand right here, I'm just going to move my finger I'm going to move my finger down. I want to be able to predict we're going to feel yeah so.

B

I said, move my finger down, but what am I going to feel well.

A

The movement down is a movement relative to my body that that the movements initially start out as a Jewish mystic movement. That's how movements are down whether it's going to land on the coffee cup depends on where the coffee cup is in its orientation at any point in time. Right so I.

B

Can't just say a movement down takes.

A

Me to the bottom of the coffee cup, because the baby doesn't maybe take our side of the property honey, both what you need to know what your motive behavior was, and you have to convert it into an equivalent allocentric.

A

So I have to start with an ego sense representation of behavior right, which is where behavior starts and I have to convert it into a new allocentric location. On the origin ago, we went to so I so that I can apply to my model of the coffee cup and.

B

Know what to expect so every object that you're looking at you've got is the dual representation of you and your relation. It is versus its representation. It's become clear lessons.

A

That the cortex infuses it everywhere yeah it's everywhere- is these conversions between reference strings going on all the time. Senses- allergens, it's everywhere now. So in this joint, maybe it's worth pointing out! I show! This is more of a conceptual diagram of these took these basic circuitry except the Alpha bond was saying hey when the body moves in generates behavior m1. That has to generate a signal which gets across, though, which of these distant sensory cortex and becomes an ala, centric location right.

B

Off and you have to know where you're going well.

A

Where am I going to be on the object after I moved to make some addition? What I'm going to feel I asked to know if I'm going to touch this microphone and move my finger I have a different expectation from such additional mapping and I. Think.

B

With an expectation of their orientation I've been to what object I'm.

A

Touching I have.

B

To know its orientation and position over.

A

My body and once I do all that I can make a prediction right, but this has to happen because you make pictures all the time so there's a process here and how this get converted from the behavior into an ALICE nitrification. We have some theories about that. We have two models that we're not exactly something exactly where it's occurring and all the different layers. So we're I'm not going to go deeper to that. Okay, but we have to occur yeah. The other thing is they have to go backwards.

A

The other way what is ahead is because, let's say I'm, manipulating something like this pin and I say myself: okay, I, want you touch the top of the pen. Well,.

B

What are you have to translate? What is the movement.

A

I have to make to create Tesla top of the pen down. Well, it depends on the orientation Ken yeah. It depends.

B

Like this I throw my hands up, it becomes like this decimal man to the left, so it depends of allocentric location. How do you get that they.

A

Tend to depend orientation or education right, so I can say: I love a movement I want to make relative to the pet enough. We should do this all the time right beyond another yeah, so I say I'm, a woman I want to create in allocentric ordinance I have to send it back to the motor cortex. You can burn it into an egocentric look. So as you can sososo, this communication has to go both ways right.

A

You have to take movements and say what is the new location on the object and you have to say I want to do something to the object. What does the movement I have to do to create that.

B

So in these maybe I could expand on this, because it's in these two different vertical structures: the what and the where commit promoters you get a helping. Everybody.

A

Finally, got up at your office, they would have been drawing. If you say you draw a little hierarchy like this, and you know, hierarchy wonders yeah, so.

B

A

Where is it going up and down and it's going up and down, and what we're saying is this back and forth between them? This other like that, which is doing, which is converting the location right.

B

So they're cooperating all throughout yes,.

A

B

That I can understand space and time essentially way.

A

I know that the more you think about this going to come to realize this, that everything you perceive in the world is its own location in space, and therefore everything has to have this out of such a rotation of social yeah. It's a pretty I mean in some sense would be some time to say out the obvious any kind of in August you can produce it, but on the other hand, it is a detailed model that this is occurring in all century regions.

A

The cortex is novel, and so, as we know, nobody even thinks aquatic society like be watching doesn't need any clothes of the certificate, because when we can now show that even a primary sensory region, epsilon V one can learn the entire structure of 3d objects and most people think it's just extracting some feature tapping on someone else, though it's far more powerful in easily anyway. So this is the drawing we've got. You guys need a one there where we in a what hierarchy and what are hierarchies and ones working analysis. Take one burgundy.

B

Eccentric wondering you.

A

Know behaviors now subspace once during the Averys that you say and they have to convert back and forth between these two coordinate rams. Those.

B

Assume this is maybe p1 prepare well, anyone do underneath this, like.

A

A quinoa just be.

B

Empty so for like that, okay, but yeah interpret other somatosensory processing or audio problem. They are those communicating with each other that they well.

A

Then we have similarities it drawing my pictures of sort of a conceptual drawing us yeah.

B

Tonight but I mean: does your vision help your your.

A

B

Sure at some point there are some lateral.

A

B

A

To an idea that actually I wrote about his own intelligence is that if you think about you, think about your sensory organs, you've got retina and you got your skin. You got your cooking.

B

These are all this two-dimensional.

A

Or flat or raised eccentric, yeah.

B

A

Mean your skin is a million bits of, and your retina is about a million Chuchu of it and so way to think about this is the brain handles them all today, pretty much you know roughly and and your your imagine.

B

Your rent is so big. You have million parmesan.

A

You can't process it all at once, so the most you can handle is a small section at any point. Oxygen model then input coming from all sections. So so you can't in one region process the entire crucial input sure so.

B

You have to have another region.

A

Which don't take that, for the first reason Tony you should have built up well, is it part to the retina are no different? They like doing the.

B

Left side of retina lights other ever can't communicate.

A

With each other, in the first layer in higher the juice too far, parties too much information to pin bottle that yeah, okay. Well, it's no different than saying parts of the retina and parts of my skin are too far apart, and so there is a convergent. As you go up the hierarchy, there's a convergence that occurs cells.

A

You know converts as they go up and there's no difference between conversions between different parts of the retina and just between vision and offshore or tsunami. They.

B

Have that something that's coordinating, but input probably doesn't know.

A

The coordinate is, it know, a predicting understand: ah it's natural we're getting a little far field there. I, don't believe there is any obvious if you can't advise them up willy-nilly sure there is a convergence or occurrence, but there is nothing that says: okay, I'm, processing, vision and auditory.

B

Now, when I'm talking about is I, see a Jaguar here is a where I know the Jaguar and some.

A

I can't overstate wears no we're saying the following: you're mister, the pardon forget we're.

B

Staying in the new theory, I.

A

Can see a Jaguar and no two Jaguar, even in this first region of the cortex, as long as the Jaguar doesn't occupy too much of my visual space I see we were saying each region, it's actually learning the entire three-dimensional or presentational object within what they can see. If.

B

I can see a part of my visual space.

A

I can represent solid 3d object in that visual state I.

B

Don't need to pass someone else saying that the jag line- this is very important thing here. This is something that, before we might have assumed, we needed a higher. Yes.

A

Everyone else up right, yeah, there's evidence by the way is we now discover them into something called border border ownership which scientists have found, which is showing exactly what I'm saying that the even being one can recognize complex objects that are bigger than you think that they can our without hierarchy in a layer within a region a raise or a level of the hierarchy show. So the way to think about this is above you've got.

A

Some you've got some large layer of some impressive sheets of destroying over one dimensional, two dimensional, sir, and it's getting this input of something we don't really. It doesn't really care with input. Coming from it's going to able it's able to within some region of this thing, it's going to be able to model 3d objects. That is some part of the inference. If.

B

The option is larger than this.

A

This one Ridge, you can't do that, but.

B

After cooperation.

A

Well, you guys I, can't I have to have another level in the hierarchy to do that right. So, but the mechanism is occurring, the same exact questions was occurring everywhere, that a single region can learn the three-dimensional structure of an object given somewhat extensive input space. It doesn't matter that vision to touch whatever it is, makes no difference, and but.

B

It might only represent a portion of an object, bursar.

A

If it's all, but in here it can represent the entire object right. So here's one way to think about it, I'm very confident now to say that when I read and I and letters are small enough that and we're just not that they fit with that of students that mention of my v1 or my retina. So.

B

I can recognize that ty.

A

Word, why do you want right? Yes,.

B

Sir, that's a ball say, but most people.

A

Think that's great yeah I'm telling us what's going on, but if that word is bigger or sentence or something will strongly more complex, it was just wrong girl, literally physically bigger, not getting more complex is.

B

Bigger to the repetitive in retina,.

A

Fish for the record- yes, I can find more of the right until and it goes beyond what I can to processing my first layer. There physically just extends beyond them.

B

But input is larger, yeah.

A

It goes like this and I can't do it all in one, and one and I have to go up the hierarchy. Cause I have to get some convergence and it.

B

Is to recognize the word not meaning.

A

Of the word, one.

B

A

The out there, okay, just to.

B

A

The word I mean just reckon worried about much making of it. So, but this is safe to say, recognize word around: let's, let's go there. There is no there's a lot of place in businesses, oh the meaning of something they don't have to so we're going to do it far field. Here we haven't even talked about what we'd actually accomplished in this materia, which were giving some background.

A

This is all about background in the brain about where the stewardship and how it where the garage belongs, which, on the stereo switch over to layer, input-output model that occurs, try to meet region every region has them both. What regions, wherever you can choose to hire work is going on.

A

The whole model is relying on the fact that I have a hollow, centric, location and and I prefer fishing house that you can equal centric and escrow, not true the west where, but in the end, we have a two layer model which does how this works, and we should talk about that paper. But because that's the message that thing we discussed.