Numenta Numenta Research Meetings, 19 Feb 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Seeing is Believing Part 1 (NRM Feb 19, 2020)

Description

This research meeting was split into two parts. This is the 2nd part of the research meeting, but the 1st part of Aries' talk.
First part of this research meeting: https://youtu.be/3fdl9O7WTHM
Second half of Aries' talk: https://youtu.be/3THc5dN-2Wg
Discuss at: https://discourse.numenta.org/t/numenta-research-meeting-feb-19-2020-part-2/7239
Paper: https://www.nature.com/articles/nn.4385

A

Sure, hey, we are recording.

B

C

B

I wanted to give this podcast.

C

I've been doing what my mom is so, as most of you know, I guess: I'm, a scientist by training experimental do something: dissonant, my PhD in a short post computer cowboys lab in Basel, that's free to Commissioner Institute, and the whole idea is that we work on the predictive processing in sensory cortex and so the.

B

Attending babies.

D

C

To like show like revision process, it's true, of course, it's like let's take this forward. I want to see how far they go. So the way that at least I think about this killed suffering is that the.

A

Brain sort of exists.

C

In a little black box right so we're you mind, many of you have might read like philosophical papers about dead sprites and we have like we're sort of detached.

B

From the outside world, in many ways there's no like outside.

C

Creates- and it has access to- let's say: yes, we have like an internal world of likes extra space in the internal space and, let's say, there's something interesting in the external world, and we want to interface with it right. So you know we can use like our eyes or noses sense of touch and all these things to feedback to the brakes. Wait. Let me ask us to these things: that's rich to the outside world, and this person has access to its own internal states. So how does it do this? How do we like you know?

C

How are we able to do anything at all right? So the leading the prevailing theories is well I'll meet those experimental II, at least to the 50s. This makes people's easels work is that sort of we have this little Lego block model which I'm sure all of you are familiar with this idea that there's okay, let's say, there's a hierarchy in cortex and that you know there's a the one. For example, the primary visual cortex just looks a little thing.

C

It has like a little receptive field somewhere and it has it really likes that one there's a little bar or a dog or something in this direction. Another direction and then you'd like a pool, a number of these cells, and you can get likes more complex for presentation. So when these two cells are active, you get like a little track now the line will look something, and then you could just like arbitrarily superimpose these things, and so you get like more complex shapes. Then you know there is some of it. This is for this.

C

We have things are good for the fusiform face area, but we like response on up to faces with you, there's also a tools and stuff, but the thing is like a lot of actually majority I would say of experimental data, how we seem to behave in the world. It isn't really consistent with this idea. We seem to like fill in a lot of things that are not really there in the outside world, and you know you I, don't think you could explain this just by saying. Oh, this is.

C

So- and you know one of the one example of this is sensory illusions, for example visual illusions being the most compelling, so you know so the baby is like okay, maybe there's some truth to this, but what happens the other way back? We would need a sort of way a more efficient way to do. This would be to say: okay, we have this knowledge about the world. How do we compare it to the sort of fine rail fine-grained detail we have of the world, so this.

B

Is one of my favorite.

C

Illusions to show how many of you are seeing this rotate counterclockwise, okay, I guess.

D

Everyone I can.

C

Get into both ways.

C

B

C

The majority of people seem to.

C

So you know we force knowledge of the world unto our perception.

C

So you know so this, but this you know this also means that we detect patterns where there aren't necessarily getting, and this I think is like a very big, a basic cornerstone of how how we work as a species. So you know, as many of you know, the one of the founding fathers of the idea of metamodels is Kenneth. Drake was a contemporary of volunteering and he sort of detailed this idea of having carrying around a small scale model of reality, and you can use this to sort of like a little physics model.

C

You can like simulate various alternatives, see which ones are nicer than others react to future situations for their eyes. You know tend to be a lot more verbose, those writings from the early century.

C

So let's see how the things like how do we implement? This is all like a lovely idea, but how you know, can we look at like a sort of model of how this can work? So one old notion is the the notion of effort is copies, so this comes from like engineering and also like you know, it's mold idea in neuroscience as well. So that allows me to do things that close my eyes and reach from our water bottle.

C

Without you know having the sensory feedback, so yeah I need to have a 3d model of the world to do this, so you know the simple way that you can do this so they're looking lasers anyway, measures up there, a simple way to do this, is you know, just a bizarre control theory scheme and say: okay, you have you produce a movement and that movement generates a sensory feedback, but also you have an efference copy. So this is just a copy.

C

Basically, the motor output, but it's sort of the difference is that it's sort of transformed into the coordinates of the sensory input. Otherwise it won't work. And then you know you can compare those two and have a little error if there's a mismatch and then that you can use that here to update your output online or you could use it to learn something about the environment.

C

So this is basically what predictive coding is right. It is you make predictions and you and you generated every singles. So what.

A

C

The necessary components to do this, so prediction is right, like I said they need to be in target, coordinates and then experimental. You know to create. A prediction needs to be learned and mile-deep will right. You can't just like wake up. You.

B

C

So you know you need to be able to alter them with time with experience, and you know maybe with those little predictions so they're not like necessarily long very projections and stuff and.

D

C

Right so so, let's say you're looking at d1 yeah right, so we're looking at v1. So our predictions coming only coming into v1 distally from other cortical areas or just view one generated conjunction yeah, sorry and then, of course, the other component is error signals. So these should be because neurons can't take the absolute of something at least that's not know to be a computation. They could do it.

C

You need to have positive and negative types of being meaning that, like this, your prediction stronger than your sensory input, or is your sensor even stronger right, because otherwise you know you just have like his hero, everyone, and ideally they should be directional. The ideas of the it shouldn't just be like. Oh there's, an arrow here deal with it. The error should be. Is.

B

Sort of like in a vector space.

C

The coordinates of the sensory motor model right says like you're off by you.

D

Know ten percent in.

D

C

We have some preliminary data that shows that it's casein, like in a simple model, the I, see.

D

D

Just sharing knowledge, what we have right, but.

C

In look in what ways would.

D

It might hit the wrong known the validators. What's the direction I'm supposed to go directional today, our clothes.

C

Look out of blue you can see with okay, so just.

D

By having the wrong grab the wrong word in the.

D

D

Representations which camera presenters, but they can also represent non vector quantities and you basically, you your error can be, which is the net. The net representation are wrong, clear, Libya. It can be a semantic error because of this, our semantic meaning you're not vector meanings. So.

B

D

Still know what part of your prediction is wrong like that's not. That word is the wrong word, because it's referring to a color and it should refer you this well. That was the wrong word, because it's come on intensive speech, but I wouldn't call those vectors call those semantic parsing parts of the part of the representation are incorrect.

D

C

Mean just you know, I'm still, learning about what you get yeah right and then one you know nice quality.

C

How conflicts and talks about it? So you know you want to have stronger signals when you're more confident but you're wrong, for example, or if you like, or anything can happen to our signals. So so there we'll see I love, I.

B

Lost some of the videos here to talk.

C

Unfortunately, so it's the video both the mouse running. This is like Eric's original experiment, it's supposed, but this is just like moving grading that could be scuffle to how the mouse moves on suspended Styrofoam ball and then sometimes you remove the coupling and then, when you.

D

Move the coupling you get this enormous.

C

Responses which releases population response, which is the feedback mismatch- and this is not explained just by the visual hall itself. So if the muscles run, it's running it like a different sort of speaker randomly, but this halt happens. You don't really see anything at all. It's called the playback cult here. It's the playback.

B

C

This is the traditional visual evoked responses very low, and then you know the second highest response is just running in darkness right. So this is one thing that is also bananas. It's just like the most active you get visual cortex to be is when there's nothing like today, yeah so.

C

A

Way that we think about.

A

C

So there's a lot of.

A

Yeah, you know it.

C

Seems to be yeah doing random things so.

B

C

B

C

Some interesting things about it so, for example, the majority population of cells that are active at running darkness, we're sort of orthogonal to the ones that are there when there's an electron. So.

D

C

As far as Anil, even if these predictions are not some sort of dreaming, like you know like when I have walked on with my eyes, closed, I could sort of visualize. What I'm doing right is it that and then just just gaps in some sensory input to compare it against, but it doesn't.

B

You know at least easily.

C

So what I might conflict that my PhD project was to sort of take this idea and sort of take it to a different domain where you could actually see predictions, because he is if you're, making sensorimotor predictions predictions will almost look unless you're using like very high temporal resolution. You know they will be concurrent to this for them to work, so you want something we're gonna slower, they precede it and that I think notion of space, you're, absolutely right, slower, yeah, we're noisier, let's say so. You.

A

Want something that so space dat.

C

Is that, like you know, if you look at the firing of a pencil or a play, so it's not like it's not a belter function right. It sort of spread out.

D

Why do you expect your prediction to be slow, I.

C

So Daniel, you could see that it's a prediction so either way we would proceed it right. So the thing is, though,.

C

B

C

C

D

C

Extremely fast.

D

The very first question we really one of the first questions. We try to guess here.

D

D

D

C

It's no problem because, like what do you call a prediction where says something? That's actually happening so there's many ways when you said something eternal. That is a prediction just because it responds to the state. So you.

D

D

C

You can take that even back further, so just by the fact that attacked at this funding.

C

C

So, instead of just sticking the reference copies reference copy and accessory, and then you can use that again, you know once you have a map, you can infer your location, ideally right.

C

And then that can really voice and expected stimulus, so things like I'm going to see this coordinates X Y. So that's very.

B

B

C

So unfortunately, I couldn't I mean the video was lost for seeing a mouse that run down the spiritual tunnel, but it is at the side of the tunnel, looks like this, so these are textures that are on the top and it is. We would have two different visual stimuli, the you know, gratings that would appear once the animal reach reach. So once the animal runs they're out, you know it's not massive.

C

You know the movement is cold right and then, once the animal is here, this grating little flashes stay there and for the rest of the traversal right. So just because we want to know when exactly it sees it, and these things are with all them. Spatial cues, basically things that hopefully help. Let us know how far along there's.

D

A feeding stimulates.

C

Ambiguous and then just for motivation, others ended up being annoying for the reviewers. We put a little water reward at the end of the tunnel just to keep the moss product, but.

B

A

Know, there's nothing the mass needed to.

C

Learn, there's no like, so what we did is we changed the variability of the braiding in position, five over the experiments. So this was the first four rows. There is hard to ba-ba-ba a DAT, sorry and then you know there. We did the experiment in lots of two days, so we always be recorded chronically so from the same neurons over eight days, which was a total experiment. So.

D

C

Call these conditions these are blocks of two days. We tried to make it blocks of like a certain number of traversals, but you know my Styles made the way you want them to so we tried to make it more realistic in system, so.

B

First condition for today's.

C

A is a hundred percent, so it'd be as if you learned that it's a then we make it be ten seven times to be an utmost. The realist and then we'd refer to.

D

Be another percent of.

C

The time that makes you reverse flip right, a ten percent of the time and then.

B

The final thing we did was.

C

The last hundred or so rehearsals what we do is, instead of showing it the different grading, whoo-hoo, just not show anything at all. It's a gray screen that.

D

C

Was followed by a condition for some condition. Fight was something that I fought.

A

With Garret to.

C

Do he didn't want me to, and just at the end of reporting I by.

D

Calling it a mission, the animal was expecting, something is.

B

D

Know you could just remember, we only had beats so so when you say to the mission, almost imagining I was running down, seeing a V and they're also one time you don't show it and shows like mm-hmm, but that's not way shown here so there's a two day, transcript so condition for condition. Five, so did you do two days in addition for and then two days of condition, five, no.

C

No, the students.

C

A

A

D

B

And these are not task variables at all. There's nothing.

B

C

Doesn't stress but they do themselves so so.

B

One of the things a lot of the things that I would like I.

C

Think it really interesting because it's my baby right but like to get cut to the chase. First thing we saw first when we're looking.

D

C

Predictive response so.

D

Here we sort of.

C

Been the tunnel you.

D

C

Of the phone has been look activity to space right, zero to zero and 100, and you can look at the traces of two neurons. These are two neurons, two different neurons and one traversal, and one this be preferring your honor I could see. You know the orientation sensitive, be preferring their on post is sort of puzzled expires. You know.

B

Classically in response.

C

To the stimulus, be the great one right, it's like becomes becomes again poop and then indicates the.

B

Southerner on started.

C

Someone have a question yeah.

B

So you're recording in v1 is this.

B

C

Video just before yes, sorry, so this is all calcium, imaging. There's a window. I was a preemie auntie, there's a window of cortex, and this is a gcamp.

A

C

Field of view is about a hundred by hundred fifty microns. But again this depends on the residents kindness in this machine, so the she kept sea.

D

C

Two three: we also have.

D

C

A motor along the objectives.

D

C

C

B

A little bit at the top of layer, two.

C

Three just to get some more cells and so yeah, so the other thing you see is that these there's.

B

A neuron here that sort of seems to predicts is, what's being it's not just like.

C

A random firing thing it actually is a specific to stimulus B. So then, what we do is we can sort of make an arbitrary threshold say like in new ones. I have their peak before this difference.

C

Right so these are so you guys know G companies, things work right to the bank, calcium. They get more. So.

D

C

We're only looking at the solenoid here so so what we see arm is almost.

D

Reduce you to there isn't any, but because we've seen so.

C

It's yeah exactly.

B

The thing is that it's like not exactly I mean it's not probabilistic, but it's chaotic and.

C

Instead said, you know a single.

B

Action potential.

C

Will usually evoke a fluorescence, you know sharp fluorescents onset. The thing is that when you have multiple action potentials you know because the things because of the kinetics little smooth occur. So you know.

D

B

C

Only a realizing.

D

Potentially, currently, for me.

C

Ya know things interpretations. How many action potentials are there? You know you might have missed some.

B

Of them, that's.

D

C

So yeah, what we do is we we look at the you know the stack and then what we do is we draw a region of interest around the body of the neuron. We try to make other Europe girl stuff may be traumatic. You just be taking me of that over number fence and that's our tricks, and then we do some other low-pass filtering stuff on it to remove noise and stuff, because movement right, there's all kinds of artifacts. That can be there right.

C

So you can probably split your neurons into predictable visual neurons, so these are still new ones. It must be selective to the visual input they're receiving soon, as they are beat in this case right. But it's selective.

B

C

The arbitrary criterion is that there are at least three times as responsive to, let's say a as you have to be the maximum peak and there's almost there's, of course, a activity threshold so that we're not capturing.

C

So what you see is that you, these.

D

Neurons have been.

C

Selected so that they're so there's a little bit of regression to the mean here, but we're over here, so these are selected to be predictive in addition for and then you see that how they get stronger with time sort of condition, one they're not there, of course, there's some regression to the mean here we did the same, select show individual neurons and there doesn't seem to be a real time.

D

C

The domain it's necessary to check to see like are these predictions they're in beginning with the mouse, didn't have time to sort of learn the environment very well. But you can also do the same analysis, which sort of mean over and over the conditions or mean of one condition. And you don't see you don't see it's pretty arrows in the beginning, which is the setting check, and this is not.

C

Yeah yeah yeah be for dictionary prediction just throw the first for. Ladies: what is the ratio? They're? A one-to-one really they're, both about 7% of the population. Yeah.

B

In a recording.

C

B

They will respond to them. Yeah I called the spatial cues.

B

They learned the spatial.

A

C

Patterns on David had like that differential peeking through one of them, maybe like this one more. But that was like one big combination on the paper and then yeah.

C

So, okay cool- we see these things. Those predictions would be one, and so next follow-up question is suppose like what do they come from right? So there's loads of huge projections into p1 from all kinds of corollaries, most bigger, especially from ACC and retro, speeding, air cortex and those are in terms of a number of Bouton's or even number. Five words are much bigger than the projections from the FCN, so.

A

C

We could have done ACC or registry neocortex, but we just did they succeed in this memory. Wasn't working on a sec expertise on that and he just decided actually without asking just do the same experiment recording from ACC accents like gives us data. Ok, great, so.

B

C

Video did survive, and this is what it looks like when you record axons, then, because of the lighting. What you're seeing here is this green bar on the right is just the velocity of the animal running and then over over the top. You have the time in seconds.

A

C

The same way, we record neurons the differences right. I didn't wanna, be so your field of view. Usually you need to make you feel a little bit smaller right. That's like you just inject your tracing. Sorry.

D

C

D

D

C

We think their design, but the things you have this accounts to discipler all right. So this is like it's just any. We we don't know exactly how.

A

A

M/M, second, if you have a 50 micron window, that means you passed through your entire field of you with Maya. In what.

A

C

40 fortieth part of the millisecond yeah. Well, you see the things this is not is not a voltage recording right. The kinetics of the.

C

Things like you know, these things are kind of like a bit of.

C

Time check I think you know we want to.

C

Okay, so just to finish up this C part, we see. Basically, the same pictures would be one receives signals, predictive and visual responses. The thing is, the visual response is one being here. This is spin was predicted response. This is Jung said they they're, never purely onset, responsive after learning. This is a predictive component to them look around and in the beginning there. Well, you only get caught, it's sort of like visual, like things in this condition one. This is addition for right.

B

So you only get visual response of the beginning. What do.

C

You name of that, so the conclusion from this for me is well there's by the the connections ACC's by direction all right, so you get v1 talking to a CCC and the other way around. So everyone's telling you see, there's something.

D

Here this is like: okay,.

C

Sending advantage like so is it just the loop I, don't know, but the the fact that these become predictive and even the visual evoked on cycle prediction that part I'm not sure how to explain. Okay I mean there's a lot of potential. You know need explanatory, but I don't have a good clean one. If anyone has.

D

Put this in perspective, we do a lot of work here about our cortical column, model things and make predictions, but if we fall under this paradigm, this is a paranoid. You know we think v1 is doing, and air connections like.

D

D

So that it all happened indeed, one and stuff with this is a pattern that we links back. This is one that would require I'm just making the distinction between different types of prediction. This is one that it's happened over a long period of time. Is it's a high order? Sequence, it's learning, maybe somewhere else in the brain and it's leading to a visual prediction, but it's not in the visual cortex itself, modeling the world I'm, just making them in the scene for our model violently recline to this situation, the.

B

Acc could still in our own descending set of booty.

D

It could be under signals, anyone and there's a general belief, a lot people don't believe being one right and we think it goes someplace else, but it doesn't it'll scale and small scale and time in space, and that would qualify here right so I'm. Just putting that I'm trying out our models onto this and say we don't require our models only require somebody projected that we want to make predictions, but again we're not trying to model this convey exactly.

C

Right so I mean there's.

D

C

Where you definitely can be like compressing right, like you, can like.

D

C

Salutely right so even.

D

In one single column, they committed for the snakes that puts going to make under many conditions. But this not one point I was like I'm trying to try to research, there's something that.

B

D

Still be here, it.

B

Might be there six, you know making predictions, because anybody he's showing that these projections.

D

Amazed to see our prediction and so I think there's an underlying assumption that that's where the predictions coming from now the ones is a manifestation.

C

Of that prediction, well as good as a scientist, you have to be cautious right. So, of course, you know when we think predictions come locally. Big one is layer, five layer, six and we're like yeah. You know this is just one potential source of predictions. You probably if he was a few reporters from Phoenix projections, so.

D

You know it's messy biology.

C

Specimens like.

D

D

Therefore, therefore, if you're going to have remapping or if you're gonna have some prediction coming place, house and I can be one researchers tell me that no.

C

That's true I mean the field is still a little bit. I'd say something traditional.

B

C

We do see that you.

B

C

Know that these are things that happened in the ACC is in the mouths of largely motor area. Right and.

B

We see a lot of things in store, other sensory motor cornice.

C

Predictions, so that was going to be what my next slides, but one thing I want to show you and comes to I. Don't.

B

Wanna shorts, our business, we do want to go through absolutely.

C

But this is just ties directly, a stadium in ties directly to what I mentioned before about predictions being in d1. So we know the omission.

A

Paradigm I showed the earlier. You see.

C

This thing that look like the slide in the introduction to figure into the introduction where you have this mean visual.

B

Evoked response, and then you have this huge emission response.

C

This is a population response and you have neurons that are, let's say, selective to the emission. But then, if you look at the ACC axons, they do not care about the image. So.

D

It seems that the error is.

C

So what does that say if there are predictions of the computed.

B

C

I mean it is I mean you just don't know where the predictions are coming from right, right, yeah, but the thing is the another thing is that it's not in the stuff we see like we just compare it's like a sort of kind of wavy compatible.

C

We can compare thee in any one trial, trial right and want to personally compare the activities of stimulus, predictor cells and visually about cells and see like oh, is there like a you know when, when you have like a strong prediction and use you flip, the grading is the visual able prediction stronger, because it's not as predicted we can predict with the opposite right kind of like a silly.

C

So that seems to be the case when you.

C

C

Yeah is when you see B 5.

A

C

C

Shoot question: what.

B

Percentage of it's.

C

Also wrong now: I have my own paper in a couple years.