Numenta Interview With a Neuroscientist, 5 Mar 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Konrad Kording Part 2 - Interview With a Neuroscientist

Description

In part 2 of this interview with Dr. Kording from the K-Lab at UPenn, Matt and Dr. Kording discuss motor representations in the brain, intentionality, time-warping in neurons, and causality.

A

Last month we released part 1 of an interview with dr. Conrad chording, who runs the k lab at UPenn, where we talked about uncertainty in the brain. Coming up is a continuation of that conversation. This is Numenta community manager, Matt Taylor, so you mentioned motor output or motor motor representation in the visual cortex. So let me talk. Let's talk about that a little bit. Why do you think motor output is produced across the entire neocortex? Why is there motor everywhere so.

B

Say me like this, of course, the motor issue of in this monument. Now, why do you have the brain to move Benton? It's the only reason why you have.

A

B

A

Everywhere, everywhere it's like so so does it feel like your whole brain is, is like basically modeling your whole body constantly all at the same time, in the space around it I mean it has to be right.

B

Yeah I don't know, but like uh you, you can make a pretty clean argument. That movement is what things. No. The only way how you affect your evolutionary success is there's somehow sending the right commands to the right muscles at the right time. Even speech is ultimately a movement act and suddenly.

A

B

Way of of the way people think is dulled. On sockets there were more simple movement circuits have.

A

You seen so I saw this x-ray recently of a mouth like with an MRI or something while it was talking, and the complicated movements of inside your mouth is ridiculous. It's like it's like a gymnastic feat. It's amazing.

B

It's it's off the chart, but then, if you think about it, if you do sports, you might spend an hour every day or if you're world-class athlete you might spend several hours every day, training in that sense, all of us are world-class athletes and speaking yeah. You spend hours every day talking to.

A

Your friend we started learning as soon as we started making noise. You know it's.

B

Destroyed so we have tens of thousands of hours of experience moving the tongue in our mouth and the way we speak makes a huge difference and on.

A

B

People perceive us, it truly is important, so.

A

When you think about it from that way, we're much further from real language understanding, then than we think you know, because I feel like I'm on the same page with you motor I, think it's essential the motor aspect of the integration with reality with your motor commands and that's so crucial to language as we've learned it I mean because that's the only way we know how language exists is through movement and feedback with other people. We're communicating with this yeah.

B

And how they perceive us and how we share ideas right.

A

B

Some of the most amazing things that people do is they have shared intentionality. Like you, wanna understand, intelligence, I, wanna, understand intelligence like it's some level. What makes life is scientists, so enjoyable is that we share that right.

A

It's a social interact, it's brains, brains with and brain, so the collection of brains. You know that really make us human I. Think yeah.

B

What makes us us that's.

A

Great ok, so I got another top. It's a technical topic, it's from one of your papers about. Well, you call it I like to use the phrase time warping, but maybe you can explain the phenomenon I'm talking about in the brain. You know when a monkey, for example, reaches or does something that does a task and and how the neurons involved in that task at least observed in that task, don't necessarily happen when the reach happens. Why is that.

B

It's very interesting if we stop at the other end, let's.

A

B

See something I show you a flash of light and I ask you touch the button as soon as you see the flash of light right? Okay, so it takes a little bit for the information to make it from your eye to your higher-order brain areas, all right! What is interesting is how long it takes the fight to make it cross. Your eyes depends on how bright it is if they give you a super bright flash, it's kind of gonna make it your retina faster than if it's a dim light. I.

A

B

So what that means is that the processing that's gonna happen in your vision. Kopecks will be faster for the bright things and slower.

A

For the not so bright.

B

Things now that introduces that the basically that's, not your own time lock to the outside world, sometimes you're faster. Sometimes he is low and he depends on things like brightness. Now we can make it more complicated. There's like this famous drawing where you have an elephant, but the legs aren't right. So it's clearly not an elephant. Mm-Hmm.

A

B

If you're in that situation, it takes you a while to pass that image, and sometimes some people have that Urich, huh oh my god that doesn't even work. Those legs element like very quickly and some of them have it very slowly and how fast or slow you have. It depends on the situation. So now what that means is that the inside of your brain isn't locked in time to the outside. It's like random time relays happening. Sometimes things go faster. Sometimes things go slower right.

A

B

Is a huge problem for the way we analyze brains, because, because what we do typically, is we give a stimulus- and we measure the neural activity that happens after the stimulus, but what? If sometimes the neural activity is early and sometimes it's late? Well, it means that it's all gonna be washed out in a way I.

A

Mean later than normal, or just the plug is always perfectly happy.

B

Let's say: let's take the easiest case. Let's say we have a brain cell. It shows me after some delay a little spike when it sees something sure now it means that if sometimes let's take it, only the brightness say but sometimes is 20 milliseconds earlier sometimes is 20 milliseconds later. So at that point of time, if I ask what the average activities of the unit will be totally washed out there, despite will sometimes be honest, sometimes it will be late right.

B

So it will look like this is a very boring, very sluggish, very smooth cell, where maybe what it does is something like it says like yes,.

A

B

A

B

Actually, very.

A

Precisely timed, but you're.

B

Not be incredibly precisely timed relative to when you actually see it, but.

A

Very specifically, to the context or something you know is it it's.

B

Right exactly, it could be really complicated and what that means is if we analyze the data, as we do at the moment, and what look like the whole brain is boring. Everything is kind of low-pass, nothing much happens there right.

A

B

Answer that in one trial, where you see it orally and in another trial, you see it late and then you can't even say which cell is earlier or later because well they will. Maybe you recognize both of them, sometimes only both of them later to later point of time, so the interpretation then gets to be difficult. Now, I think this is much more problematic. Even on the movement side, when I ask you to say, plan to attach the tip of your nose with your hands.

B

Well, sometimes you may do it like just in time for the movement or.

A

B

You might do it now, I'd like keep talking with Connard, and then you execute it and what that means is there's no. Alignment of the outside walk was what's it in the brain, but everything we do in neural data analysis of most things that we do is based on the assumption that it's locked inside words right.

A

B

Okay, so now time, warping is like a technical set of algorithms to kind of undo those kinds of problems. It's.

A

A it's: a data processing function right to help identify that exactly.

B

It's like the way we use it is. If you give me lots of neurons and I, to ask the question: well a: how fast are they like jointly stepping through that process that they normally do and it allow a basic he allows that on some that sometimes it might be earlier. Sometimes it might be later. But but this is a universal thing like your brain is not time locked to the outside world, and once you realize that anything, you analyze in your responses is getting much more complicated right.

A

Yeah, that makes sense, I mean there's certain experiences you have that seem very slow and others that seem very fast and anyway there's some evidence that has to do with. You know the balances of chemicals in your brain and stuff, like that I mean yeah for computation in your brain right.

B

In the temperature, if I heat up your brain a little bit, the weight of that.

A

B

Spikes might go, I can go through it. A little faster Oh like a little bit like you, can just do it get warmly dressed, run really hot in the sunshine.

A

B

A

Take an exam, I'll dress, very warm.

B

A

Okay, doctor courting a couple more questions from our forum. We have a couple almost a couple thousand people on our HTM forum and I sort of let them know sometimes who I'm gonna I'm going to talk to and some a couple questions. I'll give you from the forum somebody read your research page and said that they quoted the page, saying that you sort of have these two angles and- and one is- and this is for addressing information processing on the nervous system.

A

One angle is analyzing an explaining electro physical data and and you study what neurons do and the other being analyzed in explaining human behavior, which is sitting with all those neurons, do together and I thought. This is an interesting. His question is: how do you begin to model that huge gap? You know those are those seem like they're, going in two different directions. So he'd like to hear you talk about that yeah.

B

So this is a huge problem that there's this huge gap between basic key behavior, which is complicated, includes billions of neurons and what an individual neuron does and I'm not sure how to cross that gulf. In fact, like I've written a couple of papers were kind of, like voiced, the worries that I have about that. So when we make that bridge in neuroscience, we are often very imprecise, see we take some brain area. We find that there's some neurons that do something we say. Oh yeah, therefore, that part of the brain solves the face.

B

Recognition, part right.

A

B

And that logic doesn't walk up like that. It's basically finding that this, a difference if I show different face doesn't mean that these cells, don't do other things and the correlation with faces doesn't mean which role it has in communication of look for all that I know.

B

If I give you a task where you press your right finger when you see a face that that, like muscle, has a really strong correlation with there being a face and we're arguing that your muscle processes faces is perfectly pointless right, and so yes, there is that huge gulf between those two views and it's a little unclear how to how to bridge it, and you might argue that it's it could be impossible to naively bridge the gap. But let me kind of make the point on how it could be impossible.

B

So it could be the way how all the neurons interact would say. I, look at you and I like phrase, a sentence that that way is of such a complicated nature that people could never learn it, and the analogy that I want to use here is like deep learning systems. Let's say you take imagenet a big data set of images that are labeled and we have like, since Alex Ned. We have like good, deep longing systems that can solve that yeah.

B

They aren't very deep because they have Amazon or laksa here who just decided to turn itself on when they used it. So I know when they used its name. It.

A

Happens, a all the time.

B

So, basically, if the brain is something like a neon network, what which means that it optimizes his properties, it has plasticity it dad's to the outside walls, then, at some level, the way how your brain operates is as complicated as the Walton, which it is. So you can't properly describe my brain unless you also describe how dinosaurs.

B

Right right, ooh bring in an example the dinosaur anything that they say, sir, so basically any reflection of the thing anything that I know must be reflected in a satisfactory model of camera. No, which means there cannot be a compact model of camera because Conrad knows stuff about dinosaurs. So if you cannot compress them all off comma, then in a way we can't produce something that is both a workable model of Conrad and can be in distance. Perhaps.

A

However, my counter would be I mean a model that has learned how to be Conrad, I would say absolutely you're right, but there's also the substrate in which that model learned reality.

B

And they agree with you at some level. Maybe the reason why the golf is so big is because behavior as people exhibit kind of, contains all those things that we long for from the world and rightfully.

A

So that's absolutely right. Yeah and.

B

If it contains all those things, then maybe we are barking up the wrong tree. Maybe what we should rather say is: okay, what is this substrate? What is the learning algorithm? What are the cost functions that the brain may be optimizing? Is it using an optimization algorithms? Those questions then become very central, whereas the question of kind of like how does it work at the the low.

A

Level like okay, how.

B

Do noorans contribute to your behavior might be the wrong. No like the fact that, like somewhere in Conrad's Oliver's your context, neons have Gobbo wavelets. That might not be meaningfully part of a description of her color box right.

A

It's so hard to decide what to study and what is contributing to the overall model. Yes,.

B

In that sense, my answer to that Golf is the Gulf might be of the nature that we need to rethink what we want to study on both sites. Maybe the way we study a behavior isn't quite right. Maybe the way we study nouns isn't quite right, but how those two can come together. That is something that people tend to expect it. Someone else will solvent.

A

Right well, I'm I'm hopeful in neuroscience right now, because, as I don't know how much you know about what we're doing in the Mensa, but we're really excited about grid cells and we've incorporated into our neocortical theory. You know that knowing that I've talked about grid cells, a lot another interview, so I don't think I need to like give a basic description of what grid cells are and I'm sure you know what they are. Okay,.

B

Look: here's the problem, so great cells. You refer to the trimming of cells. You we thought you're tuning cells that basically, as you keep walking in some direction like periodically like go up and down in the activity, maybe.

A

B

Is something that characterizes the specific environment in which rooms are raised where, like that kind of representation, is useful? If you had different mice, it might be totally different or different humans present. The question is to which level tuning of neurons really the right level to reason about intelligence, because the prom with tuning like like grid cells, is that it reflects the experience in our world and.

A

B

That it might be usually dependent on the world on which you grow up. That.

A

Makes total sense, even in the grid cell community right now, because there's still questions about do rats create a two-dimensional representation of space. Is that as the grid cells they're creating only two-dimensional versus other animals that move through 3d space? Are they fundamentally different in the way that they write.

B

And what, if you gave my slake little flying things which they could move.

A

Through the 3d space, well, we know the brain is so plastic and malleable. Who knows I mean you just don't know, but I definitely agree with you, like my grid cells, that work in my brain were built off of my experience with the world, then I, don't think that they would work with it for anybody else. You know and maybe for any other species.

A

For sure I mean there could be some some things that are the same within species I, don't know I'm getting way out of my league here, but but the way everyone interacts with reality and has a specific specific. What I like to call I always go to max tegmark because he described these different layers of reality. There's an internal reality that everyone has, that is unable to be shared.

A

I cannot share my what I read is to me with you, except through a consensus reality which is language to where we've both labeled these things and we have symbols to represent them and we can understand them and then there's actual reality, which we all try and understand. As best we can and communicate about with consensus reality and yeah. This whole idea is that my internal reality, the grid cells that I have are a part of that the grid cells, a mouse has are a part of its internal reality.

A

So it's really hard to differentiate what those mean to them versus us versus that right.

B

Exactly so, that's just why I'm like worried a bit like which role findings like grid cells should have in the way we conceptualize intelligence right.

A

And that's a that's an open question. There's.

B

A bit this is to come back to the question that was asked. It's it's like how to bridge that gap. I, don't know and I'm pretty convinced that right at this moment, very very few people have thought hard about it. It's it's it's a huge gap and it's a gap that we need to acknowledge that we don't know the solutions. If we pretend that we do them, we will misguide people the.

A

Dangers of this gap is that there's, it's so big I think there's so many crazy ideas between one and the other that it's really hard to differentiate between who's idea is crazy versus who's is brilliant I mean in this space it's hard to say sometimes no.

B

One knows exactly.

A

So anyway, dr. cording- it's been a pleasure talking to you thanks a lot for taking your time and giving it to us in our community. It's been really great. So is there anything that you have a soapbox on that you want to talk about. While you have this opportunity, absolutely.

B

I, wouldn't talk about causality, okay right, so I think a lot of the language that we want to use when we talk about brains is causal. Language were to ask how neurons make things happen, how brain areas make things happen or how neuromodulators might make learning happen?

B

Those are all cause of questions we know how like we want to know how one thing makes another thing happen when you look at the bulk of the approaches in neuroscience: they're correlational findings, I show your face and I see what the activity in your brain is and those two a very different statement, so correlations are not really indicative about underlying causality and.

A

B

Want to encourage everyone who wants to think of it intelligence to stop thinking about causality. The the the prom we solve in the world is understand the causal chains in the world. We don't care. What's currently did we care about which things we could do through the world to make the world more pleasant for us and we in same thing as scientists. We fundamentally.

A

B

Causality, which things make, which other things happen, and it's just so easy to measure, correlations and I, believe that large parts of the community therefore effectively start equating the two of them and that's something that we should have. What that's.

A

A good point: what do you think companies like us that are trying to work in this space can do can benefit from that type of perspective, so.

B

I think for a company like Numenta, if you want to I, mean like ultimately, what do you into your models is a causal chain? You say this is what this noon does to that other neuron. So in that sense, when interpreting the existing literature, you could benefit from thinking about it in terms of causality. What does what did the experiment actually say, and what do they not say about causality all right, but also, then, when it comes to say, if you're implicitly, building in objectives that the system has the thing?

B

The question is: what are the meaningful objectives? What's their causal role? How do they cause behavior in the end, and so in that sense, I think it came? The concept can be usefully implement that in in in any model of your activity,.

A

Right yeah, it's so difficult to put lots of the models together. You know in a way that makes sense for everybody involved.

B

But the concept of causality: that is something if you ask yourself how you think about intelligence intelligence. The concept of causality is what makes it intelligence, but.

A

I mean at any point in time: I've got neurons that are predicting. What's what's gonna be happening in my environment right now you know that's sort of the brain as a prediction. Engine sort of idea right this and the causality of those predictions being made involve vast amounts of past experience, not just the past second or the past minute, but years years this.

B

Is this district and you could, but you can, for example, view the prediction: if you, if you view the prediction engine like that, you can say that the wanting to predict things is the thing that causes the Chinon Castle.

A

Oh yeah, because I can see that wanting to predict things you.

B

Know the goal of trying to predict things is what gives rise to how they compute I, see.

A

I'm the impression that the the predictions not a goal is just something that happens as a part of the mechanism of the neural network, at least in in HTM. You know we have a sequence, memory, theory, algorithm and the predictions just occur if you connect them to the input the right way. This.

B

Destroyed but the the way how you set up the connecting them to the input in the right way is such that they change themselves so that they get better at predicting yeah.

A

B

A

Like the world is topological, so you've got to have these all these localized computations. That I mean it's super complicated but I. You.

B

A

Agree with you mean causality is super important and we can't make any assumptions about why we're seeing what we're seeing that for monitoring neural populations, necessarily unless we know the. If and we can never look at the internal reality of the system to verify it anyway. So we had to be very careful about the assumptions that we're making right and.

B

I want to have one more thing. The back propagation of error algorithm is also just a causal inference algorithm. It tries.

A

B

Out which changes in neural properties would make performance be better yeah, so you, local prediction versus global optimization, leads to a very similar logical structure where you have an objective in longing that gives rise to computation right.

A

Hey, oh, that sounds right to me all right. Well, dr. Corning, thanks again for joining.

B

A

Problem talk to you later see you.