Numenta Live Streams, 24 Feb 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Seeing is Believing Part 2 (NRM Feb 24, 2020)

Description

This is the 2nd part of Aries' talk. The first half is here: https://youtu.be/L8PzquwMV6A
Discuss at: https://discourse.numenta.org/t/numenta-research-meeting-feb-19-2020-part-2/7239
Paper: https://www.nature.com/articles/nn.4385

A

Okay, excellent, uh so just to pick up where we left off uh suicide.

B

Asked me to add.

A

Some um some connectivity data uh so you're, just starting like with an image. This is all rabies, tracing data. uh If you want, if you you don't know what brady's tracing is. Basically the rabies virus uh jumps monosynaptically uh in the sort of uh retrograde direction. That means that, if you inject it into a neuron, it will not go down the neurons axon and to the next neuron that it projects too, but will go up through the synapses through the dendrites, sorry and the neurons that connect that project to it.

A

So if you inject this into v1, this is just a general balance injection to be one, not player specific. uh Here you see just to get like uh you know, an eyeball of the projection density. This is lgm which is considered the sort of main feed forward uh projection to uh to v1, and, as you can see, it's a lot less dense than uh these other areas here. So this is acc, which I talked about a little bit last week, uh and this also was m2 so m2. Is this part?

A

uh No sorry m2 is this part of acc? Is this part, and then like here at the border? You have this various dense integration um and it's we call it acc in the mouse it's just because of like vague anatomical homologs. It has a primate cortex. It's mostly like a motor area, slash association area. You know just it's big and there's probably a bunch of different areas that do different things in it. This is retrosplenial cortex. This is all the way in the posterior. Part of the brain is very close to b1.

A

This is a very. This is the biggest uh input to v1 in general, and then here just for some see the diversity of inputs. You see, like the amygdala projects, a little bit the classroom which we just make fun of, and the uh the orbital frontal cortex uh part of it, and then, of course, there's these intriguing connections from the lateral uh into rhino, cortex and the ca1 and hippocampus. These are both very sparse yeah.

C

And I believe in primates we don't see that projection to be one from hippocampal areas. Is that right.

A

No, I think, that's mainly um that's a tool, that's technique thing, so people uh can't I haven't been able to do this kind of um you can't do rabies tracing, uh but you use like to just pick up the signal you use like these older techniques and it's not something that's been done. So it's still like an open question, so people haven't reported them but nobody's like we checked and we didn't find them. You know.

C

I think in development, in venice and for example, they don't have.

A

That yeah yeah, but that was in the 90s and it was harder to do these things yeah. So.

D

One thing- that's not shown here, of course, is like uh the effect of these connections right, so it's well.

B

D

That the majority of connections to the b1 are not not from lgm, but I believe that that's the primary driver of v1 yeah um so and this I assume this covers all layers of the cortex, so we're not being discriminated about which, when you do this.

A

Injection is non-discriminative, we do have quantification of layer, specific and cell type, specific quantifications.

D

That I can ask to pull up so yeah, so you know the thing so at some point we look at this and some people make the wrong conclusion. Oh look, most of the inputs of you want to come in from acc. Well, it could be mostly inputs to some parts of b1 or bc most input. Some other parts are going or.

C

Not, and it could be that these are.

D

Largely modulatory, some people call them not drivers. um So it's hard to put too much. You can just it's more numerical than it is meaning.

A

Sure yeah, of course it requires some clarification, so um I don't know about retrosplenium on the other areas uh in acc. The projection to ace from ac to v1 is by stratified it projects, layer, 1 and layer 6..

A

uh While we were imaging last what I was showing imaging last time was obviously layer 1, because you it's incredibly hard to do. Axon imaging layer, 6. um and yeah. There have been a number of old papers, and so.

D

That makes sense in some sense, if you think of that, to layer, six we've always thought of that, as uh you know, representing those sort of the good cell areas down there, and you would want a motor connection to that. I know we think of that, as coming in the primate from like uh not from motor core to whether it would be from motor cortex too, but that.

C

Could be explaining what's.

D

Going on there, you know I'm just saying this: these are interesting things. We can't speculate too much about but sure yeah a couple of uh questions on the experimental technique um like what do we know about the efficacy.

C

Of like these rabies pricing, like how complete are they like? What number of you know when.

A

D

C

All to the end.

D

A

All of them actually propagate back and forth.

C

The ones that actually can like light up in the time frame that you have for expressing it, because, of course, you have to wait a little bit.

A

Right and if you wait too long, it kills the cells yeah.

C

Exactly because that's what.

A

Babies does that's why rabies is bad for you, um yeah yeah, that's a good question, uh so I'm not, I think, the serotypes that we use. So all this affinity and stuff depends on the stereotype, so basically the viral, the capsid, basically in the receptors on the capsid, um and I think the stereotype that most people use uh it's bas90. I think it's called um it's sort of like a decent trade-off between what you said. So it doesn't kill the cells too quickly and it seems to be uh pretty.

D

A

Then again, you know, there's all kinds of bites that people haven't described yet right. So we know many aev serotypes or like to avoid layer four for some reason, unless you like only restrict them, they'll only be able to express in layer four and that's how you get there for expression. um Is there any known.

C

Bias regarding the strength of the synapses that are jumped.

A

Yeah, that's also a good question, uh so you would imagine like naively that uh well the more the more.

E

Synapses there are the more.

A

Likely that the rabies will be able to jump right, bigger standards just like easier.

C

To jump than smaller energies.

A

That's also true, so then you yeah, so if, if the trade, if that trade-off, was perfect, sorry, if the tradeoff were perfect, then the you know lgn and acc would both be fairly displayed like it would be a fair representation. I don't really know, I don't think anyone really knows uh but yeah. I could look into this. uh If you want to ask one more question.

B

D

Distinguish between.

B

The particular neurotransmitter system used at the synapse and or whether it's an image yeah.

A

So, as far as I know, I don't think it discriminates between excitatory and inhibitory cells. That being said, most cells are excitatory right right of 80, and this could also be a gfp expression issue. So yeah, it's. um You know these things.

C

D

They're kind of like.

A

Like jeff said, this is kind of like an eyeballing uh yeah. This is the problem with experiments. Well,.

D

It says every technique has its biases.

B

D

You just have to until you're very very familiar with all these different techniques. You can be misled about.

A

Absolutely yeah, so just one last question.

C

When we see like you, know like one bright dodge in these things right, you know they're sometimes like like they stick.

A

Out for the masses is that literally, like one cell yeah, that's so.

D

Here, like these are cells and.

A

Their uh projections like right, dendrites and maximums this.

D

A

An artifact this.

E

Stuff happens in the preparation.

A

B

So uh a better.

D

Quantification.

A

This is just like fluorescence quantification over like uh slices oops uh and so like the um director splinos, the strongest projection, uh lateral, v2, auditory, cortex and then fourth biggest is uh the area two four b, which is uh that's part of mcc that we talked about yeah. I thought acc was big m2 up. Where is that slash m2, so m2 is uh so.

D

Where is that, on the chart.

A

There, I'm sorry sorry over here: okay, that's the right one! Yeah.

C

And where's ac is acc also in that same bar.

A

So we call it slash m2, because uh we're not confident that we will that's all our accounts. Don't include m2, uh so it's more like hey. You know a lot of caution there. So the thing.

C

Yeah it borders the region and there's a lot of so when you are 24 b you mean acc acc,.

D

Exactly slash m2, because this is about just spinning cortex. We were talking about that a few months ago. It's like a sort of a hybrid type of chordal region, it's uh and primates. It's not quite standard.

A

uh Because it has granular gradients.

D

Right yeah, it's it's sort of like a transitionary.

D

You know you get to the edge of the neocortical sheet, literally the angelino cortex and the hippocampus on the edge of the intercontinental sheet. Is they fold up underneath the brain? You can follow the surface of these cells around they, so that and- and I think the retrophenol cortex releases primates- it's under it's in the center. uh You know it's v1 fold into the surface.

D

um It's sort of like it has slightly different structures.

E

It's not classic neocortex.

D

It's somewhat in between so it's I'm.

E

Just pointing it out.

D

It's not a quote standard, neurocortical region, it's uh it's a sort of a hybrid something I forget what we know what it does, maybe mark as you remember, but it's.

A

D

It's like a mysterious thing. It.

A

Is it is very interesting region, so uh one like I just mentioned it has a there's, a granular and non-granular part, meaning it has or doesn't have a layer for um and that sort of changes in the mouse. I think the more lateral part is the a granular, but it can be wrong uh and connectivity wise. It has very dense connectivity to the uh foundation.

D

Yeah, so it's like halfway between, so you know you go to the far end. You hit the campus and you get. You know anthropologists, you got retrospective projects and then then you kind of transition to being more classic. So I just think it's from an evolutionary point of view and from a functional point of view, we just.

A

Keep those in mind. It is interesting, yeah. So, for example, the orbital.

D

A

In the um in the mouse, uh the orbital cortex of the mouse is just like blob, but.

C

It does have some structure.

A

To it, but it doesn't have any of this like uh classic. You know layer, one through six situation.

C

And is it uh from what you how you responded to flo's question? Is it true that so post-synaptic layer, four is not represented in here.

A

Oh, that would be aav uh so with rabies. We think it's represented, so we've.

C

Done we've also done yeah. I will forward ourselves so this includes projections too. Therefore,.

A

Yeah most likely, so uh you know it's a thing where you get all this labeling, because we have like a local, a different floor for it's local and we see we confirm that there's labeling, that's mostly well distributed in the layer across layers and to make sure it's in v1.

A

um This is all monocular v1 by the way, probably with a little bit of binocular too I'll, ask jorg, for um I don't know how we classified the data, but we have like next to give his office a massive poster with, like all of this cell type, specific and layer, specific rabies, tracing results, so I'll try to at least get him to send a picture.

A

um I have a feeling.

D

Having gone to the meteor talking here, we're just looking at this sorry, I don't think we've even gotten any out of your talk yet.

A

It's pretty good.

A

All right right so um anywho, so uh we went over this last time. Just as a reminder, we found a population of stimulus, predictive neurons that required experience to form so condition one's the block of the first two days and they're. Not there then, uh and the visual neurons seem to be more. uh They don't seem to have any characteristic uh evolution with time right uh and they're. These are about 50 50., like they're, both about seven percent of the cortical neurons.

A

We found I showed you the acc axons and how they basically do the same thing, but I think I didn't dwell on this as long so. um Basically, you see the same types of signals in the acc axons and v1. Now, subject of debate is like: do we know that these synopsis, terminator and b1?

A

Are they just passing through uh in the video uh it looked like uh you know these little dots, these grains, uh we think, are boutons and that's what other people think too, but you know always take everything you see the grain of salt and experimental uh results, but you.

C

Did show that acc has a strong projection.

A

Of these particular axons.

B

Absolutely right so.

D

We're like how much of.

A

The signal is actually spent to be.

C

A

In layer one and how much she's just passing by, of course, this v1 is that they're like um it'd, be weird to be passing through there, where.

C

Would it go right, yeah, exactly.

A

A

A

Yeah exactly it's a good working hypothesis. Thank you um and uh right. So we see, for example, so let's say.

D

You have an external segment.

A

That likes the stimulus b right, if you remember this, a b, a b.

E

A

uh When we show to just see if these are visual or not, um you know, you show the uh other grading when you flip it remember. We had this condition where we flipped the grading 10 of the trials uh when you flip it, you show that this, like weird anticipatory component, is still there, but there's no like, if there's a visual response as much attenuated and vice versa. Right, if you have an ax.

E

A

E

D

Unexpected there is a visually important.

A

One for a yeah right, given this talk in my lab so many times.

A

um Right, okay, so we have this, uh you can ignore this. We can talk about this later. If you want, um we have.

E

A

Projection uh from ac to v1 and one of the prerequisites.

C

We had for like this predictive coding thing.

A

To work is that you need the coordinate transformations to work right, so you need the you may need to make sure that the projection from acc to v1, for example, and all these projections need to talk uh to the other, the other cortical area in the language of that cortical area right. So um one thing- and this is in marcus's paper- um is.

E

That is it not me. This is the this person's name is marcus lineweber just for people.

C

A

Marcus's paper uh is that you know you could inject two different fluorophores into like adjacent areas of acc and see if there's like a retinotopical, if there's a gradient right in the projection to be one, is it uh just like almost together or is there a gradient and, of course, oops? We found a gradient. uh So, for example, this you know you eject a red in a green fluorophore into acc. You get a red and green innervation being adjacent to each other, and this is the only quantification. You know you can quantify this. There.

C

Are some interesting patterns? This just says: there's topology, it doesn't say, there's a transformation.

D

A

Yeah I mean the other part of the transformation is that it seems to be in these visual or visual spatial coordinates. So, of course, it's not a conclusive thing, but sort of a coordinates in the brain, not the coordinates in the visual world.

A

I mean it is a supporting finding that they would be called like visual ward coordinates, since they were it's in the topic right yeah I mean we think important.

D

It's like you know, like fit cells. Like three-dimensional reference frames, yeah yeah. We can talk about coordinate transformations for the brain for a different purpose.

A

Yeah right right, so these are coordinate transformations in this concept they're already in the topic and in the sense that the signal seems too much type of signal. That is, that v1 processes exactly yeah, which.

D

Is a non-trivial.

A

Thing to do by the way, uh and so and another thing is that these predictions should be plastic right. If they're learned right, you should be able to reverse them and play with them. So this.

D

Is something that actually happened by.

A

Accident and ended up being the main point of marcus linebacker's paper, but which I will briefly talk through. So his project was uh they had this tunnel right and uh it was like a 2d tunnel and then.

D

Vr, this is very hard for.

A

The not very hard it takes a while for the mice to learn to do this because they walk they're like running on this, like a little styrofoam ball that just keeps spinning around. So it takes them a while to like learn to control that so the in the first days. It was very short just to encourage and you get they get a reward.

B

Again, just to.

A

You know get them to where and then over time. You know they would. You could progressively make it longer, and what happened is that in this tunnel, this 2d tunnel, uh you would introduce some perturbations in the angle of the tunnel right. So imagine the.

E

A

Doesn't do it very often, but imagine the mouse is running perfectly straight. You have.

E

A perturbation.

D

B

That it's like 30.

A

Degrees off on one side or the other, and then the mouse has to correct for that right um good. So we, the marcus, imaged axons again in v1 same paradigm uh but the um but in left and right, v1 as well, and what you see is that each area so left or right v1 is the axons are most responsive uh when the.

C

A

Is contralateral, meaning that in left v1 you're more the the axon's most likely when there's more visual flow on the right right? That's looking at that part of the brain and visual flow seems to be a signal that they like. So this could be via perturbation or just by steering right um and likewise for everyone. It likes uh left uh for right, v1, sorry, uh it likes left.

A

Turning and, of course, just to simply get you know, the mouse would have to turn in the other direction, because it's you can get confused with like direction of.

C

A

The visual flow that produces right um so the way that you.

C

Just like get the stripe right so the way that you produce the perception.

A

Of the tunnel's, turning is that you had an increased optical flow on one side of the of the half, the the visual half field, no.

E

D

No, uh so we weren't perturbing so.

A

It was completely symmetrical right what what this she says is that if the mouse turns there's more flow, if it's like closer, for example, one edge, there's more flow on the right side than there is on the left right, because you're you're turning.

B

A

Turning left, yes, exactly right or you make the perturbation where you switch, you know you switch it 30 degrees to the right or to the left. So the the.

D

Visual, the tunnel is the same, but the visual flow is not.

A

Symmetric right right and uh right so.

D

What marcus did by accident.

A

It turns out- and the champion would be- the main part of the paper was that um he accidentally reversed uh reverse design of the coupling such that when the miles would be trying to turn. You know turning right and you get left visual.

C

A

They instead got right, visual flow and they turn right somewhere, like these.

C

Goggles, you could.

B

C

That right left right and right.

A

Into left is absolutely confusing, but.

C

You learned it after a while right.

A

Last prediction: error yeah, um so what they.

B

A

In the first day, of course, uh the these statistics here so the the preference of these axons was the same right. If you have left right and regulate virtual flow, but over time they it reversed. Basically, this is saying that the uh acc axons didn't really care. So this is not about the motor input they're getting it's about the visual flow input, and this would you would need this.

A

If you were able to uh predict if they were predicting visual flow, they would need it would need to be about the statistics of the environment basically um and yeah. So I mean this is just a tuning index. It shows that it doesn't affect performance. If you have normal visual flow, you know the performance stays the same. This is on the y-axis. Sorry yeah performance is here, and then this is the tuning preference and it's sort of uniform in the beginning. Then it changes, but the performance is more or less.

A

This is the performance of the mouse right, how good so it's quantified as a percentage of time spent within 30 degrees of the target. Okay um and- and this is a tuning preference of the individual axons- do they prefer controversive or subversive visual flow right right, so uh negative would mean they're changing from controversy to subversive uh yeah and then, when you.

B

Invert, the visual flow and you compare.

A

It first in the last days you see that the performance is you.

D

Know on the more or less unaffected.

A

But compared to normal visual flow right, they still learn right, like one of them didn't but like, uh uh but the tuning performance, the tuning preference changes, it becomes negative. So it goes to the uh ipsilateral flow right.

A

So it's not just about like uh topology yeah, so each of the ritual uh yeah yeah yeah, sorry uh right. So I think I showed you all this video at the end last night, so this we're talking about uh the prediction error. So this is when this is when the expected grading is there- and this is the tunnel right so time, zero. The expected rating happened.

A

um And now this is this is the response uh when you don't see it so I'll, go through it again so time zero is when the expected rating happens over here.

A

You see like it doesn't really do much, because d1 doesn't do.

A

C

This is just one example like.

A

This is a very uh ubiquitous thing we found across all mice uh except one mouse, um and so just two because biology right um so.

B

E

When we so yeah so here, instead of showing.

A

uh The other grading, ten percent of the time we just didn't, show anything ten percent of the time right and this is all ice aluminum. This wasn't like a brightness change effect and you find that emitting the stimulus drove by far the biggest uh responses in v1. If you compare this to like the mean response to the a or e.

C

A

It's there, but it's much much smaller. um It's.

B

A bit noisier because of the the fraction of trials.

A

Is only 10 right, you'll always get this, um and you can. You know just make up an arbitrary selectivity criteria and say: look these are neurons that are selectively responsive to the emission because we quantified them as such right.

A

And, as I mentioned previously, the emission doesn't drive any changes in the acc axons, it's only locally in v1, whereas the predictions, if you want to call them that and other visual signals- are in the axons. So this would support and we have other data with sensory motor mismatches that supports that the the error computation.

A

uh If that's what you want to call it, it happens locally in v1 in layer 23. Specifically, we don't really see it in layer 5., um because the the argument.

B

Is otherwise the emission neurons get that strong signal from so to speak?.

A

Exactly so, this is like a sort of you're subtracting the input, the visual input from the prediction, but how does that necessarily prove that it's local, because it could come also from theoretically right from any of the other.

C

Answers that you show doesn't.

A

Prove it it supports the argument right. uh It's consistent with the argument right, yeah. Well, of course uh it could be, but uh we've.

D

Looked at other.

A

Many other projections so far we don't see mismatched signals. We could be doing something wrong, but I of them as much signals. We've seen are local tv one and of course they happen in v2 uh and stuff as well.

B

A

Of sensory motor mismatch, that is all.

C

Right cool, so this has.

A

Been likely uh pretty much what my project has done and they've added some of the marcus linebacker's awesome work um and we sort.

D

A

uh Alexander who's in lisa jokomo's lab at stanford. Now, uh if you guys want to talk to him and bill wyatt who's at ucla and for scared, had this amazing paper, where they looked at like uh sensory motor learning for experience, they had like mice that were dark weird and then they only learned like uh they learned how they have them learned uh like to run on the ball, but the flow had nothing to do with their running.

A

It had to do with what the next mouse was doing, and they did not have these mismatched signals. But this is not what I'll get into here. It's just a lovely.

B

A

For you to read it's like the most thorough thing I've ever seen in my life, uh but like sort of they, they did a bunch of like uh cell type, specific imaging and perturbations off the genetic and dreads chemogenetic uh virtualization. So you had.

D

A

Chemical that like selectively silences or activates one cell type, and they found that this sort of uh this classic sst vip- uh uh somatostatin. um Sorry, yes, intestinal peptide right. um uh So this is like uh adam capex and a couple other people have found to consider this to be a very canonical type of circuit and uh cortex, at least in layer. 23 right, where you have pyramidal neurons, are inhibited by somatostatin.

E

Neurons that are.

A

Themselves, inhibited by vip, so the vip activation disinhibits, the pyramidal cells, so I'm sure you're all yeah very familiar with this.

A

So, for example, if you inhibit, um if you inhibit some understanding neurons here, for example, the uh you don't get a mismatch signal when, when you get like the mismatch of that occurring sorry and try to understand that right, yeah, please uh so you know this is this was all done with sensory motor as much so this is just like the mouse is running and there's visual flow coming to the back, and then you interrupt that visual flow. All the bus is still running right.

A

That's the sensory motor, that's much paradigm, and normally it looks like this. The population average right. So this is just a signal here so hold on that red circle. On the diagonal to the left says, I have either knocked out or deactivated yeah, so this is expressing cells. uh Yes, sorry, this is crimson r. uh It's crimson wait.

A

It looks rc, I think rk or dobson is the inhibitory one right, yeah. Okay, crimson is the excitatory one.

D

What is the overall um learning from this paper? I mean like, what's by the details, right.

A

Right yeah, so uh the learning is that somatostatin neurons uh seem to be um uh seems to be visually driven. They they don't have very modern words. Vip neurons um and npy neurons are seems to be more motor driven um and by having these two layers and by you know, you can invert uh where the.

D

So let me just think about this, follow the thing, so you have this primary inhibition, which is coming from visual input and then that can be disabled by a motor input yeah and how? What do you? What do you take away from that.

A

Good question, so uh you can see this in this diagram here.

B

Let's say you have a.

A

Top-Down input, let's say the set like motor prediction: if it were just a motor input and it can innervate either the uh apical, sorry the diesel uh dendritic tree or uh or the one of these internal types right. Let's say it's uh well.

C

That's the sst.

E

A

Well, let's say it's: uh it's it's a long-range projection, so it's excitatory. So let's say this is vip. This would inhibit the sst, for example. Yes, uh so this gives you a way to have two different signs to your uh prediction: error, the positive and negative sign, meaning it's a prediction: minus the input or the input minus the prediction right. That's the idea of a synaptic circuit like this. Then you can use the same with an input.

C

That splits and you can do two things you can either target an internal that inhibits your target or it can, you know, help excite it exactly so is.

D

This all in support the whole predictive coding idea is that, where this is leading like is the evidence for that I'm trying to bump at a pipe level. If I can.

A

Right right, so the the evidence I mean the evidence is that like, if it would happen, this would be a circuit, that's consistent with it, and it came to be very consistent with like how the what we know about the circuit. So if.

D

Someone a statin.

A

Is visually driven and you have a weak motor input? It's not disinhibited by the it's not inhibited by the vip, and so the mismatch neuron. If you have.

B

A

Neuron here it would be inhibited uh by the visual input right and then, of course, if you arrange it differently, you would have it excited by the visual input right just because of where the the top down, if you want to call it innervation happens to go in that neuron, that little little micro circuit of one neuron into uh or several more ssd and vip stuff is everyone following for this. I think you've, probably.

D

Lost a few people.

A

C

D

Is you know the following is two different things in my world like you can follow the details of his experiments and you can just walk down every little point here, but then you can also just keep track of the big picture. Like yeah yeah, you know, okay, yes, we showed that this exists just like this, but what's the point of it all it's like I mean.

E

Absolute value in understanding these.

C

D

But there's also the theorists you she tried like okay. What did we.

A

Learn about this, you know mostly one of the bigger picture implications.

D

Of this, which is sometimes very difficult to get out of an experimental paper.

B

And that's what I'm trying to that's what I'm trying to get here.

D

So the details- oh yeah, I can follow.

A

Because you know this thing happens to me, you know frequently this transition from like details to bigger pictures, and you know you're you're, going.

C

A

B

A

All absolutely yeah, I'm saying if you imagine you can do it, the bigger picture is going to be too simple to describe it's going to be like yeah. This 90 of what we understood but like ten percent, is something we don't really understand something much subtler, which you need to understand the actual circuit for yeah. I don't think this is the case here, but yeah in general. Now um so right. So the.

C

Policy, if you will.

A

Is uh this, this was from pierre's uh review with tom vs logal. uh It's great great review, highly recommend it.

D

So the idea is this:.

A

The you have a top down prediction, and this can make two types. This can lead to two types of error signals you have one where it inhibits here.

A

It inhibits the pyramidal neurons and then the bottom-up stimulus excites them, and this would be what they call a type one uh prediction error, which is that you know if you're, if your input matches your, if your prediction matches your sensory input, of course, it should be silent, but like this would look like a what a visual style, a visually evoked uh response would look like right, so we're arguing it's not necessarily just like the feed forward connectivity, plus all the pooling and stuff that layer form might do.

A

This is actually a subtraction uh with a prediction. So, if you're seeing something.

C

Unpredictable the visually evoked.

A

Response will be stronger right. That's that I mean the main thing about predictive coding. It minimizes activity when everything is is when the world behaves as it should behave according to the learned model.

A

You see very very little exactly and that's why, like one reason why we think layer, two three is very sparse right, um because your brain mostly gets it right um and you have the other type of error signal which is you know if you, if you accept the distort the distal dendrites and then the bottom up sensory input, you know inhibits the neuron, then you're subtracting prediction. You know you're doing prediction minus right, so you have these two types of air signals and.

E

A

Here somewhere, it must be an internal representation, because you can't only have predictions. Why can't you only have predictions? Well, I mean you could, theoretically, if you're an engineer, come up with a little way that you know your prediction errors, you have predictions and you could have predictions. You could recover the state right, of course, but um the general presentation neurons seem to be something to uh we're using this mainly to explain what we see in layer five and what we've seen later. Five is very dense activity. It's probably still considered.

A

Basically, this I mean you need to have some sort of. Is that a representation of what right.

D

B

Representation.

D

B

Whatever it is that this.

A

Part of cortex is happening to compute at this time, so it could be like a conjunction of motor information, sensory, so.

D

I mean in some sense you have to have like an internal model to make a prediction to begin with right.

E

Exactly is that what this is is that.

D

The internal model, because your predictions are coming from elsewhere in this spot, in your experiment, you're inside the long range conditions- and I was arguing in the last meeting I was saying- but we think most of these predictions are occurring locally and magicians, but there's certainly a lot of local and is that what that is? Is that your internal model? Yes?.

A

So it is that it's the predictions and those this sort of thing that would be projected to like, let's say.

D

Another cortical area to serve as a source of predictions well or or.

C

Maybe could also directly yeah, it could be just local.

A

Too yeah yeah it could also be yeah, but you know we think this happens in the deep layers. This is just a sort of educated conjecture here, so it's not uh so this could be if this were acc. For example, you'd see the same circuit, but let's say these internal representation neurons could.

E

Somehow, directly, indirectly,.

A

Be leading to this like integration of layer of v1, for example, this could be like the prediction here.

C

Because, essentially, that's computing, a difference of.

B

Two different errors right.

A

I guess, like the the left.

C

Circuit uh is essentially computing. The the the bottom up input minus the top down prediction, whereas the right circuit does the exact opposite right, so you get two different kinds of arrows and what the circuit underneath is then essentially doing is because you're subtracting them from each other you're, creating like a difference of the two. So I'm not exactly so yeah.

A

With respect to the recurrence that happens, there like you're feeding back that output, that difference between the two errors back into the.

C

Two circuits, I'm I'm not sure I follow.

A

Right yeah, I don't remember what gary's point was with that feeding it back. So let's say if you have, uh if you're subtracting two types of error signals, let's say since something that looks like a sensory signal, something that looks like predictive signal. If you subtract them uh subtract two errors: what is the computational benefit of that.

A

Whatever it is to not get caught up in time,.

C

It might be sort of you get all the you add up all the errors, and that gives you what how you need to move your state, how you need to change your state yeah, that's right, and that would that would as a machine learning person. That's the kind of thing I'd want. So hypothetically.

B

Errors, if the other, what does that mean.

A

It would mean that you um yeah, I know right so well,.

C

No, I wouldn't cancel each other, is you might have more negative.

D

And less positive at one point in.

C

Time somewhere some other time, you might have more positive and less negative. But if you.

D

Add them together, you get the.

C

Gradient you get the overall direction that you need to change right so.

B

If they're exactly equal, then it'll be zero yeah, but it's a noisy system.

C

So you want to add it all up.

B

What is it, what does it mean if, if this guy says, I think we're errored in this direction? This guy says, I think, we're here with this direction and they cancel out. What does that mean? Well,.

C

They cancel out. That means the average is zero, but in general it wouldn't cancel out in general. One would be higher if you're thinking, scalar numbers in general, one would be higher than the other right. Just like in deep learning. You know you, you have a node that projects to many other nodes and you accumulate the grade, add up all the gradients, and that gives you an overall delta for that knowledge. So that particular case we would not.

B

Have a you could have arbitrary magnitude of errors, but from the two modalities, but you can't make a decision based upon you know if they're not they're, not agreeing with each other as to the well.

C

It's not too different modality. It's the same modality! It's just that you know it's not two things, there's hundreds of these things and they're all noisy and they're all error-prone themselves. So, by averaging together all of the errors you get an overall, more accurate estimate of which, what okay? This is not a detailed thing where this particular.

B

C

That's what my interpretation of that that convergence is okay, you're.

D

Aggregating multiple error.

C

Signals into one- and you need something like that- to update your internal representation in our world. You know that might be the location space where you are in the location space, and you need to update that based on some notion of.

B

Which direction you're moving so right there? If the error signals are.

C

No, if the error signals are exactly opposite and equal, that means the average is zero. It's not.

D

That you don't know what you to do. It's just you're, saying.

C

Your average is zero, so then you don't move. That means you're, not doing anything right right, yeah and when you do move you'll, be updating in that direction.

C

There's something.

B

That says, I'm confused now.

C

The prediction error is the confusion, so there's a ton of confusion right, there's a if.

D

The error is high.

C

That means you're not predicting well, that is the confusion, but the confusion in this case.

B

Could be positive or negative.

C

A

In some sense, the parameters on the left here are visually, responsive.

C

Neurons, but they somehow, you know, get potentially attenuated by the prediction, whereas the right ones are the predictive neurons that potentially get attenuated by the original stimulus. So in some sense.

B

That circuit, underneath.

C

That is computing. The difference of those is still still a. You know, visually selective neuron, but it takes an inhibitory input from some.

A

Prediction around the wrong yeah um so that so that.

C

Seems fine now I just still don't understand what the currency.

A

B

A

Yeah uh so let's say it's: let's say it inhibits this one and it potentiates this one so because it feeds back on itself, which.

C

Means like in in theory, whatever signal you're producing, gets.

A

Cancelled out and those those kinds of circuits exist, a lot throughout the course become this handshake protocols all the time right. So when an area projects from one area to the next, it also silences the incoming area and the step after so, if you have strong predictions from layer 4 today, 2 3 there's also going to be projection to inhibitory nodes that then shut down layer 4..

A

If you're, projecting from layer, 2 3 to layer, 5, there's going to be some inhibitory neurons that then shut down the inputting layer, two three nodes, meaning whatever input you are getting um you're saying. Thank you very much and I'll shut up. um So I think what I mean, what yeah so in the lineup. What this could be is it's saying?

A

Okay, uh I just calculated this difference here, and this is what I am, what I think I'll be seeing uh in next delta t right, right and then yeah and then you're, comparing that, so this could be like some about the dynamics of how the uh I mean.

E

This would make the.

B

A

Would make a lot more sense in a temporal domain yeah and because, if like.

C

Looking at these circuits like this, it's like all linear and all the inputs are equal and all happens. At the same time, yeah.

A

Of course that's subsumed right right and whereas, in reality also just unfolding in time- and that plays a big role in how this thing actually plays out right yeah, we discussed a model of um of visual streams on.

C

O'reilly paper that.

A

Made some interesting argument about you know predictions essentially being computers through different phases through thermo-cortical protections um like using the temporal difference between two and essentially the brain is switching between. You know, like uh prediction, and you know, stimulus input and those neurons are essentially computing. The temporal uh difference between those two phases- um there's no phases here but of course, there's a temporal dynamics of this uh in these signals- um and I don't think we can tell what this looks like unless we'll make some basic.

C

Assumptions about these cell types and actually simulates that circuits yeah, like that is just I mean that's just.

A

A drawing right- um that's not a micro circuit like a microcircuit unfolds in time, there's actual spiking neurons right, yeah.

D

A

I'm going to try.

D

It's I see you're on slide, 20 of 30.. Oh no, these are the others, are all.

A

Five: okay, so how.

B

Close, do we do the end of what year this.

D

Is the end? Oh, damn all right, okay, correct for this! This is the bigger picture. Yeah, there's, not a bigger, bigger picture.

D

I think it's interesting to try to reconcile um the whole predictive coding world um with our understanding of what we study here um and um and I've never never really liked the predictive coding one aspect of it where people talk about what's passed up. The court of a hierarchy is the error. I just don't believe that I don't think that's right, but it's, but it's possibly partially right. It's like just let you see if I can just reconcile our kind of views of the world.

D

I believe that a quarter of a column on its own, you know, millimeter square quarter. Column is a complete sensory motor entrance system. It's got. Every quarter of a column has a motor output every chord column, some sort of input which can be sensory from other regions. The brain has to learn a model of the world. Now, when we talk about rats running amazing, that's a very particular type of model right.

D

We think more generally, about models of objects and things you see and so on, and so you've got this model of the world in here and the model has to be a predictive model. So the models you just look at a single quarter of a column, it's going to be making predictions it's going to be generating a motor behavior. It's basically building some model of what's observed in its entirety. It doesn't require other parts in general.

D

It doesn't require a lot of other parts of the cortex to do this, but then, of course, these these chordal problems have to communicate with each other right, and so I believe, there's predictions going on within within the core of column and many of those.

E

D

They don't they're, as I've talked about like the neuron sticks, uh internal depolarization, and then you have like okay, I have another quarter of a column, someplace else, and these have to communicate with each other right all right. So maybe this part of the cortex can tell this guy what they expect. We've always talked about it here, instead of a voting, it's like what I know can influence what you know and what you know. We can always common consensus about this. You just think of voting, perhaps as a type of prediction. It's like saying.

D

I've already figured something out or I have. I have some knowledge about the world and I'm telling you I'm from you're. You know this.

D

But the many corner columns, all these long-range connections in the cortex are basically voting to try to reach against some section. That's observing the same thing. um So let's say I'm touching touching this pan and I'm and I'm looking at the pen and I'm feeling its temperature and maybe it's even making my sound all those have to be united into the perception of this pen and so everybody's voting the hands. The touch areas are saying. I think I'm touching your pants.

C

But I think I'm seeing a pen.

D

Yeah we're doing that anyway.

C

We don't think of that as a.

D

We haven't talked about it as a prediction: we've been talking about it as uh reaching a consensus about what should be expected at any moment in time, and this morning it says: there's a dynamic and temporal time. So these things might somebody's knowledge may precede what the other one senses. So then you could look at it as a prediction right, um I'm predicting I'm.

D

I think I got this and therefore you should see this, and this is you know, I'm seeing this, but therefore you should feel that um so you can think of so, but these, of course, can't be silent. These are long-range productions. They have to be your axons that are productive and you have they have to be active, whereas maybe nine percent of the predictions occur here do not so um and- and so we could think of. Maybe it was our voting circuitry as sort of this sort of uh predictions you're talking about here.

D

um So I I'm just trying to when I listen.

C

To your talk, okay, I believe everything.

D

You say: I'm not gonna challenge anything, I'm just thinking like how do I cook and how do I reconcile these kind of views of the world uh in some ways? I'm just just throwing out some thoughts of that.

A

Absolutely and what you just mentioned recently, like with longer traditions, that's exactly what we think is going suspect is going. It's like this kind of I'm expecting you to see this, because I have this type of information yeah, uh and it's also like it concurrently says. I'm like this is still happening. What's yours.

D

Yeah we just we haven't used the word prediction for that.

A

Okay, yeah, I'm just.

E

Like it's, the.

D

Same thing in sometimes you could talk, I'm just pointing out that the word when we talk about voting between columns. That is a some sense of type of prediction. It's it's absolutely, um even if it could be vague in our case, so the voting can be ambiguous. It could be like I don't know what it could be: a b or c all those are possible, so I wouldn't call it very then it's kind of harder to call it a prediction: it's like being a convolution. Well, no more like I have uncertainty.

D

These are the possibilities of the things that might be happening. So we've we've talked about how our this voting. Essentially, everybody could be ambiguous, uh like the different parts of your skin they're touching the topic and bingoes, but together they know it's a coffee cup uh individual finger can't know, that's called the government's together. They can, and so each finger has uncertainty and it's like it's more like this union of possibilities, and then you narrow down to the only common thing uh so.

E

I can still call that a prediction, but.

B

It's not the kind of prediction.

D

Most people talk about like oh, you should be predicting.

A

X now well that still needs to happen, though right because imagine you know like, like this type of spatial prediction, you're gonna walk here like I know that this is what's gonna there.

B

Is a temporal predictive component.

D

A

uh And like yeah, I know I'm moving my uh I'm moving my arm. I need to know that it's going to end up here because I need to I need to have that confident. I need to like plan ahead to be able to do that thing and.

E

This gives you a mechanism for.

A

Planning ahead by the way like for simulating.

E

C

Yeah, so that is something the brain.

A

Needs to do eventually like how it fits into the union of both these models or the difference of both. These models is uh well.

D

Each one of these is essentially modern confusion or something they said that each one of these is a sensory motor model. So if I know I'm moving the guy.

E

In the left can say yeah I have, I know I'm moving.

D

My fingers of my finger this distance. If I know I'm going to feel- and um um I mean or something along these lines- is falling apart, but but so then, at that point in time you say yeah well, you should be seeing that or something like that, um but that's a dif I mean there's a couple things going on there. I don't know for if we're actually passing, um I'm not sure what actual information you're passing there am. I passing like the motor information.

D

It's more like. I think I'm passing here's what I'm expecting given my movements, um because the I the eye region in the sonata center region, don't can't talk to each other in terms of their individual.

C

Movements, they don't know what.

D

They don't know what they represent. They'll the ibm area doesn't know what movements it's here. They.

E

Don't actually know what the area represents. It's just saying. There's another sphere. There's another area there that says they're.

D

Expecting something can I you know, transformation, yeah, well, yeah. I think I think we try to keep the chord again our world according to transformations like real, coordinates and right right. That could be local.

A

We cannot point out that there's two kinds of transformations happening here so um so so one thing is of course, yeah. The topology needs to map, because otherwise you.

C

Can't do that kind of thing in the in the topological local circuit right, so that of the circuit here it's corresponding to the local circuit in the acc and that's.

A

Fine and those topological mappings exist, so that's fine, but the.

C

Other thing that happens: it's not really.

A

I mean you're. Not this model does not presume that there's some kind of prediction error being passed down right there. The error is computed locally, uh with respect to the model that is locally within that.

C

Column right as jeff would put.

A

C

E

A

Communicated, I mean between areas.

D

A

So the question is: what exactly is being communicated? It's clear that you are talking about a signal at that maps onto a very specific spot in in space right on the retina, so there's that that is guaranteed by that mapping.

A

But that doesn't mean that, like that voting signal right as we refer to, that is understood uh directly by that local model. So there's some kind of transformation, an incorporation which is why the the actual error is like produced locally. So like what you saw in the in the submission response that they that there's no.

C

Firing from the acc's right essentially.

D

Saying there is no vote yeah. I think I think the way, maybe all right we might. We all be talking using the same words differently.

B

Yeah, if you're saying you don't believe.

D

An error is propagated, I'm with you on that. I don't think error is a product and that's the core hypothesis of the predictive coding scheme, which you could be saying you could be passing uncertainty which is kind of like an error. You could be saying I could even know this is a x or I don't know it could be x, y and z, in which case you could be. You know, so you might think that, like that's an error, but that's not, I can. I can.

C

Describe it in a more bayesian sense: okay, um yeah! So it's what we're passing.

A

C

Is the distribution of states that are possible, given the evidence you have so.

D

Okay right, but is that an error? No, that's, not an error, but it.

C

Would be if you're unsure that distribution would be larger, yeah.

D

If you're really sure that.

C

Would be one thing if you're 100.

B

C

You would be passing here's exactly what I think it is, um and you would pass that on if you're, if it could be two.

D

Or three things: it'd be a larger signal.

C

That's a union.

D

C

Several kinds and that's like.

D

Almost a pure bayesian thing: it's the distribution of possibilities that.

C

Are consistent with the evidence so far and that will.

B

C

In its analogous to error, in the sense that it'll be high when you're confused, yes and it'll be yeah, but it's much more powerful than an error signal. Right.

A

It can do everything an error signal can do, but it's much more powerful. Well, the thing is we need to think about. Is.

C

A

What do we think of the fact that, in the emission case, acc.

D

Axon sounds silent right yeah, so that would tell you, let's do that in the emissions case right so when it is actually ambiguous running what happened, which they.

A

So you don't show the you know, actually don't show the stimulus and then you want to get this huge response, but you don't get anything right yeah. So that is no vote. That's what I.

D

Am right but but they then the visual cortex itself can make this prediction and say: there's an error right.

E

Right, I don't need the acc, I don't.

D

Again, that's my point: every cop is a self-contained unit right.

A

D

The things you you need to have.

A

Like yeah, of course, you know, the v1, for example, does have its own motor components and from what I.

C

A

It seems to have a lot to do with projections to the um superior.

D

Collective, yes and yeah, of course, but they have like.

A

Other types of kind of motor coordinates and motor frames, in which, like you, would need to have a type of prediction such as, like you know, just visual flow, for example, um and sort of you know things that are coming from.

D

When you run for.

B

A

Your visual collection, the superior collectors, have more to do from my recall, which they're making.

D

Well, yeah, but I think in it you know it's classically viewed as maintenance of cods in the primate, but my understanding is that uh superior colliculus is actually a much more broad-based, uh primitive motor system. Yeah. It also does have a head.

E

D

I wouldn't be surprised: it removes responsible, running things in the mouse. It's like a it's a it's, a primitive motor, sensory system and um more hardwired. um So it's not limited to vision and isocops. It's a large part of it.

A

Absolutely yeah.

D

A

You know there are like other types of more global or other kinds of motor sensory motor problems, which uh you know. Obviously we have other that other cortical areas exist that wouldn't seem to take some responsibility in computing. You.

D

Know one of the things nothing just I'm sorry to jump all around you. But another thing: when you talk about prediction, the brain is kind of always remember that norms, don't know anything and the court of regent doesn't know anything. It's just getting some inputs and just trying to model those inputs. I have no idea what it represents and if I got a part, if I'm getting a quarter of a column, getting input from this amount of sensory input, and I don't get individual input, they don't know that. So how do they communicate?

D

What could they possibly tell each other? They can't assume anything about knowledge of the other person. The only thing they can assume the only way our voting works. The only way these long-range connections work is that we can assume they're both observing the same thing in the world, the same structure in the world, even though they're observing it differently.

D

D

So that's the only thing that they can share. I, my error, like my visual, is.

B

I have a visual error.

D

uh Oh, I was expecting this video input and I got this input, so I calculate some error. Visual error. I can't share that with anyone, because it doesn't mean anything to anybody else. It's like uh it's! That's that's got to be handled internally, because the language of visual error is not a language, that's being shared of another column, but everybody could say.

D

I have ambiguity about what I'm viewing, because we're all doing the same thing, and so, even though I don't know how you're viewing it like, you may be doing it doing as a vision, not viewing it through touch with someone else's hearing we're all sensing the same thing. uh At least we can. We can try to vote on that because, even though I don't know how you're sensing it we're all agreeing that this is the same thing, so I think about these long-range connections. To me uh an error cannot be.

D

It can't be an error in a particular central modality. It has to be something I I don't think I don't think I don't believe in the error propagation. I did a bit of the coding because I don't think another column or region could understand.

E

My error, but it would have some symmetry with uh how, for example, the retina, how it sort of is outputting, diffs, so sort of outputting things how things have changed if you interpret the error as being something how things change from what it was before uh something or is.

D

It just an observation.

E

You can um you, could you could format that as an error and.

C

A

Well, if the prediction is gonna stay the same then optical flow is an error signal.

A

B

There is a projection from ac to b1. There's.

A

A projection of one to ac and they're. Now their layers are different, so like wherever these, uh these are well there's there's two broad classes of.

D

Cortical projections, ones that are asymmetric and ones that are symmetric in terms of layer by layer. So um there's a lot of so we try to make that distinction, because there's a lot of connections that are like, oh from you know, maybe cortical thalamic projections or layer. You know layer, um uh I don't know layer three to layer four or something like that um or backwards to layer, one. Those are asymmetric in terms of the anatomy that clearly has to do something about some hierarchical structure and that's the classic fellow in essence.

D

But there's a lot of connections like layer, five to layer, five, there's smaller layer, five cells.

C

B

Which are just they just go everywhere and they go back and forth so to us. That's.

D

The voting area- those are the voting connections, they tend to be everywhere. They go all over the place, lacking of uh hierarchy, lacking of topology they're, basically,.

B

Voting, you can vote with anybody that doesn't matter where they are as long as observing.

D

The same thing: whereas, if you have these asymmetrical connections, this is what felon vanessa focus on then that's clearly some more hierarchical type of structure, which is, we think, is related to the compositionality, but I just want to make that distinction, because it's that took me many years to figure out um the difference between those two. So when we talk about voting, we're tending to talk about the layer, the common layer, iso layer, layer, um similar, not this asymmetrical connections- just that's a sure different thing.

D

um So we have to be careful when we're talking about which ones right.

A

So you see this, uh you see this flat line in acc in the projection, but you know there is that doesn't refute that the there's a projection from v1 to acc, another layer that says like hey this is the error acc then has to. If acc gives like a spatially one thing, I didn't focus on so much that I probably should focus on, and these are these predictions have to have have to happen in this facial context. They aren't spatially invoked predictions- it's not like you're, just sequencing.

A

So what you need to get from acc, at least for in this case, is like a spatial like. We should change on that. Topology. Sorry,.

D

I mean spatial in terms of.

A

Visual uh of like.

D

A

Grid cell space like space in the environment, uh so so.

E

It's not just temporal sequences.

A

So, in our model, this, the computation that they would have to do is like okay,.

C

I have a place that I think.

A

I'm in and then I need to out of that, I need to decompose a prediction of what v1 should see given these coordinates given.

C

This argument being that the x-axis here is time, but it really ideally shouldn't be um right, because.

A

Underneath that is space right, that's.

D

How I became the viewer to even in our sequence, memory yeah.

B

That I used to be thinking. Oh, these are the elements in.

D

Time and now I think about no- these are the these- are the locations in the in the melody that would have to be ordered, but they're really just location, space right and then.

A

C

Reference frame is linear,.

A

B

A

But um yeah right.

C

A

The first experiment I ever did- I was gonna. uh My project was gonna, be to investigate the projection from ca1 and the to to rhino cortex, uh which is much sparser in this.

D

Experiment I did there's no like mismatch.

A

Or anything here, it's like a a tunnel like the other one with a bunch of different textures on it and the mouse would just run right. So these are uh lec, slash, ca1, accents and ones. You can't really disambiguate them, because you, you know you have to make an injection. You know it's going to take so.

C

That projections from ca1.

A

Into the v11 yeah right.

B

A

See into layer, v1, uh v1, layer, 1., okay, so.

B

uh Assume there's some mixture, but this is one axon.

A

I'm showing you so.

E

This axon segment seems to like.

A

This is a space right, this distance along the tunnel, and this is just a number of trials, so each child is like a roster.

B

So it's like it seems to like these two.

A

Patterns a lot specifically the last one right but except like you,.

C

Know almost all patterns stand out over time like right, there's a lot of activity in the first 20 trials, yeah.

B

Except for the last pattern.

A

For some reason that stays.

C

Very active yeah.

A

Yeah, let's say maybe like this is just a competition like, let's say the dynamical process in which it settles.

D

A

D

By the way that the circular oval patterns are selected for both of them and then it becomes a little bit more discriminatory, one could argue that the first three times the rat ran through the maze. It didn't just notice the difference between those two circular things and so yeah. Those.

A

Are the same things that, like they're, slightly different yeah yeah? I didn't do.

D

A

Modification on this, because it looks like that.

D

But yeah this is like coming around somewhere around trial. 25 say, wait a second. These are different.

A

um And when I, what I did is, I typically just turned off the projectors: it's not a very clean uh experiment. Let's.

B

C

A

uh Direct outline and right, it turned off the projector, and this.

B

And a couple other songs.

A

That's not seeing anything yeah so here.

E

They're not seeing anything, it's dark, they can still see it's not like you control.

A

Dark they can still see some of their surroundings. It's very.

C

Cool but they're not seeing those patterns.

D

You know virtual reality.

A

Exactly so, it's still fire. This is.

A

Like the whole brain is so super active in the dark, uh and what this I mean this is noisy, uh but I saw this in a few other cells. There seems to be something like a phase procession, not a procession where, with each trial, there's like an increasing uncertainty of where you are- and I don't know if you've seen these sort of uh uh daylight, entrainment, circadian clock and training and things.

D

So does the rat know when it gets to the end or you just keep going and going? Yes, that's my.

A

Oh yeah, so it's basically it's off.

D

And it keeps getting further and further.

A

Further you're inferring that diagonal line there yeah it's it's accumulating air yeah.

A

Yeah because there's no decoration signal, so you keep drifting, you keep lifting yeah, that's pretty.

D

Accurate, though, actually be off by that amount, these are pretty long trials right. How long is running how long? What is the sort of equivalent distance of one of these.

A

Oh, the equivalent distance would probably be a couple of meters, so.

D

So you that's running here, we're talking a couple meters or something 20 meters.

B

C

So- and it's still.

B

D

Of off by just it's just a little bit off each time, it's pretty good.

A

uh Yeah, they are very good at this um and um yeah, so this shows that this signal isn't it's visual.

B

Spatial, it's not just visual and not just spatial right. It's video spatial because it has to do.

A

Sorry, what's going on okay, um it cannot only be visual because well.

D

There's there's no visual here well.

A

It is on the left on the right. There.

D

Isn't so the right is progressing, it has to be spatial right.

A

It has to be, but you see, there's discrimination between the different textures. There is a visual component yeah over there absolutely, but it still happens over here when you reset it right, yeah yeah. It's the same. So it's doing the same thing, but when you turn off the, when you give it uncertainty about its location, it drifts, yeah and.

E

A

See this in our recordings from uh ca, one two: where are they.

A

Right, we didn't find any place else in ca1 when we did the same experiment. We found these things. We found these things that seem to like ratings from different orientations, but they have these weird much wider fields about almost like a great.

D

You know I I do. I do this little thing, which probably a lot of experiments would just like shutter, but um I imagine myself being in a rat and running along this maze, and I so I see these patterns going down and what would I think, but I think that I'm in a linear track and I'm feeding how far how long I am or am.

C

I sitting there.

B

D

Oh, it's! You know it's gotten a better b pattern b, better nate done, and in this case, if you don't find place cells, it's because it might be the right doesn't think of it as a place. That's just like saying: okay, I'm keeping track of work. What's coming up next, it's more like listening to a melody and and yeah okay. I've got this pattern yeah and I have to progress the melody by.

B

Running but I don't mention that my mental model isn't like I'm on this track,.

D

I'm talking down the hallway and how far down I am. It might be easier just to imagine going squares and circles the little squares, the circles. Now I'm done um sure, and that's why I might not find place health because it's it's all it's like accounting.

D

A

Be but it could also be that the computation did just look like these things. Just look like play cells, for example. This would be like the extreme uh argument.

C

They look like play.

D

A

Of the types of environments we're showing them.

D

People have found placeholders.

A

When we find place cells in other experiments, it's because of the how we show the the environment and these uh so in vr, in these linear, one-dimensional borders, people do find places all the time until she yeah.

D

I do I agree, but but.

A

Sorry so, first of all, this.

E

Is these predictions and everything they have to happen in.

A

A spatial context, the.

B

A

Where you basically see if it's temporal, if it's temporal or um or spatial right, and if it's temporal, if you sort of align the uh if you align the zero here, to be sorry, if you, if your alignment activity to be like after the offset of the previous grading or of the previous stimulus in general, and if it were only temporal, then they would line up right if you match for speed. So basically these peaks would coincide right, uh but they don't they spread out.

A

So if you hear a line on previous stimulus right and you've been in traversals by faster slowly, I lost you on this.

D

One sorry, okay, sorry I just I was I zoned out so I'll say it again now right.

A

So imagine you align uh you align your responses either on the gradient, because you know we take like a mean subtraction right of what's happening before and we see the evoked response right. So you.

D

Line it on the grading.

A

That's about to see, and that would be times zero or you aligned it on the previous stimulus, and that would be time minus two how much, how far you've traveled from the previous year, how much time has progressed, yeah exactly right, yeah and if you bend it by slow and fast reversals, you would see that like, for example, the if it's temporal it wouldn't matter if the traversals are slow or fast. You know like it's a time axis, yeah exactly um so.

A

If you align on space and fast and slow traversals, they these things overlap completely yeah right. But if you align on fast and slow, if on the previous stimulus, there's a smear.

D

And they're temporarily offset.

A

And the difference is exactly proportional to the speed exactly so it has to be spatial by that regard, and the other thing we found- and this was figure one in the paper- is that neurons visually selective neurons, for example, uh not.

C

A

Recognize if they're looking at a or b, but they recognize where they're looking at it yeah.

D

We know I can understand.

A

That too, but but so some understanding of space has to be there well.

C

Is there some cell of some sort, space and stimulus or.

A

It could just be like either yeah yeah that or you know, there's enough. Let's say modulation that you're saying I'm not only looking at a I'm looking at a3, because I need to know where I'm looking.

D

But that's like a melody. It's like, I have the you know the same refrain over and over again. I know that I don't get. You know you hear the beginning of beethoven's fifth, it's repeat many times. You don't get confused every time it repeats. You know where it is in the belly, but.

A

You know that's also like uh yeah, but that's also kind of like a space within the yeah.

D

But I don't, it doesn't mean I have to visualize it as a space, but I I I don't know I don't want to focus on this point. If I look at a hallway and I walk up and down the hallway- and I see the hallway in front of me- I have a sense where.

D

If I was in some sort of a stranger environment where these weird patterns would appear, as I'm pretending to run on a ball, I might not.

C

Visualize that there was a hallway I might just come.

D

Up with some of the.

C

Bottom of my head. What the.

D

Hell is going on. This is a strange thing. Animals, and this is not a normal environment. um Well, I.

A

Mean that's hard to control, I mean I.

D

Know it's hard to talk. I don't answer that. I know I know, but my point is we have. The results can be influenced by that absolutely.

A

They can and the fact that it's linear right I mean it- would be better if this were a two-dimensional, uh dimensional virtual environment, so yeah- and I mean literally all the the simplest thing you could do is like a sort of conjunction of space, of visual stimulus and time and one dimension- that's space right, uh I mean I, I don't see another way.

D

We have to be careful with.

A

These, oh absolutely right, um but you know this is like all consistent with what uh other people have shown before. I'm.

D

Just trying to get but the big thing I think we got.

A

On this back, we're just talking about what kind of error signals.

D

Can be propagated right um and I I think that I'm just going to stick with it at the moment I haven't learned.

D

World is, I don't believe, is something that can be understood by another region.

D

The same results can be explained by slightly different language right.

A

Well, I would disagree with you, but I mean it's still. I feel so informative.

A

But we have a couple projects in the lab. Looking at yeah.

D

My thing is it's more of a fundamental principle like how could a bunch of neurons know what another bunch of neurons the error would be. I my assumption is that when the brain is the cortex is sort of built, there isn't specific knowledge, the specific knowledge and the connectivity between them, because that has to be determined primarily by genetics or long-range connectivity, but what that, what that those axons represent, how to interpret them? It can't be. This is something founding assumption.

D

You really can't know where it where those signals are coming from and what they represent it has to. It has to be a message that can be interpreted locally, my own knowledge of the world. So, when you're sending me an error that you calculated an error in vision, I not necessarily can't understand what that means. I don't know what vision is. It has to be in the language of the object, a language.

A

Absolutely, but we see that already one one of these projections at least prediction projection is probably in the local language. So there's.

B

No reason to assume that that.

D

Rejection here cannot be transformed. How did you get that? How do you know this in the look again in terms of the local language.

A

And again, this is not like conclusive, of course, that it's a local language, so to speak, uh being one being the right not to be in one. The fact that the these are visual signals that are stimulus, selective visual and predictive signals are stimulus selected in the topic. So that's.

D

Just because that's because they become active, it's not that they're visually selected, it's! What is every one of these signals is a as a sparse representation of activity. It's not a single axon, and so how do you interpret that as far as representation? Yes, it's going to be visually selected or visually activated, but what's its interpretation, how do I encode? How do I interpret that set of active axons at any point in time, um they're meaningful only. This is like a general principle of the brain.

D

They're only meaningful the local circuit that could generate once you predict um when you take a bunch of axons you project them some place elsewhere in the brain the person receiving. It has no idea what the hell they mean.

A

Absolutely right so the but the things these look enough, like visual responses and the predictive responses. Oh.

D

A

That you could assume that, if this is uh something that lands on the pistol, uh dendrite of uh sort of the ethical dendrite of uh uh pyramidal cells like the problem also would be like this is like the other things I get locally right.

A

C

This is completely consistent with the voting idea too yeah yeah.

A

I don't say it's a good so.

C

Yeah, sorry yeah, that's uh so.

A

The only other thing you need would be the symmetric thing in which, like the error, if it's fed forward would need to be locally computed, or you know in one of the two areas- would need to be locally sort of transformed into whatever the other cortical area.

D

Yeah, that's a question because like when I think about when I've read about predictive coding and I'm.

D

But predictive coding is idea that I have an expectation of which is some sparse activation. I have an actual, sparse actors, an actual input comes in, I somehow calculate the difference and that difference would be in the local language of the region. It would be like I I need, and if I'm going to send that to someone, I have to understand how the receiving person can get meaning out of it, not just that there was an error but understanding exactly how that receiving region can take advantage of the errors.

B

D

I just don't see that still, maybe I'm just missing. I.

A

Mean I think it's something that uh is implicit in. uh I think it's implicit.

B

A

Someone who has thought about the theory it needs.

B

A

Right so um we are looking for this. I mean.

D

I mean you could don't you don't have to do.

C

Any neuroscience.

D

You just ask yourself: how could that signal be interpreted by a set of neurons? What are the possible mechanisms for that? Oh.

A

Yeah sure- and this is something that is like- I think, a little bit under under determined right, yeah.

D

Yeah, so I, but as I thought about, I can't think of a way. So that doesn't mean it doesn't exist and I.

A

Can't think of one so, but it seems to happen in the top down projection, so it should be in theoretically, it could happen in the forward projection uh if you want to think of it that way, yeah um and then another thing in the last line is uh basically the idea that it's we shouldn't think this is a hierarchy, uh at least not in the classical sense, because all these you know we find these values yeah, and I was very happy to read that on your uh a thousand brains, yeah yeah.

D

A

And then again, it's symmetric right. Both areas send prediction errors and predictions to each other, um which would need to happen for for this. This whole thing to work: yeah.

C

But that's like our voting signal. The body signal.

B

C

Both in one signal absolutely right, the question.

A

Is just how explicitly is that an error, and how explicitly is that a prediction in in the language of the area that you're projecting to or not that is a little bit what the disagreement is about right.

D

Exactly how much processing is done with.

A

Those inter-area connections and.

D

Again, I I I think we have to be really careful, because there are interior connections which are a laminar, laminator, laminate or equivalent, and and then there's these other ones which are more classic hierarchical and I think they have different meanings and they have different ways of interpreting. So we have to be careful when you say top down. I can agree that v2 rejected the v1 type of thing could be much more in the line.

D

That's going to be different than v1 projecting the s1 uh which we see you know, we see those kind of connections, those.

C

D

I argue can't be a predictive coding, you know b2 to v1. Maybe I I.

D

We see so many.

A

D

E

I personally start steering away from.

A

Like thinking of d1 as a visual area, as far as the somatosensory area, because.

D

A

All modalities in each area: well, that's.

D

Consistent with the.

A

Voting because, basically anybody can vote so anybody.

E

Anywhere, who knows anything about.

C

D

You're, observing can inform v1 uh like hey. Maybe you should be looking for this thing. You know and that's why we see. uh We see these effects of almost every modality.

A

And everywhere.

D

Else but v1 only gets no one else. You know, v1 gets input uh century for the thalamus from the eyes doesn't get from anywhere.

A

D

A

D

Just lgm, but through relay neurons I mean I don't think v1 gets any somatosensory input. You know, there's a meteorogenic nucleation. That is, you know. I don't think everyone gets important.

A

Oh probably not yeah, there's.

D

uh Other, like.

A

D

Other very interesting saliva.

A

Guys that project too.

D

Yeah there's a lot of weird stuff going on yeah.

A

And that, of course, is why this simplistic.

C

Model cannot be complete.

A

D

Thank you, that's great. What does that mean.

A

Somewhere in the alps near.

D

Did you go there to take this picture where you just.

A

A

E

You all right all right, I'm stopping recording.