Numenta Numenta Research Meetings, 18 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: COSYNE 2020 Recap Part 3 - Numenta Research, Mar 18

Description

Florian Fiebig discusses his attendance at COSYNE 2020.

Discussion at https://discourse.numenta.org/t/cosyne-2020-recap-numenta-research-mar-9/7268
Part 1: https://youtu.be/BLBPqIOyMgo
Part 2: https://youtu.be/JM5DE2BChT0

A

Yeah I wish I had a countdown and we are live.

B

Right, hello, everybody to this: uh what is it now? The third part of me going through some of the interesting things I ran into at cosign.

B

We already did some other conference organization stuff and some of the posters and took a look at the list of pis that attended the grid cell workshop and talked about some things like some of the discussion about the purpose of grid cells, and we talked about scaling environment like the work of barry caswell and about flying bats.

B

um We talked about the the the talk bad with moser like this richard gardner work about this. You know pinging sweep of cells um and so what I want to turn to now, but because we've of course, been thinking a lot about wider um uses of grid cells beyond just navigation.

B

um I would like to turn to the work of beth buffalo, who is giving a talk about gridlike responses and primate and tarano cortex, but not spatial, but instead visual grids.

B

So so this talk kind of started off with a bit of a reminder of the work that she did in 2017. That's this killian 2017 paper down here where they had primates freely viewing um images and recording from media entering cortex and what they showed. Is that obviously on any particular image um you know where the cards, the eye movements right are going and stopping uh depends a lot on the individual image. So you never get like a nice, even coverage of the image.

B

But if you have lots and lots of images to explore- and you know these monkeys love to look at these images, so these are like tetra recordings, um then you can get a very nice covering of the space and uh can start you know looking for signatures of grid like cells uh in in you know, free viewing of pictures.

B

um So you don't have much problem motivating these monkeys because there's lots of interesting pictures to look at and, as you can see, you can trace out the the cards and if you overlay many many of these pictures- and you look at the firing responses in media entrepreneur cortex, then you do find good, like cells.

B

So there was evidence for for grids and even like for for something like boundary vector cells, so that fire only when the when the monkey is looking along the border of the picture um and one of the one of the questions was I'm sorry, you have a yeah. I don't.

C

Understand this, yet um okay, so uh these this like grid like pattern. This is the this, this red dots here, that's one cell they're recording from.

B

There yeah that's spikes of one cell.

C

For a particular image or over multiple images.

B

I think it's over multiple images, but I don't know I would have to take a look in that uh particular.

C

It seems like it would, uh you know, grid cells re-anchor in different different environments, and so you would think that they might re-anchor in different images.

B

Yeah, but the interesting thing is that, apparently, they anchor also to things like the borders of their images or the border of the screen, as actually both of those exist, and so those are of course invariant over many pictures.

C

The borders would be, but not the the not the grids themselves, so the picture with the red dots.

B

Apparently, they're stable enough that they can, you know, claim to to find these cells. And if you look at the autocorrelation map of this one, I mean it looks quite nice like this around that.

C

If they recorded from a single cell as a rat in different environments, they wouldn't see this. It's only in a in a particular environment. Would you see the grid like pattern so, and so the question here we're seeing multiple images and uh and now we're seeing this grid like sort of grid like pattern to it. um That could have been one image or if it's multiple images, and then that implies that the grid cells.

D

Are not re-anchoring? The other relevant question is whether these images are learned or, if the, if the, if the monkey's looking at different images every time, if they're all novel.

B

C

B

A really good question: I don't have the answer to that.

E

Sorry can I interject uh I'm sure uh they addressed this, but uh looking at these responses I mean: did they make sure that these aren't just like those locations in visual space? The monkey is fixating that have some periodicity and obviously there tend to be a lot of them at the borders.

E

So it would look like there's a lot of grid cells and border cells.

B

I'm not sure if I understand you're thinking whether the whether the images have periodicity these are it's just a giant catalogue of of pictures and then made sure that this card span the full. You know image.

E

No, no, I mean if the if, where we see firing, is just where the monkeys fixate for a little bit of time before moving making their next account. If this is basically just the map of the cicadas, instead of something more.

B

Yeah, that is highly unlikely, given that this is, I guess, average of a hundreds of pictures right.

E

It is okay yeah, because then that would explain border responses, but.

B

Yeah yeah, I guess um anyway, so the point of the talk was not much like really this original finding, but to take it a step further and ask um I mean that is like free viewing, but is that also true in when with with covert attention?

B

So they changed the task a little bit uh and I think in interesting ways, so they now had the the monkey fixate on a dot and the task was quite simply to release um a lever when a moving dot that is outside of the fixation changes color right, so you have a fixation onset.

B

Then you, you start like some some moving dot that moves around on, like you can see these these outlines here um and at some point between 700 and 2000 milliseconds, that um that moving dot will change, color and then you're supposed to have a 700 millisecond time window in which you can like release the liver. And if you do that right, you, you get a reward, and the interesting thing about that is of course this is now covert attention.

B

um So this is not psychotic. Eye movement, in fact, like the trials, are aborted when the when the monkeys move the ice and the the the almost all of the failures in the in this task are, as a consequence of monkeys not being able to fixate, rather than monkeys, not being able to use, convert attention to follow the the dot, um and um so the the the curves that these um that the dot is doing is.

B

I think it's called a hamiltonian or something there's like a special movement pattern, but kind of make sure that you evenly cover the space that there's no bias with regards to um sort of um attention on a specific um part of the picture, and so um so that that should, like.

B

You know, also potentially address uh your your previous concern areas um so like they made sure that the full trials always cover the space evenly, as you can see like from these uh snake like outlines here and so, and they did find even on a covert detention with no psychotic eye movement. uh Gridlock firing, as you can see from like the this one of this example cell. Here, you can see in red the the individual like traces here.

B

Like start and stop points- and I think this particular cell you- you actually have yeah, so this is like one trial, which consists of multiple.

B

um You know attempts to follow these snake lines, so that's like starting at stopping points and all of these together nicely evenly cover the space and what you see with the black dots here are the individual spikes on one trial and in the background you have a multi-trial average, and so what you see is that there's that most of the spikes in this one trial are occurring in the hot zones of the multi-trial average heat map, so that just goes to the question of sort of what is the consistency of this?

B

Well, it's good enough, at least for I think, like in one session. They do um I mean these are like a total of like six of these um snakey lines.

B

I think, and then I do like a set of five of those like two two to seven, I think, is something it's a in the paper, and so that is then what you get for, like an average map, and you see in the autocorrelogram that okay, these are like grid-like uh responses, and that was then the this became this 2018 paper, android cortex receptive fields are modulated by special tension, even without movement, um and so one question one might ask okay.

B

If, if we're going to have grid cells now does that mean we also have border cells, as it also means we have something like head direction: cells uh like the direction of cards, for example, in the freezer, counting task or the direction of the attention.

B

um So you sell some of that here. um I guess these are yeah here. You can see like a cell right that fires exclusively on the left border here. um So there you go like a heat map here which actually has like a strong heat spot there, or is that a separate cell?

B

I'm not sure you get like these band-like cells um together with these grid cells, and they did some analysis to show that actually, the relation uh like the prevalence of the different types of cell types is the is the the ratio that you would expect from uh from spatial grid cells and border cells, and you know, vector cells.

B

So this was an analysis. They did on cigar direction cells and it turns out they kind of neatly divide into two categories. One is cells that fire um prison card, so uh sort of upcoming, um like when you're about to exploring.

C

Are we are we switching topics now because this up to this point attention now we're talking about.

B

Yeah right, this obviously relates to the to the task with the sarcastic right. That was like a previous paper that was um yeah. That was a 2015 paper. Actually are you.

C

Reviewing um you're giving a talk that covers multiple papers is that right.

B

Yes, yeah I mean the topic is visual grids. I guess um maybe I should have shown this one first, um like they've, ordered it slightly off.

C

In the covert attention task, it looks like the grid was essentially representing screen space. I mean um you know it's just like it's a it's a meaningless um there's, no depth to this thing. There's no structure to it. It's just the xy position on the screen.

C

um Yeah we're in the images before one could imagine that uh it might be representing something else it might be. You know you might be thinking about the the where you are in this scene right, but so I'm really confused as to what what the what grid cells are representing here. Is it just screen space? Is it something more than that? I.

B

Mean like in the in the cover task, there's a very even exploration of this 2d screen with the use of these um with the use of these. You know random, but especially um unbiased, uh trajectories.

C

Yeah, that's this one's clear, this one's! It's like, okay, we're just representing screen space! There's nothing else to it um right. There can't be anything else too. Well,.

B

It's more complicated of, of course, in the free viewing of these pictures you're perfectly right because there's more to it, but that's why they're average over many many pictures here.

C

Yeah, but here it's not clear whether whether you're representing the screen space, where something is relative x y on the screen or where it is relative to depth on the beach or you know, there's a there's, a gazillion, complicated factors here, which I don't understand.

B

C

Anyway, anyway, so we're off the covert attention that was one paper. This is the talk and now they're going to talk about something else.

D

So one thing with the covert attention one is um they call it covert attention? You could put a totally different narrative on this. It's just like where, uh let's see you have this image coming into the monkey's visual field, it has a 2d variable of where is the moving stimulus, uh and basically it's just it's. um It almost ignores space here. Think almost like the stretchy bird thing, you have a two-dimensional variable, that's being encoded using a grid-like code, and I don't know the word. Attention might not even be needed to explain this right.

F

I mean, of course, if it's yeah, if you don't, you know, if it's just a change in color, you can do that pre-attentively, there's no covert! Attention really required for that or even any spatial thing.

B

Well, I mean like the the thing is, of course, the the monkeys are fixating on the dot in the middle right and the somehow you need to without giving the space itself, meaning you have to make sure that the monkeys, you know, follow the spatial evolution of the dot right, which you get is by the fact that they have to respond quickly when the color changes.

F

Yeah, so I don't know if this is marcus's point or not, but you don't have to follow the evolution. You just have to look for a color change from dark to red.

B

They could completely ignore the space if they wanted to.

C

Well, my assumption you didn't say this, but my assumption was uh usually in these kind of tasks like this. They make it so that the animal would not see the change unless they were attending to it. uh That is the change is small enough um um that it would be right in it'd, be very difficult to see.

G

Now you can say that I was just assuming.

C

That I've done these brain exercises, where this is what they do they. Basically you have to be. You have to attend corporate attention. Otherwise you wouldn't see the thing. It's too subtle. So if it's a big color change here, you wouldn't have to do that at all, but I just assumed that there were many.

F

Yeah so hopefully they have done enough controls and stuff to that spatial uh tracking of some sort actually required. You can do that in a few different ways, but you know, as you've described it here so far, there's no attention really needed.

D

Or even if attention is needed, uh this might be something that's far downstream from that. It's it's like this is enter angle cortex, it's like. Sometimes it represents the neck, the length of the neck of a bird, and in this case it's representing a screen location but like attention, is just like an implementation detail that led to internal cortex being fed this information.

B

All right, I mean at least it's interesting that um well first, this is not. um This is not navigation and, yes, it is space, but it's not even moving in the space right. The eyes are not moving and that's. I think what makes this task interesting.

C

It's also what makes it very very hard to interpret as to what the hell is going on here. So just be honest, it's it's! You know, at least when we talk about iraq moving in an environment, and we have very we understand. There's a very lot of literature. Explain what's going on here here. I I'm confused as to what's actually being.

B

um Where's the confusion I mean, I, I think this is pretty straightforward. It's just.

C

Not you know, I'm trying what the animal is. What are the grit cells mapping right? What is the what what this is the grits like a metric space they're used to map something, but what is it is it the screen? Is it the scene is the thing in the scene and the covert attention part? You can see we're all kind of confused by that, so it it may be clear with the authors, um but it's not real clear to me what you know.

C

What's what is being it's not is it it's is it where the dot is, um and why would that? Why does that require covert attention? uh Why would could it be done if they had animals tracking with their eyes? Would you get the same results.

B

Yeah I mean that was the point right of the of the previous papers, where you had. The monkeys like free, be free being able to the card, but the problem was that it you, you need to like kind of disentangle, then the the movement of the eyes and that's tricky right. If you want to see if, if the screen space is coded, irrespective of the movement, then you would do exactly the the.

C

Laptop that was included- I don't it. Maybe it was good, but it wasn't clear to me you're saying they previously showed that even when eye movements that you get this great, like representation.

B

Yeah I mean these are the cards right. The yellow outlines here.

C

I understand that, but it's not I couldn't understand you know. My first question was: is it is it independent or what the image is um where the covert attention test implies that it's independent? It's a screen space. So maybe that's the assumption here. They're, just mapping screen space xy locations.

B

Yeah, that is, that is the assumption that this is just the metric for the space and there's nothing to do, in particular with um the particulars of a image, and it has nothing to do with the.

D

B

Of any specific trial in the cover potential.

C

That's not a way. I think that bristles would be very useful when I look out at the world and my eyes are moving around. I wouldn't expect grid cells to map out the xy coordinates of my visual scene. I mean the point of the they. As we know, if you watch a video game, they map out the location in the in the video game, not the screen space.

B

Right- and that is a great question, but of course, in this case it's a 2d screen at a constant distance from the monkeys.

C

Yeah, but that's true in a video game, it's uh you know if you want so so in the in the video game example with humans, they they they've shown this grin cell, like responses representing where the ant, where the person is mentally moving through the space rep, you know not the screen space, but the space of the game.

F

Yeah and they've shown that in monkeys too, the exact.

C

Same thing, so that's that's what I'm expecting and now we're saying. Oh no, it's mapping out the xy screen space.

B

To be clear there, she said that there's like upcoming works with with monkeys playing like vr games, so they have like this thing with collecting food rewards. You know, there's bananas in a 3d environment and the monkeys get to walk around.

C

Yeah, okay, so when I that's that my point is when I saw the first image is I'm assuming that the grid cells would be representing the space of the image? When you see the beach and you look at the the horse is further away, then you look at something close away, just like a video game yeah, but that's not what's being claimed here. It's been claimed here, you're saying it's just an xy screen space. So it's just confusing to me.

B

Yeah, you mean you mean in the free viewing task, yes yeah. Apparently that is what they are saying.

C

Because that's a conflict with the idea of watching the video again.

B

C

Maybe there's also a certain amount.

B

Of like abstracting out that you can do when you have, you know, lots and lots of images which have you know clearly also different viewing angles, as you can see. So maybe in the average, what you will get is a is a screen um as a 2d.

C

Yeah again, but that would be that's why my other question was that would be inconsistent with the idea that grid cells remap to learned environments now maybe marcus's point was that maybe these aren't learned. um But if these were learned environments then- and I averaged out over multiple different environments- I would not expect to see any bridge cell like.

B

uh I mean it seems that you have questions that would actually like require the you know like a proper review of the paper. We can do that if you want.

C

Well, I just don't want to get lost in the paper in your review here, because if I don't understand what the hell is being presented, I'm just getting a bit confused.

E

So either I mean in the case of the um in the case of the images, it could be something like what you said that just based on the statistics of the kind of images that they showed, that there tended to be areas of interest there that just showed up in the average. um The other thing is uh versus.

E

You know when, when there's just like an empty screen and you're tracking a little dot or something you could hypothetically visualize it as a different kind of space than the a sort of dynamic space that could be in the in pictures or videos or.

C

I think you're right harry so in this in this image here with the covert attention. Okay, that makes sense to me because it is a different space and it's it's it's uh it's one. You can't even say it's learned because there's no features in it, um so maybe this is the generic space of screens and and so okay great. I get that so that I understand it's the other one which is confusing to me when you're, showing horses.

E

Yeah I mean I, I don't know to what extent the monkeys can like infer the dynamism of a scene with humans and buildings and objects in it. When they've just lived in like a lab.

C

Good question, although I would assume monkeys, are pretty good at that stuff. I mean they were definitely attending to the the horses and the horse's heads, and I would assume that I don't know, but I would assume that.

E

Yeah definitely yeah, they seem to be good at it, but you know I there could be some weird confounding factors.

B

Right um so the the last point.

B

I just wanted to to to point out that in in the uh task where there are the cards right there, you can apparently find cells that are uh tuned towards the direction either of the upcoming task or to the just happened, um um sakat, so presa card and poster card tuning, as you can see here in gray and in light blue and some of those tend to be very sharp as you can see, and they cover all possible angles, as you can see from this, uh this compass here on the right.

B

So that would be something similar to you know: head direction, cells except their movement, not in 2d navigation space, but the movement on a screen.

E

uh Is this new in to rhino cortex, because, if I'm not mistaken, these kind of cells have been characterized in parietal areas like lip and vip.

B

I, I honestly don't know okay, I mean this is obviously android cortex. All of this yeah yeah right, but.

C

Yeah, it's it's new to me too, so I don't know if it's new, but it strikes me as like. Oh it's kind of a little bit surprising.

B

All right, I would like to move on uh to something that um left me quite uh intrigued, which was uh the work of cliff cantros on the interaction between play cells and grid cells, and something that he calls artificial remapping.

B

um So this all starts out with um so the the the the talk was distinct functional rules for distinct entering inputs into hippocampus, but it really like explored a lot of ideas about grid cell placer interactions.

B

In particular, how does the firing of grid cells influence, placeholders and, of course, the the idea that play cells come from grid cells like in the way that theoretical models of lafitte, for example, might would have been a round for a long time? It's part of the canonical view, but there's some problems with it like, for example, in novel environments. It's the place cells that tend to stabilize first before the grid cells, stabilize uh also with interventions that you can do to it. um You're not like the the code is not really um like.

B

The different modules are not independent, so you don't get sort of like this free compositionality.

B

You can do all kinds of interventions to mess with grid cells and that does not always produce changes in the place cells like so. For example, if you, um if you block input from the medial septum like you're, gonna, obliterate, theta and the hexagonality of the grid, is gonna go away, but most of the place cells still stay place cells and they mostly stay in the same place too.

B

uh The same goes for different kinds of pharmacological or optogenetic interventions that people have tried, lesion studies and whatnot, and so it's not always like so clear whether like what role do the grid cell inputs play, and so one way of like sort of doing this more directly, is um to like try to design an intervention that is very specific to meteor, enter rhino cortex layer, 2 stellate cells, which is of course a sub-population of a sub-population of a sub-population of the brain, and so you would need something very specific, and so, in order to get that kind of specificity um they came up with um with there was like proceeding work.

B

I think like a development in 2015 or 16, or something that came up with what they call dread, so uh designer receptors exclusively activated by designer drugs, so genetic toolkit, which allows you to insert specific receptors into very specific cell types in specific areas that will not activate anything else, because they only activate on like this specific designer drug which doesn't do anything in control animals because they don't have receptors for that. So it's a designer receptor for designer drug.

B

So that gives you a lot of control, and so apparently they came up with two different uh versions that would either excite or inhibit stellar cells in middle anterior and the cortex layer. Two. um So you can see this here very nicely how how these um map out exclusively in media and rhino cortex in this standing here and that they're pretty much exclusively layer. Two. They don't really target layer three um and that they even leave out like these these islands, these uh carbondine uh positive islands.

B

um uh So there's like these are like the the reelin ones right, which are like the the majority of it and it's the stellate cells, which are the major like the vast majority of the of the grid cells, and so you now have a very specific target and with a designer drug you can now temporarily make these cells more excited or more inhibited, so depolarizing hyper polarization, and with that amount of control, now you can take a look okay.

B

What how are the place cells going to respond to this and what they found is that they get um they get very interesting remapping, so they they take these mice out of their their environments, apply this transient, depolarization or hyperpolarization, which now makes the middle and toronto cortex layer tools, delayed cells, transiently, more excitable or less excitable, and when you are hyperpolarized, nothing happens.

B

The place cells that you record stay the play cells, the firing obviously of the grid cells goes down, but when, instead, you depolarize, you get a complete remapping of the place cells, so you can see some of this here. So this is what he calls artificial remapping of ca1 placeholds. Following depolarization, so cno is this drug that they that they apply to to do this uh remapping and, as you can see, the the firing fields, change they're, really like not the same anymore in very rare cases right they do stay in the same spot.

B

um So um and when you look at what's happening in the middle and trying to cortex, where that you know drug is applied where that drug acts, you can see these different effects, so they have. This hm3 type. Is the type that depolarizes upon this cno drug. So you see that the firing rate goes up. The firing fields stay in the same place right from here to here right, there's, some like uh in the different types. You know the the one that hyperpolarizes and the one that depolarize uh there's different reactions.

B

So you see, for example, in the hm4 type, the cno drug. It preserves the firing fields, but it decreases the firing rate as opposed to the hm3 type, which again, like also preserves the fields but increases the firing of main in the vast majority of the of the cells.

B

And so what you now have is essentially is a regulator on on the grid cell activity that allows you to do a quantitative change on the grid cells, but what you get out of that is a qualitative change in the ca1 firing fields.

B

This of course, flies a little bit in the face of the of the canonical model, which puts a lot of importance onto this idea of um that, the that the places are composed of the fields of the grid cells. So it seems, like the absolute rate and the rate relationships between griff and grid cells are actually way more important for the coding.

B

This artificial remapping that they see typically holds as long as the mice then like stay in that environment, and you can transiently take them out and put them back, they're still going to remain in that new map, so the place cells like changing in that in that place, unless you take them out for quite a long time. So it's not just the the the odometry so to speak right, the tracking of head direction and whatnot. That, like is, is part of that. So there's some actual.

B

Like change of the map and that takes a while to stabilize this- is like something like 15 minutes and returns back to baseline when you remove them from the environment for a longer time, um which is which is quite interesting, and that is like that is not just reversible but also repeatable. So the kinds of cells that would remap uh also remap a second time when you do that, and they that also you know, returns to baseline.

B

A second time um which, which I found uh quite you know, puzzling, and I think, there's something to be learned here.

B

The the fact that it tends to be the same cells that react that remap and that they remap not just to any space but to like a rather particular space, uh could be seen as a sign that there's a hidden, attractor landscape of sorts, um so that so it's kind of um predetermined by the recurrent connectivity. I guess um where these cells uh can go, which is in some sense.

B

Also vindication of the of the like ela feeds perspective that the placers are, of course, not free to go where they want to go, um but on the other hand, it puts a lot more weight. Emphasis now on the idea that the actual rates, rather than just the firing fields of grid cells, are quite important.

G

Florin, uh how did they come to the conclusion that the uh natural remapping was instantaneous.

B

Oh, that is well known uh like when you have maps in two different environments, they're familiar with both put them from one into the next there's immediately. The new map is fully there. It needs no time to stabilize it's immediately fully active.

B

So all the hexagons you know, hexagonal patterns are full and the and the placers are all in the place where they are, and they don't change, as opposed to this, where it like takes 15 minutes for this new map to to to form. So that's a difference between like the the natural remapping and the and this artificial remapping.

B

Any any any questions.

B

So um right, okay, so I guess no question um so for I I found this uh very, very, very puzzling, because it kind of needs uh different models to to explain this. uh The big takeaway for me was um like we. We need to pay attention to the information content that is in the differential or grid zone firing rates, and so firing fields are not binary on off.

D

So I guess I I'll point out that, like this, this does align with other things. We've seen the one classic thing we bring up. All the time is the is the david tank paper with the missed fields, uh that's kind of similar to like rates being different in different uh firing fields, um there's also something from kate, jeffrey dori.

D

Der dickman uh involving the paper, is called something like grid cells, encode local position info and that's just showing that consistently the firing rate and different firing fields of a good cell is different, such that you can actually get more information about where the animal where the ra is uh than just yeah, then just which cells firing. You can also use the right uh so so there this does align with other things. We've seen right.

B

I'm like I'm, I'm a little surprised that, like you guys, don't react more strongly to this. I found this like to be. You know quite interesting and puzzling.

C

uh I have to admit during uh foreign my mother called me, and I spoke to her for a few minutes, so I lost something.

B

C

So it's my apologies, I'm not taking your call or seeing what's going on so.

B

Yeah, it's all good um so so to just be be clear right, so the cells that remap are uh most often cells with the second firing field and and um and so which again like supports this idea that there's some hidden attractor landscape, which is formed by recurrent connectivity.

C

Do me a favor: um this is a complex paper and a lot of biology and experimental technique involved. um If you think it's really interesting, I'd like to read it, um so maybe I can just get it right off the screen here um is this: is this a publish paper? Is that the name of the paper.

B

uh Yes, uh let me quickly find it if.

C

You could just put it on slack or send it to me. Then.

B

Right, it's this one.

C

Required a more thorough understanding right if I could, if I could easily capture that screen, I would but I don't know how to do that.

B

C

Can you just, can you just stand in slack or something.

B

Oh yeah sure yeah. I can send that out all right.

B

All right cool um next, what we might want to do. I mean like in this context. It may be interesting to take a look at uh new map permission. I don't know how we're doing on time.

B

um You guys can tell me.

B

So um there's also a presentation by uh by julia krupitch, who is talking about the the the real-time formation of maps because of course, like most of the time, we're talking about like these stable maps all the time, but whenever we put a rodent into a place like these maps are uh forming, and can we track that formation in real time? If we really think of that whole hippocampal, android or cortex system of doing like simultaneous localization and mapping with the robot assist, call it right, then we should.

B

You know, want to follow that process online, and so one way to do that is to track map formation in real time and to like you need a lot of density for that. So these neural pixel probes are great for that and then you can like switch animals between uh environments.

B

uh So you give them like uh segments of of of learning time. So when they go into environment for like three hours and then you can break that apart into like 20 minute segments um to to track sort of. How does the map change over over time?

B

And these are kind of like the the notes I took down from the talk, which is that as a yes.

B

Cool thanks, so I guess should be enough to go through this, at least um so. What I mentioned before right, as um as uh cliff centros mentioned, as one of the things that is sort of a little bit critical of this idea, is that somehow the playsets come from with cells is that the play cells are typically present much earlier, but stabilize rather slowly, so they still increase their firing rate and they tend to move around a little bit until they sort of are always reliable in the same fixed spot. Right, keep in mind.

B

This is so over the course of like three hours right, I don't know the exact. I don't remember the the you know exact time course of this, but the the point was to say you you get to the stable position and stable rate uh quite slowly, but they're there earliest and then um so with with the grid cells. That tends to be a bit different in that there tends to be like one of the fields you know of like. Let's say that grid cell has like four or five firing fields in this arena.

B

um It tends to be one that is first and then it tends to fill on like the hexagon around it over time. So there's like one of the other that kind of like pops up at the at the right distance. So there's a there's, a filling in of sorts, um so um so that the change of that of those fields is also like slowly, um let's see, um yeah, and so the grid cells tend to change, monotonically, meaning you.

B

You have um like one field and then you fill in the other ones, whereas the place cells sometimes jump before they settle into their final point and then build out the right firing rate.

C

What kind of environment was this? You know.

B

These are my enclosures for the rodents.

C

I don't know if they're circular rectangular.

B

Boxes like they.

C

Need to so there might not be anything interesting in them. They're just boxes.

B

Yeah, I think I mean like obviously in order to get nice special coverage over the course of you know this time and to be able to follow the map in real time. They need to make the rodents move, so I'm pretty sure they have an automatic food drop.

B

I don't remember that, but um I'm I'm sure she said something like that.

C

I mean the reason I ask you is that you know I have a model in my head that we've discussed a lot about how good cells and place cells are interacting and, um and I'm always testing these papers against that model to see if that model is wrong, um and so some of these things, I'm I was just imagining here, like imagine yourself- being dropped into some big sort of box- that there's no real landmarks to to judge where you are.

C

Your grid cells are going to be very inaccurate, initially you're going to be relying almost completely on uh path, integration, and- um and so you just can't expect.

B

C

Right, there's no landmarks to really you know. It's a you know. Grid says they're not going to work they're, just not going to do their job in a situation where you can't concrete landmarks to know where you are and then place fields, I always think of them, as is primarily being sensory, driven and so they're going to be there from the start. um There's going to be some century landmarks, something otherwise you wouldn't have any place fields.

C

I put you in a white field of no nothing. You wouldn't have any places. um So so, as I'm just I'm just pointing out that when I, when I look at these kind of points that you're making here, I'm just testing our model of what we think is going on and and if I don't find a contrary evidence, then I say: okay, it's fine, um but these seem consistent with what my model in my head about how place cells and grid cells interact and how they form how an animal learns an environment.

G

Lauren did uh was there any hypothesis as to what the mechanism was for the place cells jumping suddenly.

B

uh Yeah, well, I mean like I could answer this with a pointing to the talk by cliff cantos right. The fact that the when the when the grid cells, like start filling in and like they typically have an anchor somewhere right, um there will be changes in the rate relationships right between the grid cells, and it is as clear control showed, is exactly these changes. These differences in the rate relationships of the grid cells.

B

Even if there's no big shift in space right, it's only a quantitative change that leads to a qualitative change in the in the place cell map right. That was. That was exactly the the point that cliff centros was was making that from quantitative changes in grid cell firing. You get qualitative tension in ci1.

B

That is obviously only like a correlation that doesn't answer your question as opposed to mechanistically what happens there, but I guess the the the answer is that you would hypothesize that there is some hidden attractor landscape um and somehow, by having different stimulating rates, you fall into different attractors, which is why the placer then flips to a different possible field and either stabilizes this one or the other one. But it's not going to be free to choose any point in space.

G

Yeah, I was, I was kind of uh the combination of those things I was in in in my universe. I would call it mode locking or a number of other things where all of a sudden there's a phase relationship that reinforces so uh that was why I was just wondering if she had speculated whether this is part of the attractor model or part of the you know some of the other models we've looked at.

B

Like people have been, uh you know thinking about how this possibly what would happen. I've been piecing this together, but I I think this was the very first, like you know, conclusive demonstration with like these high density probes of following this whole map formation process, while it was going on so even though none of this, like really contradicts our our current understanding, I think it was very important to um to to demonstrate that um rather clearly, and it seems like that they have done that.

C

You can imagine easily how uh just again imagine yourself in these environments and how uh it would be easily difficult to know where you are and the more time you spent in sort of a pretty much uh of an environment, a door that looks the same everywhere and it's devoid of almost any interesting things in it. That you would your perception of where you would uh initially would be confused a lot and and over time, as you spend more and more time in the environment.

C

You'd pick up on subtle clues and you would get better and better at knowing where you are just it's just sort of naturally occurred um so to me that that you know given such these are such impoverished environments.

C

To begin with that the place field, you know the animal, just the fact that the place still shifts means all of a sudden that the animal is getting a better idea where it is onto these sensory inputs, and it is it's just saying: yeah, I'm learning the space, and I know where I am better now before I was just confused all the time.

C

I don't find that hard to imagine.

B

B

Although it's interesting also that, like the the grids, um really become like you know, canonical after after a longer time right, so they say that you know, like you, get sort of the full convergence of like the stability of the rates at the different fields that are mapped out and the full hexagonality of it. You you get after a week, which kind of suggests that there might be some offline processes that also play a role here like whether I'm I'm not sure about that.

B

But I would hypothesize right that there's like offline processing that also helps with like making these grids very nice like what you would get doing. You know from from from replay, if, indeed the grid cells are. You know tagging along during that replay, then that also is another opportunity to refine the the the the grid.

C

Replay may make an important part of it, but I mean it's like any task. You learn right if you're learning how to play a new piece on the piano or something it takes days, uh you get continually better at it, so that, as a general function of learning in the brain, that's just not unexpected, um but you know it's an interesting question. Whether replay is an important part of it. Probably is.

B

um Right and what we might get like from from the fact that the the grid surfing tends to like have sort of an early anchor from which it starts filling out the neighboring fields, um and that seems to be there, irrespective of the fact that you don't have me like an environmental anchor yet because the environmental anchors right, your your place, cells they're, like jumping around, so you kind of that that goes well with the with the attractor idea that somehow there is already some some fixed point right, a bump, um and that you can.

B

Then you know, use that because you have surrounding inhibition, and that opens up the possible uh fields um outside of that yeah anyways. I think we should. You know, probably concluded that this um unless there's you know like more specific questions,.

C

No, I just want to. I want to read that paper that you uh you mentioned.

B

Right, I will send that out. I appreciate that.

B

C

C

B

C

We'll be back in uh 45 minutes or something like that.

B

C

A

Okay, um thanks for watching uh my name is matt taylor. I work at dementa and um during. uh I just wanted to note that we're going to continue trying to live stream, our research meetings, while everybody's sort of down working from home sheltering in place whatever in.

F

California, where.

D

A

At it's a big deal, there's a whole. Everything is shutting down and we're all working from home. So that's why you see the the zoom meetings instead of seeing this meeting being in the office, um but we're I'm still. I have the capabilities to continue to live stream. All this stuff, I've already set it up.

A

I've been doing it for almost a year now, so we're going to continue to share and have these research meetings and and probably more stuff over the next month since everybody's going to be working from home, and I could do it and let me maybe we'll have a couple virtual talks. I know we're going to have a big htm hackers hangout april 8th. If anybody wants to join that go to our forum, I'll put in it's a discourse.numenta.org.

A

I just threw that into chat. I hope I spelled it right um or just search for htm forum um and you can find out there about the htm hackers hangout, I'm gonna be have a big big zoom meeting, get anybody as many people as possible. Apparently we can fit a hundred people in a zoom meeting under our plan, so we're going to do that april. 8Th 4 p.m, pacific daylight savings time.

A

So please join.

C

A

If you want we're going to I'm going to, I'm not sure exactly what we're going to go over, but we're going to do something, I'd like to sort of give a report on community code projects, and so it's a research task of mine that I'm working on.

A

um So I am looking forward to seeing all of you guys there all your familiar faces and chat over there. I appreciate you guys watching. Please spread the news that we're still streaming and share these videos with your friends like this video. Please give it a thumbs up. It's very helpful and subscribe to our channel, as you saw somebody earlier subscribed and that brain pops up so that's fun um so signing off. uh I will see you guys with another stream on monday.

A

I don't think we'll have one friday, but usually there's one every monday and wednesday around 10 15, pacific time, adios.