Numenta Numenta Research Meetings, 21 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 'Eigen-view' on Grid Cells - September 21, 2020

Description

In this week’s research meeting, Marcus Lewis presents his ‘Eigen-view’ on grid cells and connects ideas in 3 underlying papers to Numenta’s research. He discusses the mapping of grid cells in terms of eigenvectors, and evaluates eigenvectors in terms of the Fourier transform (space) and the non-Fourier transform, called “spectral graph theory” (2D graph).

Marcus’s whiteboard presentation: https://miro.com/app/board/o9J_klMK6P0=/

Papers referenced:
► "Prediction with directed transitions: complex eigen structure, grid cells and phase coding" by Changmin Yu, Timothy E.J. Behrens, Neil Burgess - https://arxiv.org/abs/2006.03355
► “The hippocampus as predictive map” by Kimberly L. Stachenfeld, Matthew M. Botvinick, Samuel J. Gershman - https://www.biorxiv.org/content/10.1101/097170v4
► “Grid Cells Encode Local Positional Information” by Revekka Ismakov, Omri Barak, Kate Jeffery, Dori Derdikman - https://www.cell.com/current-biology/fulltext/S0960-9822(17)30771-6
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a cohesive theory, core software technology, and numerous software applications all based on principles of the neocortex. Our innovative work delivers breakthrough capabilities and demonstrates that a computing approach based on biological learning principles can do things that today’s programmed computers cannot do.

Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest

Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter

Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta

Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/

Our Website:
https://numenta.com/

A

Okay uh yeah: this is going um today, I'm presenting a thing.

A

That's been lingering in the back of my head for a couple years now uh that I've been just trying to make sense of um and it gets into other people's perspectives on grid cells and how sometimes they use um analogies to eigenvectors or to the fourier transform uh to talk about grid cells, uh and in this presentation, I'm going to lay out um a mapping to that to that terminology where most of the things we talk about with good cells can be described in that terminology um and I'll try to make this very um like picturesque.

A

It's not it's, not a bunch of symbols. It's I'm trying to kind of bring it to life and and take what we've talked about map it into this um and it'll serve a couple purposes. One is that it. You know it gives us a way to look at what we've done from another angle.

A

It can also it also provides suggestions of of where we can go um and I'll also consider the alternate take on grid cells, where, when sometimes sometimes people talk about grid cells as being principal components of place, cells or principal components of location, and I want to um show how that kind of fits into this and say what I like and dislike about that idea.

A

So I'll go ahead and and share this just a second uh and go into presentation mode. um Okay. So in this I I this is like a big whiteboard, essentially and I'll zoom out to show you kind of oops. That's in I'm, going to zoom out not like that, still figuring out how to use this.

A

There we go, and so so the way this is kind of structured is uh going straight down.

A

Kind of the straight line going down through is um is taking what we've already talked about and putting it in this language and going off to the side as as as alternate directions that you can go so so kind of the main, through line of this presentation, is down through the center, but I'll take a couple off shoots that where there are these interesting ideas that can be discussed uh so in the in the first part, I'm going to just lay out what I mean by an eigenvector and what and how that fits into this.

A

uh The context of grid cells, then I'll I'll jump into uh the question of, should we think of end to randall cortex in the hippocampal formation as processing graphs, or should we think of it as processing space and you'll, see what I mean when I say that um I ultimately come back to thinking about it as space.

A

But there's some interesting things to talk about over here, uh and so, if finally I'll say that, like now that we see that all this resembles the fourier transform or eigenvectors so much what new ideas can we draw from that? uh So this will be where I'm not just putting new language on it anymore. I'm I'm showing new cool ideas, okay, so I'll I'll start at the beginning and I'll hide this just to make it less crowded.

A

A

Here I want to tell you a just to give you a simple definition of eigenvector and what you the mental image you can have when we talk about eigenvectors, so imagine you're, representing the location of an animal or any 1d variable on a like a location of an animal on a 1d track using a vector yeah using a vector of like states, basically saying that it's a one hot vector this rat is here because uh you're encoding that using a one there and now say you, take these states and treat it as a markov chain.

A

uh And here I've put these probabilities on here of, like you know, 25 percent walk, left, 25, walk, right, 50, stay still and um and- and I've just said, say you use this uniform random, walk policy for this. For this animal, uh you can put these probabilities in a transition matrix uh that this transition matrix.

A

That then takes in this vector, and if you apply this matrix to the vector like using a standard, matrix, vector multiplication in this case, what you would get is you would kind of diffuse the one outward it's like uh that one analogy people have used. Is this like a droplet of ink on paper and and you watch the ink slowly fade to the side. That's kind of what is happening here. uh This random walk policy will kind of diffuse the bump outward.

B

Is that is that.

C

Suggesting that there's uncertainty to where the animal is or just representing the probabilities of where it will be.

A

uh It's it's both it's um okay, probabilities of where it will be. uh You could interpret this as if you look away and you're trying to if you're tracking someone else's location, if you're tracking a rat's location- and you look away- this is in your head. You might have this going on, where you're keeping track of where the wrap might be or if the rat doesn't have any sensory motor system. It just has you know time-based system. This might be how it keeps track of where it is well.

C

Like I was just trying to say, if this is, this probability is where it is or where it will be, it seems like it's obviously where it will be, but it seems like if you could observe the animal you would know exactly where it is so.

A

Right right, so so that is what that is. One kind of strange thing here is that um if you were actually simulating a random walk, you wouldn't diffuse this, you would choose randomly. You would choose one direction and go uh so it's more of a probability distribution of where it's going to be in the future.

A

uh So so this this is a like. It started out as like a one hot vector, encoding, a location, but for some vectors, if you, if you just freely, can paint these any color, you want or set them to any number. You want certain vectors.

A

When you apply the random walk policy, uh they they just they kind of fade. They get scaled down uniformly. They don't they don't change their shape. They just become a less like a less extreme version of their shape, and these vectors are eigenvectors the ones that, in other terms, you could say the high dimensional direction. The vector points doesn't change only it's only its magnitude changes, but you can always just picture it as like paint fading as colors fading by by the same scaler and the reason uh eigenvectors get talked about a lot.

C

I'm sorry you I'm sorry marcus. You lost me already and I'm afraid I'm afraid I, if you lose me, I'm gonna lose the rest of it. So um I mean what my knowledge of eigenvectors goes back many years ago and that doesn't seem to really too much what I'm seeing here. So so, mostly in my ignorance, um but I don't, I don't understand the idea of this fading. um You say successively applying a.

C

um Can you just try to give me the verbal equivalent what's going on here again, okay, yeah, uh and why there's two eigenvectors? What are they you know um and- and here and also in the coloring scheme here you showed it repeating- is that is that the repeating equivalent to the repeating grid cell module is that what that is.

A

That's going to come into this soon.

C

I mean because why are there two red spots uh or three red spots and two blue spots? Is that what that represents.

A

Because the way this is analytically handled, it's it's it's a wrap around track.

C

Okay, so that is the then that is the repetition of the grid cell. Is that right? Okay, because in your in your vector above you didn't, show enough points to show that um right, you just showed uh one.

D

C

Being active so now we're now we're showing you a longer track. So no so.

A

So so this was representing. This is a one-hot vector for where the rat is, and then I made a point that this is not an eigenvector, because it's not simply being scaled, it's being kind of diffused, it's it's, it's it's shape is changing.

A

These are not one hot vectors, but they they. I could have plugged numbers into here to get these or to get discrete versions of these. I guess.

C

I'm losing it here.

E

um Yeah, I'm also not sure um so when you say successively applying a is you take that single position up top and you apply a once and you get that the the yellow and orange vectors that we're seeing and if you keep applying a multiple times to that same position.

E

No, that you don't get these right. So what? What is? How do you get from? I'm I'm a little confused where how you get from ax to those eigenvectors. I guess.

A

Yeah, so there is a special set of vectors that you can solve, for uh that are not this one, that that are not this one, that that have this property that they're in a sense they're.

A

This will always be a shade of red. This will always be a shade of blue. This will always be white.

E

But what matrix are you taking the eigenvectors off? That's just.

A

One figure this this matrix- that is a diagonal of 0.5 on the diagonal and 0.25 just on one off of the diagonal and.

E

Everything else is zeros, yes,.

C

Okay, I'm still I'm still stuck, and I don't want to if.

E

It's not successively, so these eigenvector ones and twos are eigenvectors of a it's, not eigenvectors after successively applying a. I guess. That's.

A

Trust, though correct no, this is an eigenvector of a and no and this isn't just being scaled up and down.

C

B

C

These arguments, I can no longer uh interpret them as positions on the track. Is that right? These are.

A

um Right or like you could almost think of them as like a uh as a union of positions on the track, uh or you could think of them, as, um as you know, in visual cortex on and off subregions, uh you see color maps of where the cell responds and where the cell doesn't respond.

A

This is becoming more similar to that. This is more of where a you could imagine a unit that is responding to these to these locations, but not to these locations. Those are two different ways of looking at this. You can. You can say that this is either a representation of three locations, or this is the the rate map for a cell.

C

And that individual cortex we were referring to, that was a non-moving 2d type of thing. um But how does that apply now to the again I'm trying to? Is there any mapping between these color-coded bars and the position on of the rat on the on the track? Would they say that this particular cell might respond to multiple places on the track, but it's not actually a grid cell you're saying it's a it's. It's.

A

I'm bill I'm building up to where this is a visualization of of a ring, but I haven't gotten there yet.

C

Okay, because that looks like a ring so um all right and then and the idea of fading. What's the event, what's the point of that what's, uh why are we doing that? What's where's, that going? Why are you changing the scalars.

A

Well, okay, the definition of an eigenvector is, um is so so look at it in this view, with the numbers. This is a what a an eight-dimensional vector pointing in eight-dimensional space, um and I could have drawn these that same way, uh eight-dimensional vector point and eight-dimensional space, and when you apply a when you apply this random walk, that vector continues pointing in the same direction, but it just gets shorter.

A

uh This is the definition of an eigenvector, okay and um and.

C

The reason gone but you're showing four different shades here. I, those are all eigenvectors right.

A

Yeah that this is this is an eigenvector that um that is being scaled up and down. But it's it's all the same. Eigenvector.

C

I guess I guess I'm trying to the scaling thing: why is it important? Why are we doing it? Where is that going to go? Is that that's.

B

C

That's not part of what eigenvectors are is it? Is this it's just something we've decided to do in this case. Is I'm just trying to understand.

A

No, that's what eigenvectors are the the the fact that the fact that, when I apply this matrix to this vector uh it becomes a lower magnitude version of itself, it's gets scaled down. Every point on this track is scaled.

C

But why are we doing it successively? What's the logic behind that.

A

Well, you you don't just doing it once is: okay,.

C

It sounds like we're trying to you show a progression of these as if it's something we want to do, I'm just.

A

No I'm just I'm stating this is what makes it an eigenvector that no no matter how many times did someone say.

D

Just a quick question which might.

A

D

Relevant to uh so because here the the vector is kind of representing transition probabilities, um and so, when you're scaling, um can you help me kind of visualize what's happening to the transition probabilities? Around kind of? Where is the idea that it's kind of becoming more uniform well.

A

Okay, so so the vector the vector which tells you the probabilities of where you might be uh yeah is yes, it's becoming more uniform. It's it's it's slowly and as time approaches infinity, it's going to become totally uniform.

D

It's going to become yeah, okay, now that I think that helps to yeah.

F

Could you explain to me when you say successfully applying a I'm thinking, you're, taking a to some power and applying it to x? Is that what's going on? Those are the same thing? Yes, okay! So in effect, if you are, uh if you, if you take a squared, what does that look like.

E

That would be the second row. I guess.

F

No, I mean what does.

A

It look like no. No, no, a a squared would be this except spread out further. Okay,.

F

So you're, essentially you're actually heading toward a gaussian because you're taking a block thing and multiple multiplying it multiple times. So it's it's assuming a gaussian characteristic right in terms.

A

Of I guess, that's not the direction I'm going with this. uh If, if.

A

You so the whole point of this part is to define what an eigenvector is in the context of location.

F

A

F

Understand but you could you could.

A

There's not going to be a point in this talk again, where I apply a successfully successively.

F

Okay, but what I'm saying, if you're, if you're going to have that scaling factor that that fading basically would be absorbed in the eigenvalue right? Yes, so these things remain normalized it's just. The eigenvalue is somehow decreasing.

A

And since the term has come in uh the vector, the scaling factor that keeps get keeps getting applied successfully is its eigenvalue okay, um and so I'm just since you brought in the term, I decided to introduce it, uh and- um and in this case these waves are cosine waves, uh because a is translation, invariant and it's a random walk and it's symmetric, uh and so that's why that's why cosigns have come out of this?

A

uh The the general mental image I want, though, is like is it to to convey, is um how can you paint the track? How can can it yeah? How can you paint the color colors on this track uh so that a random walk um leads to it? Fading like this and and not the color is not like not shifting over at all or becoming different colors. uh But how do you make it where it, where?

A

How do you paint the track so that uh so that it fades and that, if you, if you, if you find one of those, that's an eigenvector.

F

Okay, so let me let me try to motivate one just one more uh fraction. So when you talk about successfully applying a uh could that correspond to you, you made the mention of you. You, your head, turning away and you're mentally making a model of where the thing is. Is that if you, if you continually to keep your head turned away and you allow time to elapse, things diffuse more, is that kind of what you're trying to convey.

A

Yeah, that's what that's what I meant by what how you can interpret this? What why you would ever have this fade out like this okay, it's your uncertainty about a thing in the world that you're not currently observing.

F

And and basically there's a there's a this is a model of how you might extend.

F

You know the fact that the rat could be you know in one second could be somewhere in a circular radius of you know, you know six inches and if you basically have an intuitive sense, if you wait long enough, it could be at further and further radius. So if I'm trying to recapture the image you know the rat, I I'd have to look at a wider area. Is that kind of where we're going.

A

F

A

Oh, that's! That's what that's what this part? That's what this definition does, but that's not the main topic here. I'm not going to be talking about simulating random walks for multiple time steps. I'm I'm just introducing what an eigenvector is in this context. Does.

F

That give you more of a model jeff.

C

uh Slowly, I have one more question: why are you showing two eigenvectors of different frequencies? uh Where did that come from.

A

I was showing two examples of of eigenvectors and I translate.

C

That into like two grid cell modules at different uh spatial uh scale, scales.

E

Is that yeah? I might interpret that.

C

E

All right and would, if you were to show all the eigenvectors would they be sort of offset? There could be eigenvectors that are like eigenvector one, but just where the blue and red are flipped or.

A

uh It's actually is close: it's not flipped, it's shifted by 90 degrees. It's like a relationship between a cosine and a sine, but yes, there's a there's, a there's, a cosine and a sine at each frequency.

C

But that's not, I think, maybe where you I thought, where you're coming from subatomize the idea that we in a grid cell, we can activate the same different cells, that you know re-anchor different points, and but that's not what's happening here, because this this 90-degree thing wouldn't be re-anchoring right or would it.

E

I was just saying: that'd be like different grid cell in a given module would be.

F

C

Yeah yeah so, but but that doesn't sound like that here, because if it's restricted to 90 degrees and that's in the grid cell module, you have to find the granularity than that. So.

A

Yes and I'm going to I'm going to make a better mapping than that, the sines and cosines of these are not a good way to describe the phases. I'm going to do something better than that in here.

E

Yeah eigenvectors will also give you like a minimal set. The exact minimal set you need, which might not be exactly the same as what we have in a module.

C

Is that what the next thing says is like they're, good basis vectors, because they're not over complete, is that the idea is that.

A

It's it's not it's not because they're over completeness, uh I don't know. Maybe it's that's not what I was going to say though, but I I I'll jump down to here. um Yes, the reason we talk about eigenvectors is they they act as a sort of natural basis for uh the thing you're representing um and there are. There are two general motivations here and really there are two branches of this presentation.

A

They get the go of these one, the key one, the key benefit of using an eigen basis of using the eigenvectors as your basis is that it allows you to um perform computations locally, where, where you only have to um okay, it's probably the clearest way to say to this group is the same way: grid cell modules break path, integration into multiple local nodules.

A

When you move into when you move into representing something in its eigen basis, a very analogous thing happens and I'm going to depict the below the second motivation. That really is uh quite different, in my mind, at least as um is dimensionality reduction, uh if, like the whole principle component analysis view of representing stuff um and there's there's intersection between these two, but I'm kind of keeping them separate for this presentation. Okay,.

F

One more quick technical question, so yeah you, you showed one eigenvector from one particular system. There's there's multiple ones, because you know the dimensionality of the system, so you chose to show the eigenvector. Presumably it had the largest eigenvalue. In other words, the most prominent one. Is that kind of what I'm intuiting right here.

A

No, I kind of jumped to showing ones that would demo better uh the the the first one is very boring. It's just uniform. uh The second one would have been a single. uh um It would have been a single cosine wave.

B

A

So it would have been a scale of one, so I started with scale of two in scale of three so that I just chose two that would demo well. Okay,.

A

So uh now this one- I did this in 1d to introduce the concept. um So now now we move on to 2d, and the question is: how can you paint 2d space sort of how this is a painting of 1d space? How can you paint 2d space so that so that a random walk will scale each pixel and- and there are a couple, different ways of getting answers to this?

A

Some people have used computational solvers, like in python, there's numpy and you get things like this um and and it's a little bit evocative of good cells and and so that that has that has uh that has spurred a line of thought, um but I'll also point out that if you solve it a different way, if you solve it, analytically, you get cosines and sines pointing in different directions, um and- and these are actually the same as these in some sense- um they're.

A

I've grouped these in a way uh where, uh where normally, if you take two eigenvectors and add them together, you don't get another eigenvector.

A

If you take these two paintings of space uh and average them together uh that the the result is not going to be something that has this magical, diffusing property, but sometimes sometimes it is the case that they do and- and this is one of those cases- um sometimes multiple vectors form in eigenspace. uh These are like these four vectors are like a basis where every single linear combination of them is also an eigenvector, um and these are are linear combinations of these. So I've put this little box over here.

A

Sometimes linear combinations of eigenvectors are also eigenvectors and it happens when their eigenvalue is the same when their scaling factor is the same, and so I just wanted to throw this out there that um that, in some papers, you'll see something like this and in some other papers, you'll see something like this and they're, both two ways of showing the same eigen spaces.

A

Basically, these are four orthogonal vectors in that space. These are four orthogonal vectors in a space and they're kind of rotations of each other, uh and so that's mostly just drawing a connection to um to other work uh to to where other people have have gone. With this.

C

Go ahead and yes, I think it was difficult.

D

Is there any kind of advantage or something in terms of kind of encoding efficiency to the kind of combined uh representation? Then the the combined, uh the eigenvector that captures the entire eigenspace in one vector.

A

My real answer is, I don't know um it, there may be, but I don't know of it. Okay, these may be completely equivalent or there might be a key benefit of this.

E

Yeah so you're saying uh marcus that any rotations of this a b, c and d will give you another set of eigenvectors right.

A

Yes, I I, if you insert scalars into a b c, indeed, you'll get you'll get an eigenvector yeah.

E

So the chances of getting you know, there's all you know, infinite number of rotations of these, so the chances of getting rotations that line up exactly along the axis are really small right. So most rotations are going to look more like the upper ones right most solutions. Yes, and that's probably why computationally you end up with the upper ones and because the chance of solving it computationally and exactly getting access aligned once like you show, the analytical ones are really small. Yeah.

F

So I I have a question about your sidebar when you said this happens with any set of eigenvectors that have the same eigenvalue. That would imply that the set of eigenvalues are degenerate because typically eigenvalues are not identical, but you know you can apply them. You know the the associated eigenvalue against each of the eigenvectors, so I'm just wondering you know.

F

Am I missing something here, um because if if there, if, if basically when you, when you solve the characteristic equation, it sounds like you have multiple identical roots in order to have eigenvalues have the same value.

A

So I mean I'm going to talk about fourier transform in a little bit and there, for example, I mean I'm just kind of jumping ahead or being using technical terminology, but if, uh in a fourier transform you have the positive sign and the negative sign, uh positive rotations and negative rotations and in many cases those have the same eigenvalue of um of how of the positive frequency and the and it's and it's conjugate, it's it's negative. It's corresponding negative frequency. Those.

F

A

Same eigenvalue.

F

Okay, so that's that.

A

And in fact, that's what's going on here, I'll I'll even tell you. That is why uh that is why simple cosine waves are um are eigenvectors, because that's going on here.

F

So you're, basically looking at in in complex space, you get conjugate pairs, so you're gonna, you're. Okay, I get you hopefully that helped.

C

I have a very simple I just something I don't understand about this diagram yeah, so you're saying you can combine the the those linear ones on the bottom and you'll get this other representation at the top. And then you say: okay, a are a b c and d scalars. In this case. What I mean? Yes, okay, so you have to pick those scalars carefully to get those the ones you show up above or I'm you know.

A

Just sometimes yeah some of them, for example, you can see this one right here is almost this one, it's it's it's been so so um you do have to choose them carefully. Yes, uh yeah! It's I'm not gonna. Just say this that this is this time plus this you have to.

A

But but this is a linear combination of these.

C

Four, okay, but it's I guess, I'm a little confused, but I just don't know where these abcd come from. That's just you know: they're picked out of a hat. What do they represent?.

A

It's really this sentence that this is a picture of this sentence. Linear combinations of eigenvectors can also be eigenvectors.

C

Okay and then this was.

A

Me depicting what a linear combination is.

C

Okay, but then you show four of these linear ones down below, and you show four ones up above, but by different linear combinations. I could end up with a whole ton of different ones up above um yeah, so so it's not like these four map into those four. It's like these four can map into an infinite number of ones. Up above is that right.

A

Yes, correct: okay, uh is it just just they're, not four random ones, that I'll just throw in these are orthogonal to each other. It's four orthogonal basis vectors. This is four orthogonal bases.

C

So then, that would that would restrict what my scalars abcd are right. Yes, okay.

A

Or or at least yes, yes, there is a restriction, but technically any combination. Okay, any.

C

Combination work, but if you want them orthogonal, then you then, then you have specific combinations. Okay, it's just some more of a mathematical statement than representing anything in particular.

A

C

A

Yeah so, okay, this was my section where I was introducing the picture of painting a space so that so that it diffuses, so that the paint fades when you do apply a random block. um That was that was the the image I wanted to convey of eigenvectors.

A

So um the the paper that really put all this on on my radar is this work from kim stockenfeld, matt, batman, nick and sam gershman. uh The hippocampus has a predictive map, um and in this they go a different direction than talking about good cells in terms of space. They I the way this is me putting my spin on it. How I would describe there is how I would contrast it.

A

They they treat it as graphs as thinking of 2d space as like a um like a 2d, a 2d graph I'll show a picture of what I mean there, um so, but here so there's kind of a this. This presentation kind of diverges into two parts here. One is thinking of eigenvectors like in a way that could have. I could have just said: non-fourier, uh not not in a way. That's like the fourier transform, but where the fourier transform is kind of just a special case here uh in this field.

A

There's like a there's this field, this area of study called spectral graph theory where they fit good cells into that. um So there are things I want to uh bring up here. So this brings me into.

A

Is it useful to think of the hippocampal formation as a graph processor, rather than a space, processor or all right or and and what are grid cells uh role in that? And all of that um one thing just independent of of this, but to motivate it a little bit. I want to focus on a thing that people don't always talk about with grid cells because they like focusing on the nice parts.

A

Is that something that kind of I don't know plagues the field, especially the experimentalists, is that the is that grid cells, their firing fields, are often uh distorted and not not clean in the way that you would, you would ideally expect them to be.

A

um There are lots of papers showing this, uh so so some intentionally put them in weird rooms and that- and that might seem like kind of an edge case to you, but even like the moser group did this one where all they did was used a much larger box and even within the room, uh the the grid has distortions and just a big, clean box, and- um and you can talk about other uh other variations on this.

A

This is as far as I know that this isn't published yet, but these two groups, uh the 3d grid cells, um it's not published, so I got these very mediocre photos of a screencast. I was at this presentation, but I didn't want to take out my phone. uh So um what you'll see what you would see here if they were good photos, is that grid cells in 3d, both in bats and in rats?

A

Are they have these? You know blobby firing fields, but they're not really cleanly arranged in space like you would want them to be. uh Oh and they're. Also, they also don't seem to be.

A

uh You know the 1d bands that we would be an alternate theory on on how it could work uh so so the the general theme here is there's kind of an elephant in the room uh around grid cells and good cells as a metric space, and that's that, like to record these grid cells, we make rats, perform random foraging in simple environments uh and the simple grid might be an artifact artifact of the experiment set up.

A

It might be a classic case of you get what you measure like you're you're, causing the grid to arise by the experiment paradigm, but there's not a better way. So it's it's a tough problem to wrestle with, uh and so so one one way a theory kind of can get um can get ahead. Is by explaining these distortions, and that's where this alternate view of good cells come in I'll, just jump ahead and say I'm going to say that there's something that this.

A

This theory is not covering that that that I think is not good, namely it doesn't handle the the multiple spatial phases of grid cells, but to spend a couple minutes on it.

C

Could you just say it again just want to make sure I understand it doesn't handle the multiple phases I mean it doesn't show. How individual cells will repeat is that.

A

It doesn't show how uh that there will be multiple cells with the same grid, but shifted translated, also known as grid cell modules.

C

Uh-Huh so I mean that seems like a requirement for path, integration, yeah, so another way to say this doesn't have a path integration component to it. Is it.

A

Not that I can not that I've been able to deduce uh it doesn't have a set of phases for.

C

A given scale that seems like a huge hole that that has to be at least recognized.

A

Yeah, that's the same conclusion I come to here uh down here. uh So yes, yes, that is, that is a huge hole there. It's possible that they that uh that some people who work on this have found a way to bring phases into it. But I haven't been able to find it myself by by trying.

E

To understand yeah, it must be when they relate sort of principle, components, analysis or into neural coding. I think uh bruno also did this versus they.

E

It's cleared out that you have to have these over complete bases uh to to really represent neural responses, and I I wonder if, if you have over complete bases and sparse coding, whether now you'd suddenly get all of those, uh you know all of the shifts that we want in a module.

C

B

That's not, I mean the.

C

Words made sense: I don't understand why making over a complete basis set, would would all sudden introduce path. Integration.

E

Well, not path integration, but give you all of the small shifts that you need in a module. The the problem with just pure eigenvectors and principal components, analysis is that it creates a really minimal, minimal encoding of things. So you don't really, if you don't have to have all of those shifts at the same frequency to to reconstruct the output, it's not going to create them.

C

But this seems to me the critical component is the path integration component and you you could have pass integration without having all the shifts the small shifts it seems somebody could.

A

I think it seems to me you're, coming at this with a theory and other people are coming at it with uh experimental data uh and those are both useful ways to come at this. But I, but if you talk about the fact like it's been observed, that grid cells come in multiple phases, uh I want to explain that um this doesn't do that uh and yeah okay.

C

B

A

Super tight may have actually just provided a solution. I I can't speak to that right now. Remember.

C

The uh recently I've been arguing that release exploring the idea that um you have these uh many 1d modules, but the 1d model may not be linear in in space right. They could represent any kind of movement vector, and I'm wondering if that that to me, would suggest that when you look at the combinations of those uh those individual 1d vectors that are not necessarily linear um that they could be kirk's curving and things like that. If you look at the combination, those you might, you would end up with a sort of a odd shaped.

C

You know 2d grid assemblies, um I'm just wondering if I could explain this same sort of issue, but anyway I'll. Let you I'll, let you keep going. This does not explain the the multiple phases and the path integration.

A

Right, I I want to spend a minute saying how this did kind of expand, my imagination and- and it made me want to get it to work, but I still couldn't get uh it to work. At least I couldn't get this to explain grid cells, but it still is an interesting idea that um so I I was I'm hoping just a couple.

A

Pictures might be able to expand someone's imagination- maybe you've thought of these things, but this was helpful to me to think of it this way, um so you can choose to look at 2d space as a continuous graph, where, like every every point in this, is kind of a tiny state and they're almost infinitesimally tiny.

A

This is a point of view on thinking of space as as graphs and their theory here um I'll come back to this in a second. I want to continue with the run. I'm doing um so, you can construct 2d space from a graph, but graphs are more general than 2d space. This isn't the only thing you can construct with this.

A

uh If you think of this, as like a thread or like you're you're, weaving together a space using this graph, but you can weave together all sorts of spaces, for example, maybe entering a cortex can create these strange hybrids of of a 1d track connected to a 2d space. That's connected to a 3d space.

A

You can imagine your mental models as you learn things as you learn tasks as you learn video games whatever it is, your brain's doing you. It is compelling the idea that uh that your brain is kind of stitching together, something like this, but where it's able it's like graphs with co, where graphs built with continuity, graphs that can thread that can kind of create spaces, and this line of logic. What was compelling to me was that you can create these and I'm just trying to. Are you saying these.

C

Are these are separate graphs that somehow are combined together? I'm not sure.

A

I could have drawn a bunch of little circles here, a bunch, a bunch of tiny dots here across the surface. I could have drawn a hundred tiny dots just going across this line, and I could draw a bunch of tiny dots, filling this volume, yeah and and the idea of building up spaces like that and somehow using grid cells to navigate them uh has haunted me for a while, and I wanted to understand if, um if this whole line of thinking could um I'm.

B

Just trying to understand is that, like.

C

What I think is the one would, I say, would I think, of the 1d as a 1d module and the 2d is a 2d module and there's another module, that's 3d models, or is it one great cell module that has all of these things? That's that's part, I'm having trouble imagining.

A

I would say if, if you tried to, um if you tried to do this with good cells, probably the first thing I would do is as you'd use 1d modules for all of it and, and it kind of goes with what you were saying a little while earlier that um 1d modules for all of it.

A

But then you get to this the space over here now the 1d modules point in different directions, um but I guess I'm what I'm really trying to do is sell the big picture of a mental model of what hippocampal formation could be doing uh and and how grid cells could fit into that and and the fact that um that say say you build up a model of hypochemical formation uh like this, and then you, you do have a set of cells taking the eigenvectors here.

A

So this gets back back to painting the space uh painting the space where random walks, because because the color to diffuse in a uniform way or in a by a scaler um by the way, this is the weirdest part of this talk, I'm going to bring it back to things you're more familiar with soon. uh So so I- but I really wanted to what I want to do- is give credit that this expanded my imagination before I say what that I don't like that it doesn't cover phases.

A

So I think I'm going to move on from here, other than just to say that this, what this has going for is a good. It does a good job of describing uh distortions of the grid uh and and painting and coming up with kind of a different theory of what good cells are doing. That does account for the distortions.

C

Could you I mean, could you derive those I mean I mean the distortions come because is this learned or is it um uh can you relate back to reality like okay? So you have this sort of mathematical model that explains these distortions, but how did the? How did the? How does the animal get to that point? um Did it learn these distortions? I I just I'm missing something I just okay. If there's mathematical formalisms, they could explain why there'd be distortions, but so where did it come from? You know.

C

I'm missing that.

A

Yeah best thing I can say it's learned, I mean, like you said there is kind of a mathematical formalism. How do you paint the space as mathematical as that sounds, uh but but- um and I would say my best answer I can give- is that it's learned, but I also am not someone who's spent years trying to defend this.

C

So they didn't want a better that wasn't part of this paper or they just didn't go there is that right.

A

Not that I was able to pick up okay.

F

Just just to kind of interject that picture you showed is is represented with something called a simplicial complex.

A

F

And it basically that's a way of navigating multi-dimensional spaces. I mean you can think of the distance of any point from any of those four vertices as a as a one-dimensional construct, basically gives you uh barycentric coordinates which allows you to navigate higher dimensional spaces with a set of coordinates.

F

So it's a cool concept. I don't know if it's reflected in the in the grid cells, but if, if, if you're tr, if you're pushing an idea that you want to represent, you know sets of of one-dimensional things to represent high-dimensional spaces. It's not a bad starting point.

A

And I would say that's exactly the kind of thing that people who work on this are thinking about, but okay, uh this gets into now I'm kind of going to be preaching to the choir here. But at least my opinion here is the most interesting fact about grid cells, isn't that they fire in a grid uh that they have good like firing fields. The most interesting fact is, that uh is, that multiple cells have the same grid, shifted and, and not everyone knows this, but these shifts are stable across environments and, really importantly, uh I've.

A

I've picked this part up by talking to people like posters and stuff, but but the the grids are also um stable across distortions uh so so across, even when there is a distortion in the grid. It kind of stretches this out to where this is from the original seminal paper about grid cells.

A

This is showing where multiple grid cells fire and this you can see that blue, blue and green, the blue cell and the green cell are always at the same spatial relationship to each other and even in the case where there's distortions they're still at the same spatial relationship to each other and to me this still gives a strong evocative image of of space.

A

There is a spatial representation, even if it's distorted, even if a physical space is mapped into it uh in a distorted way uh and and in this graph view of it, it's not there's not so much of a space. It's different, it's it's fundamentally a graph. It doesn't have these phases. I I I want to see the graph view um um explain the multiple phases of the grid that that's uh that's. What I feel is kind of lacking on this, and uh so that's what I said here with someone about to say something.

D

I was just curious um because you showed earlier this canon this, uh like cone-shaped, room or uh or pyramid shaped room.

D

I was just wondering: if um do you know, are there examples of the kind of distortion even in the distortion? Oh sorry, no. um The experimental example of the, not a cone, sorry a um triangle: it was a bit further up.

A

D

F

Was the trapezoidal room.

D

Yeah that one yeah I.

A

D

Yes, our trapezoidal, um I was just trying to visualize what kind of, because you mentioned that the spatial relations are still maintained despite distortions, and I was just trying to uh picture uh what that would look like do you know, are there kind of um examples, or is that just kind of stuff you've seen around.

A

So I don't think it has been studied and these dramatic distortions like this. It has been studied with this data set right here. They they did study as the moser group. They studied it on this particular data set and they found that um in this case the relationships are preserved. uh I.

D

Don't know with the really open rooms. No! That's interesting, though. Okay, thank you.

A

Yeah, uh so uh just to close on this part, uh let's see I I do like that. I do think that they they bring an interesting perspective to understanding the distortions and that, like the distance um distance on the you know, the grid cell map um might not be measured in meters or might not be measured in actual physical space.

A

It might be more time based where, if you, if you tend to move through a certain area of space faster than you, want it to be kind of more compressed, and you can go even more dramatic with this.

A

If there are spaces that you're unlikely to move to, if there are parts of a room, you're unlikely to move, you might distort the grid that did not really give much much memory space to that part of the room and- and they do these kinds of things in the way they set this up, and I think that that's going to be important for understanding the grid cell understanding the distortions.

C

That so that sounds familiar to the point I've been making recently that I asked myself: how does the? How does the brain learn? The dimensionality of the thing it's observing and to me the I've kind of backed into this belief that the only way you can learn the dimensionality is by observing movements that movements themselves define the dimensionalities of the space. It's not linear movements. It's not linear distances, it's just movements!

C

So what where you can move, it defines where you can model and how fast you move in a direction defines what that looks like. So it seems consistent with this idea um that, and it also is consistent with the idea that they're distortions, because movements aren't perfectly linear and they don't have same speeds and so on. So I'm just pointing that.

C

I'm saying this just to remind myself that that I'm that's how I want to that's how I'm trying to solve these problems is to get like think about it, as movements as the defining um basis of how to measure space, and so it was consistent. What are you just saying, marcus, yeah.

A

I agree I yeah. I agree that you've been saying something similar to this, so okay, rewinding going back to the more familiar land. um Now, I'm now I'm going to talk about things that are familiar to you, but in different terminology, so you're going to be able to map this on to your mental model, but it's gonna be in ways that you haven't most of you haven't thought of before or seen before.

A

uh So now I'm going back and just to call out the two papers in some ways the the paper that caused me to finally put all this together is: is this one from changmin, you tim barron's and neil burgess, uh where they kind of take all of the the they take navigation and path integration um and put it into kind of eigen language talking about it in terms of eigenvectors, um uh but I will say it's it's a very similar paper to this other one that came out in 2013 from jeff orchard, but he describes it in terms of the fourier transform and but they're really kind of the same thing here I just say like I, when, when you're dealing with 2d space eigenvectors of 2d space, are the fourier transform?

A

So so the rest of this presentation you can treat is, if you want to start ignoring the word eigenvector and think of fourier modes, instead feel free to think about that. Instead, both these papers make a lot of the same connections. They both talk about velocity, controlled oscillators. How both of these lead to a theory of what you, what might be 1d grid cell modules or voltage velocity controlled, oscillators vcos?

A

uh So so there's a lot of the similar things covered here, but this one really dives into the eigenver eigen language and talks about a few other things and does a few other experiments now. So I'm going to use some of the ideas from these papers, but but kind of describe it in a way that connects a little bit better with with this group.

A

So um so these, if you remind if you remember where we left off back when we were uh right here, um these start to resemble 1d good cell modules, uh but in ways that we we've talked about. It's kind of um it doesn't feel exactly like them.

A

It just kind of vaguely resembles them, and now I want to make it be exactly them, and so so some of you will have seen this kind of thing before, but um I want I want you to look at these color maps a little bit differently, rather than thinking of it as a single unit and how it fires at different locations.

A

Think of it as showing what phase is active at different locations, showing what phase and this phase can be you could you could imagine a ring attractor sort of like head direction cells except for movement?

A

um Oh, it's like a 1d grid, sound module um or you could imagine this ring being some kind of temporal phase like a velocity, controlled oscillator where this, where the unit is firing at a particular time relative to some other, some other oscillation, or if you want to do this all with engineering you can.

A

You can do this with complex numbers if you're doing this on computers and there's there's an analog I want to draw here is in neuroscience: if you see people using complex numbers in a model, it's often a useful idea to say like wait, can I plug in a ring for this, or can I plug in a vco because often they're representing the same thing? That's often the reason people use complex numbers. Is they want something that that goes around a unit circle like this?

A

So often a ring is an analog for a complex number or the complex unit circle.

D

What does the vco say.

A

Yes, so a a velocity controlled oscillator is is like a um you, can think of a neuron that fires at a particular time relative to the the theta frequency. So so you have these oscillations going through through the hippocampal formation in the cortex, and you have a unit. That's firing at some time relative to all of that.

D

Like a fixed time or a fixed phase,.

A

A a fixed phase relative to that.

C

Is that right, marcus? I would have described it differently, so maybe okay go on um the term. Volta controlled oscillator means you have something that's oscillating and the frequency at which it's oscillating is controlled by voltage and um and that the um the basic idea here is that uh there's a theta frequency um that is a base, theta frequency and you have other cells that are also firing at the base rate of frequency, but um based on movement. The frequency of the other cells increases slightly and so the the cell now is, then it's it's.

C

It's its phase is progressing slowly relative to the base frequency. So you you increase the. I got a frequency of 100 and now something's going on 101, just picking numbers yeah, and so it and so then, when you stop moving it's the the frequency stops. It gets back to the theta frequency. So now that cell is oscillating at the fader frequency, but at a different phase than it was earlier.

D

C

I'm not, I think, what you what you said, marcus didn't strike me that I I.

A

Was trying to say the same thing you just said so I may have misspoken, but in my in my mind we both just said the same thing.

C

Okay, well, my apologies for that. uh Then I might.

A

Have miscommunicated.

C

What's really confusing, that makes sense. Thank you. What's confusing me about the vultures of oscillators, and I'm looking at this picture is confusing here again too. Is that the sort of the need for this variable frequency that's slightly faster than at times when you're moving? It seems to be pretty uh hard to get around this because of the phase precession observed.

C

um But what is the actual thing? That's it. So in one one proposal, there's a set of neurons each one oscillating at a slightly different phase, but they're all being controlled their frequencies all being controlled together, um and the other example we heard about was different uh dendrites on a an individual neuron um oscillating at different frequencies. uh Sort of progressing uh different phases. Excuse me, um so that's all very confusing to me.

C

So when you show these pictures of these little circles around them on the next or the colored bars, and I was trying to figure those five different neurons you're referring to or is that one neuron I I was confused by that one.

A

Right: it's because I'm drawing a picture that can be interpreted either as a ring or as vco. So in a ring there would be different neurons and a ring attractor, but and in a vco it would be the same. Neuron firing at different phases relative to the base, frequency, okay, well,.

C

That's complex for me, but all right: okay, good.

A

Enough, I I don't know a better way to depict that.

C

It's all right, it's good.

F

Can I ask a question about your representation? You've got one cycle of the phase, but you've got it a uh a bounced. I mean, in other words the bottom and top phases are both mapped to the same color. So I was just wondering if that was semantically uh interesting or it just happened a chance because of the color map you picked.

A

So so I wondered if this question would come up so uh here I'll come back to this in a second technically, the full set of eigenvectors includes includes phases going in both directions, uh and so so what what? What happens is as you move in one direction, a ring is or a vco is, is orbiting and you're in you're right. That creates this ambiguity of like there's this light blue is being mapped to two different phases. uh That's just I don't know a better way to visualize it. I guess you could have used it yeah.

A

I don't know a better way to visualize it. So.

F

I I would just start from uh red and cycle through to red in one cycle rather than doing a bounce cycle. If you want them to be unique,.

A

This is a totally valid point uh that would.

C

Be missing the point, a bounce cycle- I don't know what that means. I I thought you were asking: why is it red twice and this is.

F

Going yeah he's going red.

C

F

And purple back to red instead of one cycle of red cyclically back to red, so.

A

Why the real question is, why is light blue appear here twice? Is that.

C

Is that the same question? Yes, okay,.

A

If I'd used a different color map that never repeats the color, this would have been better. Okay, all right.

F

I I I I'll just read that properly then.

A

Yeah you're that that was a good point. I I'd never considered that that you could use a different color map that oscillates rather than you know, going to blue and then back or going to purple. Then back um so I I won't say any more of anything more about the um the complex eigenvectors there, but but there is a technicality there.

A

uh So so I guess I did draw this thing like often in neuroscience, when you see maps like this, it's talking about how a single unit responds, but now I'm drawing a map like this and I'm talking about how a single uh 1d grid cell module or a single vco a single set of yeah yeah, a single vco cell responds, uh and so is it just a different way of looking at color maps as kevin points out, there's a better. You could use better color maps.

A

For that idea, uh though I guess what what I can say for this is it's good because it's showing um in terms of eigenvectors- and this is showing the real component of an eigenvector uh okay, bringing this back home to um so so you can treat!

A

You can represent space using a set of rings or a set of vcos.

A

If you wanted, you could represent two of those rings at once. You could have one population of cells representing two different rings, and that would be a grid cell, except it would be a square grid, but.

A

And so you can get, you can work your way toward a location system based on square grids simply by following like the eigenpath, uh but simply by saying how would I do all this with eigenvectors?

A

um And I'm not saying this is how grid cells are derived, but is sort of a proof for why you can do a lot of operations with with a taurus which that is composed of two rings.

A

um Now now I'm going to talk about displacement cells as as we're familiar with, and uh so so this gets into where, um where I said earlier, that, if you think of what are the eigenvectors of location, what are the eigenvectors of 2d space uh or uh um how? If you, if you follow that line of logic, and you build a system based on it, you land on something that closely resembles good cells and use, has the same computational uh tricks as good cells.

A

So um so here I've shown here I'm showing the basic operation of applying a displacement vector to a location. A location is represented, as a set of in this side show 1d phases or 2d phases and, and the kind of the key trick of the kind of the reason we use an eigen basis for computation is. Is it does this? uh It makes computation really convenient where to figure out the new value for this ring. We don't. We only have to look at the the old value for this ring and and its displacement vector.

A

This does not depend on this this or this um and and the second ring.

C

I thought I I think I know what you're saying, but the words didn't match up in my head. um When you said you only have to look at, would you just go through that sentence again? Would it.

A

Yeah, so so here I'm showing um you know when, when somebody walks through why we talk about eigenvectors, they'll, show a picture that I'm showing right here, except they'll, be putting numbers they'll use numbers in all of these locations, rather than rings.

A

So I'm I'm I'm showing how.

A

Okay, grid cell modules perform uh at least in our mental model of them they perform their updates locally. The grid cell module only needs to know how to update itself. It only needs to know its own displacement. It only needs to know its own previous state. It doesn't need to know the state previous state of others.

C

But but here these four circles- they're, showing uh those are four different grid cells in some sense- is that what you're saying.

A

There are, this is.

C

Four different.

A

Grid cell modules, four different 1d grid cell modules, four different vcos of that are different, um different scales. uh You can, you can interpret it as multiple things. Yes, four different grid cell modules, especially this right here. This is a square.

C

Grid yeah, yeah and then uh and then the sentence where you said you only have to look at something. What was that sentence.

A

Sure uh so matrix multiplication when, when, when a matrix is diagonal, that means that this dimension this yeah, this coordinate only depends on um on this one and and and this value uh that so matrix multiplication. Normally you have to do like kind of across yeah. Everything has to look at everything, but that's the kind of the special thing about a diagonal matrix is that you don't have to do that. Everything's local.

C

uh Okay, that's just basically saying our displacement modules as we defined them are, there's one per grit cell module and they don't need to learn. Anything else. Yeah is that equivalent to that.

A

Yeah, I'm I'm very I'm right now. I am 100 taking an idea. You already understand and putting different language on.

C

B

C

Still trying to make sure you understand it, yeah, okay, I appreciate you trying to dump it down for me, okay,.

A

Okay and uh and quick technicality here, um if you do this, if you kind of, if you kind of stay in the um keep your mind on the fourier tracks, you might reach a conclusion that uh that this is gonna. Give you a particular set of skills.

A

It's gonna give you a scale of like one, a scale of two a scale of three a scale of four, uh because this is what the fourier modes are, um but I, but that is true, if you, if you analyze this by thinking of space as a 1d, track that wraps around or 2d a 2d room that wraps around.

A

But if you think of space instead as extending often to infinity, then any in this case sine wave, because I'm using 1d space any sine wave is a valid um eigenvector as a valid for uh yeah a valid eigenvector, even even though this starts on red and ends on blue, uh if you think of it as extending off into infinity, it's still a valid eigenvector, which means that, like some people hold themselves to this constraint, as if like this theory requires it, but the theory does not require this constraint.

A

You don't have to have the skills, be this exact set of factors you.

B

C

Can I interpret this? I know these are now. um These are not grid cells. We're looking at here, but there's some analogy to them. I guess yeah um it.

C

You know. There's this big problem with grid cells and the big problem with grid cells. Is that the calculations we want to do don't work if the cell repeats within a particular environment.

C

That is if we want to determine the uh the movement vector to get from one point to another, and we can look at the displacement cells that calculate them, but it fails if that cell reports multiple times um it's it's like it's like you want. We want the path, integration, properties of grid cells, but we don't really want the repetition property within the environment we're working with, because the calculations we want to do get messed up.

C

At that point, I don't know if I have to explain why, but so you know originally grit cells were missed because they didn't they weren't.

C

They were at such a large scale that they didn't repeat within the room right, that's why they were missed and when you, if we assume that they don't repeat within the room, then all the calculations we want to do the displacement calculations and the and the movement vector calculations are solvable, but if they start repeating well that still it works makes sense from a path integration point of view, but the calculations we want to do using displacement cells start to fail.

C

um So I I I don't know if that makes sense, but the reason I'm mentioning that is that to me almost all the things we want to do with grid cells. That are interesting. Really we want to do them in grid shell modules, where the scale is larger than the thing we're modeling. So so.

B

I don't actually.

C

Get repeats within that.

B

Does that make sense.

A

Yeah, uh that's I wonder if that that might be the right mental model in that mental model, you don't need the combinatorial code anymore. It's more so you just need different skills for handling different types.

B

Well, what I guess it just struck me is: I don't need this thing to wrap around perfectly it. Just in some sense, as you said just goes on infinitely, I don't actually get to the repetition.

C

In my model, I don't get to the point where the cell repeats, um because I want this. I want the extent of the module to be larger than the thing I'm modeling.

C

um That was one of the problems with good cells and with displacement cells. They don't really, you can't really calculate distances and displacements if the scale of the grizzle module is smaller, it repeats within the within the size of the thing you're modeling.

C

If you know do you know what I'm talking about marcus well,.

A

Let's see the if you use the whole multiple grid cell modules uh to to get a very long coding range, uh then you can get displacement representations over long distances.

C

um I didn't follow those words um just.

B

C

Just take this looks like a single linear grid cell module and- um and I want to determine how what movement I should make to get from point a to point b.

C

Well, um I can calculate that as long as point b doesn't repeat multiple times within the environment and if it repeats it multiple times, then I can't calculate it and that same problem extends to multiple modules. um It's same with our displacement cells, our displacement cells say hello. I can, I can form a unique representation where this new thing is relative to the old thing, which is spice themselves just one way in one of its forms. It calculates movement vectors, um but if the destination point is repeats, I can't do it. So it's just I.

C

I feel that we always show these pictures of the grid cells repeating within an environment, but I've come to the sort of belief that they don't work in that space you have to. You have to find a scale, that's bigger than the environment, um and then then everything works. I think I think this problem has been. We identified this problem and other people identified it too. I think in other terms, but am I wrong about that or.

A

Is there so? Well, I guess: do you acknowledge that multiple grid cell modules, two different scales, can achieve a scale together that is longer than yes.

C

But if we want to, if we want to use displacement, vectors to calculate a movement or to calculate displacement between two objects, those are the two two things we use displacement vectors for. Remember the.

B

C

Vector idea is similar to the idea of how to calculate uh a movement vector right. It's like oh. I have two points. How do I get from point? A to point b, I can do that on a module one nodule basis, and we had the same thing for displacements is the same thing. So it forms a unique representation, as you said, um but we can't do the calculations that displacement cells want. We want to do.

B

A

That you can do you can do long displacements.

C

B

C

What you said once before marcus you said: yeah.

B

We could do it, but we have to learn it. We have to learn.

C

B

All right, but I.

A

Don't want to exactly I don't I don't. I agree. I I just like that too. You can represent them, but you can't decode them.

B

Yeah, so if I, if I and so that's to me, was like.

C

A show stopper, it's like okay, yes, it's unique, but I have to learn all these unique vectors impossible. If I constrain myself to a module, that is that its repeat rate is larger than the thing I'm modeling. Then I don't have that issue. I can just basically sum up all the movement individual movement vectors. Remember I remember, I'm thinking of each grid cell is representing a movement vector, and so I can just sum up all the individual movements- and I get my answer.

C

I don't have to remember anything um anyway, so I just I only mentioned that because I think that might relate to the issue you just talked about here, where you're not forcing these things to wrap around perfectly. Maybe.

E

It does maybe it doesn't.

C

It just reminded me of it: I'm gonna, if I I'm gonna, try to talk about some of this stuff on wednesday. um Come back to this again.

A

Yeah here I was one thing I was saying was that if you set this up analytically, you might over constrain yourself technically over the plane over the plane of infinite space. Any sine wave translated in any way is, is an eigenfunction or eigenvector. So um so you can. You can use this framework and still get the long coding ranges that come from using just kind of random scales and like here I showed that here this has the same.

A

It has a repetition of the six module representation on both sides of it, but this is a total this. This never repeats over that span, if you just use random skills- uh and that was more uh a caution to to people who who who who come at this from this angle that, like it, really does not supply this constraint, some people- sometimes people- have looked at this and said, like you know how, um with grid cell modules, the scales go up by a factor of about 1.4 they've.

A

Looked at this and said: is this roughly 1.4 and they say like yeah kinda like, but but that logic may not need to apply here?

A

um Okay, I want to jump into an interesting uh so so this was. This was like taking this language saying you can use fourier, eigen language to talk about displacements and everything um now now I want to see.

A

Does that take us if we go further, if we, if we continue to embrace fourier okay, are there any cool ideas that appear, and at least I want to give you a picture of this idea- um yeah a picture of this so so far we're talking about using multiple modules to represent a single location, so so far we're representing a single location, but the fourier transform can be used for much more than that.

A

You can represent, for example, multiple locations simultaneously, uh fourier transform is often used to represent high detailed images, and here we're using it to represent a single dot here. I'm laying out just a perspective that, like there might be something really interesting here um if the grid cell, if grid cells are indeed doing something resembling fourier transform, we should consider.

A

Are they actually doing more, uh really powerful fourier transform uh tricks? So, for example, um can grid cells represent locations relative to three different landmarks uh and and for those of for anyone here? Who knows what I mean when I say union, uh I'm not talking about a union, I'm not talking about having each one be multiple things at one time um in our traditional use of the word union. I'm talking about. Can this? Can you set these values?

A

Can you set each module's value in a way that encodes that I, the rat and at this location, next relative to a landmark at this location relative to the landmark and this location relative to the landmark? um Sometimes you hear place cells described as being something like this, where place cells are defined by your surroundings, by your the surrounding boundaries, and it's almost like a place cell might represent something like this.

A

It is your surroundings where, where the landmarks are relative to you or where you are relative to the landmark and you can draw boundary vector cells in this using the same idea, this might be a little bit of mind shift, but this picture is showing. Where am I me being a rat? uh Where am I relative to boundary and and and it and you start to reach like this computational need of you- almost want to paint a picture of all my locations like.

A

I am right here next to this landmark here and here and I'm right here next to this boundary, uh so so like say a rat walks into an environment like this. That sees all of these. um It's it's boundary, vector cells. Tell it this is object, vector cells tell it this.

A

Can you take that and map that into a grid cell representation? um If grid cells are performing an exact fourier transform? The answer is 100. Yes, you can do this, uh and so I'll jump before saying why this is cool, because I it might sound a little magical right now. uh What you would need to make this work is, if grid cell modules have both a phase and a magnitude. Here I drew little numbers inside the ring, uh so this is a little harder to get working with a vco.

A

You can definitely get it working with the ring if it has a phase and a magnitude, um it suddenly becomes capable of representing complicated uh um things like this, like, like this line and the set of dots, uh and so you so what you would get from this is some grid cells in some places would would fire at higher rates or have some kind of higher magnitude than other good cells, but path integration works, as is you just you just move the bump around you, you, you update the phase and keep the magnitude the same.

A

uh Everything would work. You just need the magnet. You need the grid cell module to be able to have a magnitude, uh so this idea would be cool because you would be able to take. You know a picture like this from your surroundings, your surrounding boundaries, your surrounding landmarks, convert it into a grid cell representation, and that would be like a path, integrable, fingerprint or in computational terms, a hash for the current environment.

A

You could you could take a novel environment, find its hash and not have to learn anything you just you just next time you enter this you're going to use the same trick and activate the same grid cell representation, and this would be a really cool thing for associating memories with locations.

A

um So so the fourier transform uh provides an interesting angle here uh and- and this is a possible explanation for why grid cell rates do vary across um to get that working. You'd have to do a little bit more, but um if anyone's interested.

C

Well, that's interesting! I don't remember us talking about this. This idea before this empirical.

A

No, this is my first time presenting.

C

This I mean the empirical idea that they've observed good cells and code uh they're firing rape.

A

Well: okay, remember uh the missed fields and the the the paper, the david.

C

Yeah, but those are those are more like binary: they they just disappeared. It wasn't like uh it wasn't described as a scale or it was more described as hey. They just stopped working. It was more of a not a rate coding, but more of a you know.

C

A

That still fits into this, but I okay, I would expect it to be sometimes rate, sometimes binary.

B

C

Well, that's! Well! That's that's how it was that's if I, if I remember that paper, they described it more like hey these cells. Just didn't respond at all here. You know, but we expected him to and it wasn't like it wasn't described as scalar.

B

C

That's could be an experimental bias or something like that, but that's how they described it.

A

This one was much more scalar and it was. It was a little bit. This was more of an appealing experiment because it was a. It was a rat moving around in 2d space, where this was a mouse on a virtual track running in 1d, uh the misfields one was was running in 1d, so so this is more just like uh less less statistical analysis had to go into it. It's just, they can simply say the firing rates of in this field are lower than in this field.

C

I mean bridge housing called local positional information. I mean, isn't that always the case I mean.

A

Individual grid cells, uh like you, can tell from a grid cell's firing rate which of its fields it's most likely to be in. Oh.

C

That's interesting, I didn't see that um I want to get that paper, so I mean we better write it down, um read sales and call vulcan position.

A

Yeah um yeah I've gone on for a while, uh so this I I've painted like a fun picture of something grid cells could do. I I think it's probably best for me to work on closing this presentation, so um I'll go back and just to quickly summarize um so this fourier eigenview.

A

I think it is useful to point out to people that if you understand grid cells using this lens this this mental model, there are some ideas you're likely to miss, because it's they just are weird. And this when described in this language.

A

So we've talked about how displacement cells, something that can represent a displacement between two locations, can also do this thing where, where you represent the spatial relationship between two groups of locations or reference frames- um and this is harder to see you have to you- have to know about location spaces and the the fact that you can assign a set of unique locations to one object instead of other unique locations to another object.

A

um So so, for this I would recommend like sometimes you need other mental images, here's something that marco drew from our uh from our other paper. uh So when you, when you sometimes this can be useful to think if you want to think about this analytically. Think of like multiple grid cell modules as creating a high dimensional torus, where you can only move in certain directions on the torus through movement, but the displacements can can represent any direction. The displacements can move you in any direction on that taurus.

A

um That was a quick blurb, a cautionary tale to people that this will limit your imagination, and but people in this room understand these ideas already.

C

It's funny I I as many times I've heard the the the the representation that you and merkel like I still just it just doesn't work for me. I'm sorry, it's like, I feel, like I'm a failure here.

A

I'm speaking to the subset of people who think the fourier view is intuitive, I'm saying those people will be able.

C

To get this, you know my limitations. Well, I guess.

A

Yeah, I don't know, and- and I guess just like fine final point I wanted to make- I I'm going to say this in like 20 seconds- grid cell distortions. I said a little bit of before. I think that, like um good cell distortions uh are probably a feature not a bug, the brain is probably using it in an interesting way and I'm fond of two different ideas.

A

One is that you could call this a conformal mapping where, um where your your, your, your actual cognitive, I don't know your actual grid cell map with these clean grid lines is mapped down to space and this uh in a pretty flexible funky way. But that has certain constraints where, for example, the grid lines always remain orthogonal, which is to say your head direction. Cells always update linearly. No matter where you are, uh this is a well.

A

This is a study, well-studied area that conformal maps can be really good for, um are useful computationally and you can still path integrate over them. If you're, if you, if you're careful about it uh and then second like or you, can look at distortions as a form of generalization uh that that, like you, have you have a small number of mental maps that you stretch onto different environments and the fact that this map is being distorted is actually like. uh It's a convenience.

A

It's because you're using your knowledge from the past and stretching it onto your current uh situation, um so so in general. I still think that, because of like you know the different phases of good cells, that the representations are inherently spatial but the but space can be mapped into it. Physical space can be mapped into it in a distorted non-linear way, but the, but the representation itself is still spatial.

A

uh So I kind of raced through this end part because time was coming short, but this is this is where I'm ending here.

D

Do you know if there's any experimental evidence for that second idea, like um distortions being less significant in a in a familiar environment,.

A

The irony is kind of the opposite, uh as as an.

D

Environment becomes.

A

Better learned: uh it's that's when, like the the grid kind of distorts a little bit more.

D

Because I feel like that then supports the first one that maybe, as as you, develop a better sense of the kind of space temporal or whatever, like how your movement actually affects the environment, then definitely.

A

I would say yes, yes, I totally agree.

C

Keep coming back to the idea that um the brain doesn't it's no knowledge about the world, it has to discover everything. You know the cortex at least, and so the only thing it has to work with in terms of figuring out the spaces it's its dealing with are movements it's it. It knows it says I have a representation of a movement. I don't know what the movement is.

C

I have no idea what the movement is, but I know that it's a movement vector of sort and um and from that I have to build up my spatial representation.

C

um They can't sit down, looking say: oh, this is three-dimensional or two-dimensional or one-dimensional, there's no way of doing that. It just says yeah I move, and these are the changes that I get my sensory input and so again I feel I've said it earlier to say it again somewhat from my benefit.

C

Is this that um the whole idea of space is is really defined by movements it can't be defined any other way and um and therefore it's going to be distorted by what movements you are capable doing and and if you can't move in certain directions, you will not represent that direction, and things like that. So when I think about this and trying to understand all these distortions, I would be coming back to that perspective. That doesn't mean anything you just presented here is inconsistent with that. I'm trying to merge the two.

C

I'm just saying that I think we can't ignore this idea of thinking about movements uh as the foundation for representing space. We have to do that. That has to be sort of the foundational metric in some.

A

A

Yeah, that makes sense to me- and I think that is quite compatible, especially with um this first one kind of.

C

Yeah and personally, I find it very hard to do this translation from the you know, the fourier and the eigenvector spaces, and um I feel like I should study up this on this more and work at it. But it's it's difficult for me.

A

And if you want, you could just treat this as you know that the connection is there and on demand, you can go pick it up when you need it, but uh all of the things we talk about with displacements and path integration can be described in a way. That's super analogous to fourier or eigen.

C

Yeah but as you point out, you got insights from thinking about it this way so yeah, I don't want. I don't want to miss those insights. You know that's the problem.

A

It's possible there's a big idea right here. It's also possible, I'm just totally crazy and it's not uh it's not what's happening.

A

Okay, uh so we can call this the end of anybody.

C

Else anyone else have any questions.

G

Yeah I had a short, quick question, marcus um on that on your slides or under your presentation. You had that last section go full for you, where you show those three different orange dots at once so were you were you trying to imply that grid cells are representing those three locations at once?.

A

Yeah, I'm I'm saying that grid cells- if they have both a phase and a magnitude, we know they have a phase, we kind of know they have a magnitude. I mean they have different firing rates. uh uh This could be in fact, what they're representing they in the same way that a fourier transform is. You know it takes a bunch of waves and reconstructs an image by putting different uh coefficients on these by by shifting this and scaling this shifting the scaling.

A

That's doing that it can reconstruct an image, and I'm saying if that, if that's literally what anariyama cortex is doing, this is what I would use it for. I would use it for representing a set of locations and a set of even stripes. These stripes, like saying where the boundaries are yeah.

C

So they don't even apply something else, so I mean someone else has to read out that code right um if the grid cells are representing, that we want to associate it with something, and so another neuron has to be reading out that code, but that other neuron then has to be reading out the code based on the set of not just the combination code, but also scalar values of those firing cells, um which is kind of hard to do on, is that right.

A

I mean one one part you can imagine is that so I have this question mark here but, uh like you could imagine a connection between the grid cells and the boundary vector cells where in one direction you're doing a fourier transform the other direction you're doing inverse fourier transform, so you can recover either from either.

C

My point is a boundary vector cell then, would have to be paying close attention to the scalar value of the grid cells, um they're, fine yeah.

A

Yeah and you have to have an inverse fourier, transform and neurons well.

C

Okay, recall yeah, if you want to call it that way, sure uh but yeah, which I never think of neurons capable of doing that right. It's it's like it's like. Oh, this synapse has to be at this frequency, and this synapse has to be that frequency and this synapse has to be that frequency. And if we have that combination of those three inputs at these three different frequencies, and this then I'll recognize it. I don't know how neurons could do.

A

That I haven't tried to set up a neural version of that.

D

I'm just gonna ask if, um if this can be done with the kind of four-year transform, what's to stop uh someone kind of implementing a system like this, not necessarily in a neural network based uh like system, but it just it seems like this- would be really powerful um as kind of yeah a module. If you want to do things like path integration, um but so I was just wondering if, if there's anything that makes that difficult to do uh even like ignoring neural constraints,.

A

Not that I I nothing comes directly to mind, I don't think it's a directly hard computational problem.

A

I I would expect that um that if, in fact, good cells are performing something similar to fourier magic, uh they probably do it in clever ways that would that would expand our imaginations where, where like actual fourier might be really rigid and constrain us in ways like you might will find that the brain has done it in some clever flexible way. But but that's just a general hunch right.