►
From YouTube: Discussion on Grid Cell Distortions and Reference Frame Transformations - May 17, 2021
Description
In this research meeting, Marcus Lewis discusses the importance of explaining grid cell distortions, and to generate discussion he proposes a possible explanation, showing some results from an experiment he conducted. He hypothesized that an animal localizes by detecting distance from various points of boundaries and those points “vote” on the location. The weight of the votes is determined by nearness, and distortions occur when the animal's idealized map differs from the actual environment. The team then discusses the hypothesis and raises further questions.
Jeff then explores the possible processes and mechanisms that underlie reference frame transformations in the neocortex. He describes a few problems with a previous hypothesis he proposed about rf transformations in the thalamus and further explores the role of the thalamus. He then suggests the relationship between rf transformations and temporal memory.
Papers from Marcus’ presentation:
“Framing the grid: effect of boundaries on grid cells and navigation” (2016) by John O’Keefe et al.: https://physoc.onlinelibrary.wiley.com/doi/full/10.1113/JP270607
"The hippocampus as a predictive map” (2017) by Stachenfeld et al.: https://www.nature.com/articles/nn.4650
“Flexible modulation of sequence generation in the entorhinal–hippocampal system” (2021) by McNamee et al.: https://www.nature.com/articles/s41593-021-00831-7?proof=t
0:00 Marcus Lewis on Grid Cell Distortions
43:08 Jeff Hawkins on Reference Frame Transformations
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
B
B
I
should
give
an
actual
full
presentation
on
this
because
there's
it's
just
a
nice
little
idea,
a
nice
compact
little
set
of
results
so
so
yeah
here
I
guess
I
called
this
a
possible
explanation
for
good
cell
distortions,
and
so
so
first,
I'm
just
going
to
give
you
some
background.
Why
I'm
talking
about
grid
cell
distortions
and
what
I'm
referring
to
when
I
say
good
cell
distortions,
so
I
made
this
a
little
dramatic.
B
B
First,
let
me
just
read
the
quote,
because
it's
otherwise
it's
just
kind
of
glaring
down
here
I
I
I
want
to
emphasize
that
a
common
sentiment
among
people
who
study
grid
cells
is
that
we're
not
so
sure
it's
providing
a
metric,
because
it's
just
messier
than
we'd
like
it
to
be
so
I'll.
Just
read
the
quote.
B
I
don't
know
what
john
o'keefe's
current
thinking
is,
but
he
said
like
the
spectacular
good
cells
which,
until
very
recently,
we
thought
we're
providing
something
like
a
metric
for
the
map,
but
we're
not
so
sure
anymore,
and
he
was
he
said
that,
because
of
this,
this
figure
essentially
and
something
from
his
lab,
which
I'll
also
show
in
a
couple
slides
where
he
was
using
trapezoidal
rooms,
but
even
just
I
I
feel
like
this
is
a
a
better
example,
because
it's
just
a
big
box.
It's
not
an
oddly
shaped
room.
B
You
could
make
the
case
that
rats
don't
deal
with
box-shaped
things,
but
still
it's.
This
makes
me
nervous
about
the
metric
interpretation
of
grid
cells
and,
and
so
the
in
this
presentation,
I'm
going
to
offer
a
defense
for
the
metric
interpretation,
but
right
now,
I'm
just
doing
the
background
of
why
why
it
needs
a
defense.
Looking
at
this,
looking
at
this
figure,
I
mean
I'll
just
draw
your
attention
to
a
couple
parts
up
here.
B
These
fields,
like
the
distance
between
them,
is
much
much
smaller
than
like
the
distance
between
these
two
fields.
In
fact,
if,
in
fact,
a
theme
in
this
room
is
that
the
upper
left
corner
of
it
has
this
very
small
scale
compared
to
the
scale
over
here
and
just
in
general
like
there's,
there
are
slight
orientation
changes.
B
B
A
Always
confused
me
about
this,
and
maybe
you
can
answer
this
question.
We
know
that
grid
cells.
They
have
this
phase
procession
right,
so
so,
when
the
cell
fires
in
the
the
theta
cycle
various
how
far
it
is
from
the
center
of
the
receptive
field
in
some
sense
right.
So
when
they
show
figures
like
this
there's,
there
seems
to
be
two
conflicting
things
going
on.
One
is
that
the
rat
hasn't
traveled
everywhere
in
this
room.
So
you
can.
A
You
can
see
that
some
parts
of
the
where
the
rat
like
in
the
bottom
left
and
even
in
the
upper
left
corner
that
the
rat
hasn't
gone
as
often
so
we
don't
know
what
those
data
points
are
right
and
then
the
second
thing
is,
you
know:
what
are
they?
What
are
they
saying
every
time
the
cell
fires,
and
so
so,
if
I
only
showed,
but
when
itself
fired
right
in
peak
with
a
phase,
would
be
much
sharper
these
these
rough,
these
receptor
fields
would
be
much
sharper.
B
A
A
Yeah,
because
that's
a
confounding
situation
here,
because
you
can
say
hey,
this
cell
is
not
very
accurate,
but
what
we
know
is
that
the
cells
are
actually
much
more
accurate
than
they
look
like
I'm
not
saying
you
wouldn't
have
these
distortions,
I'm
not
countering
it.
I'm
just
saying
that
the
image
here
would
be
much
more
accurate
or
precise.
A
B
A
And
and
there's
all
these
holes,
you
can
see
this
whole
bunch
of
holes
here.
You
see,
oh
there's
nice,
clear
spaces
between
these
flops,
but
there's
places
where
the
rad
has
never
been
yeah.
So
it's
like
well,
we
can't
really
read
too
much
into
that
either
again,
not
not
countering
the
distortion,
I'm
just
saying,
because
I've
always
looked
at
these
images
and
say:
oh
they're,
so
messy
you
know
they
don't
look.
A
B
So
so
this
has
been
in
the
back
of
my
mind
for
quite
a
while,
because
I
I'm
open
to
other
hypotheses
about
grid
cells
there.
Maybe
there
are
other
ones
and
and
the
first
one,
the
the
first
quote-unquote
other
hypothesis
that
that
seemed
plausible
to
me.
I'm
not
convinced,
but
it's
definitely
on
my
short
list
of
things
that
might
be
true.
Is
this
one
from
kim
stackenfeld,
matt,
botmanic
and
sam
gershman,
and
that
one?
It
would
take
too
long
for
me
to
go
into
detail
about
it.
B
I've
talked
about
it
a
little
in
the
past,
I'm
going
to
put
some
of
my
own
words
on
it,
and
those
words
would
would
be
that,
like
okay,
the
spatial
view
of
grid
cells
is
not
quite
right.
The
right
way
to
think
of
it
is
the
grid
cells
are
being
used
to
simulate
a
linear
system.
Those
linear
systems,
like
so
I'll,
just
continue.
Reading.
We
engineers
know
that
solving
a
linear
system
of
differential
equations
requires
converting
to
an
eigen
basis,
conveniently
grid
cell
firing
fields.
Look
a
lot
like
an
eigen
basis.
B
It
looks
a
lot
like
encoding
location
in
a
eigen
vector
basic
based
basis,
and
they
provide
this.
This
line
of
thought
has
been
thought
provoking
to
me
and
also
I'm
I'm
sort
of
backdating
some
information
here,
they've
they've
done
some
more
recent
work
involving
also
magma
and
and
so
me
calling
a
like
differential
equation,
a
linear
system.
Some
of
that
actually
comes
from
this
more
recent
paper,
but
I
think
that
something
like
that
was
in
their
heads
back
then,
and
so
I
still
like
this.
It's
thought-provoking
to
me.
B
The
early
one
that
I
really
liked.
I
was
calling
stretchy
maps.
I
think
I
might
have
been
under
the
influence
of
tim,
baron's,
stretchy
birds
when
I
was
using
those
words.
So
the
idea
that
we
take,
we
have
a
set
of
maps
that
we
reuse
in
a
lot
of
contexts.
We
have
a
map
and
we
stretch
it
on
to
different
environments
and
once
we've
stretched
it
onto
the
environment.
You
know
it's
like
you
have
a
a
playbook
of
actions
that
you
could
take
a
playbook
of
things.
You've
done
in
situations
like
this.
B
Proposing
yes,
yeah
yeah
I'd,
say
that's
I'll
get
to
that
the
2021
one
here
I
took
a
total
this.
This
2020
one
was
a
little
bit
influenced
by
the
stack
and
felt
idea
I'll
I'll.
Just
summarize
it
I've.
I
gave
a
presentation
on
it
in
december
and
it's
still
interesting
to
me.
You
could
call
it
taking
the
fourier
transform
and
making
it
more
general
and
calling
it
a
basis
for
representing
sensory
input.
B
So
it's
like
you
have
a
set
of
reusable
parts
that
you
that
you
sort
of
lock
onto
the
things
around
you,
and
then
you
learn
that
arrangement
of
reusable
parts
and
but
it's
closely
related.
In
fact,
the
rest
of
this
presentation
is
equally
about
both
of
these
I'm
not
gonna,
I'm
not
actually
to
use
parts
here.
I'm
the
rest
of
this
presentation
is
is
going
to
be
evaluating.
B
Is
this
this
general
idea
that
the
stretchy
maps,
the
reuse
of
parts
idea
a
good
explanation
for
good
cell
distortions
and
in
a
simulation
I
ran
that
that
I
thought
that
I
think
was
was
nice.
It
doesn't
provide
an
unambiguous
answer
to
this
question,
but
I
think
it's
a
nice
result.
It's
a
fun
result
at
the
very
least,
and
so
I'll
go
ahead
and
jump
into
what
I
did.
B
What
I
did
so
the
general
experiment
is:
can
I
explain
grid
solar
distortions,
like
actual
ones,
actual
ones
from
papers
using
this,
this
cartoon
the
mental
model
suggested
by
this
cartoon
and
that
that
cartoon
is
like
the
agent
a
rat?
What
the
the
the
problem,
a
problem,
it's
solving
when
it's
localizing,
when
it's
deciding
which
grid
cell
to
activate,
is
it's
looking
at
its
surroundings
and
trying
to
sort
of
simplify
them
into
an
idolized
into
idolized
objects
or
into
an
idolized
map.
B
B
I
don't
know
things
operating
in
the
world
to
do
is
that
they
take
their
input
and
they
kind
of
describe
it
in
terms
of
idolized
objects,
and-
and
so
this
to
me
is
an
elegant
explanation
for
why
you
would
expect
distortions,
but,
okay,
it's
elegant,
but
does
it
explain
the
empirical
ones
as
well
as
what
I
had
to
experiment
with?
I
liked
the
idea
of
going
in
I
just
needed
to
know:
does
it
explain
empirical
distortions?
B
So
what
I?
What
I
did
was
I
took
these
distortions.
This
is
the
same
field.
I
was
showing
you
before,
but
it
does
a
heat
map,
and
this
is
this
is
another
one
in
a
trapezoidal
room
from
o'keefe's
lab,
where
you
can
see
that
the
orientation
kind
of
changes
also
the
scale
changes
over
here.
These
fields
are
closer
together.
These
ones
are
further
apart
and.
B
Yes,
yes,
so
yeah,
they
depict
it
as
the
fact
that
this
this
line,
that's
connecting
these
dots
is
curving
but
yeah.
So
over
here
you
would
say
that
this
has
a
different
quote-unquote
orientation
than
this,
so
I
did
not
reach
out
to
these
labs
and
get
their
data.
Rather
I
copy
and
pasted
their
figures
into
illustrator
drew
dots
over
theirs
at
the
center
of
the
fields
and
basically
imported
this
and
tried
to
explain
these.
B
So
so
I
took
theirs
kind
of
simplified
them
into
this,
set
of
dots,
the
centers
of
the
firing
fields
and
and
tried
to
explain
those
and
so
I'll
go
on
and
show
the
rest
of
my
setup
so
the
algorithm
here
and
for
the
for
anyone
familiar
with
the
huff
transform.
This
is
going
to
be
a
little
bit
like
that,
so
the
algorithm
here
is
like
the
way
that
the
animal
localizes
is.
B
B
B
Is
the
writer
one
the
earlier
ones
are
the
confusing
ones
so
yeah?
So
so
it's
like
you,
you
look
around
you
look
at
how
far
you
are
where
you
are
from
points
on
the
wall
and
you
compare
it
to
your
idolized
kirby
wall
and
each
of
those
sort
of
each
of
those
points,
sort
of
votes
on
where
you
are-
and
I
show
this
is
like
all
these
arrows
land
on
the
same
point
here,
but
they
land
on
different
points
here
and
that's.
C
B
C
A
B
I
mean
what
you're
the
words
you're
saying
makes
sense.
Yes,
I'm
I'm
a
little
confused
by
this.
I'm
I
I'm
in
the
state
of
trying
to
decide
why
wouldn't
the
rat
use
straight
parts,
maybe
it
uses
parts
that
are
more
similar
to
gabor
filters
in
the
sense
that,
like
hey,
visual
cortex,
doesn't
use
straight
lines.
Visual
cortex
uses
gabor
filters.
Maybe
there's
actually
unintuitively
a
more
useful
type
of
part
than
what
we
would
intuitively
choose.
A
A
B
So
to
show
you
the
rest
of
the
setup,
oh
oh,
I
should.
I
never
said
this
last
part.
You
have
to
weight
the
votes
by
nearness,
which
is
to
say,
if
you,
if
you're
taking
in
like
a
bunch
of
sensory
observations,
there's
an
object
over
there
an
object
over
there,
you
use
the
nearby
one
you
give
it
more
weight
and
and
voting
on
where
you
are.
A
A
Not
I
mean
because
you're
not
really
taking
the
account
of
the
you
know,
basically
that
all
I
have
is
local
information
about
the
wall
and-
and
I
wouldn't
even
know
I
wouldn't
even
know,
I'm
moving
in
some
sense.
I'm
like
like
I'd
almost
it'd,
be
almost
like
a
boundary
seller.
I
can
always
compute
boundary
versus
border,
but
it's.
A
Is
you
can't
cross
over
once?
You
can
right,
so
it
just
seems
like
all
I
had
was
the
local
closest.
If
all
I
was
using
was
the
information
was
the
closest
part
of
the
wall.
Then,
as
I
move,
I
wouldn't
actually
know
I'm
moving.
I
mean
I
wouldn't
be
able
to
judge
my
position
differently.
I'd
have
to
be
we
talked
about
this
last
week.
I'd
have
to
be
looking
at
the
end
points
to
really
know
where
I
am.
B
A
B
B
So
I
I
found
a
way
to
figure
out
what
is
the
I:
what
is
an
idealized
map
that
leads
to
grid
cell
firing
fields
that
look
like
this,
so
I'm
solving
for
what's
in
the
thought
bubble
and
that's
the
funny
thing
about
differentiable
programming,
is
it
can
do
things
that
surprise
you,
like
all
of
the
things
I
just
described
to
you
like
the
weighted
voting?
That's
all
differentiable!
So
you
you
just
set
this
up.
You
set
a
variable
of
grid
cell
scale.
B
You
set
a
variable
for
good
cell
orientation,
another
one
for
the
phase,
and
then
you
find
a
way
to
to
describe
where
these
boundaries
are,
and
you
can
just
set
this
up
and
have
it
discover
what
parts
where
you
have
to
discover
what
idealized
map
will
cause
this.
So
that
brings
me
to
results
so
here's
what
pops
out
and
and
for
these
for
these
two
different,
these
two
different
rooms.
If
I
fit
the
the
map
the
to
the
grid,
what
pops
out
the
the
real
output
here
is.
B
Is
this
right
column
in
the
middle?
I
show
what
the
simulated
grid
cell
looks
like
that's
these
white
bumps,
that
are
these
white
fields,
and
I
overlay
the
actual,
the
red
ones,
and
what
I
find
that,
if
this
explanation
of
grid
cell
distortions
is
correct,
here
is
what
the
idealized
map
probably
looks
like,
and
I
mean
the
things
I
was
saying
before.
How
the
scale
over
here
seems
to
be
smaller
than
the
scale
over
here
is
reflected
by
how
this
is
small.
A
Wouldn't
that
make
him
bigger
there
if
it's
blown
up
in
that
left-hand
corner,
it
seems
like
if
I
stretched
out
the
corner,
I
would
be
stretching
apart
the
points
as
opposed
to
compressing
them,
and
then,
if
I,
the
compressed
lower
left-hand
corner
or
any
part,
you
think
they
would
be
closer
together.
I
mean.
B
A
So
but
maybe
that's
not
the
right
way
interpreting
it,
but
that
seems
like
what
would
have
happened.
I
would
have
just
squished
everything
together
at
the
bottom,
and
things
at
the
top
would
be
more
rarefied,
but
is
this
the
opposite?
It
seems
like
the
place
at
the
top.
We
have
the
most
space
you
just
this
receptor
fields
are
closer
together
missing
something
it's
intuitively
sort
of
backwards.
What
I
would
have
guessed,
but
maybe
I'm
not
thinking
about
it
correctly,.
B
B
A
Okay,
I
see
what
you're
saying
I
was
assuming.
I
had
a
square
thing
with
equal
fields
and
then
I
squished
it
to
look
like
this
ideal
map
and
then
it
wouldn't
produce
what
you
said.
I
would
have
closer
spaces
on
the
bottom.
However,
I
think
what
you're
saying
is,
if
I
put
an
I
a
regular
array
of
grid
cells
on
the
idealized
map,
so
they're
perfectly
they're
perfectly
spaced
apart
on
the
idealized
map,
and
then
I
stretch
the
idealized
map
out
to
be
square.
A
A
I
see
what
you're
saying.
Yes,
I
see
what
you're
saying
it's
like
it's
like,
if
I
had,
if
I
had
a
perfect
grid
cell
map
in
this
in
this,
in
the
don't
call
it
a
map,
the
idealized
map,
that's
like
the
that's
what
the
room
is
like
if
the
room
was
like
that
distorted
little
room
there
and
I
put
a
perfect
good
celery
on
it
and
then
I
stretched
the
room
out
to
be
square.
B
A
C
A
Now
you're
left
with
this
question
is
like
well.
Why
do
the
rats
think
the
room
looks
like
this
weird
stretchy
thing?
That's
that's
what
you're
saying
you're
saying
the
rat,
the
rat.
You
know
thinks
of
a
room
this
way.
This
is
because
there's
no
landmarks
in
this
room.
It's
just
a
square
room,
there's
something
else
right
so
for
somebody,
who's
wrapped
in
this
square
room
and
it's
modeling
it
as
if
it's
this
you
know
distorted
room.
C
A
A
A
You
know
who
knows
maybe
they're
navigating
to
some
light
fixture
in
the
ceiling,
and
we
don't
even
know
that
and
it's
you
know
what
I'm
saying
and
so
so,
but
it
begs
the
question:
why
would
the
rat
think
of
the
room
like
this
distorted
room
and
it's
weirdly
distorted
it's
asymmetrical?
It's
got
curves,
it's
got.
You
know
it's
like
what
the
hell.
A
You
know
it's
like
okay,
yeah,
okay,
that
would
explain
it,
but
why
it
kind
of
it
kind
of
makes
the
the
idealized
component
idea,
which
I
liked
a
lot,
that
it
kind
of
it
kind
of
says
they
still
don't
look
like
idealized
components
at
all.
These
look
like
some
weird
thing,
that's
going
on
here,
yeah.
I
wonder
I
wonder
if
it
could
be
stuff
in
the
room
itself.
You
know,
I
guess
it's
around
everyone
there.
A
These
are
open
boxes
and
there's
stuff
on
the
ceiling
and
people
walking
around
in
machines
and
rats
can
see
this
stuff
and
they
don't
even
talk
about
it
and
that's
how
the
rats
orient
themselves
mostly
to
the
room,
because
in
this
case
maybe
the
room
has
no
distinguishing
landmarks,
so
for
the
wrath
to
even
orient
itself
at
all,
it's
oriental.
It's
orientating
itself
to
things
above
the
box
that
we
can't
we
are
not
told
about,
and
that
could
lead
to
these
distortions.
B
For
the
ceiling,
no,
I
don't
have
a
good
answer
to
that,
and
I
mean
the
random
walking
is
the
random
foraging
comes
from
them
dropping
in
food
food
at
random
locations,
yeah,
so
something's
going
on
above
the
rat.
That
is
dropping
food
in
I
don't
know
what
that
looks
like.
I
don't
think
it's
just
a
person
because
I
think.
A
A
A
Well,
even
the
trapezoidal
one
you
could,
whatever
it
is,
it's
non-tropical
quadrilateral
one
there
you
could.
You
could
imagine
that
there
are
some
distinguishing
features
of
the
room.
The
corners
are
not
identical
right,
so
I
could
you
could
put
me
in
a
a
room
like
that
and
I
could
orient
myself
on
it
if
there
was
nothing
else
other
than
the
shape
of
the
room
but
the
square.
A
I
couldn't
this
there's
four
equally
valid
orientations,
and
how
would
I
know
which
one
so
I'm
wondering
I'm
wondering
marcus
is
in
trying
to
preserve
your
hypothesis,
but
the
hypothesis
would
be
that
the
rat
is
the
it's
the
things
it's
observing.
The
chunks
are
not
the
anything
part
of
the
room,
because
there's
nothing
in
room
to
suggest
that,
but
it
might
be
something
the
rat
is
observing
overhead.
A
C
A
B
Yeah
that
that
last
general
idea
is
definitely
promising,
and
I
think
some
people
sort
of
think
that
way
that
that
the
map
is,
I
mean,
there's
there's
literally
a
grid.
There's
literally
a
paper
called
like
the
grid
cell
map
is
attracted
to
goals
something
along
those
lines,
and
the
idea
is
that
it
gives
more
spit
more
good
cell
space
to
places
that
it
is
more
often,
though,
that
that
just
starts
to
seem
a
lot
less
metric.
That's
just
different.
A
A
Well,
it's
it's
sort
of
like
well
we're
partially
there
there.
You
can
partially
explain
some
mechanism
here,
but
it
doesn't
it
really.
It
almost
suggests
that
something
else
is
going
on,
because
there's
some
other
there's
some
other
thing
which
is
leading
to
the
distortions
other
than
the
idealized
objects
versus
real
objects,
something
causing
these
distortions,
as
they
have
shown
here.
C
B
This
is,
this
is
a
good
question.
I'd
say
I'd
say,
like
maybe
the
worst
part
of
this
of
this
theory
mine,
the
the
idealized
map.
One
is
that
it
can
be
used
to
prove
any
firing
fields.
I
could
come
up
with
any
weird
shape
to
prove
just
about
anything
and
so
it's
hard
to
falsify
which
doesn't
make
it
wrong,
but
it
doesn't
make
it
right.
It's
just
what
a
issue
there.
I
don't
have
a
good
answer
to
that
other
than
it's.
What
we're
doing
right
now?
B
B
Maybe
there's
this
other
thing
where,
like
the
amount
of
time
you
spend
in
a
certain
part
of
the
room,
is
given
preference
and
then
so
maybe
it's
a
connector
to
a
second
part
of
the
theory,
but
either
way.
That's
that's
the
way
this
takes
on.
I
you
can't
get
something:
that's
immediately,
falsifiable
in
a
in
a
in
an
automatic
way.
A
You
know
going
back
to
your
original
proposal
like
oh
well,
we
build
up
these
rooms
of
these
idealized
objects.
In
some
sense,
I
feel
like
it
has
to
be
true.
No,
it
does
have
to
be
true.
I
mean
everything
we
know
is
some
sense,
an
idealized
version
of
it
there's
more
detail.
I
think
think
of
any
object
than
oh
it's.
The
chair,
I'm
still
looking
next
to
me
here.
A
A
It's
not
it's
not
an
accurate
portrayal
of
the
chair
and,
and
so
that's
like
a
a
given,
and
I
I
think
the
idea
that
when
we
when
we
enter
a
new
environment
or
we're
exploring
something
new,
we
will,
if
there's
things
in
it,
that
sort
of
familiar
we'll
just
assume
it's
like
the
one
we
had
before,
and
so
I
think
that
idea
almost
has
to
be
true.
I
don't
know
if
it
has
to
lead
to
good
cell
distortion.
A
That's
a
that's
a
separate
question
so,
but
I
think
this
picture
here
said
that
the
grits
of
the
sources
are
more
complicated
than
that.
There's
something
else
going
on.
You
know
there's
another
distortion,
too.
We've
talked
about
a
lot
which
we,
I
think,
still
begs
explanation.
I'm
gonna.
Throw
it
in
just
for
completeness
is
that
we
know
that
grid
cells,
don't
always
fire
when
we
expect
them
to,
and
they
do
so
in
a
repeatable
way
right.
So
they'll
have
missing.
A
A
A
A
B
A
Don't
think
you've
succeeded
in
that?
No,
I
don't
think
so
because
you've
given
us
a
square
box
which
I
would
have
said,
okay,
the
square
box
is
not
going
to
have
any
distortions,
because
that's
the
most
idealized
thing
I
could
possibly
have
and
they're
like,
and
there
we
see
all
these
distortions.
I'm
like
holy
crap.
You
know,
because
in
the
beginning,
you're
showing
the
distortions
near
a
corner
or
near
the
mountain
range
or
the
corners
in
your
paper.
Okay,
I
got
that.
A
Apparently
it
is
a
puzzle,
so
I
think
I
mean
we've
come
up
with
two
hypotheses
and
why
that
is
I
mean
there's
something
has
to
lead
to.
I
mean
there's
something
in
the
environment
which
is
leading
just.
I
assume
that
this
is
a
consistent
grid
that
this
result
is
consistent,
that
these
distortions
of
the
grid
cells
are
not
moving
around.
A
B
C
A
B
A
That's
weird:
I
was
going
to
say
there.
Okay,
let's,
let's
remind
you,
there
has
to
be
some
information
to
tell
the
rat
its
orientation
in
this
room
and
to
tell
the
rat
why
the
room
should
be
distorted
the
way
it
is
now.
Maybe
the
orientation
is
sufficient
for
that,
but
but
there
has
to
be
some
information
to
tell
them.
So
as
the
problem
is
posed,
there
isn't
the
information
provided.
There
must
be
something
else,
whether
it's
there's
generally,
the
orientation
is
determined
by
environmental
features
which
are
not
shown
here.
A
C
A
Simultaneously
to
the
orientation
which
is,
there's
got
to
be
something
which
breaks
the
symmetry
here
and
we
don't
know
what
that
is.
So
I
think
we
just
don't
have
enough
information
here,
there's
more
information
that
these
two
good
cells
are
being
based
on
than
we
know
and
we're
not
presented
with.
B
That
data
yeah
yeah,
so
my
my
final
slide
is-
is
nothing
new,
but
basically
this
left
side
is
an
elegant
explanation
for
distortions.
At
least
I
mean
it
sounds
good
and
it
may
explain
some
distortions
if
it's
correct,
then
a
rat's
map
for
this
environment
probably
looks
like
this,
but
basically
this
is
a
big.
If
and
that's
that's
where,
or
rather
if
this
is
the
explanation
for
these
distortions,
it
means
this.
C
A
I
think
it's
very
somewhat
successful,
but
yeah
and
I
think
interesting,
but
I
think
what
was
successful
is
bringing
up
a
lot
of
issues
that
and
clarity
on
the
problem,
because
it's
there
are
more
constraints
now
that
have
to
be
solved
than
before,
but
we
have
a
clearer
image.
Well,
I've
learned
a
bunch
of
things
already
I
didn't
know
there
was
these
distortions
in
square
environments
like
that,
I
thought
the
destructions
came
about
when
you
put
barriers
in
or
you
morph
the
room
or
things
like
that.
You
know.
A
A
I
think
I
ordered
two
weeks
ago
and
talked
about
the
idea
that
the
thalamus
might
be
doing
transformations
using
orientation
for
both
absorb
feature
and
the
movement
vector
and
that
that
could
could
fit
within
the
thalamic
relay
cell
model
that
that
subtitle
civilized
I'm
working
on
and
then
but
I
I,
but
there
were
some
problems
with
that
mom
I,
the
problems
with
the
physiology
doesn't
necessarily
match
up.
So
we're
we're
talking
about.
What's
your
name
about
it,
I
can't
remember
anybody
herman.
A
And
there's
another
problem
with
it
too,
and
the
other
problem
is
there's
a
lot
of
reference
frame
transformations
that
have
to
occur
off
many
of
them
now,
when
we
think
about
this,
we're
thinking
mostly
about
the
hippocampal
complex
system
and
and
there's
tons
of
things
going
on
there,
where
you
know
these
object,
vector
cells
is
there's
ego
object,
vector
cells,
there's
motion
cells
with
the
velocity
cells
and
border
cells.
There's
boundary
cells
use
all
these
things
in
both
ego
and
allocentric
spaces,
and
and
so
there's
all
these
transformations.
A
Now,
when
we
get
to
the
cortex,
we
don't
know
if
all
those
things
have
to
occur
and
and
and
and
they
can
have
to
work
on
different
principles
right
so
so
we
we
don't
have
to
assume
everything
is
going
on
over
in
the
hippocampal
complex,
nothing
in
the
cortex,
but
the
cortex
must
solve
these
problems
and
it
has
to
solve
some
of
them,
at
least
in
some
cases,
on
a
column-by-column
basis.
We
think
about
somatosensory
cortex
eats
finger
or
eats
patchy
skin.
It
can
have
a
different
orientation.
A
That's
not
true
with
division,
so
you
know
there
might
be
a
different
solution
for
vision,
but
but
on
a
column-by-column
basis,
it's
almost
so
small
patch.
You
have
to
be
able
to
solve
these
reference
transformations
of
touch
for
nothing
else,
so
it
occurred
to
me
and
then
we
and
then
you
read
papers,
people
saying
oh
like
okay,
so
I
have
I
have.
I
have
to
do
a
reference
transformation
from
my
retina
to
my
head
right,
so
my
head
centric
breath
and
centric
over
my
eyes
to
my
hands
or
my
head.
A
A
It's
limited.
If
there's
only
it
can
only
do
a
few
transformations
that
there's
a
whole
ton
of
transformations
that
have
to
occur
and
it
doesn't
seem
like
the
thalamus
is
in
a
position
to
do
them
all
it's
just.
You
can
only
do
the
ones
that
go
through
the
thalamus
and
it
seems
to
be
a
lot
of
other
transformations
that
have
to
occur.
That
probably
wouldn't
go
through
the
thumb.
A
So
I
then
said
to
myself:
well
what
other
mechanisms
could
achieve
this,
and
we
have
that.
We
talked
about
this
basic
idea.
We
have.
I
figured
we
had.
The
figure
of
you
know
this
f2
at
two
dimensions
and
there
was
a
n
n
by
n
and
we
had
n
square
these
conjunctive
cells
in
between
and
so
and
then
we
talked
about
whether
it
didn't
have
to
be
n
squared.
If
you
recall
it
could
be
less
than
n
squared,
we
could
reuse
some
of
those
components
and
so
on
so
it
occurred
to
me.
A
A
So
the
mini
columns
represent
the
feature
and
it's
a
egocentric
space,
and
now
I
have
an
orientation
and
I'm
going
to
pick
a
single
cell
to
be
active
in
each
of
the
columns
that
are
active
in
the
representation
and
so
those
are
conjunctive
cells.
Those
are
the
cells
that
are
this
feature
at
this
orientation,
and
I
could
read
that
out.
At
that
point,
the
active
cells
would
be.
It
would
uniquely
represent
that
feature
at
a
different
orientation.
I
mean
it's
a
yeah,
it's
a
different
orientation.
Does
that
make
sense?
C
A
A
We
think
about
a
section
of
cortex
and
it's
got
all
these
mini
columns
and
it's
going
to
be
anywhere
from
a
few
hundred
to
five
thousand,
depending
on
the
size
of
the
column
and
and
now,
but
that's
not.
The
minicom
is
not
uniform,
there's
a
bunch
of
cellular
layers
in
it
a
whole
bunch
of
different
types
of
cellular
layers
and
if
we
just
take
a
very
rough
approximation,
say
well,
each
cellular
layer,
let's
say
behave
like
our
temple
number.
Each
cellular
layer
has
a
set
of
cells,
maybe
eight
or
ten
twelve,
something
like
that.
A
A
A
I
have
to
do
a
pooling
operation,
because
the
the
current
set
of
conjunctive
cells
are
only
correct
for
one
particular
feed
orientation,
a
different
orientation,
the
the
the
egocentric
representation
would
be
different
and
have
a
different
set
of
cells
acting.
So
if
I
were
taking
an
object
and
rotating
it,
you
know
the
egocentric
representations
are
changing,
but
the
alicentral
representations
wouldn't
be
changed.
A
So
I
have
to
pull
those,
and
so
that's,
like
you
know,
our
basic
pool
and
pooling
operation
where
we
would
have
to
say:
okay,
there's
a
stable
representation
and
we're
trying
to
map.
That's
like
what
we
did
in
the
temple
memory,
and
so
the
questions
could
this
become.
You
know
it.
Could
this
be
a
principle
applied
throughout
the
cortex?
A
So
I
have
an
object,
vector
cell
representation,
I'm
going
to
represent
it
by
some,
so
many
columns,
somehow
I'm
still
doing
another
transformation,
I'm
trying
to
convert
it
into
another
representation
which
is
stable,
while
one
representation
is
changing
and
there's
going
to
be
some
something
I
have
been
used
to
do
that
with.
I
haven't
figured
how
to
do
that
yet,
but,
but
there
might,
this
might
be
a
very,
very
generic
idea
that
you
just
say.
A
Yeah
I
I
did
well
like
the
game
field.
I
never
liked
game
fields
for
something
I
just
have
to
think
about
something,
but
but
I
realized,
then
we
just
a
couple
weeks
ago,
we
said
they're
kind
of
just
like
conjunctive
fields,
right
and
they're,
just
like
our
and
I
and
I
realize
these
are
all
the
same.
I
don't
think
there's
a
distinction.
So
so
I'm
not
looking
at
the
game
field,
literature
other,
I'm
just
sort
of
putting
it
aside.
I
think
this
has
got
to
be.
A
This
is
probably
the
same
thing
yeah
and
but
but
but
I
think
it
could
be
more
generic
than
just
orientation.
There
could
be
any
there
could
be
that's
what
I'm
trying
to
do
like
displacement
cells.
This
way
too
anyway,
it's
on
there
might
be
an
uber
theory
here
of
of
processing
which
of
course,
as
a
theorist
you'd
like
to
find
those
right.
Ideally
it
doesn't
even
exist
which
could
explain
a
whole
bunch
of
stuff,
and
our
temporal
memory
in
some
sense
is
one
example
of
this.
A
The
temporal
memory
as
and
we've
played
a
point
now
many
times,
it's
it's
it's
another
version
of
this,
and
and
it's
just
solving
a
different
type
of
problem
anyway.
So
that's
the
idea
working
on
the
thing
which
is,
which
is
hanging
me
up
a
little
bit
and
and
marcus
you
might.
You
might
help
me
think
about
this
is
if
I
think
about
how
many
things
I
can
represent.
A
Like
let's
say
I
want
to,
I
want
to
go,
I
walk
into
a
room
and
I'm
trying
to
I'm
seeing
a
new
object.
I'm
trying
to
build
up
I'm
looking
around
and
my
visual
cortex
is
trying
to
build
up
a
model
of
this
new
object.
Okay
and
the
new
object
has
all
these
different
parts.
Let's
say
it's
a
bicycle
right:
it's
got
all
these
parts
and
the
parts
are
complicated.
A
These
aren't
simple
parts,
you
know
the
sprockets
and
chains
and
pedals
and
crankshafts
and
tubes,
and
you
know,
seeds
and
springs
and
wires
and
stuff,
and
so
by
looking
around
the
bicycle,
I
can
see
all
these
components.
That
means
that
the
input
to
a
column
has
to
be
able
to
look
recognize
a
lot
of
different
components.
You
know
the
your
your
your
basic
building
blocks
right,
and
so
I
have
to.
A
In
my
space
report
a
lot
and
when
I
and
when
we
think
about
like
object,
vector
cells
or
like
v1
response
properties,
there
aren't
a
lot
of
things.
It's
like
there's
a
you
know
this
system,
just
some
you
know
10
different
orientations
or
something
as
an
object.
Like
yourself,
I
don't.
I
don't
know
what
it
is.
I'm
just
I'm
worried
about
capacity.
A
A
C
Weakness
with
that,
the
that
sort
of
learning
I
mean
there's
tons
of
these
reference
strength
transformations
you
have
to
do
and
every
object
has
its
own
kind
of
scale
and,
and
you
have
to
learn
all
of
these
relationships
and
it
you
have
to
learn
everything
sort
of
independently
and
it's
you
need
you
would.
It
would
be
very
time
consuming
and
you
need
a
lot
of
examples.
But
I
think
that
if
you
combine
that
basic
idea
with
the
stretchy
maps
idea,
it
may
become
really
feasible
and
efficient,
because
you
know.
A
A
It
started
me
to
think
about
hierarchy
a
bit,
because
the
output
of
a
column
would
would
be
a
you
know,
would
be
a
complex,
a
representation
of
a
complex
object,
and
so
now
that
becomes
the
input
to
the
next
columns.
So
I
have
a
whole
bunch
of
complex
objects
that
I'm
now
using
the
building
blocks
for
the
next
column.
A
Is
we
still
have
our
scale
issue?
I
mean
there's
only
several
things
we
have
to
solve.
We
have
to
solve
orientation,
we
have
to
solve
like
location
displacements
and
orientation
and
then
scaling.
You
know,
because
this
gun
is
another
issue
and
like,
like
you
know,
how
big
is
this
particular
object
or
how
far
away
is
it?
So
how
much
do
I
have
to
move
my
eyes
to
attend
the
different
parts
of
it?
So
it
might
be
that
the
thalamus
is
dealing
with
scale.
A
Three
is
the
first
place
that
pools
and
now
we
then
layer,
three
would
have
the
aloe
centric
reference
ring,
yeah
and-
and
but
you
know
so,
I'm
I'm
just
thinking
through
fit
function.
So
that's
it.
I
have
nothing
else
to
report,
but
it
sounds
like
hey.
You
already
thought
of
something
similar
to
this
before
the
nick
one
and
and
we've
talked
I
know,
we've
talked
in
the
past
about
the
the
temple
pooling
algorithm
sort
of
being
more
of
a
generic
type
of
thing.
A
A
Anyway,
well
it
is,
but
you
know
additionally,
if
you
think
about
it,
though
I
what's
different,
is
I
don't?
I
think
it.
The
flux.
Capacitor
suggests
that
they're
and
in
the
game
field
or
the
way
it
wasn't,
that
broken
burgers
purchase
paper.
They
they
complied
those
these
conjunctive
cells
in
the
middle
and
you
went
through
them
both
ways.
A
But
I
don't
think
that's
the
case
here.
I
think
it's
one
directional,
it's
more
like
a
triangle
or
something
like
that,
you
know
or
separate
arrows
like
I
can
go
from
a
a
layer,
four
representation,
a
mini
column,
space
of
a
egocentric
feature.
You
can,
then
the
individual
cells
in
layer
four
become
the
conjunctive
features,
and
then
I
pull
someplace
else,
but
I
can't
go
back
through
those
same
cells.
If
I
want
to
go
back,
I
have
to
go
the
layer.
A
Three
cells
would
have
to
have
their
own
conjunctive
cells,
which
then
project
back
to
something
else.
You
know
what
I'm
saying
it's
like.
You
can't
reuse
those
cells.
You
can't
go
two
ways
through
them,
which
is
one
thing.
I
always
bother
me
about
the
diagrams.
Like
the
cancer
purchase
paper,
it
cells,
don't
do
bi-directional
stuff.
So
so
it's
a
little.
It's
like
the
flux
capacitor
with
some.
I
don't
know
we
have
to
come
up
with
a
better
metaphor.
A
Also
it
doesn't
as
we
you
and
I
talked
personally
marcus
last
week.
It
doesn't
you
can't
use
it
to
resolve
orientation
like
orientation
is,
is
there's
three
variables
you're
trying
to
solve
it's
the
ego,
centric
and
allocentric
orientation,
but
you
can't,
I
don't
believe
you
can
use
it
to
resolve
the
orientation
that
is,
the
orientation
has
to
be
given
and
you
can
go
between
the
other
two,
but
I
don't
think
you
can
say.
Oh,
I
know
my
alicentral
and
my
egocentric.
I
can
now
resolve
to
the
orientation.
B
C
A
So
I've
been
thinking
about
this
every
night
in
my
sleep
for
the
last
few
days,
and
I
haven't
I'm
still
stuck
on
fire
speeches,
but
I'm
excited
about
it,
I'm
working
on
it.
I
had,
I
think,
there's
at
least
there's
a
possibility
that
a
very
big
uber
principal
come
out
of
it.
They
could
find
a
lot
of
stuff.
B
Further
sure,
one
quick
one
thought
on
the
other
thing
you
mentioned
in
passing
that
maybe
thalamus
is
doing
scaling
just
to
put
something
probably
back
on
your
radar.
We
haven't
talked
about
this
before,
but
I'm
guessing
you've
seen
it
so
bruno
olsausen
before
his
sparse
coding
stuff.
One
widely
cited
paper
he
has
from.
I
think
93
involves
scaling
of
the
visual
input,
but
he
doesn't
do
it
in
thalamus.
He
does
it
across
levels
of
hierarchy.
Rather
he
does
it.
B
C
A
Yeah
I
asked
him
about
it:
yeah,
that's
right,
but
but
here's
you
know,
but
but
scaling
is
very
much
involved
in
attention
like
attention.
We
know
that
the
we
know
that
the
thalamus
is
involved
in
attention.
There's
a
whole
bunch
of
papers
on
that,
but
they're
all
kind
of
wonky,
but
but
the
idea
that
to
attend
to
something
it's
it
could
in
a
sense.
In
some
sense
you
could
say:
okay,
I'm
moving
my
gaze
towards
it
or
you
know.
I
move
my
eyes.
A
A
A
You
basically
have
to
attend
to
something
to
know
how
to
scale,
but
like
okay,
I'm
gonna
manipulate
this
from
a
distance.
When
I
move
my
eyes
different
amounts
or
I'm
going
to
you
know,
I
have
to
move
my
fingers
different
amounts.
The
whole
object
model
has
to
scale
I
need
to
attending
to
it
means
I
am
now
going
to
figure
out
the
scaling
factor
to
manipulate
so
that
kind
of
fits.
It
also
fits
in
the
idea
that
I've
always
had
the
matrix
cells
during
scaling
of
sequences
and
time.