►
From YouTube: Jeff Hawkins on Object Modeling in the Thousand Brains Theory (Part Two) - September 9, 2021
Description
In continuation to last week’s meeting, Jeff first shares his new ideas about voting. We used to think that voting between cortical columns only communicates object ID of the things you attend to, but now he hypothesizes that at a minimum, voting communicates object ID, object state and location / orientation relative to body.
Jeff then describes what a model (created by cortical columns) is and the characteristics of a model introduced in our previous papers. As our knowledge continues to expand, our understanding of what a model is has also evolved. We used to think that models in a column were based on grid cell metric reference frames, but now we deduce that vector cells are involved too. We hypothesize that models are represented using vector cell modules, not grid cell modules as we previously thought. And grid cell reference frames are used in determining movement from one location to another. The team then asks questions and discusses.
➤ Columns paper: https://numenta.com/neuroscience-research/research-publications/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world/
➤ Frameworks Paper: https://numenta.com/neuroscience-research/research-publications/papers/a-framework-for-intelligence-and-cortical-function-based-on-grid-cells-in-the-neocortex/
Other papers mentioned:
➤ “Neuronal vector coding in spatial cognition”: https://www.nature.com/articles/s41583-020-0336-9
➤ “Population coding of saccadic eye movements by neurons in the superior colliculus”: https://www.nature.com/articles/332357a0
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
So
I
have
a
lot
of
material,
unfortunately,
but
the
good
news
is,
I
didn't
finish
my
slides,
so
we're
going
to
get
through
most
of
it,
but
I
won't
get
to
the
end
of
it,
but
I'll
probably
be
about
the
right
amount
of
material
and
in
some
ways
this
is
a
continuation
of
what
we
were
talking
about
earlier
in
the
week,
but
in
some
ways
it's
a
new
material
too.
I've
been
very,
I
feel,
like
I've,
been
very
productive
mentally
this
last
couple
weeks.
A
A
This
is,
you
know,
we're
talking
about
the
our
our
roadmap
and
we
want
to
get
to
the
you
know,
reference
frames-
and
I
mentioned
yesterday-
you
know
when
I
was
in
the
office
that
really
what
we
want
to
get
to
is
not
reference
frames.
We
want
to
get
to
more
object,
modeling,
you
know
structured
modeling
of
objects,
and
so
most
of
this
talk
will
be
about
that.
What
does
that
mean
to
build
a
model?
A
And
it's
going
to
be
all
phrased
in
terms
of
neuroscience,
unfortunately,
and
but,
but
that's
not
because
we
want
to
implement
the
neuroscience
just
where
we
are.
That's
where
I
think,
and
so
when
I
go
through
this,
it's
not
like
these
are
all
things
that
are
important,
that
we
have
to
implement
from
our
machine
learning
point
of
view,
and
it
doesn't
mean
we
have
to
do
it
this
way
at
all.
A
B
A
A
A
So
this
is
not
about
the
mechanisms
about
voting,
but
some
more
ideas
about
voting,
so
new
ideas
about
voting.
I'm
going
to
talk
this
one
slide
on
that.
I
think
there's
a
to
me.
Some
of
these
ideas
are
really
actually
pretty
exciting,
but
they're
just
ideas
at
this
moment
and
then
I
will
switch
completely
we'll
talk
about
object
and
modeling
of
objects,
so
new
ideas
about
voting.
This
is
how
we
typically
show
it.
A
We
show
that
there's
like
oh
these
voting
neurons
on
layer,
two
three
that
go
between
columns,
that's
true,
but
there
is
another
set
of
voting
neurons.
We
don't
talk
about
it
too
often,
but
it's
very
well
documented.
Well,
they're
not
documented
voting
neurons.
There
are
other
sets
of
neurons
in
layer,
five
that
do
the
same
kind
of
long-range
projections
as
the
cells
in
layer
two
three,
so
these
long-range
rejections
are
well
documented,
not
that
anyone
else
has
any
idea.
A
What
they're
doing-
and
I
interpret
this
as
there's
actually
at
least
two
populations
of
cells
that
are
voting
because
in
layer,
2,
3,
there's
different
cell
populations,
they
may
be
voting
on
more
than
one
thing.
There
may
be
two
different:
you
know:
cell
populations
are
voting
on
two
different
things
and
there
are
fibers.
At
least
there
are.
There
are
two,
maybe
three
cell
types
in
layer:
five,
depending
how
you
count
them
and
so
they're
going
to
be
at
least
doing
something,
and
so
I
assume
those
are
voting
as
well.
A
They
have
the
same
kind
of
anatomical
connections.
As
you
see
in
layer,
two
three
are
very
similar.
So
when
we
say
these
voting
neurons,
remember
we're
saying,
like
a
particular
type
of
cell,
let's
say
on
layer,
five
is
connecting
to
the
very
same
particular
type
of
cell
in
columns
and
other
layer
fives,
so
they're
all
voting
on
the
same
thing.
A
A
It's
not
it's
not.
There
is
still
information.
That's
passed
between
regions!
That's
more
of
this
hierarchical
connections
like
between
layer,
two
three
and
layer
four,
but
the
idea
is
that
what
you're?
Conscious
of
is
these?
This
sort
of
spread
of
this
large
spread
of
activity?
This
stable,
there's
a
very
fast
communication
system?
It's
saying
large
parts
of
the
brain
are
all
be
exposed
to
the
same
thing
and
they
can
all
know
about
it,
and
this
is
really
interesting
because
we've
talked
about
voting
in
terms
of
object
id.
A
A
I
perceive
that
it's
a
coffee
cup
I
perceive
its
pose
relative
to
my
body,
so
I
know
where
it
is
and
its
orientation
to
me.
You
know
the
coffee
cup
is
at
a
particular
position
at
a
particular
orientation
to
me
and
if
the
object
has
some
sort
of
state
like
the
stapler,
I
also
perceive
the
state.
I
I
see
perceive
the
staple
as
open
or
closed,
or
this
app
is
open
or
that
app
is
open,
and
so
these
these
are
the
stable
representations.
A
These
are
the
things
that
you
know,
as
my
eyes
are
moving
and
I'm
looking
at.
You
know
something
the
pose
the
object
id
it's
it's
positional
to
my
body,
whatever
state
it
is,
those
are
all
stable
and
and
therefore
I
assume
that
those
are
voting,
those
are
being
voted
on.
It's
not
just
the
object
id.
Otherwise
I
just
have
some
sort
of
you
know
ethereal
idea,
there's
a
coffee
cup
out
there.
No,
I
know
where
it
is.
A
A
So
there
might
be
other
things
that
are
voting
on,
but
I
use
this
sort
of
conscious
perception
stability
of
perception
as
sort
of
the
metric
right,
because
there's
lots
of
things
we
don't
perceive
things
that
are
going
on
in
the
columns,
we're
not
perceiving
our
eyes
moving
we're
not
perceiving
the
changing
inputs
from
the
retina,
but
the
stability
of
our
perception
is
sustained
so,
and
this
is
whatever
you
attend
to
so
I
tend
to
a
chair
to
my
left
right
now.
I
see
this
particular
position
orientation
to
me.
B
About
yeah
yeah:
what
about
motor
actions
like
I'm
aware
that
I'm
about
to
reach
and
touch
my
laptop
screen
or
whatever
you
know
I
can.
I
can
consciously
think
about
that
and
make
that
movement.
You
know
that
that
you
know
when
we
have
to
make
a
motor
action.
I
could
imagine
there's
a
voting
process
involved
in
that
as
well
yeah.
I.
A
Think
you're
probably
right-
and
this
is
why
I
was
careful
to
say
at
a
minimum-
although
the
motor
action
could
be
handled
a
couple
of
different
ways,
I
could
clearly
imagine
that
opening
the
stapler
and
I
could
imagine
in
its
open
state
and
if
I
imagined
it
and
it's
open
and
said
I
might
be
sharing
that
so
I
think
you're
right.
I
I
want
to
be
careful
here.
A
B
A
Yeah,
I
think
that's
that's
right,
I
mean
well.
If
the
motor
command
is
to
move
to
yeah,
I
think
that's
a
little
confusing
because
it
is
yeah
yeah.
Let's,
let's
put
that
and
file
that
away
as
like.
Let's
think
about
that,
okay,
can
we
do
that
sure
yeah
we
haven't
really.
You
know,
we've
never
addressed
at
all
the
idea
of
coordinated
motor
motor
movements
between
columns.
We
haven't
really
done
that.
You
know
I
could
say.
A
Oh,
I
want
to
reach
for
the
coffee
cup,
but
how
is
it
that
all
the
different
joints,
and
so
on?
We
do
that,
and
so
we
just
haven't
done
that
at
all,
and
I
don't
want
to
go
there
today,
but
you're
right.
As
I
said,
other
things
should
be
voting
on.
It's
not
a
you
know,
it's
it's
a
limited
number
of
things.
It
can't
be
too
many.
I
mean
I
could
argue
that
I
I
could
maybe
come
up
with
five
possible
sets
of
neurons
that
are
voting.
A
It
might
be.
Just
two
might
be.
Three
could
be
four,
but
you
know
given
what
I
know
that
the
anatomy
I
could
maybe
imagine
five
meaning
separate
cell
populations
that
are
reaching
consensus
about
different
things.
We
do
know
that
the
actual
motor
cells
that
generate
actual
moments
are
also
in
layer
five,
but
they
don't
seem
to
do
this
kind
of
long-range
connection,
so
they
don't
seem
to
be
doing
this
sort
of
voting
so
but
the
this,
the
pyramidal
cells
next
to
those
motor
cells,
do
have
these
long-range
connections
and
they're.
Fine.
A
The
first
one
is
labeled
train
across
modalities.
We
have
used,
as
for
those
who
are
new
here,
you've
heard
us
joke
about
that.
We
call
this
disjoint
pooling
and
then
marcus
recently
said.
Why
don't
we
just
get
rid
of
the
term,
but
we
used
it
a
lot.
It's
it's
a
type
of
knowledge
transfer
and
it
could
be
very
simply
described
as
the
following
I
reach
into
a
a
box.
I
feel
something
with
my
index
finger
and
I
learn
three
different
objects.
A
Using
my
index
finger,
I've
never
seen
these
objects,
but
I
felt
them
with
one
finger
now.
I
can
reach
into
that
same
object
and
use
a
different
finger
and
infer
each
object.
So
how
is
that?
I've
trained
my
index
finger
column
on
the
object
model,
but
now
I
can
use
my
pinky
and
my
pinky's
never
been
exposed.
This
is
how
the
columns
and
the
pinky
learn
this
model.
How
do
they
know
how
to
infer
it,
or
I
could
go
with
my
left
hand
or
another
way
thing
about
it?
A
A
A
So
and
that's
what
you
perceive
you,
you
remember
my
perception.
If
I
reach
or
see
something
it's
where
the
the
object
is
relative
to
my
body,
so
assuming
that's
being
voted
on,
then
a
lot
of
columns
are
being
exposed
to
hey.
There
is
a
an
object
at
some
location,
pose
relative
to
your
body
and
in
some
state,
and
even
if
I'm,
a
visual
cortex,
a
visual
column,
we'll
be
getting
that
from
my
my
finger
and
the
visual
column
says.
Well,
I
know
what
that
object
is.
I
know
what
an
edge
is.
A
I
know
what
a
candle
is.
You're
telling
me
that
there's
a
handle
here
at
this
position
and
next
to
it,
there's
a
rim
and
next
to
it,
there's
a
cylinder
and
so
on.
I
can
visualize
that
as
you're
touching
it
with
your
finger,
and
so
literally
when
you
touch
something
with
your
finger
in
a
black
box,
you
can
visualize
what
it
looks
like.
How
is
that
happening?
A
It's
the
voting,
I'm
virtually
certain
of
this,
the
voting
they're
into
sending
this
around
saying,
hey,
I'm
a
finger
and
I'm
sensing
these
objects
at
these
positions,
and
these
you
know,
orientations
and
everybody
else.
Who
knows
those
sub-objects
can
say
yeah.
I
can
visualize
that
too,
and
if
I
can
learn
that
same
model
as
you're
learning
it,
I
don't
have
to
see
it.
I
can
just
you
can
tell
me
about
it
and
I'll
build
up
this
model.
A
A
They
consist
of
things
like
this.
They
consist
of
what
you've
perceived
at
different
positions
at
different
orientations
in
different
states
relative
to
your
body.
So
and
it's
very
it's
the
specific
episodic
memories
are
relative
to
your
body:
they're,
not
generic
memories
like
a
model
of
something
so
I'll,
say
yeah.
Where
was
I
sitting
when
I
had
my
muffin
this
morning?
Oh,
I
remember
where
I
was
sitting
but
chair
was
in.
Where
was
the
muffin
on
the
counter?
Next
thing
was
it
to
my
left
is
my
right.
I
remember
all
these
things.
A
A
A
C
I
have
a
brief
question
yeah.
This
might
be
kind
of
dumb,
but
is
this
voting
also
responsible
for
reconciling
differences
between
two
objects
of
the
same
kind?
So
when
you
see
a
coffee
cup,
the
first
thing
that
you
don't
think
of
is
that's
a
coffee
cup.
You
think
of
that's
my
coffee
cup
versus
someone
else's
yeah.
A
Well,
let's
assume
you've
built
the
model
of
your
coffee
cup
right.
You've
already,
you've
already
learned
your
coffee
cups,
yeah,
okay,
so
no
object
id
in
this
case
obvi
is
is
includes
that
specific,
like
you
know
my
I
have
a
coffee
cup
next
to
me
right
now.
It's
a
green
one,
and
I
know
this
coffee
cup
I
use
at
home
all
the
time,
and
I
have
I
have
a
specific
model
of
that.
I
don't.
I
don't
need
the
different.
A
I
mean
I
have
a
model
of
that
cup
and
and
I
have
models
of
other
cups,
but
my
method,
coffee
cup,
is
a
different
model,
so
I
don't
need
voting
to
do
that.
Those
are
just
unique
object.
Ids
if,
if
it's
something
I've
never
seen
before,
if
it's
a
new
coffee
cup
that
I've
never
seen
before
well,
then
I
can't
form
a
stable
representation
of
it
right
away
because
it's
new
right
and
so
what
you
will
do,
then
is
you'll
you'll
attend
to
the
individual
components
of
that
new
coffee
cup.
Maybe
it's
square!
A
A
I
said
I
I
think
it's
very
clear
to
me
that
you
know
that's
the
way
the
world
is
right.
You
you
look
around
and
you
when
you
attend
to
things.
You
always
intend
you
always
narrow
down
your
focus
until
something
you
you
recognize
is
something
you've
learned
before
and
then
that's
what's
going
to
be
voted
on,
and
so
I
can.
I
can
encapsulate
an
entire
coffee
cup
as
one
thing,
if
I,
if
I've
learned
it,
but
if
I
haven't
learned
it,
I
will
transmit
as
a
series
of
sub-objects
at
different
locations.
E
Did
yeah
I
wanted
to
talk
about?
I
guess
your
thoughts
on
the
difference
between
the
sort
of
long
range
voting,
that's
going
on
in
layer,
two
three
and
what's
happening
in
layer,
one
as
something
akin
to
like
a
global
signal,
because
when
I,
when
I
think
of
yeah,
I
guess
I
just
wanted
to
hear.
What
do
you
think
is
the
difference
between
those
two
things
right.
A
It's
a
good
question,
so
layer,
one
is
very
unusual
for
those
who
don't
know
it
has
very
few
cells.
It
has
cells,
sometimes
like
people
describe
it
as
no
cells.
That's
not
true.
It
has
cells,
they're,
very
weird,
looking
cells,
and
but
it's
mostly
just
connections
you
could
think
of
like
the
white
matter,
underneath
the
cortex
below
layer,
six
is
just
connections,
that's
what
the
white
matter
is
wire
is
going
back
and
forth
all
over
the
place
and
in
layer
one.
A
A
So
a
typical
example
will
be
there's
these
cells
in
the
thalamus,
these
called
matrix
cells
and
they
project
to
the
cortex
and
one
of
the
places
they
project
to
is
layer
one
and
they
spread
over
long
distances.
So
these
matrix
cells
are
going
to
send
a
signal
over
a
large
area
of
the
cortex.
Just
like
we
talked
about
the
voting
neurons
and
I
believe
those
cells
are
representing
timing
information.
A
A
So
that's
a
broad
picture.
I
think
a
difference
between
voting
and
something
like
the
matrix
cells
is
voting
is
always
between
cells
of
the
same
type
right.
I
have
this
layer,
5a
cell,
let's
say,
and
it
exists
in
all
my
columns
and
they
are
connecting
to
layer
5
8
cells.
That's
it
voting
is
between
like
and,
like
you
know,
we're
all
representing
the
same
thing:
we're
all
going
to
come
to
agreement
where
layer
1.
Is
it's
not
like
that
layer?
1
signals
are
spread
long
distances,
but
they're,
not
from
cell
type,
to
cell
type.
A
A
Medium,
but
it's
not
a
voting
medium
and
and
there's
a
lot
of
mystery
about
lay
one
a
lot
of
things
project
broadly
up
there
right.
Another
example
would
be
the
rhythms.
You
know
a
lot
of
the
theories
about
how
the
grid
cells
work
requires
these
theta
rhythms
and
some
of
the
theta.
The
theta
rhythm,
is
generated
in
the
thalamus,
and
so
you
know
I
don't
remember
what
we
know
about
this,
but
one
way
you
could,
you
could
distribute
that
would
logically
be
in
layer.
A
Well,
that's
a
good
observation,
although
you
know
I'm
what
I'm
what
I
start
off
by
saying.
Let's
just
talk
about
voting
and
I
can
deduce
that
voting
has
to
include
object.
Id
object,
state
and
object
pose
right.
I
can
just
conclude
that
that's
it's
there.
That's
voting
or
doing
that,
and
the
question
is
what
could
I
do
with
those
signals
and
when
I
talk
about
transfer
learning,
I
really
focusing
on
what
this
joint
disjoint
pulling
things
here,
whereas
like?
A
How
is
it
that
I'm
able
to
you
know,
learn
something
with
my
finger
and
then
recognize
it
visually
or
learn
to
put
my
left
finger
and
then
recognize
my
right
finger.
I
mean
that's
a
fundamental
problem
with
the
thousand
brain
theory.
We
knew
it
had
to
be
solved,
and
now
I
have
a
solution
to
it.
I
threw
in
the
language
stuff,
because
I
only
because
it
was
very
it's
very
suggestive.
As
I
said
it's
just
like.
Oh
you
know,
a
lot
of
language
is
describing
the
same
things
like
you
said.
A
Like
imagine
something
left
here
next
to
your
foot
and
imagine
there's
a
little.
You
know,
pin
cushion
there
and
you
know
you
can
visualize
this.
You
can
imagine
what
it
would
feel
like
and
so,
but
I'm
describing
physical
artifacts
that
you
know
in
relative
poses
to
your
body
and-
and
you
can
just
by
just
my
language,
I
instate
that
same
state
of
visualization
in
your
brain,
and
so
that's
just
feeling
like
well
voting.
A
That
sounds
a
lot
like
what
I
just
described
voting
is
doing
so
I
don't
I'm
not
saying
that's
it
for
language,
I'm
just
saying
that
it
seems
like
that's
a
big
component
of
it,
so
I
just
want
to
leave
that
as
a
it's.
A
again
this
this
that'll,
be
a
language
is
not
something
we
focused
on
a
lot,
but
but
it
seems
like
I
had
a
new
foot
in
the
door
on
it.
A
If
you
will
but
think
about
voting
and
and
you're
right,
there
may
be
other
things
too,
but
clearly
just
the
the
ability
to
visualize.
Imagine
any
a
sensory
perception
based
on
another
century.
A
real
sensory
perception
can
influence.
Imagine-Centric
perceptions
is
a
lot
of
what
language
is.
It's
just
recreating
states
in
your
brain
that
I
have
in
my
brain,
so
language
in
some
sense
is
an
extended
voting
system.
A
I
can,
I
can
imagine,
being
seeing
something
I
turned
that
into
language
just
describes
the
exact
same
things
that
my
neurons
are
voting
on.
Now
your
neurons
are
voting
on
it
and
and
or
they're
sending
the
symbol
around
they're,
not
voting
they're.
Just
sending
that
signal
around
saying.
Yes,
we're
all
imagine
you're
seeing
this,
you
know,
pin
cushion
next
to
your
foot
and
bingo
there.
It
is.
E
A
They're
not
not
voting
in
the
sense
they're
trying
to
reach
a
consensus,
but
the
mechanism
could
be
a
way
of
spreading
spreading
states
of
perception
right.
So
not
voting
your
brain's
not
voting
when
I
describe
something
visually
to
you,
but
you
could
be
using
that
same
mechanism
to
say,
like
hey,
there's,
no
bottoms
up
input
for
this,
but
given
our
horizontal
voting
connections,
we
can
all
reinstate
this
condition,
and
you
can
imagine
it.
A
Yeah
I
mean
we
have
to
sometimes
just
be
willing
to
accept
that
we
don't
understand
things
completely,
but
but
if,
but
again
I'm
really
excited
about
this,
because
it
is.
It
feels
like
a
very
deep
insight
to
me
that
that
you
know
in
some
sense,
language
is
just
a
way
of
voting
between
brains.
It's
it's!
It's
sort
of
the
same
mechanism,
so
all
I
have
to
do
is
get
my
perception.
A
G
B
A
Well,
I
think
they
wouldn't
change
the
stage
of
layer.
Four
layer,
four
is
the:
is
the
classic
input
layer
right
and
my
thinking
and
it
just
I'm.
You
know
this
is
just
my
speculation
is
vivian,
but
my
thinking
is-
and
I
think
most
neuroscientists
would
agree
that
layer
four
represents
reality
right.
It's
it's
like
this
actually
happened.
You
know
this
is
the
real
thing.
Remember
the
voting
neurons.
A
They
have,
they
have
this
contextual
input
and
they
have
a
bottoms
up
input
right.
So
when
I'm
talking
about
spreading
this
information,
there
is
no
bottoms
up
input.
It's
just
contextual
input,
and
so
we
can
reinstate
the
state
in
the
voting
neurons
from
contextual
input.
But
there's
no
there's.
No,
it's
not
fighting
with
the
there's.
No
bottoms-up
input
saying
no.
What
really
is
happening
is
x.
So
therefore,
this
column
really
can't
vote.
It's
just
being
given
a
contextual
input.
A
I
don't
think
it
would
invoke
the
layer
of
four
cells
to
become
active
because
that
that
would
mean
that
then
the
column
would
have
no
way
of
knowing
what's
real
and
what's
imagined,
you
know
we
kind
of
have
to
say
like
yeah.
We
have
we
have
this
hypothesis
that,
what's
out
there
is
the
following
thing,
but
if
you're
getting
layer
4
input,
then
how
would
any
neural
tissue
know
that
it's
not
real?
A
A
Okay,
it's
obviously
getting
input.
It's
representing
it
in
context!
That's
all
true,
but
when
you
start
extending
it
to
like
what
part
of
the
model
does
layer
four
represent,
I
don't
wanna
go
there.
Yet
I
wanna
I'll
come
back
to
that.
So
we
can.
We
can
invoke
the
model
in
the
column.
We
will
definitely
do
that
because
I
can
say
imagine
you're
looking
at
the
nomento
coffee
cup.
We
can
all
visualize
it
right
now.
You
all
have
that
model
in
your
head,
you're.
A
B
D
D
B
Maybe
I
should
point
out
also
that
the
exact
models
that
are
in
different
cortical
different
cortical
columns
are
not
going
to
have
the
exact
model
of
the
same
object
either
the
you
know
the
somatosensory
cortical
columns
going
to
be
quite
different
from
the
visual
cortical
columns.
They
just
need
to
be
able
to
associate
in
order
to
be
able
to
vote.
You
need
to
be
able
to
associate
these
representations
together.
A
Yeah,
that's
a
good
point.
I
could
say
I
could
say:
okay,
there's
a
a
rounded
edge
on
top
of
this
object
and
you
can
all
you
can
visualize
that
and
you
can
imagine
what
it
feels
like.
But
if
I
said
the
rounded
edge
was
yellow
you
couldn't
imagine
what
that
feels.
Like
you
know,
you
just
couldn't
do
that.
A
A
I
think
we
need
that
we're
gonna
go
into
like
the
basic
what
it
means
to
model
something:
okay,
okay
sounds
good
and
I
think
that
might
it
will
expose
more
questions
or
problems.
So
so
we
say
columns
learn
models,
but
what
is
a
model?
You
know,
what
is
it
and
our
understanding
of
this
has
been
evolving
so
then
you'll
see
the
latest
evolution
of
it.
We
started
off,
as
I
talked
last
time
about
the
columns
paper.
Remember
the
big
thing
about
the
columns
paper
was:
there
were
two
big
ideas
in
the
columns
paper.
A
Now
we
proposed
and
actually
do
a
simulation
of
how
the
model
would
be
created
in
the
model
we
did
in
the
columnist
paper.
I
would
call
it
a
feature
model.
I
just
made
that
up
and
and
the
feature
model
says
a
model
of
the
of
an
object
is
a
set
of
features
at
locations.
That's
how
we
described
it
in
that
paper.
A
So,
as
you
move
your
finger
on
the
coffee
cup,
you
discover
features
at
different
locations
in
some
reference
frame
and
and
that's
how
you
build
up
it's
a
set
of
those
features
at
locations
that
was.
There
was
a
lot
of
issues
with
that,
but
that
was
what
the
best
we
could
do
at
that
time
and
we're
pretty
excited
about
it.
But
again
the
main
thing
we
were
trying
to
do
in
the
paper
saying
that
it
could
be
done.
A
A
In
the
frameworks
paper,
we
propose
a
different
type
of
model,
I'll
just
call
this
one:
a
compositional
model,
okay.
So
what
this
was?
He
says:
okay,
an
object,
isn't
a
set
of
features,
but
an
object
is
a
or
a
model
is
a
set
of
displacements
between
component
objects,
so
so
objects
a
model
of
something
is
built
of
a
set
of
other
things
in
some
relationship
to
each
other.
An
example
we
used
was
the
logo
on
the
coffee
cup
and
we
say
yes,
I
don't.
A
I
don't
need
to
when
I
learned
this
new
coffee
cup,
I
I
learned
it
as
there's
a
logo
at
some
position
relative
to
the
cost
to
this
to
a
cylinder
or
coffee
cup,
and
now
I
have
a
new
object
composed
of
logo
and
a
coffee
cup,
it's
compositional,
and
then
everything
would
be
like
that.
Everything
you
know
is
composed
of
other
things
in
relative
positions.
A
A
It's
very
efficient
in
storage
in
in
this
particular
thing,
you
can
imagine,
there's
a
cell
active
in
each
in
each
displacement
cell
module.
So
let's
say
I
had
20
displacement
cell
modules,
there'd
be
20
active
cells;
this
is
a
very
rough
approximation
and
that
those
20
active
cells
would
represent.
A
That's
all
you
take
to
represent
the
logo
on
the
coffee
cup,
it's
very,
very
efficient.
I
only
have
to
form
20
new
synapses
if
you
will
to
form
this
new
relationship
between
these
two
objects-
and
it
happens
very
quickly
happens
instantaneously,
there's
no
extra
effort,
it
happens
every
time
you
attend
to
anything
and
then
so
it's
very
efficient
in
storage,
and
it
enables
continuous
learning
because
you're
continually
calculating
these
displacements.
A
You
never
stop
in
your
life.
You
know
when
you're
awake
and
so
the
example
I
use
in
the
book
and
we've
used
in
the
office.
A
few
times
is,
you
know,
you're,
you
walk
into.
You
sit
down
at
the
dining
table
and
you
look
and
you
see
where
the
you
know:
the
bowl
of
potatoes,
mashed
potatoes
are
and
the
green
beans
and
where
the
wine
bottle.
Is
you
just
look
at
them?
A
One
one
one
and
now
you've
built
up
a
model
of
where
those
things
are
on
the
table,
and
so,
if
I
said
close
your
eyes
and
tell
me
you
point
to
the
you
know
the
green
beans,
you
can
do
it
you've
already
built.
You
know
where
it
is.
You
built
this
model
of
the
table
just
by
glancing
at
things
and
knowing
where
things
are
relative
to
everything
else,
and
so
we
may
not
remember
that
for
very
long,
but
I
formed
those
memories
constantly.
A
We
all
do
this
constantly.
Just
where
do
I
leave
my
glasses?
You
know
what
that
is
just
do
here,
but
you
know
all
these
things.
So
it's
a
sort
of
a
part
and
parcel
of
continuous
learning,
independent
of
the
the
mechanism
of
dendrites.
It's
just
saying
it
provides
a
structure
for
learning
some
of
your
models
of
the
world
continuously,
just
by
tending
the
things.
So
that
was
all
really
cool,
but
there
were
a
bunch
of
problems
with
the
compositional
model,
as
described
in
the
frameworks
paper.
I
want
to
make
a
distinction.
A
The
compositional
model
is
right,
but
the
as
we
described
in
the
frameless
paper
is
had
some
problems.
As
I
mentioned
last
time,
the
displacements
they
didn't
represent
orientation
and
scale.
There
was
no
way
to
say.
Oh,
the
mental
logo
is
at
an
angle
on
the
coffee
cup,
it's
at
a
45
degree
angle
or
that
it's
a
small
logo
or
a
big
load.
We
had
no
way
of
doing
that
with
our
displacement
cells.
A
Another
big
problem
with
the
frameworks
paper
is
that
required
many
grid
cell
modules.
We
were
relying
on
this
idea
that
you're
going
to
be
sampling
across
multiple
grid
cell
modules
that
were
slightly
different,
and
this
is
how
you
came
up
this
unique
location
and
our
displacement
cell
mechanism
required
that
too.
The
problem
with
this
is
the
evidence
we
had
suggested
that
couldn't
be
that
the
best
knowledge
we
have
and
it's
partial
information.
A
I
would
argue
that
there's
probably
one
grid
cell
module
per
column,
that's
what
that's,
what
the
evidence
would
suggest
and
we
need
you
know
I
ideally
would
want
20..
Maybe
I
could
get
away
with
eight
or
ten,
but
you
can't
get
away
with
two
and
you
know
one
forget
it:
it's
not
gonna
work,
so
we
really
scratched
our
heads
over
this
for
a
long
time.
Looking
for
evidence
on
this,
and
we
only
evidence
we
really
had
was
from
the
hip
it's
from
the
toronto
cortex.
A
A
We
couldn't
explain
how
you
could
learn
or
sense
or
infer
from
a
distance
like
the
whole
system
is
built
on
the
idea
that
the
column
knows
where
your
finger
is
relative
to
the
object
you're
touching,
but
when
you're
looking
at
something
your
eye
is
not
on
the
object,
it's
a
distance
from
the
object,
and
so
how
do
we
reconcile
that
and
that's
true
for
hearing
as
well?
So
that
was
another
problem
we
didn't
address
in
that
paper.
A
So
those
are
some
pretty
significant
problems,
but
there
are
also
some
really
cool
ideas
and
things
about
frameworks
paper.
You
know
compositional
models
and
this
one-shot
learning,
efficient
storage.
All
that
seemed
right.
It
just
seemed
too
good
to
not
be
correct.
Yet
we
had
these
fundamental
problems
so
so
now
I'm
going
to
talk
about
modeling
from
a
different
perspective
and
point
out
where
our
error
was,
and
before
I
do,
is
there
any
questions
on
this
slide.
B
A
That's
an
interesting
question
and
maybe
marcus
you
can
answer
this
question,
because
I
need
to
spend
more
time
thinking
about
this
you're,
going
to
see
in
a
moment
that
we're
going
to
swap
out
grid
cell
modules
for
for
vector
cell
modules
and
the
question
then
becomes
do.
I
need
multiple
vector
cell
modules.
A
H
A
With
me
today,
so
that
would
that
would
be
my
guess
too,
because,
because
only
because
we
have
to
fit
this
all
in
a
column,
and
I
can
easily
imagine
fitting
one
of
these
modules
in
the
column-
that's
no
problem.
I
know
we
have
to
do
that,
but
getting
a
lot
of
them,
and
so
there's
there's
a
for
those.
Many
people
won't
understand
this
comment,
but
this
is
what
super
tight
is
getting
at.
A
We
require
multiple
modules
as
a
way
of
sort
of
taking
a
representation
like
a
gridson
module
doesn't
represent
a
lot
and
in
order
to
make
it
rich
enough,
we
had
to
create
multiple
modules,
but
there's
another
way
of
creating
richness
from
a
simple
representation,
and
that
is
the
the
temporal
memory
model.
That
is
the
idea
that
you
have
a
mini
column
of
cells,
and
you
pick
one
this.
The
column
represents
something,
but
you
pick
one
cell
in
each
of
the
active
columns
and
you
get
a
much
higher
representational
space.
A
That's
that's
our
context
and
I'm
trying
to
see
if
the
whole
model
can
work
just
using
that
which
would
be
much
more
elegant.
That
is
that
what
we're
going
to
end
up
with
you're
going
to
see
well
we're
going
to
eventually
see
that
the
column's
going
to
it's
going
to
have
good
cells.
It's
going
to
have
vector
cells,
but
maybe
the
combination
together,
along
with
the
mini
column
trick
of
that
would
be
sufficient.
A
Another
another
thing
was
yeah,
that's
a
big
constraint.
Another
thing
is
on
the
on
the
mini
column.
Temple
memory
trick
right.
We
have
many
ten
cells
that
say
all
represent
the
same
thing.
You
pick
one
to
be
active
and
need
some
minicoms
there's
a
huge
amount
of
empirical
evidence
supporting
that
and-
and
so
I
feel
comfortable-
that's
actually
happening
and
it
and
then
I
can
easily
extend
and
say
well
that
could
be
happening
in
each
cellular
layer.
That
would
be
a
really
obvious
extension,
because
the
inhibitory
cells
are
there.
A
Everything
is
there
to
support
that.
The
idea
that
you
have
multiple
modules
like
multiple
grid
cell
modules
and
multiple
splicing
cell
modules.
We
have
almost
no
empirical
support
for
that,
and
the
empirical
support
we
have
says
suggest
it's
not
true.
So
so,
if
that
alone
would
suggest,
like
you'll
probably
have
to
abandon
that,
even
though
it's
so
damn
cool
the
way
it
worked,
it
was
it's
so
we
got
so
enamored
with
it.
By
the
way
we
didn't
invent
that
one
of
you
could
tell
us
who
invented
this.
A
H
B
H
A
F
A
I
I
guess
I
mean
it's
kind
of
it's
one
of
the
things
you
you
know
you
can't,
you
know,
divide
a
baby
in
half
and
treat
the
two
halves
separately.
You
know
it's,
I
mean
easy,
it's
it's
like.
I
don't
know
what
the
in-between
is
either
columns
are.
All
columns
are
all
doing
the
same
thing
right.
So
we
let's
accept
mount
castle's
proposal
mike,
as
I
said,
every
column
is
doing
the
same
thing
and
therefore
the
correlate
of
that
is
everything
the
cortex
does
is
done
by
every
column.
A
That's
a
logical
follow-on
proposal,
and
so
it's
kind
of
hard
to
go
halfway.
It's
either
you
reject
the
whole
thing
and
say
no
he's
wrong.
You
know.
Different
parts
of
the
cortex
are
doing
different
things.
It's
a
hierarchical
thing.
You
know
the
model
of
the
coffee
cup
only
exists
in
higher
levels.
It
doesn't
exist
in
these
lower
levels.
Then
you're
back
to
square
one.
That's
what
most
people
think
today.
I
don't
see
how
you
so
I
don't
see
how
to
to
be
half
pregnant.
A
You
know
what
I'm
saying:
it's
like
you
go
for
it,
you
don't
and,
and
at
the
moment
I
don't
want
to
throw
it
away
kevin,
because
I'm
I'm
absolutely
confident
we
can
build
the
entire
model
in
the
column.
So
I
you
know
if
I
have
to
be
really
really
backed
into
a
corner
where
the
logic
proves
that
it
couldn't
happen
before
I'm
willing
to
abandon
that
it's
it's
it's
just
too
good.
H
I
I
I
will
say
that
that
the
the
idea
kevin
just
brought
up
just
in
case
so
in
the
columns
plus
paper,
the
one
that
the
grid
sells
and
the
moving
and
stuff
I
made
sure
to
include
a
result
where
we
did
use
only
one
module
and
showed
that
it
actually
can
still
move
around
and
infer
a
location.
But
it,
of
course,
can't
make
predictions.
H
A
F
A
A
F
Well,
I'm
I'm
I'm
willing
to
accept
the
fact
that
that
the
that
that
is,
is
no
sense
in
just
apply
occam's
razor
into
this
that
you
might
as
well.
If
you,
if
you
can
explain
everything
as
a
as
an
isolate
cortical
column,
embodying
the
model
until
you're
forced
to
abandon
that
proceed
along
those
lines,
and
so
remember.
F
A
Okay,
well,
it's
a
good
thing
to
ponder,
but
I'm
not
going
to
ponder
it
for
too
long.
So
I
mean
we
got
this,
it's
the
thousand
right
there.
It's
really
it's!
It
solves.
So
many
constraints,
it's
just
it's
just
we
want
to
fight
for
it
and-
and
I
and
I
think
well,
it'll
turn
out
to
be
right.
I'm
very
confident
that,
but
let's
talk
about
this,
we
we
didn't.
We
did
realize
all
along
the
way
in
both
the
columns
paper
and
the
frameworks
paper
and
the
columns
paper.
A
A
I
think
we
talked
about
the
issues
of
displacement
cells.
I
think
we
might.
I
don't
know
if
we
talked
about
the
issues
too
many
grits
on
modules,
but
but
we
knew
these
things.
We
weren't
hiding
it
from
anybody,
but
it
didn't.
You
know
we
didn't
want
to
let
this
stop
us
in
our
tracks.
We
said:
okay,
there's
some
things
that
are
right
about
this
and
I'm
featuring
those
things
you
know
a
compositional
model
is
correct.
A
That
is
how
the
world
is
structured
and-
and
so,
if
we
didn't
get
it
right,
the
first
time
we'll
get
it
right.
The
second
time
and
the
benefits
of
computational
models
are
correct.
So
it's
like
okay,
let's
refine
our
models,
let's
go
on,
so
let's
just
talk
about
models.
I've
shown
these
slides
a
couple
times.
Maybe
some
of
you
have
seen
it,
but
let's
just
go
through
them
again.
This
is
the
model
you
know
what's
a
model.
This
is
a
model.
A
This
is
a
physical
model
that
an
architect
might
make
of
a
house,
and-
and
why
do
we
build
models
like
this?
We
build
models
like
this
because
it
allows
us
to
see
the
structure
before
it's
built.
What
it
would
look
like
from
different
angles.
We
can
say
what
it
looks
like
in
this
position.
It
looks
like
nitrogen.
You
can
imagine
standing
on
the
terrace
and
saying
what
would
my
view
be.
If
I
look
in
this
way
in
that
way,
you
can
also
use
it
to
solve
problems.
A
You
could
say
well
if
I
want
to
get
from
the
driveway
to
the
pool,
what
would
be
my
path
to
get
there
and
what
obstacles
might
I
have
and
things
like
that.
So
that's
where
we
build
models
like
that,
but
now
we
often
we
build
them
in
computers.
So
here's
a
model,
a
computer,
a
model
of
a
house
that
was
designed
on
a
computer.
A
A
It's
very
simple,
as
you
all
probably
know,
they
define
a
reference
frame
for
the
house
in
this
case
they
this
reference
name,
will
have
an
anchoring
point,
a
zero
point,
if
you
will
in
cartesian,
coordinates
we
place
at
the
corner
of
the
property.
Perhaps,
and
now,
with
that
reference
frame,
we
can
locate
things
relative,
you
know
and
we
can
place
things
in
the
house
now.
The
way
this
works
is,
if
I
want
to
place
a
door
in
the
house,
I
don't
specify
the
location
every
component
of
the
door.
A
In
fact,
I
have
another
model
of
the
door
which
has
its
own
reference
frame
reference
here
in
green,
and
now
I
locate
that
reference
frame
at
a
location
in
the
blue
reference
frames
like
oh
the
door,
is
at
this
location
relative
to
the
corner
of
the
property.
Let's
say
now,
but
let's
think
about
it.
So
a
model
like
this,
a
computer
model
or
any
model,
is
a
set
of
objects
at
relative
locations.
A
A
A
The
object
itself
is
represented
by
a
reference
frame
and
we
have
to
specify
the
orientation
of
that
reference
frame
at
that
location
and
the
scale
of
that
object
that
we're
going
to
place
at
that
reference
frame
and
that's
how
computer
models
are
built
right.
Exactly
like
this
using
reference
frames,
these
cartesian
coordinate,
type
reference
frames
and
just
to
reiterate,
objects
such
as
the
door.
A
A
Now
we
had
deduced
that
models
in
the
brain
are
also
constructed.
This
way,
we
were
confident
we
are
confident
of
it.
It
was
a
deduction
that
this
is
the
way
models
are
built
in
the
brain,
but
when
I
say
this
is
the
way
they're
built
in
the
brain,
I
mean
specifically
that
the
columnar
models
are
sets
of
objects
at
relevant
locations,
orientations
and
scales
that
all
comes.
We
can
see
all
that
just
coming
right
from
the
coffee
cup
with
the
logo.
Those
issues
are
there
right
from
the
start.
A
No
all
right,
I'm
getting
that
point
right
away.
Here's
I
say
specifically,
what
am
I
saying
they're
the
same
they're
compositional
in
that
way,
only
they're
compositional
that
instead
of
a
model,
is
a
set
of
objects
at
relative
locations,
orientations
and
skills.
Now
we
get
to
the
problem,
we
made
a
fundamental
error.
A
We
had
already
deduced
that
columns
have
a
metric
reference
frame
made
of
grid
cells.
We
deduced
this
from
the
the
thing
the
thought
experiment
about
moving
your
finger
relative
to
coffee
cup.
That
told
us
if
you
could
predict
where
your
finger
is
going
to
be
when
it
moves
that
tells
us
there
had
to
be
a
metric,
regular
type
of
reference
frame,
not
necessarily
a
cartesian,
coordinate
referencing,
but
grid
cells.
We
hypothesize
because
path
integration
requires
it
path.
Integration
requires
a
reference
frame
like
that.
There's
no
other
way.
You
can
say
I'm
moving
my
finger.
A
You
know
in
some
from
point
a
in
some
direction
and
I've
never
done
this
specific
movement
before,
but
I
want
to
be
able
to
predict
where
I'll
be
in
that
reference
frame.
You
know
I
need.
I
need
a
reference
frame
for
that,
and
so
we
pre
we
deduced
the
reference
room
source
exists.
We
hypothesized
that
they
would
be
built
on
grid
cells
and
now
we
know
there
are
good
cells
in
every
quarter,
they're
finding
grid
cells
in
quarterback
columns.
So
that's
all
great
we're
confident
about
that.
A
But,
honestly,
the
reason
the
only
thing
we
know
for
certain
is
that
path:
integration
which
which
columns
can
do
requires
reference
frame.
But
then
we
made
the
assumption
that
models
in
a
column
were
based
on
grid
cell
metric
reference
frames
like
the
computer
model
now
they're
not
cartesian
the
grid
cell
model.
We
we
did
with
multiple
grid
cell
modules.
A
There
is
no
coordinate,
there's
no
zero,
zero
point,
there's
some
significant
differences,
but
in
some
sense
they
have
the
same
quality
their
their
metric,
and
that
is,
you
can
determine
the
distances
from
any
point.
In
any
other
point,
there's
this
regular
structure
like
like
a
cartesian
coordinate
system,
and
so
we
were
going
with
that
and
we
were
trying
to
figure
out
how
to
make
them
work
in
3d
and
all
the
issues
associated
with
that.
But
that
was
a
logical
assumption.
A
We
just
said:
okay,
we
know
that
we're
going
to
have
these
compositional
models.
We
know
they're
going
to
kind
of
have
these
objects
composed
of
referencing
objects
composed
of
other
objects
and
orientations.
We
knew
there's
already
a
metric
reference
frame
in
each
cortical
column.
So
that's
how
they're
built,
but
that
was
a
mistake.
A
A
So
we
know
these
cells
also
exist
and
that,
as
marcus
proposed,
the
displacements
now
are
not
derived
from
changes
in
grid
cell
modules,
as
we
propose
in
the
frameworks
paper,
but
are
derived
from
vector
cells
that
you
can
calculate
a
displacement
between
two
vector
cell
representations
of
displacement,
substances
and
polar
coordinates,
and
and
that's
where
the
displacements
are
derived.
So
the
whole
model
is
built
around
these
vector
cells.
So
why
do
we
have
grid
cell
reference
frames?
A
Well,
we
still
need
to
do
path,
integration
of
movement,
and
so
the
the
basic
theory
is
that
we
built
these
models.
Using
these.
You
know
that
we
can
represent
the
relative
position
of
structures,
the
sub
models
of
each
other,
but
we're
going
to
represent
them
using
these
more
polar
coordinate,
like
representations
and
and
we're
going
to
calculate
the
displacements
using
differences
between
these
vector
cells,
and
yet
we
still
have
to
have
a
reference
frame,
because
when,
when
I
move
I'm
going
to,
I
still
have
to
figure
out
like
if
I'm
moving
straight.
A
Where
am
I
going
to
be
after?
I
move
for
you
know
one
second
at
this
speed
I
need
a.
I
need
a
a
metric
reference
frame
for
that,
so
so
the
grid
cell
places
sort
of
a
ancillary
role.
Here,
it's
not
an
integral
part
of
the
model,
it
you
know
where
my
sensor
is
at
any
point
in
time
and
and
is
not
really
important
to
the
model.
This
is
a
model.
A
That's
independent
of
my
particular
viewing
position,
and
but
I
do
need
to
have
good
cells
to
track
where
my
sensor
moves
as
it
moves
around
the
model,
but
it's
not
part
of
the
model
itself
and
that's
the
big
insight
that
that
marcus
had
marcus.
Do
you
want
to?
Did
I
say
anything
that
you
would
want
to
add
to
or
change.
A
I
know
you
don't
describe
it
in
these
very
same
words,
but
this
is
what
we've
put
for
me
now.
I
have
a
set
of
slides
that
walk
you
through
how
this
is
done,
but
I
didn't
finish
them.
I
just
didn't
get
them
done
and
so
they're
not
finished
and
I'm
not
gonna,
I'm
not
gonna
tell
you
the
unfinished
slides.
I
think
I
think
that
might
be
enough
for
one
day
anyway.
So
this
is
like
the
cliffhanger.
Okay,
we've
abandoned
our
old
ways.
A
B
A
Okay,
okay,
so
let
me
let
me
do
that,
but
I'm
going
to
have
to
visualize,
I'm
not
going
to
show
you
a
picture,
maybe
marcus.
You
can
find
a
picture
to
show.
But
let
me
visualize
it.
Let
me
describe
it
visually
for
you
and
then
maybe
marcus
can
show
you
a
picture
of
it.
So
imagine
we
have,
let's
start
with
the
grid
cell
module
right.
A
grid
cell
module
is
a
bunch
of
cells
that
are
arranged
in
some.
A
A
So
an
individual
cell
active
in
a
grid
cell
module
will
tell
you
where
you
are
within
some
area
of
space,
but
then
the
whole
thing
is
repeated
in
as
you
keep
going
further
and
further
distance,
so
individual
cells
in
the
gridzone
module
can't
they
can
tell
you
something
about
your
location.
You
can
say
well
you're
in
one
of
any
of
these
locations.
A
Equally
spaced
out
on
a
grid,
but
you
can't
tell
which
one
of
them,
so
that's
a,
but
that's
that's
what
grid
cells
represent
now
imagine
I
have
another
set
of
cells,
maybe
the
same
area
in
size,
but
now
these
cells
represent,
they
can
represent
all
space
and
here's
how
they
do
it.
You
can
imagine.
I
have
some
cells
that
represent
that
there's
something
like
a
they're
representing
a
position
of
relative
to
me,
I'm
in
the
I'm
in
some
place,
and
I
can
have
a
cell
that
says.
A
Oh,
this
cell
represents
a
location,
that's
at
30
degrees
from
you
and
one
center,
a
meter
away,
and
I
might
have
a
whole
set
of
these
cells
that
represents
one
centimeter
away
from
it
like.
Okay,
there's
something
that's
30
degrees,
40
degrees,
50
degrees,
60
degrees,
70
degrees,
80
degrees.
Not
you
know
that
kind
of
thing
go
around
the
circle,
so
I
got
a
set
of
cells
now
that
some
of
my
vector
cells,
they
all
represent
a
position
relative
to
me
at
some
distance.
A
Now
I
have
another
set
of
cells
that
represent
the
same
sort
of
angles,
but
further
away,
but
the
the
the
but
they
this
maybe
the
first
cells,
represent
anything
between
zero
and
one
centimeter.
The
next
set
of
cells
represent
between
one
and
ten
centimeters,
let's
say,
and
so
now
I
have
a
set
of
cells
to
say,
there's
something
between
well,
let's
say
one
and
five
centimeters
between
one
and
five
centimeters
away
at
30
degrees
and
one
five
centimeters,
like
90
degrees.
A
Things
like
that
and
I
have
another
other
cells
in
all
the
same
module
here
that
make
what
some
will
represent
things
that
are
even
further
away
but
much
less
precision.
So
the
further
away
you
get
the
less
precise
you
can
get.
So
you
know
your
the
cells
that
represent
the
furthest
distances.
You
could
say
you
know
it
could
be
really
far
away.
I
don't
know
if
you
know
anywhere
between
you
know
10
meters
and
100
meters.
A
I
don't
know
something
like
that,
but
in
this
direction,
so
you
can
imagine
these
cells
arranged
in
like
a
little
rosette
around
you
they're,
saying,
okay,
when
there's
something
at
this
direction.
This
distance
this
will
become
active
and
there's
something
at
this
distance
in
this
direction.
That's
something
we'll
call
it
now
imagine
so.
That's
a
grid
cell
module,
it's
a
finite
set
of
cells,
but
in
theory,
can
represent
all
space
because
the
resolution
gets
very
fuzzy
the
further
away
you
get,
so
they
don't
have.
A
A
A
Well,
let's
say
typically
an
object,
vector
cell-
let's
imagine
your
your
rat
in
a
room
and
I'm
at
some
location
and
let's
say
I'm
in
the
I'm
in
the
just
some
location
in
the
room
and
in
some
a
little
way.
For
me,
it's
behind
me
there
is
this.
There's
a
piece
of
cheese
right
now
the
object
vector
cells
will
represent
that
piece
of
cheese
relative
to
me,
but
the
the
vector
cells
are
anchored
to
the
room
in
some
sense.
Well,
let
me
put
it
this
way.
So
now
the
cell
says.
B
A
E
H
A
A
A
H
Yeah,
the
the
bottom
right
here
and
also
you
could-
we
could
make
a
point
here.
So
the
bottom
right
is
the
idea
that
you
have
a
set
of
cells
that
respond
to
when
an
object
is
at
a
certain
location
relative
to
you
and
one
benefit
of
this
type
of
coding
scheme,
although
this
is
still
speculative
even
on
the
part
of
this
paper,
is
the
idea
that
the
resolution
changes
as
you
go
further
out.
H
It
would
be
surprising
if
that
weren't
the
case,
but
it
is
still
speculative
and
one
of
the
kind
of
questions
one
of
the
things
that
varies
across
these
cells.
Some
of
them,
are
in
the
reference
frame
of
the
environment
where,
like
this
direction
like
going
up
into
the
left
here,
what
is
that
is
that
is
that
in
your
reference
frame?
A
B
So
if
there's
some
set
of
cells
where,
as
you
turn
around
you
know,
this,
the
same
set
of
cells
will
keep
within
the
code
will
be
the
same.
It's
the
same
set
of
cells
that
are
firing,
as
you
turn
around
the
room
and
then
there's
others
which
are
just
relative
to
use.
As
you
turn
around,
that
thing
will
be,
the
set
of
yellow
cells
will
be
going
around
in
there.
It.
A
A
You
know
I
mean
like
one
of
those
yellow
pentagons
it's
and
so,
as
I
turn
there's
an
object
at
that
point
and
as
I
turn
the
the
relative
well,
not
the
cell,
the
cells
will
be
changing
but
they're,
representing
that
here's
here's
the
way
to
look
at.
As
I
turn
the
active
cells
will
change,
but
the
active
cells
represent
the
same
location
in
the
room
right.
It's
changing
relative
to
me,
but
it's
the
same
location
in
the
room
so.
A
B
A
B
A
H
Yeah,
what
one
piece
of
terminology
some
robotics
people
might
be
able
to
relate
this
to
is
an
occupancy
grid
or
an
occupancy
map.
There's
a
lot
in
common
with
that
and-
and
those
can
similarly
either
be
in,
like
the
room's
reference
frame
or
in
your
cell's
reference
ring.
It's
the
same
general
idea.
If
that
helps
anyone.
A
B
A
And
I
think
that
makes
that's.
I
think
I
think
that's
what
we're
saying
here
too.
These
are
both
existing.
The
same
principles
apply
here.
You
can
put
that
way.
I'm
trying
to
go
see
how
far
you
can
go
with
is
just
say
substitute.
You
know,
vector
modules
for
grid
cell
modules
and
and
see
how
far
you
can
go.
You
know
basically
they're
playing
the
same
roles.
H
A
B
A
A
It's
interesting
whether
this
you
say
this
is
speculative.
I
I'd
be
curious
about
the
the
changing
of
scale,
because,
if
it's,
if
it
doesn't
change
in
scale
as
suggested
in
this
picture
and
the
way
I
described
it
then
you're
back
to
the
same
problem
you
had
before
with
with
grid
cell
modules
right.
They
can't
really
represent
your
space
correctly,
and
then
you
have
to
somehow
overcome
that
it's
very
appealing
that
they
change
in
scale,
because
you
know,
if
I
have
a
goal
and
I'm
going
to
represent
the
goal,
you
know.
A
Let's
say
I
want
to
get
from
my
front
door
in
my
door
in
my
backyard.
I
want
to
go
someplace.
Well.
If
I
want
to
go
just
you
know
to
the
just
a
few
feet
to
a
chair,
I
have
a
pretty
accurate
representation
of
the
distance,
but
if
I
want
to
go
from
that
door
to
you
know
a
fence
at
the
very
far
end
of
the
yard,
my
sense
of
how
far
to
go
is
not
very
good,
but
it
doesn't
matter.
I
know
the
right
direction
to
go.
I
know
I
have
to
go.
A
Always
I
can't
say:
oh
that's
going
to
be
26
and
a
half
steps.
I
can
just
say
it's.
You
know
it's
going
to
be
a
certain
amount
of
distance
and
that,
as
you
get
closer
of
course,
then
your
resolution
gets
better
and
better
and
better.
So
the
idea
that
you
could
have
this
sort
of
course
sizing
is
very
appealing
because
you
get
the
right
direction
roughly
to
get
going
in
the
right
place,
but
and
you'll
refine
your
your
trajectory
as
you
get
closer
to
the
target.
A
We
had
the
same
question
with
grid
cells,
and
you
know,
let's
just
review
the
problem
with
grid
cells.
The
problem
with
grid
cells
is
that
they
look
like
they're
two-dimensional
right
and
we
need
three-dimensional,
and
then
there
was
two
questions.
There's
two
ways
we
could
address
that
one
was
like
hey
they're,
just
doing
this
in
rats
and
rats
kind
of
live
in
a
two-dimensional
world.
So
that's
what
it's
going
to
look
like
for
a
rat-
and
you
know
we
shouldn't
you
know.
A
Maybe
they
work
in
3d
for
other
animals
or
the
other
possibility
is
no
grid.
Cells
are
fundamentally
2d
and
we
have
to
compensate
for
that.
Somehow
the
paper
that
marcus
did
with
merco
is
one
way
of
compensating.
If
you
have
a
whole
bunch
of
grid
cell
modules,
you
could
show
that
a
bunch
of
two-dimensional
grid
cell
modules
can
represent
in-dimensional
space,
but
then
that
again
requires
a
lot
of
good
sum
modules,
which
we
don't
really
think
we
could
do
so.
Those
are
the
two
options
that
to
deal
with
this
and
the
same.
A
I
guess
the
same
two
options
would
occur
here
in
the
vector
cell
space.
You
can
either
say
but
vector
cells.
You
know,
as
measured
in
the
rat,
mostly
look
like
2d,
because
that's
what
the
world
rats
live
in
and
or
maybe
they're,
fundamentally
3d
and
or
no
we
have
to
compensate
from
somehow
marcus.
I
don't
know
if
you
know
anything
about
that.
You
can
comment
on
that.
G
H
About
object,
vector
cells
right
now
and
whether
what
happens
in
with
other
dimensions,
the
way
I
frame
the
conversation
in
my
head,
as
is
you,
what
you
need
is
an
unsupervised
learning
system
that
can
rep
that
can
learn
relations
between
self
and
other
and
in
the
space
of
2d
space
or
3d
space
they're.
Just
straight
up,
they're
object,
vector
cells,
they're
they're,
these
vector
cells.
These
are
the
results
of
that
unsupervised.
A
A
F
There
is
a
there
is
a
1d
equivalent
to
what
you
were
talking
about.
That's
direction,
cosines,
which
work
for
in
dimensions,
and
you
can
think
of
them,
as
you
know,
displacement
from
some
virtual
axis
or
as
angles
in
that
axis
or
a
variety
of
other
things.
But
that's
the
reason
why
I
was
bringing
it
enough
is,
is
that
I.
F
F
Multiple,
no,
no,
I
I'm
speaking
about
abstract
coordinate
systems.
So
what
I'm?
What
I'm
saying
is
that
direction
cosines
is,
if
you
imagine,
the
x
y
z
coordinates
in
three
dimensions:
it's
the
you've
got
some
vector
out
there
direction.
Cosines,
basically,
are
the
angle
from
each
axis
to
the
vector,
so
you
have
three
direction:
cosines
and
they're
all
the
same
type
of
thing:
they're
they're,
they're,
not
they're,
they're
symmetric
in
that
sense,
and
you
can
extend
that
to
n
dimensions.
F
A
F
A
I
don't
know
yet
we
don't
know,
I
mean
it's
it's
well
in
terms
of
the
brain,
I
mean
again
again,
we
can
differentiate
between
how
someone
would
build
this
in
a
machine
learning
architecture
from
how
the
brain
does
it.
We
don't
have
to
do
it
the
way
the
brain
does
it,
but
in
the
brain
we
we're
trying
to
figure
out,
like
you
know,
okay,
well,
what's
going
on
there
and
you
know
and
many
times
we've
been
through
this,
I
keep
forgetting.
What
did
we
learn
about
3d
grid
cell
modules?
A
I
think
we
learned
that
they
do
represent
3d
right,
it's
most
likely
yeah,
most
likely.
I
found
a
bat,
you
know
so
that
it
would.
It
would
formulate
more
towards
the
hypothesis
that
hey,
if
you
study,
rats
the
world's
going
to
look
flat,
but
if
you
study
bats,
the
world's
going
to
look
three-dimensional,
and
so
that
would
be
the
ideal
situation
that
then,
then
the
these
object,
vector
modules
would
represent
three
dimensions.
A
Actually,
the
the
theory
I'm
working,
on
which
I
I
never
really
got
through
with
everybody
here
about
how
how
the
cortical
column
learns
of
the
dimension
of
its
space
by
observing
movements,
would
support
which
support
the
idea
that
a
a
module,
whether
it's
a
grid
cell
module
or
object,
vector
cell
module,
would
learn
n
dimensions.
If
that's
what
the
world
was,
it
would
learn
whatever
the
dimensional
space
that
it
observes
as
opposed
to
sort
of
pre-operator,
knowing
what
it
would
be.
So
that
would
be
the
I
to
me.
A
That
would
be
the
ideal
answers
like
hey.
You
have
this
model.
You
have
some
set
of
cells
and
they're
going
to
go
through
a
sort
of
spatial
pooler
type
process
on
inputs
relating
to
movement
and
they're,
going
to
figure
out
okay.
Well,
I'm
just
going
to
figure
out
all
the
boom
vectors
you
have
and
if
that's
two,
it's
two
dimensional,
that's
one
dimensions,
one
dimensional,
but
it's
four-dimensional!
It's
four-dimensional
that'd,
be
the
nice
answer.
I'm
not
I'm
not
saying.
F
A
It
just
says:
I'm
going
to
move
this
direction,
I'm
going
to
use
grid
cells
to
figure
out
where
I'll
be
when
I
move
in
this
direction,
a
certain
amount
but
but
the
but
the
vector
modules
are
how
we're
going
to
build
the
model
of
the
object
and
know
where
I
am
relative
to
it,
and
so
I
don't
have
to
use
good
cells
for
that
per
se.
Okay,
but
there's
this
there's
a
dimensionality
question
again,
the
evidence
would
suggest
that
we
we
don't
have
a
lot
of
modules
right.
A
We'll
have
to
live
with
that
constraint,
so
in
that
case
it
would
be
nicer
if
these
these
modules
could
handle
whatever
dimensionalities
are
thrown
on,
and
I
think
that's
possible,
theoretically,
it's
possible.
I
can
see
why
that
how
it
would
work,
but
the
evidence
for
that
is,
is
you
know
who
knows,
I
don't
think
anyone's?
Maybe
I'm
not
too
many,
but
maybe
people
have
done
experiments
with
3d
environments
of
object,
vector
cells.
I
don't
know.
A
Yeah,
I
know
that
they're
just
crazy.
They
had
to
do
these
crazy,
bad
experiments
where
they're
measuring
themselves
as
a
batch
of
flying
freely
around
you
know
long
tunnels
and
so
on.
It's
like
crazy,
very
hard
to
do
that
stuff
and
there's
a
lot
of
other
experiments
where
wraps
were
climbing
walls.
Marcus
went
through
some
of
these
papers
and
it
looked
like
you
know
the
funny
little
weird
tricks.
The
rats
were
doing
when
they
went
up
different
heights.
It
wasn't
like
they
were.
A
B
B
So
it's
you
know
you
can
see
like
if
you
poke
the
cells
at
this
b
over
here.
You
know
you
get
this
movement
because
it's
sort
of
20
degrees
up
and
you
know
five
degrees,
amplitude
and
then
a
would
be
sort
of
horizontal,
but
you
know
slightly
longer
and
this
one
would
be
down
20
degrees
and
even
longer,
and
the
reason
I
bring
this
up.
Is
you
know
if
the
if
the
representation
for
locations
and
so
on
in
the
neocortex
is
fundamentally
based
on
these
vector
style
representations?
B
A
A
So
this
is
the
signal
the
cortex
is
getting
and
we're
trying
to
the
cortex
again
in
the
the
the
details
of
how
the
cortex
learns
motor
behaviors
in
that
theory
that
I've
worked
through
it
has
to
it,
has
to
have
a
corresponding
representation
in
the
cortical
column,
as
is
coming
from
the
superior
collectors,
and
so
the
fact
that
the
spirit
colliculus
is
sending
a
sort
of
polar
coordinate,
representation
means,
like
I
don't
have
to
do.
Any
sort
of
you
know
weird.
You
know
geometric
gymnastics
to
convert
it
into
the
representation.
A
That's
in
the
cortex,
if
the
quartrix
is
also
using
polar
coordinates
for
all
these
things,
then
it's
a
very
easy
correlation
to
learn.
Yeah,
and
so
you
know
this
was
a
big
epiphany
that
came
about.
You
know,
thinking
about
what
marcus
has
proposed
to
me
all
sudden
realize,
oh,
my
god,
everything's
going
to
be
done
in
these
polar
coordinate
systems.
That's
that's
the
natural
representational
space
for
movement
for
like.
A
If
I
want
to
say,
oh,
I
have
to
I'm
standing
here
and
I
want
to
go
to
the
kitchen
now
I
have
to
turn
a
certain
amount
and
go
a
certain
distance.
I
mean
that's
just
that's
the
way
it
works
right
and
if
I
want
to
calculate
like,
oh
in
my
xyz
coordinates-
and
I
want
to
get
to
those
x
y
k
coordinates,
then
I
have
to
do
some
math
to
figure
out.
Oh
well.
What
angle
am
I
going
to
go
and
how
far
am
I
going
to
go?
A
A
And
it's
also
interesting
I'll
talk
about
this
next
time,
but
you
know
I
talked
about.
We
haven't
really
handled
orientation
like
orientation
of
the
sensor
of
your
finger
or
turning
your
head
or
things
like
this.
Well,
we
have
to
path
in
a
great
orientation
too
right.
You
know
like
I
have
to
say
if
I
move
my
eyes
to
the
right
at
some
speed
for
some
distance.
Where
will
I
be
pointing-
and
this
is
it
that's
a
path
integration
in
in
an
angle
in
some
sense,
and
so
we
have
these
two
competing
things.
A
We
have
to
do
path,
integration
and
changing
orientation.
We
have
path,
integration
and
location,
and
so
now
the
hard
part
is
path.
Integration
of
location
and
we're
going
to
use.
This
crux
called
the
grid
cell
module
help
us
figure
that
out,
but
all
the
other
orientation
changes
all
the
other
path.
Integration
happens,
naturally,
in
some
sense,
with
with
vector
cells.
I
used
to
comment,
you
know
you,
you
could
if
you
want
to
learn
a
room,
what
a
learner
looks
like.
A
What's
the
first
thing
you
do
you,
I
put
you
in
the
corner
of
the
room:
you're
not
going
to
walk
you're
going
to
sit
in
that
corner.
You
stand
there
and
look
around
all
the
different
things.
Everything
you
can
see
from
that
corner
of
the
room.
That's
how
you're
going
to
build
your
model,
the
world,
not
by
walking
around
first
you're,
not
going
to
say.
Oh,
I
see
the
chair
here,
I'm
going
to
walk
over
and
look
at
that
chair.
I
know
you're
going
to
like.
A
Oh
the
chair
territory,
thing
big
thing:
you
can
build
almost
the
entire
model
room
by
just
standing
one
place
and
moving
your
head
and
changing
your
orientation.
In
some
sense,
that
is
the
first.
The
first
order
approach
to
learning
something
visually
is
just
to
and
you
can
build
the
entire
model
that
way
I'll
show
and
and
then
only
then
well,
they
go.
Oh
now
we're
going
to
walk
over
there.
A
Okay,
I
have
to
work
too,
but
now
this
just
introduces
error,
good
cell
path,
integration,
which
is
not
as
accurate,
but
I
can
do
so
anyway.
Other
people
follow
that
or
not,
but
it
feels
like
these
vector
cells
are
like
oh
yeah.
Of
course
that's
how
it
works,
and
I
don't
know
if
marcus
have
you
thought
about
yet
how
to
actually
calculate
displacements
in
vector
cells.
E
A
A
Yeah,
okay!
Well,
I
guess
I
can
see
that
right,
just
like
we,
but
the
problem
I
always
had
with
the
displacement
cells
is
that
I
always
puzzled
how
we'd
learn
it.
You
know
it
was
simple.
It
was
a
simple
pairing,
but
I
always
felt
like
how
are
we
going
to
learn
that
so
it
seems
to
me
here
with
the
vector
cells.
There
ought
to
be
an
easy
way
to
learn
it.
Somehow.
A
G
Just
more
of
a
comment,
I
guess
that's.
I
think
it
could
be
interesting
to
try
and
think
of
these
encodings
in
like
an
action-oriented
way
as
well,
because
I
mean
space
perception
can
be
very
subjective
and
it
can
be
very
influenced
by
our
action
possibilities
and
the
effort
that
is
required
to
perform
a
certain
action
like
there
are
behavioral
studies,
for
example,
if
you
have
to
throw
a
heavy
ball,
the
distance
seems
further
than
if
you
throw
a
light
ball
or
if
you
have
to
reach
for
an
object.
G
It
seems
further
away
if
you,
if
it's
out
of
your
reach,
then
if
you
have
a
tool
that
you
can
use
to
get
it,
and
it
was,
for
example,
also
match
with
this
resolution
increasing
or
decreasing
with
distance.
So
if
you
everything
that
is
in
your
reach
has
a
pretty
high
resolution,
you
can
get
to
it
with
one
grasp,
but
your
mailbox
outside
it's
like
a
pretty
long
action
sequence
to
actually
get
there.
So
so
I
feel
like
a
lot
of
these
things,
you
could
interpret
in
being
encoded
in
a
action-oriented
way.
A
I
mean
that's
interesting
because
the
way
I've
been
thinking
about
how
the
brain
figures
out
these
dimensions,
like
you
know,
the
space
you're
in
is
just
from
that
purpose.
It's
like
observing
what
actions
actually
occurred
like
that's
how
it
knows
what
space
is
it
says
I
have.
I
can.
I
can
observe
actions
and
those
become
the
basis
set
for
my
my
my
spatial
dimensions.
It's
it's
not
like.
There's
some
sort
of
pure
you
know
like
a
geometric
space
or
a
pure
polar
coordinate
system.
It's
more
like
hey,
based
on
the
movements.
A
I've
seen
I'm
going
to
assume
those
movements
traverse
space
and
therefore
those
will
be
the
basis
set
of
of
my
dimensions
of
space,
which
I
think
it's
very
much
aligned
with
what
you're
saying
vivian
it's
like.
In
some
sense,
the
dimensions
of
space
are
defined
by
our
movements.
It
doesn't
lead
to
those
little
weirdo
things.
You
were
mentioning
about
like
why
why
your
perception
is
incorrect.
I
have
no
idea
about
that,
but
but
the
idea
that
the
basic
idea,
I
think
you're
saying,
is
that
space
is
defined
or
integrated
with
movement.
F
F
I
agree:
I
mean
your
first
fundamental
requirement
system.
You
learn
is
what's
within
arm's
reach
right
I
mean
you
and
and
then,
if
you're,
if
you're
sighted,
then
there's
the
correlation
of
okay.
I
can
reach
this
far,
and
this
is
how
it
appears
to
my
eye
and
you
learn
all
that
you
have
to
learn
all
that
so
but
anything
beyond
your
arm's
reach
is
a
prediction
yeah
and
therefore
increasingly
more
problematic.
F
A
A
You
have
you
have
information
about
flow
across
the
retina
that
things
are
flowing,
not
just
it's
not
a
it's
more
like
things
are
moving,
not
what
it
is
and
there's
this
whole
visual
system
built
around
that
right.
This
is
the
the
magnum
cellular
system,
and
so
now
imagine
you
have
this
field
of
view,
and
you
have
all
these
flow
bits
and
every
action
you
can
take
represents
is
represented
by
every
movement
action,
whether
you're
turning
your
eyes
or
stepping
sideways
or
going
forward
or
going
backwards
or
rotating
your
head.
A
A
A
Has
no
idea
what
these
things
are:
it's
just
a
bunch
of
moving
bits
and
says:
okay
I'll,
take
whatever
those
bits
are
and
I'll
run
through
a
spatial
pooler
I'll
end
up
with
a
representation
of
them
instead
of
many
columns
and
each
minicom
will
be
a
sort
of
an
approximation
of
one
of
the
types
of
movement
vectors
that
it
observed.
It's
going
to
take,
there's
an
infinite
continuum
of
movement
vectors,
but
we're
going
to
particulate
it
into
a
set
of
many
columns.
A
You
know
a
a
a
commonly
or
some
equally
commonly
observed,
set
of
flow
bits
and
therefore
they
represent
different
types
of
movement
and
then
those
same
mini
columns
if
you're
following
me
that
represent
like
one
might
be
going,
one
minimum
column
might
represent
going
straight
forward.
Another
one
straight
mind
returning
your
head.
Whatever
things
like
this,
each
minicom
has
its
own
layer,
five
cells
which
generate
movement,
and
so
the
logical
conclusion
would
be
the
layer.
Five
cells
in
that
minicom
are
going
to
represent
movement
in
that
direction.
A
So
if
I
want
to
go,
if
I
want
to
go,
do
the
same
thing,
which
caused
all
the
flow
bits
to
come
out
like
this,
which
we
would
say
would
be
going
forward,
there
would
be
a
mini
column
that
represents
that
flow
bits
and
say
my
cell
is
going
to
represent
when
I
want
to
move
like
that,
I'm
going
to
be
active,
and
I
will
that
I
will
that
will
about
what
I
represent.
I
move.
A
I
represent
that
set
of
movements,
and-
and
so
now
you
have
a
a
basis
set
of
all
the
movements
that
represent
all
the
different
types
of
patterns.
You've
observed
and
and
and
now
you
can
reconstruct
movements
by
assembling
a
set
of
these
basis
vectors
over
time-
and
you
know
like
open
this,
I'm
gonna
do
this.
I'm
gonna
turn
my
head
and
go
up
and
down
do
this,
so
this
basic
idea
would
apply
to
any
central
modality.
It
doesn't
have
to
apply
just
division.
A
It's
it's
a
basic
fundamental
property
of
observing
movements
in
space,
not
knowing
what
it
means
partially
adding
on
to
a
bunch
of
basis
vectors
one
from
any
column.
Those
many
columns
represent
that
now
I
can
create
movements
too,
because
I
can
just
re.
I
can
just
string
together
the
things
I've
observed.
A
So
that's
the
basic
idea
and
that's
why
I
say
it
could
learn
n
dimensions
because,
as
long
as
it's
represented
in
the
flow
bits
of
of
the
sensor,
then,
if
it's
one
dimension
it'll
do
that
if
it's
two
dimensions
it'll
do
that,
if
it's
three
dimensions
it'll
do
that
and
presumably
if
it's
five
dimensions,
it'll
figure
that
out
too,
I
did
ask
recently
in
a
recent
meaning.
Is
that
possible
yeah
stupid?
I
thought
it
was
so,
but
that's
the
basic
idea
if
you
follow
that,
if
you
could
visualize
what
I
just
said.
A
A
A
A
You
know
and
then
generating
movement,
because
we
have
to
figure
out
how
to
do
that,
and
you
know
now
I
can,
and
so
now
it
brings,
and
it
brings
back
in
the
mini
column,
the
hypothesis
from
mount
castle
and
and
ties
it
together
in
a
nice
way.
So
I
I
think,
we're
really
close
to
figuring
this
all
out.
There's
a
lot
of.
I
know
difficult
concepts
that
are
floating
around
here,
but
they're.
Actually
we're
really
close
to
putting
it
all
together.
A
A
All
right
again
before
we
leave,
I
want
to
emphasize
this
is
not
stuff
that
we
have
to
do
in
our
machine
learning
work
right.
We
could.
We
may
end
up
doing
a
very
simplified
version
of
this.
We
may
do
it
very
differently.
This
is
more
just
to
get
education
about.
What's
the
nature
of
the
problem
that
we're
trying
to
solve
when
we're
modeling
things
and
movement,
and-
and
you
know
the
the
details
exactly
how
it's
done
and
we're
not
going
to
do
that
in
our
ai.
A
A
The
same
properties,
you
know
you
know
it's
an
interesting
question:
take
the
the
the
the
sort
of
rectilinear
metric
space
versus
the
polar
coordinate.
You
know
vector
space.
Well,
it's
not
clear
to
me
pretty
clear
that
the
polar
coordinate
system
is
the
way
the
brain
works
on
right.
So
do
we
have
to
do
it
that
way?
I
don't
know.
I
mean
people
build
a
lot
of
robotic
systems.
The
other
way,
there's
some
real
nice
properties
of
the
polar
coordinate
system
that
we're
just
talking
about.
A
A
Probably
I'm
guessing.
We
would
probably
do
it
that
way,
but
I
don't
know
or
maybe
the
way
we
would
do
like
the
instead
of
using
grid
cell
type
reference
frames.
We
might
just
use
a
standard,
cartesian,
coordinate
reference
frame
for
doing
path,
integration,
and
maybe
I
don't
know
who
knows
I
mean
it's
like:
why
do
we
have
to
do
it
the
way
the
brain
did?
In
that
case,
we
might
still
need
the
same
functions,
but
I
don't.
I
don't
have
to
answer
those
questions.
I'm
just
saying
it's.
B
A
Well,
marcus's
graph
model,
which
I
was
didn't,
have
finished
my
slides
on.
You
know
there
is
no
zero
point.
It's
everything
is
relative
to
everything
else.
There's
a.
I
was
talking
to
marcus
yesterday
about
a
property
that
solves
another
problem.
We've
had,
and
this
is
a
really
esoteric
problem.
So
I'm
sorry
for
those
who
don't
know
this
problem,
but
with
grid
cells
we
had
a
problem.
A
And
now,
if
I
try
to
combine
the
two
guitar
mappings,
I
would
lose
all
my
knowledge
about
at
least
one
of
the
rooms,
because
you
know
you
can't
remap
and
keep
all
your
knowledge.
It
just
doesn't
work.
So
that
was
a
problem,
but
in
the
new
situation,
where
you
have
a
set
of
objects
relative
to
a
set
of
objects
all
with
these
displacement
between
the
objects,
then
you
can
take
two
maps
room.
A
map
and
room
b
map,
and
all
you
have
to
do-
is
start
connecting
the
edges
of
them.
A
Together
with
the
displacements
and
all
the
other
displacements
come
along.
It's
like
it's.
I
don't
have
to
worry
about
remapping
them.
It's
just
like,
oh,
I
have
this
a
graph
here
and
a
graph
here
and
as
long
as
I
could
connect
the
edges
together
then
now
I
have
one
new
graph
and
it's
really
simple.
I
don't
have
to
forget
everything.
I've
learned
about
this
one.
A
I
have
to
get
a
revolution
that
comes
out
of
marcus's
hypothesis,
about
models
being
built
on
displacements
between
components
where
the
way
we
had
it
in
the
rough
in
the
frameworks
paper
and
the
columns
and
the
columns
plus
paper,
we
couldn't
do
that.
You,
you
couldn't
combine.
Two
previously
learned
objects
like
two
rooms
together
into
one.
It
was
very,
very
difficult.
A
So
now
we
have
an
answer
to
that
question,
so
that
would
imply
again
the
the
model
should
be
based
on
polar
coordinate
displacements
in
some
sense,
because
there
is
no
there's
no
zero
point.
I
guess
I'm
I'm
answering
your
questions,
but
I
we're
not
gonna
have
a
zero
point,
because
because
the
zero
point
means
I
have
to
reconcile
two
zero
points-
and
I
have
zero
point
room
major
point
through
two:
how
can
it
be?
I
can't
make
them
the
same,
but
if
I
don't
have
any
zero
points
I
just
have
relationships.