►
From YouTube: Jeff Hawkins Brainstorms on Minicolumns - July 27, 2020
Description
In this research meeting, Jeff Hawkins presents several new ideas, in a continuation of a recent concept he presented: "minicolumn is a movement vector."
A
You
good
yes,
we're
recording,
so
this
is
gonna
be
kind
of
a
hodgepodge
research
meeting.
I've
spent
I've
been
working
on
some
really
exciting
ideas
and
I
spent
several
days
trying
to
make
a
presentation
out
of
it
and
I
kind
of
failed.
There
were
too
many
unknowns
too
many
things
bouncing
around
and,
and
so
I
said
screw
it,
I'm
just
gonna
present
a
bunch
of
ideas
and
make
it
more
of
a
discussion
as
opposed
to
a
presentation.
A
So
I
really
would
like
this
to
be
as
interactive
as
possible
and
I
have
a
bunch
of
ideas,
but
I'm
not
going
to
necessarily
present
them
in
a
logical
order.
So
I
apologize
for
that
before
I
get
started.
I
I
spent
an
hour
talking
to
quran
last
week
and
it
reminded
me
of
a
couple
of
things.
First,
karan's
knew
he's
only
been
here.
Two
months
he's
never
like
he's
been
in
the
office,
he
hasn't
been
around
us
or
the
office
very
much,
and
so
a
couple
things
came
out.
I
just
started
reemphasize
again.
A
B
A
Assume
super
tight
and
lewis
and
marcus
will
pick
up
on
lucas.
Perhaps
but
not
everybody,
and
the
second
thing
is
that
we
had
some
conversations
about
the
role
of
biological
details
in
artificial
intelligence
and
I
just
want
to
re-emphasize
one
more
time,
because
quran
asked
some
questions
that
I
often
get
from
people
who
are
new
to
this
field,
which
is
like
he
didn't
say
it
this
way.
So
don't
don't
take
offense
around,
but
it's
worth
like.
You
know.
Why
do
why
do
these
biological
details
matter?
A
You
know
what's
wrong
with
mathematical
approaches.
You
know
why
are
we
considered
belief,
propagation,
various
probability
theories
and
so
on
and
of
course
we
have
in
the
past,
but
but
always
I
I
felt
I
felt
with
certainty
for
over
40
years
that
we're
never
going
to
figure
out
how
to
build
real,
intelligent
machines
without
understanding
the
biology
in
detail,
and
nothing
has
shaken
my
opinion
about
that.
So
we're
going
to
go
through
some
crazy
biological
details
today,
I
don't
think
they're
optional,
so
I
think
they're
these
kind
of
over
and
over
again.
A
Every
time
I
learn
is
when
people
say
yeah,
I
got
some
inspirational
biology,
but
I
don't
need
to
know
all
the
details
yet
that
leads
to
obstacles
and
you're,
just
not
going
to
get
past
that
at
some
point
in
the
future.
When
you
understand
enough
we'll
be
able
to
say
yeah,
more
biology,
more
biology,
it
doesn't
matter
so
this
stuff
I'm
going
to
talk
about
today,
even
though
it
seems
crazily
biological
it
it.
I
don't
think
these
are
optional
things
for
ai.
A
The
text
here
whoops
is
just
from
the
email
I
sent
out
earlier,
so
I'm
going
to
just
start
going
through
this,
and
then
I
have
a
bunch
of
slides,
some
of
which
I
had
before
so
I
figured
I
would
go
through
a
quick
review
of
some
of
the
material
I
presented
earlier
about
how
many
columns
my
phone
is
ringing.
I'm
gonna
just
disable
that
excuse
me.
One
second
shoot
I'll
move
my
nose.
You
guys
see
that
pop.
A
Zoom
is
presumably
showing
me
stuff
that
that
you're
not
seeing
it
keeps
popping
down
things
over
my
screen
review
of
the
material
I
presented
before
about
how
many
columns
might
represent
space
and
dimensions
and
and
then
I'll
talk
about
a
bunch
of
topics
and
I'll
go
through
these
in
the
slides
ahead.
Here,
I'm
looking
at
them
right
now,
so
I
I
don't
need
to
read
these
texts
right
here.
A
Let's
just
go
right
into
the
to
the
review
and
then
I'll
come
back
to
this
later
and
see
if
I
missed
anything
so
just
to
remind
ourselves
where
we
were
before
that
many
columns,
our
physical
entity
of
the
brain
they
exist,
they
are
part
of
how
the
brain
develops
and
and
many
people
have
speculated
that
many
columns
are
the
functional
unit
of
the
near
cortex.
But
as
far
as
we
know,
there
really
are
no
theories
about
what
many
columns
do.
A
But
one
of
the
consequences
of
this
is
that,
if
a
mini
column
in
theory
has
all
the
cell
types,
that
is,
the
immune,
complex,
might
have
a
220
cells
and
they
have
all
they
have
representatives
all
of
all
the
cells.
So
there
aren't
other
types
of
cells,
they're
in
the
very
cortex
or
in
the
mini
column.
There's
some
evidence.
I
think
I
read
once
it
says,
maybe
some
of
the
inhibitory
cells-
and
this
is
not
true,
but
I
have
to
go
back
and
try
to
find
that
anyway.
A
The
point
of
this
is
is
that
if,
if
this
is
true,
then
every
mini
column
must
have
at
least
one
or
more
of
the
types
of
cells
we
think
about
like
red
cells
and
place
cells
and
orientation
cells
and
displacement
cells,
because
if
they
don't
exist
in
every
column,
every
minicom,
then
they
don't
exist.
You
know
they
have
the
biggest
name,
every
meaning
comes
because
everybody
comes
the
same.
A
That's
a
really
interesting
constraint
to
keep
in
mind.
This
is
just
I'm
not
going
to
go
through
this
again,
but
this
is
just
the
argument
that
people
say
it
is
a.
You
know
the
function
of
minecraft
is
currently
difficult
to
detect.
It
is
a
fundamental
processing
unit,
cortex,
no
one
knows
what
it's
doing
and
that
we
have
a
lot
of
data
from
v1,
not
much
data
from
other
parts
of
the
brain.
A
A
It
was
an
observation
that
what
we
think
of
as
complex
cells
and
we'll
talk
about
a
moment
are
really
they're:
they're
sort
of
they're
scalar
representations
of
movement
in
a
particular
direction
and
they're
not
necessarily
features
of
the
world
they're,
more
movement,
vectors,
and
so
each
minicom
would
represent
a
singular
dimension,
and
I
said
it
may
come
as
an
open,
complete
basic
set
of
space
and
the
the
other
part
of
this
is
that
many
columns
learn
what
they
represent
from
their
sensory
input.
That
is
they.
A
A
Excuse
me
one
second,
janet
excuse
me,
janet
sorry,
she's,
making
noise
that
many
comms
represent
movements
and
they're,
not
they're,
not
really
dimensions
of
space.
And
so-
and
I
got
to
this
observation
by
just
imagining
what
kind
of
flow
patterns
a
column
a
visual
column
could
learn
and
and
it's
going
to
learn
movements,
but
it
doesn't
necessarily
mean
dimensions
of
space,
and
so,
if
I
move
straight
in
a
linear
line,
it'll
it'll
learn
that's
the
type
of
movement.
A
I
have
there's
a
certain
flow
pattern
that
appears
on
the
screen.
When
I
see
that
or
my
vision
system,
I
see
that,
but
any
common
movement.
If
I,
if
I
had
if
I
tend
to
walk
in
circles,
it
would
normally
represent
that
too,
and
it
would
also
learn
to
represent
changes
in
orientation
as
a
movement
vector,
and
so
it's
not
correct
to
think
of
them
as
actually
sort
of
like
linear
dimensions
of
space.
A
They're
really
just
you're
you're,
you're
parsing
the
world
into
movement,
vectors
and
those
moving
vectors
may
may
or
not
correspond
to
linear
dimensions
in
space,
and
therefore,
complex
cells
represent
the
speed
of
movement
of
the
sensor
or
observer
along
the
minicom's
movement
dimension
and
and
every
mini
column.
I'm
going
to
go
with
the
idea
that
every
mini
column
performs
a
1d
path,
integration
along
its
movement
dimension.
A
Again,
the
big
thing
here
is
one
of
the
big
insights
I
had
recently
is
that
changes
in
orientation
are
no
different
than
changes
in
linear
movement.
From
a
column's
point
of
view,
changing
your
orientation
is
just
a
different
type
of
movement.
It
has
a
different
flow
pattern
on
the
screen
and
and
therefore
there'll
be
many
columns
that
represent
orientation,
changes
and
and
those
mini
columns
will
perform
path,
integration
and
all
the
other
functions.
We
assume
that
happens
with
the
grid
cells
along
its
movement
dimension.
A
A
So
I'm
going
to
go
into
this
a
little
more
detail.
Everything
I'm
thinking
about
right
now
is
in
sort
of
one
dimensional
path,
integration
modules,
and
so
that's
pretty
much
the
the
theories
that
people
have
about
grid
cells
that
they
start
off
with
a
one
dimension.
A
multiple,
one-dimensional
path
integrations
which
then
combined
to
look
like
a
2d
grid
cell
and
I'm
assuming
that
2
degree
cells
are
an
artifact
of
later
processing.
A
I'm
not
going
to
go
into
that
more
now,
but
everything
I'm
thinking
about
is
using
one-dimensional
vectors
going
through
this.
So
that's
sort
of
an
overview.
I
don't
think
I'm
going
to
jump
into
some
review
some
of
this,
but
anyone
has
any
comments
yet
before
I
get
going,
can
you
still
hear
me
yeah.
A
A
You
do
not
need
anything
else,
and
the
in
the
sensory
flow
input
is
a
substitute
for
a
motor
input.
It's
basically
saying
I'm
not
getting
a
motor
command,
I'm
observing
the
movements
in
the
world.
Therefore,
I
have
a
primary
data
source
to
say
I
can
observe
what
movements
are
occurring
through
a
sensory
input
and
I
can
build
my
model
that
way.
A
A
We
talked
about
the
manual
and
probable
cellular
cells.
I
I
don't
need
to
go
into
them
in
great
detail
here,
but
just
remember
that
there
are
two
basic
sensory
inputs
to
a
visual
cortex,
there's
a
magnetocellular
cell,
which
is
a
very
large
respective
fields,
the
fast
response
they
only
trigger
an
ounce
at
an
offset
there's,
no
tonic
response
and
those
are
clearly
ideal
for
doing
flow
detection.
And
then
there's
that
part
of
a
cellular
cell
which
are
slower.
They
have
now
receptive
bills.
A
They
have
a
tonic
response
and
these
are
ideal
for
feature
detection,
and
I
might
just
get
that
now.
I
mentioned
this
idea
before
too
this.
This
whole
idea
came
apart
from
observing
the
fact
when
you're
watching,
if
you're,
watching
someone
navigate
a
character
through
some
virtual
world
or
even
natural
virtual
world,
just
watching
a
movie
and
some
you're
watching
a
first
person
view
of.
A
A
Let's
keep
going
so
now.
Imagine
if
all
the
bits
on
this
screen
are
moving
in
one
way
or
another,
they
could
be
converging
or
diverging
or
sweeping
left
or
sweeping
right
or
just
rotating
in
the
plane.
If
all
the
bits
are
moving,
it
means
my
body
is
moving
relative
to
the
world.
That
there's
only
that's
the
only
explanation.
This
is
like
being
in
an
imax,
theater
and
you're
surrounded
and
while
the
bits
are
moving
on
the
screen,
you
feel
you're
moving.
A
If,
however,
only
a
subset
of
the
bits
in
the
field
of
view
are
moving-
and
maybe
maybe
some
central
portion
or
some
some
subfield,
and
then
that
means
your
body's
not
moving
you
you're
not
moving
to
the
world.
If
you
were
moving
the
world,
every
bit
would
always
be
moving,
but
if
only
some
of
the
bits
are
moving,
then
that
means
my
body
is
stationary
relative
to
the
world
and
part
of
the
world
is
moving
relative
to
me.
I
thought
about
this
a
lot.
A
It's
a
fun
exercise
to
sit
there
and
just
look
at
the
world
and
try
to
as
you
move
around
trying
to
imagine
which
bits
are
moving
in
your
field
of
view,
which
or
not-
and
this
seems
to
be
pretty
conclusive-
that
if
you,
if
all
the
bits
are
moving,
your
body
interprets
as
you're
moving
and
when
some
of
the
bits
are
moving.
The
body
interprets
it
that
your
body's
not
moving
all
through
the
world,
something
else
in
the
world.
That's
moving.
D
A
D
A
Think
everything
I'm
going
to
talk
about
here
applies
to
other
sensor
models,
and
I
haven't
spent
a
lot
of
time
in
that.
But
I've
got
enough
time
to
convince
myself
that
everything
that
I'm
talking
about
here
could
apply
to
other
sentiment
dollars,
but
it's
easiest
to
think
about
vision,
because
we
have
more
data
on
vision.
A
I
think
this
general
principle
you'll
see
in
a
moment
applies
to
everything.
That
is
what
a
column
is
looking
at,
something
even
the
output
of
other
columns.
It's
going
to
have
these
two
field
of
views,
it's
going
to
have
a
large
one
and
a
small
one,
and
it's
going
to
be
building
models
based
on
those
two
field
of
views.
A
A
That's
another
general
characteristic.
I
can
show
the
papers
on
that
again,
but
a
much
larger
field
of
view
in
the
lower
layers,
and
so
this
is
the
way
it's.
It
basically
looks
like
you
have
complex
and
simple
cells
on
above
you
have
complex
and
simple
cells
below
but
below
in
layer.
Six
is
a
bunch
of
different
types
of
cells,
they're,
simple
cells
and
other
things
too.
A
There's
course
these
these
communications
between
specific
layers
between
the
upper
and
lower
parts
of
the
the
laminar
structure
and
we've
talked
about
these
a
lot
in
the
past
about
the
purpose
of
these
in
or
in
our
papers
as
well.
But
the
point
is
that
you
have
these
two
sort
of
separate
systems.
They
kind
of
look
like
separate
systems,
but
they're
they're
bi-directionally
connected
in
multiple
ways
and
the
general
theory
we're
working
on
here
is
that
when
people
talk
about
the
inputs
or
corporate
com,
they
have
usually
mentioned
layer.
A
Four,
that's
the
primary
input
layer,
but
then
there
are
actually
it's
been
very
well
documented.
There's
these
other
inputs
right
at
the
bottom
of
layer,
three
and
right
at
the
bottom
layer.
Five-
and
it's
been
shown
that,
like
the
inputs
at
the
bottom
of
layer,
three
is
sufficient
for
generating
complex
cells.
You
don't
need
any
input
into
layer
four.
So
the
idea
that
the
complex
cells
are
derived
from
simple
cells
is
not
true,
so
complex
cells
are
derived
must
be
derived
from
that
upper
input
there.
A
So
the
theory
that
we're
working
on
here-
and
I
have
no
data
to
suggest
this-
is
that
the
that
they're,
that
the
two
different
parvo
and
the
magnolia
cellular
layers
are
projecting
differently
to
these
inputs,
and
so
the
particle
cellular
would
be
going
to
the
layer
four,
which
is
where
your
spatial
pattern.
You're,
detecting
and
you'd
have
two
different
magnetocellular
layers
inputs
to
define.
A
Well,
you
know
it's
interesting.
Okay,
as
I
said,
I
don't
have
any
specific
evidence
for
this,
but
logically
this
makes
sense.
You
know
I
I've
asked
a
couple.
People
in
the
past
is
there
any.
You
know
like
well,
mostly
emerge
german,
but
maybe
some
others.
I
can't
remember.
Is
there
any
differentiation
between
the
the
where
the
different
laminar
of
the
thalamus
projection
the
cortex
and
he
said
no,
but
I'm
not
sure
I
believe
them
in
the
same
way
that
you
remember
how.
A
Last
time
I
presented
the
data
that
complex
cells
are
not
orientation
selective,
they
work
equally
well
with
random
bit
patterns.
In
fact,
they
work
better
with
random
bit
patterns.
So
it's
possible
that
and
that's
something
so
the
common
belief
about
complex
cells
is
wrong.
It's
just
wrong
data.
There's
data
that
people
forgot.
A
That
says:
that's
not
how
they
work,
they
work
differently,
and
so
I
I
I'm
not
sure
I
want
to
believe
the
data
here
yet
to
the
time,
because
there's
good
theoretical
reasons
to
to
explain
why
it
should
be
like
this,
I'm
not
denying
it
yet,
I'm
just
working
through
the
theory.
What
the
theory
would
tell
me
when
we
come
back
later
and
see
is:
is
the
data
really
conclusive
about
this,
and
what
does
the
data
tell
us?
A
You
know
it's
possible
that
I
could
have
manual
probable
cellular
inputs
to
layer
four
and
that
I
in
the
theory
could
accommodate
that,
but
I'm
not
thinking
about
that.
Yet
I
think
the
idea
that
they're
there
that
the
parvo
and
magno
cellular
layers
are
treated
equally
in
the
cortex
is
almost
patent.
It
has
to
be
false.
A
A
E
A
Yeah
well
the
way
they
do
this
is
they.
They
show
a
heck
of
an
anesthetized
animal
or
maybe
not
a
nest.
They
take
an
animal
and
they
and
they
put
some
typically
put
a
sinusoidal
grading
at
some
orientation
right
or
they
have
a
bar
or
line.
Maybe
it
goes
to
a
bar
and
one,
and
then
they
find
cells
that
respond,
and
then
they
make
that
bar
wider,
wider
and
wider,
and
then
they
see
when
the
cells
stop
responding
and
when
they
keep
responding
and
so
on.
A
So
the
testing
methodology
assumes
that
these
are
all
centered
at
the
same
point
in
space,
which
I
think
makes
sense,
but
the
testing
methodology
is
also
flawed
in
the
sense
that
again,
this
idea
that
complex
cells
respond
to
orient
line,
orientations
that
are
moving
is
false.
It's
it's
that
they
respond
to
anything
moving
in
that
direction,
including
random
bits,
and
so,
but
the
way
they
test
it
makes
it
seem
like
its
receptive
field
is
oriented.
A
E
A
D
A
Okay,
so
that's
interesting
because
in
in
primate
vision
we
have
this.
These
extra
layer,
4
cells,
which
some
people
have
argued,
it's
mislabeled
that
they
usually
shouldn't,
be
called
layer.
Four
cells.
They
should
be
called
extra
layer,
three
cells,
that's
what
some
people
have
argued
for
that,
but
so
there
they're
still
being
they're
still
separated
right.
So
we
don't.
We
don't
know
why.
There's
there
are
extra
layers
in
v1.
The
stride
could
be
one.
Presumably
it's
for
good
reason.
A
Not
all
mammals
have
stripe
v1,
so
it's
not
always
required
and
I'm
not
even
trying
to
address
that
at
the
moment.
I
think
the
point
we
can
say,
though,
is
that
they're
still
segregated
so
thompson
said:
arguing,
they're
still
segregated
they're,
just
in
a
different
calling,
a
layer,
a
b
and
c
whatever
it's
still
segregated.
Is
that
would
that
be
correct?.
A
No,
I
would
argue
they
do.
The
point
of
this
is
that
parbo
is
not
about
color.
Parvo
is
about
static
properties
that
can
be
observed
visually,
so
whether
that's
a
line
or
color
doesn't
really
matter.
It's
just
it's.
It's
a
it's
a
property
of
the
feature,
the
observed
feature
of
that
point
in
space,
so
you
still
need
parvo,
it's
not
just
about
color.
You
would
still
need
pyro
because
they
are
they
remember.
They
have
a
tonic
response.
I
could
go
back
to
this.
They
have
a
tonic
response.
A
That
is
it's
not
just
you
could
do
without
color.
You
still
need
a
tonic
response,
slow,
responding,
basically,
a
feature
that
can
be
detected
where
the
magno
cellular
cells
are
only
respond
on
and
off
and
in
the
very
fast
response,
and
so,
if
you,
if
the
eye
stops
moving
for
a
second
you
fixate,
then
the
magnum
cells
themselves
stop
stop
firing,
but
the
par
vesicles
continue
to
fire.
A
Think
it's
really
just
whether
they
have
a
my
impression
you
might
be
right
kevin.
My
impression
is
that
there's
a
difference
between
mammals
that
have
a
striate.
You
want
males
that
don't
have
a
stripe,
so
I
don't
know.
If
that's
this,
there
may
be
a
correlation
between
that
and
also
the
laminate
in
the
in
the
and
the
thalamus.
By
the
way
you
know
if
I
could
go
bring
up
that
slide.
I
have
it
here.
A
Oh,
where
is
it
or
this
one
here
passed
earlier?
Remember,
there's
like
four
there's
four
parallel
cellular
layers
and
there's
two
magnolia
layers
and
they
alternate
the
left
and
right
eye.
So
there
can
be
a
lot
going
on
here
in
the
thalamus.
This
is
a
picture
of
the
thalamus.
You
know
going
through
it,
I'm
not
I'm
I'm
trying
to
stick
to
the
real
sort
of
theoretical
implications
of
all
this,
as
opposed
to
you
know,
I'm
just
looking
at
that.
E
A
A
E
Right
and
I
I'm
trying
I'm
trying
to
leverage
as
you're
saying,
because
you
get
a
sense
of
motion,
you
know
from
those
optical
layers,
but
the
fact
that
there
is
more
than
just
two:
it's
not
like
it's
in
in
the
neocortex,
where
you
have
left
right.
You
know
bifurcation
yeah,
but
there
there's
other
types
of
of
motion,
that's
being
being
dealt
with
in
there
other
than
just
the
pure
visual.
E
E
I
I'm
I'm
I'm
saying
that
if
I
infer
that
stereo
is
happening
there
and
that
you're,
basically
differentiating
between
left
and
right,
visual
fields,
I'm
just
thinking
that
there
might
be
additional
processing
there.
That,
in
in
your
sense
of
generalizing
motion
to
about
this
being
like
visual
motion,
but
any
sort
of
sensorial
motion.
I'm
just
wondering
if
there
is
a
argument
to
be
made
that
the
intricacy
of
that
structure
means
that
it's
processing
different
types
of
deltas.
If
you
wish.
A
Maybe
I
think
it's
possible,
although
we
would
want
to
look
and
see
what
is
equivalent
in
the
thalamus
for
or
touch
for
example,
because
that's
there's
no
stereo
touching,
if
you
will
auditory,
could
be
similar
division.
So
there's
no
doubt
about
it.
But
from
a
theoretical
point
of
view,
I
don't
want
to
rely
on
having
two
sources,
like
you
know,
by
thinking
about
a
cortical
region
higher
up
in
the
cortex.
A
It's
not
getting
from
your
eyes
directly
or
anything,
it's
just
getting
from
other
regions,
and
so
we
need
a
very
generic
understanding
what's
going
on
here,
so
yeah,
but
let's,
let's
keep
going.
I
think
this
idea
that
that
you,
the
cortical
the
column,
gets
input
of
these
two
different
sources,
and
I
mean
a
sort
of
a
motion,
source
and
static
source,
and
also
that
there
are
two
different
size
receptor
fields
is
is
would
apply.
It
could
apply
everywhere
in
the
cortex
and
not
just
sensory
input
areas.
A
It's
a
general
purpose
way
of
modeling
space.
If
I
think
about
it,
this
way,
if
I
think
about
initially,
to
primary
or
secondary
visual
cortex,
where,
as
I
argued
earlier,
if
all
the
bits
are
moving,
a
very
large
receptive
field
is
moving.
That's
an
egocentric
movement
and
and
if
every,
if
only
a
subset
of
the
field
of
bits
of
flow,
is
moving,
then
that's
going
to
be
an
allocentric
movement.
A
It's
going
to
be
moving
up
something
in
the
world,
and
so
now
we
can
see
right
here
that
we
have
in
every
quarter
column.
This
argument
that
we
have
every
quarter
column
is
dealing
with
this
transformation
between
one
reference
frame
space
in
another
reference
frame
space
which
in
this
point
we
can
call
it
egocentric
and
allocentric.
It's
not
clear
to
me
other
parts
of
the
cortex.
We
want
to
use
that
term,
but
that's
what
that's
what's
happening
here.
We
see
in
our
model
of
the
quarter
of
a
column.
A
So
so
I'm
I'm
thinking
that
every
column
converts
back
and
forth
between
eagle
and
alice
center
reference
frames,
or,
more
generally,
between
a
larger
space,
an
independent,
smaller
space
and
two
different
moving
things
in
the
world.
If
you
will,
I
think
I'm
gonna
just
skip
and
skip.
I
want
to
go
back
to
my
list
of
things
at
the
top
here.
A
I'm
going
to
talk
about
this
number
five
here
again.
I've
already
mentioned
this,
but
I
think
this
is
a
really
big
idea
that
I've
started
thinking.
If
there's
two
things
going
on
here,
imagine
these
magnesium
bits
are
coming
into
the
cortex.
One
thing
that's
interesting
about
them
is
they're
not
they're,
not
directly
sensitive
in
the
thalamus,
so
the
cortex
isn't
getting
directionally
sensitive
bits.
It's
just
getting
bits
that
are
changing
on
and
off
very
rapidly,
because
the
inputs
are
changing
on
and
off
it's
it's
there's
no
directionality
to
it.
A
A
That
apparently
has
to
happen
in
the
cortex
with
so
right.
At
the
moment
we
don't
have
a
mechanism
for
that.
I've
proposed
earlier,
I
said:
hey
these
magnus
cellular
bits
could
be
coming
in.
We
run
it
through
a
spatial
pool
type
process
and
that
spatial
pull-up
type
process
will
figure
out
a
set
of
movement
vectors
that
represents
the
all
the
possible
flow
patterns.
A
But
in
addition,
there
has
to
be
something
that
says:
hey,
I'm
not
getting
flow
bits,
I'm
getting
it's
on
offense
and
I
have
to
somehow
determine
that
a
sequence
of
these
is
occurring
very
rapidly,
as
movement
occurs,
and
I
only
mention
this
because
well,
it
has
to
occur,
and
there
are.
There
are
observations
about
pyramidal
cells.
That
say
that
the
order
in
which
the
activation
along
a
dendrite
occurs
is
is
important.
A
That
is,
if
you
have
a
pattern
going
from
distal
to
proximal
along
the
dendrite,
the
cell
will
respond
much
more
strongly
than
it
was
going,
the
other
way
or
it
wouldn't
respond,
they're
all
going
that
way.
We
have
not
accommodated
these
in
our
theories
of
the
dendrites
in
the
neurons
so
far,
but
this
is
suggestive
that
we're
going
to
have
to
have
something
like
that,
and
that
is
there
is
data
on
that
which
suggests
that
there
may
be
inherent
to
at
least
some
of
the
cells
in
the
cortex.
A
This
directionality
very
fast
directionality
dependent
movement,
because
the
cortex
has
to
determine
that
these,
the
sequence
of
bits
that
are
going
on
and
off
fairly
rapidly
as
your
eyes
move
or
as
things
in
the
world
move
has
to
be
detected
in
the
neurons,
and
we
currently
don't
have
a
method
for
that.
So
that
has
to
occur.
A
But
then
the
other
thing
is
that's
that's
in
here,
and
I
mentioned
this
ebola
number
five
is,
and
I've
said
this
before.
Is
that
to
me?
Also
nice,
you
know
in
when
we
think
about
the
hippocampal
complex.
We
think
of
head
direction.
Cells
as
one
thing
and
grid
cells
is
another
thing,
they're
like
two
different
worlds
that
have
to
interact
and
we've
drawn
pictures.
I've
drawn
pictures
in
the
past
and
saying
oh
well,
to
know
which
way
my
grit
cells
are
to
be
upgraded.
A
A
I
think
that
that
when
you
think
about
the
flow
patterns
that
cortex
may
see,
they
can't
tell
the
difference
between
a
flow
pattern.
That
means
changing
orientation
and
flow
pattern.
That
means
I'm
moving
forward
and
moving
backwards.
They're
just
flow
patterns,
the
cortex
doesn't
know
this.
It
just
discovers
whatever
flows
are
out
there
and
it
makes
it
runs
into
a
spatial
pool
type
process
and
says:
okay.
A
These
are
the
movements
that
I'm
observing,
but
it
doesn't
know
that
one
movement
is
an
orientation
change
and
another
movement
is
linear
or
something
else
it'll
just
learn
to
represent
your
behaviors,
and
so
this
study
made
me
think
of
like
what.
If
I
just
say,
there
is
no
distinction
in
head
direction,
cells
or
orientation
cells
and
what,
if
it's
just
there's
just
movement
vectors
and
those
movements
do
not
have
to
correspond
to
linear
dimensions
in
space,
so
we
have
a
physical
space,
a
bunch
of
x
y
z,
type
of
dimensions.
A
If
you
want
to
consider
that,
but
that's
not
what
the
cortex
is
representing,
the
cortex
is
representing
movements
to
that
space
and
which
may
or
may
not
capture
the
dimensionality
of
the
real
space.
It
depends
on
how
I
move,
but
it
models
everything
in
these
movement
vectors,
not
in
spatial
dimensions.
So
there
are
movement
dimensions
but
they're,
not
spatial
dimensions,
they're,
not
necessarily
corresponding
to
linear
dimensions
in
the
space
of
the
object
of
being
modeled,
and
this
is
a
very
liberating
idea.
All
of
a
sudden,
I
can
start
thinking
about
orientation
completely
independently.
A
It's
not
really
orientation,
it's
just
movement
vectors
and
some
will
correspond
to
turning
and
some
I
could
have
a
movement
vector,
it
might
be
a
spiral
motion
and
so
on,
whatever
is
observed,
that's
what's
going
to
be
represented,
and
then
I
can
think
about.
Then
everything
that
applies
to
what
we
think
about
applies
to
grid
cells
such
as
path,
integration
and
displacement
cells
are
occurring
in
all
these
dimensions
of
movement,
including
what
we
might
call
a
change
in
orientation.
A
So
if
I
was
thinking
about
a
head
direction
cell
and
I'm
moving
left
and
right,
sweeping
left
and
right
and
the
head
direction
cell
would
be
changing.
Well,
it's
really
just
doing
path.
Integration
in
that
movement
vector-
and
I
would
also
calculate
displacements
between.
If
I
want
to
do
it's-
you
know-
figure
out
the
relative
displacement
of
one
space
in
another
space.
A
I
would
do
it
across
all
these
dimensions,
including
orientation,
so
I
would
represent
changes
in
orientation
as
part
of
a
displacement
vector
and
I've
been
working
through
this,
and
I
think
this
solves
the
orientation
problem
in
displacement
cells
that
we've
talked
about
in
the
past.
So
I
haven't
been
able
to
work
that
out
in
great
detail
yet,
but
I
I'm
convincing
myself
that
it
would
happen.
C
A
A
How
do
I
determine
which
way
I'm
moving
in
in
physical
space?
It's?
I
don't
need
to
answer
that
question.
I'm
just
moving
this
direction
in
this
in
this
movement
space
and
by
the
way,
each
of
these,
these
I'll
call
them
one-dimensional
path.
Integrations
they
have
to
re-anchor
all
the
time.
Just
like
we
do.
A
People
talk
about
good
cells,
and
so,
but
I
can,
I
can
somehow
it's
a
lot
of
problems
that
come
about
trying
to
go
back
and
forth
being
movement
and
space
to
disappear
when
you
work
all
in
this
movement
pointing
in
space,
let's
say
so
number
six
is
what
we
just
talked
about.
Minicom
represents
a
1d
movement
vector
and
only
only
it
only
means
that
path.
Integration
displacements
are
in
one
d:
it's
not
that
they're
that
this
one
dimension
is
like
a
dimension
in
space
or
even
that's
a
linear
dimension
in
space.
A
It's
just
the
movement
space.
I've
been
kind
of
struggling
with
the
term.
I
I
want
to
call
these
like
1d
grid
cells,
but
they're,
not
really
grid
cells
and
and
it's
very
confusing,
so
I've
been
calling
them
like
path,
integration,
cells
or
something
like
that.
I
don't
know
what
to
call
them
and
then
the
set
point
number
seven.
I've
already
made
how
path
integration
and
displacements
apply
in
the
point
orientation
of
whatever
type
of
movement.
A
A
What's
going
on
in
hippocampal
complex,
maybe
the
hippocampal
complex
has
got
more
a
specific
solution,
but
I
would
if
I
had
to
guess
I
would
guess
that
we've
been
misinterpreting
what's
going
on
in
the
hippocampal
complex,
remember
when
people
look
for
grid
cells,
they're,
actually
they're,
far
more
cells
that
are
combinations
of
grid
and
orientation
and
so
on.
Then
there
are
pure
grid
cells
and
and
even
the
pure
grid
cells,
don't
work
reliably
as
grid
cells,
as
we've
shown
in
the
tank
paper.
A
C
A
A
Yeah
and
I
think,
by
the
way,
I
think
the
whole
two-dimensionality
of
good
cells
is
also
an
artifact.
I
think
what's
going
on,
and
I
I
can't
work
on
detail
if
I
mentioned
this
before
I'll
mention
it
again,
I
think
everything
happens
in
one-dimensional
spaces,
one-dimensional
movement
vector.
So
why
would
you
see
a
2d
grid
cell?
What
we,
as
the
hybrid
models,
have
shown?
A
It's
like
it's
like
I'm
taking
these
1d
vectors,
I'm
projecting
them
down
into
a
2d,
a
2d
physical
space
in
the
cortex,
putting
a
can
on
it.
So
I
can
maintain
stability,
but
then
I
would
probably
project
back
to
the
individual
path,
integrated
neurons
for
movement
and
so
on.
So
it's
this
whole
idea
that
the
that
grid
cells
are
literally
two
different,
really
representing
two
dimensions
in
space.
It's
an
artifact,
the
path
integration
doesn't
occur
in
2d
and
path.
A
Integration
occurs
in
1d
and
all
the
all
the
models
about
how
we
create
grid
cells
make
that
assumption
they're
all
making
the
assumption
that
we
have
these.
You
know
voltage-controlled
oscillators
in
one
dimension,
that
would
take
a
bunch
of
them
to
intersect,
and
then
you
have
a
2d
grid
zone
anyway.
I
think
what's
really
interesting
about
this,
and
I'm
going
to
run
out
of
steam
here
is.
Is
I've
been?
A
I
started
off
by
trying
to
understand
how
we
could
do
displacements,
because
I
think
I
think
displacements
are
are
actually
happening
in
the
brain.
We
have
to
go
between
a
a
specific
do.
I
have
a
slide
on
that.
I
can't,
if
I
put
a
slide
in
about
that,
I
was
working
on
this.
Let
me
see
if
I
have
something
on
there.
A
No,
I
don't
want
to
slide
on.
I
have
drawings
of
it
in
my
notebook,
just
let's
just
blank
that
for
a
second
just
recall,
you
know
we
were
trying
it
with
displacements.
We
were
trying
to
figure
out
how
it
is.
We
can
do
composite
objects
like
the
coffee
cup
with
the
logo,
and
we
were
struggling
with
this.
A
How
is
I
can
represent
a
new
object
by
saying
here's
one
object,
which
is
a
logo
and
another
object
which
is
a
coffee
cup
and
how
to
make
a
new
object,
which
is
the
combination
of
the
two,
and
I
want
to
do
that
very
quickly
and
with
very
few
synapses,
and
so
I
can't
remember
who
came
up
with
it.
I
know
that
scott
was
involved
and
marcus.
A
You
were
involved,
I
don't
know
if
ua
was
still
here
or
not,
but
anyway
you
guys
came
back
with
this
proposal
for
a
grid,
what
I
call
displacement,
cells
and
and
despite
themselves,
basically
saying
hey.
If
I
have
two
different
locations
I
can,
I
can
calculate
an
invariant
representation
that
shows
the
relationship
between
them.
That
is
and
so,
and
so,
if
I
had
a
bunch
of
this
grid
cell
modules,
I
had
a
bunch
of
displacement
cell
modules.
A
Then
I'd
have
a
very
simple
and
unique
representation
for
the
how
the
logo
aligns
with
the
coffee
cup.
The
problem
with
that
is,
it
only
worked
if
the
two
spaces
were
exactly
the
same
sort
of
size
and
they
did
no
changes
in
orientation
between
them.
So
it
was
the
right
idea,
but
it
had
some
very
serious
flaws
and
we
pointed
those
out
in
our
papers
and
pointed
out
that
doesn't
really
work
for
scale
and
rotation,
so
it
needed
more
work.
A
I
I'm
thinking
so
I
started
trying
to
solve
that
problem
and
the
first
thing
I
did
I
said:
let's
try
to
do.
I
I
don't
have
any
slides
on
this.
I
said.
Let
me
try
to
do
scale
in
one
dimension.
Like
imagine,
I
have
a
one
dimensional
space
and
I
want
to
represent
two
one-dimensional
objects
at
different
scales
and,
and
so
there
there
clearly
has
to
be
a
mechanism
to
do
that.
That
allows
me
to
change
the
scale
of
an
object
to,
and
I
think
this
is
going
on
in
the
thalamus.
A
We
have
a
mechanism
for
this
or
a
proposed
mechanism.
You
could
change
the
theta
frequency
and
I
would
change
the
scale
of
something
and
the
more
I
thought
about
it.
The
more
I
realized
even
to
do
basic
inference.
You
have
to
have
a
scale
factor
that
scale
is
an
inherent
part
of
the
problem
of
doing
inference
and
learning
and
displacements,
and
so
the
the
working
hype
assumption
we
have
is
that
scaling
is
occurring
in
the
interaction
between
the
cortex
and
the
thalamus.
So
I
didn't
solve
that
problem.
A
Then
I
started
thinking
about
the
the
the
change
in
orientation
of
the
logo
on
the
coffee
cup
and
I
think
this
new
way
of
thinking
about
movement
vectors
solves
that
problem.
That
is
you.
If
you
take
all
the
displacements,
including
displacements
of
of
orientation,
then
you
can
look.
You
can
learn
quickly,
assess
the
orientation
of
an
object.
Voltage
is
another
object,
and
I
thought
about
that.
There's
some
complexity
to
them.
A
I
think
it
works,
and
so
now
I
now
basically
have
to
say:
okay,
we
have
to
add
in
this
sort
of
scale
scalar
part
as
well,
and
then
I
think
you
have
all
the
components
that
are
necessary
to
make
this
work.
So
the
more
I
thought
about
this,
I'm
pretty
confident.
This
whole
thing
I
just
talked
about
is
correct.
A
You
know,
even
though
there's
very
little
data
about
it,
it's
just
one
of
these
things
that
just
hangs
together
so
well,
so
many
problems
and
constraints
that
it
seems
right
to
me
and,
as
I
dig
into
it,
I
find
more
physical
evidence
that
supports
it,
even
though
you
know
some
of
it's
equivalent
anyway.
So
that's
that's
all
I
have
for
today.
A
I
I
just
felt
like
I
needed
to
talk
about
this
in
a
group
and
and
just
see
if
I
could
present
it
make
some
sense
of
it
and
I'm
struggling
how
to
talk
about
it.
What's
the
right
process,
if
I
wanted
to
write
this
up,
how
would
I
write
it
up?
I
don't
really
know
so.
I
thought
maybe
just
presenting
it
today
would
help
you
think
about
that
and
I'd
love
to
hear
questions
and
more
comments.
If
you
have
it
or
debate.
C
Yeah,
so
I
have
one
other
one
connection
I
want
to
draw
here
and,
and
for
me
it's
almost
tempting
for
me
to
collapse
your
idea
onto
this
other
thing,
but
I
I'm
keeping
them
separate
right
now.
Another
area
like
known
as
manifold
learning,
where
you
map
you
know
sensory
inputs
into
some
low
dimensional
euclidean
space,
which
might
be
2d
might
be
3d,
might
be
60
whatever
this.
This
has
a
lot
of
resemblance
to
that,
and-
and
at
least
one
thing
I
want
to
contribute
back
from
the
from
that
idea.
C
C
A
D
C
Understand
it
so,
let's
see
I
mean
that
that
is
a
good
reason,
though,
because
that
lets
you
predict
multiple
steps
ahead
by
by
you,
don't
have
to
have
the
constant
feedback
of
seeing.
If
your
predictions
are
right,
you
can
predict
what's
going
to
happen
10
seconds
from
now,
or
maybe
a
shorter
time
span,
but
the
the
constant
rate
part
is.
I
don't
know
this
is
just
your.
This
is
just
a
principle
for
mapping
sensory
inputs
into
a
space
and
figuring
out.
What
are
the
directions
of
the
space
here?
C
C
We
need
some
principle
telling
us
what
it
should
be
so
like
if
an
animal
moves
in
kind
of
a
strange
motion,
or
something
like
that,
maybe
this
should
cause
a
straight
line,
but
if
they
as
they're
as
they're
curving,
maybe
in
you
can
bring
up
other
examples.
You
could
bring
up
like
when
I've
thought
about
it
a
little
bit.
C
E
Linear
you
know
for
some
time
span
so
that
you
have
some
degree
of
predictability
and
but
you
still
might
have
a
sense
that
if
you
know
you
see
a
line
continuing
to
infinity
you
know
and
it's
static,
then
you
might
be
able
to
hold
that
and
saying
I
can
kind
of
keep
that
prediction
going
and
go
off
and
look
at
what's
delta.
You
know
other
things
are
going
on,
so
I
I.
C
A
Yeah,
it's
not
just
I
mean
linear
linear
in
is
linear
to
what
I
guess
it's
like
to
me.
If
I,
if
the
way
I
would
phrase
it
you've
got
slow
bits
on
the
screen
and
your
view
and
those
are
moving
in
some
pattern.
That
pattern
has
to
continue
that
pattern,
can't
change
in
the
middle
for
one
mini
column,
one
dimension,
it's
like
if
I'm
turning.
If
I'm
turning
left
that
that
column
would
always
represent
turning
left,
it
can't
represent
turning
left
one
moment
and
then
facing
up
another
moment
right.
A
You
can
go
left
faster
or
slower.
Turning
left,
I
can
move
slowly
to
the
left,
so
I
can
turn
quickly
to
the
left,
but
the
same
flow
pattern's
occurring
now
to
me.
That's
that
to
me,
if
that's
what
you
mean
so
it's
for
me,
it's
not
piecewise,
linear!
It's
continuous!
If
I
think
about
linear
in
the
movement
vector
it's,
it's
always
linear
in
the
movement
vector
it's
weird,
because
it's
not
a
line.
A
Active
yeah,
so
the
way
I
handle
that
it's
like
that's
how
I
think
about
the
space
of
pulicon.
If
I,
if
I
have
a
pattern
that
works
into
another
pattern,
well,
a
different
set
of
mini
columns
become
active,
so
any
particular
movement,
but
imagine
I'm
moving
through
space.
Unless
I
have
200
mini
columns,
some
subset
of
them
will
be
active
at
different
rates.
A
The
the
mini
column-
that's
representing
you
know,
forward
movement
will
be
partially
activated
if
I'm,
if
I'm
moving
and
turning
to
the
left
or
and
so
you'll
have
like
you'll,
be
you'll,
be
a
combination.
You're
always
going
to
get
like
a
sparse
representation
of
graded
responses
from
a
set
of
a
subset
of
the
many
columns,
but
the
individual
minicom
will
always
represent
the
same
thing,
and
so,
if
I
stop
turning-
and
I
want
to
go
forward
now,
then
that
minicom,
that
was
representing
tournament,
is
going
to
be
silent.
A
A
E
Right,
that's
that's
selectivity,
but
I'm
thinking
about
how
you
learn
the
thing.
If
something
is
basically,
let's
use
your
left
turn
example.
So
you're
going
along
one
direction
and
then,
if
I
was
just
thinking
you
know
mentally,
as
you
know
how
this
you
know
direction
projects
upon,
you
know
like
a
like
a
you
know.
You
know
cosine
type
of
thing:
it's
going
to
respond
less
and
less
to
that
thing
and
more
and
more
with
something
else.
So
you
have
spatial
pullers
in
front
of
both
of
these
things,
saying
I'm
moving
in
this
direction.
E
A
Well,
I
I
I
don't
know,
I'm
not
sure
I
understand
question
remember
the
spatial
cooler
is
a
continuously
learning
system
and
it
just
price
it
tries
to
figure
out
the
a
common
basis
set
of
patterns
and
and
it'll.
If
the
pattern
change,
it
changes
what
it
learns
changes,
but
it's
it's
basically
trying
to
figure
out.
You
know
a
set
of
the
most
common
patterns
it
can
observe,
and
so
I
don't
that
same
issue
applied
when
we
use
the
spatial
polar
for
static
patterns
as
it
would
apply
here.
A
So
I
think
I
I
again
in
the
end
at
any
point
in
time,
a
mini
com
would
represent
a
particular
type
of
pattern
and
if
the
behavior
of
the
animal
changes
and
then
that
behavior,
you
know
if
the
animals
started
walking
in
circles
all
the
time.
Well
then
it
would
change
those
patterns
would
change,
but
again,
I'm
not
sure
if
you're
arguing
that
the
column
has
to
do
both.
I'm
not
sure
if
you're
arguing
about
that.
E
A
No,
I
think
the
minicom
doesn't
represent
anything
more
semantic.
It
doesn't
represent
anything
semantic
or
anything
like
that.
It's
just
a
it's
just
in
this
case.
It's
just
the
flow
pattern.
It's
seen
and
okay,
semantics
semantics
comes
later.
Essentially,
the
mini
columns
are
defining
a
space,
but
the
space
is
a
bunch
of
movement
vectors.
It's
not
it's,
not
a
bunch
of
linear
dimensions.
A
A
A
E
A
E
A
A
A
B
E
I
I
I
think
the
interesting
thing
that
marx
is
bringing
up
is
is
that
you
jeff,
have
defined
a
space
and
it's
a
question
of
what
navigation
looks
like
in
that
space
when
something
departs
from
say
whatever
the
thing
that
was
exciting
along
this
one
dimension
thing
goes
someplace
else.
You
have
all
these
other
things
that
are
now
responding.
Instead
of
that,
and
can
you
say
anything?
Can
you
visualize
a
representation.
A
A
But
but
you
know
even
a
non-learned
spatial
puller
works,
it's
like
you
just
you
can
have
start
off
with
a
random
parceling
of
the
input,
but
it
gets
better
if
you
have
training.
So
it's
not
like
the
system
would
fail
of
all
of
a
sudden
I
started
seeing
input
patterns
occur
that
you
know.
Maybe
I
have
some
physical
defect
and-
and
I
start
walking
always
never
in
straight
line,
always
working
sort
of
a
semi
curve
or
something
like
that.
A
Well,
at
first
it
wouldn't
be
the
best
representation
I
have
in
my
brain.
I
would
be
confused
at
times,
but
but
it
would
learn
that
and
it
would,
but
it
would
work
it
wouldn't
be
a
failure.
I
don't
know
how
to
better
describe
it
than
that.
The
spaceship
always
parses
up
the
input
into
something
meaningful,
not
a
matter
how
it's
set
up
even
randomly
it'll.
Do
it.
E
No,
I
I
wasn't
suggesting
it
would
it
would,
it
would
fail.
I
was
I
was
looking
at,
is
that
you
you've
staked
out
you've
staked
out
a
space
where
all
these
things
have
have
flows
and
in
you
can
represent
that
as
a
as
a
giant.
You
know,
you
know
hyper
dimensional,
vector
that
all
these
things
have
certain.
You
know,
you
know
speak
to
something
that
is
received.
E
It
received
input
and
it's
either
active
or
not
active
or
less
active
or
active,
and,
and
that's
that
it's
fine,
I
I
can
believe
that
at
the
solar
level,
there's
there's
no
semantics
and
you
would
want
to
see
something.
You
know
you
want
to
feel
something
like
that.
I
I
think
what
what
marcus
is
trying
to
do
is
that
if
you
take
that
as
a
as
a
very
high
dimensional
space,
are
there
organizational
principles
that
you
can
pull
from
the
math
in
that
space?
E
That
might
actually
reflect
the
the
the
next
order
up
where
you
go
up
to
the
complex
cell.
Excuse
me,
where
you
go
up
to
the
to
the
column,
are
or
things
like
that
kind
of
intuited
inferred
and
learned
in
some
way
that
we
could
actually
say
something
about
it.
You
know
and
say
this
operation
is
going
on.
It
could
be
a
language
to
express
that
it.
A
Could
be,
although,
as
I
started
off
in
my
comments
here,
I
find
that
the
language
of
math,
in
my
case
at
least
it
always
has
to
follow
the
understanding,
the
intuition
of
what's
going
on
at
a
mechanism
level
and
that
I've
never
been
able
to
have
the
math
lead
me
to
the
right
place.
It's
the
math
basically
colors
and
describes
better
something
I
already
understood.
But
if
I
follow
the
math,
then
you
end
up
with
with
principles
that
almost
never
match
the
biology.
E
A
You
need
to
do
both
yeah,
but
until
I
understand
physically
what
the
neurons
are
doing,
I
don't
trust
the
math,
I'm
almost
just
like
well
yeah,
okay,
maybe
it's
an
eigenvector
thing
above
about!
Maybe
maybe
I'll
get
some
insight
from
that,
but
I
can't
let
I
I
gotta
get
I
end
up.
I
have
to
come
back
for
the
biology.
I
was
like
okay,
these
cells
are
doing
this
and
they're
connecting
this
way
to
the
part
of
the
dendrites
and
that's
why
it
works.
I.
E
I
I
understand,
and,
and
that's
fair
I
would
just
say
that
if
you
have
a
representation
in
the
math,
what
you're
talking
about
is
there
are
constraints
on
what
we
could
say
about
that
representation
which
you're
you're
trying
to
intuit
from
you
know.
What
does
the
biology
tell
you,
but
I
think
it's
expressible
in
there
as
a
as
a
set
of
overlapping
constraints.
That
say
what
actually
is
happening
is
this
is
constrained
in
the
following
fashion
for
these
following
reasons,
and
that's
that's
something
that.
A
A
Email,
community,
you
know
it's
possible
that
you
know.
I
always
use
the
example
of
a
computer
right,
the
computer,
the
cpu
when
the
computer
works.
I
can
understand
it
from
an
engineering
point
of
view,
but
it's
really
difficult
to
put
any
kind
of
mathematics
on
it
and
and
so
maybe
there's
some
parts
you
can.
You
can
play
shannon
information
theory.
You
can
apply
bus,
the
mathematics
of
how
buses
work
and
and
so
on,
but
but
it
doesn't.
E
A
A
Know
how
computer
works
it's
so
again
it
needs.
You
know.
You've
got
to
have
both
but
which
one
leads
is
the
question
for
me.
So
I
think
how
do
you
you
know
we
have
to
understand
what's
going
on
and
then
the
math
can
come
along
and
tell
us
it
can
form
it's
like.
Oh
it's
a
clear
explanation
for
it.
Let
me
just.
A
Idea
that
you
know
we're
really
doing
with
movement
vectors,
there's
no
way
that
a
mathematical
analysis,
I
think,
would
have
told
you
that
there's
no
way
you
could
do
all
the
math.
You
want
on
spaces,
and
you
would
never
come
up
with
that
idea,
but
a
simple
observation
about
what
you
see
when
you're
watching
a
video
game
does
it
so
anyway,
it's
just
a
bias
of
mine,
but
but
it's
it's
more
than
a
bias.
I
think
it's
actually
an
essential
component.
So
again,
I
welcome
that
now.
A
A
C
C
If
I
mean
at
the
risk
of
having
collapsed
your
idea
down
too
much
right
now,
I'm
right
now,
I'm
picturing
a
space,
maybe
2d,
maybe
3d,
maybe
some
other
relatively
low
number,
with
sensory
inputs
being
mapped
into
it
and
and
the
dimensions
of
that
space
are
kind
of
like
your
flow
dimensions
and
but
it's
still
low
dimensional.
So
there's
something
missing
there,
because
if
every,
if
every
mini
column
is-
and
it
is
an
entire
dimension-
that's
very
high
dimensional
there's.
B
C
Right,
yes,
okay,
I
got
it
got
it,
got
it
cool
you
that
that
explains
it.
Then
then,
so
the
fun
idea
here
is
like
being
able
to.
So
when
we
talk
about
displacement
cells
and
their
ability
to
do
composition,
we've
compared
them
to
wormholes
we've
compared
them
to
just
like
taking
spaces
and
putting
them
on
top
of
each
other
and
and
lining
them
up,
and
that
ability
combined
with
this
is
it
sounds
magical
and
interesting,
and
it
sounds
like
it
would
be.
C
C
A
I
don't
remember
even
the
lining
of
the
two
space
I
mean.
What
I
remember
was
I
mean
displacement
of
representation
is
essentially
an
environment
representation
and,
and
what
I'm
proposing
now
is
that
this
is
done
in
one
dimension.
So
even
if
I
had
10
cells,
let's
say
I
have
ten
cells
that
represent
the
different
path:
integration,
neurons-
that
are
cycling
through
this
one-dimensional
movement
vector.
A
So
I
go
from
one
two,
three,
four
five
up
to
ten
back
to
one
again,
so
it's
cycling
through
over
and
over
jim,
then
I
would
have
ten
displacement
cells
and
they
would
be
displaced,
and
so
I
still
have
to
figure
out
how
to
learn
that,
but
the
hell
of
a
lot
easier
than
the
two-dimensional
spatial
displacement,
cells
behind
and
and
so
and
so
I'd
have
to.
Basically
one
of
those
displacement
cells
represents,
any
sort
of
you
know
get
movement
of
two
or
whatever.
A
It
is
in
the
ten
up
above
or
something
like
that.
So
to
me
that
each
displacement
is
a
one
dimensional
displacement
vector
which
is
much
easier
to
calculate
and
and
I'm
I'm
going
back
to
this
idea-
that
every
mini
column
is
its
own
dimension,
because
then
I
have
enough
of
them.
I
could
have
200
to
900
mini
columns
in
a
column
and
each
one
would
be
like
a
10
cell,
one
dimension
path,
integrator
and
displacement
thing,
and
so
now.
A
A
C
A
Yeah,
although
you
know
it's
funny
the
way
I've
been
thinking
about
this
lately,
I
realized
that
we
want
to
say:
oh
there's,
a
bunch
of
dimensions
which
define
a
space,
and
now
I
have
the
space
and
I
can
think
about
it.
The
way
I've
been
thinking
more
like
is
like
I
never
really
that
that
larger
space
doesn't
really
exist.
A
It's
just
a
whole
bunch
of
one-dimensional
things
and
each
one
of
the
natural
things
calculates
its
own
thing
and
oh,
I
can
look
at
the
set
of
them
and
I
can
say
here's
where
I
am,
but
actually
when
it
comes
to
movements
and
when
it
comes
to
figuring
out
displacements
and
when
it
comes
to
like
how
do
I
update
my
my
my
my
cells,
I
don't
think
of
the
larger
space.
I
only
have
to
think
about
my
one
dimension.
A
A
I
don't
think
I
have
to
do
that
anymore.
I
just
have
to
have
this
one
dimensional
thing
and
it
path
integrates
based
on
flow
information.
If,
if
the
whole
thing
rotates,
it
will
flow
information
changes
and
nobody
actually
has
to
think
about
the
entire
space.
As
in
terms
of
a
metric,
I
think,
of
the
entire
space
in
terms
of
assigning
some,
you
know
location,
but
all
the
updating
of
the
space.
These
are
just
independent
vectors.
This
goes
back
to
the
you
know.
A
It's
only
when
I
want
to
represent
a
point
in
that
larger
dimensional
space,
and
I
look
at
all
the
individual
pieces,
but
all
the
metric
properties,
distance
path,
integration
so
on
motor
behaviors,
all
independent,
which
is
a
very
freeing
concept.
I
no
longer
have
to
figure
out
how
to
combine
music.
I
know
that
helps.
I
don't
know
if
that
even
really
is
what
you
said.
C
Somewhat
I'll
jump
to
my
second
thing
and
now
hopefully
my
second
thing
is
the
shorter
thing.
So,
on
the
topic
of
the
metric
of
this
space,
we're
treating
it
less
like
the
metric.
Is
you
know
meters?
It's
not
it's
not
some!
It's
not
some
fixed
metric
map,
but
maybe
there
is
a
metric,
but
it's
more
something
else.
One
easy
thing
that
comes
to
mind
is
time.
Two
things
that
are
the
distance
between
two
points
in
this
space
is
the
typical
amount
of
time
that
passes
between
them.
A
C
A
B
C
C
A
That
would
be
true.
It's
funny,
you
know,
think
about
like.
I
want
to
get
from
point
a
to
point
b
and
I
did
the
displacement
all
of
my
all
of
my
different
dimensions.
Well,
the
ones
that
represent
orientation
will
just
say.
Oh
you
need
that.
You
need
to
turn
this
much.
You
know
and
another
one
says
you
need
to
go.
You
need
to
you
know
once
you
turn,
you
have
to
go
like
an
interaction,
but
I
think
it's
like
oh
give
me
where
I
am
right
now.
A
If
I'm,
if
I
was
a
linear
dimension
and
I'm
and
I'm
linear
dimension,
I'm
not
put
in
the
right
direction
right
now,
so
my
displacements
are,
you
don't
have
to
move
at
all,
but
as
soon
as
the
as
soon
as
the
orientation
direction
starts
moving
you
then
my
linear
directions.
Oh
no,
I
do
have
to
start
moving
in
this
direction.
I
already
pointed
out
this
this.
This
leads
to
this
is
again
the
separation.
All
these
different
movements.
I
don't
have
to
coordinate
each
little
column
on
its
own.
Many
columns,
don't
figure
out.
A
Should
I
be
moving
now
or
not
and
as
things
move,
they
all
sort
of
update
until
everyone
heads
in
the
right
direction,
so
I
don't
know,
I
think
it's
a
really
hard
thing
to
think
about
anyway.
I
I
I
don't
even
know
how
to
begin
writing
this
up.
I've
tried
writing
this
up.
Already,
I'm
going
to
work
on
next.
I
think
I'm
going
to
try
to
understand
the
scaling
issue
better.
A
A
D
Yeah,
I
think
the
language
has
to
be
a
little
more
precise
and
stuff,
but
it
seems
like
well
it's
like
what
what
about?
I
think
the
concepts
are
pretty
it's
just
it's
just
about
like
displacement,
displays
displacement,
vectors,
for
example.
It's
just
hard
to
think
about
differences
in
movement.
You
know
it's
just
because
movement
itself
is
are
already
different.
It's
just
a
little
bit
harder
to
think
about,
but
it
seems
like
it
should
be.
It
should
be
possible
to
describe
it
very
clearly.
A
D
A
D
D
A
A
D
A
It
could
literally
be
if
I,
if
I
have
to
move
two
cells
forward,
then
the
displacement
is
too
cell
phone.
You
know
it's
like
no
matter
where
I
am.
I
gotta
move
two
cells
forward
or,
if
so,
if
I'm,
if
I'm
turning
left
two
cells,
meaning
you
know
two
path:
integration
cells.
If
you
go,
then
no
matter
where
I
am,
I
have
to
turn
left
to
south.
That's
the
displacement,
so
it
doesn't.
A
D
A
Like
it's,
it's
a
displacement
in
the
way
we
defined
it
in
the
paper
that
you
have
a
path
integration
system
and
the
displacement
is:
is
a
relative
movement,
independent
of
a
location
in
that
path,
integration
system
right.
So
if
it's
you
know
in
the
2d
sense,
it
was
two
cells
over
and
one
cell
up
or
something
like
that.
Yeah.
D
A
D
A
Bunch
of
cells
represent
orientation
right.
It
says
no
matter
what
your
current
orientation
is.
You
want
to
do
two
shifts
to
the
left,
no
matter
what
it
is.
So
it's
a
it's
an
environment
representation
of
movement
in
that
in
that
path,
integrator
so
from
wherever
you
are,
it
doesn't
matter
it's
a
relative
shift
from
wherever
you
are.
D
A
D
D
A
D
A
You
know
it's
it's
a
scale.
Every
minute
column
is
a
path
integrator,
that's
what
it
is.
It's
just
a
pattern
grid,
so
you
have
and
and
how
quickly
you
move
through.
Your
active
cells
is
depending
on
the
velocity
of
your
movement
along
that
direction.
So
it's
no
different
than
before.
It's
just
like
a
two-dimensional
grid
cell,
except
these
are
one-dimensional
cells
right
and
they're,
not
representing
spatial
dimensions
representing
something
else.
A
So
I
the
same
principle
if
I
want
to
form
an
environment,
well,
either
representation
of
an
object,
or
if
I
want
to
assign
the
location
of
one
object
to
another
object,
then
it
seems
like
the
displacement
is
the
thing
I
need
I
need
to.
I
need
to
say
how
much
do
I
to
go
from
one
I'm
looking
at
a
point
on
the
coffee
cup,
and
I
want
to
get
a
point
on
the
logo
I
have
to.
I
have
to
accommodate
all
the
different
I
have
to.
A
D
D
A
If
I
understand
yes
but
but
but
but
remember
if
I
stop
moving,
there's
a
set
of
cells
that
stay
active,
they
represent
where
I
am
right,
so
the
movement
is
just
how
I
update
them.
It's
like
grid
cells,
get
updated
by
movement
too
right.
We
just
assumed
that
was
movement
into
x
and
y
dimensions.
I'm
saying
the
movement
there's
still
there's
still
a
static
cell,
that
represents
a
location
and
when
I
move
that
it's
updated,
but
if
I
stop
moving,
there's
a
static
cell.
A
So
if
I,
if
I
were
to
look
at
if
I
were
to
take
a
flash
image
and
nothing's
moving
on
the
screen,
then
I'd
have
a
set
of
cells
that
are
active
that
are
not
really
representing
movement.
They
represent
a
location
along
the
movement
direction,
movement
vector
it's
just,
but
it's
a
location,
longer
movement,
it's
just
a
it's.
No
different
than
saying
a
grid
cell
represents
some
repetitive
pattern
in
some
linear
dimension.
Here
I
have
a
repetitive
pattern
in
some
movement
dimension,
but
it's
still
a
static
pattern.
A
A
A
D
A
It's
it's
the
same
as
before.
Right
a
grid
cell
represents
a
set
of
bit
cell
modules.
If
I
look
at
the
active
cells
instead
of
grid
cell
molecules,
it
represents
a
static
location
right,
so
you
can
now
imagine
doing
that
with
one
dimensional
grid
cell
modules.
Right,
and
so
this
is
exactly
like
that
a
set
of
the
set
of
cells
that
are
active
represents
a
specific
location,
but
it's
not
it's
it's
a
location
so.
D
A
Yeah,
but
it
will
but
except
the
orientation
may
be
different
when
you
reach,
so
it
depends,
but
if
you
have
a
single
finger
on
the
same
spot,
the
same
location
doesn't
matter
you
got
there
like.
I
think
everything
that
applied
to
grid
cells
applies
here.
It's
just
the
difference
is
we've
always
assumed
that
grid
cells
represent
linear
dimensions
in
physical
space,
and
now
I'm
saying
they
represent
quote
some
sense,
linear
meaning.
I
don't
even
want
to
say
that
they
don't
represent.
They
don't
represent
linear
dimensions
in
physical
space.
A
There
are
no
movement
cells,
okay,
the
mini
column,
has
a
general.
The
cells
in
the
minicom
have
a
general
activation
level,
which
is
greater
when
you
move
quickly
and
slower
when
you're
moving,
slowly
or
lower
than
when
you're
right
so,
but
the
the
mini
con
itself
represents
a
dimension
of
movement,
but
there
aren't
movement
cells.
A
A
It's
just
that
the
dimension
of
where
your
cells
are
representing
are
not
dimensions
of
space,
but
the
dimensions
of
movement
in
the
space,
and
it
could
be
a
linear
dimension.
I
could
be
going
forward
and
say:
yes,
I'm
going
to
have
a
series
of
cells
represent
how
far
I've
moved
forward,
but
it
could
also
be
turning
or
works.
D
I
think
what
would
help
me
is
a
is
a
concrete
example
of
you
know.
We
can
use
logo
on
a
coffee
cup
or
some
other
object,
but
a
set
of
objects.
But
you
know
here's
kind
of
what
the
mini
columns
represent.
Here's,
how
I'm
moving
here's,
how
the
representation
changes
and
here's
how
the
location
we
don't
have
to
figure
out
the
mechanisms
for
it,
but
just
understanding
the
representation.
A
Okay,
well,
I'm
working
on
that.
I'm
trying
to
figure
out
you
know,
but
there's
a
lot
of
pieces.
I
think
to
get
that
right
for
me.
So,
for
example,
I
think
I
think
you
have
to
understand
how
you
go
between
these
upper
layers
and
the
lower
layers.
I
think
that's
going
to
be
part
of
the
solution,
because
I
think
what
happens
is
I
think
you
go
in
the
cortex
gets
its
information
and
it's
in
this.
A
It's
very
sort
of
the
egocentric
pose
position,
I'm
using
the
term
the
marcus
pose,
and-
and
so
it's
like.
Okay,
I'm
observing
this
at
a
particular
direction,
a
particular
distance
at
a
particular
orientation.
I
now
need
to
quickly
switch.
I
need
to
convert
that
to
an
environment
representation,
and
so
I
have
now
this
sort
of
pose
egocentric
location
position
and
an
invariant
displacement
representation,
and
these
have
to
go
back
and
forth
continually.
A
So
if
I
want
to
get
to
you
know,
so,
if
I
update
the
pose
position,
I
update
the
displacements.
If
I
want
to
get
to
a
particular
position,
independent
in
the
model
of
the
object,
I
have
to
come
back
to
the
pose
position,
so
I
think
we
have
to
solve
this
sort
of
upper
layer,
lower
layer
or
wherever
it
is
that's
occurring.
A
A
D
But
if
each
mini
column
is
representing
a
flow
or
a
movement
at
a
particular
rate
of
change
right,
wouldn't
you
get
scale
from
kind
of
the
overall
sdr
of
when
you
look
at
a
whole
bunch
of
these
mini
columns,
each
one
responsive
to
different
directions
and
speed.
You'd
scale
would
kind
of
come
out
of
that.
A
A
I
it's
it
what
it
requires.
It
requires
that
imagine
the
upper
cells
and
the
lower
cells,
the
upper
layers
and
the
lower
layers
have
to
be
operating
at
different
speeds.
Maybe
it's
like
you
know
like.
If
an
object
is
further
away
from
me,
I'm
looking
at
it
then
to
move
over
the
object.
I
have
to
move,
make
smaller
movements.
If
it's
closer,
I
have
to
make
larger
movements
to
achieve
the
same
result.
So
somehow
I
have
to
take.
Maybe
that's
the
clue.
A
I
have
to
take
some
sort
of
notion
of
how
far
away
this
thing
is
to
judge
them
is
distance
and
scale
kind
of
combined
together.
Somehow
I'm
not
sure
if
we're
making
any
sense
here,
but
I
I
I
don't
think
it's
simple.
Sometimes
I
I
hope
you're
right,
but
I
don't
think
it's.
It
seems
complex
to
me
every
time
I
try
to
work
it
through
the
details.
I
get
a
little
muddied
in
my
head,
but
I
think
it's
a
good
goal.
I
I
think
your
inputs
might.
A
I
was
I
was
hoping
to
try
to
solve
this
problem
in
one
dimension
like
solving
a
one-dimensional
scale
problem.
I
got
two
one-dimensional
spaces
and
and
trying
to
figure
out
how
to
make
that
work,
and
then
I
could
make
it
work
in
any
number
of
dimensions,
but
I
think
the
idea
of
like
the
coffee
cup
logo
thing
would
be
ideal
that
we
could
solve
that.
That
would
be
the
way
of
walking
through
that.
A
D
A
This
is
like,
I
don't
think
it's
any
different
than
we
thought
about
grid
zone
right.
It's.
The
only
difference
is
that
the
dimensions
of
the
mini
column
are
not
aligned
linearly
with
the
space
they're,
just
they're,
just
they're
demand
they're,
just
it's
just
a
line
that
might
curve
or
screw
or
whatever
it's
in
movement
space.
So
everything
else
seems
to
be
applying
the
same.
A
So
this
idea
that
I
think
maybe
I've
somehow
confused
people
to
think
like.
Oh,
these
represent
movements
where,
before
grid
cells
didn't
represent
movements,
oh
great
cells
were
activated
by
movements,
and
here
too,
the
cells
are
activated
by
movements.
It's
just
that
the
dimension
is
a
a
typical
type
of
movement,
as
opposed
to
in
the
past.
We
might
think
the
dimension
as
a
straight
line
movement
through
space.
Here
it's
no
longer
a
straight
line:
movement
to
space.
A
It
could
be
a
turning
movement
or
a
screwy
movement.
It
doesn't
really
matter
it's
just
whatever
is
common,
so
you
know
I
went
through
all
these
scenarios
like
you
can
sit
there
and
say:
okay,
I
can
turn
my
head
left
and
right.
I
can
move
sideways
through
space.
Look!
My
hands
are
moving
relative
to
my
face.
You
know,
there's
all
these
different
things
you
can
do
that
you
could
detect
different
types
of
movements
you
can
detect
just
by
flow
patterns.
A
A
C
D
A
C
A
Can't
do
it
so
somehow
I
think
you
got
to
do
the
whole
thing.
I
think
you
could
try.
Here's
like
I'll,
try
back
again
I'll,
try
again
to
do
talk,
think
about
simpler
optic.
Think,
like
one
dimensional
objects,
it's
easy
to
think
about
like
okay,
just
I
have
a
linear
track
and
I
have
some
I
have.
I
have
two
different
tracks.
If
you
will
two
different,
one-dimensional
spaces
and
I
want
to
assign
one
to
a
location,
the
other
one,
it's
a
bit
abstract,
but
it's
easier
to
deal
with.
A
You
can
just
think
about
it.
I
don't
even
know
if
orientation
or
any
concept
orientation
makes
sense
in
this
case,
but
but
I
could
try
to
solve
the
scale
problem
using
one-dimensional
things
and
that
might
bring
up
some
general
concepts.
But
once
you
go
beyond
that,
I
guess
I
guess,
then
we
could
go
to
like
a
two-dimensional
space
like
like
a
room
with
a
rug.
Remember
we
talked
about
that
a
lot.
I
could
do
that
too.
That
might
be
maybe
more
fruitful.
A
D
A
All
right:
well,
I
find
it.
I
find
the
dimensionality
hard
when
I
start
thinking
of
three-dimensional
structures.
It
gets
much
harder
for
me
so
I'll,
but
I
think
maybe
a
happy
ground
would
be
a
two-dimensional
space,
because
then
I
can
have
a
variety
of
I
could
I
could
if
I,
if
I
tried
to
solve
the
carpet
in
the
room
problem
then-
and
I
think
if
it's
a
two-dimensional
space
then
I
might
I
can.