►
Description
Jeff Hawkins brainstorms how sensorimotor models might be built up from purely sensory data, how this might fit into a cortical column, and the importance of magnocellular and parvocellular cells.
For a continuation of this discussion, view the next video: https://www.youtube.com/watch?v=jQCtuK9XbTE
B
This
is
just
I've
been
working
on
this
mini
column
idea
and
I
present
a
little
bit
of
on
multiple
times
that
some
more
on
it
today.
So
this
is
I'll,
try
to
do
a
little
bit
of
review,
but
I
just
want
to
keep
going
forward.
So
it's
just
good
for
me
to
put
this
down
and
try
to
explain
it
to
people
and
see
if
people
understand
it.
So
this
is
a
slide.
B
I
showed
last
time
where
I
was
or
a
couple
times
ago,
where
I
was
explaining
how
we
used
to
think
of
how
I
used
to
think
about
a
column
where
you
get
sensory
input,
and
then
it
gets
this
an
efference
copy
motor
input
from
something
below
and
had
generates
motor
outputs.
And
then
then
we
proposed
that
I
propose
an
alternate
way,
which
is
the
new
idea.
Is
that
that
a
column
can?
B
But
I
think
this
is
going
to
apply
everywhere
so
now
the
idea
that
you
have
two
types
of
sensory
input,
a
static
input
and
the
flow
input
and
that
flow
input
is
allow
the
cognitive
path,
integration
and
later
does
it
learn
to
associate
a
motor
input,
efference
copy
and
it's
motor
output
with
some
central
pattern
generator
below.
So
this
should
be
somewhat.
This
is
review,
but
maybe
you
don't
remember
it
because
that
you've
all
doing
something
else,
but
anyway,
I
was
orient.
B
That
input
a
century
and
to
recall
into
a
column
is
divided
into
us,
static
and
flow
portions,
and
when
we
run
the
flow
bits
through
the
spatial
pool
or
like
thing,
we
define
many
columns
and
those
many
columns
essentially
represent
movement.
Vectors
like
how
the
different
sort
of
basis
of
movement
vectors
that
that
the
system
can
behave
in
is
essential,
beginning
of
the
path
integration,
modeling
system.
B
So
just
taking
this
idea
here,
the
static
foot
and
input
and
flowing
foot
I
went
back
and
looked
at
the
the
visual
pathways
again,
and
this
is
stuff
we
all
some
of
us
know:
I
hadn't
looked
at
in
a
while,
but
I
went
back
and
reviewed
it
again.
So
this
for
some
people.
This
would
be
review
some
people,
this
video,
but
when
input
comes
so
we're
going
to
look
at
how
the
input
from
the
eyes
goes
to
the
cortex,
and
it
goes
through.
B
This
thing
called
the
thalamus,
which
is
in
a
lateral,
geniculate
nucleus,
and
this
is
sort
of
a
a
pictorial
diagram
of
it,
and
the
important
thing
I
want
to
get
here
is
that
there's
there's
basically,
three
types
of
cells
that
pass
information
from
the
eyes
to
the
to
the
two
v1
v1
is
not
shown
here.
Is
that
these
pink
and
blue
things
are
part
of
the
family.
So
this
is
lgn
and
there's
a
left
and
right,
one,
so
they're,
showing
both
sides.
B
Importantly,
that
the
there's
six
layers
here
of
cells
and
two,
the
layers,
the
blue
ones,
are
called
magnocellular
cells
and
for
the
layers
are,
peat
ones
are
called
parvocellular
cells.
It's
those
those
are
the
primary
divisions.
The
magnet
just
means
that
the
cells
are
babies
or
the
large
cells
and
poggle
I
think
so
now
it
means
small
automation
that
there's
a
third
type
of
cell
called
a
coma
cell
yourself,
which
are
which
are
much
poorer,
understood
and
they're
sort
of
in
primates
they're
interspersed
between
the
layers.
B
So
this
is
there's
a
natural
division
here
between
the
magnocellular
and
the
provost.
This
is
very
well
established
in
neuroscience.
There's
really
almost
no
explanation
for
this,
but
here's
how
we
can
think
about
it.
Magnus
area
cells-
these
are
some
of
the
properties
of
them.
They
have
a
very
fast
response.
I
mean
they
very
quick
to
trigger.
They
have
broad
receptive
fields,
they're,
not
very
precise
in
terms
of
the
part
of
the
Brett
and
their
infant
from
they
are
not
tonic
mean
they.
B
If
you
look
at
a
cell,
it
becomes
active
when
when
it
when
it's
input
changes.
So
if
you
have
an
onset
of
a
signal
or
an
offset
of
a
signal
fire,
but
it
will
not
continue
to
fire,
it
triggers
on
the
on
the
transition
between
all
phenomena.
There's
no
color
representation
in
the
Magnus
other
cells-
and
this
is
ideal
for
flow
detection-
is
exactly
what
you
would
want
for
flow
detection.
You'd
want.
B
B
The
parvocellular,
my
contrasts
are
slower,
so
they
don't
respond
as
quickly.
They
have
very
narrow,
receptive
fields,
so
they're
finely
tuned
and
they
do
explain
a
tonic
response.
So,
if
I
put
a
dot
of
light
into
a
receptive
field
of
a
harvester
yourself,
it
stays
active
and
they
do
represent
color,
and
this
is
ideal
for
feature
detection.
So
I
don't
know
if
I
mentioned
this
last
time,
but
the
the
speculated
two
roles
of
two
different
types
of
sensory
input
to
a
column
are
clearly
there
and
they
match
exactly
what
I
would
expect.
B
I
knew
this
stuff,
I
hadn't
thought
about
it.
When
I
presented
this
last
time,
I
had
not
thought
about
this.
A
long
time
and,
and
so
I
said
so
I
think
that's
a
really
nice
fit
interesting
to
come
to
cellular
cells
is
very
little
known
about
them.
I
get
once
before,
speculated
a
role
for
them
and
therefore,
if
you're,
following
in
the
earlier
models,
we
thought
about
this
way.
We
thought
a
column.
The
way
a
column
would
know
how
movement
was
occurring.
B
It
had
to
have
an
efference
copy
and
motor
reference
copy,
so
it
had
to
be
receiving
information
from
a
motor
area
through
the
cortex,
and
when
we
looked
at
you
know
from
the
from
the
thalamus
from
the
LGN
to
the
cortex.
We
needed
a
signal
that
might
represent
motion
a
behavior
in
a
movement
and
so
I
speculated
that
the
conus
other
cells
would
be
that
motor
efference
copy.
That
speculation
still
could
be
true
because
I'm
not
saying
there
isn't
a
motor
African
copy.
There
is
a
motor
reference
copy.
It's.
B
B
C
B
Yeah,
that's
right.
We're
the
inputs
to
the
connoisseur
yourselves
I
think
are
coming
from.
I
didn't
mention
that,
but
I
think
we
verified
that
before
they're
coming
I
somebody
I
asked
I
said
we
need
a
signal.
That's
coming
from
the
support
on
this
particular
close
to
the
cortex.
Where
might
that
be,
and
I
figured
if
I
asked
that
question
too,
but
I
asked
several
people
over
the
years
and
some
nurses
suggested
it
might
have
been
the
corner,
sell
yourselves
and
I.
Don't
remember
if
there's
a
lot
of
good.
B
B
So
so
perfectly,
maybe
that
was
not
the
finger,
but
certainly
pretty
close,
say.
I
think
this
is
I,
think,
first
to
point
to
the
Magna
Steyr
and
the
parmigiano
cells.
This
is
almost
the
Condor
solar
cells.
It's
still
a
bit
speculative,
but
we
do
need
a
signal
from
the
spurred
colliculus
to
go
to
the
v1.
It
wasn't
clear
up
front
that
it
would
have
to
go
through
LGN.
That's
not
clear
to
me.
It
just
has
to
get
some
spur
clickers
to
b1
or
b2,
and
so
on
now
so
anyway,
that
seemed
like.
B
No
one
else
knew
what
those
cells
are
doing
inside.
Well,
then,
it
could
be
okay,
so
so
that
I
thought
was
really
nice
matching
up
with
the
theory.
With
with
this
observation
and
I
I
would
bet
almost
anything
that
we,
if
we
go
dig
into
the
data
for
the
flick
relays
for
touch
and
audition,
we
would
find
a
similar
separation
I'm,
not
saying
that
the
thalamus
would
have
the
same
organization,
but
we
would
see
two
pathways.
B
B
Sense
here:
well,
I,
don't
know
if
that's
true
I
think
you
still
need
a
union.
Now
our
receptive
fields,
Anna
tonic
response,
whether
it's
color
or
not,
right,
you
need
a
union
impo.
Is
it
going
to
basically
represent
the
feature
of
something
and
not
the
movement
of
the
field,
so
I
would
think
you
would
still
have,
but
you'd
have
to
have
you
don't
have
a
division
of
two
types
of
sensory
input,
regardless
of
whether
your
color
or
not
the
colors?
B
It's
an
interesting
thing,
because
there
are
certain
sensory
features
which
don't
really
relate
to
movement,
and
but
it
would
be
useful
for
object,
discrimination
and
those
those
would
be
in
that
second
pathway.
The
I
think
it's
a
really
nice
mapping
here
and
any
animal.
That's
gonna
be
able
to
model
the
world
through
vision.
If
they're
taught
your
hearing
is
gonna
have
I'm
going
to
predict
it's
going
to
have
this
separation
of
these
two
different
sensory
pathways.
B
It's
I
think
this
is
actually
going
on
in
region,
the
region
and
the
cortex
as
well,
but
I
just
wanted
to
just
throw
this
out
here
right
now.
This
is
a
very
there's.
No
hugs
I
know
there's
no
actual
explanation
for
why
they're
magnocellular
cells
and
parvocellular
cells
there's
just
observations
about
what
they
are
and
now
I
have
a
very
strong
sabbatical
proposal
why
there
are
two
different
pathways
and
that
they
should
be.
We
need
them
and
there
should
be
everywhere.
Okay,
let's
go
on
unless
there's
more
questions
about
that.
There's
any
more
questions.
B
Okay,
here
now,
this
whole
thing
got
started
by
an
observation
of
mine.
The
observation
was
I
forget
how
long
ago
we
did
this,
but
you're
so
ago,
I
made
an
observation.
If
you're
watching
someone
playing
a
video
game
like
like
this
first-person
scene
up
here
and
you're
just
watching
it,
you
know
how
that
player
is
moving.
You
know
when
they're
going
forward
and
backwards
and
turning
left
and
turning
right,
you
can
learn
a
model
of
the
environment
just
by
watching
the
screen.
B
You
can
predictions
by
what's
gonna
happen
and
so
that
what
that
proved
to
me
was
that
you
didn't
mean
an
efference
copy,
mortar
command,
to
to
track
your
location
and
to
update
your
grid
cells
and
play
cells
that
it
wasn't,
and
so
that
told
me
that
the
data
to
do
that
must
be
coming
from
the
sensory
visual,
the
visual
stream.
In
this
case
and
now,
what's
the
genesis
of
part
of
this
whole
idea,
it
was
like
hey
I,
don't
need
an
efference
copy.
B
I
can
do
this
all
from
sensory
data,
so
just
by
watching
we
sense
how
we're
moving
we're
able
to
track
our
location
and
orientation
and
therefore
sensory
data
flow
is
sufficient
for
updating
grid
cells,
play
cells,
etc
and
I
see
here
the
motor
efforts
copies
advantageous
but
not
required,
meaning.
It
is
actually
useful
to
know
if
I'm
in
control
of
this
system,
the
cortex.
We
want
to
know
what
movements
I'm
January,
because
that's
faster
than
then
relying
just
a
sentry,
their
sentry
data
is
reactionary.
B
Think
I
see
the
thing
starting
to
move
to
laughing
Oh
remove
left,
but
if
I'm
actually
controlling
the
joystick
myself
I
can
take
advantage
that
I
can
say
I'm
about
to
move
and
before
anything
changes
on
the
screen.
My
brain
can
predict
what's
gonna
happen,
which
can't
happen
if
I'm
not
just
observing
it
so
I'm,
not
removing
the
importance
of
motor
reference
copy
I'm,
just
saying
it's
not
necessary
and
it's
not
in
fact
I.
Don't
think
it's
the
primary
means
by
which
we
do
this
so.
C
C
B
Here
is
you,
the
idea
is
here's
a
column
and
it's
getting
input
and
there's
got
a
static
input
and
a
flow
input
that
would
be
the
that
would
be
the
Fargo
and-
and
this
itself
is
how
we
learn
the
structure
of
the
input
space.
This
is
this
is
how
the
instruction
here
previously
saw.
Calm
builds
up
its
model,
and
after
it
does
that,
or
while
it's
doing
that,
it
can
then
learn
to
associate
the
movements
that
is
detected
in
this
flow
input.
It
can
associate
them
with.
B
He
says:
oh
I
know
how
you're
moving
you're
moving
left.
Are
you
moving
forward
you
moving
back
and
then
it
could
associate
that
whatever
system
is
generating,
that
behavior
and
and
then
it
can
also
associate
the
output
of
this
system
with
this
as
well.
So
this
is
a
secondary
step.
It's
not
it's
not
really.
The
key
thing.
The
key
thing
is
that
you
we're
learning
this
all
from
this
is.
This
is
the
whole
idea
here
now
is.
This
is
the
system
that
is
learning
how
the
world
works.
B
This
is
a
secondary
system
that
allows
me
to
control
the
thing
I'm
attached
to.
So
you
can
imagine
a
column,
it's
just
observing
the
sensory
data.
It
says,
I'm,
building
a
model
of
the
world-
oh
I,
can
see.
What's
going
on,
I
figured
out
this
model,
but
until
it
has
just
on
itself
here,
I
can't
do
anything
about
the
model
here
and
now
it
says:
oh
I
can
now
control
the
thing.
That's
moving
and
I
can
now
string
together
new
movements
to
do
new
goals
that
the
old
system
couldn't
do
on
its
own.
B
C
One
thing
that's
interesting
to
me
is
the
fact
that
I
think
you
probably
like
I
would
suspect
that,
when
I
agree
that
you
can
like
infer
all
position
and
orientation
and
anything
just
from
sensory
data,
one
thing
that
I
think
might
be
happening
is
that
you
also
like
generate
fake
efference
copies
of
like
you
know
you
invert
the
model
and
see
like
what
actually
create
this
and
you're
sort
of
playing
along.
Why
would
you
do
that?
Why
do
you
need
to
do
that?
Well.
Look!
C
B
Important
thing
here
that
the
real
key
insight
here
is
that
the
model
is
learning
through
sensory
and
not,
and
we
can
learn
a
sensory
motor
model
through
sensory
data
that
the
flow
data
allows
me
to
figure
out
what
movements
are
being
performed
in
the
world
and
I
can
use
that
to
build
a
build
up,
a
model
of
a
metric
space
and
all
the
things
and
how
behavior
operates
in
that
space.
But.
B
C
B
B
It
just
says:
I'm
sitting
on
top
of
and
I'm
observing,
I'm
observing
and
essentially
the
results
or
somebody
else
moving
the
sensor
around
and
I'm
just
gonna
watch
it
and
see
what
happens
and
then
from
that
I
build
up
a
model
of
the
thing
that
this
cpg
is
interacting
with
the
old
brain,
so
that
you
can
think
of
it.
The
old
brain
you
can
think
about
another
part
of
the
cortex
doesn't
really
matter.
There's
a
comma
sitting
there
going
I'm
looking
at
something
I'm,
getting
some
sensory
input,
I'm
going
to
figure
out.
B
This
is
but
there's
an
assumption
that
there's
somebody
else
generating
that
behavior
and
and
I'm
gonna
observe
the
results
of
their
behavior
and
by
reliving
them
results
in
their
behavior.
I
can
now
build
a
model
and
once
I
have
that
model,
I
can
associate
my
model
with
the
actual
behaviors
of
being
generated
below
and
I
can
now
control
them.
I
can
now
stream,
together
novel
sequences
of
behaviors
to
achieve
new
goals
that
were
not
achievable
by
the
old
system
you
can
think
of.
In
some
case
the
simplest
case.
B
C
B
C
A
B
C
B
Based
on
I'm
claiming
that's
not
true
and
I'm,
almost
certain-
it's
not
true
in
this
case,
because
you
there's
other
reasons:
I
haven't
gotten
into
you.
Can't
the
column
cannot
know
anything
about
what
motor
its
output
can
do.
It
has
no
way
of
generating
behaviors.
Initially,
it's
those
dotted
lines
have
to
be
learned,
and
so
a
cortical
calm,
never
controls
muscles
directly
and
then
ever
does
not
a
single
layer
of
feinstone,
the
cortex.
B
It
goes
to
a
muscle,
so
you
can't
the
cortex
can't
have
any
behaviors
until
it
has
learned
a
model
the
world
and
and
how
the
thing
is
talking
to
and
generating
behaviors,
and
then
it
can
associate
its
model
with
those
behaviors
I'm
gonna
leave
is
that
we
can
debate
it
later,
okay,
okay,
anyway,
so
so
I
want
to
go
back
to
this
thing.
So
this
is
the
observation
which
led
me
to
this
idea
that
you
don't
need
an
efference
promoter
copy.
B
All
you
need
to
do
is
look
at
century
day
and
I
was
just
imagine
that
the
column.
This
is
some
brain.
That's
looking
at
this
scene,
the
cortex,
and
it
has
no
idea
what's
behaviors
are
being
generated,
but
it
just
immediately
says:
oh
I
know
what
behaviors
are
I
can
do.
I
can
infer
I
can
learn
what
behaviors
are
going
by
just
observing
how
the
flow
bits
change.
I,
don't
think
this
is
hardwired.
I
think
it
lends
how
the
flow
be
exchanged.
It
says,
oh,
those
are
the
behaviors
that
are
being
executed
here
now.
B
I
had
a
similar
observation
and
this
is
getting
a
bit
more
speculative.
Now
it's
a
similar
idea,
but
now,
as
you
imagine
now
that
I'm
I
couldn't
figure
how
to
get
a
movie
in
my
PowerPoint
presentations,
you
have
to
accept
that,
like
this
imagine
now
I'm
a
holding
an
object
in
my
hand
or
somebody's
holding
it
or
just
observing
an
object
like
this
coffee
cup,
and
now
the
coffee
cup
starts
moving.
B
It
can
be
rotating
around
yes,
okay,
okay,
so
this
observation
is
the
following:
if
I
move
an
object
like
like
the
cup
and
I,
rotate
it
around
the
different
positions
or
any
object,
why
do
I
not
feel
like
I'm
moving
through
space?
What
I
feel
is
the
object
is
moving,
so
I
have
flow
bits
changing
just
like
I
hit
on
the
left
panel
here
I
flow
bits
changing
now
for
bits
on
the
right
change,
but
I
don't
have
a
different
perception
perceptions,
I'm,
not
moving.
B
The
perception
is
the
object
is
moving
and
which
is
a
curious
thing.
Why
would
that
be?
Well,
you
know,
what's
the
difference?
Well,
the
difference
here
is
that
in
the
left
case
there
was
a
the
flow
bits.
There
was
flow
occurring
over
the
entire
field
of
view
in
the
right.
The
flow
is
only
occurring
over
a
small
part
of
the
field
of
view,
and
that's
the
only
difference.
So
why
would
you
know
that
tells
me
this
is
a
very
important
clue.
It's
just
I
don't
feel
like
I'm
moving
anymore
I
feel
the
object
is
moving.
B
B
The
idea
is
remember
we
go
back
to
our
cortical
column
and
on
the
Left
we
know
there
are
three
separate
points
where
input
from
the
l
GN
is,
if
there
be
one
column,
enter.
There's
this
the
lower
layer,
3,
there's
a
lower
layer
5
in
this
layer
4,
and
what
I'm
going
to
argue
here.
Is
that
really
so
the
idea
and
I
call
this
speculation?
Is
that
when
we,
when
we
think
about
the
manual
cells,
this
is
the
one
that
would
be
the
flow
cells.
B
Those
are
the
ones
I'm
arguing
on
going
into
lower
layer,
3
and
lower
layer
5.
They
are
determining
the
movement
commands,
but
I'm
gonna
argue
that
there's
a
narrow
field
of
view
for
the
upper
ones
and
a
broad
field
of
view
for
the
lower
ones
and-
and
if
you
recall
that
we
set
the
fields
in
the
layer
5
and
their
6
a
much
wider
than
the
receptive
fields
in
layer
2
and
before
plus
layer,
2
3,
&
4
have
n
stop,
meaning
that
if
you
get
larger,
they
stop
responding.
B
So,
there's
clearly
already
a
well-established
idea
that
upper
layers
have
an
hour
of
field
of
view
than
the
lower
layers,
and
now
we
see
this
sort
of
carnal
thinking.
We
have
this.
We
might
have
two
separate
recognition
systems
going
on
here
in
the
bottom,
where
we
have
a
large
field
of
view,
that's
like
looking
at
their
gamer
and
we're
looking
and
what
you,
if
you
walk
through
what
happens
there?
What
you're
really
doing
is
modeling
egocentric
movement,
you're,
modeling,
how
my
body
is
moving
through
space
up
to
my
bodies.
Turning
left,
my
body
swing
right.
B
It's
going
forward,
it's
going
backwards
that
kind
of
stuff,
but
if
I
tap
this
this
narrow
field
of
view,
which,
which
is
the
right,
the
rotating
cop
I
no
longer
perceive
I,
don't
think
of
it
as
movement.
I
think
it's
the
object
is
moving,
and
so
what
I
would
be
doing
is
I
would
be
I
would
be
modeling
the
movement
of
the
object,
as
opposed
to
the
movement
of
my
body.
B
And,
of
course
we
have
to
go
back
and
forth
between
them.
We
have
to
in
somehow
be
able
to
figure
out
how
we're
going
between
the
eccentric
and
egocentric
reference
frames.
In
this
particular
case,
I,
don't
have
any
details
on
that
yet,
but
it's,
but
it's
an
intriguing
idea
at
least
I
and
I
thought
I'd,
just
throw
it
out
there,
because
it's
kind
of
interesting
it
fits
a
lot
of
data
and
we
need
to
make
this
transition
from
egocentric
allocentric
space
someplace
in
the
cortex.
That's
a
requirement.
It
happens
right
away
the
right
soon.
B
As
you
get
to
v2,
you
start
seeing
more
eccentric
type
of
representations
so
anyway,
this
is
the
idea
I'm
working
on
which
I
think
is
pretty
a
pretty
interesting
idea
and
then
I'll.
Just
throwing
this
thing
is
layer
five,
of
course
these
are
motor
cells.
Here
only
some
of
them
are
and
those
would
be
egocentric
motor
commands.
So
you,
which
is
what
we
need.
B
We
need
to
be
able
to
send
a
signal
back
down
to
some
part
of
the
brain
that
says:
yeah
I
want
you
to
move
in
this
egocentric
way,
that's
what
the
commands
to
the
body
have
to
be
in,
but
then
we
might
be
going
back
and
forth
between
these
two
spaces
represented
here.
So
that's
that's
the
idea
I'm
working
on
right
now.
B
A
B
B
Okay,
so
this
is
a
real
quick
thing:
here's
a
great
paper
I'm
reading,
I've
read
it
twice,
but
I've
already
gotten
halfway
through
it
both
times
and
look.
Louis
will
remember
this,
but
happy
episodes.
People
Louis,
remember:
I,
brought
on
those
little
wire
screens
once
these
two
pieces
of
wire
screens,
maybe
some
of
I,
don't
I,
showed
anyone
else.
I
know
showing
the
Louis
yeah.
B
And
I
think
these
remind
me
of
grid
cells
right
we
and
these
two
screens,
you
rotated
and
slide
him,
and
you
get
all
these
more.
A
patterns
Anderson,
here's
this
paper,
2007
paper,
it's
about
how
it
isn't
a
mind-blowing
paper,
I,
don't
know
why
Florian
didn't
bring
this
up
when
he
was
talking
about
models
of
grid
cells.
This
is
an
alternate
model
for
grid
cells.
That
is
speculating
that
they're
really
more
a
interface
or
interference
patterns.
Exactly
the
thing
I
served
in
that's
on
those
two
missions
and
I
won't
walk
you
through
it.
B
Yet
I
will
do
it
in
a
future
time.
It's
a
great!
It's
a
really
fascinating
paper.
I,
don't
know
if
it's
right,
but
it's
fascinating,
but
there's
they're,
basically
showing
how
they're
arguing
that
grid
cells
are
essentially
derived
from
smaller,
feel
smaller
grid
fields
and
they're,
and
what
you're
really
observing
is
a
more
a
pattern
and
and
it
it
it,
it
explains
a
whole
bunch
of
experiment.
The
other
people
have
explained
it's
got
some
issues
to
so,
but
it's
fascinating
and
I'd
mention
it
because
I'm
going
to
talk
about
it
in
the
future.