►
Description
Jeff Hawkins explores the relationship between the thalamus and the neocortex. He also examines whether reference frame transformation can occur in the thalamus. He first gives an overview of the Thousand Brains Theory, specifically how reference frame transformation happens in the brain. He then explores cortex feedforward pathways. Jeff proposes that the anatomy and physiology of the thalamus suggests that thalamocortical relay cells might be implementing a multiplexor. Lastly, he discusses a few issues about the theory and the team gives various perspectives and ideas.
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
A
How
that
you
know
how
we,
how
that
could
be
used
for
reference
frame
transformations
in
the
cortex,
and
so
I'm
just
going
to
talk
about
that
issue
and
talk
about
some
of
the
issues
related
to
it,
and
so
it's
I
call
this
a
proposal
and
exploration
because
I'm
not
not
proposing
this
is
the
way
it
works.
This
is
just
like:
hey
here's.
An
idea
had
some
really
nice.
It
has
some
really
nice
actions,
but
I
don't
know
if
it's
right
or
not.
A
All
right
we're
going
to
go
now,
some
of
the
a
lot
of
material
here,
it's
going
to
be
very
familiar
to
some
people,
but
some
people
are
going
to
it's
probably
going
to
be
new
to
them.
I
can't
really
keep
track
of
who's.
Who
knows
what
and
how
much
people
understand
so
again
any
feedback
on
like
like
the
pace
or
something
don't
need
to
do
this
again.
Jeff,
let
me
know
so
it's
been
known
for
a
long
time
that
the
court,
the
brain,
has
to
do
reference
ring
transformations.
A
In
the
world
and
we're
moving
around
the
world
and
therefore,
as
we
manipulate
objects
in
the
world
we
have
to,
we
have
to
do
these
transforms
between
our
the
body-centric
reference
frames
and
the
reference
rings
in
the
world,
and
the
thousand
brain
theories
is
more
specific
about
that.
The
thousand
brain
theory
says
that
every
cortical
column
must
be
doing
response
transformations.
A
So
it's
it's.
It
could
be
happening
elsewhere
in
the
brain
too.
I'm
not
saying
it's
not
helping
in
some
sort.
It
could
be
happening
in
many
subcortical
structures,
but
it's
going
to
be
happening
in
the
cortex
and
specifically
in
every
cortical
column,
and
the
reason
we
say
that
is
that
you
know
that
the
thousand
brain
series
is
every
common
learned,
sensory
model
models
of
objects
in
the
world,
and
again
I
use
the
word
objects.
A
You
can
think
of
it's
physical
objects
like
a
coffee
cup,
but
it
could
be
any
kind
of
structure
that
exists
in
the
world
and
then
to
learn
a
model,
a
sentry
motor
model.
The
column
must
get
two
types
of
inputs.
It
has
to
get
one
type
of
input
which
is
representing
what
a
sensor
is.
Actually
sensing,
what
is
what
is
the
details?
A
I'm
sensing
at
this
moment
in
time,
and
the
other
thing
it
has
to
know
is
how
the
sensor
is
moving,
and
it's
only
by
combining
the
movement
and
knowledge
about
the
movement
and
which
direction
some.
The
sensor
is
moving
and
what's
sensing,
can
you
build
up
the
sort
of
the
structure
of
the
world
a
model,
the
structural
world
and
because
the
object
being
modeled
and
the
sensory
system
may
be
at
different
orientations
or
different
poses
relative
to
each
other?
A
The
columns
must
transform
a
sense
of
movement
data
to
be
aligned
with
the
object.
So
the
example
we've
often
used
I'll
just
talk
about
stop
sharing
for
a
second
here
is,
you
know.
We
talked
about
the
coffee
cup
in
the
paper
staplers,
and
things
like
that.
You
know
your
finger
is
at
some
orientation
to
the
cup
and
and
and
when
you
move
your
finger
where
it
moves
on,
the
cup
depends
on
the
orientation
of
the
cup
and
what
you
feel
depends
on
the
orientation
of
the
finger
on
the
cup.
It's
just
this
happens.
A
Even
in
all
the
time.
Even
something
simple
is
like:
if
you
look
at
every
one
of
these
pictures
right
now,
often
our
heads
are
tilted,
10
degrees,
one
way
or
the
other,
and
you
know
that
means
that
if
I'm
like
reading
a
book
or
something
like
that,
the
eye
movements
required
to
to
read
a
line
of
text
in
the
book
are
different
depending
on
my
head,
so
they
went
away
the
other
way
and
there's
a
there's.
A
transformation
has
to
occur
there.
A
I
can't
just
say
to
read
my
text
move
my
eyes
to
the
right
now.
I
have
to
sometimes
move
in
different
directions,
and
so
on
so
there's
that's
another
example
of
what's
going
on
in
it's
just
constantly,
if
you
think
about
it,
you'll
realize
that
this
is
happening
all
over
the
place.
Okay,
back
to
the
presentation
here,
sure
there
we
go.
So
why
did
I
stop
losing
my
ability
here
we
go
so
the
general
theme
here
is
the
inputs
to
every
cortical
column
pass
through
the
thalamus.
A
So
the
question
we
want
to
ask
is:
could
the
thumb
to
the
thalamic,
the
thalamocortical
relay
cells
in
the
thalamus
be
doing
a
reference
frame?
Conversion?
Could
they
be
converting
an
input
from
one
reference
frame
to
another,
and
so
that's
the
thing
I'm
going
to
talk
about,
I'm
going
to
start
a
little
teeny
bit
of
just
just
to
remind
you.
I've
been
talking
about
this
before
this
is
a
slide
I
presented
sometime
in
the
last
few
months.
A
I'm
not
sure
when,
where
I
was
talking
about
how
these
reference
frame
transformations
could
be
occurring
in
different
layers
within
the
cortical
column.
So
this
picture
was
a
presentation
part
of
a
presentation
I
gave
earlier
I'm
talking
about
the
cortical
column,
and
I
was
talking
about
the
different
representations,
simple
cells
and
complex
cells,
and
how
it's
possible.
Perhaps
some
of
these
layers
are
representing
allocentric
movement
and
some
of
them
represent
essential
movement
and
there
will
be
a
transformation
going
back
and
forth
between
them.
A
I
haven't
abandoned
this
idea
completely,
but
just
just
to
put
in
perspective
today,
I'm
going
to
talk
about
a
different
approach
for
doing
this.
That
would
be
an
alternate
to
this
type
of
thinking
here,
so
it's
we're
still
trying
to
figure
out
how
this
happens.
So
in
this
case
we're
going
to
go.
Look
at
the
thalamus-
and
this
is
a
little
bit
review-
what
the
subatai
showed
last
week
that
the
traditional
way
of
thinking
about
the
thalamus
is
that
it
was
a
as
a
gateway
into
the
cortex.
A
A
So
so
the
idea
is
that
information
is
passing
directly
from
reeds
in
the
region
and
the
reason
they
they
call.
The
thalamus
as
a
relay
is
because
they
often
could
see
that
one
spike
into
the
thalamus
from
the
sensor
could
lead
to
one
spike
out.
I
mean
it's
really.
You
could
you're
you're
on
the
right
conditions,
you're
you're.
A
They
weren't
well
characterized
so
people
just
sort
of
said:
yeah,
there's
other
parts
of
the
thomas
that
there's
some
kind
of
loops
between
the
cortex
and
the
thalamus,
but
it
was
generally
ignored
and
you
can
see
that
in
this
famous
phenomenon,
vanessa
diagram
where
this
is,
you
know
1991
where
this
this
is
really.
That
became
the
sort
of
the
the
the
standard
of
how
we
think
about
cortical
flow,
and
here
they
show
the
retina
this
first
block
at
the
bottom.
They
show.
This
is
the
visual
cortex
of
a
monkey.
A
They
show
lgn,
that's
the
thalamus
here
and
then
this
is
v1
this
next
purpley
thing,
and
after
that,
all
these
connections
are
cortical
cortical,
there's
no
other
thalamic
interaction
shown
here
at
all
no
other
connections
to
the
thumbs,
but
then
sherman
guillory
again
reaches
repeating
with
with
super
tight.
Last
week,
sherman
and
guillory
had
proposed
a
different
alternate
explanation
or
an
alternate
architecture,
and
their
world
feed
forward
pathway
primarily
goes
through
the
thalamus
each
time.
So
you
have
a
sensor
which
goes
to
what
they
call
a
first
order.
Thalamic
relay
goes
to
layer.
A
Four,
then
layer,
five
is
being
an
output
of
the
cortical
region,
goes
feds
into
the
higher
order,
thalamic
relay,
which
becomes
input
to
layer,
four,
which
it
goes
to
layer,
five
and
so
on,
and
so
there's
this
constant.
The
feed
flow
pathway
goes
through
the
thalamic
relay
cells.
Every
time
in
this
situation,
it's
well
known
these
layer,
five
cells,
that
project
to
the
thalamus
split
their
axon
splitted
two
and
they
project
some
place
below
the
cortex
generating
motor.
A
So
these
are
the
motor
output
cells
of
the
cortex,
and
so
this
would
say
like
well
what's
being
passed
between
region,
one
and
region.
Two
is
a
motor
command,
it's
a
motor
being
sent
through
the
thalamus,
but
over
here
you
see
in
the
we
say.
Well,
there's
this
input
from
the
sensor,
that's
being
passed
through
the
thalamus.
Well,
that
seems
kind
of
weird
if
we
think
about
it.
A
In
this
scenario,
the
thalamus
process
is
feed
forward
input
to
all
cortical
regions,
not
just
the
first
region,
but
then
there's
this
weird
thing,
like
I
just
said
the
first
order,
thalamic
really
past
the
sensory
data
and
the
high
order
ones
are
passing
motor
commands.
This
clearly
can't
be
right,
it's
it's
asymmetrical.
It
doesn't
make
any
sense,
and
this
bothered
me
for
many
years
something's
wrong
about
this
picture.
A
We
can't
have
one
region,
processing,
sensory
data
in
other
region,
processing,
motor
data
and
a
thousand
brain
theory
says
every
column
needs
input,
both
sense
and
motor
data,
so
you
know,
and
they
also
every
column
has
to
have
a
motor
output.
These
are
the
three
things
we
know
that
a
column
has
to
do.
They
actually
have
an
input
from
the
sensor,
an
input
that
represents
movement
or
motor
data,
and
it
also
has
to
generate
behaviors
well
in
this
picture.
Here
we
see
that
every
column
or
every
region
has
cells
that
project
and
create
motor
outputs.
A
That's
good
that
this
layer,
five
cells,
but
here
we're
showing
that
this
thing
is
getting
sensor,
and
this
is
getting
motor.
So
this
not
something's
wrong
here.
Just
can't
be
right,
then
what
got
me
down
this
path
of
thinking?
What
I'm
gonna
explore
further,
and
this
is
a
slide
I
showed
earlier
too
at
least
parts
of
the
slide.
A
I
was
making
the
observation
that,
if
you're
watching
a
so
again,
just
a
repeat
for
some
people,
if
you're
looking
at
a
a
first
person,
a
role-playing
game
like
imagine
this
someone's
marching
through
some
environment
here
and
I'm
just
watching
it-
I'm
not
controlling
it.
I'm
just
watching
someone
play
this
game.
A
I
can
tell
just
by
looking
at
this,
what's
go
what
the
agent
is
doing.
I
can
tell
if
the
agent
is
turning
left
or
right,
I
can
tell
the
agent
is
going
forward
or
backwards.
I
can
tell
the
agent's
looking
up
or
down.
I
can
tell
the
agent's
moving
sideways
left
or
right.
I
can
even
tell
if
the
agent's
moving
in
a
circle.
You
know
circling
an
adversary.
A
I
can
also
tell
if
the
objects
in
the
environment
are
moving
relative.
You
know
to
the
environment,
all
this
is
coming
through
purely
sensory
data.
That
is,
we
used
to
think
that
the
the
only
way
to
know
what
the
what
the
body
is
doing
is
through
what
they
call
a
reference
motor
command
like.
I
have
a
signal
telling
me
how
I'm
acting
and
and
then
the
cortex
could
use
that
which
is
still
true,
but
this
says
I
don't
have
to
have
that.
You
know
in
terms
of
vision.
A
At
least
I
can
determine
all
the
movements
that
an
agent
is
doing
just
by
looking
at
the
the
visual
input
and
I
don't
need
a
motor
reference
copy,
and
so,
when
you
think
about
this
and
I've
thought
about
this
quite
a
bit
everything
you
need
to
know
how
the
sensor
is
moving
in
the
world
is
available
by
observing
relative
flow
patterns,
and
so
and
these
ells
and
flow
patterns
must.
These
flow
patterns
must
be
present
in
the
optical
nerve.
Otherwise
you
couldn't
do
this
test.
A
You
couldn't
see
what
was
going
on
and
when
I
say
relative
flow
patterns,
if
you
think
about
what's
the
difference
between
all
these
different
type
of
actions,
one
can
take
the
only
difference
between
turning
left
and
going
forward
or
looking
up
or
going
inside
was
a
circling
anniversary
and
so
on.
The
only
difference
is
how
flow
bits
of
flow
is
changing
and
relative
in
different
parts
of
the
scene
and
sometimes
it'll
be
going
forward.
A
There'll
be
flow,
but
it's
moving
to
the
right
on
the
right
side
and
to
the
left
and
the
left
side
type
of
thing,
but
if
you're
turning
left
they're
all
moving
one
way
and
it's
a
relative
flow
that
determines
how
you
can
see
all
these
things
and
just
intuitively
know,
what's
going
on,
there's
no
thinking
involved
at
all.
Just
you
just
automatically
know
how
the
object
is
moving.
A
So
now
we
come
back
to
our-
and
this
is
I
showed
one
of
these
diagrams
previously,
and
I've
just
got
two
similar.
These
two
diagrams
are
really
showing
the
same
thing,
just
two
different
ways
of
looking
at
it,
and
and
so
what's
really
nice
about
this
is
there
are
two
different
types
of
sensory
data
coming
from
the
retina?
A
There
are
these
particle
cellular
and
and
manual
cellular
cells
and
they're
segregated
in
the
thalamus
that
is
they're,
not
mixed
in
the
thalamus.
So
these
this
picture
shows
the
thalamus
here.
This
is
the
thalamus
down
here.
As
well
and
that
there's
layers
in
the
thalamus
that
are
segregating
the
in
the
kids,
take
the
pink
or
the
parmesan
cells
and
the
blue
are
the
magnocellular
cells
and
they
have
different
distributions.
A
On
the
eyes
too,
the
parva
cellular
cells
are
primarily
in
the
fovea
in
the
very
center
of
the
magnesium
cells
are
all
around
and
the
magnetocellular
cells
are
ideal
for
for
doing
flow
detection.
They
have
very
fast,
responsive,
broad
receptor
fields,
there's
no
tonic
response,
meaning
that
if
they
only
fire
when
something
is
changing-
and
they
don't
involve
color,
so
they're
ideal
for
flow
detection
and
whereas
the
problems
of
themselves
are
the
opposite,
they
have
a
slower
response.
A
A
So
now,
let's
look
at
the
let's.
Just
look
at
the
I'm.
Sorry
look
at:
let's
look
at
the
a
more
detailed
view
of
what's
going
on
in
the
cortex,
and
so
this
is
all
pretty
well.
No.
This
is
I'm
not
making
any
of
this
up
here.
First
of
all,
now
on
the
left
here
I
show
in
it's
all
b1,
so
we're
talking
about
this
is
going
to
be
lgn
here.
We're
just
talking
about.
Excuse
me
all
vision,
we're
talking
about
now,
so
we
have
the
retina.
We
have.
A
It's
not
known
how
what
happens
at
this
point,
how
they're
mixed
or
not
mixed,
that
is
not
known
as
flash
I've
been
have
been
able
to
find
out.
I've
asked
people,
no
one
seems
to
know
so
we
don't
know
what
mixing
is
going
on
here.
If
there
can
mean
segregation
up.
We
do
know,
though,
that
when
the
input
goes
to
the
core
cortical
column
or
region,
there
is
not
just
to
layer.
Four
there's
three
separate
areas
we've
received
input
from
the
thalamus
one
is
in
lower
layer.
Five
and
one
is
lower
layer.
A
Three,
there
are
more
cells,
projecting
the
layer
four,
so
that's
the
one
that
people
talk
about,
but
these
are
separate
inputs
and
and
they
independently
drive
these
cells.
So
even
if
layer
four
is
disabled,
these
layer
piece
valves
will
respond,
and
then
we've
talked
about
this
quite
a
bit
that
these
these
are
complex
cells
and
simple
cells
and
complex
cells.
A
Complex
cells
are
cells
that
respond
to
movement,
simple
cells,
do
not
they're
they're,
basically
just
sort
of
pattern,
mechanizes
and
I
showed
some
old
literature
and
that,
where
the
complex
cells
they
they're
described
as
like,
oh
a
line
moving
to
the
right
or
a
line
moving
to
the
left,
that's
what
make
these
cells
respond,
but
the
actual
the
actually
that's
not
true.
They
respond
to
flow
anything
moving
to
the
right
and
anything
moving
to
left
makes
a
complex
cells
respond.
A
We've
talked
about
that
in
our
papers,
but
this
is
a
little
problematic
in
this
case,
because
this
tells
me
that
whatever
this,
these
cells
are
telling
the
thalamus
to
do
they're
also
telling
something
to
layer,
four,
which
I
don't
quite
understand
completely
right
now.
So
this
is
a
more
complete
picture
of
what's
going
on,
even
though
it's
not
complete,
we
don't
really
understand
it
and,
of
course,
there's
many
other
connections.
Like
our
voting
connections.
We've
talked
about
from
other
layers
that
go
across
cortical
cortical
connections.
A
There
are
a
lot
of
cortical
cortical
connections,
but
this
is
basically
representing
so
the
feed
flow
or
flow
into
the
cortex
from
region.
The
region,
in
this
case
the
output
of
a
v1
layer,
5
cell,
on
a
region
these
project
to
the
superior
colliculus,
which
is
the
old
motor
center
for
for
many
parts
of
the
body,
but
especially
for
the
eyes,
so
in
sense
that
these
cells
in
v1
and
v2
are
able
to
control
eye
movement.
So
we
have.
We
have
movement
information
coming
in
on
the
magnets
of
the
cells.
A
We
have
details,
retinal
information
coming
in
on
privacy
cells,
they're
maintained.
We
now
have
flow
data
going
on
in
the
cortex
and
and
then
we
have
an
output
motor
command
which
is
coming
back
so
we
have.
We
now
have
all
of
our
pieces.
We
don't
know
what's
going
on.
I
don't
know.
What's
going
on
in
these
higher
order
groups,
these
higher
order
relay
cells,
thalamic,
centers
or
whether
there's
something
equivalent
to
par
bro
and
magnum,
so
they're-
that's
not
known,
but
I
can
make
it
work
either
way.
A
It
also
comes
about
the
fact
that
the
idea
that
these
cells
are
you
know
one
spike
in
and
one
spike
out.
Well
that
doesn't
seem
like
they're
doing
much
does
it,
but
that
would
make
a
lot
of
sense.
If
you're
trying
to
multiplex,
where
that
spike
went
too,
you
could
say,
oh
one
spot
comes
in,
but
it
might
go
out
in
a
different
direction,
different
stuff,
so
we
we
spent
a
lot
of
time.
A
Thinking
about
that,
whether
that
could
be
what's
going
on,
because
the
thalamus
is
clearly
essential
to
everything
going
on
in
the
cortex
and
it
needs
to
have
an
important
role
and
so
we're
trying
to
figure
out
what
it
might
do.
So
now
we
have
okay,
we
have
this
multiplexer.
So
how
could
the
multiplexer
idea
explain
reference
ring
transformations?
A
That's
the
question
we're
going
to
look
at
next,
so
here's
the
basic
idea-
and
I
think
you
know
we-
this
came
up
from
talking-
you
know
with
super
tight
and
others,
and
and-
and
we
actually
in
our
slack
chat
this
last
week.
I
talked
about
this
idea,
so
just
just
gonna
try
to
put
it
in
a
picture
to
it.
As
best
I
can
imagine
you
have
a
patch
of
retina,
it's
a
small
patch
of
retina
and
you
have
a
quarter
of
a
column
that
is
getting
input
from
that
pattern.
A
A
So
we
have
I'm
showing
four
magnacellular
thalamocortical
cells
and
we're
going
to
say
that
each
of
these
cells
is
going
to
represent
a
movement
in
this
patch
of
retina
in
some
particular
direction.
So
if
the
things
are
moving
to
the
left
in
a
particular
patch
of
retina,
then
the
cell's
going
to
fire
and
things
are
moving
to
the
up,
but
this
cell
is
going
to
fire
and
so
on,
but
actually,
but
that's
not
real.
That
was
misstatement.
That's
that's
how
we
think
of
it,
but
actually
we
want
this
to
represent.
A
A
So
I
said
it
here
right:
relative
the
object
in
the
world
an
alicentral
flow,
yet
we
need
to
detect
the
flow
relative
to
the
patch
of
retina,
and
so
these
down
here,
I'm
showing
for
each
of
these
phallocortical
cells,
I'm
showing
the
sort
of
the
representation
of
four
dendrites.
So
these
these
little
blocks
are
representing
dendrites
on
this
cell.
These
four
represent
dendrites,
and
this
on
these
four
representative
cells
and
independently,
the
dendrites
are
getting
input
from
the
retina
and
the
dendrites
represent.
A
This
is
what
I
should
have
said.
The
dendrites
represent
flow
in
a
particular
direction
in
the
retinal
center
coordinates,
so
this
dendrite
will
become
active
when,
when
the
flow
is
moving
to
the
left
on
the
retinal
patch
and
this
one
when
flows
moving
up
on
the
metal
patch
and
this
one
floating
down
rapidly.
But
all
four
of
these
project
to
this
cell
and
this
cell
represents
when
flow
is
reaching
the
left
in
the
object's
point
of
view,
and
so
you
can
map
different
directions
of
flow
on
a
on
these
different
dendrites.
A
So
this
is
this:
is
a
retinal
centric
flow
to
object-centric
flow
by
enabling
or
disabling
different
dendrites.
So
now
the
basic
idea
is
a
cortical
column
from
its
layer.
6A
cell
sends
feedback
to
the
cortex,
this
blue
line,
and
the
idea
is
that
this
layer,
6
feedback,
enables
individual
dendrites
on
each
side
of
a
critical
cell.
So
this
this
will
say
under
some
context.
I
want
you
to
pass
up.
You
know
this
information
or
this
information
of
that
information.
A
So
under
a
different
context,
I
might
enable
different
dendrites,
and
so
in
this
case
I
would
say
well
when
the
flow
information
is
moving
in
this
direction.
On
the
retinal
pact,
I
want
to
report
it
as
moving
this
direction
to
the
cortex,
so
in
some
sense
I've,
rotated,
90
degrees,
clockwise
and
same
thing
here.
So
okay,
when
it's
going
to
the
left
on
the
cortex,
I
want
to
be
going
hold
to
the
left
on
the
right
now.
A
I
want
to
be
going
vertical
on
the
cortex
to
the
cortex
or
the
object,
and
so
you
can.
You
can
basically
take
these
flow
patterns
and
you
can
you
can
rotate
them
any
way.
You
want
in
any
point,
and
so
over
the
entire
cortex.
You
could
be
rotating
all
these
flow
patterns
and
therefore
the
movements
that
are
detected
in
the
world
relative
to
the
retina
can
be
can
be
translated
into
movements
relative
to
the
object
being
observed.
A
C
A
A
So
you
know
in
some
way
you
can
say
that
the
thalamus
is
a
black
box,
but
the
other
way
you
could
say
there
are
hundreds,
probably
thousands
of
empirical
data
papers
on
the
thalamus.
So
there's
a
hell
of
a
lot
of
data
there.
It's
unbelievable,
I
used
to.
I
think
I
got
rid
of
it,
but
I
used
to
have
a
two-volume
book
on
the
thalamus
that
was
written
like
20
years
ago
by
what's
his
name.
A
I
can't
remember
now,
but
you
know
that
was
hundreds
of
papers
just
on
top,
but
it's
there's
so
much
data
about
this,
but
no
one
knows
what
it
what
the
data
means.
That's,
so
you
it's
not
it's
not
true
to
say
we
couldn't
in
some
sense
verify
this
theory.
Also.
I
again,
as
I
wrote
in
the
in
the
thousand
brains
and
I've
talked
about
many
times.
A
If
we
can
come
up
with
a
system
that
is
copacetic
with
all
the
known
details
of
physiological
and
anatomical
details
in
the
thalamocortical
system
and
that
that
very
complex
system
does
exactly
what
we
need,
this
complexity
system
to
do
that
we
have
a
mapping
between
a
very
detailed
theoretical
requirements
and
very
detailed
empirical
results.
Then
you
almost
be
certain
it's
correct,
it's
just
because,
if
you're
solving
so
many
constraints
at
the
same
time,
so
I
think
this
is-
I
don't
think
this
is
untestable.
I
think
this
is
quite
testable.
A
D
I
guess
it's
worth
pointing
out,
probably
that
I
mean
a
lot
of
the
data
is
with
without
active
movement,
and
in
that
case
you
couldn't
really
just
you
wouldn't
be
able
to
make
this
distinction
right.
You
wouldn't
be
able
to
tease
it
apart.
If
there's
no,
if
only
the
object
is
moving
in
the
world
and
and
the
animal
is
head
fixed
and
you
know
their
eyes
are
fixed
and
stuff.
You
couldn't
really
tease
this
apart
right.
You
need
some
sort
of
active
movement.
A
D
I
will
say
with
our
our
model
is
pretty
consistent,
not
yet
consistent.
It's
not
the
right
word
like
it.
The
you
know
carmen
and
I
think
the
the
anatomy
and
physiology
suggests
that
this
is
very
possible.
You
know
whether
it
actually
does
it
or
not.
There's
a
difference,
but
this
sort
of
multiplex
thing
is
is
very,
I
think.
A
Yeah,
I
think
you
know
the
interesting
idea
of
doing
multiplexing
I
think
came
came
a
long
before
we
even
talked
to
carmen.
If
I
recall,
I
mean
it
in
my
mind,
it
was
like
hey
given
this
idea
that
there's
these
you
know,
as
they
said
earlier,
that
there's
these
you
know
relays
a
single
spike
type
of
thing.
That
says
well,
there's
only
so
many
things
you
can
do
with
the
relay
like
that.
A
You
know,
and
my
original
idea
was
going
to
be
using
those
triadic,
synapses
and
and
I've
forgotten
that
that
we'd
disprove
that
but
last
week
we
talked
about
that,
and
that
was
not
right.
But
then
in
the
in
the
poster
that
suicide
went
over
last
week,
there
was
the
alternate
proposal
about
the
different
dendrites
getting
in
and
out
of
these
different
states
and
which
does
fit
with
it.
A
So
even
there
I
think
the
original
idea
of
multiplexing
came
before
we
even
knew
a
mechanism
for
doing
it,
and
then
we
looked
at
it
was
one
of
the
proposals
we
had
at
least
one
of
the
posts
I
thought
about
for
many
years
and
and
then
we
looked
at
the
details
by
talking
to
carmen
and
others,
and
they
say
hey,
maybe
it
really
does
fit
it
fits
out.
It
could
explain
it
so
anyway,
but
you'll
see
in
a
moment.
There's
testable
hypotheses
here.
B
So
yeah
one
question:
I
have
early
in
the
presentation.
You
had
kind
of
kind
of
a
fork
in
the
road
where
you
had
two
two
things
you
could
choose
between
and,
and
I
feel
like
you
took
the
opposite-
one
that
I
would
have
naturally
taken
and
that's
when
you
have
the
picture
of
the
of
the
diagram,
how
some
of
the
some
of
that
is
motor
command.
Some
of
it
is
sensory,
the
the
various
levels
of
hierarchy,
and
there
must
be
something
wrong
with
that
picture.
A
I
don't
actually
I
I
don't
think
that
I
didn't
think
everything
about
promoter,
I'm
only
talking
about
the
motor
part.
Clearly
there
is
motor
and
sensory
in
in
the
in
lgm
right
and
what
I
would
like
to
get
to
is
you'd
have
motor
and
sensory
here
too,
and
the
high
order
ones.
I
think
you
need
both.
I.
B
A
A
If
I
try
to
work
through
the
details,
it
doesn't
work
for
me
yet,
and
so
I
don't
know
how
to
do
that.
So
I'm
just
focused
on
the
one
part
right
now
not
to
the
because
the
exclusion
of
the
other
one,
because
the
other
one
has
to
occur.
I
just
don't
know
how
to
make
it
occur
yet
so
it's
just
to
gun
that
point
a
little
bit
further.
If
you
look
at
this
diagram,
it's
very
it's
very
interesting.
A
We
have
these
else,
these
layer,
six
cells
which
are
controlling
the
thalamus
in
some
sense,
but
they're
also
making
these
massive
projections
to
layer
four
and
so
so
what
the
hell
is
going
on
there
right
it's
like,
and
and
is
it
possible
that
that
this
is
a
two-stage
transformation
that
the
same
cells?
Here,
let's
say
these
cells
represent
sort
of
orientation
of
the
object
to
the
to
the
to
the
sensor.
Let's
just
make
a
stab
like
that.
A
Well,
okay,
that
orientation
was
that
would
be
the
signal
I
would
need
to
to
run
this
transform
I'm
talking
about
the
thalamus,
but
it
would
also
theory
could
be
doing
some
similar
transform
up
in
layer
four.
In
fact,
this
idea
that
this
idea
that
different
dendrites
could
be
representing
different
movement
vectors
and
that
they're
enabled
or
disabled
independently
could
apply
to
a
lot
of
neurons.
A
In
fact,
I
think
it's
almost
definitely
going
to
apply
in
under
this
scenario,
has
to
apply
to
the
layer,
five
output
neurons
I'll,
get
to
that
in
a
second.
So,
just
going
to
point
it's
it's
so
damn
complicated,
because
maybe
the
I
can't
figure
out
I
can.
I
can
work
myself
through
how
the
manual
cellular
cells,
if
they
represented
flow
bits,
could
then
lead
to
representing
movement
vectors
in
the
cortex
and
outputting
movement
vectors.
I
do
not
today
know
how
to
do
the
problem.
A
A
Well,
I
can't
rotate
all
the
edges
of
the
dog
90
degrees
and
still
have
a
dog
that
doesn't
work.
I
can
rotate
the
flow
bits
that
way
and
I
still
have
my
movement
commands,
but
I
can't
read
the
the
the
edges
of
the
dog.
These
each
edge
can't
be
rotated
90
degrees
because
they
might
have
a
messed
up
dog.
A
Another
thing
to
consider
marcus
is
that
we
still
have
the
connections
from
layer
three
to
layer,
four
that
were
shown
in
the
earlier
thinkings
of
the
cortex,
so
it's
possible
that
the
sensory
data
has
to
go
through
the
thalamus
the
first
time,
but
doesn't
have
to
go
to
thalmus
the
second
time.
I
don't
know
why.
I
I
just
possible
that's
the
case,
so
there's
a
lot
of
unknowns
here,
I'm
not
picking
the
winners,
yet
it's
it's
just
too
many
balls
in
the
air.
At
this
point.
D
A
D
Yeah,
it
looks
like
that's
what
the
dendrites
are
doing.
They
can
control
whether
the
cell
bursts
or
is
in
tonic
mode.
So
that's
the
way,
that's
the
signal
that
would
get
sent
back
up
to
the
cortex
and
the
other.
I
guess
the
other
thing
we
have
carmen-
and
I
haven't
talked
at
all
about
oscillatory,
stuff
or
any
sort
of
modulation
of
the
oscillations
or
synchronization
or
anything
like
that.
So
that's
another
kind
of
yeah.
A
Well,
we're
very
independent
of
this.
We've
talked
about
other
roles
for
the
thalamus.
We've
talked
about
the
matrix
cells
in
the
thalamus,
doing
timing
and
we've
also
talked
about
the
theta
oscillations
between
the
thumbs
and
the
cortex
controlling
speed
right,
and
I
think
those
are,
I
think
those
are
also
also
going
on.
I
just
didn't
talk
about
those
today
right
today,
I'm
just
focusing
on
this
specific
pathway,
the
magnet
cellular
pathway,
and
could
we
be
doing
a
movement
reference
frame,
transform
you
know
basically,
and
I
can
make
that
work.
A
I
think
I
think
I
can
make
this
whole
thing
work
where
I
could
say
a
movement.
I
can
take
a
movement
in
one
direction
and
transfer
into
a
movement
in
another
direction.
I
mean
moving
in
a
reference
frame
of
my
one
reference
frame
to
a
movement,
a
different
reference,
the
rest
of
it.
I
can't
make
work
yet.
C
A
I
don't
think
so
at
all
the
way.
The
way
I
think
about
this
quran
is
that
every
column
is
doing
the
same
thing.
That's
the
that's!
The
mount
castle
proposal
right,
common
core
value
and
and
and
as
you
it
every
column,
doesn't
know
what
its
inputs
are.
In
fact,
the
columns
get
their
inputs
from
all
over
the
place.
A
You
know
an
individual
region,
the
cortex
doesn't
just
get
you
know.
V1
v2
doesn't
get
just
input
from
v1.
In
fact,
you
know
some
of
these
cortical
regions
get
input
from
seven
or
so
other
regions
at
the
same
time.
So
there's
a
huge
mix
of
stuff
going
on
here,
and
so
all
you
can
say
is
that
any
particular
column,
let's
say,
there's
one
I've
labeled
v2
here
it
doesn't
know
what
its
inputs
represent.
A
It's
going
to
build
its
own
model
of
the
world
and
it
can't
assume
any
knowledge
above
anyone
else's
model
of
the
world.
It
just
can't
know
anything
else
is
going
on.
Just
can't
know
it.
He
says:
okay,
I'm
going
to
get
some
inputs.
Those
inputs
are
going
to
represent
some
sort
of
spatial
pattern
and
a
movement
in
some
space,
and
I'm
going
to
build
a
model
of
that.
A
So
if
you
just
assume
that
there's
this
ignorance,
that's
going
on
between
all
these
different
columns-
and
this
goes
on
for
dozens
of
dozens
of
regions
in
the
cortex,
we
can't
assume
there's
a
similar
transform
that
the
same
transform
can
be
used
everywhere.
In
some
sense,
you
have
to
start
over
again,
which
is
which
is
interesting
because
I
want
to
get
to
in
a
second
here,
see
this
layer,
five
output.
A
Well,
my
assumption
is
this
is
back
into
an
egocentric
reference
frame
that,
if
the
layer,
if
v1
or
v2,
is
going
to
control
the
eye
movements,
it's
going
to
have
to
do
it
in
it's
gonna
have
to
send
a
signal
back
down
to
some
part
of
the
brain
which
says
you
know.
You
know,
I'm
moving
my
eye.
You
know
from
the
from
the
center
of
my
cell
phone
to
the
to
the
to
the
power
button.
But
to
do
that,
you
need
to
move
the
eye.
A
You
know
left
to
right
or
something
like
that,
and
so
this
has
to
be
this.
Motor
output
here
has
to
be
an
egocentric.
These
inputs
have
to
be
egocentric
and,
and
so
what
you're
actually
feeding
into
this
next
column
is
a
motor
command.
That's
an
egocentric
motor
command,
it's
not
it's!
It
hasn't
been
it's
no
longer
in
the
in
the
reference
frame
of
the
object.
A
So
it's
it
just
says:
okay,
this
this
column
on
its
own
says.
Okay,
I
get
some
input.
I
transfer,
I
model
the
system.
I
make
motor
outputs,
the
motor
outputs
are
going
to
be
an
eco-centric
reference
frame.
I
need
to
do
that
and
then
it
says
the
next
column
says:
okay,
I'm
independently
getting
some
eco-centric
movement
data
and
I'm
gonna.
You
know
and
some
sensory
data
and
I'm
going
to
model
it
and
make
an
output.
So
I
think
they
all
have
to
be
independent.
They
all
have
to
be
doing
their
own
thing.
A
A
When
I
just
go
on
and
talk
about
some
issues-
and
I
just
wrote
these-
I
already
said
this
here-
so
the
layer,
six
feedback
enables
individual
dendrites
and
each
slam,
a
cortical
cell
and
subatai
was
saying.
That's
done
to
this.
This
bursting
mode,
tonic
versus
bursting,
and
I
said:
okay,
here's
how
you
could
remap
them.
So
there
are
some
issues
here
and
I
just
thought
I
just
these
literally.
I
wrote
down
10
minutes
before
the
the
meeting.
A
These
salama
cortical
cells
here
are
all
reported
to
have
center,
surround
or
center
on
center
off
receptive
fields.
They're
not
reported
to
have
flow.
They
don't
report
to
have
movement
direction,
it's
it's
like
you
know,
and
and
that's
a
problem
for
this
theory.
I
need
these
cells
to
represent
movement.
I
can't
just
represent
center,
surround
responses
so
for
this
theory
to
work.
A
That
observation
has
to
be
wrong,
and
I
said
here
could
that
be
an
example
of
experimental
setup.
So,
as
super
tight
said
earlier,
if
I
took
an
animal
and
I
anesthetized
it
or
even
worse,
if
I
took
a
a
brain
slice
and
I'm
doing
this
in
a
petri
dish,
you
know
sometimes
they
do
this.
You
know
what's
going
to
happen
with
these
cells.
What's
going
to
happen
with
these
dendrites
is
one
going
to
enable
one,
not
convenient
well
one
of
the
possibilities
that
they're
all
enabled.
A
So
imagine
all
these
centroids
enabled
they
all
would
pass
information
through.
So
what
would
this
cell
response
look
like?
Well,
it
would
respond,
no
matter
which
way
the
movement
and
flow
was
occurring
and
that
might
look
like
a
center
surround
receptor
field.
You
would
say
yeah
well,
I,
and
you
can
think
of
that.
This
way
you
can
think
of
center
surface
receptor
field,
there's
a
union
of
all
the
possible
movements.
A
It's
sort
of
saying,
like
I
don't
know
which
movement's
going
on,
I'm
not
sure
which
movement
I'm
not
sure
how
I
should
orient
my
movement.
So
I'm
going
to
respond,
no
matter
which
way
is
moving.
It's
a
it's
a
union
of
all
possible
movements,
but
maybe
only
under
intentional
gaze
of
the
animal
actually
looking
at
something
and
saying:
oh
yeah,
I'm
looking
at
a
coffee
cup.
D
I
I
think
this
comes
from,
and
I
also
think
this
is
an
outdated
view
now,
but
I
think
it
comes
from
retinal
ganglion
cells
having
center
surround
properties,
and
I
think
that
that
that's
one
of
the
reasons
they
thought
these
were
relay
cells,
they're,
just
kind
of
mimicking.
The
response
properties
interesting.
E
D
I
thought
I
saw,
but
I
thought
I
saw
a
paper
where
they
said
that
that
there's
a
more
recent
paper-
that's
a
pretty.
That
is
the
traditional
view,
but
they
found
that
the
the
retinal
cells
are
much
more
sophisticated
than
that.
There
are
many
many
different
types
of
response
properties
and
then
those
are,
then
you
know
feed
back
up
to
the
thalamus,
so
that
could
be
included
in
that
is
flow
and
and
yeah
things
like
that.
A
Also,
you
know,
as
as
we
talked
about
last
week,
if
you
look
at
these
these,
these
thalamic
cortical
cells-
these
guys
here
and
as
as
a
sherman
report,
they
have
about
6
000
synapses
about
15
of
them
are
driver
from
the
retina,
so
that
means
they're.
Getting
each
of
these
cells
is
getting
about
900
synapses
from
retinal
ganglion
cells.
That
certainly
doesn't
sound
like
a
relay.
Does
it.
E
A
Well,
it
turns
out
that
actually
the
retinal
ganglion
cells
in
my
in
this
theory
here
the
red
and
ganglion
cells-
could
still
be
thinner,
surround,
but
imagine
each
of
these
dendrites
now
there
might
be
more
than
four
here
right.
I've
only
showed
four,
because
it's
the
one
that
dropped
but
yeah.
It
depends
on
where
you
you
take
the
branching
point,
but
there
could
be
more.
There
could
be
easily.
There
could
be.
A
You
know,
15
or
so
of
these
these
things
here
each
one
is
going
to
get
hundreds
of
of
inputs
from
the
retina
and
even
if
those
inputs
from
the
retina
were
center
surround,
these
dendrites
could
easily
recognize
a
fl
that,
by
looking
at
a
pattern
of
those
inputs
in
the
retina,
could
the
dendrites
could
then
say.
Oh
I'm,
going
to
respond
to
a
flow
of
these
center
sounds
in
a
particular
direction.
Well,.
E
A
D
A
So
this
theory
makes
no
statement
about
that,
but
it
does
make
a
statement
that
the
phalaemocortical
cells
must
represent
flow
and
cannot
be
center
surround
under
normal,
attentive
conditions.
So
we
can.
We
can
accommodate,
what's
whatever
happens,
going
on
in
the
retina,
because
we've
got
all
these
converging
inputs
onto
these
retina
onto
these
dendrites,
but
so
that
we're
we're
we're
ambivalent
about.
But
we
can't
be
ambivalent
about
here
that
we
can't,
we
can't
be
the
each
dendrite
has
to
represent
flow,
and
each
cell
then
has
to
represent
flow
anyway.
A
I
think
it's
quite
possible
that
you
could
still
observe
these
cells
these
these
pharmacocortical
cells
as
having
centers
around
response
properties
only
because
they're
they're
testing
under
conditions
where
none
of
the
you
know
there
hasn't
been
a
selection
of
which
dendrites
are
responsible,
and
so
all
dendrites
are
reporting
correctly
they're
all
responding.
Then
the
cell
will
look
like
a
center
surround
it'll
respond
to
a
flow
in
any
direction.
Therefore,
it's
not
flow
sensitive,
it's
not
directional
sensitive.
So
I
think
that's,
that's
it.
That's
a
very
getting
back
to
karan's
early
point.
A
That's
a
very
testable
hypothesis.
You
know
you
could
easily
probe
these
cells
and
say
you
know,
we're
looking
for
flow
patterns
and
we're
looking
for
a
directional
sensitivity
that
changes
under
context
as
the
object
being
viewed
changes.
So
it's
very
testable
theory
and
there
might
be
data
about
that
already.
The
second
that's
a
big
issue.
A
The
second
issue
we've
already
talked
about
is
like
what's
happening
in
the
part
of
a
cellular
pathway
as
marcus
brought
up
the
same
mechanism
could
work
for
changing
the
orientation
of
features
that
in
in
you
know,
static
features
in
the
input,
but
that,
as
I
point
out
that
isn't
sufficient,
you
can't
rotate
it.
It's
only
part
of
the
problem
and-
and
I
haven't
thought
about
it
sufficiently,
but
it's
clearly
this
it's
more
complicated
here,
but
ideally
that
would
be
nice.
A
It
would
be
nice
that
you
know
we
got
these
relay
cells,
they
they're.
They
exist
in
all
the
different
layers
in
the
thalamus,
so
there's
relay
cells
in
the
proper
cellular
pathways.
They
should
be
doing
something
similar,
and
so
they
probably
are,
but
we
that
alone
does
not
seem
to
be
sufficient
to
really
solve
the
problem
of
of
orientation.
You
know
reference
frame
transforms
for
visual
objects,
you
know,
maybe
it
doesn't.
I
just
haven't
thought
about
it
clearly.
A
Yet
I
already
pointed
out
that
layer,
6a
cells
also
projected
layer
4,
which
is
very
suggestive,
saying,
hey,
maybe
maybe
layer
4
is
doing
something
similar
to
what's
going
on
in
the
thalamus.
Maybe
remember
these
cells
in
layer,
four:
are
these
they're,
not
pyramidal
cells?
There's
these
stellite
cells,
which
which
appear
throughout
the
brain
in
that
cortex
and
and
the
hippocampal
formation
in
very
specific
parts
of
the
they,
and
so
maybe
there's
something
going
on
there.
Maybe
these
cells
really
are
are
doing
part
of
the
transform.
C
A
A
I
used
to
run
my
finger
along
the
edge
of
the
the
eraser
panel
at
the
bottom
of
the
whiteboard
and
as
I
and
as
I
ran
my
finger
along
it,
like
my
thumb
along
it,
my
thumb
would
rotate
as
I
ran
it
along
it
and-
and
I
said
to
myself
my
perception-
I
don't
perceive
that
change,
as
my
thumb
is
my
orientation.
My
thumb
is
changing.
Is
it
going
along
this
edge,
and
so
the
pattern
of
my
thumb
is
changing?
My
perception
is
that
I
have
this
line.
A
This
edge
is
straight
and
consistent
and
not
changing
in
front
of
me.
It's
like
it's
like
it's
it's
somehow.
We
go
right
to
having
this
constant,
horizontal
edge
in
front
of
me,
and
I
don't
change.
I
don't
have.
I
don't
sense
the
changing
pattern
on
my
thumb
and
so
it
could.
It
could
explain
something
like
that.
I
think
it
could
say
hey
you
know
what
I've
got
it
could
do
that
see
that,
but
that's
not
the
whole
object.
That's
just
like
some
edge,
it
could
be
transferring
an
edge.
A
A
The
last
thing
is
something
else
that
we
just
we
already
marked
about
briefly
here-
that
these
these
layer,
five
motor
cells,
the
ones
that
project
supporting
a
neurons
that
also
predict
to
the
higher
order,
thoramic
relay
systems
they
have
to
go
back
to
being
egocentric
orientation.
I
mentioned
that
here
I
have
to
if,
under
this
scenario,
all
the
processing
going
on
in
a
quarter
column
is
in
the
reference
frame
or
the
orientation
of
the
object
being
modeled.
A
So
I've
converted
somehow
my
inputs
into
the
to
the
object's
reference
frame.
The
processing
is
all
occurring
in
the
object
reference
frame.
I
now
have
a
motor
output,
but
the
motor
output
has
to
be
converted
back
into
an
egocentric
reference
ring.
So
to
achieve
a
certain
goal
relative
to
the
object
means
I
move.
I
make
a
movement
relative
to
the
object,
but
then
I
have
to
convert
that
in
the
movement
relative
to
the
the
effector
of
the
motor
of
the
body.
A
If
you
will
so-
and
this
is
not
going
back
to
the
thalamus
there's
no
equivalent
here
so-
but
it
did
occur
to
me
that
these
layer,
five
cells,
that
that
do
this
they're,
they
are
noted
because
they
have
this
very
unusual
physiological
property,
they're
called
intrinsically
bursting
cells
and
similar
to
the
cells
in
the
thalamus.
They
have
sort
of
a
bursting
and
atomic
mode,
and
so
it's
possible
that
they're
doing
the
exact
same
thing
as
these
cells
down
here,
but
they're
doing
it
up
here.
A
That
is
the
it's
possible
that
the
dendrites
on
these
layer,
five
cells,
are
representing
movement
in
the
allocentric
object-centric
reference
frame.
But
then
I
map
it
to
a
specific
movement
in
the
egocentric
reference
chain
by
enabling
one
of
those
dendrites,
and
so
because
that
has
to
occur
as
far
as
I'm
concerned.
It
just
has
to
happen
someplace
and-
and
so
it's
so
that's
a
nice
little
twist
that
you
see
this
similar
type
of
bursting
tonic
mode.
A
So
that's
another
thing
that
is
required
of
this
theory
and-
and
all
this
seems
super
super
complicated,
just
just
keep
in
mind
that
it
has
to
occur.
These
things
have
to
occur.
It's
not
like.
We
have
an
option
here
and
so
they're
gonna
occur
someplace.
So
it's
not
like.
Oh,
this
is
a
bad
idea.
We
don't
need
to
do
this.
No,
we
gotta
do
this,
so
we
have
to
find
the
mechanisms
for
doing
it.
A
A
It
says
every
cortical
column
has
to
take
some
input,
translate
it
into
a
local
reference
frame
process
it
and
then
pass
it
back
out
in
the
in
the
original
reference
frame,
and-
and
so
that's
a
it's
a
nice,
it's
a
it's
really
the
first
time
we
have
a
that
I've
known
of
that
there's
a
real
purpose
for
the
thomas.
It's
it's
like.
I
I.
A
A
So
it's
it's
a
critical
role
in
the
whole
system
and
we
need
to.
We
need
a
critical
function
for
it
and
referencing
transformation.
Just
strikes
me
as
like:
oh
that's,
a
beautiful
thing.
If
it
did
it,
it's
just
a
beautiful,
singular
purpose
that
every
column
needs
to
do
and
there's
a
mechanism
for
it
and
that's
and
that's
one
of
the
reasons
it
exists
along
with
the
other
things
we
talked
about
earlier.
A
A
B
Strikes
me
as
you're
thinking
about
it
too
quickly
and
just
just
pausing
for
a
second.
If
you
picture
like
here's,
what
I
think
your
mental
model
is
like
picture,
a
four
by
four
grid
of
cortical
columns
and
the
coffee
cup
is
kind
of
landing
on
it,
landing
in
that
grid
and
you're
saying
it's
a
it's
a
problem.
If
you
take
all
of
the
16
squares
of
that
grid
and
rotate
them.
Oh.
A
B
A
B
A
A
The
beauty
of
the
thousand
brains
theory
is
that
each
column
looks
only
a
teeny
part
of
the
world
and
nobody
has
to
assemble
it
into
an
image
right.
All
they
have
to
do
is
say
this
teeny
part
of
the
world
is
at
some
location
in
a
reference
frame,
and
this
teeny
part
of
the
world
is
some
location,
a
reference
I
mean
this
pinot
noir
or
some
location
reference,
no
one
reassembles
the
image
anywhere,
and
so
I
think
what
you're
saying
well.
A
This
is
what
popped
in
my
head,
when
you
were
just
talking,
was
that
the
whole
thousand
brains
theory
says
yeah,
I
don't
need
to
reassemble.
There's
nobody
looking
at
this
image,
in
fact,
if
it's
like
yeah,
if
I
were
to
look
at
the
cortex
I'd
see
all
these
rotated
lines,
but
no
one's
looking
at
the
dog,
and
so
each
rotated
line
on
each
column
is
basically
saying.
Is
this
the
right
thing
I
should
be
seeing
at
this
location
on
the
object
and
the
answer
could
be
yes,
it's
it's
like.
A
A
I
can
take
a
let's
take.
Take
my
thumb
right.
I
can
take
an
edge.
That's
going
across.
My
thumb,
you
know,
take
off
and
stop
sharing
here
show
my
thumb.
I
take
a
line.
That's
going
across
my
thumb
and
that's
that's
running
along
the
edge
of
the
the
whiteboard
eraser
contain
tray
right
and
wherever
it's
on
it's
on
this
coffee
cup.
A
Here
right
and
I
say:
okay,
but
now
I
rotate
my
thumb
and
I
get
the
different
impression,
but,
what's
being
passed
to
the
cortex,
just
take
this
one,
it's
being
passed
to
the
cortex,
even
though
it's
changing
on
my
thumb,
the
cortex
is
getting
the
same
input.
The
cortex
is
saying
relative
to
the
cup.
There's
a
line
horizontal
relative
to
the
cup
it
doesn't
and
as
I'm
rotating
this,
the
different
dendrites
on
those
cells
are
changing.
A
A
A
A
D
B
D
A
Well,
then,
I
would
think
that,
and
then
I
turned
my
head
rotate
my
head
like
this,
so
the
input
on
my
retina
is
changing
as
I
do
this,
but
my
perception
is
that
it's
stable
here.
So
that
means
I
would,
I
would
expect
the
retinal
output
be
rotating,
but
I
wouldn't
expect
the
layer
four
cells
to
be
rotating.
That
is
the
same
layer.
Four
cells
would
be
responding
and
I
don't
think
that's
I'm
not
aware.
A
D
E
C
D
A
A
So
here's
a
here's,
a
very
succinct
experiment:
you
have
a
monkey
and-
and
the
monkey
is
looking
at
an
object
like
this
coffee
cup
and
is
attending
to
it,
and
then
the
monkey
turns
the
head
left
and
right
right.
We
know
that
the
the
pattern
on
the
retina
is
going
to
change.
We
know
that
in
the
classic
view
the
layer
four
cells
should
change,
but
in
this
view
the
layer
four
cell
should
be
stable.
B
A
So
you'd
have
to
characterize
the
cells
first
and
then
you
could
see
it.
Yes,
this
so
getting
back.
I
think
was
quran's
question.
There's
a
lot
of
ways
you
could
test
this.
In
fact,
maybe
it's
already
out
in
the
literature
there
already.
So
this
is
so
many
damn
papers
to
look
at
I
you
know
this
is
really.
I
call
this
a
proposal
and
I'm
not
really
certain
about
it,
but
it's.
A
It
really
feels
good.
It's
just
like.
Oh,
I
really
want
this
to
be
true,
it's
just
so
much
going
for
it.
It's
better
than
the
other
proposals.
I've
been
looking
at
by
far
doesn't
mean
it's
right,
but
it
makes
you
want
to
make
it
right
so
anyway,
that's
it
for
today.
That's
all
I
had
to
present
on
this.