►
From YouTube: Jeff Hawkins - Human Brain Project Keynote [Screencast]
Description
On October 15, 2018, Numenta Co-founder Jeff Hawkins gave a keynote presentation at the Human Brain Project Summit Open Day in Maastricht, the Netherlands. Because we were not able to get a recording of that talk, we created a screencast of Jeff presenting the material in our office. The material covers our research paper that was released two days prior to the Human Brain Project Summit, "A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex."
https://numenta.com/neuroscience-research/research-publications/papers/a-framework-for-intelligence-and-cortical-function-based-on-grid-cells-in-the-neocortex/
A
Hi,
it's
Jeff
Hawkins,
with
momentum,
we
could
go
on
October
15th
2018
I
gave
a
talk
in
the
Maastricht
Netherlands
at
the
European
human
brain
project
summit.
It
was
a
keynote
address
at
a
major
neuroscience
conference
and
our
intention
was
to
record
that
talk,
but
that
didn't
happen.
So
we're
recreating
the
talk
in
this
screencast.
It
was
an
honor
to
speak.
A
At
this
event,
the
human
brain
project
is
a
large
10-year
effort
by
the
European
Union
to
understand
the
human
brain
and
I
was
invited
to
speak
because
we
coming
up
with
a
new
theory
about
how
the
human
neocortex
works,
and
it
was
a
great
coincidence
because
we've
been
working
very
hard
on
writing
a
paper
on
this
new
theory
and
that
paper
became
available
to
view
on
a
preprint
server
by
archive
just
two
days
before
my
talk
and
my
talk
followed
that
paper.
So
that's
the
context
for
what
we're
about
to
see
here
great.
A
So
the
title
of
my
talk
is
a
location,
Location
Location,
a
framework
for
intelligence
and
cortical
computation.
So
let's
just
jump
right
into
it.
This
gentleman
here
is
Francis
Crick
and
he,
of
course,
famous
for
being
one
of
the
co
discoverers
of
the
double
helix
structure
of
the
DNA
molecule,
and
he
want
to
know
about
prize
for
that,
the
latter
part
of
his
life.
He
spent
thinking
about
the
other
big
problem,
which
is
human
intelligence
and
the
brain
in
1979.
A
He
wrote
this
essay
called
thinking
about
the
brain
which
had
a
huge
impact
on
my
life,
and
this
was
a
critical
essay.
He
said
you
know
what
people
have
been
studying
the
brain
for
a
long
time,
even
back
in
1979,
and
they
had
amassed
a
huge
amount
of
data,
but
he
said
in
spite
of
this
accumulation
of
detailed
knowledge
how
the
human
brain
works
is
still
profoundly
mysterious.
A
He
said
we
would
clear
their
gonna
get
more
data
over
the
time
and
we're
gonna
come
up
with
new
techniques
for
measuring
and
understanding
how
the
brain
works.
He
says,
but
it
may
not
matter
because
we're
just
not
thinking
about
the
problem
correctly.
He
said
to
understand
the
brain.
We
need
new
ways
of
thinking
about
it.
More
experimental
data
will
not
be
sufficient,
so
this
is
a
real
sort
of
wake-up
call
for
neurosciences.
A
You
know
we're
just
not
going
about
this
in
the
right
way
and
then
here's
a
longer
quote
I
want
to
read
to
you.
He
says
what
what
is
conspicuously
lacking
is
a
framework
of
ideas
within
which
to
interpret
all
these
different
approaches.
It
is
not
that
most
neurologists
do
not
have
some
general
concept
of
what
is
going
on.
The
trouble
is
a
concept
is
not
precisely
formulated
touch
it
and
it
crumbles
out
he's
being
really
critical
here.
A
He's
saying
you
know,
people
act
as
if
he
understands
this,
the
brain
or,
what's
going
on,
we
said
we
really
don't
and
to
me
when
I
read
this
article
I
said
wow,
that's
incredible.
I
want
to
spend
my
life
working
on
that
and
that's
what
we
do
in
the
mentum.
We
work
on
this
developing
this
framework
for
understanding
your
cortex.
A
Now
that
we
think
about
the
human
brain,
the
neocortex
is
about
75%
of
the
volume
of
the
brain.
That's
a
picture
of
the
neocortex
right
here.
The
other
25%
is
consists
of
like
the
brainstem
and
the
spinal
cord
and
the
cerebellum,
and
it's
like
a
post.
It
sticks
up
inside
of
the
neocortex,
the
old
part
of
the
brain,
the
old
25%
of
the
brain.
This
seems
like
autonomic,
control
of
breathing
and
heart
rate
reflex
reactions,
maybe
even
running
and
walking
or
controlled
by
that
emotions,
but
want
to
think
about
things
everything
we
think
about
intelligence.
A
It's
really
the
neocortex
everything
about
perception
and
language
and
thought
and
planning
all
your
conscious
perceptions
are
all,
and
then
your
cortex,
and
even
today
it
how
it
works,
is
still
a
mystery
is
also
my
opinion,
the
most
important
scientific
problem
of
all
time.
Not
only
does
it
or
we
as
humans,
really
basically
a
neocortex
and
all
of
our
issues
and
problems
in
society
are
related
to
our
brains.
It
is
how
we
think
it
is
all
of
our
Arts
and
Sciences
and
I
believe
understanding.
A
The
neocortex
will
be
important
for
the
long-term
survival
of
our
species.
I'll
get
back
to
that
more.
At
the
end
of
my
talk,
so
here's
my
talk,
outline
I'm
gonna
go
through
some
background
material,
because
I
can't
assume
everybody
knows
a
lot
of
neuroscience.
I
want
to
get
you
up
to
speed
where
we
are
and
then
I'm
going
to
introduce
this
new
framework
for
intelligence
and
cortical
computation
and
then
the
end
of
the
talk.
A
I'll
talk
about
some
of
the
implications
for
the
new
theory,
all
right,
let's
go
into
some
background
material,
here's
a
model
of
the
neocortex,
it's
dinner
napkin,
and
if
you
could
take
your
neocortex
out
of
your
head
and
iron
it
flat,
it
would
be
about
this
big
and
about
this
thick
a
little
bit
thicker.
It's
it's
about
1,500
square
centimeters
in
area
and
two
and
a
half
millimeters
thick
in
this
tissue
in
this
neocortex
there's
somewhere
between
15
and
20
billion
neurons.
A
Now
one
of
the
tenets
of
neuroscience
is
that
every
perception
and
every
thought
and
everything
you've
ever
occurred
to
you
and
perceived,
is
basically
the
activity
of
neurons.
So
some
of
these
neurons
are
active,
but
most
of
them
inactive
any
point
time
and
the
ones
that
are
inactive
represent
your
current
thoughts
and
perceptions.
The
neurons
are
connected
together.
There
are
thousands
of
synapses
or
connection
between
each
neuron,
so
there's
somewhere
between
50
and
100
trillion
synapses
in
the
neocortex
and
the
second
10
of
Neurosciences.
Those
synapses
contain
all
the
knowledge
you
have
about
the
world.
A
So
everything
you
know
every
member
learned
is
stored
in
those
connections.
Now,
if
you
look
at
the
surface
of
the
neocortex,
it's
very
uniform.
You
don't
see
any
demarcation,
but
we
now
know
that
that
different
areas
do
different
things.
This
was
first
discovered
when
people
had
trauma
and
they
had
an
injury
one
spot
and
they
said
oh,
they
can
no
longer
see
or
they
can't
do
language.
They
can't
think
about
certain
types
of
things,
but
we
now
have
mapped
out
the
near
question,
great
detail.
A
We
know
that
these
different
regions
or
connected
to
each
other
with
bundles
of
nerve
fibers
through
the
white
matter
and
some
of
those
regions
are
assigned
to
different
things.
So,
let's
just
talk
about
that
and
go
back
to
the
slides
you
so
now
we
have
here:
I've
shown
some
of
those
regions.
I've
been
highlighted
in
blue
some
visual
regions,
some
auditory
regions
on
some
at
essentia
regions.
Those
are
multiple
regions
in
those
blue
highlighted
areas.
A
You
see,
there's
a
large
part
of
the
cortex,
so
you
can't
easily
associate
with
vision,
auditory
somatic
input
and
the
basic
way
people
think
about.
This
is
the
following.
They
say
the
input
comes
in
from
some
sensory
organ
touches
all
right.
Now
it
projects
you
the
first
visual
region
in
the
back
to
your
brain
and
that
visual
region
extract
some
sort
of
simple
features
and
then
that
those
simple
features
are
passed
to
the
next
region,
which
extract
some
complex
features
which
projects
to
the
next
region.
A
This
is
saying
basically,
people
believe
the
same
basic
thing
is
happening
at
different
modalities,
such
as
touch,
and
so
your
scheme
projects
to
this
somatosensory
regions
in
a
similar
type
of
process
and
somehow,
in
these
most
in
your
cartridges,
some
sort
of
multimodal
or
associations
of
multiple
mobile
objects
and
so
on.
This
is
the
basic
idea
that
how
many
people
think
about
the
brain
today,
it's
actually
not
really
true
and
I'll.
Get
to
that
in
a
moment
when
you
actually
look
at
the
map
out
how
regions
the
brains
connect
to
each
other.
A
This
has
been
done
when
the
first
real
great
map
was
done
in
1991
by
two
people,
Feldman
Vanessa.
This
is
what
they
reported.
This
picture
on
the
left
shows
each
of
those
little
rectangles
is
a
region
of
a
monk,
a
macaque
monkeys,
neocortex
and
showing
the
it's
basically
showing
the
somatosensory
regions
on
the
left.
The
visual
regions
on
the
right
and
the
lines
are
bundles
of
nerve
fibers,
millions
in
nerve
fibers
that
are
going
between
each
of
these
regions,
and
you
can
see
it's
very
complicated.
A
There's,
there's
connections
going
all
over
the
place
and
there's
parallel.
Horizontal
connections
and
parallel
regions
and
there's
level
skipping
going
on
so
we
can
sort
first
of
all
just
say
this
is
a
very
complex
structure.
It's
not
a
simple
flow
chart
like
in
the
upper
right.
It's
certainly
not
strictly
hierarchical.
In
fact,
in
1991
they
reported
that
40%
of
all
possible
connections
between
regions
exist,
which
is
much
greater
than
you
would
get
in
a
hierarchy,
and
now
we
know
that
number
is
much
much
higher
yeah
with
new
techniques.
A
We
know
that
there's
a
far
greater
connectivity,
so
that's
kind
of
confusing.
Now
the
strange
thing
about
is
not
aware.
You
look
in
the
in
the
near
question.
If
you
look
at
any
one
of
these
regions,
the
local
circuitry
anywhere
is
remarkably
the
same.
So
you
have
all
these
regions
doing
different
thing
is
connected
in
the
cranium
way,
but
then
every
region
looks
like
it's
doing
the
same
thing.
A
This
was
first
noted
by
perhaps
the
most
famous
neuro
sciences
of
all
time
or
run
in
Kahala
stanard
and
right
around
the
8th
and
the
late
1800s
and
early
nineteen
hundred's.
He
started
mapping
out
all
different
types
of
parts
of
the
brain.
They
had
just
discovered
these
staining
techniques
where
you
could
take
a
bunch
of
neural
tissue
and
it
would
stain
just
some
of
the
neurons,
and
this
was
important
because,
if
you
staying
to
all
the
neurons,
it
would
be
big
black
mass.
A
So
by
sticking
some
of
the
neurons,
they
could
start
seeing
where
they
look.
What
they
look
like
so
here
on
the
left
of
a
picture
here
is
a
slice
of
the
New
York
cortex.
You
see
it's
two
and
a
half
No,
and
this
stain
highlights
the
cell
bodies,
so
the
neuron
bodies-
and
you
can
see
they
have
different
shapes.
They
have
different
sizes
and
they
have
different
packing
densities
and
they
started
noticing
that
they
appeared
to
be
layers.
A
And
so
there
was
they
started
saying
it
looks
like
there's
six
layers
of
neurons
in
the
neocortex,
and
this
looked
the
same
everywhere.
Then
the
stain
on
the
image
on
the
right
is
a
different
stains.
It
not
only
stains
the
cell
body,
but
it
stains
the
the
dendrites
and
the
axons
that
coming
out
of
the
cells.
So
now
you
can
see
how
they're
connected
you
can
see.
There's
a
lot
of
vertical
connectivity
going
on
across
the
layers
and
some
layers
have
more
horizontal
connectivity.
A
So
for
the
last
120
years
many
neuroscientists
been
mapping
out
what
these
circuits
look
like.
What
these
neurons
look
like
how
they're
connected
together
it's
an
incredibly
rich
field
of
data
people
like
ourselves,
who
are
theorists
recreate
that
we
create
models
from
this.
So
we
build
pictures
like
this,
where
we're
looking
at
the
different
cell
layers
and
trying
to
figure
how
they
connect
together
and
what
they're
doing
and
the
different
types
of
connections
between
them.
When
you
just
summarize
some
of
this
first,
we
could
say
there
are
dozens
of
neuron
types
in
in
the
neocortex.
A
They
are
organized
in
layers
that
how
many
layers
you
get
depends
on
how
you
measure
them
it.
You
know
you
could
say
six
to
ten
and
depending
on
which
way
you
break
them
apart,
but
there
are
prototypical
projections
across
layers,
meaning
that
everywhere
you
look
the
same
type
of
connection
and
the
information
basically
goes
vertically
comes
in
one
layer
and
it
goes
up,
and
that
goes
down
and
back
and
forth
across
the
layers.
There
are
horizontal
connections,
but
they're
more
limited,
and
they
only
come
from
certain
layers.
A
Another
thing
that
was
discovered
was
that
all
regions
is
in
you
know:
neocortex
have
a
motor
output
and
I've
shown
that
here
labeled
this
one
of
the
green
house
coming
back
out-
and
this
is
surprising
you'd
think
like
he
might
have
heard
that
there's
a
motor
section
of
the
New
York
cortex,
but
we
only
did
all
parts
of
the
air
cortex
or
motor
cortex.
So
you
said
well
what
would
the
visual
cortex
have
to
say
about
movement?
A
It's
complex
the
first
person
who
really
made
some
sense
of
this
was
a
gentleman
named
Vernon
mal
castle,
and
he
had
this
huge
idea
by
the
way
he
is
like
one
of
the
fathers
our
neuroscience
and
he
was
really
the
guy
who
who
has
some
of
the
most
elegant
writings
about
the
neocortex.
He
he
was
at
Johns
Hopkins,
and
he
had
this
big
idea
in
in
1978.
He
published
a
monograph
and
in
that
monograph
he
made
the
following
claims.
A
He
says:
well,
look
the
reason
that
all
areas
of
the
neural
cortex
look
the
same
is
because
they
perform
the
same
basic
function.
They're
all
doing
the
same
thing
and
what
makes
one
region
a
visual
cortex.
Another
region
in
an
auditory
region
is
what
it's
connected
to
literally
saying
you
could
take
a
part
of
the
neocortex
and
stick
an
auditory
nerve
into
it
and
it
would
become
auditory
cortex.
A
Well,
you
could
take
the
neck
and
if
you
stuck
a
visual
nerve
until
it
become
visual,
cortex
and
then,
which
is
an
amazing
idea
and
then
he
said,
a
small
area
of
the
cortex
about
a
one
millimeter
squared
area
which
he
called
the
cortical
calm,
is
the
unit
of
replication
and
contains
the
Sokoto
common
cortical
algorithm.
He
chose
a
millimeter
square
because
in
that
square
memory
you
have
all
the
different
cell
types,
all
the
different
connections,
all
the
different
physiological
response
properties.
It
wasn't
that
this
cortical
column
is
a
physical
thing.
A
It
was
that
it
was
a.
It
was
a
small
enough
unit
that
contained
everything.
We
sometimes
make
pictures
like
this
to
illustrate
this.
So
here
we
can
imagine
a
slice
of
the
neocortex
and
you're
looking
at
the
new,
your
cortical
sheet
and
we're
showing
these
individual
columns
packed
in
there
in
a
human
there'd,
be
150,000
of
these
columns.
Now
again,
they're,
not
physical,
like
this.
If
you
actually
look
at
the
cortex
without
it
with
just
a
few
exceptions,
you
wouldn't
see
this,
but
this
is
the
way
to
think
about
it.
A
A
lot
of
nurse
scientists
do
is
what
this
column
looks.
Like
you
know
now,
Casas
idea
was
one
of
the
biggest
ideas
ever
in
science.
I
put
right
up
there
with
Darwin
Darwin
said
that
we
have
this
tremendous
tremendous
diversity
of
life
and
it
all
of
it
comes
about
because
of
a
single
algorithm
repeated
over
and
over
again
a
mountain
castle,
saying
we
have
this
tremendous
diversity
of
intelligence
everything
we
think
about
if
intelligence,
whether
it's
language
and
music
or
arts
or
physics
and
mathematics,
it's
all
based
on
a
single
algorithm.
It's
an
incredible
idea.
A
It's
so
incredible
that
many
neuroscientists
today
have
trouble
dealing
with
it's
like
they
don't
know
how
to
interpret
it.
What
to
do
about
it,
but
it's
clearly
true,
and
it
was
it's
just.
It
should
be
a
foundation
of
all
neocortical
theory.
Okay,
so
the
next
question
I
want
address
is
what
does
the
neocortex
do,
and
over
the
years
people
propose
different
ways
of
looking
about
it,
but
I
and
other
neuroscience.
If
many
of
us
have
come
up
with
this
perspective,
it's
a
thing
about
the
New
York
cortex.
What
it
does
is.
A
It
learns
a
model
of
the
world,
so
when
you're
born
you
don't
know
about
the
world,
you
don't
know
about
buildings
and
cars
and
people,
you
don't
know
about
trees,
you
don't
know
about
computers,
you
don't
know
about
everything
and
you
don't
know
any
languages,
you
don't
know
any
word.
You
have
to
learn
all
of
this,
and
so
you
have
to
learn
this
model
of
the
world
and
in
your
cortex
you
have
a
model
of
the
world
and
and
when
you
interact
with
that
model,
though
you'll
see
it
makes
predictions.
A
So,
let's
talk
about
what
does
that?
No
Idol
consists
of
its
there's,
thousands
and
thousands
of
things.
You
know
you
know
how
they
look,
how
they
feel
and
how
they
sound
objects
in
the
world
that
you
interact
with
every
day.
You
know
where
these
objects
are
located
to
relative
to
other
objects,
so
I'm
sitting
here,
looking
I'm
in
a
room
and
I
see
a
door
and
a
screen
and
floor
and
a
table
and
chairs
these
things.
A
I
recognized
by
they
also
have
relationship
to
each
other,
and
I
would
not
expect
to
see
the
door
on
the
ceiling
or
on
the
floor.
I
know
how
objects
behave
so,
for
example,
a
door
has
hinge
ISM,
it
can
open
and
close,
and
it
has
a
latch
that
goes
up
and
down,
or
my
computer
has
all
kinds
of
behaviors
that
it
changes
as
I
interact
with
it,
and
we
do
this.
We
also
learn
this
for
both
physical
and
abstract
objects.
A
So
I
can
model
physical
things
that
are
in
front
of
me,
but
I
also
model
abstract
concepts
may
like
democracy
or
places
I've
never
been
to,
and
yet
I
still
have
models
of
them
and
then
finally,
it's
a
predictive
model.
So
the
you
have
this
model
in
your
cortex
and
it's
constantly
making
predictions
about.
What's
going
to
happen
next,
and
this
is
its
value
now,
it
makes
predictions
at
all
levels.
So
even
every
time
I
touch
something
I
have
a
prediction:
what
I'm
gonna
feel
and
every
time
I
move
my
eyes.
A
I
would
predict
what
I'm
gonna
see,
even
if
I'm
not
consciously
aware
of
this,
but
I
also
can
make
predictions
about
long-term
things
like
if
I'm
trying
to
apply
to
get
a
grant
for
my
science,
I
might
say
what
so
I
apply
early
or
later
so.
I
use
this
language
of
that
language
to
improve
my
chances
of
getting
accepted,
so
we're
trying
to
predict
the
outcomes
of
our
actions.
All
the
time
I
wrote
an
entire
book
about
how
the
cortex
builds
predictive
models.
It's
called
on
intelligence.
So
this
is
what
the
brain
does
we
want.
A
This
is
an
incredibly
complex
model,
the
world
that
it's
stored
in
your
neocortex,
and
we
will
understand
how
it
happens,
how
it
happens.
So
the
question
we
want
to
now
is:
how
does
the
neocortex
learn
this
model
using
the
circuitry
that
we've
talked
about
so
now
we
can
sit
to
our
new
framework
for
this,
so
I'm
gonna
start
with
a
thought
experiment.
We
this
occurred
just
a
couple
of
years
ago,
and
it
was
a
real
revelation
and
sort
of
a
bunch
of
things
came
out
of
this
thought
experiment.
We
were
asking
how
it
is.
A
A
What
does
the
cortex
need
to
know
to
predict
what
I'm
gonna
feel
when
I
move
my
finger
to
a
new
location,
place
it
down
on
top
of
the
cup,
so
I'm
going
to
touch
the
rim
of
the
cup
here
and
I
can
imagine
what
I'm
gonna
feel
before
I
feel
it?
The
cortex
needs
to
know
several
things.
It
needs
to
know
that
it's
touching
a
coffee
cup
it
needs
to
know,
says:
oh
I
know
what
this
object
is
because
it's
that's
a
requirement
to
know.
A
It
also
needs
to
know
where
the
finger
is
going
to
be
on
the
cop.
After
it
the
finger
comes
down.
If
I
move
the
finger,
a
different
direction,
I'll
touch
something
else
and
I'll
feel
something
different,
so
I
has
to
know
where
it's
gonna
be,
and
it
needs
to
know
what
object
it's
touching
now,
if
I'm.
A
It's
not
lost
in
my
body
the
same
thing:
recurved
the
cup
is
at
a
different
angle,
and
so
on
so
the
cortex
needs
to
know
a
location
in
the
reference
frame
of
the
cop.
It's
kind
of
hard
to
imagine
it
would
do
that,
but
it
not
snow.
It
muscle
needs
to
know
that
now
with
you
realize
that
when
you
touch
the
cup
with
multiple
fingers
at
the
same
time,
over
the
your
hand,
all
parts
of
your
skin
are
making
the
same
type
of
predictions.
A
A
Every
part
of
my
input
has
to
have
a
location,
and
so
this
tells
us
that
in
the
primary
sensory
cortex
in
the
neocortex,
in
this
case
touch,
but
the
same
is
gonna
happen,
a
vision,
an
audition
there
has
to
be
a
representation
in
the
neural
tissue
of
the
location,
on
the
object
that
I'm
touching.
This
is
a
really
interesting
idea
and
we
ran
with
it.
Ok
back
to
the
slides.
A
year
ago,
just
about
a
year
ago,
we
probably
published
our
first
paper
on
this.
The
idea-
and
it
was
called
the
paper
title.
A
The
paper
was
a
theory
of
how
columns
in
the
neocortex,
enable
learning
the
structure
of
the
world
and
I'll
give
you
a
just
brief
synopsis
of.
What's
in
that
paper,
we
argued
that
a
single
column
in
the
cortex,
it's
a
receiving
input
from
the
tip
of
your
finger.
They
have
a
little
incomplete
model
to
the
objects.
How
does
it
do
that?
I
mentioned
I
had
to
have
a
location
signal
which
I'm
illustrating
here
in
blue,
so
we
have
an
input
coming
from
the
finger.
A
You
have
another
input
coming
from
a
different
layer
in
the
cortex
retrieve
believe
is
in
layer
6,
and
that
represents
the
location
of
the
input
relative
to
the
object.
So
now,
I
have
two
things:
I
have
the
actual
sensation
coming
from
the
finger
and
I
had
the
location
on
the
object,
these
arriving
layer
4.
This
is
not
well
known,
Anatomy,
and
so
now,
if
I
think
about
that
I
said:
okay,
I
know
the
sense
and
where
it
is
and
if
I
move
my
finger
multiple
times,
I
can
basically
build
up
a
model
of
the
object.
A
What
the
different
features
are
different
locations
and
we
propose
that
that's
being
assembled
in
an
upper
layer,
layer,
2
3,
this
object
layer
is
a
state.
A
representation
like
this
would
represent
the
coffee
cup.
It's
the
stable
reputation,
meaning
as
I
move
my
finger.
The
input,
changes
and
location
changes,
but
the
representation
in
the
output
layer
is
stable,
so
I
associated
a
set
of
features
and
locations
with
an
object.
We
then
went
on
to
show
that
what
would
happen
if
I
multiple
columns
at
the
same
time,
so
imagine
our
three
fingers
touching
at
the
cop.
A
At
the
same
time,
each
one
is
sensing.
A
different
location
has
a
different
input.
Each
one
on
its
own
would
not
be
able
to
determine
what
the
object
is,
but
they
can
by
voting,
meaning
that
this
object
clave.
They
can
also
be
on
cert
and
say
well,
I,
don't
really
know
what
this
is.
It
could
be
a
B
and
C
and
the
other
one
says
this
could
be
B,
C
and
D,
and
Aleman
says
this
could
be
being
X
and
Q
and
they
say:
oh,
it
must
be
B
the
coffee
cup
we
modeled
this.
A
A
You
almost
certainly
couldn't
recognize
what
the
object
is,
but
if
I
can
touch,
if
I
have
to
move
my
finger,
multiple
patients,
but
if
I
can
grab
it
with
my
whole
hand,
at
once,
I
can
usually
get
in
a
single
sense,
so
multiple
columns
can
infer
objects
in
a
single
sensation
by
voting
on
an
object
identity.
This
is
touch,
but
you
should
realize
that
the
same
thing
is
going
on
in
vision
and
other
sensory
modalities.
When
you
think
about
the
the
eyeball
of
the
retina,
it's
not
a
single
thing.
It's
like
it's
like
the
skin.
A
It's
a
set
of
sensory
organs,
arranged
in
on
the
retina
and
each
one
of
those
is
projecting
to
a
column
and
each
one.
It's
like
the
cortex
doesn't
funeral
cortege
doesn't
look
at
an
image.
It
looks
at
lots
of
little
pieces
of
it
just
like
when
you
get
the
inputs
from
your
fingers,
so
the
same
basic
structure
works
in
all
modalities.
A
We
had
a
big
question,
though,
like
we
propose
that
this
location
representation
is
in
layer,
six
or
one
of
these
lower
layers.
But
how
could
that
be?
Where
could
it
come
from?
What
does
it
look
like?
How
would
the
brain
know
a
location
and
what
does
it
mean
to
know
a
location
on
an
object
in
this
paper?
We
didn't
answer
that
question.
That
is,
we
left
it
as
a
question,
but
we
suggested
where
to
look
to
finding
answer
and
that
turned
out
to
be
correct
what
we
suggested.
A
We
should
look
in
something
called
the
entorhinal
cortex.
Now,
what's
going
on
in
the
entorhinal
cortex,
you
may
have
heard
of
these
in
the
entire
quarter.
So
some
things
called
grid
cells
here
I
have
a
picture
of
a
rodent
in
a
human,
and
it
shows
these
two
older
in
smaller
brain
structures.
They're
not
part
of
the
neocortex
called
the
hippocampus
and
the
answer
I
know
cortex.
A
You
can
see
in
the
human
they're
like
the
size
and
maybe
or
pinky
they're
sort
of
wrapped
around
on
the
inside
of
the
towards
you
old,
a
part
of
the
brain.
A
grid
cells
are
a
very
hot
topic.
A
lot
of
people
have
been
studying
and
Nobel
Prizes
have
been
awarded
for
them
they
and
what
they
do
is
they
represent
the
location
of
your
body
relative
to
an
environment
so,
and
we
have
them
too.
A
So
let
me
just
give
me
what
do
I
mean
by
representing
the
location
of
your
body,
relative,
the
environment,
so
I'm
in
a
room
right
now
and
I
have
a
sense
of
where
I
am
in
the
room
and
remember
when
you
have
a
sense
for
something,
there's
cells
that
are
representing
that
that's
the
grid
cells
in
my
my
in
Toronto
cortex
and
they're,
representing
where
I
am,
and
even
if
I
close
my
eyes
and
I,
take
a
couple
steps
over
here.
I
have
a
I
have
a
different
perception
of
where
I
am
in
them.
A
I
know
now
that
the
stool
is
further
away,
that
wind
goes
further
away
and
I
step
back
here.
I
know
I'm
closer
to
it
again,
so
this
sense
of
where
I
am
is
actually
these
grid
cells
in
the
entorhinal,
cortex
and
they're,
updating
as
I
move
and
because
I
can
they
update,
even
if
I'm
not
looking
anything.
You
know
that
they're
updated
by
my
movement
themselves,
I,
don't
need
a
sensory
input.
To
tell
me
this.
The
brain
says:
you've
moved
I,
know
you're
in
a
different
location.
I'm
gonna
represent
a
location
differently.
A
Okay,
let's
go
back
to
this,
so
these
grid
cells
represent
the
location
in
the
body
relative
in
environment.
The
big
idea
we
have
is
that
grid
cells
also
exist
in
the
neocortex
that
they
were
preserved
in
evolutionary
time
but
they're.
Now
they
used
for
something
different.
The
cortical
grid
cells
represent
the
location
of
a
sensory
input
relative
to
the
objects,
you're
sensing
and
I'll
go
into
this
more
in
detail.
A
First
of
all,
I
need
to
tell
you
how
grid
cells
work.
Now
this
is
complicated
and
and
if
I
lose
you
on
this
we'll
come
back
and
you
don't
need
to
all
the
details
about
this,
but
the
details
are
important
and
they're
interesting.
So
let's
just
talk
about
some
basic
the
things
about
grid
cells
highly
representing
occasion.
A
Typically,
this
is
done
with
the
rodent,
such
as
a
rat
or
Mouse,
and
here
we
have
the
rodent
walking
around
in
a
room
and
if
I
were
to
stick
a
probe
into
one
of
the
grid
cells
in
its
internal
cortex
and
say
well,
what
is
that
grid
cell
become
active?
When
does
it
fire?
You
would
see
that
it
becomes
active
at
different
locations
in
this
environment
whenever
there,
the
rodent,
is
in
one
of
those
red
spots
that
cell
becomes
active
and
when
it's
not
in
the
red
spot.
A
It's
not
it's
relatively
inactive,
and
this
occurs
matter
how
the
animal
moves
in
a
room.
That's
it
and
that's
where
the
term
grid
comes
from,
because
it's
sort
of
a
grid-like
pattern
in
the
room
where
the
wood
this
cell
becomes
active.
Now
we
know,
as
I
just
mentioned
a
moment
ago,
that
the
grid
cells
activity
is
updated
by
motor
commands,
because
you
can,
this
even
happens
in
the
dark
of
the
animal
moving
around
in
the
dark.
It
doesn't
have
to
see
anything
for
these
grid
cells
to
say,
hey
we're
in
the
new
location.
A
Now
now
this
isn't
very
useful
to
know
exactly
where
you
are
because
it
could
be
any
one
of
those
spots.
If
we
then
probe
the
next
cell
over
one
right
close
to
the
first
one,
a
different
grid
cell,
we
might
might
see
that
it
becomes
active
in
these
blue
areas,
and
so
it's
very
that
you
can
see
they're
very
similar,
spacing
and
similar
tiling
going
on
here.
It's
just
representative
one
little
further
over.
A
In
fact,
there
are
grid
cells
that
represent
every
spot
in
this
in
this
room,
but
they're
all
sort
of
the
same
tile,
so
good
cells
on
themselves
can
tell
you
something
about
where
you
are
in
the
room,
but
they
can't
represent
a
unique
location.
So
how
does
the
brain
get
around
that
the
basic
way
we
believe
and
other
people
believe
it
gets
wrong
as
as
follows?
Imagine
I
had
two
grid
cell
modules,
meaning
two
sets
of
grid
cells,
module
one
and
module
two,
and
they
differ
slightly.
A
They
might
differ
in
their
spacing
of
the
where
the
firing
fields
are
and
they
might
differ
in
their
orientation
relative
to
the
room.
Now,
if
I
wanted
to
know
where
I
am
in
the
room,
if
I
looked
at
the
cells
in
module,
1
I
might
say
well,
it
could
be
at
any
one
of
those
red
spots
because
that's
cells
active
and
they
looked
at
module.
A
And
so
now
the
animal
can
know
exactly
where
it
is
one
of
the
things
that
I
won't
explain
it
in
more
detail,
but
state
is
that
the
representation
location
is
different
than
the
kind
of
thing
you
learned
in
high
school.
You
know
it's
not
like
XY
and
Z,
and
in
this
case
the
U
location,
meaning
which
sells
a
fire
which
Grizzles
are
firing,
is
unique
to
the
position
in
the
room
and
to
the
room.
A
A
Even
though
they're
the
same
size
and
I've
marked
three
different
locations
or
labeled,
three
different
locations
in
each
room
and
room,
one
I'd,
labeled
location,
a
B
and
C
in
room
to
a
location,
D
en
F.
Remember
every
location
in
the
room
has
a
unique
representation
and,
as
the
animal
moves,
these
updates.
These
location
representations
change
notice
if
I
go
from
A
to
B
to
C
or
if
I
go
directly
from
A
to
C
I'm
always
going
to
get
to
see.
A
This
is
called
path,
integration,
so
no
matter
how
I
get
there,
you
always
going
to
get
the
same
location.
Now,
if
you
think
about
this,
since
each
of
these,
all
these
locations
are
unique
to
the
room
that
a
room
can
be
defined
as
a
location.
Space
of
all
the
possible
location
representations
in
the
world,
there's
a
unique
set
that
are
assigned
to
each
of
these
rooms,
and
and-
and
so
we
can
think
about
a
room
having
this
location
space.
A
Even
if
the
animals
never
been
to
some
corner
of
the
room,
it
still
has
a
representation
for
that
location.
Now,
what
we're
proposing
is
the
following:
the
neocortex
does
something
very
similar,
but
instead
of
the
animal,
your
body
moving
out
in
the
room,
it's
your
sensory
patches
or
your
sensory
organs,
moving
relative
to
objects
in
the
world,
so
in
the
cortex,
the
grid
cells
represent
the
location
of
a
sensor,
input
relative
thing
optical
in
this
case,
where
my
finger
is
relative
to
the
pen
or
relative
to
the
coffee
cup.
A
I've
labeled
three
locations
on
the
cup
XY
and
Z
I
really
build
four
locations
relative
to
the
Pam
and,
as
you
move
your
finger,
the
cortex
updates
its
location.
Well,
two
of
those
objects
notice
that
I
went
I
go
from
V
to
T
and
I
go
through
location.
W
W
is
not
on
the
pen,
but
it's
still
in
the
location
space
of
the
pen,
so
the
location
space
is
bigger
than
just
the
object.
The
objects
contained
in
this
location
space
and
so
the
cortex
we
need
tracking,
where
that
finger
is
as
it
moves.
A
So
every
object
in
the
in
the
world
now
has
its
own
unique
location
space,
and
this
is
the
key
to
understanding
how
the
cortex
models,
the
world
objects,
have
their
own
location
spaces
and
we
have
to
start
thinking
along
those
lines.
So
now
we
go
back
to
the
drawing
I
showed
you
earlier
and
I
said.
Well,
we
didn't
know
how
the
location
Rose
did.
The
object
was
represented.
A
We
don't
believe
that
our
grid
cell
modules
in
each
cortical
column,
I
showed
those
by
these
little
green
rectangles
here,
and
so
they
provide
the
mechanism
for
representing
the
location
and
everything
we've
learned
about
grid
cells
have
now
applies
to
the
cortex
as
well.
We
had
another
paper
which
is
just
posted
just
a
few
weeks
ago,
which
talks
about
the
mechanisms
being
how
these
grid
cells
and
in
layer,
six
and
layer
four
actually
interact.
It's
a
more
detailed
paper
subset
of
this
whole
overall
theory
and
that's
mentions
louisette
al.
A
So,
let's
review
our
proposal
so
far,
we
proposed
that
the
grid
cells
exist
in
every
cortical
column
and
they
represent
the
location.
The
input
to
the
column,
relative,
the
object
being
since
each
column
learned,
is
now
able
to
learn
complete
models
of
objects,
because
it
knows
these
location
signals
and
the
objects
in
the
world
have
their
own
unique
location
space.
So
once
we
understood
this,
there's
a
whole
series
of
other
problems
that
started
that
we've
been
puzzling
over
for
years,
it
became
clear
how
to
solve
them.
A
I'll
just
go
through
a
few
of
them
and
as
I
mentioned
here,
this
is
this:
basically,
that
finds
a
location
based
framework
for
understanding
and
in
your
projects.
So
let's
talk
about
one
of
these
things.
One
of
these
things
it's
compositional
structure.
Everything
in
the
world
is
composed
of
other
things,
so
a
door
imagine
a
door,
it
has
has
panels
and
a
shape,
but
it
also
has
a
handle
that
goes
up
and
down
or
turns
rotates.
It
also
got
a
little
hospitals
in
and
out.
A
It
also
has
hinges
and
hinges,
have
pins
and
the
screws
and
so
on.
So
everything
every
object
that
world
is
like
this
cars
are
consist
of
other
things,
and
so
the
cortex
has
to
learn.
Objects
is
composed
of
other
objects,
arranged
in
particular
ways
the
example
we're
going
to
use
again
in
our
coffee
cup.
In
this
case
we
have
the
coffee
cup
with
an
omental
logo
on
it,
the
cup
is
a
previously
learned,
object,
I
mean
I,
know,
coffee
cups
and
the
logo
is
a
previously
learned.
A
Object,
I've
seen
that
elsewhere
in
the
world
and
I'm
trying
to
learn
a
new
thing,
a
new
composition,
which
is
the
cup
with
the
logo
and
never
seen
that
combination
before
and
I
want
to
learn
it
very
quickly
and
efficiently.
I
don't
want
to
have
to
relearn
the
cup
or
relearn.
The
logo.
I
want
to
be
able
to
say
here's.
A
new
object
consist
of
something
I
already
know
and
something
else
I
already
know
bingo
it's
done.
Here's
the
new
object
and
I
want
it.
A
A
The
same
is
true
of
B
and
Y
and
scene
one.
So
those
blue
arrows
in
some
sense
represent
a
circuit
transform
a
way
of
saying
if
I
can
go
between
the
point
in
the
space
of
the
cop
and
the
points
in
space
of
logo.
I
can't
fine
wear
the
logos
on
the
copper
to
find
a
new
object,
and
so
every
point
in
the
logo
and
every
point:
a
cup
there's
a
one-to-one
correspondence
between
those
two.
So
basically,
this
one
sort
of
blue
arrow
will
sort
of
transform
that
can
be
thicker
and
represent
this
new
object.
A
And
how
can
that
be
done?
We
are
proposing
a
new
type
of
cell
to
do
this
and
we
call
it
displacement
cells
that
solve
this
problem.
It's
a
fundamental
problem.
It
has
to
be
solved
by
the
cortex
here,
I'm,
going
to
illustrate
a
little
bit
about
how
displacement
cells
work
just
to
give
you
a
flavor
for
it
now.
What
I'm,
showing
here
are
three
grid
cell
modules.
A
These
are
the
actual
imagine
these
little
rectangles,
actually
populations
of
neurons
in
your
columns
and
your
cortex
and
I'm,
showing
at
one
time
the
green
dot
represents,
which
cell
is
active.
It's
it's
axis
representing
an
active
cell
in
module
one
and
another
active
cell
in
module
two
and
another
active
cell
in
the
third.
You
know
module
an
and
let's
say
that
those
three
active
are
those
inactive
cells
represent
location,
a
on
the
cup.
That's
how
locations
are
represented
in
grid
cells,
location,
a
on
the
cup.
A
Now,
in
the
next
moment,
I'm
going
to
let's
say
I,
represent
the
same
grid
cells
can
represent
location
X
on
the
logo.
You
know,
X,
on
the
logo,
is
it's
a
different
location
space?
It's
the
same
physical
space
when
the
logos
on
the
cup,
but
because
it's
it's
representing
in
the
location
space
of
the
logo,
the
same
physical
space
is
two
different
representations.
A
and
X.
So
I've
switched
from
one
representation
at
one
moment
of
time
to
another
reputation
in
another
moment
time,
but
they're
physically,
the
same
location
in
space.
A
The
act
of
grid
cells
in
each
of
these
models
changed
and
unites
represented
by
the
blue
hours.
Here
from
the
previous
active
onto
the
new
active
one,
and
we,
if
we
can
represent
that
transition,
then
we
have
a
way
of
representing
the
entire
position
of
the
cup
to
the
logo.
So
we
just
were
proposed
that
there's
another
set
of
cells
called
displacement
cells
that
are
also
in
modules.
A
They
work
very
similar
to
grid
cells,
but
they
do
is
they
represent
the
transition
from
one
point
in
the
other
to
another
point
another
and
I'm
just
going
to
state
this
without
any
further
a
justification
that
if
I
look
at
the
act
of
displacement
cells
across
a
set
of
modules,
it
is
very
unique.
It
unique
to
the
locale
of
the
logo
on
the
cup.
At
a
particular
position
that
is,
is
even
as
small
as
20
active
cells
can
represent
the
entire
logo
in
the
entire
cup.
A
At
a
particular
position,
it's
a
very
efficient
representation
and
it
allows
us
to
go
back
and
forth
between
cups,
space
and
logo
spaces,
this
displacement
cell
representation.
So
what
we
think
is
going
on
is
that
these
displacement
cells
are
on
a
different
layer
in
each
cortical
column
in
one
of
the
cell
types
in
layer
five.
A
So
we
have
this
displacement
cell
modules
that
we
have
grid
cell
modules,
and
this
allows
each
cortical
column
to
build
complex
models
of
objects
composed
of
other
other
objects,
and
we
talked
about
how
these
interact
in
the
paper
that
we
just
released
a
few
days
ago.
Okay,
so
the
next
thing
we
want
to
talk
about
is
object.
Behaviors
and
I
mentioned
earlier.
Objects,
have
lots
of
behaviors
doors
open,
but
take
like
a
smartphone,
for
example.
It's
an
object,
but
what's
on
the
displays,
changes
continue
as
I
touch
it
and
manipulate
it.
It's
it's
changing.
A
A
So
we
use
a
simple
example
here
of
a
stapler:
a
staple
has
set
of
behaviors,
meaning
it
it
changes
its
morphology
or
its
shape,
and
it
does
various
things
I'm,
just
illustrating
one
of
them
here,
which
is
when
you
can
take
the
top
of
the
stapler
and
rotate
it
up
and
open,
now
think
about
the
stapler
as
similar
to
the
coffee
cup,
the
top
of
the
stapler.
This
could
be
considered,
like
the
logo
relative
to
the
entire
statement,
which
is
like
the
coffee
cup
and
so
the
top
of
the
space.
A
The
stapler
has
a
position
relative,
the
entire
stapler,
and
so
it
when
it's
closed,
I'll
call
that
displacement
a
and
when
it's
open,
I'll
call
it
displacement
n,
it's
literally
it's
as
if
the
coffee
cup
example
earlier
I
was
moving
the
logo,
but
in
that
case
it
was
fixed
in
this
case.
Part
of
the
part
of
the
object
moves
relative
to
the
other
part
of
the
object
and
therefore
the
displacement
changes
to
learn.
This
behavior
I
just
need
to
learn
a
sequence
of
those
displacement
cells.
A
How
they're
changing
over
time
and
then
I
will
learn
this
behavior.
We
have
previously
proposed
a
detailed
model,
how
sequence
memories
learned
in
the
quarters
that
was
a
2016
paper.
It
would
apply
directly
to
this,
so
object.
Behaviors
can
be
represented
and
learned
as
sequences
of
displacements,
and
it's
a
very
powerful
mechanism
and
it's
a
very
efficient
mechanism
for
doing
that
and
has
to
be
solved
by
the
neocortex
I'm
going
to
go
onto
one
more
topic.
That's
impacted
by
this
theory,
and
this
goes
I'm
gonna
go
back
to
the
hierarchy.
A
I
mentioned
earlier
sort
of
this
classic
view
of
how
information
people
think
how
information
flows
in
the
cortical
hierarchy
and
what
their
this
framework
suggests.
A
new
interpretation
of
that
you
still
have
the
same
sort
of
regions
connected
together
as
we
saw
before,
but
now
we're
saying
that
these
regions
are
divided
in
two
columns
and
each
column
is
learning
complete
models
of
objects.
A
So,
if
I
observe
a
coffee
cup
in
this
case
there
are
models
of
the
coffee
cup
in
the
visual
cortex
and
there's
models
of
the
coffee
cup
in
the
somatosensory,
or
touch
cortex
at
different
levels
and
in
different
and
in
different
columns.
These
are
all
running
in
parallel,
they're,
not
identical,
because
each
model
is
built
to
built
on
these
specific
inputs
to
that
column.
So,
even
in
the
visual
cortex,
though,
it
might
be
models
that
are
biased
towards
color.
A
Some
bias
towards
motion
some
are
bias
towards
shape
and
so
on,
or
some
would
be
different
parts
of
the
retina,
but
they're
all
modeling
the
same
thing,
and
so
we
call
this
a
thousand
brains,
theory
of
intelligence.
And
now,
if
you
realize
once
you
have
all
these
models
running
all
over
the
place
in
the
cortex,
that
connections
that
are
not
hierarchical
makes
sense.
Going
back
to
what
I
mentioned
earlier.
A
If
two
columns,
any
two
columns
are
modeling
the
same
object,
then
there's
a
make
sense
have
long-range
connections
between
them
because
they
can
vote
so
even
primary
sensory,
visual
cortex
can
send
signals
over
to
primary
somatosensory
cortex
saying:
look
we're
both
observing
the
same
thing
here
and
let's
vote
on
what
that
is.
This
solves
what
I
call
the
the
sensor?
Well,
many
people
call
the
sensor
fusion
problem.
A
The
centrifuge
problem
is
as
follows:
people
say
look
the
inputs
coming
in
from
your
sensors
and
they
go
into
the
brain
and
they're
divided
they're
all
over
the
place.
There's
different
parts
of
the
visual
regions
of
approach
somatosensory.
Where
do
they
get
combined
into
a
single
model,
a
single
perception
of
the
world
and
what
the
thousand
brains
theory
of
intelligence
says.
There
isn't
a
single
model
of
the
world.
There
are
thousands
of
them,
and
but
they
crystallized
together
through
this
voting
across
long-range
projections,
and
so
there
isn't
a
views.
A
It's
a
distributed
sensor,
fusion
solution:
you
have
distributed
models
that
all
vote
together
to
create
a
common
perception,
but
the
models
are
separate.
It's
a
very
different
way
of
looking
about
the
brain
than
most
people
think
about
it
today.
Now,
let's
see
how
we
did
Francis
Crick
argued
that
we
need
new
ways
of
thinking
about
the
brains
we
need
a
broad
framework
or
which
interpret
experimental
results,
I'm
proposing
new
ways
of
thinking
about
the
brain
today
and
a
broad
framer.
What
are
the
two
major
components
of
this?
A
You
look
at
the
level
of
the
cortical
column
and
we
look
at
the
level
of
cortex
as
a
whole.
So
this
is
a
review
at
the
cortical
column
were
arguing.
They
are
far
far
more
powerful
than
currently
believed
that
every
column
is
learning
complex
models
of
objects
and
their
compositions,
so
confident
columns,
learn
complete
models
of
objects,
including
subs
objects
and
behaviors,
and
that
grid
cells
and
displacement
cells
are
part
of
this
model
and
they
define
location
spaces
for
objects
and
their
relative
positions.
A
So
this
is
the
way
of
thinking
about
how
the
cortex
learns
and
manipulate
objects
in
the
world.
Second
thing
is
the
New
York
Electric
as
a
whole.
We've
composed,
we
proposed
that
New
York
cactus
is
composed
of
thousands
of
models.
Those
models
differ
based
on
their
inputs,
which
is
what
mal
castle
said
back
a
long
time
ago.
A
He
said,
regions
were
were
basically
doing
the
same
thing
and
he
said
even
the
common
algorithms
in
your
cortex
of
the
column
and
we're
saying
he's
right,
the
commas
just
it's
far
more
powerful
than
anyone
thought
and
that
these
long-range
connections
allow
the
columns
for
votes.
Also,
we
have
a
different
interpretation
of
the
cortical
hierarchy.
It's
not
that
there
isn't
hierarchical
processing,
but
the
most
of
the
connections
in
the
your
cortex
do
not
fit
in
that
hierarchy.
They
fit
in
this
sort
of
voting.
This
scheme
that
goes
across
modalities
across
columns
all
right.
A
A
Reentered
structures,
so
I
could
have
an
object
composed
of
an
object
which
is
composed
of
the
same
object
and
so
on,
and
you
can-
and
these
are
the
kind
of
structures
that
linguists
use
and
we
think
about
language,
nested
structures
and
reentrant,
see
and
so
on.
All
that
can
be
supported
in
this
model.
So
there's
a
long
way
to
go
here
to
understand
this,
but
the
evidence.
It's
just
very
strongly
that
everything
we
think
about
and
high-end
sort
of
high-level
thinking
can
be
understood
in
this
basic
framework.
A
There's
a
lot
of
experimental
support
for
this
framework
that
I
proposed
today
and
I'm
not
going
to
go
through
all
the
details
here,
I'm
just
going
to
give
you
sort
of
the
basic
categories.
There
is
evidence
that
there
is
object,
centric
locations,
signal
available
in
central
regions.
This
is
a
surprising,
but
people
found
that
certain
cells
in
the
primary
centric
cortex,
whether
it's
in
v1
or
v2,
seem
to
know
where
they
are
on
objects
in
the
world.
Then
they
fire
differentially,
there's
evidence
now
that
there
are
actual
grid
cells
in
the
neocortex.
A
There's
evidence
that
prediction
in
the
central
regions
is
based
on
movement,
sensory,
motor
prediction
and
there's
even
evidence
that
the
your
cortex
evolved
from
all
the
parts
of
the
brain,
such
as
the
hippocampus
and
in
Toronto
cortex.
So
this
complex
machinery
that
might
have
been
evolved
first
for
understanding
where
we
are
in
the
world
now
is
being
used
in
mammals
and
in
humans
who
not
model
the
world
as
a
whole.
All
right
now
sits
of
the
last
part
of
my
talk,
which
is
the
implications
this
will
be
fairly
short.
A
I
won't
go
into
too
much
detail
clear,
there's
a
lot
of
implications
for
neuroscience.
There's
a
tremendous
amount
of
neuroscience
data
which
can
is
today
is
unassimilated
into
any
kind
of
theoretical
framework,
and
this
all
of
a
sudden,
as
I
just
mentioned
in
the
previous
slide,
there's
a
lot
of
experiment
of
the
data
which
didn't
really
kind
of
make
sense,
which
kind
of
mal,
but
it
does
make
sense
in
the
neuroscience
world.
So
it's
my
hope
that
these
these
sort
of
top-down
theories
really
will
influence
our
understanding
of
experimental
neuroscience.
A
A
They
made
it
really
clear
to
me
that
you
know
we
teach
kids
how
to
learn,
but
we
really
have
no
idea
of
what's
going
on
in
their
brains
and
if
we
understood
how
what
what
knowledge,
how
was
really
representative
brain,
we
might
be
available
to
do
better
jobs
of
teaching
people
and
I
was
like
struck
by
that
and
I
think
all
of
a
sudden.
Now
we
have
a
ability
to
start
thinking
about
things
like
this,
also
how
we
form
beliefs
and
when
do
we
form
false
beliefs,
is
a
big
problem
for
humans.
A
We
also
might
be
able
to
start
understanding
what
the
limits
the
human
intelligence
are
right
today
we
don't
really
know
what
those
are
and
finally
have.
Some
of
the
diseases
of
the
mind
could
be
a
perhaps
understood,
theoretical
foundation
for
them.
How
does
a
healthy
brain
do
something
would
help
us
understand
how
it
might
be
diseased
I
think
some
of
the
biggest
impacts
are
going
to
be
on
artificial
intelligence
and
robotics.
Now
let
me
just
step
up
and
talk
about
a
few
things
here.
A
A
lot
of
people
talk
about
artificial
general
intelligence,
a
true
AI
and
so
on,
and
most
people
today
believe
that
the
current
name
ie
models
are
very
limited
and
we
need
something
else,
I'm
going
to
argue
the
following.
You
know.
Ultimately,
we
go
out
in
the
future
and
we
talk
about
intelligent
machines.
They're
gonna
have
some
of
the
things
I've
talked
about
here.
They're
going
to
have
they're
gonna
be
based
on
distributed
models
which
we
call
the
thousand
brains,
theory
of
intelligence.
A
It's
not
one
big
monolithic
model
which
they're
building
today,
but
thousands
of
models
distributed
in
each
of
those
models
is
going
to
be
built
using
object,
centric
locations
and
location
spaces.
I,
don't
think
you
can
get
away
from
that.
You
cannot
learn
the
structure
of
the
world
without
knowing
the
locations
of
things
in
the
world,
then
today's
neural
networks
don't
do
that.
They're
gonna
have
to
be
designed
to
understand
compositional
structure.
How
objects
are
composed
of
objects
today
they
don't
do
that
and
they're
going
to
be
based
on
learning
through
movement.
A
We
learn
through
a
movement,
that's
how
we
have
to
move
my
finger
over
to
learn
what
it
is,
and
today's
AI
doesn't
do
them
this.
It
says,
of
course,
then
we
have
to
have
embodiment
you.
You
can't
have
an
AI
system
that
doesn't
have
the
ability
to
move
its
sensors,
and
so
AI
and
robotics
are
really
not
separate
problems.
They
are
really
the
same
problem
and
today
we
view
them
as
separate
problems
that
that's
not
gonna,
be
true
in
the
future.
A
There
are
some
things
that
two
AI
does
not
have
to
do
the
same
as
a
human.
Obviously,
we
can
make
brains
that
are
faster
and
larger
or
even
smaller.
We
can
use
very
different
types
of
sensors.
We
don't
have
to
use
of
sensors
that
humans
have
and
we
can
have
new
types
of
embodiments.
We
don't
have
to
think
about
these
as
being
like
humanoid,
robots
and
including
virtual
environments.
A
Virtual
embodiments
excuse
me,
so
it's
possible
have
an
AI
that
its
movements
are
moving
through
the
internet
or
you
know
following
links
and
so
on.
Basically,
it's
basically
as
I'm
moving
my
sensor
to
send
something
else
and
it
doesn't
have
to
be
physical.
It
could
be
virtual.
Okay.
Having
said
that,
what
are
some
of
the
big
impacts,
I,
think
going
forward
here
and
I
think
that's
gonna
be
a
huge
impact
on
humanity
in
the
not-too-distant
future.
First
of
all,
we
can
build
purpose-built
brains.
We
forget
what
we
could
build.
A
The
mountain
brain
for
mathematics
or
one
for
physics.
Einstein's
brain
was
very
much
like
you're
an
eyes
your
mind.
It
was
this
slightly
different
and
maybe
was
a
little
bit
more
this
a
little
less
of
that
we
don't
really
know.
But
the
point
is
it
wasn't
built
on
different
principles,
and
so
we
should
be
able
to
build
brains
that
are
really
tuned
for
certain
types
of
problems.
They
never
get
tired
and
they're
really
good
out.
We
can
build
virtual
brains,
as
I
mention
from,
for
example,
cybersecurity,
they're
roaming,
the
internet.
A
Looking
for
things
and
trying
to
you
know
solve
problems,
and
then
there's
the
the
real
big
impact
I
think
on
humanity's
it
will
be
what
I
call
real
robotics.
You
know
today's
robotics
are
really
in
the
dark
ages,
we're
sort
of
a
stone
Age's
with
robotics
that
you
know
fluid
smart
systems
that
act
independently
and
no
commutes
for
industry,
but
I
think
also
it's
gonna
be
a
sort
of
larger
human
human
effort.
A
Here,
there's
a
lot
of
people
today
who
wanted
you
know,
live
on
Mars
or
you
know
populated
planet
or
explore
space
I,
don't
think
we
can
achieve
this
without
having
really
smart
robots.
Yeah
engineer,
scientist
things
on
the
wrong:
go
autonomously
out
there
and
prepares
some
place
on
Mars
for
us
to
live.
It's
just
not
gonna
happen
until
we
do
that,
that's
my
opinion,
and
this
will
allow
us
to
really
sort
of
branch
out
of
our
earth.
A
In
some
sense,
I
should
mention
here
that
all
the
resources
that
I've
talked
about
you
can
find
on
dementia,
calm
the
papers
and
there's
even
a
there's,
a
companion
paper,
which
is
a
subtle,
a
person's
interpretation
of
all
the
stuff
I
just
talked
about,
so
you
can
find
that
there
and
videos
and
other
great
resources,
I'm
gonna
end
now,
with
a
few
of
just
personal
good
comments.
I
just
wanted
to
come
back
to
a
couple
things
I
talked
about
one.
Is
this
whole
theory
this
framework
in
hindsight
it
seems
like
it
should
have
been
obvious.
A
Why
I
say
that
you
know
as
you
as
we
look
out
upon
the
world.
Everything
has
I,
perceive
you
or
people
or
doors
or
objects,
and
out
here
they
all
have
some
location
relative
to
me.
They're
out
there
I
do
not
perceive
them
on
my
retina,
where
I,
where
the
light
enters
my
head,
I,
don't
perceive
them
inside
of
my
head,
I
just
seen
him
out
there
and,
as
I
said
earlier,
everything
you
perceive
is
basically
based
on
a
set
of
neuron
firings.
So
for
me
to
perceive
something
a
distance.
A
There
have
to
be
neurons
that
represent
that
thing
at
a
distance
and
when
I
see
two
objects,
I
immediately
see
the
relationship
between
them.
I
know
how
far
apart
they
are
that
that
has
to
be
represented
by
neurons
and
that's
what
we
argue
is
that
the
displacement
cells,
so
these
things
it's
in
hindsight.
A
It's
obviously
these
representations,
our
entire
perception
of
the
world,
is
based
on
locations
and
relative
positions
of
things,
and
so
this
this
is
the
basically
the
language
in
them
the
data
types
of
the
new,
your
cortex,
it
should
have
been
obvious,
but
it
wasn't
not
off
of
this
second
thing:
I
want
to
go
back.
To
is
what
I
just
mentioned
a
moment
ago,
is
how,
at
the
beginning
of
talk,
how
important
understanding
how
the
human
brain
works
is.
This
is,
in
my
mind,
the
most
important
scientific
endeavor
of
all
time.
A
It
is
I
think
it's
actually
gonna
be
essential
to
understand
how
our
brains
work
for
the
long-term
survival
of
our
species,
and
this
is
something
we
should
all
be
concerned
about,
and
we
should
all
be
working
hard
too
to
understand
and-
and
we
should
be
saying
and
think
about
the
long-term
implications.
Good
and
bad
about
these
things.
A
I
believe
that
we've
made
some
significant
progress
in
this
direction,
and
that's
I
talked
about
that
in
this
talk
and
it's
in
the
papers
and
I
encourage
you
to
go
if
you're
interested
in
this
stuff
and
many
people
are
to
try
to
understand
this
material
I
think
it's
important
for
us
as
a
species.
All
right
I
want
to
end
with
just
a
picture
of
the
dementia
team.
Here
is
all
the
people
who
working
at
dementia
this
summer
on
the
left
there.
A
The
my
five
collaborators
on
this
particular
research
to
Pattaya
mada
I,
want
to
call
out,
because
he's
been
my
long-term
partner
and
all
this
stuff
and
helped
me
write
this
paper.
Marcus
Lewis
was
the
guy
came
up
with
the
displacement
cell
idea,
so
kudos
to
Marcus
on
that
and
with
that
I'm
done.
Thank
you
for
your
time
and
listening
to
this.