►
Description
This video introduces a new concept in HTM theory about sensorimotor inference with multiple layers in one region of cortex.
You should have some background in HTM before understanding these videos. See http://numenta.org/htm-school/ for more videos explaining basic HTM theory.
Music: "Holy Roller" by YACHT
(used with permission from Free Music Archive)
A
B
A
For
joining
us,
I'm
Matt
Taylor
from
defenses
I'm,
Jeff
Hawkins
is
fundamenta,
desperate
friend
and
co-founder
of
lavenza.
So
we're
here
to
talk
about
hierarchical
deferral,
memory
and
especially
emphasis
on
an
extended
theory
on
student
hierarchical
memory
that
include
sensory
motor
integration.
I
think
a
good
place
to
start
with
that.
If
you
have,
if
you
don't
know
what
how
HTM
works
or
HTM
Siri,
you
can
watch
these
videos
that
explain
it
from
the
ground
up
and
then
from
there.
You
probably
have
a
good
idea.
What
we're
going
to
be
talking
about
this
video
so
just.
B
An
evolving
theory
of
the
neocortex
us
what
it
is,
and
so
at
any
point
in
time
can
be
extended
in
that
doesn't
replace.
What's
there,
it's
just
we're
having
more
pieces
to
it.
So
we've
been
going
through
that
process
in
this
last
year
about
just
about
a
year
ago,
we
had
a
significant
advance,
which
you're
talking
about
yes
and
I'm,
going
to
do
that.
B
A
B
So
the
exact
you'll
relate
various
pending,
but
we
asked
and
how
you're
counting
so
I
there's
always
been
liberty
in
this,
but
I'm
going
to
I'm
going
to
talk
about
business
it
always
it
sort
of
common
layer
for
and
as
others
layer.
Three.
If
people,
often
this
hayley23
I'm,
going
to
actually
divide
that
into
lower
layer,
three
3
B
3,
a
and
two
and
in
the
past,
when
we,
if
you're
familiar
I
work
up
to
recently,
we
came
up
with
a
model
for
how
one
of
these
Legos
learned
sequence,
J,
true
temple
memory.
B
B
Play
someplace
else:
okay,
when
I
come,
plays
out
in
more
detail
the
model
we've
done
so
far,
if
you
think
about
it
as
ATF
secret,
summary,
I
believe
it's
actually
going
on
in
Leonard
rebate
and
the
reason
for
that
is
not
like
the
agreement.
There
is
that
you
have
influence
coming
from
your
centers,
so
that
could
be
your
eyes
and
ears,
and
so
we
have
the
sensory
input
being
the
brain.
It
actually
goes
through.
B
Something
comes
down
on
on
the
way
to
its
the
cortex,
but
that
point-
and
these
axons
is
a
little
axon
from
there
on
goes
up
and
it
makes
it
connections
on
me
cells.
These
are
some
known,
others
drove
one,
but
when
we
model
temporal
memory
we
have
a
spatial
and
temporal
memory.
That's
all
going
on
many
columns
in
one
layer,
so
that's
pretty
much
it
and
and
these
lower
layers,
three
cells
actually
get
these
connections
directly
from
it,
and
then
they
make
a
lot
of
horizontal
connections
within.
B
The
memory
of
sequences
comes
from
so,
if
you're
difficult,
those
are
the
distal
connections,
so
the
green
is
the
sensory
input
goes
on
to
the
proximal
synapses
and
then
the
horizontal
connections
are
the
discipline.
So
when
we
model
temple
memory,
it's
really
just
one
layer
cell,
the
inputs
coming
out
of
the
proximal
decides
and
all
the
cells
connect
to
each
other
and
that
produces
sequence
map.
So
that's
tomorrow,
we've
done.
B
B
Yeah
four
years
ago,
and
it
kind
of
beaten
the
pulp
out
of
this
thing,
because
we
would
characterize
that
we
publish
it,
we
we've
done
all
kinds
of
things
and
people
mifflin
it
and
hardware
so,
and
this
is
a
well
understood
these
conceptually
and
mathematically
think
like
now.
What
was
this
all
about?
If
you
think
about
the
brain-
and
you
think
you
this
is
your
of
your
cortex-
we
just
some
or
one
region
here,
but
all
doing
the
same
thing.
B
You've
got
these
inputs
coming
in
from
your
senses
and
they're
changing
right,
and
one
of
the
things
we
know
is
that
the
cortex
makes
a
predictive
model
of
the
world.
It
learns
the
structure
below
and
in
its
predictive.
It's
always
trying
to
predict.
What's
going
to
happen
next
I
mean
that
was
this
core
pieces
along
intelligence
right
until
this
temple
memory
is
a
predictive
model
of
sequences.
It
says:
okay,
when
the
path
has
changed,
it's
so
predictable
way.
I
could
predict.
B
I
can
model
it
on
one
of
the
sequences
I
make
predictions
about
it,
but
the
there's
another
way
that
the
inputs
can
change
and
that's
when
you
move
your
body
so
in
terms
of
axes
meant,
the
vast
majority
of
the
changes
that
are
coming
into
your
brain
is
because
you
will
I'm
moving
my
head
constantly
here.
I'm
moving
my
eyes
three
to
five
times
a
second
I'm
moving
my
fingers
and
touching
things.
My
body.
B
Yes,
right,
you
could
say
yes,
it
is
where
I
could
sit
here
and
close
my
eyes.
Would
you
speak
that
sensory
input
this
stuff,
because
of
my
own,
but
hearing
myself
speak
right
now,
is
because
my
own
behavior
yeah,
so
that
is
the
vast
majority
of
the
world-
is
changing
on
your
senses,
because
you
move,
and
this
turns
out
to
be
the
primary
way
you
learn
about
the
structural
role.
You
have
to
combine
your
movements
with
the
changes
of
sensory
input,
and
this
is
basically
all
the
very
hands
work
on
right.
B
This
concept
was
known
back
to
my
lady
in
Harvard.
Town
Hall
talked
about
it
because
he
realized
that
when
you,
hunger,
I
suppose
the
input,
your
brain
changes
and
somehow
you
have
to
combine
the
and
you're
not
aware
most
of
time
you
just
not
even
where
your
eyes
are
moving,
yes,
the
inputs
running
away,
and
so
why
does
the
world
seem
stable
when
the
inputs
are
changing
and
the
only
way
you
can
explain
that
is
that
the
brain
has
to
come
fast,
wrong.
B
Alleys
understand
what
movements
are
being
occurring
at
what
movement
you're
making
and
using
as
part
of
the
inference
product,
perhaps
of
the
exact
angle.
Yes,
most
people
think
a
lot
of
people
think
that
oh
okay,
the
brain
is
trying
to
remove
the
influence
of
movement.
But
that's
not
true
at
all.
The
brain
is
actually
taking
advantages
in
movement
to
learn
how
the
world
works
and
what
we've
done
last
year
is
we
figure
out
essentially
the
mechanism
equivalent
mechanism
for
temporal
memory,
but
the
mechanism
by
how
the
brain
uses
movement
to
make
a
predictive
olive
oil.
B
A
A
B
We
had
tried
to
just
few
years
ago
actually,
and
we
didn't
we
had
some
problem.
The
big
insight
we
had
last
year
is
that
you
took
the
exact
same
model
that
we
have
the
temporal
memory
same
columns
they
should
pool
or
at
all,
stop
and
instead
of
just
alive,
they
cannot
sell
to
connect
via
the
distal
dendrites
between
themselves
within
matter.
Within
that
reason,
if
you.
B
A
B
You
have
finished
art
about
something
which
I
wouldn't
normally
do
if
I.
Normally
we
talk
about
the
new
Siri
I
was
struggling
to
do
that
you
go
through
and
when
we're
going
to
go
into
that.
But
but
if
we
want
to
be
really
precise,
what's
going
on
here,
is
that
I'm
a
drug
little
picture
here,
usually
a
for
liquor
stores,
of
course
very
well
known
as
the
input
layer
to
the
fourth
pick,
and
so
the
same
actions
between
guises
that.
B
Connections
of
here
actually
make
connections
myself
do
too.
Instead,
what
we
believe
into
Georgia
are
this
is
not
a
positive.
We
believe
is
going
on
is
that
layer,
four
in
lower
layer
tree
are
basically
both
doing
station
pulling
they're
both
doing
the
same
in
cotton,
but
the
layer,
four
cells
are
actually
learning
the
sensorimotor
inference
and
the
layer,
3
B
cells
are
learning
the
high
order.
Sequence
memory
will
do
the
same
layer
and
this
wealth
within
these
two
layers
yeah
just
two
of
this
yeah.
B
B
And
I
want
to
be
here,
but
I'm,
so
much
lower
layer.
Three,
the
part,
that's
getting
the
sensory
input.
Okay,
the
other
layer
tree-
does
not
get
that
centric.
But
so
you
know
it's
kind
of
unfortunate.
They
call
them
layer
three,
because
it's
really
two
thin
layers
and,
as
you
said,
Nick
terminology
anyway,
so
the.
B
We
had
a
true
temporal
memory,
put
a
new
signal
into
it
and
run
it
the
same
way
as
before
and
you'll
get
statistical,
inference
we'll
talk
about
how
that
works
in
reality,
I
think.
What's
going
on
the
cortex,
these
things
were
occurring
together
so
right
on
top
of
water
on
there
so
part
of
the
same
basic
they're,
basically,
the
same
spatial
Pooler
and
music
same
mini-com,
and
until
let
us
whether
a
slice
of
cortex
is
getting
an
input
from
the
century
organs
and
then
inputs
changing
it
tries
to
model
it
both
ways.
It
says.
B
B
B
Really
just
what
has
different
input,
it's
the
same
algorithm
actually
but
you'll,
see
in
a
minute,
so
I
mean
I,
don't
want
to
focus
on
that.
That's
a
little
bit
more
speculative
there
and
what
we've
been
working
on,
what
we
have
been
working
on,
essentially
modeling
layer,
four
and
the
upper
layer?
Three
and
true,
and
and
that's
those
two
are
required
to
understand,
sent
remember
so
this
is
sort
of
in
the
context
of
what
we've
done
recently.
B
If
your
HCM
follower
new
pic
follower,
you
know
what
we're
going
to
talk
about
here
is
not
a
new
theory.
It's
not
a
replacement
for
we've
done
it.
It's
actually
a
direct
extension
of
what
we've
done,
but
in
the
papers
were
writing
out.
We're
actually
not
going
to
talk
about
the
temple
memory,
part
of
it
we're
just
starting
with
the
sensorimotor
inference
part
of
it.
So
what
we're?
What
we
are!
B
What
we
are
we're
currently
in
the
process
is:
writing
the
research
writing
papers,
we're
talking
about
essentially
a
layer,
four
and
another
layer
up
here
who
call
they
will
say
later:
upper
layer,
2
and
layer
3
days
you
want
and
two
layers
where
you
know
the
input
comes
into
this
layer.
We
have
this
other
signal.
What
you're
going
to
talk
about
this
later
projection
of
this
layer.
This
plays
project
back
this
way
project
is
when
it
dislike,
and
so
we
have
this
two
layer
model
amount
of
the
sensorimotor
interrupts
and.
A
B
I
mean
literally,
we
did
not
write
a
new
code
go
out,
be
just
be
big
and
just
put
another
signal
into
the
into
the
existing
simple
memory
code.
We
did
have
to
create
this
pool
and
layer
up
here,
which
so
created
a
second
layer
but
again
using
a
Jim,
neurons
and
we've
had
to
build
the
models,
a
two-layer
circuit
to
do
this
right,
but
it's
a
very,
very
powerful
system.
When
we
do
that,
and
so.
B
A
B
B
Well,
I
want
to
tell
you
what
it
is.
So
the
term
is
this
determines
used
a
lot
neuroscience,
but
not
you
know,
I,
don't
push
it.
It's
called
an
allosteric
representation
and
we
call
the
authentic
representation
of
location.
Otto
Kendrick
just
means
it's
means
other.
It's
the
opposite
of
egocentric,
so
egocentric
means
something
relative
to
your
body.
I
was
tempted
me
something
relative
to
something
else.
Some
other
thing
because.
B
Yeah,
so
here
is
we
use
coffee
cups
a
lot,
so
we're
going
to
do
that
nurse
that
stuck
here,
yeah,
here's,
my
reusable
coffee
cup
and
I
can
I
can
have
a
locations,
coffee
cup
relatives
in
my
body
that
we
go
sensor.
Location
right
if
I
wanted
to
know
where
this
hole
is
relative
to
the
rest,
the
litter,
where
this
particle
just
bottle
works
any
feature
Europe
roses
to
the
copy
house.
Now
being
the
reference
trim,
the
carbon
right
so
so
trick.
B
A
B
Yeah,
well,
it's
a
pivot.
If
I
have
a
model
of
a
coffee
cup,
it
is
on
I.
Do
I.
Have
this
model
come
because
every
time
I
touch
it,
my
brain
is
predicting
what
it's
going
to
feel
like
I've
used
itself
everyday
I.
Understand
it
very
well.
That
model
has
to
be
in
Allison
framework.
It
cannot
be
in
an
egocentric
chamber
because
it
moves
every
time.
B
So
knowing
where
the
feature
as
well
just
my
notebook
very
good
because
of
your
differently
overhearing
there
so
in
hindsight,
is
clear
that
we
must,
under
the
world
in
absentia,
coordinate
in
the
in
the
parts
of
the
brain
where
they've
studied
is
the
most.
It's
usually
related
to
some
room
like
a
rat
will
walk
through
a
room,
and
the
rat
knows
where
it
is
in
the
room.
It's
relative
to
the
room.
These
are
called
the
play
cells
me
so.
B
And
I
know
where
we
are
relative
to
this
room
in
the
kitchen
and
I,
usually
do
yeah
I
usually
go
bigger
pocket,
so
the
key
insight
we
had
here
was
that
allocentric
location.
This
is
a
consistent,
an
SDR,
the
represented
location
on
some
physical,
something
out.
There
is
a
used
pervasively
throughout
the
neocortex.
It's
everywhere,
it's
not
just
in
some
places
of
rain
and
it's
a
key
ingredient
coming
into
even
primary
sensory
regions.
So
what
this
means
is
when
I,
if
I,
trust
this
cup,
my
finger
is
feeling
some
sensation.
B
A
B
Yes
at
first,
you
may
not
know
where
you
are
on
topic,
so
that's
a
challenge.
We
have
to
do
maybe
every
dressing,
but
once
you
know
where
you're
arming
the
object,
then
you
essentially
to
learn
what
a
coffee
cup
is
or
a
pan
or
anything
else.
It's
literally.
It
is
literally
saying
this
feature
at
this
location.
This
feature
at
this
location.
This
feature
at
this
location.
A
B
Well,
that's
because
that's
the
entrance
part
right,
there's
like
so
I'll-
make
a
subtle
connection
here,
if
you
think
about
that,
for
those
who
really
understand
the
temple
memory
very
well,
this
state
of
the
cell,
at
any
point
in
time
when
you're
in
a
sequence,
is
a
high
order,
state
very
sparse.
That
state
represents
both
the
sequence
and
where
you
are
in
the
sequence
I'm
at
the
third
note
of
beethoven's
bit
or
something
like
that
right
now,
the
constant.
B
A
B
B
If
you
learn
a
model
of
how
the
world
looks
a
field,
what
a
structure
is
in
reality
independent
of
what
we
resolved
into
my
body,
we
build
a
predictive
model,
the
coffee
table
or
predictive
model
of
a
camera
or
whatever
all
the
stuff
is
stored,
your
head,
so
you
know
the
core
texture
of
the
deal,
so
that
was
the
big
insight
was
that
allocentric
locations
are
available
even
in
primary
sensory
cortex.
We
now
know
now
that
we
know
what
to
look
for.