►
From YouTube: HTM Retrospective
Description
Subutai Ahmad talks about the history of HTM algorithms at Numenta. This video was recorded at the 2015 HTM Challenge.
Did you realize that the development of HTM algorithms at Numenta has been going on for over 10 years? Subutai (who has seen it all) will step you through the sequence of HTM developments, from our very first demos, algorithms and products, to our current research on cortical algorithms.
A
Working
so
this
this
kind
of
it
there's
a
pretty
informal
talk.
It's
something
that
Matt
asked
me
to
do.
It
actually
came
from
a
talk.
I
did
at
Numenta
a
few
months
ago
and
the
context
for
that
was,
we
were
having
Donna
our
CEO.
Does
these
annual
big
budget
planning
off
sites
where
the
whole
company
gets
together
and
we
talk
about
our
plans
and
our
budget
and
stuff
well
I
had
just
the
previous
week
already
talked
about
the
research
plans
within
the
company
and
the
research
budget
is
really
boring.
There's
only
a
couple
of
us.
A
This
really
not
much
to
it.
So
I
was
really
struggling
with
what
to
talk
about
at
our
company
off-site,
and
so
you
know
so.
I
asked
here's.
The
conversation
I
asked
Donna.
Well,
what
should
I
talk
about?
Well,
she
said
just
make
it
fun
like
okay,
but
I
told
her
well,
the
research
budget
is
really
boring.
I
could
do
that
in
30
seconds
and
she
said
well
make
it
30
minutes.
I
was
like
okay.
A
What
am
I
going
to
talk
about
so
I
decided
after
some
deliberation
to
do
something
completely
different
and
there
I
kind
of
realized
that
Numenta
is
now
10
years
old,
so
was
founded
in
2005
and
it's
2015.
Now
it's
a
little
over
10
years
and
there's
a
lot.
That's
happened
within
that
time
frame
and
from
an
algorithms
perspective.
A
Jeff
and
I
are
really
the
only
ones
who
really
been
working
on
the
algorithms
throughout
that
entire
10
years
and
Jeff
is
not
a
good
historian.
He's
got
a
many
things,
but
he's
not
a
good
historian.
That
kind
of
falls
on
me
and
so
what
I
thought
I'd
do
is
kind
of
give
a
history
off
htm'
the
algorithms
perspective
over
the
last
10
years
from
my
kind
of
personal
point
of
view,
just
looking
at
everything
in
there.
A
So
a
little
bit
of
prehistory
just
before
Numenta
founded,
you
all
know,
unintelligence
Jeff
wrote
that
and
was
published
in
2004,
and
in
that
in
that
book,
Jeff
outlined
a
bunch
of
the
key
properties
that
we
think
are
embodied
in
the
cortex,
and
these
are
key
to
understanding
intelligence.
These
are
principles
that
are
embodied
in
the
cortex
that
we
want
to
understand,
computationally
and
then
build
systems
based
on
that.
So
many
of
you
here
have
already
read
the
book,
but
I'll
just
list
them.
A
It's
a
common
cortical
algorithm
the
notion
that
every
part
of
cortex
is
doing
the
same
basic
algorithm,
the
notions
of
hierarchy
and
invariance,
very
critical
for
solving
large
problems,
sequences
and
prediction
notion
of
continuous
learning
feedback,
the
importance
of
behavior
and
sensory
motor
stuff
attention
and
the
laminar
structure
of
the
cortex.
So
these
were
all
kind
of
discussed
in
the
book
as
key
aspects
of
the
cortex
that
we
want
to
understand.
So
this
is
kind
of
how
we
measure
ourselves
is
against
these
principles.
A
A
Ok,
so
Jeff,
founded
Numenta
in
2005
and
the
state
of
Gaul
has
been
very
consistent.
That's
over
the
last
10
years
is
to
discover
the
operating
principles
of
the
cortex
and
then
to
build
intelligent
machines
based
on
those
principles.
So
it's
kind
of
a
split
mission.
We
have
a
scientific
component
to
what
we're
doing,
which
is
really
trying
to
understand
the
brain
and
a
more
pragmatic
component,
computer
science
kind
of
component,
which
is
taking
those
principles
and
really
building
something
and
to
this
day
I,
don't
think,
there's
any
other
organization.
A
That's
still
trying
to
do
both
of
these
things.
I
just
went
to
a
neuroscience
conference
last
week
and
a
lot
of
they're
doing
trying
to
understand
number
one,
but
not
number
two
or
you
have
people
are
trying
to
do
number
two,
but
not
number
one,
but
very
few
people
are
trying
to
do
both
and
it's
a
very
interesting
kind
of
position.
We
feel
I
think
we
think
it's
a
really
important
position.
Those
reasons.
A
A
Okay,
so
most
of
what
my
talk
is
actually
focused
on
the
first
part,
the
the
first
generation
algorithm
is
because
a
lot
of
that
has
been
lost
over
time.
We
actually
did
some
really
interesting
things
there
and,
and
then
I'll
briefly
talk
about
the
last
two,
because
that's
a
lot
more
familiar
to
people
now,
I
think,
okay,
so
the
first
generation
called
the
Zeta
one
algorithms
I'm
gonna
stand
here,
I
think
that's
okay.
A
So
we
would
look
at
patterns
in
the
in
the
input
data
and
look
to
see
what
patterns
would
cook
her
in
time
and
those
would
become
temporal
groups.
So
it's
very
similar
in
spirit
to
some
of
the
pooling
stuff.
That's
done
like
max
pooling
and
so
on.
If
you're
familiar
with
that,
but
these
groups
were
actually
learned
from
the
data
they
were
not
hard-coded
like
the
max
pool
stuff
is.
A
This
is
an
example
showing
how
some
of
those
groups
would
be
formed.
So
you
can
see
on
the
left
is
an
image
and
we're
sweeping
a
little.
You
know
rectangle
throughout
the
image
in
the
middle,
you
see
the
actual
what's
under
that
rectangle.
So
these
are
the
pixels
of
the
image
and
on
the
right,
you
would
what
we
build
up,
what
we
call
a
time
adjacency
matrix.
This
is
to
see
what
patterns
follow,
another
pattern
in
time,
okay
and
so,
where
it's
white.
That
means
that
a
particular
pattern
follows
another
pattern.
A
So
this
goes
on
for
a
while,
as
it
sees
more
and
more
patterns,
the
matrix
gets
bigger
and
bigger,
and
then
eventually
you
get
a
nice
kind
of
block
diagonal
structure
and
from
there
you
can
build
groups,
groups
of
patterns,
and
you
can
do
clustering
on
this
and
you
basically
find
that
things
like
vertical
edges
would
cluster
together
into
one
group,
horizontal
lines
would
all
cluster
together
in
another
group.
Maybe
diagonal
lines
would
cluster,
so
you
get
these
kind
of
common
clusterings
of
input
patterns.
A
B
A
A
So
what
we
would
do
is
we
would
mimic
what
happens
with
our
eye,
which
is
you
know,
moving
around
saccadic
movements
exactly
yeah,
so
so,
but
also
you
know
in
in
real
life.
Pictures
do
have
time.
You
know.
Even
if
you're
staying
staring
at
one
place,
you
know
the
objects
moving
around
or
your
bodies
moving
around
so
and
so
there's
a
natural
contiguity
to
the
pixels
and
that
contiguity
is
actually
important.
A
We
then
you
know
and
I
again,
I'm
not
really
going
into
the
algorithms
in
depth
here,
and
this
is
just
to
give
you
a
high
level
picture
of
what
we
did
over
a
quick
retrospective.
We
don't
were
also
able
to
use
these
temporal
groups
to
do
inference
in
very
noisy
scenarios.
So
this
is
an
example
of
a
very
noisy
picture
on
the
left,
but
because
we've
already
built
up
these
temporal
groups,
we
know
what
patterns
are
likely
to
follow.
Another
pattern
and
noise
noise
doesn't
fit
into
that.
A
So
you
can
use
those
several
groups
to
do
much
better
inference
than
you
would
without
the
temporal
groups,
and
we
could
show
it
extremely
noisy
images.
You
could
really
improve
your
image,
your
recognition,
so
here's
an
example
of
that
we
call
time-based
inference
and
the
blue
curve
shows
the
accuracy,
as
you
add,
increasing
amounts
of
noise
without
using
the
temporal
inference.
Ii
and
the
green
curve
shows
the
improvement
you
can
get
with
temporal
inferencing.
A
Okay,
so
you
can
get
a
pretty
significant
improvement
in
there,
so
this
was
this
was
interesting
and
even
today
you
know,
as
we
think,
about
temporal
pooling
and
feedback.
These
concepts
apply,
even
though
the
implementation
is
totally
different.
Now
this
this
concept
of,
if
you
know
the
temporal
group
and
the
temporal
pooled
representation,
you
can
do
much
better
in
a
noisy
scenario
than
otherwise
so
that
that
concept
still
holds.
A
You
just
throw
in
your
images
and
we
would
figure
out
it's
just
a
good
classifier
for
it
and
the
way
we
did
it
is
we
would
pre
train
a
vision,
hierarchy
on
thousands
and
thousands
and
thousands
of
natural
images,
so
those
were
all
pre
trained.
All
the
temporal
groups
who
are
created,
and
so
we
would
freeze
all
of
that
and
what
happened
in
the
toolkit
is.
A
When
you
added
your
own
images,
we
would
just
train
an
SVM
classifier
at
the
top,
so
it
would
just
be
a
little
bit
of
training,
but
the
bulk
of
the
visual
system
was
pre
trained
on
natural
images,
and
this
is
the
how
the
training
and
GUI
look
like
you
could
drag
in
your
own
images
and
very
easily
create
a
network
out
of
it.
We
had
an
internal
competition
on
this,
just
to
see
what
you
could
do
with
it.
I
don't
know
Donna's
here.
This
was
what
she
did.
A
She
tried
to
classify
chairs
versus
sofas,
I
I,
had
my
personal
photos,
library
had
a
lot
of
pictures
of
dogs
and
a
lot
of
pictures
of
kids
and
I
wanted
to
separate
those
two.
So
I'd
want
to
do
a
puppy
detector
and
then
a
bunch
of
people
did
a
whole
bunch
of
other
things.
So
just
shows
you
the
kind
of
the
flexibility
of
this
tool
you
could
throw
in.
You
could
figure
out
any
set
of
categories,
just
apply
it
to
the
thing
and
it
would
actually
work
reasonably
well.
A
This
shows
some
of
the
accuracies
that
we
received
from
just
not
many
of
the
folks
had
knew
nothing
about
vision
you
just
they
just
collected
the
images
stuck
it
in.
It
actually
got
quite
good
accuracy,
so
we
applied
this
to
em
NIST
back
then
so
in
remember,
then,
this
particular
network
was
not
really
designed
for
EM
Ness.
A
The
image
resolution
was
totally
different
and
it
was
pre
trained,
unnatural
images,
but
we
tried
it
on
the
amnesty
to
set
and
which
is
a
data
set
of
individual
digits
if
you're
familiar
with
that,
there's
about
60,000
training
images,
10,000
test
images
and
the
best
pub
results
were
in
the
range
of
95
percent
to
ninety
nine
point:
six
and
no
pick
at
that
time
got
ninety
eight
point,
four
percent,
which
is
very
respectable,
especially
considering
there
was
no
special
purpose.
Training
for
him,
missed
it's
on
par
roughly
with
a
human
accuracy
on
M
miss.
A
We
released
a
vision
prod.
We
fight
Amin
D
company
that
we
work
very
closely
with
released
a
vision
product
based
on
this
technology,
and
we
worked
very
closely
with
them
to
train
system
to
recognize
people
from
everything
else
in
video
images.
So
this
shows
you
little
demo
of
that.
Let's
see
what
so
you
can
see
here,
there's
there's
a
security
camera
pointed
out
of
a
window
there's
cars
passing
by
and
that
was
classified.
It's
not
a
person,
a
person
walks
in
there.
It's
green,
that
is
a
person.
A
A
A
Some
reason
I
can't
skip
here.
Okay,
this
is
the
UI
that
they
put
together
vitamin
D.
It
was
a
really
great
UI
on
top
of
the
vision
system.
So
what
you
could
do
is
here's
a
shows.
A
camera
pointed
out
again
a
security
camera
pointing
out
at
a
parking
lot.
It
would
recognize
people
and
what
you
could
what
you
could
do
on
the
left
hand,
side
is
filter
very
quickly,
got
jump
straight
to
the
events
where
people
were
detected
or
were
not
detected.
A
A
A
A
Okay,
we
did
a
little
bit
with
sequences,
but
not
much.
We
had
temporal
groups,
but
they
really
didn't
have
much
notion
of
directionality
in
there.
It
was
just
that
they
cook
her
in
time
or
not.
This
demo
shows
an
initial
experiment:
I
did
you?
Can
the
vertical
axis
shows
different
temporal
groups
and
it
was
able
to
kind
of
distinguish
between
someone
walking
forward
versus
someone
walking
backwards.
That's
an
example
of
directionality
and
it
kind
of
worked,
but
not
it
wasn't
really
great
at
that.
It
really
didn't
have
a
concept
of
true
sequences.
D
B
A
Again
it
kind
of
worked
in
those
scenarios,
but
it
you
know
it
was
I
think
roughly
equivalent
to
state
of
the
art.
It
wasn't
like
way
better
than
state
of
the
art
all
right.
We
got
a
whole
bunch
of
publicity
for
it
and
there's
a
picture
from
an
article
that
showed
up
I,
don't
know
who
thought
ographers
out
of
this,
but
did
jeff
hacks
the
human
brain
yeah?
Oh
really,
she
had
no
idea
was
gonna,
be
yeah.
A
The
algorithms
were
really
sophisticated
mathematically
and
there
was
even
a
mapping
to
cortical
columns
that
this
was
all
described
in
that
paper,
but
the
problem
was
that
it
was
getting
really
far
from
the
biology
and
then
the
real
issue
was,
if
you
go
back
to
this
set
of
criteria
and
then
you
try
to
judge
what
we
were
doing
against
this
said
you
could
say:
okay,
we
we
worked
on
hierarchy
worked
on
invariance.
We
that
kind
of
work
which
we
showed
that
was
working
well,
but
sequences
and
prediction.
A
You
know
we
were
kind
of
doing
a
little
bit
of
that,
but
not
a
huge
amount
of
that.
And
then,
if
you
look
at
all
of
the
other
stuff,
we
didn't
even
address
any
of
those
things,
and
it
was
really
difficult
to
figure
out
in
this
kind
of
formalism
how
we
were
going
to
get
those
other
properties.
So
something
like
continuous
learning,
for
example,
was
very
difficult
to
do
in
there.
You
know
things
like
attention
and
you
know
feedback
and
behavior.
It
was
gonna
hard
to
figure
out
how
we
would
extend
that
extended
technology
there.
A
So
that's
kind
of
takes
us
to
2009
and
I.
Remember
this
viscerally,
you
know,
Jeff
was
really
rethinking
everything
at
that
time
and
he
went
back
to
kind
of
the
basic
principle
that
no
Manta
was
founded
on,
which
is
that
we're
really
kind
of
at
the
intersection
of
neuroscience
and
computer
science,
and
we
had
kind
of
lost
sight
of
that
and
there
was
a
kind
of
a
six-month
period
where
Jeff
was
really
rethinking.
Everything
came
up
with
a
completely
different
way
of
thinking
about
things.
A
That's
what
we
wrote
up
in
this
white
paper
and
the
idea
behind
there,
as
you
all
know,
was
really
to
go
back
and
look
at
what
we
know
about
neurons.
What
we
know
about
the
laminar
structure
and
go
back
to
the
biology
and
come
with
algorithms
that
are
based
on
the
biology.
So
we
threw
away
all
of
the
math
and
went
back
to
neuroscience
intuitions
and
built
up
from
that.
Okay,
so
that
it's
kind
of
the
genesis
of
the
second
generation
of
algorithms
and
I'm
not
really
going
to
go
into
the
algorithm
again
in
details.
A
But
if
you
then
stack
it
up
against
those
criteria
with
the
CLA,
we
did
really
well
with
sequences
and
prediction.
I
feel
like
we
at
understand,
understood
that
really
well.
I.
Understand
that
really
well.
The
CL
is
really
good
at
continuous
learning
and
we've
shown
that
in
a
number
of
different
scenarios,
and
then
there
was
a
new
thing
with
the
CLA
there
was
a
new
property
sparse,
distributed
representations
that
was
actually
not
discussed
in
on
intelligence.
That
we
now
know
is
really
important
to
understanding
how
the
brain
works.
A
So
that's
something
that
we
really
addressed
with
the
CLA
hierarchy
and
invariance.
We
haven't
really
worked
much
on
that
and
if
you
are
on
the
mailing
list,
you
know
that
that's
an
issue
that
you
know
we're
pretty
open
about
it,
that's
something
that
we
haven't
done.
Even
though
it's
data
one
with
the
first
generation,
we
did
focus
on
that
stuff
and
so
far
we
you
know
back.
Then
we
hadn't
done
any
of
these
things.
Okay,
but
the
CLA
still
worked
really
well
on
a
number
of
different
practical
applications.
A
We
did
a
project
with
Forbes
where
we
looked
at
web
click
prediction.
So
the
idea
was
look
at
a
sequence
of
web
clicks
in
a
news
in
a
very
heavy
traffic
news
site
and
then
try
to
detect
or
try
to
predict
what
the
new
pages
would
a
user.
You
know
wanna
see
and
we
were
able
to
show
that
with
because
we
were
able
to
learn
the
sequence
of
clicks
really
well,
that
we
could
actually
do
a
really
good
job
predicting
the
possible
next
categories.
A
If
you
did
not
use
the
sequence
memory
you're,
the
best
accuracy
you
could
get
was
about
28%,
but
once
he
used
HTM,
this
sequence
prediction
except
you,
accuracy,
really
jumps
to
45
percent,
and
that
was
a
huge
improvement.
If
you
think
about
how
click-through
rates
and
stuff
are
measured
in
in
websites,
this
is
a
really
good
result,
normally
predicting
what
a
user
is
going
to.
Click
on
is
really
low
percentage.
So
we
were
able
to
show
this.
This
works
really
well.
A
A
A
This
entire
building
in
France-
and
so
this
was
I-
believe
this
is
the
DMV
and
in
a
village
of
press,
it's
a
pretty
fancy
DMV,
but
they
had
there.
It
was
doing
anomaly
detection
on
the
entire
building
and
we
went
to
France
several
times
to
work
with
this
company.
So
we
learned
a
lot
about
HTM
and
how
they
work.
In
these
scenarios,
we
released
a
whole
bunch
of
demo
applications
in
all
of
these
different
areas.
That
most
of
you
are.
A
And
they're
discussed
on
our
website
quite
a
bit,
so
that
kind
of
brings
us
to
where
we
are
today
and
the
third
general
I
think
about
as
a
third
generation
of
algorithms.
So,
unlike
the
difference
between
Zeta
1
and
C
la
well,
in
order
to
go
make
that
jump,
we
completely
throw
away
that
whole
branch
of
algorithms
and
start
it
from
started
again
with
this
generation
we're
actually
building
on
the
neuroscience
stuff.
A
A
lot
more
and
the
focus
of
what
we're
doing
is
to
fill
in
some
of
those
other
things
that
we
hadn't
really
worked
on.
So
we're
starting
to
think
about
hierarchy
and
invariance.
When
in
our
work
on
temporal
pooling
and
we're
starting
to
think
about
feedback,
we've
done
a
little
bit
with
behavior
in
sensorimotor.
We
haven't
really
done
a
lot
of
that.
Yet,
even
though
we
discussed
that,
but
so
right
now
that
the
work
that
we
are
doing,
Numenta
is
really
focused
on
I,
think
the
temple
pooling
hierarchy
and
invariance
and
and
feedback
okay.
A
And
then
what
is
my
next
slide?
Oh
so
you'll
notice
that
for
both
of
these
things,
I
kind
of
left
out
the
common
cortical
algorithm
and
the
laminar
structure
stuff.
So
what
is
that?
So?
Where
is
the
common
cortical
algorithm
and
Jeff
is
going
to
talk
about
this
a
little
bit
more
in
his
slide,
he's
talked
a
lot
about
the
laminar
structure
of
the
cortex
and,
if
you
look
at
it,
there's
a
very
regular
repeating
structure
everywhere
in
cortex.
A
If
you
look
at
the
inputs
that
are
coming
in
and
outputs,
and
what's
known
about
the
anatomy
and
the
physiology
we
can
play
certain,
you
know
we
have
a
pretty
good
sense
that
you
know
all
all
of
these
layers
are
implementing
some
form
of
sequence
memory.
It
looks
like
layer.
Six
is
very
heavily
involved
in
attention.
Layer
five
sends
out
motor
commands
to
the
rest
of
the
body,
and
so
we
it's
involved
in
motor
output.
Layer
four
is
involved
in
sensory
motor
inference
and
so
on.
A
We've
talked
about
this
stuff
before,
but
the
key
thing
here
is
that
this
structure
is
repeated
everywhere
in
cortex,
and
so
these
operations
actually
apply
to
all
cortical
regions.
They
apply
to
all
sensory
motor
motor
modalities,
and
my
view
is
that
this
really
is
the
common
cortical
algorithm.
So
if
you
understand
what
the
circuitry
is
in
these
layers
and
understand,
you
know
how
they
interact,
how
they,
what
their
properties
are
and
how
they
connect
together.
A
This
really
is
the
key
to
intelligence.
This
contains
all
of
the
properties
that
we
think
about,
think
of
as
being
essential
to
a
really
flexible,
intelligent
system,
and
so
a
lot
of
what
we're
focused
on
is
really
building
out
of
this
building
out
this
circuit
and
I.
Think
as
we
look
ahead
beyond
what
we're
doing
right
now,
the
idea
would
be
to
fill
in
all
of
these
other
things
and
I
believe
the
key
to
it
is
actually
figuring
out
the
behavior
piece.
There's
a
lot
of
what
we've
done
is
the
feed-forward
piece.
A
We
haven't
really
thought
about
the
output
too
much,
and
that
introduces
a
lot
of
complexities
in
the
circuit
and,
and
you
know,
figuring
out
what
the
interactions
are
with
the
different
layers
and
if
we
can
figure
out
the
behavior
of
peace
well,
I
think
all
of
these
things
will
fall
really.
Well,
so
you
know
generation
three
is
kind
of
what
we're
focused
on
right
now
and
if
you
think
about
our
roadmap,
it's
basically
filling
in
all
of
those
other
pieces.
Okay,
yeah
questions,
yeah.
C
A
Yes,
all
of
those
areas
are
very,
they
all
have
that
same
structure.
So,
even
even
if
you
think
about
like
motor
commands,
there
are
v1
outputs
motor
commands
and
it
is
involved
in
isakov's,
for
example,
so
yeah
it's
it's
sort
of
counterintuitive,
but
every
aspect,
even
what
we
traditionally
think
of
as
the
inference
areas
are
very
involved
in
motor
output
as
well.
B
F
This
particular
issue:
there
are
some
variations
that
occur
if
you
have
the
cortex,
so
in
some
animals,
some
mammals,
v1,
is
what
they
call
straight.
It
has
a
layer,
four
split
into
multiple
layers,
so
but
not
all
mammals.
So
it's
it's
not
required
for
all
vision,
but
primates
and
humans
have
a
striate
before,
which
is
one
of
the
few
exceptions,
really
that
it's
really
noticeable.
F
So
when
you
ask
the
question
about
v1,
I
didn't
know
if
you
were
referring
to
that,
because
that
is
an
example
where
there's
a
variation
on
a
theme
now,
even
though
they
a
force
put
into
multiple
layers
in
v1
in
a
human,
for
example,
the
other
layers
are
there
and
the
tsuba
they
said
layer
5
is
still
motor,
projecting
back
to
the
superior
colliculus
which
moves
your
eyes.
And
so
it's
not
true
to
say
that
these
you
know
I.
F
F
So
if
a
dog
doesn't
have
a
striated
v1
Oh
dogs
can
see,
but
maybe
they
don't
see
as
well
as
humans,
so
we
don't
want
to
those
slight
differences
or
so
on,
but
we
want
to
focus
on
the
common
properties,
but
it's
not
to
say
that
there
aren't
any
differences
between
cortical
regions.
Many
cortical
regions
are
very,
very
very
similar.
Some
have
some
notable
differences.
V1
is
the
most
notable
one,
also
the
primaries.
What
they
call
the
primary
motor
cortex
is
also
another
one.
It
doesn't
have
any
layer
4,
but
those
are
the.
A
A
So
I
just
wanted
to
give
you
a
retrospective
history
of
what
we've
been
working
on.
Why
we've
been
working
on
it?
Maybe
it
helps
you
understand
like
why
when
done
hierarchy
recently,
but
we
haven't
forgotten
about
it,
we
are
going
to
work
on
it
and
just
give
you
a
sense
of
these
things.
Take
time.
You
know,
we've
been
really
patient,
Jeff's
been
really
patient
over
decades,
and
this
is
kind
of
where
we
are
and
where
we're
going.
So
thanks.
D
E
If
you
could
describe
beyond
the
intuitive
definition,
what
attention?
Yes,
because
I
just
assumed
I,
know
because
I
know
what
the
word
means,
but
in
a
neurobiological
context,
you
know
what
what
what
properties?
What
what?
What
are
we
looking
to
achieve
when
we
talk
about
you
know
tackling
attention
attention.
A
And
okay,
so
there's
yeah,
that's
a
big
question:
has
it
lots
of
different
properties
to
it?
So,
typically,
you
know
from
a
psychophysics
standpoint.
They
typically
classified
into
two
types
of
attention:
there's
overt
attention
and
there's
covert
attention,
so
over
at
attention
is
when
we're
making
actual
muscle
movements
and
directing
our
focus
to
different
places.
So
you
can
think
about
isopods.
That's
moving
around
my
head
moving
around
is
a
form
of
overt
attention,
and
so
that's
a
very
obvious
form.
What's
perhaps
not
as
obvious
is
that
there's
a
covert
form
of
attention?
A
So,
even
if
my
eyes
and
muscles
are
perfectly
still
I'm
still
able
to
internally
attend
to
different
things,
so
I'm
able
to
internally
attend
to
different
ports,
portions
of
the
visual
field,
I'm
able
to
bias
my
perception
based
on
you,
know
previous
thoughts.
I
might
have
things
like
that.
So
I
don't
know
if
you've
seen
the
dalmatian
picture.
I
think
that
was
in
your
book
the
there's,
a
picture
where
it's
like.
Just
it's
a
very
famous
picture:
it's
just
black
and
white
splotches.
A
And
if
you
look
at
it,
you
can't
you
have
no
idea
what
it
is,
but
if
someone
tells
you
oh,
it's
a
dog,
then
immediately
the
shape
kind
of
makes
sense
and
you
can
see
it's
a
dog
drinking
water,
that's
a
form
of
covert
attention.
You
know,
that's
that's
a
form
of
feedback
or
you
could
say:
okay,
if
it's
a
dog.
How
would
you
interpret
if
you
know
it's
a
dog?
How
do
you
interpret
this
stuff
so
kind
of
bias?
You
can
think
about
it
as
a
form
of
prediction.
A
You
know
it's
kind
of
biasing,
your
your
in
you
know
how
you
perceive
the
input,
so
there's
a
lot
of
different
components
to
it.
Another
a
SPECT
is
when
there's
when
the
world
is
extremely
confusing
in
some
way,
like
it's
very
noisy
or
lots
of
different
aspects,
you
have
to
kind
of
focus
in
illuminate
a
lot
of
things
in
order
to
be
able
to
do
accurate,
inferences
and
there's
tons
and
tons
of
experiments
on
different
scenarios
where
you
have
to
have
attention
in
order
to
actually
accurately
do
your
inference.
E
E
F
Saw
this
chat,
I'll
add
on
to
something
super
tight
say
you
asked
about
this
or
maybe
you
might
have
been
asked
about
the
neural
neurobiology
substrate
of
attention,
and
we
know
a
lot
about
that
in
the
in
the
over
detention
that
Siebert
I
was
talking
about,
which
is
movement
in
the
cortical
region.
There's
certain
layer,
5
cells
that
generate
behavior,
and
so
that
would
be
the
the
output
of
the
overt
attention
from
system
and
then
we
know
a
lot
about
the
covert
attention.
F
A
lot
of
we
don't
know
about
it,
but
it's
generally
believed
to
involve
the
thalamus,
which
is
we
don't
normally
show
in
these
pictures,
but
I'm,
not
sure
in
mine
in
a
moment
which
is
very
intimately
involved
in
the
in
the
transfer
information
from
one
region
to
another
in
the
cortex
and
the
thalamus.
Is
this
a
little
gateway
that
all
this
a
lot
of
interests?
Information
goes
through
and
it
appears
to
be
involved
in
the
overt
attention.
F
We
also
know
that
the
cells
in
layer
6
just
below
layer
5
many
of
those
cells
project
back
to
the
thalamus
to
control
attention.
So
you
have
the
two
attentional
mechanisms
lined
right
up
on
top
of
each
other
in
the
cortical
region,
layer,
5,
being
the
over
detention
layer,
6
being
to
covert
attention
and
the
evidence.
This
evidence
that
I
have
am
I
thinking
about
this,
that
they
actually
they're
not
completely
independent.
So
what
an
example
I
like
to
give
if
I
say
well,
look
at
super
tie
and
you're
looking
at
them.
F
F
Now
I
believe,
actually,
if
you
restrict
the
movement
of
the
eyes,
it
automatically
starts
attending
too
small,
a
part
to
the
visual
space,
and
if
you
would,
if
the
eyes
move
in
broader
areas,
you
can't,
if
I
say
just
scan
over
super-tight
you'll,
never
be
able
to
see
the
details
of
its
glasses,
so
they're
actually
kind
of
interesting.
Your
movements
actually
dictate-
and
this
is
all
well
known
actually
in
literature-
there's
some
evidence
for
this-
that
the
movement,
the
over
movements,
actually
affect
the
covert
attention.
F
B
A
A
The
question
is:
I
think
if
I
understand
it
correctly,
there
there's
two
aspects
to
inference
this
kind
of
the
bottom-up
flow
of
information,
so
you're
looking
at
the
data
that's
actually
coming
in.
But
how
do
you
match
that
to
prior
ideas
of
what
that
image
might
look
like
where's
that
okay,
so
this
yeah.
A
So
every
modality
is
the
sensory
system
is
converting
things
to
sparse,
distributed
representations
and
then
each
sort
of
sensory
area
of
the
cortex
is
operating
on
those,
and
then
you
can
the
idea
behind
tempo
pooling
and
creating
invariant
representations.
You
can
imagine
that
that
can
happen
in
for
each
of
the
senses
independently,
but
once
they're
in
this
kind
of
common
format
of
sparse
distributed
representation,
another
part
of
cortex
can
easily
do
correlations
between
them.
They
can
easily
do
sequences
and
make
inferences
across
modalities.
A
F
I
would
only
add,
I.
Think
I
made
this
argument
in
on
intelligence
who
created
me
many
years
ago
that
yeah,
we
think
about
all
a
multi-sensory
like
oh
there's,
visual
and
auditory
and
somatic
senses,
but
you
can
think
of
in
as
soon
as
I
said
the
reason
a
cortex
has
no
idea
what
it's
looking
at.
It's
just
there's
no
idea.
F
It's
just
you've
got
century
bits
coming
in
here,
essentially
bits
coming
in
here
they
can't
be
correlated
at
the
first
level
of
the
hierarchy.
They
have
to
be
correlated
higher
up
in
the
hierarchy.
So
I
just
want
me
to
point:
is
it
really
not
different?
Multi-Sensory
is
the
same
problem
as
since
yeah
visual
hierarchy.
It's
not.
A
Really
different
yeah:
this
is
a
really
big
thing.
That
I
think
many
people
have
not
need
to
get
over
is
that
there
really
is
not
a
difference
in
addition
and
vigil
in
somatosensory,
it's
once
you've
encoded
things
properly.
It
doesn't
matter
where
the
information
is
coming
from
they're
all
spatial
temporal,
it's
just
stream
of
signals
coming
in
and
you
can
make
inferences
and
direct
behavior
in
those
modalities
without
knowing
anything
about
the
modalities
themselves
and
there's
a
really
powerful,
simplifying
principle
and
yeah.
We
just
have
to
forget
that
there
aren't
differences
here.
A
Yeah,
so
that's
the
feedback
pathway,
that's
the
part
that
you,
the
part
that
we're
working
on
right
now
that
can
show
that
if
you
have
some
prior
expectations
or
some
prior
knowledge
about
something,
you
can
use
that
to
bias
how
you
interpret
you
know
the
sensory
signals
that
are
coming
in
and
you
can
do
a
better
job,
the
more
prior
expectations
you
have.
You
know
even
in
very
noisy
scenarios
or
very
ambiguous
situations.
So
that's
something!
That's
a
current
active
area
of
research.
We.