►
From YouTube: AI With the Best: Matt Taylor's HTM Presentation
Description
This is the presentation Matt gave at the ai.withthebest.com conference in September 2016.
A
Hi,
my
name
is
Matt
Taylor
and
I
am
from
a
company
called
the
meta
I'm
gonna
talk
about
our
machine
intelligence
technology
called
hierarchical
to
portal
memory
and
first
off
I,
just
want
to
say
that
I
have
a
lot
of
content.
I'm
gonna
go
pretty
fast,
and
this
I'm
going
to
provide
you
with
a
feeling
for
how
these
algorithms
work,
but
not
all
the
exact
details
about
how
they
work,
but
I
will
give
you
some
resources
at
the
end.
So
you'll
know
where
to
go
to
learn
more.
So,
let's
jump
right
in.
A
A
A
Ai
has
arrived
because,
when
I
think
of
truly
intelligent
machines,
I
want
them
to
be
able
to
perform
lots
of
different
general
tasks,
something
that
can
help
me
with
my
budget
help
me
plan
my
vacation
as
well
as
notifying,
if
my
biorhythms
are
abnormal
or
something
so
I
disagree
with
this
statement.
Ai
has
not
arrived,
we're
still
in
the
in
the
details
of
trying
to
figure
out
what
it
is
and
how
it's
going
to
be
implement.
A
Granted
there
has
been
a
really
interesting
resurgence
and
AI
technology
over
the
past
decade.
That's
been
really
cool.
Deep
learning
can
do
some
really
interesting
things,
especially
with
spatial
pattern,
recognition,
image,
classification
speech,
recognition
and
that
has
been
awesome,
but
I
must
deposit
that
machine
learning
is
not
intelligence.
A
Everyone
has
a
different
definition
of
intelligence,
so
my
definition
of
intelligence
may
be
stricter
than
some,
but
I
don't
believe
that
what
we
have
today
is
intelligence
and
other
people
agree
with
me
to
this
Oreo
videos.
From
from
deep
mind,
also
says,
deep
learning
is
not
Sciences
from
Stanford
seminar
just
from
a
month
or
so
ago,
and
the
reason
that
I
think
it
is
is
because
the
neuron
model
for
an
artificial
neural
network
is
vastly
simplified
compared
to
a
biological
neuron
and
back
when
an
ends
were
created.
A
We
didn't
know
a
lot
about
how
brain
works.
Since
then
it's
been
decades,
we've
learned
a
massive
amount
on
about
neuroscience
about
how
neurons
interact
with
each
other
about
how
the
neocortex
works.
So
our
model
of
an
HTM
neuron
is
a
much
more
complex,
and
so
this
is
one
of
the
reasons
that
I
stayed
I,
don't
think
current
machine
learning
techniques
based
upon
this
simple
neuron
model
has
been
produced.
A
What
some
people
call
strong
AI
now
our
mission
at
my
company
is
dual
one
is
to
understand
how
intelligence
works
in
biology
and
the
mammalian
neocortex
and
two
is
to
create
software
based
upon
those
principles.
So
we're
very
focused
on
biologically
plausible,
biologically
constrained
algorithms
for
creating
intelligence,
not
so
focused
on
results
were
much
more
focused
on
creating
something,
that's
biologically
plausible
and
are
trying
to
create
interesting
things
along
the
way.
A
I
believe
that
these
biologically
constrained
methods
of
creating
intelligence
have
a
much
higher
chance
of
producing
what
we
would
refer
to
as
strong
AI
I
hope
that
you
are
going
to
learn
more
about
this
alternative
approach,
because
that's
what
I'm
going
to
talk
about
so
getting
these
algorithms
right
takes
us
a
long
way
towards
a
working
system
because
of
the
homogenous
nature
of
the
cortical
tissue.
So
let's
talk
about
brains
a
little
bit.
A
Here's
a
cross-section
of
the
cortex
and
one
of
the
interesting
things
that
you
see
is
the
cortex
has
a
rather
homogeneous
structure
throughout.
And,
if
you
look
inside
of
this,
it's
a
cellular
structure
in
each
one
of
these
cells
does
a
is
a
neuron
and
we're
trying
to
take
advantage
of
this
idea
that
this
homogeneous
structure
that
exists
throughout
the
cortex
has
a
common
set
of
learning.
Algorithms.
That
applies
to
all
sensory
input
and
all
input
coming
from
other
regions
of
the
cortex,
so
that
is
part
of
our
alright
be
up
here.
A
So
I'm
gonna
go
right
into
some
of
this
theory
and
based
entirely
on
this
idea
of
sparse
distributed
representations.
This
is
the
data
medium
I
like
to
call
it
a
data,
medium
of
the
brain
so
think
about
it
like
this,
every
big
section
of
kornek
cortex
in
your
head.
If
you
just
take
a
little
small
section
of
cortex,
it's
getting
feed-forward
input
from
a
ton
of
different
nerves,
nerve
axons.
This
is
sort
of
like
a
fiber
optic
cable.
A
If
each
each
section
of
cortex
has
a
view
to
this,
this
part
of
fiber
optic,
cable,
each
fiber
on
that
cable
could
represent
a
neuron
whether
the
the
optic
is
lit
or
not,
is
on
or
not
means
that
the
neuron
is
active.
Also,
each
bit
has
some
semantic
meaning
it
means
something
in
some
context
now
the
cortex
has
no
idea
where
these
feed-forward
inputs
are
coming
from.
They
could
be
coming
from
senses,
so
they
could
be
coming
from
other
regions
of
the
cortex.
It
also
has
no
idea
what
they
mean.
A
It
has
to
learn
that
over
time,
so
that
is
part
of
the
problem.
We're
trying
to
solve
here
so
I'm
gonna
talk
about
STRs
I'm,
going
to
show
you
some
visualizations
of
STRs,
starting
with
an
SDR,
is
capacity
all
of
these
visualizations
you'll
be
able
to
see
as
well
I'll
give
you
links
to
the
after
the
presentation,
but
here's
an
example:
I'm
gonna,
I'm
gonna
create
an
SDR
so
think
about
that
fiber
optics,
cable,
that's
what
I'm
talkin
about
here!
This
is
like
a
representation
of
a
fiber
optics
people.
It's
just
a
bit
array.
A
It's
a
bit
array
of
2048
bits
right
now.
Only
five
of
them
are
on.
I
can
change
this
if
we
want
to
make
it
more
brain
like
we
make
it
about
two
percent,
because
that's
the
typically
in
your
brain.
How
many
neurons
are
active
at
any
point
in
time.
So
my
point
here
is
with
a
representation
of
a
sparsity
within
this
sparse,
distributed
representation.
A
The
capacity
of
this
and
put
at
2048
bits
is
astronomically
large.
It's
very,
very
big.
Your
you
will
never
ever
run
out
of
whiz
to
encode
data
in
the
space.
If
we're
using,
you
know
40
bits
and
2048
bits.
So
my
point
here
is
this:
medium
is
enormous.
Okay,
now
another
interesting
thing
here,
that's
very
important
is
understanding
how
to
compare
these
STRs,
so
the
properties
of
STRs
lean
make
us
make
it
impossible
for
us
to
do
really
interesting
comparisons.
A
The
primary
one
is
an
overlap
score,
so
we've
got
one
SDR
on
the
left,
one
on
the
right,
and
then
this
one
over
here
on
the
far
right
is
their
comparison.
So
what
I'm,
showing
right
now
is
an
overlap
that
is
essentially
a
binary
and
meaning
that
if
the
bit
is
on
in
both
of
these
spaces,
it
will
be
on
in
this
space.
So
this
gives
us
an
indication
knowing
that
eleven
of
these
bits
are
similar
between
these
two
STRs.
A
It
gives
us
an
indication
of
similarity
and
and
not
just
whether
it's
similar
or
not
that's,
sort
of
an
analogue
representation
of
similarity.
We
can
also
show
a
union
which
is
a
binary
or
to
sort
of
combine
the
semantics
of
these
two
bits
as
well.
So
the
overlap
score
is
more
important.
That's
what
I
want
to
emphasize
is
we're
going
to
compare
STRs
right
here.
Look
the
next
one.
So
let's
talk
a
bit
about
matching.
A
So
in
this
example,
I
have
an
SDR.
An
original
SDR
on
the
right
are
on
the
Left.
Excuse
me,
and
in
the
middle
here
I
have
added
33%
noise.
So
a
third
of
these
bits
are
wrong.
You
know
they
don't
match
the
original
and
what
I
want
to
show
here
is
I'm,
going
to
compare
them
both
and
show
their
overlap
score
so
because
of
the
noise.
If
there
were
no
noise
at
all
zero,
though
we'll
have
score,
be
40
right,
because
there's
no
noise,
it's
exactly
right,
so
I'm,
gonna,
I'm,
gonna,
add
some
noise.
A
The
overlap
score
is
now
27.
So
the
idea
here
is
that
sparse
distributed
records
in
Asians
are
noise,
tolerant
because
I
don't
have
to
match
every
single
bit.
I
can
move
my
theta
theta
being
how
many
of
those
bits
need
to
be
the
same
before
I
will
consider
them
the
same
so
I'm
going
to
change
that
to
27
so
that
this
one
matches
so
with
33
percent
noise.
I
can
get
a
match
here
and
the
probability
of
this
being
a
false
positive
match
is
again
astronomically
small.
A
So
we
have
this
property
of
being
able
to
match
STRs
and
recognize
whether
we
have
seen
an
SDR,
not
even
with
the
presence
of
a
decent
amount
of
noise.
Let's
bump
it
up
to
like
70
percent
move
my
feet
down
until
we
have
a
match
so
around
here.
So
in
this
case,
even
with
71
percent
noise,
I
can
still
recognize
this
and
a
chance
of
the
being
a
false
positive
is
still
very
small,
1.9
times
10
to
the
negative
12.
A
So
on
the
Left
we
have
an
original,
SDR
and
I'm
going
to
show
you
how
I
don't
even
need
to
compare
all
of
the
bits
to
that
original
SDR
I
could
subsample
it
so
I'm
going
to
take
50%
of
those
bits
I'm
going
to
throw
this
one
away
and
just
use
these
50%
and
then
I'm,
going
to
take
a
random
SDR
here
and
and
I'll
show
the
comparison
over
here.
What's
the
laugh-
and
this
button
just
like
throws
another
random
SDR
there
and
shows
you
the
overlap.
A
The
point
I'm
trying
to
make
here
is
that
if
I
saw
the
original
SDR
I
would
definitely
be
able
to
match
against
it,
because
my
theta
is
10
and
I
mean
we
have
control
over
this.
We
can
change
whatever
we
want,
but
in
the
presence
of
faults.
If
some
of
those
neurons
are
just
stopped
working
right,
the
the
inputs
to
the
neuron,
that's
deserving
this
STR
just
stop
working.
A
A
So
these
these
someone
who
is
chatting
about
what
it
is
represented
right,
so
each
one
of
these
bits
is
going
to
represent
some
semantic.
Meaning
look.
Let
me
let
me
explain
this
I'll
try
to
explain
this
a
little
better
if
we
get
a
false
match.
Let's
see,
let's
do
this,
my
go
substance
yeah
sure,
so
if,
for
some
reason,
I
get
a
random
SDR
here
and
I
and
one
and
however
many
million
billion
chances,
one
of
them
is
falsely
matched.
A
That
means
that
the
semantics
of
that
specific
data
point
is
similar
enough
to
what
we
have
stored,
that
it
probably
doesn't
matter,
because
it's
all
of
these
STRs
have
semantic
meaning
they
mean
something
all
the
bits
mean
something.
So
if
this
random
thing
that
comes
in,
if
we
miss
categorize-
and
we
think
it
is
something
that
we've
seen-
but
it's
not
it's
probably
close
enough
to
the
thing
that
we
have
seen
that
it
doesn't
matter.
If
you
think
about
you
see
a
furry
thing
and
the
side
of
your
vision,
it
may
be
a
doggie
cab.
A
You
know
it's
some
type
of
mammal
a
little
now
it's
sort
of
like
that.
Okay,
so
I
have
to
keep
going
again,
use
QA,
I'll
answer.
All
your
questions
and
I'll
tell
you
where
you
answer:
ask
more
questions
later
so
now
we're
going
to
talk
about
encoding
data
into
STRs
and
keeping
the
semantic
meaning.
So
this
is
part
of
the
hard
thing
to
do
with
HTM
systems
and
it's
encoding
data.
So,
for
example,
this
is
encoding
in
a
scalar
value
between
1
and
100,
and
here's
the
value
I'm.
A
As
you
can
see,
here's
26
and
I'll
just
I'll
turn
comparison
on
and
I'll
skip
forward
to
27,
and
you
can
see
that
there's
a
three
bit
difference
between
26
and
27,
semantically,
26
and
27
in
this
representation
are
similar
right
if
I
jump
ahead
to
from
31
to
49.
Those
are
not
there's
no
overlap
there,
so
there
will
be
no
semantic.
Similarity
between
these
two
I
could
change
the
size
of
the
bucket
in
which
I'm
representing
these
values,
so
that
the
difference
between,
for
example,
34
and
52
does
have
some
semantic
similarity.
A
It
really
all
depends
on
your
input
space
and
how
you
need
to
how
you
think
you
need
to
encode.
The
data
is
38
very
close
to
39.
It
really
depends
on
the
problem
space,
so
this
is
an
example
of
a
very
simple
scalar
encoding
mechanism,
II
and
there's
more
than
one
way
you
can
do
this.
Here's
another
scalar,
encoding
mechanism
that.
A
Distributes
that
bucket
throughout
the
space
and
expands,
so
you
don't
have
to
give
it
a
minimum
and
Max
value
it
sort
of
expands
by
itself,
and
this
is
called
a
random
distributed,
scalar
encoder.
So,
as
you
can
see,
the
difference
between
510
and
511
is
just
one
bit
if
I
jump
ahead,
there's
a
big
difference,
there's
only
one
bit
between
588
511.
That
is
similar,
so
it's
very
dissimilarity
still
so
there's
different
ways
for
encoding
different
things,
and
these
are
just
two
methods
of
doing
it.
A
One
more
in
Coto
I
want
to
show
you
is
date
encoder,
and
this
shows
you
how
different
coatings
can
be
combined
to
encode
different
semantics.
So
here's
an
encoding
of
data
week,
here's
one
for
whether
the
day
as
a
weekend
or
not.
This
is
time
of
day
in
season
and
as
I
move
this
day
forward,
you
can
see
the
day
of
week.
Semantics
are
changing
and
as
I
go
to
weekend,
the
Wiggins
exchange
and
as
I
move
from
September
to
October
in
semantics,
changed
right,
so
I'm
encoding
all
the
semantic
information
babe.
A
We
did
an
anti
love
day
if
I
go
and
start
moving.
The
time
of
day
and
those
are
all
being
combined
just
concatenated
into
this
encoding,
which
encompasses
all
of
that
semantic
meaning.
So
this
entire
encoding
has
Dave
week
as
weekend
as
time
of
day
and
as
the
season
they
all
encoded
with
that,
so
that
these
are
just
some
encoding
methods
that
could
be
used
to
translate
a
problem
space
into
a
sparse
distributed
representation
so
that
HTM
system
can
operate
upon.
A
So
now
we're
going
to
get
into
some
of
the
actual
algorithms
I'm
going
to
talk
now
about
spatial
pooling.
So
the
major
goals
of
spatial
pooling
are
number
one.
You
have
to
translate
that
input
space
into
another
space
with
a
fixed
sparsity,
because
the
input
space
could
be
very
chaotic.
It
may
change
over
time.
A
A
The
power
consumption
the
time
of
day,
whether
it's
a
weekend
or
not,
and
those
are
the
only
things
on
the
coding
time
of
Day
weekend
and
the
consumption
and
in
the
input
space
down
there,
you
can
see
it
changing
as
I
move
this
along.
This
is
the
power
consumptions
going
up
and
down
with
the
days
and
then
time
of
day
and
weekend
are
also
represented
down
there.
So
the
thing
I
want
to
explain
here
is
this
is
just
one
way
to
represent
this
particular
stream
of
data.
A
In
the
input
space
there
could
be
hundreds
of
ways
to
represent
this.
This
is
the
way
I've
chosen
to
do
it.
I
can
also
use
that
random
distributed
scalar
encoder
that
I
showed
earlier
to
encode
the
power,
and
that
would
work
just
fine,
because
that
encodes
the
semantic
information
just
the
same
way
that
the
other
one
knows
just
with
the
bits
distributed
throughout.
A
There
could
be
hundreds
of
other
ways
to
to
take
this
particular
data
set
and
express
it
in
this
medium
I'm,
just
showing
you
this
as
an
example,
so
the
spatial
cooler
and
here's
where
we
really
start
getting
into
the
meat
of
HTM
theory
remember.
This
is
meant
to
translate
an
input
space
into
a
space
that
has
a
normalized
sparsity,
and
we
call
this
the
spatial
pooler's
columns.
A
So
I'll
talk
a
little
bit
about
this
in
a
minute,
but
each
one
of
these
bits
in
the
columns
represents
a
main
column
in
the
neocortex
and
a
mini
column
is
just
a
bunch
of
different
neurons
who
are
stacked
on
top
of
each
other
that
share
a
dendritic
Stegman
to
the
inputs
based.
So,
for
example,
this
column
that
I'm
highlighting
right
here
has
a
specific
relationship
to
the
synthesis
right
now.
A
I'm
showing
I'm
highlighting
all
of
the
potential
cells
in
the
input
space
at
this
column
might
be
connected
to
because
it's
sort
of
randomly
initialized
and
each
each
column
in
this
spatial
cooler
is
randomly
potentially
connected
to
a
different
set
of
the
input
space
they're,
all
just
randomly
initialized
now.
I
also
want
to
show
you
that
when
I
click
on
this
each
of
these
lines
off
that
right
now,
this
column
also
is
initially
created
and
has
connections
already.
A
Based
on
this
permanence
value,
so
these
red
squares
they
show,
are
not
connected
and
like
Here
I
am
over
here,
because
the
connection
threshold
is
0.1
and
the
current
purpose
for
this
connection
from
this
column,
to
that
input
cell
is
only
0.8
if
I
go
into
this
cell,
it's
beyond
the
connection
threshold,
so
there's
a
blue
dot
there.
So
there's
actually
a
connection
from
this
column
to
this
input
space
right
now,
all
the
red
ones.
There
is
no
connections.
So
that's
that's!
A
A
Do
random,
espy
yeah.
Let's
talk
about
overlap,
learning
rules
here,
so
this
is.
This
is
instantiating,
see
all
right.
So
here
we
have
an
input
space
on
the
left
and
a
representation
of
some
input.
This
is
I'm
just
throwing
in
some
rent
some
some
input
here,
just
to
show
you
an
example
and
here's
a
spatial
cooler's
relationship
to
that.
Not
only
the
input
space,
but
also
this
particular
input
so
I'm,
so
we
can
see,
for
example,
this
column
has
a
an
app
score
of
37.
A
That
means
there
are
37
green
dots
that
overlap
the
Elliots
connections
that
overlap
with
the
current
input
space,
which
is
the
blue
shaded
area.
Here.
All
of
these
gray
dots
are
connections
that
fall
outside
of
that
space.
Okay
and
each
one
of
these
columns
has
a
completely
different
relationship
with
that
space.
A
completely
different
overlap
score.
This
one
is
30,
for
example,
this
one's
33.
Here
the
green
ones
are
the
ones
that
are
having
highest
amount
of
overlap
with
the
input
space.
A
So
what
I've
done
over
here
on
the
right
is
I've
sorted
all
of
the
overlap,
scores
every
single
column
and
stacked,
ranked
them
here
and
I've
drawn
a
line.
You
can
see
like
right
here
and
I'm
gonna,
say
everything
above
this
line.
Those
columns
will
now
become
active,
and
this
is
exactly
how
your
brain
is
also
doing
this
at
every
step.
It
decides
which
columns
are
going
to
be
activated
based
upon
their
relationship
with
the
input
space
take,
for
example,
this
call
it
has
44
bits
that
fall
within
this
input
encoding.
A
It
has
become
activated
now
that
it
has
become
activated,
it
will
learn.
So
what
that
means
is
that
all
of
the
gray
dots
that
fall
outside
of
this
encoding
in
the
input
space,
all
of
those
permanence
values
from
this
column
to
each
gray
dot
will
be
decremented.
That
means
it
will
start
forgetting
those
connections.
All
of
the
green
dots
that
fall
within
this
particular
input
encoding
will
be
incremented,
thus
strengthening
this
columns
relationship
with
those
input,
cells
and
I'll,
show
you
how,
over
time
this
looks
and
in
our
next
visualization
and
as
so
yeah.
A
So
let's
talk
about
learning
so
in
this
visualization
I'm,
going
to
run
two
spatial
coolers
we're
going
to
have
one
which
I
call
a
random
spatial
bubbler
which
has
learning
turned
off.
So
all
it
is,
is
just
those
initial
random
permanence
--is
that
and
it's
mapped
to
the
input
space.
It
doesn't
learn
it
all
throughout
its
lifecycle,
the
other
one
learning
spatial
cooler
down
here.
A
It
is,
it
has
learning
turned
on
so
we'll
start
to
see
over
time
that
the
learning
spatial
cooler
starts
to
deviate
from
the
random
one
and
create
representations
that
better
match
the
semantics
of
similar
input
data
that
it's
seen
previously,
and
it's
going
to
take
a
little
while
to
progress
here
because
of
got
video
recording
on
this
is
a
compute
heavy
process.
But
what
I
can't
show
you
here?
Let
me
get
to
a
good
point,
so
these
dots
sure
how
well
you
guys
can
see
this,
but
oops.
A
A
The
red
is
the
most
dissimilar
to
this
current
state
and
it's
showing
exactly
what
I
would
expect
it
to
the
points
in
the
past
that
are
most
similar
to
this
point,
which
would
be
you
know
the
beginning
of
the
day,
not
on
a
knot
on
one
of
these
downturns.
Those
are
the
most
similar
and,
as
as
we
continue
to
move
through,
you'll
see
that
happen.
A
You
know
as
this
as
we
go
down,
the
green
bars
go
to
the
bottom
as
we
go
up,
the
green
bars
go
to
the
top
and-
and
it
performs
better
with
the
learning
spatial
cooler
and
generally,
you
can
take
a
snapshot
and
see
right
away.
Obviously
this
is
better
because,
where
I'm
at
now
is
more
semantically
similar
to
the
same
points
in
the
past,
then
then
they
random
spatial
colors
showing.
So
this
is
showing
you
that
the
learning
is
actually
making
a
difference
and
helping
better
represent
the
spatial
correlations
within
this
space.
A
So
the
last
visualization
I'm
going
to
show
you
is
a
beta
of
something
that
I've
been
working
on.
So
this
is
the
temporal
memory
algorithm.
So
the
whole
point
of
this
spatial
cooling
thing
is
to
create
a
normalized
space
that
we
can
run
temporal
memory
algorithm
upon,
and
so
this
space,
like
I,
said
each
one
of
these
is
a
column
of
cells
and
that's
what
we
get
represented
here.
For
example,
here's
an
active
cell
and
the
spatial
cooler
columns,
which
is
an
active
column.
A
So
the
key
is
the
blue
ones
are
currently
active
cells
that
were
correctly
predicted
for
the
last
time
stuff.
The
red
ones
are
predictive
cells
that
it
thinks
these
cells
think
they're
going
to
be
active
next
and
remember
the
compute.
The
computing
objects
in
this
in
this
model
are
the
neurons
themselves.
So
this
cell
is
making
the
decision
whether
it
thinks
it
will
be
active
next,
based
upon
all
of
its
connections,
to
the
input
space
and
to
two
neighboring
cells,
which
I
haven't
talked
about
at
this
point.
A
But
I
don't
have
time
to
go
into,
but
this
temporal
memory
system
also
has
intercellular
relationships
as
well,
so
it
can.
It
observes
what
it's
seen
around
it
and
and
to
help
it
make
predictions.
So
so,
as
we
move
forward
it,
if
we're
not
seeing
any
yellow
cells,
yellows
are
inactive
cells
are
I'm,
sorry,
yellow
are
active
cells,
but
that
have
not
been
correctly
predicted.
A
If
we're
not
seeing
yellow
cells,
then
it's
it's
perfectly
predicting
the
the
input
now,
the
the
very
first
part
of
the
sequence
it
it
never
quite
knows
about,
because
I'm
resetting
the
sequence
at
the
end.
So
the
first
step
in
the
sequence
is
always
sort
of
a
Thomas
once
it
sees
it,
though
it
knows
what
the
next
step
is.
Gonna
be,
and
it's
an
it's
spot
on
throughout
the
next
all
the
next
steps
of
the
sequence.
A
So
the
interesting
thing
here,
I'm
just
gonna-
let
this
run
a
little
bit
because
I'm,
basically
you
know
I'm
training,
it
I'm,
just
I'm
establishing
a
learning
here.
It's
it's
learning
this
spatial
pattern,
even
though
it's
a
very
trivial
pattern.
It
does.
It
takes
some
time
for
all
of
those
connections
to
to
become
establishes
permanence
values
to
increase
and
decrease.
A
So
now
what
I'm
going
to
do
is.
Let
me
pause
this.
Let
me
go
a
little
further
here.
I've
got
I'm
gonna
shake
it
up
right.
So,
instead
of
going
all
from
top
to
bottom,
there's
going
to
be
a
little
jitter
in
here,
it's
gonna
go
like
one
two
and
then
bounce
down
and
then
bounce
back
up
and
then
continue.
So
the
first
pattern
that
we
see,
we
know
we're
not
sure
what
it
is,
because
it's
just
the
first
thing
we've
seen
but
we're
making
a
prediction
for
the
next
one.
A
Yes,
we
were
correct
about
the
prediction,
and
now
the
next
prediction
is
predicting.
These
cells
are
going
to
be
active,
but
when
we
move
forward,
the
jitter
hasn't
happened
yet
it'll
happen
in
the
next
cycle.
So
let's
do
this
again.
Okay,
so
now
it's
predicting
that
these
red
cells
can
be
active.
We
move
forward
now,
that's
something
unexpected
happen,
because,
instead
of
moving
into
this
section
of
cells,
it
jumped
ahead.
So
the
some
of
the
cells
that
it
predicted
apparently
were
we're
correctly
predicted,
but
a
bunch
of
them
that
actually
activated
were
not.
A
So
you
can
tell
at
this
point
that
something
unexpected
happen
and
again
it
goes
back
up
now
and
again,
something
unexpected
happened.
It's
it
sort
of
has
some
predictive
cells
that
thinks,
because
it
seemed
those
bits
before,
but
just
not
in
the
sequence
so
now
I'm
going
to
just
let
this
run
for
a
while
and
let
it
learn
that
there
are
basically
two
patterns
here.
One
goes
all
the
way
down
the
next
one:
just
does
a
little
jitter
all
the
way
down
the
next
one
does
a
little
chipper
okay.
A
So
now
it's
eventually
it
will
learn
both
of
those
patterns
and,
at
some
point
it
has
to
decide
what's
coming
up
next.
So
let's
stop
right
here
now
that
it's
learned
these
some
of
these
patterns
and
I'm
going
to
step
forward,
and
at
this
point
this
is
the
point
where
it
could
go
here
or
to
go
here.
So
look
at
all
these
predictive
cells,
it's
many
more
predictive
cells
than
last
time.
It
is
because
it's
seen
both
of
these
things.
It's
seen
this
happen,
and
it's
seen
this
happen.
A
It's
seen
both
of
those
contexts
when
we
move
forward.
Some
of
those
are
gonna
be
correct,
and
some
of
them
will
actually
totally
correctly
predicted
that
so,
but
as
it
should
at
some
point
get
so
that
yeah.
So
this
was
the
one
that
had
got
confused
on,
but
I
think
it
probably
just
needs
to
to
see
this
pattern.
A
few
times
typically
will
it'll
seem.
You
know
thousands
of
data
points
before
we
would
consider
this.
A
This
model
to
have
enough
knowledge
about
the
input
space,
so
so,
thanks
for
bearing
with
me
on
this
one
I
just
created
this
visualization
on
Friday,
so
I'm
still
sort
of
working
through
it,
but
now
I
am
going
to
go.
Go
ahead
with
my
slides.
I
might
have
a
little
bit
of
time
to
take
questions
at
the
end,
but
let's
I'm
gonna,
finish
out
with
these
slides
and
there'll
be
more
resources
to
talk
about.
A
So
one
of
the
big
points
I
want
to
make
is
that
these
are
really
just
the
foundational
aspects
of
neocortical
inspired
intelligence.
These
are
the
building
blocks
of
creating
an
intelligent
thing
based
upon
the
neocortex
and
I,
truly
believe
that
this
is
the
future.
Now
that
AI
I,
don't
like
the
term
AI.
A
So
I
have
a
lot
of
resources
for
learning
more
about
this
I've
been
working
over
the
past
year
on
a
youtube
series
that
takes
all
of
these
concepts
that
that
I've
been
explaining
to
you,
except
for
temporal
memory.
I
haven't
done
this
yet,
but
I
will
be
working
on
those
soon
and
breaks
them
out.
In
short,
you
know
fifteen
minute
videos
so
start
with
bit.
Arrays
I
mean
these
are
for
absolute
layman.
You've
never
heard
of
this
before
you.
A
Even
if
you're,
not
a
computer
scientist,
you
could
still
follow
along
and
then
talking
about
s
yards
or
capacity.
You
comparison
and
the
overlaps
that
subsampling
et
cetera
and
all
the
way
up
into
spaceship,
Euler
and
learning
and
I'll
be
working
on
temporal
memory
episodes
within
the
next
several
months.
A
Just
a
reminder:
this
is
all
open
source
code.
All
of
our
HTML
meant
a
shion's
are
open
source.
We've
got
a
strong
community
around
these
HTM
technology.
We've
been
open
source
for
over
three
years.
Now
we've
got
a
forum,
we'd
love
for
you
guys
to
join
our
community
and
help
us
try
and
build
these
intelligent
machines.
There's
a
lot
of
discussion
on
our
forums
called
HTM
forum
and
oh
by
the
way,
to
find
this
HTM
school
series.
Just
google
HTM
school,
it's
the
first
thing
that
comes
up.
A
It's
not
an
ad,
and
also
you
can
google
HTM
forum.
It's
the
first
name
comes
up,
that's
not
an
ad
and,
as
our
also
go
to
discourse,
Numenta
org
I
read
this
forum
all
the
time.
If
you
have
any
questions,
this
is
a
very
good
place
to
add
to
ask
your
questions
so
one
one
last
thing
I
want
to
note-
is
that
all
of
this
being
said
this
approach
in
no
way.
Trivializes
the
work
of
the
machine
learning
pioneers
like
these,
who
have
been
working
on
these
really
hard
problems
for
decades.
A
I
have
great
amount
of
respect
for
all
of
our
machine
learning,
pioneers
that
have
been
tirelessly
trying
to
implement
intelligence
in
and
artificial
neural
networks
for
years.
We
absolutely
need
people
like
this.
It's
just
that
from
my
approach,
I'm,
not
by
trade
or
by
education.
A
computer
scientist,
so
I
came
through
this
approach.
Reading
this
book
called
on
intelligence.
This
book
was
a
product
of
the
Redwood
Neuroscience
or
the
Redwood
centre
for
theoretical
neuroscience.
So
this
guy
Jeff
Hawkins
is
my
boss.
You
don't
know
who
he
is.
A
He
founded
palm
created,
the
Palm,
Pilot
and
handspring,
and
all
that
stuff,
but
his
passion
has
really
always
been
intelligence
and
the
brain
so
when
he
was
done
with
palm
and
mobile
computing,
he
founded
the
Redwood
science
for
Theoretical,
the
redwood
centre
for
theoretical
neuroscience,
and
then
that
was
a
big
source
of
the
book
on
intelligence.
I
read
this
book
in
2006
and
I
was
extremely
intrigued
by
it
and
motivated
to
try
and
figure
out
I
thought
this
is
the
right
way
to
try
and
build
truly
intelligent
machines.
A
You
know
start
with
what
we
know
and
move
on
from
there
so
from
that
Numenta
was
founded
based
upon
the
work
of
the
Redwood
Neuroscience
Center,
and
it
was
donated.
The
Redman
Center
was
donated
to
Berkeley
and
still
exists
today
as
a
theoretical,
Neuroscience
Institute.
So
so
I
hope
you
understand
that
we're
just
coming
from
a
different
approach.
A
I
feel,
like
our
approach,
has
a
better
chance
of
getting
in
us
to
truly
intelligent
machines
than
the
current
machine
learning
landscape,
not
that
that
trivializes
all
the
work
that
all
of
these
great
pioneers
have
done
for
decades
in
the
past.
So
that's
basically
my
presentation.
So
thanks
everybody
for
for
and
I'll
see
if
I
can
address
any
of
these
questions
here,
real
quick
before
my
time's
up,
let's
see
these
grids
are
moves.
I,
don't
know
much
about
SEPA
Tron's
and
it
is
ok
for
something
to
be
wrong.
A
A
So
the
encoding
zs--
were
all
created
by
hand
there.
The
encoding
algorithms
are
basically
just
code
that
and-
and
we
have
a
suite
of
encoders
already
that
exist
in
new
pic
new
pic.
Is
the
new
Binta
platform
for
intelligent
computing,
go
to
our
forums
or
go
to
demented
organ
invented,
,
find
out
more
about
it,
but
we
already
have
encoders
like
date/time
encoders
I
showed
you
see
our
encoders
dealt
encoders.
We
even
have
a
geospatial
coordinate
encoder,
which
is
really
cool.
A
It
actually
will
translate
the
semantics
of
geospatial
information
into
STRs,
so
you
can
do
stuff
like
anomaly.
Detection
on
routes
of
objects
moving
through
time
space,
which
is
really
useful,
I
think
it's
one
of
the
coolest
things
that
we
can
do
see
a
cortical
columns.
So
what
I'm,
but
this
is
representing,
is
actually
a
very
small
cross-section
of
layer,
2/3
of
one
region
of
cortex.
A
So
we're
talking
like
one
known
I
mean
this
is
extremely
small
scale,
but
typically
I
think
we
create
a
grid
of
65,000
or
so
cells,
32
columns
per
cell,
and
that
is
a
representation
of
layer,
2
3
of
the
cortex.
There
is
no
hierarchy
in
our
current
implementation,
although
we
have
implemented
HAARP
in
the
past
or
we
wanted
to
get
just
one
layer
right.
How
do
you
persist
the
data
we
don't?
The
data
is
not
persistent.
A
It
is
learned
over
time,
so
the
only
representation
of
the
data
that
exists
htm'
system
at
any
point
time
exists
in
the
permanent
values
between
the
cells.
So
you
can't
go
back
and
replay
that
input
data
ever
just
like
in
your
brain.
You
remember
what
it
feels
like
to
be
6
years
old,
but
you
can't
reverse
time
and
go
back
and
feel
and
get
the
exact
input.
You've
got
when
you're
6
6
years
old,
the
sensory
input
flows
through
your
brain
and
and
goes
away.