►
From YouTube: HTM Spatial Pooler
Description
Numenta engineer Yuwei Cui walks through how the HTM Spatial Pooler works, explaining why desired properties exist and how they work. Includes lots of graphs of SP online learning performance, discussion of topology and boosting.
See corresponding paper at https://discourse.numenta.org/t/the-htm-spatial-pooler-a-neocortical-algorithm-for-online-sparse-distributed-coding/1548
This was recorded in the Numenta office during an engineering lunch meeting on Feb 8, 2017.
A
Facial
cooler
and
because,
as
you
know,
this
is
a
recent
paper
and
the
review
in
compliance
merely
discussed
the
properties
of
facial
cooler
and
come
up
with
a
bunch
of
metrics
that
can
quantify
the
performance
of
facial
fillers.
In
addition
to
these,
two
papers
are
also
a
bunch
of
other
resources,
a
good
line,
the
HTM
School,
where
math
talks
about
the
properties
of
special
tutor
extensively,
so
that
small
approachable
results
and
what
I'm
going.
A
Of
academia
like
talk
that
covers
the
paper
basically
so
cause
also
special
puller
is
open
sourcing
in
your
pick,
so
at
a
very
high
level,
Sabri
is
a
powerful
streaming
analytics.
Engine
is
continuously
receives
a
vast
amount
of
information
from
periphery
sensors,
and
this
is
in
the
form
of
millions
of
light
chains.
So
a
fundamental
question
in
Europe
science
is:
how
does
the
neuron,
in
the
context,
learn
to
respond
to
specific
input
passing?
How
do
they
decide,
which
include?
Should
they
respond
to
and
on
the
population
level?
B
B
A
360-
and
it
also
satisfies
a
bunch
of
properties-
that's
important
for
downstream
processing,
actually,
because
the
spatial
cooler
is
at
a
very
upstream
in
the
whole
HTM
system
is
critical
to
get
a
facial,
fuller
right
in
order
to
have
any
learning
down
for
sequence,
memory
and
the
question
we
often
get
up
by
the
communities
what
properties
to
SP
the
special
cooler
achieved.
Why
we
need
learning
SP?
Why
can't
we
just
use
random
act
II
and
was
a
function
of
boosting
SP,
so
I'm
going
to
talk
about
properties,
various
properties
or
SP,
this
presentation?
A
So
that's
the
background,
so
I've
start
by
describing
the
algorithm
of
spatial
for
us.
This
is
not
a
very
detailed
description
to
like
more
like
a
high-level
summary
of
the
activation
and
learning
algorithms
in
the
spatial
polar
and
then
I
will
focus
on
the
various
properties
and
discussed
the
founder
of
metrics.
That
can
only
buy
these
properties.
They
have
done
a
lot
of.
A
So
the
HTM
models,
a
layer
of
cells,
is
the
context
and
it
contains
a
set
of
cells
are
organized
into
what
called
mini
columns,
which
is
the
word
instead
of
vertically
aligned
itself
and
the
HTM
theory.
We
model
each
neuron
as
having
three
different
types
of
input,
including
the
piece
for
the
input
that
targets
on
the
proximal
dendrite
next
segment,
and
the
contact
should
includes
happy
down
to
the
lab
through
lateral
includes
that
technology
so
basal
to
join
excitement
and
the
feedback.
A
The
special
puller
receives
input
from
a
set
of
input,
neurons
and
each
mini
column
typically
connect
to
a
subset
of
the
input
space.
So
by
default
it
has
if
the
input
space
has
topologies
the
special
food,
also
models,
pathologies,
and
by
that
I
mean
nearby
HP.
Mini
columns
will
receive
input
from
nearby
a
region
nearby
subspace
of
the
input
space.
So.
B
A
A
D
B
A
B
A
Row
coefficient
regions,
the
top
pay
percent
of
the
main
columns
that
has
most
of
the
overlaps
column
will
become
active
and
your
videos
percent
any
column
to
become
active
interspatial
product.
So
it's
a
very
sparse
activation,
but
at
the
same
time
is
also
each
abilities.
You
have
more
than
one
columns
at
you
at
any
time,
so
here's
the
synaptic
connections
are
modeled
by
a
binary
connection
lee.
So
this
is
almost
the
neural
network
models
that
feels
accurate,
synaptic
weights.
A
A
So
you
may
notice,
there's
another
here.
Obviously,
composting
in
facial
Porter
that's
useful
to
ensure
homeostatic
accessibility
control.
So
basically
the
input
is
weighted
by
this
booster
factor,
which
is
kind
of
like
a
factor
that
determines
how
expectable
the
current
in
economies-
and
that
depends
on
the
past,
activation
history
of
that.
The
recent
activation
history
of
that
mini
column,
so,
depending
on
the
system
tracking
the
boots,
a
factor
as
a
function
of
the
recent
activities,
frequency
of
that
mini
column.
A
So
if
it's,
the
activation
frequency
is
exactly
the
same
as
a
target
level,
which
is
about
two
percent.
There
will
be
no
boot.
Boot
tracker
will
be
one
which
is
no
boost.
This
is
large,
actives
and
desires
that
mini
column
will
be
more
accessible
than
its
neighbors
and
active.
Too
often,
it
will
be
like
acceptable
to
its
neighbor.
So
using
this
mechanism
we
can
encourage
all
many
columns
to
participating
in
representing
the
input.
This
is
called
homeostasis
in
your
mind,
and
I
will
show
you
why
this
is
important
later
with
the
metrics.
A
A
A
A
To
capture
this
underlying
structure,
with
idea
of
the
actual
freaking
inter-korean
approach
to
the
federal
representation
to
random
noise
after
learning,
so
we
quantify
what
we
mean
by
better
with
a
bunch
of
properties
and
a
little
goodies,
so
this
include
person
there
should
be
on
page
apart
knees.
This
is
like
an
inherent
property
of
the
special
Cooper.
In
addition,
those
other
properties
required
learning
such
as
distributed
Colleen,
to
use
our
columns
to
represent
something
to
preserve
the
systemic
sing
aloud.
A
A
So
the
first
property
that
is
it
has
you
need
to
form
of
nations
that
have
relatively
fixed
smartness.
So
in
this
experiment,
I'm
showing
you
on
the
top
graph
is
a
bunch
of
inspectors
that
fell
into
the
spatial
cooler
and
it
has
variable
of
5.
Star
parties
sounds
happens,
a
sensor
some
patterns
of
butter
and
on
the
bottom
I'm
showing
the
output
of
the
spatial
quarter.
It
has
the
purposes
of
neotec
minimum
variability.
A
So
here
we
define
Spartan
is,
as
the
population
Spartan
is
basically,
as
at
each
time
point
you
calculate
how
many
HP
mini
columns
are
active
and
divided
by
that
by
the
total
number
of
mini
columns.
But
typically
we
have
around
2%
of
the
mini
columns
as
giving
the
spatial
cooler,
and
this
is
out
of
1024
2048
columns
in
the
entire
system.
Here,
I'm
only
showing
you
a
fraction
of
small
fractions
between
columns
because
of
the
spacing
and
the
diversity
still.
A
A
Basically,
we
want
every
single
many
columns
to
participate
in
representing
the
input,
so
this
is
important
from
an
information
theoretic
perspective,
so
you
can
imagine
if
I
see
four
columns,
that's
not
active
at
all.
It
hasn't
coming
at
information
and
four
columns
of
active
a
lot
for
a
lot
of
time.
They
also
doesn't
convey
information,
so
we
want
to
ensure
every
column
has
some
action
in
it,
so
this
guy
needs
to
come
true
only
bodies
which
first
calculate
the
activation
frequency
of
each
mini
column.
B
A
How
they
are
not
asking
for
any
of
these
inputs
and
not
the
burning.
We
have
every
column
acting
for
about
2%
of
these
new
cooks
and
we
treat
each
mini
column
as
a
binary
variable
and
compute
the
entropy,
the
binary
entropy.
For
that,
and
then
the
entropy
of
the
entire
spacial
cooler
is
complete.
The
summation
of
entropies
of
individual
mini
columns,
so
here
I'm
showing
the
entropy
before
during
and
after
learning.
So
exactly
there's.
The
difference
is
not
huge.
This
is
suggesting
that
given
adrenaline
seen
as
a
reasonable
w
this,
but
still.
B
A
Second,
property
of
XP
is
knowledge
blackness.
This
is
after
learning,
you
want
the
output
of
the
spatial
cooler
to
be
relatively
insensitive
to
not
in
the
input,
especially
those
inputs-
that's
frequent
occurrence,
so
here
I'm
marrying
the
amount
of
change
of
your
special
power
output
as
a
function
of
the
noise
level.
Noise
here
means
in
activating
a
small
fraction
a
fraction
of
the
activities
and,
at
the
same
time,
you
also
active
the
same
amount
of
the
activists
to
keep
the
responses
of
thing
in
this
experiment.
A
A
B
A
This
curve,
this
is
basically
function
for
occupying,
and
this
is
the
resist
knowledge
about
these
matrixes
performance
before
before
training
and
after
chain.
So
there
is
a
pretty
significant
improvement
of
knowledge
about
skills.
In
this
case,
we
can
monitor
all
these
properties.
All
these
metrics
as
a
function
of
learning
continuously
so
here
on.
The
x-axis
is
a
box
of
time
of
training.
A
How
many
iterations
go
through
the
entire
dataset
and
I'm,
showing
you,
the
engineering
you
created
by
a
logical,
Baptist,
also
another
important
matrix
is
the
theology,
so
we
don't
want
acti
to
be
changing
all
the
time.
So
if
the
same
encode
is
present
is
twice,
we
want
to
have
similar
facial
Pula
output
and
that
ability
matches
is
basically
qualifying.
A
What's
the
percentage
of
difference
on
the
spatial
color
output,
given
the
same
input
across
the
across
room
at
all
and
on
the
bottom,
the
true
graph
is
showing
the
number
of
musing
axes
and
the
number
of
synapses
is
eliminated
from
the
spatial
footer
at
the
bottom
train.
So
it
pretty
much
stabilizes
after
50
at
all.
A
We
can
understand
why
that's
the
case
by
looking
at
the
activation
frequency
distribution
on
the
new
dataset,
so
right
after
into
the
new
data
set
70%
of
the
special
cooler
mini
columns
are
not
responding
to
anything
good.
So
this
is
a
pretty
significant
and
we
see
this
big
data
discovering
the
entropy
number
entropy
matrix,
but
after.
B
A
A
So
if
the
input
changes,
the
spatial
fuller
will
adjust
the
connections
to
by
the
representative
input
and
another
interesting
property
of
spatial.
Cooler
is
fault
tolerance.
So
in
this
experiment,
while
we
damage
a
fraction
of
the
physical
remaining
columns
in
the
center
and
the
monitor
a
bunch
of
metrics,
both
before
the
damage
and
after
damage
to
see
how
to
recover
the
one
thing
is
the
really
special
for
many
columns.
We
computed
a
receptacle
Center,
which
is
basically
we
look
at
all
the
critiques
for
a
given
mini
:
and
compute.
A
A
B
A
B
A
This
is
not
sure
you
where
clearly
this
graphic
or
what's
happening
here
is
so
I
have
for
the
damaged
column
near
the
boundary
tends
to
actually
losses
in
its
neighbor,
so
it
has
a
lower
boosting
factor.
So
this
because
of
this
a
difference
of
booster
factor,
you
will
see
a
shift
of
the
receptive
field.
Stencil
cabal
Colossus
of
damaged
region.
We
have
physics
movie,
so
here
is
the
damage,
and
you
can
see
the
receptive
field
starts
moving
slowly
towards
the
demonstration
to
cover
the
damaged
part
of
the
spatial
cooler.
A
In
a
second
experiment,
we
damaged
part
of
the
enclosed
space.
Is
that
simply,
similarly
with
blocks
part
of
the
input
neurons
and
just
don't
allow
any
activations
in
part
of
the
input
space
so
biologically?
This
is
similar
to
a
lesion
in
the
retina,
for
example,
a
focal
lesion
in
the
retina
and
the
prevents
damage
experiments
like
a
lesion
in
the
visual
cortex,
and
we
do
monitor
the
same
metrics
and
in
this
case,
because
the
lesion
is
in
the
input
space
right
after
the
little
engine,
nothing
happens.
A
Course,
their
cells
connecting
to
the
to
the
center
part
of
the
input
space,
but
after
a
while
special
cooler
realizes,
there's
actually
nothing
to
be
represented
in
that
part
and
the
connection
to
that
region.
Regions
you
get
eliminated
due
to
the
happy
learning
room
and
after
the
ecologist
epithelial
my
gosh
after
the
astronaut
reorganization,
no
doubt
almost
no
tells
how
receptive
field
in
the
center,
because
the
others
there
is
nothing
to
be
representing
the
past.
They
have
to
remove
a
way
to
maintain
your
activation
level.
A
So
if
you
look
at
the
booster
factor
right
after
the
lesion
mean
columns
that
represent
the
lesion
part
will
help
you
don't
have
any
input.
The
activation
frequency
will
go,
go
down.
You
have
lower
activation
frequencies
in
neighbors,
and
the
boost
factor
will
increase
the
soil
physics,
the
right
block
in
the
center,
and
they
start
to
completely
the
neighbor
mini
columns
and
such
represented
inputs
that
close
to
the
region,
property
inspector.
B
A
Requires
this
increase
of
accessibility
for
those
mini
columns
to
to
learn
to
represent
other
stuff?
If
you
don't
have
this
Magnum,
this
digging
ecology
to
stay
there
and
it
will
not
be
used
for
anything
and
this
actually,
unlike
the
previous
experiment,
this
happens
fairly
fast.
So
it's
almost
instantaneously
so
nice
because
it
takes
a
couple.
Maybe
five
apples
from
go
back
here
almost
instantaneously.
It's.
A
With
some
experimental
study
where
they
found
out,
if
you
do
a
focal
lesion
in
the
retina,
the
cells
in
the
visual
cortex,
we
augment
your
receptive
field
by
artifacts
within
several
minutes.
But
if
you
have
a
stroke
that
damaged
part
of
the
visual
cortex,
that
reorganization
occurs
at
a
much
slower
time
scale
and
takes
month
is
for
to
recover.
From
a
stroke,
the.
A
A
A
Really
talk
about
this
basically
map
simulate
includes
two
similar
output.
It's
kind
of
showing
you
this
nozzle
busting
in
courses.
Your
change,
that
includes
by
a
small
amount,
doesn't
affect
the
output
much
after
learning,
also
continuous
learning
and
simulation
so
that
how
so
the
resources
are
online
and
balanced
by
the
paper
and
also
the
HTM
school
is
a
good
shop
to
learn
about
special
color
I.
D
I'm
kind
of
get
some
ginger.
If
there's
two
things
here,
one
is
what's
going
on
the
biology
and
how
do
we
notice?
What
do
we
model
and
saw
in
the
biology
is
rolling
around
SE?
The
tips
of
the
dendrites
are
constantly
growing
out
and
trying
to
find
connections,
and-
and
so
we
can't
just
decide-
I
need
to
go
to
X.
Some
distance
away
have
to
so
feel
its
way
there,
and
it
will
continue
growing
unless
it
forms
new
synapses
that
are
useful
right.
D
So
it's
a
much
more
organic
processor
in
so
after
we
don't
model
that
we
don't
have
to.
We
can
just
define
this
region
of
potential
synapses
and
willy-nilly
just
say:
we're
going
to
connect
any
one
of
them,
because
we
also
worry
about
physically
keeping
the
health
of
a
dendrite
of
keeping
the
ACP
going
on
electronics,
but
so
we
model
in
a
simple
form
and
I'm
modeling
the
growth.
So
we
achieve
this
exact
same
result
by
just
having
a
larger
potential
pull
up.
Sternum
right.
D
D
I
think
so
it
does
yeah
I.
Think
there's
some
movies
of
this
to
do.
You
can
see
that
online,
but
it's
a
much
more
dynamic
process
that
people
realize
this
hats
on
having
super
fast
and
when
you
could
slow
it
down
and
speed
it
up,
and
she
does
it
sort
of
like
this
little
you
know
seems
moving
in
and
out
trying
to
find
new
connections
and
it
will
split
and
it'll
just
keep
going.
C
A
B
A
A
We
convert
the
same
receptive
field.
There
has
to
be
some
mechanism
to
ensure
that
these
are
the
same
thing,
so
you
cannot
just
rely
on
this.
The
topology
of
the
input
space-
that's
very
accurate
I,
had
think
that
when
you
look
at
the
branch
and
further
singles
I'm,
a
cotton
fiber
is
much
more
broader
than
the
scale
of
the
receptive
field.
So
that's
part
of
the
paper
I
haven't
talked
about.
This
arises
very
enemies
paper.
They
speculate
about
two
potential
mechanisms.
Mechanisms
for
this
to
happen.
Why
is
you?
A
Maybe
you
rely
on
me
heavy
trainer
on
ensuring
that
all
the
cells
in
the
mini
columns
I
have
the
same
research
field?
The
other
is
maybe
there
is
a
especially
during
development.
They
have
a
sub
planer
on
the
first
versions
of
the
piece
powers
of
the
field,
and
then
she
causes
stress
observances
itself
and
it's
a
mini
column,
to
learn
the
same
thing
so
could
be
both
so.
D
We
draw
our
eyes,
are
beautiful,
yeah
I'm,
not
sure?
That's
exactly
the
question
you're
asking
not
exactly
like
that.
I'm
doing,
question
I
think
you're,
attractive
and
there's
a
reason
why
they
should
prove
could
be
applied
to
this
neuron
to
be
columns
with
no
elevator
there.
Yes,
first
among
Tourette,
occur.
Point
of
view
does
not
become
haters.
Would
look
that
a
wheeze?
That's
not
useful,
because
the
way
the
whole
system
works
and
and
be,
and.
D
The
context
of
slices
of
this
you
a
was
just
saying
all
the
neurons.
We
know
that
reducing
stores
in
put
on
the
other
proximal
synapses
are
a
mini,
cost
and
share
economies
of
the
fields
in
committee.
Come
so,
but
technically
you
could
just
think
of
the
many
Commons
our
honor,
yes,
but
not
any
other
Spadina
fitted
white
dress.