►
From YouTube: Boosting (Episode 9)
Description
Learn about boosting and inhibition in this episode of HTM School with Matt Taylor.
Intro music: "Books" by Minden: https://minden.bandcamp.com/track/books-2
A
A
This
episode
is
about
homeostasis.
The
idea
is
that
life-forms
maintain
some
kind
of
physiological
constancy
or
stability
in
spite
of
external
conditions.
This
type
of
phenomenon
is
also
observed
in
your
brain
and
your
neocortex
think
about
it.
You
could
get
a
tons
of
different
input
from
your
senses.
Some
could
be
very
dense.
Some
could
be
very
light.
The
brain
has
mechanisms
to
keep
homeostasis
in
cortical
networks.
A
Let's
see
here,
homeostatic
regulation
of
neuronal
excitability
refers
to
the
collective
phenomena
by
which
neurons
alter
their
intrinsic
or
synaptic
properties
to
maintain
a
target
level
of
electronical
activity,
HTM
Theory.
We
call
this
boosting
in
order
for
a
column
in
the
spatial
Pooler
to
express
itself.
It
must
be
selected
as
a
winning
column.
You
remember
from
our
last
episode
when
I
talked
about
the
topmost
columns,
with
the
highest
overlap,
with
the
input
space
being
selected
as
winning
columns.
A
So
the
columns
with
high
overlap
with
the
current
input
are
more
likely
to
be
selected
as
winning
columns.
Looser
columns
are
inhibited
from
expressing
themselves.
Only
winter
columns
can
update
their
permanence
'as
with
the
input
space.
This
process
of
preventing
looser
columns
from
learning
is
called
inhibition.
The
boosting
mechanism
I'm
about
to
describe
can
change
a
columns
overlap
score
before
this
inhibition
phase
even
occurs,
giving
less
active
columns
a
better
chance
to
express
themselves
and
inhibiting
columns
that
seem
to
be
overactive.
A
So,
let's
take
a
look.
First
at
active
duty
cycles
of
spatial
Pooler
columns.
This
visualization
is
going
to
show
us
how
active
columns
are
over
time
and
you're
going
to
see
as
I
hit
this
button,
that
a
very
small
minority
of
the
available
columns
are
going
to
be
actively
trying
to
interpret
the
spatial
information
in
this
input
space
over
time.
In
fact,
you
can
see
these
red
ones
are
the
ones
that
have
the
most
activity.
Active
duty
cycle
is
essentially
just
how
many
times
has
the
column
been
active
over
a
certain
period
of
time.
A
In
this
case,
these
red
ones
have
been
active
every
time
step
because
Mac
the
active
duty
cycle
maximum
is
one
that's
a
hundred
percent
of
the
time.
So
all
of
these
very
red
dots
have
never
turned
off.
They've
been
active
the
entire
time.
So
far,
some
of
these,
the
the
orangish
ones,
the
yellow,
which
ones
there's
somewhere
in
between
as
we
now.
A
Finally,
some
of
those
turned
off,
so
the
maximum
active
duty
cycle
is
now
in
the
90
percentile
range
and
all
of
the
ones
that
are
zero
have
never
been
active
and
in
many
cases
they
never
will
be
active.
So,
let's,
let's
talk
about
this
very
explicitly,
as
you
can
see,
I
made
this
little
visualization
of
connections
over
here.
So
let's
take
this
this
guy
right
here.
If
we
look
at
him
his
overlap
with
the
input
space
is
19
and,
as
you
know,
that
means
19
input.
A
Bits
are
overlapping
with
its
connections
that
it
currently
has
with
the
input
space
and
we're
not
boosting
it
at
all,
and
so
we'll
talk
about
boosting
in
just
a
minute.
So
if
we
were
to
turn
boosting
on
which
I'm
going
to
do
right
now,
I'm
just
going
to
make
it
two,
this
is
sort
of
the
maximum
boost
is
a
as
a
field
that
we
use
to
specify
how
aggressive
our
boosting
algorithm
is
going
to
be
to
try
and
do
this
homeostatic
normalization
sort
of
deal.
A
So,
as
I
start
playing
here,
I'm
going
to
show
these
boost
factors
on
the
right
now
the
boost
factors
are
associated
with
each
column.
So
it's
a
one-to-one.
Each
column
has
a
boost
factor
and
it's
just
a
multiplier
for
the
overlap
score
for
that
column.
So
before
we
move
to
the
inhibition
phase
and
inhibition,
if
you
remember
from
previous
episodes
when
we
stacked
up
all
the
winter
columns,
you
know
and
we'll
take
many
of
the
first
40
winners
with
the
highest
overlap
score.
Those
are
the
winners
and
the
rest
don't
learn
anything
right.
A
Only
the
winners
learn.
That's
inhibition.
We
inhibit
all
of
the
loser
columns
from
learning.
So
if
we
were
to
change
the
overlap
scores
of
the
columns
before
we
did
that,
in
addition
phase
we
can
affect
which
columns
end
up
learning
and
which
ones
end
up
expressing
themselves
and
which
ones
don't
that's
what
boosting
does
in
this
case,
all
of
the
very
weak
columns.
A
For
example,
all
of
the
ones
that
are
green
over
here
have
very
little
overlap
with
the
input
space,
but
we're
going
to
artificially
boost
them
up
so
that
they
can
start
expressing
themselves
with
what
they
see
of
the
patterns
in
the
input
space.
So
in
this
case
here
I'm
looking
at
this
cell
right
here,
it's
overlap.
Score
is
17.
There's
17
input,
bits
that
are
overlapping
with
its
connections
to
the
input
space,
but
the
boosted
overlap
scores.
Seventeen
point
six,
eight
because
of
its
boost
factor,
so
that
column
has
a
particular
boost.
Factor.
A
A
That's
part
of
the
boosting
algorithm
I'm
not
going
to
talk
explicitly
about
that,
but
it
it's
basically
looks
at
the
active
duty
cycles
for
a
for
a
cell
and
all
of
its
surrounding
cells
and
just
to
get
an
idea.
Is
that
one
way
too
high
way
too
low?
How
should
we
affect
it?
It's
you
know,
maybe
10
or
20
lines
of
code
that
does
this
little
calculation
to
get
us
these
boost
factors,
so
that
is
what's
providing
this
homeostasis
that
we're
talking
about.
A
We
want
all
these
columns
to
to
represent
themselves
to
express
themselves,
but
what
they
see
in
the
input
and
this
boosting
mechanism
prevents
some
of
the
strongest
columns
that
come
out
from
the
spatial
representation
from
dominating
the
representation
entirely
kind
of
squelches
those
down
and
encourages
the
weaker
columns
to
take
part
and
say
what
you
think
about
the
patterns
that
are
going
on.
So
it
gives
them
a
chance
to
express
themselves.
So
I
also
want
to
show
what
this
looks
like
when
you
compare
a
spatial
Pooler
that
has
boosting
on
versus,
has
boosting
off.
A
So
in
this
comparison,
I've
got
one
with
boosting
off
entirely
and
the
right
hand,
or
the
bottom
graph
and
on
the
right
hand,
grid
as
boosting
at
ten,
so
I'm
going
to
run
this
ahead
for
a
bit
and
then
we're
going
to
talk
about
what
we
see
here.
But
you
can
already
tell
immediately
that
with
boosting
on
the
spatial,
Pooler
is
much
more
efficient
because
it's
using
more
of
its
columns
to
represent
the
data
that
to
model
this
data
with
boosting
off
there's
just
a
small
minority
of
columns
that
are
even
contributing
to
the
model.
A
So
let
me
step
in
and
talk
about
some
of
what's
going
on
here
in
this
representation.
So
one
of
the
things
you
you
might
have
noticed
is
that
the
weakened
representation
is
certainly
easy
to
discern
from
both
these
patterns,
but
there
seems
to
be
a
higher
granularity
in
the
spatial
recognition
happening
with
boosting
on,
for
example,
at
this
point
in
the
day,
this
is
a
Monday
Tuesday,
Wednesday
Thursday.
So
it's
a
weekday.
A
We
have
not
only
similarity
with
other
days
at
that
general
power
level,
but
we
have
a
specific
time
based
similarity
as
well,
so,
for
example,
with
eating
off
this
point
is
very
similar
to
all
the
different
power
levels
that
we've
seen
on
a
weekday
no
matter
what
time
of
the
day
that
occurred
with
boosting
on,
we
have
a
much
higher
similarity
with
not
only
that
power
level
we've
seen
in
the
past,
but
at
that
time
that
we've
seen
in
the
past.
So
at
each
point
in
the
day
you
can
you
can
sort
of
identify.
A
There
was
a
a
time
where
we
was
most
similar,
whereas
in
the
previous,
the
one,
without
boosting
on
it's
harder
to
discern
at
what
point
in
that
day
is
the
most
similar
point.
So
there's
there's
that
to
think
about,
and
you
can
also
tell
as
we
bounce
along
here
and
we're
going
to
move
we're
moving
into
a
Friday.
A
A
This
has
semantic
similarity
with
this
entire
swath
of
data,
which
was
a
weekend,
but
when
it's
boosting
on
again
we're
going
to
have
higher
similarity
with
that
time
of
day,
then
then
we
are
with
the
entire
weekend
which,
in
in
the
case
of
boosting
off
it's
mostly
making
the
association
with
oh
I'm
in
a
weekend,
so
I'm
more
similar
to
times
I've
been
in
a
weekend
with
boosting
on
since
we've
got
that
more
spatial
granularity.
We've
got
more
of
those
weaker
columns.
A
So
it's
it's
a
higher
granularity
of
spatial
recognition
that
you
can
get
with
boosting,
and
this
is
a
range
effect
you
can
go
from
1
to
as
high
as
you
want
from
X
boost.
So
if
you
want
to
distribute
the
active
completely,
you
can,
and
you
can
still
get
some
semantic
information
out
of
it
and
every
column
will
have
the
exact
almost
the
exact
same,
active
duty
cycle.
If
you
make
boosting
max
boost
something
like
a
hundred
so
you'll,
see
that
when
you
do
that,
it'll
really
spread
the
activity
out
throughout
all
of
the
columns.
A
None
of
them
will
be
very
much
higher
in
active
duty
cycles
than
the
others.
We
need
boosting
to
distribute
the
cellular
activity
throughout
the
spatial
Pooler
to
take
advantage
of
each
of
the
columns
potential
to
contribute
to
the
model
that
we're
creating
the
boosting
squelches
these
overactive
columns.
It
encourages
the
less
active
columns
to
become
more
active
more
often
and
that's
basically
the
the
gist
of
it.
It's
not
too
complicated,
but
it's
a
very
important
mechanism
in
HTM
serie.
Otherwise
we
don't
get
the
efficiency
that
we
need
to
get
out
of
spatial
pooling.