►
From YouTube: Column Swarm Reinforcement Learning - Eric Laukien
Description
2015 HTM Challenge Submission. 2nd Place Innovation.
A
B
A
Okay
and
we
have
Eric
obvious
skype
right
now,
so
thank
you,
Eric
for
your
submission
good
to
see
you
so
let
me
first
say
eric
has
been
involved
in
our
community
for
a
while.
Now
doing
some
really
interesting,
cutting-edge
integrations
between
HTM
theory
and
other
machine
learning
techniques
that
he's
been
investigating
so
he's
he
I
would
call
him
one
of
those
mad
scientist
who's
just
taking
this
into
areas
that
nobody
else
is
exploring
right
now
and
I
commend
him
for
that.
C
I
have
a
question
it
like
Matt's.
A
lot
of
it
is
think
beyond
my
expertise,
but
it
I
was
curious
about
what's
going
on
with
the
the
animated
robot
it
looks.
It
reminded
me
somewhat
of
a
like
genetic
algorithms,
words
you're
testing
out
something
see
if
it
works.
If
it
doesn't,
if
it
does,
you
keep
adding
to
it,
and
is
that
the
that,
what's
going
on
there.
D
Sort
of
it's,
it's
actually
reinforcement
learning,
so
it's
an
attempt,
I'm
reproducing
the
way
humans,
learn,
basically
uses
a
sort
of
HTM
like
hierarchy
that
predicts
its
next
actions
and
then
their
little
reinforcement
learning
units
at
each
node.
These
are
my
malls
of
the
column,
which
is
basically
each
column,
is
a
reinforcement
learner
in
my
model,
which
might
not
be
realistic.
But
the
thing
is
the
reinforcement.
Learners
can
act
like
a
real
column.
They
can
learn
how
to
be
a
real
column.
D
C
E
I
have
a
follow-up
question
on
that
I
think
at
the
beginning
of
the
video
you're
right
that
you
use
Hugh
learning
as
one
of
the
layers,
the
reinforcement
learning
algorithm
that
so
the
reward
function
is
just
going
right.
Then,
when
you
show
that
it's
going
backwards,
then
how
does
that
work?
Is
it
also.
F
So
I
have
a
question:
is
that
I'm
trying
to
stand
the
overall
motivation
for
the
work
I
mean
you
could
be
saying:
hey
I'm,
trying
to
combine
HTM,
but
the
particular
other
types
of
learning,
algorithms
or
you're
you're,
saying
no
I'm
actually
trying
to
move
HTM
in
a
different
direction,
or
you
might
be
saying
no
I'm
trying
to
solve
a
problem
and
I'm
using
a
mixture
of
tools
to
do
that.
So
it
wasn't
clear
what
the
overall,
what
you
wrote
about
motivation
is
and
how
that
would
help
us
to
interpret.
D
So
I'm
working
on
that,
I
actually
just
now
created
a
GPU
version
of
the
algorithm
of
just
the
predictive
hierarchy,
not
the
reinforcement,
learning
it
and
I
adamite
tested
it
yesterday
and
it's
able
to
predict
stuff
like
HTM
it
uses,
unlike
HTM.
It
uses
like
some
different,
sparse
coding,
algorithms,
with
explaining
away
properties,
but
most
of
the
idea
is
still
the
same
way
of
the
body.
Directional
hierarchy
that
you
strike,
features
and
stuff
upwards
and
predict
downwards.
G
D
Say
my
columns
are
supposed
to
learn
how
to
be
HTM
columns.
They
use
reinforcement,
learning
because
I
don't
know
exactly
how
the
HTM
Callum's
work
to
be
honest
or
how
real
columns
work
so
I,
just
basically
kind
of
winged
it
and
made
a
reinforcement,
learn
that
tries
to
mimic
a
real
column
through
Ward
signals.
So
it
can.
It
can
learn
to
approximate
actual
columns
and
it's
able
to
gate
attention
and
predictions
and
everything.
D
D
D
Well,
I
haven't
really
run
it,
for
that
long
is
busy.
Usually
learns.
How
to
get
to
decent
speed
within
a
few
seconds
is
I.
Have
the
speed
up
key
that
I
pressed
in
the
demo,
so
it
I
don't
really
know
what
the
maximum
theoretical
boundary
is
for
this
problem,
but
it's
obviously
limited
by
the
environment
to
some
extent,
but
I
have
a
demo
where
I
tried
like,
for
instance,
increasing
the
strength
of
the
motors,
and
then
they
basically
sometimes
learn
to
glitch
out
the
physics
and
just
fly
all
over
the
place.