►
From YouTube: NuPIC Office Hour: CLA Quiz (Nov 12, 2013)
Description
NuPIC Office Hour: CLA Quiz (Nov 12, 2013)
See the CLA Quiz here: http://numenta.org/blog/2013/11/11/cla-quiz-office-hour.html
It didn't show up in the video, so wherever you see the Numenta logo on a blank screen, I was trying to display the quiz.
B
Can
you
see
our
video?
No,
I
only
see
your
the
icon.
A
A
A
D
C
F
A
Hey
guys
so
I'm
displaying
the
quiz
here
to
ask
you
guys
can
would
you
mind
muting
your
mics
and
if
you
have
something
to
say
unmute
it
otherwise
I'll
have
to
control
your
volume
from
here.
A
Okay,
so
so
I'm
just
going
to
switch
back
and
forth
to
the
drive
between
your
video
and
the
quiz
here,
and
I
think
what
we're
going
to
do
is
is
ask
subutai
and
jeff
to
kind
of
go
through
these
quiz
questions
we'll
see
how
far
we
get,
starting
with
the
first
one
and
kind
of
define
the
question
and
get
some
audience
participation.
If
anybody
wants
to
take
a
stab
at
answering
some
of
the
questions
and
then
we'll
have
some
discussion
about
each
one.
Does
that
sound
good?
G
H
Can
do
it
once
I
mean
this
is
not
it's
not
quiz
time?
This
is
office
hours
yeah
right.
I
think
we
just
thought
it'd
be
a
good
way
of
breaking
the
ice
or
something
yeah
just
spurring
some
discussion
sure,
but
if
people
have
something
they
want
to
talk
about,
instead,
we
should
let
you
know
that
as
well
right.
I
So,
hey,
so
why
don't
you
start
off
with
the
first
one
here?
Okay,
so
do
you
see
our?
Do
you
see
a
video
of
us
right
now.
D
All
right,
okay,
cool,
so
well,
hopefully,
if
this
works
out
well,
we
won't
need
to
answer
any
of
these
and
you
guys
will
know
the
answers
and
answer
everything
which
would
be
great.
D
D
So
just
as
a
background,
this
is
some
questions
that
we
often
give
to
our
employees,
who
want
to
work
on
the
algorithms
and
really
need
to
understand
some
of
these
concepts
in
detail.
We
go
through
some
of
these,
and-
and
you
know
I
don't
know
how
far
we'll
get
down
here.
But
at
least
some
of
the
basic
ones
are
are
pretty
cool
properties
of
the
spatial
pool
or
something
don't
worry
about
getting
involved.
Yeah
yeah,
don't
worry.
E
D
J
D
Okay,
yeah,
that's
right,
you
know
if
it's
a
random
hash,
then
what
a
random
hashtag
should
do
is
take
two
inputs,
even
if
they're
just
slightly
different
and
map
them
to
two
completely
separate
representations
and
the
cla
spatial
pooler
won't
do
that,
like
you
said,
if
it's
you
know
similar,
the
input
is
similar.
The
outputs
are
going
to
be
somewhat
similar
and
that's
actually
the
desired
problem.
That's
the
desire,
that's
what
we
want
yeah
and
if
you
know
about
stuff
like
bloom
filters,
you
know
we
often
look
at
bloom
filters.
D
Some
of
the
math
is
appropriate,
but
bloom
filters
are
a
random
hash.
You
know
they
will
try
to
spread
out
the
output
as
much
as
possible.
So
that's
good,
and
the
next
question
with
is
sort
of
a
consequence
of
that.
So
what
happens
to
the
output
of
the
spatial
pooler?
If
you
randomly
change
one
bit
in
the
input.
D
Okay,
yeah
and,
and
you
know
which
output
bits
would
change
you
get
if
you
have
a
42
40.
If
you
have
40
out
of
2048
that
we've
picked
as
the
ones
that
are,
you
know,
represent
the
current
sdr
and
you
change
one
bit
and
the
output.
Can
you
quantify
which
of
those
bits
would
change.
D
One
bit
in
the
input,
sorry
and
which
of
the
output
columns,
would
no
longer
be
active
and
which
would
become
acquired.
How
would
you
just?
How
would
you
know,
how
do
you
decide
which
one
of
those.
B
The
ones
I
think
that
the
columns
that
are
active
or
inactive
are
are
connected
to
the
input.
The
columns
that
are
connected
to
that
input
will
be.
B
By
the
change,
so
if
it's,
if
it
was
on,
if
you
turn
it
off,
then
the
ones
that
were
connected
to
it
and
are
active
are
less
likely
to
be
active
now
and
the
ones
that
are.
If
you
turn
it
on
from
off,
then
the
ones
that
are
connected
to
it
are
more
likely
to
be
active
when
they
were
not,
and
then
the
inhibition
will
cause
the
this
will
decide.
D
D
Yeah
yeah,
so
you
have
a
you
know
this.
The
columns
have
an
overlap,
score,
that's
computed
with
the
input
and
and
the
ones
that
are
affected
by
that
input
bit.
Their
overlap
will
score,
will
either
go
up
or
go
down
and
those
will
be
the
ones
and
hopefully,
the
ones
which
are
right
at
the
edge
of
when
you
pick
the
top
40
there's
going
to
be
some
which
are
right
at
the
edge
of
being
selected
and
not
selected,
and
if
they
in
you
know,
interact
with
this
input
bit.
D
D
Or
not,
if
you're
working
with
the
spatial
cooler,
this
kind
of
stuff
is,
is
very
helpful,
but
if
not,
then
it
might
not
be
interesting.
So
the
next
one
is
kind
of
was
part
of
the
focus
of
my
cla
deep
dive.
If
any
of
you
saw
that
at
the
hackathon
or
saw
them
so
the
question
is:
can
you
do
spatial
pooling
with
small
numbers
and
so
more
specifically,
is
it
reasonable
to
have
a
spatial
fulo
with
just
20
columns?
Will
you
get
the
properties
you
want?
D
If
not,
why
are
large
numbers
important
in
sdrs
and
so
there's
sort
of
two
specific
questions
we
can
ask
there
suppose
look
at
a
what's
the
difference
between
picking
five
columns
out
of
50
or
50
out
of
500.
Both
of
them
have
10
sparsity,
but
why
is
the
second
one
better
than
the
first?
You
should
have
a
little
buttons.
Oh
yeah,.
G
D
K
D
D
M
K
On
the
on
the
mailing
list,
the
other
day,
but
I'm
not
sure
it
was
related,
it
was
more
about
the
temporary.
D
Yeah,
it
is
actually
a
little
bit
related
to
the
email
that
you
send
out
they're
all
actually
kind
of
around
this.
You
know
the
number
stuff
is
they
all
kind
of
get
at
the
same
kind
of
concept?
You
know
I
want
to
take
a
crack.
J
D
Yes,
maybe
one
way
to
rephrase
that
you
know:
if
you
pick
five
columns
out
of
50,
then
you
have
50
choose
five
possible
outputs.
Is
that
what
and
if
you
make
500
out
of
50,
that's
a
much
bigger
a
50
out
of
500.
Sorry,
that's
a
much
bigger
set
of
possible.
H
I
would
just
go
into
the
google,
so
if
you
just
taught
five
choose,
50
you've
got
the
answer
and
you
can
choose
52
500,
500,
250
or
52
is
five
yeah.
You
just
typed
in
and
it'll
tell
you
that
you
get
it.
You
start
getting
a
sense
for
the
size
of
these
spaces
and
how
they
vary
yeah.
So
that's
a
good
one.
So
that's
a
good
one.
500
choose
50
is
a
much
much
bigger
space
than.
B
Determining
that
your
prediction
has
is
correct
or
incorrect,
like
if
the
actuality
is
is,
was
correctly
predicted.
D
Yeah,
that's
a
that's
exactly
right!
That
was
the
second
thing
I
was
going
to
get
into.
Is
that
if
you
pick
five
columns
out
of
50
the
chance
of
another
random
pattern,
having
a
false
positive
is
much
higher
than
if
you
did
50
out
of
500..
D
So
if
you
pick
50
out
of
500,
you
know
the
chance
of
another
pattern.
You
know
matching
closely
or
exactly
as
well
that.
D
B
G
It's
only
if
you
have
a
really
impoverished
world,
you
know
to
say
all
your
inputs
have
a
lot
of
overlap,
meaning
you
basically
don't
have
a
lot
of
different
inputs
and
we
actually
ran
at
the
problems
when
we
used
the
cla
with
actually
very,
very
small
spaces,
and
we
had
to
do
some.
D
Yeah,
so
if
you
have
very
impoverished
spaces,
it
you
know
doesn't
matter
too
much
and
the
higher
numbers
will
work.
Fine,
too.
Okay,.
A
So
we
have
a
guy
on
on
chat
that
doesn't
have
a
microphone,
so
I'm
going
to
butcher
his
name
anselmo
talona,
and
he
says
it
has
an
impact
on
the
resolution
of
your
result.
So
the
dimension
depends
by
the
accuracy
you
need.
I
guess.
D
Yeah,
so
the
comment
was
that
it
impacts
the
resolution
of
the
output.
I
guess
that's
another
way
of
saying
the
same
thing:
it's
a
much
bigger
space,
and
so
you
can
yeah.
I
guess
it's
a
slightly
different
point.
That's
that's
true.
If
you
have,
if
you're
picking
five
columns
out
of
50,
you
know,
if
you
can,
you
can
distinguish,
you
know,
similarities
up
in
gradations
of
one-fifth,
whereas
if
you
have
50
out
of
500,
you
can
make
finer
distinctions
of
the.
G
More
cancer,
generalization
and
and
catching
extinction-
I
just
did
the
numbers
here
so
50
choose
5-
is
about
2
million
and
500
she's
50
is
about
10
to
the
69th.
So
there's
a
big
difference.
It's.
D
A
big
difference
and
that's
one
of
the
really
important
properties
of
sdrs,
is
you
really
need
to
have
large
numbers
and
to
get
into
these
really
comfortable
spaces
like
10
to
the
69.
wow?
That's
a
small
one
yeah!
So
that's
the
first
one.
It's
kind
of
the
the
impact
of
the
number
of
active
columns.
D
The
next
one
is
what's
the
difference
between
picking
50
out
of
100
versus
50
out
of
a
thousand,
so
in
this
case
they
both
have
50
ones
coming
out.
So
if
you
turn
look
at
resolution,
for
example,
there's
no
difference
between
not
much
difference
between
them.
What
would
what
is
the
big
difference
between
these
two
50
out
of
100
versus
50
out
of
1,
000.
yeah,
and
this
gets
is
but
more
important.
This
is
closely
more
closely
related
to
the
temporal
pooler
issue,
because
it's
almost
the
same
question.
D
You
just
asked
right:
isn't
it
it's?
It's
yeah,
yeah,
okay!
Well,
these
both
have
10
sparsity,
whereas
here
the
the
number
of
active.
D
A
little
bit
yeah
with
you
know,
picking
50
out
of
100,
you
have
a
slightly
higher
chance
of
false
positives.
That's
true!.
D
That's
not
as
but
that
it's
an
impact,
but
it's
not
as
as
big
an
impact
as
the
other
issue.
B
The
50
out
of
1000
there's
more
sparsity,
which
means
that
only
the
best
call
only
the
best
columns
will
learn
to
represent
the
input
rather
than
a
lot
of
columns
trying
to
represent
the
same
input.
So
you
might
be
able
to
distinguish
between
inputs
better.
D
That
you're,
seeing
rather
than
maybe
yeah
I
mean,
maybe
these
50,
would
better
represent
the
input
than
the
50
out
of
100..
They
would
well
they
would.
They
would
learn
to
be
more
specific
in
their
representations,
they're
more
specific
in
their
representation
yeah.
So
that's
another
possible
thing,
because
that's
just
more
subtle
one.
D
What
I
was
thinking
of
is
actually
the
super
superposition
thing
that
I
talked
about
at
the
deep
dive,
which
is
that
the
way
that
temporal
pooler
works
with
sdrs
is
when
you're
making
predictions
in
the
column
space
you
can
actually
or
together
multiple
different
inputs
and
still
be
able
to
tell
whether
the
original
inputs
are
in
that
or
representation
or
not,
and
that's
one
of
the
ways
that
tempapula
can
make
multiple
predictions
about
the
future,
and
so
with
50
out
of
a
thousand
you
can,
you
know
you
can
or
together
many
more
patterns
than
you
can.
D
If
it's
50
out
of
100
and
the
capacity
the
ability
to
have
multiple
patterns
represented
simultaneously
increases
greatly
with
50
out
of
1000..
When
we
were
when
we
were
looking
at
vision,
I
just.
G
You
know
unbelievably
huge,
and
so
one
of
the
ways
to
think
about
this
is
that
the
more
columns
you
have,
even
if
you
only
have
50
active
at
a
time,
the
more
columns
you
have
each
column
will
be
able
to
represent
a
more
richer
coincidence
pattern
in
the
input
space,
and
so
each
one
will
be
able
to
represent
a
little
bit
more
discriminate
a
little
bit.
You
know
more
active
bits,
more
active
synapses
on
it.
You
can
only
go
so
far.
G
You
know
you
could
say
well,
I
could
just
have
a
gazillion
columns.
Each
one
would
represent.
You
know
a
totally
unique
input
space
pattern,
but
it
doesn't
work
that
way
the
numbers
don't
look
down,
but
anyway
the
the
more
you
have
overalls
and
other
benefits
that
each
coincides
with
the
is
represents
a
larger
coincidence,
a
more
more
active
synapses.
K
I
have
a
question
at
this
point
about
the
oring
together
of
sdrs
for
basically
yeah
superimposition.
K
I
believe
from
what
I
remember
from
the
original
literature
on
the
subject
that
different
mechanisms
were
suggested
for
superimposing
vectors
which
are
not
simply
or
but
it's
more
about
counting
a
number
of
active
bits
and
have
inactive
bits
in
in
the
original
vectors,
and
then
doing
decisions
based
on
majority
and
so
on
and
and
and
this
kind
of
super
position
would
remain
more
faithful
and
degrade
than
so.
What's
the
engineering
choice
that's
been
made
to.
G
Actually,
I'm
not
familiar
with
the
literature
on
this
and
we
didn't
come.
We
didn't
derive
this
from
literature.
I'd
like
to
see
those
if
there's
literature
on
this
property.
This
is
a
property
that
we
just
sort
of
discovered
and
realized
on
our
own.
I
think
so.
I'm
not
aware
of
other
attempts
at
this
and
the
different
approaches
for
doing
it
canary
talks
about
it.
A.
D
But
in
a
different
way
right
he
doesn't
talk
about
like
the
ordering
of
predictions.
No,
I
I
forget
he
might
have
used
xor.
I
don't
remember
exactly
what
operation
he
used,
but
he
you
talked.
G
G
And
I
think
biologically
the
oring
together
makes
a
lot
of
sense.
That's
it
fits
beautifully
the
so
I
know
that
I
I
don't
know
if
another
way
the
brains
could
do
it,
but
that
doesn't
mean
we
have
to
stick
to
that.
You
could
potentially
use
other
methods,
but
I
haven't
thought
about.
K
K
G
You
know
it's
funny
because
when
we
talk
to
people
who
study
like
sparse
coding
like
bruno
also
housing-
and
you
know
you
talk
about
this
property-
those.
D
G
D
I
think
what
they're
trying
to
do
is
they
do
this
l1,
minimization
and
they're,
trying
to
create
a
sparse
encoding?
That
is
almost
like
a
compression,
so
you
can
reconstruct
the
original
input
as
faithfully
as
possible.
That's.
D
Yeah
we
should
ask
him
and
of
course,
a
lot
of
the
deep
learning
folks
use.
You
know
the
bruno
type
of
smartphone
as
well.
So
that's
a
okay
that'd
be
great.
If
you
post
that
to
the
list,
maybe
we
can
skip
by
number
two,
but
that's
really
very
down
in
the
details.
D
The
number
let's
stick
with
go
to
number
three,
which
I
find
like
a
really
cool
property
of
sdrs.
So
this
question
is:
how
does
the
sdr
representation
of
input
a
in
isolation
and
input
b
in
isolation,
compare
with
the
sdr
representation
of
input,
a
overlapped
with
with
b?
What
do
you
mean
input
a
and
if
they
are
or
diagonal
or
just
or
together
yeah?
So
an
example
would
be
you
know?
Let's
say
you
have
a
horizontal
line
and
then
it
moves.
D
D
How
does
the
sdr
representation
of
the
cross
compared
with
the
horizontal
and
the
vertical
line
and
nice.
M
G
D
D
I
think
you
can.
I
think
this
is
a
very
cool
property,
so.
H
D
It
takes
a
little
bit
of
thinking,
but
let's
see
I.
B
I
do
have
an
into
intuitive
approach.
Okay,
I
thought
about
it.
I'm
not
sure
if
it's
correct
about
it
is
that
there
are
properties
of
there's
a
way
to
represent
a
horizontal
line
and
a
way
to
represent
a
vertical
line
and
a
cross
has
shares
some
of
the
representation
of
both
right
it
has
so
if
you
were
to
represent
a
cross,
you
would
be
representing
part
of
the
horizontal
line,
part
of
a
vertical
line.
The
part
of
your
representation
would
include
both.
B
So
when
you
represent
just
a
horizontal
line,
you
the
bits
learned
to
that
are
active,
are
basically
the
ones
that
best
represent
the
horizontal
line
and
same.
B
The
vertical,
but
when
you
have
them
both
together,
you
have
to
represent
it
with
the
same
number
of
bits
because
of
the
inhibition,
so
the
cross
representation
would
have
to
be
the
one
that
best
represents
the
cross,
which
is
the
combination
of
the
horizontal
and
vertical
line.
So
I'm
kind
of
thinking
about
it
from
the
perspective
of
the
best
bits
that
represent
the
input,
so
you
it
would
share,
it
would
have
some
of
those
bits
would
overlap
with
the
bits
that
represent
the
horizontal
line.
E
B
Think
of
it
from
another
bits
that
represent
the
horizontal
line
are
saying:
hey,
I'm
seeing
a
horizontal
line.
The
ones
that
are
representing
vertical
line
are
saying:
oh
I'm
seeing
a
vertical
line
and
then
they
fight
and
then
the
ones
that
best
represent
the
combination
both
of
them
together
ends
up.
So
you
have.
D
Something
so
let
me
let
me
just
rephrase
that
and
fergal,
then
we
can
get
to
your
point
sure.
Let
me
just
rephrase
what
you're
saying
so,
let's
you
can't
see
the
board
yet,
but
suppose
you
have
bits
0
through
39
that
represent
the
horizontal
line
and
you
have
bits
40
to
79
that
represent
the
vertical
line
by.
H
D
Horizontal
vertical
lines-
yes,
there
are
sub-sampling
from
that,
so
none
of
these
bits
are
exactly
exactly
right,
so
you
have
the
zero
and
third,
nine
are
the
ones
that
best
represent
the
horizontal
overlapping
and
through
79
best
overlapped,
the
vertical
line
and
day
one
and
chetan.
What
you're
saying
is
when
you
present
the
cross,
you'll
have
the
the
highest,
the
ones
that
were
the
best
overlapping
of
the
horizontal
line
from
the
first
40
20
of
those
say,
and
maybe
20
of
those
from
the
vertical
line
that
best
represent
the
vertical
line.
D
Some
number:
okay,
it's
actually
correct,
they're,
very
close,
so
purgal,
let's
get
to
you
and.
N
N
Yeah,
so
what
he
was
doing
was
he
said
that
if
you
have
just
one
neuron
that
represents
the
horizontal
and
just
one
neuron
that
represents
the
vertical,
then
the
answer
would
be
a
combination
of
both
neurons
or
both
of
those.
But
in
the
case
where
he
had
things
where
they
were
mixed
up,
that
it
was
exactly
as
cheating
had
said,
that
the
you
get
a
a
mixture
of
some
of
the
bits
from
the
first
representation
and
some
of
the
bits
from
the
second.
D
Okay,
no,
that's
yeah!
Okay!
So
that's
pretty
much
what
chetan
said.
So
I
think
it's
that's
also
not
quite
correct.
It's
actually
very
cool.
What
happens
so?
What
what
you
get
is.
So
you
have
the
40
that
represent
the
horizontal
line
and
you
have
40
that
represent
the
best
that
best
represent
the
vertical
line.
Then
you
have
a
whole
bunch
of
other
columns
that
didn't
win
up
right
and
they
represent
that.
There's
going
to
be
some
in
there
that
overlap
with
both
the
horizontal
and
the
vertical
line.
D
F
D
Right
and
so
what
will
happen
is
when
you
show
across
you
will
get,
let's
say,
a
third
of
the
ones
that
best
represent
the
horizontal
line.
Roughly,
they
will
win
out
a
third
of
the
ones
that
best
represent
the
vertical
line
would
come
out,
and
then
you
have
a
new
set
of
columns
that
are
going
to
come
in
that
represent
the
cross
together
that
that,
by
chance
overlap
that
by.
G
D
Represent
the
cross,
so
you
actually
get
three
different
things
in
that
representation,
something
that
represents
the
horizontal
line,
which
is
one
of
the
parts,
the
vertical
line,
which
is
another
part,
and
then
another
set
that
kind
of
represent
that
just
tall.
If
you
will
off
the
cross
because
it
it
represents
the
whole
pattern
better.
A
Hey
subatai
open
up
your
chat
box
in
the
top
left
of
the
hangout
window
and
and
somo
got
that
right
in.
C
Chat
there
we
go.
Oh
yes,
very
good
and
so
much.
K
D
So
I
think
this
is
a
really
cool
property.
Is
you
get
the
parts,
but
you
also
get
the
whole
in
the
same
representation,
and
you
can
actually
now
reason
about
so
you
had.
If
you
had
sequences
that
were
dependent
on
horizontal
lines,
you
would
still
kind
of
activate
them,
but
you
could
also
have
sequences
that
depend
on
the
cross
and
not
any
of
the
parts
by
themselves.
D
G
Really
cool,
if
you
turn
this
into
a
training
sp,
if
you're
learning-
and
you
show
it
these
three
patterns,
the
the
the
the
bits
of
the
columns
will
start
migrating
through
these
different
three
patterns
and
the
preference
there.
However,
how
often
you
see
a
hard
line
line
alpha
in
your
vertical
line,
however,
you
cross
will
dictate
where
you
emphasize
it.
So
if
I
show
a
lot
of
crosses
and
a
few
horizontal
vertical
lines,
then
you
get
you'll
end
up
with
a
bunch
of
costly
things
and
you
have
fewer
cooler
representation
of
the
horizontal
vertical.
H
To
the
the
high
precedent
to
high
prevalent
representational.
D
G
You
train
on
horizontal
lines,
moving
a
vertical
line
grouping
now,
so
you
see
you'll
see
a
cross
yeah
and
you.
N
Yeah,
but
I
think
the
important
thing
is
that
in
this
case-
as
if,
if
what
you're
saying
is
correct
and
I'm
taking
it
that
it
is,
then
what
it's
doing
is
it's
generating
a
truth
that
isn't,
in
the
sequence
of
the
training,
set
exclusively
yeah
yeah.
That's
generalization
right,
okay,
so,
in
other
words
it's
it's
postulating
that
if
it
sees
vertical
lines
and
it
sees
horizontal
lines,
it
accommodates
the
fact
that
maybe
the
two
can
occur
simultaneously,
even
if
it
hasn't
seen
those
things
occur
simultaneously.
D
G
G
You
know
I've
learned
a
bunch
of
animals
and
how
to
behave,
and-
and
now
I
see
one
with-
I
see
one
with
four
legs
and
and
fish
scales.
I've
never
seen
that
before
yeah,
so
I
would
say:
well,
it
probably
runs
and
it
probably
swims
yeah.
G
D
Yeah,
so
this
you
know,
another
kind
of
simplistic
way
of
saying
is:
maybe
the
whole
is
greater
than
the
sum
of
its
parts
exactly
and
the
part
says.
But
you
have
this
whole
thing,
which
is
different.
B
Actually
that
that
maybe
it
offers
the
origins
of
fiction
or
creativity
in
a
novel
setting
where
you
imagine
something
that
doesn't
exist
or
something
you've
never
actually
seen
before
by
combining
bits
and
pieces
from
all
the
things
that
you
have
seen
before.
G
G
The
word
binding
problem
applies
to
lots
of
different
things,
and
so
you
got
to
be
careful
if
you
use
that
term.
But
the
idea,
if
someone
says
well
you've,
never
seen
a
volkswagen
beetle
with
with
polka
dots.
So
you
know
how
do
you
represent
that
in
your
brain?
It's,
like
you
know.
If
you
try
to
do
this
with
structures
and
software,
it's
very
hard
to
do
that
because
you
know
you
didn't
anticipate,
but
here's
the
sdrs,
it's
trivial.
You
just
see
it's
kind
of
it's
like
in
here.
G
N
N
F
G
G
M
G
Well,
I
don't
know
I
would
no.
I
don't
think
I
think
you
can
don't
don't
take
it
too
far.
I
mean
it's
just
a
it's
an
analogy,
but
I
think
it's.
I
don't
think
I
need
a
hierarchy
to
explain
that.
Literally
it's
just
like
the
heart.
It's
just
like
the
the
horizontal
line,
the
vertical
line,
except
it's
a
vw
bug
and
you
know,
and
polka
dots.
B
M
G
G
A
nested
composition,
or
something
like
that
with
inside
this
str,
but
that's
I
mean
the
basic
idea:
sdr
can
be
a
novel
combination
of
features
that
have
never
been
seen
before.
In
fact,
in
a
real
brain,
every
input
brain
is
always
normal.
It's
never
true
twice
and
therefore
you're
always
doing
this.
D
Okay,
well,
that
was
those
are
probably
the
main
ones
in
here.
Some
I
don't
know
if
we
really
want
to
get
into
before
we
kind
of
went
over
some
of
this
already.
What
happens?
If
you
add
noise.
D
A
Do
you
want
to
open
it
up
for
for
questions
about
any
of
these
other
other
quiz
quiz
items.
I
Hey
subatai,
actually
so
looking
through
the
the
different
problems
that
that
you
guys
have
already
tried
and
experimented
with
new
pig.
How
come
spatial
anomalies
didn't
work
out.
G
I
mean
like
this:
I
guess
we
never
really
even
wanted
to
do
that.
It's
pretty
simple
right!
I
mean
we're
trying
to
go
the
opposite.
We
were
trying
to
deal
with
novel
inputs
and
know
how
to
handle
them
as
opposed
to
detect
that
they're
anonymous,
it's
more
like
hey.
We
have
novel
inputs.
How
do
I
get
this
to
work
with
with
and
not
look
not
looking
armless.
D
Yeah,
there's
a
bunch
of
you
know:
techniques
for
doing
spatial
anomalies.
You
know
what
were
you
thinking
when
you
say
it's
pretty
simple.
I
Well,
I
mean
you
got
in
your
sdr's
right
would
be
a
simple
comparison
to
compare
all
the
previous.
You
know
what
the
field
looks
like
you
know,
or
what
the
state
of
the
region
is
compared
to
what
the
input
looks
like
right.
D
G
We
never
we
never
even
thought
about
doing
that.
I
mean,
for
the
reasons
I
stayed
it
was.
It
was
the
we
were.
We
wanted
the
opposite
property.
We
wanted
novel
inputs
to
look
familiar.
I
mean
one
one
way
to
think
about
the
space
of
pool
the
way
I
just
thought
about
it
is,
is
you
have
you
have
an
input
space
of
some
dimension,
some
large
dimension,
and
it
has
some?
G
It
has
the
properties
of
sparsely
stupid
representations,
but
this
varsity
can
vary
quite
a
bit,
so
you
might
have
a
very
sparse
or
less
sparse
and
the
number
of
bits
you
have
is
some
number
of
bits
and
you're
trying
to
map
that
onto
another
sparse
distributive
representation,
which
is
say,
2048
columns,
and
you
want
to
have
a
fixed
varsity
and
you
want
the
fixed
parts,
because
you
need
it
for
the
spatial
pooler,
so
you're
trying
to
do
this
mapping
from
some
less
constrained,
sparse
representation
to
this
more
constrained,
sparse
representation,
which
is
the
output
of
the
spatial
cooler.
G
Again,
we
want
to
have
a
fixed
number
of
columns
and
we
want
to
have
because
that's
what
the
brain
has
fixed
columns
and
we
wanted
to
have
a
relatively
narrow
range
of
sparsity
and
what
the
spatial
puller
does.
It
does
a
very
logical
mapping.
In
fact
it
does
it
could
it
could
come
from
a
much
much
larger
input.
G
Space
you'd
be
coming
from
10
000
bits
of
which
you
know
2000
or
1000
are
active
or
something
like
that
and
you're
trying
to
map
it
down
to
2048,
of
which
40
are
active
and
so
you're
doing
this
a
typical
way
of
thinking
about
the
brains,
you're
doing
this
massive
space
reduction
and
you're
trying
to
map
it,
and
what
we
want
is
the
the
patterns
in
the
input
space
that
are
overlapping
leads
to
patterns
in
the
upper,
the
sp
that
are
overlapping
and
it
does
that.
That's
basically
what
it
does.
G
It's
a
it's
a
part:
it's
a
space
conversion
from
one
space
size
to
another
space,
five,
those
properties
and,
and
then
what
happens?
Is
you
go
up
the
hierarchy
it
does
this
over
and
over
again,
because
this
means
that
any
region
in
the
cortex
can
be
receiving
inputs
and
a
dozen
other
reasons
which
it
typically
does,
and
it
doesn't
really
matter
how
many
other
ones
you
got
you
don't.
The
brain
doesn't
have
to
worry
about
the
dimensions
of
all
the
input.
How
many
input
bits
are
going
to
a
region?
G
The
spatial
pool
says
you
give
me
any
number
of
input
that
you
want
I'll
figure
out
the
best
way
of
representing
them
in
a
logical
way,
with
the
property
that
the
overlapping
patterns
will
be
overlapping
in
the
output
of
the
sp.
It
allows
you
to
build
hierarchies
that
are
very
complex
and
messy,
and,
and
we
could
recalibrate
every
time
you
enter
a
new
region.
G
So
that's
that's
the
thinking
behind
the
spatial
pooler
why
it
came
about
and
what
what
its
purpose
was,
but
it
also
matches
what
we
know
the
biology
goes
to.
D
D
But
you
know
in
the
past
we
have
created
multi-level
hierarchies.
You
know
we
went
up
to
as
high
as
three
I
would
say.
I
think
two
we
got
working
reasonably
well,
I
think
three,
it
was,
it
got
a
lot
harder
and
we
were
trying
to
train
it
so
that
the
expeditional
levels
of
the
hierarchy
actually
added
some
benefits,
so
they
were
more
invariant
or
there
were
slower.
You
know
the
the
output
would
change
slower.
They
were
more.
K
D
Environment
in
times
more
invariant
in
time,
so
they
would
represent
larger
and
larger
concepts
in
the
input
space.
Where
concept
can
be
a
spatial
temporal
thing.
So
that's
that's
what
we're
trying
to
do
and-
and
we
know
in
the
brain
and
the
visual
system
in
four
levels,
you
can
do
really
complex
reasoning.
You
can
detect
faces
and
all
sorts
of
very
complex
objects.
D
So
that's
kind
of
how
many
levels-
and
I
think
it's
really
hard.
We
found
it
very
difficult
to
get
it
working
very
well
with
with
three
three
levels,
and
you
know
the
training
becomes
very
difficult.
You
need,
you
need
lots
and
lots
of
inputs
and
you
need
to
you
know
the
systems
run
much
slower.
So
you
know
finding
the
right
set
of
parameters
and
all
of
that
stuff
gets
that's
really
hard,
so
that.
G
Was
part
of
the
problem?
Part
of
the
problem
was
just
to
get
very,
very
slow
to
run
these
experiments.
Part
of
the
problem,
I'll
be
honest
with
you,
is,
I
think,
there's
aspects
of
how
it
works
in
the
hierarchy.
We
don't
understand
yet
so
we
have
a
you
know.
The
temporal
pool
is
named
the
temple
pool
because
it
does,
it
can
do
pooling
over
time,
meaning
it
could
create
stable
outputs
for
sequences.
That's
not
currently
enabled
the
new
pic
and-
and
I
and
although
it
theoretically
makes
sense
and
it
works
pretty.
G
Well,
I'm
I,
as
I
told
some
people
at
the
hackathon,
I
had
a
couple
side
conversations
I
think,
run
with
chettin
that
there
is
issues
with
that
and
we're
not
sure
exactly
how
that
works,
and
so
I
think,
that's
an
area
that
that
needs
exploring
more
it's
it's.
You
know
a
lot
of
the
parameters,
a
lot
of
the
constraints,
but
I
don't
think
we
know
the
exact
we
might
have
some
of
the
details
wrong
so
and
it's
kind
of
hard
super
time
saying
it's
kind
of
hard
to
do
the
experiments
to
explore
these
details.
N
G
Whether
it
was
getting
invariably
so
we
were
trying
to
create
artificial
data,
sets
that
we
at
least
know
had
the
right
properties,
so
we
could
make
sure
the
system
was
working
properly,
but
it's
very
difficult
to
create
those
rich
artificial
data
sets
one
time
we
actually
started
doing.
You
started
looking
at
3d
modeling
right
right
right
and
then
we
had
bill
atkinson.
He
was
doing
a
lot
of
photography
yeah.
There
was
a
lot
of
time
spent
on
this
and
it
just
was
very
difficult
to
do
so.
G
N
F
G
D
Went
so
slow
compared
to
that?
Well,
the
moving
is
a
whole.
Another
thing
is,
you
know
with
a
baby.
I
think
it
can.
The
baby
can
move
and
and
go
towards
inputs
that
it
hasn't
seen.
Yet,
whereas,
if
you're,
just
showing
images,
you
know
or
a
set
of
training,
you'd
be
showing
you'll
be
quickly
showing
it'll
become
very
inefficient
because
you'll
be
showing
over
and
over
again
stuff
is
already
learned,
yeah
and
so
the
percentage
of
time
it's
seeing
new
things.
It
gets
smaller
and
smaller.
That's
another
factor.
Yeah.
G
K
Yeah
unmuted
yeah
yeah
so
because
I
just
was
worried
that
that
this
officer
would
come
to
a
close
soon.
Yes,
the
iic
channel
is
still
part
of
officially
part
of
the
new
peak
community.
I
saw
that
we
lost
all
the
operators
there
and
I
didn't
see
a
lot
of
meta
stuff
hanging
out
there
and
the
activity
on
isc
was
also
quite
low
during
the
hackathon.
So
is
this
still
part
of
something
that
that
you
want
to
maintain
and
encourage
to
use,
I
mean
it's
a
question
for
matt
yeah.
A
Yeah,
so
I
definitely
want
to
continue
maintaining
the
irc
channel,
but
I
have
not
had
time
to
be
on
the
irc
channel
a
lot,
especially
during
the
hackathon.
I
was
engaged
pretty
fully
with
a
bunch
of
other
stuff
scott
and
I
were.
A
First
started
everything
and
we
had
some
good
conversations,
but
unfortunately
we
we
didn't
get
any
logging
in
place
and
our
chat
bots
died
when
the
aws
instances
for
our
own
product
were
killed.
So
we
I've
put
a
call
out
for
anybody
who
is
interested
in
hosting
our
chat
bots
to
maintain
that
irc
channel
I'd
be
welcome
to
help
us
out,
but
I'm
I'm
planning
on
continuing
to
maintain
it.
I
don't
want
to
let
it
go
and
I'm
gonna
try
and
have
a
larger
presence
there,
I'm
actually
on
it
right
now.
A
Sorry
yeah
that's
actually
really
useful
to
me
it
because
it's
it's
an
indicator
that
some
that
there's
activity
going
on,
and
so
whenever
I'm.
K
A
The
chat
room-
that's
that's
what
I'm
looking
at
so
I'd
hate
to
move
that
we
might
want
to
have
a
new
dev
chat
room
as
well
as
just
a
general
community
chat
room,
because
the
ones
that
are
actively.
A
Like
mark
and
scott
or
austin,
or
whoever
that's
pretty
useful,
just
to
see
as
in
real
time
what
what's
happening.
So
that's
something
to
think
about
making
another.
J
Rick:
okay,
thanks
you're,
aware
of
the
the
mailing
list
correct,
that's
quite
active.
K
Of
course,
I
have
posted
a
number
of
times,
but
I
feel
that
there's
some
barrier
to
entry
to
posting
on
the
mailing
list.
You
know
your
thoughts
have
to
be
well
thought
out,
and
you
know
it's
not
as
interactive
as
irc
I
mean
that's.
That's
the
whole
reason
why
we're
having
this
office
r2
here,
because
you
know
one-on-one,
one-on-one
and
and
face-to-face
and
and
interactive-
is
just
a
different
thing
than
than
email
right.
A
G
You
know
matt
tried
to
get
me
set
up
on
irc
and
we
we
didn't
really
succeed.
Matt.
G
You
know
if
I
could
turn
on
my
computer.
It
was
always
there
I'd
participate
but
didn't
seem
to
work.
That
way.
Give.
K
Okay
sounds
good.
I
have
one
more
comment
on
the
hackathon
that
took
place
two
weeks
ago,
which
is
that
I,
like
the
format
that
was
like
half
hacking,
half
presentations.
I
guess
in
part
I
I
took
part
back
in
2009
in
in
the
htm
workshop,
and
I
was
you
know,
a
2010
workshop
got
announced,
but
then
never
never
took
place.
So
this
this
hackathon
seemed
like
a
good
substitute
for
for
continuing
this.
K
This
series
of
workshops,
where
you
know
you
have
some
presentations
and
some
one-on-one
conversation,
and
so
if
future
hackathons
are
not
just
about
hacking,
but
also
about
discussing
theory
and
these
things
that
that
would
be
something
I'd
be
very
interested
in.
Thank
you.
That's.
G
A
good
input
I
mean
we
thought
about
that.
You
know
after
we
realized,
we
had
a
bunch
of
presentations.
I
said:
well,
it's
a
little
bit
like
like
a
workshop.
You
know
and
we
were
worried
that
people
would
be
just
you
know,
might
not
get
any
hacking
done
because
they'd
be
listening
to
all
the
talks,
but
I
hear
from
you
ricky
thought
it
was
a
good
balance.
So
is
that
yeah?
So
maybe
we'll
try
to
strike
that
balance
again
in
the
future.
I
think
you
know
one
of
our
math
goals.
G
One
of
our
goals
is
also
to
to
make
it
easy
for
people
to
get
productive
hacking.
It's
still
pretty
hard
to
get
started,
so
you
know
so
you
might
be
able
to
say.
Oh
yeah,
I'm
gonna
spend
a
bunch
of
hours
listening
to
people
talk,
but
I'll
still
be
able
to
get
going
really
quickly.
On
a
project
where
you
know
some
people
still
just
spend
hours
just
getting
installed.
B
Any
other
pretty
high
level
question
about
the
whole
theory.
Is
there
anything
known
anything
at
all
about
goal
seeking
behavior
about
how
I
mean
it's
a
prediction:
inference
engine,
but
is
there
any
component,
that's
known
about
it
over
how
it's
a
goal
seeking
engine?
Because
that's
the
way
we
sort
of
learn
unsupervised,
because
we
we
try
to
achieve
something
and
then
we
fail
at
it
like
trying
to
return
a
ball
in
tennis
and
you
just
get
better
automatically
by
trying
more
and
more
times
until
you
reach
your
goal.
B
G
Your
questions,
I
assume
you
mean
anything
known
in
the
cla
world
yeah,
so
I
mean
to
me:
that's
a
that's
an
important
part
of
the
century
motor
integration
problem
and-
and
we
did
have
some
conversations
about
that-
the
hackathon
about
that
and
there's
a
lot
of
psychological
literature
and
some
neuroscience
literature
on
this,
which
we
talked
a
bit
about
it's
probably
more
than
we
want
to
get
into
here.
G
But
I
think
this
is
an
open
problem
when
we
say
okay,
we're
going
to
add
motor
behavior
we're
going
to
make
a
sumo
region
that
has
a
cla
for
inference
and
also
for
generating
behaviors.
Like
I
talked
about
in
the
presentation,
then
you
have
to
you
have
to
address
that
question
about
goal
seeking
behavior
one
way
or
the
other.
Ideally
you
know.
We
know
that
in
the
brain
there's
some
complex
mechanisms
that
are
subcortical
that
evaluate
different
motor
behaviors
in
the
basal
ganglia
we'd,
rather
not
try
to
have
all
that.
G
We've
got
to
come
up
with
some
simple
way
of
handling
this
and
ian
danforth,
and
I've
had
some
conversations
about
this
and
how
we
might
go
about
it.
So
I
guess
I'm
saying
we
don't
really
have
any
answers
to
this.
We
have
a
lot
of
clues.
It
really
makes
sense
to
address
it
when
you
start
adding
a
behavioral
component
and
and
it's
it's
a
field
that
has
yet
you
know,
there's
lots
of
speculation.
We
don't
have
a
good
answer
to
it.
Yet,
okay,
we
can
speculate
on
it.
I've
been
thinking
about
it
thanks.
G
You
know,
by
the
way,
just
just
give
you
a
flavor
there's.
It's
generally
believed
that
there's
two
ways
of
people
learning
this
kind
of
stuff
one
is
a
goal
which
is
essentially
to
a
goal
to
sort
of
increase
the
entropy
of
the
data
you're
getting
is
to
say
you
know
you
have
a
goal
to
explore,
and
so,
if
you're,
not
learning
new
things,
you're
not
happy,
so
that's
one
type
of
goal
and
and
then
there's
another
goal
which
is
like.
Oh
no.
G
G
I
don't
have
to
have
a
specific
goal
like
you
know,
eat
something
I
just
need
a
goal
to
learn
and
therefore
my
my
behavior
should
be
built
around
ways
of
increasing
my
knowledge
of
the
world
as
opposed
to
solving
a
problem
that
would
be
a
that
would
be
my
first
inclination
to
try
to
tackle
that
problem.
N
F
N
G
G
N
G
N
G
Psychological
literature
about
this,
which
I
don't
always
find
too
helpful
but
yeah
it
talks
about
these
two
and
and
speculations
on
the
different
motives
and-
and
I
know
one
researcher
at
berkeley-
who's
studying
the
mathematics
of
this
fitz
sommer,
s-o-m-m-e-r
and
he's
got
a
paper
out
recently
about
mathematical
approaches
to
increasing
that
sort
of
learning
rate
type
of
thing.
G
So,
but
you
know
in
the
end
we
have
to
build
stuff
in
the
community
and
we
gotta,
you
know,
make
things
work
and
I
think
we
don't
get
too
theoretical
about
it.
J
Get
off
of
riffing
off
of
that?
What
what
would
could
we
imagine
what
would
be
the
simplest
way
to
implement
some
kind
of
a
feedback
loop
where
the
new
pick
has
some
control
over
its
input.
G
G
You
know
two
two
class
attached
to
each
other
one's
associatively,
linked
to
some
to
a
motor
behavior,
and
I
you
know
to
me
it
would
be
like
okay,
create
a
system
that
has
two
degrees
of
freedom
and
you're
just
and
it
has
some
built-in
you
know
built
in
a
pollution
avoidance
type
of
thing,
and
now
I
want
to.
I
want
to
control
that
and
make
it
explore
some
virtual
world.
G
That
could
be
pretty
simple,
so
I
think
it
could
be.
You
know
you
could
start
some
very
simple.
You
know
exploratory.
You
know
basically
learning
a
maze
if
you
will
at
least
a
set
of
rooms
and
building
a
model
of
it.
J
Or
in
a
practical
application
like
you,
are
with
energy
systems
and
so
forth,
if
the
you've
got
a
massive
amounts
of
data,
that's
too
much
to
put
in
it.
K
G
J
G
I
don't
know
how
to
you
know,
I'm
sure
this
is
going
to
be
big.
I
just
don't
know
how
it's
going
to
play
out
yet,
and
so
I'm
personally
I
mean
I'm
not
trying
to
tell
anyone
else
to
do.
But
personally
I
would
I'm
going
to
look
for
a
very,
very
simple
problem
that
I
can
get
my
head
around
completely
and
just
understand
exactly
what's
going
on
before
I
branch
out
on
something
practical
I
feel
like.
I
just
need
to
do
that.
That's
how
we
that's
how
we
did
the
cla
too.
A
Left
we
could
probably
take
one
more
question
anybody's
up
for
it.
A
A
So
I
apologize
to
anybody
who
didn't
make
it
in.
Did
they
hear
you
saying
this
if
they
didn't
make
it
in
if
they
watch
youtube?
Okay,.
J
F
L
F
J
A
D
Yeah,
it
would
be
nice
to
have
more
than
10
people
available,
live
as
well
yeah
yeah,
so.
G
Yeah,
you
know
we're
just
we're
still
learning
how
to
do
all
this
stuff
so
and
we're
still
learning
how
to
do
it.
D
A
Yeah,
we
should
probably
also
try
and
get
it
get
irc
up
and
get
people
on
irc
to
chat
and
ask
questions
while
they're
watching
the
youtube
video
but
I'll
I'll
look
into
this
offline.
Thank
you
jeff
and
subutai.
Very
much
and
thanks
everybody.
A
Yes,
so
we'll
work
we'll
continue
working
on
this
and
continue
trying
to
engage
with
you
guys
as
much
as
possible.
See
you
on
the
mailing
list.