►
From YouTube: Temporal Pooling Fireside Chat with Jeff Hawkins
Description
Recorded at 2014 Spring NuPIC Hackathon
C
B
B
So
I
was
asked
about
temple,
cool
and
so
I
thought
I
would
just
as
real
special
movement.
That's
question
can
be
helpful.
Do
you
have
a
question
for
me,
so
I
thought
I
start
off
with
what
the
idea
of
cooling
is
a
gentleman
focus
real
basics.
So
imagine
this
has
been
done
many
times
you
moon
is
a
waitress
goals,
open
you're,
ready
to
earn
surgery
and
the
recording
fees
element
in
the
high
level.
Visual
cortex
and
animate
show
pictures
of
this
person
and
they
find
out.
B
There's
a
cell
that
responds
to
celebrity
like
I,
go
click,
and
so
any
time
she'll
picture
Bill
Clinton
to
this
person
to
sell
attacks.
It
doesn't
matter
what
capitulated
way,
we're
working
on
the
picture
the
cell
becomes
active,
and
but
if
you
look
at
somebody
else,
they
tell
doesn't
matter
so
it's
a
girl
friend
zone
and
that
those
self
exist.
Sometimes
they
find
that
it's
like
okay,
only
when
it
opens
looking
sideways
as
a
post
poet.
So
now
we
can
understand
that
cells
being
part
of
a
sparse
root
of
representation.
B
But
the
point
of
this
thing
that
bill
Klem
cell
is
staying
on,
while
your
eyes
might
show
the
picture
of
a
dope
n
to
the
person
every
time
there
is
movement,
probably
the
new
innovations
coming
around
one
and
if
you
show
up
it
to
third
picture
completely
alliteration
coming
up
on
to
put
renal
cortex.
So
pooling
is
the
idea
that
you
have
all
these
different
patterns
that
map
onto
a
relatively
stable
pattern.
You
know
so
you
could
say
here's
that
is
the
stable
pattern.
B
B
That's
the
basic
idea
is
fully
it's
saying.
Multiple
pattern
get
relatives,
mobile,
relatively
changing
to
relatively
stable
and,
of
course,
you'd
be
trivial.
To
do
this,
if
you
just
say
turn
all
these
cells
in
the
model
the
time,
so
we
want
stability
with
uniqueness
right.
We
don't
want
this
all
some
things,
I'll
stay
on.
That's
true,
but
you
know
we
want
to
be
stability
with
uniqueness
of
a
show
bill.
Clinton's
liability
meters
are
so
the
other
one
personally
publisher
to
her
Aniston
and
so.
B
B
You
could
group
two
patterns
together
and
say
they
the
same
thing
but
they're
only
doing
because
I
share
the
same
bits,
so
I
haven't
I
have
a
picture
with
a
bunch
of
bits
on
it,
and
I
could
just
turn
some
of
the
kids
off
at
some
of
the
bits
on
that
would
be
those
too
proud
interviews
very
similar
because
they're
sharing
northeast
and
bits.
So
it's
a
very,
very
it's
a
Hamming
distance
type
thing.
Basically,
there's
a
facial
overlapping
team.
Two
patterns
can't
do
much
with
it.
B
B
It's
an
important
step,
but
it
turns
out
that
even
if
you
don't
train
the
space
cool
and
you
use
a
fixed
one,
it
works
pretty
well,
so
you
don't
get
a
lot,
it's
beneficial
to
train
doesn't,
but
you
know
it's
not
the
most
critical
thing,
the
critical
thing
for
learning
and
infants
in
this.
By
the
way,
it's
just
an
inference
process.
B
You
take
a
bunch
of
patterns
and
producing
a
comment
out,
but
that's
information,
I'm
also
a
current
in
temples,
and
the
idea
here
is
that
if
in
yours,
if
I
consider
these
sequences
that
these
patterns
over
time
then-
and
I
can
predict
from
one
moment
to
the
next-
what's
going
to
happen
because
for
various
reasons
was
that
sequence
has
occurred
for
an
occasional
predicted.
If
I
can
predict
over
the
series
of
time,
then
we
basically
the
brainless
name.
Evolutionists
said
these
predict
abilities
means
they
must
have
an
underlying
cause.
B
It's
the
same,
therefore,
I
will
pull
the
ones
that
are
predictable.
So
what
an
unpredictable
I
want
to
have
something
stable
up
here
and
if
it's
not
particularly
that
I
shouldn't,
do
that,
because
it's
something
different,
that's
the
basic
idea
of
temporal
it's
a
numerous
scientists
have
come
to
the
same
conclusion
that
this
wasn't
the
way.
B
If
you
look
back
in
the
world
today,
I
it's
not
where
they
started
from
oh
and
thought
about
this
big
inning,
but
the
party
people
that
have
come
to
the
same
conclusion
that
temple
pooling
is
really
how
it
was
all
inference
works
entering
this
is
the
key.
So
when
your
eyes
are
moving
around
them
or
something
it's
not
shifting
an
image,
that's
not
what
it's
doing!
It's
a
pletely
new
pattern.
Every
time
your
eyes
move
completely
new,
it's
distorted!
It's!
It's
not
a
transformation!
It's
a
different
pattern.
B
So,
as
you
Sekot
over
my
face
or
my
body
you're,
getting
a
completely
different
set
of
things,
but
somehow
your
brain
is
able
to
say
predict
from
12,
not
evil.
What's
its
kind
of
what
is
going
to
see
and
if
it
does
enough
of
a
prediction,
it
says:
okay,
I
wanted
a
stable
counted
over
here.
For
that,
that's
Jeff,
or
that's
this
way
to
work
or
something
like
that
any
restaurant
owner
is.
B
So
you
should
hopefully
have
at
least
have
a
guest
of
that
tail,
because
I
said
look
as
long
as
I
can
predict
the
next
element
here
right
then
I
can
prediction
that
supplement
that
I
want.
You
took
the
pooling,
so
one
way
to
predict
the
next
elements.
Learn
sequences
of
these
things
right
and
that's
that
was
a
there's.
There's
two
basic
types
of
temple:
court.
B
That's
the
first
type
and
there's
simplest
way
to
think
about
this
way.
This
is
not
what's
going
on
when
your
eyes
moving
okay.
So
let's
don't
think
about
that
one,
but
this
would
be
something
like
imagine.
The
simplistic
example
here
would
be
on
melody
and
these
are
select
notes
in
the
melody
and
it's
silly,
the
sooner
it.
But
if
you've
learned
this
melody
connects
off
the
next
up
the
next
on
the
next
on
the
next
hour,
then
you
might
say:
oh,
that's,
that's
a
pattern,
I
hurry,
or
is
that
here's
another
one?
B
Here's
a
I'm
going
to
say
I'm
going
to
say
a
word
variation
kept.
Now
you
probably
don't
recognize
that
whatever
shindig
that's
three
syllables,
but
if
I
reverse
the
order
who
might
get
dictionary
right
so
dictionary
is
a
sequence
you've
heard
before
you
were
able
to
predict
and
you'll
form
a
stable
representation
for
that,
but
area
should
dick
does
one?
Do
you
like
the
hell?
B
B
B
B
B
Plus
you
have
a
copy
of
whatever
motor
command
was
just
executed,
copy
of
motor
commands,
meaning
I
see
you
haven't
the
cortex
gets
it
then
you
can
make
the
prediction
and
I'm
not
going
to
walk
through
this
right
now,
but
basically
we've
argued
that
in
the
cortex
this
is
what
they
are
three
is
doing,
and
this
is
what
layer
4
is
doing.
These
are
the
two
basic
types
of
sequence
member
you
can
do.
B
You
can
have
a
high
water
steam
looks
like
a
melody
or
you
can
have
a
city
which
is
not
a
high
or
to
see
because,
as
you
can,
it's
not
always
repeatable
sequence
of
elements,
but
if
you
know
what
the
copy
of
it,
you
know,
what
behavior
you
do
like
I
knew
my
eyes
and
relaxes
amount
to
write.
This
around
I
can
still
make
the
correct
prediction,
and-
and
if
I
can
make
the
correct
prediction,
I
can
form
a
stable
pattern.
Up
here
did
I
answer
is.
B
B
Imagine
if
I
have
a
stable
pattern
here
that
stabled
matter,
where
I'm
going
on
the
image
and
how
my
eyes
are
moving.
That's
invariance
right!
It's
a
doesn't,
really
matter
what
the
time
is.
There's
there's
there's
a
lot
to
this.
Well
on
your
generalization.
Got
the
business
isn't
about
the
hierarchical
kind
of
stuff.
I!
Don't
want
to
go
there
with
you.
The
today's
just
talk
with
us
to
talk
a
bit
about
what
tempo
polling
is
and
how
we
are
going
to
implement.
So
that
was
the
first
part.
B
Is
so
I
just
thought,
it'd
be
good
to
everyone
understand
what
the
hell
pulling
it
I.
Don't
you
know
that
that's
what
this
is
a
poignant.
It's
multiple
patterns
get
mapped
to
a
relatively
stable
pattern.
I
think
relatively
it
does
not
be
perfectly
stable.
The
parts
of
it
keep
changing.
It's
all
about
increasing
stability
as
you
go
up.
The
hierarchies
increasing
stability
over
the
tune.
B
B
B
Would
say
it's
either
high
or
signal,
and
maybe
someone
who
knows
just
to
talk
me
out
on
this
one.
It's
it's
not
sustained,
given
this
I
can
take
this.
It's
saying,
given
this
plus
the
motor
command,
which
is
about
the
magic
you
I
can
predict
this.
Given
this
cost
about
humanity,
actually
I
predictive,
you
know
in
the
CLA,
if
you
notice,
when
you
make
a
successful
prediction,
it's
much
more
support
by.
We
are
with
oneself
economy,
so
you
go
from
the
column
of
representation
to
a
seems,
only
converse,
you
get
a
very
sparse
representation.
B
When
you
make
a
successful
prediction,
you
get
very
sparse
and
that's
we
see
that
in
the
brain,
so
it
doesn't
matter
which
of
these
you're
doing
when
you're
in
this
one
or
this
one,
you
have
same
basic
affect
the
next
representation.
If
you're
able
to
successfully
predicted
to
can
be
very
sparse
in
one
case,
you
can
predict
it
just
by
knowing
the
sequence,
the
other
one
you
had
to
say
no
I
need
to
know
where
I
was
plus
the
motor
command
here,
where
I
was
plus
the
motor
command.
H
B
Yeah
this
one,
basically,
so
they
could
you
write
in
that
sense.
Okay,
I'm,
misunderstood
your
question.
Change!
Sorry!
No!
Yes
in
this
one,
you
could
be
affected
by
the
entire
sequence
right.
What
stayed
here
could
be
dependent
or
something
that
here,
but
in
the
second
one
here,
actually
not
it's.
It's
really
basically
saying
no!
It's
like
you
know
what
I
now,
where
I
am
now
and
the
statement
now
and
where
I'm
going
to
go
kind
of
it
occur.
C
G
B
B
Thank
was
the
world's
looking
for
thank
you
for
life
ins.
We
go
is
over.
Thank
you.
Thank
you.
Super
touch
kind
of
the
world's
like
that
aim.
It
discreet
it
doesn't
make
me
comfortable.
These
are
not
discrete.
States
trains
are
distributed,
sparse
distributed
states,
and
so
he
doesn't
be
careful
about
falling
into
that
trap
of
thinking
about
okay,
here's
a
state
machine.
I
could
put
them
on
a
graph.
It's
not
like
that.
It's
everything
all
the
time
is
there's
a
distributed
pattern
that
some
parts
of
our
be
predicted
in
some
parts
can't
be
predicted.
B
D
B
B
So
you
know
sometimes
you
can
select
conceptually
for
well.
You
could
say:
okay,
final
x
games,
you
may
help
you
think
about
right,
but
you
gotta
be
careful
not
to
not
to
start
thinking
about
that
way
too
much
about
some
of
this.
We
don't
really
understand
deeply.
Oh
I,
don't
know
have
all
of
us,
but
you
can
definitely
say
that
it's
very
rare
that
these
would
be
very
exactly
repeatable
things.
B
B
Yes,
there's
lots
of
evidence
of
synchronized
spiking.
You
don't
use
any
of
that
in
HTML,
CLA,
three.
Okay,
at
the
moment,
we
think
that
is
a
plumbing
issue
for
biology
and
it's
not
an
information
powering.
That
is,
that
is
not
an
opinion
that's
held
by
everybody.
Some
people
think
the
synchronization
is
the
key
to
consciousness
and
all
the
smell
of
the
stuff.
I
think
it's
bunch
of
nonsense,
but
it
could
be.
We
don't
know,
so
we
don't
need
to
synchronize
the
actual
spikes
reading
the
house
spike
sincerely
right.
B
E
You're,
coming
about
layer
for
being
a
copy
of
the
motor
command
coming
in
there
yeah.
Are
you
saying,
in
the
general
case,
throughout
cortex,
you
believe
that
layer
4,
has
projections
from
motor
for
length
smarts.
B
That's
what
record
everywhere?
Is
this
really
true
that
leg
for
is
getting
accomplished,
motor
commands
out
and
round?
Yes,
there
is
evidence,
and
but
I
need
to
be
more
a
little
bit
more.
You
want
to
know
that
I
need
to
get
a
little
bit
detail.
Okay
to
imagine,
have
a
hierarchy
of
cortical
regions
and
I'll,
just
so
too.
Okay
and
here
you're
some
sensory
input
down
here
it
might
be
preprocessed
by
some
cortical,
some
political
or
that,
and
it
then
comes
in
to
this
region,
their
role
everywhere
as
far
as
they
bought
their.
B
There
are
other
things
down
here
that
we're
doing
generating
behaviour,
and
so
in
brain
these
fine,
these
are
subcortical
Star
Trek
right
in
the
brain.
These
could
be
anywhere
from
the
spinal
cord
to
the
brainstem
tube
is
becoming
Leo
things
like
that,
it's
pretty
cool
interest
and
so
on,
and
these
guys
were
paid.
These
are
your
reflex
reactions,
you're,
walking
everything
you
do
and
these
guys
generate
outputs
which
make
things
happen
and
everywhere
you
go.
You
see
that
these
guys
split
off
the
same
action
split
up
in
the
calendar
quarter.
They
also
post
a
comment.
D
B
General
is
that
if
you
look
at
the
first
region
the
context
every
one
of
these
things
either
gender
and
behavior
will
split
their
at
rons
and
you'll.
Send
them
up
here.
So
this
guy
gets
both
sensory
and
motor.
That's
the
general
rule
and
I
think
there
was
some
discussion
of
the
nice
book
by
sherman,
jewelry
or
naturally,
but
very
sherman
position
show
mango
this
news,
demonic
what
they
have.
You
can
talk
about
this
very
nicely
in
there.
Rafa
diamonds
do
this
and
then
then
next
question
is
ever
be
to
the
cortex.
B
Well,
I
think
the
data
sure
is
a
lot
of
fun.
What
we
do
know
is
the
following:
is
that
some
of
the
lower
readers
so
get
some
of
this
stuff,
but
maybe,
as
you
go
higher
up
a
dozen,
but
we
do
know
is
it
follow.
Me
is
every
region
of
the
cortex.
When
I
talked
about
this
in
my
meat
of
talk
recently,
everybody
reason
the
cortex
has
cells
in
layer
5
that
generate
that
basically
project
should
tornado
slipper.
B
These
cells
project
and
projection
ease
generating
things
have
these
are
motor
generating
visual
motor
traders
and
so
every
every
region.
The
cortex
has
an
output
which
impacts
these
lower
level.
They
don't
drive
cells
and
look
like
neurons
from
cell
I
mean
dumped
our
muscle
structure.
They
associated
the
link
to
these
guys
and
so
every
reason
the
cortex
idiot
motor
complement,
which
goes
someplace
below,
but
also
these
things
then
project
off
to
the
next
region.
So
you
can
look
at
this
the
same
the
dis
region,
even
if
it
didn't
get
this
pathway.
B
This
region
is
getting
a
motor
copy.
Here,
it's
getting
a
copy
of
the
motor
commands,
remove
region
below
it
as
well
as
sensor
data
from
here,
and
that's
exactly
what
you
say:
we
see
a
direct
path
and
a
indirect
path
like
that,
so
I,
even
a
mite,
go
higher
open.
The
cortex
I
could
say:
yes,
it's
getting
motor
behavior,
but
it's
getting
them
from
the
region's
forward
and
below,
so
that
that
sort
of
hierarchical
parallelism
exists
but
subcortical
hands
in
the
cortex.
B
B
Cares
what
I
think
is
going
on
and
I
talked
about
this
basically
I
think
I,
don't
remember
where
I'll
be
going
around,
but
if
I
look
at
a
region
with
this
breaking
of
a
couple
of
spots,
you
can
say
here's
layer,
41
students
by
three
here's
way
for
and
then
let's
say,
there's
a
five
and
six
down
here.
Anybody
comes
in
basic
is
the
basic
schematic
are.
You
can
becomes
in
here
mrs.
sensor
data,
and
it's
motive
is
motor
cotton
or
bikinis
all
right.
B
So
that's
what
goes
into
labor
for
that's
the
primary
pathway
and
then
personally
or
projectile
83
and
my
screen
projection
a
parsec.
This
is
there's
a
canonical
circuit
and
the
cortex
is
not
simply
this
way
how
textbooks
describing.
What
I
think
is
going
on
is
that
you
come
in
here.
There's
a
spatial
pulling
going
on
when
it
comes
in
and
inverting
sequences.
B
Pulling
at
it's
just
learning
the
transition
temple
pulling
is
forming
the
stable
pattern
at
the
top.
This
is
just
learning
the
sequence.
Not
just
her
question
was
you
know?
How
do
I
come
to
stay
will
happen.
This
is
just
like
the
transitions
like
okay,
I.
Don't
want
time
to
count
of
three
predicting
right,
then
what
happens
and
you
get
to
the
when
this
goes
a
live.
Three,
you
do
temple
pulley,
/
spatial
pulling
and
then
now
high-order
sequences.
B
So
what
does
this
mean?
This
means
that
if
I
can
predict
the
changes
in
the
input
due
to
some
motor
command,
then
this
this
this
layer
will
do
a
good
job
of
predicting
it
and
then
I'm
stable
presentation
of
here
and
now,
but
if
I
can
achieve
the
change
we
could
pass
through.
So
you've
got
two
choice
and
get
two
chances
here.
This
is
happening
pretty
much.
B
Every
region
you
say,
can
I
make
up
kind
of
build
a
model
of
the
data
from
the
from
motor
commands
and
if
I
can
great,
then
I'll
look
for
the
changes
that
can't
be
predicted.
I
thought
a
command
or,
if
I
can't
believe.
Indeed,
even
a
modicum
and
I'll
try
to
find
all
the
changes
here.
Basically,
this
guy
gets
the
best
job
and
the
remainder
to
pass
something
and
the
media
content
to
John.
So.
H
G
G
B
Cortexes
of
action
is
a
sense,
it's
very
poorly
representative
in
cortex,
and
it's
not
already
is
what
you're
going
to
argue
about
this,
but
mostly
on
the
way
I
do
just
pull
faction
is
absolutely
nothing
to
do
this
I.
Don't
you
don't
have
it
it's
like
an
old
census,
very
poorly,
representing
projects.
There
is
no
primary
or
factory
cortex.
In
your
context,
there
is
a
whole
factory
cortex,
but
its
not
New
York.
Our
next
is
it's
a
different
structure,
almost
completely
so
the
very
little
temple
to
data
processing
manufacture.
B
But
you
see,
if
you
sniff
again,
you
spend
again
it's
not
the
sequence
of
patterns.
It's
just
over
and
over
again,
it's
almost
pure
spatial
patterns.
If
you're
trying
to
take
one
of
thousands
of
chemicals
combine
together
some
spatial
pattern-
you
probably
spatial
pullin,
probably
pretty
good
way
of
recognizing
motors
but
I,
don't
think
its
temporal
all
and
son
in
some
representative
quarter
hierarchy.
That's
you
know,
think
about
its
rarity.
It's
very
driven
the
sense
for
us.
We
don't
have
much
high
acuity.
B
B
B
H
B
Well
then,
it's
only
one
could
argue
that
in
auditory
space
one
could
think
like,
oh,
maybe,
all
four
spaces
less
motor
behavior
right
but
I
don't
know
if
that's
true
I
mean
like
what
mostly
what
I'm
hearing
right
now
is
based
on
my
own
bit,
my
favorite.
So
I'm
not
surprised
to
hear
you
know
if
you
ever
notice
that,
like
a
different
listens
like
you
scratch
your
head,
you
know
how
loud
that
is
it's
really
loud,
but
you
don't
hear
it
at
all.
B
G
B
F
B
B
F
B
The
motor
community
generalizes
attention
to
I,
don't
know
I,
don't
think
I.
Don't
we
had
a
long
conversation
about
some
of
this
earlier
I
think
would
be
unwise
to
go
down
that
pathway
right
now.
I,
don't
I,
don't
think
would
be
fruitful
to
have
that
conversation
right
now.
Okay,
I
I
think
its
simplest
way
to
think
about.
It
is
like
there's
a
this.
The
part
of
their
brain
called
the
superior
colliculus
that
moves
your
arms.
B
B
So
let's
not
put
anything
else
on
moment.
It's
just
physically
a
copy
of
a
motor
thing
and
by
the
way
by
the
basal
ganglia
and
it
might
project
to
the
brain
stem
and
which
might
projection
the
spinal
cord,
and
so
a
copy
of
this
gets
up.
Well,
this
is
the
more
higher
level
motor
command
right.
It's
done
here.
B
It's
driving
a
muscle,
it's
just
something
subcortical
that
is
basically
going
to
make
something
else
happen
somewhere
in
the
world
and
and
so
we
don't,
the
cortex
can't
know
about
all
that
stuff
can't
know
all
that
crap,
the
top
it
has
to
have
some
way
of
working,
regardless
of
what
the
rest
of
the
subcortical
stuff
is
doing.
Yeah.
G
B
B
Of
comments,
but
a
real
neuron
is
a
cell
and,
as
a
part
of
the
cell
coming
out
called
the
axon,
which
is
such
a
risk
in
tune
and
and
if
I
look
at
if
I,
look
along
that
acts
on
a
spike
is
a
is
a
temporary
spike
change
in
the
voltage,
the
differential
in
that,
and
it
moves
quickly
down
here
and
so
in
a
Cell
each
other.
It's
all
its
output,
it
as
a
spike
lotus
pond
again.
B
Another
spike
typically
do
a
train
of
these
guys
going
down
the
axon,
and
sometimes
it's
more
sputter
spikes
and
there's
rape
to
them,
and
things
like
that.
We
don't
want
that
at
all.
We
basically
say:
okay,
any
comments
generating
spikes
per
arm.
Okay,
spunk
areas,
its
binary
on.
We
also
have
another
state
which
almost
no
I
don't
think
any
other
neural
network
model
has
which
is
wheel
on
when
the
cell
is
not
generating
spikes,
but
it's
depolarized.
We
need
a
the
cell
is
in
a
state
people.
B
It's
in
it's
a
voltage
level.
Is
it's
raised
a
little
bit,
so
it's
almost
ready
to
spike,
but
it's
not
spiking,
that's
our
predictive
state.
So
this
is
something
that
we
we
do
so
we
have
a
couple
states,
it's
nothing.
It's
the
polarized,
it's
active
and
all
I'm
saying
is
that
we
do
not
distinguish
one.
We
do
not
model
spikes,
we
don't
models
by
timing,
respite
latency
or
spike
to
those
patterns
and
things
like
that.
Some
other
neural
network
models.
Do
then
that's
what
I
meant
by
that?
Ok,
no
more
spiking,
okay.
B
So
what
we
haven't
talked
about
yet
and
that
the
purpose
is
target
first,
we'll
describe
what
temple
pulling
it.
We
want
to
somehow
get
stable
patterns
away,
so
player
for
can
make
a
good
prediction.
I
want
to
get
stable
patterns
in
layer,
3
inflated,
three
convictive,
a
good
prediction.
I
want
to
get
stable
patterns
in
the
next
layer
that
in
projective
and
I
want
this
stable.
If
I
can
make
a
prediction,
I
want
to
be
able
to
stable
the
neck
region
or
more
stable
and
actually
over
and
over
and
over
again,
whatever
the
pap.
B
It
doesn't
really
matter
which
way
we're
going.
But
as
long
as
I'm
going
upward,
somehow
forward
I
want
to
say,
if
I
can
predict
sequences,
then
I
want
to
have
more
stability.
The
next
11,
that's
what
tempo
cooling
it
separate
from
spacers
for
the
sequence
memory.
It's
the
next
thing.
After
that,
the
thing
we
added
this
last
year
was
realizing
that
temple
pooling
is
occurring
between
layers
ourselves
and
there's
evidence
for
this,
and
it's
never
occurred
to
me
before
think
of
it.
That
way,
but
so,
for
example,.
B
B
You'll
see
cells
that
start
out
that
they,
they
represent,
say
individual
cortex,
a
cell
orientation
like
this
and
then
you'll
see
the
cell
response,
no
matter
where
it
is
in
different
patterns,
so
the
cell
is
fun
if
a
little
mining,
your
visual
field,
it
says:
okay,
well,
that's
how
we
respond
to
the
lines
they
are.
They
are
they
over
there.
There
doesn't
really
matter
where
it
is:
that's
a
complex
cell,
so
you
start
out
with
this
online
orientation.
B
But
now
you
have
one
that
where
they
go
from
layer
to
layer,
3
I
find
yourself,
it
represents
it'll,
be
active
through
any
of
these
positions.
Well,
that's
temporal
pooling,
that's
basically
saying
I'm.
Taking
all
these
different
patterns
and
the
cells
going
to
be
active,
regardless
of
where
it
is.
So.
That's
that's.
Essentially,
the
output
temple
pooling
that
would
be
going
from
I
would
argue
from
going
from
Leah
for
delivery.
Then
the
next
thing
you
see
is
you
see,
cells
that
are
that
are
active
only
if
thats,
that
sequence
of
patterns
goes
into
particular
direction.
B
Those
are
directionally
sensitive,
complex
cells
that
will
be
output
of
layer
3.
That
would
be
essentially
saying.
Oh
here's
a
sequence
is,
is
a
high
order
sequence.
This
is
I
or
sequence.
It's
moving
in
one
direction:
it's
not
moving
the
other
directions,
not
jumping
around,
so
I
might
find
in
late
for
cells
and
say:
okay,
any
where
that
line
is
I,
don't
care
I'm
going
to
respond
active,
but
three
I
might
say:
oh
here's,
a
bunch
of
cells
only
when
it
was
in
that
direction.
That's
consistent
with
this
basic
concept:
okay,
yeah.
F
B
That's
particular
problem
about
melodies
and
different
keys
I'm
going
to
punt
on
it's
a
hard
problem.
I
haven't
really
quite
figured
it
out
or
I
had
various
hypotheses
over
the
years,
so
I
I
don't
want
to
be
wanted
to
uncomfortable
state
trying
to
speculate
on
it
right
now.
I
could
have
some
so
there's
lots
of
things.
We
don't
understand,
okay,
so,
but
it
does
explain
a
lot
of
other
embarrasses.
Okay.
This
is
a
form
of
events
right
here.
B
It
doesn't
matter
where
the
client
that's
a
form
of
embarrassed,
so
the
tournament
barents
and
very
reputations
I
use
it
in
the
blog
on
intelligence.
You
know
it's
a
concept,
but
we
have
to
be
really
precise.
What
we
mean
by
it
for
going
to
use
you
so
I
prefer
to
talk
about
simple
things
like
stability
and
sequences
and
prediction
that
are
really
hardcore
things.
I
could
say
exactly
what's
going
on
there.
All
right
I
want
to
wrap
this
up.
So
I
want
to
talk
about
how
it
is.
We
actually
do
temporal
pulling.
B
Okay,
exactly
we
don't
know,
but
we
have
some
speculations
in
the
original
in
the
original
ceiling
white
paper.
You
talked
about
a
mechanism
for
doing
it
and
we
implemented
that
mechanism.
We've
now
abandoned
that
mechanism
and
I'm
just
going
to
give
you
our
current
thinking
on
the
lumen.
Okay
and
I've
done
a
lot
of
reading
about
this
trying
to
find
biological
evidence
and
tell
us
exactly
me
how
much
fun
I
read
on
told
it
to
get
about.
A
month
ago,
I.
B
Years
I
think
I,
don't
think
I
registers
I
wrote
and
it
was
very
interesting.
I
read
through
it
and
I
said:
okay.
Well,
what
parts
am
I
getting
free,
massage?
Oh,
the
one
maniac
sure
was
pretty
good
I
thought,
but
there
was
the
part
where
I
talk
about
temple
pooling
I
didn't
use
the
word
type
before
I
didn't
know
that
name
we
didn't
know
about.
B
Some
are
stupid
representations,
but
even
like
columns
in
Miranda
stuff,
but
we
talked
about
I
talked
about
sequencer
me
and
I
talked
about
the
concept
of
temple,
calling
about
forming
a
stable
reputation
for
sequence
and
I,
said
into
Indian
on
intelligence.
I
said
you
know
what
we
know.
The
brain
has
to
be
doing
this,
but
there's
many
ways
it
could
be
doing
it.
B
We
don't
really
know
enough
to
know
which
way
it's
doing
it,
and
I
said
I
will
just
described
one
just
to
show
you
that
it's
possible-
and
I
described
one
in
the
book-
is
in
chapter
6,
where
everyone
gets
lost.
This
is
the
part
you
really
good
one
and
and
it's
good,
because
the
thing
I
described
it
almost
certain
is
wrong,
but
but.
H
B
Say,
I,
don't
know
it's
done,
and
this
is
just
an
exemplary
that
could
be
done.
So
even
back
then
I
didn't
know,
a
temple
pulling
I
had
no
really
idea
how
it
would
implement
and
then
our
first
implementation.
It
did
it
one
way
and
now
we
have
an
idea
for
how
to
do
it,
and
let
me
see
if
I
can
remember
it
right
here.
Sometimes
it
takes
a
lot,
but
these
things
click
back
into
my
hair.
Let's.
B
I
have
what
I
have
is
a
series
of
columns
you
can
think
of
this
as
later
for
and
some
region
of
the
cortex
and
then
I
have
another
series
of
columns.
Another
CLA,
if
you
will,
we
think
of
it
as
a
three
or
maybe
the
two
regions
of
hierarchy.
It
could
be
a
better
scene,
but
basically
the
information
is
coming
in
this
way,
and
it
goes
here
and
I
want
to
I
want
to
do
temporal
pulling
here
from
sequences
here.
B
B
Let's
just
talk
about
the
situation
where
I
can't
temple
pool
what,
if
I
can't
learn
in
a
sequence
here
now
remember
what
does
it
mean?
I
mean
I,
have
a
sparse
representation
here
in
column,
space
and
some
sales
active
and
that's
changing
all
over
the
time.
I
can't
draw
that
picture
out
of
you
afraid
today,
right
I'm
going
to
stay,
but
this
guy
is
not
predicting.
Then
we
say
all
the
cells
in
the
active
columns
are
not
write.
Something
like
get
me
and
and
what
we're
going
to
do
here
is
we're
going
to.
B
Predict
is
it
could
all
these
columns
of
bursting
and
what
we're
going
to
do
is
this
kind
of
pass
goes
through.
Literally,
these
are
the
dishes
later
three.
This
is
platform.
These
are
literally
just
the
same.
Columns
are
just
communing
up
in
that
region,
so
we're
going
to
pass
those
through.
So
this
is
the
simplest
situation
that
you
consider
is
saying:
okay.
Well,
if
this
guy
couldn't
predict
just
pass
out
to
the
next
guy,
he's
basically
do
nothing.
B
This
isn't
Noah
right,
but
now
that's
easy,
but
now
what
if
all
of
a
sudden
I
could
predict
the
next
summer?
So
let's
say:
let's
just
a
little
bit
and
we're
back
to
here
now
I
am
now,
let's
say:
I
have
a
new
input
and
I'm
able
to
put
into
that
to
infest
is
like
here
and
here
and
here:
okay
now
I
only
drawing
one
cell,
because
this
is
a
predictive
input
right.
So
that's
now.
B
F
B
It
means
I
me
how
this
guy
has
to
be
stable.
Well,
this
guy
is
changing,
and
so,
as
long
as
these
guys
are
very
sparse
in
predicting
this
guy
has
to
stay
the
same.
That's
the
whole
point
and
it
has
to
learn.
It
has
to
learn
that
all
these
patterns
are
occurring
down
here
are
equal
and
this
guy
has
to
represent
in
the
same
way.
So
again,
I
did
a
lot
of
research
trying
to
get
the
biological
evidence
on
this,
its
basic
Lucifer.
B
So
what
we're
going
to
do
and
I'm
not
sure
it
matters
if
you
get
exactly
why
it
or
not,
but
what
the
simple
way
we're
thinking
about
it
now
is:
ok
all
right
if
I
went
from
being
unpredicted
to
predicted
as
long
as
I'm
convicted
I
want
to
keep
the
same
pattern
and
I
want
these
cells
up
here
to
learn
to
recognize
all
the
patterns
down
here
that
are
predicted
in
sequences.
Like
you,
cells
appear
at
the
state.
B
They
have
to
learn
to
recognize
all
the
sparse
patterns
and
the
melody
down
that
this
was
a
melody
and
I
was
going
through
the
melody
and
I'm
going
through.
All
these,
and
they
said
I-
could
learn
to
recognize
anything
Amy
estate
that
melody
that
clear,
ok,
so
what
we're
literally
going
to
do
is
we're
going
to
say:
ok.
B
We
do
just
try
to
learn
the
recognize
these
patterns
from
some
subsampling
of
the
Act
ourselves
down
there.
So
all
these
cells
that
are
bursting
of
all
these
columns
are
essentially
representing
this
sequence
down
here
and
if
I
ever
got
back
into
a
state
where
I
was
any
one
of
these
patterns
on
your
existence.
I
would
invoke
this
pattern
up
here
and
this.
G
B
Of
stays
the
same
until
this
guy
is
no
longer
predictable.
That's
that's
the
end
goal
out
defriended
and
exactly
the
neural
mechanism
making
this
happen,
and
you
know
why
does
they
have
to
do
this
and
that's
a
little
tricky
and
I
don't
get
it
I
can't
fully
I
can't
prove
that
this
could
work
so
good
work,
I
can't
I,
don't
have
enough
evidence
to
say
I'm,
confident
this
is
exactly
what's
going,
but
it
solves
the
problem
that
we
want
to
solve.
It's
pretty
simple
and
solve
the
problem,
achieves
the
results
we
want
to
achieve.
B
B
Think
about
vision,
I'll
leave
you
on
this
point,
visions
tripped
up
so
many
people
when
you
actually
look
at
something
your
eyes
or
ciccotti
over
and
over
and
over
all
over
the
place
patterns
completely
changing
you're,
not
even
aware
right.
Your
perception
is
stable.
Your
perception
of
what
you're
seeing
is
stable.
In
reality,
this
bottom
stuff
is
changing
all
over
the
place,
but
some
of
you
goes
nothing
levels
up
in
the
hierarchy.
Three
or
four
levels
of
the
hierarchy
is
stated,
so
we
do
example
pulling
now.
The
tricky
thing
is
is
I.
B
Show
you
an
image
and
I,
don't
give
you
a
chance
to
sakana
all
I
can
do.
Is
one
image
and
you
it's
flashing
from
your
eyes
and
you
don't
have
a
chance
to
get
a
second
image.
You
can
still
do
a
lot
of
inference.
You
can
still
say
what
was
that
picture,
even
though
you
didn't
have
a
chance
to
go
ding
ding,
ding
ding
to
predict,
which
is
the
normal.
B
So
it
turns
out
that
if
you,
if
you
train
a
system
like
this
and
then
you
you
throw
it
so
you've
trained
it
over
the
over
Tsikata,
you've
trained
it
over
sequences,
and
now
you
just
flash
it
in
front
of
it.
What's
going
to
happen,
you're
not
going
to
get
a
sparse
representation
down
here,
you're
going
to
get
a
complex.
Are
you
going
to
bursting
because
it
was
unpredicted?
There
was
no
prior
state
and
boom
there.
It
is
well.
B
What's
going
to
happen
well,
even
though
this
these
cells
appear
learned
a
very
sparse
pattern
down
here,
this
input
is
a
superset
of
that
sparse
pattern.
So
maybe
it
was
in
this
cell
and
this
cell
and
this
cell
I
will
still
invoke
these
celibacy
cells
appear.
We'll
still
see
that
pattern
so
like
it's
like,
there's
a
whole
bunch
of
pattern,
superimposed
on
you
because
I'm
not
bursting
these
columns
and
it
might
it
possibly
might
invoke
some
other
interesting.
B
Is
you
but
unlikely,
and
soon,
as
I
did
a
little
test
on
this
already,
and
I
think
you
could
basically
right
you
saying
yes,
so
this
is
a
really
nice
result.
It
says
that,
even
though
I
learned
by
psychotic
and
interacting
with
the
world
at
least
envision
is
how
they
may
be
not
anywhere
else,
but
least
envision
after
I've
learned
that
way
and
I
just
flashing
phonezoo,
the
learning
that
was
occurred
through
ciccotti
will
actually
invoke
the
right
representation
here.
G
B
Several
others
at
the
same
time,
but
very
quickly
filters
out
to
say
nope,
that's
the
one
that
is
so
I
can't
expect
you
to
to
God
that
right
now,
I
just
want
to
point
out.
That's
a
nice
result
of
this
whole
mechanism
is
that
it
does
do
spatial
inference,
ongoing
vision,
problems
where,
but
it
shows
that
it's
a
subset
of
the
learning
process
of
psychotic
and
motion,
and
that's
that's
exactly
what
we'd
like
to
see.
We
want
to
be
able
to
explain
how
it
is.
B
D
So
on
that
last
point
it
seems
like
in
theory
it
should
be
possible
to
train
a
system
to
do
that
without
any
of
the
temporal
cool
line
right
and
you
don't
limit
yourself
any
specific
impulse
implementation
in
theory,
there
should
be
some
way
to
do
that.
Do
you
do
what
to
train
a
system
simply
to
recognize
an
image
without
having
to
use
a
temporal
process
to
training.
B
Well,
I
would
say
no
question
was:
maybe
people
here
you
know
seems
like
there
ought
to
be
a
way
of
training,
a
system
recognizing
image
without
temple.
Food
well,
I
think
you
know,
I
take
Google's
shown
that's
possible.
They
use
deep
learning.
Networks
with
you
know,
gazillion
things
into
images.
Somehow
they
got
some
basic
who
would
occur.
I
would
say
that
the
brain
that
is
it's
an
odd
thing
and
at
the
brain,
uses
temper
pulling
that
her
question
about
in
giant
representations.
B
This
is
the
mechanism
by
which
I'm,
very
confident
of
what
environment
comes
about
in
the
brain
is
to
temple
pool.
And
could
you
do
it
some
other
way?
You
know,
could
you
could
you
create
a
vision
system
that
recognizes
you
know
caption
dogs,
by
training
on
a
deep
learning
network
with
16
million
pictures
problem?
I
guess
you
can
I
mean
I,
don't,
but
that's.
D
B
G
B
B
Like
the
biggest
problem,
I
have
but
I've
been
this
huge
advocate
for
time
in
artificial
intelligence
and
neural
models
in
vegan
and
I
think
you
will
never
ever
get
close
to
machine
intelligence
if
you
do
not
fully
incorporate
time
into
your
thinking,
and
so
you
know,
our
CLA
is
part
of
a
framework
called
hierarchical
temporal
memory
and
that
framework
we're
now
able
to
plug
in
spatial
a
sensory
motor
inference.
We
are
now
able
to
plug
in
motor
generated
behavior
by
the
cortex.
B
B
Do
some
cool
things
in
vision
without
time,
yeah
I'm
sure
you
can,
but
that
is
not
getting
at
the
essence
of
machine
intelligence
that
you're
just
going
to
get
stuck
in
some
local
minima
someplace,
and
so,
if
you
just
just
think
like
all
right
could
I
have
a
system
that
doesn't
have
time
to
those
that
the
has
behavior
I,
don't
think
so
I
think
that's
impossible
and
can
I
have
a
system
that
does
auditory
inference
without
time?
No,
it's
not
possible.
So
vision.
G
B
Just
because
of
vision
and
vision
is
it's
forty
to
fifty
percent
of
the
cortex.
It's
dedicated
division
because
it
sets
a
huge
massive
memory
problem
to
solve
that
so
I
think
that's
a
really
red
hair
man.
We
don't
you
don't
want
to
get
down
that
path.
So,
when
I
talk
to
people
who
are
the
oiling
people,
I
say
what
about
time?
Well,
how
are
you
going
to
act
into
your
time,
makes
yeah
we'll
need
any
rate
time
you're
right,
but
we're
trying
to
do
that.
B
We're
stuck
on
it,
whatever
I
think
we
know
how
to
do
that
part
and
and
and
you
need
to
integrate
motor
and
you
need
attention
all
these
things
are
time-based
things
so
yeah
you
can
do
some
stuff
without
time.
But
why
bother
you
aren't
voting
machine
telligent
you
gotta
have
time,
that's!
That's
it
yeah.
J
J
B
G
J
B
Ok,
I
hope,
I
answered
the
question.
The
problem
sometimes
I'm,
not
good
at
hearing
the
question
you
said:
isn't
that
special
for
one?
What
I
decided
is
the
mechanism?
That's
a
spaceship
or
the
temple
pool
are
really
the
same.
Maybe
that's
what
you're
saying
it's
really
it's
an
extension
that
we
merge
them
together.
In
some
sense,
it
in
a
nice
sense,
you're
right.
In
that
sense,
the
saying:
okay,
this
guy's
littering
to
represent
a
bunch
of
patterns
but
notice.
Those
pants
are
no
longer
spatially.
B
Well,
there
are
also
going
to
be
bullying
basically
they're,
all
the
proximal
ones,
but
I
don't
want
to
talk
too
much
about
it,
this
one
because
they
don't
know
if
I'm
you
even
talk
what
you're
talking
about,
but
essentially
we're
extending
the
concept
of
spatial
pulling
with
the
proximal
dendrites
to
recognizing
multiple,
unique
on
orthogonal
patterns
down
here,
and
the
only
thing
you
have
to
do.
We
have
to
keep
these
cells
active.
We
basically
have
to
say
okay
when
once
the
self
becomes
active.
B
Active
they'll
keep
learning,
there's
long
as
I
keep
learning
or
learning
about
it
anyway
and
I've
done
the
math
on
this.
You
can
interdigitate
these
multiple
patterns,
all
these
different
condoms
on
the
same
kinetic
segments,
it's
not
going
to
get
confused,
so
you
don't
have
to
have
separate
dendrites,
for
this
is
reckon,
is
one
pad
or
the
other
time
you
just
throw
them
all
the
other
there.
You
know
separate
out
those
patterns
as
long
as
we're
very
sparse
in
the
arbor,
by
definition
the
first
course.
B
So,
if
your
question
is,
isn't
this
like
space
a
hole
in
it
before
I
like
it,
but
we've
extended
it
to
recognize
orthogonal
patterns
as
it,
because
it's
filling
the
role
of
God
from
one
region
to
the
next?
That's
it's
like
spatial
pulling
in
that
regard,
and
it's
determining
which
of
the
active
columns.
It's
like
spatial
in
that
regard,
but
it's
now
low
under
just
spatially.
Similar
patterns
is
now
temporal
predicted
secret
crackers
or
thank
God
okay
and
one
last
customer,
like
a
not
gonna
end
this
you.
I
Got
right
so,
if
we're
taking
the
region
port
right
now
we're
taking
it
looking
at
the
crowd
booing,
so
we
have
the
first
pump
triggering
the
second
one.
That's
reading
the
third
one,
because
it
seemed
that
this
ABC,
like
18
ABC
report,
then
when
ccp
thinks
his
aides
good,
trigger
beam
and
see.
That's
that's!
That's
temporal
bullying
by
the
region,
I
think
I,
don't
add
my
baby.
She,
okay,
so
hey
zippers,
calling
all.
B
Sparse
activation
with
arms,
but
it
said:
there's
no
column
a
there's,
a
there's,
a
pattern,
a
which
is
this
pattern
and
another
time
I'll
have
so
may
God,
say:
okay,
I,
get
this
column,
this
column
and
this
column
and
B
I
get
this
column
and
this
column
in
this
column
and
see
I
get
this
column
this
column
and
this
con.
Something
like
that
right.
So
my
question.
I
B
Not
technically
that's
secret
among
Syrians,
equus
memories,
yes,
I'm,
separate
I,
invited
I've
said
this
on
the
line,
but
unfortunately
we
use
the
term
temple
pooling
in
the
white
paper
referred
both
sequence
memory
and
typically-
and
it
really
should
be
separate,
so
the
sequence
memory
and
then
there's
temper
point
secrets:
memory
occurs
within
here:
learning
what's
ABC
and
the
attack
will
pull
into
the
courage
when
I
go
up
to
here,
which
is
saying
group
those
together.
My.
I
I
B
Ag
you're
absolutely
right,
and
that
is
another
talk.
I
really
think
I've
made
some
progress
on
that
how
this
guy
feels
got
to
this
guy
and
what
it's
doing,
why
it's
doing
that
and
I'll
do
that.
Another
time,
that's
a
lot
of
stuff
to
talk
about,
but
you're
right.
It
turns
out
that
yeah,
you
know
well
not
later
I
shoot
I'll
take
about
not
letting
through
the
way
before.
But
if
I
go
back
to
erase.
A
B
I
risk,
if
I
go
back
to
there
is
there's
really
no
feedback
for
money
through
that.
Fourth,
but
what
you
do
have
is
the
following:
this
is
what
you
really
have
you've
got
a
region
in
the
cortex.
We
just
got
layer
three,
four
and
five
and
six
for
this
thing.
You
got
another
region
which
is
hierarchy
below
it.
Three
or
four
five
be
another
asian
feeling
like
this.
They
have
three
four
and
five.
What
you.
B
Have
also
sticks
go
to
there
because
from
player
six
you
see
these
cells
that
massively
projector
hear
from
later
six.
You
see
the
cells
that
massively
projected
here.
There's
this
big
feedback
path
we
going
down-
and
this
is
a
few
third
pathway-
and
this
entire
region
tells
this
region.
What
to
expect
in
this
entire
region
tells
this
region
want
to
inspect,
and
I
think
I
understand
that
pretty
well
now,
and
that
would
be
another
talk.
I
B
Works
like
is
our
cells
and
layer,
sex,
all
of
them
wats
on
that
project,
and
they
spread
their
actions
right
broadly
overlay
a
one
year,
then
cells
in
layer,
3
and
five
have
what
are
called
apical
dendrites
to
go
up
into
layer
1
in
addition
to
the
basement
entrance.
So
when
we
model
the
remodel
cells
in
the
CLA,
we
just
talking
with
a
basil
filter,
that's
the
sequences,
but
this
these
other
dendrites
that
come
up
here
and
what
these
guys
are
getting
is
feedback
from
this,
and
so
what's
actually
happening
on
setting
one
sentence.
B
B
Union
of
all
STRs
in
the
sequences
that
are
represented
by
the
state
are
represented
by
this
data
stick,
so
it
would
be
like
saying,
I'm
telling
you
to
do
a
motor
command,
and
this
that
says,
do
this
motor
command,
and
this
is
going
to
take
all
those
later
five
cells
and
invoke
all
the
possible
motive.
Sequences
that
could
generate
that
behavior
are
all
going
to
be
default,
lies
at
once
and.
B
B
A
minute
predictor
state
shows
a
union
of
all
the
things
that
could
possibly
be
participating
in
this
stable
state.
It's
like
all
the
notes
in
the
melody
or
invoked
at
once,
we're
expecting
to
get
those
notes,
and
that's
not
a
good
example,
but
it
gorillas
a
better
one,
but
anyway,
and
and
and
so
now
these
guys
says.
Okay,
all
I
have
to
do.
I've
got
a
union
of
all
these
things
now
I
have
to
do
is
take
an
input
and
that
input
will
select
where
I
am
in
that
sequence
and
bias
it.
B
So
if
I
see
a
fuzzy
pattern
house
they-
oh
it's
the
grill,
because
it
looks
a
little
bit
like
a
gorilla
and
I'm
biased
to
see
a
gorilla
or
I'm
going
to
say,
okay
I'm
in
the
kitchen
and
to
get
the
drink
I
need
to
go.
This
way,
don't
be
like
that.
So
that's
the
basic
idea.
There's
this
feedback,
which
would
vote
communion,
just
the
opposite
tip
of
pulling
it's
a
good
way
to
end
this
up-tempo
food
is
taking
a
series
of
pounds
here,
form
a
stable
pattern.
B
You
now
check
the
stable
pattern
here
and
go
to
the
union
of
the
sequences
that
led
to
the
stable
pattern
so
that
I
now
can
take
an
input
and
uniquely
select
one
of
those
elements
and
until
I
actually
low
road
up
there
on
I
couldn't
believe
it
automatic.
But
I
didn't
understand
the
back
to
her
before
so
it
is
boot.