►
From YouTube: Brains@Bay Meetup - Continuous Learning
Description
What is Continuous Learning? Borrowing from a recently released survey [1], "Continual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective changes through time, or where all the training data and objective criteria are never available at once".
We will try to answer a few questions, and reviewing the existing literature is a way towards answering those questions:
- How is it defined in the literature?
- What are the most common approaches?
- How do current approaches relate to biology and neuroscience?
- What are the limitations and open questions?
- What are future research directions?
As a reminder, our meetups are journal clubs, so we invite participants to read on the topic beforehand and join the discussion.
https://www.meetup.com/BraIns-Bay/events/262647238/
Review papers:
[1] Continuous Learning for Robotics (https://arxiv.org/abs/1907.00182)
[2] Continual Lifelong Learning with Neural Networks: A Review (https://arxiv.org/abs/1802.07569)
Additional Literature:
[3] Overcoming Catastrophic Forgetting in Neural Networks (https://arxiv.org/abs/1612.00796)
[4] Continual Learning via Neural Pruning (https://arxiv.org/abs/1903.04476)
[5] Superposition of Many Models Into One (https://arxiv.org/abs/1902.05522)
0:00 Lucas Souza
1:18:09 Sam Heiserman
A
E
G
D
D
As
a
group
discuss,
the
two
papers
on
the
field
should
try
and
cover
they
say
if
they
are
so
what
I
did
is
I
got
a
field
surveys
and
a
few
papers
and
I
went
through
and
then
I'm
just
gonna
go
through
that
mainly
that
survey
and
some
of
the
other
stuff
I
read.
But
the
idea
is
that
whenever
I
say
here
is
just
your
spark
the
description,
so
it's
not
like
me
giving
a
lecture,
especially
because
I'm
not
an
expert
on
it,
so
you
probably
know
a
lot.
D
So
whatever
the
transfer
is,
what
is
continuous
learning?
How
is
it
defined
in
the
literature
where
the
current
approaches,
the
benchmarks,
the
research
directions,
and
mainly,
how
is
this
all
brain
related?
So
I'm,
not
a
neuroscientist
I
just
started
inventor.
So,
but
there
are
neurosciences
in
the
room
and
I
hope
they
can
point
out
to
how
it's
been
related
and
how
it's
not
so.
D
D
Probably
already
has
a
concept
of
what
Sadaqat
just
isn't
on.
It's
called
the
dog,
so
that's
Marlin
duck.
We
learn.
We
have
all
this
data
coming
in.
They
are
from
different
pasts,
who
can
say
different
distributions,
different
type
of
stuff,
and
sometimes
you
have
a
label.
Sometimes
we
don't-
and
this
you
have
this
continuous
stream
of
data
coming
in.
I
A
I
J
E
I
mean
I,
think
television
system
like
a
hero.
They
I've
learned
that
you
were
drinking
a
beer
right
now
and
that
type
of
cop
you
doing
that
constantly.
You
never
stop
right.
So
if
you're
gonna
be
how
the
presence
in
the
world-
and
you
definitely
remember
what
you
just
did
where
you
went
the
table
in
front
like
the
all
right
now-
I'm
it
wasn't
there
this
afternoon
and
to
that's
how
you
know
DISA
be
a
sort
of
perceptual,
an
agent
in
the
world.
E
E
D
K
D
D
L
D
J
Seems
also
up
to
me,
I
mean
these
ones
first,
like
in
creating
machine
learning
in
a
our
systems.
It
seems
odd
that
this
wasn't
one
of
the
things
that
they
kind
of
held
it
to
then
it
seems
to
be
worried
about
this
in
the
creation
of
multi-layer
perceptron,
deep
learning,
because
you
can
do.
Obviously
you
can.
You
can
do
continuous
learning
and
those
methods,
but
if
just
looking
at
the
way
that
information
is
spread
out,
so
it's
really
not
maybe.
E
F
E
J
J
A
Anything
that's
tractable
in
statistics.
You
need
to
have
an
idea
suction
round,
and
if
you
don't
have
that,
you
can't
really
derive
a
lot
of
the
self
yeah,
and
so
the
flip
side
is,
if
you
want
to
mathematically
an
analytic
to
understand
something,
you
kind
of
have
to
assume
their
idea
sumption,
which
is
completely
covered
to
continuous
learning,
yeah
I.
Think.
K
D
G
A
D
L
D
D
E
E
D
J
The
way
the
information
is
spread
in
deep
learning
system,
like
theoretically
all
of
the
information
is
spread
everywhere
in
the
network
yeah.
So
how
do
you
learn
something
new
and
spread
if
everywhere,
though,
something
else
that
spreads
everywhere?
Where
is
like
yeah,
this
part
of
the
network,
big
gross
business
party
members
response,
wasn't
that,
and
it's
not
so
I
had
that's
also
makes
it
black
box.
So
you
see
that
the
learning
happened.
The
basically
D
hesitation
happen,
but
refuse
to
say
so.
N
I
D
D
Want
the
label
form
in
order
to
maximize
our
learning
in
practice,
we
use
a
lot
of
active
learning,
but
there's
not
much
discussion
of
a
complete
etutor.
Other
related
things
is
online
learning.
That's
just
learning,
for
example,
to
be
stochastic,
waited
and
said
they
have
few
short
learning
and
you.
D
H
D
D
Just
going
to
go
through
tonight,
that's
it
so
curriculum
learn
is
learning
progressively
harder
task.
So
we
do
that
our
law
enforcement
learning,
so
you
want
to
make
a
Robert,
go
all
the
way
to
be
on
the
full
moon.
You
can
first,
you
first
learned
how
to
take
a
stab
and
then
learn
how
to
take
Chris
tap
and
then
I
learned
how
to
turn,
and
then
you
give
him
a
task
to
go.
D
I'll
be
recording,
so
that's
curriculum
learning,
so
you
have
a
better
learning,
which
is
you
learn
from
made
of
the
metadata
from
previous
tasks.
So,
for
example,
you
can
come
out
something
from
the
distribution
of
previous
tasks
and
use
that
somehow
to
guide
your
retaining
of
the
current
task,
transfer
learning
we
discuss
and
explain
so
all.
K
O
O
E
O
D
K
J
D
Some
of
the
tellings
in
the
field
or
a
catastrophic
forgetting
which
is
basically
you,
learn
with
us,
and
then
you
learn
new
tasks.
Let
me
forget
about
the
old
us:
it's
about
happens
in
transmitter
salon.
You
are
actually
looking
for
get
the
stress
of
forgetting.
How
do
you
have
no
memories?
So
one
of
the
ways,
one
of
the
approaches,
continuous
learning
and
we're
going
to
discuss
just
keeping
a
very
big
with
play
buffer.
So
how
do
you
handle
that?
We
could've
all
date
and
you
take
blatant
representations.
Castro.
E
I
I
J
E
E
J
H
Q
E
Q
I
have
new
people
new
person,
a
new
set
up,
that
person
will
fail
or
something
we
can
do.
Okay,
by
going
new
data,
it
doesn't
on
the
same
task.
It
probably
gets
a
little
better
I
feel
very
small
I
think
the
thing
to
fail.
It
is
your.
The
task
is
okay,
ie
clinic
on
detecting
people
and
I
wanted
praying
how
you
have
images
for
now
for
cats
and
at
the
end,
you
cannot
attack
people
anymore.
It's
a
new
task
right
is
the
sequence
of
tasks
and
completely
feasible.
G
D
Being
a
difference
in
area,
then
so
there's
a
lot
of
research
going
on
how
to
detect
that
situation
of
shape
so
I'm
just
gonna
go
over.
The
current
approaches
will
quit
and
they
are
mostly
from
the
review
as
well.
So
there
is
dynamic
architectural
approaches,
and
that
means
changing
the
neural
network
architecture.
So.
D
G
D
D
D
Dynamic
changes,
which
are
a
lot
more
interesting,
and
that
means
you
use
the
same
network
and
but
you
change
the
way
you
use
parameters.
So,
for
example,
we
were
doing
some
work
on
sparse
new
networks,
you
and
when
you
have
sparse
narrow
networks.
That
means
you
have
a
lot
of
connections
which
are
not
being
used,
so
those
connections
could
potentially
be
used
to
learn
another
test,
so
you
can
have
the
same
neural
network
learn
to
three
or
even
ten
different
tasks.
So
there
is
a
lot
of
work
on
that
in.
D
Changing
the
forward
pass,
you
have
different
forward
paths
for
each
test
using
the
same
government
to
learn
several
tests
in
the
same
network.
I
think
that's
very
interesting
research
direction
and
one
might
probably
look
at
mainly
because
you
can
still
use
the
same
structure.
Of
course
there's
a
limit
to
how
much
you
can
scale,
but
you
can
still
use
the
same
neural
networks
working
for
that
or
they're
approaching.
H
D
D
D
So
it
falls
on
the
back
of
the
regularization
approach
so
using
some
sort
of
constraints
or
penalties
in
order
to
avoid
your
most
important
ways
of
being
change,
but
they
are
still
changing
a
little
even
using
a
double
series
to
change
the
dynamic
architecture
is,
is
the
idea
that
you
use
some
sort
of
mask
or
you
can
use
the
same
network
to
learn
several
tasks,
but
you
just
use
different
edges
or
different
neurons
or
different
forward
paths,
but
within
the
same
network.
So
that's
the
idea
there
do.
D
D
Have
a
fast
learner
with
so
they
related
to
brain
research
and
I.
Think
that's
the
whole
idea
here.
So--But,
the
fast
network
could
be
the
hypo
campus
for
is
so
storing
a
short-term
memory
and
it's
learning
fast
and
slow
learning
will
be
the
neocortex
and
then,
from
time
to
time
you
get
whatever
you
learn
in
the
fast
learner
and
you
move
to
the
slow
learner.
So
there's
no
learn
is
the
one
consolidating
learning
from
several
tests,
while
the
fast
learner
is
getting
boots
and
learning
online.
D
D
J
Like
injecting
things,
it
sounds
neo
expression
for
one.
It's
a
whole
like
by
saying.
Okay,
you,
you
neurons
new
synapses,
basically
are
responsible
for
looking
at
this
part
of
the
distribution,
because
you're
only
going
to
respond
with
misfires,
so
we
won't
again
have
everything
spread
everywhere.
You
know,
I
mean,
as
opposed
to
these
sort
of
like
not
having
everything
be
involved
in
everything.
It's
just
two
separate
systems
will.
G
G
E
D
D
G
D
Sort
of
knowledge
of
distillation,
so
in
knowledge
distillation,
you
have
one
network
learns,
one
network
learns
a
and
you
want
to
transfer
that
knowledge
to
another
network.
So
you
you
make
the
other
network
learn
the
output
of
the
previous
network,
like
the
I'd
say
that
the
last
season
there.
So
it's
like
a
soft
target
in.
K
M
D
D
H
D
L
B
D
M
D
D
G
H
R
Kind
of
have
two
networks,
and
then
the
one
is
keeping
an
exponential
moving
of
the
network
weights
of
the
fastener
and
then
basically
at
this
serene
it's
called
polyak
averaging
and
then
I'm
not
sure
if
you
can
actually
apply
it
to
these
tasks,
but
that's
kind
of
the
it's
less
aggressive
than
copying
the
way
it's
right.
Instead
of
copying
the
weights,
you
update
the
way
to
be
an
average
of
the
new,
fast
learner
weights
and
all
praise,
and
then
you
basically
have
a
parameter
and
if
you
said
that
permit
at
one
there's.
D
I
R
N
K
G
A
Yeah
we
wouldn't
use
drop
out
because
drama
is
a
very
random
force
way
of
killing.
So,
but
this
would
be,
the
sparse
weights
would
be
very
specific
to
the
tasks
that
are
and
when
you
learn
new
tasks,
you
would
have
a
new
set
of
sparse
weights
for
that,
and
he
is
that
with
sparse
ways,
they
don't
collide
with
one
another
and.
G
A
A
A
A
H
O
O
D
D
World
I
mean
I,
think
I
put
in
the
mirror
by
Bruno
from
so
he's
using
the
same
set
of
weights
and
decomposing
that
so
using
context,
he
decompose
that
into
several
different
neural
networks,
and
one
of
them
is
used.
For
instance,
I
could
do
like
in
a
Fourier
series.
You
decompose
the
sign
to
several
Sigma
X
or
so.
D
D
D
Enforcement
Larry
and
you
just
keep
the
memory
of
past
transitions,
and
then
there
are
smart
ways
of
keeping
that
to
prioritizing,
and
then
you
replay
them
from
time
to
time.
So
we
start
to
forget
that
we
just
grabbed
that
memory
and
we
play
to
the
network
continues
memory
being
the
trainings
yeah.
They
memory
being
the
sample.
The
you
know,
input
and
label
or
firstnet
learning
is
the
transition.
D
What
do
you
forget
yeah,
so
in
prioritize
experiences,
so
he
explains
to
play.
The
idea
is
that
you
keep
the
memories.
So
there
is
this
notion
of
prioritizing
students
to
play
that
every
time
you
sample
from
your
batch
and
you
use
that
badge
to
learn
right
and
then
you
calculate
the
temporal
difference,
error,
which
is
kind
of
a
measure
of
surprise.
So
if,
if
those
experiences
have
a
big
impact
in
your
network,
it
means
you
haven't
learned,
so
you
keep
those
in
the
bathroom,
but.
D
D
G
D
H
D
H
H
A
D
They
end
up
learning
a
sequence
of
tests
without
forgetting
test,
so
there
are
two
benchmarks
in
the
field:
some
benchmarks
are
about.
Shifting
the
distributions
or
having
like
known
IND
distributions
and
permitted
in
East
Bay.
Is
that
so
you
are
constantly
learning
in
a
distribution
which
is
shifting
and
that's.
A
D
Premise
of
machine
in
line
is
that
you
have
an
IFV
distribution
so
so
that
the
we
get
coverage,
then
the
main
issue
of
continuous
learning
and
other
benchmarks
are
related
to
sequentially
learning
data
set.
So
you
can
get
like
speed,
at
least
so.
You
learn
from
zero
and
one,
and
then
you
have
to
learn
from
two
and
three
and
then
four
and
five
and
six
and
seven.
You
also
have
the
same
thing
for
cypher
I
think
it's
called
eyesight
fun
or
you
also
have
sequentially
learning
tasks.
You
have
this
a
benchmark
from
business
from
Vincennes.
D
G
D
D
D
So
there
are
not
a
lot
of
benchmarks
and
that's
actually
the
main
problem.
So
the
purpose
of
the
review
is
people
are
trying
to
set
standards
for
benchmarking
and
reproduce
ability.
So
continuous
learning
is
something
not
a
lot
of
people
are
learning,
it's
not
like
mainstream
and
there
is
lack
of
matrix
I'm
saying
and
some
of
the
things
they
point
out.
Is
you
have
to
define
those
things
like
how
what.
D
Of
prior
knowledge,
you're
allowed
to
have
where
the
memory
of
competition
constraints
like
you
could
just
do
a
model
that
you
know
keeps
memory
of
everything
and
they
constantly
learn
from
that.
That
would
work.
But
how
can
you
compare
that
to
a
model
which
doesn't
keep
any
memory
at
all,
so
it's
hard
to
compare
both.
So
you
have
to
kind
of
define
the
problem,
so
people
can
actually
go
and
back
to
the
problem
and
the
amount
and
type
of
supervision
you
are
allowed.
D
So
here
we're
taking
these
problems
with
all
labels,
or
do
we
have
like
a
bunch
of
problems
with
no
lamer
than
some
labels
and
what
are
the
performance?
Metrics
are
looking
at
so
in
supervised,
learning
everybody's
looking
at
accuracy
and
that's
kind
of
the
main
theme
there.
But
here
you
have
to
look
at
other
things
like
you
have
this.
How
much
did
you
forget
the
previous
task
when
you
learn
a
new
task,
so
you
have
to
define
a
set
of
metrics
that
work
specifically
for
continuous
life.
It's
not
it's,
not!
D
Okay,
just
to
measure
a
bad
person,
there
are
more
of
limits
to
scale
commit
methods,
so
they
can
scale
to
6
10
12
20
tests,
but
we
need
more
escape
to
a
thousand
tasks.
So
I
think
this,
the
beginning
and
I,
don't
know
what
else
you
brought
me
I
put
that
there
with
some
empty
spot,
so
people
can
also
contribute
to
it.
So
this
is
mainly
a
review
of
the
video.
D
I
G
J
M
L
I
M
M
I
Of
advertising
areas-
and
you
have
advertising
everywhere-
that's
not
the
optimal
thing
to
do,
because
you
won't
be
learning
so
you
should
spend
ten
percent
of
your
budget.
There's
actually
a
way
to
figure
out
the
exact
amount
but
spend
ten
percent
of
your
budget,
and
you
double
up
and
cut
in
half
your
advertising
in
temperature
high
places,
so
that
you
learn
for
next
time
that
this
level
advertise
it
was
the
optimum,
so
you're
learning
and
that
actually
comes
that
optimal
control.
Theory
comes
out
of
flight.
D
D
O
D
D
D
A
You
still
very
well
and
the
reason,
if
you
think
about
a
continuous
street
where
things
are
constantly.
Yes,
the
sensor
is
accentuated
by
streaming
in
and
then
something
changes
if
you
keep
your
entire
buffer
and
you
retrain
over
and
over
again
it'll
be
a
long
time
before
the
change.
It's
a
significant
percentage
of
your
bat.chat
that
you
actually.
A
R
Major
to
this,
if
you
I
don't
know
if
this
might
have
setting,
then
the
other
aspect
is
that
I
mean
in
your
Baha.
You
will
kind
of
have
all
the
tasks
right
and
if
you
try
and
model
to
do
n
tasks,
that's
really
diverse
on
the
one
task
you're
interested
in
at
the
current
time
and
the
model
that
is
kind
of
adapting
to
that
one
task.
So
that's
another
reason
right
I
mean
I
I'm,
not
sure.
R
C
I
D
D
D
M
G
D
B
G
I
D
I
D
E
E
And
then
the
final
one
would
be
the
synapses
themselves,
the
way
they
they
it's
more
like
they're,
not
really
like
scalars,
it's
more
like,
but
the
function
of
them
and
their
permanence.
We
use
the
term
permanence.
What,
when
how
synapses
decides
to
stay
around
or
be
forgotten,
is
a
very
complex
function.
It's
not
a
simple
thing:
yeah,
it's
not
as
simple
as
the
Anka
card,
for
example,
so
for
something
that
was
salient
when
you,
you
know,
like
you,
almost
have
life
and
death
experience.
E
One
moment
that
synapse
could
be
with
you
for
the
rest
of
your
life.
Biologically
is
different.
There
was
a
different
type
of
internal
structure
and,
and
so
there's
there's
different
methods
by
which
synapses
decide
what
they're
forgetting
ready
to
their
learning
rate.
It
is
and
their
permanence
is
it's
not
a
simple
rule
that
can
apply
continuously.
D
E
E
Synaptic,
synaptic,
efficacy
or
weight
could
be
a
more
complex
function.
You
might
have
a
more
there
could
be
parameters
associated
with
the
synapse
in
I,
wonder
who
that
would
be
helpful,
because
those
things
do
this
right,
so
as
you're
going
through
here
deciding
what
to
forget
what
to
learn.
Just
the.
E
E
E
E
I
didn't
parameter
that
I
should
be
remembered
longer
than
anybody
else
right
along,
because
this
was
an
important
thing.
So
it's
it's
just
that
you're
going
from
a
single
single
rule
applied
to
synaptic
ways
to
make
it
more
complex
function,
I,
don't
even
know
what
that
would
be.
The
function
is
network.
E
D
D
E
It
means
is
how
important
it
is
or
great
significance.
So
when
you
go
through
these,
these
benchmarks
I
think
that
there
isn't
any
concept
of
you
know.
Some
patterns
are
more
important
to
dollar
than
others.
I
is
it?
Is
there
I
don't
know,
but
in
the
real
world
that's
the
case
and
some
errors
are
more
damaging
than
other
errors
and
so,
from
a
practical
implementations,
point
of
view
from
a
real
system.
That's
doing
something
that
might
be
useful.
G
E
Purpose,
yes,
yeah
or
something
like
that
anyway,
you
might
think
about
this
in
the
brain.
With
a
lot
of
this,
it
occurs
through
this
sort
of
these
release
of
these
neuromodulators.
So
you
have
all
these
synapses
rules,
everything
updated,
but
when
something
bad
happens
or
something
good
happens,
it
sort
of
floods,
the
neocortex
with
a
chemical,
we
said.
A
E
G
G
F
G
E
O
D
First,
the
planning
we
do
that
we
use
this
degenerate
to
decide
which
experience
to
throw
off
or
you
keep
so
you
use
this
together
as
a
measure,
but
it's
different
from
what
Jeff
saying
cause
it
just
things
that
you
only
see
that
once
and
you
know
that
one
time
that
you
should
never
gather
she
will
never
forget
to
fire
price.
So
I,
don't
know
if
you
have
like
you
have
the
area
like,
but
then
you
have
to.
D
L
E
J
F
E
M
M
A
A
P
P
D
P
M
P
P
L
D
H
D
E
M
R
R
G
R
The
default
mode
Network
is
not
associated
to
any
sv
as
far
as
I
know,
so
it's
not
related
to
a
particular
motor
activity.
And,
moreover,
if
you
kind
of
perturb
that
activity
during
sleep,
then
the
Rani
will
be
affected.
So
if
you
try
to
Mouse
today
on
something
and
perturb
the
consolidation
phase,
then
it
won't
have
that
have
that
motor
memory
in
the
next
day,
so.
G
D
E
F
M
I
E
E
I
E
J
I
F
G
Q
A
J
G
G
N
M
F
N
G
N
N
More
of
us
to
list
that's
a
good
question.
I
feel
like
it
might
have
essentially
like
added
a
little
bit
of
noise.
You
can
see
cuz,
it's
like
it's
the
same
task
every
time.
So
the
idea
is
like
if
you
please,
if
you
played
exactly
the
same
way,
the
little
old
data
that
you
generated
would
be
listening,
but
if
you
have
a
different
conditions,
but
that
was
big.
That
yeah
okay.
J
N
Okay,
so
they
may
generate
these
training
sets,
but
everybody
train
the
model
right
and
then
there's
a
face
to
really
smaller
test
sets,
go
through
the
Train
models
and
then
they
create
teams
of
rankings.
So
what'll
happen
is
so
here.
We
have
an
example
of
the
model
of
one
versus
three
subjects,
with
the
data,
so
I
say
this
is
like
just
you
know
this
your
model,
all
right,
just
the
one
thing
from
you
and
it's
but
there's
three
different.
Three
three
separate
in
several
ways
like
I,
say.
N
N
N
N
I
N
N
N
N
N
N
N
Quickly
suppose
so,
if
you
look
at
these
two
different
sequences
right,
two
different
time
series
of
all
sequences
ABCD
here,
an
x,
vc
y.
What
we
see
is
that
in
this,
in
this
total
space,
you
have
different
parts
of
the
different
parts
of
the
states
that
are
allocated
to
different
potential
inputs
within
the
distribution.
So
this
is
categorical
right.
So
here
you
have
when
you
see
a
these
comments.
Actually,
so
whenever
you
see
a
only
those
columns
left
when
you
see
me
these
different
contracts
right,
so
only
these
columns
and
only
big
knobs
are
involved.
N
A
N
N
Right
so
we
have
a
representation
here
that
that
uses
the
same
columns
because
it's
a
B
but
use
of
the
same
columns
here
it's
here,
but
it's
a
seat,
but
because
the
context
is
different,
it
takes
a
different
cell
needs
caller.
So
Larry
you
have
basically
done
spatial
and
temporal
feature
detection
at
once
in
a
release
clause
space
without
having
the
patents
all
over
each
other
right
because
even
separate
you
can
recognize
this.
G
N
G
N
Right
just
a
little
bit
more
tangible
example
here
pretty
much
similar
to
the
last
thing,
so
two
different
sequences
right,
but
they
all
share
common
common
inputs.
Here,
retrieving
is
like
category
so
here
right,
double.
Finally,
prime,
like
they
have
same
colonies,
different
neurons
to
represent
them
different
here
many
they
also
same
finding
the
columns
and
then
cakes
and
pies.
These
are
two
different,
so
they
have
a
totally
different
set,
so
even
separated,
but
we
can
separate
too
many.
It's
the
same
way
you
can
separate,
but
essentially
all
right.
So
right.
N
Is
very
very
quickly
concept
is
the
heaven
style
earning
right
so
as
opposed
to
having
the
system
where
you
have
a
lot
of
initializer
weights
to
do
is
change
around
a
lot,
but
here
it
he
started,
and
so
we
no
no
it's
inevitable
in
the
very
being
this
pregnant
on
long
finger,
I
am
starting
at
me,
but
and
he
form
synapses
as
we
need
to
in
order
to
represent
sequence
of
dignity.
So
it
first
is
surprised
by
everything,
but-
and
it
won't
surprise
you
or
any
more
money.
N
G
N
Okay,
so
these
many
cells
connect
specifically
to
these
so
and
I
think
this
is
this
sort
of
exemplifies
a
real,
because
you
can
have
essentially
Markov
property,
one
right
all
these
cells
only
connecting
in
lockstep
prior,
but
because
these
cells
before
continue
specific
contextual
information.
That
allows
it
to
kind
of
remember
everything
else
in
that.
N
N
N
N
But
theoretically,
if
you
have
a
really
big,
if
you
have
a
big
bat,
that's
neat
things
and
it
will
learn
to
eventually
connect
those
things
depending
on
me,
sort
of
what
else
goes
on,
but
it
is
a
long
gap
that
you
never
ways
to
wait
like
the
way
that
it
stores
a
lot
of
information
on
time
is
like
like
here.
This
is
such
a
specific
instance,
contextual
instance
of
Maine.
That
has
wanted
to
connect
a
surface
specific,
the
actual
exclusive
heat
then.
G
N
N
N
N
N
S
N
N
G
N
F
N
N
N
Q
R
N
Good
so
there's
some
predictive
activity
and
then,
when
the
new
input
comes
in,
basically
the
the
parts
of
the
space
that
showed
up
contradicted
do
the
learning
and
the
parts
that
were
created
don't
so,
if
you
have
essentially
more
predicted
like
more
period
to
speak,
ability
thinking
about
structures,
they
kind
of
go
find
it
essentially
then
read
segments
that
sort
of
like
becomes
strong
and
kind
of
by,
and
you
don't
need
that
many.
But
if
you
have
a
more
complex
pattern,
longer
patterns,
more
noise,
you'll
fill
it
more
the
space,
so
they
continue.