►
Description
Michaelangelo Caporale reviews and evaluates a continual learning scenario called OSAKA, outlined in the paper “Online Fast Adaption and Knowledge Accumulation: A New Approach to Continual Learning.” He first gives an overview of the scenario and goes through the algorithms and methodologies in depth. The team then discusses whether this is a good scenario that Numenta can use to test for continual learning.
Paper: https://arxiv.org/abs/2003.05856
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
Are
there
potentially
better
continual
learning
scenarios
that
we
can
be
testing
against?
Not
just
osaka,
but
are
there
other
ones
that
maybe
that
we
know
of
so
far?
Let's
see
so
I
have
a
hard
time
changing
the
slide.
Okay
yeah.
So
the
osaka
stands
for
online
fast
adaptation
and
knowledge
accumulation.
A
So
that's
the
the
title
of
the
paper
and
the
link
is
in
the
bottom
right.
This
is
a
paper
out
of
element,
ai
and
and
mila,
so
it
offers
both
a
general
scenario
for
continual
learning.
So
like
an
example
that
would
be
task
incremental
or
class
incremental
learning.
It
also
offers
an
algorithm
to
train
in
this
scenario.
A
That
does
particularly
well
in
this
scenario
and
beats
out
sort
of
other
sort
of
similar
kind
of
algorithms,
such
as,
like
mammal
and
animal
and
sort
of
other
ones
familiar
with
and
ones
that
we're
not
familiar
with
I'm
not
going
to
go
over
number
two,
I'm
not
going
to
go
over
the
algorithm.
Just
I'm
just
going
to
go
over
the
scenario,
because
that's
sort
of
where
what
I,
what
I
think
would
be
sort
of
more
fruitful
to
talk
about
continual
learning
scenarios,
yeah.
A
That's
meant
to
be
singular.
Sorry.
The
paper
say
the
paper,
that's
a
good
question:
okay,
yeah
yeah,
so
high
level.
This
is
the
the
continual
learning
scenario.
So
in
the
scenario
you
have
a
pre-training
phase
that
your
model
starts
off
in
and
then
you
have
a
continual
learning
phase
in
which
you
actually
test
the
ability
to
continue
to
learn
so
potentially
in
the
pre-training
you're,
not
trying
to
do
any
kind
of
continual
learning
you
could,
but
it's
just
an
arbitrary
pre-training
phase
on
a
fixed
amount
of
data.
A
So
and
then
the
continual
learning
phase,
it's
a
it's
a
little
I'll
go
into
more
depth,
but
one
of
the
things
that
sort
of
marks
it
is
that
you
can
visit
old
tasks,
oftentimes
in
continual
learning,
you'll,
just
sort
of
sequentially
go
down
class,
0
class,
1
class,
2
and
you're
never
going
to
revisit
class
0,
for
instance.
But
this
thing
you
have
the
potential
to
visit
old
tasks
and
then
all
there's
also
an
emphasis
on
at
a
distribution
test.
So
it's
not
just
you
know.
A
Permuted-
let's
see
here,
it's
not
just
you
know
seeing
the
same
type
of
handwritten
digit,
but
potentially
you
could
visit
images
of
an
entirely
different
thing.
Let's
say
like
and
you
know,
you're
continue
learning
on
digits
and
then
you
can
be
continual
learning
on
an
alphabet.
So
in
that
sense
you
can
visit
at
a
distribution
test
and
I'll
give
a
more
concrete
example
of
that
later
on
and
then
the
third
point.
C
Hey
michelangelo,
this
sounds
almost
identical
to
kind
of
what
I
was
discussing
on
monday
and
then
there's
like
a
pre-training
phase,
where
you
learn
the
basic
representations
of
the
input
domain
or
you're
allowed
to
learn
the
basic
representation.
And
then
you
switch
to
more
of
a
continual
learning
phase.
C
A
Okay,
cool:
let's
see
reports
online,
oh
yeah,
so
for
the
third
point
one
thing
that's
also
kind
of
interesting
is
that
it
reports
the
online
average
performance.
So
you
know
that's
just
to
say:
is
that
how
to
describe
it?
The
effect
of
this
is
that
you
don't
you
can
actually
forget
about
previous
tests
if
you
actually
haven't
seen
them
right.
So
if
you
sort
of
do
well,
the
current
tasks
that
you're
looking
at
but
you're
sort
of
forgetting
about
a
task
that
you've
seen
like
10
20
100
tasks
ago.
A
That's
perfectly
fine.
The
sort
of
presumption
here
is
that
you
know
you
allow
the
model
to
forget
about
tasks
that
it's
not
really
seeing
as
often
and
if
it
does
see
it
more
often,
then,
because
reporting
the
average
performance
it.
Ideally,
it
should
be
doing
well
on
the
ones
that
it's
seeing
sort
of
more
often
than
not,
so
I
so.
A
These
are
three
things
that
I
all
three
things
that
I
sort
of
like
like
about
it
and
sort
of
gives
probably
the
the
main
kind
of
the
big
big,
the
big
points
of
the
scenario.
Okay.
So
this
is
just
sort
of
like
pseudo
code
of
what
the
of
what
the
training
looks
like
it's
sort
of
a
repetition
of
what
I
said,
but
maybe
just
put
a
little
bit
more
concrete.
It's
right,
the
pre-training
phase.
A
You
can
sort
of
think
about
this
as
like
you
can
either
do
supervised,
training
or
meta
learning,
but
effectively
you're
just
sort
of
doing
some
sort
of
supervised
pre-training
for
the
continual
learning.
The
context
makes
more
of
a
difference,
so
you're
actually
gonna.
The
context
are
like
this:
this
markov.
This
markov
process
is
smart
conference
process
because
they're
going
to
be
randomly
sampled
and
then
you're
going
to
sample
data
within
that
context,
and
that
these
targets
will
actually
change
for
the
given
context.
A
The
context
is
a
combination
of
the
task,
so
let's
say
which
images
you're
training
on
like
whether
or
not
that's
like
10,
mnist,
digits
or
like,
let's
say,
10
omni
glock
characters,
and
then
it's
also
the
labels
that
are
associated
with
those
digits.
So
what's
interesting
about
this,
and
I'm
not
sure
exactly
how
I
feel
about
this
part
of
the
about
the
scenario
the
labels
can
change.
So,
let's
say:
label
zero
for
mnist
digi
in
a
particular
context,
may
be
labeled
zero
or
a
digit.
A
D
E
Yeah,
wait
just
to
make
sure
I'm
understanding
this
correctly,
so
the
context
is
basically
a
pairing
between,
like
a
certain
type
of
data
and
like
certain
labels
right,
it's
not
some
sort
of
like
it's
not
the
same
as
the
con
prefer
dendrite
stuff
right.
This
is
this
is
like
information
about
the
specific
task
scenario:
okay,.
A
C
Maybe
is
this
one
way
to
think
about
it,
like
you
know
how
you
think
about
an
object
depends
a
lot
on
the
context
if
you're
looking
at
a
you,
know
porsche
car
and
you
have
minivans
and
you
know
sedans
and
other
cars
nearby.
You
might
say.
Oh
it's
a
sports
car,
but
if
you
had
lamborghinis
and
ferraris
and
other
things,
then
you
might
say:
oh,
it's
a
porsche
right.
So.
B
E
A
There
is
a
notion
of
a
task,
the
the
task
would
be
sort
of
which
images
you're
actually
using
so
like
which
10
characters
of
omni
got
you're,
using
which
10
characters
of
mdis.
Well,
obviously,
there's
only
ten
digits,
but
are
you
using
ten
endless
digits,
ten
fashion,
mnist,
ten
omni
guy
characters?
That's
sort
of
the
task.
E
A
Yeah,
okay,
so
which
task
to
train
on
so
every
single
step.
In
the
continue
learning
phase,
you
decide
whether
or
not
you're
going
to
choose
a
new
task,
a
new
set
of
10
classes-
and
these
are
you
sort
of
decide
whether
or
not
you
want
to
switch
sarcastically.
So
with
probability,
1
minus
alpha,
you'll
switch
and
alpha
is
going
to
be
fairly,
is
going
to
be
fairly
high,
so
you're
not
really
going
to
be
switching
super.
Often
it's
like
two
percent
of
the
time.
A
You're
gonna
switch
to
a
new
set
of
ten
classes,
so
the
boundaries
are
unknown
to
the
model.
So
the
model
is
not
going
to
be
told.
Okay,
now
you're
training
on
these
set
of
ten
classes,
or
anything
like
that.
So
it's
we'll
have
to
sort
of
be
able
to
infer
that
a
task
has
been
changed
and
it
should
be
able
to
adapt
its
outputs
accordingly.
A
A
Not
only
may
you
change
data
sets
given
the
new
context
or
change
the
10
classes
that
you're
training
on
given
the
new
task,
but
to
the
second
point,
the
labels
of
those
images
may
change,
given
the
sample
of
context
right.
So
the
end.
This
digit
zero
may
now
be
labeled
one,
for
instance,
yeah.
So
there's
that
there's
that
example,
that
I
keep
on
giving
yeah
so
and
then
part
of
this
too,
is
that
tasks
can
be
revisited.
A
So
if
you
decide
to
go
back
to
mnist,
there
are
only
10
digits
in
there
and
you're
going
to
select
all
10
of
them
to
train
on
you'll,
assign
labels
to
them
and
you'll,
be
training
on
mnist
again
and
then
you'll
keep
on
going
until
you
decide
to
switch
to
a
new
task
in
a
new
data
set.
So
what's
interesting
about
this
is
you're
most
likely
going
to
see
one
again
throughout
your
training,
as
opposed
to
another
continued
learning
scenarios
where
you
only
may
see
the
category
of
one
only
once
okay.
A
So
this
is
a
little
bit
more
granular
of
how
the
task
switching
works.
So
you
can
sort
of
start
on
the
on
the
left
side
say:
you're
working
with
omni
glot,
with
some
probability
alpha
like
98,
you're,
gonna
stay
in
omnicloud,
okay
and
then,
let's
see
here
or
I
should
say,
you're
going
to
stay
within
the
same
10
classes
that
you're
working
with
in
omniclot.
A
So
the
consequence
of
this
is
that
the
pre-training
phase-
and
I
should
have
been
a
bit
more
clear
about
this-
the
pre-training
phase
is
done
on
omniplot
and
then
the
continual
learning
phase
includes
these
new
data
sets
fashionabness
and
mnist.
So
the
effectively
50
of
the
time
you're
going
to
train
on
the
stuff
you
pre-trained
on
continual
learning
on
the
stuff
you
pre-trained
on
and
then
50
percent
of
the
tr
percent
of
the
time
you're
going
to
see
out
of
distribution
classes.
C
C
A
A
Yeah,
that's
a
good
point.
This
is
this.
This
is
probably
my
fault.
They
do
use
the
word
task
and
context
in
the
paper,
and
that
was
honestly
very
confusing
to
me
as
well.
So
it
may
actually
just
be
easier
to
say,
sampling
context
that
continue
learning
time.
I
think
that
is.
I
think
that
this
sort
of
make
make
a
lot
more
sense
if
anything
yeah,
maybe
there's
a
subtle
distinction
to
be
made,
but
it
is
just
sort
of
simpler
to
think
about
context.
B
A
A
B
B
I
mean
obviously
there's
some
yes
right
or
there's
some
degradation
that
occurs.
You
know
in
our
lives
that
we
forget
things,
but
there's
things
I
that
I
remember
that
I
haven't
dealt
with
for
months
or
years,
and
I
still
remember
them
not
everything,
some
things
I
forgot
quite
quickly,
but
but
there's
many
things
that
I've
learned
very
well
and
I
and
I
have
to
go
back
and
revisit
them.
You
know
it's
like,
so
it
just
seems
like
this
is
a
little
bit
sort
of
like
well
they're,
really
doing
continuous
learning.
B
B
E
B
C
C
A
B
A
Yeah,
so,
okay,
so
I
guess
something
that
could
help
with
this
is
I'll
say
that
omniglot
when
it
says
50,
it's
somewhat
misleading,
so
the
pre-training
phase
is
done
on
half
the
classes.
That's
I
think
it's
done
on
like
a
thousand
classes
and
then
leaves
about
600
something
that
it
hasn't
seen.
So
it's
actually,
although
the
split
is
a
little
bit,
it's
closer
to
like
25,
it's
closer
to
like
say,
25
percent
of
the
omnium
images
or
not
25.
What
is
it
a
good,
a
good
600
out
of
1600
classes?
A
B
I
mean
the
way
I
would
have
thought
you
would
coming
in
naively
into
this.
I
would
have
said:
okay,
someone's
gonna,
do
this
pre-training
on
the
clock,
whatever
all
categories
whatever
and
then
now
you're
going
to
start
loading
new
tasks,
fashion,
numbness,
mnist,
whatever
it
is,
and
as
you
add
these
new
tasks
you,
then
you
go
back
and
test.
How
well
am
I
doing
on
the
ones
I
learned
earlier
and,
of
course,
there'll
be
some
degradation,
but
what
is
the
percentage
of
degradation?
B
F
B
F
B
G
A
A
I
think
it
overemphasizes
the
sort
of
testing
the
ability
to
sort
of
adapt
quickly,
because
the
the
labels
are
constantly
changing,
and
I
think
this
is
potentially
part
of
it
is
that
it's
not
really
sort
of
getting
you
know
to
a
more
fundamental
core
say,
continue:
learning
where
you're
testing
the
ability
to
learn
without
forgetting
and
then
towards
subutai's
point
about
whether
or
not
they
actually
made
an
algorithm
just
tailored
for
it.
I
don't
think
that's
the
case,
considering
that
they
titled
the
paper
after
the
scenario
and
they
explicitly
say
in
the
paper.
B
B
B
C
Yeah,
I
think
the
yeah.
The
most
important
thing
is:
is
this
just
some
really
artificially
constructed
set
of
rules,
or
is
there
some
real
intuitive?
You
know,
motivation
for
this.
Are
there
application?
Is
this
a
practical
scenario?
Is
this?
You
know.
Is
this
a
natural
scenario
that
that
you
might
an
autonomous
agent
might
encounter,
or
is
there
some
underlying
motivation
that
that's
somehow
intuitive
and
appealing
yeah.
B
A
Yeah
exactly
so,
I
messed
up
the
animations
but
yeah.
So
I
think
there
are
some
sort
of
the
first
question.
I
think
there
are
some
improvements
over
other
continue
learning
scenarios
so
like,
for
instance,
over
class,
continue
learning
tasks
in
class
incremental
learning.
I
like
the
fact
there's
a
pre-training
phase,
there's
an
emphasis
on
other
distribution
tasks
compared
to
meta,
continual
learning
say
like
oml
and
animal
that
we're
working
on
we've
been
working
with
once
again,
there's
an
emphasis
on
out
of
distribution
tasks.
C
C
A
D
A
C
E
F
I
I
have
a
quick
comment
to
the
od,
even
though
animo
and
noah
ml,
they
don't
deal
with
all
the
tasks.
I
think
we've
seen
before,
that
the
general
direction
in
the
meta
learning
community
and
even
matter
of
continued
learning
is
to
deal
with
all
these.
So
the
last
data
set
we
reviewed
the
metadata
set
that
contained.
You
know,
10
different
data
sets
into
one,
so
that
was
mainly
to
deal
with
that
and.
C
F
A
F
A
A
I'm
not
entirely
sure
I
think,
just
as
I
just
mentioned
lucas,
there
are
other
improvements
in
this
realm
as
far
as
sort
of
testing
against
ood,
and
then
there
are
still
other
things
as
far
as
you
know,
just
sort
of
finding
a
scenario
that
really
does
sort
of
test
continual
learning
in
a
and
in
a
sort
of
faithful
light.
You
know,
for
instance,
as
jeff
pointed
out
not
retraining
on
tasks
on
pre-trained
tasks
too
often
and
things
of
that
nature.
A
A
B
One
way
to
think
about
it
is
what
would
be
most
impressive
to
people?
What
would
something
that
you
know?
What
kind
of
benchmark
would
you
have
with
people
would
say
yeah
that
gets
to
the
heart
of
the
matter.
That's
you
know,
you
know
just
put
a
different
spin
on
think
about
what
other
people
would
think
about
it.
B
F
So
the
way
we're
thinking
about
this
in
the
battery
ai
project
jeff
is
it's
kind
of
similar.
There
are
a
lot
of
similarities
to
what
michael
just
showed
so
we're
thinking
about
the
pre-training
phase,
but
the
pre-training
would
be
unsupervised
as
opposed
to
supervised.
So
it's
just
like
the
agent
is
throwing
it
environment
and
it
has
some
time
to
explore.
It
has
to
learn
from
that
and
then
after
the
pre-training
and
we'd
have
a
series
of
a
few
shot,
continue
learning
tasks,
and
then
we
want
to
know.
F
If
you
know
we
can
have
an
agent
learn
a
good
model
of
the
world
during
the
pre-training
phase
that
can
easily
be
extended
for
continual
learning
during
the
validation
phase.
That's
that's
kind
of
the
idea,
we're
going
for
yeah
and
I
don't
know,
I
think
I
think
it's
at
least
on
my
perspective.
It's
it's
a
lot
closer
to
how
humans
learn.
B
D
Well,
I
think
what
lucas
means
is
like,
like
from
an
agent's
point
of
view,
learning
behavior
doesn't
always
have
to
be
like
task
oriented.
It
can
just
learn
from,
like
its
own
end,
quote
curiosity
to
build
like
some
sort
of
representation
of
the
world
and
then
use
that
representation
when
presented
with
specific
tasks.
B
B
Saying
it's
like
so
I
it
makes
sense
in
that
environment,
but
I'll
just
try
to
relate
to
okay,
we're
going
to
use
the
same
bob's
going
to
show
continuous
learning
on
the
data
sets.
That
michelangelo
was
just
talking
about
what
what
would
you
know?
How
does
that
strategy
relate?
To
that
I
mean.
Does
that
I
just
I
don't
know
I
don't.
I
don't
really
have
good
thoughts
about
this.
I'm
just
asking
questions.
I.
F
C
B
More
so
what
would
be
the
domain
if
I'm
looking
at
these
sort
of
character,
data
sets
and
all
the
glass
stuff?
And
so
on
I
mean,
is
it
you
know
it's
like
it's.
I
can
imagine
a
robot
running
around
the
house
and
saying,
okay,
I
got
unsupervised
learning
about
the
house
or
something
like
that.
I
don't
know,
but
what's
the
equivalent.
C
B
Okay,
so
then
I
mean
then
you'd
really
maybe
yeah
you'd
want
to
move
away
from
the
whole
thought.
Then
then,
you
move
away
from
the
whole
thought,
like.
Oh
we're,
training,
pre-existing
training
on
this
omigod
set.
Now
we're
going
to
look
at
you
know
eminence,
or
something
like
that.
This
is
more
like:
hey
here's,
the
world,
here's
real
world
images
and
that's
what
we're
going
to
train
on
and
now
we're
going
to
look
at
some
subset
of
the
real
world
which
might
be
character,
sets
or
might
be
cats,
or
something
like
that.
A
It's
a
it's
the
first
one,
it's
the
it's,
the
general,
that's
a
general
continual
learning
scenario,
so
I
gave
them
this
some
fashion,
ms
omnipot,
as
an
example,
but
they
also
had
other
data
sets
as
an
example.
So,
for
instance,
they
even
had
tiered
image
net
right
where
imagenet
is
split
up
into
broader
categories,
and
then
those
broader
categories
are
used
to
define
sort
of
outer
distribution
tasks.
A
I
mean,
I
think
the
the
paper
is
general.
I
don't
know
exactly
yeah
if
you
wanted
to
sort
of
test
against
theirs
and
yeah,
I
think
you'd
have
to
either
yeah
like
there
would
have
to
be
some
kind
of
standardization
of
the
other
data
sets
that
the
data
sets
that
you
use,
but
I
don't
see
why
you
couldn't
necessarily
adjust
things
if
you
really
wanted
to,
and
then
you
know
test
against
theirs,
but
then
also
say:
okay.
Well,
this.
B
C
Yeah
even
like
the
metric,
I
think
you
mentioned
earlier,
the
metric
is
some
sort
of
a
moving
average
of
recently
seen
stuff.
Yes,
and
you
know,
do
they
specify
okay?
What
is
the
coefficient
of
the
moving
average?
You
know
what
is
it
you
know
it's
a
huge
advert
that
could
make
a
huge
difference,
or
is
it
a
window
of
time?
You
know
what
is
that
window.
A
C
A
C
E
C
A
A
C
It
seems
like,
like
every
paper
or
every
other
paper
seems
to
have
its
own
definition
of
continuous
learning
and
the
benchmark.
I
think
that's
there's
so
many
ways
that
continued
learning
can
be
evaluated.
It's
really
confusing
and
yeah.
We
would
like
to
find
something
that
makes
sense.
Yeah
I
mean.
B
B
F
B
E
B
D
I
think,
like
there's
a
pretty
good
case
from
the
paper
I
read
about
how
reinforcement
learning
is
a
good
framework,
because
they're
like
in
supervised
learning,
there's
no
notion
of
like
an
agent's
past
or
future.
It's
just
like
the
data
in
front
of
it
at
the
moment,
whereas
in
reinforcement,
learning
like
just
the
objective
itself,
has
it
within
it
baked
like
the
agent's
lifelong
future
and
its
past
so
like
in
order
to
act
optimally,
it
has
to
optimize
something
that
already
like
accounts
for
that.
B
C
A
But
one
of
the
things
I
wanted-
I
thought
would
be
interesting
to
have-
is
a
place
to
test
the
dendrites,
necessarily
I'm
not
necessarily
looking
for
something
to
test
animal
or
oml
or
meta,
continue
learning,
but
the
dendrites
work
and
then
been
working
on.
I
find
a
good
sort
of
a
a
good
application
for
that.
A
good
testing.
C
C
Idea
of
there
being
a
pre-training
phase
and
just
echoing
what
lucas
said,
ideally
that
would
be
unsupervised,
but
I
like
the
idea
of
there
being
a
pre-training
phase,
the
fact
that
there's
different
contexts
associated
with
I
think
that
seems
pretty
nice
as
well.
But
it's
just
there
are
a
few
weird
things
here.
A
F
A
F
We
have
been
discussing
like
ways
of
applying
dendrites
in
the
enforcement
learning.
I
mean
akash
came
up
with
a
few
ideas
you're
doing
as
well.
So
I
mean
we're
not
talking
in
the
research
meetings,
but
if
you
want
to
join
the
discussions
yeah
more
than
welcome,
but
yeah
we've
been
thinking
about
that.
So.
D
C
I
mean
I
can
come
up
with
a
completely
different
objective,
which
is
kind
of
training
speed.
It's
the
you
know.
The
real
power
of
continued
learning
you
could
argue
is
that
you
can
really
quickly
learn
new
things
without
you
know,
seeing
this
you
know
huge
batch
of
stuff
over
and
over
again,
so
you
know
another
metric
could
be
as
you're
learning
new
stuff.
C
You
know
how
quickly
can
you
get
to
the
accuracy
as
if
you
had
seen
the
entire
data
set
over
and
over
again,
you
know:
how
fast
can
you
learn
stuff
and
we
know
humans
can
learn
extremely
fast,
so
this
is
a
little
bit
like
few
shot
learning,
but
in
a
continually
learning
scenario
right,
it's
just
you
should
be
able
and
that's
a
that
has
tremendous
practical
applicability,
yeah
large
production
networks,
and
you
want
to
update
it
with
some
new
data.
You
don't
want
to
retrain
the
entire
thing,
huge
energy
usage.
B
C
F
C
F
C
F
C
F
F
G
Yeah,
I
I
guess
I'm
trying
to
look
at
something
where
it's
tending
toward
to
the
degree
that
the
model
gets
to
some
being
able
to
generalize
as
opposed
to
particularize.
For
these
various
cases.
I
mean
that
kind
of
is
when
I
think
of
how
I
consolidate
memories
and
various
other
things,
eventually
things
window
down
to
more
compact
representation.