►
Description
Michaelangelo Caporale presents a summary of two papers that apply self-attention to vision tasks in neural networks. He first gives an overview of the architecture of using self-attention to learn models and compares it with RNN. He then dives into the attention mechanism used in each paper, specifically the local attention method in “Stand-Alone Self-Attention in Vision Models” and the global attention method in “An Image is Worth 16x16 Words”. Lastly, the team discusses inductive biases in these networks, potential tradeoffs and how the networks can learn efficiently with these mechanisms from the data that is given.
Next, Lucas Souza gives a breakdown of a potential machine learning environment and benchmark Numenta could adopt - Interactive Gibson. This simulation environment provides fully interactive scenes and simulations which allows researchers to train and evaluate agents in terms of object recognition, navigation etc.
“Stand-Alone Self-Attention in Vision Models” by Prajit Ramachandran, et al.: https://arxiv.org/abs/1906.05909
“An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” by Alexey Dosovitskiy, et al.: https://arxiv.org/abs/2010.11929
iGibson website: http://svl.stanford.edu/igibson/
0:00 Michaelangelo Caporale on Self-Attention in Neural Networks
1:09:30 Lucas Souza on iGibson Environment and Benchmark
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
Here
yeah,
so
I
thought
it'd
be
good
to
talk
about,
or
this
is
something
that
subutai
had
sort
of
ran
by
me
a
while
ago,
some
papers
where
they
apply
self-attention
to
vision.
So
self-attention
is
this
pretty
popular
technique.
An
llp
is
increasingly
popular
and
sort
of.
There
have
been
recent
papers.
That
said,
okay,
well,
you
know
applied
well
to
nlp,
and
so
can
we
apply
well
division,
so
this
I
thought,
could
be
a
good
time
to
one
go
over
this
application
of
vision.
A
C
A
Yeah
so
yeah,
so
I
thought
I'd
reiterate
on
self
attention.
Once
again,
we
talked
about
it
kind
of
go
into
the
details,
a
little
bit
more
also
talk
about
it
in
the
context
of
context,
so
in
the
dendrite
work
that
we're
sort
of
doing
you
know
we're
thinking
a
lot
about
making
context
and
foreign
representations,
as
I
interpret
self-attention,
is
sort
of
you
know
the
machine
learning
community's
way
of
or
one
way
the
machine
learning
community
sort
of
deals
with
context,
in
particular
context.
A
You
know
within
very
long
sentences
and
then
there's
also
some
kind
of
interesting,
which
is
just
sort
of
like
why
self-attention
is
just
so
very
popular
these
days
and
sort
of
why
it's
getting
the
results
that
it's
getting
and
it's
somewhat
just
a
iteration
of
the
just
build
a
bigger
mentality
of
machine
learning.
You
know
you
scale
the
model,
you
throw
more
data
at
it
and
things
just
sort
of
you
know
seem
to
work
better.
A
You
know,
at
least
on
the
benchmarks,
so
kind
of
like
sort
of
circle
back
to
this
and
sort
of
just
pose.
A
question
of
the
group
see,
as
I
said,
self-attention
has
been
applied
in
nlp.
This
is
like
the
gpt-2
and
gpt
3
models
that
just
sort
of
have
just
done
very
well
in
benchmarks
and
sort
of
have
you
know
just
sort
of
a
very
impressive
results:
impressive
demos,
online
and
sort
of
the
main
thing
that
it's
doing
as
a
sort
of
help
it's
helping
establish
long-range
dependencies
in
text.
A
So
you
can
kind
of
think
in
you
know,
processing
sequences
sort
of
how
do
you
preserve
the
information
of
previous
inputs
and
that
can
kind
of
be
difficult
with
an
rnn
if
you
only
are
sort
of
dealing
with
one
hidden
state
at
a
time,
and
so
someone
came
along
and
said
instead
of
instead
of
processing.
One
word
at
a
time
what,
if
you
just
do
every
word
at
the
same
time,
and
so
that
kind
of
gets
to
the
difference
between
an
rnn
and
and
a
and
a
self-detention
network.
A
So
this
is
just
an
overview
and
kind
of
describe
the
architecture,
but
so
in
rnn
you
can
kind
of
think
of
it
as
you're
baking.
The
understanding
of
the
previous
words
into
the
just
the
one
that
you're
processing,
this
sort
of
sort
of
happens
in
as
the
temporal
memory
algorithm,
where
you're
going
to
create
a
context,
informed
representation
of
your
current
input,
given
the
ones
from
previous.
But
you
only
sort
of
get
one.
You
know
hidden
state
in
a
sense
to
sort
of
process
and
sort
of
to
inform
the
representation.
A
D
A
Do
that,
for
I
mean
every
word
in
the
sentence
like
you're,
going
to
look
around
you're
going
to
say
you're
going
to
go
to
this
first
word
the
and
you're
going
to
say
I
have
a
representation
for
that.
But
how
do
I
also
incorporate
representations
of
all
the
other
words
in
the
sentence
and
then
you're
going
to
go
to
the
word
animal
and
you
say
I
have
a
representation
for
animal,
but
how
do
I
incorporate
representations
for
every
other
word
in
the
sentence
and
so
on
and
so
on?.
D
D
B
D
E
F
A
A
I
should
say
you
have
representations
for
every
word
and
sentence
and
then
you're
sort
of
going
to
combine
them
in
some
intelligent
way
to
sort
of
form
one
representation
for
it.
So
it
sort
of
makes
sense
to
serve
to
preserve
a
lot
of
the
animal
because
it
is,
as
is
what
is,
it
is
identifying
the
animal
so,
but
how
do
you
know.
D
A
Essentially,
what
it
boils
down
to
is
you're
going
to
form
a
representation
for
it
and
you're
going
to
form
a
representation
for
the
animal
and
then
you're
going
to
look
essentially
how
similar
those
two
things
are
and
then
the
more
similar
they
are.
Then
the
more
you're
going
to
use
the
animal
in
your
final
representation.
So.
D
B
Well,
I
think,
there's
two
different
things
one
is
each
word,
has
a
representation
which
is
like
an
encoding.
So
that's
that
you
know
we
could
that's
done.
You
know
separately,
but
then
then
it
learns
in
a
brute
force
way
that
if
you
have
animal
here-
and
you
have
these
kinds
of
words
near
you
and
you
have
an
animal-like
word
in
the
middle,
what
other
parts
might
you
want
to
pay
attention
to,
and
that's
learned
for
every
possible
word
in
every
possible
context,
and
that's
why
you
know
that's
why
you
need
these?
F
F
I
don't
know
if
you
remember
so
the
8
is
going
to
be
so
you're
going
to
use
the
query
for
the
word
e
and
you're
going
to
match
against
the
key
for
each
one
of
the
other
words
on
the
left
side,
or
I
don't
know,
I
don't
know
how
the
word
queer
and
key
is
but
one
of
either
but
you're
going
to
try
to
match
one
with
the
other
and
see
where
it
fits.
So
it's
going
to
learn
those
weights
that
they're
going
to
tell
you
know
it
fits
with
animals.
F
D
I
guess
I'm
still
struggling
a
little
bit
understanding
how
it
determines
that
I
mean
yeah,
it's
obvious
to
us,
but
but
I
mean
super
tight
said:
there's
two
things
I
think
one
is
that
maybe
the
basin
coding
for
it
and
the
animal
has
some
sort
of
pre-learned
similarity
and
then,
but
then
in
one
sentence
like
this,
you
wouldn't
be
able
to
figure
this
out.
So
somehow,
you're
gonna
have
to
do
this
over
many
many
different
sentences.
To
figure
out.
D
F
D
A
D
D
I'm
going
on
what
stupid
I
said.
The
whole
learning
process
here
was
trying
to
predict
the
next
word
in
the
sentence
and
using
that
prediction
as
the
as
the
teacher
to
figure
out
what
would
be
the
best
previous
and
forward-looking
components
that
are
are
related
to
that
and
best
make
that
prediction,
I'm
just
asking
so
that
given,
if
that,
given
what
stupid,
I
said,
is
that
the
basic
definition
of
self-attention
in
this
case.
B
Yeah,
I
think
the
self-attention
is
like
michael
interested.
One
way
to
do.
It
is
in
a
temporal
way
and
like
the
rnn,
where
you're
keeping
the
context
somehow
in
the
representation
like
we
do
in
temporal
memory,
but
self-attention,
I
think
it's
just
it's
all
done
in
one
feed
forward
way.
Every
possible
hairwise
thing
is
represented
in
this
feed.
Four
things
so
self-attention
is
like
in
one
input
pattern
it's
attending
to
different
parts
of
itself.
If
that
makes
sense,.
D
Okay,
all
right,
I
see
so
it's
less
the
it's
less
the
training
paradigm
or
the
learning
paradigm.
It's
more
of
the
result
of
that.
Given
a
given
sentence
or
2000
words,
whatever
I
learned
to
attend
some
components
of
that
of
its
own,
the
thing
I'm
trying
to
understand
I'm
attending
the
components
of
that.
So
it's.
D
D
I
mean,
for
example,
I
could.
I
do
all
the
words
in
the
right
here.
In
parallel,
I
mean
I
could
do
them
in
any
order
right.
It
doesn't
matter
if
you're
not
taking
advantage
of
the
fact
in
the
training
method.
We're
not
taking
advantage
of
the
fact
we're.
Not
you
don't
have
to
go
through
the
words
in
order
right,
you
can
do
them
all
at
once.
You
can
do
them
backwards
or
as
long
as
each
word
is
compared
to
every
other
word
is
that.
D
B
F
D
A
A
B
A
A
So
it's
sort
of
like
a
fully
connected
network
in
the
sense
that
you
know
you're,
going
to
sort
of
you're
going
to
combine
your
representations
for
all
of
your
inputs
in
a
fully
connected
manner.
So
in
this
case
we
have
three
to
three
and
you're,
going
to
have
a
sort
of
weighted
connections
in
what
is
it
in
sort
of
three
different
ways
for
nine
nine
weights
total.
So
this
is
just
a
different
example,
I
just
say
the
cat
etc.
A
Just
because
it's
sort
of
easier
to
draw
a
diagram
of
a
of
a
with
three
inputs
and
three
outputs.
This
red
is
now
going
to
sort
of
kind
of
correspond
to
a
new
representation
for
rep,
for
the
this
red
will
correspond
to
new
representation
for
cat,
and
this
red
will
correspond
to
new
representation
for
just
the
rest
of
the
sentence,
or
you
know
just
this
one
word
etc.
A
So
I
said
the
first
thing:
you're
going
to
you're,
going
to
generalize
your
inputs,
and
so
really
all
this
is
just
you're
going
to
take
your
encoding
and
you're
going
gonna
pass
it
through
a
neural
network.
This
is
why
I
say
at
the
top:
it's
using
three
neural
networks
in
a
way
to
make
one
neural
network,
so
these
three
will
sort
of
combine
to
make
one
generalized
neural
network.
So
this
is
the
first
one
you
have
a
set
of
weights
called
wv.
A
These
are
going
to
put
these
quote-unquote
values,
so
you
go
from
encodings
to
values,
and
this
transformation
is
applied
in
the
same
exact
way
for
the
same
exact
way
for
cat
and
same
exact
way
for
eccentric.
So
these
weights
are
shared
right
here,
so
x,
sort
of
gives
you
these
general
inputs
and
then
now
you're,
going
to
take
linear
combinations
of
these
of
these
vectors
of
these
values.
A
So
to
generalize
the
connections
you
kind
of
just
start.
Oh
we'll
start
looking
at
this
for
sort
of
first
one
we're
going
to
create
a
generalized
representation
for
the.
Maybe
you
want
to
keep
a
little
bit
of
the
you
want
to
keep
a
good
amount
of
cat
in
your
representation
and
maybe
there's
a
little
bit
of
information
in
the
rest
of
it
and
the
etc
that
you
want
to
keep
as
well.
A
One
is
going
to
be
a
query,
neural
network,
this
wq
and
that's
sort
of
represented
by
the
purple
and
sort
of
once
again
it's
going
to
be
shared
among
all
your
encodings
and
then
your
wk
is
your
quote-unquote,
your
keys,
that's
representing
the
orange
and
so
you're
going
to
use
the
same
set
of
weights,
wk
against
the
same
set
of
weights
for
cats,
same
set
of
weights
for
etc.
So,
once
again
we're
just
working
on
this.
A
The
more
you're
going
to
keep
the
less
symbol
are
the
less
they're
going
to
keep
and
then
you're
going
to
look
at
the
query
of
cat
or
sorry
the
query
of
the
because
we're
just
working
on
the
and
then
the
key
of
cat
and
you're
going
to
say
how
similar
is
the
and
cat
and
it'll
be
reasonably
similar.
So
you
keep
about
a
little
bit
more
and
then.
Finally,
you
look
at
you
know
once
again
query
of
the
key
of
etc.
A
And
then
you
decide
how
much
sort
of
how
similar
these
two
vectors
are
and
then
that's
sort
of
how
much
you
keep.
So
these
three
values
are
going
to
end
up
being
your
sort
of
deciding
the
linear
combination
that
you
take
on
the
left,
and
then
you
repeat
that
same
process.
Very
you
know,
similarly
for
cat
and
very
similarly
for
etc,
to
get
your
sort
of
decide
all
the
connections
that
you
will
end
up
using
so
does
that
make
sense.
G
A
Yeah
and
then
you
also
have
to
normalize
these
values
in
some
way
to
make
sure
they
add
up
to
one.
But
that's
you
can
maybe
consider
that
just
a
smaller
detail,
for
instance,
this
may
output,
like
one
five
and
then
three
and
then
you
just
out
you
know
normalize
it
to
point
two
point:
five
point:
three
in.
A
A
C
D
A
And
k1
here,
but
it's
already,
I
okay,
okay,
but
yeah.
I
guess
just
in
summary:
it's
just
using
three
neural
networks
to
make
to
make
one.
You
generalize
the
connections
and
you
generalize
the
inputs,
and
these
are
just
sort
of
decide
on
the
fly
to
determine
how
much
is
one
relate
word
related
to
another
word,
and
then
you
sort
of
take
linear
combinations
to
sort
of
preserve
the
relevance
of
the
important
words
okay.
A
So
this
is,
I
guess,
the
first
point
of
you
know
sort
of
why
this
works.
This
is
my
possible
explanation,
as
opposed
to
an
rnn
which
sort
of
has
this
issue
of
keeping
long-range
dependencies.
Now
also,
you
have
tight
links
between
things
that
may
be
related
to
one
another,
as
you
create
I'll
call
them
context
and
form
representations.
A
I
say
tight
links
because
you
can
relate.
You
know
the
tenth
word
in
the
sentence
directly
to
the
first
word
in
the
sentence,
as
opposed
to
having
to
relate
the
tenth
word
in
the
sentence
to
some
wrapped
up
hidden
state
of
all
the
previous
words,
and
then
it
also
helps
that
you
know
these
links
are
very
general
and
that
these
links
are
fully
connected,
as
in
you
can
connect
the
tenth
word
to
the
first
second,
third,
fourth,
and
so
on.
A
A
Yeah,
exactly
the
important
headlines
are
learned
over
time,
so
yeah.
So
I
guess
in
in
in
the
dendrites
case,
I
think
I
was
just
trying
to
make
an
analogy
here
there.
It
also
works
as
a
at
least
in
temporal
memory
would
work,
and
you
just
sort
of
have
the
previous
state
to
work
off
of
the
other
sort
of
major
difference
is
that
the
important
to
have
links
would
be
sparse.
A
A
Let's
see
here,
I
guess
there
is
a
sort
of
a
caveat
of
having
this
to
work,
which
I'll
talk
about
in
a
little
bit,
which
essentially
that
it's
so
general
that
you
need
a
lot
of
data,
which
is
that
part
of
the
thing
that
I
was
saying.
It's
sort
of
the
usual
story
of
just
sort
of
make
it
bigger
so
I'll
get
to
that
in
like
a
little
bit
but
just
kind
of
going
into
the
the
papers
that
that
sort
of
originally
motivated
and
going
into
this
topic.
A
So
someone
came
along
and
said:
well,
it
worked
really
well
for
nlp,
and
so
can
I
work
well
for
vision.
I
don't
necessarily
go
super
deep
into
the
details
of
these
papers
because
I
think
if
the
basic
idea
of
self-attention
is
is
grass,
then
it's
sort
of
you
can
kind
of
see
how
it
can
apply
to
vision.
But
there
are
sort
of
two
ways
that
it's
done.
One
is
local
attention
and
that's
in
this
paper
and
this
other
one
is
global
attention.
So
let's
sort
of
just
describe
what
that
means.
B
A
Yes,
that
is
true.
I
think
one
thing
that
they
pointed
out
in
in
the
global
attention
paper
is
that
you
get
to
create
links
between
pixels
but
like
this
pixel
now
gets
to
have
a
link
between
this
one
much
earlier
in
the
hierarchy,
so
that
is
sort
of
one
one
difference
but
sort
of
like
how
soon
you
can
actually
relate
information
on
one
side
of
the
image
to
the
other
side
of
the
image.
A
B
A
One
thing
that
they
mention
in
this
paper
because
they
use
they
basically
take
the
resonant
architecture
and
they
just
sort
of
swap
swap
in
self-attention
layers
just
to
replace
the
comp
layers
and
just
make
sure
the
you
know
the
sizing
is
maintained
and
whatnot
the
input
size
and
the
output
size
among
each
layers
and
such
but
gonna
say
they
mentioned
that
most
likely.
There
is
a
sort
of
better
architecture
that
they
could
actually
use
to
apply
local
attention.
A
This
also
has
the
benefit
by
the
way
that
there
are
already
optimized
gpu
kernels
to
do.
You
know
self-attention
in
this
way,
whereas
this
one
to
do
it
pixel
by
pixel
and
sort
of
moving
across
the
image.
It's
just
as
of
right.
Now
there
isn't
sort
of
optimized
gpu
kernels
for
this.
So
this
is
another
motivation
of
the
16
by
16
words.
F
A
A
F
A
Okay,
so
these
are
just
the
general
results
so
for
the
local
attention
once
again,
they're
just
performing
to
the
resnet
baseline,
just
replacing
replacing
up
the
comp
layers
so
using
the
same
amount
of
data.
They
end
up
using
fewer
flops
because
they
the
way
how
the
tension
works,
because
you
can
share
your
weights,
you
don't
have
to
use
as
many
and
you
can
use.
A
A
Is
that
right,
really
similar
number
of
parameters,
or,
I
should
say,
apple
samples
in
terms
of
the
resonant
baseline
of
using
the
same
number
of
flops
they
outperform
resonant
but
generally
for
a
given
amount
of
accuracy.
They
can
achieve
that
accuracy
in
fewer
flops.
But
there's
this
huge
huge
caveat,
which
is
that
it's
like
trained
pre-trained
on
this.
On
this
I
call
a
modest
300
million
images.
That's.
D
B
B
A
It's
like,
I
think
they
basically
have
they
they
put
in
one
patch
at
a
time,
but
then
they
also
include
a
like
a
an
input.
That's
it's
supposed
to
be
similar
to,
like
a
start
token
or
something
like
that,
and
it
goes
through
the
transformer
and
then
in
that
place
of
that
start
token
and
output,
some
representation
and
then
they'll
do
classification
on
that
output.
It's
something
like
that!.
B
B
B
F
A
A
A
So
it
does
just
there's,
I
guess,
there's
like
small
caveat
here,
which
I
won't
necessarily
I
guess
maybe
get
into,
but
I
say
this
is
possibly
due
to
you
know:
content
based
interaction,
so
I
know
this
is.
I
think
this
is
how
they
sort
of
refer
to
it
in
the
paper
that
you
can
actually
sort
of
like
sort
of
create.
A
F
Michelangelo,
just
isn't
those
two
things
opposite
like
I
agree.
They
have
a
general
model,
so
you
can
even
learn
convolution
through
the
that
attention
model.
But
that
means
you
have
a
worse
inductive
bias
at
this
start
right
because
convolution,
you
already
have
a
better
inductive
bias
that
it
was.
F
You
came
up
through
studying
how
vision
works
and
all,
and
then
you
you
probably
need
well
less
data
if
you
have
a
better
inductive
bias.
But
in
this
case
I
understand
it's
a
more
general
method
that
learn
anything
and
if
it
were
just
a
lot
of
data,
it's
going
to
learn
a
better
inductive
bias
through
training,
but
at
the
start
it
has
a
worse
inductive
bias.
That
is
my
understanding.
F
B
E
B
Which
not
necessarily
worse,
it's
just
it's
making
less
assumptions
and
because
there's
less
of
it's,
because
it's
a
weaker
bias,
you
need,
you
have
to
have
more
data
in
order
to
learn.
Yes
right,
the
stronger
the
bias
is,
the
less
data
you
need.
But
if
your
bias
isn't
exactly
right,
then
it's
not
the
network's
not
going
to
be
work
as
well.
A
B
F
C
B
A
You
sort
of
need
enough
data
to
actually
to
actually
get
you
there,
and
so
I
guess,
that's
probably
a
better,
a
better
interpretation
of
sort
of
what's
happening
here
versus
the
better
inducted
biases.
As
you
guys
say
here
we
made
the
inductive
biases
more
general,
but
then
we
were
able
to
support
it
with
more
data.
A
A
A
A
And
not
only
that,
but
then,
if
you
sort
of
scale
it
up
and
make
it
even
more
general
and
then
add
add
adding
more
data,
then
obviously
well,
we
always
know
we
get
improvements
this
way.
But
there
is
something
to
say
that
it
is
something
slightly
of
a
better
inductive
bias
than
to
come,
but
yeah
yeah.
E
A
So
getting
back
to
this
point,
I
guess
this
kind
of
made
me
think
about
our
work
and
this
sort
of
brings
to
the
last
slide.
So
it
has
very
powerful
inductive
biases,
but
these
inductive
biases
are,
if
I
the
way
how
one
way
you
may
think
about,
it
is
they're
sort
of
giving
you
more
bang
for
your
buck.
A
So
it
makes
me
wonder
that
sort
of
like
what
kind
of
constraints
I
guess
hardware
may
be
given,
and
then
I'm
wondering
sort
of
how
much
you
can
kind
of
get
out
of
your
bang
for
buck
using
sort
of
fewer
inductive
biases.
If
you
sort
of
are
enabled
to
just
sort
of
throw
more
power
at
it
and
throw
more
data
at
it,
then
I
sort
of
wonder
what
kind
of
potential
trade-offs.
A
I
guess
that
you
can
make,
while
still
making
a
good
amount
of
progress,
if
anything,
just
to
give
one
possible
answer
to
this.
This
may
sort
of
give
a
bit
of
validation.
I
guess
to
sort
of
the
kind
of
current
work
that
we're
doing,
which
is
just
seeing
how
far
we
can
get
in
injecting
the
thousand
brains
theory
into
machine
learning,
where
maybe
not
a
lot
of
the
subtleties.
A
You
know
the
brain
can
sort
of
be
properly
incorporated
into
it,
and
maybe
some
of
that
stuff
will
have
to
sort
of
be
kind
of
sort
of
just
sort
of
give
way
to
sort
of
general
neural
network
mechanisms,
but
nonetheless,
just
sort
of
nonetheless
we'd
sort
of
still
be
injecting
powerful,
inductive
biases,
but
even
with
more
data.
So.
G
A
G
More
or
less
or
that
it
it
does
equally
well,
I'm
just
you
know
just
using
humans
as
as
a
as
a
kind
of
drop
in
point
at
some
point,
you
understand
enough
about
the
world
that
it
doesn't.
Your
processing
of
images
doesn't
can
stop
at
some
level
your
higher
order,
cognitive
of
how
do
you
interpret
some
of
the
things
might
keep
on
going,
but
it's
like
the
world
is
not
going
to
present
you
with
too
many
new
surprises.
G
D
D
D
I'm
taking
out
humans
because
I
don't
think
human
brains
work
like
this
at
all
so,
but
if
you
just
look
at
these
networks
and
they're
of
a
constrained
size,
they
can't
and
you're
assuming
the
world
is
static,
it's
not
changing
on
you
over
time.
Then
they
can't
get
better
infinitely
that
this
they
have
to
saturate
at
some
point
in
time.
That's
the
way
it
is
right,
there's
only
so
many
parameters
and
that's
that
you
can't
go
beyond
the
best.
D
D
I
don't
think
this.
This
is
this.
This
can
the
way
these
things
are
working
here.
There's
this
kind
of
attention
is
completely
different
than
the
kind
of
intention
that
we
know
that
humans
do
and
the
whole
structure
of
knowledge
based
on
reference
frames.
This
is
how
you
know,
I'm
certain
that's
how
brains
work
and
none
of
that's
in
here
so.
G
D
I
was
just
making
a
comment:
if
I
have
a
network
of
any
type,
I
don't
care
what
it
is
or
how
it's
trained.
If
it's
got
a
fixed
number
of
units
and
parameters,
you
can't
continually
improve
its
performance
forever.
That's
just
not
possible.
Unless
the
world
is
not
static,
then
you
can
always
sort
of
update
to
the
latest
model
of
the
world,
but
just
just
basic.
You
can't
improve
something
infinitely
right.
D
H
H
Sure
one
one
very
simple
rebuttal
to
this,
not
necessarily
well
I'll,
just
I'll
just
state
it
one
very
simple
rebuttal
is
this:
this
argument
assumes
that
gradient
descent
can
discover
all
of
the
brain's
algorithms.
But
if
that's
not
true,
if,
if
gradient
descent
can't
discover
the
important
intelligence
algorithm,
then
then
just
you
know
throwing
more
data
and
more
computation
at
it
isn't
gonna
help.
So
so,
like
this
argument,
you're
making
is,
is
good
if
gradient
descent
is
able
to
discover
solutions,
but
it's
not.
D
But,
but
why
why
does
it
matter?
It's
basically,
I'm
making
an
argument.
Doesn't
matter
what
what
learning
mechanism
you
have
you
have
a
network
of
n
number
of
neurons.
It
could
be
a
real
brain.
It
doesn't
really
matter
if
the
world
is
static,
meaning
it's
not
changing
and
you
have
a
fixed
number
of
parameters.
D
Then
there
is
it's:
it
doesn't
matter
what
it's
doing
it
can't
go
on
forever.
It
can't
get
better
forever
this.
What
does
that
mean
to
get
better
forever?
I
don't
see
how
that
could
it
just
doesn't
make
any
sense
to
me.
It
doesn't
matter
what
what
learning
algorithms
you
have
or
anything.
It's
just
there's
a
limit.
D
H
D
F
Yeah,
so
this
the
second
point
and
as
a
response
to
marcus
as
well.
That's
exactly
the
question
that
meta
learning
is
trying
to
answer,
and
I'm
not
talking
about
memo-
and
I
mean
memo
is
just
a
very,
very
small
part
of
metal
learning.
But
metal
learning
is
about
learning
everything,
the
architecture,
the
optimization,
algorithm,
everything
we
use
in
deep
learning
and-
and
you
can
frame
it
as
you-
want
to
learn
the
inductive
biases
and
there
is
kind
of
the
two
approaches
in
metal
learning.
F
So
I
went
through
the
matter
learning
workshop
now
it
was
really
good,
but
I
could
say
there
was
clearly
a
divide
between
the
community
and
half
of
the
community
was
thinking
about
learning
a
good
inductive
biases
through
meta
learning
and
then
getting
those
inductive
bias
and
incorporating
into
the
network,
and
that's
essentially
what
efficientnet,
for
example,
is
that's
the
state-of-the-art
network
we
have
right
now
that
was
learned
through
a
network
of
texture
research
by
google
and
even
swish.
The
activation
function
we
we
talked
about
was
also
learned
using
program
search.
F
So
this
is
one
way
of
approaching,
and
the
other
way
of
approaching
is
is
thinking
that
we
don't
need
to
learn
the
inductive
devices
at
all.
We
can
just
make
it
a
very
high
level,
optimization
process
that
goes
through
like
levels
and
then
you
can
just
learn
the
whole
thing,
but
you
don't
need
to
get
the
inductive
bias
out
of
it.
You're
just
gonna
make
it
so
that
you'll
always
learn
the
whole
thing
at
once,
and
it
doesn't
have
to
be
true
grade
and
descent.
So
meta
learning
is
a
lot
of
things
you
can
do.
F
Yeah
and
yeah,
I
think
that's
a
very
good
workup.
If
you
have
some
time,
you
should
go
to
it
and
it
it
answers.
Your
second
question
I
mean
that's
what
they're
trying
to
do.
There
is
a
work
by
a
tone
kirsch.
He
is
a
phd
student
with
schmidt
hubber
and
he
has
his
work
out
general
matter
learning,
which
is
exactly
trying
to
learn
the
whole
thing
at
once,
like
even
the
architecture,
the
delineate
everything
and
it's
trying
to
abstract
all
the
way
up.
So
you
don't
need
any
inductive
bias
at
all.
F
B
B
Yeah,
but
even
in
10
years
it
seems
like
I
mean
they're
sort
of
replicating
evolution
in
some
sense
at
a
very
high
level,
right,
maybe
through
gradient
descent,
but
it's
still
kind
of
learning
the
structure,
but
it
seems
you
know
I
can
imagine
it
for
smaller
things,
but
even
in
10
years
can
it
scale
to
just
learning
everything
matter.
Learning
everything
seems
really
hard.
D
B
D
B
Well,
they're
doing
that
too.
Well,
there's
something
called
network
architecture
search,
I'm
sort
of
lumping
that
in
with
this
general
category
of
stuff,
so
you
would
start
with
very
you
learn
the
rules
by
which
neurons
operate.
You
learn
the
structure
of
the
neurons,
the
network.
You
learn
the
learning
rules
that
it
operates
by
I
mean.
D
Just
take
something
taste,
something
like
you
know:
the
various
rhythms.
So
we
understand
now.
Hotel
rhythms
are
essential
for
doing
path,
integration
and
learning
reference
frames,
and
so
on
I
mean
it
seems
to
me
that
these
networks
aren't
even
capable
of
capable
of
learning
that
it's
like
it's
it's
not
baked
into
the
the
the
the
overall.
It's
just
not
part
of
I
mean
it's
hard
to
imagine.
Yeah.
D
Maybe
I
don't
understand
it
well
enough.
It
just
seems
impossible
to
do
that,
given
their
their
assumptions
about
what
the
networks
look
like
you
know,
I
mean
they
don't
start
with
a
blank
slate
and
say
let
this
thing
evolve
for
a
billion
years
and
figure
out.
You
know
they
start
with
some
assumption
about
what
the
neural
networks
look
like,
and
then
they
see
it
come
out.
Neural
network
learn
to
do
all
these
things
is
that
am
I
even
correct
about
that.
D
G
D
G
You
go
ahead,
but
I
just
to
what
lucas
offered
to
do.
I
would
be
interested
if,
because
he
brought
up
this
point,
that
meta
learning
doesn't
have
to
proceed
from
assuming
gradient
to
set.
If
you
have
a
taxonomy
of
things
that
would
substitute
for
gradient
descent
or
you
have
that
in
mind
or
you
you
encounter
that
that
would
make
for
an
interesting
talk
as
to
what
alternatives
there
are
to
gradient
descent
that
fit
within
the
metal
learning
environment.
You
know
how
rich
is
that.
F
D
No
evolutionary
algorithms,
you
know
like
evolution,
evolution
like
biological
evolution
works
at
the
genetic
level
and
and
so
through
genes.
You
can
create
all
different
types
of
cells.
You
can
create
all
different
types
of
networks.
You
can
create
all
different
types
of
chemicals
and
interactions,
and
you
know
it's
so
you're
evolving
at
that
sort
of
real
basic
level
that
almost
anything
can
come
out
of
it
or
at
least
there's
some
very
large
large
universal
things
that
can
come
out
of
true
biological
evolution
because
it's
working
at
this
sort
of
protein
gene
level.
D
But
if
you,
if
you
start
with
an
assumption
about
what
neurons
are
like
or
what
the
types
of
you
know,
connections
between
neurons
are
like,
then
then
then,
you've
limited
yourself
and
evolution,
an
evolutionary
algorithm
applied
to
a
complex
standard,
neural
network.
Is
it's
just
not
going
to
be
able
to
get
to
the
complexity
of
the
brain?
What
it
does
because
you're
doing
the
evolution
at
the
parameter
level
or
you're
doing
the
evolution
at
the
you
know,
basically,
at
the
parameter
level,
maybe.
D
Then
even
what
the
learning
rules
are
so,
but
how
I
guess
I'm
just
I'm
ignorant,
then
I'm
sorry,
I
don't
understand
how
every
time
I've
seen
evolutionary
algorithms
applied,
which
is
a
you
know,
a
good
candidate
substitute
right.
It's
been
applied
at
that
level,
not
the
more
fundamental
level
of
you
know
like
okay,
we're
going
to
start
with
biochemical.
G
They
don't
have
to
start
with
a
point
neuron.
The
the
genetic
algorithms
that
were,
like
you
know,
started
out
in
the
80s.
Basically
was
just
a
string
of
bits
that
coded
for
something
coded
for
use
of
a
function
coded
you
know
and
basically
used.
You
know
the
typical
biological
mechanism
of
recombination,
random
selection
and
things
along
those
lines,
but
but
the
elements
of
those
things
how
those?
If
you
want
to
call
them
networks,
they're
they're,
not
they
necessarily
have
to
be
neuronal
networks.
G
There
were
functional
networks
that
you
know,
inputs
come
in,
something
comes
out
type
of
thing,
very
general.
You
can
constrain
it.
You
know
to
try
to
get
interesting
results
like
some
of
the
ones
used
to
have.
The
elements
of
the
networks
were
electrical
components
like
just
a
resistor
capacitor
inductor,
and
try
to
drive
it
to
an
evolution
for
a
particular
type
of
filter,
yeah.
D
G
D
Level,
this
gets
something
you
know
the
basic
theme,
at
least
my
life,
and
the
theme
of
you
know.
Momentum
in
some
sense.
Is
that
a
belief
that
the
you
you
won't
get
there
yeah
unless
you
first
understand
how
the
brain
does
this
stuff,
and
I
I
understand
that
most
machine
learning
people
don't
want
to
do
it
that
way
they
want
to
come
up
with
principal
mathematical
approaches
that
they
can
understand
that
you
know
moves
them
a
long-term
trajectory
towards
a
useful
solution.
Even
if
it's
not
the
solutions
the
brain
implements.
D
G
G
D
Understand
I
understand
that
I
I
just
the
process
is
very
fraught
with
problems,
even
just
what
problems
you're
trying
to
solve,
like
you
know,
is
vision.
Looking
at
images,
no
vision's,
not
looking
at
images
at
all
vision
is
what
happens
when
you
walk
around
and
you're
moving
through
the
world
and
looking
at
different
things
in
different
directions
constantly
several
times
a
second
none
of
it
just
captured
in
any
of
the
benchmarks
that
people
use
in
vision,
computer
vision
and
so
even
there's
just
so
many
ways.
D
You
can
go
wrong
in
this
process
that
I
just
felt
like.
Even
that
insight,
which
is
such
a
simple
insight.
That
vision
is
not
looking
at
images.
There's
nothing
like
it.
You
know
we
have
a
phobia.
We've
got
this
hugely
distorted.
You
know
you
know
weird
stuff
going
on.
That's
constantly
changing
going
into
your
brain.
D
Somehow
you
end
up
with
this
perception,
you're
seeing
an
image,
but
it's
not
so
even
that,
even
that
it's
just
built
in
from
the
beginning,
and
so
I
feel
like
that's
an
insight
that
comes
from
brains
of
studying
the
animals.
That's
what
animal
vision
is
like.
It's
like
this
complex,
dynamic
thing
as
you
move
around
the
world
and
nothing
at
all
like
processing
images,
so
you
can
it
just
there's
so
many
things
that
you
can
dismiss.
If
you
just
think,
okay,
I'm
gonna
apply
some
evolution
algorithm.
Well,
I
think
it's
an
interesting
debate.
D
You
know
we'll
see
right,
we'll
see
how
it
turns
out.
I
just
I'm
just
pointing
out
that
as
long
as
I
thought
about
this,
I've
never
been
convinced.
You
can
engineer
your
way
there,
even
through
evolutionary
type
algorithms
it
just
don't.
I
don't
believe
it's
okay,
I
think
the
far
quicker
way
to
get
there
is
to
first
understand
the
principles
of
how
the
brain
works
and
they
want
to
understand
those
principles,
you're
going
to
have
ways
of
implementing
or
maybe
even
applying,
evolutionary
algorithms
to
them.
D
I
gave
the
examples
of
theta
rhythms
and
you
know
the
oscillations
and
so
on,
and
you
know
it
just
seems
so
hard
for
people
looking
at,
like
computer
vision,
image
recognition
that
they
can
figure
that
stuff
out,
because
this
number
is
going
to
be
important
in
that
path.
That
they've,
given
it
I'm
just
giving
my
biases
here.
H
With
this,
with
this
slide,
michelangelo
kind
of
took
like
a
big
step
back
and
brought
us
on
the
topic
of
like
inductive
biases,
and
can
you
just
do
do
these
big
evolutionary
searches
and
stuff
stepping
back
forward
to
just
like
the
transformer
specifically
and
attention
like
this
one?
One
reminder
we've
talked
about
this
before.
D
D
Yeah,
I
understand
there's
a
clever
algorithm
here
that
yeah
yeah
biologically,
and
I
think
it's
I
think
this
might
be
really
great
for
image
recognition
right.
I
just
don't
think
it's
vision
and
it's
not
really
vision.
So
when
I
think
about
this,
I'm
I'm
sort
of
going
forward
to
like
what
real
machine
intelligence
is
going
to
be
like
it's
not
going
to
be
image
classification.
If
that's,
what
we
want
to
do
is
image
recognition,
image
classification
then
great.
I
think
these
things
could
be
wonderful.
D
D
Sufficient,
no,
it's
not
images
at
all.
There
are
no
images
coming
into
the
brain.
You
have
highly
distorted,
foveal
points
in
the
world
and
you're
scanning
it
around
constantly
as
you're
moving
your
body
and
so
you're.
Looking
at
the
world
different
parts
of
the
world
as
you're
moving
around
constantly
of
different
orientations,
and
so
it's
nothing
about
image,
recognition.
It's
about
learning
a
model
of
the
world.
G
D
I
I
guess
I
guess
you
know.
The
thousand
brain
theory
says
you
get
this,
this
very
complex
motor,
driven,
it's
motor,
driven,
it's
not
a
presented.
Almost
everything
changing
in
your
visual
system
is
because
you
move
and
multiple
different
ways.
You
get
this
very
complex
mode,
driven
signal
and
from
that
you
have
to
learn
to
model
the
world.
It's
not
the
correct,
it's
not
a
build
the
correct
output,
it's
just
to
build
a
model,
and-
and
I
it
just
processing
images-
doesn't
get
you
there.
It
just
doesn't
even
come
close
to
it.
D
D
All
right,
so
mostly
what
we
do
is
we
we
don't
generally
look
at
images
anytime,
we
can.
We
can
take.
I've
always
felt
that
the
the
idea
that,
oh
yes,
a
human,
can
look
at
an
image
and
tell
you
what's
in
it.
Therefore,
let's
solve
that
problem,
that's
a
a
small
subset
and
a
misleading
subset
of
what
vision
is
so
it's
like
saying.
D
Real
biological
vision,
has
the
ability
to
look
at
images,
but
that
is
but
that's
a
subset
of
what
a
very
very
small
set
of
what
vision
is
and
what
how
it's
useful
in
an
intelligent
agent
like
a
human.
So
we
don't,
we
just
don't
want
to
get
hung
up
on
that
process.
It's
like
that's,
not
the
goal
of
vision.
F
E
D
So
that's
yeah,
I
I
think
that's
right
and
and
in
some
sense
that's
what
we
did
with
sparsity
right,
we're
applying
sparsity
in
a
way,
that's
really
not
in
a
networks
that
aren't
biological
networks
at
all,
but
and
so
we're
some
sense.
Oh
here's
a
piece
of
our
theory
and
and
then
you
know,
that's
only
a
piece
and
then
we're
gonna
do
the
same
thing
with
dendrites
and
you
know,
and
and
reference
frames
and
so
on.
But
I
guess
maybe
there's
two
ways
to
look
at
this.
D
I'm
constantly
thinking
about
what
is
the
ultimate
form
of
machine
intelligence,
not
what's
going
to
be
useful
a
year
or
two
years
or
even
three
years
from
now,
and
so
when
I'm
like.
Oh
I'm,
when
I'm
expressing
my
grievances
or
my
my
biases
about
this,
I'm
thinking
like
okay,
where
do
we
have
to
go
completely
in
the
end
and
I'm
not
arguing
what
we
should
do
in
the
next
year?
D
I'm
just
pointing
out
like
this.
These,
the
what
I'm
hearing
is.
I
still
think
there's
going
to
be
you're
not
going
to
get
there,
but
we
still
have
to
finish
our
work.
We
still
have
to
finish
how
the
brain
really
represents
spaces
and
how
it
really
does
transforms,
and
we
have
to
figure
all
this
stuff
out
is
my
argument.
Ultimately,
we
don't
have
to
do
it
right
now.
I
guess
you
know,
does
that
make
sense
yeah
that.
A
Makes
sense-
and
I
yeah
I
do
see
the
value
in
keeping
the
end
goal
in
mind.
I
think
I
also
like
the
idea
of
just
maybe
maybe
this
is
just
from
like
age.
I
don't
know.
I
have
a
harder
time
thinking
in,
like
you
know,
a
lot
longer
term,
or
at
least
more
rather,
I
should
say
kind
of
like
a
like
an
eagerness.
I
guess
just
to
sort
of
see
good
results
in
the
short
term.
D
Yeah
yeah
all
right.
I
understand
that.
But
that's
you
know
that's
how
people
felt
about
machine
learning
techniques
over
the
decades,
and
you
know
maybe
you
know.
Definitely.
My
attitude
is
an
outlier.
There's
no
question
about
it
that
you
know
like
hey.
You
have
to
understand
how
the
range
of
this
first
a
lot
of
people,
don't
want
to
do
that,
for
maybe
it
takes
too
long.
Maybe
they
just
find
it
too
hard
to
think
about
brains
and
biology
because
you
know
maybe
it's
like
you
said
hey.
I
want
to
get
results.
D
So
that's
my
that's
my
bias.
That's
the
whole!
That's
what
the
mentos
all
about
and
that's
you
know
what
I've
been
trying
to
do
all
right.
Enough
said
about
that.
I
think.
Maybe,
unless
someone
wants
to
talk
more
about
it,
I
want
to
make
sure
if
michelangelo
is
going
to
or
something
lucas
you're
going
to
talk
next.
Is
that
what
you
said?
Yeah.
F
F
F
So
one
thing
that
I
came
across,
which
I
found
it's
really
cool,
it's
this
environment.
So
I'm
going
to
divide
it
into
two
parts
when
it's
the
environment
and
the
other
is
the
benchmark
and
it
doesn't
mean
we
have
to
use
the
benchmark,
we
can
just
use
the
environment
which
we
want
so
interactive,
gibson
is
was
built
to
be
an
image
net
for
robotics.
F
F
So
what
interactive
gibson
is,
is
they're,
photorealistic,
simulators
or
they're
built
from
actual
photos
and
what's
different
from
currently
state
state-of-the-art
simulators
is
that
it
includes
physical
interactions
with
objects,
and
it's
allowed
and
it's
encouraged
so
usually
photo
realistic
simulators.
You
can
move
around.
But
if
you
bump
on
a
chair,
the
chair
won't
move.
So
in
this
case
they
they
build
it.
So
it's
it's
an
actual
yeah,
so
the
hub,
you
can
interact
with
the
objects.
F
There
is
a
it's
physics
based
and
what
you
can
do
is
you
can
train
and
evaluate,
robotic
agents
that
use
visual
signals
to
solve
navigation
manipulation,
tests
like
opening
doors,
picking
up
picking
up
or
placing
objects,
searching
cabinets?
It's
just
moving
to
some
place
and
they
encourage
use
from
all
disciplines
in
robotics
planning,
learning,
control,
et
cetera,.
D
F
Yeah
not
just
not
just
and
I'm
gonna
get
there.
So,
okay,
you
can
use
all
sorts
of
inputs.
If
you
want,
you
can
just
use
vision,
input,
but
the
environment
is
gonna,
provide
all
sorts
of
data
like
the
location
of
the
robot,
even
others
like
tap,
sensors
and
other
things,
and
you
can
either
choose
to
use
it
or
not.
F
F
D
F
F
F
Yeah,
they
start
with
real
images
and
that's
what
I
was
getting
at
go
back,
so
they
start
with
real
images,
and
then
they
build
this
model
and
what
they're
doing
now
is
to
bring
interactive
into
a
static
model.
So
it's
all
deep
learning
based
they
do
a
3d
reconstruction
match,
doing
semantic
segmentation
and
then
they
identify
which
objects.
There
are
right,
they
do
instant
segmentation
and
then
they
go
so
they
have
these
proposals
of
where
which
objects
are
where
and
they
use
amazon,
mechanical
turks.
F
So
actual
humans
go
and
review,
and
you
know
just
confirm:
okay,
that
there
is
a
couch
there.
The
chairs
are
just
to
make
sure
it's
correct
and
when
they,
when
they
have
this
data,
annotated
like
here's
a
chair,
here's
a
table,
they
use
a
physics-based
environment,
they
use
a
bullet
which
is
open
source
to
enable
interaction.
F
Yeah
exactly
so,
you
can
turn
over
the
coffee
table
and
since
it's
built
from
photos,
it's
easy
to
transfer
back
to
actual
environment,
so
those
seem
to
real.
So
you
can
transfer
back
to
your
robot
since
it's
it's
built
from
an
electronics.
I
could
take
a
picture
of
your
house
and
go
through
all
this
process
and
I
could
interact
with
your
house.
F
Yeah
exactly
and
so
it's
built
for
research,
it's
mit
license.
It
uses
open
source
bullet
for
physics
simulation,
so
the
entire
pipeline
is
in
python,
spy,
opengl
type
and
there's
just
c
plus
plus
for
the
rating
part
of
the
part
b.
Don't
actually
change
it
supports
up
to
a
thousand
frames
per
second
and
256
by
256
resolution
runs
even
in
common
computers,
the
render
image
they
go,
direct
to
tensions
and
gpus,
and
that's
important
for
us,
because
you
don't
have
to
download
it
host
memory
and
then
transfer
to
pytorch.
D
F
F
Supposed
to
be
an
army
two
mobile
manipulators
with
an
arm
there
is
a
legged
rob
and
a
quadricopter,
so
you
can
choose
what
agent
you
use
and
some
of
the
inputs.
So
you
have
visual
signals.
That
includes
like
image,
but
you
can
also
have
as
input
like
semantic
segmentation
depth
mats,
and
aside
from
that,
you
can
have
information
about
their
own
configuration
like
where
they
are
in
the
floor.
Prem
velocity
collisions
with
environments
and
objects,
motion
and
information
about
navigation
tests
like
position
of
the
goal
closest
waypoints.
So
all
this
information
is
available.
G
G
F
And
actually
so
there
is
a
previous
project
called
gibson,
which
is
from
2018
and
the
difference
between
gibson
and
I
gibson.
It's
interaction.
So
gibson
didn't
have
this,
and
so
you,
you
could
see
the
the
environment.
You
could
navigate
through
it,
but
you
couldn't
like
flip
a
table.
You
if
you
walk
through
the
chair,
the
cargo.
D
G
F
B
F
There,
okay
yeah,
so
I
wanted
to
break
down
into
two
things
because
one
is
the
environment.
So
this
environment
supports
all
sorts
of
tasks.
You
can
define
any
task,
you
want
right
and-
and
they
encourage
use
from
our
disciplines
and
the
idea
here
is
they
give
you
the
environment,
and
then
people
from
like
planning
are
going
to
define
a
planning
task.
F
People
from
learning
are
going
to
find
a
learning
task,
et
cetera,
but
and
just
to
close,
this
they're
working
on
more
semantically
guided
interactions,
for
example
like
turning
on
the
stove
actuation
of
articulated
objects,
like
you
have
an
arm
in
the
arm
has
a
few
joints,
but
you
know
it's
not
as
realistic
as
it
could
be.
Is.
F
D
The
way
they've
broken
it
down
here,
two
separate
things:
the
world
you're
interacting
with,
has
objects
in
it
and
they
can
be
and
that's
articulated,
which
you
know
they
have
behaviors.
That
applies
to
everything,
but
then
the
other
side,
if
you're
trying
to
do
an
articulated,
robotic
arm,
they're
sort
of
leaving
that
out
of
the
picture
here
right,
that's
like
that's
up
to
you.
E
F
D
D
F
Part
of
gibson,
so
they
have
the
agents
available
and
when
you
define
a
basket
you
you
can
choose
the
agent
that's
going
to
play
by
that
right.
So
you're
going
to
choose
an
agent.
Let's
say
you
choose
the
major,
coherent
and
the,
and
that
agent
has
a
set
of
degrees
of
freedom
that
you
can
use
it
to
manipulate
the
agent
right.
So
yeah.
D
You
know
one
of
the
things
that
struck
me
right
away,
but
I
mean
I
think
this
is
wonderful
by
the
way.
One
of
the
things
that
struck
me
right
away
is
you
know
when
we
interact
with
the
world.
So
much
of
what
we
do
is
through
some
out
of
sensory
interactions.
It's
not
even
aware
of
it.
You
know,
like
I'm,
I'm
going,
I'm
gonna
open
the
fridge.
D
I
may
take
a
quick
glance
of
the
fridge
and
then
I
I'm
looking
somewhere
else
as
I
reach
out
and
grab
the
door
of
the
refrigerator,
I'm
not
even
looking
at
it.
When
I
grab
it
and
I
know-
and
I
and
by
touch
I
know
where
my
hand
is,
and
how
adjusted
to
get
there,
so
I
don't
have
to
look
at
the
door
the
handle
refrigerator
when
I'm
opening.
I
only
have
to
like
locate
it
once
then,
I'm
gone,
and
so
it's
so
much
of
what
we
do
like
that.
D
Imagine
like
putting
on
a
t-shirt
right,
it's
in
a
pile
in
front
of
you,
the
pile
of
clothing.
I
can
put
on
a
t-shirt
in
the
middle
of
the
night.
I
do
this
all
the
time
I
can
just
feel
with
my
hands
and
I
feel
where
the
collar
is
and
the
where
the
seam
is
and
just
do
the
stuff
automatically
and-
and
you
can
manipulate
this
really
complex
folded,
piece
of
fabric
and
get
it
over
your
head
in
the
correct
direction
without
ever
seeing
it.
D
So
this
there's
this
blurring
between
what
we
do
with
vision
and
what
we
do
with
tactile
sensations.
So
I
always
feel
like
you
really
can't
you're
going
to
be
limited
in
this
world
if,
if
your
robot
is
relying
completely
on
vision
to
solve
every
problem
without
sort
of
a
true,
so
not
a
sensory
system,
it's
not
it's,
not
a
criticism
of
this.
It's
just
pointing
and
just
making
that
observation.
So
I
was
wondering
if
they,
how
much
are
they
specifying
the
somatosensory
part
of
it?
F
Well,
they
have
some
data
like
they
have
collisions
with
environments
and
objects.
So
that
means
I
mean
if
I
touch
a
chair,
that's
going
to
be
part
of
my
input
data.
If
I
want
to
see
that
yeah,
of
course,
it's
going
to
be
very
far
away
from
the
kind
of
somatosensory
input,
yeah
yeah,
but
I
think
it's
a
process
and
you
need
to
start
and.
E
C
D
No,
I
think
it's
great
again.
I
think
it's
great,
although
it
does
need
to
go
in
that
direction.
It
otherwise
becomes.
You
know
you
know
you
see
these.
You
know
sock,
folding,
robots
or
whatever
they
are.
You
know
they
take
10
minutes
to
pull
the
socks
because
they're
constantly
it's
like
you
know
it's
like
you're
doing
it
with
a
set
of
you
know
like
joysticks
and
you're.
Looking
at
in
a
box,
you
know
trying
to
control
some
arm
with
a
set
of
joysticks.
That's
what
the
robot's
doing
without
it's,
not
a
sensory
system.
F
Yeah,
so
the
normal
questions
on
this
distribute
to
the
tests,
so
the
benchmark
and
the
tasks
they
defined.
So
this
is
not
associated
with
the
environment.
So
I
think
what
they're
proposing
is
two
different
things,
and
this
is
one
task,
one
benchmark.
They
propose
as
something
that
I
gibson
could
do
so
they
proposed
this
benchmark
because
it's
a
convex
combination
of
path
and
effort,
efficient
scores
and
and
here's
a
combination
and
a
path
efficiency
is
so
you
define
a
goal
like
the
agent
has
to
get
across
the
room.
F
F
F
Yeah,
so
that
path
is
whether
you're
getting
there
to
your
objective
or
not,
if
you're
taking
the
shower's
path
and
after
effort
is,
if
you're
hitting
something
in
the
way
like
if
you're
spending
any
effort
like
I
have
to
move
a
table,
I
have
to
move
a
chair,
so
it's
a
combination
of
both
and
you
can
define
you
know
this.
This
alpha
parameters,
how
much
you
want
to
optimize
for
path.
F
You
just
want
the
shortest
path
or
how
much
you
want
to
optimize,
for
if
efficiency
means
I
don't
want
to
bump
into
anything,
and
they
have
this
open
data
board
where
the
test
several
like
enforcement
learning
agent
for
this
test
and
like
ddpg
ppo
sex.
So
these
are
like
state-of-the-art
enforcement
learning
agents
and
here's
how
they
define
alpha.
So
if
alpha
is
zero,
it's
going
all
the
way
to
effort
efficient
like
I
don't
want
to
bump
into
anything.
F
So
I
want
the
solution
where
I
go
like
around
the
room
and
get
there,
and
if
alpha
is
one
I
don't
care
about
bumping
and
having
to
move
chairs.
I
just
want
the
shortest
path.
So
I
want
this
straight
line,
even
if
I
have
to
move
everything
aside
and
there's
like
somewhere
in
between.
So
this
is
a
task
they
defined
to
test
this
environment.
F
And
one
nice
thing
that
we
kind
of
discussed
about
that
is
usually
in
reinforcement
learning.
There
is
no
diffusion
between
training
and
validation
scenes
and
in
this
case,
that's
something
they
thought
about,
and
since
you
have
this
option
of
randomizing,
you
can
actually
do
that.
So
you
have
this
distinction
between
training
and
evaluation.
So
you
don't
actually
see
during
test
time.
You
don't
actually
see
the
same
data
you
soldering
training,
which
I
think
for
reinforcement
learning
it's
a
huge
step.
D
F
D
Funny
the
word
scenes
bothers
me,
but
I
don't
want
to
over
or
emphasize
that
it's
more
like
it's
like
you
know,
the
training
shouldn't
be
a
set
of
scenes.
The
training
is,
you
know,
movements
through
the
space
right.
Are
these
systems
trained
by
just
showing
a
bunch
of
images,
or
I
assume
it's.
It's
an
interactive
movement.
F
It's
interactive,
so
scenes
means
scene
is
like
the
room
where
the
agent
is
going
to
try
and
navigate.
So
I
give
you
a
room
and
those
are
all
rooms,
so
I
gave
you
a
room
to
get
across
that
phone.
So
that's
what
they
are
calling
a
scene
and
then,
if
I
give
you
a
different
room,
that's
going
to
be
another
scene,
but
the
learning
is
is
like
we
discussed
before.
There's
an
agent
and
the
agent
has
to
navigate
through
that
yeah.
D
It's
funny,
I
would
it's
kind
of
weird,
because
the
way
I
think
about
brains
doing
this-
it
wouldn't
be
like
that
and
if
I
show
me
a
different
room,
I
very
quickly
build
a
model
of
it.
I
don't
I
don't
necessarily
generalize
from
the
previous
room.
I
just
said:
oh,
this
is
the
new
room.
I
see
what
it
is.
I
built
the
model
of
it
very
very
quickly
as
opposed
to
having
to
go
through.
You
know
a
lot
of
training
you
know
like
I
can.
I
can
navigate
in
a
new
house.
D
F
D
It
I
got
it,
it
just
shows
a
bias
in
the
thinking
about
it.
I
I
I
don't
want
to
over
it's
just
a
sort
of
bias
and
thinking
about
what
the
solution
might
be
like
it
doesn't
say
you
have
to
do
it
that
way.
Yeah
I
could
be
learning
during
the
evaluation
yeah.
So
I
would.
I
would
argue
that
there
really
shouldn't
be
two
different
things.
F
C
B
C
E
B
D
D
That
is,
you
know,
as
far
as
we
know,
that's
something
that
we
can.
It
can
be
handled
in
the
hippocampal
complex
right.
That's
what
you
know:
grid
cells
and
placeholders
do
and,
as
we've
been
talking
recently,
the
the
process
of
say,
applying
that
same
basic
algorithm
to
manipulating
an
object
like
a
stapler
or
a
smartphone,
or
something
like
that.
It's
not
it's
not
exactly
the
same.
We
were
talking
recently
about
scale
and
the
difference
of
types
of
multiple
sensors
at
different
orientations
and
so
on.
D
So
there
is
a
you
might
end
up
with
a
set
of
solutions
that
work
very
well
for
just
navigating
like
okay,
I'm
facing
this
direction.
I
have
one
you
know
I
can
go
forward
or
you
know
I
can
turn
and
go
forward
back
up.
That
kind
of
stuff
may
not
generalize
to
the
more
the
more
general
solution
of
hey.
How
does
a
hand
pick
up
an
object
and
how
does
even
vision
work
when
I
rotate
the
object
in
front
of
me,
it
may
not
apply
to
that.
D
If
people
may
just
first
settle
on
me,
hey,
let's
figure
out
how
to
get
the
robot
to
navigate
from
point
a
to
point
b
and
not
hit
things
which
is
fine.
I
just
want
to
point
out
that
that's
a
subset
of
what
I
think
is
going
on
the
cortex
and
subset
of
what's
necessary
to
do
the
full.
You
know
machine
intelligence
thing,
but
it's
still
a
useful
thing
to
do
so.
F
D
Right
it
sounds
like
they
had
object,
behaviors
in
there
right,
yeah
yeah,
you
know
so
the
stapler
could
be
picked
up
and
rotated
and
opened
and
pressed
down,
and
that
gets
that
really
gets
to
an
interesting.
You
know
that
gets
that
brings
in
those
other
problems,
because
now
I'm
looking
at
it
from
a
distance,
it
really
would
help
to
have
the
somatocentric
motor
part
of
that
so
anyway,
none
of
this
is
to
be
critical.
I'm
just
sort
of
openly
thinking
about
it
as
you're
presenting.
F
F
B
F
Was
just
released
right
now,
so
they
themselves
they
define
this
task
and
they
implement
it
with
enforcement
learning
agents
to
show
the
capabilities
of
environment.
Okay,
so
they've
done
it,
they
have
done
it
so
right.
I
showed
that
table
of
results
there,
but
I
think
they
just
released.
I
don't
know
if
people
picked
up,
but
you
know
you
know,
for
affiliate
groups,
it's
very
people
behind
imagenet,
so.