►
From YouTube: NeurIPS 2020 Recap - December 16, 2020
Description
Subutai Ahmad, Lucas Souza and Karan Grewal give a recap on The Conference and Workshop on Neural Information Processing Systems (NeurIPS) 2020, which was held virtually on Dec 6-12.
Subutai first gives his impression on the conference and highlights the positives and negatives. He then gives an overview of three papers from the conference that are focused on contrastive learning. Karan Grewal gives some highlights on the workshops he attended, specifically talks on the community being hyper-focused on achieving benchmark performances. Karan also highlights a panel session he attended on the rising trend of bias in machine learning. Lastly, Lucas shares his experience at the conference - highlighting the tutorials, workshops and industry talks.
Papers Subutai mentioned:
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
https://proceedings.neurips.cc//paper_files/paper/2020/hash/70feb62b69f16e0238f741fab228fec2-Abstract.html
Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
https://proceedings.neurips.cc//paper_files/paper/2020/hash/f3ada80d5c4ee70142b17b8192b2958e-Abstract.html
LoCo: Local Contrastive Representation Learning
https://proceedings.neurips.cc//paper_files/paper/2020/hash/7fa215c9efebb3811a7ef58409907899-Abstract.html
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
So
nurip's
2020
was
last
week,
so
this
is
a
really
really
brief
trip
report.
I
didn't
attend
the
whole
thing
I
sort
of
sub
sampled
from
it.
I
tried
to
get
a
bunch
of
stuff
out
of
it.
Lucas
I
think,
tried
to
attend
the
whole
thing
beginning
to
end
as
if
he
was
really
there
full-time,
and
I
think
karan
did
something
somewhere
in
between.
So
this
is
going
to
be
a
pretty
quick
trip
report.
B
A
A
I
think
this
was
definitely
the
best
online
conference
so
far
that
I've
been
to
I've
been
to
three
others
at
least,
and
it's
like
far
better
experience
than
the
other
ones,
I
would
say
from
the
other
ones.
I
got
almost
nothing
out
of
it,
whereas
at
least
with
mirips
I
got
something
out
of
it
and
I
think
clearly,
a
huge
amount
of
work
has
gone
into
creating
the
structure
and
really
kudos
to
all
of
the
conference
organizers
for
putting
a
huge
amount
of
work
into
it.
A
A
So
clearly,
this
type
of
conference
is
far
more
accessible
and
and
it's
hard
to
overstate
the
importance
of
this,
I
think
it
was
extremely
cheap
and
if
I
recall
correctly,
they
were
very
generous
about
just
giving
waving
the
registration
fee
altogether.
For
anyone
who
even
said
they
could
had
a
hard
time
affording
it.
So,
basically
anyone
could
attend
it.
A
There
was
no
limits
on
registration,
as
far
as
I
could
tell
their
schedule
was
around
the
clock
they
had
stuff
going
on
for
every
time
zone
in
the
world
which
is
really
interesting
and
and
nice
it
was
fantastic,
I
think,
for
students,
those
with
physical
challenges.
You
know,
if
you're
in
a
wheelchair
or
anything
like
that
international
researchers,
anyone
with
funding
issues
and
so
on
so
pretty
much.
You
know
every
everyone,
almost
everyone
can
get.
The
entire
conference
experience
again
it's
hard
to
overstate
this.
A
I
think
it's
more
so
this
time
around
and
the
full
conference
materials
are
available
afterwards.
So
this
is
going
to
be
a
great
resource
for
later,
as
well
and
they're,
going
to
open
it
up
for
everyone
soon,
and
I
felt
the
poster
session
was
really
the
best
part
of
it.
Originally,
it
felt
hokey
there
was
this
gather,
town
ui.
A
I
think
I
showed
that
in
a
previous
research
meeting,
where
you
had
this
sort
of
virtual
field,
where
there
were
like
these
simulated
tables
any
little
icon
that
would
walk
around
and
and
go
to
different
things,
and,
as
you
got
closer
to
a
poster,
you
would
start
seeing
the
other
people
there
and
then
you
could
press
a
key
and
you'd
see
the
full
poster
in
front
of
you
and
the
videos
of
everyone
there
and
you
could
then
start
interacting,
and
it
was
all
based
on
kind
of
this.
Physical
proximity.
A
It
was
for
the
poster
sessions
and
for
some
of
the
socials
they
did
the
same
thing
and
maybe
four
workshops-
I
don't
know,
but
it
when
I
first
heard
about
it,
it
felt
really
hokey,
like
people
were
trying
to
force
feed
physical
thing,
but
actually
it
was
really
fun
and
work.
Well,
I
thought
so
that
was
that
was
that
was
pretty
nice.
A
Oh,
I
should
say
this
is
far
better
than
let's
say
what
happened
at
nisits
with
posters,
where
it
was
really
almost
zero
interaction
with
the
posters.
As
far
as
I
could
tell
so
this
this
was
really
nice.
I
think
the
overall
ui
for
the
conference
was
better
than
past
years.
I
think
because
they
were
forced,
they
knew
everything
was
going
to
be
online.
It
wasn't
a
secondary
thing.
A
A
C
A
A
A
Yeah
yeah
in
the
ui
it
was.
It
was
like
a
space
invaders
kind
of
field.
If
you
can
imagine
like
a
grid
of
posters
with
a
lot
of
space
in
between,
but
right
great
up,
posters
and
you
could
see
everyone
else's
icon,
so
you
could
see
where
people
were
clustered
and-
and
you
know
like
in
the
real
europes,
it
was
fairly
crowded.
A
D
A
D
A
You
can
glean
that
from
the
interface
it
was
a
little
hard
because
nerfs
is
so
huge.
They
had
many
post
many
of
these
gather
town
rooms,
so
it
was
a
little
hard
to
go
back
and
forth
between
them,
but
in
in
there
were
aspects
of
this
that
were
better
than
the
physical
interaction
like
the
three-minute
video
recording
of
the
poster,
and-
and
you
know
you
could
actually
in
europe's
normally
the
poster
session-
is
huge,
very
crowded.
It's
actually
hard
to
move
it's
it's
sort
of
it's
like
a
japanese,
crowded
rush
hour,
subway
thing.
A
If
you're,
you
know,
if
you
remember
my
pictures
from
last
year,
but
it's
actually
hard
to
move
from
place
to
place,
whereas
here
it's
easy
to
move
from
place
to
place,
and
so
that
was
that
was
kind
of
nice.
So
there
are
aspects
of
this
actually
better
than
the
physical
experience,
and
you
know
I
think
you
saw
from
my
slack
message.
I
I
was
really
taken
by
the
keynote
by
charles
isbell.
A
I
think
he
took
he
exploited
this
online
format
and
did
a
really
high
production
value,
video
with
some
great
messages
and
very
thoughtful
video
on
there.
So
that
was
pretty
cool.
You
don't
normally
see
that
in
talks
and
clearly
this
is
great
for
the
environment.
There
was
no
plane,
you
know,
no
one's
flying.
Planes
are
around
the
world,
you
don't
have.
You
know
ten
thousand
people
flying
into
vancouver
or
wherever
it
is
so.
A
A
The
real
benefit
for
me
of
conferences
was
not
so
much
the
talks
as
really
meeting
people
and
having
meaningful
conversations
and
I'm
sure
it's
possible,
but
I
didn't
really
experience
any
of
that.
I
think
if
I'd
done
it
in
depth
as
much
as
lucas
did
and
really
really
forced
myself
to
find
people
and
do
this-
maybe
I
could
have
done
it,
but
I
didn't
really
it
sort
of
happens.
Naturally,
in
a
physical
conference.
I
didn't
it
didn't
happen,
naturally
here
so
to
me
that
that's
a
negative
for
me
also.
A
I
I
really
like
having
a
few
times
a
year
where
I
just
go
to
a
different
physical
location
and
I'm
there
for
several
days.
It's
a
very
helpful
change
of
context
for
me
and
yeah.
I
take
advantage
of
that.
I
have
you
know
chunks
of
time
to
myself
in
a
different
place.
That's
and
I'm
often
I've
often
been
very
creative
from
a
research
standpoint.
A
Every
year,
which
I
didn't
really
get
in
2020
and
this
you
know
kind
of
like
jeff
was
saying
you
know:
there's
gather
dot,
town
there's
actually
like
a
plethora
of
services
and
accounts.
You
need
to
create
for
to
exp
these
various
things.
I
found
that
really
annoying
and,
for
you
know
some
of
the
things
like
socials
they're,
like
three
or
four
different
accounts.
A
You
have
to
create
and
passwords
and
all
of
this-
and
I
know
I'm
gonna-
get
marketing
messages
now
from
them
and
it's
just
annoying
and
there
was
no
integrated
thing
and,
and
it's
understandable,
it's
not
a
fault
of
neurops
there's
no
way
they
could
have
created
all
of
these
services
themselves.
So
it's
nice
that
they're
there,
but
at
the
same
time
it
would
have
been
nice
if
the
signing
and
authorization
process
was
sort
of
taken
care
of
so
karan
pointed
this
out
to
me.
A
You
know,
despite
all
of
this,
they
didn't
have
the
wova
app
which
they've
had
for
the
last
couple
of
years
in
physical
conferences.
This
is
an
app
that's
on
your
phone
that
you
use
to
connect
with
other
people
at
conferences,
and
I
met
a
lot
of
people
through
the
volvo
app
in
the
conferences
and
I
think
something
like
that
would
have
been
actually
really
useful
here
and
they,
as
far
as
I
know
they
didn't
have
that.
At
least
I
didn't
see
it.
A
A
That's
what
I'm
hopeful
for
down
the
road.
So
there's
not!
This
shouldn't
be
a
negative.
I
guess
this
is
kind
of
a
aside.
So
that's
what
I'm
hoping
will
happen
and
it's
it's
kind
of
in
retrospect.
It's
good
that
in
2020
a
lot
of
this
stuff
was
people
were
forced
to
innovate
and
forced
to
do
things
faster
than
they
might
have
otherwise,.
C
A
They
could
at
least
do
it
so
that
if
you
weren't
there
physically,
you
could
still
get
a
pretty
good
experience,
and
so
some
of
the
stuff,
for
example,
having
these
little
recordings
of
the
posters,
are
having
a
lot
more
focus
on
a
nice
ui.
Things
could
be
good.
I
think
it'd
be
actually
kind
of
cool
to
have
a
physical
and
a
virtual
poster
session
for
every
poster,
because
each
one
has
sort
of
different
benefits
to
it.
I
don't
know
if
that's
possible,
there's
the
problems
with
that,
but.
A
C
A
Couldn't
have
been
live
yeah
exactly,
they
could
do
the
stage
performance.
Yes,
I
think
you
know
you
could
select
some
people
to
have
longer
video
recordings
that
they
could
submit
in
advance
and
maybe
make
sure
I
mean
they
all
have
posters.
Anyway,
all
the
orals
have
posters,
I
think
so.
You
just
have
a
poster
session
for
them
and
they
maybe
just
get
a
spotlighted
recorded
talk.
I
don't
know
it
makes.
A
B
A
A
A
A
Think
they
had
spotlights,
I
think
they
had
so
in
the
past,
what
they
would
well,
I
don't
know
actually
in
the
past
what
they'll
do
is
they
have
oral
talk,
which
is
you
know,
20
minutes
or
whatever?
Then
they
pick
a
few
posters
that
that,
where
they
each
have
two
or
three
minutes
talk
to
to
give
a
spotlight
on
their
poster
session,
so
they're
in
front
of
everyone
in
this
big
room
and
they
get
three
minutes
to
explain
their
their
poster.
B
But
during
the
conference
the
difference
was
in
the
sessions
there
are
sessions.
The
oral
talks
were
a
bit
longer
like
10-15
minutes
with
questions
afterwards
and
the
spotlight
were
like
five
to
eight
minutes
and
they
were
like
in
the
sequence
of
three
or
four
and
just
time
for
questions
at
the
end.
So
the
difference
was
just
like
time
allotted
for
it.
Yeah
yeah.
A
And
yeah,
so
that
that
you
know,
maybe
you
could
still
do
that
and
maybe
some
posters
get
more
than
three
minutes
to
talk
about
them.
I
found
the
three
minute
recording
to
be
ideal
because
you're
gonna
see
the
details
in
the
poster
in
the
paper
anyway
and
the
three
minute
you're
forced
to
kind
of
describe
the
highlights
of
it,
and
I
found
that
actually
more
helpful
than
the
longer
talks.
B
A
A
B
D
A
No,
there
was
nothing
like
that.
As
far
as
I
could
see
just
physically,
you
could
see
where
people
you
know
their
people
are
gathering.
I
know
there
were
some
meetups
that
were
that
would
highlight
specific
papers
or
posters
after
the
fact.
So
there
were
things
like
that,
but
nothing
like
a
ratings
or
a
review
system
or
anything
like
that.
Yeah.
D
The
thing
I
used
to
like
about
siggraph,
I'm
not
sure
they're
still
doing
it,
was
that
they
finally
came
down
to
a
format
where
you'd
all
gather
in
a
room,
but
what
they
do
is
each
each
paper
would
basically
give
like
a
one
minute.
Quick.
You
know,
blitz
of
you
know,
you
know
with
the
iconic
here's
the
images,
here's
the
concept,
here's
the
thing
please
come
see
our
talk
type
of
thing.
It
was.
A
D
A
For
me,
I'm
not
sure
I
would
find
a
rating
system
helpful
because
my
my
interest
and
to
be
different
from
90
of
machine
learning
people,
so
I
would
rather
have
a
much
better
ui
for
finding
things
by
topic
or
keywords.
I
found
the
search
functions
not
to
work
too.
Well,
they
had
this
visualization.
If
you
want,
I
can
show
it
where
they
clustered
different
papers
and
they
had.
You
know
you
could
look
at
what
these
clusters
were.
I
didn't
find
that
very
helpful.
D
B
Yeah,
but
I
I've
missed
something
like
that-
I've,
I
really
didn't
like
the
representations.
I
think
the
categorization
of
papers
were
not
good,
so
there's
a
category
like
continuous
lash,
meta,
slash,
missed
learning
and
they're,
like
all
sorts
of
paper
in
there
yeah
each
other
at
all,
so
it
was
kind
of
a
mess
and
I
gave
up
on
them
real,
quick,
but
the
workshops
were
amazing
or
the
best
I've
ever
seen,
and
I
think
the
workshops
they
do
a
really
good
job
of
like
filtering.
A
A
Exactly
okay,
so
I
picked
three
papers
to
just
talk
about
briefly
and
then
we
can
switch
to
either
karana
lucas,
and
maybe
one
of
you
could
go
over
the
workshops,
because
I
didn't
spend
as
much
time
on
that.
So
I
just
picked
three
papers
on
the
same
topic,
which
is
contrastive
learning,
and
I
I
thought
there
was
a.
There
were
a
lot
of
interesting
papers
on
this
topic
in
at
nurifs.
So
to
me
that
was
kind
of
had
to
pick
one
focus
that
would
be.
A
A
You
know
if
you
have
a
model
of
an
object
you
want
to
as
you
move
around
it
you
want
to.
You
want
to
be
able
to
predict
what
you're
sensing,
so
this
is
defined
very
generically
and
broadly
as
just
different
examples
of
the
same
category.
So
you
want
to
have
this
forces
the
network
to
have
representations
that
are
as
not
quite
invariant
but
informative
of
the
same
category.
A
So
if
you
have
two
dogs,
they're
going
to
have
very
similar
representations
versus
a
dog
and
a
mountain
or
something
right
and
and
the
way
they
do,
that
is
so.
You
want
to
do
a
good
job
at
predicting
different
examples
of
the
same
category
and
you
want
to
do
a
bad
job
at
predicting
examples
from
different
categories.
C
A
A
A
D
Yeah,
because
because
I
was,
I
was
thinking
here,
what
it
interprets
as
a
difference
could
be,
they
can
make
a
poor
choice
as
to
what
a
difference
is,
as
as
it
was
kind
of
as
jeff
was
kind
of
mentioning.
So
if
you
have
a
pre-trained
one,
but
then
starting
with
a
whole
new
set
of
categories,
at
least
you
have
some
basis
on
on
similarity
to
informant.
Otherwise,
I'm
just
wondering
what
it
actually
comes
up
with
as
a
difference.
A
Yeah
and
I
think
in
most
cases
it's
just
whatever
happens
by
chance
and
most
things
will
be
quite
different
and
very
few
things
will
be
similar
if
you
have
a
large
enough
data
set,
and
so
it
yeah.
But
maybe
you
could
do
I'll
talk
about
a
few
variations
of
this
that
that
I
thought
were
interesting
that
maybe
speak
to
this
stuff.
A
A
I
think
the
distinction
is
kind
of
blurred
between
unsupervised
and
self-supervised.
So
we'll
do
that.
So
you
know,
of
course,
no
labels
are
required
for
the
most
part,
except
you
have
a
very
simple
classifier
at
the
top
for
actually
doing
the
categorization,
and
this
has
been
gaining
traction
over
the
last
couple
of
years
really
over
the
last
year
and
there
were
a
bunch
of
interesting
papers
on
this
at
europe's.
I
picked
three
to
talk
about
and
I
I
should
caution
you.
A
Well,
that's
a
good
question.
There's
different
variations
of
it,
so
maybe
I
should
I
should
change
this
in
the
classic
one.
You
do
need
you.
You
do
have
labels,
because
you
need
to
know
they're
different
or
the
same.
So
you
do
need
to
know
that
yeah,
but
in
in
some
of
these
variations
you
don't
actually
have
labels,
except
for
the
classifier
right
I'll,
see
yeah
so
that
yeah.
So
I
should.
I
should
tweak
the
sentence.
C
A
Yeah,
so
some
of
some
of
what
they
do,
for
example,
is
they
take
a
batch
of
images,
you're
learning
one
image,
one
and
you
just
assume
all
the
rest
of
the
images
in
the
batch
are
from
a
different
category,
and
so
that's
not
going
to
be
strictly
true.
But
if
you
have
a
thousand
categories
you
know
the
vast
majority
of
them
are
going
to
be
from
different
categories.
C
C
A
A
B
A
B
A
A
There's
a
clustering
thing
there,
where
you
try
to
find
views
that
don't
do
well
at
predicting
one
another.
So
that's
kind
of
the
contrastive
piece
in
some
sense
and
they
also
have
a
way
of
doing
multi-resolution
augmentation.
So
you
can
also
compare
views
at
different
scales
and
the
nice
thing
about
this
is
that
it's
all
done
with
augmentations.
A
So
you
can
do
it
all
in
the
same
memory
footprint
essentially-
and
you
don't
need
to
do
all
these
pairwise
comparisons
with
other
images,
and
so
it's
very
efficient.
It
can
be
all
done
in
kind
of
the
same
loop
and
what
they
found
is
doing
this.
I
think
this
was
the
best
results
I
saw
with
resnet50.
They
got
about
75.3
top
one
accuracy,
which
is
really
close
to
the
best
supervised
examples
which
are
somewhere
around
70.
A
B
A
A
So
if
you
have
a
view
of
an
object
like
if
you
have
a
view
of
a
car
from
the
side
versus
one
from
the
bottom
there,
this
different
views
of
the
same
object,
but
you
can't
really
it's
not
easy
to
predict
one
from
the
other,
and
so
you
might
want
to
have
both
of
you
might
want
to
have
both
of
those
views,
as,
as
you
know,
clusters
of
this
object
so
anyway,
I
need
to
spend
more
time
on
this.
I'm
I'm
probably
decim.
You
know
massacring
the
actual
technique.
A
The
second
one
was
one
where
another
one
where
you
don't
really
need
negative
samples,
and
what
they're
doing
is
this
is
the
the
top
one
is
the
main
network,
you're
learning,
and
then
they
have
a
moving
average
of
the
top
network,
as
your
contrastive
network
is
the
one
below.
So
it's
a
slowly
moving
average
of
that.
A
It's
not
learned
as
well
as
this
network
and
what
you
do
is
you
take
the
same
image
and
you
do
different
views
or
augmentations
of
it,
and
you
try
to
make
sure
that
they
don't
are
not
good
at
predicting
one
another.
So
it
uses
an
older
version
of
the
same
network,
essentially
as
the
negative
target,
and
so.
A
Yeah
yeah,
it's
just
this
is
a
network,
that's
not
as
good
as
this
one.
In
some
sense,
it
was
kind
of
intriguing
to
me
because
you
have
different
from
a
neuroscience
standpoint.
You
have
different
scales
of
plasticity
going
on,
and
so
I
don't
know
if
there's
any
correlation,
but
anyway,
this
is
a
slower
learning
network
and
this
is
the
more
current
network
there.
B
B
A
Yeah,
so
I
need
to
look
at
that
a
little
bit
more
and
the
third
one
was
in
the
vein
of
local
contrastive
representation
learning
so
that
the
traditional
one
is
an
is
on
the
left.
Here
you
have
a
traditional
feed
forward
network
and
you're
doing
this
contrastive
error
right
at
the
top,
and
you
still
have
to
do
back
prop
all
the
way
back.
A
A
So
this
is
much
more
biologically
plausible,
there's
no
sort
of
backdrop
throughout,
but
you
you
still
need
to
do
this
contrast
of
loss
at
every
level
and
she
got
nice
results
there,
but
not
as
good
as
the
full
end
to
end
and
what
this
paper
loco
does
is
sort
of
builds
on
cindy's
work
by
adding
a
feedback
connection
between
two
layers.
So
it's
somewhere
in
between
here
and
there
it's
like
a-
and
here
they
are
doing
back
prop
in
this
red
arrow
they're.
A
Still
doing
back
prop,
but
it's
intriguing
to
think
about,
can
we
do
it
as
an
actual
real
feedback
connection?
That's
more
of
a
you
know:
backwards
prediction!
If
you
will
and
this
by
doing
by
adding
just
one
of
this
backward
connection,
they
were
able
to
get
significantly
better
accuracies
than
cindy's
thing,
so
you
just
have
need
a
little
bit
more
information
between
levels.
You
can
do
it.
A
Yeah
so,
overall,
you
know
it's
just
very
intriguing
to
think
about.
What's
going
on
with
contrastive
loss
and
how
it
relates
to
prediction
in
the
brain,
I
might
not.
I
think
the
intuitive
idea
of
finding
things
that
you
want
to
be
different
from
is
a
really
nice
idea
and
if
it
would
be
really
it's
just
really
interesting
to
think
about.
That
is
something
like
that
going
on
in
the
brain,
as
you
can
see
lots
of
different
ways
of
doing
it,
it's
not
really
the
labels.
A
C
Yeah,
I
don't
know
the
only
thing
that
jumps
out
of
me
is
like
yeah,
the
sparsity
plus
boosting
right.
You
know
you're
forcing
representations
to
spread
apart,
but
but
this
is
I
I
have
to
think
about
it.
It's
not
clear
to
me
immediately,
like
nothing
jumps
to
my
mind,
saying
oh
yeah.
This
is
a
major
principle
be
going
on
the
brain.
It's
like
I
don't
know
it's.
A
A
little
strange,
it's
a
little
strange,
some
of
the
one
work
they've
done.
Is
they
you?
You
try
to
predict
the
image
patch?
That's
right
next
to
you,
so
you
can
think
about
a
cortical
column
trying
to
predict
what's
in
the
neighboring
cortical
column,
which
is
similar
to
how
our
lateral
connections
work.
A
C
Know
I
mean
I
mean
that
doesn't
even
seem
right
to
me.
It's
like
well,
depending
how
far
away
you
are.
I
mean
you
would
if,
if
an
image
was
occupying
large
partner
retina,
you
would,
it
seems
like
a
goal,
would
be
everyone's
trying
to
reach
the
same
consensus,
we're
all
looking
at
the
dog
or
look
at
the
mountain,
and
I
just.
A
C
C
F
Yeah,
so
it
is
intriguing
to
me
because
variations
on
this
are
tempting
to
me
when
I
think
about
building
predictive
models
when
I
think
of,
if
you
want,
if
you
want
say
a
cortical
column,
to
learn
a
representation
of
its
input,
not
not.
For
categorization,
I
mean
I'm
just
putting
that
aside.
Okay,
you
want
cortical
column
to
do
something.
Spatial
pooler
like
you,
wanted
to
represent
the
input
somehow
and.
F
F
The
critical
column
to
learn
is
as
you,
you
want
this
input
to
have
a
low
anomaly
score
when
this,
when
this
input
arrives
again,
you
you,
you
want
to
not
be
surprised,
but
if
a
different
input
arrives,
you
want
to
be
surprised
like
if
the
input
from
like
five
meters
over
arrives
or
turn
the
head
to
the
left.
That
input
arrives.
You
want
that
to
cause
surprise,
and
so
this,
this
kind
of
gives
you
a
a
a
complete
objective.
Where
you
want
the
current
input
to
map
to
something.
F
C
A
Yeah
so
the
question
the
to
translate
into
kind
of
plasticity
rules
it
would
be,
you
know,
you'd
want
to
do
something
like
and
learning
or
you
know
for
situations
where
you
know
it's
it's
the
same
thing
and
you
want
to
do
anti-have
been
learning
from.
If
you
somehow
know
that
it's
has
is
different
in
some
way
and
and
it
you
know,
this
paper
shows
that
that
could
be
as
simple
as
just
the
representations
from
some
time
ago.
A
A
To
me,
cindy
low's
work
was
really
eye-opening,
because
it's
it's
the
first
time.
I've
seen
local
completely
local
learning,
rules
that
do
as
well
as
a
completely
end-to-end
global
training
system,
and
it's
been
very
difficult
to
figure
out
how
to
stack
layers
where
the
stacking
actually
helps
in
traditional
unsupervised.
Learning.
B
B
So
they
started
using
saving
older
versions
of
the
agents
and
play
against
those
and
and
in
order
to
specialize
against
something
they
started
to
create
like
forks
of
those
agents
that
were
specialized,
for
example,
attacking
I
don't
know,
attacking
something
or
building
something
else
like
in
starcraft
and
then
they
would
play
against
those
versions
of
agents,
so
they
would
like
somehow
somewhere
along
the
way
they
would
save
versions
of
the
mod
and
use
those
like
to
to
train
their
newest
version.
So
yeah
this.
A
Is
something
actually
jerry?
Tesara
talked
about
a
lot
in
his
background
and
td
game
and
stuff,
because
he
was
worried
about
that.
When
you
know
a
lot
of
what
alphago
does,
I
think
just
builds
on
jerry's
work,
so
in
td
gammon
he
was
very
worried
initially
that
that
the
network
would
just
go
into
some
sort
of
a
corner
case
where
it's
just
playing
in
a
very
simple
version
of
the
game
and
just
and
not
really
learning
strategy.
A
So
what
it
it
because
you're
just
playing
yourself,
you
could
just
do
a
very
simple
version
and
and
do
okay.
You
wouldn't
necessarily
do
well
against
a
master
background
player
or
someone
who
might
have
a
very
different
strategy.
So
he
did
a
lot
of
stuff
with
randomization
and
that
gaming
is
normal,
naturally
random.
Because
of
the
because
there's
a
role
of
a
die
which
really
helps
and
in
go.
You
don't
have
the
role
of
a
die.
A
C
E
Yeah,
I
can
share
a
few
things
about
the
workshops
that
I
noticed
so
a
couple
of
trends
that
I
I
think
I'll
touch
on
two
points
from
the
workshops,
which
are
my
favorite
part.
So,
first
there's
this
big.
I
guess
concern
right
now
in
the
field
about
you
know
right
now,
people
are
really
focusing
on
getting
high
accuracy
in
their
models
and
and
that's
really
what's
driving.
You
know,
publications
forward,
and
so
you
know
a
lot
of
people
like
especially
joshua
benji,
for
example.
E
He
had
a
talk
at
a
workshop
where
he
talked
about
you
know.
This
is
not
the
right
incentive
structure
for
researchers
and
he's
proposed
a
couple
alternative
methods
that
he
thinks
would
be
better,
and
so
that
means
like
focusing
less
on,
I
guess,
having
fewer
conference
papers
and
focusing
more
on
having
high
quality
papers
as
opposed
to
a
lot
of
the
papers.
We
see
today
because
a
lot
of
the
papers
we
see
today
at
conferences.
E
You
know
the
accuracy
is
really
high
in
all
these
models,
but
I
think
in
terms
of
the
impact
in
the
long
run
it
doesn't
really.
It
doesn't
really
go
too
far,
and,
and
so
his
talk
made
me
remember
three
years
ago,
when
I
was
at
europe's
I
spoke
to-
I
was
speaking
with
someone,
and
they
were
telling
me
you
know.
70
of
the
papers
here
are
probably
going
to
be
forgotten
in
about
three
months
from
now,
and
I
feel
like
that
is
many
times
the
case,
so
that
that
was.
E
Yeah,
so
he
he
did.
I
didn't
write
it
down
exactly,
but
from
what
I
remember
it
was.
It
was
along.
The
lines
of
you
know
have
have,
I
guess
people
they
shouldn't
when
reviewers
are
evaluating
papers,
one
met
one
thing
they
should
look
at
apart
from
you
know,
accuracy
on
benchmarks
is:
how
far
is
this
going
to
go
in
the
future
like
how
like
is
it
going
to
have
a
really
long
term
impact?
C
Five
years
ago
I
had
this
lunch
with
andrew
ing,
and
I
was
talking
about
this
topic
and
he
he
said
to
me
says
you
know
your
work
is
not
going
to
be
paid
attention
by
anyone,
because
all
they
care
about
is
you
know,
benchmark
performance
and
you're.
Not
talking
about
that,
and
so
no
one's
going
to
pay
attention
to
your
work,
and
I
would
argue
with
at
the
time
like
well,
you
have
these
new
ideas
and
the
ideas
are
important
because
no
one
cares.
He
said.
So
it's
interesting
that
just
interesting.
F
A
Well,
maybe
it's
not
the
opposite,
but
I
I
could
the
thing
about
doing
this
online
conferences
and
stuff.
Is
that
there's
no
physical
limit
to
what
you
can
have
in
the
conference
right
and
so
why
not
make
it
a
much
broader
pool
instead
of
having
a
really,
I
think
what
you're
saying
is
he's
saying:
I
have
a
smaller
set
of
papers.
A
I'm
saying
have
a
larger
set
of
papers
have
a
very
lightweight
review
system,
maybe
it's
sort
of
open-ended
and
stuff
and
make
the
reviews
openly
available
and
some
ways
to
sort
by
reviews,
but
everyone
can
sort
it
themselves
and
why
and
then
those
who
are
interested
in
benchmarks?
That's
fine!
I
mean
let
them
let
them
find
the
papers
that
have
good
benchmark
reviews,
but
for
those
who
are
looking
for
novel
things,
let
them
let
them
find
it,
and
if
you
have
something
novel
that
you
know,
allow
you
to
present
it
at
this.
A
C
F
A
C
That
I
think
in
the
way
you
looked
at
the
different
sides
that
I
looked
at
I
looked
at
hey.
We
should
be
trying
to
put
a
light
on
people
who
aren't
just
doing
benchmark
performance.
You
know-
and
maybe
that's
what
you
said
karen
they
had,
but
but
the
idea
you're
pointing
out
that
he
was
saying,
let's
limit
the
other
ones.
I
agree
with
you.
We
shouldn't
limit
anything.
You
know
that
yeah.
C
Yeah,
I
I
heard
that
around
saying
we
should
we
should
give
more
credence
and
attention
to
papers
that
aren't
like
that,
which
I
could
still
argue
it
just
says
do
more
as
opposed
to.
I
could
argue
that
I
like
that
you
know
again.
My
conversation
with
andrew
eng
was
like
disappointing
right,
so
I
liked
it.
C
A
But
but
who
decides
what's
high
quality
if
you
have
fewer
papers,
it's
like
it
gets
more
and
more
elite
and
it's
you
know
someone
still
has
to
decide
what
high
quality
means
and
it's
really
hard
to
predict.
What's
a
good
paper
yeah,
you
know,
what's
really
going
to
be
impactful
years
from
now,
it's
really
hard
to
predict
that.
A
B
E
Why
we
see
so
few
is
because
you
know
the
accuracy
generally
tends
to
be
the
bottleneck
for
people
just
because
a
lot
of
these
initial
methods
aren't
able
to
get
aren't
able
to
compete
with
the
accuracy
that
a
lot
of
these
other
techniques
are
getting
that
you
tend
to.
You
tend
that
this
area
tends
to
be
very
underexplored
in
conferences
like
in
europe's.
E
I
guess
the
machine
learning
field
is
bias
in
machine
learning
and
so
the
link,
the
talk
that
subatai
linked
to
by
charles
isbell,
talked
a
lot
about
this.
How
you
know
machine
learning
models
can
be
inadvertently
racist
because
that's
just
the
data
that
they've
been
trained
on
and-
and
so
I
was
listening
to
a
panel
with
with
with
a
couple
people
and
then
they
they
brought
up.
E
So
as
compute
has
gotten
better,
it's
the
simpler
methods
that
have
won
out
and
all
the
really
fancy
ideas
have
sort
of
died
away,
and
so
rich
sutton
is
arguing
here
that,
like
you
know
it's
it's
the
compute,
that's
going
to
ultimately
drive
changes
in
the
future,
and
I
think
most
people
in
the
field
do
agree
with
that.
But
one
thing
this
panel
brought
up,
which
I
found
very
interesting,
was
that
they
said.
Okay,
sure
I
mean
it
depends
what
your
objective
is
in
the
end
right.
E
If
your
objective
is
just
to
get
higher
accuracy,
then
sure
maybe
this
is
higher.
Compute
is
going
to
win
out,
but
the
cost
of
this
is
that
you
know
we're
seeing
things
like
bias
and
machine
learning,
which
is
a
result
of
if
you're,
just
you
know,
having
fewer
and
fewer
inductive
biases
than
just
relying
on
compute,
and
so
that's
I
think
so.
E
B
If
I
can
just
put
things
in
perspective,
I
think
we're
talking
about
accuracy
as
like,
supervised
learning
is
the
only
paradigm
machine
learning,
but
half
of
it
is
reinforcement.
Learning
that's
helpful.
That's
where
reach
is
coming
from
yeah
reinforcement,
learning.
There
are
several
ways
of
defining
reward
and
the
way
you
define
your
reward
or
whatever
or
a
function
to
define
intrinsic
reward,
defines
the
problem.
You're
optimizing
for
so,
for
example,
multi-agent,
reinforcement,
learning
you
can
define
objectives
which
are
not
related
to
maximizing
the
reward.
B
You
can
define
objectives
related
to
fairness,
for
example,
making
sure
that
everyone,
every
agent
gets
at
least
a
little
bit,
or
at
least
you
know
enough,
or
you
can
define
objectives
related
to
maximizing
the
total
utility
if
you
want
so
there
are
several
different
ways
of
defining
that
reinforcement
learning.
So
it's
not
you
can
still
customize
and
make
it
fair
through
the
way
you
create.
You
define
the
problem
right
and
that's
different
from
just
the
accuracy
problem
supervisor.
E
E
The
one
thing
that
was
really
great
from
a
workshop
perspective
at
this
conference
is
that
you
know
usually
I
I
want
to
go
to
a
lot
of
different
workshops
because
they
all
have
panels,
and
I
like
listening
to
panel
discussions
and
getting
people's
views
on
things
but
and
then
usually,
if
you're,
if
you're
there
in
person,
you
have
to
run
around
between
workshops.
But
here
it's
like
I
could
be
in
four
workshops
at
the
same
time
and
just
have
three
of
them
muted.
I
could
go
between
them,
so
yeah.
A
B
Yeah
in
terms
of
experience
having
some
time,
that's
everything
what
I
can
add
is
I
was
excited
to
watch
it
live,
but
the
conference
for
me
was
five
to
eleven
a.m
and
five
to
11
p.m,
which
was
really
bad
time
for
me,
especially
because
at
evenings
I
have
a
hard
time
managing
the
pain.
So
I
mean
it
wasn't
an
option
for
me
to
watch
a
conference
five
to
eleven,
so
I
had
to
discard
the
live
part
of
it.
B
I
think
it
was
a
great
format,
but
it
was
kind
of
unusable
if
you
really
like,
try
it
and
use
it
because,
at
least
for
me
I
had
a
lot
of
connection
issues
and
by
connection
issues
I
mean
I
was
like
talking
to
someone
and
it
would
drop
for
like
five
seconds,
but
those
five
seconds
were
enough
for
me
to
like
lose
the
trout
of
the
conversation,
and
that
happens
so
many
times
at
some
point
you
just
give
up.
You
know.
B
So
if
there
are
no
connection
issues,
I
think
it
would
be
amazing,
but
with
the
connection
issues
after
two
or
three
days
it
just
give
up,
you
know
I
I
was
just
like
walking
around
the
posters
and
checking
the
videos
and
that's
how
I
just
gave
up
on
talking
to
people.
So
I
think
there's
potential
there,
but
yeah.
A
They
were
they
kind
of
anticipated
that,
and
I
think
the
idea
was
that
if
there
are
too
many
people
or
but
they
also
had
a
zoom
link
set
up
for
every
poster,
so
you
could
switch
over
into
zoom.
But
I
didn't
see
anyone
using
that.
It's
just
too
awkward
yeah.
You
know
if
you're
in
the
flow
of
having
people
coming
in
and
out,
it's
just
really
hard
to
say:
okay,
let's
go
into
a
zoom
link,
so
I
didn't
see
anyone
use
that.
B
A
E
It
wasn't
too
bad
for
me,
but
one
thing
I
found
that
was
really
strange
was
that
when
I
was
in
a
group
where
people
are
someone's
talking,
sometimes
if
I
joined
the
group
half
the
people,
I
can't
hear
them.
So
I
just
thought
that,
like
people
are
just
here,
silently
not
doing
anything,
but
it
turns
out
they're
listening
to
someone
else
who,
for
some
reason
I
can't
hear
which
sounds
like
a
bug
on
their
end.
B
B
B
Yeah
yeah,
I
think
it's
true
it's
true
in
general,
it's
just
that
in
previous
conference
I
was
looking
for
the
novelty
and
but
I
walk
into
this
new
rips
with
a
different
perspective.
I
wanted
to
be
more
like
a
summer
school.
I
wanted
to
learn
and
not
like
really
like
chase
the
novel,
because
I
thought
there
are
several
areas
that
I
wanted
to
work
on
and
there's
a
lot
of
material
like
a
lot
of
opportunity
and
this
year
I
think
the
the
highlights
for
me
were
the
tutorials
and
the
world
cups.
B
They
were
both
amazing
for
that,
especially
the
tutorials.
So
I
I
kind
of
focused
on
that.
So
I
tried
to
go
to
somewhere
else.
I
thought
they
were
really
bad,
so
I
just
kept
going
back
to
the
tutorials
one
after
the
other
and
then
when
I
saw
it
was
already
thursday
and
then
I
just
moved
straight
to
the
workshop.
So
I
didn't
really
do
a
lot
of
paper
digging,
but
the
papers
are
going
to
be
there,
so
I'm
slowly,
I
I
went
to
the
poster
session
wrote
down
like
a
papers.
B
Yeah,
so
I
started
with
the
tutorials.
Well
there
there
are,
the
industry
talks
on
sunday,
they're,
actually
pretty
good.
I
like
it
and
what
I
can
highlight
about
the
industry
talks
and
the
tutorials
is
that
there
are
a
lot
of
time
for
questions
so
especially
the
interest
stocks
were
like
15
minutes,
but
there
was
one
hour
for
questions
and
the
talk
by
itself
was,
you
know,
like
uneventful,
but
the
discussions
were
really
good
and
that's
where
the
meat
of
the
thing
is.
C
B
C
B
Talk
and
very
interesting
discussions
you
can
get
like
a
peek
of
what's
going
on
inside,
you
know
and
they
were
like
answering
all
the
questions,
so
that
was
nice
and
one
thing
I
noticed
on
that
and
I
noticed
on
tutorials
and
I
noticed
on
the
invited
talks
that
there
was
a
lot
of
time
for
questions.
There's
like
one
hour
for
questions
and
people
would
not
be
shy
of
asking
those
questions.
B
There
were
like
a
lot
of
stuff
going
on
in
the
forums,
so
I
thought
the
discussion
that
generated
was
even
better
than
in
person
because
in
person
just
like,
it's
always
the
same
people
that
ask
always
the
same
questions
but
a
lot
of
people.
You
know
go
up
and
actually
ask
something,
and
even
me
I
don't
even
ask
anything,
usually
ask
anything
on
those
conference,
but
I
just
felt
like
I
wanted
to
join
the
conversation.
I
felt
him
kind
of
invited,
so
I
was.
A
D
I
intended
nips
like
maybe
about
six
years
ago,
and
it
was
smaller,
was
growing.
It
was
quadrupling
year
over
year
at
the
time,
but
there
was
just
like
four
main
industry
sponsors,
four
or
five.
You
know
netflix
facebook,
amazon,
etc,
and
and
when
the
industry
side
of
it
came
up,
it
was
interesting.
D
The
problems
that
they
encountered
in
deploying
at
scale
things
that
you
wouldn't
really
think
about,
but
where
the
stuff
would
interact
with
humans
and
the
facebook
guy
guy
was
talking
about
how
they
would
they
would
basically
dynamically
change
some
of
their
algorithms
and
then
inject
them
into
the
stream
and
see
what
the
results
were
so
they're
dynamically.
You
know
trying
things
on
out
and
they
would
get
these
weird
results
every
once
in
a
while
they're
going
along
and
all
of
a
sudden.
D
You
know
the
number
of
clicks
that
they
got
just
dropped
dramatically
and
what's
going
on
here,
it
turns
out
that
someone
injected
something
that
they
didn't
expect
the
the
ui.
They
moved
this
one
click
button
like
four
pixels
higher
next
to
something
else
that
was
more
obscure
and
then
it
just
human
perception
being
what
it
was.
They
they
lost
a
bunch
of
clicks.
You
know,
and
it
was
like
you
know
they
pounded
on
the
guy
who
had
done
it,
but
it
was.
It
was
just
this.
A
A
B
B
B
What
is
the
name
raven
test
like
data
sets
like
arc
from
francois
so
which
are
kind
of
data,
sets
that
deep
learning
doesn't
perform
well,
like
you
can
try
anything
and
a
lot
of
people
tried
anything
that
is
even
like
dpt3
and
they
don't
perform
well
in
those
data
sets
but
program
based
search
methods
perform
well.
So
that's,
I
think,
one
of
the
frontiers
that
people
are
exploring
now
and
there's
a
whole
workshop
on
this,
and
there
was
a
whole
tutorial
on
this
and
I
think
those
are.
I.
B
B
C
D
B
So
those
are
really
cool,
at
least
for
me
more
for,
like
a
learning
perspective,
I
also
spent
a
lot
of
time
on.
There
was
a
tutorial
in
approximate
inference
and
I
was
going
over.
You
know,
like
the
basics,
operation,
inference
mcmc
and
asian
neural
networks,
which
is
just
something
I
wanted
to
learn
more
about.
I
spent
quite
some
time
on
that.
There's
also
workshop
on
that,
and
there
are
some
things
on
neuroscience
which
karen
described
there
is
a
tutorial.
B
Very
introductory,
like
just
showing
the
relationship
between
neuroscience
and
machine
learning,
but
from
several
different
perspectives.
It's
kind
of
cool,
like
from
the
perspectives
of
defining
the
architecture
and
from
the
cognitive
science
you
know
like
defining
what
functions
should
be
learned.
So
it
was
a
very
high
level
review,
but
it
was
super
interesting
and
and
workshops
results.
They
were
also
true,
the
one
from
raymond
shure
from
mila.
B
I
think
that's
a
biological
one
and
the
one
from
cindy
lowe-
and
let
me
see
oh
yeah
and
I
think
for
me
the
best
workshops
one
I'm
still
going
through.
So
one
was
for
meta
learning,
specifically
because
we
were
working
on
that.
I
think
it
was
a
great
workshop.
It
really
like
broadened
my
view
of
better
learning.
I
knew
very
little
about,
I
still
know
very
little
about,
but
you
know
more
than
I
knew
like
last
week
and
there
is
another
one
on
self-supervised
learning,
that's
the
one
I'm
still
going
to.
A
F
A
B
Can
go
back
and
get
everything
because
everything
was
recorded,
like
even
the
questions
you
know
they
answer
some
ques,
you
you're
not
you're,
not
going
to
be
able
to
see
the
questions
in
the
chat,
but
they
answered
a
lot
of
the
questions
after
the
talks
after
each
talk
they
go
and
answer
the
question,
so
you
can
see
that
in
the
recording
and
they
have
a
panel
as
well.
All
the
workshops
have
panels
and
I
think
for
me,
that's
always
like
the
meat
of
it.
You
know.
That's
where
every
question
people
ask
is
our
answer.
B
B
B
I
don't
know
if
you
guys
saw
that
and
that's
from
the
same
group
that
organized
ml,
collective
plus
a
lot
of
other
people,
and
there
are
a
bunch
of
surveys-
and
you
know
like
discussions
of
where
we
are
in
machine
learning.
How
far
have
we
gotten
how
where
we
have
to
go?
I
thought
that
was
a
very
like
open-ended
night,
not
not
like
focusing
you
know
at
all.
This
model
beats
accuracy
on
that
benchmark.
That
was
not
in
focus
at
all.
B
That
was
really
cool
and
the
other
workshops.
What
I
realized
is
they
have
a
really
good
blend
of
old
and
new,
like
surveying.
What's
out
there,
some
of
the
new
papers
we've
seen
this
conference
and
what
is
the
path
ahead,
but
I
I
think
it's
the
interesting
question
like
where
we
have
to
go
and
you
could
see
the
perspective
from
different
communities.
You
know
like
meta
learning.
People
are
like
going
in
one
direction.
Self-Supervised
people
are
going
another
direction,
there's
some
like
work
a
between
the
two
fields.
B
A
B
Yeah,
I
think
you
should
check
that
one
out
so
yeah
some
some
papers
would
like
bridge
the
gap,
but
you
can
see
that
the
communities
are
very.
You
know
like
close
within
themselves,
and
everyone
has
different
ideas
of
where
machine
learning
should
go.
Like
the
symbolic
reasoning,
people
have
like
a
different
idea
and
for
me
it
was
super
interesting
just
you
know
being
there
like.
B
B
Well,
if
I
could,
I
would
go
to
the
physical
one,
of
course,
yeah
I
go
to
the
physical
one.
I
mean
the
workshops
were
great.
If
I
could
be
there
in
person,
they
would
be
even
better,
but
I
I
love
that
all
the
all
the
information
is
online
and
I'm
going
to
spend
the
next
two
months
going
through
all
the
videos
so.