►
From YouTube: 2021-01-19 - Md Abul Hayat & George Stein - Self-Supervised Representation Learning for Astro Images
Description
NERSC Data Seminars Series: https://github.com/NERSC/data-seminars
Title: Self-Supervised Representation Learning for Astronomical Images
Speakers: Md Abul Hayat (University of Arkansas, Berkeley Lab), George Stein (UCB, Berkeley Lab)
Abstract: Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning recovers representations of sky survey images that are semantically useful for a variety of scientific tasks. These representations can be directly used as features, or fine-tuned, to outperform supervised methods trained only on labeled data. We apply a contrastive learning framework on multi-band galaxy photometry from the Sloan Digital Sky Survey (SDSS), to learn image representations. We then use them for galaxy morphology classification, and fine-tune them for photometric redshift estimation, using labels from the Galaxy Zoo 2 dataset and SDSS spectroscopy. In both downstream tasks, using the same learned representations, we outperform the supervised state- of-the-art results, and we show that our approach can achieve the accuracy of supervised models while using 2-4 times fewer labels for training.
Bios:
Md Abul Hayat is a 4th year PhD student of Electrical engineering at the University of Arkansas. His research is on predicting hydration status using signal processing and statistical learning techniques from biomedical signals. In past years he has worked at Berkeley Lab and Nokia Bell Labs as a summer intern. Before joining the PhD program, he has worked as a telecommunications system engineer at Telenor Bangladesh.
George Stein is a postdoc at LBL/BCCP, with research centered on machine learning for cosmology. Areas of focus include cosmological simulations, generative models, anomaly detection, and of course self-supervised learning.
A
A
His
research
is
on
predicting
hydration
status,
using
signal
processing
and
statistical
learning
techniques
from
biomedical
signals.
In
the
past
years
he
has
worked
at
berkeley,
lab
and
nokia
build
labs
as
a
summer
intern.
Before
joining
the
phd
program,
he
has
worked
as
a
telecommunications
system
engineer
at
telenor
bangladesh,
and
we
also
have
george
stein
the
duo
speakers
today.
A
George
george's
research
is
centered
on
machine
learning
for
cosmology,
with
areas
of
interest
and
focus,
including
cosmological
simulations
generative
models
and
anomaly
detection
and,
of
course,
self-supervised
learning
without
further
ado
I'll.
Let
you
start
your
talk,
please
go
ahead.
Thank
you
again
for
joining
us.
B
For
the
introduction,
am
I
audible?
I
guess
I
am
yes:
okay,
cool,
so
hi
everyone.
So
today
I'm
going
to
talk
about
our
work
on
self-serverized,
representational
learning
for
astronomical
images.
So,
as
mustafa
said,
this
is
the
work
we
did
together
with
george
peter
zaria
and
verstappen
last
summer,
and
we
have
already
uploaded
the
the
manuscript
of
the
work
you
can
go
through
the
slides.
I
have
pasted
just
pasted
the
link
in
the
chat.
B
B
So
you
can
guess
from
the
number
of
the
images
that
it
is.
That
is
very
likely.
There
are
many
images
that
are
never
that
are
not
going
to
be
seen
by
any
human
eye,
and
but
we
need
to
analyze
those
large
number
of
images,
because
you
that
they
are
going
to
help
us
to
understand
the
evolution
and
nature
of
our
universe.
B
We
can.
We
can
try
to
find
out
rare
objects
and
anomalies
and,
like
the
kind
of
work
we
have
tried
to
do,
is
to
do
some
classification
tasks
like
to
find
out
the
galaxy
types
that,
if
the,
if
some
galaxy,
is
a
spiral
or
disk
or
something
like
that
or
probably
any
kind
of
regression
tasks
like
predicting
the
red
shift
from
the
images.
So
you
have
the
image
and
you
want
to
try
to
predict
the
red
ship
of
the
galaxy,
so
one
common
way
that
people
have
done
it.
B
This
kind
of
thing
for
a
long
time
is
we
all
know,
is
the
supervised
learning.
In
this
case
we
have
the
data
that
can
be
imaged
or
can
be
something
else,
and
if
we
follow
usually
machine
learning
techniques,
the
classical
mentioning
learning
techniques,
we
might
use
a
support,
vector
machine
and
other
kind
of
things.
But
in
this
case,
if
we
think
of
the
neural
networks,
then
we
actually
fix
the
the
architecture
of
the
network.
B
In
this
case,
probably
cnn,
our
feed
forward
neural
network,
then
we
split
the
data
set
on
training,
testing
and
validation
and,
most
importantly,
we
have
the
labels
for
of
the
data
and
what
is
the
common
source
of
these
labels?
Usually
people
are
used
to
annotate
these
labels,
so,
for
example,
but
but
having
these
labels
is
very
time
consuming
the
imagenet
dataset.
That
is
very
much
used
in
machine
learning
community
at
the
very
first
stage
it
collected
almost
like.
B
I
remember,
three
million
images,
annotated,
three
million
images
and
it
took
like
two
and
a
half
years
to
do
that,
and
obviously
it
did
take
a
lot
of
time,
money
and
obviously
in
in
a
specialized
kind
of
data
set.
It
also
involves
specialized
knowledge
on
annotating
the
labels,
so
that
is
actually
the
the
most
critical
part
of
supervised
learning,
and
once
we
have
this
training
and
validation
data
set.
We
train
the
model
in
using
back
propagation
in
in
neural
networks,
and
then
we
have
this
model.
B
Other
kind
of
paradigm
involves
the
unsupervised
learning
that
involves
pca,
distance-based,
learnings
and
so
on,
and
then
and
supervised
learning
does
not
take
into
account
of
any
levels,
but
it
is
somehow
gets
very
difficult
to
infuse
domain
knowledge
in
in
unsupervised
learning
paradigms.
That
makes
it
really
difficult
to
beat
performance
of
supervised
learning
paradigms.
C
B
I
have
a
large
number
of
level
data.
The
performance
is
not
improving,
then.
In
that
sense,
the
the
other
possible
improvement
scope
is
that
what
about?
If
I
have
less
number
of
leveled
images,
but
I
still
can
get
two
or
three
times
better
performance
than
than
having
that
what
we
have
been
doing
so
far
and
based
on
that
there
have
been
pretty
much
decent
work
going
on
for
a
long
time.
B
B
B
All
of
them
are
unlabeled.
So,
for
example,
if
we
have
this
image
in
the
blue
border,
we
take
the
image
we
we
do
some
augmentation
on
the
image
to
come
up
with
two
versions
of
the
same
image.
For
example,
in
this
case,
the
xq
is
actually
the
rotated
version
of
the
image
x
and
x.
K
is
also
rotated,
but
we,
if
you
can
see
that
is
it,
has
more
noises.
B
And
then
both
these
images
go
through
encoders.
This
encoder,
we
call
it
just
a
simple
encoder,
and
this
encoder
should
be
appropriately
called
the
momentum
and
encoder,
and
this
both
these
encoders
have
the
same
network
architecture,
but
the
parameters
are
going
to
be
different
and
going
through
these
encoders
this.
The
this,
the
two
augmented
version
of
the
same
image,
gives
us
two
different
representations:
zq
and
zk,
plus
we
call
them
positive
samples
and
also
from
previous
exam
previous
images.
B
We
stored
the
representations
in
the
momentum,
encoder
branch
and
you
and
we
call
this
the
stored.
The
stored
representations
are
stored
in
a
queue
and
in
the
moment
of
momentum
and
in
the
contrastive
learning
approach.
What
we
are
trying
to
do
is
that
we
want
to
maximize
the
similarity
between
zq
and
zq
zk
plus,
and
try
to
minimize
the
similarity
between
the
other
samples
in
the
queue
and
in
this
process
we
actually
try
to
learn
the
representations,
given
the
augmentations
that
are
actually
inherent
are
semantically
useful.
B
The
q
length
can
be
the
the
q
that
we
have
used
is
actually
five
percent
of
the
whole
data
set,
so
it
actually
is
a
is,
is
a
good
representation
of
all
the
variation
in
representations
we
have
throughout
the
the
input
image
set
and
the
encoder
part.
Actually
the
moment
of
encoder
part
actually
changes
slowly,
so
that
we
can
we
can.
We
can
trace
out
the
changes.
B
B
We
are
using.
In
this
case
we
are
using
the
cosine
similarity
to
to
measure
the
similarity
between
the
representations
and
regarding
the
the
augmentations
we
have
used,
so
some
augmentations
can
be
useful
in
some
efficient
tasks
and
in
our
case
we
found
out
that
these
are
possibly
the
the
good
set
of
augmentations.
B
One
is
rotation,
and
so
we
rotate
the
image.
So
the
input
of
the
the
input
image
you
collect
from
the
sdss
data
set
is
107
pixels
and
we
rotate
that
randomly
between
0
to
360
degree,
and
then
we
move
the
center
of
the
image
uniformly.
That
can
be
from
-7
to
plus
7
left
to
right
and
top
to
bottom
just
to
move
the
center
of
the
image,
and
then
we
introduce
gaussian
noise
based
on
the
channels
and
also
to
take
into
account
of
the
foreground
dust
noise
from
the
galaxies.
B
We
also
augment
the
images
with
galactic
extinction
based
on
the
established
model
of
the
stasis
data
set.
So
all
these
pre,
all
the
all
the
simulations
on
all
these
things
we
have
in
this
paper
or
work,
is
based
on
the
sds
status
and
also
you
take
into
account
of
the
variability
of
the
point
spread
function.
You
also
used
kind
of
blurring
that
goes
into
that
models.
The
point
spread
function
so,
finally,
the
the
image
we
have.
B
After
all,
these
segmentations
is
actually
64
times
64,
and
this
is
actually
seen
by
the
encoder
and
finally,
when
we
are
done
with
all
these
representations
and
when
we
can
say
that
yeah
we
are
getting
some
representation
that
is
actually
you
know
trying
is
very
much
good
at
distinguishing
our
the
cq
with
gq
minus
and
finds
out
very
much
similarities
with
zk
plus.
Then
we
stopped
the
pre-training
approach,
so
the
whole
approach
I
have
been
talking
till
now
is
actually
the
pre-training,
and
then
we
almost
discard
the
momentum
encoder.
We
just
remain
with
the
encoder.
B
B
B
That
is
that
that
is
in
the
following,
slides
and
also
we
can
distinguish
just
using
a
linear
classifier.
We
can
get
some
really
good
performance
on
galaxy,
morphology
or
glass,
galaxy
classification
and
also
redshirt
prediction.
So
for
the
redstone
prediction
that
has
been
the
main
focus
of
the
work
I
have
been
doing
in
the
summer,
we
have
beaten
the
state-of-the-art
model
with
the
fully
supervised
and
also
with
the
self-supervised
model
we
have
developed.
B
So
these
are
the
all
possible,
exciting
opportunities
of
work
that
we
can
do.
Obviously,
with
this
huge
data
set
with
fewer
labels,
but
still
getting
reasonable
and
good
performance-
and
this
is
the
data
set-
we
use-
sjss-
is
a
five
band
data
set,
and
also
we,
the
the
total
data
set
we
have
been
working
with,
is
actually
1.3
million
images
both
including
labeled
and
unlabeled,
and
1.3
million
500.
B
We
have
morphologic
classification
from
not
that
expert
people,
but
that
is
based
on
a
statistical
model
that
is,
that
is,
that
is,
that
is
actually
trying
to
source
the
labeling
process
and,
as
I
said
before,
the
the
encoder
we
have
been
using
is
resonance
50
with
a
slight
modification
and
what
total
images
that
has
been
trained
is
actually
1.3
million
and
we
are
using
moco2.
That
is
actually
the
algorithm
behind
the
social,
robust
learning
and
is
also.
B
There
is
another
algorithm
that
is
called
sinclair,
that
is
from
google
mapo
2
is
actually
from
facebook
and
they
actually
started
appearing
at
the
very
beginning
of
last
year.
They
they
started
to
beat
each
other
and
they,
you
know
improved
on
on
each
other,
and
our
motivation
of
work
is
actually
coming
from
these
two
and
and
and
we
started
working
with
moco2
because
sinclair
is
computationally
more
expensive
and
requires
tpus.
B
We
don't
have
that
and
yep.
So
we
add
labels
and
then
do
the
downstream
tasks.
So
the
following
I'm
going
to
stop
my
presentation
now,
but
I'll
be
in
the
background.
So
now
george
is
going
to
take
over
and
he's
going
to
explain
the
results
and
yeah
just
feel
free
to
ask
any
question
in
between.
So
I'm
handing
it
over
to
george
now.
C
B
C
All
right,
perfect:
well
thanks,
so
much
hyatt
for
giving
a
really
good
introduction
on
on
the
on
the
topic
and
how
the
self-supervised
contrast
framework
is
working
for
this.
For
this
application,
I'm
going
to
focus
on
the
downstream
tasks
that
haya
briefly
mentioned
when
he
was
showing
the
illustration
of
the
network
and
there's
a
number
of
different,
downstream
tasks
that
we're
targeting
here.
C
We
do
these
self-supervised
pre-training
with
no
labels
and
what
we
do
is
we
boil
down
the
1.3
million
images
into
these
20-48
dimensional
representations,
where
these
representations
should
be
invariant
to
the
augmentations
that
we
threw
in
the
rotations,
the
jitter,
the
gaussian
noise
and
all
this
so
before
highlighting
specific,
downstream
tasks
we
use.
The
first
thing
we
did
was
was
to
visualize
what
information
is
contained
within
these
representations
and
to
do
that.
C
C
C
C
C
Yet
we
see
that
there
is
a
clear
you
know:
a
clear
separation
between
galaxies
of
different
morphological
types,
different
colors
and
and
all
the
different
things
that
are
happening
in
this
1.3
million
images.
So
in
the
top
left
here
you
know,
we
see
a
bunch
of
brighter
large
large
galaxies
along
the
left-hand
side,
we're
seeing
more.
You
know
more
edge
on
galaxies
and
and
interestingly
in
this
visualization
was
all
these
things
over
and
here
which
at
least
to
to
me.
I
was
not
an
expert
on
this
sdss
data
set.
C
I
was
not
expecting
to
see
a
lot
of
these.
You
know
these
green
images
here
these
red
images
here
some
clusters
of
like
green
lines
through
the
images
which
are
all
various
just
imaging
artifacts
from
the
from
the
telescope
and
from
the
data
reduction
pipelines
that,
at
least
to
me
we're
not.
I
was
not
expecting
to
see
all
of
these
in
the
data
set,
but
you
know
the
representations
in
an
unsurprised
way.
C
Contrastive
learning
has
learned
to
separate
all
of
these
different
types
of
galaxies
from
one
another,
and
if
we
zoom
into
this,
you
know
these
are
every
little
thing
is
an
individual
galaxy.
This
is
probably
easier
to
see
and,
like
I'm
saying
right,
you
can
scroll
through
this
thing
for
hours.
You
know.
I
found
this
very
interesting
to
look
at
exactly
what
information
has
been
separated.
C
You
know
there's
merging
galaxies
here
and
galaxies
next
to
each
other,
and
it's
getting
dimmer
as
we
go
down
all
the
way
down
to
you
know
here
we're
seeing
possibly
some
some
stars,
or
maybe
some
other
things
that
were
not
expected
in
this
galaxy's
galaxy
image
data
set
and
then
we
can.
We
can
start
to
bring
in
the
labels,
so
here
on.
The
left
is
just
that
same
image.
C
So
we
have,
as
hyatt
mentioned,
we
have
crowd
source
labels
based
on
the
morphology
of
the
galaxy.
So
these
are
things
like
you
know,
citizen
scientists
were
asked.
Is
this?
A
dispute
edge
on
which
you
know
is
is
like
a
line
like
one
of
these
things
here
or
is
it
face
on
yes
or
no?
Is
it
a
smooth
galaxy
or
does
it
have
features
our
disk,
and
we
see
that
these
representations
right?
C
We
just
color
the
the
points
by
the
by
the
answers
to
these
classification
tests,
and
we
see
that
you
know
it
has.
It
is
the
self-sufficient
learning
has
separated
face
on
galaxies,
these
dark
blue
ones
from
edge
on
galaxies.
These
you
know
pink
lines
to
the
to
the
order
where
I
can
almost
just
draw
a
you
know.
A
classification
boundary
by
I
say
anything
to
the
right
of
this
is
face
on
anything
to
the
left
is
edge
on
and
you're
going
to
get
a
quite
a
high
accuracy.
C
Just
from
these
very
similar,
sim
or
simple,
you
know,
by
eye
classifications,
what
we
actually
use
is
linear
classifications,
and
then
you
know,
I
show
that
we
show
this
for
a
number
of
of
questions
right
like
features
and
disks
are
all
these,
these
brighter
larger
ones.
Here
we
also
show
the
regression
labels
that
we
have.
These
are
the
the
redshift
where
low
redshift
galaxies
are
are
here
and
high.
C
Redshift
galaxies
are
are
over
in
the
middle
of
this
space,
and
we
see
that
these
representations,
at
least
by
eye
in
this
umap
visualization
and
various
other
visualizations.
We
did.
They
seem
to
have
very
nicely
separated
galaxies
based
on
the
semantic
information
in
the
image,
which
was
the
whole
point
of
doing
the
self-supervised
learning,
so
that
was
some
visualizations
and
here's
some
more
concrete,
downstream
tests
that
we
did.
C
So
you
know
broadly
stated,
given
an
image,
can
we
find
other
similar
images
in
the
data
set,
and
this
is
similar
to
how
like
google
image
search
works
where
you
paste
in
an
image,
and
it
goes
and
finds
other
images
that
are
that
are
similar
to
this
image?
And
it's
not
doing
this
based
on
an
image
level
right,
it's
not
taking
an
image
and
doing
some
dot
product
with
every
other
image
and
then
sorting
by
the
largest.
C
So
these
five
here
and
these
five
here
and
the
following
five
images
are
the
most
similar
images
from
the
data
set
according
to
a
similarity
search
on
the
representation
space,
and
we
see
that
you
know
this.
We
we
get
returned
a
number
of
interesting
images
if
you're
looking
for
you
know
just
simple
edge
on
dim
galaxies.
Well,
here's
five
other
ones
that
semantically
look
almost
identical
at
least
to
to
me.
C
You
know
brighter
edge
on
galaxies,
maybe
you
know
merging
galaxies
or
at
least
galaxies
that
are
projected
near
each
other
ones
that
are
quite
close
to
each
other,
a
lot
farther
than
this
spiral
galaxies.
You
know
elliptical
galaxies
and,
interestingly,
like
like
how
it
was
saying
too.
This
is
you
can
use
this
sort
of
as
an
anomaly
search.
C
Maybe
it's
not
an
automated
anomaly
search
but
say
you
find
one
weird
image
that
you
weren't
expecting
or
one
weird
instance
in
your
data
set
such
as
this
one
here.
Well,
when
we
saw
this
we're
like
we
were,
you
know
a
bit
surprised
to
see
it.
So
what
we
did
was
was
did
a
similarity
search
and
out
popped.
Many
other
images.
You
know
not
thousands,
but
but
a
decent
number
of
similar
looking
images
which
we
think
are
you
know
these
are
imaging
artifacts.
C
Given
one
example,
you've
isolated,
maybe
not
all
the
other
examples,
but
a
significant
number
of
similar
examples,
and
then
maybe
what
you
want
to
do
is
is
throw
out
these
images
or
you
want
to
flag
them
for
further
investigation
or
a
huge
number
of
anything.
You'd
want
to
do
with
the
similarity
search,
many
many
applications.
C
So
that's
one
that
requires
absolutely
no
labels
right.
You
just
train
this
self-supervised.
You
do
a
similarity
search
and
you
get
similar
images.
The
second
downstream
task
that
I'm
that
we're
showcasing
here
is
a
classification
tasks.
A
number
of
classification
tasks
where
the
goal
here
is
is
directly
from
that
20
48
dimensional
representation
is
to
predict
galaxy
z2
morphological
labels
directly
from
the
representations.
C
Yes,
no
is
there
a
bar
how
many
spiral
arms
all
these
detailed
questions
on
the
the
on
the
morphology
of
a
galaxy,
and
this
is
a
crowdsource
platform
where
hundreds
of
thousands
of
people
you
know
volunteered
their
time
to
help
label
galaxies
and,
in
the
end,
at
least,
for
galaxy
2.
There's
there's
about
a
couple
hundred
thousand
galaxies
that
have
all
of
these
questions
labeled
for
them.
C
So
what
we're
going
to
do?
Instead
of
you,
know
what
you
do
in
a
supervised
training
or
a
supervised
framework?
Is
you
design?
Some
cnn?
You
take
your
images.
You
try
to
predict
for
each
question.
You'd
split
it
into
you
know,
design
a
network
to
predict
each
question,
but
here
we're
just
going
to
do
a
number
of
these
classification
tests
directly
from
the
representations.
C
So
so
what
this
actually
looks
like
in
practice
is
just
a
linear
layer
from
the
2048
dimensions
to
one
where
that
one
dimension
is
just
the
probability
of
it
being
negative
or
positive.
In
a
binary
classification
test-
and
it's
you
know,
this
is
super
quick
right.
It's
just
a
2048
dimension
for
each
thing.
C
You
know
the
morphological
types
of
galaxies,
but
here
we're
showing
three
different,
separate
classification
tasks.
Three
questions
from
the
galaxy
decision
tree
the
first
one
here
is,
is:
does
it
have
features
our
disk
here
is?
Could
this
be
a
disk
viewed
edge
on,
and
here
is?
Is
there
a
sign
of
a
bar
feature
through
the
center,
and
these
top
three
panels
show
with
a
very
limited
number
of
labeled
samples
right.
C
So
this
is
250
labeled
samples
here
this
is
the
prediction,
and
this
is
the
true
label
and
we're
seeing
that
you
know
we're
starting
to
converge
upon
quite
an
accurate
answer,
at
least
for
what
we
call
high
quality
samples.
So
these
these
labels,
you
know
it's
crowd
source.
Sometimes
the
images
are
very
blurry
or
hard
to
determine.
C
So
it's
not
you
know,
really
a
binary
task.
There's
there's
a
lot
of
mislabeling.
There's
there's
a
lot
of
various
errors
going
on
in
these,
but
even
with
you
know,
a
very
limited
number
of
samples,
250
we're
able
to
get
quite
an
accurate
prediction
and
using
the
full
training
set.
You
know
we
get.
We
get
a
highly
highly
accurate
classification
task,
our
classification
prediction,
at
least
for
these.
C
So
this
is,
you
know,
a
classification,
a
set
of
classification,
downstream
tasks
and
a
third
thing
we've
we
looked
at,
which
was
the
focus
of
the
summer
work.
We
looked
at
this
in
a
lot
of
detail,
was
the
photometric
redshift
prediction,
so
this
is
given
an
image.
What
is
the
redshift
of
this
galaxy,
which
you
know
is?
It
is
a
what
is
the
distance
of
this
galaxy
and,
like
hyatt
said,
we
have
500
000
labels,
which,
which
seems
like
a
lot.
You
know.
C
You
know
we
never
use
the
labels
with
the
images
at
the
same
time,
but
here
as
we're
training,
this
linear
layer
we're
also
updating
the
encoder
with
a
very
small
learning
rate.
So
the
encoder
itself
is
is,
is
you
know
it
has
the
loss
back
propagated
through
it
and
the
weights
are
being
updated
and
we
we
show
here
three
different.
You
know
quality
metrics
of
your
redshift
prediction.
C
C
So
here
first
this
this
pascal
dial,
this
is
the
you
know
previous
state
of
the
art
on
on
this
specific
sdss
data
set
and
these
500
000
labels.
This
is
a
fully
supervised,
convolutional
neural
network
setup,
and
we
show
here
on
the
x-axis,
the
the
quality
metrics
as
a
function
of
the
amount
of
labeled
data
used.
C
So
this
is,
you
know,
previous
state
of
the
art,
and
this
blue
here
is
our
resnet
50
in
a
fully
supervised
setting,
and
we
see
here
that
using
the
full
training
set
full
set
of
labels.
You
know
we're
getting
very
similar
answers
on
the
on
the
quality
metrics.
As
the
previous
work,
which
is
a
you
know,
very
good
sanity
check,
and
then
here
in
the
red.
This
is
the
the
key
one
to
look
at.
This
is
the
results
of
first.
C
You
know
so
these
are
the
number
of
outliers
has
decreased
by
like
30
percent.
The
dispersion
has
has
decreased
and
we're
just
all
around
getting
more
accurate
redshift
predictions
from
this
labeled
data
set,
but
perhaps
more
interestingly,
is
to
look
at
the
plot
in
a
different
way
and
just
take
a
constant.
C
C
C
All
of
these
downstream
paths
were
just
with
the
same
trained
network,
so
you're
not
having
to
train
the
full
resnet
50
from
scratch
for
every
classification
test,
every
regression
task.
You
know
every
similarity
search
task.
It's
just
one
network
that
you
then
apply
to
everything.
You'd
want
to
do
for
the
sky
survey.
C
C
Just
you
know
show
how
a
sketch
of
how
this
works
in
other
fields,
which
is
pretty
much
exactly
as
we
talked
about
here,
for
whatever
experiment
or
any
data
set.
You
have
you
just
take
all
your
data
with
or
without
labels.
This
doesn't
have
to
be
images,
it
can
be,
1d
can
be
spectra
or
whatever
you
like
can
be
3d
and
you
just
construct
data
augmentations
that
reflect
changes
in
the
data.
C
You
want
your
network
to
be
agnostic
to
so
various
types
of
noise,
various
jitter,
maybe
there's
some
smoothing
of
your
data
or
scaling
making
things
bigger
or
smaller.
This
is
you
know,
domain
specific
and
whatever
downstream
task,
you're
shooting
for
and
whatever
data
set
you
pick
and
choose
what
you
think
would
be
useful.
C
Please
check
it
out
online
if
you're
interested
in
learning
more
and
please
send
any
questions
to
any
of
the
authors,
we're
all
very
interested
in
hearing
what
people
think
we
can
do
with
this
and
and
various
different
things
that
people
are
interested
in
with
this.
So
I'll
finish
there
and
please
ask
any
of
us
any
questions
you
you
want.
D
C
So
because
of
the
augmentation
one
of
the
augmentations
we
chose
was
to
randomly
orient
by
or
randomly
rotate
by
zero
to
360
degrees.
The
orientation
has
been
washed
out,
but
that's
not
maybe
this
one's
easier
to
see,
but
that's
only
a
result
of
the
augmentation
that
we
chose.
If
you
were
targeting
downstream
tasks
where
you
cared
about
the
orientation,
you
would
not
put
that
augmentation
in
and
then
the
representation
should
learn
the
orientation.
So
you
can
separate
things
based
on
the
orientation,
but
here
it's
it's
not
so
much.
C
E
Hi,
I
have
a
sort
of
related
question
to
that
actually,
which
was
about
you
said
you
could
make
this
embedding
available
for
other
people
doing
other
tasks,
for
example,
but
then
linked
with
that,
like
what
you
just
said,
what
you
know,
what
term
augmentations
you
use
is
is
kind
of
important
to
its
usefulness
for
the
tasks
so
is
there?
Is
there
a
way
of
dealing
with
that
with
like
sort
of
pre-training
and
re
building
from
the
same
embedding
or
is
it
like?
You
have
to
just
make
different
ones.
C
C
So,
if
you
were,
you
know,
we
tried
to
keep
it
simple
so
that
any
any
downstream
test
can
be
done
on
this,
whereas
in
if
you
were
targeting
something
specific,
maybe
if
you
didn't
care
about
the
color,
you
would
add
inside
augmentation
that
changed
the
color
and
you
know,
therefore,
you
wouldn't
be
learning
the
color
information,
but
I
mean
maybe
hyatt.
Do
you
have
any
on?
If
say
you
wanted
to
add
another
augmentation,
do
you
need
to
retrain
from
scratch,
or
do
you
need
to?
B
E
F
Hi,
just
a
quick
question
there
about
the
extensibility
of
this,
so
this
is
set
up
with
s
has
been
run
on
sdss,
but
obviously
there's
new,
bigger
and
bolder
and
scarier
catalogs
coming
down
the
line
like
elasticity
in
gaia.
How
viable
will
this
with
this
technique
be,
or
how
quick
would
it
be
to
transfer
to
these
new
formats?
Do
you
know.
B
Regarding
the
one
we
trained
with
1.3
million
it
took
so
if
we
train
for
50
epochs
the
whole
set,
it
took
like
eight
hours,
but
initially
we
were
training
even
for
more
longer
duration,
but
it
seemed
that
probably
50
epochs
or
just
eight
hour.
Training
is
enough,
and
also
one
important
thing
aspect
of
this.
This
probably
we
don't
need
to
train
on
the
whole
unlabeled
data
set,
because
it's
so
big
if
it's
representing
the
distribution
of
images
in
a
fairly
fair
way,
then
probably
you
know
we
don't
have
to
train
the
300
million
images.
B
Probably
just
3
million
would
be
enough,
provided
that
we
are
sampling
from
uniformly
and
getting
the
same
kind
of
distribution,
but
there
is
always
a
trade-off
of
missing
that
out,
but
yeah,
and
also
regarding
the
question
of
scaling
this
relating
that
to
other
surveys.
Probably
there
are
some
other
factors
may
in
that
case
we
have
to
take
into
account
like
resolution
and
the
the
the
scope
of
per
pixel
and
how
much
it
is
representing
in
the
real
domain.
B
B
B
Yeah
it
has,
I
guess
it
can
work
equally
good
in
that
discipline
also,
but
also,
obviously,
that
we
need
some
kind
of
domain
knowledge
of
the
image
set
like
it
is
very
important
to
come
up
with
a
useful
set
of
augmentation
so
that
we
retain
the
property
of
the
images
and
we
do
not
do
any
augmentation
that
actually
distorts
the
inherent
inherent
properties
that
we
are
trying
to
figure
out.
So
the
the
framework
would
remain
the
same,
but
probably
have
to
come
up
with
some
innovative
way
of
domain.
Specific
non-knowledge,
specific
augmentations.
D
Well,
in
all
different
sorts,
but
you
have
the
black.
You
know
biological
things
that
can't
be
crystallized
that
are
kind
of
big
and
floppy,
but
can
be
captured
in
under
micro.
You
know
frozen
in
water,
often
in
a
microscope
slide.
You
have
many
many
repetitions
of
the
same
macromolecular
component
in
different
orientations.