►
Description
Ralph Kube from PPPL talks about machine learning in fusion research
A
So
to
illustrate
the
the
breadth
of
the
field,
I
took
all
the
abstracts
from
that
meeting
and
generated
a
word
cloud
and
here
stripping
away
all
filler
words
like
and
with
etc.
The
how
large
a
word
appears
in
this
plot
gives
you
some
sense
of
the
relative
of
the
frequency.
These
words
appear
so
first,
let
me
say
that
fusion
energy
sciences
can
be
roughly
separated
into
magnetic
confinement
and
inertial
confinement.
Fusion
and
this
mini
conference
covers
both,
so
this
workout
gives
you
oh.
A
Certainly
neural
network
appears
very
large
that
reflects
the
fact
that
most
machine
learning
practitioners
in
fusion
energy
sciences
do
deep
learning
in
all
kinds
of
varieties
that
can
be
multi-layer.
Perceptrons.
You
see
the
phrase
convolutional
neural
network
popping
up.
You
see
deep
neural
network.
There
are
some
people
who
do
time
series
modeling
with
a
recurrent
neural
network,
neural
networks
and
then
more
of
coming
from
the
from
the
physics
side.
You
see
the
big
phrase
simulation
and
that
reflects
the
the
research
thrust
in
the
field
to
replace
expensive
numerical
simulations
with
machine
learning
surrogate
models.
A
That's
certainly
an
aspect,
many
people
look
into,
and
then
you
see
many
smaller.
You
see
many
smaller
terms.
For
example,
anomalous
transport
coefficients
beam
emission
spectroscopy
disruption
prediction,
inner
shell
target
and
those
are
all
terms
that
reflect
the
different
physics
modalities
so
to
say
that
come
that
that
are
important
from
the
from
the
exp
from
the
experiments,
and
you
also
see
some
smaller
words,
for
example,
gaussian
process
regression.
A
A
So
here
on
the
top
left,
you
see
a
token
mac,
a
machine
for
use
for
magnetic
confinement,
fusion-
and
here
this
is
an
illustration
of
the
upcoming
spark
token
mac,
one
of
three
token
mags
that
is
currently
under
construction
in
the
us
and
then
in
the
top
right.
I
just
took
some
random
visualizations
of
data
that
are
measured
in
plasma
discharges
and,
as
you
can
see,
all
these
five
plots
look
nothing
alike,
and
that
is
because
the
data
we,
the
measurement
data
we
sample
infusion
plasma,
varies
by
a
lot
it
can.
A
A
There
are
diagnostics
and
sort
that
are
sensitive
to
electromagnetic
radiation
in
all
kinds
of
the
spectrum,
for
example,
visible
light
infrared
which
targets
the
material
walls
cyclotron
radiation,
which
is
kind
of
micrometer
range
and
they're.
Also,
for
example,
diagnostics
that
are
based
on
particle
flux,
so
yeah,
a
broad
array
of
diagnostics,
is
used
to
sample
the
various
aspects
of
the
plasma
and
we
use
those
to
try
and
reconstruct
the
plasma
state.
Now
these
diagnostics
also
have
different
output.
Some
diagnostics
just
give
a
zero
zero
dimensional
quantity,
for
example,
a
pressure
sensor.
A
A
So
let
me
just
quickly
illustrate
three
kind
of
machine
learning,
research
topics
that
are
popular
at
pppl
and
since
we
are
associated
with
princeton,
also
at
princeton
university
on
the
left,
you
see
an
illustration
that's
supposed
to
be
plasma
control
and
here-
and
that
is
exactly
what
the
name
says.
If
we
have
a
discharge,
we
have
a
real-time
feedback
system
that
tries
to
steer
the
plasma
away
from
unstable
configurations,
kind
of
kind
of
just
keep
it
confined,
and
there
are
some
there
are
some.
A
B
A
Daddy
has
to
talk
to
us
guys,
I'm
sorry,
I'm
having
a
daycare
situation
today.
Okay,
right,
so
we
we,
we
have
the
xgc
code,
which
is
very
compute
intensive
and
in
preparation
for
running
more
physically
accurate
models.
We
would
like
to
replace
some
compute
intensive
parts
of
the
codes
with
machine
learn
with
machine
learning
surrogate
models.
A
A
A
Some
things
we
are
very
happy
with
is
that
tensorflow
and
pi
charge
modules
are
readily
available,
also
in
modern
versions
and
a
typical
job
for
machine
learning.
Practitioners
are
smallish
problems;
they
don't
require
much
compute
time.
So
for
this
we
rely
heavily
on
jupiter
notebook
and
we
really
appreciate
the
gpu
support
we
have
in
there.
A
A
Some
bottlenecks
I
personally
have
experienced
were
that
when
we
train
large
training
jobs
which
require
multiple
terabytes
of
data,
data,
loading
becomes
a
bottleneck
and
we
have
inspected
this
with
debuggers
such
as
tau
and
then
so.
We,
we
are
actually
very
happy
with
the
software
that
is
provided.
So
I'm
actually
going
to
start
wrapping
up
my
talk
now
and
talk
about
some
new
contenders
and
there
are
the
ideas
that
that
is
basically
using
automatic
differentiation
in
combination
with
scientific
simulation.
A
A
So
there
are
two
tools
that
I
gather
are
popular
among:
machine
learning,
practitioners
and
those
are
julia
and
jax.
So
julia
is
a
language
that
is
developed
from
the
scratch.
It's
very
young
automatic
differentiation
as
a
first
class
citizen,
and
the
entire
language
is
just
in
time
compiled,
so
it
runs
very
fast
and
jax
is
used
in
conjunction
with
python.
A
We
are
talking
about
crossing
petabytes
of
data
per
day
and
it
is,
and
we
are
looking
into
how
machine
learning
can
be
used
with
the
with
these
kinds
of
data
sets.
So
some
technologies
we
are
discuss.
I
have
discussed
internally
with
colleagues,
are
first
custom
machine
learning
hardware,
and
this
goes
under
the
name,
wafer
scale
engines.
Sierra
brass
recently
released
one
of
these.
It's
basically
chips
designed
for
machine
learning,
tasks
specialized
hardware
and
combine
them
into.
B
A
A
What
what
they're
doing
from
which
google
released,
I
think
in
2017,
but
those
I
believe
will
be
commercially
available,
also
in
conjunction
with
the
big
data
age
of
fusion,
if
you
so
will,
is
a
move
towards
transformer
neural
networks
and
those
those
are
by
design
more
general,
but
require
more
data
to
to
to
work
on
the
tasks.
So,
there's
some
research
going
on
how
to
use
these
networks
for
fusion
energy
right
now,
yeah.
A
Actually,
no-
and
I
I
have
I
to
the
colleagues
I
talked
in
preparation
for
this
talk,
I
really
haven't
heard
anything
the
anything
that
they
feel
is
missing
or
would
really
increase
their
productivity
when
they
do
machine
learning
at
nurse.
So
I
get
the
impression
that
the
facilities
are
quite
yeah,
where
they
need
to
be
right
now.
Well,
we'll
see
later
when,
when
we
really
do
machine
learning
with
the
with
these
big
data
that
are
supposed
to
be
generated
from
ether.