►
From YouTube: 2020-05-01 - Stephan Hoyer - Deep learning for PDEs, and scientific computing with JAX
Description
NERSC Data Seminars Series: https://github.com/NERSC/data-seminars
Deep learning for PDEs, and scientific computing with JAX
Abstract: This talk will give an overview of how deep learning can be combined with traditional numerical methods to create improved methods for scientific computing. I will highlight two recent examples from my research: using deep learning to improve discretizations for solving partial differential equations [1], and using deep learning to reparameterize optimization landscapes for PDE constrained structural optimization [2]. I will also briefly introduce JAX [3], an open source library from Google for composable transformations of Python/NumPy programs, including automatic differentiation, vectorization and JIT compilation for accelerators. JAX is particularly suitable for scientific applications, including hybrid machine learning / simulation codes.
[1] Bar-Sinai*, Y., Hoyer*, S., Hickey, J. & Brenner, M. P. Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences 201814058 (2019). doi:10.1073/pnas.1814058116 [2] Hoyer, S., Sohl-Dickstein, J. & Greydanus, S. Neural reparameterization improves structural optimization. arXiv [cs.LG] (2019). https://arxiv.org/abs/1909.04240 [3] https://github.com/google/jax
A
Maybe
I'll
get
started.
Thank
you
all
for
joining
I
see
almost
40
people
on
the
call.
That's
great,
and
hopefully
a
few
more
will
come
in
it's
my
great
pleasure
to
introduce
Stefan
he's
a
scientist
at
Google
research
in
vehicle
accelerated
science,
team,
I,
guess
he's
one
of
our
own
in
in
that
he's
a
PhD
in
physics
from
Cal,
and
he
is
working
on
some
very
exciting
topics
that
are
of
great
relevance
to
a
lot
of
the
recent
theory.
Scientific
computing.
A
Sorry,
scientific
machine
learning
calls
that
are
coming
up,
especially
for
applications
in
the
physical
sciences,
systems
governed
by
PDS,
etc,
but
also
broadly
he's
working
on
other
scientific
computing
projects
with
Jax,
and
thank
you
Stefan
for
taking
the
time
to
join
us
remotely
and
I'll.
Leave
it
to
you
from
now
great.
B
Looking
we've
been
seeing
that
industry
we've
seen
that
deep
learning
is
that
really
I
think
a
really
transformative
impact
on
how
we
do
a
lot
of
we
make
a
lot
of
systems
at
Google
and
really
over
just
over
the
past,
be
the
past
roughly
seven
years
or
so,
and
so
we
looking
into
how
we
can
use
the
technology.
Deep
learning
and
sort
of
the
related
technologies
that
sort
of
come
to
sort
of
come
to
rise
along
with
deep
learning,
is
see
how
that
can
also
potentially
transform
scientific
computing
okay.
B
C
B
That
makes
it
really
easy
to
access
a
lot
of
compute
and
to
and
and
the
implements
automatic
appreciation.
So
you
don't
have
to
write
trips
by
hand
we've
seen,
especially
in
recent
years,
with
projects
like
like
tensor
flow,
I,
torch
and
Jack's
another
project
in
the
same
tradition.
That
makes
it
the
sort
of
that
sort
of
developed
specifically
for
deep
learning,
but
I
think
also
makes
it
gives
you
a
lot
of
access
to
these
capabilities
in
terms
of
compute
and
automatic
Association.
B
That's
actually
rolling
for
a
lot
of
science
as
well,
so
that's
kind
of
that
and
so
yeah.
So
when
we
tell
you
about
Jack,
why
Jack's
interesting
so
Jax
is
a
system,
it's
a
Python
library
for
for
writing,
composable
transformations
of
Python
and
numpy
code.
So
the
idea
with
Jax
is
that
you
can
you
take
take
a
program
to
do
almost
exactly
what
you
would
write,
you'd
write
with
with
an
numpy
or
sort
of
MATLAB
style
code,
and
they
can
be
efficiently
sort
of
compiled
and
transformed,
so
we
can
compile
them
to
run.
B
Tron
accelerators
like
like
GPUs
and
NTP,
is
with
with
with
with
a
very
little
overhead,
and
we
can
also
do
the
other
function.
Transformations
and
I'll
go
into
the
sort
of
what
these
are
see.
This,
the
slight
pointer
is,
is
viewable,
hopefully
yeah,
okay,
so
the
rock
so
Jax
also
has
these
these
function,
contributions
and
all
so
some
examples.
B
These
four
think
for
things
like
addition
to
the
JIT
compilation,
but
for
things
like
taking
gradients
or
or
sort
of
batching
or
paralyzation,
and
so
this
this
the
system
lets
you
write,
two
really
concise
and
and
powerful
in
powerful
programs
in
this
sort
of
numpy
programming
paradigm.
So
this
is
work
largely
done
by
the
Jack's
team.
I'm,
not
I'm,
mostly
sort
of
an
eager,
I
I'm,
sorry
I'm,
mostly
a
sort
of
a
user
or
sort
of
occasional
contributor.
B
This
is
an
example
of
training,
of
sort
of
how
you
might
imagine
sort
of
setting
up
the
training,
a
very
simple
neural
network,
and
this
is
and
I'm
most
of
you
may
have
seen-
maybe
a
little
bit
of
deep
learning.
I
think
that
mostly
most
of
this
program
with
things
like
this
predict
function
is
just
something
you
do
something
you
do
right
in
numpy
I
was
a
normal
Python
structions
like
like
loops.
You
have
a
few,
and
we
have
a
few
of
these
extra
sort
of
funny
functions
that
have
highlighted
here.
B
These
imports
of
grab
and
do
you
map
so
she's
also
show
you
talking
to
reach
those.
So
first
what
happens
when
we
so
right?
We,
you
can
write
these
programs
just
like
numpy
programs
and
when
you
do
that
sort
of
by
default.
Everything
is,
is
everything
just
evaluated
for
operation
by
operation
and
sort
of
as
you
as
each
function?
It's
called
it's.
B
It's
a
little
different
from
how
it's
done
so
in
tensorflow
and
different
tetrafluoride
torch
from
jacks
operations
like
like
gradients
are
function
transformations,
so
everything
is
so
Jack's
programs
are.
Are
everything
is
functionally
pure
there's?
No,
there's,
no,
there's,
no,
there's,
no
beautiful
state,
and
and
in
order
to
make
a
already
cut
in
order
to
to
cut
the
ingredient.
For
example,
you
you
apply
a
gradient
function
to
another
function
which
transforms
that
function
into
something
that
let's
do.
The
calculus
creates
a
set,
and
so
Jack
says,
as
the
gradients.
B
B
There's
also
this
this
transformation,
for
which
stages
out
sort
of
a
function
compiles
a
function
as
as
a
program
for
Excel
a
compilation,
that's
something
that
you
can
that
again
can
be
sort
of
composed
with
with
these
other
transformations.
So,
for
example,
you
can
say
I
want
to
compile
greediest
I.
B
Think
one
really
important
I
think
aspect
that
the
the
authors
of
Jack's
did
really
nice
they've
been
a
big
focus
on
sort
of
compositionality,
so
you
can
you
can
compile
so
you
can
sort
of
compose
grad
and
get,
for
example,
sort
of
any
arbitrary
what
you
want
and
build
a
really
rather
fairly
large
sort
of
sophisticated
programs
that
combine
these
transformations
and
then
the
lab
through
the
last
sort
of
family
transformations.
That's
actually
a
really
powerful
Jackson
there,
and
there
are
a
few
others
as
well
that
are
I'm
going
to
say
that
mental.
B
B
As
if
you
wrote
them
in
terms
of
arrays
with
higher
dimensions,
and
so
for
something
for
so
this
is
so
as
I'm
sure
most
of
you
have
used
that
business
system
like
like
numpy
or
mail
before
to
get
or
to
get
efficient
code,
you
need
to
typically
need
to
write
its.
You
need
to
write
operations
in
terms
of
sort
of
in
terms
of
operations
that
happen
to
happen
on
large
arrays,
and
often
that
means
that
your
code
ends
up
being
a
lot
more.
B
You
end
up
sort
of
juggling
around
a
lot
of
extra,
so
instead
of
bacteria,
do
you
have
matrices
I'm?
Instead
of
instead
of
matrices,
you
have
like
3d
tensors,
like
you
end
up
with
a
lot
more
with
a
lot
of
sort
of
much
bigger
arrays.
It's
just
like
kind
of
logically
keep
track
of.
What's
going
on,
so
the
map
is
really
nice
because
you
can
take
code
that
it
really
takes
some
code
that
you
might
be
directs
a
on
a
sort
of
little
or
the
natural
input
like
here.
B
This
predict
function,
acts
on
a
single
vector.
Now
it
turns
out
to
do
in
deep
learning
to
efficiently
run
a
multi-layer,
neural
net.
You
we
almost
always
write
things
in
terms
of
batches
of
examples,
so
a
whole
bunch
of
so
we
in
so
instead
of
doing
it.
For
us
take
one
example
at
once:
we
have
to
do
a
whole
batch
and
that
music,
the
code,
if
we
wrote
this
and
I
wrote
this
in
terms
of
in
terms
of
batch
of
examples
at
once.
B
I
heard
something
three
similar,
but
it
have
to
think
much
more
carefully
about
sort
of
how
the
dimensions
line
up
and
in
contrast,
if
I
use
something
like
the
XenApp
transform,
I
can
I
get
those
batch
dimension.
I
can
sort
of
I
can
sort
of
inject
it
in
those
those
Batchelor
mentions
of
the
transformation,
and
it's
really
just
as
efficient
as
if
I
wrote
as
if
I
wrote,
the
the
the
vector
is
code
by
hand,
but
I
don't
need
to
do
that.
B
B
B
We
have
this
also
working
on
on
sort
of
multi
node
systems
as
well,
so
we
actually
just
pretty
routinely
inside
Google
now
for
sort
of
multi
node
systems
with
with
potentially
with
hundreds
or
thousands
of
CPU
cores,
and
we
and
there's
sort
of
some
some
prototype
level
approach.
Some
prototype
versions,
this
for
also
doing
this
for
multi
a
multi,
node
GPU
systems
as
well.
If
you're,
you're
interested
in
I
think
we
love
to
eventually
potentially
collaborate
on
that
as
well.
Stefan.
D
I
had
a
quick
question:
yeah
Google's
done
some
evaluation
of
programming
languages,
high-level
productivity
programming
languages
for
deep
learning,
and
you
know
right
now,
the
syntax,
at
least
that
you
had
on
this.
It's
for
Python,
Jax
and
Swift
and
Julia
very
very
similar
I
was
wondering
if
you've
reached
a
conclusion,
I
guess
at
Google
on
the
way
you
wanna
go
forward.
B
B
In
again,
Jax
is
also
sort
of
you
know
again,
fairly
experimental
sort
of
a
lot
of
these
are
sort
of
I.
Don't
think,
there's
like
a
like
a
single
answer
for
the
right
way
to
the
right,
even
in
terms
of
whether
that's,
whether
that's
like
both
tensorflow
and
jacks,
or
because
you're
very
actively
developed,
for
example,
okay,
yeah,
yeah
I
think
there's
a
lot
of
merits
of
different
different
program
models,
especially
depending
depending
on
the
use
cases.
B
Is
it
really
is
powerful
research?
Is
it
I
think
it
has
possibly
the
most
sophisticated
Autodesk
systems
in
any
sort
of
modern
sort
of
deep
learning
framework?
And
this
these
this
tends
to
be
a
particularly
valuable
right
now,
mostly
for
sort
of
research
use
cases
but
I
think,
especially
for
like
scientific
computing.
This
becomes
world
for
things
like
forward
mode
differentiation
which
is
sort
of
a
sort
of
first-class
part
of
how
Jax
works.
It
actually
builds
up
reverse
mode,
which
you
need
to
be
flirting
out
of
forward
mode.
B
It
has
support
for
complex
numbers
and
also
support
for
sort
of
some
more
advanced,
auto
deaf
use
cases.
Things
like
features
like
like
gradient
checkpointing,
which
is
important
for
saving
memory
in
very
in
in
very
long
rain.
Simulator
simulations
where
you
have
save,
where
you
have
say
solving
in
2d.
B
There
are
multiple
ways
you
can
define
derivatives
for
this
operation,
so
one
way
to
do
it
is
to
define
the
is
the
calculate
derivatives
sort
of
sort
of
through
the
operation.
So
it's
like
going
through,
for
example,
going
through
the
the
each
step
of
the
Ute
solver
and
and
using
the
automatic
differentiation
rules.
B
That's
almost
never
how
you
want
to
do
it
so
for
all
these
four
cases
where
you
can
just
sort
of
ranking
that,
in
terms
of
the
equation,
the
sort
of
the
right
way
to
find
derivatives
is
typically
sort
of
often
by
sort
of
differentiating,
differentiating
an
implicit
function,
and
this
can
often
save
you
a
lot
of
memory
and
speed
as
well,
and
so
this
is
something
that
anyway.
So
we
have
to.
B
We
have
some
examples
of
this,
but
this
in
practice
this
is
often
turns
out
to
be
pretty
important
for
doing
for
if
you
want
to
actually
use
these
say
inside
as
part
of
larger
programs,
that
involves
some
deep
learning
components.
I
mean
a
few
prototypes
to
visit
today
in
Jackson
we're
very
interested
in
sort
of
trying
to
helping
to
build
building
this
out.
With
your
help,
okay,
so
what
it
sort
of
switch
gears
a
little
more
now,
a
little
more
towards
the
science
and
kind
of
sort
of
sort
of
things.
B
But
really
just
as
this
notion
that,
instead
of
that
kind
of
the
interesting
thing
about
deep
learning
is
maybe
that
it's
a
way
of
writing
parametrized
computer
programs
that
we
have
a
procedure
for
for
for
optimized.
So
it's
a
notion
that
we
can
write
sort
of
various
flexible
programs
and
we
can
sort
of
tune
all
the
Cramer's
sort
of
automatically
and
this
paradigm
I
really
like
for
for
scientific
computing,
because
even
more
so
than
a
lot
of
sort
of
conventional
applications
of
machine
learning.
B
In
scientific
computing,
we
had
a
lot
of
domain
knowledge
and
we
have
really
sophisticated
programs
for
this
sort
of
exploit
that
domain
knowledge,
and
so
the
this
difference.
What
programming
paradigm
is
really
exciting,
because
it's
a
way
that
we
can
complain
that
together
so
concretely,
we
have
their
machine
learning
problems
that
they're
sort
of
eventually
approximation
problems
and
and
for
approximation
problems
where
there's
a
rule
of
thumb.
Machine
learning
is
like
is
is
neat.
B
Learning
is
a
really
effective
way
to
do
that,
like
neural
nets
are,
are
great,
other
other
other
machine
learning
techniques
are
also
great.
If
you
have
something
like
a
it's
really,
the
best
we
can
do
is
like
solve
something
like
solve
something
like
a
regression
problem.
We're
an
interpolation
problem.
Machine
learning
is
really
effective.
B
Now,
unfortunately,
for
many
rural
problems,
we
actually
that
was
kind
of
done
for
a
lot
of
science
applications
there
are
we
have.
We
have
numerical
methods
that
that
we've
been
we've
developed,
I,
guess
really
over
the
over
say
the
past
I
guess
maybe
70
years
of
scientific
computing,
that
building
a
holida
main
knowledge
and
the
lettuce
into
the
both
in
terms
of
to
be
knowledge
of
sort
of
sort
of,
like
say,
like
the
physics
we're
solving
and
also
how
to
solve
those
equations,
and
so
it's
being
able
to
make
use
of
that.
B
Those
looking
America
doesn't
sort
of
build
on
that
sort
of
established.
Those
established
check
means
is
really
important
and
that's
something
that
we
can
do
with
deep
learning,
because
deep
learning
is
sort
of
a
mode
of
a
mode
of
machine
learning
that
that,
with
this
with
this
differential
program,
paradigm
lets
you,
we
can
integrate
together,
sort
of
six
components
of
programs
and
a
little
components
of
programs,
and
so
I'll
show
some
examples.
But
we're
talking
about
PDEs.
But
things
like
we're.
Solving
like
the
fact
that
we're
like
the
resolving
we're
solving
ordinary
different
equations.
B
We
have
things
like
conservation
laws.
If
we
can
build
that
structure
into
into
our
programs,
that
makes
that
makes
them
empirically
work
much
better,
and
these
are
the
things
that
turn
out
to
be
really
hard
part
for
machine,
or
so
it
is
I
think
basically,
is
an
unsolved
problem
for
Ferguson
learning
techniques
to
learn
to
learn
hard
rules
about
about
how
the
universe
works,
and
so
we
can
build
in
that
prior
knowledge
interest
into
our
programs.
It
works
much
better.
B
This
first
application
area
abet
worked
on
for
using
deep
learning
for
discretizing
PDEs
and
rising
in
fluid
dynamics.
Okay,
so
I
probably
don't
need
it,
and
this
is
work.
It's
a
bunch
of
colleagues.
From
from
from
Google
and
Harvard
we
published
last
year
in
PNAS,
we
have
some
some
follow-on
paper
work
here
that
that's
in
a
archived
paper
and
we've
been
sort
of
continuing
to
work
to
work
in
this
area
and
kind
of
the
framing
here
is
that
there
is
which
I'm
sure
needs
no
introduction
for
this
crowd.
B
But
there's
the
world
is
full
sort
of
many
assistants
systems
that
we
care
about
model
in
the
world
are
spread
by
PD,
ES
and
PD
ease
are
remarkably
hard
to
simulate,
particularly
when
we
look
at
something
like
a
like
a
state
of
the
art
with
a
forecast
model
which
divides
the
world
up
into
you
know
into
a
little
grid.
We
these
systems
are
sort
of
described,
are
modeled
with
with
a
with
a
sort
of
vastly
inadequate
resolution.
B
First
of
resolving
all
the
four
sort
of
resolving
all
the
physics
and
detail
that
we
care
about,
and
and
in
order
to
tomorrow,
things
accurately,
we
need
things
to
solve
things.
One
really
ideally
solve
things
on
really
fine
grids,
and
we
kind
of
have
we
can't.
We
know
that
in
principle
or
methods
for
solving
PDEs
would
should
converge
and
between
accurate
solution.
B
If
we
could
only
solve
things
on
a
coarse
enough
grid,
I
started
finding
this
grid,
but
we
can't
afford
to
do
that
because
computationally,
it's
computationally,
it's
it's
too
expensive,
I
mean
one
I,
guess
one
important
point
years:
we
don't
we
don't
really
care
about,
as
necessarily
a
finish
for
a
hurricane
sure,
we'd
love
to
see.
We
it's
not
even
so
important
that
we
that
we
maybe
see
where
every
last
little
cloud
is
in
this
picture,
but
we
really.
B
We
definitely
want
to
model
accurately
where
the
hurricane
overall
ends
up
and
doing
end,
and
that's
something
that
the
kind
of
the
only
way
we
know
how
to
do
that
with
real
physics
based
approaches
is
to
model
the
entire
thing
tomorrow,
the
entire
thing
accurately
and
that's
and
that's
something
that
we
really
need.
High
resolution
is
very
challenging
to
afford
some
kind
of.
B
At
this
at
this,
the
super
resolution
problem
in
computer
vision,
and
so
this
isn't
it
so
I'll
show
you
an
example
of
this.
So
here
I
have
a
very
pixelated
image
of
a
this
is
a
sort
of
cartoon
face
here
and
the
super
resolution
problem.
The
challenge
is
to
try
to
to
try
to
figure
out
what
the
what
the
high
resolution
version
of
this
picture
should
look
like.
So
this
is
an
ill-posed
problem.
There
isn't
a
clear
answer.
One
way
to
do
it
is
to
kind
of
smooth
things
out
use.
B
B
Eight
years
we've
we've
we've
learned
that
actually
deep
learning
to
be
remarkably
effective
for
solving
problems
like
this,
so
this
was
from
this
is
actually
2016
now,
but
people
that
showed
that
that
they
could
use
neural
nets
based
on
Gans
to
sort
of
reconstruct
a
much
higher
resolution
image
here
this
face,
and
a
lot
of
the
detail
here
is:
maybe
not
not
entirely
right.
So
if
you
look
at
compared
to
the
ground
truth,
the
like
the
detail,
this
necklace
is
like
this
wrong.
B
B
It
was
a
much
better
job.
Look
we're
faithful
job
of
sort
of
capturing
the
statistics
of
these
images,
and
so
this
is
good
love
to
be
able
to
do
something.
The
hope
is.
Maybe
we
do
something
like
this
or
with
for
sort
of
physical
solving
equations
as
well,
so
so
so
that
we
can
look
on
that,
we
can
effectively
achieve
sort
of
a
virtual
higher
resolution
for
salt
for
salt
for
solving
PDEs
when
today,
basically
every
every
numerical
method
that
we
use
use
something
much
less
byte.
B
Basically
it's
a
bit,
so
we
the
the
methods
we
use
for
solving
PDEs
today,
make
very
few
assumptions
about
how
about
the
about
the
the
form
of
the
equation
is
really
just
assuming
that
things
are.
Things
are
smooth,
so
it
seems
like
we
should
be
able
to
perhaps
or
capture
more
more
of
the
statistics
of
equations
and
do
a
better
job
assaulting
them.
So
talk
a
little
bit
about
what
it
looks
like
concretely
to
solve
this
all
PD
easin
again,
why
this
is
exciting,
so
so
here
I'm
showing
a
very
simple
PD.
B
B
It
turns
out
that
if
we
could
solve
this
again,
if
we
could
solve
this
on
a
much
coarser
grid,
that
would
actually
be
okay
for
many
many
applications.
We
don't
necessarily
need
the
super
high
resolution.
If
we
could
sort
of
do
the
exact,
coarse
graining,
where
we
could
just
resolve
the
PDE
on
a
coarser
grid,
that's
still
good
enough
for
making
predictions
so,
for
example,.
C
C
B
A
lot
of
purposes
for
making
sort
of
useful
decisions
based
on
that
information
that
that
actually
be
perfectly
fine
and
in
principle,
if
we
can
do
this,
is
a
very
really
fantastic,
because
you
can
do
things
because,
when
we
reduce,
we
have
a
factor
of
four
from
the
three
space
and
one
and
one
time
dimension.
This
is
power
forest.
It
really
is
really
tremendous,
so
we
make,
for
example,
we
making
this
four
times.
Coarser
would
be
250
times
faster
than
principal,
with
the
same
numerical
method.
Of
course.
B
In
practice
this
doesn't
work,
okay
and
it
tends,
and
things
tend
to
break.
When
you
solve
things
we
run
when
you
use
methods
on
course,
with
grids
tends
to
break
in
two
ways.
One
way
is
that
equation
methods
often
become
unstable,
and
that's
because
the
fundamental
assumptions
is
that
these
methods
are
derived
say
for
an
assumption
of
sort
of
smoothly,
continuous
smooth
in
continuity,
and
when
that
breaks,
the
numerical
method
breaks
too,
and
so
sometimes
that
they
break
them
like
really
sort
of
gross
and
sort
of
terrible,
obvious
ways.
B
Alternatively,
there
are
of
course
techniques
they
can
ensure
that
that
things
don't
break,
and
typically
that's
so
it
took
the
techniques
that
computational
fluid
dynamics.
Other
techniques
like
like
putting
or
slope
or
slope
limiting,
but
things
things
that
basically
ensure
that
you
can
never
insure
that
you
can.
The
physics
can
sort
of
never
never
blow
up,
but
those
tend
to
kind
of
smear
out
solutions
instead,
so
we
have
to,
let
me
actually
have
to
compromise
accuracy,
so
we
have
a
choice
between
sort
of
stability.
B
B
You
kinda
have
to
make
this
make
this,
because
this
unfortunate
trade-off,
of
course,
in
practice,
everyone
we
take
stability.
Is
it's
better
to
be
stable
than
to
like
this
is
not
as
bad
as
a
solution
entirely
blows
up,
but
it's
still
not
not
great,
and
this
this
is.
This
is
kind
of
the
big
challenge
of
trying
to
coerce
green
peas.
So
we
were
looking
this
and
sort
of
hoping
that
we
can
do
a
better
job
of
solving
this.
This
coarse
grain
simulation
problem
with
machine.
B
So
let
me
show
sort
of
step
you
through
what
it
looks
like
I'm
sure
this
is.
This
will
be
very
familiar
to
most
of
you,
but
again,
just
as
a
reminder
what
it
looks
like
we
actually
did
when
we
solve
a
PDE,
even
in
a
minute,
in
a
very
very
simple
case,
so
we
can
imagine
that
so
I'm
gonna
fall
off
here,
one
day,
PDE
in
time,
so
one
space
in
one
time,
dimension
and
kind
of
the
first
thing.
B
We
do
magic
that
remodeling
PDE
these
are
these
are
these
are
continuous
curves
in
space,
and
we
want
to
see
on
these
curves
and
when
we
do
that,
we
the
first
thing
we
do.
Is
we
pick
a
diss
causation?
We
pick.
We
represent
this
PDE
based
on
values
on
on
a
grid,
for
example,
and
kind
of
the
simplest
version
like
like
a
finite
difference
scheme.
We
just
represent
the
PDE
based
on
the
values
on
these
grid
points,
so
we
can
do
that
and
then
we
kind
of
pretend
that
we
don't.
B
We
don't
know
what
the
these
curves
look
like
anymore,
and
now
we
need
to
figure
out
how
these
pleats
evolves.
So
I
have
these
discrete
equations
change
over
time
and
to
do
that,
we
can.
We
need
to
know
how
to
sort.
We
need
to
kind
of
evaluate
the
the
equation
for
solving,
and
particularly
when
we,
what
we
need
is
we're
gonna
need
estimates
of
the
spatial
derivative
on
this
this
on
this
grid,
so.
B
Gonna
need
to
figure
out
what
the
slope
is
at
each
of
these
points,
and
once
we
have
that
slope
we
can
evaluate
every
term
in
the
equation
of
resulting
so
great.
How
do
we
figure
out
what
these?
What
these
slopes
are?
This
is
kind
of
the
ground
truth
here,
they've
shown
in
purple.
Well,
one
way
to
do
it
is
and
again
the
simple
approach.
Problems
would
be
to
say:
do
a
polynomial
interpolation.
B
This
is
kind
of
makes
very
few
assumptions
about
the
about
the
underlying
equation
that
it's
smooth,
and
so
that's
that's
as
a
reasonably
to
do
for,
if
you
know
nothing
else
about
about
the
about
the
equator
which
should
look
like,
but
in
this
case
it's
not
very
good.
As
you
can
see
these
these,
you
have
all
these
extra
overshoots
and
it's
and
it's
not
gonna-
do
a
very
good
job
of
estimating
the
slope
here.
In
fact,
if
we,
if
we
actually
use
this
in
practice,
we
these
these
errors
would
even
compounded
over
time.
B
It
would
look
really
bad,
so
this
would
actually
be
basically
unusable.
It
turns
out
now,
fortunately,
we
don't
have
to
use
in
polynomial
interpolation.
That's
because
we
actually
have
prior
knowledge
about
what
solutions
to
PDEs
look
like
and
if
we
in
particular
here
we
know
again,
we
know
the
equation,
we're
solving
those
cases.
It's
a
burgers
equation
that
we're
solving
and
we
can
solve
this
equation
a
bunch
of
times.
We
can
do
it,
we
can
do
it
on
computer
and
you
can
see
it.
B
We're
solving
I
mean
see
the
Paszek
sort
of
character,
six
shape,
so
burgers
equation
builds
up.
Shockwaves
and
they'd
have,
in
this
case,
sort
of
a
typical
width.
This
were
related
to
the
strength
of
the
dissipation
and
the
and
the
so
the
forcing
and
viscosity
here,
and
so
these
typical
the
dictionary
solutions
we
can
use
potentially
to
sort
of
train
models,
do
a
better
job
in
population.
So
we
can
train
machine
or
model
on
this,
and
we
do
much
better
and
if
we
train
too
much,
you
learn
them
all
to
do.
B
Interpolation
here
it
gets
the
it
solves.
It
can
solve
this
interpolation
problem
almost
almost
personally
right
almost
perfectly,
and
that's
because
there's
actually
there's
enough.
If
you
know
the,
if
you
know
the
equation,
you're
solving,
there's
enough
information
here
that
reconstructs
almost
exactly
we
can
reconstruct
the
the
solution.
But
you
need
to
know
you
need
to
know
something
about
the
equation:
you're
solving,
first,
okay,
so
that's
kind
of
the
idea
sort
of
in
theory
and,
of
course,
for
sort
of
what
look.
What
this
is
kind
of.
This
is
the
interpolation
problem.
B
This
is-
and
this
is
the
only
essentially
the
only
part
of
our
the
only
place
where
we
wanna
use
machine
learning,
because
there's
actually
there's
all.
We
have
a
lot
of
other
sort
of
tricks
for
miracle
methods
that
work
really
well
were
sold.
P
DS,
so
I'll
show
you
how
we
put
it
together.
So
again,
the
core
idea
is
that
we
know
the
equations
were
solving.
We
don't.
We
don't
need
to
wonder
that.
B
So,
instead
of
being
coefficients
like,
like
you
know
like,
like
plus
1/2
minus
1/2,
here
we're
going
to
we're
going
to
sort
of
dynamically
produce
these
these
coefficients
with
with
a
neural
network-
and
this
is
this-
is
where
we're
gonna
plug
proximation.
So
now
we're
gonna
sort
of
now,
then
so
so.
F
B
In
this
case,
were
so
we
pick,
we
happen
to
the
one
thing
we
haven't
done
here
to
eat,
we're
still
using
a
fixed
representation,
that's
up
for
the
PD,
so
we're
solving
it.
So,
on
a
grid,
the
we're
solving
we
are
so
these
coefficients
are
so
they
they.
They
could
be.
I
guess
like
an
example
of
so
these
are
effectively
take
the
place
of
say
finite
difference
coefficients,
but
what
we're
actually
like
we're
letting
them
be
for
a
dynamically
determined
by
from
some
local
by
the
local
solution.
B
We
and
you
need
to
do-
is
you
calculate
this,
this
equation
for
a
flux,
and
that
requires
that
requires
evaluating
in
terms
spatial
derivative
terms
and
sort
of
and
they're
kind
of
the
various
ways
you
could
learn.
You
could
various
rules
you
could
use
for
out
or
how
to
calculate
spatial
derivatives.
But
a
very
common
approach,
like
a
finite
difference
approach
is
to
use,
is
effectively.
B
It
looks
like
a
limit
play
a
convolution
kernel
which
can
approximate,
which
can
approximate
these
terms
and
so
some
some
sort
of
some
sort
of
some
sort
of
something
like
a
smoothing,
kernel
and
they're
different
versions
of
this.
For
example,
we
do
a
if
a
video
to
say
a
second
order
or
fourth
order
approximation
or
if
we
do
things
with
a
or
we
could
say
a
spectral
approximation
answer
different
different
different
models.
B
We
can
use
different,
different
sort
of
rules
for
sort
of
calculating
these
coefficients,
for
example,
and
typically
they're
just
they're,
just
something
that
you
might
sort
of
look
up
at
a
table
or
you
try
different
sort
of
sort
of
see
what
you
see
what
works
best,
and
so
here
we're
determining
so
here.
The
really
the
only
difference
is
that,
instead
of
these
numbers
being
fixed
numbers,
we're
saying
let's
let
these
beep
sort
of
parametrized
those,
let's
have
some
sort
of
small,
simple
primary
choice-
function:
that's
going
to
generate
what
these
coefficients.
B
And
so
in
this
case,
it's
so
this
case.
It's
it's
a
small
like
it's
a
three
layer,
convolution
on
their
own
neural
network,
and
so
then
we
take
this
and
we're
then
we're
going
to
plug
it
into
the
rest
of
the
Miracle
Method.
We're
got
capped
at
a
flux.
We're
gonna
count
their
time
derivative
we're
going
to
use
the
method
of
lines
for
sort
of
advancing
in
time,
and
that's
kind
of
that's
that
that's
what
the
us
with
the
computation
looks
like
is
that
is
that
helpful
or
useful?
You
still
questions
there
about
I.
F
B
The
use
okay,
so
in
this
case,
so
they'll
escape
that
use
would
be,
for
example,
the
value
of
the
this
is
a
1d
PD,
but
it
this
would
be
something
like
the
value
like
if
we
in
this
case
something
with
it,
you
might
resent
to
say
the
sell
average
value
at
each
at
each
points
across
the
solution
to
this
equation.
So
it's
so
we
just
pick
so
here
we're
picking
a
12-2
selling
things
on
a
grid.
B
C
B
And
there's
a
lot
of
other
and
there's
been
a
lot
of
interesting
work
in
terms
of
other
ways
to
represent
Pease
with
things
like
things
like
some
sort
of
like
embedding
space
or
like
as
and
there's
a
lot
of.
That's
not
something
we've
done
here
here
we
haven't
been
really
learning
the
representation.
We've
just
been
learning
how
we're
gonna
solve
things
given
us,
given
this
fixed
replication.
Okay,
thank
you.
Can.
G
B
G
B
Yes,
absolutely
so
we
can
plot
this
out
and
I
think
we
have
some
examples
in
the
paper,
but
you
can
look
at
these
two
coefficients
and
they're
going
to
be
different,
different
parts.
The
solution,
particularly
here
they're
gonna,
obviously
member
got
very
most
strongly
where
the
solution
varies
most
from
as
well
so
at
these
weight
at
these
shock
fronts,
they're
going
to
be
really
they're
going
to
vary
pretty
dramatically
and
where
it's
smooth,
they're
gonna.
B
Okay,
so
I'll
show
you
a
little
a
little
bit
here,
just
kind
of
example,
what
this
looks
like,
and
so
we
can
solve
so
there's
the
interesting
thing
we
can
do
with
this
is
when
the
models
we
train,
let
us
list
simulate
accurately
on
much
coarser
grits.
So,
for
example,
we
can
represent
this
now
we
person
this
equation
with
shown
here
when
we
say
16
points
and
we
actually
get
a.
B
We
get
a
solution
that
looks
both
qualitatively
and
quantitatively
is
actually
better
than
what
we'd
see
was
solving
this,
even
with
with
a
say,
a
simple
finite
difference,
method
with
would
say
four
times
as
many
points
in
space
and
that's
largely
because
we
can,
with
the
even
with
so
few
points
we
can
actually
accurately
you
can
kind
of
infer
if
you
switch
with
the
graphing,
even
just
based
on
these
on
these.
On
these
on
these
orange
boxes,
you
can
see
where
the
curt,
where
were
they
shock
fronts,
are
not
turned
out
the
most
this
equation.
B
H
B
Some
experiments
where
we've
try
it
so
it's
a
very
natural
question
to
just
try
it
thrown
away
the
PDE
entirely
and
it's
what
we
found
is
that
keeping
the
PDE
structure
actually
seems
to
really
help
in
terms
of
the
both
in
in
terms
of
these,
for
the
accuracy,
the
model
that
you're
fitting
and
also
its
stability.
So
it's
really
easy
to
get
these,
especially
for
some
of
these.
B
So
some
of
the
PDS
we
looked
at
that
are
sort
of
more
more
pathological
in
some
ways
so
systems,
for
example,
that
didn't
have
dissipation,
or
that
mean
this,
for
this
one
has
forcing
as
well
so
say
without
forcing
it's
one.
I
think
one
of
the
challenges
in
those
fields
actually
is
is
that
coming
up
with
techniques
that
are
stable,
so
you
can
train
it
on,
say
one
time
step
or
a
handful
of
time
steps
and
then
that
you
can.
B
That
will
still
give
good
results,
even
when
you
run
it
out
for
thousands
of
time
steps.
That's
I
think
really,
obviously
really
important
for
a
method
that
you
can
count
on
and
if
you
don't
build
them,
the
statistical
structure,
I
guess
our
general
experience
was
that
that
was
that
the
the
pure
neural
network
approaches
tend
to
be
very
unstable.
They'll
give
good
predictions
for
a
few
steps,
but
then,
if
you
transfer
a
few
steps
and
then
run
it
for
a
few
more
steps,
they
tend
to
blow
up
Thanks.
H
B
B
Those
through
these
systems,
even
through
neural
networks,
because
we
have
we
have
auto-da-fé-
is
we've
done
some
experiments
with
that
and
there's
a
few
interesting
papers
on
this
as
well.
For
example,
trying
to
like
trick
trying
to
like
add
instability,
penalties
or
constraints,
I
think
it's
still.
C
B
B
So
compare
a
bunch
of
these
different,
these
different
methods,
and
so
we
solved
this
on
very
different
grids
at
various
different
Florida
support
systems.
Can
we
sort
of
see
how
accurately
can
solution
and
kind
of
up
here
on
the
left?
Are
the
the
top
are
sort
of
this
sort
of
more
standard
like
a
first
order
volume
method,
and
so
these
methods
that
they
don't
work
great,
but
they're
they're
their
baseline?
B
You
can
also.
There
are,
of
course,
much
more
sophisticated
methods
for
solving
for
solving
different
types
of
PD
es
I.
Think
inferred,
particularly
for
burgers
equation.
Probably
one
on
the
most
relevant
F
is
the
technique
called
Tippi
called.
We
know
which
uses
I
guess
I
would
say,
use
it
if
my
probation
is
sort
of
very
sort
of
clever
rules
for
for
how
to
kind
of
interpret
late
and
the
sin
of
your
shock
weights
such
that
it
never
overshoots.
B
With
this
learned
representation,
and
we
can
also
be
a
few
other
variations
here,
where
we
can
serve
built
in
other
types
of
some
of
these
other
physical
constraints,
sort
of
more
sophisticated
finite
volume
methods
like
it's
good
enough
technique,
and
we
can
also
even
do
very
simple
models.
We
don't
even
have
necessarily
train
a
full
neural
network.
We
can
also
try
just
instead
of
saying
well
I'm
only
gonna
do
a
first
order
method
or
a
third
order
method.
B
I
can
also
do
something
like
say,
I'm,
going
to
allow
myself
I'm
just
gonna
like
learn,
constant
coefficients
and
end
and
see
how
well
that
works
as
well,
so
I'm,
just
gonna
try
learning
some
set
of
finite
difference
coefficients
instead
of
using
the
ones
I.
Look
at
that
I
might
look
up
in
a
table,
for
example,
and
with
that
we
can
actually
also,
in
some
cases,
have
a
have
a
significant
improvement
for
some
of
these
some
of
these
standard
techniques.
So
you
don't
necessarily
need
to
really
sustain
fancy.
B
B
Some
people
actually
care
a
lot
about
advection
for
the
modeling
same
modeling
systems
like
like,
like
solution
flow
and
the
atmosphere
and
also
is
like
a
real
real
building
block
of
of
what
we'd
use
for
in
a
full
sort
of
2d
or
3d
computational
fluid
dynamics
scheme
in
actually
it's
kind
of
one
of
the
hardest
parts
of
solving
now
because
those
equation
to
resolve
is
is,
is
the
convection
term.
So
here
we're
gonna,
look
at
we're,
gonna,
look
at
it,
sort
of
we're,
gonna,
look
at
velocity
fields.
B
We
extracted
from
from
sort
of
a
decaying
two-dimensional
turbulence
and
we're
gonna
see
how
well
we
can
do
for
sort
of
firm
for
falling
sort
of
passive
scalar
infection
in
these
velocity
fields,
and
so
I'll
show
you
just
an
example.
Hopefully
this
shows
it.
Okay
is
what
is
an
example
of
what
what
it
looks
like
to
solve.
B
To
use
sort
of
standard
methods
for
salting
infection
on
on
these,
and
what
you
see
is
that
if
you
have
really
high
resolution,
you
can
make
some
really
really
pretty
pictures.
But
if
we,
if
we
solve
things
like
coarse
grids,
we
see
this
phenomenon.
I
showed
earlier
everything
the
solutions
get
really
speared
out,
and
they
don't
look
very
good.
And
that's
because
we
have
to
this.
The
methods
need
to
sort
of
void
these
sort
of
very
sharp
discontinuities.
B
I
A
question
yeah
so
in
the
context
of
where
neural
networks
have
been
applied
for,
say,
image
classification.
Yes,
there
is
something
understanding
that
the
reason
they
work
so
well
is
because
perhaps
these
new
networks
are
able
to
learn
like
representations
much
more
effectively
and
and
and
people
are
extracted
information
from
the
from
the
hidden
layers
to
see.
Why
is
it
working
so
well?
So
is
there
a
wondering,
could
you
provide
intuitive
understanding
of
why
they
work
so
well
with
automatically
architecture
with
a
set
of
neurons?
B
A
great
question
so
so
we
did
so
we've
had
sort
of
I
guess
some
success,
but
not
not
a
lot
of
success
in
terms
of
interpret
understanding
exactly
what
the
models
for
John.
So
one
thing
that
we
will
be
found
that
we
discovered
actually
and
I
wish
I
had
to
slide
on
this
is
here,
but
you
can
see
if
you
look
at
our
paper,
is
that
there's
a
technique
for
solving
PDEs
called
called
up
wounding,
which
is
the
that
the
they
to
sort
of
accurately
model
a
fluid
flow?
B
You
should
preferentially
look
in
the
direction
the
wind
is
coming
from
and
that
helps
you
get
causality
right
because
if
because
causality,
only
flows
downward,
and
so
this
is
something
that
most
sort
of
sophisticated
numerical
methods
used.
Methods
like
this,
like
a
solver,
uses
this,
but
actually
we
didn't.
B
If
you
compare,
for
example,
if
you
look
at
again
looking
at
these
burgers
equation
systems
like
these,
like
really
simple
methods
like
this
baseline
method,
fine
fine
volume
method
doesn't
do
that,
and
we
found
actually
is
that
we
were
able
to
sort
of
look
at,
because
we
have
this
interminable
structure
where
we
we
know
that
we're
producing
these,
these
sort
of
essentially
interpolation
coefficients
or
these
misfiring
difference
coefficients.
We
can
look
at
those
coefficients
and
we
can
actually
notice
things
like.
B
I
Is
a
follow
up
to
that
so
even
again,
in
the
context
of
image
classification
now
they're
kind
of
yes,
some
tasks,
I
mean
people
have
discovered
these
bags
of
tricks
like
Bravo,
regularization
or
or
you
know,
changing
at
an
age
which
you
should
be
classically
used
to
review,
or
a
leaky
raloo
yeah
I'm
talking
are
those
those
same
tricks
apply
even
to
PDS,
or
is
that
a
different
bag
of
tricks
that
you
need?
That's.
B
A
good
question
and
then
it
could
be
back
it'd
be
a
whole
other
talk
actually
I'd
say
some.
The
same
same
techniques
work.
My
experience
has
been
that
sometimes
they
they
don't
I
mean
particularly
I.
Think
probably
the
biggest
challenge
is
maybe
not
so
much
humans.
A
lot
of
this
work
is
when
it
comes
up
a
lot
of
times.
You
end
up
doing
effectively
doing
some
sort
of
higher
order.
B
Differentiation
for
when
you're
doing
say,
like
a
some
kind
of
physics,
constrained
some
sort
of
business,
constrained,
deep
learning
system
and
in
those
cases,
a
lot
of
the
standard
trick.
Tricks
don't
work
so
well
because,
for
example,
a
real
ooh
is
is
just
step
function.
You
can,
if
you
take
a
take
a
we
take
its
derivative,
so
it's
so
the
higher,
and
so
there
are
some
I
think
there
are
some
useful
tricks.
Maybe
a
handful
of
useful
tricks,
they're,
sort
of
a
little
different
for
sort
of
physics,
applications,
but.
B
E
Do
you
have
any
sense
of
how
if
it's,
if
it
is
possible
to
have
some
sort
of
metaphysics
learner,
where
you
don't
have
to
have
a
specific
or
physics
based
constraint
for
every
single
different
type
of
simulation
that
you
are
building
up
towards?
Oh,
is
it
first
of
all
possible
and
if
it
is
possible,
so
how
how
would
the
sort
of
the
current
system?
So
there
is
some
work
on
long
range?
It
was
something
like
that.
Would
those
be
the
kind
of
like
like
architecture,
searches
or
architecture
developments,
which
would
aid
in
that
particular
respect.
E
B
That's
a
very
question.
So,
yes,
I
got
some
other
work
that
I
guess
you
can
I
don't
have
in
this
talk,
but
you
can
see
with
with
some
other
other
work
on
using
sort
of
lagron
sort
of
Lagrangian
structure
for
for
search
for
some
of
these.
Like
these,
these
that's
where
you
learn,
say:
learning
affleck,
flexible
lagrangian,
then,
let's
say
solving
that
with
automatic
enunciation
and
that
can
have
some
some
simple
cases.
I
think
the
challenge.
The
real
challenge
is
that
we
is
okay.
B
Well,
so
I
guess,
there's
in
my
mind,
is
it's
still
not
entirely
clear
when
those
techniques
would
actually
be
useful
and
that's
because
I
guess
most
of
the
time
we
have
a
lot
of
physics
we
actually
we
do
have
a
lot
of
physics
problems.
We
like
we
know
the
general
equations
were
falling
in
case.
We
really
need
to
discover
the
equations
that
were
solving
and
that's
I
think
that
it
becomes
a
much
harder
learning
problem
and
I
know.
B
In
addition
to
learning
sort
of
some
of
the
finer
details
about
salt
and
my
mind,
they're
not
entirely
appropriate
for
us,
you
might
I
may
attend
my
personal
perspective
is
that
AI
works
best
today
is
when
you
think
about
it,
for
solving
interpolation
problems
and
take
problems
like
that
or
not
really
when
you're
trained,
and
so,
if
you
can
find
a
way
to
turn,
you
know
learning
equations
into
an
interpolation
problem,
that's
fantastic!
That
might
work,
but
that's
kind
of
that's
kind
of
the
challenge
we
need
I,
don't
think
AI
has
really
solved
anything.
H
B
Oh,
those
kids
have
intended
the
challenges
suffered
in
sambal
of
different
solutions
to
the
equation.
So
we
kind
of
pick
some
random
noise
or
any
initial
conditions
we
could
have.
We
have
heaven
in
samba
love,
so
maybe
hundreds
or
in
this
case
these
are
a
small,
fast
PG.
So
we
have
a
few
thousands
of
examples
and
then,
in
this
case
we
didn't
pick
a
domain
size.
We
did.
We
do
test
these
models
on
large
amazing.
They
do.
They
do
generalize
nicely
to
because
they
do
they
do
generalize
to
sort
of
much
larger
domains.
B
B
Definitely
I
mean
the
cleverness
is
that
you
need
to
be
I
mean
right
like
again,
because
machine
learning
is
just
those
only
solving
tribulation
problems.
You
every
sort
of
scenario
you
solve
needs
to
be
covered
effectively.
So
the
locally
you
need
to
see
every
every
phenomena
that
you
hope
to
model
we
need
to
see.
It
needs
to
be
resolved,
at
least
once
in
the
training
data,
and
that's
that's
why
it's
really
important
that
we
design
our
neural
network,
for
example,
based
on
the
convolutional
structure.
B
H
B
Yes,
the
main
rule
of
thumb
would
be
to
make
sure
that
your
mock,
that
your
I
mean
I,
think
that
at
least
it
so
I
think
one
is
for
me-
is
we're
kind
of
really
helpful
to
be
sure
that
it's
something
that
at
least
with
your
human
insight?
You,
like
you,
really
think
believe
that
you
can
solve
the
problem
and
so
like
if
I
look
at
something
like
something
like
this
problem
here.
Are
this
windy
interpolation
problem
I?
B
Look
at
this
I'm
like
yes,
I
can
interpolate
that,
like
I
can
follow
those
Kurt's
like,
and
so
the
challenge
is
really
sort
of
breaking
it.
The
tasks
down
to
the
simple
level
where
you
believe
that
it
is
actually
that
looks
like
something
that
with
something
like
like
it
like
a
human
visual
system
which
neural
nets
can
do
pretty
a
pretty
good
job
of
approximating.
B
We
think
they'd
be
interpolation
problems
that
we
can
solve
them
that
yes,
they're
tractable.
If
we
think
that
sort
of
based
on
like
looking
at
examples,
if
give
it,
if
not,
then
I
think
it
could
be
pretty
pretty
hopeless
and
if
you
have,
depending
on
the
size
of
your
training
data
set-
and
they
might
but
also
could
be
a
helpful
guide
for
like
looking
you're
trained
as
well
I
mean
sometimes
neural
nets.
Can
work
really
well
even
on
very
few
examples.
B
So
I'm
not
gonna
talk
about
the
other,
the
other,
so
the
other
project
that
I
have
made
some
slides
for
you
today,
but
we
have
it
again,
even
as
little
as
like
one
example
can
actually
work
work
well
for
neural
networks,
if
you,
but
that,
but
it
needs
to
be
something
we're
at
the
very
least
that
there's
maybe
some
some
hope
with
human
insight.
I
think
very.
G
So
in
terms
of
this
chart
that
you
talked
about,
there
are
two
questions
about
the
cost
versus
also
accuracy.
Yes,
so
because
you
calculate
these
coefficients
at
every
time,
step
because
there's
very
specific
to
the
grid
as
well
and
when
there's
let's
say
these
fronts
are
moving,
then
you
have
to
reevaluate
their
so,
like
you
know,
it
is
not,
which
is
not
so
costly
to
get
keep
doing
it.
B
And
so,
while
in
here,
but
we
actually
did
do
some
comparisons
to
I
think
this
is
really
important
whenever
looking
up
some
fancy
a
neural
net,
that
is
it
seriously
how
expensive
it
me
see
if
I
have
that
I,
don't
think
I
actually
I
kept
the
numbers
here,
but
basically
we
found
it
in
this
2d
advection
case.
We
sought
resolving
things
on
a
four
times,
coarser
grid,
and
it's
you
might
hope
that
it's
like
four
times
that
it's
that
it's
like
250
times
faster.
It
turns
out
it's
actually
about
the
same
expense.
B
B
You
do
need
to
really
important
to
actually
benchmark
how
fast
your
neural
net
is
because,
when
you're
solving
things
in
the
in
your
inner
loop
but
I
solve
already
doing
it
and
you're
yourself
and
you're
you're
you're
running
that
solver
bunch
of
times
the
neural
nets
can
become
very
expensive.
The
good
news
is
that
is
that
neural
net
flops
are
much
cheaper
in
some
sense,
then,
then,
the
flops
you'd
spend
on
on
a
conventional
PD
solar.
B
B
A
I'm
gonna
interrupt
for
a
moment
because
it
is
past
the
hour
and
I
do
want
to
thank
Stephan.
I
know
this
is
the
virtual
talk,
so
I'll,
clap
but
I'm
sure
everybody
else
is
virtually
flapping.
Obviously,
it's
a
very
interesting
talk
and
I.
Think
lots
of
people
have
lots
of
questions
so
you're
welcome
to
stay
we're
going
to
continue
on
this
channel
for
maybe
30
minutes,
but
those
that
want
to
leave
are
obviously
welcome
to
leave.