►
Description
NERSC Data Seminars Series: https://github.com/NERSC/data-seminars
Speaker:
Manolis Papadakis
NVIDIA
Title:
Legate: High Productivity High Performance Computing
Abstract:
The Legate project (https://github.com/nv-legate) aims to provide distributed and accelerated drop-in replacements for popular scientific / data-science libraries (NumPy and Pandas so far). Our goal is to allow programmers to prototype their applications on their local machine, then be able to transparently scale up to large clusters, utilizing the available acceleration hardware, without having to rewrite their application in an explicit parallel programming model like MPI or Dask. By building all Legate libraries on top of Legion's distributed data model and runtime we can achieve seamless asynchronous execution across libraries, with minimal blocking and copying.
Bio:
Manolis Papadakis is a Senior Software Engineer at NVIDIA, working on the Legate project.
Host of Seminar:
Laurie Stephey
Data & Analytics Group
National Energy Research Scientific Computing Center (NERSC)
Lawrence Berkeley National Laboratory
A
All
right,
I
think
we
have
a
quorum
so
hi
everybody
welcome
to
the
nurse
data
seminar.
Thank
you
very
much
for
coming.
I'd
like
to
first
introduce
our
speaker,
so
we're
really
lucky
to
have
manolis
papadakis
from
the
nvidia
legate
team.
He
recently
received
his
phd
from
stanford
in
computer
science
and
now
he's
a
senior
software
engineer
working
on
nvidia's
the
game
project.
B
Thank
you.
Thank
you.
Thanks
for
the
introduction,
thanks
everybody
for
having
me
yes,
so
league,
what
what
is
legate,
what
is
this
project?
Let's
see
all
right?
Well,
let's
start
with
a
little
bit
of
motivation.
What
we're
trying
to
do
here
is
our
vision
for
running
hpc
applications
with
python.
So
what
does
it
look
like
to
write
a
to
do
scientific
computation
in
python
on
a
single
chord
a
day?
Let's
say
I
want
to
do
a
combustion
simulation
with
chemistry,
natural
choice
would
be
would
be
numpy
I
have.
B
I
can
have
multi-dimensional
arrays
to
express
the
different
degrees
of
freedom
that
I
want
to
simulate.
I
can
use
nice
slicing
syntax
to
to
update
with
with
update
every
cell
in
parallel,
so
that
that
is
good
for
my
simulation,
let's
say
later
on,
I
want
to
do
a
statistical
analysis
on
the
results
of
this
simulation
on
the
order
of
the
different
species.
B
I
might
pass
the
data
on
to
pandas.
I
can
do
that
on
a
single
node
without
incurring
additional
copies
without
needing
to
transfer
the
data.
The
different
libraries
generally
know
can
recognize
each
other's
data
format
through
through
introspection
through
the
dynamic
type
system.
So
all
of
this
works
nicely,
I
can
pass
it
in
everything
runs
on
the
same
node.
I
can
run
the
two,
the
two
applications
in
walk
step
or
at
different
times,
and
they
all
run
together
nicely.
B
I
can
even
have
the
the
the
decisions
of
pandas
flow
back
into
the
simulation
suggest
different
configurations
see
if
the
simulation
is.
If
the
results
are
stable,
it
all
works
nicely.
It
can
all
work
in
the
same
context,
and
I
can
even
on
this
on
the
one
node
that
I
am
I
can.
I
can
visualize
the
results
of
the
simulation
I
can
even
use
on
the
side
of
pandas.
B
I
can
use
classifier
to
process
the
results
of
the
simulation
and
uncover
uncover
different
characteristics
of
the
of
the
flamewave
fronts.
I
can
visualize
that
either
in
a
new
library
or
can
even
reuse
the
different,
the
library
that
I've
used
previously,
so
it
all
works
together.
It's
all
pretty
seamless
and
that's
on
the
single
node
and
that's
what
we
we
think
the
situation
should
be
on
on
the
super
computing
front
as
well.
However,
the
the
reality
of
it
today
is
is
a
little
different.
B
Let's
say,
you're
trying
to
program
a
supercomputer.
Now,
the
of
course
the
single
node
numpy
implementation
is
not
going
to
be
sufficient
anymore
you're
going
to
need
to
switch
to
a
an
explicit
parallelism
framework
like
mpi.
Now
you
need
to
do
a
management
of
resources.
You
need
to
decide
where
the
simulation
will
run,
where
pandas
will
run,
where
your
visualization
will
run
and
the
visualization
might
not
even
run
on
multi-node.
B
So
now
you
have
this
latency,
this
added
latency
to
think
about,
and
of
course,
it's
it's
much
harder
to
to
move
data
back
and
forth
have
the
different
parts
of
the
application
control
each
other.
Now
again,
there
is
no.
There
is
no
one
version
of
the
data
that
that
you
can,
that
that
exists.
There
are
different
parts
of
it
that
you're
managing
yourself.
B
So
you
need
to
manage
the
communication.
You
need
to
decide
the
the
minimum
amount
of
data
that
you
need
to
transfer
and
when
you
need
to
do
that,
also
you
have
your
accelerated
hardware,
so
you
need
to
you
need
to
make
use
of
it
you're
going
to
try.
You
may
want
to
try
and
use
an
accelerated
version
of
your
of
your
linear
algebra.
B
Now
you
have
more.
Decisions
have
to
do
with
having
to
do
with
the
heterogeneity
of
the
hardware
and
how
you
launch
the
the
work
you
have,
for
example,
to
decide
whether
you're
going
to
run
in
a
process
per
node
or
process
per
gpu,
and
that
might
there
might
be
different
capabilities
that
each
of
them
allows
you
to
do,
and
one
of
them
might
be
might
be
easier
harder
than
the
other.
B
So
all
of
these
are
obviously
are
not
insurmountable
problems,
but
they
make
they
add
a
lot
of
friction
in
their
their
far
cry
from
how
you
you're
running
python
applications.
On
a
single
note
today,
and
the
goal
of
this
project
is
to
strive
towards
something
closer
to
the
previous
slide,
even
on
an
hpc
setting
so
in
in
broad
terms,
the
the
goal
of
this
project
is
to
enable
enable
users
to
program
these
large
heterogeneous
accelerated
machines.
B
B
B
No
explicit
parallelism
whatsoever
have
a
single
specification
of
the
program
that
can
run
on
anything
from
from
a
gpu
to
a
cluster
and
maintain
the
sequential
semantics
and
and
take
advantage
of
the
of
the
of
the
ample
parallelism
that
exists
within
these
apis
within
a
action
like
a
like.
A
matrix
multiply
on
numpy
discover
that
parallelism
and
take
advantage
of
it
automatically
without
having
the
user
pointed
out
to
us.
B
So
that
is
the
high
level.
The
high
level
goal
of
the
of
the
project
where
we
stand
is
that
we
had
an
open
source
release
on
github
earlier
this
year
and
we
have
two
libraries
so
far.
We
have
numpy
and
pandas
with
with
support
for
some
some
some
high
use
operations,
some
critical
operations,
and
this
is
very
much
a
work
in
progress.
We
are
adding
we're
adding
support
for
different
things
and
we'll
talk
later
about
how
you
can
influence
that
here
is
an
overview
of
the
legate
stack.
B
B
Runtime
came
out
of
an
hpc
setting
and
is
is
used
I'll
show
you
in
the
next
few
slides
how
we
use
it
to
to
run
applications
like
legate,
numpy
and
pandas
on
a
distributed
setting,
what
the
the
pertinent
features
are
and
in
the
bottom,
when
we
reach
the.
When
we
reach
the
level
of
a
single
node,
we
want
to
reuse
all
the
all
the
excellent
work
that
has
come
before
us
with
the
different
accelerated
libraries.
I
note
here
a
lot
of
the
libraries
available
for
a
single
gpu
out
of
nvidia.
B
We
also
use
open
blast
for
for
for
cpus,
for
example.
So,
yes,
on
on
a
single
node,
we
have
a
lot
of
a
lot
of
work
to
stand
on
and
I'll
show
you
how
we
can
take
advantage
of
these
accelerating
libraries
in
in
a
distributed
execution.
There's
two
features
of
legion
that
are
very
important
to
us:
the
the
transparent
parallel
execution
which
I'll
show
in
this
slide
and
in
the
common
data
model
I'll
show
in
the
next
one.
B
B
B
So
that
is
the
first
part
of
legion,
the
that
we
use
the
the
asynchronous
execution.
The
other
very
important
part
is
the
is
the
common
data
model,
so
legion's
a
notion
of
a
region
is
that
this
is
a
distributed.
Collection
that
is
can
also
be
partitioned
across
is
can
also
be
partitioned
across
the
machine,
and
the
different
libraries
in
league
8
can
understand
the
the
underlying
data
or
presentation,
the
underlying
legion
data
representation,
and
can
avoid
can
avoid
copies,
what's
it.
B
Different
libraries
can
query
the
the
existing
partitions
of
the
data
and
either
you've
either
reused
them
to
avoid
any
repartitioning
or
require
a
partitioning
for
the
particular
for
the
specific
operation
they're
about
to
they're
about
to
compute.
So,
for
example,
numpy
might
prefer
a
special
decomposition
of
the
data
where
pandas
might
prefer
a
column-based,
column-based
decomposition.
B
All
of
these
can
coexist
and
they
are
kept
in
sync
by
the
runtime,
which
makes
sure
to
only
on
in
when
repartition
is
necessary.
It
will
only
move
the
minimal
amount
of
data
to
make
sure
the
the
new
partition
is
valid.
So,
with
these
two,
with
these
two
features,
we
built
the
different
libraries
on
top
of
on
top
of
a
common
back
end,
all
right.
So
that's
that
was
the
the
high
level
overview
of
how
how
the
gate
works.
B
I
will
now
switch
to
a
demo
I've
prepared
for
for
legate
on
corey
gpu.
What
this
demo
uses
is
the
the
quick
start
scripts
I'll
have
a
link
to
this
page
later
in
the
talk.
Also,
these
are.
These
are
specialization
specialized
installations
of
of
legate
on
different
machines,
and
I've
done
I've
made
these
configurations
for
query
gpu.
So
I'll
show
you
how
you
can
you
can
you
can
run
the
same
commands
on
on
your
own?
B
B
So
I
have
this
simple
this
this
short
code,
the
the
part
to
focus,
is
here
the
main
loop,
where
we
perform
a
stencil,
a
four-point,
stencil
and
computer
norm
and
then
print
out
some
statistics,
every
100,
iterations
and
then
on
every
iteration.
We
check
if
we've
reached
the
norm
target,
so
we
can
stop
the
we
can
stop
the
the
iteration
short
we're
not
going
to
reach
that
in
this
presentation.
B
B
And
then
I
will
add
my
my
program
and
then
just
I'll
I'll
go
through
this
option
in
a
minute.
But
for
now,
let's
just
start
it
you
see,
it
will
will
create
an
output
directory.
It
will
show
you
what
what
command
is
being
used
to
submit
a
batch
script.
It
it
does
all
this
internally.
You
don't
need
to
run
a
write.
B
B
What
what
I
added
here
is
this
profile
profile
option
so
that
we
will
get
a
a
legion
profile
of
this
run
without
it,
then
this
would
have
been
the
only
output
that
we
get,
but
here
we
got
the
profiler
output
and
then
the
the
script
that
processes
it.
So
now
we
have
some
visualization
files
to
look
at,
so
I'm
gonna
switch
to
my
local
node
and
pull
the
files
that
we
just
created
on
my
local
machine.
B
B
B
B
So
this
is
just
executing
the
python
script
and
really
nothing
ran
on
the
gpu
there's
only
this
small
finalization
piece.
So
what
happened
here
here?
It
was
the
original
the
original
script
and,
as
you
can
see,
the
size
of
the
matrix
was
a
hundred
by
a
hundred.
So
what
ligate
decided?
What
legates
heuristics
decided
is
that
this
is
not
worth
going
through
the
overhead
of
legion
to
execute
and
going
through
the
overhead
of
launching
kernels
and
transferring
data
back
and
forth
between
the
cpu
and
the
gpu.
B
So
it
decided
to
just
run
it
as
as
unchanged
numpy.
Let's
see
what
happens
if
we
increase
the
size
a
little
bit,
so
I'm
going
to
make
a
change
to
this
script.
I'm
going
to
use
a
different
version
where
this
matrix
is,
is
increased
to
7k
by
7k,
so
I'm
going
to
run
the
same
command
for
b,
dot,
pi
so
again
on
one
gpu
and
same
example,
larger
larger
array.
A
B
Yep,
let
me
see
if
this
is
still
running
still
on
this
page
yeah.
This
is
there
there's,
there's
a
lot
more
configuration
you
can
do
when
you're,
when
you're
building
league
8,
for
example,
skip
support
for
gpus
or
to
other
things,
and
even
more
on
how
to
run
so
running,
for
example,
multi-run
multi-rank
on
the
same
node,
but
these
scripts
should
be
enough
for
quarry
gpu.
B
B
So,
as
you
can
see,
there
is
more
a
lot
more
activity
on
the
gpu
on
the
gpu
timeline.
I'm
gonna
just
hide
these.
For
now
there
is
some
use
of
the
of
the
frame
buffer
or
or
gpu
device
memory,
and
there
is
some
activity
on
the
channel
or
the
the
copy
where
we're
tracking
the
copies
happening
between
different
memories.
B
So
let
me
zoom
in
a
little
bit
to
see
what's
going
on,
as
you
can
see
here,
these
are
the
the
gpu
tasks
that
are
running,
so
we
have
something
like
an
add
operation
and
multiply
and
reductions
so
things
that
are
used
to
implement
the
the
loop.
The
main
loop
of
the
program-
and
it
looks
like
there
is
the
python
task,
which
is
waking
up.
There's
some
there's
some
gaps
in
the
timeline.
B
It
looks
sorry
there.
It
looks
like
there's
the
python
task
that
is
waking
up
and
the
gpu
is
pausing
and
then
the
gpu
is
resuming
and
there
seems
to
be
some
transfers,
and
this
is
between
the
same
memory.
So
this
is
a
this
is
a
an
intra
device
copy
and
this
is
about
the
size
of
373
megabytes.
That's
about
the
size
of
of
one
array,
so
so
better
better
than
before,
but
there
is
still
this.
B
B
This
is
a
common
thing
to
watch
out
for
when
building
on
top
of
legion
or
similar
synchronous
execution
frameworks,
the
the
framework
will
will
work
best
when
the
control
code
can
run
ahead
and
emit
work
ahead
of
the
actual
of
the
actual
gpu
or
other
processor.
That's
performing
the
computation,
and
that
is
because
that's
how
we
can
hide
the
the
latency
of
of
dynamic
runtime
analysis.
B
So
the
whole
program
has
to
wait
for
all
the
work
on
the
gpu
to
finish
before
it
can
continue
and
emit
more
work,
and
that
that
is
something
that
sometimes
you
can
you
can
change.
You
can
change
the
code
to
to
avoid
having
that
and
give
the
give
the
program
a
little
bit
of
a
little
bit
of
way
to
move
ahead
and
and
queue
up
a
lot
of
work
and
then
have
the
gpus
be
be
more
full
more
of
the
time.
B
So
that's
what
we're
going
to
do
here,
so
I'm
going
to
have
another
copy
of
this
program
where,
instead
of
checking
the
the
the
norm
every
time
step
instead,
I
am
only
going
to
compute
it
every
100
time
steps
and
then
block
on
it.
So
we
will
give
the
the
control
program
a
little
bit
more
room
to
to
go
ahead
and
emit
emit
useful
work,
and
this
you
know
this.
This
might
be
something
you
can
or
might
you
might
be
able
to
do
for
your
application?
B
It
might
converge
a
little
bit
slower.
You
know
a
few
more
iterations.
It
might
take
a
couple
more
iterations
for
it
to
to
converge
because
you're
not
checking
as
often
but
the
added
the
added
parallelism
may
well
make
up
for
it.
So
I
will
just
run
this
same
program
with
under
these
conditions,
and
let
me
just
go
up
a
little
bit
to
see
how
much
the
older
code
took.
B
This
example
was
inspired
by
a
a
question
on
the
on
the
lee
gate,
bug
tracker,
where
somebody
was
asking
why?
Why
is
this
being
so
slow
and
part
of
the
part
of
the
story
was
the
size
was
not
really
something
that
legate
thought
was
worth
bothering
with
and
part
of
it
was
the
the
waiting
on
the
main
loop
that
was
causing
it
was
causing
the
execution
to
stall.
B
Okay,
so
that
is
11
27
10..
So
let's
go
look
at
that
real,
quick
and
here
again
let
me
hide
some
irrelevant
stuff
and
open
up
these
and,
as
you
can
see,
you
can
see
the
different,
the
the
the
10
hundreds
of
iterations,
and
you
can
see
that
there
is
still
some
some
blocking
every
100
iteration,
but
at
from
that
point
forward,
there's
there
there's
the
the
copy
that
is
required
to
perform
the
the
the
norm
calculation.
B
But
then
you
can
see
that
from
this
point
forward
after
we've
checked
the
normal,
we
started
emitting
work
again,
even
though
the
the
python
program
wakes
up
once
in
a
while,
once
the
the
queue
becomes
full.
The
the
gpu
will
pick
up
work
the
once
the
queue
empties
out
a
little
bit.
The
python
program
will
be
resumed
and
it
will
add
a
little
bit
more
work
to
the
queue,
but
this
continues
with
minimal
blocking
on
the
on
the
gpu.
B
B
Not
too
much
so
here
the
profiler
is
really
running
on
the
cpu
processor,
so
it
shouldn't
it's.
It's
all
these
tiny
tasks
and
if
they're
running
on
a
separate
core
that
that
is
allocated
for
for
cpu
work,
but
in
this
case
just
just
just
the
profiling,
I
don't
have
concrete
numbers,
but
I
it
hasn't
been.
I
mean
it's,
not
it's
not
affecting
the
the
gpu
work
very
much.
B
C
B
So
that's,
okay,
so
so
legion
doesn't
make
these
decisions
for
you.
Legion
will,
so
that
the
underlying
layer
does
not
make
these
decisions
for
you.
It
expects
it.
It
provides
you
the
ability
to
run
on
on
anything.
You
need
and
will
make
sure
that
wherever
you
choose
to
run
will
give
you
the
same
result
but
or
but
the
decision
itself
is
not
made
for
you
it's
made
by
the
by
the
application.
In
this
case,
it's
made
by
legate
legate
has
some
heuristics.
B
B
So
all
of
those
decisions
are
made
by
by
the
part
of
the
application.
That's
called
the
mapper
and
the
mapper
can
make
these
decisions
based
on
the
specifics
of
the
machine,
the
so
in
ligate
we
have
some
heuristics
to
do
that
different
application
different.
When
we
move
to
different
libraries,
the
heuristics
may
need
to
change
for
the
for
the
specific
needs
of
those
libraries,
so
yeah.
B
There
is
a
so
the
mapper
interface
has
there's
capability
for
auto
tuning.
We
have.
There
is
a
project
in
the
in
the
broader
legion
ecosystem
called
the
auto
mapper,
which
does
tries
to
auto
tune
this
decision,
and
it's
it's
in
development
for
legate
in
particular.
We
would
like
to
come
up
with
something
that
requires
can
work
with.
No,
we
can
make
these
decisions
based
on
the
program
itself
and
not
not
can
work
without
any
input
from
the
user.
C
B
B
It
will
work.
There
is
a
dash
dash
and
ms
option
on
on
run
on
only
on
the
ligate
on
the
legate
script,
runner,
it
will
dump
out
the
inside
profiles.
You
can
load
those
or,
as
as
normal,
what
what
those
will
show.
You
are
a
similar
timeline,
but
like
in,
in
this
view,
we
essentially
see
these
blocks
what's
happening
within
every
one
of
these
blocks
is
that
a
kernel
is
being
launched
and
data
is
being
copied.
So
what
the
insight
systems
profiler
will
show.
B
B
B
That
is
exactly
the
size
of
a
of
the
boundary
between
two
between
the
the
two
two
halves
of
the
domain.
So
what
happened
here
is
legate
numpy
split
the
the
array
between
the
two,
the
two
gpus
that
are
gonna
run
and
on
every
iteration
it
needed
to
to
do
a
halo
exchange.
This
is
a
this
is
a
one
point
stencil.
So,
however,
liga
decided
to
do
the
the
the
split
either
a
length,
column,
wise
or
row
wise.
It's
going
to
be
the
same.
B
B
So
that's
all
for
demos
doing
okay
on
time.
Yes,
so
the
slides
will
also
have
the
same
pictures
if
you
want
to
review
those
later.
Here
is
the
the
quick
start
page.
If
you
want
to
try
it
there's
a
little
bit
of
a
build,
we'll
talk
about
that
in
a
second
here's,
some
more
some
larger
benchmarks,
we've
run.
This
is
a
this.
These
are
all
on
a
internal
nvidia
super
pod
with
a800
dgx's,
a100
dgxs.
This
is
a
conjugate
gradient
example
that
you
see
on
the
left.
B
This
is
a
code.
That's
similar
to
the
one
that
I
showed
on
the
demo.
This
is
taken
from
a
cfd
course
on
python
cfd
on
python.
Taken
we
take
it.
We
took
the
same
code
and
run
it
and
scaled
it
up.
Just
replacing
numpy
with
legate
numpy,
again
good
scaling
up
to
hundreds
of
thousands
of
gpus
and
similar
on
one
node
to
kupai
escaping
from
from
legate
numpy
a
little
bit
a
lot
of
exciting
results
on
pandas.
B
B
B
So
that's
it
that's
all
I
had
to
that's
all
I
had
to
show
you
on
only
gate.
I
showed
you.
I
gave
you
a
demo.
I
I
talked
a
little
bit
about
how
league
8
works
internally
and
then
showed
you
how
you
can
use
it
and
gave
you
a
show.
The
commands
that
you
can
use
yourself
to
run
on
encore
gpu
right
now
and
if,
if
we
get
access
to
to
perlmutter,
we'll
also
do
the
same
there
if
possible.
B
B
Please
report
those
if
you
have
an
existing
code
that
uses
one
of
our
libraries
that
we've
already
that
we
already
cover
we'd
love
to
hear
from
you,
especially
if
you
have
a
program
that
derives
its
its
scale
by
operating
on
large
arrays.
That's
how
that's
the
best
way
for
legate
to
extract
parallelism,
have
operations
with
large
on
large
collections.
B
B
If
you
just
want
to
tell
us,
you
know
this
is
this:
something
is
this
something
that
intrigues
you
or
is
this
something
that
is
there?
Something
we're
missing
in
how
real
scientists
are,
are
scaling
real
applications,
we'd
love
to
know
and
on
the
on
the
more
on
the
more
practical
side,
once
you
take
a
look
at
the
the
scripts
plea,
please
let
us
know
what
what
would
make
it
easier
for
you
to
try
it
out.