►
From YouTube: Using Python and Jupyter on Perlmutter
Description
Part of the Using Perlmutter Training, Jan 5-7, 2022. Slides and more details are available at https://www.nersc.gov/users/training/events/using-perlmutter-training-jan2022/
A
Hi
everybody
thanks
for
being
here
in
this
short
30
minutes.
We're
gonna
try
to
cram
in
a
lot
of
information.
So
please
bear
with
us.
We'll
have
three
presenters
I'll
be
the
first.
My
name
is
laurie
steffy,
I'm
a
scientific
data,
architect
and
you'll
also
be
hearing
from
daniel
marcol
and
roland
thomas.
A
Okay,
so
what
we're
going
to
try
to
cover
here
is
general
python
advice.
A
lot
of
this
will
apply
to
both
cory
and
promoter.
We're
going
to
talk
about
using
python
on
gpus,
I'm
sure
a
topic.
Many
of
you
are
curious
about
we'll
talk
about
using
jupiter
on
promoter
and
we're
going
to
try
really
hard
to
leave
about
five
minutes
at
the
end
for
an
open
q
a
so.
This
would
be
a
good
opportunity
for
you
to
ask
us
questions
all
right.
So,
let's
get
started,
I
assume
most
of
you
have
used
corey.
A
So
there
are
a
few
differences
and
I'll
try
to
highlight
the
most
important
things
here.
One
thing
hopefully
you'll
notice
and
like
is
that
we'll
have
a
more
uniform
python
and
jupiter
environment,
so
the
python
jupiter
3
kernel
will
be
based
on
the
current
default
python
module.
So
right
now
that
it's
a
little
bit
different,
but
this
should
make
it
easier
to
keep
track
of
your
packages.
A
A
That
said,
many
of
the
kind
of
best
practices
we've
been
advertising
on
quarry
are
still
in
effect,
so
for
good
performance.
You
can
install
your
software
stack
on
our
global
common
file
system.
This
is
a
project-based
file
system,
so
you
don't
have
your
own
directory,
but
you
can
install
in
your
projects
directory.
A
If
you
run
out
of
space
and
a
ticket,
we
can
try
to
get
you
more.
Another
best
practice
is
to
use
a
custom
combat
environment
for
any
kind
of
software
you
need
that
is
not
in
the
module
we
provide.
This
is
a
nice
sandbox,
it's
easy
to
get
rid
of
if
something
goes
wrong
and
it's
also
very
easy
to
use
in
jupiter
via
the
ipython
kernel
conversion
and
finally,
of
course
we
are
still
encouraging
shifters,
especially
at
large
scale,
and
this
is
also
easily
used
in
jupiter
as
a
kernel.
A
A
What
we
tell
you
can
help
avoid
some
of
these
issues.
So,
first,
these
systems
are
sharing
file
systems,
and
that
means
a
lot
of
system.
Customizations
like
in
your
dot
files,
for
example,
will
be
shared.
You
most
likely
don't
want
the
same
kinds
of
software
packages
and
environments
on
these
systems
so
be
careful.
First
of
all,
try
not
to
put
too
much
in
your
dot
file,
and
if
you
do,
you
need
to
periodically
review
it
make
sure
you
still
want
it.
A
A
One
tip
is
to
append
the
system
name
to
your
environment,
so
you
can
keep
track,
but
of
course
you
can
do
this.
However,
you
like
and
pip
can
be
a
little
more
dangerous
now
with
the
shared
system,
so
we'll
cover
that
okay,
so
one
big
change
on
perimeter
now
and
actually
coming
shortly
to
promoter
is
that
you'll
finally
be
able
to
use
the
conda
activate
command.
Some
of
you
may
be
familiar
with
our
current
setup,
which
requires
either
source
activate,
which
is
here
I
don't
know
if
you
can
see
my
mouse.
A
So
this
is
the
current
setup
or
using
a
conda
init
conda
activate,
which
is
not
so
good
because
it
adds
customization
to
your
battery
c.
So
now,.
B
A
New
setup,
which
is
on
promoter
today,
you
can
just
load
the
python
module
and
conda
activate
and
deactivate
with
no
changes
to
your
setup.
So
that's
great!
Please
try
it
out.
If
you
want
to
know
more,
we
have
some
pending
updates
to
the
docs
that
you
can
check
out
all
right
so
for
parallels.
A
Achieving
parallelism,
mpi
for
pi
is
a
very
common
library
and
so
a
lot
of
our
user
questions
have
been
around.
How
do
I
build
and
use
mpi
for
pi,
so
we
provide
it
in
the
python
module,
so
it's
already
there.
You
can
load
it
and
test
it
out,
but
if
you're
going
to
build
your
own,
like-
I
think
many
of
you
will.
This
is
the
recipe
that
we
have
iterated
on
and
settled
upon
for
now.
A
So
we
recommend
the
gnu
programming
environment
because
at
least
last
I
tried
we
can't
build
mpi
for
privacy
and
video
compilers
you'll
need
kudo
tool,
kit
and
python,
of
course,
build
your
custom
environment.
You'll
need
this
flag
here
so
that
you
can
access
the
gpu
and
finally,
there's
some
other
stuff
in
here.
We'll
talk
about
on
the
next
slide,
so
note
that
when
you've
built
mpi
for
pi
with
cuda
support,
it
will
ask
for
the
crew
libraries
at
runtime.
A
B
C
A
That's
how
it
works,
okay
and
finally
be
careful
with
pip
pip
will
try
to
be
clever
and
help
you
and
it
will
search
our
shared
file
systems
for
packages
and
say:
oh
hey,
like
I
found
an
mpi
reply.
Let's
use
that,
but
if
it's
finding
one
that
you
built
on
corey,
you
won't
want
that
because
it
doesn't
work
so
be
careful.
A
A
Also,
when
you're
pip
installing
use
the
force
flag,
this
will
force
a
rebuild,
so
it
will
prevent
them.
It
will
prevent
pip
from
using
packages
on
the
wrong
system.
This
appeared
in
our
mpi
for
pi
recipe.
Okay.
Finally,
if
you
do
use
pip
user-
and
I
know
some
of
you
do-
it
will
install
to
places
specified
by
the
python
user
base
environment
variable
that's
here,
so
this
is
in
your
search
path,
as
defined
in
the
module,
which
can
be
a
little
bit
tricky
depending
on
how
you've
set
up
your
environment.
A
C
Hi
thanks
lori
yeah,
so
today
I
hope
to
just
give
in
a
couple
of
minutes
a
few
I'll
present,
a
few
slides
on
how
to
get
started
with
gpus
and
python
on
on
the
nerf
systems,
particular
perlmutter.
C
C
But
there
are
many
python
gpu
frameworks
out
there,
some
of
the
very
popular
ones,
kind
of
serve
as
drop-in
replacements
to
those
core
backbone.
Libraries
like
numpy
and
scipy,
as
well
as
pandas
and
scikit-learn.
So
those
those
popular
gpu
frameworks
are
are
coupei
and
there's
also
a
number
of
them
in
the
the
rapids
library
ecosystem.
C
There's
also
coup
numeric,
which
is
not
currently
available
or
working
on
on
promoter.
But
it's
something
that
aims
to
be
a
drop-in
replacement
for
for
numpy,
but
also
scale
across
many
many
gpus
there's
also
the
popular
machine
learning
libraries
such
as
pytorch
and
tensorflow
that
also
support
a
lot
of
general
gpu
computing
features
such
as
working
with
array,
data
structures
and
performing
linear
algebra
operations
on
those.
So
if.
C
C
And
one
nice
thing
about
some
of
these
libraries
in
the
ecosystem
in
general.
Well,
I
guess
it's
there's
kind
of
two
sides
to
it.
There's
all
of
these
libraries
are
kind
of
you
know,
they're
they're,
trying
to
accomplish
different
things
and
they've
implemented
things
in
different
ways,
and
so
some
of
them
are
better
at
certain
things
than
others,
and
so
there's
a
lot
of
trade-offs
that
you
have
to
consider
when
choosing
a
framework.
C
There
is
some
effort
in
various
places
to
try
to
coordinate
and
kind
of
share
a
common
data
structure
format.
So
you
can
pass
these
array
structures
that
are
created
in
these
different
libraries
between
each
other
and
then
there's
also
some
effort
in
the
community
community
to
kind
of
standardize
what
it
means
to
have
an
array
api.
So
on
the
right
here,
there's
a
little
screenshot
of
the
api
for
the.
A
C
Average
function
from
a
bunch
of
these
libraries
and
they
all
have
a
slightly
different
interface.
So
that
could
be
a
little
bit
of
a
headache
when
trying
to
mix
and
match
these
various
libraries.
But
the
good
news
is
that
there's
there's
some
effort
to
to
coordinate
and
improve
that
going
forward.
C
So
getting
started
on
pearl
butter,
as
lori
mentioned,
you'll
want
to
load,
use
the
the
toolkit
and
python
modules,
and
you
can
you
start
by
loading
those
modules
and
you
have
to
pay
attention
to
the
version
that's
brought
in
from
the
kudo
toolkit
module.
So
in
this
case
here
I'm
highlighting
that
the
version
of
cuda
is
11.4
currently
by
default.
C
So
that's
important.
For
example,
if
you
wanted
to
use
coupey
when
you
install
coupey,
you
have
to
make
sure
you
specify
the
version
that
corresponds
to
cuda
and
then
it's
just
as
simple
as
that,
as
after
installing
coupe,
I
you
can
kind
of
import
coupe
I
as
cp,
just
as
you
would
import
numpy
as
np,
and
you
can
create
an
array
on
the
gpu
device
and
print
that
out.
So
there's
just
a
few
lines
of
code
and
you're
already
doing
gpu
computing
in
python.
C
C
C
And
I
don't
think
we
have
time
to
really
go
into
kind
of
this
more
advanced
gpu
programming,
but
you
you
can
do
things
this.
This
slide
here
is
meant
to
kind
of
show
an
example
of
writing
your
own
kind
of
lower
level
kernels
using
using
the
number
just
in
time,
compilation
to
write
a
kernel
and
then
also
how
you
can
use,
mix
and
match
these
kind
of
number
and
coupon,
as
well
as
using
kind
of
this
generalized
array
api
to
operate
on
that
that
data
as
well.
C
Another
thing
that's
very
useful,
as
you're
kind
of
starting
off
and
trying
to
understand
what
your
what
your
gpu
code
is
actually
doing
is
to
visualize
is
to
to
run
a
profiler
and
visualize.
What's
what's
actually
happening
running
on
the
gpu
versus
cpu,
so
you
can
use
the
nvidia
insight
systems,
profiling,
tool
to
profile
your
application,
and
you
can
use
the
kind
of
standard,
nvtx
markers
and
ranges
to
decorate
your
code
and
add
your
own
labels
that
you
would
be
able
to
visualize
in
the
profile.
C
C
And
finally,
you're
probably
wondering
if
you're
just
getting
started
is
my
code,
a
good
fit
for
a
gpu.
So
there's
a
lot
of
discussion
about
this
in
the
presentations
earlier
today,
and
so
just
reiterating.
Some
of
those
points
of
cpus
are
good
at
doing
things
very
quickly
through
kind
of
low
latency
and
gpus
are
better
at
high
throughputs.
C
A
B
Hi,
I'm
rollin
thomas
I'm
from
the
data
science
engagement
group
at
nurse
now,
and
I'm
going
to
talk
to
you
a
little
bit
about
jupiter
on
perlmutter
what
you
need
to
know.
Moving
from
corey
to
perlmutter,
I've
got
about
seven
slides
to
answer
kind
of
three
high
level
questions
that
I
think
everybody
will
be
on
people's
minds.
These
are
basically
how
do
I
make
sure
that
I
can
run
notebooks
on
promoter
phase
one.
B
How
do
I
actually
make
sure
that
I'm
using
the
gpus
from
jupiter
on
promoter
and
is
there
anything
special
that
you
need
to
keep
in
mind
when
moving
workflows
from
cory
to
promoter
using
jupiter
all
right,
so
the
first
thing
you'll
need
to
do
is
just
make
sure
that
you
can
access
promutter
with
jupiter
by
logging
in
at
jupiter.nurse.gov,
and
if
you
could
advance
to
the
animation
just
real
quickly
what
you
ought
to
see
when
you
log
in
and
go
to
the
home
or
the
console
page.
B
Is
this
extra
new
row
above
the
quarry
set
of
buttons?
There
should
be
probably
three
buttons
there
for
you
if
that
row
doesn't
show
up
for
you,
then
what's
needed
is
for
you
to
be
to
have
promoter
added
to
your
list
of
server
logins
in
iris.
If
you
can't
figure
out
how
to
get
that
done,
file
a
you
can
file
a
ticket.
You
everybody
here
in
the
training,
probably
should
be
able
to
see
this
line
of
buttons.
A
B
All
right,
what
do
the
different
buttons
actually
do
when
you
click
them?
The
far
left
button
labeled
shared
shared
cpu
node
on
promoter.
What
that
does
is
that's
the
equivalent
of
running
on
the
query
shared
cpu
node,
it's
just
that
it's
on
promoter,
so
these
are
shared
login
nodes.
They
are
not
chart.
Your
usage
is
not
charged
and
will
not
be
charged
in
the
future.
There
are
no
limits
currently
on
cpu,
gpu
or
memory
usage
right
now
and
like
on
quarry.
B
You
can
use
this
configuration
mainly
for
debugging
testing
and
developing
jupyter
notebooks
things
that
are
not
supposed
to
be
kind
of
compute
intensive
or
really
long
running
things.
That's
really
more
for
jobs,
the
middle
button,
exclusive
gpu
note
what
that
does.
Is
that
gets
you
a
a
notebook
running
on
a
promoter
phase,
one
compute
node,
all
to
yourself.
B
When
we
start
charging,
the
account
that
will
be
charged
will
be
some
sensible
default.
That
might
be
your
default
gpu
account.
If
we
have
something
like
that,
there's
a
six
hour
time
limit
there,
so
you
can
run
a
notebook
for
up
to
up
to
six
hours,
which
is
the
longest.
You
can
run
a
job
right
now
on
promoter
anyway.
B
This
is
good
for
interactive
gpu
work
where
you're
using
the
gpus
a
lot
and
things
that
maybe
need
to
run
a
little
bit
longer,
so
things
that
are
maybe
kind
of
more
compute
intensive,
some
things
that
will
not
fit
in
the
resources
you
get
on
the
shared
node
and
then
finally,
the
button
all
the
way
to
the
right
pops
up
a
menu
that
allows
you
to
configure
the
parameters
of
the
job
that
your
notebook
is
going
to
be
running
in,
and
this
is
a
way
for
you
to
be
able
to
use
more
than
one
node.
B
You
can
use
up
to
four
nodes,
so
16
gpus
here
you
can
customize
how
your
slurm
allocation
works.
If
you
don't
want
to
use
your
default,
your
default
account
you
can
switch
to
another
account,
use
a
reservation
or
whatever.
B
What
you
want
to
use
this,
for
is
when
you
need
to
scale
up
to
using
multiple
nodes.
If
four
nodes
turns
out
to
not
be
enough
for
your
needs,
please
contact
me
I'd
love
to
hear
about
your
needs
and,
and
we
can
work
on
on
getting
you
the
resources
you
want
to
use
next
slide
again.
How
do
you
actually
make
use
of
the
gpus?
What
do
you
actually
have
on
the
login
nodes?
I
think
it
was
mentioned.
Probably
earlier
in
the
training.
B
There
is
one
kind
of
shared
a100
on
the
login
nodes
right
now,
it's
kind
of
a
free-for-all,
so
if
you're
using
it
you're
the
person
using
it,
if
somebody
else
is
using
it
you're
not
going
to
be
able
to
use
it
in
the
future,
we
might
have
multi-instance
gpu
set
up
on
on
the
login
nodes
so
that
more
people
could
use
the
gpu
actually
interleave
work
on
the
gpu
more
cleanly,
the
exclusive
login,
the
exclusive
gpu
node
gets
you
for
a100s
and
then,
of
course,
on
the
configurable
option.
B
You
can
have
up
to
16
a100s
next
slide,
and
how
do
you
make
sure
that
you're
actually
using
the
gpus
on
the
node
that
you're
actually
on
so
you've,
probably
seen
nvidia
smi?
You
can
do
that
from
the
notebook
you
can
do
that
from
the
terminal
tab
in
jupiter
lab.
B
There
is
a
lab
extension
that
we
have
installed.
There's
this
kind
of
funny
icon.
I
think
it's
a
gpu
card.
It's
supposed
to
depict
a
gpu
card,
it's
the
third
icon
down
on
the
side,
on
the
left
hand,
side
of
the
jupiter
lab
user
interface
there,
if
you
click
that
what
you'll
see
are
a
few
different
dashboards
that
you
can
open
up
for
gpu
utilization
memory,
consumption,
pcie,
throughput
and
things
like
that.
This
lets
you
monitor
gpu
usage
on
the
node
that
you're
on.
B
B
If
you're
a
user
of
python
who
likes
to
use
the
das
framework
for
distributed
parallelism,
we
it
is
possible
to
use
the
das
dashboard
to
monitor
everything.
That's
happening
in
your
cluster,
observe
progress
profile,
your
workflow,
which
is
really
what
it's
for
right
now.
The
way
that
you
need
to
do.
That
is,
you
need
to
open
up
a
separate
browser
tab
and
then
copy
the
url
into
the
url
search
bar.
For
that,
however,
there
is
another
lab
extension
that
we
used
to
have
installed
a
long
time
ago,
but
was
kind
of
problematic.
B
B
Maybe
the
next
deployment
of
the
jupiter
lab
stack
that
we
do
and
that
will
provide
a
similar
kind
of
interface
to
what
you
can
get
from
mv
dashboard,
which
is
what
I
showed
on
the
previous
slide
except
it'll.
Let
you
see
all
of
the
activity
happening
across
all
the
nodes
you're
using
in
your
das
cluster.
B
Okay,
this
is
my
last
slide.
What
are
the
high
level
things
you
should
know
moving.
You
know,
as
a
jupiter
user
from
corey
to
perlmutter.
B
One
big
difference
is
that
on
the
shared
node
notebook
configuration
on
corey,
there's
only
four
nodes
dedicated
to
jupiter
use
that
are
repurposed
login
nodes,
basically
out
of
all
24
login
nodes,
we
took
four
and
set
those
aside
for
jupiter
use
and
so
about.
250
users,
usually
on
a
daily
basis,
are
kind
of
crammed
into
those
four
nodes,
and
so
there's
cpu
and
memory
limits
in
place
to
keep
those
nodes
healthy.
B
On
the
promoter
side,
there
are
many
more
login
nodes,
there
are
40
login
nodes
and
we
don't
restrict
jupyter
notebooks
to
any
particular
subset
of
those
nodes
right
now.
So
that
means
that
jupyter
notebooks
run
alongside
ssh
based
login
sessions.
We
don't
have
any
resource
limits
in
place
right
now,
so
all
kinds
of
fun
things
can
happen
there
on
the
exclusive
and
configurable
notebook
side
of
things.
B
On
the
exclusive
note
notebooks
on
the
cpu
side
on
corey,
if
you've
used
that
you
might
have
noticed
it's
a
little
bit
flaky,
the
number
of
resources
that
are
available
are
is
actually
pretty
small
and
you
need
to
ask
to
be
let
into
a
special
qos
to
access
those
on
promoter.
Jupiter
jobs
are
much
more
first
class
when
you
click
the
exclusive
node
button.
You
really
shouldn't
wait
too
long
to
get
a
notebook
that
starts
up.
B
We
have
no
plans
for
for
requiring
people
to
have
access
to
a
special
qos
for
that
we're
keeping
an
eye
on
the
allocation
success
rate.
So
when
you
request
a
jupiter
notebook
from
the
hub
in
on
a
compute
node,
are
you
actually
going
to
be
able
to
get
that
quickly
or
do
you
have
issues
with
that?
We're
keeping
an
eye
on
that
and
we'll
make
adjustments
to
policy
to
try
to
keep
that
as
responsive
as
possible?
B
Last
point
I
want
to
make
here
is
to
touch
on
setting
up
your
your
python.
Sorry,
your
jupiter
kernels,
of
course
on
corey
all
the
documentation.
That's
there!
That's
on
the
jupiter,
sorry
on
the
jupiter
page
on
docs.nurseco.
All
of
that
basically
still
applies
to
promoter
that
applied
previously
to
cory.
One
thing
that
you'll,
probably
notice
is
that,
as
you
put
in
things
like
the
nvidia
libraries,
sorry
about
the
background,
my
my
daughter's
at
home
doing
school
today-
I
don't
know
if
you
can
hear
her.
Your
content.
B
Environments
will
pick
up
a
lot
more
kind
of
bigger
libraries
and
you,
you
may
need
to
avail
yourself
of
using
the
google
common,
the
global
common
software
file
system
right.
That's
it
for
me,.
A
Okay,
thanks
so
I'll,
just
quickly,
wrap
up
and
try
to
point
you
all
in
a
helpful
direction.
So
we
have
a
lot
of
documentation.
We've
been
updating
our
docs
almost
daily
now,
since
we've
installed
promoter,
so
we
hope
you
find
them
helpful.
Some
of
the
pages
you
might
check
out
are
just
the
general
promoter
page.
We
have
a
whole
page
about
python,
so
this
talks
about
khan
environments,
condo
channels.
We
have
python
on
promoter
where
we
point
out
system
specific
things.
A
You
should
know
about
most
of
what
we
just
talked
about
here,
the
jupiter
page
rolling
just
mentioned,
and
we
have
a
pretty
handy
search
bar
so
check
it
out.
You
can
type
in
mpi
for
pi
and
find
what
you
need
pretty
quickly
and
if
you
can't
find
what
you
need
in
your
docs,
please
submit
a
ticket
we're
very
friendly
and
we
we
want
to
help
you
and
make
sure
you
can
get
your
work
done.
A
So,
in
summary,
welcome.
We
are
here
to
help
and
make
sure
you
can
get
make
productive
use
of
this
brand
new,
exciting
system
check
out
our
docs
file
ticket,
and
I
think
we
left
a
few
minutes
for
questions
so
we're
all
here.
If
you
have
questions
about
gpus,
jupiter
or
just
general
python,.