►
From YouTube: 12 Python at NERSC
Description
Part of the NERSC New User Training on June 16, 2020.
Please see https://www.nersc.gov/users/training/events/new-user-training-june-16-2020/ for the training day agenda and presentation slides.
A
Hi
everybody-
this
is
funny
okay,
so
I'm
here
to
tell
tell
you
about
python.
At
nursk
my
name
is
laurie
steffy,
I'm
in
the
data
analytics
services
group
and
if
you
have
any
trouble
with
python
and
submit
python
tickets.
They'll
come
to
me.
So
my
goal
here
is
to
help
you
use
python
as
well
as
you
can,
without
needing
my
help,
all
right.
So
for
most
of
you,
I
probably
don't
need
to
tell
you
about
python.
Many
of
you,
I
imagine,
are
probably
python
users.
A
If
we
were
in
a
real
life
setting,
I
would
ask
for
a
show
of
hands,
but
unfortunately
can't
do
that.
I
think
this
xkcd
comic
captures
what
I
like,
and
I
think
what
a
lot
of
people
really
like
about
python.
It's
clean.
It's
it's
easy
to
get
started
and
instead
of
focusing
on
array
syntax,
you
can
focus
on
your
science
problem.
So
for
that
reason,
python's
popular,
it's
growing,
I'm
gonna
talk
to
you
today
about
kind
of
standard
python.
A
I
won't
cover
any
of
the
machine
learning
parts
of
the
python
ecosystem,
but
mustafa
will
cover
that.
I
think
in
today's
last
talk
all
right.
So
can
you
use
python
at
nursk?
Yes,
although
this
is
sort
of
a
recent
phenomena,
maybe
in
about
ten
years
ago,
using
python
was
not
really
a
given
languages
like
c
and
fortran
were
dominant,
but
fortunately
times
have
changed
so
now,
I'm
sure
you've
heard
we
have
over
7000
users.
A
A
lot
of
these
users
use
python
for
at
least
some
part
of
their
work,
and
this
number
is
growing.
So
I
think
this
is
great.
We
are
here
to
help
you
with
python
art
nurse.
It's
a
first-class
citizen.
It's
fully
supported.
A
There
are
several
ways
that
we
could
provide
python,
but
our
current
preferred
way
is
anaconda
python
I'll
talk
more
about
the
details,
but
we
give
you
the
ability
to
either
use
our
pre-built
environment
or
to
build
your
own.
So
you
can
kind
of
choose
your
own
adventure
I'll.
Just
reiterate
that
we're
here
to
help
we're
very
friendly.
If
you
have
questions
or
run
into
any
issues
with
python
we're
here
to
help
you
get
done
what
you
need
to
get
done
and
my
last
point
is
to
get
started
using
python.
A
All
you
need
to
do
is
modulate
python,
so
I
hope
that's
straightforward
enough.
Okay,
so,
as
I
mentioned,
there
are
kind
of
two
major
ways
we
encourage
people
to
use
python
at
nursk.
A
There
is
a
pre-built
python
where
we
have
done
the
work
for
you
to
assemble
this
environment.
So
this
is
like
our
prefabricated
house
here
and
that's
great,
if
you're
just
trying
to
do
something
straightforward,
if
you're
not
trying
to
use
any
custom
library,
so
you
can
be
off
doing
what
you
need
to
do
if
you
need
to
install
something
that
your
domain
uses
like
a
custom
package.
Just
anything
that's
not
in
the
base
environment.
A
Our
solution
is
that
you
build
your
own
custom,
con
environment.
So
that's
that's
the
house
under
construction
here.
So
that's
easy
to
do.
These
are
about
the
five
lines
of
code
you
need.
If
you
would
like
to
build
your
own
environment,
of
course,
don't
forget
module
and
python
if
you
don't
you're
going
to
end
up
with
user
bin
python,
which
is
kind
of
old
crusty
python,
it's
not
not
really
up
to
date
doesn't
really
have
good
packages,
so
always
always
monitor,
load
python.
A
You
can
create
whatever
environment
you
like
once,
you
are
inside
your
environment,
which
is
what
source
activate
does
for
you.
You
are
the
king
or
queen
of
your
environment.
You
can
conda
or
pip
install
anything
you
like
so
do
do
what
you
like
import
anti-gravity
when
you're
done
you
can
deactivate.
A
So
that's
it!
I
I
really
like
this.
I
do
this
all
the
time.
It's
easy
to
convert
one
of
these
custom
environments
into
a
jupiter
kernel,
if
you
like
jupiter,
and
if
something
goes
wrong,
no
problem
just
delete
your
environment
and
start
again,
because
it's
so
easy
to
make
your
environment,
it's
probably
quicker
to
build
a
new
one
than
to
try
to
troubleshoot
what
went
wrong
in
your
old
environment.
So
that's
all
there
is
to
it.
A
Okay,
so
since
you're
here
at
nursk,
you're,
probably
interested
in
how
you
can
run
your
python
at
some
scale,
bigger
than
your
laptop,
that
is
a
complex
question,
but
I
just
want
to
give
you
an
overview
of
the
landscape.
A
A
A
I
think
that's
my
favorite
part
about
desk
yeah.
If
you
have
more
information
or
if
you,
if
you'd
like
to
know
more
about
desk,
submit
a
ticket
and
we
can
help
you,
okay,
the
other
option
for
scaling
your
code
is
the
tried
and
true
mpi
mpi
for
pi
is
kind
of
a
nice
python
interface
to
mpi,
and
it
lets
you
run
as
big
as
the
whole
machine.
A
If
that's
what
you
need
to
do,
with
the
major
caveat
that,
if
you're
using
mpi
for
pi,
you
should
also
very
seriously
consider
using
a
container
I'll
talk
a
little
bit
more
about
this,
but
at
really
large
scale
your
performance
will
get
slow
because
of
file
system
traffic,
so
mpi
should
should
be
kind
of
synonymous
in
your
head
with
docker
and
shifter.
So
check
check
out
our
docs
for
for
more
information
on
that.
A
Okay,
so
part
of
my
job
here
today
is
to
help
you
get
running
on
corey
with
python
and
help
you
avoid
common
problems
life's
better
for
all
of
us.
If
you
don't
have
problems,
so
I've
been
answering
python
tickets
for
about
a
year,
I'm
still
relatively
new,
but
I've
sort
of
seen
some
general
patterns
emerge
as
a
result.
So
I
want
to
help
you,
the
user,
avoid
these.
The
first
one
is
trying
to
install
things
into
our
base
environment.
A
The
second
major
pain
point
I
see
is
related
to
mpi
for
pi,
and
the
third
major
problem
is
configuration
settings
in
your
dot
files.
So
I
will
talk
now
about
how
I
can
help
you
avoid
these
okay.
So
the
first
problem
I
said
was
that
people
try
to
make
changes
to
our
default
environment.
If
you
remember,
our
default
is
like
this
prefab
house
and
unfortunately
you
can't
change
it.
The
reason
is
because
we
all
share
this
same
module,
so
we
don't
want
7,
000
people
making
changes
to
it.
A
So
if,
if
you
want
to
do
something
or
install
something,
that's
not
there,
no
problem,
you
just
need
a
custom
environment.
If
you
do
forget
and
try
to
install
you'll,
see
things
like
permission
denied
and
then
that
should
ring
a
bell
on
your
head.
Oh
right,
I
can't
install
into
the
default,
so
that's
it's
an
easy
solution
to
that
problem.
A
Okay,
other
problems
we
see
commonly
are
people
installing
mpi
for
pi.
So
in
the
in
the
case
that
you're
using
our
prefab
environment,
just
the
module
load
python,
we
have
already
provided
mpi
for
pi.
So
it's
there,
you
can
use
it.
You
don't
have
to
worry
about
installing
it.
The
confusion
comes
when
people
are
using
a
custom
environment
and
then
they
they
find
that
they
need
to
install
mpi
for
pi.
A
So
if
you
remember
anything
from
this
talk,
please
remember:
do
not
install
mpi
for
pi,
just
pip
install
or
conda
install
it
that
works
on
your
laptop.
That
does
not
work
on
our
very
complicated
supercomputer.
What
you
need
to
do
are
to
follow
our
directions.
It's
maybe
just
five
lines
of
code
to
build
mpi
for
pi
against
our
and
pitch
mpi
libraries.
A
So
if
you
just
do
that
and
just
follow
our
directions,
you
should
have
working
mpi
for
pi
in
your
content
environment,
just
one
last
gotcha
for
mpi
for
pi,
you
will
not
be
able
to
use
it
on
our
login
nodes.
A
So
if
you're,
finding
errors
double
check,
make
sure
you're
on
a
compute
node.
A
A
So
here's
what's
happening
every
library
that
you're
importing
numpy
sci-fi
extra
pi,
whatever
your
favorite
library,
is,
has
to
be
transferred
to
every
node,
and
that
means
that
there's
a
lot
of
data
moving
across
our
file
systems
to
each
of
your
compute
nodes,
so
that
can
be
slow
and
now
imagine
that
there
are
eight
other
people
doing
that
simultaneously.
A
So
you
can
see
how
that
can
become
a
bottleneck.
The
solution
is
to
instead
have
your
code
in
a
container.
So
that
way,
the
libraries
that
you
need
are
there
and
you're
not
moving
things
across
the
file
system.
So
this
will
make
your
startup
fast
and
will
also
make
your
code
resilient
to
anything
that
may
or
may
not
be
happening
on
our
file
system.
So
yeah
definitely
want
to
consider
containers.
A
Okay,
the
last
major
family
of
problems
that
python
themes
that
come
up
in
python
tickets
are
are
really
not
even
from
python
itself,
they're
from
how
you've
configured
your
user
environment,
so
some
of
you
may
be
familiar
with
dot
files.
These
are
like
your
bash
rc
bash
profile.
A
These
files
are
cool
because
they
let
you
customize
your
setup.
They
every
time
you
get
every
time
you
have
a
fresh
log
on
to
corey.
They
will
kind
of
set
things
up
for
you
in
an
invisible
way.
This
is
good
and
bad,
because
it's
easy
to
put
things
in
these
files
forget
about
them
and
then
five
years
later,
some
some
setting
has
changed,
and
now
it's
causing
problems
for
you.
So
when
people
submit
python
tickets
and
say
you
know,
your
content
tool
is
broken
almost
always.
A
A
Okay,
so
imagine
you've
run
into
some
python
problems.
I
log
into
your
environment
to
see
what
you've
got
going
on
and
then
I
see
this
in
your
back
rc.
So
this
is
a
bad
example,
and
I
show
this
to
you
to
help
you
avoid
this
situation.
So
if
the
user
has
a
problem
with
their
python
environment,
it
becomes
really
really
tricky
to
pinpoint
where
the
problem
is
coming
from.
A
A
A
Okay.
Another
question
you
might
have
about
python
is:
should
you
use
python,
2
or
3,
and
the
answer
is
3
python,
2,
officially
retired,
at
the
beginning
of
this
year,
developers
are
no
longer
supporting
python
2..
You
will
see
on
corey
that
we
do
have
some
python
2
environments,
but
we
don't
promise
to
keep
them
forever
and
we
are
not
going
to
put
them
on
perlmutter.
A
So
if
you
have
python
2
code,
now
is
the
time
to
transition
it
to
python
3
and,
if
you're,
using
someone
else's
python
2
code.
If
you're
not
the
developer,
make
sure
that
they
have
a
plan
to
convert
their
code
to
python
3.,
so
yeah,
it's
worth
reminding
them.
That's
an
important
thing
to
do.
A
A
It's
a
good
question,
so
I
guess,
as
python
users
we've
become
a
little
bit
spoiled
in
having
a
nice
import
statements
at
the
top
of
our
code,
that
let
python
do
magic
things
and
unfortunately,
there's
no
there's
nothing
quite
like
that
for
gpus
at
the
moment.
What
I'm
about
to
tell
you
may
be
different
in
six
months:
it's
it
seems
to
be
changing
pretty
quickly,
but
I'll
give
you
the
best
information
that
I
have
at
the
moment.
A
A
In
one
of
these,
you
can
write
your
own
gpu
kernel,
which
I
understand
that
for
python
users
they
may
not
want
to
do
that.
They
may
not
have
the
expertise
to
do
that
or
just
may
not
have
time.
A
So
we
understand
there's
some
balance,
but
in
order
to
fully
move
your
code
to
a
gpu,
you
may
need
one
or
both
of
these
kinds
of
frameworks.
Depending
on
what
you
need.
I
won't.
I
won't
talk
in
a
lot
of
detail,
but
if
you're
a
pandas
or
a
psychic
psychic
learn
user
nvidia
has
a
framework
called
rapids
where
they've
basically
implemented
it
for
gpu.
So
you
can
check
that
out
for
more
information.
A
Okay,
so
I'll
talk
just
really
briefly
about
the
two
kinds
of
well,
the
two
kinds
of
drop-in
replacements
for
numpy
and
sci-fi,
probably
the
easiest
and
most
sophisticated
one
is
koopa.
Kupai
looks
a
lot
like
numpy,
except
it's
kuda
under
the
hood.
I
don't
know
why
people
are
chatting.
A
Sorry
me
guess
you
guys
can
see
that
okay
anyway
yeah,
so
it's
cuda,
which
is
good
it's.
It
will
get
your
python
running
on
a
gpu,
but
it's
maybe
not
good,
because
it
will
only
run
on
nvidia
system.
So
if
that's
a
concern
for
you,
if
you're
worried
about
running
on
different
kinds
of
systems,
that's
an
important
caveat.
A
A
Okay,
the
other
kinds
of
python
gpu
options
are
kernel,
custom
kernel,
so
you
can
use
number,
which
is
a
jet
compiler.
This
will
allow
you
to
write
sort
of
a
pseudo
python,
cuda
mix
which
will
result
in
a
gpu
kernel
or
if
you
really
need
something
more
powerful
than
that,
you
can
use
pi,
cuda
or
pi
opencl
pi
cuda
is
basically
an
actual
cuda
kernel
wrapped
in
python
and
pi
opencl
is
the
same
for
opencl.
A
So
if
you
have
gpu
expertise
in
cuda
or
opencl,
this
is
how
you
can
take
advantage
of
those
okay,
so
just
to
wrap
up
welcome
to
nursk.
We
work
really
hard
on
our
python
documentation,
so
we
hope
that
you'll
read
it.
We
hope
that
it
will
save
you
some
time
and
difficulty
if
you'd
like
to
make
contributions
to
the
docs.
We're
happy
for
that
too.
A
To
avoid
common
problems
in
python,
don't
try
to
install
into
our
base
environment,
make
your
own
be
careful
using
mpi
for
pi
and
don't
put
too
many
customizations
in
your
dot
files.
That's
all
I
have
so.
Thank
you
very
much.
B
Okay,
hi
laurie.
So
there
is
one
question
in
the
google
doc
that
I'll
read
you,
which
is
that's
not
related
to
r
or
r
studio,
which
is
how.
A
I
don't
know
I'm
actually
not
familiar
with
pycharm
for
ides.
I
use
spyder
myself
and
I
usually
use
spyder
on
my
local
laptop
yeah.
I
don't
know
that
we
have
a
great
way
to
have
an
ide,
but
we
can
look
into
that
and
get
you
more
information.
A
Oh
yeah,
we
do
so
that's
going
to
be
covered
in
our
last
talk
today.
A
B
Okay,
I
don't,
I
don't
see
any
other
python
questions
in
the
document,
at
least
under
the
python
nurse
section.
But
if
there
are
questions,
of
course,
you
can
get
in
touch
with
lori
or
file
a
ticket
at.