►
From YouTube: New User Training: 14 Python and Jupyter
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Okay,
so
I
guess
we're
a
couple
minutes
ahead,
but
what
I'll
do
is
I'll
I'll
use
that
time
for
a
live
demo
thingy
or
a
live
coding
thing
for
the
Jupiter
part.
So
this
talk
is
about
using
Python
and
Jupiter
here
nurse
and
before
I.
Give
this
talk
I
usually
like
to
see
if
there
are
Python
users
in
the
audience.
So
could
you
like
raise
your
hands
if
you
use
Python,
not
even
at
nurse
just
like
in
your
daily
life?
You
know
like.
A
So,
if
you
see
anything
up
here
like
that,
you
use
regularly
like
I.
Don't
want
to
go
through
all
of
these,
but
like
are
there
a
lot
of
you
know:
numpy
sci-fi
type
of
users
in
the
room,
so
I'm,
seeing
some
nodding
heads
of
pandas
people,
maybe
not
as
much,
maybe
how
about
scikit-learn
tensorflow
of
the
machine
learning
kind
of
toolkit
not.
B
A
A
A
lot
of
this
is
automated,
so
a
lot
of
this
is
actually
just
people
logging
in
and
they
have
module
load
Python
in
their
bash
RC
or
whatever,
but
people
are
using
it
for
production
jobs
for
some
lots
of
smaller
jobs,
but
there
are
a
few
larger
scale,
jobs
that
are
using
Python.
An
HPC
I.
Think
it's
interesting
to
note
that
I
have
a
friend
that
this
was
before.
I
worked
at
nurse
wrote
into
the
consultants,
maybe
about
10
years
ago.
A
A
You
need
to
know
about
how
you
use
Python
on
the
Cray
systems
and
and
how
we
deliver
Python
to
users,
there's
just
a
few
things
that
you
should
probably
know
that
will
probably
save
you
time
down
the
line,
and
one
of
them
is
knowing
that
we
do
have
documentation,
there's
kind
of
six
or
seven
kind
of
big
pages
about
how
to
do
different
things
with
Python
and
why
we
provide
Python
the
way
that
we
do.
We
try
to
keep
it
up
to
date.
A
There's
a
frequently
asked
questions
page,
which
is,
is
pretty
new,
but
there
are
some
questions
that
always
do
come
up
so
check
there.
First
before
you
file
a
ticket,
I
think
that
would
save
you
some
time,
there's
also
a
page
for
advice
or
gotchas.
If
you
want
to
try
to
use
Python
on
on
KML,
so
again,
I
can't
say
this
enough,
like
probably
like
once
a
week,
I
get
a
ticket
where
actually
the
person
is
using
the
wrong
Python.
Basically,
the
point
is:
don't
don't
use
this?
A
Don't
use
user
bin
Python,
that's
the
Python
that
is
installed
when
they
deliver
the
system
to
us
and
it's
usually
pretty
old
by
the
time
you
get
around
to
using
it.
So
instead
always
be
sure
that
you
do
a
module
load,
Python
of
some
sort.
You
can
pick
whether
or
not
you
want
to
be
a
Python
2
7
person
or
a
Python
3
6
person,
and
in
fact,
if
you
don't
want
to
use
the
modules
that
we
that
we
provide,
you
can
install
your
own,
you
can
do
whatever
you
want.
A
Yeah,
let
me
I'll
come
come
back
to
that,
to
tell
you
about
the
defaults
and
and
what
we're
doing
there
so
for
a
while
now
nurse
Python
is
anaconda.
Python
are
people
familiar
with
the
anaconda
distribution
for
Python.
It
seems
to
be
the
way
that
people
are
using
Python
today
is
they're
using
anaconda
Python.
So
we
see
that,
and
so
we
decided
that's
that's
a
way
to
provide
Python
to
users
if
that's
especially
what
they
like,
it's
nice
and
familiar
to
them,
so
we've
retired
our
own
builds.
A
There
is
a
cray
Python
there,
but
it's
pretty
it's
pretty
minimal
the
modules
that
I
mentioned.
They
exclusively
leverage
anaconda
Python,
if
you
know
virtual
end,
but
you
don't
know
anaconda.
The
only
thing
you
really
need
to
know
is
that
Conda,
this
Conda
tool
basically
replaces
virtual
end
and
then
goes
way
beyond
that.
So
package
management
with
Conda
is
what
you
is
with
the
kind
of
tools.
What
you
get
of
course,
there's
hundreds
actually
there's
many,
many
more
than
hundreds
of
useful
packages.
A
Now
in
the
anaconda
Python
you
have
the
threaded
mkl
library
built
into
anaconda
Python.
It
comes
for
free.
There
are
some
additional
many
more
actually
now
in
machine
learning
tools
in
the
defaults,
Channel
and
then
there's
ways
to
get
packages
from
other
people.
Who've
packaged
up
software
through
this
mechanism
called
channels,
so
I'm
not
going
to
go
into
great
detail
about
how
Conda
environments
work.
A
But
if
you
run
into
problems
provisioning,
your
own
Conda,
environment,
I,
can
help
you
debug
that
in
addition
to
that,
we
have
a
couple
of
extra
things
that
you
need
to
know
about.
If
you
use,
if
you
use
MPI
or
HD
f5
or
h5
PI,
and
you
want
to
use
like
do,
multi
node
jobs
with
I
can
I
can
address
that
in
a
little
bit
alright.
So
the
Python
modules
on
both
systems
are
these
two
here,
two
seven
anaconda
for
four
and
three
six
anaconda
for
four
to
seven
anaconda
for
four
is
the
default
now.
A
So
if
you
just
do
a
modulo
Python,
that's
the
one
you're
going
to
get,
and
that
is
going
to
be
the
one
that
you
get
for
like
the
next
21
months
or
so
as
the
default,
because
can't
make
it
come
up.
But
there's
this
this
website
down
here.
You
can't
see
right
there,
but
it
tells
you
it
gives
you
a
countdown
for
how
long
you
have
until
Python
to
seven
is
no
more
and
you
have
to
use
Python,
26
or
Python
3,
so
4.
A
Does
that
stop
us,
but
yeah
anyway,
like
I
said
you
can
do
you
can
do
your
own
stuff?
If,
for
whatever
reason
you
don't
like
kind
of
you
know,
like
our
modules
like
we
put
them
there
kind
of
as
a
way
for
users
to
get
started,
and
lots
of
people
are
just
fine
with
that,
but
sometimes
people
want
to
customise
them
or
they
want
to
bring
in
other
packages
that
come
from
other
channels
that
we
don't.
We
don't
know
about
they.
A
A
In
fact,
if
you
want
to
use
the
global
common
global
common
software
directory
a
way
to
put
your
own
Python
installation,
there's
to
just
grab
the
mini
Conda
bash
installer
and
run
it
there,
and
then
you
can
provide
some
kind
of
Python
environment
that
you
and
your
collaborators
like
to
use
to
all
your
clam
readers
and
one
piece
of
advice:
that's
that's
pretty
important
is
if
you're
going
to
use
MPI
from
Python
or
gonna
use,
h5,
PI,
parallel.
Ok,
you
don't
just
want
to
do
Condit,
install
MPI
for
PI,
that's
not
going
to
work.
A
It
will
be
very
confusing
to
you.
What
you
need
to
do
is
you
need
to
build
MPI
for
PI
against
the
Cray
in
pitch
well,
but
it's
very
easy,
as
we
saw
this
morning,
it's
really
easy
to
use
the
nice
Cray
wrappers.
That's
basically
all
that
you
do
here.
Is
you
just
build
it,
and
you
say
point
me
at
the
at
the
right:
C
C
compiler,
and
that's
it
ok.
A
So
that's
pretty
straightforward
so
that
you
only
need
to
do
this
if
you
create
your
own
content
environment,
where
you
do
your
own
installation
with
with
this
with
this
bash
script.
Okay,
if
you
use
our
Python,
it's
already
built
that
way.
Okay,
a
few
words
about
parallelism
with
Python
within
a
node.
You
know
people
always
kind
of
interested
in
how
how
they
can
improve
the
performance
of
their
Python
code
at
nurse
within
a
node.
A
The
way
to
to
improve
performance
through
parallelism
is
using
OpenMP
threaded,
math
libraries
generally,
that's
the
easiest
way
to
go
and
knowing
a
little
bit
about
OpenMP
environment
variables,
and
things
like
that
will
probably
help
multi-processing
works
to
multi-processing
is
a
process
based
parallelism
and
that
works
okay
within
a
node,
but
that
won't
scale
really
across
a
node,
okay,
so
to
get
or
beyond
one
node.
So
if
you
want
to
use
like
ten
nodes-
and
you
want
to
do
like
a
parallel
python
processing,
you
have
to
do
something
else
and
that's
MPI
for
pi.
A
A
We
haven't
been
able
to
really
scale
up,
but
it's
not
really
built
to
scale
up,
say
to
like
the
whole
machine
or
anything
like
that
in
terms
of
how
to
combine
parallelism
to
get
both
within
a
node
and
multi
node
parallelism
is
to
combine
MPI
for
pi
and
threaded
math
libraries
combining
MPI
for
pi
and
multiprocessing,
for
instance,
doesn't
work
and
the
reasons
for
that
are
well
it.
It
will
look
like
it
works
until
the
day
you
you
really
need
it
to
work
it.
It
will
have
a
sprained,
cryptic
error
when
it
crashes.
A
Profiling
and
debugging
Python
applications
in
general.
It's
not
yeah.
It's
not
just
a
specific
question
about
like
how
do
I
you
know.
How
do
I
do
that
at
nurse
it's
kind
of
a
general
question,
but
the
you
can
do
print
that
obviously
works.
The
only
advice
I
have
is,
if
you're
going
to
put
print
statements
in
and
run
a
job,
you
get
the
output
unbuffered.
So
that's
what
these
use
here
are
for.
Okay,
because
otherwise
it
will
run
and
you'll
be
like
what's
not
producing
output,
why
isn't
it
producing
output?
A
A
B
A
For
me,
Python
see
will
mean
that
means
take
the
string
that
comes
after
this
and
interpret
it
as
Python
and
run
that
yeah.
That's
what
C
is
okay
moving
beyond
print,
though
you
want
to
be
a
little
bit
more
sophisticated,
maybe
or
you've
done
everything
that
you
can
with
print.
What
can
you
do
in
terms
of
you
know?
A
We
think
that
that
that
works
pretty
well
in
combination
with
snake
vis
or
this
tool
called
g
prof
dot,
which
you
can
see
here
like
a
kind
of
a
tree
of
your
of
an
application
that
actually
Laurie
made
in
the
back
there.
That
tells
you
like
here's,
where
the
code
is
spending
most
of
its
time
and
it's
nice
and
graphical,
and
you
can
follow
that
if
you
have
lots
of
MPI
processes,
there's
a
handy
trick
that
a
nurse
user
uses
that
this
is
actually
a
link
here.
A
That
shows
you
how
you
can
actually
turn
on
profiling
and
turn
it
off
and
you
can
get
like
one
profile
per
MPI
rank
or
just
the
MPI
rank
you're
interested
in
it's
actually
pretty
easy.
It's
like
five
lines
of
code
for
a
little
bit
of
a
deeper
dive
after
you've
done
maybe
a
C
profile.
On
your
run
on
your
application,
you
can
slap
online
profiler,
which
is
a
separate
module.
A
A
Shifter
is
gonna,
be
after
this
if
you
start
running
Python
applications
that
are
going
to
go
to
like
thousands
of
MPI
ranks
or
something
like
that,
or
even
a
few
hundred
MPI
ranks
you're
actually
gonna
want
to
put
your
your
application
stack
into
a
docker
container
and
use
shifter
to
run
in.
So
Shane
will
tell
you
how
to
do
that
after
this
talk
now
for
the
Jupiter
part
of
the
talk,
we
have
a
node
on
quarry.
A
That's
basically
a
repurposed
login
node,
where
we
allow
users
to
spawn
Jupiter
notebooks
and
we
give
users
a
place
to
do
this
so
that
they
don't
do
it
themselves
and
maybe
a
insecure
way.
In
addition
to
that,
we
have
an
external
service
called.
We
have
an
external
service
that
you
know
you
may
want
to
use.
If
say,
Cori
is
down,
but
they're
there
at
these
two
different
URLs.
A
So
this
is
the
external
one,
the
one
that's
external
to
Cori,
and
then
this
is
the
one
that
actually
runs
on
Cori
and
you
get
different
things
with
these
two
different,
these
two
different
websites.
So
these
are,
of
course
external
to
the
Krays.
They
can't
see
the
scratch
filesystem,
which
may
be
important
you,
but
they
can't
see
the
rest
of
the
nurse
global
file
system.
The
one
that's
on
Cory
can
see
Cory
scratch,
obviously,
and
you
can
actually
even
like
submit
jobs
from
a
notebook
with
this
magic
s
badge
here
all
right.
A
So
if
there's
one
thing
by
the
way
whose
are
there
Jupiter
users
in
the
room
say
people
have
used
it
before
okay.
So
this
is
like
the
number
one
question
that
I
get
probably
twice
a
week.
How
do
I
set
up
my
own
kernel
right?
A
kernel
is
a
process
that
you
launch
that
basically
runs
the
calculation
for
you
in
the
notebook
right.
A
How
do
I
make
a
Condit
environment
for
myself
and
make
that
now
a
Jupiter
kernel
right,
so
these
people
will
set
up
a
bunch
of
packages
that
they
like,
and
then
they
want
to
be
able
to
access
that
from
Jupiter
dev.
So
there's
a
few
different
ways
to
accomplish
this,
but
this
is
kind
of
the
easy
way
of
doing
that.
So
I
think
I
think
it
would
be
fun
to
do
this.
This
way,
in
fact,
I
have
to
I
have
to
log
in
so
I
even
get
to
demo.
A
Just
in
time,
okay,
so
the
first
thing
you
need
to
do:
which-which,
Python
right.
The
very
first
thing
as
I
said
before,
I
was
don't
use
this
guy
right.
Do
this
where's
my
I
want
to
make
sure
my
example.
This
is
right:
module
load,
Python,
okay,
so
which
Python!
So
it's
not
user
bin
Python
anymore.
It's
this
user,
common
software,
Python
blah
blah
stuff
right.
Okay!
So
what
we
want
to
do
now
is
we
want
to
make
a
Condit
environment
kondeh,
create
n,
let's
call
it
early
user
Python
to
b6,
okay.
A
This
is
kind
of
new
behavior.
If
you've
been
using
the
Conda
tool,
you
didn't
have
to
do
this
before.
Okay,
it
would
just
pick
pick.
It
would
just
install
the
Python
into
that
environment
that
you
already
using
that's
not
the
case
anymore.
That
was
kind
of
a
surprise
to
me.
So
you
do
want
to
do
something
like
say:
I
want
this
to
be
a
Python
three-point-six
environment.
Before
you
install
any
other
packages,
if
I
were
to
install
numpy
I,
don't
know
which
one
I
would
get
yeah.
B
A
A
Any
other
questions,
while
this
is
spinning
there
all
right,
they
were
cached:
okay,
source,
activate
user.
B
A
A
Ok,
so
I've
made
an
environment
called
early
user.
Okay
and
it's
got
one
package
in
it:
I
have
to
install
another
one,
though
kind
of
install
I
PI
kernel.
Okay,
putting
this
into
the
package
actually
kind
of
just
makes
it
a
kernel.
Already.
We
have
a
thing:
a
package
called
env
Conda,
which
makes
anaconda,
detect
your
environments
and
just
make
them
kernels
on
the
fly,
and
this
helps
it
do
that
it
also
is
required
for
making
it
into
a
natural
kernel.
A
And
check
the
next
thing:
I'm
supposed
to
do
right
is
yeah.
Alright,
here's
the
magic
part
that
makes
this
into
a
kernel,
Python
M.
That
means
run
this
module
as
a
script.
Cana
install,
give
it
a
name
like
early
user
kernel
and
then
I'm
cheating
user,
and
what
that
does
is
that
makes
this
thing
called
a
kernel.
Spec
file.
A
A
A
A
A
You
can
honestly
when
people
write
that
ticket
I
just
go.
Do
you
really
have
to
have
this,
but
this
is.
This
is
how
you
can
do
it
you
you,
you
replace
that
that
Python
executable
with
some
shell
script,
it's
executable,
you
you
set
it
to
be
executable,
and
then
you
do
whatever
module
load
you
need
in
here
and
then
you
just
pass
in
these
arguments
to
the
to
the
Python
there.
Okay
and
then
the
really
handy
thing
is
when
you
run
on
Jupiter
dev.
A
There
is
in
your
home
directory
a
Jupiter
dot
log
and
it
tells
you
like
what's
going
on
so
if
you
have
problems,
that's
interesting,
if
you
have,
if
you
have
issues
trying
to
get
this
to
work,
the
first
place
to
look
is
the
Jupiter
dot
log
in
your
in
your
directory?
Okay,
this
is
very,
very
handy,
okay,
so
that's
the
end
of
the
Python
and
Jupiter
talk.