►
From YouTube: 13. Using Shifter
Description
Learn about how to use Shifter to run containers on Cori.
Slides for all sessions can be downloaded from here: https://www.nersc.gov/users/training/events/new-user-training-june-21-2019/
A
So
for
the
next
talk,
the
next
talk
is
by
Shane
cannon,
so
I'll
switch
my
nametag
and
become
Shane
now
so
Shane
Shane's
in
Germany
and
actually
I
owe
him
because
he
gave
my
Python
talk
last
time
and
I
did
look
at
the
slides
at
least
ahead
of
time.
So
all
right
so
I'll
give
you
a
quick
intro
to
Sosh.
Iftar
is
about
containers,
and
it's
about
running
containers
on
HPC
on
our
on
quarry.
A
So
I'll
give
you
a
quick
intro
to
two
containers
and
then
I'll
talk
about
what
shifter
is
and
how
it's
different
from
docker
and
then
we'll
walk
through
it's
just
because
I'm
not
looking
at
it
right.
I'll
walk
through
the
use
of
shifter
and
docker,
actually
I
wonder
whether
or
not
this
aspect
ratio
is
worse
for
it
or
something
anyhow,
we'll
go
on
intro
to
containers
and
shifters.
So
how
many
people
have
used?
Docker
I
get
tired
of
raising
your
hand.
Okay,
so
docker
is
a
container
technology.
Okay,
so
basically
it's
about
build
ship
run.
A
So
it's
not
not
like
VM.
So
you
don't
like
build
this
virtual
machine
with
all
this
software
in
it.
You
have
this
kind
of
really
much
more
lightweight
kind
of
image
where
you
can
install
whatever
software
you
want.
They
can
move
that
thing
around
and
you
can
run
it
anywhere
where
the
docker
stuff
is
supported.
A
So
you're,
building,
edges
capture
your
applications
and
what
they
all
depend
on.
You
can
create
a
file
that
actually
kind
of
automates
the
build
process
called
a
docker
file.
You
can
push
that
around
to
different
registries
in
different
places,
either
docker
the
cloud
or
a
hosted
registry
or
your
own
private
registry,
and
share
these
images
with
other
users
and
pull
down
images
and
and
execute
them
okay.
So
they
kind
of
bottle
up
all
the
dependencies.
A
All
of
the
all
of
the
software
that
you
need
all
the
libraries
you
need
to
run
an
application
and
the
application.
Okay.
So
there's
applications
for
reproducibility
and
stuff
like
that.
What's
inside
an
image,
well,
there's
a
file
system.
So
there's
a
directory
tree,
there's
a
base,
Linux
operating
system,
lynnie
distribution
that
you
pick
stuff
for
building
software
and
running
code.
Your
code,
sometimes
data
but
probably
don't
put
too
much
in
there
probably
configuration
stuff-
is
what
what
he
means.
There
also
run
time
settings
so
environment
variables
can
be
set
inside
of
a
container.
A
What
working
directory
you
want
to
start
from
that
can
be
set
there
as
well,
how
you
actually,
what
command
to
run
by
default
and
all
of
the
arguments
that
go
with
it.
That's
also
specified
other
things
not
relevant
to
shifter.
Oh
okay,
you
can,
you
can
manage
the
there's
kind
of
a
networking
component
of
docker,
so
different
docker
containers
can
talk
to
each
other
over
a
network
and
then
who's
actually
running
alright.
So
this
talk
is
about
shifter,
but
it's
related
to
docker.
A
What's
the
difference
between
docker
and
shifter,
and
why
do
we
have
this
thing
on
Cori
called
shifter
that
we
run?
Why
don't
we
just
run
docker?
Well,
docker.
Has
this
all-or-nothing
security
model?
So
if
you
ran
docker
on
Cori,
you
would
basically
have
system
level
period.
Privileges
so
you'd
be
like
root
on
a
supercomputer
which
isn't
good.
It
also
assumes
that
there's
a
local
disk
and
like
our
compute
nodes,
don't
have
a
local
disk,
so
that
kind
of
doesn't
make
sense.
Docker
doesn't
play
nicely
with
batch
systems.
A
I
think
that
that's
really
just
a
statement
about
I,
don't
really
know
what
that
means,
but
I
think
that
I
think
that
maybe
that's
just
what
it
means
on
face
value
I,
don't
really
know
what
Shane's
getting
at
with
that
one,
but
we
we
made
sure
that
shifter
plays
nice
with
our
batch
system.
I
guess
docker
typically
requires
a
much
more
modern
kernel
than
a
lot
of
people
are
running,
so
that
kind
of
places
a
burden
on
on
system,
administrators
and
then
complexity,
real
docker,
would
add,
would
be
a
lot
more
complex
than
running
shifter.
A
We've
taken
a
lot
of
the
complexity
out
of
docker
to
create
shifter
shifter.
Was
this
research
and
development
effort
here
at
nurse,
so
was
it
was
invented
here?
It
was
put
together
with
Cray
to
kind
of
support
these
kind
of
workloads
where
people
say:
hey
I,
don't
like
your
software
stack
I
want
to
have
my
own.
Is
there
a
way
for
me
to
have
a
VM
or
something
like
that?
A
There
are
multiple
places
that
are
run
Christams
that
are
using
shifters,
so
we're
not
the
only
one
and
create
support
shifter.
So
it's
not
just
us
doing
it.
Shifters
addresses
security
issues.
Some
of
the
security
issues
I
talked
about
on
the
on
the
previous
slide,
and
there
also
is
intermeshed
with
with
with
the
slurm
workload
manager,
but
the
workflow
for
developers
or
for
people
who
want
to
use
shifters.
A
You
create
a
docker
container
and
then
you
ship
it
to
a
registry,
and
then
you
pull
it
down
onto
Cori
and
there
it's
there
you're
using
shifter
infrastructure
to
run
it.
So
it's
kind
of
got
a
runtime
piece,
which
is
the
shifter
piece?
Ok,
so
why
will
you
like
containers
and
shifter,
ok,
you'll,
be
able
to
the
thing
that
I,
like
is
I,
can
pick
the
Linux
distribution.
A
I
can
pick
all
the
libraries
I
can
pick
how
I
how
I
set
up
the
whole
software
stack
and
I
can
stick
it
in
a
container
and
I
know
what
I
did
and
if
somebody
on
Cori
say
changes
what
the
software
stack
looks
like
I,
don't
care
right,
I
know
it's
in
there.
I
can
always
run
it,
but
there's
a
number
of
other
things
that
people
may
like.
So,
of
course,
you
can
develop
application
on
your
desktop.
You
can
run
it
in
the
container
sometimes,
and
then
you
can
ship
it
to
Cori
and
run
it.
A
There
too
also
lets
you
solve
your
dependency
problems.
The
way
you
like
to
do
it
without
writing
lots
of
tickets.
To
us,
of
course,
I
just
mentioned
running
the
Linux
operating
system
of
your
choice
sometimes
actually
matters
for
performance
for
some
people.
So
you
can.
You
can
make
a
choice
that
works
for
you
best
and
in
some
cases
it
improves
application
performance.
So
I
mentioned
before
in
the
Python
talk
stuff
about
shifter
and
why
that's
the
best
place?
A
That's
what
we're
getting
at
reproducibility
your
whole
build
is
there
in
a
docker
file
and
you
can
share
the
whole
stack
with
other
people.
Okay,
so
I
think
I
talked
about
reproducibility
portability.
It
runs
wherever
you
can
run.
Shifter
reduction
of
efforts,
compile
takes
ten
hours
just
do
it
once
and
share
it
with
everybody.
So,
instead
of
everybody
in
your
group
having
to
build
your
application
software,
you
just
give
them
the
name
of
the
shifter
image.
A
Basically,
and
if
there,
of
course,
if
the
library
isn't
there
on
the
on
the
native
system,
you
can
just
put
it
into
your
container
right.
So
I
would
say
one
thing
that
that
people
probably
need
to
get
used
to
with
with
shifter
and
docker.
Is
that
once
you
push
your
image
to
shifter
its
it's
baked,
it's
done.
You
can't
get
into
your
shifter
image
and
then,
like
do
young
install
whatever
you
forgot.
So
that's
a
little
bit
of
a
complication
that
makes
the
loop
a
little
bit
more
open,
but
generally
I.
A
The
way
that
docker
works
is
that,
if
you
just
add
a
thing
at
the
end,
it
doesn't
reduce
the
whole
build,
and
so
it's
not
as
not
as
slow
as
it
sounds
all
right,
so
shifter
in
action.
How
does
it
work
so
this
is
on
your
laptop,
or
this
is
we're
pretending.
This
is
Shane's
laptop.
You
create
this
file
called
a
docker
file.
It's
name
is
just
docker
file
with
a
capital
D
and
what
you
there's
these
verbs.
A
You
need
to
learn
some
verbs
if
you
haven't
used
docker,
yet
those
are
those
things
in
all
caps
on
the
left
hand
side,
but
basically
it's
all
just
UNIX
stuff
that
you
want
to
run
okay.
So
for
bun
too,
the
thing
to
install
packages
is
the
apt-get,
so
you
do
apt-get
update,
apt-get,
install
all
the
compilers
and
all
the
libraries
and
all
that
stuff.
You
can
copy
your
application
directly
into
the
container
while
it's
being
built,
and
then
you
do
make
okay.
A
So
your
while
you're
sitting
there
this
is
running,
make
is
running
and
then,
when
it's
all
done,
you
have
this
image
that
has
your
application
in
it.
Okay,
so
the
build
step
is
the
this
thing
down
here,
sorry
here.
So
this
is
the
build
step,
and
then
this
actually
pushes
it
to
some
repository
somewhere.
Okay,
all
right,
how
do
you
actually
use
the
image
on
quarry?
The
first
thing
you
have
to
do
since
now:
it's
up
in
this
registry.
You
have
to
pull
it
down
to
quarry.
A
So
that's
the
shifter
image
pull
it's
not
docker,
poles,
okay,
shifter
image,
pull
image
name
and
then
you
can
submit
a
batch
job.
That's
got
this
guy
in
the
header.
Okay,
and
the
cool
thing
is
that
the
image
lands
on
the
compute
node
before
the
job
starts.
So
you
don't
pay
for
that.
Well,
you're,
not
waiting
for
the
the
shifter
pole
tattoo
to
complete.
While
your
job
is
starting,
you
can
actually
change
what
shifter
image
are
using
in
the
job.
A
But
then
you
will
take
the
hit
of
the
shifter
pole,
okay
right,
but
it's
fairly
straightforward.
In
fact:
I,
don't
think
module
load,
I
think
you
should
have
taken
module
load
shifter
off
of
this
example,
but
basically
it
will
get
shorter.
The
only
thing
you
need
to
do
in
your
job
script,
it's
a
little
bit
different
from
like
just
running
a
regular
application.
Is
this
shifter
prefix?
Okay?
If
you
don't
do
that,
it's
gonna
go
I!
I,
don't
know
what
my
app
app
is.
A
There's
no
such
thing,
but
this
is
the
path
inside
the
container
that
you
wanted,
that,
okay!
Alright,
how
do
you
use
shifter
with
mpi
shifter?
Has
this
built-in
support
for
mpi,
but
that
doesn't
mean
that
every
shifter
image
can
use
mpi.
You
actually
have
to
have
the
mpi
libraries
inside
of
your
container,
because,
presumably
you
want
to
build
an
application
against
some
MPI
library
right.
A
A
Okay,
then,
at
runtime,
this
cool
magic
thing
happens,
your
shifter
image
gets
started
up
and
because,
because
the
library
is
compatible
with
what
we
have
running
on
quarry
it
just
kind
of
swaps
out
the
backend
okay,
so
then
you're
using
the
cray
in
pitch
alright.
So
it
figures
all
of
that
stuff
out
for
you
at
runtime.
You
don't
have
to
rebuild
the
container
and
it
actually
is
completely
like
running
a
regular
old
MPI
application.
Natively
same
performance.
I
think
this
is
what
it
looks
like.
A
We've
been
building
up,
shifter
images
that
have
components
that
people
frequently
want
like
MPI.
So
here
the
first
line
is
the.
So
we
have.
We
have
a
few
min
jizz
that
you
can
use
that
have
commonly
requested
components
in
them.
So
here's
one
called
nurse
/
bun
to
MPI
that
has
all
the
MPI
stuff
already
compiled
into
it.
So
then,
if
you
have
hello
world,
you
can
come
along
and
start
with
that
image.
Stick
hello
world
in
there
and
it's
got
like
you
know:
MPI
init
and
then
hello
from
rank
blah.
A
A
Any
way
you
can,
you
can
download
this,
and
you
can
use
this.
You
can
build
build
against
pre
pre,
can
kind
of
images
and
use
those
for
your
for
your
MPI
enabled
applications.
Okay,
so
I
mentioned
before
you
want
to
be
careful
about
when
you
run
a
large-scale
Python
application.
If
you
have
like
a
thousand
MPI
ranks
or
even
like
a
few
hundred
MPI
ranks
to
be
careful
about
what
file
system
your
Python
stack
is
on
and
I
said
that
the
very
best
choice
was
shifter,
which
isn't
a
file
system.
It's
this
container
system.
A
The
reason
that
that's
true
is
that
your
entire
Python
stack
is
self-contained
inside
the
image.
The
image
lives
on
the
RAM
disk
on
the
compute
node.
So
looking
through
that
to
find
all
your
stuff
is
really
fast,
because
it's
it's
the
RAM
disk.
The
other
thing
is:
is
that
there's
none
of
that
involvement
of
that
metadata
server
on
the
parallel
file
system
like
on
GPFS
or
lustre?
So
there's
not
some
separate
guy
out
there,
whose
job
is
to
keep
track
of
where
everything
is
that's
inside
the
container.
A
So
you
at
most,
you
have
like
a
few
ranks
asking
the
container-
hey
where's
where's,
where
all
these
files,
so
all
the
metadata
lookup,
is
localized
to
the
to
the
compute
node.
So
it's
not
involving
the
network.
It's
not
involving
this
metadata
server.
So
it
completely
cuts
out
that
middleman
and
everybody
doesn't
care
about
where
everybody
else
is.
A
You
can
even
make
shifter
applications
do
that
lookup
faster,
because
when
you
build
the
container
your
route
and
there's
this
little
trick
for
if
you
have
a
lot
of
shared
object,
libraries
and
you
route,
you
can
run
this
thing
called
LD
config,
which
will
cache
where
all
the
symbol,
where
all
the
where
all
the
shared
object
libraries
are
ahead
of
time
and
that
can
shave
off
maybe
10
or
15%
in
terms
of
load
time.
Okay,
all
right.
A
Differences
between
I
think
that
the
last
few
slides
are
going
to
be
just
a
few
things
to
keep
in
mind
about
shifter
shifter
versus
docker
in
shifter
processes
that
run
from
your
sugar
container
run
as
you,
they
don't
run
as
root
okay.
So
that's
a
little
bit
different
from
when
you
run
docker
on
your
laptop.
A
You
go
in
it's
probably
route,
but
they
don't
allow
that
on
quarry
images
are
mounted
read-only,
like
I,
said
so,
once
you've
built
it
it's
it's
baked
and
to
make
changes
to
it,
you
have
to
push
a
new
image
home
directories,
global
file
systems,
all
of
that's
automatically
mounted.
So
when
you
do
like
you
just
get
into
an
interactive
job
and
you
do
shifter
the
only
thing
that
that
you
see
changes
like
the
prompt
changes
from
like
one
symbol
to
another,
but
all
your
file
systems
are
still
there.
A
You
can
still
see
home
directory
scratch
and
all
that
some
handling
of
special
docker
file
directives
aren't
yet
supported,
but
I
I'm
not
really
sure
what
those
are
I'm,
not
worried
about
that.
Other
things
to
think
about
there
are.
These
neat
features
like
this.
This
way
to
mount
volumes,
you
can
map
a
directory
into
another
location,
your
image
they
can
do
that
to
scratch.
People
are
people,
use
that
to
do
this
per
node,
writable
cache.
So
on
a
node,
you
can
have
kind
of
more
temporary
space,
but
it's
really
just
scratch
underneath
this
is.
A
This
is
documented
on
the
website
as
well.
I
don't
see
the
details
here
and
then
we,
we
do
run
a
private
registry.
So
you
can
push
your
images
to
our
private
registry
and
then
pull
them
down,
so
you
don't
have
to
put
them
on
docker
hub
or
whatever.
If
you
don't
want
to
put
them
there,
you
can
put
them
here
and
then
only
people
here
can
see
them.
A
A
What
do
people
use
it
for
for,
like
I,
said
scaling
up
these
kind
of
massive
Python
applications?
Here's
an
example
of
one:
this
is
a
simulation
of
a
code
from
a
code
called
toast.
This
was
this
look.
This
ran
on
all
of
the
Cori
KML
nodes,
I,
think
pretty
soon
after
we
had
them
and
that
that
application
is
a
layer
of
Python
and
then
a
bunch
of
C++
code
underneath
for
high
performance
and
what
they
do
is
they.
A
They
calculate
the
Cosmic,
Microwave
Background
and
all
these
different
frequencies
and
they
simulate
weather
and
different
sites,
and
things
like
that,
so
they
were
able
to
simulate
50,000
instances
of
their
telescope.
They
do
like
50,000
realizations
of
the
CMB
stuff
and
they
used
all
of
the
quarry
K&L
partition.
Without
it
they
would
have
never
been
able
to
start
the
application
up,
but
they
were
able
to
start
the
application
up
in
about
60
seconds
this
way,
so
MPI
init
that
takes
60
seconds
on
the
whole
machine,
isn't
so
bad?
A
A
Yes,
yeah,
but
if
you
really
want
to
know
how
to
want
to
really
get
good
at
building
docker
images,
the
best
place
to
go
is
oh
right.
There
is
the
how
to
get
started
with
docker
stop.
So
there's
a
thing
that
users
yet
another
thing
that
users
kind
of
have
to
learn
in
order
to
do
stuff
is
you've,
got
to
learn
a
little
bit
about
docker
in
order
to
use
shifter,
but
it
isn't
it's
not
that
bad
to
get
started.
I
found,
but
yeah
mm-hmm.