►
From YouTube: New User Training: 15 Shifter
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
So
I'm
shaking
I'm,
also
a
member
of
the
task
group
and
I'm,
also
one
of
the
developers
and
maintainer
zuv
shifter,
so
you've
already
heard
a
couple
of
references
to
it.
This
will
be
a
pretty
kind
of
rapid
walkthrough
of
it,
but
there's
other
resources
at
the
end.
That
I'll
point
you
to
that.
You
can
learn
more
about
them.
A
A
Tony's
sort
of
went
through
this,
but
just
in
brief,
what
sort
of
containers
and
docker
is
sort
of
the
poster
child
for
container
technologies
allows
you
to
do?
Is
it
allows
you
to
sort
of
build
ship
and
run
kind
of
applications
or
tools
or
packages?
It
provides
a
set
of
tools
to
take
sort
of
a
recipe
and
package
that
into
an
image,
and
then
you
can
ship
that
to
a
registry
either
a
public
one
like
docker.
This
is
dr.
A
cloud
here
now
it's
docker
hub
or
or
a
private
one
like
the
one
that
we
run
at
nurse
and
then
you
can
then
pull
those
images
down
and
and
run
them
on
different
hosts.
So
that
could
be
on
your
laptop,
a
workstation,
the
cloud
or
using
shifter.
You
can
run
it
at
nurse
roughly.
What
is
kind
of
packaged
up
in
an
image
is
things
like
the
file
tree
layout
that
you
would
see
you
know.
Imagine
you
log
into
a
system
all
of
the
the
directory
tree
structure
that
you
see
kind
of
gets
captured.
A
So
that
means
the
base
operating
system
based
Linux
operating
system,
all
the
libraries
and
tools
that
sort
of
come
along
with
that,
either
from
the
OS
itself
from
the
distribution.
Well,
that
you
might
install
you
know
your
own
user
code
and
applications,
and
even
data
and
there's
also
ways
to
have
it
captured
things
like
the
environment.
Variable
settings
working
directories,
sort
of
default
ways
that
it
should
execute,
and
then
there
are
other
things
that
aren't
so
much
relevant
to
how
we
run
these
containers
on
HPC
systems.
A
But
you
can
also
have
it
capture
things
related
to
network
settings
and
the
run
user,
for
example.
So
docker
sounds
really
cool
and
some
of
you
may
have
already
used
it.
So
why
not?
Why
don't
we
just
installed
our
on
the
system
and
let
users
have
at
it,
and
the
biggest
reason
is
this:
first
one
there's
some
just
fundamental
security
issues
with
allowing
a
regular
user
to
run
docker
ona
online
in
system,
and
it
effectively
allows
the
user
to
have
root
privileges
on
that
system.
A
So
this
is
an
example
where
I'm
going
to
just
take
slash
and
mount
it
into
another
M&T
space
inside
that
container
and
once
I
do
that
I'm
rooting
I
could
go
in
edit
file.
So
if,
if
I
was
a
regular
end
user
that
wasn't
supposed
to
have
any
privileges
now
I've
got
them
or
I
could
take
somebody
else's
directory
that
I
shouldn't
have
access
to
map
that
in
and
start
seeing
their
their
data,
their
files
be
able
to
edit
things.
A
So
obviously,
in
a
shared
environment
like
like
nurse,
that's
kind
of
a
non-starter
there's
other
challenges
around
sort
of
how
it
integrates
with
the
system.
They
don't
want
dab
in
too
deeply,
but
the
main
thing
is:
is
we
want
this
to
sort
of
integrate
within
the
batch
system
in
a
in
a
clean
way,
and
we
want
this
to
be
able
to
run
it
at
scale,
and
so
dr.
A
out-of-the-box
sort
of
breaks
a
lot
of
these
things
so
to
address
that
nurse
a
couple
of
years
ago
started
an
effort
to
to
figure
out
ways
to
be
able
to
run
sort
of
docker
containers
securely
and
scalable
on
our
systems
and
the
way
we
went
about
doing
this
is
we
wanted
to
leverage
as
much
of
the
docker
ecosystem
as
possible,
so
all
of
the
tools
that
used
to
build
the
images
and
define
them?
We
wanted
to
keep
that
the
ways
that
you
could
ship
those
images
to
registries.
A
We
want
to
keep
that,
but
then
we
just
wanted
to
replace
the
runtime
components
so
that
we
could
run
those
securely
on
our
on
our
systems.
And
so
that's
what
what
this
does
and
like
I
said.
It's
been
in
production
for
a
couple
of
years
now
and
we
were
really
on
the
forefront
of
this
nurse
was
one
of
the
first
sites
to
really
be
pushing
this
this
model.
A
So
why
would
you
as
an
end-user,
be
like
containers
in
shifter?
So
one
thing
is:
is
it
means
that
you
can
develop?
You
know
you
can
sort
of
test
and
develop
an
application
on
your
workstation
package
that
up
and
and
ship
it
and
run
that
exact
same
code
on
you
know
quarry
or
Edison
you're
able
to
solve
the
dependency
problems
yourself.
A
And
then
you,
you
know
it's
a
lot
of
yak
shavings,
so
you
can
kind
of
bypass
this.
All
in
a
docker
containers,
because
you
have
total
control
over
it
and
you
can
just
do
an
app
get
or
a
yum
install
instead
of
having
to
try
to
manually
build
all
these
things,
you're
not
limited
to
the
Susa
OS
that
we
run
on
the
on
the
Krays.
You
could
run
a
red
hat
or
CentOS
or
a
bun,
or
something
like
that.
In
many
cases
it
improves
application.
A
Performance,
Rowling
showed
a
slide,
I'll
show
it
I'll
show
it
again
and
talk
about
that,
and
also
these
last
two
in
fact,
I'll
go
to
the
next
slide.
I
think
containers
are
also
useful
just
for
science
in
general,
and
for
these
some
of
these
reasons
one
is
it
kind
of
helps
with
reproducibility,
because
you
can
take
everything
you
need
to
run
that
application
package
it
up
and
save
it
and
tag
it
up
in
a
registry
that
means
later
on.
You
can
pull
it
back
down
rerun.
It
know
it's
sort
of
the
same
code.
A
You
can
also,
you
know,
have
your
colleagues
and
collaborators
pull
that
image
down
and
they
can
run
the
same
thing
as
well.
You
can
even
publish
it
in
a
paper,
and
then
you
know
if
somebody
wants
to
go
and
test
it
out
themselves.
They've
got
everything
they
need
to
run
that
it
also
is
a
little
more
auditable
than
what
you
might
get
when
you're
using
the
docker
file
format,
which
I'll
talk
about
in
a
bit.
It
really
has
a
clear
set
of
steps
of
like
this
is
how
it
constructed
this
right.
A
This
is
how
I
built
this
environment.
So
if
somebody
wants
to
come
back
later
on
and
look
at
it,
they
can.
They
don't
have
to
know
that
there
were
all
these
unwritten
things
that
happen
behind
the
scenes.
They
can
look
at
that
recipe
and
basically
understand
how
you
how
you
built
things
portability,
because
you
can
now
run
that
image
across
different
systems
and
because
of
these
things,
you
also
get
sort
of
some
reduction
in
effort.
I
don't
have
to
sit
there
and
rebuild
everything.
A
A
So
again,
a
docker
file
is
like
a
recipe
on
how
you
construct
an
image
right,
so
you
can
almost
think
about
it
as
what
are
the
things
that
you
kind
of
copy
into
an
environment
and
what
are
the
things
you
run
to
start
to
prepare
that
right
and
that's
effectively
what
these
sort
of
primitives
you
see
in
these
doctor
files.
So
the
first
line
the
from
is
just
saying:
I
want
to
take
a
base.
Os
I,
don't
want
you
don't
want
to,
for
everything
have
to
start
from
from
nothing.
A
So
there's
a
lot
of
common
OSS
out
there
or
base
images
that
you
typically
use
so
in
this
case
I'm
basing
it
off
in
a
bun
to
1404
image.
This
is
kind
of
the
old
syntax,
but
it's
always
good
habit
to
go
ahead
and
put
you
know
who
the
person
is.
That's
creating
this
this
docker
file,
and
then
you
see
these
two
kind
of
primitives,
there's
run
and
add
so
run
is
saying
you
know:
go
into
the
matrix
space,
go
into
the
container
space
and
run.
A
You
know,
run
this
operation,
so
in
this
case
I'm
just
going
to
do
and
I
don't
think
that's
a
valid
syntax
I
think
it
should
be
apt-get
update
some
of
these
I
actually
just
typed
in
the
window.
I
didn't
cut
and
paste.
So
you
do
an
update
and
then
you
install
the
build
essential
tool
so
that
you've
got
GCC
and
stuff
and
then
add
as
a
way
to
say,
take
something
that
from
is
from
outside
the
container
I'm
building
and
bring
it
into
that
space.
A
So
I'm
saying
bring
the
working
directory,
Khan
tense
and
just
/my
up
and
then
on
the
next
step.
I
go
in
there
and
I
start
to
build
it.
So
that's
just
a
very
simple
example:
you
can
imagine
you
can
just
keep
kind
of
chaining
these
things
to
do
more
complicated
operations
and
then
the
other
thing
is
is
at
the
end.
What
happens
is
to
create
take
this
recipe
and
create
an
image
from
it?
These
are
the
commands.
A
I
would
run
so
it's
a
doctor
build
and
I'd,
give
it
a
name
and
then
there's
a
little
period
there.
At
the
end,
that's
easy
to
overlook
that
says:
use
the
current
working
directory
as
the
place
to
look
for
the
docker
file
and
then
once
it's
built
I
can
push
it
to
a
registry
like
that
there's
a
kind
of
a
missing
step
here,
which
is
you
have
to
the
first
time
you
use
docker
and
you're
right.
A
So
the
other
thing
that's
interesting
here
is
you
know
now:
I've
got
this
tag
damaged,
others
could
based
off
of
that
use,
instead
of
their
from
line
saying
upon
two.
They
could
say
this
and
then
add
things
to
it
too,
so
you
can
also
imagine
creating
sort
of
a
you
know.
A
base
image
that
you
might
use
for
a
larger
project
and
then
you
could
have
sub
images
off
of
that
that
have
you
know
specific
tweaks
to
it,
for
example,
and
there's
also
a
colon
and
a
number
there.
You
can
create
tags
as
well.
A
So
that
means
that's
a
good
way
to
say.
If
you
kind
of
get
something
that's
working
well,
you
could
put
a
specific
tag
on
it,
so
that
you
could
go
back
and
reuse
it
later
all
right
and
then
so,
and
here
we've
prepared
an
image
and
we
pushed
it
to
a
registry.
How
would
we
use
that
with
shifter,
so
you
can
use
shifter
to
run
images
sort
of
interactively
or
through
a
batch
a
batch
script,
so
in
this
case
I'm
showing
the
batch
batch
mode.
A
So
we
have
extensions
to
slurm
to
sort
of
understand
to
know
about
shifter.
So
you
can
actually
embed
this
this
image
to
tell
it
what
shifter
image
you,
what
doctor
image
you
want
to
use
and
that'll
make
sort
of
behind
the
scenes.
It'll
do
some
setup
for
you
so
that
later
on,
when
you
do
in
this
example,
we're
doing
an
MPI
application.
A
So
we,
the
only
thing
that's
different-
is
we
just
add
this
shifter
command
and
before
the
execution
of
the
actual
application,
and
in
this
case
that
application
exists
in
the
image
inside
that
container,
it
doesn't
exist
in
you
know
if
I
went
out
on
quarry
and
look
there's
no
slash
my
app
directory.
This
is
coming
from
the
image.
I
could
also
start
up
an
interactive
session
and
just
get
out
on
a
node
and
type
shifter
and
image,
and
whatever
image
I
want
to
run
and
use
that
as
well.
Okay,.
A
So
one
question
people
have
asked
is:
can
you
use
shifter
and
containers
and
run
MP
applications
and
the
short?
The
brief
answer
is
yes,
we
have
supported
that
sort
of
built-in
to
help
you
do
this
and
give
an
example
sort
of
a
building,
an
image
with
MPI
support
in
it
under
the
hood.
What
we
do
is
we
kind
of
map
in
the
libraries
that
are
optimized
for
the
Cray
behind
the
scene,
so
the
the
only
thing
you
really
need
to
do.
A
A
So
this
is
an
example
here,
we're
using
a
base
image
that
we've
already
constructed,
and
then
we
just
add
in
our
application.
We
build
it
just
like
we
normally
would
so
there's
nothing
that
looks
specific.
You
know
particularly
different
and
then,
when
we
go
to
run
the
application
we
just
again
just
would
do
the
s
run,
the
s
runs
outside
of
the
container
space
and
then
you
do
shifter
in
the
application.
A
A
Don't
have
to
go
all
the
way
back
to
the
lustre
file
systems,
metadata
servers.
Instead,
it's
it's
kind
of
handled
locally,
and
so,
as
a
result,
we
can
get
a
lot
of.
We
can
get
speed
ups
in
things
like
Python,
so
anything
that's
doing
a
lot
of
dynamic
library
loading
you
may
want.
If
you're
going
to
run
this
at
really
large
scales,
you
might
want
to
consider
putting
that
in
a
shifter
container.
Just
a
few
notes
about
differences
between
shifter
behavior
versus
docker.
The
biggest
thing
is
processes
run
as
you
not
as
route.
A
This
is
for
obvious
sort
of
security
reasons.
Also,
images
are
mounted
up
read-only.
So
if
you
have
an
image
and
it's
it's
behavior
is
to
go
and
edit
files
inside
the
image
at
runtime
that
will
probably
not
work
with
shifter
there's
ways
to
work
around
it,
but
that
one
is
probably
the
more
tedious
one
for
the
first
one.
You
just
need
to
make
sure
that
any
of
the
you
know
executables
or
libraries
that
you
need
to
access
that
their
world
readable
and
not
just
readable
by
route.
A
We
also
automatically
mount
in
home
directories
and
scratch
directories
projects
things
like
that.
So
there's
a
lot
of
sort
of
magic
that
just
happens
for
you
and
there
are
certain
directives
that
we
don't
support
and
docker.
But
generally,
if
you
just
want
to
package
your
application
up,
it
should
work.
A
There
are
some
other
things
that
probably
don't
have
time
to
go
into
any
depth,
but
you
can
do
things
like
volume
outs.
This
allows
you
to
take
a
directory
tree
sort
of
outside
the
container
space
and
map
it
into
a
location
in
the
interspace.
So,
for
example,
you
could
take
your
scratch
area
and
have
it
show
up
as
slash
data,
for
example,
in
the
container
space.
A
We
also
have
a
registry-
that's
really
intended
for
people
running
things
through
shifter,
that
you
can
push
images
to
you.
Just
you
can
just
log
into
that
using
your
your
regular
nurse
LDAP
password
an
account
just
quickly
since
Tony
talked
about
spin
just
wanted
to
say
so
shifter
you
generally
will
use
shifter
if
you
want
to
run
things
on
the
HPC
systems
at
scale.
If
you
want
to
run
model
million
simulation
jobs
analysis,
you
know
machine
learning,
tools,
things
like
that.
You
know
things
that
have
a
sort
of
finite
lifetime
to
them.
A
When
you
run-
and
you
typically
would
you
spend
for
things
like
persistent
services,
things
that
you
want
to
run
kind
of
indefinitely
things
that
need
to
be
externally
accessible,
and
you
know
the
difference
in
sort
of
how
they
run
is.
This
is
using
spins
using
regular
stock.
Docker
shifter
is
sort
of
a
modified
run
time
and
just
briefly
an
example
of
how
some
users
have
used
shifter.
This
comes
from
an
application
called
toast,
and
here
they
ran
at
the
pretty
much
the
full
scale
of
the
KL
side
of
quarry.
So
some
50th
out
I
think.
A
Do
you
remember
what
scale
they
ran
out?
600,000,
yeah
600,000
course:
yes,
full
the
machine,
and
for
them
that
was
really
critical
that
they
used
shifter,
because
originally
they
tried
just
running
it.
It's
a
Python
sort
of
wrapper
around
their
application
and
when
they
tried
running
at
you
know
above
a
certain
scale,
they
were
spending
all
of
their
time
just
and
start
out
trying
to
load
in
the
Python
application.
A
So,
but
you
know
this
was
kind
of
critical
to
their
success
and
getting
these
runs
to
go
through,
so
you
can
run
it
to
full
full
system
size
and
if
you
need
more
information
on
the
nurse
website
under
for
user
software,
there's
a
shifter
section
and
it
covers
most
of
what
I
just
talked
about.
There's.
Also
we've
given
a
Lisa,
Gerhardt
and
I
have
given
tutorials
at
SC
and
some
other
venues
and
there's
a
link
to
the
material
for
that
in
our
github
area
under
shifter
tutorial.
A
So
if
you
just
go
to
the
nurse
github
area,
you'll
see
it
and
then
obviously
for
doctor
itself,
there's
lots
of
resources
out
there.
So
just
google,
it
is
probably
the
best.
The
best
way
to
go
I,
don't
think
I
really
have
any
time
for
any
demos,
so
I'll,
just
I
can
take
questions
up
how
many
people
have
used
docker?
A
Okay,
you
guys
don't
count.
Okay
well,
I,
definitely
say
even
if
you're
not
sure,
if
you
have
an
immediate
use
for
it
grab
it
install
it
on
your
laptop
and
kick
the
tires
on
it.
It's
really
pretty
powerful.
It's
even
useful,
just
as
a
way
on
your
laptop.
If
you
want
a
quick
way
to
get
into
a
Linux
environment,
so
you
have
access
to
other
tools,
even
just
for
that.
It's
worthwhile
but
I
think
once
you
get
used
to
it.
You'll
see
you
know
the
the
full
value
of
it.