►
From YouTube: Containers as a Service Spin
Description
Part of the Data Day 2022 October 26-27, 2022
Please see https://www.nersc.gov/users/training/data-day/data-day-2022/ for the training agenda and presentation slides.
A
Right,
yeah,
thanks
for
joining
up
today,
I'm
just
going
to
give
a
brief,
a
brief
talk
about
spin
a
little
bit
of
a
teaser
and
I
have
a
demo
in
here
as
well,
and
so
just
enough
to
kind
of
give
you
a
taste
of
what
the
service
is
capable
of
and
a
little
bit
of
what
it
might
look
like
and
feel
like
to
use
for
those
of
you
who
are
new
to
it.
A
So,
as
Annette
said,
I'm
the
lead
to
the
infrastructure,
Services
Group
and
we
deploy
spin,
but
we
do
it
in
partnership
with
a
lot
of
other
folks
at
nursing.
Some
of
them
are
there
in
the
room.
So
it's
actually
something
that's!
You
know
involved
a
lot
of
people
across
the
organization
to
operate
and
and
support.
So
so
you
know:
where
did
it
come
from
where
it's
been
really
come
from
yet
well?
The
idea
was
that
we
kept
hearing
questions
from
from
our
user
Community
about
Services.
A
You
know
the
nature
of
these
services
that
they
is
that
they
would
be
persistent,
but
they'd
be
able
to
access
some
of
the
same
resources
that
computational
jobs
had
had
access,
so
the
file
systems,
the
networks,
there's
also
a
need
for
scale,
there's
a
need
to
run
custom
software
and
most
the
time.
Services
like
this
need
to
be
need
to
be
persistent.
You
know
for
the
duration
of
the
project,
not
for
just
the
duration
of
of
jobs
or
certain.
You
know
a
number
of
runs.
A
In
fact
they
might
even
need
to
be
the
tool,
that's
scheduling
the
jobs
or
conducting
the
the
workflow
orchestration,
and
so
that
means
staying
up
when
HPC
is
down
and
being
available
on
the
web.
Where
do
you
put
these
things
like
this?
A
Like
databases
or
data
archives,
workflow
managers,
websites,
science,
gateways,
a
lot
of
projects
actually
need,
and
so
we
created
spin
so
that
we
would
be
able
to
provide
you
in
addition
to
the
compute
and
the
storage,
an
infrastructure
for
nurse
users
to
run
those
kinds
of
services
that
help
tie
their
projects
together.
So
a
lot
of
projects
need
more
than
just
HPC,
and
that
was
what
spin
was
designed
to
provide.
A
So
the
way
it
works
is
it's
a
system
where
you
can
deploy
your
own
science
gateways,
workflow
managers,
databases
or
other
kinds
of
Network
Services
that
are
either
used
by
your
computational
jobs
or
they're
used
by
your
user
Community,
or
that
help
to
disseminate
your
data.
There's
just
a
number
of
ways
that
this
could
be.
A
This
could
be
spun
and
the
the
infrastructure
is
built
on
the
Technologies
at
the
bottom,
Docker
containers,
the
Rancher
orchestration
system
and
kubernet,
the
kubernetes
scheduler
there's
a
couple
different
kubernetes
clusters
that
are
actually
part
of
part
of
spin,
in
our
instance
of
Rancher
and
and
together.
A
These
are
very
particular
kinds
of
services
for
the
type
of
science
that
these
projects
do
so
so
Desi
there,
the
first
one
there
they
do
nightly
Sky
surveys,
and
so
there
are
daily
routines
that
transfer
data
and
process
jobs
and
collect
results
so
that
they're
ready
for
the
next
night
and
a
lot
of
those
soft
a
lot
of
those
those
tools
are
built
in
spin
or
in
some
way,
and
it's
been
going
down
through
there.
A
He
has
to
dive
as
a
data
archive
and
it
takes
advantage
of
large
storage
systems
that
we
have
at
nurse
and
then
the
spin
flat
form
for
the
front
end
to
that
jgi
does
a
number
of
different
genomics
workflows
and
has
a
number
of
different
science
gateways.
Some
of
those
live
in
spin
LZ
is
a
dark
matter.
Detector.
A
That's
that's
done
a
lot
of
really
interesting
work
in
in
spin
to
to
be
able
to
allow
analysis
of
the
tech,
the
detectors
events
to
be
done
via
web
tools
that
are
hosted
in
spin
for
the
scientists
that
need
to
take
a
closer
look
at
events
that
come
from
that
detector
in
the
materials
project
has
also
made
use
of
spin
for
some
of
their
their
web-facing
tools
that
that
allow
for
molecular
stimulation
material
simulation.
A
So
these
are
just
some
examples
of
the
projects
that
that
we
know
about,
but
there's
dozens
of
projects
that
are
making
use
of
spin
for
services
like
like
these
that
go
alongside
their
their
HPC
work
at
a
very
high
level
and
I'll
give
you
a
demonstration
to
this,
but
a
very
high
level.
A
A
And
when
you
deploy
something
it
might
look
a
little
bit.
Something
like
this.
This
is
just
a
web
service,
but
it
can
have
a
front
end.
It
can
have
a
back
end.
It
can
have
a
database,
it
can
have
a
key
value
store
and
you
define
these
using
kubernetes
constructs
or
using
the
the
UI,
and
you
build
these
images
yourself
or
you
rely
on
ones
from
from
the
from
public
repositories
like
Docker
Hub.
A
But
you
get
to
control
how
the
composition
of
those
Services
actually
makes
up
your
your
application
and
the
the
job
of
the
Rancho
orchestration
system
is
to
figure
out
what
nodes
and
spend
to
run
those
on
how
to
mount
up
the
storage
that
you've
declared
and
to
keep
all
those
things
running
and
to
plumb
the
networks
that
connect
all
of
those
pieces
and
to
keep
those
private
from
other
people's
deployments.
A
So
you
have
a
lot
of
degrees
of
freedom
in
the
kinds
of
services
that
you
can
create,
but
web
services
that
are
built
like
this
are
are
pretty
pretty
common.
A
So
let
me
show
you
how
that
works,
I'm
going
to
deploy
an
app
in
five
minutes
or
actually
I'm
going
to
show
you
a
video
of
me
doing
that
and
the
app
that
I'm
going
to
deploy
is
actually
the
one
that
we
use
in
our
training
program.
So
we
have
a
workshop
about
every
two
months
and
part
of
that
is
building
a
live
thing.
It's
kind
of
a
canned
example,
but
it's
designed
to
sort
of
exhibit
a
lot
of
the
features
of
of
spin.
A
So
this
app
looks
like
this:
it's
python
based
it
uses
some
static
files
in
CFS.
It's
got
a
little
database
back
end
we're
going
to
build
it
from
the
bottom
up
with
the
database.
First
and
again,
I
really
want
to
stress
spin
is
not
a
server
service
for
deploying
things
that
look
just
like
this
spin
has
a
lot
of
flexibility,
so
this
is
only
one
example
of
the
kind
of
thing
that
you
could
deploy.
You
could
deploy
just
the
database.
A
Many
people
do
you
could
deploy
a
headless
API
rather
than
a
UI,
or
you
could
build
some
something.
That's
that's
completely
different,
and
so
you
have
a
lot
of
degrees
of
freedom
as
far
as
the
kinds
of
the
kind
of
microservices
that
you
can
assemble
based
on
what
your
project
needs.
A
A
Oh
here
we
go:
okay,
yeah,
sorry,
yeah.
Sorry,
just
I
have
a
lot
of
Windows.
That
sort
of
looks
similar.
So
let
me
make
sure
sorry
folks,
let
me
just
make
sure
this
is.
Oh
there
we
go.
We
all
important
optimize
for
video.
A
Okay,
here
we
go.
Okay,
so
can
y'all
see
that
okay,
all
right
so
I'm
going
to
start
this
and
then
I'm
just
going
to
describe
what's
happening.
So
this
is
the
Rancher
user
interface
and
I'm
going
to
start
right
off
with
deploying
a
database.
A
So
I
click
the
deploy
button,
and
this
is
where
you
start
specifying.
What's
in
the
workload,
so
this
workload
is
going
to
be
a
database
and
it's
going
to
use
the
MySQL
version
5
image
if
you're
familiar
with
kubernetes,
you
put
things
like
workloads
inside
of
a
namespace.
Now
this
image
is
designed
to
take
a
number
of
environment
variables
that
specify
the
name
of
the
database,
the
user
that
will
access
at
the
password
the
time
zone
and
you
specify
those
in
environment
variables,
foreign,
whether
you
deploy
it
in
spin
or
anyplace
else.
A
It's
just
how
that
image
is
customized
and
then
this
particular
image
requires
certain
privileges,
capabilities
in
order
to
run
correctly
and
in
spin.
This
is
an
important
aspect,
because
it
we
limit
the
number
of
capabilities
that
you
can
run,
but
MySQL
requires
just
a
handful
of
capabilities
because
it
starts
up
as
rude
and
then
it
changes
to
a
MySQL
user.
So
it
needs
to
be
able
to
do
things
like
that.
That
database
is
running
already.
A
A
It's
actually
in
the
nurse
registry
registry
nurse.gov
that
we
Supply
as
part
of
spin
and
it's
a
galaxy
image
application
that
our
colleague,
Roland
Thomas,
Built,
For,
This
Workshop,
now
notice
here,
I'm
specifying
the
same
password
that
I
specified
for
the
database,
so
the
web
app
will
be
able
to
connect
to
it
and
then
the
other
thing
I'm
doing
here
is
I'm
mounting
a
CFS
directory,
so
I'm
setting
these
settings.
A
The
volume
is
called
images
and
it
is
an
existing
directory
on
CFS
that
has
this
path,
so
that
could
be
one
of
your
own
projects,
path
and
I'm
inside
the
container,
where
it
runs
I'm
going
to
mount
it
on
slash
SRV
static,
and
these
are
static
images
from
from
Sky
survey
now
down
in
this
other
section
here,
where
I
set
the
permission
or
the
capabilities
for
the
database
I'm
going
to
set
them
for
the
web
service,
but
it
doesn't
really
need
any
special
capabilities
at
all.
A
It's
just
an
unprivileged
layer
and
it
started
so
so
now
I've
got
a
web
application
running
in
a
container
a
database
application
running
in
a
container
they're
talking
to
each
other
and
I'm.
Adding.
What's
called
an
Ingress
if
you're
familiar
with
kubernetes,
that's
basically
a
web
proxy
so
that
I
can
get
this
web
app
I
can
get
out
the
web
application
over
the
network,
the
host
name
for
that
is
based
on
the
cluster
name
and
the
namespace,
and
the
name
of
the
Ingress
that
that
DNS
name
is
created
automatically.
A
And
what
I'm
doing
here
is
saying
when
someone
comes
to
that
hostname
direct
the
traffic
to
the
app
workload
on
Port
5000,
where
it's
listening
and
then
there's
a
little
custom
setting
in
here
below
that.
Actually,
you
don't
need
to
put
any
more
when
the
demo
was
made.
I
did
so.
This
is
basically
creating
an
entry
in
a
reverse
proxy
at
the
network.
A
Parameter
has
been
that
allows
that
web
application
to
be
accessible
on
a
port
80
externally,
and
it
takes
a
little
bit
of
time
to
actually
create
that
Ingress
and
it
will
also
go
out
and
create
that
DNS
name
within
the
spend
on
nurse.org
subdomain
and
assign
it
an
an
IP
address
and
then
once
it's
ready,
you'll
be
able
to
click
it,
and
this
is
actually
accessing
the
application
and
the
database
underneath
that
image
came
off
of
CFS
and
some
of
this
metadata
here
is
actually
stored
in
the
database.
A
A
We
get
into
a
little
bit
more
detail
of
some
of
the
things,
some
of
the
tools
that
you
can
use
in
in
Rancher
and
kubernetes
to
do
a
little
bit
more
sophisticated
deployments,
but
that's
kind
of
a
brief
version
of
actually
starting
up
a
real
web
service
in
spin
in
kind
of
a
lightning
fast
deployment,
so
I
I
actually
hate
when
presenters
do
what
I
just
did
and
say
see.
A
You
can
build
it
in
five
minutes,
because
I
know
that
that's
not
always
the
real
story,
so
obviously
there's
there's
some
prep
work
that
leads
up
to
doing
a
deployment
that
that
quickly
and
so
I
wanted
to
talk
about
that
a
little
bit.
So
you
know
what
if
he
were
to
use
a
tool
like
spin.
A
You
know
here
are
some
of
the
things
that
would
sort
of
be
different
about
your
your
world
and
so
the
biggest
thing
about
this
example.
The
biggest
sort
of
sort
of
like
person
behind
the
curtain
is
that
that
image
with
that
python
application
was
already
created
and
that
database
already
populated
with
some
metadata.
So
you
would,
you
know,
spin,
isn't
canned
sets
of
applications,
but
so
you
need
to
be
able
to
build
a
container
image
that
has
that
flask
application
in
there,
but
you're
already
going
to
do
that
anyway.
A
Right
you're
already
going
to
develop
that
application
anyway,
so
so
in
an
environment
like
spin.
This
first
point
here
is:
you
know,
you're
going
to
do
your
development
you're
going
to
in
a
conventional
environment,
you
would
have
iterated
code
changes
and
done
testing
on
a
development
VM
and
in
a
kind
of
spin
environment
or
development.
Workflow
you'd
be
iterating,
those
same
code
changes
and
testing,
but
you'd
probably
be
testing
out
in
a
local
container
deployment.
A
You'd
run
something
like
Docker,
desktop
or
Rancher,
desktop,
build
that
container
and
make
sure
that
it
works
properly
and
then
in
a
conventional
environment.
Once
you
get
to
a
good
stopping
point,
you
commit
test
release,
you
know,
push
it
out
to
a
VM
or
whatever
in
the
spin
environment,
commit
test,
rebuild
the
image,
push
the
image
to
registry.nurse.gov
and
then
release
on
spin.
So
you
know
update
it
to
a
new
version
of
the
image
it
pulls
it
over
and
restarts
the
service.
So
a
little
bit
of
a
difference
in
the
flow
there.
A
A
So
just
the
coding
and
testing
and
releasing
process
may
be
not
that
different,
but
where
the
where
spin
really
shines,
is
your
ability
to
kind
of
change
things
that
would
be
much
more
difficult
in
a
conventional
environment,
particularly
a
managed
environment.
So,
let's
say:
if
you
want
to
upgrade
the
database
or
change
some
configuration
to
get
better
performance,
that's
going
to
be
a
help
desk
ticket.
You
know
in
the
conventional
environment,
but
in
spin
redeploy
the
database
workload.
A
If
you
want
a
different
version
of
the
database,
then
you
just
specify
the
version
that
you
need.
If
you
want
to
make
a
change
to
the
configuration,
then
you
just
pass
in
a
new
configuration
file
or
or
change
the
deployment
so
that
it
passes
a
different
argument
at
startup
and
you've
made
the
change.
This
is
a
database
that
you
own,
it's
not
one
that
you
share
with
someone
else.
A
You
know
a
lot
of
the
interaction
these
these
next
points
here
about
you
know
sshing
into
the
VM
tailing
logs.
You
know.
What's
going
on
with
the
service,
you
know,
access
to
those
things
are
a
little
bit
more
abstract
in
an
environment
like
Spence,
they'd
use
the
UI
or
the
CLI
to
look
at
logs,
so
you're,
not
interacting
with
something
that
you
log
into
you
interact
with
something
that's
running
in
a
kubernetes
environment.
A
You
know,
if
there's
trouble,
you
know
if
the
thing
goes
down,
you're
going
to
be
SSH
again
and
trying
to
figure
out.
What's
going
wrong
for
a
lot
of
different
failure
modes,
you
can
kind
of
rely
on
kubernetes
to
just
reschedule
the
crashed
web
application.
Kubernetes
kind
of
has
one
job
and
that's
keep
the
containers
running.
So
even
if
that
means
we're
starting
a
crashing
container
endlessly.
So
there
are
even
things
that
you
can
put
in
called
health
checks
that
can
do
continuous
monitoring.
A
So
those
are
pretty
powerful
tools
and
then
the
last
one
here
is
that
in
a
conventional
environment,
a
lot
of
documentation
is
needed
to
kind
of
capture
details
about
what
you've
done.
The
really
nice
thing
about
an
environment
like
spin,
that's
entirely
software
defined
and
declarative.
A
Is
that
there's
a
file
that
describes
everything
that
we
just
did,
and
so
you
actually
have
a
deployment
yaml
file
that
you
can
connect
that
you
can
actually
commit
into
repository
and
check
and
and
diff
and
roll
back
that
describes
your
entire
deployment,
so
am
I,
saying
you
never
have
to
write
documentation,
I,
don't
know,
I
mean
I,
guess
that
would
be
a
great
selling
point.
Probably
not,
but
the
point
is
you
know
something.
That's
actually
declarative
and
easy
to
share
and
track
is
a
huge
benefit
of
when
things
get
complex.
A
So
obviously
this
is
a
real
brief
overview
kind
of
a
teaser
I
tried
to
keep
tried
to
keep
it
kind
of
honest
but
you're.
If
you're
interested
in
using
spin,
like
I
said
we
have
workshops
regularly
throughout
the
year.
Those
are
instructor-led,
you
you'd
build
the
same
application
yourself
and
then
for
established
projects
where
people
want
to
join
colleagues
and
help
out
with
some
existing
deployments
in
spin.
A
We
have
kind
of
a
fast
track
there
subject
to
approval
where
you
can
work
through
the
exercises
yourself
and
kind
of
get
up
to
speed
with
your
with
your
colleagues,
and
we
also
have
office
hours,
you
can
schedule
drop
in
for
interactive
help,
so
you're
not
just
at
the
mercy
of
tickets
to
get
things
done.
So
that's
all
that
I
have.