►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Kubernetes is clearly the container orchestrator of choice for cloud-native stateless applications. And with StatefulSets and Persistent Volumes, it’s now possible to run stateful applications on Kubernetes. Tools like Kustomize, Helm, and Kubeflow help tackle some of the deployment complexity for stateful applications. However, running complex stateful applications for distributed AI, machine learning, and big data analytics on Kubernetes remains beyond the reach of most users.
Enter KubeDirector. KubeDirector is an open source Apache project that uses the standard Kubernetes custom resource functionality and API extensions to deploy and manage complex stateful scale-out application clusters. With KubeDirector, you can run complex stateful clusters for AI, machine learning, and big data analytics on Kubernetes without writing a single line of Go code.
This webinar will provide an overview of the KubeDirector architecture, show how to author the metadata and artifacts required for an example stateful application (e.g. with Spark, Jupyter, and Cassandra), and demonstrate the deployment and management of the cluster on Kubernetes using KubeDirector.
A
All
right
we're
gonna
go
ahead
and
get
started.
I'd
like
to
thank
everyone
who
is
joining
us
today.
Welcome
to
today's
the
ncf
webinar
deploying
complex
stateful
applications
on
kubernetes,
with
Cube
director,
I'm,
Caitlin,
Varner
marketing
manager
at
CNCs
and
I'll
be
helping
to
moderate
today's
webinar
I'd
also
like
to
welcome
today's
presenters,
Tom
Phelan
and
Joel
Baxter
from
new
data.
So
just
a
few
housekeeping
items
before
we
get
started
during
the
webinar,
you
are
not
able
to
talk
as
an
attendee,
so
there
is
a
Q&A
box
at
the
bottom
of
your
screen.
A
If
you
have
any
questions
throughout
the
webinar,
please
feel
free
to
drop
them
in
there
and
then
we'll
get
to
as
many
as
we
can.
At
the
end,
the
session
is
also
being
recorded
and
will
be
sent
out
afterwards
and
available
online,
along
with
a
link
to
the
presentation.
So
with
that
I'll
hand
it
over
to
Tom
to
kick
off
today's
presentation
thanks.
B
B
So
today
we'll
be
speaking
about
kubernetes
and,
on
the
first
part,
I'll
review
a
little
bit
talk
about
stateful
versus
stateless
applications,
we'll
talk
about
how
AI
and
ml
and
deep
learning
are
complex,
stateful
applications
and
how
that
has
some
challenges
on
kubernetes.
Well,
then,
don't
dig
into
what
what
cube
director
actually
is
a
deep
dive
of
the
technology
technology
itself,
a
demonstration
in
some
takeaways
at
the
end,
so
to
begin
with
I'm
young
I
know
this
audience
knows
a
lot
about
kubernetes,
but
just
kind
of
do
a
level
set.
B
B
A
physical
host.
Some
are
a
virtual
machine
in
the
public
cloud
top
use
cases
for
kubernetes
up
to
this
point
has
been
the
deployment
of
stateless
applications.
These
are
also
known
as
micro
services.
It's
a
still
in
the
evolving
platform
for
stateful
applications
that
we'll
be
talking
about
that
today
we
say
stateless
application.
What
we
mean
is
each
application
service
instance
is
identical.
So
each
container
running
an
instance
of
a
stateless
service
is
identical
to
any
other
instance
running
that
same
service.
B
They
don't
have
a
persistence
or
a
configuration
that
is
unique,
any
information
that
they
need
to
store
and
access
is
stored
remotely
and
when
we
say
remotely,
we
mean
that
the
persistent
storage
has
a
life
span
different
than
that
of
the
container.
If
the
container
were
to
exit
and
restart,
it
could
connect
to
the
same
persistent
storage,
the
remote
persistent
storage
without
loss
of
context.
B
Typically,
these
sorts
of
applications
are
referred
to
as
cattle,
because
each
instance
is
like
every
other
instance
of
that
application.
This
is,
in
contrast
to
stateful
applications
in
a
stateful
application.
There
is
some
intrinsic
difference
to
each
instance
of
the
application.
That's
instantiated
within
the
container.
It
may
be
as
simple
as
a
an
IP
address
or
may
be
more
complex,
and
this
critical
information
is
stored.
Locally
locally
means
it's
part
of
the
storage,
that's
associated
with
the
container
when
a
container
starts.
B
Such
applications
are
completely
referred
to
as
pets,
since
each
instance
is
different
and
the
loss
of
one
such
instance
is
a
loss.
Now
we
look
at
stateful
applications
such
as
AI
machine
learning,
deep
learning.
These
are
some
of
the
most
stateful
applications
we
have
in
the
enterprise
today.
What
about
all
these
applications
have
in
common?
They
require
large
amounts
of
data,
and
that's,
okay.
That
data
is
typically
remote.
It's
in
a
de
Lake
external
from
the
containers
themselves,
but
these
are
complex,
distributed,
processing
solutions.
B
They
are
not
written
into
cloud
native
architecture,
they
are
typically
deployed
as
collections
of
services
running
on
a
bare
metal
host
or
in
a
virtual
machine,
and
those
services
have
inter
dependencies.
They
are
not
a
cloud
native
architecture,
there's
no
micro
services
and
there
is
an
application
state.
B
So
these
are
some
of
the
most
complex
stateful
applications
in
existence
than
they
require
special
treatment
to
step
back
for
a
moment
when
we
say
we're
running
a
Hadoop
or
an
AI
or
a
machine
learning
cluster
like
spark
or
tensor
flow
in
containers,
what
we're
talking
about
is
deploying
it
the
same
way
it's
deployed
on
bare
metal.
This
is
not
re-architecting
or
rewriting
the
AI
or
ml
application.
What
I'm,
showing
here
are
four
boxes,
which
are
the
Primo
typical
deployment
strategies
for
Hadoop,
there's
a
resource
manager
as
a
collection
of
node
managers.
B
They
run
different
services
with
containers.
What
we're
talking
about
is
putting
each
of
those
nodes
into
a
container
or
potentially
a
pod
with
with
kubernetes,
and
this
entire
set
of
containers
what's
referred
to
as
a
cluster,
so
we're
going
forward
with
the
remainder
of
this
presentation.
This
is
what
we're
talking
about,
what
we're
saying:
we're
deploying
a
cluster
of
containers,
running
one
of
these
machine
learning,
artificial
intelligence
or
deep
learning.
Applications
with
kubernetes.
We
have
a
number
of
tools
and
building
blocks
to
work
from
these
are
called
objects.
B
Pods
are
a
collection
of
containers
that
are
deployed
on
an
individual
host.
We
have
a
concept
of
a
stateful
set.
Stateful
set
refers
to
a
set
of
features
associated
with
the
kubernetes
container
that
are
necessary
for
running
a
stateful
application.
They
order.
The
service
is
in
the
pod
that
are
brought
up.
They
preserve
some
state
of
the
container
itself.
That
should
be
done
with
the
use
of
persistent
volumes
and
each
application
that's
deployed.
B
Are
we
have
via
a
stateful
set
it's
done
via
an
operator
and
operators,
the
collection
of
actions
they
can
be
performed
on
a
per
application
basis?
Typically,
operators
are
constructed
from
the
use
of
a
custom
resource
definition.
You'll
also
hear
the
term
CRD
kubernetes
operators
are
a
basic
API,
is
way
of
packaging
deploying
and
managing
an
application.
Now
that
open
source
community
has
created
an
operator
framework,
it
makes
it
easier
to
develop
an
application,
specific
operator
and
finally,
there's
another
tool
on
top
of
that
called
the
operator
SDK.
B
These
are
layers
of
abstraction
which
make
it
easier
to
develop
an
application
specific
operator,
but
still
we
require
a
different
operator
for
each
application
to
be
deployed
on
kubernetes
and
with
the
plethora
of
AI
and
ml
application
is
in
the
marketplace
today.
This
would
be
quite
a
few
operators.
This
is
very
challenging
for
the
enterprise
to
do
so.
What
do
we
need
to
do?
There's
got
to
be
an
easier
way,
and
this
is
what
blue
data
has
done.
B
We've
used
our
knowledge,
we're
helping
the
community
with
an
easier
tool,
bluh
KS,
you
might
hear
that
term.
This
is
a
an
umbrella
Apache
project
started
by
a
blue
data,
but
now
open
to
the
open-source
community
focused
solely
on
bringing
enterprise
support
for
complex
stateful
applications
to
kubernetes
there'll,
be
a
series
of
open
source
projects.
In
this
case,
and
the
first
thing
would
roll
out
is
cube
director.
This
is
the
URL
for
the
cube
director
site
on
github.
C
Off
mute
now,
okay,
so
I'm
going
to
start
with
the
background
that
Tom
laid
there
and
dig
a
little
bit
more
into
the
design
of
cube
director,
the
specific
things
that
we're
doing
with
it
and
tempt
fate
somewhat
with
a
live
demo.
It's
a
lot
to
get
through,
but
I'll
try
to
leave
us
some
room
for
questions
at
the
end
as
well.
C
C
Comparison
of
features
you
know,
does
this
solution
do,
but
I
wanted
to
do
right
now,
but
from
our
perspective
as
developers
and
looking
for
you
know,
open
source
contributors
and
the
direction
of
the
project
in
the
future,
we're
also
interested
in
asking
what
does
the
design
enable
us
to
do
as
we
go
forward?
What
things
does
our
architecture
of
our
solution
make
easy
to
do
or
hard
to
do
and
there's
not
going
to
be
a
best
solution
for
all.
C
C
When
you
figure
out
that
you
need
to
deploy
that
application
and
on
the
other
end-
and
this
is
where,
for
example,
app
specific
operators
come
in
some
other
person,
who's
an
expert
in
the
application
and
in
kubernetes
has
put
together
a
solution
for
you
kind
of
as
a
black
box,
and
it
handles
this
application
very
well.
But
if
you
want
to
tweak
it
or
if
you
want
to
manage
other
applications,
then
it's
not
the
best
solution.
C
So
if
we're
going
to
aim
for
a
spot
between
these
two
extremes,
you
know
we
we
don't
want
to
just
use
the
low
level
make
our
end-users
use
the
low
level
kubernetes
api
eyes
directly.
So
we
need
to
build
some
abstractions
on
top
of
those
to
guide
these
abstractions,
you
know
to
pick
what
features
and
what
data
model
and
what
operations
we
support.
We
need
a
focus.
You
know
we
don't
want
to
go
all
the
way
towards
just
supporting
a
specific
application,
but
we
do
want
to
focus
on
this
domain
of
big
data.
C
So
coop
director
itself
is,
what's
called
a
custom
controller.
This
is
a
process
that
watches
the
kubernetes
api
for
resources
to
appear.
So
someone
else
uses
the
cuba
Nettie's
api
to
submit
a
resource
to
create
or
change
or
delete
a
resource
in
kubernetes
persistence,
store
and
then
the
controller
notices
that
that
has
happened
and
does
something,
and,
in
our
case,
we're
actually
watching,
among
other
things,
for
custom
resources
to
appear
so
we're
not
just
interested
in
observing
the
the
pods
and
the
services
and
things
like
that
that
are
the
sort
of
native
kubernetes
objects.
C
We've
also
provided
some
schemas
to
kubernetes,
to
define
some
custom
objects
that
our
coop
director
is
specific
and
we'll
talk
a
lot
more
about
those
in
just
a
minute,
and
these
are
essentially
the
requests
that
end-users
provide
to
kubernetes
that
are
requesting
cube
director
to
do
something
on
their
behalf.
And
so,
when
cube
director
sees
this
request,
it's
going
to
again
start
using
the
kubernetes
api
to
create
stateful
sets
cradle.
C
This
is
pretty
much
what
an
app
specific
operator
does.
So
the
way
cube
director
diverges
from
that.
Is
that
the
logic
for
determining
how
to
respond
to
these
requests?
There's
no
app
specific
semantics
coded
into
cube
director
itself
cube
director
works
with
data-driven
descriptions
of
application
types,
how
you
deploy
these
applications,
how
you
manage
them
after
they're
deployed,
and
then
it
uses
those
descriptions
which
can
be
provided
by
the
end
user.
They
don't
have
to
be
created
by
the
cube
director
team
or
anybody
outside
the
users
organization.
C
Those
descriptions
completely
Drive
cube
director
through
a
state
machine
of
deploying
then
managing
these
applications,
so
one
obvious
benefit
to
that
is
that
it
is
some
there's,
a
somewhat
natural
and
easier
process
for
bringing
new
applications
into
the
fold
of
being
deployable
and
manageable
on
kubernetes,
especially
if
you
have
an
existing
application
definition,
and
you
just
maybe
want
to
roll
to
a
new
version
of
that
application.
I
use
some
different
packages,
set
it
up
in
a
slightly
different
way.
C
It's
quite
easy
to
make
that
modification
in
these
metadata
files
that
describe
the
deployment,
but
if
you
want
to
just
bring
a
completely
new,
say,
open
source
application
and
that
maybe
hasn't
had
an
operator
written
for
it.
Yet
you
can
do
that
as
well.
You
can
create
a
new
set
of
metadata
and
a
container
image
for
that
application
and
give
it
to
cube
director
and
cube
director
will
know
how
to
manage
it.
C
There's
also
a
slightly
more
subtle
benefit
from
this,
which
is
that
there's
other
objects
and
features
that
cube
director
currently
and
especially
in
the
future,
will
support
which
are
cross
cluster
and
cross
application
features
things
like.
How
do
you
access
data
sources?
How
do
you
authenticate
users
inside
your
applications,
maybe
integrating
with
your
Enterprise
ad
server?
C
How
do
you
hook
clusters
together
into
processing
pipelines
and
where
make
clusters
where
up
other
objects
you
defined
once
these
features
are
implemented
in
cube
director,
then,
by
virtue
of
providing
the
metadata
for
a
new
application
type
to
cube
director,
it
participates
in
those
features
automatically
and
different
applications.
Even
from
different
developers
can
participate
in
these
features
together
without
any
knowledge
of
each
other.
C
A
C
Or
JSON
file
that
describes
hey,
you
know,
here's
the
container
image
to
use
you
know,
maybe
pull
an
image
from
docker
hub
or
from
your
own
internal
image,
repository
and
just
to
play
this
on
kubernetes
it
starts
up,
starts
listening
to
the
kubernetes
api
and
it's
basically
ready
to
go
at
that
point.
Then
an
end
user
who
wants
to
actually
deploy
a
virtual
cluster
comes
along
and
provides
this
custom
resource
that
I
mentioned
earlier.
This
specification
of
the
users
intentions.
C
So,
even
though
kubernetes
has
been
told
about
the
schema
for
these
resources,
it
doesn't
treat
them
as
having
any
implied
actions.
It's
just
a
document
that
kubernetes
is
managing.
So
the
you
use
a
standard,
kubernetes
api
client
to
create
this
resource.
Cube
director
then
notices
that
the
resource
has
been
created
and
that
it
represents
a
request
to
create,
say
maybe
a
Hadoop
cluster
or
spark
cluster,
and
then
it
enters
what
we
call
the
reconciliation
loop.
C
The
reconciliation
loop
is
is
basically
just
working
its
way
through
a
state
machine
where
it
instantiates
some
objects
in
kubernetes.
It
takes
some
actions
on
the
kubernetes
api.
Maybe
it
creates
a
stateful
set.
Maybe
it
creates
a
service
and
then
it
goes
away,
and
then
it
comes
back
in
a
little
while
to
check
to
see.
C
Has
this
been
completed,
if
so,
it'll
go
to
the
next
part
of
the
state
machine,
so
maybe
it
has
to
set
up
a
pod
wait
until
the
containers
are
ready
in
that
pod
then
configure
the
application
inside
the
pod,
so
there's
a
set
of
steps
that
has
to
work
through
and
meanwhile
it's
updating
this
resource
that
are
provided
to
say
what
the
status
of
this
process
is
and
then
once
cube
director
is
finished
working
its
way
through
the
reconciliation
loop.
The
cluster
is
up
and
ready
to
go.
C
By
using
this
custom
controller
approach
and
also
by
taking
a
data-driven
approach
to
implementing
support
for
different
applications,
one
of
the
good
things
about
the
coop
director
design
is
that
it
supports
it
doesn't
require,
but
it
supports
the
separation
of
concerns
among
different
user
types.
So,
if
you
have
people
in
your
organization
that
have
different
knowledge
sets,
maybe
different
privileges
to
do
different
things
by
virtue
of
the
fact
that
everything
we
do
runs
through
the
kubernetes
API
uses
custom
resources
and
is
controllable
by
all
the
authentication
and
authorization
methods
that
kubernetes
provides.
C
So,
for
example,
you
can
say
these
users
in
this
namespace
have
privilege
to
create
or
modify
these
custom
resources
and
these
users
in
this
other
namespace
had
these
different
privileges.
You
can
have
this
separation
of
concerns.
You
can
have
application
experts
who
are
entirely
responsible
for
figuring
out
how
to
create
the
container
image
and
the
metadata
for
the
application,
but
they
don't
have
to
have
the
privilege
to
install
this
application
in
kubernetes.
They
don't
have
to
understand
how
kubernetes
operators
work.
They
don't
have
to
write
go
code.
C
They
just
have
to
know
how
to
write
to
this
metadata
format
and
make
a
container
image.
Then
they
can
supply
that
to
an
administrator
either
like
an
IT
kind
of
administrator
or
maybe
a
project
manager
who
can
take
this
metadata
and
import
it
into
coop
director
and
determine
which
users
have
privileged
to
make
instances
of
this
application.
C
They
just
know
that
here's
a
kind
of
application
that's
available
to
me
I,
can
submit
a
request
to
kubernetes
to
instantiate
that
application
that
application-
and
you
know,
maybe
there's
various
knobs
and
controls
available
to
the
end-user
like
selecting
the
amounts
of
resources
it's
going
to
consume
or
selecting
which
services
it's
going
to
run.
But
whoever
designs,
the
application
for
an
appointment
at
this
particular
site
can
dial
that
complexity
up
and
down
they
can
decide
how
much
complexity
is
actually
put
onto
the
end
user
and
from
our
perspective,
as
Cube
director.
We
just
want
this.
C
This
custom
resource
from
the
end-user,
this
request
to
instantiate
an
application
to
come
to
the
kubernetes
api
and
then
we'll
tackle
it.
We
don't
really
care
where
it
comes
from.
This
is
probably
a
very
rich
area
of
development
for
putting
a
nice
user
interface.
On
top
of
this
part
of
the
workflow.
C
So
thinking
about
that
that
design
that
architecture
for
cube
director,
let's
call
back
a
minute
to
the
motivation
slide,
you
know,
why
would
we
want
to
have
a
solution
architect
alight
cube
director,
as
opposed
to
some
of
these
other
things
that
are
available
to
us?
Currently
and
again,
these
other
things
are:
are
very
good
solutions
for
many
use
cases,
but
coop
director
we're
trying
to
make
it
strong
for
some
particular
use
cases
in
this
domain
of
applications.
C
So
one
thing
as
I
mentioned
is
that
we
support
these
distinctions
between
user
roles.
You
don't
have
to
have
a
power
user
who
knows
a
lot
about
kubernetes
and
the
application
or
someone
who
has
privileges
to
do
very
high
level,
privilege
things
in
kubernetes,
as
well
as
deploy
the
clusters
for
the
data
scientists.
C
Before
we
react
to
them,
we
get
the
benefit
of
all
the
authentication
and
authorization
methods
that
kubernetes
provides,
and
we
don't
have
to
worry
about
having
maybe
two
different
sources
of
authority
about
users
that
might
you
know,
have
to
be
kept
in
sync
with
each
other.
If
you're
using
a
solution
like
helm,
two
with
tiller
that
sits
out
in
front
of
the
API
where
you
ought
indicate
to
that
other
solution
first
and
then
it
sort
of
acts
like
a
super
user
to
kubernetes.
C
C
Also
because
we
have
this
process
that
runs,
that
is
long-lived
sitting
in
kubernetes,
watching
these
deployments
after
they're
created
we're
not
just
intervening
at
deployment
time
after
to
the
cluster
comes
up,
we
can
watch
it
to
respond
to
further
user
requests
to
scale
it
up
and
down.
We
can
respond
to
auto
scale
policies
to
scale
the
cluster.
According
to
load.
We
can
auto
remediate
when
there's
errors.
You
know
when
a
when
a
stateful
node
goes
down,
there
may
be
applications
specific
things
that
have
to
be
done
to
bring
it
back
up
again.
C
All
these
things
require
having
a
long-lived
manager
process
there
to
watch
it,
because
these
are
semantics
above
and
beyond.
What
kubernetes
can
understand
or
can
be
can
be
made
to
deal
with,
and
so
again,
if
you're,
using
just
a
a
purely
client-side
sort
of
template
driven
approach,
you
don't
have
this
long-lived
manager
process
to
take
care
of
these
things.
C
Now
all
those
things
on
the
previous
slide,
all
those
benefits
also
apply
to
application.
Specific
operators
there's
a
couple
of
other
ways
in
which
Cube
director
is
different
from
application,
specific
operators
and
we've
really
either
explicitly
or
implicitly
hit
this
point
a
lot
up
until
now,
and
that's
because
we
are
data
driven
by
metadata
by
documents
that
describe
application
types,
it's
easier
to
import
new
applications
to
be
supported
by
this
solution,
and
you
can
have
common
features
that
apply
across
multiple
application
types.
C
This
orange
cloud
in
the
middle
of
the
of
the
picture
we
may
get
some
other
slides
later
on,
that
use
this
same
sort
of
iconography.
That's
that
represents
just
all
the
resources
that
have
been
created
in
kubernetes
site,
so
all
the
documents,
all
the
Native
communities
resources,
as
well
as
the
custom
resources
that
we've
defined.
C
When
someone
asks
a
cube
director
to
create
a
virtual
cluster.
By
submitting
this
custom
resource
that
asks
to
instantiate
an
application
type
cube
director
is
going
to
start
creating
objects
in
kubernetes,
and
one
of
the
important
things
too
that
you'll
see
when
we,
when
we
get
into
actual
cube
director
activities,
is
the
concept
of
rolls
these
squares
inside
the
deployed
virtual
cluster
there.
C
Those
are
each
a
container
acting
as
a
virtual
host
like
Tom
mentioned
our
approach
to
to
deploying
these
stateful
applications
kind
of
a
model
that
we're
able
to
fit
on
a
large
group
of
applications
is
to
treat
each
container
as
an
entire
virtual
host.
It
has
a
file
system,
it
has
a
network
identity,
it
has
services
running
in
it
on
the
standard
service,
ports,
multiple
services
inside
each
container
and
within
this
virtual
cluster.
C
Among
these
virtual
hosts,
we
use
the
role
concept
to
distinguish
different
cluster
members,
different
virtual
hosts
that
have
different
personalities
so
for
a
particular
role
in
the
cluster
that
determines
which
container
images
used
to
deploy
the
container.
It
determines
how
many,
how
much
CPU
and
memory
and
GPU
and
other
resources
are
used
by
the
container
it
determines
which
services
are
running
inside
this
container.
C
So
that's
kind
of
our
leverage
point
for
dealing
with
the
fact
that
different
members
inside
the
cluster
may
have
different
personalities,
and
in
this
particular
case,
we've
deployed
a
cluster
that
has
role
with
one
member
and
maybe
that's
some
sort
of
controller
or
manager
host
inside
the
cluster.
And
then
we
have
another
role
with
three
members
running
different
services
than
the
previous
role.
Maybe
these
are
some
sort
of
scale
out
worker
task,
processing,
sort
of
role.
It's
a
common
common
set
up
in
this
sort
of
cluster.
C
So
when
the
container
comes
up,
the
application
metadata
may
tell
us
that
we
need
to
pull
in
these
extra
configuration
artifacts.
These
can
be
scripts
that
respond
to
lifecycle
events
that
cube
directory
triggers.
These
can
be
things
like
application.
Configuration
file
templates
that
the
scripts
use
to
help
set
up
the
application.
It
could
be
software
licenses,
it
could
be
host
certificates.
There
could
be
you
know
anything
you
want
to
put
in
this.
It's
just
a
tar
file
that
we
inject
into
the
application
at
launch
time.
C
So,
finally,
before
we
get
into
some
demonstrations
and
details,
the
customer
resources
are
really
the
heart
of
what
we
work
with
and
what
any
custom
controller
works
with
and
there's
a
few
custom
resources
already
present.
If
you
check
out
the
the
cube
director
github
repo
today
and
we're
talking
internally
and
with
with
some
other
people
about
future
features
that
are
going
to
use
many
more
custom
resources.
Basically,
anytime,
you
have
some
user
provided
information
that
you
want
to
be
persisted.
C
That's
going
to
be
represented
in
a
custom
resource,
but
to
keep
this
this
particular
talk
focused
and
to
try
to
keep
us
on
track
for
allowing
some
questions
here.
We're
just
going
to
talk
about
the
two
main
custom
resources
that
cube
director
will
react
to
when
it
sees
those
created
or
changed
in
the
kubernetes
resource
cloud.
The
first
one
is
cube
director
cluster,
and
this
is
what
represents
that
incoming
request
by
an
end
user
by
or
made
by
a
data
scientist
that
says,
I
want
to
instantiate
this
application
type.
C
So
that's
the
cube
director
cluster
resource,
the
cube
director
cluster
itself,
references,
a
different
type
of
ceará
called
the
cube
director
app
and
the
cube
director
app.
Is
that
application
type
description
that
application
metadata
that
I've
been
talking
about
so
typically,
when
you're
setting
up
the
cube
director,
you
would
provide
it
with
some
initial
set
of
cube
director
apps,
some
initial
catalog
of
application
types
that
can
respond
to,
and
maybe
those
will
get
revised
or
or
added
to
over
time.
But
they
don't
they
don't
churn
a
lot.
C
So
to
talk
about
the
details
here
and
to
to
demonstrate
cube
director
in
action,
we're
going
to
use
as
a
framework
some
of
these
user
roles-
we've
been
talking
about
this
also
kind
of
help
demonstrate
if
you
want
to
set
up
and
test
out,
try
out
cube
director
yourself.
This
is
kind
of
the
set
of
steps
you
would
have
to
go
through
personally
to
get
it
to
get
it
running,
so
the
first
user
role
we're
going
to
go
through
really
briefly,
because
there's
not
not
a
whole
lot
to
that
is
the
administration
role.
C
C
You
need
to
set
up
a
service
account
for
cube
director
to
use
to
interact
with
the
kubernetes
api
and
then
deploy
cube
director
itself
and
cube
director
can
could
technically
run
anywhere,
but
it's
easy
just
to
deploy
it
into
a
pod
in
the
same
kubernetes
cluster
that
it's
going
to
be
managing,
and
then
there
may
be
some
configuration
to
do
of
cube
director
itself.
There's
a
few
different
global
settings.
You
can
tweak
like
which
class
of
persistent
storage
should
be
used
for
the
applications,
but
that's
about
it.
C
C
So,
let's
flip
over
to
my
terminal
here
so
here,
I've
got
just
our
cube
director,
repo
from
github,
actually
just
checked
out.
I've
checked
it
out
from
the
the
latest
version
tag,
which
is
version
0.2
and
I've
got
cube.
Ctl
also
installed
here
and
my
cube.
Ctl
client
is
configured
to
talk
to
a
kubernetes
cluster.
C
This
group,
nice
cluster,
happens
to
be
in
Google
cloud,
but
we've
run
a
cube
Director
on
Amazon,
on
digitalocean
on
local
hosts
kubernetes
installed
on
bare
metal,
there's
a
couple
of
requirements
that
our
documentation
discusses
that
if
you
want
to
try
it
out
really
quickly
and
easily
yourself,
I
would
really
recommend
using
one
of
these
cloud
services.
So
anyway,
I've
got
cube.
Ctl
set
up
here
to
talk
to
my
kubernetes
cluster
and
I'm,
going
to
use
this
make
file
target
that
we
have.
C
This
is
a
shortcut
just
to
help
do
all
those
administrative
tasks
that
I
mentioned
in
the
slide.
So
this
sets
up
the
service
account
for
cube
director.
It
creates
those
custom
resource
definitions
that
I
mentioned
cube
director
cluster
cube
director
app
and
then
cube.
Director
config
is
actually
a
custom
resource
used
to
manage
those
global
settings
that
I
talked
about.
C
Then
it
creates
that
sends
a
deployment
to
accumulate
A's.
That
mentions
the
container
image
for
cube
director,
and
we
just
have
a
container
image
out
on
docker
hub
and
my
my
kubernetes
deployment
has
access
over
the
Internet
to
docker
hub,
so
it
just
pulls
it
down
and
cube.
Director
is
now
running
if
we
look
at
the
pod,
where
cube
director
has
been
deployed.
C
C
So
there's
basically
two
possibly
three
things
that
you
need
to
provide
to
cube
director
so
that
it
knows
how
to
deploy
and
manage
an
application.
First
of
all,
you
need
a
container
image
that
has
the
packages
already
installed
on
it.
You
want
to
have
this
container
image
as
ready
to
go
as
possible.
You
know
you
don't
want
to
be
downloading
packages
after
it
comes
up
just
have
it
pre-configured
as
much
as
possible
so
that,
just
once
the
container
is
launched,
you
can
grab
the
final
bits
of
runtime.
C
C
These,
don't
necessarily
have
to
be
all
services
that
are
running
inside
these
virtual
hosts
in
the
application,
but
anything
that
the
external
world
will
be
interested
in
either
an
end-user
or
maybe
other
clusters,
or
you
know
any
any
service
that
we
might
have
to
you
know
hook
up
to
let,
in
this
example,
I'm
using
Google,
so
I
want
to
hook
the
service
up
to
one
of
Google's
load,
balancers,
so
it'll
be
externally
accessible
from
my
network,
so
any
service
that
fits
any
of
those
criteria.
It
needs
to
be
described
in
this
metadata.
C
C
C
C
They
could
also
be
overridden
at
the
role
level,
although
we
don't
do
that
in
this
example
here.
So
in
the
next
page.
Here
we
define
just
two
roles
and
there's
various
things
that
could
be
defined
here,
along
with
overriding
container
image.
You
could
specify
things
like
minimum
resources
and
a
few
other
things,
but
the
most
important
bits
are
giving
the
role
an
ID
and
specifying
how
many
of
the
cluster
members
can
take
up
this
role.
C
So
in
this
case
we
have
a
controller
role
and
there's
always
going
to
be
one
only
one
of
these
in
the
cluster
and
we
have
a
worker
role
and
there
can
be
zero
or
more
of
those.
So
those
are
valid
requests
by
the
user.
The
user
could
not
ask
for
two
controllers,
but
they
could
ask
for
0
or
10
workers.
We
don't
care,
then
here's
the
section
that
describes
the
services
are
available
and
particularly
which
port
each
service
is
running
on
and
finally,
a
config
section
that
ties
all
this
together.
C
Among
the
defined
roles
which
roles
are
we
actually
going
to
deploy
by
default
and
also
which
services
then
are
going
to
be
present
on
each
of
those
roles.
So,
when
cube
director
gets
a
request
to
instantiate
this
application,
it'll
use
this
information
to
configure
the
network
access
to
the
different
virtual
hosts
in
the
cluster,
and
it
will
also
send
in
information
about
the
selected
role.
C
C
C
There
could
be
anything
you
want
here
that
you
want
to
inject
into
the
container
after
it's
been
launched,
but
the
thing
that
we
most
typically
have
in
this
tar
file
is
a
set
of
scripts
that
react
to
lifecycle
events
that
cube
director
will
trigger.
So
these
are
life
cycle.
Events
like
hey,
you
have
just
appeared
as
a
new
container
do
whatever
you
need
to
do
on
startup.
They
can
also
be
life
cycle.
Events
like
another
container,
has
appeared
inside
your
cluster.
In
this
role.
Do
you
need
to
react
to
that?
C
C
To
add
its
fqd
and
their
config
files
and
this
sort
of
application,
specific
semantics
is
handled
by
these
lifecycle,
hooks
that
cube
director
triggers,
and
that
runs
a
script
that
the
application
expert
has
provided
when
they
were
importing
this
application
and
these
scripts,
especially
the
the
initial
scripts
to
set
things
up,
can
get
pretty
interesting,
but
I
I
think
I'll
just
have
to
direct
you
to
our
example:
applications
and
packages,
and
our
are
a
github
repo.
If
you
want
to
learn
more
about
that,
but
that's
the
gist
of
what
happens
with
this
package.
C
Because
time
is
is
moving
along,
let's
go
ahead
and
actually
do
that
here.
Let's
go
into
the
deploy
example,
catalog
directory-
and
so
we
have
these
example-
application
type
definitions
here,
I'm
going
to
send
this
one
to
kubernetes,
and
then
it
will
exist
there
to
be
referenceable
by
cube
director
when
someone
asks
to
instantiate
this
kind
of
application.
C
So
finally,
we
come
to
the
end
user
and
the
end
users.
Job
is
by
design
pretty
simple.
We
want
to
give
the
tools
as
much
as
possible
to
let
the
administrators
and
the
application
experts
bacon
all
the
complexity
into
a
policy
that
the
the
end
user
can
just
say,
give
me
one
of
those
and
not
have
to
like
decide
what
container
image
to
use
or
how
many
resources
to
apply
in
this
case.
This
is
a
completely
valid
request
document
for
instantiate,
again
spark
cluster.
C
There
are
other
things
you
could
add
to
this
request
to
make
it
more
complex,
but
this
this
sort
of
request
here
will
work,
and
all
this
is
saying
is
that
for
the
controller
role
I
I
do
want
to
limit
the
memory
to
four
gigs
I'm,
adding
that
into
the
request
and
then
for
the
worker
role
I
want
to.
So
that's
all
I'm
really
saying
is
the
end
user.
C
Sort
of
do
a
by
play
of
what
cube
director
is
doing
when
we
try
this
out
in
practice.
So
again
in
the
deployed
directory.
We
have
this
example:
clusters
subdirectory,
which
has
a
bunch
of
example,
instantiation
requests
that
you
could
use
to
throw
a
cube
director
I'm
going
to
do.
This
is
a
slightly
simpler
version
of
the
example
spark
cluster.
C
C
So
if
we
start
here
at
this
line,
I'm
highlighting,
we
can
see
that
it's
notice,
there's
a
new
document.
That's
requesting
an
application
of
this
spark
type
that
we
defined
earlier,
and
this
document
says
we
have
a
controller
in
worker
role
and
we
validated
those,
as
you
know,
valid
roles
for
the
spark
type
and
we've
created
a
stateful
set
to
represent
each
of
these
roles
and
set
its
replica
account
for
the
stateful
set
to
the
number
of
members
that
have
been
requested
for
each
of
the
roles.
C
Then
we
have
gone
into
the
containers
and
injected
a
just
JSON
file
that
completely
describes
the
request
and
kind
of
the
current
state
of
the
cluster,
the
fqdn
x'
the
services,
anything
that
a
setup
script.
One
of
these
lifecycle
reaction
scripts
inside
the
cluster
might
need
to
know
in
order
to
do
its
job,
so
we've
injected
that
file
into
the
cluster
and
then
because
there
was
this
configuration
artifacts
package
mentioned
in
the
application
metadata.
We've
pulled
that
down
over
the
web
injected
that
into
each
container
and
run
the
script
for
new
container.
C
Your
starting
up
is
a
new
container.
Do
whatever
you
have
to
do,
and
so
that
script
has
the
like
spark
specific
knowledge
in
there
that
are
our
spark
application
expert
provided
to
take
this
cluster
configuration
information
and
do
final
setup,
whatever
is
necessary
to
make
sure
that
the
the
spark
master
knows
what
it's
doing
and
the
spark
workers
are
connected
to
spark
master.
C
So
that's
what
you
see
where
it
says
running
app,
config
setup
and
running
app
config
in
these
pods,
it's
creating
and
then,
as
part
of
this
state
machine,
this
reconciliation
loop.
It
is
trigger
this
configuration.
Then
it
goes
away
and
then
it
comes
back
later
and
says:
are
you
done
with
the
configuration
yet
in
this
case,
no,
the
initial
configuration
is
still
ongoing.
C
One
other
thing
I
probably
should
mention
here
is
our
our
requests
mentioned
some
persistent
storage
for
the
controller
role,
so
it
asked
for
40
gigs
of
storage.
One
of
the
things
that
cube
director
will
do
in
this
case
is
it
will
bring
up
a
persistent
volume
attached
to
this
particular
container
and,
depending
on
the
application
metadata,
that
application
type
will
specify
which
directories
in
the
file
system
of
the
container
image
actually
need
to
be
persisted,
and
so
we
will
copy
the
initial
state
of
those
directories
like
maybe
it's
/opt
or
maybe
slash
data.
C
You
know
you
don't
care
about
all
your
binaries
and
/bin
those
aren't
going
to
change.
If
your
container
goes
down
and
comes
back
up,
you
can
just
start
fresh
from
the
container
image,
but
there
may
be
other
directories
that
you
really
do
care
about
any
changes
and
persisting
them,
and
so
we
will
copy
the
initial
state
of
those
directories
over
into
the
persistent
volume
and
mount
those
back
at
their
usual
location
inside
the
container
and
then
any
changes.
C
C
Whoops
and
let
me
actually
do
describe
rather
than
get
because
if
you
do
a
describe
you'll
get
the
events
that
have
been
posted
as
well,
so
this
is
the
the
type
of
object
that
we
created
and
there's
only
one
of
them.
So
it's
only
going
to
list
this
one
object,
and
so
this
is
the
this
is
our
custom
resource.
This
is
the
request
we
sent
to
kubernetes.
C
So
if
we
look
at
the
spec,
this
spec
is
essentially
the
document
we
passed
in
it
just
persisted
what
we
asked
for
you'll
notice
that
there's
a
few
other
things
have
been
populated
that
we're
not
included
in
our
initial
spec
there's
like
the
storage
class
name,
there's
the
service
type.
These
are
some
defaults
that
got
filled
in
because
we
didn't
specify
something
different.
So
the
spec
part
is
not
that
interesting.
That's
not
going
to
change
until
we
change
it.
C
The
maybe
the
more
interesting
part
is
the
status
section
and
the
status
section
really
communicates
two
things.
One
is
the
overall
state
of
the
cluster
like.
Is
it
in
some
sort
of
transitional
change
or
is
it
ready
to
go
and
the
other
is
all
the
different
objects
that
have
been
created
on
behalf
of
this
custom
resource?
So,
for
example,
the
controller
here
has
a
persistent
volume
claim
if
we
wanted
to
inspect
that,
we
could
see
now
what
what
kubernetes
object
was
set
up.
C
There
I'm
gonna
skip
ahead
kind
of
to
the
punchline
here
in
the
interest
of
time
and
look
at
this
service
for
the
controller.
So
a
service
object
in
kubernetes
is
a
representation
of
a
collection
of
ports
and
we
use
it
to
say
for
this
particular
virtual
host.
What
are
all
the
externally
accessible
service
ports
that
that
we
need
to
deal
with
so
we've
defined
like
port
22
for
SSH
port
8080
for
the
SPARC
master
web
UI?
Actually,
let's,
let's
look
at
that
one.
C
So
this
service
is
telling
us
that
for
our
controller
node,
if
we
were
to
go
to-
and
this
is
the
virtual
controller
of
our
virtual
spark
cluster-
that
we
just
created-
if
we
were
to
go
to
this
IP
of
a
Google
load
balancer
and
look
at
port
8080,
we
should
see
our
spark
master
web
UI.
So
let's
go
ahead
and
try
that
out
so
yeah
there
it
is.
You
can
see
that
we've
got
it's
reporting,
three
workers.
C
C
And
remember:
we're
just
sending
a
change
in
a
document
to
kubernetes
and
then
cube
director
is
responding
to
that
change
and
interpreting
our
change
document
as
a
request
to
change
the
cluster.
So
we've
increased
the
worker
count
from
two
to
four
I'm
going
to
apply
that
change,
and
then
we
can.
Let's
describe
our
object
again
to
look
at
the
events,
so
we
can
see
on
the
events
that
are
being
posted
to
our
cluster
object.
It
notices
that
we're
expanding
the
role
from
two
to
four,
and
so
it's
reconfigured,
the
stateful
set
to
have
more
replicas.
C
It's
waiting
for
those
pods
to
come
up
when
those
pods
do
come
up.
It's
not
only
going
to
trigger
the
lifecycle
event
for
new
container
inside
those
pods.
It's
going
to
go
into
all
the
existing
containers
and
trigger
the
lifecycle
event
for
there's
been
a
new
member
added
to
your
cluster,
and
so
if
this
application
has
a
script
that
reacts
to
that,
it
can
do
that
as
well
at
this
time.
C
C
So
let
me
skip
past
these
guys.
These
are
images
and
also
with
a
lot
more
descriptive
text
on
our
wiki
I.
Think
we're
pretty
much
done
at
this
point.
I
really
I
hope.
We've
Illustrated
these
these
first
two
key
takeaways
and
I,
really
encourage
people
to
you
know,
have
a
look
at
our
github
repo,
not
only
the
wiki
but
also
the
other
documentation.
There,
there's
I
think
a
pretty
good
QuickStart
document
for
how
you
can
just
close
the
repo
and
instantly
run
Kubb
director,
I
I,
did
see
out
of
the
corner
of
my
eye.
C
Another
question
about
whether
we
can
use
cube
director
with
AWS
or
GCP,
and
the
answer
is
almost
certainly
yes,
you
know,
we've
tried
it
with
AWS
Google,
digitalocean
I
think
some
other
cloud
service
there's
just
a
few
requirements
for
what
sort
of
kubernetes
you
want
to
deploy
to
and
once
those
are
met
we
should
work
anywhere,
but
I
think
I'll
I'll
call
it.
This
point
wait
see
if
tom
has
any
closing
remarks
he
wants
to
make,
and
then
we
can
squeeze
in
a
few
more
questions.
B
B
C
So
not
currently
there.
This
is
the
category
of
things
where
we
want
to
make
sure
we
have
the
architecture
to
address
this
going
forwards.
So
you
know,
kubernetes
has
the
idea
of
health
checks
that
you
can
define
for
containers
and
that
may
be
sufficient
for
a
lot
of
cases,
but
if
you
want,
you
know,
if
you
need
a
more
detailed
set
of
examinations
that
need
to
be
done
in
order
to
determine
exactly.
C
What's
you
know,
what's
up
and
what's
down
and
what's
going
wrong,
then
you
know
that
again
is
some
application-specific
semantics
that
would
have
to
be
provided
in
this
package,
that
of
importing
the
application
to
cube
director
and
I.
Think
the
way
we're
talking
about
approaching
this
a
lot
of
the
concepts
that
we're
doing
with
coop
director
are
things
that
we
kind
of
test
vetted
with
our
with
our
application
called
epic,
our
management
platform,
where
we
do
a
lot
of
this
same
sort
of
data-driven
application
deployment
and
one
of
the
things
we
have
there
is.
C
We
have
an
agent
that
we
can
insert
into
the
clusters
that
can
watchdog
the
defined
services.
You
know
these
services
that
are
presented
in
the
metadata
and
reports
service
status.
All
the
way
back
into
the
the
management
for
what
we're
talking
about
doing
for
kube
director
is
not
injecting
agents
into
the
containers,
but
rather
having
like
a
sidecar
container
inside
the
same
pod.
That
can
do
all
sorts
of
things
really.
C
B
All
right,
thanks
Joe,
all
the
other
thing
I
want
to
talk
about
a
little
bit
is
the
use
of
existing
application
operators.
So
does
this
require
the
use
of
cube
directory
just
wire
a
complete
rewrite?
My
answer
is
no,
you
can
you
can
take
the
existing
application
operators
and
wrap
them
with
the
additional
functionality
of
cube
director?
Is
that
correct?
Oh
yeah,.
C
B
Okay,
good
and
from
the
final
question
before
you
turn
it
back
to
to
Kaitlin,
is
there
was
a
question
around
K
needed?
Okay,
so
I
can
ativ
is
a
kind
of
a
tool
that
Google
is
released.
They
allows
stateless
applications
to
inherit
some
common
functionality.
Is
there
anything
equivalent
tonight
in
in
cube
director.
B
C
An
auto
scale
feature
that
we're
working
on
in
epic
right
now
that
we,
like
many
things,
we'll
be
thinking
about
how
to
represent
that
in
cube
director
that
allows
you
to
specify
you
know:
load
based
policies
for
particular
roles
for
hosts
for
particular
hosts
and
particular
roles
within
a
cluster,
and
so
it's
a
kind
of
an
application
agnostic
way
of
specifying
an
auto
scale
policy
that
interacts
with
the
way
the
application
is
modeled
to
Kubb
director.
So
you
know,
for
example,
and
that's
for
the
regime
if
you've
modeled
the
application
correctly.
C
If
you've
given
cube
director
of
valid
application
metadata,
then
it
automatically
gets
the
benefit
of
being
able
to
work
with
an
auto
scale.
Policy
like
that,
so
again,
I'm
kind
of
hand
waving
that,
because
this
is
this
is
future
work.
But
it's
it's
definitely
the
sort
of
thing
that
we're
trying
to
enable
through
this
approach
is,
you
know,
get
all
the
goodness
of
having
this
these
data-driven
ways
of
talking
about
these
applications
and
that
lets
us
build.
You.
C
B
A
Yeah
we
are
coming
right
on
the
top
of
the
hour
here,
so
thank
you
so
much
tom
and
Joel
for
the
great
presentation
and
everybody
who
joined
us
today
and
asked
those
great
questions.
The
webinar,
recording
and
slides
will
be
online
later
today.
So
look
out
for
those
and
look
forward
to
seeing
you
at
a
future.
Cnc
webinar
have
a
great
day.
Everyone.