►
From YouTube: Laying the Foundation: Real World Kubernetes Deployment Patterns - Josh Rosso & Craig Tracey, VMware
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Laying the Foundation: Real World Kubernetes Deployment Patterns - Josh Rosso & Craig Tracey, VMware
Enterprise deployments often find themselves gridlocked in debates concerning the "correct" approach for deploying Kubernetes. Should we use immutable images or a complex configuration management solution? What about leveraging one of the managed service offerings? Or even one of the opinionated open source projects? The possibilities are endless but can also be dizzying. In this talk, Josh and Craig provide an overview of both the successful (and sometimes not so successful) deployment patterns they have encountered across dozens of production Kubernetes users in nearly every industry vertical. They will identify the critical decision making processes that will enable "Day 2" operations, empower stakeholders, and ultimately yield successful Kubernetes outcomes.
https://sched.co/MPYg
A
All
right,
hello,
everybody
welcome
back
to
another
session,
so
you
are
in
laying
the
foundation
real-world
kubernetes
deployments.
My
name
is
Josh
and
I
am
a
employee
of
VMware.
I
came
over
from
the
heavy
acquisition
prior
to
VMware.
I
worked
at
core
OS
for
about
a
year
and
a
half's
we've
been
doing
kubernetes
stuff
for
quite
some
time.
My
focus
around
kubernetes
is
working
in
the
field
and
working
with
a
lot
of
different
customers
who
have
a
lot
of
different
deployment
strategies
around
how
they're
going
to
roll
their
kubernetes.
A
So
this
talk
is
going
to
be
really
really
interesting.
Kind
of
I
guess,
retrospective
of
a
lot
of
the
collective
experience
that
Craig
and
I
have
had
working
with
many
of
these
customers.
Some
of
the
really
great
patterns
we've
seen
in
some
of
the
areas
where
we've
seen
folks
cut
themselves
a
bit
on
some
of
the
sharp
edges
that
we
see
in
the
kubernetes
ecosystem.
So
again,
I'm
Josh-
and
this
is
Craig
yep.
B
So
there's
a
lot
of
opinions
about
kubernetes
kubernetes.
Some
of
the
paintings
are
some
one
hysterical,
I
love,
Paul's,
ARS
tweets
all
the
time.
If
you
don't
know
Paul's
or
you
should
definitely
follow
him,
get
some
great
takes
hot
takes
on
technology,
especially
kubernetes,
but
what
you'll
find
if
you
look
for
opinions
about
kubernetes,
is
that
a
lot
of
them
are
somewhat
critical
of
how
hard
kubernetes
is
and
I
think.
A
lot
of
them
are
expressing
some
frustration
with
how
it
is
how
hard
it
is
to
actually
get
kubernetes
deployed
and
operable.
B
But
I
think
said
when
all
said
and
done,
carré's
is
actually
not
that
hard.
There
are
ways
that
you
can
look
at
the
patterns
from
successful
customers
and,
as
Josh
pointed
out
some
of
the
less
successful
cases
in
this
talk.
We're
going
to
try
to
give
you
some
examples
of
the
way
most
customers
approached.
B
Kubernetes,
where
they
start
to
trip
up
and
what
types
of
problems
they
have,
we
won't
go
super
deep
technically,
although
at
the
end
of
the
we
did
set
aside
some
time
for
questions,
if
you
want
to
go
deep
on
technical
stuff,
we
can
go
deep
there.
We
only
have
35
minutes,
so
we
want
to
hit
the
high
points.
B
The
first
thing
that
customers
struggle
with
is:
where
are
we
going
to
put
this
kubernetes
cluster
or
their
clusters?
There
are
a
variety
of
choices,
as
you
may
know,
and
we'll
walk
through
each
one,
these
and
kind
of
talk
through
some
of
the
advantages
or
disadvantages
of
each.
The
first
obviously
is
magic.
Uber
Nettie's
here
we're
talking
about
the
AKS
is
the
e
KS
is
the
g
KS
of
the
world,
and
these
these
platforms
are
really
great.
B
They
provide
a
easy
turnkey
solution
for
customers
to
get
up
and
going
with
kubernetes
it's
in
a
managed
environment,
but
this
comes
at
a
bit
of
a
price
right,
so
you
don't
typically
have
access
to.
Maybe
all
the
things
that
you
need
if
you
have
a
compliant
workflow
or
a
compliant
application
that
has
to
like,
adhere
to
specific
security
constraints.
You
may
not
be
able
to
deploy
here.
So
an
example
of
this
would
be.
When
you
get
to
a
lot
of
these
manage
kubernetes
platforms,
you
don't
have
access
to
the
control
plane.
B
Maybe
you
need
access
to
Etsy,
D
or
access
to
audit
logs.
That's
not
something!
You
typically
get
with
a
managed
kubernetes,
offering
the
second
area
that
you
might
explore
something
like
a
distribution.
So
there's
things
like
PKS
or
open
shift,
and
what
this
requires
is
that
you
have
a
little
bit
more
of
a
team.
You
know
perhaps
a
couple
of
operators
that
can
can
help
manage
the
distribution
and
they
give
you
a
little
bit
more
control.
B
Is
you
know
once
you,
if
you're
constrained
within
tightly
within
the
box,
that
a
distribution
offers,
then
things
are
great
as
soon
as
you
start
to
stray
a
little
bit
outside
of
those
realm
outside
of
that
realm,
things
may
start
to
break
down
for
you
and
things
that
you
may
require
may
not
be
supported
and,
of
course,
the
third
area
that
this
is
the
area.
We're
gonna
focus
on
in
this
talk
is
rolling
your
own.
What
does
it
take
to
roll
your
own?
Obviously
it's
going
to
take
more.
B
You
know,
person
power
to
make
that
kubernetes
cluster
happen.
You're
gonna
have
to
have
deep
insights
in
how
the
community
is
moving
like
where
the
which
directions
they're
moving
what-what
is
in
a
cluster
of
life
cycle.
Upgrade
strategy.
Look
like
these
sorts
of
things
it.
It
is
not
as
hard
as
most
people
make
it
out
to
be,
but
there
are
things
that
you're
going
to
need
to
know.
A
So
talking
a
little
bit
about
some
of
the
missteps
that
we
sometimes
see
with
our
customers
when
working
with
these
different
kubernetes
deployment
models,
one
is
a
lot
of
times
we
go
out
to
customers
and
they
are
very
focused
on
going
all-in
with
managed
kubernetes
now
manager.
Kubernetes
is
extremely
awesome,
especially
conceptually
who
here
is
very
excited
to
not
manage
their
control
plane
at
some
point
in
the
future.
Johanns
whoo
yeah.
We
are
psyched
right.
However.
Sometimes
we
get
so
all-in
with
this.
A
We
don't
really
take
on,
or
are
honest
with
ourselves,
about,
the
cons
that
some
of
these
services
do.
Have.
You
know
an
example,
as
far
as
costs
that
are
associated
with
someone
else,
managing
your
cluster
I,
don't
necessarily
mean
monetary
costs.
I
mean
some
of
the
things
you're
gonna
have
to
take
by
not
having
access
to
perhaps
the
control
plane.
A
It's
kind
of
like
a
Roomba
right,
like
delegating
your
vacuuming
to
Roomba,
is
awesome,
but
if
it
snags
your
toilet
paper
and
drags
it
all
around
your
house,
you
need
to
think.
Like
am
I
cool
with
my
toilet
paper
being
dragged
all
around
my
house,
maybe
I
could
raise
my
toilet
paper
a
little
bit
higher.
Maybe
my
toilet
paper
is
not
a
concern
for
me
right.
So
it's
just
those
things.
You
have
to
be
honest
about,
there's
trade-offs
with
everything.
Of
course.
A
We
all
know
this
super
well,
but
this
is
included
in
managed
services
and
we
really
need
to
consider
these
types
of
things
and
then
another
thing
we
run
into
folks
cutting
themselves
on
quite
a
bit
as
far
as
the
sharp
edges
is
having
a
one
size
fits
all
tool.
That
does
everything
from
start
to
finish
right,
so
I
click,
this
box,
I
hit
OK
and
then
BOOM.
A
My
infrastructure
is
there
everything's,
configured
and
kubernetes
is
installed
and
running
and
if
you're,
using
like
an
enterprise,
fully
baked
offering
that's
great
because
you
have
someone
to
support
it
and
so
on.
But
when
you
take
on
one
of
these
one-size-fits-all
installers
in
the
community,
I
think
there's
definitely
some
some
considerations
you
kind
of
have
to
make
right
one
is
that
the
infrastructure
that
you're
using
is
probably
different.
A
If
you
look
to
the
left
and
to
the
right
of
you
assuming
you're
not
sitting
next
year,
coworker,
they
probably
very
different
setup
than
you
currently
do
right
and
trying
to
make
these
installers
fit
that
model
of
our
very
unique
infrastructure.
It's
not
that
easy
to
do,
and
then
another
thing
that
folks
don't
oftentimes.
Consider
is:
are
you
comfortable
with
supporting
this
thing
again?
A
If
you
have
a
vendor
backing
you,
that's
awesome,
you
can
go
to
them
if
something's
wrong
or
you
need
to
get
it
fixed,
it's
great,
but
if
using
an
open-source
offering
do
you
treat
that
tool
like
a
big
black
box
with
a
couple
knobs
in
front
of
it
right
and
if
you
were
to
go
home
right
now
and
turn
some
of
those
knobs?
Do
you
know
what
those
knobs
actually
impact
and
another
thing
that
we
run
into
quite
a
bit
when
you
don't
actually
understand
the
pipeline
that
builds
your
clusters?
A
Is
you
can
get
yourself
in
really
weird
situations
when
you
know
crap
hits
the
fan
right,
like
a
run,
see
vulnerability
comes
out,
and
now,
all
of
a
sudden,
you
have
to
roll
updates
throughout
your
entire
cluster,
but
you're
really
not
sure
how
the
update
structure
works.
You
just
kind
of
run
it
in
the
past.
It's
been
okay,
like
it's
kind
of
sort
of
work,
but
if
something
breaks
in
that
big
black
box,
big
machine,
that's
updating
your
nodes,
you
might
be
in
a
bit
of
a
sticky
situation.
A
I
mean
sometimes,
we
actually
advocate
for
our
customers
to
consider
smaller,
more
purpose-built,
tooling
right
where
they
understand
the
config
management.
Perhaps
they
understand
the
infrastructure.
Provisioning
know,
there's
trade-offs
that,
because
guess
who
has
to
own
that
you
right?
But
if
you
know
how
it
works,
you
can
better
enable
yourself
to
handle
those
intense
situations
should
they
come.
B
B
The
first
thing
we
work
with
customers
on
developing
is
a
plan
for
how
they
get
to
what
their
ideal
state
is,
and
a
lot
of
customers
will
approach
us
and
say,
like
they
get
very
locked
into
these.
These
design
decisions,
upfront,
saying,
okay,
well,
I'm
thinking
about
ingress
and
I've,
decided
that
this
is
the
right
ingress
controller.
B
For
me,
the
light
the
nice
thing
about
kubernetes
is
you
don't
have
to
do
that
for
the
most
part,
you
can
decide
what's
right
for
you
today
and
what's
right
for
you
down
the
road,
so
oftentimes
the
world
with
customers-
and
they
say
you
know
today,
nginx
ingress
controller
will
work
fine
for
us
and
that's
great
that
gets
you
to
production
sooner.
You
may
find
that
down
the
road,
that's
something
like
contour
or
one
of
the
other
ingress
controllers
would
be
better
suited
for
you,
kubernetes,
because
it's
declarative.
B
We
can
declare
a
state
and
say:
okay
today,
our
English
controller
is
nginx
and
then
six
months,
eight
months,
12
months
down
the
road.
It's
going
to
be
something
different,
so
don't
get
hung
up
on
the
the
details
of
you.
What
your
actual
deployment
looks,
like
the
other
thing
that
folks
will
tend
to
do
when
they
approach
a
kubernetes
deployment,
is
like
okay,
we're
doing
this
kubernetes
thing:
everything's
green
field,
we're
gonna,
start
from
scratch,
we're
gonna,
deploy,
spinnaker
or
whatever
new
tools
out
there
today,
and
that
just
typically
does
not
work.
B
What
you
want
to
do
is
find
the
integration
points,
with
the
tools
that
you
have
today.
So
let's
say
you
have
Jenkins,
that's
a
perfect
tool
use
Jenkins
to
deploy.
The
application
is
to
the
currents
cluster
in
a
way
that
makes
sense
for
your
users
and
speaking
of
users
enable
your
stakeholders,
you
wouldn't
a
Matt,
you
wouldn't
believe
the
number
of
times
we
work
with
customers
where
they
forget
about
the
the
end
user.
B
So,
if,
depending
on
the
organization
you
work
with,
you
may
have
a
small
DevOps
team,
but
you
may
also
work
in
a
large
enterprise.
It
has
separation
of
concerns,
so
you
may
have
a
group
of
operators
that
are
responsible
for
the
uptime
of
the
application.
This
is
a
pretty
typical
pattern
and
I
work
with
customers.
B
Aggregation
is
like
how
do
I
do
role
based
access
control
against
logs
these
sorts
of
things
and
think
are
critical
to
your
deployment,
because
over
at
the
end
of
the
day,
the
number
of
stakeholders
who
are
bought
and
tacones
will
significantly
affect
your
success
rate
with
tip
on
kubernetes.
In
your
organization.
A
So
this
is
this
is
interesting
right,
it's
like
you,
go
to
a
car
dealership
or
something,
and
you
look
at
one
package
of
car
and
then
you
just
kind
of
talk
yourself
into
the
next
package
of
car
in
the
next
package
of
car.
When
reality
is
you
just
needed
something
to
commute
you
to
minutes
every
day
you
up
with
this?
You
know
$80,000
crazy
vehicle
right.
A
lot
of
our
customers
fall
into
this
trap.
A
You
know
one
of
the
really
common
ones
that
we've
seen
at
least
over
the
last
year
is
the
idea
of
whether
to
use
a
service
mesh
right.
So
all
of
you
in
this
room
right
who
have
run
micro
micro
services
or
are
thinking
about
it,
you
can
conceptualize
the
problems
that
service
meshes
solve.
It
absolutely
makes
complete
sense.
In
fact,
it's
fantastic!
Why
would
you
not
want
to
service
a
mesh?
You'd
have
to
be
crazy
to
not
use
one
right.
A
However,
like
do
you
actually
have
the
problem
it's
solving
for
today,
or
are
you
on
the
cusp
of
actually
needing
to
solve
that
problem?
A
particular
customer
that
we
worked
with
decided?
You
know
our
kubernetes
isn't
quite
operationalized.
We
don't
really
understand
it
that
well,
but
we
can
totally
tell
that
we're
gonna
need
service
mesh.
A
So,
let's
just
add
this
on
now
and
then
spent
the
next
six
months
figuring
out
that
not
only
was
the
technology,
they
chose
not
quite
ready
because
it
was
still
in
the
early
days,
but
also
they
added
so
much
complexity
to
their
system
to
solve
problems
that
they
didn't
necessarily
have
to
Craig's
point.
They
weren't
solving
business
value
immediately
and
they
created
a
fragile
system
that
was
delaying
the
deployment
to
production
of
their
kubernetes
infrastructure.
A
So
it's
really
thinking
about
you
know
what
problems
do
I
need
to
solve
what
problems
do
I
have
and
that
kind
of
leads
into
the
next
one,
which
is
perfect,
is
the
enemy
of
done
so
another
show
of
hands.
Who
here
is
in
the
planning
stage
of
kubernetes
in
their
organization
and
is
thinking
about
rolling
kubernetes
out
to
do
production
at
some
point
in
the
future,
all
right,
those
that
you
have
your
hands
up?
Please
please,
please
don't
try
to
boil
the
ocean
and
build
the
most
perfect
thing
from
start
to
finish
right.
A
We
run
into
this
all
the
time.
All
the
time
think
about
how
can
I
add
business
value
early
on
getting
a
kubernetes
cluster
up
and
running,
even
if
it's
slightly
naive,
it's
so
valuable
to
get
some
operational
experience
around
how
it
runs
if
you're
new
to
kubernetes
welcome,
if
very
glad
to
have
you,
but
you
have
so
much
to
learn
right
and
don't
just
waste
all
of
your
time.
A
Cycling
through
cycling
through
perfection,
perfection
perfection
only
to
realize
that
there's
a
bunch
of
problems
that
you're
gonna
need
to
solve
for
that
you
could
have
learned
about.
Had
you
gotten
an
MVP
like
thing
up
and
running,
maybe
just
a
couple
workloads
rather
than
trying
to
shift
your
entire
organization
over
to
this
brand-new
beautiful,
perfect
platform,
because
chances
are
you're.
Gonna
you're
gonna
predict
a
couple
things
wrong.
If
you
go
all
in
like
that,
so.
B
We
talked
previously
about
some.
You
know
the
nice
thing
about
currents.
Is
that
it's
composable
you
can.
You
can
declare
state
and
have
multiple
technologies
running
in
parallel.
There's
some
areas
where
you're
not
this
is
not
true
right.
So
there's
some
things.
You're
gonna
want
to
think
of
think
about
upfront
you're
going
to
want
to
get
these
things
right.
The
first
is
container
networking.
You
should
choose
a
container
networking
plugin,
that's
right
for
you.
There
are
various
options
out
there.
So
you
look
at
something
like
calico
nice
thing
about
calico.
Is
it
provides
linux
basic?
B
You
know
normal
linux
networking
it
scales
pretty
wide,
but
it
maybe
doesn't
come
out
with
coming
with
some
other
features
like,
for
instance.
If
you
need
something
like
hard
multi-tenancy,
that's
an
area
where
something
like
maybe
anise
X
would
shine.
You
may
have
a
requirement
or
something
like
multicast.
A
lot
of
the
C&I
providers
do
not
provide
support
for
multicast.
You
may
need
also
something
like
hardware
hardware,
offloading
of
NICs
and
again
seeing
that
mostly
nice
pro
scene.
I
providers
do
not
do
that.
B
The
second
one
is
persistent
storage
and
you
would
be
shocked
at
the
number
of
times
we
approach
an
engagement.
This
is
an
on-premises
engagement
and
customers
have
not
thought
about
storage.
I,
think
it
stems
from
the
fact
that
custom,
the
customers
that
we
engage
with
are
typically
trying
to
deploy
an
application.
They've
elected
to
take
on
kubernetes
and
part
of
the
reason,
I
think,
is
that
they
don't
want
to
talk
to
a
central
idea
about
getting
storage,
so
they
don't
think
about
storage.
B
So
we
go
into
these
engagements
and
all
us
in
there
becomes
an
immediate
need
for
storage
and
then
we're
talking
about
things
like
rook
or
open
EBS,
and
how
do
these
things
scale?
How
these
things
perform
so
definitely
give
a
thought
to
persistent
storage
before
you
approach
an
on-premises
deployment
of
criminales
and
the
third
area
would
be
connectivity,
so
they
have
their
applications.
They
deploy
these
applications,
let's
say
they're
even
stateless
and
they
go
to
create
a
service
type
of
load
balancer
or
they
deploy
a
an
application
that
plums
through
an
ingress
controller.
B
A
Hopefully,
you
have
similar
experience
that
folks
are
very,
very
quick
in
this
community
in
this
industry,
to
wake
up
in
the
morning,
see
a
new
project
on
github
and
be
ready
to
bet
their
entire
business
on
it
right.
That's
a
really
really
rough
model.
We've
seen
people
try
it
and
sometimes-
and
if
your
team
is,
you
know,
extremely
extremely
capable
engineers
who
understand
the
project
that
can
work
but
you're,
taking
on
a
lot
of
risk
right.
A
So
when
you
see
that
new
CNI
plugin
come
out,
when
you
see
those
new
things,
I'm
not
saying
don't
use
them,
you
know
keep
track
them.
There
there's
a
lot
of
really
interesting,
especially
in
CNI,
there's
a
lot
of
interesting
stuff
going
on
with
EBP
and
all
this
stuff
right,
but
there's
a
lot
of
projects
that
have
been
around
for
really
long
time
that
have
been
doing
the
job
super
super.
Well
right
and
again,
this
is
your
bedrock.
A
Do
you
want
tons
of
instability
or
unknowns
in
your
bedrock
for
a
couple
new
features
that
you
think
are
really
cool,
so
you've
got
a
balance
that
out
when
you're
thinking
about
this
stuff,
and
then
lastly-
and
we
probably
all
know
this
super
well-
open-source
is
certainly
not
free
right.
It
requires
diligence
so
a
lot
of
times
we're
trying
to
help
our
customers
navigate
whether
they
should
adopt
an
open-source
project.
I'm
sure
a
lot
of
you
go
through
the
same
thing
in
your
organization.
Right,
like
I
need
to
do.
A
I
am
authentication,
AWS,
do
I
use
Khayyam
or
do
I
use
cube.
I
am
what
are
the
differences?
What
is
the
project?
Health
and
while
it
might
seem
kind
of
silly
genuinely
gauging
the
community
health
of
something
that
you're
trying
to
take
on
is
an
extremely
valuable
tactic.
Right
going
on
the
github
page-
and
you
know
obviously
stars
are
all
you'll-
need
a
check
right,
but
yeah
thanks
so
yeah,
so
stars
are
really
important,
because
it'll
show
some
subtraction
in
the
project,
but
more
realistically,
if
you
go
into
issues
right
are
the
maintainer
z'?
A
Is
the
community
responsive,
helpful
right?
You
know
our
our
pool,
requests,
getting
reviewed
and
merged
and
considered?
Is
there
guidelines
for
how
to
contribute,
or
is
it
just
to
read
me
with
like
some
go
code
and
who
knows
what's
gonna
happen
right?
You
really
want
to
look
into
this
stuff,
because
again,
these
are
technologies
that
you're
potentially
betting,
your
business
on,
and
it's
really
valuable
to
have
a
gauge
for
what
level
of
risk
you're
gonna
take
on
with
the
health
of
that
project
over
time.
A
B
Right
at
the
end
of
day,
you
own
your
uptime
right,
so
the
other
area
that
people
often
overlook
is
the
security
model
around
what
criminate,
what
their
equipments
deployment
will
look
like,
and
when
we
were
happy,
we
kind
of
came
up
with
this
idea
of
multi
team
versus
multi
tenant
I.
Think
it's
a
really
powerful
one.
The
way,
the
the
way
that
we
think
about
it
is
multi
team
would
be
for
perhaps
like
the
same
organization.
You
have
a
certain
level
level
of
trust
between
members
of
your
same
organization
versus
multi-tenant,
where
you
are
taking.
B
You
know
some
random
people
from
the
internet,
placing
the
moniker
brands
cluster
together.
There
are
different
design
constraints
you
want
to
think
about
when
it
comes
to
security
for
sure
and
a
lot
of
people.
A
lot
of
folks
overlook
this
point,
especially
with
regard
to
things
like
how
I
perform
builds
on
my
cluster.
Let's
say
I'm
using
my
closer
for
performing
some
some
docker
image
builds.
This
is
something
you
definitely
want
to
take.
B
A
look
at
by
and
large
communities
does
not
have
a
a
hard
multi-tenancy
design,
and
you
can
certainly
enable
those
sorts
of
things,
but
doesn't
come
out
of
the
box.
The
other
thing
the
next
thing
would
be
consistent,
authentication
and
authorization.
You
would
probably
not
be
surprised
at
how
many
folks
we
engage
with
that.
B
They've
used
something
like
cube,
ATM
or
another
tool
that
spun
up
their
cluster
and
then
spits
out
this
x.509
certificate,
and
then
you
know
that
person
then
goes
in
males
into
the
rest
of
the
team
for
how
they're
going
to
access
this
cluster.
That's
the
wrong
way
to
do
it.
Use
something
like
open
up.
Ids
are
Oh
a
DC
and
connect
your
existing
user
Federation
systems
to
kubernetes.
B
It's
actually
really
simple,
and
it
enables
all
sorts
of
other
things
that
will
relate
to
policies
these
in
their
next
area,
that
people
forget
they
go
off
and
they
deploy
their
kubernetes
cluster
and
number
of
times.
We
see
people
that
forget
about
resource
quotas
or
pot
security
policies
is
like
pretty
much
every
single
cluster.
B
These
are
the
things
that
are
both
going
to
secure
your
cluster
as
well
as
give
you
the
levels
of
utilization
that
you
want
in
your
cluster
and
we'll
talk
about
that
in
a
little
bit
couple
further
slides
and
the
final
one
will
be
backup
and
restore
EDF.
Do
we
built
this
tool
called
arc
now
known
as
Valero?
It's
a
great
tool
for
both
making
sure
that
the
applications
that
you
deploy
to
kubernetes
can
be
moved
to
different
clusters.
B
If
you
encounter
a
situation
that
you,
you
know
a
disaster
situation
where
you
have
to
move
clusters,
it
can
also
be
used
for
things
like
CI,
CD,
workflows
or
just
basically
xeroxing
one
namespace
or
a
collection
of
resources
from
one
cluster
to
another.
So
it's
a
I
would
definitely
take
a
look
at
if
you
haven't
already
it's
a
very
powerful
tool
that
we
have
added
to
the
community.
A
Alright,
so
missteps
that
we
see
in
security-
and
this
is
definitely
one
of
the
the
highest
friction
points
quite
frankly
that
we
run
into
with
our
folks.
So
a
lot
of
times
there's
this
desire
when
thinking
about
kubernetes
as
a
piece
of
your
platform
for
orchestration
right
is
to
have
this
one
mega
cluster.
A
That
rules
them
all
right,
and
you
know
when
we,
when
we
start
out,
especially
those
of
you
who
are
just
getting
started
on
this
journey
right,
you're,
gonna,
start
out
and
you're
gonna
start
onboarding
workloads,
but
then
these
workload
requirements
are
gonna
come
on
that
are
going
to
require
some
things.
You
perhaps
didn't
think
about
like
when
someone
says
all
right,
I
got
a
workload
it
needs
to
be.
You
know
it
needs
to
be
conforming
to
PCI.
A
You
know
make
me
a
namespace,
that's
a
PCI
namespace
right
and
if
you
try
to
engineer
this
like
it
is
really
hard
to
do
right,
not
to
mention,
if
there's
some
other
obscure
requirement
that
you
may
or
may
not
want
to
introduce
to
the
entire
cluster
for
one
workload.
But
you
don't
want
to
necessarily
say
absolutely
no
kubernetes
stay
on
the
mainframe
forever
right,
but
you
don't
want
it
to
kind
of
impact,
your
entire,
your
entire
initiative,
so
thinking
more
about.
A
When
does
it
actually
just
make
sense
to
start
stamping
out
clusters
for
our
folks
and
with
manage
kubernetes
service
our
offerings?
Hopefully
this
will
become
even
easier,
but
when
you
have
these
more
edge
cases
or
maybe
security
requirements,
it's
not
necessarily
a
bad
model
to
stamp
out
a
more
secure
cluster.
That
has
other
considerations.
A
I
mean
you've
been
doing
this
forever
right,
like
if
you're
a
bank,
you
segregate
the
info,
the
stuff
that
requires
additional
info
SEC
stuff
outside
of
the
the
more
general
stuff
right
to
ensure
that
you
don't
lose
agility
and
the
more
general
stuff.
Perhaps,
and
then
this
is
one
that
it's
you'd
be
surprised
how
much
this
happens,
but
not
getting
your
security
team
involved
early
right
show
of
hands
who
would
say
they
work
primarily
in
the
security
realm
at
their
work.
A
Okay,
those
of
you
who
raised
your
hands-
thank
you
for
being
here
and
we're
so
psyched
to
have
more
security
focus
people
in
the
kubernetes
community,
but
this
is
something
that
we
find
really
interesting
because
a
lot
of
security
teams
that
are
out
there.
They
don't
really
know
that
much
about
kubernetes.
Sometimes
they
don't
know
anything
about
containers
for
the
most
part
right
we're
starting
to
see
that
shift
change.
But
you
have
to
keep
in
mind
that
this
isn't
just
a
journey
for
you
right.
This
is
a
journey
for
your
security
team.
A
This
is
a
journey
for
your
apps
team.
You
have
to
bring
everyone
along
with
you
and
make
sure
that
you're
keeping
people
up
to
speed
and
honestly,
if
you're,
the
operator
of
these
clusters
or
the
person
architecting
them
you're
gonna
have
a
lot
of
Education
on
your
hands
right
and
that
especially
goes
with
security.
You
want
to
make
sure
that
it's
not
the
11th
hour
and
Security's
going
Oh.
What
do
you
mean?
Containers
can
potentially
run
his
route
right.
A
A
Ok
with
the
events
that
are
currently
unfolding
right,
so
you
can
see
how
this
can
be
really
really
messy
over
time
and
in
show
hands
who's
tried
to
add
pod
security
policies
to
a
running
cluster
before
all
right,
I'm,
so
sorry
wow,
that
is,
that
is
arduous,
to
say
the
least:
who's
tried
to
add
network
policies.
After
the
fact
when
there
was
no
network
policies
in
the
cluster
wow,
that's
crazy.
A
So
we
know
that
when
you
change
the
security
posture,
when
you
change
how
your
cluster
works
from
a
security
standpoint,
it
causes
a
lot
of
headache
to
many
developers
now.
There's
ways
to
ease
this
a
little
bit.
But,
let's
be
honest,
like
the
last
thing
you
want
to
do
is:
insert
a
pod,
secure,
sorry,
a
network
policy
in
a
namespace
and
all
of
a
sudden
that
allow
all
rule
that
your
developers
were
used
to
in
vs.
to
a
deny
all
and
then
only
things
that
were
explicitly
allowed
are
allowed
right.
A
That's
a
big
model
change
you
don't
let
them
to
be
submitting
their
containers.
They've
always
been
submitting
and
all
of
a
sudden,
it
says
sorry
but
you're
running
his
route
and
then
they're
blocked
all
of
a
sudden
right
and
they're.
Looking
for
a
new
nginx
image,
because
the
nginx
image
runs
its
route,
you
know
you
can
see
the
headaches
that
this
causes,
so
moral
of
the
story
bring
security
along
with
you
during
this
ride.
During
this
transformation.
B
And
then
we
have
to
work
with
customers
about
how
they're
gonna
scale
out
their
clusters.
This
often
times
comes
early
in
the
conversation.
Typically
in
the
sales
cycle,
you
know
we
have
customers
who
come
to
us
and
we've
had
customers
that
say
we're
gonna
build
the
mm,
no
bearnaise
cluster
and
of
course
you
can
look
at
the
community
benchmarks
for
the
kubernetes
clusters
and
how
big
how
big
they
would
scale
I
can
promise
you
that
2,000
is
an
inappropriate
number
for
scaling
out
Coburn's
clusters.
B
The
the
nice
number
that
we
like
to
work
with
I
think
is
around
five
hundred
nodes
and
at
five
hundred
nodes.
That's
where
you're
gonna
start
to
run
into
issues
around
having
to
actually
tune
the
components
of
kubernetes.
So
if
you
don't
want
to
start
to
tune
coumarins,
which
can
be
an
arduous
process,
I'd
say
around
500
is.
The
note
is
the
is
a
number
of
notes,
but
you
want
to
be
in
the
multi
cluster
mindset
anyway
right.
So
we
talked
about
features
before
that:
you're,
not
gonna
change
things
like
the
CNI
networking.
B
B
Then,
of
course,
when
we
talk
about
multi
cluster,
the
question
always
arises
around
Federation.
How
am
I
going
to
have
a
very
seamless
integration
with
my
app
deployment?
How
am
I
going
to
say
you
know
this
application
needs
to
be
deployed
to
us,
east
and
us
west
for
redundancy
sake.
By
and
large,
we
would
advise
against
using
generic
tools
for
Federation.
These.
These
tools
are
generic.
They
don't
fit
your.
B
They
probably
don't
fit
your
specific
business
use
case,
and
there
are
other
ways
to
do
it
so
implement
things
like
Jenkins,
workflows
or
pipelines
to
get
your
applications
to
the
places
that
you
want
them
to
be.
With
the
business
rules
that
you
you
can
apply,
we
had
a
project
called
well
a
cig
called
cig.
Federation,
cig
Federation
was
actually
disbanded
for
a
cig,
multi
cluster
and
there's
a
reason
for
that.
This
problem
is
really
really
hard
solve
it.
B
With
the
tools
that
you
have
in
your
house
in-house
today,
and
then
we
talked
about
before
about
resource
policies,
things
especially
around
quota,
a
lot
of
people
approach
kubernetes
for
the
with
the
desire
that
they
want
to
increase
utilization
of
their
infrastructure.
So
maybe
previously
you
were
scaling
your
application
at
the
ec2
instance
layer.
Now
you
can
scale
with
a
container.
But
to
do
this
you
have
to
implement
resource
quote
resource
quotas
right.
You
need
to
make
sure
that
one
consumer
of
your
cluster
is
not
overtaking
the
cluster.
A
Cool
and
this
one's
pretty
short
and
sweet,
so
one
of
the
missteps
we
see
kind
of
going
along
with
what
we're
just
talking
about
with
stamping
out
clusters
is
folks
who
try
to
bootstrap
mega
clusters.
Let's
call
them
right,
and
you
know
you
can
have
big
clusters
that
run
large
workloads,
especially
as
you
get
more
and
more
kubernetes
operational
expertise,
but
if
you
put
all
your
eggs
in
that
basket
and
then
the
Train
shoots
off
the
rails,
you
know
you
have
a
big
blast
radius
there
that
you
have
to
worry
about.
A
A
So
generally,
we
just
again
want
you
to
consider
the
the
possibility
of
running
smaller
clusters,
not
just
for
security
reasons,
but
also
for
what
Craig's
been
talking
about
about
being
able
to
slice
things
up,
handle
multi-tenancy
in
a
cleaner,
easier
way
than
you'd
otherwise
have
to
engineer
purely
with
a
large
cluster
with
many
namespaces
inside
of
it.
Yeah.
B
We've
missed
that
on
the
multi-tenancy
piece,
if
you
like,
this
is
a
very
common
use
case
where
folks
have
maybe
have
a
super
secure
or
super
compliant
application.
Let's
say
PCI
workloads.
If
you're
already
in
the
mindset
of
building
multiple
clusters
right,
that's
a
perfectly
fine
tendency
model.
You
can
get
around
some
of
the
hard
hard
tendency
models
within
currents
itself
through
namespaces
by
just
deploying
additional
clusters,
and
you
should
plan
to
extend
kubernetes
right.
So
a
lot
of
folks
we
encounter
they
they
see.
B
Kubernetes
is
this
thing:
I
apply
some
llamo
and
an
application
comes
out
the
other
side
somehow
deployed.
They
don't
really
understand
the
mechanics
of
how
that
works
internally.
But
it's
actually
not
that
complex.
There's
through
things
like
custom
controllers
and
custom
resource
definition,
you
can
define
your
own
types.
You
can
add
functionality
that
may
not
exist
in
the
community.
A
perfect
example
of
this
is
is,
if
you
have
an
existing
legacy
infrastructure,
let's
say
ec2
again,
and
you
have
some
some
form
of
service
discovery
there.
Let's
say
DNS.
B
How
do
I
now
reflect
the
new
model
with
kubernetes
into
this
existing
service
discovery
model?
We've
seen
a
whole
host
away
that
ways
that
people
do
this
using
a
controller
and
operator
do
something
that
is
a
really
simple
way
to
do
it.
It's
not
that
hard.
You
could
write
it
in
any
language
that
you
want
and
it's
a
really
good
way
to
move
forward.
B
So
every
customer
that
I've
worked
with
has
extended
commands
in
some
way
whether
it's
something
like
deploying
calico,
ware
calico
comes
with
C
or
DS,
and
they
don't
even
realize
that
the
big
sin
in
kubernetes
all
the
way
down
to,
in
fact,
most
cases
that
work
with
customers
are
actually
building
their
own
operators
for
their
very
specific
business
use
cases.
So
you
should
plan
on
that.
You
should
look
into
operators,
see
how
those
would
potentially
benefit
your
deployment
and.
A
The
misstep
that
we
most
commonly
see
with
extending
kubernetes
is
when
you
kind
of
do
that
smell
test,
and
it
seems,
like
the
extension
point,
isn't
so
kubernetes
native
right.
So
you
know
a
lot
of
us
when
we
shows
a
lot
of
go
programmers
in
the
room.
A
lot
of
us
learn
how
to
write
good
go
by
looking
at
the
standard
library
right.
A
lot
of
us
can
learn
a
lot
about
extending
kubernetes
by
understanding
how
kubernetes
itself
works
right,
an
example
that
I
can
give
you.
A
As
you
know,
this
is
a
real
life
example
say
you
have
a
service
discovery
situation
and
then
we've
walked
into
customers
before
where
they
needed
to
articulate
that
some
pod
was
up
to
a
service
discovery
mechanism
and
the
method
that
they
did.
That
was
they
set
up
in
an
it
container.
The
pod
came
up
the
init
container
said:
ok,
the
pod
seems
like
a
scene
that
container
came
up
first
said:
okay,
hey
pod
information,
go
to
service
discovery,
shot
that
up
to
service
discovery,
and,
as
you
can
imagine,
while
that
does
work,
it's
pretty
fragile.
A
You
end
up
with
a
machine
that
looks
a
little
bit
like
this
right.
Something
knocks
something
over
knocks
something
else
over
and
if
something
goes
wrong
in
that
chain
of
command,
then
you
could
get
yourself
in
a
bit
of
a
sticky
situation.
But
when
you
look
at
the
kubernetes
kind
of
primitives
and
the
way
in
which
kubernetes
works,
you
think
about
things
like
reconciliation,
loops,
an
eventual
consistency,
and
how
can
you
kind
of
take
that
same
model
and
move
it
into
there?
A
Well,
it'd
be
pretty
easy
to
just
have
a
very
small
control,
a
couple
lines
of
code,
ideally
or
many
lines.
If
you
make
it
advance
that
just
kind
of
goes
through
and
understands
when
these
things
are
coming
up
and
then
updates
service
discovery
accordingly,
right,
that's
a
much
cleaner,
easier
to
troubleshoot
method,
so
just
be
sure
that
you're
doing
a
bit
of
the
smell
test
when
you're,
when
you're
extending
kubernetes
and
be
sure
that
you're
not
going
completely
off
the
rails
as
far
as
building
really
complex,
fragile
machines,
yeah.
B
B
So
we
have
some
resources
tomorrow,
at
10:30
and
3:30
we'll
be
giving
away
the
managing
kubernetes
book
at
the
VMware
booth.
If
you
want
to
stop
by
in
this
talk,
we
only
talked
at
the
very
high
level.
This
book
goes
a
little
bit
deeper
on
a
lot
of
these
topics
and
we're
happy
to
take
any
questions
now.
Thank.