►
From YouTube: Kubernetes VMware User Group Intro: Best Practices for Running on VMware - Steven Wong & Myles Gray
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes VMware User Group Intro: Best Practices for Running on VMware - Steven Wong & Myles Gray, VMware
This will be an inaugural presentation by organizers and members of the recently formed Kubernetes VMware User Group. This group addresses running all forms of Kubernetes on VMware infrastructure. Come to this session if you want to meet other users, along with those who develop, test and support running K8s on vSphere and desktop hypervisors. This group is also here to support those building/using cluster installation tooling for VMware hypervisors. Agenda: - Intro to the vSphere cloud provider and related storage plugins - Recent features/changes: What are they? How to use them? - Running Kubernetes on "desktop" hypervisors - How to get involved in the User Group
https://sched.co/ZewL
A
Hi
this
session
is
presented
by
the
kubernetes
vmware
user
group.
This
group
is
about
running
all
forms
of
kubernetes
on
the
vmware
infrastructure
platforms.
From
a
user
perspective,
I'm
steve
wong,
co-chair
of
the
user
group.
I
work
on
kubernetes
and
a
few
other
open
source
projects
as
an
employee
of
vmware.
Today,
I'm
joined
by
miles
gray.
The
other
co-chair.
A
A
B
Thanks
for
that
steve,
so,
like
steve
said,
I'm
going
to
talk
to
you
about
the
vsphere
cloud
provider,
the
updated
auditory
cloud
provider
and
the
container
storage
interface,
so
we'll
cover
off
the
vsphere
cloud
provider.
First
and
cloud
providers
are
a
concept
that
sort
of
makes
kubernetes
cloud
native.
It
provides
an
abstraction
layer
between
the
apis
from
the
underlying
infrastructure,
whether
that's
vsphere
or
google,
cloud
or
amazon,
or
what
have
you
and
the
primitives
that
kubernetes
understand.
B
B
It
really
sort
of
ties,
the
underlying
infrastructure,
so
the
vm
concept
into
the
kubernetes
layer,
so
it'll
apply
labels
into
particular
nodes
in
kubernetes,
based
on
what
their
vmid
is,
and
that
kind
of
thing.
So
the
cloud
provider
vsphere,
is
an
updated
out
of
tree
version
of
the
vsphere
cloud
provider
and
the
vsphere
cloud
provider
is
something
that's
known
as
an
intri
cloud
provider.
B
Those
are
all
being
sort
of
phased
out
by
about
kubernetes
1.20
1.21,
the
the
migration
timelines
change
a
little
bit,
but
I
think
it's
1.20
at
the
minute,
and
basically
the
idea
is
the
kubernetes
maintainers
want
to
remove
all
of
the
entry
cloud
providers,
because
what
it
does
is
add
dead
code
into
the
code
base.
It
opens
it
up
to
security
vulnerabilities,
it
adds
bloat
to
the
core
kubernetes
binaries.
B
It
doesn't
need
to
be
there
and
it
means
that
if
there
are
changes
that
need
to
be
made
to
a
cloud
provider
the
vendor
for
that
cloud,
provider
has
to
make
changes
to
the
core
kubernetes
code
base,
rather
than
just
updating
their
own
individual
plugin,
which
you
can
appreciate.
If
there's
a
bug
that
you
want
to
fix.
That
means
you
have
to
upgrade
your
entire
kubernetes
platform,
so
not
really
a
great
solution.
So
they've
pushed
all
of
the
vendors,
including
ourselves
to
creating
out
of
tree
version.
So
more
of
a
plug-in
type
architecture.
B
B
So
we'll
start
from
a
storage
perspective,
because
storage
is
is
sort
of
my
thing
and
we've
already
talked
about
the
vcp
and
basically
it
was
a
minimum
viable
product.
It
was
an
intri
cloud
provider
that
allowed
for
dynamic
provisioning
of
storage
on
kubernetes.
So
if
a
pod
was
scheduled
and
it
required
a
100
gig
volume,
the
vsphere
cloud
provider
would
take
that
call
and
translate
it
into
something
that
vsphere
would
understand.
B
So
it
would
essentially
just
create
a
100
gig
vmdk
and
mount
it
into
the
appropriate
kubernetes
worker
node
so
that
that
pod
can
access
some
storage.
So
that
was
all
fine
but,
like
I
say
it
was
a
minimum
viable
product.
It
had
kind
of
a
hacky
way
of
of
deploying
storage
and
basically,
what
it
did
was.
It
would
create
a
shell
vm
that,
had
you
know
nothing
in
it
other
than
a
vmx
file
and
a
vmdk.
B
B
So
we
saw
this
move
from
entry
to
iof3
as
an
opportunity
to
rewrite
that
integration
in
its
entirety
like
we
basically
just
went
from
scratch
and
we
used
a
completely
different
backend
that
doesn't
have
some
of
those
inherent
problems.
So
that's
what
we
sort
of
released
with
with
vsphere
6.7
u3,
something
called
the
cns
platform,
which
is
cloud
native
storage
platform,
and
it
has
basically
the
same
features
as
the
vsphere
cloud
provider,
but
in
a
much
more
supportable,
much
more
scalable
manner,
so
you've
still
got
your
dynamic
provisioning
of
volume.
B
So
if
your
applications
require
storage,
they
can
still
request
that
and
have
it
assigned
automatically.
There's
no
need
to
have
a
developer
talk
to
a
sysadmin
to
get
stuff
provisioned
and
frankly,
that's
just
not
the
way
stuff
works
in
this
world.
So
having
dynamic
provisioning
is
a
must
that
vm
provisioning
workflow.
I
talked
about
where
it
created
a
vm
with
a
vmdk
attached
and
threw
away
the
vm.
Clearly,
a
lot
of
api
calls
and
a
lot
of
functions
going
on
there.
That
didn't
really
need
to
happen,
so
we
changed
the
back
end
from
that.
B
What
we
call
just
the
legacy:
vm
provisioning
backend
to
something
called
first
class
disks
and
first
class
disks-
are
really
nice
because
they
don't
have
this
orphan
storage
problem.
If
you
unmount
one
of
these
first
class
disks
from
vm,
we
still
track
where
it
is.
We
still
track
how
much
storage
it's
consumed.
You
know
where
it
lives
on
the
file
system,
all
that
good
stuff.
So
even
whenever
you
scale
out
an
application
say
a
stateful
set
and
you
scale
it
back
in
from
say
three
to
two
nodes:
it'll
keep
the
the
disk
there.
B
The
first
class
disk
will
still
exist.
You
know
persis,
it
will
still
persist,
that's
the
point
of
it,
but
we
won't
lose
track
of
it.
So
the
next
time
it
goes
to
scale
out,
say
from
two
to
three
nodes:
it'll
automatically
take
that
volume
and
remount
it
back
in
and
then
the
application
can
do
a
delta
sync
on
top,
so
it
removes
some
of
the
challenges
around
operationalizing
it.
In
addition,
we
added
a
ui
into
vcenter
for
all
this
stuff.
B
So
whenever
you
get
a
dynamically
provisioned
volume,
all
of
those
volumes
show
up
in
the
vcenter
ui,
so
that
your,
if
you're
a
developer,
your
sysadmins
can
help
you
in
debugging
things.
If
you
give
them
a
ppc
id,
they
can
just
paste
it
into
our
ui
and
it'll
show
them.
You
know
right
the
way
down
to
the
physical
tin,
what
disk
that
volume
actually
exists
on
so
a
lot
of
the
refinement
and
and
feature
additions
that
we
made
were
really
around
making
this
more
scalable,
making
it
more
operationally
viable.
B
So
that
was
our
release
with
6.7
u3
and
then
in
7.0.
We
continued
on
that
architecture
and
we
added
a
few
more
bits
and
pieces
in
there.
So
if
any
of
you
are
familiar
with
vsphere
storage,
we
added
support
for
basic
vivo
primitives.
So
if
you
want
a
block
volume,
that
is
to
say,
if
you
want
a
read,
write
once
volume
in
kubernetes,
you
can
now
use
vmfs,
nfs,
vsan
or
vivo,
so
all
storage
types
for
vsphere
knives
supported.
B
In
addition,
we
had
quite
a
few
requests
for
rewrite
many
volumes.
So
if
you
have
a
visa
and
data
store,
we
can
now
automatically
provision
nfs
file
shares
for
you
just
dynamically.
The
same
way,
we
would
for
read
write
once
volumes
and
they
will
get
mounted
into
you
know.
However,
many
pods
and
worker
nodes
you
assign
that
single
read,
write
many
persistent
volume
to
so.
B
So
if
you
have
a
node
failure
and
it
gets
moved
to
another
vm,
that
volume
will
be
re-keyed,
so
it
can
be
re-read
by
the
target
vm,
which
is
quite
nice
as
well,
and
we
have
persistent
volume
snapshots
now
I'll
highlight
this.
Isn't
the
csi
snapshot
feature
that
we
don't
have
interrupt
with
the
csi
snapshot
api
yet?
But
this
is
a
plug-in
for
valero.
B
We
also
added
a
first
pass
at
volume
resizing,
so
we
can
now
actually
support
offline
volume
resize.
So
you
would
essentially
remove
it
from
your
pod.
Delete
the
pod
change.
The
specification
of
the
pvc
it'll
automatically
expand
it
in
vsphere,
and
then
you
can
reattach
it
to
your
pod
and
we've
added
support
for
a
mixture
of
tooling
as
well.
B
So
if
you
want
to
use
wavefront,
you
can
use
wavefront,
but
we
realize
that
almost
everyone
here
uses
prometheus,
so
we
added
native
prometheus
exporters
into
esxi,
as
well
as
created
a
bunch
of
grafana
dashboards
for
those
that
actually
expose
those
metrics
and
tell
you
how
to
use
them
as
well.
So
if
you
have
prometheus
in
your
environment,
you
can
now
use
that
to
scrape
metrics
from
esxi
as
well
directly
and
have
your
applications
react
to
changes
on
the
underlying
infrastructure.
B
B
So
when
you
install
that
it's
just
automatically
there,
so
the
first
thing
you
do
is
you
create
your
spbm
policy
in
vsphere
and
if
you're
not
familiar
with
vsphere
or
spbm
policies,
it's
essentially
just
a
storage
class
like
they're,
basically
identical
concepts,
but
an
spbm
policy
allows
you
to
specify
things
like
you
know
how
many
copies
of
this
data
do
you
want?
You
know
where?
Should
it
be
located
on
what
site
that
kind
of
thing?
B
So
you
create
your
spbm
policy,
and
then
you
create
a
kubernetes
storage
class
that
points
at
that
spbm
policy,
so
super
simple,
they're
just
related
by
name.
Then
we've
got
our
csi
driver
and
the
csi
driver
is
what
does
the
translation
between
the
kate
storage
class
and
the
spbm
policy.
So
whenever
a
volume
is
provisioned?
B
That
is
how
vcenter
and
vsphere
know
in
what
manner
to
create
that
volume
and
where
to
mount
it
into
which
kubernetes
node
and
then
what
pod
so
you've
got
a
cassandra
deployment
here,
and
it
says
I
would
like
a
100
gig
volume
from
this
storage
class
so
that
storage
class.
If
you
look
at
how
that's
constructed
there
is
a
provisioner,
and
that
would
say
this
is
the
vsphere
csi
provisioner.
So
it
says:
okay
well
I'll
hand.
This
call
off
to
the
vsphere
csi
provider.
B
B
Cns
looks
at
the
request
and
says:
okay,
this
is
a
read,
write
once
volume
and
it
requires
this
spbm
policy.
So
it
sends
that
to
the
spbm
engine
and
then
it
just
creates
that
volume
based
on
what
the
policy
said.
So
it
will
create
two
copies
of
the
data
mounted
up
to
into
the
appropriate
kubernetes
worker
node
format
it
with
a
file
system
and
mount
it
into
the
container
so
end
to
end.
B
B
In
addition,
it
will
actually
change
the
back-end
disk
type
from
just
a
standard
vmdk
to
a
first-class
disk
so
that,
whenever
you
are
importing
all
of
these
volumes,
you
get
all
of
the
nice
new
ui
based
tracking
and
all
that
stuff
that
comes
along
with
cns
as
well,
so
the
upgrade
would
actually
be
transparent
to
the
application.
You
don't
need
to
change
any
of
the
application
manifests.
We
basically
just
intercept
all
of
the
calls
that
would
go
to
the
vcp
via,
like
a
a
web
hook,
shim
layer
and
they
get
pushed
into
the
csi
instead.
B
So
we
just
take
over
all
of
those
calls.
So
this
is
really
nice
from
like
a
day
two
perspective.
Whenever
you
are
upgrading
your
case
clusters
in
future
you'll
be
able
to
just
take
your
existing
volumes
and
import
them
into
the
csi
driver
so
that
they
get
automatically
included
in
the
vcenter
ui,
which
is
really
quite
cool,
and
we're
just
going
to
take
a
brief
look
here
through
some
of
the
recently
added
features
in
the
vsphere
cloud
provider.
B
Before
I
hand
it
back
to
steve
and
he's
going
to
take
you
through
desktop
hypervisors,
so
just
real,
quick,
we're
gonna
hit
off
the
new
things
here
and
then
you
can
click
on
the
individual
links
with
the
deck
afterwards,
so
we
actually
have
nsxt
based
load
balancers,
getting
added
into
the
cloud
provider.
So
if
you
specify
a
load
service
type
of
load
balancer,
we
can
interrupt
with
nsx
directly
and
ask
for
those
to
be
provisioned,
which
is
really
quite
cool.
B
Additionally,
we're
moving
from
like
an
ini
type,
config
config
to
a
yaml
based
config,
just
because
you
know
that's
kind
of
how
things
work
these
days.
We
are
updating
the
cloud
provider
interfaces
image
account
to
be
a
non-root
account,
which
I'm
sure
you'll
agree
from
a
security
standpoint
is
a
good
thing
to
do,
and
just
adding
a
bit
more
robustness
to
the
whole
zones
and
regions
labeling
we're
looking
at
or
we
are
adding
a
folder
and
resource
pool
traversal
for
those
features
as
well.
B
So
if
you
are
using
zones
or
tags,
it
will
now
be
able
to
traverse
all
of
the
resource,
pools
or
folders
under
that
environment
and
figure
out
where
stuff
actually
lives
and,
like
I
say,
the
rest
of
it.
You
can
read
through
in
your
own
time
with
the
deck
afterwards
and
you
can
click
on
the
individual
pull
requests
and
see
how
those
things
actually
work.
A
A
A
The
fusion
hypervisor
is
the
version
for
mac
and,
like
workstation,
it's
known
as
a
type
2
hypervisor,
meaning
it
runs
on
an
os
rather
than
directly
on
bare
metal.
It's
also
worth
noting
that
these
desktop
hypervisors
implement
virtual
networking
to
allow
creation
of
multiple
networks
connected
to
the
vms
they
host.
A
A
A
A
The
top
line
here
shows
the
download
links
for
getting
the
production
releases
of
the
desktop
hypervisors
and,
by
the
way,
the
the
fusion
hypervisor
for
mac
version.
11.5.5
is
already
out
with
this
container
support,
so
if
you're
running
11.5,
first
fusion,
you
can
upgrade
to
11
5.5
right
now
to
get
the
container
support.
A
The
next
line
here
has
links
to
the
free
tech
preview
beta
and
a
warning
if
you
use
the
earlier
tech
preview
beta,
that
was
out
at
the
beginning
of
the
year.
Uninstall
it
before.
Moving
on
to
the
newer
version,
also
be
careful
about
using
google
to
find
this
beta,
because
the
older
version
tends
to
show
up
first
in
the
search
results
just
because
a
lot
of
blog
posts
got
out
talking
about
it.
A
If,
if
this
had
been
a
physical
conference
as
originally
planned,
I
would
have
done
it
as
a
live
demo,
but
since
we're
going
to
do
it
online,
it
can't
be
live
anyway,
and
I
decided
that
the
screen
content
snaps
suffer
from
less
from
bad
internet
and
also
allow
cut
and
paste
directly
from
the
slides
in
the
deck.
If
you
choose
to
follow
along
later
at
home,
so
I'm
going
to
show
this
demo
on
windows
instead
of
linux.
A
So
I
wanted
to
be
inclusive
and
help
people
get
a
bridge
from
those
millions
of
windows
laptops
into
the
world
of
kubernetes
if
you're
on
a
mac
or
a
linux
machine,
don't
fret
it's
very
similar
to
what
I'm
about
to
show.
Mostly.
The
difference
is
that
you
can
cut
out
a
few
steps
so
start
by
downloading
the
five
items
listed
here
if
you're
watching
from
the
recording
these
links
are
to
specific
versions
that
probably
have
a
shelf
life
of
a
month
or
two.
A
A
The
summary
here
is:
I'm
assuming
you've
already
downloaded
it
from
your
collection
of
downloads,
start
by
installing
the
hypervisor,
then
golang
get
finally
build
a
driver
from
the
golang
source,
install
cube,
cuddle
and
then
mini
cube.
The
these
downloads
are
installer
packages,
so
they're,
pretty
simple.
To
figure
out,
I'm
not
going
to
make
you
watch
step
by
step
with
these
installs
just
find
the
installers
that
you
downloaded
and
install
them
in
the
order.
Here
you
should
be
fine.
A
Next,
we're
going
to
configure
minicube.
The
effect
of
these
configuration
settings
is
to
specify
the
vm
that
kubernetes
is
going
to
run
in
the
settings
I'm
showing
here
are
generous.
I
did
these
big
to
allow
enough
to
run
pretty
demanding
things
such
as
installing
and
running
an
istio
service
mesh.
A
By
the
way
I
show
an
optional
step
here,
because
I'm
changing
the
default
network
cider
and
chain.
This
change
is
going
to
require
some
additional
setup
in
the
workstation
hypervisor
that
I'll
get
to
in
the
in
a
moment.
I'm
doing
this
simply
because
most
mini
cube
usage.
Examples
that
you
find
out
on
the
internet,
use
the
192
168
99
cider
range,
and
I
wanted
to
make
it
easy
to
cut
and
paste
out
of
any
examples
you
might
find
off
the
internet.
A
So
do
the
config
steps
shown
here,
but
when
you're
done
don't
close
the
command
prompt
that
you
do
this
in
because
we'll
be
back
to
it
shortly.
You'll
follow
the
steps
to
configure
each
of
these
settings
one
at
a
time
and
that
final
step
of
mini
queue,
config
view
just
echoes
back
your
configuration
so
that
you
can
verify
that
you
didn't
have
any
typos.
A
Now
I'm
showing
the
configuration
of
the
vmware
workstation
hypervisor,
like
I
say
it
comes
with
virtual
networks.
There
are
some
created
by
default.
You
can
create
more
if
you
need
them,
but
what
we're
going
to
do
is
change
the
the
ip
subnet
that
is
being
used
by
one
of
these
pre-configured
networks.
A
A
A
A
The
the
vm
is
going
to
be
underneath
the
dot
mini,
cube,
slash
machine,
slash,
mini
cube
directory
inside
your
home
directory
on
windows,
and
later
you
can
add
that
file
to
your
workstation
inventory,
using
the
file
open
menu.
If
you
want
to
see
it
in
the
workstation
gui
after
the
command
line,
shows
done
with
an
x
exclamation
point.
A
A
We'll
move
to
a
moderately
advanced
demo
now,
but
you
could
do
whatever
you
want.
Mini
cube
starts
pretty
lightweight,
but
I'm
going
to
turn
on
some
actual
optional
add-on
features.
Since
I
gave
this
vm
plenty
of
cpu
in
memory
to
allow
this
we're
going
to
deploy
a
load
balancer,
something
you
might
need
if
you're
going
to
play
with
service
mesh
or
interact
with
the
outside
world,
we're
also
enabling
the
kubernetes
dashboard.
A
When
you
do
invoke
the
minicube
dashboard
command
at
the
command
line,
this
will
open
up
a
browser
windows
with
the
kubernetes
ui.
The
metal
lb
load.
Balancer
is
capable
of
issuing
ips
from
a
pool
like
an
ipam,
but
we
need
to
configure
it
in
order
to
specify
the
ip
address
pool
that
it
issues
ips
from
when
it
provides
load
balancing
services.
A
A
A
Now
we're
going
to
display
deploy
a
service,
and
this
is
the
hello
mini
cube
service
example
that
comes
straight
out
of
the
mini
cube
docs.
We
use
cube
cuddle
to
create
a
deployment
of
a
web
service,
as
shown
here,
then
we
use
cube
cuddle,
expose
to
create
a
load
balance
service
and
make
this
web
server
publicly
accessible.
A
After
this
completes,
you
can
find
a
link
to
the
service
in
the
dashboard,
or
you
could
also
get
the
url
from
the
command
line.
I'm
not
going
to
show
it
here,
but
if
you
do
this
later
at
home,
a
hidden
slide
in
this
deck
shows
the
steps
to
clean
up
the
service,
the
deployment
and
even
the
mini
cube
vm.
A
So
this
content
is
brought
to
you
by
a
kubernetes
user
group
and
if
you're
using
kubernetes
on
vsphere,
I
encourage
you
to
formally
join
this
group.
We
have
a
meeting
each
month
where
we
discuss
tutorials
and
best
practices.
The
agenda
is
user
driven,
so
members
are
encouraged
to
nominate
presentation
and
discussion
topics,
including
discussions
of
feature
requests.
A
A
You
can
go
to
the
kubernetes
community
calendar
link
shown
here
to
get
a
conversion
to
your
local
time
zone
and
to
add
it
to
your
calendar.
You
can
become
a
group
member
by
joining
them
joining
the
mailing
list.
That's
shown
in
the
link
here
and
finally,
here's
a
link
to
the
group's
slack
channel
so
miles
and
I
are
going
to
hang
around
for
q
a
there's
a
link
at
the
top
that
will
get
you
this
deck.