►
From YouTube: Kubernetes VMware User Group: Using GPUs with K8s on vSphere - Steven Wong & Myles Gray, VMware
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes VMware User Group: Using GPUs with K8s on vSphere - Steven Wong & Myles Gray, VMware
An increasing number of applications and services can benefit from GPUs, yet costs and other constraints often prohibit installation in all compute hosts. “Landlocked” GPUs resources often lead to underutilized cycles and wasted spending. This session will describe how a pool of available GPU resources within a vSphere cluster can be shared across a broader number of Kubernetes cluster nodes to accelerate workloads like AI, deep learning and inference. This can provide full or partial GPU compute capacity at scale to Kubernetes workloads, even when these are running in pods on hosts without an installed GPU. The session will show an example based on running a TensorFlow workloads on Knative. The K8s VMware User Group shares best practices for hosting K8s on VMware infrastructure, and we will close the session with details on how you can participate in the group.
A
A
A
A
word
about
the
user
group,
we're
inclusive
of
all
users
running
all
forms
of
kubernetes
on
vmware
infrastructure.
So
what
I'm
talking
about
doesn't
just
apply
to
people
running
a
vmware
distribution.
This
should
be
useful
to
you
if
you're
running
distributions,
like
anthos
eks,
anywhere
openshift,
rancher
or
pure
upstream
kubernetes,
open
source.
A
A
Well,
advancements
in
both
hardware
and
software
have
changed
the
terrain,
and
today
you
want
the
chickens
now
before
some
nitpicker
in
the
audience,
looks
at
that
picture
carefully
and
points
out
that
those
are
actually
ducks
and
not
chickens.
Give
me
a
little
leeway
and
I
want
you
to
paint
an
even
different
picture.
Visualize
this
in
your
head.
You
can
keep
the
ox,
but
you
know,
lose
the
ducks
or
chickens,
but
I
want
you
to
bring
in
a
shark
and
a
school
of
piranha.
A
Now
imagine
somebody
is
giving
you
the
goal
of
creating
a
museum
exhibit
of
an
ox
skeleton
at
the
la
county,
natural
history
museum
and
that
museum
by
the
way
is
just
a
short
distance.
That
way-
and
I
highly
recommend
it
living
in
los
angeles
if
you've
got
some
spare
time
to
kill,
but
back
to
the
mission,
we
start
with
an
ox
and
we
want
an
ox
skeleton.
A
You
could
take
that
shark
oops
accidentally
invents
my
slide,
you
could
take
that
shark,
put
it
with
the
ox
in
a
pool
and
that
shark
is
going
to
take
big
bites
and
digest
them
bigger
bites
than
a
piranha
could.
But
if
you
unleash
that
school
of
piranha,
they
are
going
to
be
able
to
access
that
from
the
pool
on
all
the
sides,
the
top
and
the
bottom
you're
just
going
to
see
a
cloud
burst
of
activity
and
poof.
A
A
A
And
by
the
way,
this
isn't
just
time
to
result.
There
are
often
energy
advantages
to
using
these
gpus.
If
you
can
get
the
same
job
done
by
while
burning
fewer
kilowatt
hours,
it
helps
the
planet
another
possible
tie-in
that
might
save
energy
if
you're
doing
something
like
deep
learning
applications.
Usually
the
training
stage
is
very
gpu
intensive,
but
then,
after
you've
done
the
training
and
take
it
to
the
run
time
to
actually
execute
using
your
model,
you
can
get
by
on
a
fraction
of
a
gpu's
capacity.
B
B
B
So
each
one
of
these
has
a
nvidia
tesla
d4
attached
to
it
and
in
case
you're
not
sort
of
aware
of
how
bitfusion
works,
but
fusion
is
a
client
server
model,
so
these
servers
in
this
instance
are
vms
and
they
have
a
gpu
passed
up
into
them.
Your
clients,
then,
can
dial
into
the
servers
and
request
gpu
resources.
B
So
say
you
have
a
application
and
you
say
I
would
like
it
to
consume
half
of
the
gpu,
then
the
bitfusion
client
running
in
either
inside
the
container
or
on
the
nodes
and
we'll
get
into
that
in
in
a
second
here
is
going
to
request
from
bit
fusion.
You
know
half
of
a
gpu.
If
it
can
fulfill
that
request,
then
what
will
happen
is
bitfusion
will
intercept
all
the
cuda
calls
that
go
to
the
cuda
apis.
B
That
would
need
gpu
acceleration
and
instead
sends
those
over
the
network
to
the
bitfusion
server,
which
does
the
calculation
on
the
gpu
and
sends
back
the
results.
So
bitfusion
abstracts
the
gpu
from
the
workload
itself.
So,
there's
no
need
to
have
a
gpu
mounted
into
every
single
vm
that
is
consuming
or
has
a
workload
that
would
like
to
consume
a
gpu.
B
B
Instead,
we
can
slice
up
that
gpu
into
an
arbitrary
number
of
frame
buffer
allocations,
so
each
of
these
gpus
has
16
gigs
of
vram
or
frame
buffer
more
accurately.
So
you
can
divide
that
up
into.
However
many
slices
that
you
like
and
gpu
and
bitfusion
will
allow
the
client
to
claim
that
much
of
the
gpu
so
you'll
see
as
we
get
into
this,
why
this
is
particularly
beneficial
to
kubernetes
workloads,
given
the
non-deterministic
placement
of
them.
The
fact
that
there
are
many
of
those-
and
only
you
know
a
limited
subset
of
gpus.
B
So
anyway,
we've
got
our
bitfusion
servers
there.
They
are
going
to
be
the
things
that
are
going
to
do
the
actual
processing
of
our
application
code.
That
requires
acceleration
inside
this
resource
pool.
Here
you
can
see
tce
management
and
a
workload
cluster
here,
tanzu
community
edition
that
is
so
steve
that
set
this
up
for
me
very
kindly
last
week,
so
you
can
see,
we've
got
our
management
cluster
and
we've
got
our
workload
cluster
here
and
we've
also
got
rancher
in
here
as
well.
B
B
B
B
So
the
way
that
fusion
works
is
it
assumes
a
gpu
as
a
service
model,
so
we're
assuming
that
the
gpus
are
sort
of
being
leased
out
to
these
tenants
or
internal
customers
for
and
and
their
builds
based
on,
how
much
time
they
have
used
those
for
so
each
of
these
clients
is
actually
kept
as
a
historical
record.
So
you
can
say:
okay,
this
consumed
30
minutes
of
gpu.
This
consumed
an
hour
of
gpu,
so
you
can
do
chargeback
and
that
kind
of
stuff.
B
So
the
first
thing
that
we're
going
to
do
here
is
we're
going
to
add
bitfusion
or
the
bitfusion
credentials
more
accurately
to
our
kubernetes
cluster
to
our
tce
cluster,
and
this
is
brand
new
in
bitfusion4
and
to
be
honest,
I
think
this
is
a
really
really
killer
feature.
So
if
we
go
kubernetes
clusters,
we
click
add
it
says:
cubeconfig
and
I've
got
my
tceq
config
file
here,
so
click
upload
and
we'll
call
it
tce01,
and
you
can
see
it's
grabbed.
The
ip
address
from
the
cubeconfig
file
automatically
connected
and
pull
back
the
namespaces.
B
B
So
I'm
going
to
add
that
to
three
namespaces,
we're
going
to
add
it
to
cube
system,
we're
going
to
add
it
to
flower
market,
which
is
where
we're
actually
going
to
deploy
our
application
and
we're
going
to
deploy
it
to
the
bitfusion
with
kubernetes
integration
namespace
as
well.
So
we
clicked
add
okay.
So
now
these
are
our
target
clusters
right.
So
you
can
see
here.
This
is
sort
of
a
one-to-many
relationship.
B
You
can
have
many
kubernetes
clusters
and
you
can
even
have
different
tokens
for
different
name
spaces
inside
the
same
cluster
depends.
What
way
you
want
to
do
your
permissions
model,
so
we've
added
our
cluster
and
now
we
need
to
create
a
client
token,
so
that
those
workloads
can
actually
connect
to
bitfusion
and
consume
some
gpu
resources.
B
So
we'll
click
create
and
we'll
just
call
it
flower
market
token
and
we're
just
going
to
add
it
to
all
three
of
those
you
can
see.
You
can
pick
and
choose
here,
so
you
can
have
many
tokens
if
you
want
to
to
do
individual
permissions
models
and
that
kind
of
thing,
but
we're
just
going
to
pick
all
of
them
and
it
says,
activate
token
after
creation.
So
that
means
that
it'll
deploy
the
token
to
the
kubernetes
cluster
in
the
form
of
some
secrets.
And
actually
let
me
show
you
that
before
we
do
it.
B
So
if
I
do
k
get
secret
and
hopefully
we
won't
see
yeah.
So
there
should
be
three
secrets
in
here,
though,
and
you'll
see
them
whenever
they
pop
up,
but
basically
that
will
be
the
ca
servers
and
the
client
configuration
file.
So
they
do
not
exist
here.
So
what
it's
asking
us
here
when
we
say
activate
token
after
creation,
is
once
it's
populated
those
secrets
into
kubernetes.
B
Is
that
token
able
to
make
requests
of
the
bitfusion
server
and
you
can
disable
it
at
any
time
as
well?
So
if
you've
decided
that
you
need
the
gpu
resources
for
something
else,
you
can
deactivate
that
token
without
actually
deleting
it
and
changing
the
configuration.
But
anytime
someone
connects
they'll
just
get
it.
No
sorry,
your
token
has
been
deactivated,
so
we're
going
to
go
ahead
and
click
create.
You
can
see.
It
says
it's
activated,
it's
added
it
to
these
namespaces.
B
B
So
that's
good.
That
means
that
we
can
now
mount
those
secrets
into
an
integration
that
allows
us
to
connect
to
the
bitfusion
instance
and
the
nice
part
of
this
about
it
is
it's
dynamic
right.
You
just
add
the
cube
config
file,
you
choose
your
namespaces
and
it
auto
populates.
It's
not.
You
know
download
credentials,
create
generic
secret,
all
that
kind
of
stuff,
and
you
can
rotate
secrets
a
lot
easier
with
this.
B
The
way
the
integration
is
built,
so
the
first
thing
we're
going
to
do
is
we're
going
to
take
a
look
at
the
bitfusion
with
kubernetes
integration
and
I'll
just
pull
up
the
github
page
here.
So
you
can
get
this
at
github.com,
vmware
bitfusion
with
kubernetes
integration
and
the
bit
that
you're
interested
in
here
is
the
bitfusiondevice
plugin.
So
kate's
has
a
concept
of
device
plugins.
It's
generally
used
for
like
hardware
that
you
have,
that
has
specific
drivers
that
need
to
run
on
the
gas
os
of
the
the
node
itself
alongside
cubelet
to
make
them.
B
You
know
work,
so
we
have
a
similar
sort
of
thing
with
bitfusion
here
it
goes
about
things
a
little
bit
differently,
but
it
does
install
some
libraries
and
some
drivers
that
are
required,
but
basically
it's
in
two
parts
right
and
we'll
look
at
the
the
architecture
here
to
just
to
exemplify
that.
So
what
you
see
here
is
your
standard
in
green.
By
the
way
is
your
deployment
and
the
job
for
that
deployment.
B
If
you're
doing
job-based
batch,
processing
or
whatever
and
then
you've
got
your
bitfusion
device
plug-in
and
that
is
essentially
doing
the
connection
between
the
worker
node
and
the
bitfusion
server,
so
that
creates
that
pipe
between
cuda
on
the
client
and
cuda
on
the
server
to
pass
those
calls
over
the
network.
So
you
can
see
here.
B
So
I've
already
deployed
this
because
it
takes
a
little
bit
to
deploy
and
I'm
just
going
to
show
you
how
this
actually
mounts
the
secrets
in
just
that
you're
aware
of
where
the
integration
from
bitfusion
server
in
4.0
and
the
bitfusion
kubernetes
integration
is.
So
if
we
look
at
our
actual
deployment
yaml
file
here,
it's
passed
in
as
a
config
map,
and
if
we
go
down
to
the
bottom,
you
can
see
here
it's
mounting
in
ca,
client
and
servers.conf,
so
that
is
then
telling
the
device
plugin.
B
Here's
how
you
connect
a
bit
fusion
here
are
the
servers
here,
your
credentials,
and
this
is
how
you
authenticate
the
connection
right.
So
we've
already
deployed
the
fusion
with
kubernetes
integration,
we're
not
going
to
look
at
that.
What
we're
going
to
do
is
deploy
an
app
and
I'm
going
to
show
you
the
docker
file,
just
to
show
you
that
we
don't
have
any
pre-baked
bits
in
there
right.
There's,
there's
no
bit
fusion
client
in
there.
There's
nothing
like
that.
B
It's
just
a
a
standard
container
just
runs
python,
that's
it
and
what
the
bit
fusion
with
kubernetes
integration
is
going
to
do
is
look
for
the
annotations
and
we'll
look
at
the
annotations
in
a
second,
and
if
it
sees
those
it
will
automatically
inject
a
init
container
onto
that
deployment
and
put
the
fusion
client
into
it
and
then
append
all
of
the
the
entry
point
for
that.
Docker
file
onto
the
end
of
the
command,
essentially
proxying,
the
entire
container's
connection
to
to
cuda
through
the
network
completely
transparently.
B
We
are
then
telling
it
we
want
to
execute
this
benchmark
python
script
right,
it's
just
a
a
standard,
tensorflow
benchmark.
It
uses
gpu
acceleration,
but
it's
not
actually
doing
any
real
work.
It's
just
demonstrating!
You
know
work
on
the
gpu,
so
we're
specifying
the
number
of
batches
the
batch
size,
data
format.
You
know
model
all
that
kind
of
stuff,
so
this
is
essentially
just
setting
up
tensorflow
and
telling
it.
What
benchmark?
Are
you
going
to
run?
B
What
model
are
you
going
to
use
and
what
are
you
going
to
do
the
processing
on
so
local
parameter
device
gpu?
So
it's
going
to
do
it
use
a
gpu.
If
we
come
down,
we
just
pip
install
some
packages,
because
this
exposes
some
information
to
prometheus.
We
install
a
prometheus
client.
We
then
clone
the
repo
for
itself
down
so
that
it
can
get
the
python
script
and
then
we
execute
the
python
script.
B
So
you
can
see
here,
entry
point
python
and
then
the
name
of
the
file
along
with
all
the
other
model
stuff
that
we
saw
further
up
as
well
as
telling
it
to
run
on
a
gpu.
So
that
is
the
container
we're
working
with
right.
It's
essentially
a
base,
nvidia
tensorflow
container,
it's
just
running
python:
that's
it
there's
no
other
bits
or
pieces
in
there.
So
this
is
like
your
standard
machine
learning
container.
B
Now
what
I
want
to
show
you
is
that
we
can
take
that
standard
machine
learning
container
which,
if
we
just
ran
it
on
the
cluster
without
any
gpus
attached,
it
would
fail
right
because
it's
trying
to
use
a
gpu
and
there's
no
gpu
directly
attached
to
the
cluster.
The
gpu
is
mounted.
You
know
somewhere
else
in
the
infrastructure
entirely,
so
this
wouldn't
actually
work.
B
So,
let's
have
a
look
at
our
manifests
here.
The
only
one
we're
going
to
look
at
is
the
deployment
all
the
rest
of
it
is
fairly
uninteresting.
We've
got
our
name.
Namespace
service,
monitor
service,
horizontal
pod,
autoscaler,
that
kind
of
stuff,
but
the
the
bit
that
actually
ties
this
to
the
bitfusion
integration
is
just
some
annotations
here.
B
So
you
can
see.
This
is
just
a
standard
deployment.
It's
called
a
new
hope,
wookie,
there's
a
whole
other
presentation
on
that
which
you'll
find
if
you,
google,
the
dutch
v-mug
from
from
last
year,
you'll
find
what
this
whole
story
is
about,
but
essentially,
we've
got
our
same
container
that
we
were
just
looking
at
here.
The
a
new
hope
worker
latest
and
you
can
see
the
the
python
command-
is
exactly
the
same
as
the
one
that
was
that
was
in
the
container
itself
and
we've
added
these
three
annotations.
B
So
we've
got
auto
management
bit
fusion
set
to
yes,
so
that
essentially
says
enable
the
bit
fusion
integration
for
this.
This
deployment,
please
what
os
would
you
like
it
to
use
to
do
the
bootstrapping?
So
I
just
chose
ubuntu
20,
because
it's
the
latest
one
and
what
version
of
the
bitfusion
client
would
you
like
to
use?
This
generally
needs
to
be
matched
to
your
server,
so
my
server
is
running
4.0.1,
so
my
bitfusion
client
version
is
4.01,
so
that's
essentially
it.
We
then
add
our
limits
to
our
container
just
under
our
resources.
B
So
you
can
see
we
are
requesting
bit
fusion
gpu
and
one.
So
that's
the
number
of
gpus,
so
one
gpu,
and
what
percentage
of
that
gpu?
Would
we
like
to
use
so
we're
asking
for
100?
So
we
are
essentially
expecting
to
get
a
full
gpu
allocation
here.
So
I'm
going
to
go
ahead
and
do
a
k
apply
f
and
it's
in
I'm
in
the
wrong
folder.
So
let's
fix
that
first.
So
it's
the
new
hope,
app
and
we'll
do
k
apply.
Dash
f,
manifests.
B
B
You
can
see
that
it's
15
seconds
old
and
it's
already
up
and
running
and
if
I
do
a
k
get
pod,
you
can
see
that
that
container
is
running
now.
What
I
want
to
show
you
before
we
look
at
the
container
logs
or
maybe
maybe
we'll
look
at
the
container
logs,
really
quick.
I
will
do
k
logs
and
it's
going
to
be
a
new
hub
and
we'll
follow.
B
B
B
So
if
we
have
a
look
up
here,
you
should
see
that
there
a
container
wookie,
which
is
the
standard
one,
that's
this
one
that
is
in
the
manifest
here
as
you
can
see,
but
we
should
also
see
a
init
container
as
well.
Here
you
go
so
there's
our
init
container
and
you
can
see
that
I
have
not
I've
not
specified
an
init
container
in
this
manifest
anywhere,
that's
been
injected
by
the
integration.
So
you
can
see
it's
doing
some
command
line
foo
here.
B
It's
copying
the
bitfusion
client
service.conf,
as
well
as
the
the
debian
file
or
the
bits
to
actually
run
the
client.
It's
copying
those
into
the
workload
container
and
then
it's
running
it's
adjusting
the
command.
So
you
can
see
here.
It
says
the
command
for
the
wookie.
Now
is
actually
not
just
the
python
stuff
that
we
had
already
specified,
but
rather
it
has
injected
the
bitfusion
client
in
and
is
now
executing
the
command
through
bit
bit
fusion.
So
you
can
see
it's
doing
a
bit.
B
Fusion
run
dash
n1
number
of
gpus
one
and
how
much
of
the
gpus
so
dash
p
is
partial
1.0.
So
it
means
the
whole
gpu,
so
you
can
see
that
is
injected
the
bitfusion
client
and
it
is
now
running
that
benchmark
through
the
the
gpu
there.
So
if
you
look
up
at
the
top
here,
you
can
see
it
says
requesting
gpus
with
15
gigs
of
memory
right.
It's
a
16,
gig
gpu,
obviously
they're
using
different
units
here,
but
you
can
see
that
that's
now
running
in
the
background.
C
B
B
So
if
we
go
back
in
here,
we
should
see
once
it's
finished.
You
know
process
so
many
images
per
second
and
that
kind
of
thing
from
the
benchmark
script.
But
you
can
see
here
that
essentially
this
container
has
zero
knowledge
of
bit
fusion
whatsoever
right.
It's
just
a
python
container
and
the
bit
fusion
with
kubernetes
integration
along
with
the
bitfusion4.
B
You
know
token
copy
mechanism
allows
these
containers
to
connect
to
a
bitfusion
server
without
any
knowledge
of
the
client
without
having
to
be
adjusted
in
any
way.
I'll
just
show
you
for
for
clarity,
how
I
used
to
have
to
do
this.
So
if
we
go
into
my
docker
file
here,
you
can
see.
I
have
a
whole
bunch
of
stuff.
That's
been
commented
out
here
and
that's
the
way
that
we
had
used
to
have
to
do
it
with
bitfusion
3.5.
B
So
I
had
to
to
bake
into
the
container
the
bitfusion
client
bits,
as
well
as
all
the
configuration
for
those.
So
I
had
to
bake
in
the
number
of
gpus
how
much
of
the
gpu
any
variables.
Along
with
like
the
servers
file,
the
ca
file,
it
made
it
really
fragile.
It
worked,
but
it
meant
that
any
time
the
servers
list
changed,
you'd
have
to
rebuild
your
container
right
or,
if
you
needed
to
issue
a
new
token,
you
had
to
rebuild
your
container,
it's
just
stuff
that
you
don't
want
to
have
to
deal
with.
B
It
wasn't
necessary,
so
you
can
see
all
of
this
complexity
around
installing
the
bitfusion
client.
Everything
like
that
is
completely
removed
from
the
container,
and
instead
now
we
can
take
off
the
shelf.
Gpu
accelerated
containers
pass
them
through
the
bit
fusion,
with
kubernetes
integration
that
we've
released
again:
github.com
vmware
bitfusion
with
kubernetes
integration,
and
this
will
just
automatically
bootstrap
bit
fusion
into
any
of
them
and
allow
you
to
allocate
gpus
to
containers
without
any
modification
whatsoever,
and
I
think
with
kubernetes.
B
This
is
particularly
powerful
because,
with
you
know,
more
standard
step,
steady
state
workloads
like
vms
or
even
you
know,
containers
running
in
vms
under
docker,
or
something
like
that,
where
it's
it's
not
being
constantly
spun
up
spun
down
descheduled
or
it's
not
job-based,
or
you
know
that
kind
of
thing
where
you
could
just
mount
a
gpu
into
a
vm
and
it
was
fine
to
live
there
with
kubernetes.
Because
placement
is,
you
know,
pseudo
non-deterministic.
B
That
means
it
could
land
anywhere
in
the
cluster,
even
on
a
different
cluster
or
whatever,
so
instead
of
having
to
buy
gpus
for
every
single
one
of
those
just
in
case
well,
now
that
you
can
have
a
consolidated
set
of
gpus
and
just
allow
the
bitfusion
client
to
do
all
of
the
connection
stuff
for
you,
so
it
means
that
you
don't
have
to
continually.
You
know
unmount
and
remount
gpus
into
things
or
you
know
worry
about.
Is
this
being
underutilized
over
here?
Are
they
actually
using
that
resource?
B
So
that
was
our
kind
of
quick
high
level
whistle
stop
tour
of
the
fusion
4
integration
with
kubernetes?
So
that's
this
bit
here
under
our
tokens,
you
can
now
add
kubernetes
clusters
direct
to
bit
fusion,
as
well
as
the
open
source
bit
fusion
with
kubernetes
integration
project
that
allows
you
to
take
standard,
off-the-shelf,
unmodified
containers
and
run
them
with
gpu
acceleration.
B
A
Thanks
mile,
so
these
are
just
some
links
to
some
of
the
things
miles
showed.
I
did
add
one
thing
that
he
didn't
cover.
This
was
based
on
a
conversation.
I
had
a
hallway
track
here.
I
wanted
to
practice
my
delivery
of
this,
so
I
showed
it
to
somebody
with
a
lot
of
experience
using
gpu
resources
in
an
organization
and
the
person
planted
the
seeds
for
this
aha
moment
where
he
pointed
out
that
the
organization
has
a
lot
of
users
running
these
jupiter
notebooks.
A
You
know
it's
a
it's
sort
of
like
a
ui
for
data
scientists,
and
one
way
to
do
that
is
to
install
this
provision.
Individual
users
with
gpus
in
high-end
laptops,
but
it
the
jupiter
notebook
just
is
a
web
application.
So
you
could
centrally
deploy
that
in
a
server
farm-
and
there
happens
to
be
a
the
jupiter
project
itself
has
put
up
a
helm
chart
for
running
this
at
scale
in
kubernetes.
A
You
know
provision
a
thousand
jupiter
notebook
users
and
I
kind
of
think
that
using
this
as
a
back
end
of
that,
because
the
scenario
with
that
jupiter
notebook
is
people
are
typing
things
in
almost
like
a
spreadsheet,
with
a
lot
of
dead
time.
You
know
it's
think
time
for
the
user
until
they
kick
off
whatever
the
operation.
Is
it's
not
going
to
use
a
gpu
this
technique
of
pooling
it
might
be
saving
money,
energy
maintenance?
Overhead?
A
So
if
you
like
this
presentation,
this
was
sponsored
by
the
kubernetes
vmware
user
group.
We
have
meetings
I'll,
tell
you
in
the
next
slide
how
to
join
it.
It's
me
miles.
We
have
a
couple
user
sponsors,
one
of
them,
joe
cersei
of
t-mobile.
I
don't
see
him
in
the
room,
but
he's
walking
around
the
conference.
Oh
there,
he
is
right
there,
but
you
know
we.
We
have
these
monthly
meetings,
sometimes
there's
a
speaker
giving
talks
on
a
topic.
I
don't
know
like
this
other
times.
A
We
just
declare
them
bring
us
your
problems
or
presentations
on
best
practices.
Once
again,
these
apply
to
using
any
form
of
kubernetes
on
top
of
the
vmware
infrastructure.
So
picture
the
base
vsphere
as
the
equivalent
of
a
public
cloud
platform.
You
can
bring
any
kubernetes
distro
you
like
there.
This
isn't.
This
isn't
vmware
product
centric.
This
is
whatever
kind
of
kubernetes
you
got
if
you're
running
it
on
vsphere
on-prem.
A
A
If
you
want
to
get
a
hold
of
the
speakers,
these
are
our
github
and
twitter
handles.
But,
to
be
honest
with
you,
the
it's
probably
easier
to
get
a
hold
of
us
in
that
slack
channel,
and
I
think
we
only
have
a
couple
minutes
for
q.
A
and
miles
is
not
physically
here,
but
he
was
going
to
be
monitoring
in
that
kubernetes
slack
channel,
which
is
our
normal
one.
A
So,
even
if
you
have
questions
after
this
is
over
later
today,
next
week
go
ahead
and
ask
them
in
that
channel,
and
then
this
is
a
link
where
you
can
download
this
deck.
A
lot
of
the
things
that
I
showed
up
here
on
the
screen
are
hot
links
behind
them,
so
you
should
be
able
to
conveniently
go
to
these
resources
that
we
just
talked
about,
and
with
that
said,
maybe
we
have
a
minute
for
a
question
or
two.
If
we
run
out
I'll
hang
around
in
the
hallway.
D
A
You
want
to
have
fast
network,
so
I
would
not
recommend
this
if
you
have
less
than
10,
gig
e
and
probably
40
is
better
100,
100
gig
ethernet,
even
better,
and
it
depends
what's
going
on,
is
really
you're
feeding
a
pipeline
that
goes
to
the
gpu
and
usually
what
comes
back
is
relatively
small
compared
to
what
has
to
go
over
there
and
it's
a
scenario
where
there's
a
little
latency
on
getting
that
pipeline
filled.
But
if
you've
got
jobs
that
run.
You
know
this.
A
This
probably
wouldn't
pay
off
for
jobs
that
run
100
milliseconds
or
something.
But
it's
my
understanding
that
many
of
these
applications,
some
of
them,
might
even
run
for
a
day
or
more,
even
with
the
gpu,
many
of
them
run
for
minutes
and
hours,
and
that
initial
pipeline
effort
is
just
kind
of
a
one-time
thing.
A
It
is
algo,
it
is
dependent
on
your
algorithm
and
the
ratio
of
kind
of
gpu
think
time
to
actual
data
that
has
to
be
loaded
into
that
fast
access
frame
buffer
over
on
the
gpu,
so
your
mileage
would
would
vary,
but
it
works
for
very
many
popular
jobs.
And
it's
my
understanding
that
when
you
get
gpu
service
and
some
of
the
public
cloud
providers
that
you
might
have
a
similar
thing
going
on
too,
depending
on
how
they
chose
to
implement
it
under
the
covers.
E
A
A
This
is
up
to
you.
It's
like
you,
just
write
it
as
if
you
were
running
your
machine
learning,
deep
learning,
app
inside
docker
or
inside
a
kubernetes
pod,
and
this
is
just
taking
on
virtualizing
a
cuda
interface
out
of
out
of
physical
gpus
that
aren't
necessarily
on
your
worker
node.
So
in
terms
of
what
goes
on
in
the
stages,
if
you're
using
it
for
deep
learning,
this
doesn't
get
there
that
that
would
have
to
be
in
the
code
that
you'd
implement
in
the
kubernetes
pod.
A
A
You
know
literally
mounted
in
a
pci
express
box
on
the
very
box
you're
on
you
know
there
isn't
this
isn't
quite
up
to
the
same
bandwidth
and
there
is
going
to
be
latency
associated
with
the
network.
How
big
a
deal
that
makes
is
kind
of
application,
dependent
so
longer
running
jobs.
I
think
that
might
not
show
up
so
much
and
jobs
that
don't
need.
You
know
the
less
data
that
has
to
flow
in
there
to
fill
the
frame
buffers
the
better
off.
A
Yeah
it
should,
I
can't
say
I
personally
tried
it,
but
I
think
that's
orthogonal
to
this
as
well.
Thank
you,
okay!
So
I'll
get
you
in
the
hallway.
I
see
we
have
one
more
question,
but
I
think
we're
over
time
so
catch
me
in
the
hallway,
but
thanks
for
attending,
like
I
say
if,
if
you
have
more-
and
you
can't
meet
me
in
the
hallway
just
reach
out
join
the
group
join
our
monthly
meetings,
but
thanks.