►
From YouTube: Kubernetes UG VMware 20221006
Description
October 6, 2022 meeting of the Kubernetes VMware User Group.Agenda: recap recent updates to vSphere CSI storage driver;
recap recent updates to vSphere Kubernetes Cloud Provider;
vSphere 8 announcement at the VMware Explore conference; Finding and removing stale CNS volumes in vSphere see: https://vrabbi.cloud/post/finding-and-removing-stale-cns-volumes-in-vsphere/
A
Hi
welcome
to
the
October
6
meeting
of
the
kubernetes
VMware
User
Group,
where
we
talk
about
running
all
forms
of
kubernetes
on
top
of
VMware
infrastructure
in
the
agenda
for
the
meeting
today.
I've
nominated
topics,
including
recent
changes
to
the
vsphere
CSI
storage
driver,
as
well
as
recent
changes
to
the
VMware
vsphere
cloud
provider,
will
cover
lightly
some
of
the
announcements
related
to
vsphere
that
were
made
at
the
recent
VMware
Explorer
conference.
The
reason
it's
light
is
I
think
when
it
comes
to
kubernetes.
A
You
know
the
support
for
running
it
on
that.
Well,
frankly,
the
even
though
the
ga
was
originally
slated
for
late
September,
it
got
delayed,
so
it's
not
even
out
there
yet
and
it
will
take
a
while
for
generic
kubernetes
distros
to
support
it,
but
I'm
just
going
to
give
you
some
links
to
where
you
can
learn
more
about
it
primarily
and
then.
Finally,
we'll
touch
on
the
discussion
of
user
groups,
just
because
at
the
upcoming
kubecon
conference
there
is
a
session
I'm
hosting
on
User
Group
formations.
A
Before
we
hit
record,
I
was
talking
informally
about
this
with
Scott,
but
we'll
go
on
during
the
meeting
and
go
over.
What's
going
on
there
and
I
just
wanted
to
see
poll
people
for
anyone
on
this
call
who
might
be
physically
attending
because
we've
in
the
past
tried
to
get
face-to-face
Gatherings
just
to
see
you
know
who's
there.
Sometimes
it's
nice
to
actually
meet
people
that
you've
been
meeting
on
zoom
and
meet
them
face
to
face.
A
A
So
I
already
said
what
the
agenda
items
are
and
by
the
way
we
might
have
time.
So
if
anybody
wants
to
bring
an
additional
item
that
isn't
here,
go
for
it,
I
haven't
had
a
chance
to
go.
Look
at
the
agenda
notes
document
to
see
if
anybody
formally
added
an
agenda
node
stock,
but
we're
so
small.
You
can
just
like
up
your
mic
and
do
it
verbally,
so
the
CSI
driver
updates.
We
haven't
really
had
a
meeting
since
June.
So
it's
been
a
number
of
months.
A
The
2.60
came
out
that
added
support
for
kubernetes
1.24.
Also
added
this
enhanced
volume
topology
feature
I'm,
not
sure
how
many
it
pertains
to,
but
for
people
who
got
started
in
their
Journey
or
running
on-prem
data
centers.
This
is
a
feature
that
would
let
you
land
a
kubernetes
cluster
on
a
situation
where
you've
got
a
high
availability,
utilizing
storage,
replication
to
a
I'll
call
it
a
backup
data
center.
A
If
you
will-
and
these
are
based
on
active
passives
storage,
so
the
idea
would
be
that
your
backing
store
for
persistent
volumes
in
kubernetes
would
nominally
be
recording
to
the
local
one,
that's
hosting
everything.
But
if
you
had
a
situation
either
a
planned
or
an
unplanned
failure,
let's
say
you
had
a
hurricane
coming
in
and
wanted
to
potentially
set
yourself
up
to
fall
back
on
an
alternate
location.
A
You
could
take
advantage
of
storage
to
do
this,
so
it's
pretty
specialized
and
I'm
going
to
publish
this
deck.
I
haven't
uploaded
it
to
a
share
yet,
but
I
will,
when
this
meeting's
over
and
put
the
link
in
the
agenda
note.
Stock
I'll
also
drop
it
in
the
slack
Channel,
but
I'd
encourage
you
to
follow
the
doc
link,
because
it
goes
into
a
fair
amount
of
detail
on
this,
and
that's
where
that
picture
came
from
the
other
things
that
made
it
into
six.
A
Oh,
are
migration,
so
this
would
be
for
people
who've
been
using
kubernetes
for
a
while
that,
while
being
so
long
that
you
got
started
out
using
the
in-free
storage
driver,
so
I
think
that
would
put
you
back
at
least
a
couple
of
years
and
if
you're
now
the
fact
is
you're
at
some
point,
you're
forced
to
move.
So
this
would
be
a
feature
that
provides
for
support
of
the
that
sort
of
migration.
A
From
the
entry
storage
Shriver
to
the
modern
CSI
based,
there
is
a
feature
that
supports
read,
write
many
so
read
write.
Many
is
something
that's
been
there
for
NFS
for
a
while
I
think
it's
probably
most
commonly
used
for
people
doing
web
apps.
That's
that
utilize
shared
back-end
storage
with
multiple
writers.
So
this
feature
is
something
that
would
enable
you
to
do
that
kind
of
scenario,
with
a
vsan
backing
store
and
then
finally,
this
last
feature
only
applies
to
Google
amphibos,
but
well
you
could
read
it.
A
A
A
To
read
these
to
you,
you
can
probably
read
them
yourself
and
once
again
they're
in
the
release
notes,
there
are
some
known
issues
that
I
think
people
do
run
into
these.
Occasionally
these
aren't
all
the
known
issues
by
the
way,
I
just
highlighted
some
that
I
think
people
run
into
more
than
others,
and
once
again,
these
known
issues
are
in
the
release
notes.
But
if
you
were
to
upgrade
vsphere
itself
the
underlying
hypervisor
platform,
it's
you
might
potentially
run
into
this
issue.
A
B
A
Yeah
I,
you
know
I'm
not
working
on
the
development
of
that,
but
it
sure
would
be
nice
if
you
made
it
impossible
to
handle
to
to
happen,
but
I
suspect
that
it's
harder
than
you
might
think,
because
I
think
that's
been
there
for
a
while.
Anyway,
it's
there
and
I
just
wanted
to
call
it
out,
because
I
think
it.
B
A
It's
there
moving
on
from
the
storage
to
the
cloud
provider,
the
1.24
came
out
now
this
actually
came
out
in
I
think
last
week
of
June,
but
it's
been
a
while,
since
we
touched
on
this
subject,
so
just
calling
it
out
there
it
it
the
the
main
and
only
thing
I.
Think
in
one
two
four
was
the
support
for
kubernetes
one
two
four,
but
there
are
a
number
of
subsequent
patch
things
for
these
issues
called
out
here,
related
to
remote
off
I'm,
not
really
sure
what
that
Legacy
para
virtual
is
about.
A
Vsphere
8
was
announced
in
when
it
was
announced
at
VMware
Explorer
they
aspired
to
putting
it
out
there
for
download
in
late
September
that
gets
delayed
and
it
still
isn't
out
today
I
believe
I
kind
of
rushed.
This
I
was
working
on
this
deck
the
hour
before
the
meeting,
so
I
didn't
have
time
to
actually
look
at
the
new
date.
A
If
you
only
want
to
devote
like
five
minutes
to
pick
it
up,
if
you
got
more
time
all
of
the
sessions
from
VMware
Explorer
are
now
available.
I
think
William
lamb
did
the
best
job
of
cataloging.
These
I
sort
of
like
it
better
than
going
to
the
official
site
myself
but
hey,
maybe
you
maybe
you
like
the
Explorer
site
bottom
line.
Is
you
can
get
these
now
for
free?
A
Although
to
play
them,
you
do
have
to
establish
a
free,
login,
ID
and
password,
but
once
you've
done
that
the
William,
what
lab
site
will
Point
give
you
the
titles
Vector
you
directly
to
the
individual
sessions
and
you
can
play
them
at
will.
There
were
a
few
people
who
you
know
the
the
VMware
Explorer
ones
would
be
the
official
and
they're
going
to
run
over
30
minutes
each
and
cover
cover
them
in
some
detail.
A
A
I
believe
that
some
of
the
other
things
that
aren't
really
a
technical
thing
related
to
kubernetes
is
that
you
know
vsphere
8
tends
to
be
licensed
in
a
different
way,
and
things
like
that
that
might
have
some
impacts
on
how
you
might
go
about
taking
advantage
of
it
too,
so
that
was
it
for
the
material
in
my
slide
deck.
So
the
final
item
that
I
had
put
on
my
agenda
was
talking
about.
A
If
you
asked
for
one
and
I,
don't
to
me
that
signaled
that
you
know
they
needed
to
cut
back
on
session
count
so
rather
than
even
try,
I
just
voluntarily
didn't
submit
this
time.
However,
I've
got
another
session.
That
is
a
generic
session
on
user
groups,
and
this
is
the
idea
that
Scott
and
I
started
talking
about
this
before
this
meeting
formally
started
that,
in
my
opinion,
this
user
group
isn't
as
good
as
it
potentially
could
be,
because
we've
got
kind
of
light
attendants.
A
You
know
when
we
started
this
over
two
years
ago,
who's
to
say
what
we
expected,
but
I
kind
of
dreamed
that
we
might
get
attendance
up
there.
You
know
approaching
50
to
100
people,
but
it
never
really
happened.
I
think
we've
probably
had
a
few
sessions
that
we
maybe
briefly
cost
over
40,
but
the
average
rope
probably
has
been
10,
and
that
might
be
a
signal
really
that
we
went
too
narrow
in
Focus
or
that's
what
I'm
thinking
anyway.
A
A
Right
and
the
idea
there
was
to
cover
not
just
home
online
worldwide
user
groups
like
this,
but
before
covid
there
were
quite
a
few,
very
healthy
local
user
groups,
often
on
meetup.com
that
physically
met
where
I
live
here
in
Los
Angeles.
It
got
so
popular
that
actually
two
user
groups
were
forked,
I,
think
and
the
attendance
was
very
high.
A
If
you're
going
to
be,
you
know
turning
it
into
a
social
thing
so
anyway,
in
Los
Angeles
things
got
so
popular
that
a
west
side
and
an
Eastside
group
got
formed
because
La
is
kind
of
sprawling
too
to
where,
if
you're
on.
If
you
live
or
work
on
east
side,
but
the
meetings
on
West,
it
might
take
you
over
an
hour
to
get
to
the
thing.
So
the
kubernetes
meetup
groups
forked
into
two,
but
then
when
covet
hit.
It
just
died,
and
a
few
of
us
have
been
trying
to
reinvigorate
that.
A
However,
there,
the
companies
that
used
to
host
it
still
have
lingering
hesitation
related
to
covet
here
and
we
haven't
managed
to
have
our
first
physical
meeting
group.
Yet
there
are
a
few
user
group
meetings
that
moved
online
very
successfully.
So
one
of
the
people
who
was
active
in
this
group,
Joe
Cersei,
who
was
with
T-Mobile
at
the
time,
is
one
of
the
co-organizers
of
the
kubernetes
Atlanta
user
group
and
they
managed
to
move
online,
get
good
speakers
and
get
zooms
going.
A
That
might
have
had
a
hundred
people
or
more
in
attendance,
and
they
did
very
well
some
of
the
others,
never
really
pulled
off
that
successful
move
to
zoom,
exercise
and
I
think
a
lot
of
them
are
potentially
trying
to
get
back
to
physical
meetings.
At
this
stage.
You
know,
I
think
that
one
of
the
attractions
of
physical
meetings
are
that
people
can
join
these
user
groups
for
a
number
of
different
reasons.
A
I
think
work
better
when
you've
got
the
social
aspects
of
a
face-to-face
meeting
going
on
anyway,
this
session
at
kubecon
that
I'm
going
to
be
hosting
I'm
looking
forward
to
trying
to
have
attendance
by
both
User
Group
organizers,
as
long
as
well
as
users
and
just
reach
out
for
any
ideas
people
might
have,
there
are
going
to
be
people
from
the
cncf
there
as
well.
So
joining
me
will
be
Taylor
Taylor
Who's,
the
I
think
his
title
is
head
of
community
or
kubernetes,
or
maybe
it's
head
of
community
for
the
cncf.
A
But
in
any
event
he
is
going
to
be
joining
me
for
that
birds
of
feather
and
we're
looking
forward
to
maybe
coming
up
with
some
ideas
for
how
to
best
structure
user
groups
under
the
cnco
one
of
the
things
I'm
going
to
throw
out
there.
But
I'm
going
there
more
to
listen
but
I'm,
going
to
put
this
idea
on
the
table
that
maybe
it
would
be
best
to
structure
these
user
groups
along
the
lines
of
one
for
on-prem
users
and
a
second
one
for
public
Cloud
users.
A
My
thinking
being
that
you
know
this
group
has
been
on-prem
clearly
but
for
people
doing
on-prem
on
vsphere,
but
I'm,
not
sure
how
much
Delta
there
is
between
that
and
people
running
on-prem
on
bare
metal
or
on
some
other
hypervisor
solution.
I
think
there
are
a
lot
of
things
that
all
these
people
have
in
common
need
for
load.
Balancers
would
be
just
one
example,
but
also
posting
your
own
storage
being
concerned
with
implementing
your
own
availability
zones.
A
B
Yeah,
no
just
one
thing
that
I
will
throw
out
there.
That's
it's
something
you
had
mentioned
load
balancers
on
premise,
and
it's
something
that
we've
talked
about
a
lot
here
in
the
group.
I
think
one
of
the
interesting
ones
that
people
can
take
a
look
at
is
we've
mentioned,
like
Cube,
VIP
and
metal
lb
in
the
past
is
lightweight
solutions
for
when
running,
on,
vsphere,
there's
actually
now
Andrea,
which
is
an
open
source.
Cni
has
a
service
type
load,
balancer
implementation
itself
as
well.
B
A
B
Yeah,
it's
pretty
easy,
it's
I
think
that's
1.6
or
1.7
of
Andrea.
That
was
added
we're
on
1.8.
Now,
though,
it
is
feature
gated,
so
you
have
to
enable
the
feature
gate,
but
you
basically
create
an
IEP
pool.
It
is
only
Layer
Two,
so
it's
the
same
network
as
your
nodes,
but
it
will
publish
the
service
type
load
balancer
for
you.
A
It's
kind
of
interesting
because
that
whole
like,
like
I
mentioned
earlier
I'm
active
in
kubernetes
on
edge
and
load
balancer,
is
a
tough.
It's
it's
a
challenge.
There
I
mean.
Sometimes
people
don't
even
want
to
public
put
services
on
the
air
from
Edge
that
are
neat
accessibility
from
the
outside.
But
if
you
do
you're,
typically
resource
Challenge
and.
B
To
examples
and
when
you're
running
on
edge
nodes,
every
pod
counts
right,
so
even
just
Cube
VIP,
which
runs
as
a
demon
set.
It's
a
very
lightweight
demon
set,
but
it's
still
another
demon
that
you
need
on
every
node,
plus
the
cloud
controller
manager
for
it
as
well.
It's
just
extra
compute
that
you
don't
necessarily
need
to
use.
If
you
already
have
the
compute
running
as
part
of
your
cni.
A
Okay,
I'll
give
that
a
shot
that
that
intrigues
me
I,
I
hadn't
been
tracking
Andrea
for
over
a
year,
so
I'll
I'll
take
a
look
at
that:
I'm,
actually
I'm
running
a
whole
Lab
now
myself
and
have
been
doing
Cube
fit,
but
you
know
historically,
all
of
these
load.
Balancers
kind
of
go
through
pendulum
swings,
I.
Think
too,
where
you
know
the
one
that
was
sort
of
leading
in
popularity
a
year
or
two
ago
has
often
shifted
to
a
different
one,
with
the
original
not
disappearing
for
the
most
part.
B
B
Another
issue
that
We've
encountered
a
lot
I
just
wrote
a
blog
post
on
this
as
well
with
some
scripts
within
it
to
help
one
of
the
biggest
issues
that
we've
seen
recently
with
CSI
on
vsphere
is,
if
you
delete
a
cluster
without
deleting
all
the
persistent
volumes
in
advance,
they
all
stay
in
vsphere
and
I
ended
up
with
over
seven
terabytes
of
leftover,
vsphere
or
first
class
discs,
and
there's
no
clear
way
in
vsphere
UI
to
actually
manage
this.
There's
no
bulk
option
for
deleting
there's
no
filtering
capabilities
in
the
UI.
B
Really
that
are
useful
for
being
able
to
find
these
things
so
using
go.
Vc
actually
has
some
great
capabilities
in
there.
So
I
wrote
a
blog
post
on
how
to
kind
of
pull
out
a
report
of
all
of
the
persistent
volumes
which
cluster
they're
a
part
of
what
namespace
persistent
volume
claim,
data
that
gets
bubbled
up
to
vsphere
and
was
able
to
clear
seven
terabytes
of
yeah.
That
sounds
great
I'll.
A
Take
a
look
at
that
for
sure
one
thing:
I
wanted
to
throw
out
there
that
I
discovered
within
the
last
six
months
is
when
you
start
reading
about
these
vsphere
CLI
tools.
You
know
whether
it's
go
VC
or
Govi
mommy,
or
even
the
Powershell
things
that
I
sort
of
grew
up
in
an
era
when
I
think
it
was
because
Powershell
became
so
popular
for
examples
that
you
felt
you
had
to
bring
up
for
convenience,
a
Windows
node
to
be
running
this
stuff,
but
I
went
and
backfilled
and
tried
these
days.
A
Powershell
runs
perfectly
well
on
a
Linux
host
and
I
went
and
installed
the
Powershell
Linux,
and
you
can
kind
of
script
deployment
of
a
Linux
VM
that
has
all
of
these
tools.
Pre-Installed,
including
the
go,
go
VC
and
go
V
Mommy
to
just
kind
of
make
a
generic
control
point
to
be
doing
all
of
these
utilities.
That
can
do
things
that
you
can't
easily
do
or
can't
do
at
all
in
the
vsphere
UI.
A
C
B
C
B
A
B
A
B
Exactly
well,
and
it's
especially
for
things
like
vsphere
or
on-prem
Solutions,
where
AWS
is
one
version,
they
need
to
really
support,
which
is
the
AWS.
That's
live
right
now
you
need
to
support.
You
have
people
still
using
versions
before
six,
seven,
which
is
why
people
are
still
releasing
bug
fixes
for
the
internal
entry
provider
in
six,
seven
and
seven
and
all
the
versions
of
seven,
and
so
any
new
feature
that
comes
in
has
to
be
backwards,
compatible
always
with
the
latest
supported
versions,
and
it
just
becomes
a
nightmare.
A
It's
been
a
while
years
ago,
I
worked
on
the
backup
features
in
vsphere,
and
one
of
the
things
that
makes
it
even
harder
than
you
might
think
is
that
people
have
taken
backups
of
VMS
years
ago.
So,
even
though
some
of
these
ancient
vsphere
releases
are
quote
no
longer
supported,
there
are
people
who
who
took
VM
backups
and
have
a
10
year
old,
VM
backup,
and
you
really
do
want
to
give
them
an
ability
to
recover
that
backup,
because
you
know
they're
being
kept
around
just
in
case
in
some
cases.
A
A
B
It's
one
of
the
biggest
challenges
that
I've
always
brought
up
with
kubernetes
backups
and
the
difficulty
of
kubernetes
backups
is
with
the
speed.
Kubernetes
is
moving
and
the
fact
that
everything
is
becoming
crds
basically
and
they're
changing
API
versions.
Every
two
days
is
that
at
some
point
you
take
a
backup.
Six
months
later,
you
may
not
be
able
to
restore
it,
because
those
API
types
don't
exist
anymore.
It's
like
I
can
think
of
any
backup
that
was
taken
a
year
ago
was
probably
using
V1
beta,
1
ingresses
and
now
that
doesn't
exist
anymore.
A
And
you
know
I
understand
the
attraction
of
moving
things
out
of
tree
and
going
with
custom
resource
definitions
and
things
and
if
you
start
dragging
in
those
as
aspects
of
what
you
backed
up
and
then
try
to
do,
a
restore
to
a
scenario
where
maybe
you
even
I
don't
know
flipped,
kubernetes,
distros
or
something
boy
I,
it
seems
like
there's
a
I.
Fortunately
haven't
been
called
upon
to
do
that
myself,
but
I'd
start
getting
really
nervous
on
how
well
you
could
expect
that
to
work
if
at
all
and
typically
maybe.
B
Some
of
the
backup
Solutions
are
working
on
API
conversion
capabilities
where,
because,
luckily
within
kubernetes
I
mean
persistent
storage
is
persistent
storage
because
that's
just
a
Storage
level
right.
So
that's
relatively
easy
if
you're
using
something
like
restic
or
things
like
that,
where
it's
literally
just
block
data
and
you're
just
shoving
it
back
into
a
persistent
volume
who
really
cares
if
it's
moving
from
kubernetes
112
to
1
30.?
It's
you
know,
data
is
data.
The
API
types
there's
talks
about
building
in
the
capability
for
like
doing
basically
conversions.
B
C
Yeah,
it's
definitely
like
one
of
those
things
where
it's
like
a
legacy
idea
that
has
to
be
revamped.
The
same
kind
of
thing
we
have.
We
get
like
very
similar
things
where,
like
you,
get
a
Visa
admin,
who's
just
adopted
like
tanza,
or
something
like
that
and
the
it's
great
for
VMware,
because
we've
removed
the
barrier
of
Entry
of
having
to
like
set
up
kubernetes
by
yourself,
but
it's
also
hard
for
VMware,
because
you
get
a
VMware
admin
who
you
know
typically
some
people
in
there
or
they
won't
upgrade
for
years
at
a
time.
C
But
then
we
have
kubernetes
certificates
and
stuff
that
is
made
by
cluster
API.
That
are,
you
know,
set
by
default
at
a
year.
Yeah
so
like
we
probably
had
a
decent
amount
of
customers
have
hit
that
issue,
because
the
old
sense
of
like
set
it
and
forget
it
and
don't
upgrade,
has
now
affected
them
on
the
kubernetes
level
that
is
set
for
upgrade
all
the
time
and
and
minus
two.
You
know,
oh.
B
Yeah
and
that's
not
that's
not
a
tanzu
issue,
that's
how
you
know.
Overall,
we
see
that
without
a
distro,
that's
going
to
be
Rancher,
that's
going
to
be
eks
anywhere
and
those
do-it-yourself
kubernetes,
and
if
you're
doing
Cube,
ADM
you're
getting
one-year
certificates
unless
you're
generating
yourself
and
putting
in
special
Flags
telling
it
to
use
your
own
certificates
right,
Cube
ADM
is
generating
one-year
asserts
and
I
bumped
into
customers.
That
till
today
are
using
kubernetes
114..
B
C
A
You
know
back
in
the
old
days
with
you
know:
backup
has
been
in
I.T
for
decades,
and
you
know
there
used
to
be
kind
of
the
generic
solutions
to
backup
and
restore
you
know,
even
our
sinking
and
things,
but
one
of
the
reasons
the
big
vendors
earn
their
money
for
selling
high
cost
Solutions
is
that
they
had
Solutions
on
thinking
here
in
the
file
system,
backup.
But
this
might
be
a
scenario.
A
lot
like
the
stale
certificate
issue
that
when
you
did
a
com,
a
high-end,
commercial
and
backup
thing
you
would
be
backing
up.
A
A
So
you'd
restore
the
file,
but
you
couldn't
read
it
or
do
anything
with
it
and
a
lot
of
those
commercial
backup
products
would
have
all
kinds
of
complex
controls
to
recover
it
while
stripping
that
stuff
and
it
to
my
knowledge,
nobody's
done
it,
but
perhaps
a
proper
high-end
kubernetes
backup
and
restore
thing
that
round
tripped.
It
would
be
part
of
their
job
to
try
to
patch
in
certificate
Replacements
automatically.
As
part
of
that
recovery.
A
And
it
was
kind
of
interesting
writing.
The
code
for
even
those
user
substitutions
you'd
have
to
substitute
users
groups,
maybe
in
some
cases
whole
organizations,
because
another
scenario
that
happened
in
backup
is
a
company,
buys
another
company
and
merges
the
it
organizations
so
that
the
root
certificates
are
just
no
longer
in
existence
and
I
can
imagine
that
there's
an
awful
lot
of
person
years
of
work
to
get
that
to
work
right,
but
there's
probably
demand
for
it
somewhere
too
I
guess.
A
Another
approach
would
be
just
this
throw
in
the
towel
and
say
we
have
no
expectation.
We
could
ever
recover
our
kubernetes.
We
won't
even
try,
but
we
are
running
persistent
apps.
We
try
to
keep
everything
stateful
in
databases
and
we
just
back
them
up
at
the
individual
data
service
level
and
right
get
to
a
situation
where
we
can
guarantee
that's
doable
and
we
don't
aspire
to
even
taking
on
anything
more
than
that,
whether
that
really
is
workable
or
not.
Who
knows
it
certainly
is.
B
It's
where
the
regulations
need
to
adapt
to
that,
because
I
think
that's
where
like
get
Ops
is
the
right
solution
for
that
right,
like
if
you're
using
a
git
Ops
model
to
deploy.
You
have
get
revisions
back,
so
you
you
definitively
have
for
your
definitions
within
kubernetes.
If
dealing
with
kubernetes
the
correct
way
through
like
get
Ops,
you
have
a
backup
in
sense
of
the
configuration
that
was
in
a
cluster.
B
A
Yeah
and
it's
tough
to
have
in
the
in
the
old
VM
Solutions,
you
could
pretty
much
put
things
together
where
you
could
pull
off
kind
of
a
one-button
restore,
it
might
be
expensive.
You
might
have
to
buy
a
ton
of
Hardware
to
go,
recovered
and
stand
it
up,
but
I
don't
really
know
that
the
one
button
thing
is
doable
in
kubernetes
these
days.
B
I'm,
slowly
reaching
the
realization
that
I
don't
think
it
is
and
I
think
that
most
backups
today
are
just
for
regulatory
purposes
and
aren't
really
like
you,
have
the
backup
and
restore
immediately
right.
You
have
that
type
of
a
scenario
for
moving
off
of
one
platform
to
another.
You
backup
you
restore
somewhere
else
and
you're
done
they're,
basically
the
same
version
of
kubernetes.
Okay,
you
can
solve
that
or
you
have
the
short
term.
You
know
30-day
backups.
B
B
A
B
B
B
There
was
always
like
office
hours
working
groups
for
AWS
and
vsphere
I'm
wondering
if
that's
a
part
of
what
would
be
like
I
think
that
there's
a
level
of
a
user
group,
that's
important
and
I-
think
there's
also
a
level
of
a
working
group
of
vsphere,
which
would
be
a
technical
like
okay,
what's
going
on
with
vsphere
CSI
or
I'm
having
an
issue
with
VC
or
CSI.
I
think
that
those
are
two
different
things,
but
I
think
that
both
of
them
are
valuable,
meaning
I'm,
not
sure
which
one
more
people
would
be
interested
in.
A
Yeah
there
were
brief
periods
where
I
can
only
speak
for
some
of
the
ones
I
was
in
so
there
there
was
aspirationally
a
period
when
there
were
office
hours.
If
you
will
for
the
vsphere
cloud
provider,
but
I
think
what
was
happening
was
users
would
only
show
up
once
every
five
months
when
they
had
an
issue,
and
after
and
between
times
the
meeting
would
come
up
and
it
would
be
dead
air,
and
if
that
happens
over
and
over
pretty
soon,
people
are
saying.
A
B
Or
I,
luckily,
I
have
face
to
face
right,
so
I
have
enough
connect
that
I
can
turn
to
an
r
d
folk
of
whatever
whether
it's
you
know
and
I'll
also
just
send
something
in
the
slack
channels.
I
think
some
people
I'm
just
thinking
from
like
a
user's
perspective,
I
think
there
are
some
people
who
would
like
just,
for
example,
some
of
the
open
source
projects,
whether
it's
kubernetes,
you
know
sigs
or
other
things.
B
You
know
I,
some
projects
are
once
a
month
there's
an
office
hours
or
whatever,
but
where
as
like
and
then
during
those
meetings,
instead
of
them
being
dead
air,
what
you
see
a
lot
of
projects
do
is
that's
when
they'll
do
like
their
backlog.
Trimming
grooming
is
during
that
meeting
and
then
you'll
have
the
developer
team
there,
and
if
anyone
comes
that
has
questions
awesome
and
otherwise
it's
backlog.
B
A
C
A
Csi
and
you
had
devs
out
of
different
CSI
drivers
coming
together
and
kind
of
what
goes
on.
There
was
kind
of
interesting
because
it
I
think
there's
even
in
terms
of
moving
things
towards
a
goal.
Bringing
all
of
these
different
CSI
drivers
together
in
a
meeting
is
a
critical
element
to
get
uniformity
in
terms
of
when
the
things
they
are
doing
features
they're
aspiring
to,
because
the
fact
is,
if
they
drifted
or
a
kubernetes
user.
A
Eventually,
if
all
of
these
CSI
storage,
plugins
aren't
in
an
alignment,
aren't
in
alignment-
and
are
you
know,
plus
or
minus
two
years
on
feature
adoption
and
things?
It
becomes
a
pretty
ugly
world
to
actually
say
that
kubernetes
is
this
abstraction
layer
that
provides
a
proper
spot
for
cloud
native
applications
that
could
run
anywhere.
B
Yeah,
no
I
I
definitely
think
that
there's
you
know
a
lot
of
truth.
Then
I
think
there's
a
lot
of
importance
of
this
cross-cloud
provider
at
least
knowledge
sharing
that
exists
because
I
mean
you
see
this
with
I.
Just
like
take
Ingress
is
the
best
example
of
the
failure
that
has
happened
where,
luckily
we're
getting
back
to
it
with
Gateway
API
now
to
a
good
place,
but
Ingress
you
had
you
have
a
hundred
different
implementations
of
Ingress
controllers.
B
Everyone
implemented
different
annotations
and
different
ways
to
add
features
that
were
needed
that
aren't
part
of
the
Ingress
back.
So
you
have
different
Ingress
providers
implementing
the
same
core
object,
but
you
can't
back
up
a
Ingress
that
was
used
with
nginx
and
deploy
that
somewhere,
that's
using
Contour
or
whatever,
and
expect
it
to
work
yeah,
because
the
Thousand
annotations
that
nginx
has
are
nginx
specific
and
same
thing
with
contour
and
same
thing
with
you
know,
Kong
and
same
thing
with
any
of
these
different
solutions
and
CSI.
B
Is
that
same
thing,
it's
a
single
abstraction,
that's
core
kubernetes
that
is
being
implemented
with
vsan
specific
features
and
with
Google
specific
features
and
EBS
and
EFS,
and
you
know
Port
works
or
whatever
it
ends
up
being
Seth,
and
you
definitely
have
that
need
for
some
level
of
commonality.
Yeah.
A
And
if
you
could
get
users
in
there
to
kind
of
voice
the
issue
with
the
difficulties
of
all
of
this
variation
I
think
that
maybe
the
world
would
change,
but
if
the
users
aren't
even
aware
of
the
meetings
or
the
meetings
don't
exist
that
are
appropriate
for
users
to
attend
yeah
it.
It
would
get
like
your
Ingress
scenario.
I
could
see
that
happening
all
over
the
place.
A
Yeah
interesting
challenge
there
because
you
know,
even
if
you
provided
the
meeting,
that
would
be
a
sounding
board
to
Aspire
to
uniformity.
It's
almost
like,
if
you
look
at
where
things
did
largely
work,
reliably
across
vendors,
you
know
take
vsphere
storage,
for
example.
It
probably
requires
a
compliance
test
and
some
sort
of
mandatory
imposition
of
certification
that
you
really
even
meet
a
standard.
B
Well-
and
you
also
don't
want
to
stop
someone
from
being
a
of
creating
their
own
internal
CSI
driver
that
works
with
their
in-house
storage
system
that
and
they
don't
need
volume
snapshots.
So
why
do
they
need
to
implement
CSI
snapshots
right
like
there
shouldn't
be
a
need
for
that,
but
then
they're
not
portable
right.
So
it's
like
how
do
you,
what
does
actual
conformant
mean
and
that's
I,
think
where,
like
you
reach,
these
difficulties
is
what
is
conformant
and
conformant
could
be
you're
confirmed
with
the
Baseline
okay,
but
everything
in
kubernetes
isn't
Baseline
anymore.