►
From YouTube: Deep Dive: SIG Release - Tim Pepper, VMware
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Deep Dive: SIG Release - Tim Pepper, VMware
This session on the Release Special Interest Group will give a brief overview of the kubernetes release process and then shift focus to open discussion of portions of this process which give value to kubernetes vendors, distributors, and service providers, and portions of this process which could use enhancing to give better value. Input from the Chinese portion of our vendor ecosystem is especially sought.
To learn more click here: https://sched.co/FuLi
A
A
So,
just
briefly,
who
here
is
familiar
with
the
term
LTS
a
few
ok
so
briefly,
also
who
was
here
yesterday
for
the
cig,
release,
intro,
ok!
So
briefly,
I'll
talk
about
the
release
process
and
the
release
lifecycle
and
then
go
into
the
implications
that
the
way
we
do
that
has
on
vendors
distributors
and
hosted
providers
and
how
they
install
and
upgrade
their
clusters,
and
then
I
want
to
open
things
for
a
discussion,
because
I
want
to
understand
pain
points
from
the
community
here.
A
So,
first
briefly,
the
release
process.
We
make
four
releases
of
cur
Nettie's
each
year,
its
quarterly
and
on
a
month
cadence,
and
this
makes
it
look
like
a
waterfall
process,
but
it's
actually
much
more
agile
than
this
there's
always
discussion
going
on
in
the
community
about
features,
and
the
first
thing
then
to
highlight
is
that
we've
stopped
calling
it
features.
We
have
an
enhancement
process
because
there's
more
than
just
features,
there's
bug
fixes
there's
refactoring
code
cleanups,
but
about
four
weeks
into
the
cycle.
A
We
have
our
enhancements
freeze
and
at
that
point
we've
built
up
a
list
of
the
enhancements
that
are
targeted
for
this
release
and
we
asked
the
SIG's
to
focus
their
work
primarily
on
those
and
then
there's
a
period
of
time
where
coding
happens
on
these
and
and
everything
is
moving
along
at
the
rapid
pace
like
open
source
does.
But
then
we
declare
a
code
freeze
about
eight
weeks
in
and
from
this
point
forward,
only
critical
bug
fixes
are
supposed
to
be
merged
into
the
branch.
A
A
It
will
stop
getting
patches
back
ported.
The
three
supported
will
be
11,
12
and
13,
and
then
we
have
the
cycle
that
continues
year
after
year.
So
support
from
a
coding
perspective
is
back
porting.
The
critical
bug
fixes
to
the
release
branch
three
releases,
get
support
and,
like
I
said
right
now,
the
latest
stable
is
1.12,
so
the
three
that
are
getting
support
are
112-111
and
110
as
a
user.
Then
you
see
roughly
nine
months
of
support.
You
can
stay
on
a
particular
code
branch
and
know
that
you
won't
be
insecure.
A
That
important
fixes
will
come
back,
but
what
this
means,
then,
is
you
have
to
plan
an
upgrade
every
three
to
six
months,
four
to
four
times
a
year,
possibly
maybe
maybe
two
or
three,
but
if
you
get
too
far
behind
you
could
get
stuck
and
as
other
talks
this
week
at
the
conference
of
reminded
our
upgrade
path,
is
to
go
from
release
to
release.
You
can't
go
from
release
110
via
grade
2:112.
You
have
to
go
from
110
to
11
what
12
and
iteratively
take
these
single
steps.
A
So
this
then
invariably
brings
up
the
question
of
what
we
call
LTS.
Lts
is
a
term
in
the
state
of
the
art,
both
in
proprietary
software
and
an
open-source
software.
It's
it's
long-established
as
a
thing,
but
but
what
is
it
so?
Only
a
few
hands
came
up
when
I
asked.
If
people
know
what
it
is,
it's
an
acronym
and
it
stands
for
long
term
support,
and
this
amounts
to
a
policy.
Somebody
makes
a
decision
like
what
I've
just
described
here
for
kubernetes.
We
have
three
releases
nine
months
four
times
a
year.
A
A
Some,
maybe
we'll
do
bug
fixes
for
a
year,
just
any
bug
fixes
and
then
narrow
it
down
to
just
the
most
important
security
fixes
for
an
additional
period,
and
then
the
the
important
thing
to
think
about
here
is
that
long
is
measured,
usually
in
either
a
year
type
of
timeframe
or
multiple
years,
and
then
usually,
there's
also
something
for
short
term,
because
this
is
open
source,
it's
rapidly
iterating
along,
and
why
would
you
do
this?
Well,
there's
obvious
benefits
like
I've
I've
alluded
to
as
a
user
of
the
open
source
project.
A
You
don't
want
to
spend
all
of
your
time
planning
and
moving.
You
actually
want
to
have
stable
production.
Usually
your
goal
isn't
to
just
be
changing
kubernetes.
Your
goal
is
to
run
things
on
top
of
kubernetes
and
deliver
value
to
you.
So
all
of
this
churn
can
be
a
distraction
from
you
and
then
beyond.
Just
that
churn
for
your
workloads
that
are
running
on
top
of
kubernetes
having
something
like
LTS.
If
it's
done
successfully,
it
requires
that
there's
good
compatibility
between
releases.
That
means,
as
a
user
when
you
do
upgrade
your
experience,
is
better.
A
Similarly
for
vendors
and
I'll
go
into
this
in
more
detail
later,
but
every
vendor
in
the
ecosystem
they're
doing
some
level
of
support
for
their
customers,
and
surely
they
all
do
something.
That's
unique
value
add
for
their
customers,
but
if
we're
all
doing
this
in
parallel,
there's
also
some
common
portion,
and
if
we
can
do
that
common
portion
together
an
open
source,
we
can
be
stronger
and
get
a
better
result
for
the
end
users
of
all
of
our
systems.
A
Finally,
for
developers
the
idea
of
cross
release
compatibility,
a
good
developer,
wants
to
give
that
to
their
users,
but
it's
hard
to
see
the
way
we
see
that
we
have
problems
on
compatibility
is
through
testing
and
three
end-user
bug
reports.
If
we're
going
to
do
something
like
an
LTS
in
a
open
source
project,
then
we
must
have
good
testing.
So
if
we
go
in
the
direction
of
an
LTS,
it
means
that
we're
going
to
have
better
testing,
which
would
be
a
good
thing
for
our
developers.
A
Now,
it's
not
all
positive,
there's,
actually
problems,
risks
and
costs
to
doing
LTS.
Today
we
have
this
simple
scheme
of
three
releases
a
year
with
support
for
nine
months
and
up
until
this
point
it's
about
to
change
in
a
couple
of
months.
But
up
until
this
point,
not
nine
months
of
support
has
been
handled
by
one
person
in
the
kubernetes
community.
That's
that
means
that
there's
three
people
at
any
time
doing
support
for
everyone.
If
we
add
something
like
LTS,
and
maybe
we
do
one
LTS
every
year
and
support
it
for
two
or
three
years.
A
That
means,
instead
of
three
things
to
test
and
manage
and
staff
for
a
short
amount
of
time.
We
maybe
have
six
things
to
manage
and
staff
for
a
long
time,
and
that
comes
at
a
cost.
We
need
more
people
involved
and
then
testing
becomes
more
complex.
If,
if
you
have
an
LTS,
then
often
you
want
to
be
able
to
upgrade
from
LTS
to
LTS
right
now.
A
We
only
have
one
thing
to
test
the
current
version
to
the
next
version,
or
maybe
three
instances
of
that,
but
with
an
LTS
scheme
you
have
more
of
these
possible
jumps
and
then
it's
not
just
the
number
of
things
that
you
have
to
test.
It's
actually
more
complex,
technically
ensuring
API
stability
like
I'm,
a
software
engineer
and
I'll
just
say
we
are
bad
at
doing
this
and
it
only
gets
worse
when
the
time
gets
longer.
A
If
we
make
the
occasional
mistake,
maybe
we
notice
it
but
and
fix
it,
but
it's
one
thing
to
have
to
think
about
a
mistake
in
a
month
versus
a
mistake
in
36
months.
If
we
make
one
mistake
in
a
month.
Well,
then,
in
36
months,
there's
36
mistakes
and
the
users
have
much
more
problems
and
then
the
upgrade
path.
It's
no
good!
If
you
have
a
stable
support
branch,
so
you
can
be
a
production
and
have
an
easier
life
if
you're
permanently
stuck
there,
you
need
a
way
to
upgrade
to
something
newer
eventually.
A
So
again,
the
technical
complexity
of
upgrade
is
already
large,
but
the
best
case
for
upgrade
is,
if
you
upgrade
almost
every
day,
you're
only
deploying
tiny
bits
of
code,
it's
easier
to
test
and
as
an
engineer
rationalize
and
prove
that
the
upgrade
is
correct.
When
you
go
to
many
years,
you
have
a
much
bigger
technical
problem
to
understand
what
all
might
have
changed
and
validate
that
that
change
was
safe
and
correct.
This
is
very
complex
to
do.
Well,
so,
with
all
of
this
in
mind,
I
was
the
release
lead
on
the
112
release.
A
Team
and
I
was
getting
a
lot
of
questions
and
seeing
a
lot
of
problems
on
our
current
support
process,
and
that
led
me
to
say
well,
let's
have
a
broad
discussion
in
the
community
about
what
our
users
for
support
and
what
our
developers
technically
can
do,
and
maybe
we
see
some
gaps
and
can
do
things
to
improve
those.
So
initially,
we've
just
started
up
this
working
group.
We're
working
on
getting
a
charter
emerged
into
the
community
repo
describing
what
we
intend
to
do,
but
initially
we
know
we
want
to
collect
stakeholder
feedback
and
get
requirements.
A
What
do
people
need
for
support
and
then,
within
that
gage
the
willingness
of
vendors
and
distributors
to
contribute
to
the
upstream?
If
we're
going
to
do
something
for
longer
support,
or
even
if
it's
not
long-term
support,
it's
just
doing
better.
We
need
more
people
contributing
to
test
automation,
to
tools
that
allow
migration
between
different
configure,
a
P,
I
versions.
So
it's
not
just
enough
to
notice
there's
a
problem.
We
have
to
get
a
commitment
from
people
to
contribute
to
fixing
things
and
improving.
A
Now
the
simplistic
approach
would
be
to
say:
well,
LTS
is
the
community
answer,
so
is
a
working
group,
hopefully
will
will
say
what
it
takes
to
make
an
LTS
and
we'll
we'll
pick
one
in
the
future
say:
1.20
is
a
first
LTS
release
and
we're
gonna
have
one
LTS
for
a
year
and
supportive
for
two
years
and
we'll
make
upgrade
possible
all
of
this
stuff,
but
is
a
working
group.
We
don't
want
to
presuppose
that.
A
That's
what
we're
going
to
do,
because
many
of
us
don't
like
that
model
that
the
traditional
LTS
model
of
one
release
every
year
or
two
in
these
big
gaps,
because
technically
it's
very
hard
to
actually
do
well.
So
we
want
to
actually
brainstorm
new
ways
and
sort
of
not
take
this
easy
path
to
going
with
just
the
traditional
way
of
doing
better
support,
but
maybe
see
if
we
can
invent
a
new
model
here
and
especially
because
kubernetes
is
about
cloud
native
and
rapidly
deploying
applications.
A
Maybe
we
can
actually
make
it
so
it's
easier
to
similarly
have
kubernetes
rapidly
deploy.
So,
instead
of
wanting
a
long
term,
support
branch
that
you
stay
on
for
two
years,
we
make
it
to
where
you've
changed
your
mindset
and
is
technically
easy
and
safe
every
month
moved
new
kubernetes
version,
so
we're
trying
to
keep
an
open
mind
and
figure
out
what
we
can
come
up
with
proposals
and
whatever
that
is,
then
it
will
come
back
to
sig
release
to
actually
operationalize,
but
that's
gonna
be
sometime
in
2019
at
best.
A
We're
gonna
spend
many
months
now,
just
understanding
the
state
of
the
community
and
what's
desired,
but
a
key
takeaway
I
think
that
I
want
to
give
to
people.
Is
we
have
many
many
vendors
in
the
ecosystem
if
I'm
a
vendor
and
I'm
making
kubernetes
for
my
customer,
I'm
gonna
have
at
least
one
person
whose
job
it
is
to
do
the
support
to
to
bring
in
patches
and
build
it
and
manage
my
branch
of
kubernetes.
A
If
there's
80
of
us
doing
that
with
one
person
well,
first,
one
person
is
gonna,
do
probably
a
marginal
job
of
it
all
all
good
credit
to
one
of
our
past
patch
release.
Managers
who's
put
a
lot
of
work
into
this.
It's
a
very
hard
job
for
one
person,
but
if
there's
80
of
us,
if
we
could
come
together
as
a
team,
what
we
could
accomplish
is
very
large
and
then
just
to
highlight
the
vendors
here.
A
Some
of
the
vendors
here
in
Asia
there's
at
least
20
and
again
that
by
itself
is
a
huge
team
of
potential
engineers.
If
there's
one
engineer
at
each
company
doing
this,
or
maybe
there's
two
or
three,
it
could
be
40
or
60
engineers,
surely
we
could
invent
something
really
cool
and
good.
That
makes
kubernetes
wonderful
for
users
together.
So
at
this
point,
I
want
to
open
up
to
a
conversation
and
see
what
thoughts
people
have
is
kubernetes,
releasing
too
fast
or
too
slow.
Do
we
stop
support
too
fast
or
too
slow?
A
If
you've
deployed
what
problems
do
you
have
upgrading
from
1.10
to
1.11?
How
many
people
do?
Does
your
company
have
doing
this
today?
Could
we
come
together
and
do
something
more
strongly
and
how
can
together
we
work
to
improve
the
cross
release
and
and
also
cross
vendor
compatibility,
so
I've
kind
of
gone
through
this
quickly,
because
I
wanted
to
leave
about
half
of
our
time
to
talk
so
I'm
hoping
people
have
questions
or
thoughts
and
comments.
This
is
literally
my
last
slide.
B
So
the
use
case
for
ATS
has
not
arrived
like
on
our
side,
mostly
because
the
the
production
environments
are
pretty
new.
So,
like
we
run
like
one
point,
eight
point:
eight,
the
latest
yeah
exactly
the
latest
oldest
clusters.
But,
like
you
know,
LTS
comes
when
you
when
you
start
having
problems
with
your
like
old
clusters,
because
the
the
cost
of
upgrading
the
clusters
it's
pretty
high.
You
need
to
invest
a
lot
of
man-hours
in
order
to
do
like
the
upgrades
and
keep
all
the
clusters
on
the
on
the
latest
version.
B
And
then
you
need
to
be
sure,
like
the
CRTs
that
you
peeled
on
top
of
it
or
even
the
applications
are,
are
up
to
date
and
oh,
okay,
to
use
the
new
versions,
so
I
mean
I.
Think,
like
a
lot
of
I,
see
that
a
lot
of
organizations
just
keep
older
versions
and
think
of
them.
That
way,
we
don't
get
any
patches,
but
we
consider
them
LTS
because
they
didn't
broke
until
now.
Yes,.
A
B
I
think
in
production
as
well,
because
like
I,
was
speaking
with
the
security
Whitley's
yesterday
on
the
security
discussion
and
like
organizations
don't
have
culture
for
security
on
a
cloud
native
application,
mostly
they
think
if
we
can
create
a
border
like
around
kubernetes
and
we
put
some
firewalls,
then
security
will
be
it's
it's
hard
enough.
It's
strong
enough!
So
I'm
not
I,
don't
agree
with
that.
Instead,
no
need
to
waste
any
talk
to
to
work
on
the
educational
part
for
the
security,
but
yeah
I
mean
as
I,
see
like
organizations
use
kubernetes.
C
B
30
yeah
yeah,
so
like
ATS
will
be
will
be,
will
be
highly
welcome
and,
like
the
the
business
benefit,
will
be
like
reducing
the
cost
of
upgrading
and
keeping
it
to
the
because
lights.
It's
a
huge
overhead
now
yeah.
So
if
the
pitch
will
be
okay,
you
need
to
contribute
in
order
to
reduce
the
cost
of
keeping
your
applications
and
your
environments
up-to-date
and
secure
then,
should
be
pretty
obvious
for
them.
I.
B
A
Yeah,
the
security
portion,
I'm
I'm
surprised
to
hear
that
people
wouldn't
be
more
worried
about
that
it.
From
my
perspective,
that's
just
vitally
critical
to
to
ensure
the
security
and
whoever
has
the
ability
to
cube
cuddle.
You
contact
your
your
cluster
or
who
writes
the
container
code.
That's
running
on
your
cluster
I
mean.
B
B
B
C
B
Mingle,
so
like
kubernetes
is
just
one
of
the
ways
you
can
do
that
so,
like
organizations
now
are
trying
to
become
more
agile
and
to
achieve
those
goals
and
like
on
a
medium
to
longer
term,
I
think
security
will
become
will
become,
like
people
will
focus
more
on
security,
but
I
think
like
with
the
organization
that
you
are
working
with,
as
well
like
with
notice.
Even
in
large
organizations
like
digital,
like
digital
transformation
managers
or
like
business
people
that
are
focused
only
on
launching
products
as
fast
as
they
can.
B
They
try
to
overcome
the
security
issues.
They
try
to
try
to
find
workarounds
just
to
just
to
come
up
with
these
deployments,
so
like
security
because
doesn't
understand
that
much
how
they
can
like
do
con
like
contact
security
or
like.
How
do
you
do?
How
do
you
treat
like
a
zero,
zero
trust
network
or
something
like
that?
They
will.
They
will
do
just
like
a
security
at
at
the
border
level
and
do
some
firewalls
and
make
sure
that
the
kubernetes
api
won't
be
accessible
from
the
outside
world.
C
A
B
D
Exactly
right
and
and
fine
and
and
find
the
way,
that's
most
effective
to
get
them
where
we
want
them
to
be,
and
one
of
the
things
I
feel
like
I,
don't
know
enough
about
and
I'd
love
to
see
more
data,
or
maybe
we
could
brainstorm
on
the
way
to
gather
it
would
be
globally.
What
versions
of
kubernetes
are
actually
being
run
by
people
yeah.
A
These
are
vendors
and
distributions
and
certified
hosting
platforms,
but
the
if
you
look
at
the
the
numbers
of
options
versus
what's
at
each
release-
it's
there's
less
here
than
there
and
that
the
numbers
are
sort
of
a
not
quite
we
don't
have
enough
data
to
say
it's
a
Gaussian
distribution,
but
1.12
has
been
out
for
45
days.
Roughly
and
11
of
the
80
have
moved
to
it
and.
A
1.9
has
been
out
for
almost
a
year,
but
again
only
half
are
there.
So
it's
that's
a
little
disturbing
to
me.
It's
hard
to
say
what
people
are
running.
The
other
aspect
of
this
is:
maybe
people
aren't
running
these.
The
early
adopters
maybe
are
building
their
own.
They
could
be
building
something
newer
or
the
there
was
a
talk
earlier.
The
upgrading
your
clusters
talk
by
Google
a
couple
hours
ago.
B
B
A
It
doesn't
give
you
revenue,
but
the
flip
side
is,
if
you
don't
pay
for
that
development,
you
will
incur
future
expense
and
managing
costs
today
versus
future
expense
is
not
what
simplistic
business
managers
do
they
manage
cost
today
versus
future
revenue,
the
opposite
of
expense,
so
finding
how
to
convince
and
in
train
and
I'm
a
software
engineer,
not
a
business
person.
But
part
of
this
is
a
business
equation.
C
B
You
you
said
like
hi
tread
explore
like
high-profile,
exploits
or
something
like
the
Tesla
one
was
kind
of
really
well-known
and
public,
but
the
problem
with
like
enterprises,
they
don't
disclose
those
kind
of
of
breaches.
I
mean
the
Consulate
II
have
problems
with
their
software.
It
like
a
lot
of
things,
but
they
don't
disclose
them
so
by
non
disclosing
them
like
the
community,
and
nobody
knows
exactly
what's
happening
from
a
security
perspective
and
they've
mitigated
through
like
creating
more
boundaries
and
that'll,
slow,
SIG's
and
stuff,
like
that,
I
think.
A
That
there
we
can
make
some
progress
because,
from
the
perspective
of
these
vendors
and
products
platforms
and
distributions
those
people,
those
business
entities
who
are
certified
performant
and
kubernetes,
they
actually
get
early
access
to
confidential
and
embargoed
security,
vulnerabilities
yeah.
So,
on
the
release,
team
I
see
the
current
incoming
vulnerabilities
us
mitigating
them
and
then
announcing
them.
The
normal
public
only
sees
when
they
come
out,
though,
and
as
a
vendor
or
a
distribution
part
of
the
the
business
ecosystem.
If
you
choose
to
engage,
you
will
probably
be
somewhat
scared
by
what
you
see,
because
there.
A
B
So
if
you
need
to
like
human
resources
that
are
like
pretty
high
in
demand
all
around
the
world,
like
people
that
really
know
how
kubernetes
worked
and
how
you
can
do
deploy
on
them
so
like
instead
of
the
cost
of
maintaining
kubernetes,
is
pretty
high
now
so
I
I
think,
if,
like
the
update
tool,
will
become
seamless
and
will
become
just
as
a
tool
and
like
the
vendors
can
provide
a
tool
for
seeing
seamlessly,
upgrading
and
like
allowing
them
to
plan
I,
don't
know
scanning
the
clustering.
Okay,
our
back
now
has
changed
this
application.
B
B
From
a
technical
perspective,
we're
gonna
upgrade
like
pretty
straightforward
and
it
will
be
similar,
then,
should
be
I
think
it
would
be
easier
for
them
to
understand
that,
like
the
cost
will
be
just
a
tool
and
some
development
time,
rather
than
like
keeping
a
team
for
operating
the
kubernetes
cluster.
So
yeah.
A
C
So
yeah
I've
been
operating
gratis
for
a
while
now
and
we've
had
so
I.
We
were
on
one
nine
one,
nine
four
until
last
week,
essentially
on
one
cluster
right,
so
we've
normally
we're
like
I
work
for
a
start-up
and
essentially
it's
a
very
small
team
and
the
only
reason
we've
essentially
upgraded
is
because
we
used
to
be
like
we
got
stock
pods
on
one
nine
on
AWS
and
I.
Think
it's
only
surface.
C
On
one
nine,
so
yeah,
and
because
we
don't
necessarily
have
a
lot
of
old
like
we
use
a
lot
of
older
verses
of
things,
it's
very
sort
of
like
difficult
to
upgrade
and
we
did
try
to
upgrade
to
one
hand
before
but
I
think
was
one
tense
six
or
something
up.
Oh
no
one.
It
was
a
version
that
had
some
permission
she's
on
the
pots
and
it
essentially
just
destroyed
our
elasticsearch
cluster
or
where
that
one,
but
it's
a
we
roll
back.
C
It
was
let's
find
out,
but
sometimes
it's
as
I
said
it's
very
difficult
to.
Unless
you
invest
a
lot
of
time
because
I
like
when
we
think
us
it
took
us
about
two
weeks,
because
we
essentially
spent
up
a
new
class
or
the
same
version
and
put
all
the
software
there
see
what
essentially
broke
and
then
yeah.
No,
no.
So
we
can
gain
confidence
in
the
upgrade
pots.
As
he
said,
it
would
be
nice
to
have
more
sort
of
like
tooling
around.
We.
B
A
A
A
A
Kubernetes
workgroup
LTS
subscribe
to
that
list.
Also,
there's
a
if
you
look
in
the
github
issues.
There
is
an
open
issue
for
workgroup,
LTS
I,
believe
it's
cake,
unity
issue,
number
twenty,
twenty
nine,
eleven,
twenty
nine
eleven
workgroup
LTS.
If
you
search
for
that
issue,
there's
discussion
there
once
we
resolve
the
discussion.
If
you
look
at
the
SIG's
list
in
that
markdown
list
of
all
the
SIG's
and
working
groups,
you'll
see
the
LTS
there.
It'll
have
links
to
the
mailing
list.
We
have
a
slack
channel
working
group
LTS.
A
Conformance
test
is
an
area
where
the
Aaron
quicken
burger
right
now
is
talking,
unfortunately,
we're
conflicted,
but
there's
a
bunch
of
work
in
the
community
to
improve
conformance
testing.
So
all
of
this
I'm
really
hopeful
that
over
the
next
year,
this
will
really
come
together
and
build
a
better
story,
but
we
need
more
involvement
from
people
with
with
ideas
and
with
time
to
contribute
code
as
well.