►
From YouTube: Intro + Deep Dive: Kubernetes Conformance WG - Srinivas Brahmaroutu, IBM & Dan Kohn, CNCF
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Intro + Deep Dive: Kubernetes Conformance WG - Srinivas Brahmaroutu, IBM & Dan Kohn, CNCF
This session will cover the importance of the Kubernetes conformance program. Topics include how to run the conformance test suite, how to certify your company’s platform for both distributions and installers, and how to get maximum benefit out of the program for your company. In addition, the session will also provide an update on the current status of the conformance program, the breadth of coverage of the conformance test suite, and the planned future expansion of the test suite into new focus areas. Attendees will also gain knowledge on the latest tooling improvements, and a greater understanding of the conformance project’s dashboards as well as conformance program documentation methodologies.
https://sched.co/OVxV
A
Cn
CF
is
three
and
a
half
years
old.
In
that
time,
I
think
the
most
impactful
program
that
CN
CF
has
created
for
the
cloud.
Native
community
is
certified
kubernetes,
and
so
this
is
a
way
of
certifying
that
every
implementation
of
kubernetes
out
there
supports
all
of
the
necessary
api's.
And
so,
if
you
have
a
workload
that
runs
on
one
version,
it
really
should
run
correctly
on
another.
Now
in
the
real
world,
our
conformance
tests
do
not
yet
support
100%
or
cover
100%
of
api's
and
even
on
covering
a
hundred
percent.
A
There's
then
a
lot
of
subtleties
about
how
we're
exercising
and
such,
and
so
it
is
a
program
that
continues
to
improve
and
that
we
make
significant
investments
into,
but
I
would
actually
compare
us
against
some
of
the
most
successful
certification
programs
ever
say:
Wi-Fi
or
Bluetooth
or
Android
applications
as
as
being
three
and
the
fact
that
we've
gotten
the
entire
industry.
So
all
of
the
biggest
public
clouds,
the
smallest
public
clouds,
all
of
the
distributions
that
were
not
just
getting
a
part
of
it
has
really
exceeded
our
kind
of
wildest
dreams.
A
A
We
divide
the
certified
Cooper
name,
these
implementations
into
three
groups:
it's
the
distributions
up
top
meaning
that
you
can
install
it
yourself
and
you
can
see
all
different
vendors
that
we
have
here
and
then
we
have
the
hosted.
And
so
these
are
the
clouds
and
you
can
see
by
do
their
and
Huawei
and
Alibaba
and
others,
and
then
the.
A
Finally,
we
have
it
the
installers,
and
we
just
define
this
as
saying
that
they're
not
really
adding
any
other
software,
it's
all
about
just
taking
vanilla,
kubernetes
and
as
a
way
of
building
it
in
one
nice
thing
here,
you
can
see
in
the
latest
kubernetes
just
released
this
week.
Cube
ADM
has
a
logo
now
which
it
didn't
have
before.
Interestingly,
there's
five
different
installers
built
into
kubernetes
that
are
cops,
cube,
spray,
cube,
ADM,
mini
cube
and
kind,
and
it
just
speaks
to
the
fact
that
this
is
a
very
big
project.
A
People
have
different
approaches
to
doing
it,
but
we're
really
pleased
that
all
five
of
those
are
certified
kubernetes
that
it's
it
speaks
to
that
level
of
interoperability
and
and
compatibility.
So
we
continue
to
make
significant
investments
in
the
underlying
conformance
tests.
The
program
itself
for
certified
kubernetes
really
runs
quite
smoothly,
and
we
can
talk
through
that.
A
B
Thanks
that
Dan
has
kindly
found
some
time
to
come
and
give
us
a
brief
update
on
this.
I
will
walk
through
what
kubernetes
components
program
is
from
the
certification
point
of
view,
and
also
this
is
a
inter
on
deep
dive
together,
35
minutes,
it's
going
to
be
a
little
bit
of
what,
from
the
community
perspective,
work
from
the
community
side
of
things
so
on
the
development
of
the
ETA
tests
and
approaches
so
so
communities
certified
communities
is
a
software
conformance
program.
B
B
B
B
Coverage
wise
we
are
trying
to.
We
are
trying
to
make
sure
that
enough
of
coverage
is
done
so
that
it
is,
it
makes
a
real
value
to
the
offering
so
actual
software
conformance
on
the
vendor
version
requires
them
to
exercise
certain
api's.
The
the
API
is
that
are
exercised
by
the
free
version
of
community.
Is
the
community
version
of
the
community's
right.
So
that
way
we
will
ensure
that
there
is
consistency
and
the
portability.
So
if
you
are,
if
you
run
your
workload,
you
can
run
your
workload
anywhere
and
everywhere.
B
B
So,
in
order
to
to
not
have
vendor
lock-in,
it's
it's
important
that
you
run
your
workloads
on
certified
kubernetes
platforms
and
from
the
provider
point
of
view.
They
have
to
have
timely
updates
on
the
conformance
program.
They
have
to
certify
latest
versions
of
kubernetes
and
I'll.
Talk
about
how
that
works.
So,
like
dan
mentioned,
certified
kubernetes
marks
and
kubernetes
mark
are
trademarks
of
linux
foundation.
So
linux
established
this
for
a
reason
number
one:
it's.
B
To
participate
in
the
program
by
sarcastic
on
the
qualifying
offering
and
the
CNC
of,
has
a
participation
forum,
which
is
a
set
of
questions
that
you
general
questions
that
you
answer
and
then,
with
the
self
tested
results
and
this
forum
you
submit
as
a
PR
to
their
github
repo.
That
way,
CNCs
will
be
able
to
verify
your
results
and
validate
those
results
and
certify
you
and
the
program
is
not
free
for
it's.
It's
free
for
the
CL
CF
members,
as
well
as
nonprofit
organizations
they
may
be.
B
There
is
some
he
associated
with
it,
and
all
the
terms
and
conditions
of
the
program
are
listed
on
the
ciencia
website.
I'll
show
that
to
you
in
order
to
be
eligible,
certified
kubernetes,
you
need
to
be
certified
on
one
of
the
latest.
Two
minor
releases,
for
example
113
or
114.
I
know
that
115
is
released
last
week,
say,
for
example,.
B
B
B
So
there
are
two
ways
to
run
the
test:
either
you
you
use
cube
tests
to
run
the
tests
or
use
the
Sunnah
boy,
which
is
a
tool
available
and
I'll
walk
through
Sunnah
Bui
in
detail.
But
if
you
are
using
hip
tests,
luckily
you're
building
the
the
ete
test
on
your
on
your
cluster
of
sorry
and
then
use
you
access.
You
set
the
access
path
to
the
your
cluster
and
run
the
test
with
ginkgo
focus
on
two
components.
So
there
are
lots
of
hit
ideas
like
about
2,000
meters
in
the
crib
in
the
kick
here.
B
Right
and
stairs
are
a
small
subset,
which
is
like
200
tests
right
now,
we're
constantly
adding
new
tests
to
the
conformance
program,
but
it
takes
a
long
time
to
add
tests
to
the
conference
program.
I'll,
explain
also
why
it
takes
a
long
time,
but
generally
it's
a
small
sub
test
which
is
hitting
the
core
API
of
kubernetes.
B
The
other
option
is
to
run
sonobuoy
for
this.
You
set
up
a
cluster
on
any
platform
any
offering
you
have.
You
need
a
cube
control
installed
the
client
and
access
to
the
cluster
by
proper,
cube
config,
and
you
need
to
download
the
sonobuoy
program
written
ago,
and
originally
this
tool
was
available
from
here
now.
It's
is
managed
by
the
community.
So
actually
I
ran
this
a
while
ago,
if
you
here.
B
So
essentially,
IBM
cloud
I
created
a
cluster
of
three
notes
and
I
exported
coupon
fixed,
so
I
can
reach
to
the
cluster
and
I
did
a
go,
get
on
sonobuoy
I.
Think
I
did
somewhere
here,
but
assume
that
I
got
it
and
then
just
as
soon
as
we
run,
and
then
you
monitor
the
tests
run
after
a
while.
You
will
notice
that
son
of
Iran
has
completed,
and
then
you
would
do
a
sorry,
I
think
I'm
on
the
wrong
window
here.
So
it's
run.
B
If
you,
if
you
see
it's
an
OBE,
runs
a
set
of
parts
on
your
cluster
and
running
inside
these
claw
spots,
the
ete
test,
and
eventually,
when
it
finishes
by
checking
the
sonobuoy
status,
it's
complete,
you
can
go
ahead
and
do
it
sooner.
We
retrieve
what
it
does.
Is
it
captures
all
the
test
output
from
from
your
cluster
and
dumps
it
into
a
tar
file
using
the
tar
file
you
can.
These
are
all
the
contents
of
the
tar
file,
which
probably
have
not
much
interest
for
us.
B
C
A
B
B
B
Yeah
well
from
the
CNCs
side,
they're
managing
the
conformance
program
from
the
certification
point
of
view
and
the
top
on
the
vendors
and
the
client
side
right
on
the
community
side.
What
we
are
trying
to
do
is:
how
can
we
build
the
conformance
program
itself?
How
do
we
make
sure
that
all
the
vendors
have
the
same
quality
of
service?
B
Essentially,
the
the
role
is
cigar
picture
is,
is
the
umbrella
for
governing
this.
From
the
community
point
of
view,
they
decide
what
all
the
required
tests
that
should
be
run
as
part
of
the
conformance,
and
we
have
a
confirm
n
sub
project
where
we
have
development
activity
that
will
go
through
and.
B
B
Sig
related
work,
so
six
will
write
the
idiot
s
and
the
et
test
will
become
part
of
the
ete
suite
and
from
the
conformance
point
of
view,
we
will
then
go
ahead
and
promote
and
existing
italy
test
into
conformance
test
suite.
So
this
is
always
happening
so
from
every
release
to
the
next
release.
There
will
be
more
conformance
test
that
gets
added
to
the
test
suite,
so
it's
not
as
a
static
as
we
think
so.
B
Once
you
pass
the
control
meds,
it
doesn't
mean
that
you
will
pass
a
conformance
for
the
next
release
and
so
on
so
forth,
because
there
are
more
tests
and
more
tests
right
now.
I
think
we
have
about
two
hundred
and
over
two
hundred
and
ten
right
now
the
one
I
showed
you
was
for
a
version
113
where
we
had
about
one
hundred
and
ninety
tests.
So
there
are
like
2025
tests,
I
added
it
between
139
115.
B
So
there
are
several
approaches
we
take
from
the
community
point
of
view.
First,
we
decided
that
we
will
concentrate
on
parts
back
because
that's
a
general
thing
that
should
work
consistently
for
all
the
workloads
we
have
exercised.
Most
part
of
the
parts
back,
but
there
are
still
there
are
some
areas
of
the
pods
back
that
are
not
exercised
yet
so
we
are
working
on
it.
B
Like
I,
said
the
promotion
process
itself
takes
several
cycles,
so
we
decide
that
this
particular
feature
is
not
tested
like
bought.
The
pod
communication
within
a
cluster
is
not
tested.
We
write
a
UTA
test
in
this
quarter
in
this
release.
We
we
add
that
to
the
EDI
test
suite
we
make
sure
that
that
test
is
not
flaky
and
not
slow
and
so
on
and
so
forth.
I've.
B
The
test
has
been
consistent
on
our
CI
in
the
next
release.
That
test
will
be
slated
to
do
promotion
to
conformance
tests.
So
what
is
promoted
the
conformance
test
or
guaranteed
to
work?
Well
that
it's
a
couple
of
ideas
for
us.
So
at
this
point
we're
not
hundred
percent
covered
right.
I
mean
it's,
it's
it's
impossible,
so
we
have
only
two
hundred
tests
and
we
want
to
improve
the
coverage
so
of
the
core
features
of
kubernetes.
So
there
are
several
approaches
we
are
taking.
A
couple
of
approaches.
I
mentioned
here,
is
behavioral
driven
conformance
testing.
B
What
in
this
case,
we
are
trying
to
do
is
we're
trying
to
look
at
the
API
and
the
fields
and
and
see
how
they
can
be
used
in
user
scenarios
and
define
the
behaviors,
and
then
we
go
ahead
and
write
new
ETA
ideas.
The
other
approach
that
there
is
a
tool
called
API
snoop,
which
basically
analyzes
the
audit
logs
every
time
you
run
some
tests
or
workload
whatever
you
run
on
the
cluster
and
sees
what
all
the
API
endpoints
you
are
hitting.
B
So
if
you
are
not
hitting
some
API
endpoints
during
the
conformance
test
tests
that
are
run
on
the
cluster,
we
know
these
api's,
which
are
the
core
API
is
that
needs
to
be
hit.
So
that
means
we
need
to
write
more
tests
in
those
areas,
so
there
these
are
the
two
approaches
we
are
following
and
then
there
is
also
a
validation
to
its
proposal.
Right
now,
that's
in
the
works.
B
This
is
more
not
towards
the
conformance
program
itself,
but
it
is
for
things
that
are
outside
of
the
conformance
program
like,
for
example,
you
have
functionality
like
CSI
drivers
and
storage.
That's
not
part
of
the
core
features.
We
need
some
kind
of
a
suite
that
can
only
test
that
particular
module
or
networking
C
and
I
would
not
right.
So
that's
the
idea
in
there,
so
these
are
again
a
very
public
information.
So
if
you.
B
B
What
all
the
API
is
that
are
that
are
running
right
now
on
the
CI
and
see
the
coverage
there
so
about
the
conformance
tests
eating
a
test
itself
like
I
mentioned
briefly
here
and
there,
but
the
merging
process
of
an
EEG
it
is
is
a
cigarette
they
like,
for
example,
if
it's
a
networking
test,
Signet
working.
If
it's
a
no
test
is
signaled
and
eventually
they
we
analyze
that
and
say
how
this
can
be
promoted
to
conformance
it
takes
two
releases.
B
The
the
main
idea
are
the
criteria
that
the
conformance
test
has
to
meet
is
it
should
be
a
ga
feature.
It
cannot
be
an
alpha
or
beta
feature,
and
it
should
work
on
all
providers
and
on
all
architectures.
It
cannot
rely
on
network
or
binaries
special
binaries.
So
what
we
are
expecting
is
any
kubernetes
vendor
should
not
have
to
do
additional
steps
to
run
this
and
it
has
to
be
stable
and
consistent.
That's
why
we
have
to
to
release
cycles
associated
with
it.
B
So
there
are
two
PRS
one
is
to
add
the
ETA,
and
the
second
PR
would
be
to
add
that
test
to
the
components
test,
wait.
So
a
promotion
PR
has
to
be
done
to
get
that
test
into
into
the
suite.
There
are
other,
oh
I'm,
sorry
did
I
Wow.
So
that's
what
I
talked
about
here.
So
essentially
that's
the
criteria
for
the
conformance
test
weights.
B
So
eventually,
if
you
are
promoting
an
eatery,
you
should
submit
a
PR
to
the
cigar
collector.
There
are
areas
that
we
want
to
cover
in
the
current
in
the
conformance
test
weight.
Those
are
the
areas
that
needs
to
be
covered,
especially
in
the
old
part.
We
are
covering
part
right
now.
Volumes
there
are
tests
already
existing,
but
there
are
more
tests
needs
to
be
written.
B
They
need
not
have
to
know
the
technical
details
of
the
test.
They
don't
have
to
go
through
the
code,
but
they
will
know
exactly
what
the
test
is
doing.
It
helps
us
to
actually
analyze
ways
of
the
metadata
that
we
are
adding
at
the
top
of
the
test.
What
the
coverage
is
and
also
debug
if
something
goes
wrong,
fails
in
a
vendor
environment
right.
B
B
B
About
the
from
the
community
point
of
view,
the
kubernetes
conformance
program,
we
have
a
kubernetes
conformance
work
group,
it's
not
a
sake,
it's
a
work
group
and
there's
a
mailing
list.
If
you
have
any
questions
about
the
conformance
program,
you
can
use
that
actively.
All
our
development
is
is
its
public.
So
if
you
have
any
any
interest
in
joining
the
development
part
of
the
communities,
there
is
a
office
hours
that
runs
bi-weekly
on
Tuesday
noon.
Pst,
Pacific,
Standard,
Time,
you're.
B
B
B
B
If
you
see
here
all
the
issues
that
come
through
triage,
where
we
have
amber
Elijah's
on
tests,
we
want
to
write
and
then,
when
the
tests
are
written,
they
will
go
through
the
standard
progress
from
the
tests
it
in
the
backlog
and
if
somebody
reviews
the
tests,
it
goes
into
the
progress
once
the
test
is
reviewed
by
the
sig
as
well
as
conformance
tech
leads.
Eventually,
the
conformance
program.
Tech
leads,
will
review
the
tests
and
that
will
become
part
of
the
control
ments
test
suite.
You
can
follow
this.
B
This
is
also
publicly
available
project
board
and
you
can
see
what
tests
are
going
to
come
in
your
next
release.
If
you
are
interested
in
running
your
tests
on
116,
probably
this
board
will
give
you
some
indication
of
what's
coming
in
for
the
future.
So
with
that
also
I
want
to
thank
the
people
who
have
contributed
to
conformance
some
of
the
slides
I
have
used
from
Arin
freaking
Berger
from
last
cube
con.