►
From YouTube: K8s Conformance & SIG Architecture Conformance Subgroup - Hippie Hacker, Dan Kohn & John Belamaric
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
K8s Conformance & SIG Architecture Conformance Subgroup - Hippie Hacker, ii.coop; Dan Kohn, CNCF; & John Belamaric, Google
https://sched.co/VyC7
A
Hi
there
we'll
go
ahead
and
get
started.
My
name
is
Dan
Khan
and
I'm
the
executive
director
of
the
cloud
native
computing
foundation.
This
is
definitely
not
well
I.
Guess
we
officially
pitched
it
as
the
intro
deep
dive
of
conformance
but
I
think
we're
gonna
focus
this
pretty
intensively
on
a
deep
dive.
I
did
want
to
just
give
the
context
that
we
started
this
program
two
years
ago.
Arguably
of
everything
that
CN
CF
has
done,
including
cube
con.
This
has
been.
A
The
single
program
has
had
the
biggest
impact,
so
we
took
the
conformance
test
suite
that
had
already
been
built
into
kubernetes
and
then
created
a
program
around
it
to
allow
organizations
to
self-certify
when
we
launched
two
years
ago,
I
think
it
was
with
22
or
24
vendors,
which
we
were
utterly
thrilled
with
we're
at
102.
Now,
though,
which
is
a
really
spectacular
and
given
the
large
number
of
acquisitions
going
on
in
the
space,
this
number
probably
will
go
down
a
little
bit.
A
But
then
there's
also
a
bunch
of
new
folks
coming
in
that
that
may
make
it
stay
the
same
or
go
up.
So
I
think
we're
very
much
at
the
phase
of
a
conformance
program
where
we
are
at
the
end
of
the
beginning
and
there's
the
the
program
itself
in
terms
of
organizations
getting
involved
and
signing
the
the
paperwork
around
it
and
then
having
a
way
of
submitting
their
tests
and
such
is
all
working
and
and
the
program
was
designed
in
a
very
explicit
fashion,
to
require
people
to
stay
on
the
release
train.
A
B
Actually,
we'll
start
with
me:
I'm
John,
Bell
Marik
from
Google
and
I'll
talk
a
little
bit
just
kind
of
about
the
introductory
material
I'm
just
going
to
stay
down
here.
This
is
a
really
small
group
which,
by
the
way,
you
can
come
closer,
you
don't
have
to
be
so
far
away,
rather
than
standing
up
there,
where
I
can't
see
the
slides.
B
So
just
as
a
sort
of
sort
of
introduction,
as
Dan
was
saying,
this
program
is
intended
to
ensure
that
we
protect
the
brand
of
kubernetes,
so
any
any
vendor
that
is
claiming
to
be
kubernetes
needs
to
meet
a
certain
level
of
certification,
and
we
want
to
make
sure
it's
it's
so
important
for
the
confidence
and
trust
of
our
our
users
that
kubernetes
running
you
know
from
Google
and
from
Amazon
and
in
your
in
your
VMware
machines.
All
work
the
same
way
essentially
right
that
we
can
move
workloads
portably
across
things.
C
B
B
B
Again,
I
think
I've
already
sort
of
said
this,
but
for
end-users
it
means
that
they
can
have
a
lot
more
confidence
that
any
workload
they're
running
in
any
particular
environment
can
be
made
to
work
in
another
environment.
Now
every
vendor
of
course
wants
to
differentiate,
and
so
we
try
to
leave
enough
room
that
the
functionally
things
work.
But,
of
course
you
know
one
vendor
may
have
better
networking
or
something
than
others.
There
are
pluggable
interfaces
to
kubernetes
and
we
need
for
conformance
point
of
view.
B
We
need
for
the
the
distributions
and
those
pluggable
interfaces
to
work
in
the
same
manner
and
functionally
the
same
way,
but
there
can
be
there
can
be
some
differences
between
them
for
differentiation,
if
you
are
a
distributor
or
a
vendor,
you
definitely
want
to
certify.
You
want
to
get
this
little
badge
and
you
probably
want
to
use
kubernetes
in
the
name
of
your
product,
which
you
can't
do
otherwise.
B
We
split
the
conformance
program
into
two
aspects
and
Dan
alluded
to
this
a
little
bit,
but
the
CNC
F
actually
runs
the
program
of
certification,
so
the
CNC
F
is
what
you
submit
your
results
to,
and
they
have
the
authority
to
grant
you
that
badge.
What
we
do
have,
though,
is
in
the
open-source
community
in
cig
architecture.
B
There's
a
conformance
sub
project
that
conformance
sub
project
does
two
things:
one
defines
the
actual
behaviors
or
tests
or
functionality
that
is
considered
required
for
conformance
so
actually
defines
what
kubernetes
is
and
also
implements
an
open-source
set
of
tests
to
our
coordinates
that
really
other
SIG's
typically
implement
the
test,
but
coordinates
that
that
plant
that
that
platform
to
actually
do
the
evaluation.
So
the
mechanics
of
what
the
tests
are,
what
the
behaviors
are,
is
determined
by
Sagarika
texture.
B
The
actual
execution
of
those
tests
and
evaluation
of
the
results
for
a
given
vendor
is
done
by
CN
CF.
So
that's
all
on
the
sort
of
intro
level
very
high
level,
I'm
happy
to.
Maybe
what
we
should
do
is
have
hippy
go
through
some
more
information,
I'm,
a
sort
of
deep
dive
level
of
what
the
current
state
is
as
far
as
coverage
and
things
like
that.
A
lot
of
the
effort
right
now
in
the
conformance
program,
as
Dan
alluded
to
is
increasing
the
coverage.
We
have
a
limited
amount
of
coverage.
B
C
You
John
I,
appreciate
your
introduction
and
thank
you
Dan
for
setting
up
the
program
for
us
all.
I
know
that
it's
critically
important
for
me
in
my
country
that
the
software
that
we
create
together
runs
everywhere
on
the
planet
without
that
assurance.
I,
don't
have
time
for
this.
You
have
to
go.
Do
something
else,
but
I
think
this
is
one
of
the
platform
that
we
can
standardize
on
and
ensure
that
our
workloads
can
run
globally.
C
C
I'm
gonna
close
these
slides
and
I'm
gonna
close
this
slide
and
there's
all
of
our
certified
companies.
Vendors,
I'm,
gonna,
close
that
slide,
and
this
slide
and
this
slide
and
we're
going
to
talk
about
clouds
and
the
resolution
at
which
we've
been
able
to
to
see
the
clouds
that
f11
that's
screenshot.
Where's
there
we
go.
C
C
This
is
the
it's
called
small
blue
dot.
It
was
taken
from
Voyager,
you
can't
see
it,
but
in
this
little
red
band
there
is
a
quarter
of
a
pixel
resolution
of
what
it
looks
like
from
30
degrees
above
the
ecliptic
to
see
the
whole
earth
that
was
from
Voyager
1.
This
is
probably
the
most
famous
selfie
in
the
world.
This
is
called
blue
marble
and
it
was
taken
from
the
crew
Apollo
17.
C
The
resolution
is
much
higher.
We
can
see
some
land
masses
and
the
color
of
the
ocean
go
forward
a
2001.
Now
we've
got
lots
of
sensors
and
metrics
coming
in
from
all
of
satellites
everywhere,
and
so
we
actually
have
our
full
view
of
Blue
Marble,
two
from
NASA,
and
they
combined
all
those
images.
This
is
a
problem
in
a
lot
of
presentations
around
the
world.
Is
they
forget
to
include
this
small
country
down
at
the
bottom
New
Zealand,
because
of
the
aspect
ratio
of
the
way
that
we
we
focus?
C
C
The
reason
I'm
working
with
the
CN
CF
and
enjoy
their
vision
is
they
are
about
building
sustainable
ecosystems,
and
these
ecosystems
need
to
be
able
to
encourage
that
all
people
to
participate
and
to
be
able
to
benefit
from
this
collaboration.
This
global
collaboration
that
we're
doing
the
CNC
F
also
fosters
communities
like
the
kubernetes
community,
I
think
we're
doing
some
things
for
the
first
time
in
humanity,
where
we're
able
to
coordinate
and
do
things
in
an
open
governance
way
that
is
global.
C
C
C
C
This
is
probably
good
that
we
know
we
have
full
coverage
and
at
about
10
or
11,
then,
when
we
added
user
agent
support
both
to
the
API
server
and
to
our
testing
framework,
we
were
able
to
tell
when
a
test
was
hitting
kubernetes
progressing
to
113
we're
starting
to
get
a
lot
of
satellites,
or,
in
this
case,
using
test
grid
and
all
of
the
CI
automation.
To
look
at
lots
of
audit
logs.
We
set
up
some
API
snoop.
That's
the
NCS
CI
to
start
processing
those
audit
logs
and
give
us
a
view
across
time.
C
C
What
was
added
to
kubernetes
to
ensure
that
we
didn't
add
anything
new
or
create
more
surface
area,
create
more
land,
that
people
couldn't
safely
build
infrastructure
or
be
productive
on
in
the
cloud
there
was
actually
posting
of
APA
links
to
API
snoop
in
issues
and
tickets.
Saying
this
is
you've
got
three
more
to
go:
you're
almost
there.
These
are
the
exact
three
surface
area.
End
points
that
we
need
test
coverage
for
and
I'm
happy
to
say.
C
C
116
we're
still
trying
to
define
what
surface
area
means.
We
came
up
with
this
idea
that
when
you
query
that
kubernetes
it'll
tell
you
here
are
all
the
operations
you
can
do,
and
here
are
all
the
kinds
of
things
that
you
can
do
the
objects
you
can
do
operations
on
and
it
gives
us
a
almost
a
thermometer
type
check.
But
we
don't
know
the
full
health
or
behavior
yet
and
Jon's
got
some
interesting
things
that
he's
been
working
on
with
his
team
and
combining
together
with
the
database
in
query
ins.
C
We're
gonna
actually
go
through
and
talk
about
that
and
I'd
love
to
get
your
input
on
precisely
defining
what
it
means
to
measure
the
surface
area
of
the
kubernetes
itself
and
I
I.
Think
that's
pretty
much
for
the
fancy
slighty
things.
These
are
the
links
for
Kate's
conformance
and
API
snoop.
The
first
one
is
the
CN
CF
repo,
where
everyone
who
wants
to
become
certified
can
submit
their
test
results.
C
There
are
two
things
you
need.
You
need
to
submit
your
test
results
for
the
version
of
kubernetes.
You
want
to
be
certified
for
and
you
need
to
be
a
CN
CF
member.
If
you
want
to
understand,
what's
the
current
area
is
and
we'll
go
to
that
link
in
a
moment
is
API
snooped
at
CN
CF,
dot
IO.
It
helps
us
understand
the
surface
area
what's
covered
and
what
can
come
next
and
we
also
have
a
repo
that
accompanies
that
I'm.
Just
a
quick
look
at
API
snoop
itself,
oops.
C
The
surface
areas
you
see
in
this
picture
here
the
red
things
are
lava:
it's
where
new
things
are
being
formed.
New
land
is
coming
together
and
blue
is
beta.
It's
where
it's
starting
to
cool
down,
hadn't
solidified!
Yet,
but
it's
on
by
default
in
your
cluster,
so
you
can
give
it
a
try,
but
don't
depend
on
it.
Yet
it
might
change
slightly.
C
The
green
area
is
where
it's
super
safe
and
generally
available
for
everybody
to
build
their
everything,
but
we
have
to
agree
on
what
that
is
and
make
sure
that
it's
a
really
solid
foundation
for
us
all.
If
I
zoom
in
to
what
could
possibly
be
the
stable
foundation
for
us
all,
there
well
notice
it's
actually
a
very
specific
number
of
endpoints
or
operations,
and
we
can
tell
how
many
are
hit
by
tests
in
general
and
how
many
are
how
much
how
what
percent
is
hit
by
conformance
test,
it's
useful.
C
We
can
see
the
groups,
here's
one
of
our
groups,
that's
completely
covered
under
admission
registration
if
I
click
on
that
you'll
see
that
all
of
the
little
outer
wheel
is
filled
in.
If
we
go
back
to
the
green
and
choose
one,
that's
not
yet.
Let's
choose
storage,
we'll
choose
auto
scaling.
Auto
scaling
has
got
three
endpoints,
but
there's
still
ten
in
points
that
need
to
be
covered.
C
C
Would
you
prefer
bright,
color
or
a
dark?
Color
writes
good.
You
can
see.
Okay,
so
I
went
ahead.
We
have
a
template
that
we
go
through
with
our
team
to
help
them
help
identify
what
it
is
that
we're
gonna
focus
on
helping
the
community
next.
As
far
as
writing
tests,
this
template
we
go
through
and
helps
us
to
ensure
that
we're
all
doing
it
the
same
way
and
I'm
gonna
encourage
you
to
join
me
in
going
through
this
I
won't.
C
We
can
actually
share
these
terminals
with
each
other.
We
call
it
pairing
so
that
we
can
transfer
knowledge
from
people
in
one
sig
or
within
our
own
team
to
get
all
on
the
same
page.
I
have
another
window
here
on
the
on
the
right
and
I
will
as
we're
we're
going
through
this.
It
should
write
things
over
there.
C
We
go
through,
and
this
got
this
checklist,
but
I've
already
done
90%
of
it
for
you
to
go
through
and
set
up
our
cluster
and
create
a
kubernetes
cluster
using
kind
one
of
the
things
that
we've
found.
That
helps
us
is
the
audit
logs
and
you
can
actually
ask
your
API
server.
It's
still
alpha
I'm,
looking
forward
to
when
it's
beta.
So
it's
available
to
everybody.
So
we
asked
our
cluster
to
please
send
your
logs
to
a
specific
service
and
we
bring
up
this
database
of
API
snoop
inside
the
cluster.
C
So
this
this
process
that
we
go
through
helps
us
look
at
the
existing
surface
area
and
I'll
show
you
how
we
define
that
and
then
you
can
run
your
applications
that
are
important
to
you
and
see
the
areas
that
they
hit.
That
are
not
yet
conformance.
So
you
can
advocate
that
they
get
tests
written
either
by
spending
your
team's
organizational
time
to
write
those
tests
or
let
us
know
that
they're
important
to
you,
so
we
can
prioritize
them.
C
C
We
have
a
walkthrough
that
this
basically
is
through
and
does
a
coop
cuddle
apply
of
API
snoop.
So
that
says:
that's
how
simple
it
is
to
get
API
snoop
installed.
Make
sure
that
you
have
audit
logging
enabled
on
your
cluster
and
then
you
to
cuddle
apply.
Then
it
allows
you
to
have
a
the
ability
to
query
and
we're
gonna
go
through
and
do
those
queries
here
in
a
minute.
So
you
can
see
the
definition.
C
Let's
make
this
super
simple,
API
snoops
been
applied,
we
verified
that
all
of
our
pods
are
running
and
we
have
port
forwarded
things
so
that
we
can
get
access
to
the
there's
a
PostgreSQL
instance
so
that
the
simplest
way
to
help
define
coverage
is
just
to
query
the
API
itself
on
what
you're
interested
in
covering
and
looking
at
all
of
the
api's
that
have
hit
it
so
far
and
we
connect
this
file.
I
work
on
to
the
DB,
so
this
and
get
rid
of
here,
so
we
can
focus
all
in
one
place.
D
B
C
So
this
query
right
here
we
have
a
table
for
stable,
endpoint
stats
where
it's
not
the
live
data
from
your
cluster.
This
is
the
latest
CI
build
from
it
used
to
be
close,
blocking,
merge
blocking
for
master,
but
I
think
it's
merge
informing.
Now
we
need
to
what's
one
decision.
We
need
to
find
out
what
is
the
best
source
material
for
comparing
the
baseline.
The
reason
we
need
a
baseline
is:
we
need
to
make
sure
that
baseline
never
goes
down,
but
we're
always
increasing
the
percentage
of
surface
area.
C
C
C
We
go
through
a
bit
further
at
this
point.
We
need
to
let
the
DB
sit
for
a
bit.
This
is
where
we
go
through
and
identify
a
particular
feature
we
want
to
focus
on.
This
is
just
a
sequel
statement
that
says:
go
find
the
stable
things
in
core
that
have
no
test
at
all.
Oh
and
don't
look
at
the
live
feed
just
yet.
C
I'll
go
ahead
and
close
this
area
out,
that's
identifying
the
feature,
then.
Normally
we
have
another
section.
That
is
for
our
template
very
light,
because
once
you've
identified
the
end
point
to
focus
on,
we
want
to
go,
find
all
the
documentation
become
versed
in
the
vocabulary.
So
we
can
identify
the
right
people
to
talk
to
the
experts
and
create
an
issue
that
invites
them
to
look
at
the
code
that
we
will
write
shortly
and
that's
what
I'll
do
in
our
next
little
snippet
here
is
write
your
test.
C
This
is
where
you
don't
have
to
learn
how
to
write
everything
in
the
IDI
framework.
Yet
you
don't
know
how
to
how
to
even
write
full-on
go.
If
you'll
put
some
example
code
inside
this
block
and
execute
it,
it
will
run
against
your
cluster
and
will
allow
us
to
without
needing
to
be
experts
at
least.
Ask
really
intelligent
pointed
questions
with
examples
and
output
and
the
endpoints
that
it
hits
so
that
we
can
onboard
more
people
getting
us
that
we
have
10x
ago.
C
Let's
get
there
as
quickly
as
we
can
with
as
many
people
as
we
can
so
I'll
go
down
here.
I
think
this
is
a
pretty
cool
thing.
I
haven't
seen
this
in
any
other
type
of
way
of
coding,
but
we
have
these
things
that
we
do
with
code
blocks
and
inside
the
code
block.
It's
just
a
really
simple
snippet
of
code.
It's
only
what
is
that?
Not
even
15
20
lines,
long
and
it's
not
all
of
the
ghost
stuff.
It's
just.
C
C
C
Takes
a
little
bit
of
time
to
do
the
query,
but
not
too
bad.
So
this
is
our
the
upstream
job
and
the
date
that
that
job
was
run
on
it's
a
little
bit
behind
cuz
a
coop
can
prep
the
number
of
endpoints
at
that
time.
We're
talking
stable
operations,
how
many
times
it
was
hit
by
tests
and
how
many
times
was
hit
by
conformance
tests.
So
we
have
the
number
of
endpoints
hits.
We
have
this
percentage
%
tests
that
are
preference
conformance,
tested.
I
can
go
through
these
queries
with
you
and
another
time.
C
In
this
question,
we're
asking
what
was
hit
by
the
new
code
snippet
that
we
wrote,
so
we
don't
have
to
be
experts
in
it.
We
can
understand
that
hit
by
new
test
list,
core
v1
pod
for
all
namespaces
was
actually
hit
three
times
because
I've
run
this
three
times
now,
but
if
you
were
to
actually,
you
could
actually
see
exactly
how
many
things
we
could
reset
an
experiment
with
how
do
we
increase
coverage
in
a
way
that
the
experts
agree
on
that?
C
We
are
very
precise
in
our
description
before
we
go
write
any
tests
we've
been
working
hard
with
the
community
to
try
to
write
meaningful
tests
and
I.
Think
we've
discovered
that
we
need
to
wait
to
write
any
test
until
there's
some
type
of
lazy
consensus
on
the
right
idea
before
we
spend
that
time
and
I
think
this
will
help
us
help
us
get
there.
C
The
last
area,
as
far
as
our
work
flow
is
actually
to
say:
how
much
will
this
particular
issue
if
we
were
to
create
a
test
from
it?
How
much
would
it
increase
our
coverage
by
this
is
the
core
of
not
only.
How
important
is
this
test
to
be
written?
It
is
also
the
core
of
the
bot
that
says
you
shall
not
pass
unless
you
make
sure
that
our
conformance
and
coverage
for
the
basis
of
how
all
of
us
deploy
our
workloads
worldwide
is
safer
than
it
was
before
you
came.
B
One
thing
I'd
like
to
mention
I,
think
that's
useful,
also
because
we
want
want
to
avoid
tests
that
are
sort
of
accidentally
hitting
end
points
in
a
Rube
Goldberg
ii
kind
of
way.
So
if
you
look
at
your
list
of
things
that
are
hit
by
the
new
test,
you
could
you
know
if
that
list
is
really
long,
you're,
not
necessarily
gonna
count.
All
of
those
right,
so
I
mean
it
helps.
You
keep
a
lot
of
clarity
around
making
the
test
precise.
C
One
of
the
things,
if
there's
any
questions,
one
of
things
we
like
about
our
workflow
here,
is
that
I
can
actually
at
this
point,
take
this
entire
document,
including
the
code
snippets
and
stand
it
out
and
the
queries
and
the
results
and
hit
some
keys
and
post
it
into
an
issue
without
having
to
format
a
document
or
go
through
a
letter
process.
So
we're
hoping
that
the
more
people
that
learn
to
pair
and
and
cross
Sig
cross
pollinate.
Look
we
pair
together
in
this
way
and
can
create
enough
tickets.
C
D
C
Important,
it
is
not
part
of
the
conformance
working
groups-
role,
okay,
but
there
are,
and
I
can
show
you
exactly
where
the
tests
are
in
test
grid,
where
we
do
upgrade
tests
across
the
last
three
to
four
releases,
all
the
way,
all
the
way
up.
It's
not
I
per
vendor
thing,
so
it's
not
a
way
that
that's
kind
of
up
to
the
vendor
to
ensure
they
upgrade
well,
but
as
far
as
ensuring
that
the
base
components
of
kubernetes
can
withstand
and
survive.
The
upgrades
there's
a
whole
other
area
of
testing.