►
From YouTube: Kubernetes SIG Testing 2018-03-20
Description
A
Okay,
hi
everybody
today
is
March
20th
2018.
I
am
aaron
aaron
berger,
your
moderator
for
today's
sake,
testing
meeting
today
on
the
agenda.
We've
got
a
presentation
from
javelina
to
talk
about
Windows
testing
and
we've
got
Tim
pepper
from
contributor
experience.
Contributor
experience
sake
to
give
us
a
little
bit
of
a
road
show
that
they're
doing
with
a
state-by-state
basis.
And
then,
if
you
have
time.
A
C
D
D
A
D
So
all
right,
so
the
current
state.
So
why
are
we
doing
this?
To
begin
with?
The
main
idea
is
that
in
the
moment
in
which
we
had
windows
support
to
even
project,
especially
something
as
complicated
or
latest,
it
becomes
paramount
to
a
testing
to
avoid
that
people
get
regression
and
other
issues
that
typically
come
from
the
fact
that
most
of
the
ecosystem
is
not
aware
necessarily
about
the
difference
between
Linux
and
windows.
So
you
end
up
very
easily
with
a
Linux
Linux
is,
let's
say
in
the
code
base
that
will
of
course
break
there.
D
The
windows
side
of
things
so,
of
course,
the
obvious
way
to
mitigate
and
resolve
this
is
to
add
in
place
continuous
integration
testing,
which
is
actually
what
we
are
currently
doing.
So,
of
course,
this
is
a
project
that
was
already
very
well
going
before,
starting
to
added
the
reporting,
it
means
the
total
bulk
of
the
testing
infrastructure
and
the
tests
themselves
already
have
dependencies
on
Linux.
So
what
we're
doing
here
is
to
take
all
this
work
and
make
sure
that
it
works
in
the
best
possible
way
with
us.
D
So
the
goal
of
the
course
is
to
validate
bug,
fixes,
validate
new
features
and
Koechlin
gestures.
Okay,
so
the
current
status
we
we
are
currently
around
67
149
performance
tests
passing
and
Counting
okay.
So
we
keep
on
adding
more
objectives.
So
the
first
thing
is
that
we
are
adding
ACS
engine
support
in
cube,
destined
to
execute
the
push
tests
using
Windows
nodes
working
with
a
community
aka
pass
here
to
review
and
modify
existing
tests
to
pass
on
Windows
and
Linux,
and
eventually
the
goal
would
be
to
add
blocking
tests
for
Windows
intro
okay.
D
Last
but
not
least,
we
also
want
to
run
cube
tests
on
on
Prem
clusters
having
Windows
nodes
and
that
will
allows
us
to
to
test
using
a
pretty
large
set
of
possible
network
plugins.
So
starting
with
Ruby
and
blended,
more
action
plan
so
well
enable
cube
test,
deploy
Windows
Linux
VMs
on
a
teacher,
so
the
first
thing
use
a
chart
for
the
Apple.
As
we
already.
D
D
Okay,
so
we
will
need
to
replace
container
images,
so
we
need,
of
course,
windows
images
because
of
course,
at
the
moment,
at
least
in
a
moment
which
we
run
containers.
Of
course
we
deploy
containers
in
Windows.
Those
have
to
be
windows,
containers,
okay,
so
we
need,
of
course,
windows
images.
We
need
a
related
to
dark
red
files.
D
We
need
to
remove,
had
hard-coded
name,
so
here's
one
of
the
PR
that
the
Lena
didn't
let
her
go
into
did
just
afterwards,
and
not
only
that
most
of
the
containers
are
then
used
to
perform
actions
that
are
Li
not
specific.
Some
of
them
are
relatively
easy
to
port,
for
example,
everything
that
requires
busy
box.
There
is
a
busy
box
windows
windows
parking
that
we
using
other
are
way
more
complicated
or
cannot
be
ported
at
all.
D
For
example,
if
a
test
requires
iptables
well,
of
course,
there
is
no
way
that
we
can
do
anything
like
that
on
on
linux,
which
brings
me
to
this
slide.
We
basically
started
grouping
all
the
tests
in
three
categories.
One
is
the
best
possible
use
case,
meaning
that
the
tests
are
simply
running,
as
is
so
without
changes
or
with
very
minor
modifications.
D
The
second
category.
It
means
that
those
tests
can
be
ported,
meaning
that
without
too
much
hassle
that
we
can
take
them
and
convert
them
in
a
way
which
they
can
just
run
Windows.
The
busybox
is
a
perfect
example
here
or
cannot
be
ported
like
requiring
every
tables
as
I
was
mentioning
before.
Last
but
not
least,
we
also
would
like
to
add
additional
tests
which
tests
the
specific
Windows
things
that
are
not
available
in
Linux,
which
are
windows
account.
They
see
else
and
whatever
else.
What
is
completely
out
of
scope?
D
A
D
A
C
Well,
the
first
visible
specific
issue
is
that
there's
no
busybox
on
and
Windows
I
mean
the
problem.
Is
that
invisible
the
the
way
the
best
box
image
is
used
for
more
I,
saw
or
is
doing
stuff
like
this
URL
or
even
an
LS
or
stuff
like
that?
That's
not
available
on
Windows
anywhere.
If
it
is,
the
output
is
not
the
same.
So
the
best
expects
is
a
specific
output.
C
A
D
As
of
today,
yes
in
the
future,
we
hope
to
have
both
of
them.
But
let's
say
the
typical
use
case
today
is
that
they
have
a
cluster
which
makes
the
Linux
and
windows
nodes.
So
basically,
a
Linux
pause
and
little
containers
are
going,
of
course,
to
go
on
Linux
nodes
and
vice
versa
for
the
biggest
ones.
Okay
and
of
course,
the
deployment
can
be
consistent
in
the
mix,
tab
and
Linux,
plus
windows,
apps.
A
E
So
the
issues
earnings
it
was
when
the
test
is
trying
to
verify
that
the
API
or
the
interface
provided
by
the
cubelet
is
it
actually
fulfilled
right
now
you
have
tests
that
used
Linux
isms
to
verify
that
that
behavior
happened,
and
so
you
are
you
you,
you
said
you're
porting
these
images
to
then
at
runtime
and
do
a
switch
so
depending
on
which
node
we
scheduled
to
check
that
the
behavior
is
appropriate
using
a
different
tool.
Are
you
building
a
cross-platform.
B
C
As
of
this
moment,
the
we
do
it
is.
We
know
that
our
cluster
is
having
the
nodes
in
a
cluster,
our
windows.
So
basically
that's
why
we
have
a
yard
there.
It's
listed
in
the
presentation
for
switching
or
choosing
a
specific
registry
where
you
can
get
the
images.
So
basically,
what
we've
done
is
you
get
all
the
images
they're
the
same
name,
but
they
are
little
specific
because
they're
from
our
registries,
of
course,
this
is
just
a
temporary
thing,
they're
in
docker
hub
under
my
user,
but
this
is
this
is
temporary
for
testing
right
now.
C
F
This
is
Michael,
so
I
think
that
it's
kind
of
the
approach
here
needs
to
be
twofold.
The
first
part
is
that
we
need
to
be
able
to
detect.
The
host
operating
system
is
Windows
versus
Linux
and
that's
gonna
dictate
what
kind
of
test
agents
you're
using
right.
You
know
it's
cubelet
and
q
proxy
that
you're
using
gonna
be
Windows
based
versus
Linux,
based
that
that
could
also
inform
additional
tools.
You
need
to
use
whether
that's
busy
box
or
anything
else.
That's
part
one.
F
The
second
part
is
the
tests
themselves
need
to
know
what
containers
do
I
need
to
do
used
in
order
for
me
to
be
able
to
execute
a
particular
test,
whether
that's
an
acceptance
test
or
any
other
type
type
of
test,
and
that
part
is
what
Adelina
mention
later,
which
is
basically
saying.
Okay,
if
it's
a
traditional
test
that
exists
in
the
test
corpus
today
then
use
the
containers
that
are
already
defined
and
if
otherwise
go
ahead
and
use
containers
from
a
different
registry.
F
There
are
windows
based
because
you're
testing
you're
having
a
test
case,
that's
conforming
towards
Windows.
So
then,
both
of
those
parts
need
to
be
part
of
the
test
infrastructure
that
we
update
and
build
for
this,
so
that
we
have
that
flexibility
and
to
the
earliest
question
that
that
was
asked
in
the
future.
I
mean
you
you
already
can
today,
but
I
envision
us
to
add
support
for
Linux
containers,
only
windows
host
by
kubernetes
and
because
of
the
Linux
subsystem
being
available
in
Windows.
F
In
that
case,
then
that
might
make
a
few
things
easier,
but
I
think
the
first
part
is
be
able
to
recognize
that
your
host
is
Windows
or
Linux,
use
the
right
tools
and
be
able
to
execute
the
test
cases
and
then
add
additional
test
cases
that
you're
adding
for
Windows,
so
that
your
Windows
containers
are
the
ones
being
executed
versus
Linux
containers.
Yeah
I
hope
that
helps
yeah.
B
E
Is
there
does
the
C&C
of
conformance
working
group
have
commentary
on
when
this
stuff,
like
changing
the
definition
of
what
the
conformance
test
is
doing
or
how
it's
verifying
that
a
cluster
is
conforming
based
on
the
house?
I
can
expect
them
to
have
input
on
this
as
well.
B
We
haven't
talked
to
them
yet
I
mean
I.
Think
we'd
be
willing
to
do
that
as
a
next
step.
But
given
that
the
my
understanding
was
that,
since
this
working
group,
you're
sort
of
owns
these
tests,
I
think
we
want
to
identify
what
we
believe
is
the
right
path
forward
here
and
then
take
it
to
them
for
for
secondary
discussion.
Okay,.
A
So
they're
the
highest
impact
next
step,
for
you
would
be
to
get
in
contact
with
timothy,
st.
clair
and
attend.
We
have
sort
of
a
separate
working
or
sub
project
called
the
testing
common
sub
project
they're
more
about
specifically
how
the
tests
are
written
and
best
practices
they're.
Also
Tim
Sinclair
is
an
active
participant
in
the
conformance
working
group
and
helped
author
sonica,
which
is
sort
of
what
they
use
as
their
main
method
of
testing.
But
I
do
think.
A
Like
me,
we
also
like
have
opinions
from
my
perspective,
like
this
seems,
like
kind
of
saying
steps
in
the
right
direction,
I'm
having
difficulty
looking
too
far
ad
to
see
it.
This
turns
into
some
like
crazy
fractal
of
permutations,
but
I
think
these
are
logical.
Next
steps
like
turning
the
image
registries
in
the
flag
seems
pretty
obvious
and
finding
some
way
to
to
introspect
the
node.
You
know
based
on
the
label
and
then
using
some
kind
of
image
maybe
makes
sense,
but
I
suspect.
A
A
So
the
way
the
way
the
conformance
stuff
works,
it's
got
it's
literally
across
a
couple
things:
okay,
so
the
testing
Commons
sub
project
is
a
son
project
of
cig
testing
right
and
they're
about
like
what
are
the
best
ways
to
write
tests.
How
can
we
use
reusable
utility
as
libraries
frameworks
things
of
that
nature?
A
Okay,
so
they
probably
have
an
opinion
about
like
this.
You
have
to
get
e
to
the
image
thing
that
you
have
scattered
around
a
bunch
of
the
e2b
tests
like
they
might
have
an
opinion
in
Pablo
that
there's
a
different
approach
that
could
be
made
mom.
So
the
group
that
owns
like
whether
or
not
a
test
is
considered
part
of
conformance
is
the
architecture
sake,
so
they
have
ownership
over.
A
Yes,
this
test
defines
conformance
so
like
the
architecture
special
interest
group
kind
of
has
the
most
say
over
what
kubernetes
is
and
then
the
conformance
working
group
is
a
CNC
F
working
group
separate
and
apart
from
the
kubernetes
project,
that
is
populated
by
a
wide
variety
of
vendors,
who
are
all
trying
to
find
the
best
way
to
interoperate.
So
I
apologize
me
for
making
it
sound.
A
Like
you've
got
to
talk
to
four
different
groups
to
see
if
you're
heading
in
the
right
direction,
but
I
think
that's
kind
of
where
we're
at
right
now
and
and
I
personally,
am
a
huge
fan
of
of
what
you're
doing
here.
These
all
look
like
the
right
next
steps
to
me
so
I'm
happy
to
help
out
in
any
way
that
I
can
yeah.
E
I
think
my
only
they'll
need
like
first
blush
comment
that
I
guess
that
would
have
is
talking
to
cigar
architecture
would
be
good
just
because
like
as
soon
as
the
mechanism
by
wish
you
Val
today
what
the
test
is
actually
trying
to
validate
as
soon
as
that
mechanism
is
changing,
and
that
depends
on
we
further
Street
you're
pulling
the
image
from
that
kind
of
moves.
The
locus
of
control
for
that
test
away
from
the
test,
that's
actually
written
in
the
Cuban
Hays
repo
and
it
moves
it
into.
E
A
B
B
That
a
question
I
had
on
the
regarding
being
able
to
use
different
registries.
Has
it
been
brought
up
in
other
contexts,
because
I
know
that
today
it
seems
to
have
a
heavy
dependence
on
docker
hub
and,
if,
let's
say
docker
hub
were
to
go
down,
you
know
for
a
brief
outage.
You
know,
I
wouldn't
want
all
the
kubernetes
tests
to
just
fail.
Yeah.
A
G
A
G
B
One
of
the
so
I
think
it's
possible
for
some
images,
but
in
order
to
be
able
to
use
an
image
manifest
list,
whoever
owns
that
that
repository
and
in
the
case
of
the
busybox
one
it's
owned
by
I,
guess
as
TN
and
or
somebody
that
represents
them
from
the
community,
but
basically
they
they
build
those
images
and
it
would
be
up
to
them
to
take
ownership
and
in
subs
them,
and
so
the
problem
is
that
you
can't
maintain.
So
like.
Let's
say,
we've
found
a
window
specific
bug
fix.
B
G
B
And
that
was
kind
of
what
I'm
getting
at
because,
like
you
know,
looking
at
the
list
of
images
that
are
used
today,
I
see
things
like
nginx
I
see
other
things
like
that.
So
you
know
if
you're
a
reliant,
let's
say
the
default.
Nginx
image
changed
where
instead
of
you
know,
returning
out,
you
know
207,
you
know
404
not
found
or
something
like
that,
because
you
didn't
pass
in
any
static
content
for
it.
You
know
I'd
hate
for
that
to
break
the
tests
too.
So.
B
A
C
A
Perfectly
happy
having
an
umbrella
issue,
saying
adding
ACS
engine
support
acute
test
and
then
PRS
that
hi-
really
wants
a
lot
more
structure
that
works
for
me.
Okay,
cool
starting
for
moving
quickly
here.
I
just
want
to
give
Tim
pepper
enough
time
to
do
the
King,
Trebek's
approachable
to
the
next
week.
If
you
prefer.
H
B
H
Basic
idea
is
that
we
and
contributor
experience
are
trying
to
improve
they
contributor
experience
kind
of
self
obvious
plain
English
there
and
part
of
that
is
doing
some
outreach
going
out
to
the
SIG's
and
just
kind
of
letting
them
know
a
little
bit
more
about
what
it
is.
We're
trying
to
do,
and
some
of
the
specifics
and
why
and
where
that
starts,
to
touch
in
other
SIG's
y'all
have
seen
some
of
this
already
I.
H
This
really
goes
more
into
processes
and
roles
and
responsibilities
and
there's
kind
of
talks
about
what
it
is
that
the
sig
does
and
lays
it
out
in
a
way,
that's
more
understandable
for
for
members,
and
people
can
see
a
little
more
clearly
where
they
might
be
able
to
contribute
and
help
the
sig
and
again
just
in
support
of
the
the
general
idea
that
we're
trying
to
build
the
contributor
base.
So
there's
there's
information
on
that
stuff.
H
That's
linked
and
the
doc
and
I'll
just
kind
of
leave
that
at
that
there
is
a
contributor
Gard
guide
that
we
have
been
working
on
and
improving
again
linked
here
in
the
document.
But
one
of
the
things
that
we
wanted
to
call
out
to
individual
SIG's
was
that,
where
there
are
specifics
that
are
different
than
the
rest
of
the
broader
kubernetes
that
read
on
your
sig,
it
makes
sense
to
add
something
like
a
particular
contributing
file
or
a
developer
guide
portion
that
specific
to
your
SIG's.
H
It
elucidates
on
that
and
again
testing
I
think
is
one
of
the
areas
where
there
is
a
pretty
good
existing
corpus
of
info
there.
So
not
entirely
new
info,
then
mentoring.
We
have
a
whole
bunch
going
on
at
the
moment.
We
have
a
meet
our
contributors
session
on
slack,
that's
kind
of
or
slack
and
zoom,
that's
mentoring
on
demand
and
then
also
at
the
end
of
the
dock.
There's
a
user
office.
Our
is
kind
of
similar
just
trying
to
get
people
available
for
asking
questions
and
interacting
with
users
and
contributors.
H
Looking
at
cig
testing
I
noticed
that
there
are
a
number
of
pin
documents,
but
some
of
them
are
fairly
old,
like
there's
a
1.9
road
map
sitting
there.
So
these
are
just
subtle
little
things
that
for
new
contributors
coming
along
showing
up,
if
there's
information
pinned
there,
for
example,
and
it's
up-to-date,
it
just
makes
it
easier
for
somebody
new
to
come
in
and
orient
and
in
four
minutes,
that's
kind
of
quickly.
Briefly,
what
I
wanted
to
share
with
you,
guys
and
obviously
sig
can
trigger
on
slack
come
around
talk
to
us.
H
E
H
Doesn't
make
as
much
sense,
yep,
yeah,
I,
think,
there's
there's
a
history
of
kind
of
copypasta
and
let's
a
little
bits
of
things
here
and
there.
So
my
vision,
I
think
for
the
the
contributor
guide
is
to
have
less
there.
Have
it
be
as
much
kubernetes
specific
as
possible
and
then
link
to
existing
authoritative
data
elsewhere,
where
people
who
are
subject
matter,
experts
are
main
painting
it.
That's
the
most
sustainable
thing,
I
think
over
time.
We're
working
that
way.
H
A
Automatically
updated
so
like
when
there
was
a
dock
that
described
in
great
detail
how
full
requests
were
ordered
in
that
the
submit
queue
it's
like
well,
we
could
do
that
or
we
could
just
link
to
that
information
directly
in
the
submit
queue
and
make
sure
this
would
make
use
like
merge.
Apartments
are
dynamically
updated
based
on
the
configuration
that
it's
got
so
one
of
the
things
that
we
can
do
and
like
we've
made
a
lot
of
progress
there
with
bak
commands
and
plugins
and
stuff,
which
is
one
thing
but
then
like.
A
A
Dock
to
something
that's
self
hosted
on
prowl
and
then
in
the
great
effort
to
make
a
consistent
experience
across
all
repos
I
want
to
get
rid
of
people's
ability,
people's
need
to
have
directory
access
to
repos.
There
are
a
couple
things
standing
in
the
way.
Eritrea
has
added
a
slash,
milestone
command
a
little
bit
back,
which
was
one
of
the
main
things.
A
There
are
a
lot
of
labels
involved
with
the
cherry-pick
process
that
people
can
only
manually
apply
right
now,
so
there's
been
an
experiment
called
a
cherry
picker
that
seems
to
work
well
for
openshift
and
we're
moving
that
into
a
proud
book
and
Jordan
Leggett
made
some
suggestions
to
improve
the
process.
That
discussion
happened
in
contributing
experience
just
from
alike.
What's
the
experience
of
this
I
think
would
be
a
good
idea
enough
to
bring
that
into
sick
testing
discussions
as
we
improve
the
trip
across
those
through
ground.