►
From YouTube: Kubernetes SIG Testing 2018-09-04
Description
A
So
welcome
everybody
to
today's
cig
testing
meeting.
My
name
is
Aaron
Cregan,
burger
and
I.
Am
your
host
today
is
Tuesday
September
4th
weekly
sig
testa
meeting.
You
excuse
me
being
publicly
recorded
and
will
be
posted
to
YouTube
later,
so
please
smash
that
subscribe
button
and
like
with
that
I
will
hand
over
to
hippy
hacker
who
is
going
to
present
to
us
today
about
API
soup,
and
if
we
have
some
time
afterwards,
we
will
talk
about
refactoring
effort
of
the
e2e
framework.
It
be
all
you.
B
Thanks
for
the
introduction,
Aaron
so
glad
everybody
here
today,
I'm
gonna
go
through
just
some
simple
sections
of
where
things
have
been
started
and
where
things
were
heading
with
a
px
new
project.
One
of
my
favorite
things
that
I
found
this
all
of
the
different
ways
where
community
generates
artifacts
and
the
way
that
we
generate
our
clusters.
B
It's
a
very
diverse
ecosystem
and
I
have
come
to
really
appreciate
our
continuous
integration
system
provided
by
the
sig
testing
and
the
testing
for
repo
I
have
gone
a
lot
of
different
paths
and
I'm,
starting
to
really
focus
on
improving
and
trying
to
find
a
consistent
way
within
sig
testing.
Particularly
this
first
item
here.
The
test
grid
artifacts
I,
just
found
out
this
morning
that
these
are
publicly
available
and
have
all
of
the.
B
Audit
logs
that
we
need
to
process
and
they're
continuously
updated,
that
is
just
delightful
I've
pulled
these
pieces
in
and
will
continue
to
do
so
in
our
ongoing
basis
from
here
forward.
I
just
want
to
give
a
shout
out
to
everyone:
that's
been
a
part
of
making
test
grid
and
all
the
artifacts
available
I
think
we
should
look
at
more
ways
of
integrating
this
work
into
our
community
and
seeing
meaningful
outputs.
So
what
do
we
do
with
those
logs.
B
We
have
some
code
inside
of
our
repo,
that
is
some
just
some
Python
code
initially,
and
if
we
look
at
the
particular
type
of
them,
headers
that
are
available
early
on,
you
can
see
that
we
don't
have
any
user
agents.
It's
just
a
bunch
of.
We
don't
know
what
test
is
coming
in
and
we
add
it
in
a
little
later
if
I
go
through
and
we
just
look
at
all
the,
for
example,
at
1.12
and
later
we
had
this
beautiful
piece
of
code.
B
This
will
allow
us
to
see
all
the
user
agents
at
this
point.
There's
we
only
got
one
so
there's
only
nothing
with
the
test
in
there,
but
if
we
say
grab
the
thousand
of
those
you
start
to
see.
Different
user
agents,
including
some
from
our
updated
eat
and
eat,
vest
that
let's
make
this
one
that
have
the
test
info.
B
So
I'll
go
ahead
and
close
that
out.
One
thing
that
I
wanted
you
to
note
here
is
that
only
in
1.12
and
later
do
we
have
the
ability
to
do
to
do
this,
so
I'm
gonna
go
over
here
and
bring
down
just
re
te
test
and
I'm
gonna
increase
this
to
a
whole
whole
right.
So
this
is
all
of
the
e
to
e
test
endpoints
that
are
part
of
this
particular
suite
this.
B
There
may
be
something
wrong
through
the
data
I'm,
not
sure,
but
this
is
what
we
gain
a
value
that
we
need
to
find
a
way
to
back
port,
so
I'm
gonna
go
a
little
bit
further
into
that
problems.
We
progress
down
through
these
results.
This
is
how
we
process
this
information
we
go
through
and
we
find
with
the
swagger
the
definition
it
needs
to
match
the
clients
of.
B
What's
communicating
to
our
our
kubernetes
server,
so
I
just
kind
of
move
these
around,
in
which
one
I'm
going
to
look
at
right
now,
so
we'll
hit
eleven
bit
eleven
down
here
at
the
bottom
and
we'll
be
sure
to
focus
on
eleven
here.
So
we
pull
this
over.
So
you
kind
of
see
what
happens.
It's
going
to
evaluate
this
block
and
I
want
to
do
that.
I'm,
gonna,
just
for
grins.
B
So
you
see
this
run
over
here.
There's
some
there's
some
watch
this
and
thanks
what
was
really
cool
is
just
a
few
days
ago,
Aaron
was
able
to
add
some
a
pull
request
to
help
us
not
focus
on
end
points
that
have
been
deprecated.
We
also
saw
some
new
some
possible
features
in
open
api
3,
and
maybe
something
into
me.
Aaron
can
speak
to
that
later,
where
we
should
probably
not
just
be
putting
deprecated
in
the
string
field
and
or
in
the
descriptions,
but
there's
actually
some
attributes
that
make
this
a
whole
lot
easier.
B
Let's
see
so
once
we
do
this,
this
puts
it
inside
of
a
what's.
It
called
a
Monte.
B
Add
a
SQL
database
because
it's
a
kind
of
a
complex
thing
when
we
go
down
to
this
next
spot
it'll
run
a
command
to
export
that
to
a
CSV
we're
using
some
interesting
stuff
with
the
d3
library
the
simplest
way
to
feed
it
data.
Currently
it's
to
generate
a
CSV.
We
probably
will
not
be
doing
that
going
forward
will
be
using
node
and
doing
something.
That's
some
really
fun
stuff,
but
that's
I'll
leave
that
does
that
he's
been
picking
up
a
lot
of
the
flow
here.
B
B
What
is
it
called
our
test
grid
artifacts,
and
we
could
provide
a
link
from
pests
grid
when
you
click
on
that
test
grid
artifact,
for
it
to
do
this
and
generate
the
graph
that
would
not
be
an
unreasonable
and
unreasonable
math
stuff.
B
Just
you
can
kind
of
see
those
artifacts
were
generated
and
the
ones
that
they're
not
going
to
have
an
e
to
e
test
each
test
inside
the
logs.
We
have
to
do
some
porting
on
that,
and
so
I'm
gonna
go
through
kind
of
the
step
through
that
hoarding
I've
for
kubernetes,
so
for
the
III
Oregon
github
and
I've
provided
some
branches
there
for
the
the
one
110
and
111,
because
those
they
need
the
back
court
order
to
work.
B
You
go
ahead
and
once
you've
checked
that
out,
you
need
to
build
from
source
I'm
really
done
using
dine
and
particularly
I'm
interested
in
seeing
the
work
that
been
the
elder
has
been
working
on
to
increase
the
speed.
It
is
a
joy
to
work
with
dine
and
kind,
especially
when
you're
trying
to
do
many
things
at
once.
The
speed
and
iteration
loop
is
meaningful.
B
We
need
to
be
able
to
run
the
tests.
I
found
this
working
with
the
different
parts
of
the
cube
root
of
the
unity
test
system.
It's
great
to
actually
just
run
the
e2e
test
itself,
so
you
can
see
in
this
section
here.
It
actually
uses
basil
to
generate
just
the
e2e
binary
and
then
as
long
as
qu
config
is
set
I'm,
not
sure
ginko,
parallel
matters,
but
as
long
as
coop
configures
set
the
e
to
e
test
suite
doesn't
need
any
other
variables.
B
So
you
need
to
specify
everything
on
the
command
line
and
it
gives
a
little
more
precise
control,
while
you're
trying
to
do
deep
level
poking
around
in
our
test
suite,
at
least
from
my
perspective,
I'd
love
to
see
what
other
people
are
doing
in
our
community
and
then
the
last
piece
is
retrieve
the
audit
logs
from
the
API
server
I'm,
depending
on,
if
using
dined
or
coming.
You
can
just
do
a
docker
CP
to
get
it
out
of
the
container,
which
is
nice
and
then
of
course,
I
didn't
document
here.
B
But
the
next
steps
would
be
to
pull
it
out
of
the
API
server
on
on
GC.
So
that's
our
next
steps
there
and
then
here's
the
what
I
needed
to
do
to
back
port
112.
This
is
fine.
It's
great
it's
going
to
take.
The
next
part
is
the
back
porting.
So
this
is
what
I
did
to
do
the
back
part.
So
those
are
the
branches
I
made
and
there's
a
particular
cherry
tree
picks
we
need
to
get.
B
C
B
The
best
way
to
communicate
a
per
test
thing
within
our
framework
and
then
the
next
part
was
the
letting
the
API
server
log
that,
as
the
clients
are
talking,
and
that
was
was
really
was
pretty
early
on,
and
so
we
cherry
pick
those
commits
and
then
we're
we're
done.
B
So
this
is
kind
of
the
one-liner
once
we're
up
just
run
cube
test
basil
up
provider
go
and
they're
going
to
do
our
test
and
I
would
love
some
feedback.
As
I
said,
I've
been
trying
to
run
the
test
directly
using
the
88's
binary,
even
with
ginko
parallel
total
8
and
all
of
this
sometimes
I
was
really
slow
and
I'm
not
super
familiar
with
with
why
and
how
to
see
that
things
are
running
in
parallel,
but
that
would
be
yeah.
B
One
great
thing,
I
think
that's
pretty
much
it
for
the
interactive
Kodi
demo,
if
it's
the
rest
of
its
just
just
the
website.
B
Chrome
I
know
that
API
scoop
not
co-op.
That's
the
old
one
on
the
IO,
so
an
IO
that
stuff
from
our
sitcom
the
test
speed.
These
are
the
current
test
grid
artifacts
for
the
GCE
testing.
B
That's
great
thanks
cool
yeah
and
you
a
new
your
mouth,
so
we
we
can
make
this
do
whatever
we
want
right,
but
right
now,
I
just
used
a
little
short
name
because
it
was
convenient.
You
can
see
that
this
integrates
errands.
B
Please
don't
bother
us
with
the
deprecated
API
patch
deprecated
deprecated
endpoints
patch,
and
then,
though,
oh
it's
going
out,
it's
going
up
so
I
yep,
so
I
go
and
I'm,
not
sure
what
happened
here
on
12,
there's,
probably
a
data
problem,
but
that's
yeah,
it's
kind
of
exciting
to
see
where
were
where
the
numbers
are
heading
and
the
possibility
for
that
tag,
cloud
and
sig
cloud
to
be
able
to
say
which
what
would
with
this
sig
tag,
which
is
in
the
test,
but
also
doing
some
static
analysis
and
finding
which
parts
of
our
code
base
are
maintained,
like
with
the
owners
files
to
be
able
to
see
when
I
look
at
this
particular
from
when
I
look
at
the
community.
B
From
this
particular
SIG's
perspective,
what
things
might
I
want
to
see
and
I
know
that
Zack
has
been
looking
at
automatic,
adding
github
plug-in.
So
it
knows
who
you
are
when
you
come
to
look
at
this
and
giving
you
context
and
what
you
might
be
interested
in
seeing
within
our
community,
so
that
kind
of
that's
kind
of
it
for
now,
but
there's
lots
more
ideas
we
have
under
on
get
github
API.
Still.
B
There
is
some
kinds
of
projects
to
get
his
projects
to
there's
a
bunch
of
different
visualization
thoughts
that
it
might
be
interesting
to
get
people's
feedback
on
that's
known
as
quite
a
few
actually
sorry
on
this
one.
So
I
wanted
to
show
all
individual
endpoints.
Currently
we
mouse
over
they're,
not
all
there.
Let's
make
this
bigger
some
reason.
My
computer
is
kind
of
stopped
responding
yeah,
it
has
stopped
responding,
but
that's
pretty
much.
It
go.
Look
at
the
possibilities
and
vote
for
your
favorites
go
from
there.
B
C
B
A
Grouping
the
end
points
together,
just
it
might
be
interesting
to
group
them
in.
So
you
see
the
end
point
and
then
the
different
verbs
that
are
covered
for
the
end
point
I,
don't
know,
I
think
that's
the
UX
things
here
somewhere.
So
it's
also
worth
calling
out
that
Kathryn
is
on
the
call
here,
like
I,
give
full
credit
just
to
say
it
on.
The
video
like
I,
give
full
credit
to
her
for
the
deprecation
and
for
building
stuff.
A
I
ported
it
over
from
of
you
that
she
contributed
that
was
more
table-based,
allowing
you
to
drill
down
to
each
specific
component
of
the
endpoint.
If
we
find
we
need
to
differentiate
between
more
than
just
alpha
beta,
stable
or
what
group
the
end
point
belongs
to,
and
I'm
still
kind
of
in
the
midst
of
saucing
as
data
but
yeah
I
agree.
It's
been
really
interesting
to
see
this
progress
forward,
we're
using
this
for
now,
as
kind
of
a
guiding
light
on
conformance
test
coverage
efforts.
A
But
it's
not
necessarily
our
prime
driver
like
we're
not
about
to
go,
write
a
test
that
cruds
every
single
endpoint
just
to
make
that
go
up.
That's
the
kind
of
the
wrong
direction,
but
it
does
kind
of
help
us
make
sense
of
what
API
endpoints
are
we
covering
and
for
kubernetes
as
a
whole?
Is
its
API
surface
area
generally
trending
more
towards
stability
beta
or
alpha
stuff,
like.
E
A
A
F
C
F
For
the
purpose
of
getting
the
coverage
up,
are
you
certain
that
it
doesn't
make
sense
to
do
that?
I
mean
certainly
not
as
a
as
the
endpoint,
but
it
does
seem
like
just
a
really
generic
test
against
the
open.
Api
definitions
would
catch
some
just
like
screw-ups,
so
I,
potentially
some
of
the
vendors
I.
A
Have
seen
an
issue
to
that
effect
pop
up
in
the
past,
where
it
would
guarantee
whether
or
not
an
API
is
responsive
or
present,
but
that
sort
of
test
wouldn't
necessarily
guarantee
that
the
functionality
behind
that
API
isn't
in
any
way
shape
or
form
meaningful.
So
I
don't
think
it's
useful
as
a
driver
for
conformance
testing.
It
could
be
a
very
interesting
use
of
API
snoop
as
a
tool
to
encourage
further
development,
but
I
think
in
terms
of
exercising
the
behavior
of
a
kubernetes
cluster
for
conformance.
It's
not
it's
not
ideal.
F
A
And
I
guess
I'm
just
articulating
that
my
concern
is.
You
could
then
run
that
test
against
another
kind
of
script.
That
just
would
generically
responds
to
all
of
those
endpoints.
You
wouldn't
actually
be
confirming
whether
or
not
the
thing
that's
on
the
other
side
of
those
endpoints
is
a
congrat
YZ
cluster
or
not
agreed.
F
G
Aron
is
echoing
everything,
I
would
say,
sounds
good.
The
other
comment
I
wanted
to
make
with
regards
to
Akana
that
Steve
made
it
if
you
actually
track
API
deprecation
policy
with
some
of
this
you'll
find
that
we
don't
follow
our
own
policies.
We
actually
have
a
bunch
of
stale
alpha
and
beta
API
that
have
existed
for
way
too
long,
even
though
the
EP
hours
have
graduated
well
beyond
their
lifecycle,
so
we
actually
have.
G
If
you
look
inside
the
code,
we
have
like
up
to
five
layers
of
conversion
that
is
still
somehow
supported,
and
the
fact
that
even
works
actually
amazes
me
so
I
don't
know
if
we
should
somehow
raise
that
with
API
machinery
or
whatnot,
but
you'll
find
that
some
of
your
alpha
and
beta
numbers
will
be
weirdly
conflating
and
probably
are
not
indicative
of
what
should
be
the
truth.
That.
A
H
We
can
we
can
get
started.
Basically,
I
I
want
to
move
forward
with
this
proposal.
There
is
something
that
can
be
discussed
either
google
docs
or
in
the
corresponding
issue
that
I
created
a
while
back
and
and
Tim
also
kind
already
leave
some
comments,
but
I
thought
that
bringing
this
to
this
audience
here
might
might
be
a
faster
way
to
make
progress.
So
just
to
recap,
my
my
motivation
is
to
make
the
existing
test
e
to
e
framework.
Also
usable
outside
of
Cuban
ideas
outside
of
abilities.
H
I've
seen
that
both
in
my
own
project,
I'm
working
on
a
CSI
driver
that
eventually
needs
some
kind
of
into
end
testing,
I've,
seen
it
in
the
Cuban
eighties,
six
storage,
which
currently
has
unit
testing
for
its
CSI
components,
but
not
really
end
to
end
testing
outside
this
one
test.
That
is
in
divinity
score,
but
we,
we
hope
and
I'm
working
with
a
ballistic
on
that.
H
We
hope
to
expand
that
test,
set
to
come
out
more
tests,
different
scenarios
and
then
make
both
tests
available,
also
to
others
CSI
rather
implementers,
but
yeah
in
the
current
state.
People
apparently
I've
arrived
their
own
framework
for
doing
end
to
end
testing
but
humanities
or
way
somehow
do
it
with
the
current
framework.
But
it
does
certainly
not
ideal
and
looking
at
the
code,
there
are
also
various
other
to-do
items
from
past
years,
like
the
VIPRE
configuration
handling,
which
kind
of
got
started
two
years
ago,
but
then
hasn't
progressed
much
and
there
again.
H
This
whole
partial
implementation
of
vipers
abort
leads
to
questions
that
people
have
that
want
to
write
tests.
That
I
ran
into
about
myself
when
I
added
some
configuration
option
for
the
existing
a6
storage
test
for
CSI
I,
had
a
command
line,
parameter
added
as
the
flag
inside
the
test
itself,
and
then
I
was
asked
to
move
it
to
its
emperor
framework
which,
just
in
my
opinion,
wrong
wrong
place
for
it.
H
My
Google
Doc
that
I
created
that
has
some
proposal
and
there's
actually
a
prototype
ranch
that
I
created
to
try
out
this
idea.
It's
I
think
it's
working,
it's
doable.
My
open
question
really
is
how
we
move
forward
with
that.
Tim
was
making
somewhat
suggesting
that
the
long
term
goal
should
be,
for
example,
to
completely
replace
the
existing
or
current
set
of
command
line
parameters
with
something
that
fits
better
into
a
rack.
Little
Viper
configuration
file
and
well
I
do
agree
with
a
goal.
A
So
I
think,
like
my
personal
opinion,
is
I.
Would
there
are
many
people
who
want
to
see
et
framework
refactor?
There
are
many
things
wrong
with
it.
I
would
hate
for
people
for
scope
creep
to
pile
in
on
you.
So
if
I,
look
at
your
dock
and
I
see
that
your
first
goal
is
to
be
able
to
import
e
to
e
framework
as
a
vendor
dependency
without
pulling
in
the
entire
world,
thus
enabling
people
to
use
that
framework
out
of
tree
to
write
their
own
tests.
A
I
think
that
is
laudable
and
an
appropriate
initial
scope
for
MVP
I
think
everything
else
you
list
sounds
cool
I,
really
only
care
about
that.
First
bullet
point
happening
successfully.
The
rest
of
the
stuff
I
would
all
point
to
nice
to
have
okay,
we're
also
like
apologize
for
not
engaging
with
this,
because
I
feel
as
strongly
as
Tim.
Does
that
like?
A
It
is
super
awesome
that
you
were
working
on
this
I
personally
have
been
heads
down
on
other
efforts
and
we
are
also
entering
into
code
freeze,
so
I
think
getting
you
to
a
place
where
you
are
ready
to
start
ramping
up
on
this.
As
part
of
the
113
time
frame
would
be
ideal,
because
what
I
think
would
be
not
idea
what
happen
is
to
somehow
end
up
mid
refactor
when
we
cut
113.
H
H
It
just
means
that
the
flags
need
to
follow
some
certain
naming
scheme
with
a
dot,
for
example,
to
separate
different
levels,
and
then
I
already
have
one
commit
which
takes
the
existing
parameters
and
maps
them
to
Vipers
of
a
piper
basically
pulls
from
this
arc
ago,
a
command-line
argument,
and
then
you
have
full
Viper
configuration
file,
support,
so
I
don't
see
any
big
blockers.
If
we
agree
that
business
is
a
good
first
step,
then
I
will
just
proceed
with
the
implementation
and
will
will
eventually
submit
a
pull
request
after
we,
the
current
release
is
done.
G
When
you're
nearing
the
end
of
the
release
cycle,
it
doesn't
hurt
to
push
up
a
whip
near
the
end.
If
people
aren't
too
busy
just
to
say,
work
in
progress,
please
give
feedback
at
some
times.
If
it's
a
large
PR,
it
can
take
a
number
of
rounds
to
get
through
the
cycle
or
you
might
want
to
refactor
it
into
breaking
up
into
individual
discrete
pieces
that
are
easier
to
review.
That's
always
the
best
for
a
project
to
this
year.
H
I
at
this
point
its
individual
commits-
and
it
starts
so
that
the
first
one
could
be
taken
without
breaking
anything
and
I
started
actually
start.
A
great
example
request
for
we're
moving
dead
code
in
in
VA
and
to
end
tests
already
so
yeah
I
think
I'm
doing
I'm
doing
that.
That
should
that
shouldn't
be
an
issue
and
I
can
do
a
work
in
progress,
pull
request
like
I
said
probably
next
week,
I
guess.
H
A
Okay,
I
super
appreciate
you
bringing
this
to
the
group
I
apologize
that
we
ran
late,
walk
through
this
I
think
it
was
worthwhile.
I
would
say
like
so.
The
number
one
thing
I
would
like
to
avoid
is
you've
done
a
great
job
of
putting
together
this
proposal
and
doing
some
work.
I
would
hate
for
somebody
to
swoop
in
and
say
whoa
whoa.
What
are
you
doing
like?
This
is
a
bad
idea,
because
X
Y,
&,
Z
and
I
can
tell
that
Tim
is
trying
to
shop.
A
H
H
Yeah
I
don't
think
that
porting
will
work
that
well,
though,
because
people
are
continuing
to
to
change
the
tests,
there
are
some
some
there
are
some
some
trivial
code
conflicts
nothing
major,
because
not
that
much
happens
at
the
framer
itself,
but
because
I
need
to
move
code
around
yeah.
There
is
a
risk
of
of
code
conflicts
right
and
when.