►
From YouTube: Kubernetes SIG Testing 2017-08-01
Description
Meeting notes: https://docs.google.com/document/d/1z8MQpr_jTwhmjLMUaqQyBk1EYG_Y_3D4y4YdMJ7V1Kk
A
C
Will
go
ahead
and
hand
off
to
you,
so
there
was
a
comment
on
sink
Western
lifecycle.
We
were
discussing
this
last
week
not
this
week
regarding
the
fact
that
the
sink
testing
infrastructure
builds
from
the
build
directory
and
it
doesn't
actually
run
the
release
bits
in
the
comet
we
had
made
is
that
it
doesn't
antigo
doesn't
get
tested
till
the
end
of
the
release
and
by
a
person.
C
B
B
For
you
sure
that.
C
B
D
D
A
C
This
heart,
not
really
the
beige
you're
concerned,
I,
think
that
the
entire
build
products
that
we
produce
are
not
regularly
tested
so
Kubb
ADM
is
a
good
example
where
it
requires,
as
I
was
pointing
out.
It
requires
that
Deb's
and
rpms
to
be
built
right
in
order
for
you
to
validate
everything
to
be
properly
running
so
you'd
want
to
have
the
those
built
as
a
regular,
build
chain,
artifact
and
they're,
not
right
now,
they're
on
the
island.
A
A
E
E
E
F
E
F
E
E
Thing
that
I
had
noticed
this
it's
actually
kind
of
similar
if
we
started
using
some
of
the
Jenkins
pipeline
stuff.
You
end
up
in
this
weird
sort
of
fight
for
who
gets
to
control
what
the
step
definitions
are
and
Travis
only
has
the
resolution.
As
far
as
you
know,
each
of
the
steps
of
Euclid
is
that
considered
yeah
MO
and
often
it
doesn't
make
as
much
sense
to
have
a
whole
bunch
of
fine-grained
stuffs
in
there,
and
rather
nidia
makes
harder
or
something.
F
Is
anybody
aware
of
other
formats,
aside
from
j-unit,
for
sort
of
representing
a
workflow
and
steps
within
that,
because
I
yeah
I
think
you're
right?
There
were
a
lot
of
times
like
you
know,
we
have
a
j-unit
result
for
like
cube
up
which
technically
isn't
a
test,
but
that's
sort
of
a
useful
thing
to
highlight
that
that
phase
failed
and
so
right
now
we're
encoding
that
into
a
J
unit
thing.
C
Good
lab
has
their
own
ya
know
that
you
can
specify
I'm
not
intimately
familiar
with
it,
but
I
just
know
of
it
and
just
a
plus
one
to
the
readability
of
Travis.
It's
very
easy
to
find
out
where
exactly
you
fail
to
dive
directly,
where
you
care
about
versus,
like
having
this
sift.
E
F
C
E
E
F
So
is
the
essential
idea
that
we
run
our
jobs
and
then
our
jobs
produced
to
build
blog
text
and
maybe
also
some
j-unit
xml
artifacts,
as
well
as
some
potential
other
artifacts,
and
then
something
notices
that
that
build
was
uploaded
and
then
it
scans
it
analyzes
the
logs-
and
maybe
we
sort
of
say
that
we
grep
for
these
math
these
matching
lines
and
these
types
of
files.
And
what
does
it
do
with
that?
Did
it
like
spit
out
some
other
thing
with
vim
Gruber
nadir
displays
that,
as
opposed
to
the
raw
built
log,
yeah.
E
I
think
the
general
idea
that
I
had-
and
this
is
kind
of
half-baked,
slow
right
now,
but
if
we
could
take
yeah
define
you
know,
this
is
a
failure,
cause
that's
described
by
some
red
X
or
some
block
that
we
look
for,
and
here
are
the
types
of
log
files
that
we're
going
to
look
for
the
failure
inside
of
and
then
once
we
find
it.
I
was
hoping
that
we
would
be
able
to
tag
it,
maybe
with
some
opaque
tags
for
search
ability
and
then
also
whoever's,
configuring,
the
flake
or
the
failure.
E
Detection
would
be
able
to
provide
priority
or
important
values,
because
I
think
one
of
the
things
that's
always
really
hard
is
ranking
relative
importance
of
failures
but
yeah.
If
we
could
take
it
and
then
describe
it,
you
know
here's
the
snippet
of
the
log
file
use
the
links
or
you
could
find
out
online.
Here's
the
relative
importance
that
we
gave
it,
and
here
are
some
opaque
tags
that
you
know
we
know
this
is
an
infrastructure
failure,
for
instance,
and
then
it
tools
like
Google
Rayner
could
use
that,
in
addition
to
j-unit.
F
Are
people's
thoughts
about
how
the
goober
nader
goober
Nader's
output
right
now.
A
It
sort
of
seems
to
highlight
any
times
where
air
pops
up
in
the
log
I
don't
know.
I
personally,
have
don't
not
spending
a
lot
of
cycles
like
going
to
a
good
breeder
page,
seeing
the
air
text
going
through
the
log
file
finding
the
exact
part
of
the
test
where
that
air
at
troubleshoot,
but
I
assume
that
probably
the
round-trip
time,
where
I
almost
always
have
to
go
to
the
build
log.
A
A
I
sort
of
second
Eric's
comment
that,
like
I,
think
what's
helpful
for
us
is
the
ability
to
call
out
the
basis
of
different
steps
and
then
output
that
corresponds
to
those
steps
who
want
to
highlight
that
in
je,
and
it
just
seems
to
be
like
a
machine,
readable
format
that
a
lot
of
common
tools
out
there
used
today.
Other
consumers
of
data
and
GCS
to
me
would
be
cattle,
which
I
think
actually
maybe
consumes
the
GCSE
stuff
and
puts
it
a
format
that
gubernatorial
not
sure
and
then
I
also
think
it's
like
tester
in.
A
A
It
would
be
another
right
what
we're
doing
with
Google
in
four
examples:
let's
do,
if
any
put
together
a
plugin
that
runs,
go
in
on-demand
against
all
requests
and
then
just
comment
on
the
actual
pull
request
lines
where
that
stuff
is
happening
so
that
way
like
if
somebody
addresses
that
and
pushes
a
new
to
Memphis
just
go
away.
That
seems
like
really
useful
context
for
that
yeah.
That
class
of
error,
but
I
agree
that,
like,
if
there's
a
nicer
format
than
j-unit
for
this
sort
of
like
these
steps
happen,
then
this
is
the
output.
F
E
Yeah
one
of
the
things
that
I
think
we
noticed
downstream
on
OpenShift
we've
there's
a
set
of
tools
that
was
written
that
allows
us
to
generate
jameth
for
specific
lines
of
Bashan,
just
wrap
it
around
it
and
it
becomes
difficult.
I
think
some
of
these
steps
it'll
be
difficult
to
integrate
with
a
tool
like
test
grid,
because
there's
not
necessarily
this
temporal
linking
of
these
steps
between
every
job.
E
F
Would
agree
with
Aaron,
and
you
know
Tim,
that
I
have
I've
pretty
much
find
myself
always
clicking.
The
expand.
Think
is
inevitably
the
error
that
I'm
actually
or
the
actual
cause
doesn't
seem
to
be
highlighted
because
it
doesn't
have
the
word
fail
or
erred
in
it,
and
so
I'm
a
little
and
I'm,
also
a
little
bit
of
verse
to
I
sort
of
just
feel
like
if
we
I'm
adverse
to
going
down
the
path
of
like
every
error
that
we
want
is.
F
Goober
Nader
actually
process
that
right
now,
which
probably
is
not
the
end,
result
that
we
want
over
time,
but
goober
Nader,
actually
out
files
directly
and
processes
them
Ketel
does
the
same
thing
and
then
it
uploads
it
into
bigquery
and
then
test
grid
has
a
job
which
is
doing
the
same
thing
where
it
reads
all
the
build
log
and
the
JUnit
XML
and
converts
it
into
a
proto
format
that
tests
grid
understands
so,
like
you
know,
potentially
yeah
I
mean
it
doesn't
necessarily
need
to
be
temporarily
based
like,
for
example,
when
we
do
go
test
it,
those
will
be
run.
F
You
know
we
have
a
bunch
of
tests
that
run
in
parallel
a
lot
of
times
and
then
those
will
all
still
get
different
rows
and
test
grid,
and
so
you
know
I.
What
I
would
really
like
to
see
is
a
format
that
allows
us
to
sort
of
represent
like
here's.
What
happened
during
this
run
in
a
way
that's
sort
of
convenient
for
tools
to
output
and
convenient
for
humans
to
consume.
E
F
Mean
so
what
we
did
now,
it's
not
so
what
like,
when
we
convert
like
there's
two
so
bootstrap
actually
doesn't
really
care,
it's
sort
of
agnostic
to
bootstrap.
What
bootstrap
does
is
it
checks
out
your
repository
start
some
process
that
represents
your
job
and
then
logs
all
the
output
and
then
uploads
it,
and
then
it
will
also
upload
any.
Are
any
there's
like
a
artifacts
directory
and
anything
you
put
into
the
artifacts
directory.
It
will
then
upload
the
GCS,
and
so
you
know,
but
perhaps
well
I
feel
like
yeah.
F
Maybe
you
know
defining
like
either
a
proto
or
an
XML
format
or
a
JSON
format
or
a
gamma
format.
That,
like
represents
what
happened
and
at
the
end
we
can
update
test
grid
to
you,
know
up
to
a
test
grade
and
guru,
nadir,
to
read
all
of
that
and
display
that
and
then
make
it
easier
for
jobs
to
produce.
That
would
sort
of
be
something
that
sounds
nice
to
me.
F
If
yeah
I
mean
I
and
I
think
it
does
require
some
discipline
and
how
we
actually
write
our
tooling
and
I'm,
not
sure
how
palatable
that
is,
for
people,
but
I
feel
like
there's
sort
of
a
general
senses
that,
like
the
shell
script,
usage
is
sort
of
a
little
unwieldy
like
I
know.
Zack
is
on
converting
the
gke
like
cube
up
into
a
more
of
a
you
know,
essentially
getting
rid
of
all
the
cluster
DK
scripts
and
replacing
them
with
raw
cheek
loud
commands
inside
of
cube
test.
E
A
job
to
generate
the
structured
output
would
you
expect
that
to
be
something
that
is
inherent
to
commands
inside
of
the
job
being
placed
to
demarcate
okay,
we're
moving
on
to
this
next
step?
Or
do
you
would
you
expect
that
to
happen
after
the
fact,
every
child
run
and
then
pick
up
on
output,
markers
or
something
I,
don't
know
we're
going
to
be
forcing
people
into
some
sort
of
job
definition
schema
for
them
to
benefit
from
the
tool.
F
E
Seems
difficult
like
to
get
a
lot
of
buy-in
for
it.
I
think
the
approach
I
was
looking
at
definitely
has
the
downsides,
but
was
mostly
aimed
at
being
an
incremental
improvement
over
some
of
the
way.
The
good
writer
approaches
it
right
now,
where
you
know
the
cubelet
log
is
chosen
to
be
one
of
five
things
and
very
specific
things.
A
Could
you
populate
that
I
can
format
with
more
useful
information
than
what
we're
getting
out
of
paper
nature
today
and
if
it
turns
out
that
that
adds
good
incremental
value
and
we
have
buy-in
on
the
consumption
of
that
new
format,
we
could
maybe
find
a
way
to
shortcut
that
and
have
tests
write
to
that
directly
instead
of
writing
to
chain,
but
like
definitely
in
favor
of
an
incremental
approach
that
is
well
scoped
and
worth
like
yeah.
You
don't
want
to
tackle
two
hours
of
a
problem.
A
E
F
How
about
you
know
if
we
could
partition
the
build
log
into
different
phases
like
I?
Think
that
might
be
a
incremental
thing
is
sort
of
what
you're
talking
it
seemed
like
part
of
your
format
was
talking
about
I'm,
going
to
search
for
these
things
and
then
I'm
going
to
say
that
in
this
build
bug,
you
know
from
dist
exposition
to
this
other
text.
Vision
is
going
to
be
the
error.
I
think
I
might
also
be
interested
in
you
know,
knowing
exactly
when
is
the
cubic
versus
the
gingko
test
versus
the
you
know
build?
F
Where
is
that
in
the
build
log
text?
And
if
there
is
some
way
of
representing
that
that
within
we
could
highlight
in
guru
nader,
we
might
be
able
to
do
some
sort
of
tree
type
structure
because,
like
maybe
I'm,
pretty
confident
that
the
build
worked?
Fine,
so
I
just
want
to
eliminate
all
of
those
logs
or
maybe
I
want
to
expand
that,
because
that's
what
I'm
interested
in
and
I
don't
care
about.
You
know
what
happened
on
cluster
tower
teardown,
so
hide
all
that
formula.
A
Another
another
bit
of
low-hanging
fruit
that
would
maybe
help
with
the
goober
Nader
report.
Today,
not
the
bill
blog,
but
like
the
little
snippets
I
know,
we
pinched
it
a
couple
times
the
waiting
for
conditions
find
out
what
was
the
condition
that
was
waiting
for
we're,
expected
err
not
to
happen.
There's
another
great
one.
Those
are
really
difficult
to
search
for
in
an
easy
way
and
require
a
lot
more
parsing
of
law,
dense
logs
to
figure
out
what
the
actual
failure
was
at
the
actual
line.
Right,
I
think.
E
C
It
be
possible
for
us
to
maybe
a
rally
around
what
it
is:
an
absence
of
what
how
we
get
there,
what
it
is
the
ideal
case
that
we
would
want
to
see
and
then
maybe
from
there
start
to
read
lease
rally
around
the
issue
so
that
way
it
could
get
traction
over
time.
You
know
in
an
ideal
world
where
computers
didn't
matter
and
they
did
everything
we
wanted
to
do.
C
E
If
I'm
understanding
you
correctly
tim
and
eric',
but
it
is
equally
important
to
highlight
snippets
when
we're
confident
too
they
exist,
but
also
to
do
that
inside
of
something
that
gives
us
immediate
context
and
lets
you
move
away
from
that
snippet
without
having
to
look
at
the
whole
bill
at
once
enough
context.
Oh
yeah.
F
E
F
Don't
know
I
don't
have
strong
opinions
or
I.
Don't
know
I
mostly
just
want
the
ability
to
yeah,
you
know,
show
more
I
mean
not
I
mean
as
I
get
right
like
on
github.
If
there
is,
if
I'm
reviewing
a
particular
area
of
code
and
I
want
to
see
further
up
context,
I
can
click.
The
like,
you
know,
show
the
previous
lines
above
that
which
isn't
super
great.
F
You
know
if
there's
a
little
more
structure,
maybe
I
want
to
see
like
the
whole
function
or
something,
as
opposed
to
just
two
like
two
lines
of
code
around
that
so
yeah
no
I
think
it's
not
it's
kind
of
tricky.
Okay,.
A
Yeah,
like
what
has
been
really
useful
to
me
for
some
of
the
proposals
that
come
out
of
people
like
me
are
like
documents
that
include
screenshots
of
the
intended
changes
to
a
user
facing
program
just
to
show
like
what
what
the
ideal
vision
looks
like
from
the
consuming
it
perspective
and
then
next
to
the
actual
machine,
readable
thing,
that's
driving
that
you
can
sort
of
wine
back
from
there.
So
just
like
mock
screenshots
or
something
maybe
help
us
figure
out
what
we
went
ahead.
A
Okay,
that
make
sense
and
I
think
we're
basically
a
time
so
public
service
announcements
seems
like
play,
crate
is
kind
of
increasing,
or
rather
Eric
did
a
good
job
of
defining
the
difference
between
consistency
and
flakiness
right.
So
consistency
is
gain
coming
as
passes
and
fails
the
same
sessions.
Our
consistency
isn't
looking
too
good.
I
was
planning
on
like
I,
don't
know
you
said
that
even
though
the
kubernetes
does-
and
we
can
raise
an
issue
that
the
cube
root
cut
the
community
meeting
honestly
something
to
talk
about.
A
A
So
that's
basically
the
way
that
announcement
is
going
to
go
out
communities
if
people
are
wondering
whether
their
alpha
is
because
it's
we're
like
two
weeks
behind
schedule
on
an
alpha.
It's
because
we
can't
cut
one
because
the
tests
don't
pass
and
then
last
thing
features
fees
today,
not
history
of
has
any.
This
is
the
kubernetes
facing
features,
but
everybody
there
was
originally
supposed
to
be
a
meeting
this
morning
to
talk
about
what
features
we're
going
to
go
in
for
one
eight
I'll
be
happening
right
before
the
community
meeting
on
Thursday.