►
From YouTube: Test/Release Working Group Meeting 5/14/2018
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
C
C
A
A
C
B
A
A
B
Okay,
so
we
have
a
different
meeting
by
the
way
today
to
talk
about,
so
you
don't
need
you've
got
moved
from
seven
o'clock
because
of
the
conflict
to
1
p.m.
so
in
case.
Any
of
you
are
interested
in
discussing
more
about
this.
Please
show
up
to
the
other
meeting,
but
for
now
I
think
here
the
plan
should
be
to
after
cost,
since
PR
goes
out,
so
maybe
we
can
cut
the
release
tomorrow,
in
which
case
I'll
send
an
email
out
to
sto
death.
B
Today
and
let's
plan
on
having
the
community
testing
either
on
Wednesday
or
Thursday
to
any
of
you
have
any
preference,
are
you
waiting
on
any
other
peers
or
anything
else
to
happen,
one
of
those
Wednesday
and
Thursday?
No,
we
can't
kick
it
up
on
Wednesday,
but
I'm,
trying
to
figure
out
if
there
are
any
other
pending
peers
into
concentration.
B
A
A
D
Yeah
so
as
fortunately,
the
SWAT
team
effort,
we
said
you
can
have
a
new
test
framework
and
we
are
actually
in
kind
of
the
design,
slash
like
some
strong
prototyping
stage
right
now
and
are
there
some
early
feedback
on,
especially
on
the
UX
side,
at
this
point
and
how
things
fall
together,
so
that
we
can
incorporate
into
our
work
so
I'm.
Just.
A
D
Okay,
so
the
basic
idea
is
to
come
up
with
the
test
frame
or
if
it
is
thus
all
the
problems
are
outlined
here.
So
with
the
game.
Problems
are
that
we
have
multiple
test
frameworks
and
we
are
exposing
some
of
the
primitives
to
much
three
so
that
when
we
are
writing
tests,
the
daughters
are
need
to
deal
with
things
like
flakiness,
and
you
know,
one
of
things
really
need.
First
MOOC,
we
are
not
simple,
like
most
people
are
just
opting
to
use
the
sky
pipeline.
D
We
want
to
kind
of,
simplify
that,
and
you
know,
are,
like
all
of
us
use
different
types
of
environments.
Luckily,
like
I
use,
macros
and
running
those
tests
locally,
is
it
that
you
want
to
you
know,
attack,
and
one
of
the
things
that
you
want
to
do
is
like
you,
try
to
let
people
right
there
cuz
in
a
higher
level,
so
that
you
can
get
this
logic
code
and
actually
use
it
in
like
against,
like
different
environments.
Try.
D
So
if
you
want
to
support
different
types
of
environments,
we
should
be
able
to
leverage
our
existing
set
of
tests.
So
those
are
the
kind
of
problems
that
we
are
trying
to.
So
with
this
approach
so-
and
you
can
find
some
of
the
details
here-
I'm
sorry
I
didn't
get
a
chance
to
go
over
the
documents
after
the
comments
there's
some
interest
in
commencing
here.
So
this
document
focuses
mainly
on
the
authoring
model
and
you
can
actually
see
the
showcase
code
here
in
it
checked
into
github
right.
D
So
there's
a
bunch
of
different
types
of
showcase
code,
there's
also
the
the
actual
framework
code
that
we
currently
have
in
case.
You
guys
are
interested
if
you
want
kind
of
peek
under
the
hood,
but
the
idea
is
the
tests
framework
should
be
straightforward.
It
should
have
kind
of
give
you
guide
like
guide
rails,
so
that
you
kind
of
do
the
right
thing.
It
should
be
easy
to
use
right.
So
that's
one
of
the
main
things
that
we
are
looking
for,
so
opting
into
the
framework
itself
is
easy.
D
We
just
need
to
capture
test
name
and
with
that
we
tap
into
the
the
go
test
framework
go
test
infrastructure.
This
is
very
similar
to
what
the
current
framework
plus
today
we
are
pretty
much
doing
the
same
thing.
That's
just
that
you
know
it's
not
a
one-liner
and
you
know
we
are
off
to
the
races.
So
some
of
the
things
that
mean
it.
This
is
like
when
we
calculate
the
name.
D
We
can,
for
example,
expose
the
the
our
own
logging
attributes
so
when
I
say
go
test,
I
actually
see
these
attributes
that
you
see
here
these
are
they
be
coming
from
our
our
framework
right?
So
if,
for
example,
if
I
run
mixer
in
line,
I
can
actually
specify
go
test.
Log
output
level
at
attributes,
:
debug
and
can
I
just
see
the
mixer
attributes
coming
in
during
the
debug
output.
D
So
this
is
kind
of
a
grab-bag
list
of
things
that
we
have
attacked
some
UX
issues.
So
the
very
first
one
is
we
actually
integrate
experimented
with
labeling
the
tests.
So
this
is
one
mechanism
that
you
know
we
tried
and
and
is
currently
working
on,
like
test
categorization
problem.
So
this
is
kind
of
a
showcase
of
what
we
can
do
in
this
space
right
and
the
idea
is
there's
we
can
take
the
test
by
specifying
some
labels
like
this.
D
For
example,
two
tags,
the
test
using
the
networking
label-
and
if
you
will
get
here
in
the
screenshot
I,
can
actually
help
the
test
out
by
saying
networking.
First
and
you'll
see
that,
like
the
test
that
had
label
about
right
again,
this
is
just
as
a
way
to
showcase
what
we
can
do
and
like
the
Ohio,
listen
apply
is
strongly
opinionated
thing
at
this
point.
So
if
you,
by
the
way,
feel
free
to
give
feedback
at
any
points-
and
we
can
have
it
special
about
this
thing
right.
D
So
the
other
thing
that
we
did
was
essentially
a
similar
mechanism.
We
can
actually
let
the
test
alter,
specify
the
requirements
on
the
environments.
So
you
can
say
I
have
a
test
methods
and
it
requires
the
fake
apps
and
app
pilots
right.
So
we
expect
the
framework
to
make
sure
that
your
environments
either
complies
with
this,
or
it
can
actually
make
the
environments
work.
D
This
way
you
can
actually
modify
the
environments
so
that
by
the
time
the
testers
executing
you
know
the
environment
is
as
expected
so
right
now,
the
the
design
of
this
is
a
bit
nebulous
in
the
sense
that,
yes,
we
they
have
the
top-level
coding
pieces
for
this.
We
can
actually
specify
dependencies
and
we
can
actually
kind
of
check
for
them
what
the
underlying
engine
we
are
still
working
on
this
right,
but
we
expect
the
experiments
experience
to
look
like
those.
C
D
D
So
this
this
was
a
test
was
essentially
based
just
describing
your
requirements
on
the
environment
and
then
the
actual
altering
of
the
test
starts
with
accessing
a
facade
that
we
have
called
environment
right.
So,
yes,
I
should
say:
I
want
to
get
a
hold
of
the
environments
and
from
there
you
actually
start
writing
your
test
logic,
and
this
is
one
way
to
this-
is
how
you'd
normally
go.
You
actually
say:
ok,
I
want
to
get
the
environments
that
I
actually
want
to
apply.
D
E
D
I
personally
want
the
experience
to
be
as
simple
as
that,
where
you
want
to
kind
of
interject-
and
you
know,
do
these
kinds
of
things,
but
at
the
same
time
you
want
these
tests
to
be
a
bit
hermetic
and
we
may
actually
ought
to
say
look.
We
are
just
going
to
depend
on
a
local
VM
that
we
start
or
local
testing
environments
to
which
we
are
gonna
actually
apply
things,
and
only
to
specify
the
configuration
right.
You
notice
round
pick
they
can
start
and
while
doesn't.
D
Okay,
so
this
is
a
kind
of
a
strawman
mobile
of
so
once
you
compute
the
environments.
You
can
say
this
is
a
full
test.
Why
do
I?
Just
just
to
give
you
some
background,
so
the
idea
is
you're
gonna
have
a
pilots
and
a
mixer,
and
then
we
are
gonna,
do
call
from
one
service
to
the
other
one
and
see
if
you're
gonna
be
able
to
do
a
check.
Fail,
like
you
know,
do
a
check,
get
the
failure
response
or
you
know,
response
and
then
get
something
more
right.
D
So
this
is
kind
of
the
scenario
in
in
this
stroma
right.
So
the
first
thing
we
do
is
get
me
get
a
handle
to
a
fancy
back-end
that
so
this
is
one
of
the
marks.
That's
gonna
happen.
You're
gonna
have
in
the
system
where
you
can
directly
change
the
behavior
of
this
backends
s,
and
you
know
through
that
you
actually
change
the
behavior
of
mixer.
Here
we
are
saying:
okay,
you
know
from
here
on
deny
all
the
check
requests
coming
your
way,
this
part
shows
that
can
all
we
are
going
to
get
two
applications.
D
Fake
applications
handles
to
those
applications
that
are
deployed.
This
is
like
this
is
already
something
that
has
the
pilot
test,
and
then
we
are
gonna
make
a
call
from
one
to
the
other
one
and
at
the
end
we
actually
expect
the
results
to
fail
because
of
the
deny
check,
and
then
we
also
expect
them
report
to
be
done.
You
know
in
in
a
particular
way
right,
so
this
kind
of
showcases
how.
E
D
E
D
So,
just
to
be
clear,
though,
these
are
actually
using
high
level
primitives
the
backend,
the
Pacific
and
is
using
the
and
already
established
interface.
We
have
in
mixer
right.
So
next
is
expect
a
Mesa
adapter
is
expected
to
conform
to
a
particular
API,
and
this
is
going
to
be
implementing
that
if
that
API
choose
this
okay.
D
D
So
we
have
a
few
discussions
that
are
among
us
about
how
to
deal
with
certain
things
like
flakiness,
you
know
I,
think
being
able
to
control
some
of
the
actors.
The
application,
like
the
apps,
the
fake
ask
ourselves,
is
gonna,
be
a
gain
in
making
sure
that
we
can.
You
can
reduce
flakiness
in
the
system.
Also,
we
are
trying
to
expose
high-level
to
the
user
when
writing
their
tests,
so
that
most
of
the
flaky
latest
stuff
should
actually
be
doubts
and
it's
at
the
framework
level.
D
So
this
should
reduce
the
like
one
of
flakiness
issues.
That's
occurring
in
different
tests.
Also,
you
know
I,
don't
know
me,
there's
some
something
you
guys.
What
seen
it
like
on
Friday
I
showed
a
way
to
expose
the
config
state
pilots,
for
example,
through
control
Z,
and
we
can
use
that
mechanism
or
similar
mechanisms
to
actually
listen
to
the
state
changes
on
the
our
components
so
that
you
know
we
can
ensure
that
whatever
configuration
we
passed
is
actually
disseminated
correctly
before
going
forward
with
the
next
steps
of
the
test.
D
So
if
you
in
case
you
missed
that
this
is
like
some
of
the
in
terms
of
pads,
there
is
a
if
you
enable
control
Z
under
components.
It
essentially
exposed
an
H
and
through
that
end
point
we
can
expose
the
the
various
method
information
about
about
the
component
and
in
this
one,
very
exposing
the
pilots
vivo.
What
configuration
looks
like
and
that's
pretty
much
it.
A
Some
questions
about
the
configuration
model,
so
if
you
showed
up
a
little
bit
dollars,
but
more
I
mean
yeah,
so
the
the
environment
are
configured.
So
the
configuration
depends
on
the
test
requirements.
Right
say:
if
you
test
required,
find
out
that
make
sure
you
want
to
configure
both
of
them
or
how
so.
D
It's
a
bit
hard
to
come
up
with
a
model
that
allows
you
to
constraint
the
configuration
based
on
the
requirements
right.
So
that's
why
it
requires
is
more
like
a
way
to
say
to
the
system.
You
know
I'm
gonna,
be
depending
on
these
things
in
the
rest
of
the
code,
so
either
do
a
precondition,
check
and
then
show
the
user
that
look.
You
know
we
need
these
things.
Your
environment
is
not
set
up
correctly,
so
here's
a
way
to
fix
it.
D
So
yeah
we
are
actually
kind
of
letting
the
user
know
how
to
fix
the
error
so
that
they
can
run
test
or
we
can
try
to
update
the
environment
so
that,
okay,
you
know
these
studies
have
these
requirements.
You
know
it's
the
local
test.
For
example,
you
have
like
dependency
on
pilot.
Let
me
go
and
start
the
pilot
instance,
for
example,
right
which
is
kind
of
divorced
from
the
okay.
D
A
A
D
D
We
could
do
that,
so
the
so,
for
example,
here
it
shows
you
can
actually
specify
a
requires
at
the
street
level
yeah
so
most
likely
what's
gonna
happen
is
we
are
going
to
have
one
single
environments
set
up
and
either
we
are
gonna
change.
It's
only
slightly
based
on
each
test
or
like,
and
if
you
have
something
like
this,
it's
gonna
be
set
up
once
and
it
seems
to
just
work.
A
D
A
A
D
You
can
actually
I
actually
have
modeled
this
in
an
earlier
one,
and
you
can
actually
say
something
like
exclusive.
If
you,
you
should
be
able
to
specify
that
look.
I'm
gonna
be
exclusive
access
to
this
and
I,
which
will
be
a
signal
to
the
test
framework
that
says.
Okay,
you
know
once
this
guy
is
done,
I
should
try
to
restore
environment
back
to
its
original
condition.
Very.
A
E
D
D
D
E
C
Don't
think
it's
if
I
mean
I,
think
so
intent
or
if
I
understand
it
correctly,
is
that
you
are
going
to
interrupt
it.
The
current
component,
video
component,
is
server
layer
below
through
your
mixer
and
in
depending
on
how
you
run
the
test.
You
may
end
up
with
a
local
pilot.
You
may
run
with
the
real
pirate
running
in
jke
or
you
may
not
I
mean
it's.
It's
well
run
the
test.
You
decide
against
which
components
you
run
them,
but
all
cores
go
to
you
know
what
are
your
component
yeah.
F
F
B
D
So
this
is
kind
of
like
the
level
of
details.
Kind
of
thing
I
can
get
the
handle
mixer
running,
mixer
I
think
it's
awesome
port
now
I
can
do
some
like
I
can
try
to
add
them
like
a
mob,
but
hear
myself
saying
that
okay,
this
is
like
a
mock
pilot
making.
It
call
to
book
proxy
making
a
phone
mixer,
for
example
right.
So
there
is
there's
ways
to
extend
the
framework
in
in
that
way,
but
but
the
core
of
the
framer
is
about
actually
being
able
to
create
an
environment
using
the
actual
components.
D
So
if
you
can
pull
that
one
off,
if
we
can
actually
say
look,
it
is
very
easy.
It's
dead,
simple,
to
create
an
environment
where
we
are
using
all
the
real
components
and
then
based
on,
and
if
you
write
like
scenario
tests,
then
I
think
that's
a
great
place
to
be
in
and
you
don't
need
to
deal
with
marks.
It's
like
you
are
running
the
all
the
actual
components.
I
would.
C
E
C
Would
see
that
this
can
be
used
for
hundred
percent
of
the
tests,
but
not
all
tests
will
actually
what
I
mean.
30
percent
of
test
may
not
achieve
organon
environments
so
because,
since
that
this
framework
allow
you
to
write
a
test
that
would
run
against
a
pyro
training
wherever
the
only
difference
would
be
that
we
need
to
skip
some
tests.
If
the
tests
depend
on.
Let's
say
behavior
of
JK
would
depend
on
on
on
some
local
behavior.
G
C
A
F
A
C
D
Those
primitives
that
we
can
supply
it
through
the
test,
altering
experience
right,
you
know,
you're
telling
your
chest
is
okay.
This
is
it.
This
is
a
gke
test.
This
requires
G
key
right
and
then,
if
the
framework
level,
then
you
can
come
up
with
strategies.
For
example,
if
we
are
running
locally,
we
can
just
say
look.
This
is
an
error
like
any
try
to
run
into
this,
but
you
didn't
supply
me
gke
configuration
so
I'm,
gonna
or
in
the
CI
case.
C
D
Hey
conditions
right,
so
the
test
framer
can
either
make
those
preconditions
true
if
it
can,
and
it
can
be
done
very,
very
ease
of
success
right.
For
example,
if
you
give
me
a
cue,
config
configuration
and
say
this,
is
my
test
setup?
Do
what
you
will
you
know
he
can
go
and
just
really
moist
your
there,
like
you
know,
just
pave
it
over
right.
So
if
you
give
me
the
permission
to
do
that,
you
know,
for
example,
I
can
go
ahead
and
do
that
in
the
framework,
but
yeah.
D
What
I'm
trying
to
put
point
out
is
like
there's
a
dial
that
is
actually
control
controllable
at
both
the
are
like
implementation
choices
level
right.
You
know
we
haven't
done.
Some
infant
was
in
informatics
in
choices
here
and
also
that
tile
is
the
valuable
at
the
usage
level,
depending
on
which
command
line
flags
that
you
specify.
C
F
Another
question
did
you,
you
know
we're
going
to
use
open
tracing
on
all
the
little
framework
that
can
be
said
about
doing
something
like
this
I'm.
Sorry
that
again
so,
like
do
some
kind
of
tracing,
so
I
think
is
to
support
serpent
racing
right
now,
nothing
like
form,
then
we
keep
on
to
fuel.
It
be
good,
like
you
know,
if
we
modify,
let's
say
booking
for
all
those
component
that
we
use
to
actually
use
tracing
and
then
the
framework
when
you
created
this
also
does
that
then
you
know,
like
you,
run
your
test.
F
E
Yeah
I
think
that's
really
good,
and
in
the
past,
in
fact,
that
we
actually
had
a
couple
of
sto
build.
That
tracing
is
like
totally
broken,
so
we,
it
would
be
nice
to
the
framework,
should
provide
the
easy
way
for
us
to
test
like
Jay,
King
I.
Think
we're
looking
at
having
Jacob
in
default
are
for
distribute
tracing
is
still
and
also
like.
Sap
was
saying
to
actually
drill
down
to
a
particular
microservice
application
to
figure
out.
What's
going
on
so.
F
C
C
C
And
they'll
be
very
useful
for
debugging
and
this
kind
of
stuff,
so
any
problems
it
can
is
so
heavy
that
it
cannot
really
run
on
a
single,
no
Coco
curry,
but
checker
can
use
maybe
light.
So
if
we,
if
we
switch
to
Jagger
and
and
and
permit
use,
started
in
the
local
environment,
we
could
get
towards
this
debugging
and
may
be
integrated
to
that
bug.
Report
tools
that
we
have
now.
F
C
C
F
C
C
I
mean
you:
can
we
start
putting
a
rotation
on
you?
This
I
mean
if
you
see
a
new
test
being
submitted,
can
you
ask
a
it
must
have
this
annotation
you
test?
If
someone
is
ranking
at
least
for
modifying
a
test,
we
can
require
them
to
put
the
respective
validations
and
then
start
using
the
new
framework.
Yeah.
D
I
think
doing
it
at
the
testimo
is
not
going
to
be
easy
because
you
need
to
opt
into
this
framework
and
all
all
the
frameworks
that
we
have
and
I
know
is
kind
of
capturing
the
test
main.
So
there's
gonna
be
a
bit
of
friction
there,
like
you
know,
you
cannot
just
say:
oh
test
requires
and
that
stuck
using.
So
we
need
to
capture
the
pilot.
D
E
C
E
E
D
D
C
Have
an
example:
XDS
test
I
mean
in
piety
to
testing
pilot
run
locally
in
circles
here
against
in
process
pilot,
so
they
start
the
piloting
process
and
they
are
run
just
like
normal
test
amines,
you
don't
and
they
could
use
the
same
framework
and
they
run
from
90s.
They
start
a
pilot
and
start,
and
we
luckily
Monica's
processes.
Aha,.
C
C
F
H
E
H
Worries
I
was
gonna,
say
that
even
when
you're
writing
just
a
plain,
simple
unit
test
that
I
suspect
that
a
lot
of
the
nuts
and
bolts
of
our
framework
with
respect
to
local
testing
in
particular,
might
come
in
handy.
So,
for
example,
we
might
have
a
new
way
to
just
like
if
you
want
a
unit
test
pilot,
for
example,
I
might
have
a
new
clever
way
to
just
spin
up
a
local
pilot
with
defaults
that
are
useful
for
testing
locally.
H
D
You
don't
have
it
scheduled
yet
they're
still
doing
yeah
I
think
at
this
point
so
much
transitioning
from
or
in
the
process
of
transition
from
being
in
the
prototype
mode.
So
actually,
okay,
let's
try
to
get
things
running
in
time
and
I
think
the
main
litmus
test
that
we
are
running
against
is
okay.
Can
we
get
some
tests
like
from
the
existing
set
of
test
Suites
and
then
you
know
make
them
run
like
if
you
can
actually
accomplish
that,
then
I
think
that
would
be
the
right
time
to
more
concretely
about
like
schedules,
anti-union
yeah.