►
From YouTube: Automating Quality Gates in Azure DevOps with Keptn
Description
Azure DevOps is a popular choice for many organizations to automate continuous delivery of enterprise and cloud native applications. In this Keptn webinar we have Bert Van der Heyden, Performance Engineer at RealDolmen and creator of the Azure DevOps Keptn Extensions, walking us through the use case of integrating Keptn's SLI/SLO-based Quality Gates into Azure DevOps pipelines in order to automate the build validation and approval process. Join us live and bring your questions!
Get the extension here: https://marketplace.visualstudio.com/items?itemName=RealdolmenDevOps.keptn-integration
Source code: https://github.com/keptn-sandbox/keptn-azure-devops-extension
Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject
A
A
You
have
been
really,
you
know
fundamental
in
helping
us
getting
captain
integrated
with
Asia
DevOps.
So,
first
of
all,
thanks
for
that,
and
can
you
just
quickly,
you
know
tell
a
couple
of
words
about
you:
what
do
you
do
and
why
you're,
actually
you
know
interested
in
captain
and
why
you
are
helping
us
and
contributing
to
this
open
source
project
so.
B
I
will
pronounce
my
name
first
and
it's
better,
so
my
digital
performance
consultant
at
real
Dolman
located
in
in
Belgium.
So
we
were
part
of
the
GFI
group
and
we
are
a
service
provider
and
we
doing
a
lot
of
things
regarding
performance
and
that's
why
also
in
the
in
the
earlier
days,
I
don't
know
if
people
remember
that
we
used
to
be
a
product
few
years
back,
it's
completely
gone
now.
It
was
called
dynaTrace
at
home
and
it
also
had
something
like.
B
Some
test,
automation,
stuff
and
also
at
that
moment,
and
now
again
we
stay
interested
in
everything
that
can
automate
the
pipeline,
because
that's
the
the
key
one
of
the
one
of
the
most
important
parts
of
our
role
and
and
and
also
it
helps
our
customers
to
really
get
on
that
track
of
that
cloud
native
track.
And
that
means
a
new
software
delivery
way
of
working.
And
there
are
a
lot
of
challenges
and
I.
Think
captain
solves
a
few.
A
Is
a
low
based
quality
gate
approach?
To
actually
do
this
now
to
kind
of
bring
this
little
into
perspective
and
again,
this
is
something
that
we
bear
to.
You
agree
that
this
kind
of
me
to
make
sense
right
about
the
problem
statement
in
in
many
pipelines
whether
this
is
in
our
case
obviously
Asia
there
folks,
but
it
could
be
any
type
of
pipeline
if
you're,
building,
if
you're,
deploying.
If
you're
then
running
your
tests,
a
lot
of
organizations
still
have
may
not
approval
somebody.
It
has
to
look
at
data
now.
A
A
Is
this
build
really
better
than
the
other,
but
just
by
looking
at
metrics
at
graphs,
we
can
also
pull
in
into
your
into
your
DevOps
pipeline
and
the
last
part
and
bear.
This
is
also
what
you
said
earlier.
You've
been
you've
been
investing
in
APM
for
many
years
now
is
in
the
consulting
organization
and
and
an
APM
is
great,
but
it's
it's.
It
sometimes
gives
you
too
much
data
right.
What
do
you
do
with
all
this
data
again
it,
but
it's
from
dynaTrace
or
whether
it's
from
prometheus
or
any
other
monitoring
tool?
A
It's
great,
but
often
it
is
unstructured.
It's
not
linked
with
the
actual
test
and
to
build,
and
therefore
it
takes
even
more
time
to
really
come
to
good
conclusion,
and
this
is
really
the
problem
that
we
want
to
solve.
We
want
to
make
this
more
automated.
Now
the
inspiration
on
how
to
solve
this
problem
came
actually
on
the
one
side
from
Google
says
to
reap
rectus,
so
I
assume
a
lot
of
you
editing
these
out.
A
There
are
familiar
with
it,
but
let
me
just
recap:
for
those
that
don't
know
we
will
talk
a
lot
today
about
service
level
indicators.
These
are
SL
eyes.
Basically,
what
it
is.
It
is
a
measurable
metric,
like
the
error
rate
offered,
say
the
login
request
in
case
we're
execute
test
against
login.
Then
we
are
having
a
solo,
so
our
objectives.
Our
objective
is,
for
instance,
that
a
particular
s,
Li,
has
a
certain
value
over
a
certain
period
of
time,
typically
used
in
production
monitoring,
but
I
can
say.
A
Log
and
error
rate
must
be
less
than
2%
over
30
period
of
time,
a
period
of
year
period,
30
day
period,
and
then
we
have
SL
a's
and
I
think
the
difference
between
a
solos
and
SFA's
are
desolated.
Typically
they
and
also
binding
to
some
business
objectives.
That
means
hey
if
logins
are
not
reliable
and
fast
enough.
That
means,
for
instance,
if
my
login
is
not
meeting
up
to
my
expectations,
99
percent
out
of
a
30-day
window,
then
we
have
a
problem
we
may
get
penalized,
we
may
have
customers
drop
off.
These
are
restless.
A
There's
a
great
video
out
there
on
on
the
Google
cloud,
YouTube
video
channel,
so
please
check
it
out.
It's
called
slic
solos.
It's
a
lays!
Oh
my.
It's
really
great
the
top
again
I
borrowed
this
from
one
of
their.
The
XSL
eyes
drive
s
ellos
which
informs
SaaS,
but
here's
the
point.
This
is
a
practice
that
has
been
heavily
used
and
is
used
in
production
because
we
always
talk
about
a
certain
period
in
time
in
production.
Now
this
is
where
the
second
inspiration
comes
from.
How
can
we
use
this
in
pre
prod?
A
And
this
inspiration
comes
from
our
Thomas
dan
barber
and
his
team
we're,
but
they've
done
a
dynaTrace
for
the
last
couple
of
years,
every
time
they
deploy
and
you
build
into
the
continuous
performance
environment,
they
do
something
very
similar
what
they
actually
do.
They
figure
out
if
this
build
better
than
the
other
and
the
way
to
do
it,
they
are
pulling
out
individual
metrics
s
lies
so
they're,
looking
at
different
things
that
they
know
are
important
memory,
consumption,
throughput,
IO,
whatever
it
is,
they
have
objectives
for
every
SLI.
A
An
objective
could
either
be
a
hard-coded
SLI
where
they
say
it
should
not
consume
more
than
five
gigabyte
on
the
heap
after
we
run
these
tests
or
it
could
be
something
it
should
not
be
worse
than
the
last
release
build
and
then
they're.
Looking
at
all
the
values
across
and
basically
say,
based
on
all
the
SLO
is
across
all
their
slice.
A
We
consider
this
a
good
or
a
bad
bill
and
they
call
this
the
performance
signature,
and
this
is
what
has
can
inspire
us
to
build
the
captain
quality
gate
capability,
which
also
uses
an
SLI
sudol
based
evaluation
approach
and
an
opieop.
We
will
see
this
later
from
you.
That's
why
I
keep
it
short,
but
in
the
in
order
to
use
captain
quality
gates,
you
need
to
define
a
slice.
What
metrics?
A
What
indicators
are
important
for
you
to
give
it
a
logical
name
in
the
query,
and
the
query
depends
on
the
monitoring
tool
or
whatever
tool
that
delivers.
The
value
I
think
today
we're
going
to
see
dynaTrace.
So
it's
a
dynaTrace
metric,
query
indicators.
Then
you
specify
your
SLOS.
What
are
your
objectives?
You
can
define
different
thresholds
for
pairs
and
warning
criterias
for
every
SLI.
You
can
also
specify
different
weights.
You
can
also
make
one
sli
key
criteria
and
then
you
specify
an
overall
score
now.
What
happens
is
when
you
call
captain?
A
Please
start
my
evaluation
for
a
particular
time
period
in
all
the
cases
will
be
the
time
period
of
the
test.
Of
my
let's
say
my
service,
that
was
on
the
test
of
my
test
environment
and
here's
my
saliva
SLO,
then
captain
automates
that
process
of
pulling
out
all
these
indicators.
The
values
from
the
underlying
tool,
compares
them
against
race,
low
scores,
every
metric
and
then
comes
up
with
an
overall
score.
A
So
it
could
be
you
achieve
7
out
of
8,
or
you
just
achieved
five
out
of
eight
and
therefore
that's
the
total
score,
and
then
the
total
score
depends,
whether
you
made
it
or
whether
you
didn't
make
it
a
more
thorough
example
is
again,
because
we
want
to
see
how
this
works
also
cross
builds,
if
you
have
different
SL
eyes
like
in
this
case
response
time
fatal
rate
response
time
of
my
login
transaction
number
of
database
calls
of
my
login
transaction.
This
is
all
data
that
comes
out
from
let's
say,
monitor,
model
monitoring
tools.
A
Then
you
can
specify
us
to
those
on
top
of
it
where
the
static
or
relative
thresholds-
and
you
specify
an
overall
score
of
the
whole
evaluation,
then
you
can
call
captain
with
every
bill
that
say:
we'd
build
one.
It
just
deployed
it.
You
ran
a
test
for
a
couple
of
minutes.
Captain
can
do
the
evaluation
for
you
and
returns
a
score,
and
a
hundred
percent
looks
pretty
good
to
me
that
why
we
can
then
let
the
build
pipeline
go
through.
If
build
number
two
comes
along
and
something
kind
of
falls
over,
you
can
always
decide.
A
Is
this
still
good
enough?
It's
in
the
warning
range
yeah.
Maybe
we
still
let
it
pass
through,
or
we
inform
somebody
that
something
is
not
right
and
then
do
a
manual
promotion
build
number
three.
In
this
case
it
seems
that
the
response,
time
and
failure
rate
there
was
the
problem
and
build
number
two
is
fix
the
screen
again,
but
now
they
introduce
the
regression
in
the
login
transaction.
A
The
number
of
database
calls
jump
from
three
to
six,
which
is
not
allowed
and
the
response
time
jumped
from
90
to
120
milliseconds
violating
our
SLO
and
therefore
we
getting
penalized
and
therefore
we
only
get
sixty
point
six
two
point:
five
percent
bill
for
comes
along.
Everything
is
fixed
and
the
idea
is
if
we
can
automatically
integrate
this
in
every
build
and
we
get
automated
fast
feedback.
This
will
have
a
change
in
how
developers
will
actually
work,
because
you
get
fast
feedback
and
you
can
immediately
see
what
impact
you
have
any
confess
to
fix
things.
A
This
is
how
it
looks
like
in
PowerPoint.
This
is
how
it
looks
like
in
the
captain's
bridge
and
opiate
is
going
to
show
us
more
life
than
we
anyway
with
the
heat
map
and
a
chart
visualization.
But
the
point
is:
if
we,
if
we
take
captain
our
approach,
then
we
can
level
up
our
pipelines
by
integrating
these
quality
gates
the
first
thing
and
bear
this
is
what
we
have
to
solve.
A
Anyway,
when
you
run
a
test,
we
have
this
concept
of
tagging,
so
we
need
to
tell
the
monitoring
tool,
who
is
picking
up
metrics,
that
these
metrics
come
in
from
a
particular
test.
We
call
this
taking
and
then
it's
about
instead
of
having
by
to
somebody
mainly
look
there,
you
say:
captain
here's,
here's,
my
solo,
you
go
off,
you
do
your
evaluation
and
then
you
report
it
back,
and
then
this
should
be
much
faster.
A
B
A
Maybe
one
last
thing
for
people
that
was
joined.
If
you
have
questions,
please
put
them
into
the
Q&A
feature.
I
see
those
questions
coming
in
like
Vickers
was
asking
the
build
comparison
chart
that
displays
builds
and
s
ellos.
This
is
part
of
the
captain
bridge
correct.
It's
part
of
the
captain
bridge
and
the
nice
thing
is.
You
can
also
displayed
anywhere
because
they
kept
their.
The
data
underneath
can
be
also
retrieved
through
a
rest
call.
So
you
can
also
visualize
that
the
response
anywhere
but
a
reminder,
please
ask
questions.
B
B
So
at
this
moment
it's
not
included
in
in
in
the
complete
pipeline
from
going
to
from
one
to
another,
but
it's
it's
scheduled,
so
they
run
they
run
daily
at
noon
and
apparently
they
will
start
running
now
because
I'm
going
to
create
a
release.
So
that's
in
the
near
future.
We
can.
We
can
look
what
happens
so.
B
Let
me
jump
into
the
the
pipeline
itself
or
what
we've
built
so
later
on.
We
will
do
the
captain
part,
but
actually,
let's
now
see
in
practice
what
we
have
in
the
pipeline
itself
so
like
any
set,
we
have.
The
first
thing
was
the
tagging
that's
done
via
our
jmeter
scripts
themselves,
so
we
actually
use
blaze
meters
to
run
our
jmeter
scripts
here.
B
So
that's
an
important
part.
All
this
before
is
to
prepare
some
some
data,
some
some
test
data,
so
we
always
execute
the
same
scenarios
but
on
new
data.
So
we
first
freshly
generate
data.
Then
we
run
that
we
launch
the
test
and,
in
the
end,
actually
wait
for
the
results
now,
and
we
look
at
this,
you
might
see
that
the
first
thing
we
actually
do
is
prepare
the
captain
environment
and,
as
you've
seen
it's
a
it's
a
nice
little
new
task
with
an
icon
and
that's
actually
the
thing
that
that
I've
built.
B
Think
captain
and
you
can
start
using
it,
so
this
is
actually
the
integration
which
I've
built
and
which
will
continue
to
evolve,
but
that's
actually
it
small
but
useful,
so
going
back
to
the
pipeline.
So
this
was
task
number
one
being
prepare
the
captain
environments
where
we
actually
define
our
endpoint.
It's
just
something
that
we
collect,
because
it's
an
endpoint
which
you
can
manage.
Where
you
just
say
this
is
my
my
url
a
my
captain
token
and
that's
about
it.
B
B
A
That's
pretty
cool
so
to
connect
directly
to
recap.
So
what
you
have
done
here,
the
only
thing
you
really
need
is
you
need
to
connect
your
Asia
DevOps
or
you
need
to
provide
the
config,
the
endpoint
configuration
of
captain
and
then
what
you're
doing
through
the
API
calls.
You
say:
captain
I
want
to
make
sure
that
this
project
exists
with
this
stage
and
for
this
service,
and
then
you
created
in
case
it's
not
existing
it.
So
you
completely
automatic
configuration.
A
B
I've
made
it
kind
of
I
used
three
tasks.
I
could
do
it
in
one
made
three
tasks
because
I
see
a
lot
of
potential
with
in
the
future,
also
because
captain
is
even
based,
of
course,
and
you
need
to
send
an
event
to
captain
so
once
blaze
me
that
has
done
its
job.
The
next
thing
is,
by
the
way,
I
also
push
an
annotation
to
dynaTrace,
which
we
will
talk
about
later.
I
also
send
afterwards
and
if
an
evaluation
start
evaluation
event
to
do
captain,
so
these
things
are
not
yet
implemented.
B
That's
on
the
planning
of
the
near
future.
So
what
do
you
need
to
pass
with
your
evaluation
event?
Your
start
time
and
your
end
time
in
a
specific
format,
and
these
are
actually
filled
in
using
via
the
script.
So
this
script
already
gives
me
the
start
time
and
the
end
time
of
blaze
meters,
and
it's
also
not
the
start
time
of
the
pipeline
itself.
B
It's
really
the
one
that
blaze
me
that
says
I
started
then,
because
sometimes
it
can
take
awhile
before
blaze,
meter
scheduled
actually
it's
X
load
test
and
then
in
the
end,
you
actually
have
this
step
being.
Okay,
wait
for
the
results.
This
actually
waits
until
the
evaluation
done
event
comes
in
and
it
gives
me
there's
not
a
lot
of
configuration
here,
but
it
gives
me
a
pass
or
a
fail.
It
will
put
the
the
pipeline
in
in
orange
or
in
orange,
green
or
blue
I.
Don't
know
what
the
colors
are.
B
So
when
we
take
a
look
at
pipeline
still
running
place,
meter
takes
about
five
six
six
minutes,
then
it
will.
It
will
go
further.
Maybe
I
can
already
show
I,
don't
know
if
this
one
gives
me
a
lot
of
luck,
except
that,
yes,
it's
using
these
endpoints
and
this
project
already
exists
and
yes,
I've
added
these
these
files,
so
maybe
I
can
jump
into
the.
A
Quick
question
a
bit
because
this
just
came
in
in
case
the
project
does
not
exist.
You
basically
create
a
new
captain
project
with
a
simple
shipyard
that
only
contains
a
single
stage
remain
use
case.
Here
is
a
quality
gate,
so
we
basically
say:
captain
I
need
this
project.
It
has
a
single
stage
that
is
called
whatever
it's
called
and
then
you're,
adding
a
service
and
the
service.
I
think
this
is
also
something
I
learned
over
the
last
couple
of
weeks
doesn't
necessarily
have
to
be
just
one
micro
service.
A
B
Yeah
I
mean
this
projects.
We
started
with
this
in
2010
and
that
says
a
lot
because
it's
a
model,
it's
we're
on
the
on
the
track
of
breaking
it
down,
but
you
need
to
have
a
business
case
of
breaking
down
a
monolith.
So
at
this
moment
this
can
be.
Project
is
still
far
from
your
your
typical
kubernetes
running
cloud
native
application,
but
we
use
the
quality
gates
and
we
do
bi-weekly
deploys.
So
that's
all
good.
B
So,
when
I
go
to
my
repository
to
actually
show
what
is
in
our
our
files,
so
basically
you
see
that
that
I've
got
to
test
stages,
actually
they're,
both
the
same
test.
Script
running
on
two
environments,
where
one
environment
we
have
some
more
early
early
adopter
features
and
on
the
other,
it's
it's
more,
the
one
that
goes
immediately
to
production,
so
the
scripts
are
the
same,
but
the
functionality
of
the
of
the
application
itself
is
a
bit
it's
a
bit
different.
B
So
that
means
that
we
have
defined
the
same
as
a
lie.
So
this
is
a
live
file.
Is
is
used
for
for
both
these
these
projects,
I
made
them
to
projects,
and
you
see
that
this
is
not
the
American
in
large
a
bit.
This
is
not
your
typical
as
a
live
file
which
you
find
in
the
examples.
The
first
five
are,
and
then
you
have
the
second,
the
second
one
and
then
this
one,
because
our
how
we
have
a
monolithic
application.
B
So
it's
the
entire
thing
and
therefore
I,
don't
I'm
not
only
interested
in
the
response
time
and
the
true
put
in
the
error
rate
of
my
front
end
I'm,
also
interested
in
what
does
my
back
end
do
and
because
there
gave
me
some
chattiness
between
those
things
and
the
response
times
here
also
are
a
bit
different
than
minimums
there,
and
also
the
database
so
I'm
also
interested
in
especially
the
the
throughput
of
the
of
the
database.
Specifically.
B
A
A
badge
is
really
cool,
because
I
was
just
ask
another
question
similar
to
this,
so
that
means
you
can
it's
a
perfect
example
here
of
combining
metrics
across
an
architecture,
and
the
way
you
have
gone
about
is
you're.
You
are
putting
obviously,
that
the
captain
stage
tag
on
it
in
the
INA
face,
so
you
can
reuse
that
and
also
the
project.
But
then
the
the
third
tag
is
a
tag
that
you
put
on
your
web
node
on
your
R&I,
node,
your
so
no
they're
actually
and
yeah,
and
then
your
database.
B
Yeah
yeah,
these
tags
were
already
there,
then
so
so
we're
using
dynaTrace
longer
than
we
use
captain.
So
these
tags
were
already
there,
so
it
was
a
natural
choice.
For
me,
I
think
I
have
even
included
now
the
the
captain
service,
but
that's
all
going
to
be
the
same
for
for
all
of
these.
Of
these
of
these
things,
like
you
said
this,
the
service
is
just
to
contain
it
in
in
captain,
if
you've
all
of
the
gates
so
but
or
as
or
as
allows,
are
a
bit
different.
B
B
Yeah
second,
one
is
is
going
along,
so
this
one
Wren
and
it's
good
because
it
failed.
That's
part
of
the
part
of
the
show.
So,
first
of
all,
my
blaze
meters
test
also
failed
because
I
have
three
real
failures.
So
that's
that's
one
thing,
and
then
this
results
and
give
me
a
captain
context,
and
my
score
is
only
60.
Shame
on
me
and
we
have
a
failure
there.
This.
A
Also
may
answers
the
question
read
earlier.
I
know
you're
going
to
show
us
the
bridge
soon,
but
that
means
the
data.
The
individual
results
also
come
back
to
you
Julieta
devops,
so
you
could
potentially
visualize
whatever
you
want
that
and
you
decided
to
send
write
it
out
as
a
log
file
on
here.
Yeah.
B
B
So
look
at
captain.
Yes,
we
can
look
at
captain
to
actually
see
what
it
did
there.
Let
me
first
show
dynaTrace,
because
I
also
created
dashboard
here,
but
I
can
follow
up
the
difference
that
the
load
test
of
the
AC
to
environment
and
the
ACC
environment
and
actually
have
here's
my
response
time
by
jmeter
steps.
So
that's
that
first
part
the
the
tagging
time
part.
B
So
this
is
actually
our
Dream
Eater
script
and
we've
added
or
we
pass
along
some
some
headers
as
well.
So
your
execution
path,
so
this
actually,
this
integration
for
that
for
the
Heather's
actually
make
sure
that
we
have
this
distinct
stack.
So
so
we
know
that
it
was
executed
by
by
jmeter
actually,
but
we
have
a
lot
more
things
which
are
actually
added
by
place.
Meter
so
blaze
me
that
still
has
an
old,
app
Mona
integration
feature
and
it
adds
actually
also
Heather's
to
the
steps
so
and
then
by
configuring
the
the
request
attributes.
B
We
actually
know
it.
This
was
coming
from
a
test
and
it
was
coming
from
this
desktop.
So
all
the
things
you
see
here
I
know:
there's
not
a
lot
of
activity.
Other
addict
activity
going
on
at
this
hour,
but
all
the
other
things
you
see
here
and
here
those
things
are
actually
the
results
from
from
the
test
itself.
So
it
doesn't
take
care
of
any
other
requests
which
were
which
was
going
on
at
that
time
by
a
user
or
something.
B
B
Annotations
so
blaze
meter,
this
actually
were
and
one
from
from
this
from
this
noon,
and
one
here
so
this
one
was
just
was
just
pushed
so
this
one
came
from
the
pipeline
itself,
so
that's
directly
done
not
fire
Jamie
and
not
fire
captain,
but
directly
from
chromium
in
the
pipeline.
I
have
a
nice
link
to
the
results
of
of
what
blaze
meter
reports
us
and
normally
there
should
also
be
voila
the
quality
gates.
This
one
was
actually
sent
by
captain
and
nice
and
we
have
even
a
CI
backlink,
so
I
can
jump
back
to
okay.
B
A
So
great
just
to
live.
So
that
means
your
your
your
asier
dev
ops
to
Captain
integration
is
also
pushing
that
quality
gate
result
to
dynaTrace,
including
a
lot
of
additional
metadata
like
the
link
back
to
H
DevOps
yep.
Also,
the
link
to
to
the
bridge,
which
is
at
one
of
the
question
that
would
ask
you,
can
actually
generate
all
of
these
direct
links
to
an
individual
result
into
the
captain's
bridge
because
we
have
smart
links
and,
and
and
you
can
this.
A
B
A
A
there's
a
question:
let
me
just
quickly
cocaine
up
to
me
a
favor.
Can
you
go
back
to
the
bridge
again
yeah?
So
because
the
question
came
in,
how
is
the
waiting?
How
does
this
work
I'll
stay?
How
is
the
SLO
calculated
and
by
default
right?
If
you
look
at
the
heat
map
here,
you
have
one
two,
three,
four,
five,
six,
seven,
eight
nine.
How
many
do
you
have?
A
It
always
grows
a
cross,
okay,
basically
base.
We
look
at
those
SL
eyes
where
you
actually
have
a
criteria.
You
can
see
some
of
those
web
throughput
and
the
RMI
error
rate
they
are
showing
up
in
as
white.
This
is
no
criteria.
You
just
get
the
raw
values,
as
you
said
earlier,
you
may
not.
You
may
not
know
the
criteria,
yet
you
may
just
don't
learn
and
I.
A
Think
that's
why
you
can
get
the
raw
data
and
if
you
click
on
chart
on
the
top,
if
we
switch
over
to
the
chart
yep
all
right,
then
we
can
obviously
look
at
some
of
these
metrics
over
time.
But
if
you
click
on
them
in
there
in
the
legend
DB
error
rates
they
were
I
think
is
one
of
those
and
then
maybe
you
maybe
some
address
which
you
can
just
yeah,
unclicks
unclick
score
and
then
click
a
couple
of
those
yeah
exactly.
A
That's
great,
so
we
can
they,
you
can
actually
get
the
historical
data.
So
that's
one
thing
now,
if
you
do
me
a
favor
click
back
to
the
heat
map,
because
the
question
was
how
how
is
the
weighting
and
how
it
is
calculated.
So
in
this
case,
if
you
look
at
the
last
build
the
one
on
it
on
the
very
right,
we
have
one
two,
three,
four
five
s
lies
that
are
producing
a
value.
Four
of
them
are
orange.
One
of
them
is
green.
A
That
means
by
default,
every
metric
has
the
same
weight
of
one,
so
everyone
is
contributing
the
same
amount
of
points
to
the
total
that
whole
score.
The
total
score
is
100
so
and
if
we
have
one
two
three
four
five,
that
means
every
SFI
can
potentially
contribute
either
twenty
points.
If
it's
green
ten
points,
if
it's
orange
or
zero
points,
if
it
is,
if
it
is
red
and
the
question
was,
how
can
you
change
the
weight
and
you
can
change
the
weight?
If
you
do
me
a
favor
and
go
back
to
your
SLI
tokima.
A
A
But
you
can,
rather
maybe
you
don't
have
to
save
it,
but
if
you
do
me
a
favor
go
into
one
of
these,
and
just
it
maybe
after
ya,
error
rate
just
add
a
new
line
and
line
number
9
number
10
just
add
a
new
line,
and
then
you
can
say,
wait
and
go
on.
Let's
say
2,
then
this
is
this
gets
twice
as
much
weight
as
TRS,
because
by
default
every
one
is
1
and
I.
A
Think
there's
also
another
key
element
here
that
people
are
not
aware
of
I
think
that's
the
key
sli
so
instead
of
weight,
you
can
also
say
the
web
error
rate
is
very
important
to
me
and
if
that
fails,
the
whole
build
should
fail,
and
then
you
can
add
a
new
line.
You
can
say
I
think
it's
called
Kadesh
SLI
and
then
true
or
false,
and
we
have
to
look
at
this
pack
exactly
I.
Think
it's
key
sli.
A
B
B
So
yeah
very
valuable
for
us.
Let
me
jump
back
to
the
or
forward
to
the
slides.
So
this
is
just
a
recap:
the
place
where
you
can.
We
can
find
it
and
download
it
the
current
version,
so
we
have
our
three
tasks
which
actually
handle
all
the
communication
between
agile,
devops
and
captain,
and
then
the
captain
system.
That's
what
it
does.
B
So
indeed,
you
see
that
it's
at
a
wait
for
evaluation.
At
this
moment
in
agile
devops,
you
can
say
fail
or
fail
with
warnings.
I
just
say
you
fail
with
warnings,
always
so,
even
if
it's,
if
it's
a
failed
or
or
a
warning
in
captain,
but
that's
something
which
is
plans
to
change
and
to
actually
make
it
configurable
so
that
you
can
decide
what
what
do
I
want
to
do
with
it
and
also
maybe
get
some
more
more
details
of
the
result.
B
And
then
the
result
is
indeed
the
heat
map
in
in
captain
and
the
fact
that
you
don't
have
to
look
for
it.
I,
just
added
here
also
the
for
the
people
who
are
interested
the
llamo
configurations,
but
it's
just
the
same
so
here
like
all
it
already
set
in
the
prepare.
We
actually
prepare
and
just
call
the
api's
and
what
it
is
it
does
it
results
in
the
result
in
captain
in
the
configuration
of
captain
it
just
pushes
the
eds
Ally
and
the
and
the
other
load
to
the
to
the
right
folder.
B
While
we
launch
the
tests
actually,
indeed
the
mark
evaluation
and
then
and
then
trigger
the
test,
and
they
also
said
that
each
SLI
will
be
transformed
and
polled.
I
think
I've
shown
you
and
the
the
tags
which
are
actually
transformed
to
whatever
service
that
you
actually
or
service
projects
and
so
on
stage,
so
it
actually
gets
from
the
SLI
provider.
B
The
the
metrics
then
the
the
magic
which
is
not
that
magical,
in
fact
because-
and
it
just
explained
how
it
all
works
with
the
weight
and
all
that-
and
the
only
thing
we
need
to
do
is
just
wait
for
the
evaluation
done
a
lot
faster
than
manually,
even
with
a
great
thing
like,
like
the
any
trace.
You
don't
want
to
look
at
dashboards
to
manually
all
do
that.
B
Yeah,
okay
and
in
the
end
of
course,
the
captain
itself
also
sends
the
results.
If
you
have
the
dynaTrace
integration
back
to
dynaTrace,
to
your
components
which
have
those
tags,
so
that's
all
nicely
integrated,
I
think
I.
Okay,
so
you
can
find
the
code
in
key
table.
You
can
find
the
deployable
artifacts
in
in
on
a
market
place
free
to
use
free
to
contribute
just
to
make
it
very
foolproof,
install
it
configure
it
and
you're
done
well.
B
What's
next,
so,
actually
the
I'm
not
that
familiar
with
it,
yet
we're
going
to
work
on
it
anyway.
The
idea
is
to
make
to
make
that
final
step,
that
waiting
for
make
it
and
a
gate
in
DevOps
release
gate.
So
you
have
something
like
line
gates
to
actually
go
from
one
step
to
another.
The
nice
thing
is,
you
can
put
your
your
own
delay
in
there.
B
Load
tests
and
it
slow
test
date
they
take
some
time
and
when
you
put
them
in
a
pipeline,
your
pipeline
is
actively
waiting
for
it,
so
you're
you're
actually
consuming
resources
of
your
pipeline.
Well,
actually,
the
or
locking
resources
from
your
pipeline.
While
in
our
case
it's
it's,
it's
done
in
the
cloud.
It's
it's
done
by
blase
meter,
so
it's
just
waiting
and
hanging
around.
B
So
that's
one
of
the
next
things
that
I
want
to
focus
on
is
is
actually
this
thing
is
already
there
the
place
me
the
service
and
the,
and
they
already
call
it
made
it
for
me,
but
it's
still
it's
still
on
it's
still
empty.
So
the
idea
is
when
that
blazed
me.
That
is
also
triggered
from
within
captain
itself,
so
thence
from
Azure
Devils.
B
A
A
B
B
A
A
Answer
the
question
in
general:
right
captain
is
an
event-driven
control
plane,
which
means
what
captain
really
does
when
it
needs
to
execute
a
test.
It
sends
an
event,
and
in
your
case
you
are
going
to
build
the
place
meter
service
that
is
listening
to
that
event
from
captain
east
and
executing
blaze
meter
with.
B
Clearly,
it
will
be
I
mean
I
didn't
actually
I
already
did
it.
I
already
did
that
integration
to
to
place
meters
in
that
in
that
powershell
script,
which
I
showed
and
it's
just
sending
off
a
actually
it
I
just
take
the
jmeter
script
and
put
it
somewhere.
So
it's
actually
calling
some
api's
from
place
meter
and
that's
it
yeah.
A
A
Now
the
good
news
is
to
answer
the
question
of
what
loadrunner
it
means
anybody
can
write
just
a
drone
or
service.
They
can
then
trigger
loadrunner.
We
had
new
loads,
they
already
built
their
integration
for
new
load.
Now,
there's
also
a
generic
executor
service,
where
you
can
specify
either
a
shell
script
or
an
HTTP
request,
and
then
you
can
already
trigger
any
type
of
shell
script.
You
have
I
think
it's
its
best
script
or
any
type
of
HTTP
request.
A
So
if
I'm
not
that
familiar
with
the
automation
API
of
loadrunner,
but
if
you
have
a
way
to
execute
lone
runner
remotely
from
a
shell
script,
an
HTTP
call,
then
you
can
already
risk
through
the
a
generic
execution
plug-in
that
we
have
yeah,
that's
also
good.
The
other
thing
is
that
mere
it
is
out
loud.
It
really
is
full
automated,
because
I
see
that
it
runs
processes
manually
with
an
orchestrated
like
bamboo
or
Jenkins
or
another
edge,
a
service
be
missing
Rodrigo.
B
A
A
And
so
for
those
of
you
that
are
online
still,
it
would
be
great
if
we
can
get
your
support.
You
know
staris
and
get
up.
You
know
shout
shout
out
for
us
on
social
media,
join
us
like
channel
and
actually
there's
an
easier
way
of
join
this
like
channel.
It's
like
thought,
captain
or
SH,
and
also
try
out
the
tutorials,
provide
feedback
feel
free
to
contribute,
and
we
need
more
people
like
beard
who
contributes
but
really
give
us
feedback.
That
would
be
great
and
I
would
say
yeah.
Can
you
do
me
exactly
this
one
here?
A
So
if
you
want
to
reach
out
to
us,
then
you
know
you
have
our
names
and
there's
definitely
more
coming
from
us
as
more
of
these
webinars
there's.
Also,
the
kept
developer
focused
community
webinars
that
are
always
a
meetings,
they're,
always
every
Thursday.
So
in
case
you
want
to
get
in
touch
with
the
development
team.
You
want
to
see
what
they
are
doing.
You
wanna
influence
the
backlog.
You
want.
Our
contributes.
You
want
to
take
issues
on
then
Thursday
is
the
best
one
the
best
day
for
you,
the
afternoon
cool,
Beth.