►
Description
Building and then implementing a DevOps pipeline is only the beginning of your journey. Understanding your pipeline analytics is just as important and can be a key factor to your overall success and adoption. In this meetup, Chris Riley, Splunk DevOps Advocate, will help us understand pipeline analytics including:
· How pipeline analytics are a necessary practice in all high-performing DevOps environments
· How to select the right metrics to measure DevOps success
· And how to implement build a Pipeline Analytics strategy
A
Okay,
thank
you,
everybody
for
attending
the
cdf
online
meetup.
I
am
tracy
reagan
your
host
for
this.
For
today's
event,
before
we
get
started,
I
just
want
to
remind
everybody
that
our
next
meetup
is
we're
going
to
have.
Let
me
make
sure
I
know
we're
doing
using
the
jenkins
templating
engine
for
pipeline
modeling,
steve
tarana,
who
wrote
it
will
be
presenting
and
it
will
be
held
on,
may
12th
at
11
a.m,
pacific
time
and
2
p.m.
A
So
today,
chris
riley
is
going
to
talk
about
measuring
devops
success
with
pipeline
analytics
and,
if
you
don't
know
who
chris
is,
you
may
have
heard
him
before
he
does
run
a
podcast
called
sweetcode.io.
I
highly
recommend
you
go
out
there
and
check
out
some
of
the
the
podcasts
he's
done,
but
he's
also
a
developer
advocate.
He
does
developer
relationships
or
relations
for
splunk
and
he
has
sort
of
been
a
dev
up
kind
of
devops,
enthusiast
and
developer
relations
builder
over
his
career.
A
So
he's
also
works
on
devsecopdays
he's
an
organizer
for
devsecopday,
so
you
should
know
chris.
He
is
around
and
I'm
super
super
happy
to
have
you
today,
chris,
and
on
that
I
am
going
to
bow
out.
Let
you
introduce
yourself
in
a
way
you
feel
is
appropriate
and
go
ahead
and
start
telling
us
about
pipeline
analytics.
B
Thanks
tracy
yeah,
I
actually
like
how
you
introduced,
maybe
because
I
think
sometimes,
when
you
read
my
bio,
that
I
wrote
it
doesn't
come
off,
as
is
personal,
but
hopefully
I
live
up
to
to
what
you
mentioned
and,
as
a
matter
of
fact,
the
podcast
that
you
mentioned.
There's
a
really
fun
episode
with
you
and
I
on
it
and
as
a
result
of
that
conversation,
it
was
just
a
just
two
second
spot
on
pipeline
analytics.
B
B
B
Perfect
so,
like
tracy
said,
my
name
is
chris
riley.
If,
at
the
end
of
this
webinar
you
you
I've,
I've
earned
your
respect.
Please
reach
out
to
me.
You
can
scan
that
qr
code.
It
gets
all
my
social
information,
the
podcasts
and
I
love
just
engaging
with
the
community.
That's
my
job.
My
job
as
an
advocate
is
to
make
sure
I
understand
the
reality
of
how
companies
and
practitioners
are
embracing,
devops,
sre
and
modern
development
practices.
B
As
a
result
of
doing
that,
I
have
picked
up
a
really
strong
interest
in
pipeline
analytics
and
we
definitely
on
the
nerd
scale,
where
we're
at
10
when
it
comes
to
pipeline
analytics,
as
well
as
all
the
other
things
in
devops.
But
the
reason
I
really
honed
into
pipeline
analytics
is
I'm
kind
of
blown
away.
This
isn't
a
standard
practice,
yet
there
isn't
a
gardner
quadrant
for
it
and
it's
something
that
we
see
a
lot
of
organizations
do
and
consider,
but
it's
kind
of
an
after
after
thought.
B
B
B
Is
it
securable
and
do
we
have
something
measurable,
metrics
kpis,
that
we
can
learn
from
and
complete
the
loop
and
improve
our
environment
and,
as
I
said
earlier,
I've
been
kind
of
surprised
that
organizations
don't
embrace
this
sooner,
but
I
understand
why
we
are
obsessed
with
deploying
applications.
We
are
obsessed
with
deployments.
We
are
obsessed
with
what
happens
in
production
and
we
cater
to
that.
We
cater
to
delivering
value,
because
that
is
what
devops
is
about
mean
time
to
value
which
is
one
of
the
metrics
and
as
a
result,
we
often
don't
think
about.
B
B
So
I've
changed
my
own
mindset.
The
way,
I
think
about
visibility
now,
in
application
development
environments
is
both
in
depth
and
breadth.
Now
the
depth
part
of
it.
We
we
have
pretty
down
pretty
well
like
the
the
concepts
around
observability
sure,
there's,
there's
still
a
lot
of
challenges
when
figuring
out
how
to
deal
with
distributed
systems
and
creating
visibility
for
them,
but
but
it's
also
largely
understood
what
you
can
do.
B
We
also
are
pretty
good
at
incident
response
and
on
call,
and
especially
if
your
organization
has
implemented
the
sre
practice,
you
probably
have
that
pretty
dialed
in
and
there's
always
room
for
improvement.
All
of
this
is
a
journey.
So
what
happens
in
production?
The
depth
part
of
this
we're
doing
pretty
good
at
in
the
areas
that
there's
a
lot
of
growth
besides
iterating
on
observability
and
on
call
is
in
better
collaboration
with
security
and
in
devsecops.
B
What
we
have
been
neglecting
is
the
breadth,
and
that
is
your
pipeline.
What
happens
in
your
delivery
chain
and
true
code
to
cloud
visibility
so
understanding?
Well,
that
happens
from
the
point
that
a
feature
is
defined
to
what
happens
when
that
feature
is
running
in
production
has
to
include
both
of
these
and
not
just
the
the
coverage,
if
you
think
it
from
a
visibility
coverage,
but
the
relationship
between
all
of
these
elements
and
what
we
find
in
organizations-
and
you
might
be
saying
to
yourself-
we
are
really
good
at
using
prometheus
to
visualize
xyz.
B
You
do
find
pockets
of
visibility
in
enterprises
that
are.
That
is
very,
very
good.
For
example,
you
know
developers
tend
to
be
very
good
at
visualizing,
dev,
test
environments
and
what's
happening
there.
Devops
engineers
tend
to
be
very
good
at
visualizing,
maybe
what's
going
on
with
their
pipelines
and
what
what
happens
to
the
tooling
and
how
it's
being
used
and
so
forth,
but
these
silos
become
as
problematic
as
they
are
good
at
their
particular
job,
they're
very
fine-tuned
at
one
job.
B
What
what
happens
is
that
developers
are
speaking
one
language
devops
engineers
are
speaking
in
one
language.
Security
professionals
are
speaking
one
language.
Why
does
that
matter?
To
the
developer,
you
may
think:
well,
that's
just
fine!
I
want
to
be
really
good
at
my
job.
I
don't
really
care
what
happens
once
it's
thrown
over
the
fence.
B
That's
why
these
visibility
silos
are
bad,
because
your
security
professional
is
not
going
to
understand.
What's
going
on
in
your
prometheus
dashboards
they're,
just
not
you're
going
to
have
to
explain
every
time
and
I
think
in
the
development
space
we
like
to
refer
to
that
as
shaving
the
yak,
which
is
one
of
those
terms
that
I've
never
really
understood.
So,
if
somebody
out
there
understands
exactly
where
they
came
from,
please
please
tell
me
so
this
is
where
how
we
get
to
why
why
pipeline
analytics?
Why
are
we
doing
this?
B
Well?
First
of
all,
if
your
delivery
chain
is
down
no
code
chips,
so
your
customers
are
your
developers,
your
devops
engineers,
everybody
in
the
engineering
org,
and
if
your
delivery
change
shuts
down,
you
can't
deliver
a
code
that
is
the
service
that
you're
providing
also,
we
know
now-
and
I
don't
even
have
to
prove
this
point
anymore,
because
it's
been
in
the
news
so
much
that
your
delivery
chain
is
a
part
of
your
attack
surface.
So
when
it
comes
to
security,
we
talk
about
the
solarwinds
hack.
B
B
Also.
I
already
talked
about
speaking
the
same
language,
that
that
is
a
big
reason.
Why
we
do
this
measuring
business
value
so
have
you
ever
have
a
hard
time
justifying
to
your
manager,
a
tool
that
you
would
like
to
be
by
part
of
that
justification
is
being
able
to
demonstrate
business
value
and,
yes,
yeah
as
you
grow
in
your
career
and
maybe
you're
already
there
engineering
you
also
have
to
speak
business,
speak
so
being
able
to
correlate
what
you
do
in
your
delivery
chain
to
success.
Failure
business
value
is
a
big
deal.
B
The
other
two
categories,
which
are
not
as
obvious
is
this
is
also
a
way
to
to
help
reduce
technical
debt
before
it
becomes
technical
debt
and
support
shift
left,
which
is
obviously
something
that
we
all
talk
about.
It's
all
about
creating
that
framework
and
paving
that
road
to
make
sure
that
the
application
velocity
and
the
automation
you
write
is
sustainable
over
a
long
period
of
time.
B
So
now
I'm
really
going
to
dig
into
what
the
components
are
of
pipeline
analytics.
Some
of
these
are
more
fun
than
others.
It
depends
on
choose
your
poison.
I
think
the
one
that
people
get
the
most
excited
about
and
will
probably
double
click
on
and
why
it's
at
the
end
is
the
measuring
success.
The
kpis,
but
there's
other
value
here,
and
these
three
use
cases
tend
to
be
how
you
structure
your
pipeline
analytics
tooling.
So
first
is
just
monitoring
your
software
delivery
chain.
Just
like
you
monitor
your
applications
is
my
secrets.
B
Management
tool
up
is
my
ci
cd
tool
up
and
responsive
and
running
with
secrets
can
are
the
least
times
on
the
like
the
the
turnaround
time,
the
sli
for
returning
a
secret?
Is
it
fast
enough
also
measuring
value,
creating
value
stream
being
able
to
tell
people
that
you
just
invested
a
whole
bunch
of
money
in
observability
and
it
has
reduced
your
mttr
and
the
one
that
tends
to
be
the
most
boring?
B
So
how
do
you
do
this?
Well,
the
other
thing
that's
kind
of
frustrating
of
why
enterprises
aren't
always
already
doing
this.
Is
it's
not
that
hard
and
it
may
not?
Even
you
may
not
even
have
to
procure
a
new
tool
to
do
it
so
you're,
gathering,
metrics
and
logs
from
the
tools
in
your
tool
chain,
not
just
the
tools
themselves,
but
also
the
processes,
so
the
automation
that
you
run,
because
some
of
that
automation
may
not
be
directly
associated
with
the
tool.
B
You
find
that
your
ci
cd
tool
tends
to
be
the
the
low-hanging
fruit,
not
the
only
one
but
low-hanging
fruit,
and
I'm
gonna
discuss
that
in
a
second
once
you
have
the
data,
you
need
to
correlate
that
data
across
tools
and
teams,
so
understanding
from
your
ticketing
system.
How
that
relates
to
your
repository,
how
that
relates
to
your
ci
cd
jobs?
B
These
are
the
two
things
that
are
always
present
and
it
could
be
both
and
so
the
two
key
challenges,
and
thus
the
two
key
areas
that
enterprises
need
to
think
about
is
data
resolution
and
tool
sprawl
data
resolution.
What
do
I
mean
by
that?
Well,
metrics
are
different
than
logs
right.
Metrics
are
a
synthesized
version.
The
data
is
probably
coming
from
a
log,
but
it's
it's
synthesized
data
points,
the
things
you
want
to
measure
over
a
time
series,
so
it's
extremely
valuable.
B
You
can
do
a
lot
with
metrics,
but
it
doesn't
have
a
lot
of
verbosity
and
data
to
dig
into
in
in
the
future.
What
are
other
elements
of
data
resolution?
Well,
when
you
commit
your
code,
are
you,
including
the
ticket
number
from
your
ticketing
system?
If
not,
I
don't
have
the
ability
to
correlate
the
two
together.
B
B
So
you
actually
have
to
decide
early
on
with
your
tools
the
type
of
data
you're
going
to
get
how
you're
going
to
correlate
that
data
together.
Also
when
it
comes
to
data
resolution,
the
more
tools
you
correlate
together,
the
better
resolution
you
have
so
I
will
give
you
an
example.
I
can
calculate
change
failure
rate
in
three
different
ways.
I
can
calculate
it
from
my
ci
cd
tool
related
to
rollbacks.
I
can
calculate
it
from
my
incident
management
incident
response
tool
based
on
incidents
or
ticket
open
and
ticket
close
date.
B
I
can
calculate
it
from
my
ticketing
system
with
bugs
open
close
date.
The
least
accurate
of
all
of
those
is
your
backlog,
because
there's
a
lot
of
different
reasons
that
are
bugs
are
filed
in
a
backlog.
The
most
accurate
is
actually
well.
You
know.
Maybe
you
could
argue
against
this,
but
for
most
people
the
most
accurate
is
going
to
be
your
incident
response,
tooling.
Assuming
that
you
have
a
really
strong
incident
response
strategy
and
then
middle
of
the
road
is
going
to
be
your
cicd
tool.
Any
one
of
those
are
a
way
to
calculate
change.
B
Failure
rate.
However,
your
accuracy
is
going
to
increase,
as
you
correlate
the
three
together.
So
that's
another
element
of
data
resolution.
These
are
all
not
technology
problems.
These
are
all
things
that
the
organization
has
to
consider
on
their
own.
The
next
thing
is
tool
sprawl
speaking
of
tools.
B
Large
companies
have
nearly
every
tool
out
there
and
all
of
those
tools
produce
data
and
there's
a
tendency
to
say
that
we
are
going
to
consolidate
on
one
tool,
which
would
be
amazing.
That
is,
I
think,
the
the
overall
goal.
This
is
certainly
how
I
advise
our
customers,
the
path
they
need
to
go.
What
companies
want
that?
B
I
I
break
so
many
hearts
on
is
they
want
just
to
be
agnostic
of
the
tool
and
just
say
it
doesn't
matter
the
tool
just
just
get
the
data
in
and
once
the
data
is
in
we're
going
to
be
able
to
make
value
from
it.
You
run
into
a
lot
of
issues
with
that.
You
have
apples
to
oranges
problems
where
the
way
one
tool
reports,
job
start
job
end
is
different
than
another
tool
and
reconciling
those
becomes
very
difficult,
so
tool
sprawl
is
a
problem,
so
is
tool
fatigue.
I'm
sure.
B
We've
all
felt
that
in
our
industry
we
have
a
tendency
to
become
very
tool
obsessed
due
to
the
nature
of
being
builders
and
tinkerers,
but
it
can
cause
problems
and
so
tool.
Consolidation
is
a
noble
effort
and
you
can
actually
use
pipeline
analytics
as
a
way
to
enforce
tool,
consolidation
by
demonstrating
value
on
a
very
focused
tool,
and
I
could
talk
about
that
a
lot,
but
I'm
going
to
move
on
to
the
next
step.
B
So,
let's
find
out
how
we
execute
on
these
three
primary
use
cases
I'm
going
to
start
with,
monitor,
go
into
audit
and
then
measure.
We'll
finish
with,
because
that
is
the
most
fun
in
my
opinion,
so
monitoring
no
different
than
than
what
you
do
today
with
your
applications
in
production
is
it
is
your
cicd
tool
running?
Is
your
artifact
management
tool
running
if
it's
running
is
it?
Is
it
tapped
out?
Is
it?
B
Is
it
performant
and
you
can
actually
use
standard,
red,
metrics
or
golden
signals
from
a
infrastructure
standpoint
to
oftentimes
determine
if
this
is
happening,
if
you're
using
a
sas
based
repository
tool,
your
considerations
change
and
it
becomes
much
simpler.
It's
it's
all
around
availability
in
response,
but
if
you're
running
a
secrets
management
tool
on
prem,
then
it's
basically
your
standard
infrastructure
metrics,
which
makes
it
very
nice
and
easy
to
do.
B
Usually
what
you
see
organizations
do
is
create
a
dashboard
just
for
monitoring
and
it's
usually
owned
by
the
devops
team,
sometimes
sres,
auditing
and
devsecops.
So
this
is
checking
things
like
in
your
secrets.
Management
tool
are
people
generating
secrets
for
root
access
that
shouldn't
be
allowed,
probably
ever
that
should
never
happen.
B
So
if
that
is
happening,
you
want
to
know
about
it.
Are
you
getting
secrets
lease
requests
from
outside
outside
the
u.s
outside
your
your
firewall
or
your?
What's
the
proper
networking
term
here,
we'll
just
say
your
your
network
and
how
many
of
those
are?
Are
you
getting
so
these
are
all
making
sure
that
you
don't
have
bad
axers
accessing
your
tools
in
your
tool
chain,
but
also,
if
the
auditing
compliance
department
comes
down
to
you,
you
don't
have
to
stop
everything
for
two
weeks.
B
You
can
show
them
a
dashboard
and
say
here's
your
audit
trail.
This
is
how
we
do
what
we
do.
Here's
the
information,
let
me
know
if
you
have
any
questions
versus
stopping
everything
and
stop
building
functionality,
so
it's
certainly
fun
not
to
be
disruptive.
That's
one
of
the
reasons
you
do
this
again.
This
tends
to
be
its
own
dashboard
that
is
delivered
to
the
rest
of
the
organization.
B
B
The
first
thing
you
have
to
decide
is:
what
are
you
going
to
measure?
There
are
so
many
metrics.
You
can
measure,
and
so
many
reasons
to
do
that,
once
you've
decided
what
you're
going
to
measure
you
have
to
figure
out
how
you're
going
to
measure
it.
So
I
gave
you
the
example
of
change
failure
rate
in
the
various
ways
you
can
measure
it,
some
of
it's
based
on
having
the
data
and
if
you
don't
have
the
data
you
need
to
try
to
get
the
data.
B
If
you
do
have
the
data
is
a
high
enough
resolution
to
answer
the
questions
you
want
to
answer.
That's
those
two
things
have
to
happen
every
single
time,
and
this
is
why,
unfortunately,
it's
not
turnkey,
so
nearly
every
company
I've
dealt
with.
There
is
a
different
format
and
way
of
dealing
with
this,
and
unfortunately
it
comes
so
often
down
to
how
projects
and
repos
and
teams
are
named.
It
is
such
a
simple
thing
naming
conventions,
but
that
tends
to
be
the
problematic
factor
in
almost
all
the
time.
B
So
there
is
this
thing
out.
There
called
the
dora
metrics
and
tracy
mentioned
this.
Dora
was
a
research
institute
that
was
at
some
point,
acquired
by
google
and
releases
the
amazing
annual
report
on
devops
and
as
a
result
of
all
their
resource
research.
They
came
up
with
with
four
key
metrics
around
measuring
your
delivery
chain.
It's
great.
It
comes
up
in
conversations
a
lot,
but
the
reason
it's
great
is
because
it's
forcing
people
to
think
about
pipeline
analytics.
B
So
it's
very
similar
to
what
google
did
with
golden
signals,
it's
very
similar
to
what
we
see
with
red
metrics
that
they're
a
good
starting
point,
but
you're
not
done
just
because
you
implement
these
and
they
may
not
be
where
you
need
to
start.
They
may
not
be
all
of
the
metrics.
You
need
to
create
and
there's
other
out
there
that
I'm
actually
really
interested
in
there's
flow
metrics.
For
example,
there
is
I
really
like
works
in
progress.
I
really
like
the
calculations
of
unplanned
work.
B
B
So
it
certainly
changes
because
mean
time
to
value
is
not
going
to
be
the
same
when
you're
doing
blue
green
deploys
as
it
is
for
an
organization
where
the
version
that's
in
production
is,
is
the
version
that's
in
production,
so
it
becomes
very
fascinating
and
the
reason
that
this
is
so
important-
and
I
want
to
highlight
that-
is
the
fact
that
you
have
to
think
about
this.
You
can't-
and
this
is
true
for
everything
you
can't
just
expect-
an
on-call
tool
to
make
an
incident
response
strategy
for
you.
B
You
can't
just
expect
pipeline
analytics
tooling
to
solve
the
metric
question
for
you.
That's
a
question
that
you
have
to
answer.
It
can't
fix
your
team
naming
conventions.
It
can't
fix
your
project
naming
conventions
now,
one
of
the
things
that
we
suggest
is
establishing
a
common
information
model
and,
as
a
matter
of
fact,
I'm
working
on
that.
I
would
love
for
that
to
be
an
open
source
project
that
eventually
becomes
a
part
of
the
cd
foundation,
but
you
need
to
establish
these
upfront
even
prior
to
thinking
about
implementation.
B
Good
news
is,
there's
best
practices,
there's
a
lot
of
guidance
out
there
happy
to
talk
about
that
with
you,
if
you're
you're
interested
so
going
back
to
the
the
the
initial
point
in
the
punchline.
Your
software
delivery
chain
is
an
application
of
applications.
Because
of
that
you
need
to
treat
it
like
an
application.
It
needs
to
be
operable,
auditable
and
measurable.
B
It's
the
measurement
is
the
most
fun
and
that's
what
pipeline
analytics
does
for
you.
Pipeline
analytics
is
the
foundation
of
devsecops
foundation
of
a
lot
of
things
and
there's
there's
so
much
creativity,
you
can
you
can
get
here
some
good,
some
bad
one
of
the
things
I
want
to
put
out
there.
Leaderboards
leaderboards
are
really
fun,
but
they're
also
very
dangerous.
B
So
if
you're
thinking
that
you
want
to
pin
your
engineering
teams
against
each
other,
you
need
to
really
really
consider
that,
because
that
can
also
backfire,
so
pipeline
analytics
is
something
that
everybody
should
embrace,
and
I
really
appreciate
you
taking
the
time
to
join
me.
I
know
there's
a
lot
of
information
out
there,
and
so
I
respect
the
fact
that
anybody
who
is
dedicating
half
an
hour
of
their
time
on
one
of
my
sessions
is
is
being
very
nice
to
me.
B
A
A
A
A
A
So
when
you,
when
we
move
to
microservices
and
microservices,
are
all
independently
deployed,
have
you
thought
about
the
the
analytics
that
we
need
to
start
tracking
on
them
on
a
microservice
so,
for
example,
at
ortillius
and
deploy
hub
we're
thinking
about
this
stuff
in
terms
of
risk,
risk
value
and
usage?
So
you
push
a
micro
service
out
there,
but
nobody's
consuming
it.
Was
there
a
value
of
doing
what's
the
value
of
it
being
out
there?
So
have
you
thought
about
that.
B
Yeah
and
I
you're
right,
it's
a
very
hard
question,
because,
because
the
easy
response
is
a
cop-out
and
that
cop-out
is,
is
you
don't
you
don't
look
at
it
at
a
service
level?
You
you
roll
it
up
to
a
broader
level,
but,
but
I
think,
looking
at
it
at
a
service
level
is
is,
is
absolutely
critical.
What
we
found
with
organizations
is
that
the
drill
down
has
been
very
problematic,
and
so
what
tends
to
happen?
B
So
at
the
25,
let's
say
for
simple
purposes:
you
you're
rolling
out
an
application
to
100
users
and
you
start
with
25
of
the
population.
All
looks
good
50
of
the
population
looking
great
but
to
make
the
decision
at
50
to
turn
the
whole
thing
on
when
maybe
at
75
percent
everything
explodes
because
of
latency
and
performance
that
you
never
even
anticipated
and
never
saw.
So
those
are
the
reasons
you
do
this
type
of
deployment
strategy.
B
So
if
you're,
not
visualizing
and
and
building
dashboards
on
that
you're
you're
missing
out
on
that
granularity
and
opportunity,
I
don't
think
that
it's
a
technical
challenge.
I
think
that
all
of
the
data
is
there,
especially
if
you
have
good
observability,
tooling
and
apm,
but
I
don't
think
organizations
have
yet
thought
through
that
I've
seen
how
they're
going
to
make
value
from
that.
So
I
don't
have
a
good
answer.
A
B
They
are
responsible
for
owning
and
defining
those
metrics
at
the
service
level,
they
are
responsible
for
making
available
the
metrics
the
standard,
redmi
metrics.
Once
you
double
click
and
you
get
become
a
service
owner,
you're
interested
in
stuff
that
other
people
on
the
team
are
going
to
have
no
clue
about,
and
it
depends
whether-
and
this
is
the
other
thing
I
want
to
say-
it
depends
on
the
risk
level-
is
very
interesting.
If
it's
a
front
end,
it
may
have
less
risk
than
if
it's
back
end
your
pushing
changes.
B
Your
back
end,
it's
probably
more
risk
than
your
difficulty
changes
to
a
front
end.
So,
at
the
service
level,
our
service
owners
are
on
their
metrics.
I
wonder
if
you
can
do
the
same
thing
with
pipeline
analytics.
You
still
allow
the
capabilities
at
the
service
level,
but
the
service
owner
is
the
one
who
owns
those
metrics.
The
downside
of
that,
though,
is
that
part
of
this
is
demonstrating
value
to
the
entire
organization.
B
B
We
hope
and
believe
that
that
is
possible,
but
we're
going
to
have
to
settle
for
maybe
even
one
and
maybe
even
30,
because
yeah
you
can't
in
in
a
micro
services
environment,
it's
misleading
to
say
that
our
entire
application
is
doing
well.
When
you
have
one
critical
service
that
is
not
performing
well
at
all
over
over,
like
has
a
tremendously
high
change
failure
rate.
A
A
Another
kind
of
thing
that
I
and
I
think
about
this
stuff
oftentimes,
I
don't
know
where
my
brain
goes
sometimes,
but
when
we
start
thinking
about
the
pipeline
and
we
think
about
maturing
the
pipeline,
I
always
feel
like
the
maturity
of
the
pipeline
itself
should
be
part
of
our
pipeline.
Metrics
we're
going
to
start
seeing
infrastructure
as
code
be
part
of
the
pipeline.
B
A
B
Puppet
and
chef
and
terraform
terraform
is
one
of
the
most
common
tools
to
be
monitored
in
pipeline
analytics,
and
I
think
you
see
that
a
lot,
maybe
part
of
it,
is
an
artifact
that
the
devops
team
tends
to
have
more
control
of
what
they
own,
and
so
it's
easier
for
them
to
do
that,
but
I
think
you're
right
everything
needs
to
be
abstracted
to
this
idea
that
it's
it's
all
code.
It's
it's
all
software,
it's!
B
I
am
trying
to
automate
it
as
much
as
I
can
possibly
animate
and
including
you
know,
maybe
not
bare
metal.
I
know
we
talked
about.
This
would
be
really
interesting
from
the
perspective
of
mainframes,
but
it's
all
code,
it's
all
software.
It
all
has
telemetry.
We
need
to
make
value
from
that
and
I
think
actually
you
find
that
that's
easier
to
do
than
what
we
were
talking
about
with
microservices.
A
But
when
we
talk
about
things
like
that,
I
feel
like
we
are
still
thinking
about
our
monolithic
practices
and
we're
not
really
trying
to
think
of
a
service
oriented
architecture.
So,
as
we
go
down
this
road,
we
have
to
really
start
asking
the
question
about
what
are
the?
What
is
a
service
oriented
architecture?
A
B
Yeah
and
then
the
exercise
you're
going
through
right
now
is,
is
exactly
what
companies
don't
do
and
I,
more
than
once
I've
run
into
a
company
where
they're
like
okay,
we
set
up
observability.
We
set
up
incident
response.
We
have
all
this
great
stuff
going
on
now.
We
want
to
decide
how
well,
how
we're
going
to
measure
it
versus
saying
before
we
get
our
instagram
response
tool
before
we
get
our
observability
tool,
let's
decide
what
is
the
meaning
of
good?
B
How
do
we
know
that
by
deploying
these
things
we
are
doing
better,
and
I
think
that
that
is
is
true
with
everything
because,
like
like
you
said,
I
mean
serverless
functions,
get
away
from
infrastructure
like
the
up
and
down
of
surplus
functions.
That
happens
in
a
blink
of
an
eye.
Well,
maybe
not
quite,
but
it's
very
quickly
and
it,
but
it
matters
to
the
value
of
your
delivery
chain.
So
that
exercise
is
the
exercise
that
is
the
most
frustrating,
because
everybody
wants
an
easy
button.
A
B
A
B
A
And
yes,
there
was
a
question
out
there.
If
this
will
be
recorded,
it
is
recorded,
and
after
this
I
will
produce
it
and
it
will
be
posted
in
the
cd
foundation's
online
meetup
playlist.
So
you
can
go
to
the
youtube
account
or
the
youtube
channel
for
the
cd
foundation
and
look
for
the
online
meetup
and
it
will
be
posted
here,
probably
in
the
next
24
hours
or
less.