►
From YouTube: Measuring DevOps with the Four Keys Project: How to Make it Work for your Org... Dina Graves Portman
Description
Measuring DevOps with the Four Keys Project: How to Make it Work for your Organization - Dina Graves Portman, Google
Through six years of research, Google’s DevOps Research and Assessment (DORA) team has identified four key metrics that indicate the performance of a software development team. The metrics not only give insight into DevOps performance, but the research indicates that these metrics are actually predictive of improved business outcomes, including profitability, productivity, and customer satisfaction. Google Cloud Platform has created an open source project to help automate the generation and collection of the metrics. However, not every organization has a workflow that will work perfectly out of the box with the Four Keys Project. This session will outline how to update the data definitions and extend the Four Keys Project to better reflect your operations.
For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/
A
A
Okay,
let's
start
with
some
definitions,
so
we're
all
on
the
same
page.
First
of
all,
what
is
devops
well,
there
are
a
lot
of
definitions
for
devops,
but
this
is
the
one
that
we're
going
to
use.
Devops
is
an
organizational
and
cultural
movement
that
aims
to
increase
software
delivery.
Velocity,
improves
service
reliability
and
build
shared
ownership
among
software
stakeholders.
A
Now
another
word
that
you're
going
to
hear
me
throw
around
is
dora.
Dora
is
the
devops
research
and
assessment
team
and
through
six
years
of
research,
they
identified
four
key
metrics
that
are
predictive
of
success,
which
is
a
very
strong
relationship
stronger
than
correlation
and
roughly
these
four
key
metrics
fit
into
two
buckets
speed
and
stability.
A
A
A
A
A
A
To
that
we'll
be
able
to
accept
data
from
these
dozens
of
different
systems,
systems
that
we
might
not
even
be
aware
of
yet
that
you
might
be
using
a
tool
that
I've
never
even
heard
of,
and
so
how
do
we
build
something
that
can
integrate
with
all
of
these
things
we
don't
even
know
about
well.
The
solution
that
we
came
up
with
was
to
create
a
very
generalized
pipeline
that
takes
in
events
via
web
hooks
and
then
ingests
them
into
bigquery.
A
So
to
start
with,
we
have
the
events
happening
at
their
source
and
every
time
an
event
happens.
We
send
that
event
to
our
web
hook,
which
is
the
event
handler
and
per
source.
We
have
a
pub
sub
topic
and
a
little
data
parser.
That
does
some
very,
very
light
data
mapping
very
light
etl
here
and
then
puts
that
into
a
bigquery
table
that
that
is
just
append
only
so
we're
just
adding
rows
and
then
from
there
we
have
some
derived
tables
and
then
that
feeds
into
our
dashboard.
A
A
So
I'm
going
to
go
over
the
dashboard
pretty
quickly
down
the
center.
We
have
the
the
metrics
as
defined
by
the
door
research
and
they
are
separated
into
the
different
buckets
that
the
door
research
separates
them
into.
So
if
you
have
good
or
elite
performance,
then
this
will
be
represented
with
a
green
badge.
A
A
A
You
know
reasonable
defaults
and
some
a
couple.
You
know
common
data
sources
and
we've
done
the
data
mapping
for
that,
but
obviously
we're
not
going
to
account
for
every
single
use
case
right
out
of
the
right
of
the
bag.
So
how
do
you
make
it
work
for
you?
So,
let's
start
by
talking
about
adding
new
data
sources.
A
New
integrations
with
the
four
keys
project,
so
in
order
to
do
that,
we
are
going
to
have
to
edit
the
event
handler
we're
going
to
have
to
create
a
new
pub
sub
topic
and
we're
going
to
create
a
new
data
parser,
and
I
have
a
little
video
that
I
am
going
to
play
here
and
talk
you
through
showing
how
you
can
add
a
new
source
okay.
So
this
is
sources.pi
and
these
are
event
sources.
A
Each
event
source
has
a
name,
a
signature,
header
and
a
verification
function,
because
we
don't
just
want
to
accept
any
old
thing
from
the
open
internet.
We
want
to
make
sure
it's
authorized
and
some
sources
have
very
complicated
ways
of
authenticating
their
messages.
Others
just
have
a
very
simple
verification
system,
so
I
am
going
to
build
in
a
new
source
and
it
will
say
this:
is
this
very
cool
new
get
system?
A
A
A
A
Okay,
so
now
I
need
to
create
the
all
that
pub
sub
stuff
and
we
created
this
little
script
to
try
to
make
that
a
little
bit
easier
and
so
we're
putting
in
the
user
agent
of
the
source.
It's
cool
get
hook
shot,
and
then
you
saw
I
put
in
cool
git.
That's
gonna
be
the
name
of
my
worker
here,
okay,
so
it
did
all
the
logic
it
did,
the
all
the
piping
it
did,
the
permissions
it
created
the
pub
sub.
A
So
now
one
of
the
things
that
it
did
was
it
also
created
this
new
directory
in
my
my
source
right
here,
and
this
is
just
copied
from
a
template.
This
is
what
you
know.
The
workers
basically
look
like
they're
looking
for
pub
sub
messages
and
we're
going
to
process
the
new
source
event.
So
this
is
where
the
data
mapping
happens.
A
So
what
I
would
do
at
this
point
is,
I
would
look
at
the
payload
from
the
source
and
I
look
at
what
they're
calling
these
different
fields,
how
they're
nested
and
then
I
would
map
that
to
these
fields
that
my
bigquery
table
is
expecting.
So
it's
expecting
all
these
fields,
event
type
id
metadata
time
created
signature
message,
id
and
source
and
the
metadata
is
just
the
entire
body
of
the
event.
A
So
here
I
am,
you
know
doing
this
data
mapping,
making
sure
that
I'm
pulling
out
the
correct
fields
from
the
message
that
I
am
piping
through
my
little
etl
system
here
and
the
source.
We
want
it
to
be
called
cool
git
when
it
shows
up
in
the
table,
because
it
is
the
coolest
kit,
I'm
double
checking.
Okay,
everything
looks
good,
so
let's
re-deploy
this.
A
This
is
just
my
command
to
deploy
this
and
there
it
goes
so
this
is
going
to
build
the
container
it's
going
to
push
it
up
and
everything
else
is
already
set
up.
So
you
know
the
event
handler
knows
to
accept
from
this
new
source
and
knows
how
to
verify
that
it's
a
an
authorized
source.
The
pub
sub
topics
are
set
up.
A
They
have
the
correct
permissions
to
push
to
our
worker
and
now
I'm
going
to
test
posting
to
it,
and
if
I
did
it
correctly,
if
I
did
it
correctly,
we
are
going
to
see
some
data
in
our
table.
It
will
have
gone
through
that
entire
pipeline
that
whole
system
and
so
there'll
be
some
data
here.
Okay,
so
we're
just
gonna
select
star
from
events
raw
where
the
source
equals
cool,
get.
That
is
the
source
nickname
that
we
put
in
the
table.
A
And
voila
there
it
is.
That
is
the
data
that
I
just
posted
to
the
event
handler
excellent
okay,
so
once
you've
created
a
new
source,
well,
you
submit
a
pr
please
and
thank
you.
We
have
a
roadmap
that
lists
all
of
the
integrations
that
we
want
to
build.
Very
common
integrations
lots
of
tools
that
people
commonly
use,
and
so,
if
you
do
go
through
this
process-
and
you
happen
to
build
a
new
integration
with
spinnaker
or
jenkins
or
something
on
the
roadmap,
then
please
do
submit
a
pr
would
be
very,
very
highly
appreciated.
A
A
A
A
Obviously
this
will
not
be
an
exhaustive
list
and
by
the
time
I'm
done
going
through
them,
you're
going
to
be
saying:
oh
yeah,
we
get
it
you,
you
add
something
to
the
sequel,
but
people
have
asked
me
for
concrete
examples.
So
I'm
giving
you
examples
so
deployments
we're
going
to
start
with
the
traffic
percentage,
because,
if
you're
doing
some
kind
of
canary
roll
out,
maybe
a
successful
deployment
to
your
organization
is
any
traffic
at
all
has
reached
the
user
or
maybe
it's
five
percent,
or
maybe
it's
fifty
percent.
A
Maybe
it's
a
hundred
percent
and
that
will
be
distinct
and
unique
to
your
organization,
and
so
in
this
example,
I
am
controlling
my
rollouts
with
cloud
build
and
I'm
using
a
variable
for
traffic
and
so
every
time
this
runs.
This
is
going
to
post
an
event
to
the
event
handler
and
in
the
the
body
to
the
pub
sub
message.
A
It
will
have
the
variable
for
traffic
and
I'm
going
to
use
that
to
query
I'm
going
to
dig
in
to
that
metadata,
which
is
the
entire
body
of
the
message
and
I'm
going
to
extract
the
field
that
I
care
about
and
filter
based
on
that.
So
in
this
example,
I
am
only
looking
at
deployments
where
the
traffic
is
greater
than
75,
and
so
you
do
that
by
editing
that
deployments.s
table,
that
is
in
the
source
of
four
keys.
A
A
A
A
A
Scheduled,
okay,
so
we've
gone
over
how
to
add
new
sources
and
then
also
how
to
adjust
your
data
definitions.
So
now
what
so?
At
this
point,
you
should
have
a
dashboard
and
it's
bringing
in
all
the
data
that
you
want
and
you've
really
encapsulated
your
deployments
and
changes.
The
way
that
you
want
to-
and
maybe
you
see
that
your
lead
time
to
change
isn't
very
stellar
and
you
want
more
insights.
A
So
it's
important
to
note
that
the
the
dashboard
template
it's
a
starting
point.
The
metrics
are
a
starting
point,
and
so,
if
you
see
that
your
lead
time
to
change
is
poor,
then
you
may
want
to
write
some
more
queries
to
really
dig
in
to
what
is
happening.
Perhaps
before
or
after
the
peer
review.
You
know
how
much
time
from
when
the
code
is
integrated
to
deployed.
A
A
A
A
A
So
the
question
here
is,
you
know:
are
there
other
customer
measurements
for
successful
deployments
like
performance
or
number
of
bugs?
The
answer
to
that
is
yes,
absolutely
there
are
so
many
metrics,
and
especially
depending
on
your
domain,
that
you
can
use
the
nice
thing
about
the
4k
metrics
is
that
they
are
not
industry
specific.
A
So
for
me
I
would
always
want
to
look
at
the
performance
of
my
organization
using
the
four
keys,
but
then
having
other
metrics
that
I
can
drill
down
into
organizationally,
because
these
are
metrics
that
that
don't
necessarily
guide
your
your
day-to-day
operations,
they're
very
high
level,
they're,
the
you
know
when
we
look
at
it
when
we
think
about
it.
We
think
about
what
our
performance
is
like
like
over
the
past
quarter,
not
just
not
in
the
past
day,
because.
A
And
that
is,
you
know,
because
people
have
different
patterns
of
rolling
out
software
and
yeah,
essentially
they're
they're
designed
to
be
high
level,
but
there
are
there's
a
lot
going
on
underneath
the
hood,
so
I
would
you
know
I
would
encourage
you
to
to
use
those
metrics
use
the
four
keys
to
kind
of
to
isolate
where
you
need
to
pay
attention.
You
know
if
your
lead
time
is
really
really
high,
then
I
would
start
looking
at
metrics
in
that
area
to
try
to
understand
where's
the
bottleneck
and
then
drilling
down
that
way.
A
A
So
if
there
are
large
gaps
between
development,
it's
it's
going
to
look
like
a
poor
performance.
There's
kind
of
no
way
around
that,
and
I
would
not
subtract
non-working
hours
from
total
time
because
at
that
point,
you're
you're,
starting
to
like
get
into
these
kind
of
like
you're
optimizing
for
the
for
these
like
edges
and
we're
getting
away
from
the
point
and
the
point.
One
of
the
points
is
we
want
to
know
ultimately
how
our
customers
are
experiencing,
the
software
that
we're
delivering
and.
A
If
you
know
you
start
a
pr
at
3
p.m,
you
stop
working
at
five
the
next
morning.
You
finish
it
at
10
a.m.
Well,
you
know
your
total
working
time
might
be.
I
don't
know
like
three
hours,
but
that
overnight
time
that's
time
that
people
all
over
the
world
have
not.
You
know
gotten
your
new
feature,
and
so
again
these
are
high
level
metrics
they're
supposed
to
be
zoomed
out.
A
If
you
want
under
the
hood
to
like
understand
a
little
bit
better
about
you
know
your
your
batch
size
and
how
much
time
your
developers
are
actually
spending
on
it,
then
that
again
would
be
a
drill
down
metric,
so
high
level.
You
want
to
keep
it
just
as
high
level
as
you
can
and
then
anything
that
you
want
to
drill
down
into
to
better
fine
tune
your
operations.
A
A
It'll,
be,
like
you
know,
relatively
poor
performance
for
that
time
period,
but
once
you
have
that
data,
you
can
see
how
exactly
that
went
through
your
system
and
then
you
can
start
putting
in
different
guardrails
and
different
checks
to
try
to
prevent
that
from
happening
again.
Try
to
detect
it
earlier
so
yeah.
We
don't
want
to
these.
Aren't
you
know
if
you
have
a
poor
performance?
It's
not
a
penalty
like
it.
It's
not
meant
to
to
punish
anyone
any
any
particular
person,
any
particular
team.
It's
not
like
saying
hey,
you
did
a
bad
job.
A
A
Okay,
oh
brian
colfer
had
a
question
or
some
comments
about
being
able
to
use
just
open
source
tools,
and
I
think
that
is
a
great
idea.
We
we've
had
a
lot
of
people
ask
about
whether
or
not
we
could
create
a
version
of
this
that
runs
in
pure
kubernetes,
and
I
would
be
absolutely
delighted
if
somebody
submitted
a
pr
that
did
that.
A
I
think
that
would
be
great
and
on
the
one
other
note
on
the
like
pure
open
source
section
is
we
are
working
on
migrating
our
dashboard
template
into
grafana,
which
is
open
source
dashboard
instead
of
data
studio
and
again
that
also
like
runs
in
containers.
So
I
mean
right
now
that
you
you
could
you
could
absolutely
do
all
of
this
using
open
source
tools?
It's
it's
a
very
container
heavy
architecture,
and
you
know
the
setup
script
just
to
do
like
a
really
easy
bootstrap.
A
It
just
pushes
everything
into
like
cloud
run,
and
you
know
pub
sub
and
things
like
that.
But
there's
no
reason:
there's
there's
nothing
special
and
unique
about
this
about
all
these
different
tools.
There's
no
reason
why
you
couldn't
use
another
one.
The
only
exception
to
that
I
think
is
bigquery
is
the
as
a
you
know,
previous
data
analyst
data,
enthusiast
data
scientist
and
now
a
data
engineer.
A
A
Marcy
had
asked
how
poor
is
poor
performance,
for
instance,
median
time
of
commit
to
get
to
production.
So
I
would
encourage
you
all
to
go
to
the
the
google
cloud
devops
page
and
look
at
the
door,
research
and
so
like
poor
performance
is,
I
think
it's
like
you
know
like
a
month
or
something
for
lead
time
to
commit.
I
mean
for
your
time
to
change
and
yeah
it's
it's
all
about
finding
your
starting
place
and
then
working
from
there.