►
From YouTube: CDF SIG Best Practices - Nov 30, 2022
Description
For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/
B
E
D
D
E
D
F
E
D
E
Else
do
we
have
today
how
about
the
folks
who
have
never
been
here
before?
Maybe
you
could
give
yourself
a
quick
intro.
C
Yeah
sure
Colin
bowron,
the
SVP
of
product
at
octopus,
deploy
based
here
in
New
Zealand
kiwi,
a
Canadian
in
New
Zealand,
yeah
yeah.
E
I'm
sure,
where
did
you
say
you
you
were
working.
D
H
Okay,
Frederick
yeah
I'm,
one
of
the
maintainers
of
Jenkins
ex
from
Norway
and
I
work
as
a
consultant.
H
There's
no
Cloud
base
left
in
drinking
sex,
so
we
are
having
a.
We
have
a
plan
to
change
the
name
now,
but
we
can't
find
the
right
one.
E
All
right,
who
else.
G
Oh
yeah
hi,
my
name
is
Dick
benzo
I'm
working
for
fujito
and
I've
been
asked
to
visit
the
various
special
interests
groups
to
learn
a
little
bit
from
my
company.
What's
currently
going
on
on
the
various
groups.
What's
the
direction
of
the
CD
Foundation
at
the
moment,
because
Fujita
wants
to
intensify
their
efforts
in
this
area
and
so
on.
This
I
started
visiting
the
slack
discussions
and
then
the
various
meetings
watching
videos
of
previous
meetings
and
try
to
learn
where
you
guys
are
heading
to
at
the
moment.
I
Have
not
thank
you,
I'm
on
the
CDF
technical
oversight
committee
and
been
hearing
about
this
from
Terry
and
wanted
to
join
I
work
at
liquibase.
D
E
Did
we
get
an
Andreas
or
a
Dave
Bender?
Yet
no,
we
did
not
all
right
well,
I
mean,
since
you
have
a
hard
stop.
How
do
we?
How
do
we
best
get
started
on
this?
The
idea
was
to
try
and
identify.
D
E
Metrics
against
the
state
of
CD
events,
so,
let's,
let's
measurable,
that
we
can
correlate
to
a
best
practice
to
start
to
evolve,
recommendations
that
we
could
document
as
part
of
this
process
to
Terry.
If
I'm,
forgetting
part
of
this
conversation,
please
shout
out.
A
Yeah,
so
there
were
so
with
two
parts
of
this,
because
the
the
stuff
that
David
was
working
on
was
the
the
supply
chain,
maturity
piece
and
we
were
hoping
to
tie
all
of
these
bits
together
to
get
you
know,
everyone
aligned
on
on
Earth,
similar
overlapping
pieces
of
work.
D
E
So
for
those
who
aren't
aware,
I'm
not
sure
what
the
permissions
on
this
is:
I'll
drop
the
link
here,
it's
also
in
the
agenda.
Let.
E
Pardon,
like
I,
said
I'm
running
late,
so
breakfast
is
now
one.
One
of
the
topics
here
was
out
of
this
discussion.
Is
we
have
seen
the
events
which
is
a
natural
opportunity
to
have
a
trace-based
measurement
right
from
an
observability
standpoint,
so
it
felt
like
there
was
a
there
was
an
opportunity
here
to
to
start
to
actually
put
some
concrete
associations.
E
F
Can
go
to
see
the
events.com
there's
a
website.
F
F
D
F
Is
that
intentional
yeah?
Well,
the
names
are
strongly
influenced
by
tecton,
but
they
are
intended
to
be
generic
enough
to
to
manage
any
kind
of
I
would
call
them
activity
rather
or
jobs,
which
could
happen
in
a
pipeline.
So
it's
more
of
an
the
orchestration
level.
Events
on
a
different
dimensions
from
the
other.
D
E
So
I
think
one
one
thing
that
we
could
also
probably
do
in
this
is
try
to
fill
in
some
questions
about
what
we're
actually
measuring
so
like
for
code
management,
we
have
source
code,
control,
review
requirements,
linting
and
static
analysis.
What
are
we
actually
measuring
from
that
I?
Don't
know
if
Terry
did
you
did
you
and
David
have
some
discussion
on
this.
That
might
be
helpful
in
this
regard.
I
Right,
let's
start
doing
this
Tara,
are
you
talking
about
like
metrics
that
you
get
from
static
analysis?
Are
you
just
talking
about
specific
things
that
linting
and
static
analysis
is
doing.
E
So
the
the
goal
of
this
working
session,
which
again,
which
really
wish
David
was
here,
was
to
try
to
make
concrete
associations
with
key
Concepts
in
the
in
the
service
of
best
practice
guidance
right,
so,
which
is
a
non-answer
to
your
question.
So
to
specifically
answer
your
question.
A
A
E
Because
any
or
well,
hopefully,
everybody
in
this
session
knows
never
look
at
lines
of
code,
but
but
yeah
each
in
each
Target
metrics
is
going
to
be
based
on
individual
organizational
needs
right.
If
you've
got
like
a
really
old
kind
of
crafty
piece
of
infra
or
product,
you
you're,
as
you
apply
metrics
to
that
it's
probably
going
to
be
pretty
low.
E
So
you
don't
have
to
like
go
for
gold,
your
first
iteration,
but
how
do
you
even
measure
where
you're
at
and
what
your
goals
might
be
is
is
really
what
we're
trying
to
start
to
to
get
concrete
on
here
so
for
source
code
code
management.
D
I
Use
it
yeah
that
starts
going
to
use
it.
What
what
about
you
know,
there's
a
lot
of
stuff
that
a
lot
of
confusion
on
what
goes
into
source
code
control
and
I've
certainly
seen
oh
well,
no,
that
that
doesn't
go
in
source
code
control
that
goes
somewhere
else.
We
don't
need
that
over
there
I
think
this
group
would
disagree
with
that,
but
you
know
being
able
to
say
no
if
it
deals
with
your
application
from
you
know,
infrastructures
code
all
the
way
up
to
application
code.
It
all
goes
in
source
code
control.
Everything.
I
Well,
being
that
I
work
at
liquibase,
the
first
one
that
comes
in
mind
is
SQL
scripts,
but
I
don't
want
to
make
it
specific
to
that.
I
have
certainly
seen
oh
well.
We
don't
have
to
worry
about
that.
That's
not
part
of
the
application,
that's
something
else,
and
so
I'm
thinking
like
you,
know,
okay.
Well,
there
are
some.
I
You
know
at
some
companies
not
not
trying
to
shame
or
anything
they
got.
You
know
shell
scripts,
that
are
sitting
on
somebody's
computer
and
that's
what
this
one
individual
uses
to
do
their
job
and
it's
tucked
away
on
their
machine.
It's
not
in
source
code
control.
It
can't
be
improved,
it's
just
for
that
one
person,
but
it
is
like
the
basis
of
this
Fortune,
500
company
being
able
to
say,
hey,
look
if
you're
using
it
to
deploy
your
code
I,
don't
care
what
it
is.
It
needs
to
be
in
source
code
control.
C
A
live
example
that
would
be
something
like
a
QA
tester
in
their
database
reset
and
Sample
data
population
scripts
definitely
seen
a
few
qas
carry
those
around
on
USB
drives
yeah.
I
And
they
built
it
for
themselves,
and
the
value
here
is
to
sharing
it
with
others
and
that's
what
source
code
control
provides
us
I,
don't
know
how
it
fits
into
this
and
and
I'm
wondering
if
I'm
off
base
here
I'm
sorry
I
went
on
too
long.
E
C
So
scenarios
that
we
see,
because
we
do
some
run
book
automation
around
deployments-
people
will
use
scripts
for
resetting
databases
generating
sample
data.
C
Typically
you'll
see
t
SQL
scripts,
but
they
can
really
be
whatever,
but
they
they.
They
don't
see
that
as
part
of
the
core
application,
because,
as
Robert
pointed
out,
it's
like
something
I
wrote
for
myself
to
do
my
job,
but
actually
ends
up
getting
passed
around
via
email,
USB
keys
and
things
like
that
to
various
QA
teams.
C
Yeah
I
won't
name
the
organization,
but
yes
yeah,
the
the
the
other
I
guess
the
the
other
aspect
of
code
management,
Beyond
kind
of
those
ancillary
files
around
it
is
we,
you
know
in
a
past
life,
we
talked
a
lot
about
kind
of
where
in
the
code
base
is
the
most
amount
of
churn
is?
Is
this
one
kind
of
section
of
code
causing
us
lots
of
problem
or
we're
having
to
change
it
on
a
regular
basis?
C
So
people
look
at
that
kind
of
turnover
within
parts
of
the
code
base
as
a
sign
of
maturity
or
possible
problems,
or
did
that
surprise
me
in
some
way,
if
I,
if
I
see
the
stats
around,
that
I
think
TFS
way
back
when
used
to
even
do
a
code
turn
report
that
would
show
you
the
most
active
parts
of
your
code
base.
E
So
that's
I.
Don't
let's
see
I,
don't
remember
seeing
that
in
this
list.
Oh
wait,
candidate,
rollout,
latencies.
I
I
E
I'm
wondering
if
that's
actually
supply
chain,
though
there's
like.
E
J
J
C
Guess
the
way
I'd
I'd
kind
of
think
about
it
in
terms
of
the
supply
chain
is
where's,
my
at-risk
kind
of
where
where's
most
of
my
change
happening,
so
that
I
could
identify
potential
areas
of
risk,
and
you
could
even
extend
that
down
to
dependencies
like
how
often
are
these
particular
dependencies
changing
you
know
versus
what
are
the
stable
parts
of
my
code
base
that
seem
to
be
well
developed
and
not
need
a
lot
of
maintenance.
I
A
A
A
So
there's
a
there's,
a
big
gap
here
that
that
that
says:
okay,
what's
what's
the
feature,
but
we're
supposed
to
be
delivering
within
our
supply
chain.
When
does
that
feature
initiate?
Because
that's
when
the
clock
starts
ticking
on
lead
time,
and
so,
rather
than
doing
the
engineering
thing
of
trying
to
find
a
proxy
to
that
information,
we
should
actually
have
that
as
a
specific
event
and
and
push
it
back
the
other
other
way
and
say
when
does
that
event
occur
in
your
system.
D
D
A
J
E
Sorry
can
you
can
you
start
me
off
here,
I,
just
switch
into
suggest
mode
so
and
then
I
like
lost
the
track
of
what
you
were
saying.
A
G
J
A
Yeah
I
mean
those
the
realistically.
What
you're
talking
about
is
the
decision
to
implement
something
at
a
commercial
level,
so
yeah
that
that
might
be
the
decision
to
implement
a
fix
for
a
bug
or
it
might
be
the
decision
to
build
an
experiment
to
test
a
customer
Behavior.
But
it's
that
it's.
That
decision,
which
is
the
the
clock,
starts
ticking
on
lead
time.
E
A
So
the
the
the
challenge
that
people
all
always
come
across
in
my
experience
is
that
it's
easy
to
measure
things
in
the
code
loop,
so
everybody
always
goes
down
that
path
and
then
they're
not
actually
measuring
their
product.
Behavior
and
they're
not
measuring
what's
going
to
the
customer.
So
so
the
maturity
model
needs
to
be
extended
beyond
the
cicd
model.
At
both
ends.
E
A
Than
you
get
from
your
cocd
system,
so
this
is
CD.
Events
is
ideal
for
expanding
from
that
engineering,
specific
view
of
cicd
to
a
much
more
encompassing
end-to-end,
metrics
and
observability
driven
view
of
continuous
delivery.
C
A
concrete
example
of
that
Terry
would
that
be
let's
say:
I've
got
a
lending
system
and
it
is
successfully
deployed
to
a
production
environment
based
on
a
change
set
that
was
open
or
PR
that
was
opened.
That
is
about
adding
this
functionality
to
the
system
and
kind
of
closing
that
Loop
and
saying
this
this
this
original
event
that
said
PR
open,
I'm
gonna,
you
know
make
this
change
for
this
business
purpose
and
I've
now
closed.
That
Loop,
by
saying
that's
now
been
successfully
deployed,
is
that
the
business
scenario
that
you're
thinking.
A
A
E
A
Is
not
intended
just
for
cicd
systems,
it
it's
a
vocabulary
to
allow
you
to
communicate
it
with
continuous
delivery
in
general,
so
you
can
have
a
business
system
at
the
front
end
that
captures
your
decision
to
run
a
particular
feature
and
it
might
be
triggered
by
a
funding
event.
You
know,
we've
got
sign
off
to
run
this
experiment
on
this
feature.
Okay,.
A
Yeah,
so
you
so
that's
it
that
there's
a
c
you
can
have
a
CD
event.
That
says
you
know.
A
new
feature
has
been
initiated
as
a
continuous
delivery
activity,
and
then
that
starts
the
clock
ticking.
A
Other
end
of
the
process,
you
can
put
something
in
a
production
environment,
but
you
haven't
started
the
experiment
until
you've.
Let
the
customer
in
and
they've
actually
started
using
the
system.
So
there's
another
piece
at
the
other
end,
which
is
observability
on
the
platform
that
says,
there's
actual
activity
going
on
consuming
this
this
service
or
feature.
F
Emma,
what
was
your
comment?
Yeah
I
would
say,
I
mean
so
far.
We
haven't
looked
at
a
earlier
stages
than
when
a
source
code
commit
comes
in
in
the
series
definitions
of
events.
F
We
have
something
in
the
pipe
coming
in
which
we
will
call.
Is
it
issue
defined
or
something
where
we
intend
to
show
that
the
bug
or
something
has
happened,
and
that
could
also
be
probably
a
feature,
but
that's
not
defined
yet
as
far
as
I
know,
at
least
but
I
was
thinking
enough
under
this
metrics
to
consider
heading
there.
F
Because
there
it
says,
for
example,
in
the
build
stage
that
an
input
to
that
stage
is
the
software
source
change
that
which
means
that
the
build
stage
doesn't
start
until
we
have
a
software
Source
in
place.
So
it's
not
what
we're
talking
about
at
the
beginning
there
it's
not
really
about
what
is
the
coverage
so
how
much
of
our
data
our
source
files
are
in
Source
change
or
in
sem
systems
or
not?
That's
not
really
part
of
the
build
stage,
that's
something!
Maybe
that
comes
doesn't
stage
before
that.
F
F
As
the
metric
within
the
build
maturity
stage
or
whatever
you
just
call
it
box.
E
E
H
Management
system
like
jira
or
GitHub
issues,
or
something
very
Define,
your
issues,
then
they
want
to
see.
Where
is
that
issue
now
what's
happening
with
it
and
is
it
in
production
and
has
it
been
tested
and
so
on.
A
A
So
as
a
result,
there
there
are.
There
are
no
good
standard
hooks
to
hang
this
stuff
on
so
I.
Think
one
of
the
one
of
the
big
drivers
for
adoption
of
CD
events
would
be
that
it
provides
a
way
in
which
you
can
start
to
really
automate
the
measurement
of
your
Dora
Matrix,
because
it
provides
a
way
to
pass
those
types
of
events
through
the
build
process
and
get
them
out
the
other
end
and
extend
them
past
so
that
you're
getting
the
full
scope
of
the
information.
E
E
Right
and
it
means
it
is
this
supply
chain
maturity
to
make
sure
that
is,
if
it's,
that
definition.
Actually,
let's
see.
A
A
Licensing
in
Regulatory
Compliance
and
in
software
life
cycle,
as
well
as
the
security
aspects
and
and
and
and
and
and
and
and
we're
we're
going
to
see
a
lot
of
benefit
from
being
able
to
pass
things
like
life
cycle
events
up
and
down
the
software
supply
chain.
A
E
All
right,
so
in
the,
how
are
we
doing
for
time
all
right
in
the
service
of
trying
to
get
something
concrete
that
we
could
pass
out
on
and
we
lost
Emil
right?
Okay,
are
there
other
okay?
So
I
can
go
in
after
the
fact
and
make
the
associations
between
the
events
the
current
event
standard,
and
this
are
there
other
gaps
that
people
may
see
in
this
list
that
we
want
to
call
out
for
review.
E
What
is
code
I
think
is
the
other.
You
know
that
going
back
to
like
the
test,
tooling,
shell
script,
Etc,
there's
of
course
the
broader.
What's
the
scope
of
supply
chain,
the
pre-commit
post-deploy
bits
what
else
this
has
been
a
great
discussion
by
the
way.
This
is
I
think
one
of
the
best
discussions
we've
had
in
a
while.
This
is
awesome.
Are
there
other
kind
of
Gap
areas?
E
That
would
be
a
measurable
part
of
this
process
that
what
might
be
missing
here,
oh
and
by
the
way
Andrea?
Do
you
have
the
link
to
this
hear
me
drop
since
you
joined
late?
It.
A
Thank
you.
So,
under
our
dependency
management
section,
we're
definitely
missing
a
supply
chain
events
blank
there
might
be
life
cycle
events
as
an
another
way
of
putting
it.
A
Well,
that's
that's
things
like
you
know,
a
new
version
of
a
of
a
library
that
you
depend
on
is
available
or
the
library
you
depend
on
has
been
deprecated
those
types
of
events
and
then
there
will
also
be
security.
Related
events.
You
know
so
new
cve
on.
J
D
But
upgrades
e
l,
L's
LTS
pages.
E
J
A
So
what
what
we're
trying
to
do
with
this
piece
of
work
is
help
quantify
someone's
organizational
maturity
in
continuous
delivery
and
so
we're
trying
to
find
what
are
the
differentiators?
A
What
are
the
key
points
that
that
differentiate
between
a
mature
organization
and
a
less
mature
organization
and
what
we've
almost
defined
already
a
clear
differentiator,
which
is
that
a
mature
organization
will
be
thinking
about
this
as
a
product
to
customer
problem
and
a
less
mature
organization
will
be
thinking
of
thinking
and
thinking
and
thinking
and
thinking
and
thinking
and
thinking,
control
to
production
problem.
E
There
we
go
okay,
so
just
yeah
I
got
part
of
it.
Could
you
reset
restate
it
just
so
we
capture.
A
So
you
know
just
as
a
as
a
very
coarse
rule
of
thumb,
a
mature
organization
will
be
approaching
continuous
delivery
from
product
customer,
whereas
a
less
mature
organization
will
be
thinking
about
it
in,
in
an
engineering
perspective,
from
source
code
to
production,
and
so
that's
almost
like
the
the
level
one
decision
of
you
know
have
you
entered
on
to
the
maturity
hierarchy
yet
is
you
know,
are
you
still
thinking
about
this
as
an
engineering
problem,
or
are
you
thinking
about
it
as
a
delivering
features
to
customers
problem,
and
then
you
get
into
the
finer
differentiations
of
how
effectively
you're
doing
that
and
how
quickly
you're
doing
it.
E
A
Yeah,
so
so
it's
like
at
a
at
a
basic
level
of
maturity.
You
should
be
thinking
about
features
to
customer
at
a
higher
level
of
maturity.
You
should
be
pushing
your
knowledge
right,
the
way
out
into
your
supply
chain,
so
that
a
very
mature
organization
is
getting
information
about
changes
in
its
dependencies
in
real
time
and
using
that
to
trigger
changes
in
its
asset
management
for
its
product.
A
Cd
events
to
finer
and
finer
granularity
is
there'll,
be
a
need
to
react
to
lots
of
different
types
of
supply
chain
events
coming
into
the
system
so
that
those
can
be
used
to
trigger
activities
in
the
middle
and
then
we'll
almost
certainly
find
things
that
are
outputs
from
our
system
that
are
the
source
of
those
events
to
customers
further
down
the
supply
chain
from
us,
because
software
supply
chain
is
a
fractal
problem,
it's
it's
the
same
patterns
at
every
scale.
A
E
And
to
be
clear,
when
we
talk
about
the
higher
level
maturity
in
in
thinking
about
the
broader
inputs,
that
would
be,
for
example,
also
include
like
business
decision
like
so
if
there
was
I'm
just
going
to
make
something
up
here,
if
there
was
a
a
organizational
policy
that
all
new
products
or
features
used,
some
templatized
mechanism
for
being
created
that
could
trigger
an
event
that
would
actually
alert
like.
Are
you
kind
of
talking
about
that
level
of
instrumentation
I'm,
just
trying
to
understand
the
scope
of
what
what
you're
proposing.
A
Well,
classic
examples
here
are,
ideally
what
you
want
to
be
doing
is
working
towards
a
situation
where
you're
your
continuous
delivery
process
can
be
smart
enough
to
make
certain
classes
of
the
decision
for
you.
A
So
if
you
have
a
cve
of
a
certain
severity
on
one
of
your
dependencies,
the
build
system
should
be
smart
enough
to
immediately
attempt
to
upgrade
that
package
and
and
and
rebuild
while
it's
notifying
someone
of
of
that
event,
so
that
by
the
time
someone
gets
around
to
looking
at
it,
there's
already
a
you
know,
a
preview
version
in
place,
saying
you
know
here's
the
solution
with
the
new
version
of
the
library
in
it
and
it
all
builds
correctly
and
all
the
tests
pass.
So
should
we
put
it
into
production.
E
A
Taking
the
the
Reliance
on
people
having
an
end-to-end
understanding
of
what's
going
on,
pushing
as
much
of
that
into
the
system
as
possible,
so
rather
than
having
people
telling
the
system.
What
to
do.
The
system
is
advising
people
of
what
it's
done
in
reaction
to
things
that
are
happening
in
the
world.
E
E
We
have
a
supply
chain
that
involves
a
kubernetes
cluster
or
five
using
the
tecton
as
a
declarative
description
of
what
that
cluster
is
supposed
to
incorporate
the
mature
model.
Let's
say
that
an
incoming
CPE,
for
whatever
the
base
distro
image
is,
would
trigger
a
waterfall
effect
of
fig.
You
know
whatever
the
policy
is
for
staging.
You
know
rebuilding
that
container,
staging
it
in
in
some
Canary
deployment
and
triggering
a
set
of
notifications.
That
says
that
this
thing
is
ongoing.
E
Utilizing,
for
example,
CD
events
tied
into
the
overarching
pipeline,
because
the
the
plane
declaration
is
that
or
that
it
includes
the
idea
that
everything
is
up
to
date.
As
from
as
an
implemented
policy.
E
A
A
But
there's
limited
value
in
in
that
knowledge
within
the
cicd
scope.
But
it's
easiest.
It's
low
hanging
fruit,
so
where
that's
where
we've
started,
but
the
real
value
is
once
that
starts
to
spread
out
into
the
into
the
full,
continuous
delivery
problem
space,
because
that's
where
nobody's
got
a
good
technical
solution.
Yet
and
as
soon
as
we
start
to
enable
that
there's
a
whole
new
Market
opens
up
for
for
a
new
product
integration.
J
B
B
There
is
certainly
a
recording
I
can
find
the
link
and
send
it
to
you.
Sorry.
E
Cool,
we
are
down
to
the
last
couple
of
minutes.
Wow
there's
a
lot.
E
Any
other
interesting
ideas
that
we
want
to
drop
as
either
comment
for
Dave
bendery
or
a
future
topic
to
dig
in
I.
Think
we
have
a
lot
of
topics
that
we
can
continue
to
dig
in
on.
But.
I
Real
real,
quick
one,
I
I
just
said
this
whole
time.
I
was
thinking
what
about
the
things
that
are
manual,
that
we
can't
capture
with
an
API
and
and
how
does
that
play
into
this
because
I
go
back
to
the
who
who
said
it?
If
you
can't
measure
it,
you
can't
improve
it
and
one
of
the
challenges
of
those
manual
steps
in
not
so
continuous
delivery.
I
How
do
we
shine
a
light
on
that
with
CD
events
like
maybe
a
blocker
between
it
anyway,
just
wanted
to
that's
where
my
head
was
at
and
I.
Don't
know.
If
that's
imp
means
anything
here,
but
just
certainly
in
my
world
I
see
that
a
lot.
E
I
Colin's
example
with
the
the
USB
key
like:
oh,
we
we
it's
now
in
test,
but
somebody
has
to
manually
do
this
thing
before
that
release
is
deployed
and
testing
has
started.
How
do
you
track?
What's?
E
Yeah,
the
one
that
I
can
think
of
that
was
a
pain
and
now
there's
better
ways
of
doing
it.
But
I
can
imagine
many
companies
still
have
the
situation
that
for
certain
types
of
artifacts,
or
instance,
access,
let's
say
you're
in
a
super,
highly
regulated
environment,
and
you
have
a
requirement
for
compliance
reasons
that
requires
you've
got
the
the
nuclear
two
key
thing
right.
E
The
security
Engineers
had
one
key,
but
the
C
the
CSO
distinctly
both
in
order
to
access
the
Bastion
host.
In
order
to
do
a
thing
right-
and
you
know
some-
sometimes
you
think
well,
just
don't
do
that-
do
it
this
other
way,
but
there
you
know,
while
you're
still
required
to
do
that
for
business
reasons,
how
do
you
account
for
it
right?
I.
J
B
So
positive
and
something
we
definitely
talked
about
manual
things
but
I,
don't
know
if
that's
the
kind
of
manual
events
that
you're
thinking
about
so
I
mean
for
like
if
I
writing
a
test
manually
I
probably
need
to
report
that
results
somewhere
and
then
I
would
need
to
have
a
system
where
we
test
results,
attract
sending
a
CD
event
for
that
even.
I
To
the
source
yeah
that
sucks
having
people
to
do
timesheets,
you
know
I
I,
don't
have
an
answer
for
this,
but
I'm
just
saying
that
it's
just
a
really
tough
problem
and
if
we
really
want
to
as
a
best
practice,
provide
really
clear
insights
to
users,
something
to
consider
I,
don't
have
a
solution.
E
So
I'm,
remembering
for
and
maybe
some
of
you
who've
been
Jenkins
folks
in
the
past,
there's
I
think
if
it's
part
of
blue
ocean
or
but
there
was
a
mechanism
by
which
you
could
insert
into
the
pipeline
process
a
a
manual
input
or
it's
like
the
the
the
continuous
integration
flow
would
go,
and
then
there
would
be
a
widget
that
would
basically
like
at
this
point
a
person
has
to
do
a
thing
and
they
would
go
into
the
Jenkins.
You
know
UI
and
and
hit
hit
the
button,
and
then
things
would
progress.
E
It
was
like
a
like
a
queue
engineer
had
to
sign
off
that,
because
there
was
like
flaky
tests
and
other
types
of
stuff,
since
they
wanted
a
human
in
the
automation,
but
they
didn't
they
wanted
most
of
it
to
be.
Automated
I
was
like
well
that's
kind
of
Genius.
This
is
like
I,
don't
know
seven
eight
years
ago,
but
you
can
imagine,
like
you
know.
You
know
a
a
CD
event
pipeline
that
basically
sends
sends
an
event
or
publishes
an
event
that
gets
captured
by
maybe
an
element
in
the
in
the
kubernetes
control
plane.
E
E
Maybe
it's
through
notifications,
you
know
with
email
and
which
is
terrible
or
a
slack
bot
or
well,
but
some
kind
of,
but
a
notification
mechanism
that
someone
could
choose
to
attach
some
kind
of
interactive
mechanism.
That
would
that
would
the
pipeline
would
just
be
waiting.
A
It's
also
again
worth
going
back
to
the
the
methodology,
but
if
we
look
at
the
classic
example
of
you
know
where
continuous
delivery
came
from,
it's
okay,
I'm,
a
startup
I've
got
a
theory
about
what
a
customer
might
want.
Here's
an
experiment
to
test
their
behavior
I'll,
build
that
experiment,
I'll
deploy
it
I'll
run
the
experiment
for
a
week
collect
customer
data
and
then
my
experiment
is
finished
when
I've
analyzed
the
data
at
the
end
of
that
week.
H
A
If
you're,
if
you're
pushing
your
events
that
far
out,
then
any
manual
processes
are
dark
spots
on
that
Journey,
but
you
can
always
be
testing
to
see
if
you've
got
the
next
event
that
you
expected
to
happen
back
in
right
in
the
automated
world
and
that's
how
a
lot
of
manual
workflow
processes
work.
You
know
case
management
for
law
firms,
for
example,
yeah
riddled
with
manual
processes,
but
they
all
have
expectations.
A
So
you,
you
say
you
know,
here's
a
manual
process
that
started
on
this
date
and
we
are
now
waiting
to
receive
this
event
in
the
form
of
somebody
putting
in
more
information
in
a
month's
time.
Yeah.
If
that
event
doesn't
occur
within
this
time
window,
we'll
raise
an
alert.
A
I
So
inferring
from
the
the
the
length
of
the
manual
process,
whether
it's
started
or
stopped
in
the
amount
of
time
based
on
the
the
steps
before
and
after
that,
that's
reasonable.
No
argument.
Thank.
E
You
and
then
I
was
thinking
going
back
to
the
previous
example,
where
it's
like
somebody
has
to
put
the
USB
key
in
or
whatever
if
there
is
a
a
environment,
at
least
there's
a.
If
there's
like
an
organizational
policy
that
those
things
have
to
happen
within
a
certain
amount
of
time.
Like
hey
this,
this
deployment
pipeline
event
process
is
happening.
Somebody
you
know,
there's
a
QA
person,
who's
who's,
essentially
the
on-call
for
the
validation
and
they
have
24
hours
or
four
hours
or
whatever.
There.
E
So
yeah
an
internal
SLA,
there's
a
okay,
so
I
didn't
even
think
about,
but
that
that's
all
measurable
right
so
even.
I
The
manual
stuff,
but
I
guarantee
you
this
and
I,
see
this
with
dbas
all
the
time:
production,
DBA
SLA
for
a
push
to
production,
typically
72
hours,
the
average
time
that
they
use
you
go
back
to
ticketing
system
71
and
a
half
hours.
It's
like
it's
like
my
12
year
old
son.
It's
like
a
last
possible
moment.
It's
but
it's
human
nature
and
that
ain't
good.
That
ain't
good.
E
Oh,
this
was
a
lively
discussion
and
we
are
over
time.
Apologies
for
that
I
was
not
tracking
quickly.
I
think
we'll
probably
keep
hammering
on
this
drop
comments
in
there,
for
or
you
know,
however,
you
want
to
participate
in
the
supply
chain.
Fraternity
model
document
keep
thinking
the
good
thoughts
and
let's
keep
this
conversation
going.