►
From YouTube: Pipelines and the Multiverse of Madness - Christian Hernandez & Hilliary Lipsig, Red Hat
Description
Pipelines and the Multiverse of Madness - Christian Hernandez & Hilliary Lipsig, Red Hat
Gone are the days of daisy chaining Jenkins Jobs together. With modern CI/CD tools pipelines are the new high ways. But unlike a driving a car, which you can only take down one road at a time. CI/CD pipleines have the power to run concurrent processes, across multiple lanes, to get to the same destination: Code in production, fast, reliably, and automatically. Join us for a review of Pipelines, CI/CD, and how to leverage tools to get jobs done with your GitOps workflows.
A
B
But-
and
you
know
I
deal
on
on
the
other
side
of
get
ops
right-
I
I
I'm
in
the
I'm
in
the
bu
and
I
always
receive
all
the
feedback
right
from
from
from
our
customers,
so
she
kind
of
deals
with
actually
implementing
a
lot
of
things
being
an
sre
for,
and
so
she
has
a
different
view
than
I
do
of
get
up
so
so
yeah
we're
both
cartoons
there.
So,
as
you
see,
yeah.
A
It's
not
just
the
like
the
way
we
implement
it,
but
then
like
supporting
like
the
infrastructure
that
has
the
implementations
on
it,
so
operating
git,
ops
and
using
git
ops
as
a
consumer.
It's
good
times,
folks,
all
right
so
brief
agenda
today,
which
is
a
lot
of
white
space
on
this
big
screen.
I'm
not
gonna
lie
we're
just
gonna.
A
Keep
this
all
pretty
high
level
conceptual
we're
gonna
talk
about
ci,
cd,
we're
going
to
talk
about
pipelines,
we're
going
to
talk
about
non-linear
pipelines
and
then
we're
going
to
tie
all
of
that
back
into
get
ops,
okay,
so
how
it
started
and
how
it's
going.
This
is
a
real
make
file
from
10
years
ago.
I
picked
this
one
because
it's
nice
and
clean
and
you
can
see
everything
and
I'm
borrowing
this
meme
format
right.
A
Usually
we
use
this,
how
it
started,
how
it's
going
and
we're
talking
like
things
have
gone
terribly.
The
good
news
is
with
like
git,
ops
and
modern
ci
cd
things
are
actually
going
pretty
great,
so
you
know
who
here
has
ever
manually
deployed
code
to
production
right
yeah?
What
did
you
do
like
an
ftp
right
right?
You
something
like
that?
C
A
A
A
So
a
lot
of
the
the
driver
behind
this
was
this
methodology
right,
ci,
cd,
continuous
integration,
continuous
delivery
and
the
idea
of
being
continuous
means
we're
going
to
do
it
faster.
It
also
means
we're
going
to
fail
a
lot
harder,
so
one
of
the
things
that
immediately
got
kind
of
codified
into
this
was
that
testing
needed
to
be
part
of
the
way
you
delivered
code
test,
driven
development
like
ci,
cd
and
test
driven
development
came
up
around
the
same
time.
I
spent
11
years
in
quality
engineering.
A
I
promise
you
that's
how
it
went
so
we
had
a
lot
of
those
things
happening
in
in
concurrence
and
so
a
lot
of
original
cicd
pipelines
that
I
worked
on.
It
was
not
just.
How
are
we
going
to
deliver
our
code
consistently,
but
how
are
we
going
to
guarantee
our
code,
and
that
was
now
we
have
infrastructure,
as
code
is
one
way
of
doing
it.
We
have,
of
course,
the
unit
tests,
the
integration
tests.
You
have
various
different
layers
of
things
to
deliver
code,
so
I've
actually
like
logically
divided
this
by
color
right.
A
A
So
in
the
first
one
I
have
you
have
some
concurrent
processes
right
like
you
might
do
a
build
and
then
especially
like
cloud
native
where
you
can
spin
up
multiple
instances
of
things
with
pause,
you
might
be
doing
some
tests.
You
might
be
doing
some
security
verifications
all
of
these
things
by
the
way
apply
to
non-cloud
native
deployments,
because
that
is
really
where
a
lot
of
our
our
reasons
for
best
practices
came
from.
It
was
when
services
were
monolithic
and
they
weren't
broken
up
into
microservices
and
they
weren't
necessarily
as
easily
observable
again.
A
So
this
is,
I
love
observability
you're
going
to
hear
me
say
it
a
lot
just
get
with
it.
So
here
here
are
two
examples
of
like
real
ci
cd
pipelines
that
I
have
built
in
the
past
right.
So
we're
doing
some
concurrent
things
where
we're
building
we've
got
some
security
checks
that
then
check
off
at
a
separate
workflow,
and
then
we've
got
our
test
release
and
deploy
to
stage
now.
A
All
of
those
things
must
be
true
and
correct
in
order
for
you
to
progress
past
stage,
that's
why
that
ends
there,
I'm
not
like
the
best
visual
thinker,
so
my
diagrams
suck
and
then
once
you
go
to
stage
you
test
again.
This
is
an
interesting
thing
because
with
the
infrastructure
as
code
with
the
pretty
much
guarantees,
we
have
that
things
will
be
the
way
we
expect
them.
A
That
last
test
thing
is
starting
to
get
dropped
off
a
little
bit
and
and
the
last
just
like
kind
of
the
one
cd,
but
there
really
is
like
cd
to
stage
and
then
cd
like
for
real
with
the
delivery
versus
deployment
at
lasting,
had
a
great
article
on
that
a
few
years
ago.
I
don't
know
if
it's
still
up
and
so
you're
going
to
have
those
steps,
and
so
some
some
times
you're
going
to
see
it
with
or
without
the
second
deployment
model.
I
always
have
that
last
set
of
that
last
set
of
tests.
A
That's
your
just
in
case!
That's
your
oh
shoot
button.
If
you
will
like
that,
that's
the
fail-safe
method
and
you're
going
to
have
something
like
a
notification
right.
We've
we've
passed
our
last
round
of
checks
and
things
are
going
to
go
off.
So
that's
an
example.
The
next
one.
This
is
a
bit
more
of
a
micro
service
designed
kind
of
pipeline
like
this
is
again
another
real
pipeline.
So
if
you
would
imagine
something
like
a
react,
stack
right
react
native
react
just
for
web.
A
So
moving
on
so
in
these
models
I
discuss,
you
know
subtasks
right
and
this
is
you
see
these
a
lot
sub
tasks?
What
is
that
this
is
this
thing
that
you're
doing
it's
part
of
your
task
in
your
pipeline
and
it
may
or
may
not
be
blocking
right.
A
really
good
example
of
that
is
a
notification
subtask.
If
I
am
notifying
you
that
something
has
happened
or
something
has
started,
and
let's
say
I'm
doing
it
through
a
slack
integration
right.
A
If
your
slap
slack
web
hooks
are
failing
because
slack
has
gone
down,
you
probably
don't
want
to
stop
your
entire
pipeline
realistically
you're
not
going
to
stop
your
entire
pipeline,
you
can
add
some
other
fail-safes
or
even
like
attach
pager
duty
alerts.
If
you
want
to
you
know
some
parts
of
your
pipeline
failing,
but
I
don't
want
my
notifications,
failing
blocking
the
continuous
you
know,
part
of
continuous,
so
it's
kind
of
you
know
then
there's
other
things
that
would
be
very
blocking
right.
A
Let's
say
your
unit
tests
fail
well
yeah,
it's
you're
going
to
stop
there.
Realistically,
that's
a
good
place
to
stop.
If
your
unit
tests
are
failing.
You
know,
ideally,
ideally
I
mean
I've
seen
it
done
the
other
way,
just
not
to
you
know
much
success,
but
you
know
you're
gonna,
probably
like
we
showed
security
text
on
the
last
page.
You
know
those
are
types
of
things
that
you
could
run
them
as
subtasks,
or
you
know
a
pipeline
of
subtasks,
but
realistically
that
stuff
fails
you're
not
going
to
progress.
A
Your
code
you're
not
going
to
progress
your
deployment,
so
there's
lots
of
different
reasons
why
something
may
or
may
not.
It
also
might
be
about
your
maturity
model
right
if
you're
not
actually
going
to
production
right,
you're,
new
you're,
just
starting
off
something
brand
new,
it's
very
like
in
the
moment.
You
can
go
ahead
and
make
things
that
should
be
blocking
non-blocking
and
then
update
it
as
you
go
right,
get
more
mature,
get
ready,
but
the
point
of
continuous
integration
again
is
that
it
is
fast.
A
So
if
you
want
something
to
be
blocking
your
non-blocking
maturity
model,
the
maturity
of
your
application,
the
maturity
of
what
you're
doing
these
can
be
factors
as
well,
and
so
you're
basically
should
be
continuously
evaluating
it.
You're
going
to
want
to
manage
your
ci
cd
with
a
life
cycle
really
similar
to
a
life
cycle
of
an
application
right.
The
software
development
life
cycle
applies
everywhere.
A
A
I
showed
you
that
slide
with
like
two
different
pipeline
models
and
one
of
them
was
kind
of
like
there
was
a
lot
more
to
it
and
the
other
one's
kind
of
broken
out
more
nicely.
And
basically
the
advice
is
find
logical,
breakout
spaces,
where
you
can
do
that,
because
it
makes
your
overall
process
of
cacd
it's
easier
to
debug
when
something
goes
wrong,
it's
easier
to
follow
it's
easier
to
debug
you're,
going
to
be
able
to
observe
it
a
little
bit
better.
A
If
you
break
it
out
into
more
logical
spaces-
and
I
mean
feel
free
to
experiment
with
that
and
try
and
fail
right
at
the
end
of
the
day,
there's
like
copy
paste
for
a
reason
you
could
you
can
just
move
stuff
around
and
so
strive
for
dry
code,
but,
like
don't
kill
yourself
with
that,
and
you
know
really
at
the
end
of
the
day,
what
are
we
doing
when
we're
doing
ci
cd
right?
Where
did
we
start?
We
started
with
bash
right.
It
was
bash
and
now
we've
basically
created
tools
to
orchestrate
bash.
A
I
was
explaining
this
to
one
of
the
junior
engineers
on
our
team,
and
I
was
like
look
everything
we
are
doing
in
python.
You
could
do
it
in
bash.
It
just
wouldn't
be
object,
oriented
it
wouldn't
be
as
dry,
but
at
the
end
of
the
day,
we're
still
just
doing
bash
commands
we're
just
we've
made
them
faster.
We've
made
them
more
reliable.
We've
made
them
more
observable
and
we've
kind
of
regulated
right
that
environment.
A
That
infrastructure
is
as
codepiece
really
does
that
as
well
so
dry
code,
where
possible,
but
don't
don't
kill
yourself,
crawl,
walk
run.
Don't
be
afraid
to
mix
and
match
the
right
tools
right
so,
like
we
say,
oh
cicd,
what
do
I?
What
am
I
going
to
use?
What
am
I
going
to
do?
My
team
uses
a
mix
of
git
lab
and
jenkins
and
you're
like
whoa.
Why
well
there's
reasons
why
right
you?
Basically,
I
said
this
to
chris
earlier:
choose
your
technical
debt
wisely
right!
B
A
So
choose
your
technical
debt
wisely
you're
going
to
get
some
guaranteed
everything
you
pick
has
technical
debt.
You
have
to
decide
what
is
the
most
sustainable
technical
debt
for
you
and
you
really
need
to
be
mindful
of
that.
Really,
I'm
like
a
pessimist.
Honestly.
I've
seen
everything
that
can
go
wrong,
go
wrong.
So
I'm
never
like.
Oh
yeah
choose
the
advantages,
no
choose
your
debt
because
you
will
pay
it
every
single
time
and
again,
iterate
start
with
good
get
your
mvp
work.
Your
way
up
to
better
wow
we're
doing
great
on
time.
B
C
B
A
Right,
so
this
is
a
real
story
from
about
10
years
ago,
before
all
of
this
had
really
matured
right.
This
has
come
so
far.
Ci
cd
has
come
so
far.
We
have
tools
to
choose
from.
We
did
not
used
to
have
tools
to
choose
from.
We
were
like
daisy,
chaining,
jenkins
jobs
and,
like
praying
so
everybody
anybody
daisy
chain,
jenkins
jobs,
yeah,
yeah,
okay,
so
a
lot
of
you
make
me
feel
very
old
and
I'm
not
okay
with
that.
You
may
leave
so
way
way
back
in
days
of
yore.
We
did
not.
A
We
did
not
have
some
of
the
maturity
right.
We
had
to
learn
very
painful
lessons,
which
is
why
we
have
today's
maturity
and
one
such
example
of
this,
and
why
infrastructure
is
code
and
git,
ops
and
so
forth
is
so
important
is
real
story.
We
might
my
company
right.
We
had
this
big
monolithic
application.
We
added
this
new
feature.
I
was
the
head
of
quality
assurance.
I
had
spent
like
tons
of
time
testing
this
six
ways
from
sunday
it
worked.
A
So
it
was
a
very
memory
intensive
thing
and
somebody
not
me
because
I
was
not
the
sysadmin
esri
is
not
your
cis
admin
by
the
way,
just
in
case
that
wasn't
clear,
throwing
that
out
there.
I
say
that
every
time
I
get
a
chance
so
not
me,
because
I
was
not
the
sysadmin.
Somebody
had
decided
that
the
staging
cluster
or
agent
cluster
g,
so
you
can
see
how
long
I've
been.
A
A
A
Had
made
that
decision
that
it
was
okay
for
the
environments
not
to
match,
and
there
was
no
r
back
right.
Vms
did
not
have
our
back
no
role
based
access,
no
identity,
access
management,
none
of
that
existed.
It
was
just
one
person
had
the
password,
so
we
had
to
wait
for
this
one
person
to
go
in
and
figure
out
why
this
you
know
much
awaited
very
highly
lauded.
You
know,
feature
didn't
work
and
he,
finally,
like
he's,
done
a
dungeon
digging
he.
Finally,
he
finally
finds
that
right
and
he's
like.
A
C
A
C
D
C
A
To
turn
off
production
yeah,
that
was
not
a
good
day
folks,
it
was
not
a
good
day,
so
you
know
these
these
kind
of
these
lessons
that
we've
learned
these
these
stories
right.
A
lot
of
this
applies
to
bare
metal.
A
lot
of
it
applies
to
vms,
and
now,
of
course,
it
applies
in
the
cloud
native
world,
and
so
we
talk
about
these
best
practices
and
christians
about
to
go.
A
B
B
B
So
it's
not
going
to
be
as
technical,
maybe
as
you
guys
would
like.
If
you
want
to
chat
with
me
after
or
ask
questions
after
I
can
get
deeper
technical,
but
for
the
purposes
of
this
conversation,
I'm
going
to
keep
it
like.
Really
high
level
and
kind
of
like
the
three
ways,
I've
actually
seen
get
ops
tie
into
ci
cd,
so
but
first
the
how
does
get
ups
look
like
in
your
pipelines
from
a
high
level
right
traditionally
long
time
ago,
ci
cd
was
considered
like
a
thing.
B
One
thing
right
and
a
lot
of
it
has
to
do
with
jenkins
right.
I
mean
it's
kind
of
like
it's
the
tool
we
had
right.
We
had
the
hammer
and
everything
looked
like
a
nail,
and
so
we
jenkins
did
everything
with
ci
cd.
So
jenkins
did
the
ci
part
the
it
also
did
the
delivery
part
and
but
with
with
get
ops,
ci
cd
are
really
kind
of
decoupled
right
because
they
have
their
individual,
the
individual.
B
That
happens,
whereas
get
ops
is
really
asynchronous
right.
If
you
think
about
it,
like
like
flux
or
if
you're
using
argo,
there's
a
reconciliation,
loop
yeah,
you
can
use
web
hooks
to
kind
of
run.
The
great
conciliation
maybe
at
commit
time,
but
it's
really
like
it's
really
asynchronous
right,
especially
with
like
drift
detection
and
things
like
that.
You
know
you
have
to
wait
for
that.
A
reconciliation
loop
to
happen,
and
so
you
know
now
you
have.
You
know
this
idea
that
getups
controller
doesn't
necessarily
know
anything
about
your
ci
process.
B
Really,
because
now
what
you're
trying
to
do
is
you're
trying
to
integrate
a
synchronous
process
with
the
asynchronous
process
and
try
to
make
them
work
together
and
that's
kind
of
difficult
really
from
from
what
I've
seen
right.
So
I've
seen
three
solutions
out
there
right
and
if
my
did,
I
press,
maybe
oh
technically,.
B
B
Them
right,
like
this,
is
kind
of
fits
more
into
that
aspect
where
the
ci
tool,
like
basically
owns
the
whole
thing
and
and
really
get
ops,
is
kind
of
like
the
it's
kind
of
like
a
second
thought,
it's
kind
of
like
the
the
last
thing,
and
they
don't
really
know
about
each
other,
and
this
is
kind
of
where
I
see
people
start,
because
it's
they
use
the
floating
tag
right
aspect
of
it.
Ci
does
the
whole
thing
when
it's
ready
to
deploy
it
just
tags
that
image
to
dev
wait.
B
B
Yeah
and
yeah,
and
also
there's
security
implications,
and
just
because
it's
like
it
because
get
ops,
you
need
to
have
your
source
of
truth
right
and
a
floating
attack
isn't
really
a
source
of
truth,
because
someone
can
force
push
an
image.
You
know
a
bad
actor
can
can
change
an
image
if
you
need
to
roll
back,
there's
no
really
good
way
of
rolling
back
without
like
force,
pushing
a
change
so
so
yeah.
B
So,
like
you
know
the
ci
tags,
an
image
and
it
just
tags
the
image
and
just
assumes
that
the
cd
went
into
place,
but
this
is
kind
of
like
a
good
start
right,
so
it's
kind
of
like
a
where
people
start
and
kind
of,
because
it
feels
familiar
right.
I
see
them.
It
feels
familiar.
It's
like
okay,
it's
kind
of
like
you
know.
B
I
could
still
kind
of
use
my
ci
process
the
way
it
is
and
kind
of
just
put
like
that,
get
off
thing
at
the
end
and
then
and
then
this
this
next
one
is
kind
of
like
the
mid
tier,
and
it's
probably
like
what
the
most.
What
I
see
most
people
use
read
how
we
use
argo
city.
So
if
the
flux
people,
if
you
see
something
different
I'd,
love
to
talk
to
you
guys,
so
it's
essentially
the
ci
does
the
build
and
get
ops?
B
Does
the
the
cd
part
the
get
ups
controller,
whatever
it's
continuous
delivery
or
deployment,
the
ci
builds
the
application
just
like
it
normally
would
do
and
the
galaxy
control
actually
does
that
deployment
and
it
handles
and
it
hands
back
the
process
back
to
ci
to
do
whatever
testing
it
needs
to
do
you
know
it's
it's
essentially,
you
know
we're
talking
about
multiple
pipelines
in
the
you
know
the
in
the
multiverse
of
madness.
Right
like
this
is
like
when
they
start
chaining
together.
B
It's,
like
you,
know,
hand
off,
you
know,
handoff
hand
back
on
to
the
ci
process
and
continue.
You
know
running.
However
many
pipelines
you
need
to
run
so
the
ci
process
is
more
hands-on
right.
So
it's
kind
of
like
the
ci
process
is
kind
of
managing
the
the
git
ops
tool
itself
and,
although
get
ops
is
involved
right,
this
is
a
it's
a
functioning
asynchronous
process,
because
it's
handing
back
it.
You
know
back
and
forth.
B
The
git
ops
is
kind
of
just
waiting
for
that
image,
change
to
happen,
and
then
it
does
something
and
it
has
it
back
off
and
it's
you
know
you
can
it's
more
a
little
bit
more
linear
right.
It's
it's!
You
know.
Even
though
get
ops
is
there
asynchronously,
it
functions
a
synchronous
process
so-
and
this
is
things
I've
seen,
which
is
pretty
cool
ci
triggered,
but
like
the
get
ops
is
complete
completely
owns
that
process
right
and
so
the
ci
builds
only.
B
It
only
builds
the
image
and
git
ops
does
like
all
the
rest.
Essentially,
ci
creates
prs
right,
so
it's
like
more
intimately
involved
with
that
get
ops
process.
It'll
create
a
pr.
You
know.
It'll
do
gating
that
way,
right
with
prs
and
since
get
ops
controllers
act
on
a
branch
or
tag
or
whatever
the
ci
process.
That's
how
it
interacts
right
with
git,
essentially
with
with
tag
with
sorry
with
prs
and
someone
you
know,
can
manually
either
manually,
approve
it
or
automatically
prove
it
after
test
run
or
something.
B
The
ci
and
and
cd
and
in
this
design,
they're
like
literally
decoupled
right,
and
so
this
is,
you
know,
kind
of
what
he
was
saying,
you're,
using
the
best
tool
for
the
job,
you're
kind
of
building
things
with
what
ci
is
and
you're
deploying
things
with
the
best
deployment
tool
and
since
they're
decoupled,
you
can
kind
of
you
know
pick
and
choose
what
the
best
tool
for
that
you
can
make
not
necessarily
best
tool
for
the
java.
I
guess
the
best
tech
debt
is.
B
And
the
kind
of
the
the
drawback
of
this
is
no
longer
like
a
linear,
simple
process
right.
You
know
this
is
the
multiverse
of
madness
sort
of
ideas.
You
know
you're,
you
know
you
have
different
pipelines
doing
different
things
and
it's
not
necessarily
a
linear
anymore.
So
you
kind
of
need
come
a
good
management
of
that,
so
so
yeah.
Then
this
is
kind
of
the
the
idea
that
kind
of
like
the
nirvana
but
there's
kind
of
just
some
drawbacks
to
it
and
so
so
kind
of
when
you're.
B
When
you're
you
know
integrating
get
ops
into
your
your
ci
cd
functions
right
it.
You
really
need
to
do
that
kind
of
a
paradigm
shift
right.
So,
first
of
all,
it's
ci
cd
should
probably
be
decoupled
in
get
ops.
Remember
get
ops
code
is
not
promoted,
you're,
not
promoting
code
anymore.
What
you're
doing
is
you're
promoting
manifests
really
at
that
point
right:
you're,
promoting
yaml
at
that
point,
right,
you're,
not
really
promoting
code
right,
so
you're,
just
building
an
image
and
then
you're
promoting
the
yaml
from
it.
B
That
then,
just
then
deploys
so
yeah
code
update
is
just
a
trigger
for
a
pipeline
right,
and
so
you
kind
of
just
have
to
keep
that
in
mind.
It's
some
of
these
paradigm
shifts.
You
know
we
come
from
like
a
background
where,
like
you,
are
promoting
code
you're
doing
a
get
clone
onto
either
a
production
vm
or
you
know,
git
clone
and
then
ftping
that
is
old
school,
but
now
you're,
just
you
know
just
promoting
ammo
through
the
process.
B
So
you
kind
of
have
to
change
your
mindset
there
and
do
that
paradigm.
Do
that
paradigm
shift
right,
you're
kind
of
you
know
the
a
lot
of
the
times
you
try
to
fit
a
square
peg
around
whole
right.
We
have
these
old
processes
and
you
know
we're
trying
to
like
see
how
that
fits
in
this
new
in
this
new
world.
B
But
you
know
now
now,
like
you
know,
cloud
native
architecture,
immutable
infrastructure,
right,
you're,
no
longer
building
vms,
to
run
code,
you're
building,
vms,
to
run
containers
and
then,
at
that
point
the
vm,
just
kind
of
all
it's
doing
is
just
running
containers.
So
you
know
you
have
to
take
care
of
less
and
have
the
kubernetes
take
care
of
it
more
so
anyways,
that's
it
right!
You
can
find
us
at
the
I.
I
think
we
messed
that
up
somewhere.
I.
B
A
Basically
that
yeah
I
mean
it
depends
on
your
tool
a
little
bit
when
we
daisy
change
jenkins
and
it's
still
like
a
valid
model.
You
could
just
be
like
okay,
when
I'm
done
here,
then
I'm
gonna
go
like
hit
this,
and
then
that
does
the
next
thing
the
next
pipeline,
the
next
job,
whatever
it
is,
that
your
tool
has
there's
gonna,
be
some
sort
of
like
now
that
I've
done
all
of
my
things,
there's
just
one
last
little
step
most
most
modern
cicd
tools
have
something
like
that
in
there.
A
Okay,
so
this
is
like
a
really
big
topic:
okay,
I'm
an
sre
right
so,
like
promotions
to
prod
are
still
part
of
my
life,
just
like
they
were
when
I
was
when
I
was
in
quality
right,
and
so
there
are
what
was
the
word?
I
just
used
it
earlier
models,
something
maturity
models,
yes,
maturity
models;
there
are
maturity,
models
right,
so
you're
going
to
want
to
have
minimum
quality
standards
right
you
have
certain
amount
of
tests
are
definitely
passing
before
certain
things
can
promote
this
far
right.
A
C
A
So
we're
just
that's
where
we
are
today,
so
let
me
give
you
a
really
good
example:
we
have
some
services
in
red
hat
and
they're.
What's
called
a
managed
service,
what
does
a
managed
service
mean?
A
managed
service
means
you
buy
it,
you
use
it
and
we
take
care
of
all
of
the
infrastructure
and
running
it
and
making
sure
it's
up
right,
I'm
an
sre
for
that.
That's
what
I
do.
I
manage
other
people's
stuff,
and
you
know
I
don't
know
what
they're
doing
with
it.
A
A
B
C
A
In
is
really
this
is
your
longest
lived
test
and
you
can
decide
it's
a
period
of
24
hours.
You
can
decide
what
this
is
a
test
that
will
actually
send
off
production-like
alerts
to
your
operations
team
right.
This
is
the
operations
team,
saying:
hey,
guess
what
your
stuff
operates
or
not
depending
right,
so
there's
lots
of
different
little
pieces
like
that
that
we
have
and
then
once
we've
had
enough
experience
with
how
it
fails
not
how
it
runs,
how
it
fails,
then
you
can
say:
okay,
we
know
how
this
fails.
A
A
Like
manual
check,
if
you
don't
want
deployments
happening
on
the
weekends
right,
if
you
don't
want
anything
to
happen
on
the
weekend,
you're
always
going
to
leave
that
off,
because
you're
not
going
to
try
to
remember
to
turn
it
off
for
the
weekend.
That's
that's!
That
is
too
many
layers
of
human
error
that
we're
going
to
go
through.