►
From YouTube: GitOps - Operations by Pull Request [B] - Alexis Richardson, Weaveworks & William Denniss, Google
Description
GitOps - Operations by Pull Request [B] - Alexis Richardson, Weaveworks & William Denniss, Google
GitOps is the latest exciting evolution in empowering developers to do operations and CICD. Imagine describing your entire infra in Git declaratively and then continually using that to verify your state. Well, with Kubernetes, and tools like Terraform, and Ansible, you can. We've taken this forward by adding continuous diffs and alerting - and even some of our observability stack itself. An introduction is here: https://www.weave.works/blog/gitops-operations-by-pull-request
William (Google PM) and Alexis (Weaveworks, CNCF) will talk about how we jointly developed this pattern based around our own use cases. We shall make reference to other companies using the approach like Github and Atlassian. This is NOT a product pitch - we are going to teach you the PATTERNS.
About William Denniss
William is a Product Manager at Google and works on Google Cloud and Kubernetes. He has a passion for open source and open standards, is the author of several IETF Internet-Drafts including OAuth 2.0 for Native Apps, and founded AppAuth, the leading open source OAuth client for native apps.
About Alexis Richardson
Alexis is the co-founder and CEO of Weaveworks. He is also the chairman of the TOC for CNCF, and the co-founder of the Coed:Code meet ups.
Previously he was at Pivotal, as head of products for Spring, RabbitMQ, Redis, Apache Tomcat and vFabric. Alexis was responsible for resetting the product direction of Spring and transitioning the vFabric business from VMware.
Alexis co-founded RabbitMQ, and was CEO of the Rabbit company acquired by VMware in 2010, where he worked on numerous cloud platforms. Rumours persist that he co-founded several other software companies including Cohesive Networks, after a career as a prop trader in fixed income derivatives, and a misspent youth studying and teaching mathematical logic.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
B
B
We
love
to
know
communities
conformance
okay,
so
we're
going
to
talk
today
about
how
to
go
faster,
using
modern
practices
which
are,
of
course,
variations
on
older
practices,
and
we
calisca
tops
to
kind
of
get
across
the
concept
you're
driving
operations
from
get
and
I'm
going.
To
give
you
an
example
to
go
first,
this
is
a
company
called
Cordoba
which
uses
our
products
and
they
have
a
team
of
a
thing
15
or
20
people,
and
they
are
based
in
San
Francisco.
They
have
a
website
that
does
internationalization
and
localization.
B
B
So,
if
you
think
about
it,
when
you
start
to
deploy
more
than
once
a
day,
you
get
a
feeling
of
empowerment
and
control.
You're
no
longer
doing
things
manually,
watching
things
having
to
fix
them,
you're
actually
moving
with
confidence,
and
you
stop
worrying
I
asked
them.
You
know,
what's
the
real
benefit
here,
developers
stop
having
to
think
about
this
stuff,
it
frees
up
our
minds
to
concentrate
on
our
business
that
we
want
to
spend
real
developer
time
on
and
there's
this
Basin
things
like
machine
learning.
B
So
this
pattern
is
called
git
ops,
which
builds
on
DevOps
practices
and
at
the
core
of
this
is
the
idea
that
git
is
a
single
source
of
truth
for
your
whole
system.
This
is
a
really
profound
statement.
It's
not
not
been
easy
to
do
this
until
fairly
recently,
because
we've
lacked
enough
declarative
infrastructure
that
allows
us
to
describe
our
whole
system
and
then
store
it
in
gear,
but,
taken
to
this
extreme
once
you've
got
the
whole
system
describing
it.
B
You
can
make
all
your
changes
to
your
running
system
through
gate,
which
essentially
means
you
no
longer
use
things
like
cube
couple,
but
you
start
making
pull
requests
instead,
and
that
has
the
additional
benefit
that
any
developer
who
understands
how
to
do.
A
pull
request
can
also
make
operational
changes
which
lowers
the
cost
of
entry,
which
is
amazing,
plus
get
is
an
amazing
tool
for
working
in
teams.
You
get
an
audit
trail.
You
get
comments.
You
can
see
why
people
did
things
you
can
go
back
and
forward
in
time.
B
You
shouldn't
be
rolling
your
own
stuff
err.
You
should
be
using
a
tool
that
works
and
we
found
it.
We
use
it
ourselves
inside
we've
works
to
run
our
own
systems.
We
could
do
things
like
blow
up
our
entire
system
and
recover
in
minutes
just
by
using
this
technique.
So
we'll
talk
about,
but
just
to
remind
you
a
lot
of
what
we're
showing
you
is
an
evolution
of
what
people
already
do
and
have
been
calling
adopts
for
nearly
ten
years
and
here's
a
chart
I
found
on
the
Internet
woman
called
Helen.
B
It's
almost
like
you
know
if
you
you're
outside
the
club
until
you've
figured
out
what
the
llamo
is.
So
this
is
a
config
description
of
the
kubernetes
app
here,
and
the
key
point
is
that
it's
a
set
of
statements
that
you
can
verify
and
you
can
use
it
to
reproduce
a
system
from
that
set
of
statements.
It's
different
from
a
set
of
instructions.
B
So
we
use
declarative
infrastructure
up
the
wazoo,
Cuba
Nettie's
docker,
terraform
ansible,
and
our
entire
system
in
weave
cloud,
which
is
a
SAS
program,
stores,
monitoring
rules,
config
dashboards
and
the
code
all
together
and
get
with
the
full
audit
trail
around
it.
The
developers
can
contract
and
then
we
can
observe
our
running
system
and
compare
it
with
what's
written
down
and
get,
and
we
have
diff
tools
which
are
led
us
when
we
see
when
differences
occur
that
are
not
picked
up
by
the
kubernetes
Orchestrator.
B
B
So,
just
to
recap,
get
is
a
source
of
truth.
The
whole
system
you
can
observe
the
run,
the
running
system,
which
is
different
potentially
from
the
desired
state.
If
it
is
different,
you
need
to
be
told,
because
that
might
be
something
you
need
to
fix
and
when
you
do
want
to
force
convergence,
you
can
make
a
pull
request.
Well,
sometimes
the
Kuban
Eddie's
will
pick
it
up
for
you
and
orchestrate
it
away,
but
sometimes
it
won't
and
we
respond
to
anything
from
small
changes
to
full-on
crises.
This
way.
B
B
This
is
an
era
where
the
Sun
confusion,
the
key
point
about
automating
your
deployments
is
everything
has
to
be
joined
up,
and
that
can
mean
that
usually
means
build
and
deployment
and
release
automation,
but
get
as
the
desired
source
of
truth.
That
is
what
I
joined.
That
PowerPoint
is
William.
I,
take
over
sure.
A
So,
let's
talk
about
how
we
actually
join
up
these
components
into
a
pipeline.
Basically,
the
deployments
are
actually
controlled
using
the
operator
pattern.
So
you've
probably
seen
these
pattern
a
lot
in
committees.
The
operator
is
what
takes
the
desired
state
that
you
declare
and
give
Vanity's
and
turn
it
into
the
state
that
it
can
then
observe.
So
one
common
example
is
the
deployment
object
in
kubernetes.
You
will
declare
that
I
have
a
container
and
I
want
to
have
a
hundred
replicas
of
that
container.
That's
the
declared
state.
A
The
operator
is
then
responsible
of
driving
the
observe
state
in
the
cluster
towards
that
state
that
you
gave
it.
So
if
there
was
only
one
and
you-
and
you
said
it
should
be
one
hundred
it's
responsible
for
making
that
true.
Equally,
if
you're
sound,
asleep
and
you
know
a
bunch
of
nodes
disappear
and
that
replica
count
drops
down,
the
the
observe
state
is
now
not
equal
to
your
declared
state,
and
so
the
operator
is
then
driving
that
back
up
so
with
gaddafi.
A
We
have
actually
applied
the
same
pattern
used
throughout
communities
to
actually
drive
your
entire
conflict
from
get
to
be
the
the
observed
state
in
the
cluster.
So,
basically
to
make
this
work,
everything
has
to
be
in
get
all
the
config
should
be
treated
like
just
like
code
right
and
anything
that
is
not
recording.
Those
changes
in
github
is
a
is
effectively
harmful
to
the
system,
because
the
operator
is
driving
the
observe
state
towards
what
you've
described.
A
What
happens,
though,
if
you
are
starting
not
from
a
clean
slate
like
you
already
have
a
running
cluster
with,
and
it's
working
just
fine.
So
to
get
into
this
sort
of
happy
path.
You
need
to
have
all
your
config
under
get
management,
but
you
don't
have
that.
So
what
do
you
do?
Well,
fortunately,
cube
cube
control.
Has
this
export
flag
in
queue,
control
get,
and
you
can
just
go
through
one
by
one,
all
your
resources
and
extract
to
extract
that
out
of
the
cluster.
A
Be
careful,
though,
with
export,
because
it's
intentionally
dropping
some
of
the
e
fields
from
the
declarative,
config,
and
these
are
fields
that
it
thinks
are
not
required.
So
things
like
the
node
IP
address,
which
is
generally
not
used,
you
will
drop
that,
but
sometimes
it
gets
wrong.
Sometimes
it
drops
too
much.
Sometimes
it
misses
something.
So,
as
you
kind
of
do,
this
export
from
the
cluster
make
sure
you're
reviewing
the
the
content.
A
What
about
secrets,
though,
so
we're
saying
that
everything
should
be
in
get
every
single
change
should
be
stored
in
get,
but,
of
course,
secrets
are
not
very
good
to
keep
and
get
right,
because
they're
then
visible
to
people
that
that
maybe
they
should
be
visible
to
well.
Bitnami
have
a
pretty
cool
solution
to
this,
so
it's
called
sealed
secrets.
You
can
search
it
on
get
up,
it's
open
source
and
what
it
does.
A
Is
it
provisions
a
private
key
into
the
cluster
and
then
provides
a
public
key
that
the
developers
can
use
to
encrypt
the
secrets,
so
they
can
encrypt
those
secrets
and
install
them
and
get
just
like
any
other
configuration.
It
then
actually
uses
the
same
operator
pattern,
so
it
has
its
own
operator.
That
is
then
looking
at
these
in
these
encrypted
secrets
and
decrypting
the
with
its
private
key.
A
So
when
you
create
that
new
cluster,
you
bootstrap
it
with
that
one
private
key
and
then
all
the
other
secrets
come
in
and
it
just
means
that
you
can.
You
can
manage
all
the
other
secrets
through
the
the
git
ops,
workflow,
really
cool,
really
cool
project,
all
right.
So
what
is
this?
What
does
it
actually
look
like?
What
is
the
get
ops
repository
look
like
so
here
I'm
referring
to
the
config
repository.
A
This
is
a
separate
repository
for
your
code,
and
the
reason
we
keep
it
separate
is
that
you
likely
have
some
continuous
integration
triggering
on
the
code
and
you
don't
want
changes
to
the
config
triggering
an
image
because
they
kind
of
gets
a
bit
weird.
So
we
tend
to
recommend
one
repository
per
kind
of
logical
application
or
service,
and
what
I
mean
by
this
is
is
anything
that
that
is
sort
of
tightly
coupled
together.
So
could
be
a
bunch
of
deployments,
a
bunch
of
services,
any
other
kubja
Naser
resources.
A
That
sort
of
that
are
part
of
like
one
output,
typically
of
like
a
team
with
their
own
like
backwards
forwards,
compatibility
guarantees
of
whatever.
Then
what
you
can
do
is
use
a
separate
branch
for
each
environment.
So
you
know,
let's
say
you
might
have
three
staging
production
and
test
you
really
have
more.
A
But
the
idea
is
that
you
have
one
branch
per
environment
and
these
branches
map
to
a
kubernetes,
namespace
or
a
completely
standalone
kubernetes
cluster
and
the
reason
we
used
the
namespace
or
a
cluster
is
so
that
you
can
reuse
the
same
objects
from
staging
to
production.
So
if
you
have
a
deployment,
that's
called
foo
you,
you
don't
want
to
have
like
foo
staging
and
view
prod,
it's
much
simpler,
just
to
call
it
food
or
and
safer
to
be
honest
and
deploy
that
into
a
separate
namespace
or
cluster.
A
So
a
common
pattern
we
see
is
some
will
have
a
production
cluster
and
then
maybe
a
test
cluster
with
with
a
bunch
of
different
environments.
So
then
the
process
is
any
changes
that
you
want
to
make
you
first
make
them
in
get
up.
So
a
very
common
change
would
be
your
bumping.
The
version
of
an
image
right,
you've
just
released
some
code,
but
there
are
lots
of
other
changes
that
that
should
go
through
this
as
well.
In
fact,
all
changes
should
go
through
this
way,
but
things
like
health
checks,
changes
to
the
replica
counts.
A
Things
like
that
should
all
go
through
this
get
process
where
you
would
do
a
pull
request,
submit
it
to
say
the
staging
or
a
feature
branch
get
that
peer
reviewed,
submitted,
tested
and
then,
when
it's
ready
to
roll
it
out
to
production.
You
basic
just
merge
that
change
from
the
staging
branch
to
the
production
branch
and
the
benefit
of
doing
it.
This
way
is
that
even
though
you're
getting
code
review
on
this
final
merge,
you've
already
tested
that
exact
configuration
in
staging.
A
So
let's
say
you
might
have
like
fat-fingered
and
and
mistyped
the
number
of
replicas,
for
example.
Maybe
you
missed
a
zero?
You
don't
want
to
take
down
the
system,
or
maybe
you
added
a
zero
and
you're
about
to
run
up
a
really
big
bill.
All
these
kind
of
things
or
you
have
a
health
check.
That's
this
is
a
really
good
one.
If
you're,
if
your
health
check
is
buggy,
then
it's
going
to
be
declared
unhealthy
right
and
what
happens
if
everything
is
declared
unhealthy.
The
whole
system
goes
down.
A
So
that's
why
you
want
to
deploy
these
changes
first
to
staging
and
then
rolling
it
out
should
just
be
a
merge,
and
at
that
point
you
should
know
that
the
changes
are
fairly
safe.
Of
course,
there
is
one
cache
here,
and
that
is
what
happens
if
you
have
staging
only
changes.
So
at
the
very
beginning,
as
you're
saying
this
up
or
or
perhaps
you
know
throughout
the
lifecycle,
you
may
need
to
make
changes
to
staging.
In
that
case,
it's
a
little
bit
tricky.
A
There's
I,
don't
think
there's
a
really
good
analogy
here,
but
you
just
have
to
make
the
changes
in
the
staging
branch
and
you
can
use
this
get
merged
a
sales
strategy
to
effectively
say
that
you
have
merged
the
change
without
actually
applying
the
change,
and
it
just
means
that
the
next
time
someone
does
emerge.
It's
not
gonna
point
that
change
that
was
meant
to
be
staging
only
finally,
definitely
use
features
like
protected
branches
to
ensure
that
all
those
code
reviews
are
enforced
on
the
production
branch
alright.
A
So
this
is
what
the
pipeline
looks
like
and
I
kind
of
wish
I
had
a
laser
pointer
here,
but
effectively
the
the
top
pipeline
is
your
typical
part
of
CI
pipeline.
The
developer
commits
a
change
goes
in
the
git
repository
for
the
code.
The
container
builder
kicks
in
builds
out
image,
uploads
it
to
a
container
registry.
Then
we
have
that
an
extra
step
at
the
end
now
Alexis
talks
a
lot
about
this,
but
a
lot
of
people.
Basically
at
this
point
just
jam
in
their
continuous
deployment.
A
It's
not
so
great
doing
it
that
way.
What
we
do
with
get
ops
instead
is
is
rather
than
deploying
that
change
we
edit
the
staging
configuration
and
change
the
image
to
be
the
one
that
was
just
created.
So
here
you
have
like
an
automatic
push
to
staging
effect
to
it
once
the
and
that's
that
config
up
data
at
the
end
there
once
that
change
has
been
committed
in
to
get
the
operator
the
deploy
operator
that
I
mentioned
earlier,
kicks
in
and
notices
a
wait,
a
minute.
The
observe
state
in
git
is
now
different.
A
Sorry,
the
observe
state
in
the
cluster
is
now
different
to
the
desired
state
again
and
we'll
deploy
that
staging
change
automatically.
Typically
prod
isn't
set
up
on
the
same
kind
of
automatic
push.
So
then
you
have
an
operator
here,
user,
two
at
the
bottom,
doing
our
merge
from
staging
to
production.
It
goes
through
a
code
review
and
once
it's
accepted,
it
goes
into
the
git
config
in
the
production
branch,
at
which
time
the
deployment
operator
for
the
production
cluster
will
kick
in
and
do
the
same,
rollout
and
Alexis
back
to
you.
Thank.
A
B
If
you
don't
have
a
way
to
check,
but
what's
happened
is
the
right
thing
and
if
it's
gone
wrong
to
be
able
to
take
action
to
remedy
it,
so
observability
is
essentially
tied
to
get
ops,
because
it's
the
property
of
a
system
that
you
can
check
where
your
poor
requests
are
happening
correctly
or
incorrectly
at
a
service
level,
meaning
anything
from
user
experience,
which
is
the
most
important
thing.
If,
if
you
have
users
down
to
deep
Diagnostics.
B
So
you've
got
to
integrate
your
gear
ops
pipeline
with
your
tools
to
observe
things,
which
means,
if
you
do
a
service
push.
You
want
to
see
the
error
rate
on
your
service,
for
example,
as
it's
happening,
and
the
more
that
you
adopt
policies
like
Canaries
stage
deployments
the
more
important
it
becomes
to
see
the
stages
impact
on
the
system
early,
so
that
you
can
make
the
right
decisions-
and
this
is
how
you
gain
confidence
in
your
system
and,
ultimately
how
you
get
to
go
faster
and
faster
to
having
many
deployments
a
day.
B
So
it's
also
the
ability
to
think
in
terms
of
holistic.
Being
like
a
doctor,
it's
my
system,
happy!
Is
it
about
to
become
man?
Happy
is
something
licking
smelling
wrong?
Do
it
cuz?
You
really
want
to
stop
problems
happening
before
they
become
real
issues
and
then,
of
course,
obviously
fixing
them
if
they
are
so
we're.
Starting
to
see,
companies
like
this
is
from
lyft
Matt
Klein.
The
author
of
envoi
has
been
showing
this.
B
This
chart
is
a
concept
of
a
dashboard
that
unites
elements
of
observability
together
metrics,
like
error
rates,
request
rates,
latency
bars
with
events
like
a
deployment
has
just
occurred.
The
canary
went
from
50%
to
70%,
which
means
that
somebody
can
focus
on
the
services
health
as
the
deployment
is
occurring
and
as
this
is
a
short
talk,
we
won't
go
too
deep,
but
there
are
blogs
about
this.
That
you
could
read
and
control
is
the
counterpart
of
observability
and
it
basically
goes
back
to
that
principle
of
get
off
switches.
B
Do
things
through
get
to
make
sure
that
you've
got
consistent
and
correct
running
system
relative
to
your
desired
state,
meaning
if
you
change
your
desired
state
and
get
then
of
course,
your
system
must
be
updated
as
well,
rather
than
messing
around
the
cubicle.
If
you
can
avoid
it
and
the
more
things
that
we
figure
out
how
to
describe
using
there's
good
old
yeah
malls,
the
better
so
things
like
security
policy,
application
policy,
monitoring,
other
properties
and
then
good
old
orchestration
is
a
model
of
control.
B
What
kubernetes
is
actually
doing
for
you
and
other
orchestrators
like
swarm,
is
correcting
the
running
state.
Relative
to
the
desired
state
for
you
and
differencing,
we
mentioned
a
few
times.
These
are
somewhat
crude,
but
important
tools
for
alerting
you
when
something
is
actually
different
between
the
observe
state
and
the
desired
so
that
you
can
can
fix
it,
and
convergence
is
the
way
you
do
that.
So
this
is
a
picture
of
this
desired
state,
observe
state,
desired
state,
observe
state,
so
control
means
convergence
and
here's
a
picture
of
that
diff
tool
firing.
B
B
And
having
mentioned
services
a
few
times,
I
wanted
to
just
emphasize
that
this
implies
a
certain
life
cycle.
So
if
you
want
to
know
what
you're
doing
this
is
a
picture
of
it,
you
start
and
get
you
go
about
it
kubernetes,
and
then
you
use
your
monitoring
and
tracing
tools
to
see
what's
going
on,
to
construct,
observe
state,
and
then
you
have
a
control
loop
to
finish
it
off,
which
may
update
get
there.
You
go.
B
That
is
what
operations
is
in
the
Kuban
areas
world
and
that
gita'
world,
and
for
those
of
you
who
are
fond
like
I,
am
of
acronyms
there's,
a
famous
acronym,
called
Buddha,
which
came
up
in
the
60s,
observe
orient,
decide
act
and
it's
been
used
to
describe
many
things.
It
was
used
by
coda,
Hale,
famously
about
seven
or
eight
years
ago,
to
describe
how
Yama,
so
it
was.
Yes,
it
was
Yama,
do
their
operations,
so
inspired
by
that
I've
called
this.
The
rue
de
loop,
because
it's
release
driven
release,
observe
orient,
decide
and
act.
A
Right
so,
just
to
summarize
the
three
core
principles
of
get
ops
that
you
can
hopefully
take
away
and
if
you're
on
Twitter,
by
the
way,
we've
got
the
get-ups
hash
tag.
So
a
bunch
of
this
content
is
already
up
there,
but
the
three
core
principles
are
use:
declarative
conflict
to
define
your
application
and
the
services.
All
changes
need
to
go
through
a
good
review
process.
No
one
should
be
using
tube
control
directly.
So
that
means,
even
if
you
have
a
production
emergency,
don't
be
using
cube
control.
B
B
It
just
means
that
the
way
that
we
deal
with
systems
is
going
to
look
more
like
how
developers
think
and
do
things
as
we
as
we
get
closer
and
closer
and
closer
to
you,
know
the
dream
presented
eg
by
Brendon
yesterday
of
something
that's
so
simple
that
a
young
person
could
come
in
and
build
a
sophisticated
that
really
in
very
little
time
and
and
get-ups
is
just
taking
that
to
its
logical
conclusion
for
operations
using
cloud
native
tools,
cool
okay.
So
do
you
want
to
ask
us
some
some
questions
so.
A
C
Awesome
I
really
like
this
whole
thing
and
I
think
the
devil
is
in
the
details.
I
guess
with
this.
It
really
makes
sense
for
kubernetes
deployments,
but
when
we
talk
about
other
things
like
like
different
things
like
dashboards
and
criminais
and
like
console
configs
and
all
these
different
things
like
do
you
end
up
writing
your
own
operators
that
can
read
what
is
happening
and
then
dip
in
and
then
apply
it
to
production.
B
Say
that
please
so
I
can
describe
a
real
system,
which
is
we've
cloud,
which
is
relatively
complete
interpretation
of
the
get-ups
idea.
So
we
have
three
tools
that
I
mentioned
cubed.
If
ansible
different
aridif
ansible
diff
is
just
a
wraparound
ansible
zone,
diff
capability,
tara
diff,
is
something
which
tells
you
in
terraform
is
out
of
sync,
and
we
provision
on
Amazon
using
terraform.
We've
found
that
to
be
very
effective
and
cubed.
B
If
it's,
the
one
I
showed
you
an
example
of
the
one
that
we
don't
have
yet
described,
is
the
app
itself
and
I
think
that's
kind
of
next.
So
there
is
a
cig,
it's
called
app
dev
the
app
dev
cig,
and
we
would
very
much
like
people
who
care
about
declarative
definitions
of
applications
to
be
part
of
that.
So
please.
A
D
E
A
question
about
sort
of
the
multi
branch
strategy
and
actually
things
of
the
vergine
right
in
the
terraform
world,
there's
a
tool
that
rapper
called
Terra
Grande,
basically
delegates
the
terraform
and
you
just
define
the
variables,
in
other
words,
just
the
different
configurations
for
the
different
environments.
So
how
do
you
manage
in
this
thing
that
I
don't
actually
have
all
these
different
yamo
files
and
complete
chaos
versus
I?
Have
a
single
source
of
truth
for
I
want
to
deploy
this
and
just
make
these
changes
in
each
right?.
A
B
Don't
think
we're
using
get
releases
yeah
trying
to
think
of
some
good
examples.
I
mean
there
is
there
are
things
that
Cuban
Aires
is
not
aware
of,
like
machines
going
away
which
might
be
picked
up,
so
you
know
we're
two
cubed.
If
we
have
a
range
of
different
alerts,
yeah
see
me
later,
I'll
put
you
in
touch
with
engineers
and
they'll
give
you
all
of
them.
If
you
like
I.
A
Okay
rights
I
mean
I,
guess
one
example
could
be
someone
applied
manual
change
that
that's
a
sort
of
situation
like
what?
What
do
you
do
right
with
the
true
operator
pattern?
You
would
actually
just
blow
away
that
change
immediately,
but
I
think
you
might
have
had
a
situation
where
we're
all.
Actually
the
deployment
operator
will
observe
this
back.
That's
someone
modified
the
costume
manually
and
put
an
alert
through
on
slack
or
something
yeah.
B
I
mean
that's,
the
kind
of
classic
is
you've
got
somebody
I
mentioned
it
several
times,
it's
great
when
new
people
join
the
team
and
it's
really
easy
and
simple,
but
actually
also
new
people
can
join
the
team
and
do
something
which
is
unintended
and
those
that's
something.
They'll
get
picked
up
the
one
of
these
diff
tools
quite
quite
often.
Well,
somebody
goes
out
to
lunch
and
doesn't
someone
else
comes
in
and
doesn't
realize
that
something
has
changed,
but.
A
B
D
B
B
G
G
A
Right
yeah,
I
think
well,
one
advantage
of
having
everything
declaratively
defined
and
storing
it
is
that
it
should
be
easy
to
spin
up
a
new
cluster
and
and
try
that
out
with
the
new
version
of
kubernetes
yeah.
So
he
that's
one
benefit.
You
get
I
think
you're
right
that
that
we're
not
actually
solving
that
yet
I
mean
that's
a
problem.
So
there's.
B
If
you
go
to
the
website,
if
you
do
a
Google
search
for
get
ops,
you'll
find
some
blog
posts
which
refer
to
an
explanation
of
how
we've
works.
We've
cloud
solves
that
specific
problem
in
detail.
Okay,
but
it's
it's
it's
not
something.
I
would
necessarily
recommend
everybody.
Adopt
I
mean
it's
it's
a
system
that
we
built
for
us.
If
you
want
something,
that's
more
like
product
that
everyone
can
use
and
works
the
same
way,
etc.
B
A
A
Thanks
it
yeah,
that's
actually
an
excellent
point
like
anything
that
is
like
a
workaround.
This
good
thing
is
dangerous,
whether
it's
a
latest
tag
or
or
whether
your
doctor
and
me
just
like
pulling
down
like
I
heard,
someone
was
like
pulling
down
a
war
file
right
like
as
it's
brooding,
so
anything
like
that.
Just
breaks
them
all.
The
waiters
yeah,
hey.
H
A
B
H
A
A
config
so
again
this
is
kind
of
just
our
opinion,
but
because
they
continuous
integration
typically
gets
triggered
off
a
commit.
What
you
don't
want
is
to
have
like
a
code
change
which
triggers
CI,
which
then
updates
this
configure
for
staging
which
might
like
trigger
another
CI
or
you
have
to
if
to
make
sure
that
wouldn't
happen,
and
you
kind
of
get
like
a
weird
loop
or
you
have
the
way,
someone
just
changing
config
that
then
rebuilds
the
the
image
unnecessarily
so
I
think
you
can
get
it
to
work
that
way
but
yeah.
A
A
A
Would
be
in
a
config
repository
okay
with
the
way
we've
set
it
up,
but
you
know,
of
course,
if
you,
if
you
have
a
better
approach
by
all
means,
okay,
but
I-
think
yeah,
I
guess
in
my
head,
like
I,
could
never
get
around
the
fact
that
I
don't
want
the
replica
set
change
to
actually
trigger
an
image
rebuild
of
the
code.
I
mean
I.
Guess
you
can
do
that
and
then
just
sort
of
throw
away
the
image.