►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
GitOps Is Likely More Than You Think It Is - Cornelia Davis, Weaveworks
While the term “GitOps” has achieved almost mainstream use, confusion remains around what it is and the benefits it can bring. True, it involves continuous delivery (CD), but the way delivery is achieved as well as how CD interacts with workload operations, that is where GitOps gets really interesting. True, part of GitOps is leveraging Git abstractions like branches, pull requests and approval flows to manage the operational process, but GitOps is far more than the last step of CI. Instead it is about leveraging the most fundamental element of Kubernetes, reconciliation, both for CD and to link CD with the reconcilers that are automating workload operations. In this session Cornelia will cover the four key principles of GitOps, and she’ll demo those concepts with specific tools including Flux. She’ll also talk about use cases including cluster-api (CAPI).
https://sched.co/ekBA
A
A
I've
been
in
this
industry
for
a
long
time
about
30
years.
My
background
is
in
development.
I've
always
been
a
developer,
there's
a
little
hint
there.
That
says,
wasn't
ops,
but
I
can
definitely
consider
myself
a
an
ops
person
now
as
well
been
working
in
web
architecture,
so
gosh
for
well
over
15
years
cloud
native
for
nearly
a
decade,
although
I'll
definitely
concede
to
you
that
we
we
didn't
always
call
it
cloud
native.
We
certainly
didn't
call
it
cloud
native
a
decade
ago.
A
A
The
other
thing
that
I
mentioned
is
that
I'm
the
author
of
a
book
called
cloud
native
patterns
which
is
targeted
at
the
application
developer
and
architect
and
goes
over
all
of
the
patterns
that
are
required
to
make
its
software
work
really
well.
In
the
cloud
being
software
that
works
well
in
a
highly
distributed
highly
distributed
environment
that
is
experiencing
constant
change.
A
A
So,
given
that
we
want
to
achieve
certain
benefits,
the
question,
then,
is:
how
do
you
get
ops?
What
is
get
ops
and
how
do
we
do
it
now?
I'm
betting,
that
most
of
you
have,
if
an
impression
of
git
ops.
That
goes
something
like
this
we're
going
to
store
code
and
increasingly
configuration
in
something
like
it.
After
all,
git
is
the
first
part
of
the
name
get
offs
doing
things
in
git
is
going
to
cause
some
automation
to
happen
and
that
automation
is
going
to
configure
some
type
of
a
runtime
environment.
A
Now
git
ops,
of
course,
is
applicable
to
both
applications
that
are
running
in
that
runtime
environment,
but
you'll
see
as
we
go
along,
that
it's
also
very
applicable
to
the
configuration
of
that
runtime
environment
and
the
standing
up
and
maintain
maintenance
of
that
runtime
environment
as
well.
Oh
and,
of
course,
the
whole
one
of
the
whole
points
of
git
ops
is
that
the
primary
user
interface
for
this
get
ops
process
is
all
the
way
back.
On
the
left
hand,
side
of
this
diagram
on
the
get
side,
so
the
ux
is
actually
using
git.
A
Now
I
have
a
confession
to
make.
I
have
been
known
to
call
myself
a
propeller
head
before,
and
I
am
I
am
a
technologist
and
what
we're
going
to
do
in
the
next
30
minutes
or
so
is
I'm
not
going
to
present
to
you
the
principles
of
get
ops,
we're
gonna
derive
them,
so,
yes,
I'm
a
mathematician
at
heart
and
in
mathematics
we
do
derivations,
but
if
you're
not
a
mathematician,
please
stick
with
me
because
I
think
you'll
enjoy
this.
Even
if
we
don't,
you
know,
do
proofs
at
that
level.
A
A
A
They
might
be
two
different
individuals
and
they're
checking
their
code,
their
source
code
into
a
repository
they've
well
established
ci
practices
that
are
these
days
because
we're
operating
in
a
containerized
environment
generating
those
container
images
increasingly
we're
also
storing
the
application
configuration
in
git
and
then,
as
we
alluded
to
in
the
previous
slide,
we're
going
to
use
some
automation
to
draw
those
pieces
together
and
get
them
running
in
some
type
of
a
runtime
environment,
and
this
is
kubecon.
So
of
course,
we're
going
to
talk
about
that
runtime
environment
being
kubernetes.
A
Now,
if
we
take
a
look
at
this,
and
we
think
about
that
that
automation,
how
do
we
achieve
that?
Automation
that
I'm
alluding
to
that's
right
in
the
middle
there?
Well,
we've
already
got
some
automation
in
this
picture
right
well,
understood
and
well-established
automation,
and
that
is
the
ci
system.
A
A
Well,
let's
go
ahead
and
just
insert
deployment
as
a
part
of
that
automation,
but
I'm
going
to
argue
to
you
that
continuous
integration
is
not
the
same
as
delete
continuous
delivery
and
deployment
that
there
are
reasons
to
separate
those
out,
and
those
reasons
include
things
like.
If
you
are
running
in
a
regulated
environment,
you
know
that
you
are
dealing
with
separation
concerns
developers,
release
code.
They
are
the
ones
that
write
the
code
and
release
it,
but
then
operate
operators
do
the
deployment
they've
got
the
keys
to
the
production
environment.
A
A
A
You
can
also
imagine
that
there
might
be
deployments
where
we're
not
talking
about
three
targets.
We
might
be
talking
about
hundreds
of
factory
floors,
so
we're
starting
to
talk
about
edge
deployments
or
you
might
have
thousands
of
coffee
shops
or
sandwich
shops,
or
you
might
have
tens
of
thousands
of
cell
towers.
Yes,
we're
starting
to
see
kubernetes
running
in
cell
towers.
A
So
now,
in
addition
to
those
security
challenges,
I've
also
got
challenges
of
network
connectivity.
I've
got
challenges
of
scale.
How
many
environments
can
the
centralized
system
deal
with
the
centralized
cd
system?
Now,
there's
an
easy
answer:
there's
an
easy
solution
to
this,
and
that
is
to
just
spin
things
around.
A
A
A
We
don't
have
to
know
when
to
reconcile,
because
we
have
reconciliation,
loops
that
run
forever,
and
this
is
the
core
pattern
that
kubernetes
really
brought
mainstream.
Now
this
reconciliation
pattern,
I
have
to
say,
existed
before
kubernetes.
It
existed
in
cloud
foundry,
for
example,
where
we
applied
it
to
the
orchestration.
A
We
applied
it
to
the
running
of
container
images.
Now
those
container
images
predate
originally
predated
docker,
so
they
were,
it
was
well
before
oci
and
we
built
our
own
reconciliation
engine
in
in
cloud
foundry.
Now
cloud
foundry
applied
it
to
this
very
specific
use
case,
no
question
that
kubernetes
applied
it
more
broadly.
Kubernetes
created
this
reconciliation
platform.
If
you
will
and
we're
going
to
see
that
play
prominently
through
the
rest
of
the
presentation,
so
we've
got
this
notion
of
well.
A
A
The
is
watching
the
actual
state
of
the
system
and
it's
doing
what
it
needs
to
do
to
bring
those
two
things
together,
so
that
pattern
that
basic
reconciliation
pattern
is
extraordinarily
powerful
and
it
enables
a
whole
bunch
of
other
interesting
patterns
patterns
that
I
like
to
call
git.
Ops
patterns
are
cloud
native
operational
patterns.
A
So
let's
talk
through
some
of
those
now
the
first
one.
Well,
actually,
this
is
not
the
first
pattern.
This
is
more,
like
the
you
know,
maybe
third
pattern,
given
that
we
talked
about
pull
and
we
talked
about
reconciliation.
Those
are
certainly
two
important
patterns,
but
let's
talk
about
this
next
pattern:
the
pattern
that
I
taught
called
drift
detection
and
remediation.
A
So
we've
done
this
great,
get
ops
process
where
we
have
declare
put
our
declarative
configuration
in
git,
we
have
a
reconciler
that
is
constantly
drawing
things.
The
latest
things
in
from
git
into
the
runtime
environment
and
somebody
comes
along
and
does
a
coupe
cuddle
apply.
This
is
the
modern
day
equivalent
of
sshing
into
a
box.
A
But
again
here
somebody
has
done
the
modern
day
equivalent
of
sshing
and
we
have
drifted
from
that
desired
state,
as
recorded
in
the
system.
So
what?
What
do
we
do?
Well,
that
reconciler
is
constantly
running,
and
now
what's
happened
is
that
the
actual
state
has
diverged
from
the
desired
state
and
the
reconciliation
loop
can
do
some
type
of
remediation.
A
Now.
What
is
the
remediation
that
you
want?
Well,
it
can
actually
go
back
and
say
you
know
what
I'm
going
to
undo
that
change,
I'm
going
to
go
back
to
the
state
that
was
represented
in
the
git
repository
or
you
might
decide
that
you
in
fact
want
to
just
alert
on
that,
and
you
want
to
let
somebody
know
that
something's
changed.
A
That
is
your
prerogative
and
that's
what
you
are
going
to
tool
into
what
I
like
to
start,
calling
a
get
ops
pipeline,
stringing
together
these
types
of
behaviors
into
the
patterns
that
you
need
for
your
organization,
all
right.
So
that's
one
pattern.
Let's
talk
about
another
one
that
I
like
to
call
image,
update
automation.
A
A
A
A
That
pull
request
goes
into
the
repository,
somebody
approves
it
or
maybe
it's
automatically
approved,
and
then
what
happens
well.
This
is
the
pattern
that
we
already
know
and
love
is
we
have
the
reconciler
that's
running
inside
the
runtime
environment
that
pulls
the
latest
version
in
and
has
the
effect
of
deploying
the
updated
version
of
that
image.
A
But
let's
talk
about
another
because
another
pattern,
and
that
is
environment
customizations,
remember
we
talked
about
this
earlier,
so
we
talked
about
the
fact
that
I
have
dev
staging
and
prod
and
they're
all
going
to
be
deploying
this
application
and
while
a
lot
of
the
application
deployment
configuration
is
going
to
be
exactly
the
same,
there's
going
to
be
some
differences
across
those.
So
how
do
we
deal
with
that
again?
Let
me
make
a
little
space
in
this
diagram
and
I'm
going
to
talk
very
briefly
about
customize
with
the
k.
A
A
Now
those
overrides
can
be
handled
by
reconcilers,
so
we
can
actually
apply
these
customizations
as
a
part
of
our
get
ops
pipeline
and
those
reconcilers.
Those
customized
reconcilers
can
operate
independently
in
each
of
these
different
runtime
environments
so
doing
environment
customizations
is
an
also
an
important
pattern,
all
right,
so
we
had
started
here
and
those
patterns
that
I
just
described
to.
You
really
are
addressing
this
they're
saying
all
right.
There's
a
whole
bunch
of
processes,
there's
a
whole
bunch
of
detail.
A
That's
required
in
this
process
of
taking
these
application
configurations
and
these
image
images
in
the
from
the
image
registries
and
composing
them
together
into
the
actual
declarative
state.
That's
going
to
be
running
in
my
runtime
environment
and
I
like
to
call
those
set
of
controllers
that
we've
just
been
talking
about
the
set
of
reconcilers
delivery
controllers
and
hang
on
a
second.
I
said
this
is
about
deployment,
and
I
don't
know
that
delivery
of
configuration
manifests
really
encompasses
all
of
deployment.
A
A
Now,
if
we
remember,
we
were
here
recall
that
in
image
update
automation,
we
had
this
pattern.
Remember
we
had
reconcilers
that
were
watching
the
registries
and
updating
the
the
references
in
the
yaml.
Then
we
had
reconcilers
that
were
interfacing
with
git
to
do
things
like
pull
requests
or
maybe
automatically
committing
things
into
the
git
repository
and
then,
of
course,
we
had
the
reconcilers
that
were
drawing
those
configurations
doing
the
composition
like
using
using
customize,
like
I
showed
in
the
last
example
and
then
drawing
those
into
the
runtime
environment.
A
A
What
we
were
effectively
doing
with
those
reconcilers
is
getting
to
the
point
where
we
had
drawn
that
configuration
into
kubernetes
into
fcd.
So
now
we've
landed
that
in
fcd.
The
next
step,
of
course,
to
get
all
the
way
to
runtime
is
that
we
have
those
that
we
have
running
pods
in
the
end.
Now,
how
did
we
get
those
running
pods?
Well,
you
might
say
that's
kubernetes
and
of
course
it
is,
but
how
does
kubernetes
do
it?
A
A
A
A
What
can
you
do
with
that?
Well,
what
if
you
want
canary
style,
rolling
upgrades?
What,
if
you
want
blue
green?
What
if
you
want
a
b
testing
where
you're
actually
going
to
run
both
two
versions
of
something
in
parallel
and
actually
use
the
metrics
coming
out
of
those
to
make
decisions
on
you
know
who
gets
what
and
and
where
the
traffic
goes
so
different
deployment
scenarios?
A
Well,
what
you
can
do
is
you
guessed
it
you
can
have
a
reconciler
kubernetes
ex
is
allows
you
to
extend
the
api,
so
you
can
create
a
reconciler
that,
for
example,
recognizes
when
a
deployment
is
happening
and
provides
some
additional
logic.
On
top
of
that
now,
as
it
happens,
one
of
the
projects
that
we
have
worked
on
here
at
weaveworks
is
a
project
called
flagger.
A
It
is
totally
open
source
and
it
does
exactly
that.
It
allows
you
to
define
a
deployment
strategy,
progressive
delivery,
as
james
governor
had
coined
that
term,
and
it
allows
you
to
select
one
of
these
release:
strategies,
canaries,
a
b,
testing,
blue
green
and
then
it
interfaces
with
ingress,
either
kind
of
plain
vanilla,
ingress
like
an
engine
x
or
a
service
mesh,
and
it
interfaces
with
the
ingress
or
the
service
mesh
to
do
the
traffic
routing
to
provide
exactly
those
types
of
release,
strategies
that
you're
going
for
and
how
is
that
implemented?
A
Of
course,
it
is
implemented
as
a
set
of
controllers
that
run
in
the
kubernetes.
This
project
is
open
source.
You
can
find
it
there
at
the
url
and
I
won't
go
more
into
the
details,
but
what
you
can
see
there
and
really
the
point
that
I'm
getting
to
is
that?
Not
only
are
we
going
to
leverage
the
runtime
controllers
that
are
in
kubernetes,
we
can
also
extend
things
with
the
runtime
controllers
ourselves.
A
Okay,
that's
a
lot.
If
we
go
back
to
this
picture
that
I
just
showed
you
a
moment
ago,
there's
reconcilers
all
over
this.
How
can
I
possibly
manage
that?
Well,
the
first
thing
that
I
want
to
show
you
is
that,
in
fact,
this
is
a
version
of
the
very
first,
very
simple
diagram
that
I
started
with
at
the
beginning
of
the
presentation.
A
Let
me
show
that
to
you
again:
we've
got
something:
we've
restoring
some
declarative
configuration
and
git.
Then
we
have
a
set
of
delivery
controllers
that
facilitate
the
delivery
with
whatever
the
pipeline
is,
I
want
into
xcd,
then
I
have
a
set
of
runtime
controllers
that
actually
achieve
getting
that
running
in
production.
A
A
A
A
So
you
take
those
controllers
and
then
you
stitch
them
together
into
the
pipelines
that
are
relevant
for
your
organization
and
for
your
needs
and
that's
what
draws
together
get
offs
so
to
go
back
to
this
picture.
We've
now
completed
this
entire
cycle.
We've
got
continuous
integration
on
the
far
left.
We
talked
about
the
delivery
controllers.
We
added
on
the
runtime
controllers,
and
you
draw
all
of
that
together
and
you
get
get
offs
so
get
offs
is
not
just
about
delivery.
A
Now
I
had
promised
you
that
we
were
going
to
derive
principles.
We
were
going
to
go
back
to
first
principles
so
often
times
when
I've
spoken
about
get-offs
in
the
past,
I've
started
by
presenting
these
principles,
but
today
the
whole
idea
was
to
kind
of
maybe
challenge
your
assumptions
on
what
git
ops
is
and
to
derive
those
principles
from
scratch.
But
let
me
now
summarize
and
review
those
principles
that
we've
just
gone
over.
A
I'm
going
to
start
with
the
bookends.
On
the
left
hand,
side
we
have
declarative
configuration,
that's
yaml,
for
example,
on
the
right
hand,
side
we
have
the
software
agents,
so
the
deployment
controllers,
for
example,
deployment
replica
set.
These
are
two
foundational
principles
that
kubernetes
brought
to
the
party.
A
So
we
have
those
two
bookends
and
what
we've
done
with
get
offs
is
we've
added
two
more
elements
on
in
the
middle
we've
said
well,
if
I've
got
that
declarative
configuration-
and
I
now
add-
storing
that
declarative
configuration
in
a
version
version
controlled
immutable
store
that
has
semantics
like
git,
then
I
achieve
those
benefits
of
things
like
roll
forward
rather
than
roll
back.
I
can
always
get
back
to
an
earlier
configuration
that
I
know
worked
or
if
I
have
a
disaster,
I
can
recover
from
that
disaster
very
quickly,
so
that
is
another
magical
element
now.
A
Yes,
I
also
alluded
and
mentioned
several
times
that
it
is
the
user
interface.
It
doesn't
have
to
be
the
user
interface
and
there
are
some
very
interesting
other
user
experiences
that
don't
involve
necessarily
going
in
and
creating
a
pull
request
by
hand,
but
some
of
those
abstractions,
those
git
semantics-
are
important
and
very,
very
heavily
leveraged.
A
Now
I
won't
go
over
this
in
detail,
but
you've
heard
me
talk
about
these
get-ups
patterns
or,
as
I
also
like
to
call
them
cloud-native
operational
patterns
over
the
years
as
we
got
better
and
better
at
microservice
architectures,
we
learned
a
whole
suite
of
patterns
that
were
those
cloud
native
patterns
and,
in
fact
that's
what
I
cover
in
my
book.
Things
like
retries
and
circuit
breakers
and.