►
From YouTube: Simplifying Service Mesh Operations with Flux and Flagger - Mitch Connors & Stefan Prodan
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Simplifying Service Mesh Operations with Flux and Flagger - Mitch Connors, Google & Stefan Prodan, Weaveworks
Distributed Proxies have opened the floodgates for Service Meshes to provide substantial value at the Application Networking Layer, but early adopters of Service Meshes are often overwhelmed by operational complexities. How do you ensure that the proxy is distributed everywhere your software runs? How do you safely upgrade or roll back all those proxies? How can you ensure that your network config is correct - without pushing it to production and risking an outage? Following the GitOps Principles is key to simplifying Service Mesh Operations. Defining the entire service mesh declaratively - be it installation, proxy injection, or configuration - provides a mechanism to safely manage the complexities of a service mesh. Continuously reconciling declarative config with the latest service mesh release keeps you from being caught off-guard by CVEs. Progressive Delivery tools enable seamless movement from one version of a service mesh to another - and back - with minimal impact to traffic.
A
A
All
right:
well,
let's
get
to
it!
Everyone
here
should
probably
already
know
that
istio
makes
your
service
mesh
more
secure
right.
It's
the
leading
reason
for
adoption
among
our
users.
They
want
the
security.
They
want,
zero
trust
built
on
mtls.
They
want
the
ability
to
set
authen
and
off
z
policies
to
have
their
certificates
automatically
rotated,
and
it
makes
all
of
that
really
easy
and
seamless,
but
there's
a
little
bit
of
a
catch.
A
A
In
fact,
your
service
mesh
could
be
less
secure
because
istio
is
there
rather
than
more
secure.
So
why
is
this?
Why
are
88?
It's
not
that
istio
is
not
patching
their
vulnerabilities
right.
We
have
patch
releases
that
come
out
at
least
once
a
month.
Each
of
them
covers
a
bunch
of
known
cves,
but
it's
still
not
being
picked
up
in
the
community
very
quickly.
A
A
So
this
is
sort
of
a
timeline,
a
history
of
what
we
thought
was
the
problem
with
upgrades
in
the
istio
project
and
as
you
can
see,
it's
changed
a
lot
over
time.
We
first
noticed
that
no
one
was
upgrading
istio
in
q2
of
2020
and
we
thought
well,
maybe
they
don't
know.
Maybe
users
don't
realize
that
there
are
cves
in
their
istio
installation.
So
we
ran
a
survey
and
we
did
find
a
few
users
who
were
unaware
and
we
built
some
tooling
and
some
documentation
to
help
boost
that
awareness.
A
We
found
a
lot
more
users
who
were
aware
that
their
service
mesh
was
vulnerable,
but
they
didn't
feel
like
they
had
time
to
get
to
it.
You
know
these
istio
is
not
the
only
thing
that
anyone
is
running
in
production.
They've
got
their
own
apps
they've
got
telemetry
systems,
they've
got
maybe
some
ebpf
systems
that
need
to
stay
up
to
date
and
staying
on
top
of
all
of
it
is
just
so
difficult.
It's
like
a
treadmill
that
you
can
never
keep
up
with,
so
we
decided,
let's
make
istio
upgrades
easier.
A
This
was
a
part
of
the
2021
roadmap.
You
see
it
now
on
the
2022
roadmap
for
the
project.
Istio
upgrades
should
be
easy
and
we
launched
surveys
that
went
along
with
every
release
asking
users.
How
easy
was
this
and
we
saw
consistently
from
one
release
to
the
next.
The
studio
is
getting
easier
to
upgrade
easier
to
upgrade
nothing
huge
but
consistent
improvement
from
one
release
to
the
next.
So,
a
year
later
we
ran
our
production
survey
again
and
asked
okay.
A
A
What
else
well
right
now,
as
of
last
fall,
you
had
to
upgrade
seo
four
times
a
year
and
that's
a
lot.
So
maybe
we
should
move
to
something
else.
You
know
kubernetes
has
now
the
three
times
a
year
release
process.
We
looked
at
that
a
little
bit.
The
sweet
spot
we
landed
on
was
once
every
six
months
and
we
we
kept
our
release
process
the
same.
A
So
we
still
release
four
times
a
year,
but
you
can
now
upgrade
directly
from
say
112
to
one
or
sorry,
1
10
to
112
or
111
to
113
without
doing
an
intermediate
upgrade
along
the
way,
and
then
we
extended
our
support
window
so
that
110
and
112
would
overlap
by
six
weeks
giving
you
that
time
to
upgrade.
So
now
we
just
cut
your
work
in
half,
it's
really
easy
to
upgrade
istio,
and
so
we
waited
six
months
and
then
we
surveyed
production
and
we
found
out
that
88
of
all
istio
installations
still
have
known
cves.
A
So
it's
not
that
it's
too
difficult!
It's
not
that
our
customers
don't
know
they.
Should
it's
not
that
it's
too
frequent.
We
need
a
new
hypothesis,
so
this
one's
really
controversial
our
2022
hypothesis,
for
why
istio
upgrades
aren't
happening
is
that
humans
are
bad
at
repetitive,
monotonous,
labor.
A
I
know
I
know
that
super
controversial
in
a
group
of
programmers.
We
all
love
our
monotonous,
repetitive
tasks,
but
that's
what
we're
going
after
this
year.
That's
how
we
hope
to
move
the
needle
down
from
88
percent
stefan's
going
to
share
with
us
how
we're
going
to
remove
that
monotonous,
repetitive
labor
from
istio
installations,
with
some
products
from
weaveworks
sure
there
are.
B
Not
products
of
weavers
there
cncf
projects,
okay,
so
I'm
guessing
everybody
here
knows
what
githubs
is
it's.
You
know
you
already
seen
this
slide
and
this
event
10
times
per
day.
Probably
the
idea
behind
githubs.
Is
you
let
your
cluster
figure
out
its
state
from
an
external
source
and
with
flux
that
can
be?
You
know,
git
repository
or
an
s3
bucket,
or
we
have
many
ways
of
how
you
can
define
your
cluster
state
outside
of
the
cluster.
B
So,
instead
of
you
know
you
having
to
run
commands
to
upgrade
your
infrastructure
or
your
applications,
what
you
can
do
is
define
how
those
can
be
automatically
upgraded
in
your
git
repo,
for
example,
with
kubernetes
custom
resources
and
something
the
cluster
itself.
Something
running
on
the
cluster.
You
know
sees
that.
Okay,
I
need
to
upgrade
myself
from
the
source
and
I'm
going
to
do
it
automatically.
B
I'm
going
to
talk
about
the
flux
project,
so
in
the
flux
project
we
have
two
main
projects:
one
is
flux
itself,
which
is
a
collection
of
kubernetes
controllers.
We
have
12
custom
resource
definitions
and
what
flux
aims
to
provide?
You
is
a
you
know
like
a
lego
system
where
you
can
build
your
own,
continuous
delivery
solution.
That
fits
your
use
case.
B
Flux
is
not
operated
in
any
way.
We
don't
offer
you
a
ui.
We
offer
you
something
that
allows
you
to
build
your
platform
and
besides
flux,
we
have
a
second
project
called
flagger
and
the
flagger
role
is,
to
you
know,
decouple
the
deployment
from
the
release
process.
So,
if
flux
does
the
deployment
automatically
based
on
what's
in
git,
that
doesn't
mean
that
those
things
that
are
engaged
should
be
automatically
exposed
to
all
your
users
and
flagger
role
is
to
detect.
Oh,
you
want
to
deploy
a
new
version
of
your
application.
B
Let
me
test
the
new
version
on
some
of
your
users.
So
with
flagger
you
can
say:
okay,
when
there
is
a
new
version,
move
one
percent
of
the
traffic
to
the
new
version,
see
if
that
works
measure
slos,
like
I
don't
know,
error
rates,
latency
other
custom,
metrics
and
if
that
works,
add
more
traffic,
more
traffic
and
if
everything
goes,
okay,
only
then
flagger
fully
rolls
out
the
new
version
to
all
your
users
and
that's
called
progressive
delivery.
B
So
this
is
how
flux
looks
like
and
how
the
continuous
delivery
you
know
circle
is
flux,
is
fluxes,
controllers
are
even
based.
So
every
time
you
push
something
to
your
repo
flux
will
be
notified
and
it
will
reconcile
the
new
state,
but
you
can
also
ping
it
from
ci.
It
also
monitors
your
container
images.
It
can
upgrade
your
container
applications
inside
the
cluster.
You
can
say,
for
example,
I
want
for
envoy
every
time
there
is
a
patch
release.
B
It
also
works
with
cluster
api,
very
well,
okay
flagger.
I
I
mentioned
about
it.
Besides.
Integrating
with
istio
for
routing
traffic
between
versions,
it
works
with
a
bunch
of
other
service
meshes
and
also
ingress
control.
So
if
you
are
not
into
the
service
mesh
hype,
you
just
use
an
ingress
controller.
B
B
We
have
a
solution
for
this
flux
is
able
to
upgrade
itself
and
how
we
how
we
can
do
that
we
use
the
github
action
or
something
in
our
ci
that
monitors
the
flux,
github
releases
when
that
action
detects.
Oh,
there
is
a
new
patch
release
for
flux.
It
will
open
a
pull
request
for
you
where
it
can
bump
the
flux
version
directly
in
a
branch.
Then
flux
is
oh,
my
definition
from
git.
My
own
definitions
have
changed
so
now.
B
It's
time
to
upgrade
myself,
it
pulls
the
his
new
version
starts
a
rolling
deployment
and
that's
how
you
can
keep
flux
fully
automated
up
to
date
and,
unlike
istio,
you
can
jump
from
any
flux
version
to
any
other
flux
version.
So
we
had
flux
version,
2
users
who
installed
it
six
months
ago.
They
ran
the
upgrade
last
week
and
it
worked
even
if
we
released.
B
B
B
The
repository
will
write
its
own
definitions
in
the
repo
will
commit
its
definitions
to
the
upstream
and
dell
will
install
itself
on
the
cluster,
and
we
also
have
flavors
of
bootstrap
for
github
gitlab
bitbucket,
where
you
can
say,
flux,
bootstrap,
github,
and
it
creates
the
repository
for
you.
It
sets
up
the
deploys
keys
for
you
and
so
on,
but
today
we
are
using
this
command
that
works
with
any
kind
of
of
git
server.
B
Okay,
so
I'm
pressing
enter.
Hopefully
it
will
work
what's
happening
now,
I'm
clone
the
repository
locally.
It
pushes
all
its
definitions
to
get
through
using
ssh
go
on
go
on.
We
are
really
in
a
hurry,
flux
and
yeah
after
it
it
it
talks
to
the
git
repo.
After
it
it
places
there
it's
its
configuration
next.
What
it
does
is
it
configures
itself
for
a
particular
path
in
the
git
ripple,
so
you
can
use
a
single
git
repository
to
manage
your
whole
infrastructure
fleets.
B
B
Okay,
after
it
has
installed
anything
now,
flux
is
verifying
itself
if
all
the
controls
are
running
and
we
have
four
controllers
which
are
part
of
the
base,
flux,
installation
one
is
the
source
controller
which
deals
with
all
the
git
operations
and
another
one
is
customize
control
which
can
apply,
customized,
plane,
research,
kubernetes,
plane,
resources
or
customized
overlays
and
finally,
the
helm
controller
which
are
using
now
for
for
istio.
We
are
installing
istio.
B
B
What
is
what
what
the
flux
did.
So
first,
we
have.
We
have
a
thing
in
flux
where
we
can
define
dependencies
between
our
infrastructure
items
in
in
the
case
of
istio.
This
is
critical,
because
istio
has
several
charts.
One
contains
the
crds
another
one
contains
pilot
another
one
contains
the
gateway
right.
I
cannot
install
the
gateway
until
I
have
the
crds.
I
cannot
install
pilot
and
so
on.
So
in
in
the
repo
we
have.
We
can
define
this
declarative
and
say
install
all
things
in
a
particular
order
and
also
do
the
upgrade
in
that
order.
B
So
when
there
is
a
new
sql
release,
we
first
upgrade
the
crds,
then
pilot,
then
gate
and
so
on,
and
you
see
here,
for
example,
dependencies
tld
is
not
ready.
So
what
flags
does
it
checks?
The
it
looks
at
the
health
check
of
each
deployment
and
blocks
until
all
list.
Your
components
are
ready
and
it
it
deploys
them
in
the
in
the
particular
order.
So
now
we
should
see
that
everything
is
done.
Yes,
so
we
have
estia
installed
on
the
cluster.
All
our
applications
are
there.
The
gateway
is
set
up.
A
The
the
only
progressive
delivery
api
that
I
could
find
that
worked
directly
on
deployments
so
that
I
did
not
need
to
change
anything
about
how
the
customer
was
defining
their
workloads
was
flagger.
I've
found
other
tools
that
will
allow
you
to
do
progressive
delivery
with
a
pipeline
where
you
define
your
own
steps,
but
I
didn't
really
want
to
do
that
on
behalf
of
the
user.
Also,
stefan
already
did
most
of
the
work,
the
get
ops,
repo
that
we're
demoing.
Today
you
saw
the
link
actually
back
here.
A
Stefan
proud
on
get
ups
istio
has
been
around
for
some
time.
I
just
recently
joined
stefan
in
making
some
tweaks
to
it,
which
we'll
be
showing
off
this
afternoon.
So
how
is
this
going
to
work
for
istio?
Well,
flux
is
going
to
manage.
What
we're
going
to
see
first,
is
that
flux
is
managing
the
sdo
control
plane.
A
So
when
you
merge
an
update
which,
by
the
way
can
happen
when
someone
makes
a
change
to
your
istio
installation
or
when
github
actions
notice
that
there
is
a
new
version
of
istio
available,
we'll
get
a
pull
request,
saying:
hey
new
version
of
istio
go
ahead
and
install
it
and
there's
checks
and
tests
that
will
run
on
that
pull
request.
Just
like
you
all
should
have
for
your
ci
systems.
Once
it
gets
approved,
flux
will
see
it
in
the
main
branch
and
we'll
reconcile
it
along
with
flagger
prometheus,
etc.
A
A
I
am
almost
certainly
going
to
slip
up
and
say
data
plane,
sidecar
and
proxy
interchangeably
or
the
envoy.
Those
are
all
the
same
thing.
It
sounds
like
really
overwhelming
vocabulary,
but
we're
really
talking
about
one
thing:
it's
a
proxy
that
runs
everywhere
everywhere.
You
have
an
application.
You
need
a
proxy
right
next
to
it.
So
it's
in
the
pod
running,
alongside
your
application
and
managing
things
we
just
like
to
give
it
four
names,
because
istio
so
istio
is
divided
between
a
control
plane
and
a
data
plane.
A
We
just
saw
the
control
plane
get
installed,
and
now
what
we're
going
to
demo
is
having
a
control
plane
upgrade.
Oh
before
the
demo.
I
wanted
to
talk
about
this
along
the
way
we
found
stefan
and
I
found
that
it
is
really
difficult
to
integrate
istio
control
with
github
actions,
so
we
launched
our
own
get
istio
control
action.
If
there's
anything
you'd
like
to
do
in
github
actions
in
your
ci
system
get
istio
just
downloads,
the
binary
for
you
into
your
path
so
that
you
can
run
it.
A
A
Okay,
oh
actually,
no.
I
was
already
where
I
wanted
to
be
that's
the
problem.
So
this
is
where
we're
running
our
demo
from
again
I've
forked,
this
repo
from
stefan's
and
that's
actually
your
first
step
in
your
process.
If
you'd
like
to
do
the
same
thing,
fork.
Stefan's
repo,
don't
fork
mine
for
various
reasons,
but
you
can
see
that
we've
got
our
istio
installation.
We've
got
our
apps
installed
here.
We've
got
cluster
definitions
and,
interestingly,
if
we
look
at
pull
requests,
we
have
a
pull
request
that
was
just
created
two
hours
ago.
A
A
The
only
thing
that's
changing
here
is
the
version
of
the
control
plane
and
you
can
see
that
from
the
diff
here.
We
can
also
see
that
there
are
two
automated
checks
that
have
run
for
this
and
one
that
seems
to
still
be
running
after.
Oh,
it's
only
been
six
minutes,
oh
because
of
the
changes
pushed
by
the
bootstrap.
A
B
So
what
we've
been
doing?
We've
been
running
kubernetes
kind
inside
the
github
action
runner
and
in
there
we
are
streaming
down
istio,
bringing
down
the
requests
the
limits
everything
down.
So
we
can
run
everything
in
there
and
make
sure
that
the
new
version
of
the
control
plane
works,
and
we
do
that
with
flux.
So
flux
is
deployed
inside
kind,
pulls
the
branch
from
the
pull
request
and
runs
an
end-to-end
test
suite
for
all
the
istio
components
right
before
everything
goes
out
to
our
staging
cluster,
we'll
do
some
real
testing
and
only
then
to
production
cluster.
A
A
A
We
don't
know
the
version
of
your
proxy
in
istio
by
default
is
determined
when
the
pod
starts
up
and
that's
a
little
bit
problematic,
because
it
means
all
the
way
through
your
test
infrastructure,
your
units,
your
integration,
test,
your
staging
environment,
you
don't
actually
know
what
proxy
you're
going
to
be
running
with,
and
it's
a
fairly
important
component.
It's
something
you
want
to
test
about
against.
So
that's
not
great.
Also,
this
doesn't
yet
use
revision
based
update
upgrades.
A
So
let's
talk
a
little
bit
more
about
this
problem
with
proxies
right
in
the
software
development
life
cycle,
the
proxy
is
defined
at
the
very
end.
It
doesn't
even
necessarily
result
or
isn't
necessarily
the
result
of
a
kubernetes
deployment.
You
could
have
a
deployment,
that's
running
an
old
proxy.
You
upgrade
the
control
plane.
You
scale
that
deployment
up
the
new
pods
will
get
the
new
proxy.
A
The
old
pods
will
still
have
the
old
proxy,
it's
very
unpredictable
and,
as
you
can
see,
that
reduces
determinacy
in
the
whole
process,
it's
very
uncontrolled
and
the
way
to
fix
that
is.
You
run
global
restarts
on
your
cluster,
which,
like
I,
I
don't
really
want
to
recommend
that
you
do
in
production.
So,
let's,
let's
talk
about
how
we
can
make
this
better.
A
What
we
want
to
do
is
shift
the
declaration
of
that
proxy
or
the
specification
of
the
proxy
left
in
the
software
development
life
cycle
to
the
point
of
get
ops,
get
ops
needs
to
happen
at
the
test
environment
so
that,
from
test
to
release
to
deploy
to
operate,
you
have
a
consistent
piece
of
software
that
you're
operating
across
all
those
environments.
Your
test
results
are
meaningful
and
then,
of
course,
your
your
rollbacks
are
just
to
get
revert.
If
something
goes
wrong,
you
know
how
to
get
back.
It's
not
quite
so
simple.
A
If
you're
using
sidecar
injection
in
istio,
how
do
you
define
a
sidecar
in
git?
Well,
it
turns
out
there's
been
a
command
to
help.
You
do
this
for
some
time
in
the
istio
community
called
cube.
Inject
it
will
take
a
yaml
that
could
be
a
deployment
or
a
job
or
a
replica
set
or
a
stateful
set,
and
it
will
output
a
modified
version
of
that
deployment.
That
includes
a
sidecar,
pretty
simple,
and
so
we've
built
that
in
and
baked
it
into
a
github
action.
A
So
what
we've
already
seen
is
a
new
version
of
istio
was
available,
so
we
upgraded
our
control
plane
yaml
in
git,
they
created
a
pull
request.
We
had
kind
tests
and
and
our
analyzed
tests,
we
approved
the
pull
request
and
it
got
committed
and
deployed
to
kubernetes,
but
that
commit
now,
hopefully
we're
going
to
see
in
just
a
second
has
triggered
a
second
github
action,
which
saw
oh
your
control.
Plane
version
now
does
not
match
your
data
plane
version.
A
A
Moment
of
truth
here,
pull
requests
all
right,
update,
istio
sidecar
to
113
4.
again,
our
end-to-end
test
hasn't
passed,
it's
extremely
thorough.
It
actually
does
a
roll
out
that
goes
through
the
entire
canary
process,
and
you
don't
want
canaries
to
run
terribly
quickly.
But
for
that
reason
I'm
not
going
to
stand
around
and
wait
for
the
end
to
end
test
to
pass.
Today
I
want
to
upgrade
my
data
plane.
A
A
B
B
Okay
has
upgraded
the
gateway,
which
is
the
custom
resource
of
the
gateway
that
worked
and
now
he's
doing
what
he's
doing
constellation
in
progress
still
the
gateway.
So
you
see
the
revision,
how
it
moves
from
three
to
four
right
and
for
all
this,
all
these
upgrades
that
flux
is
doing
once
has
upgraded
something
or
started
to
upgrade
something
to
let
you
know
through
you
know,
slack
channels,
team,
microsoft,
teams
and
so
on.
So
you
can
actually
see
those
if
you
have
other
thing
sorted
out.
Okay,.
A
A
And
the
good
news
about
a
nail
biter
like
that,
is
it's
given
me
enough
time
that
I
think
I
should
be
able
to
check
our
data
plane
and
see
the
same
thing
updated.
We
should
be
at
113,
113
4.,
so
we've
gone
ahead
and
done
a
live
upgrade
of
istio
we'd
separated
between
the
control
plane
and
the
data
plane.
Also,
one
added
benefit
of
having
the
proxy
defined
actually
in
the
deployment
spec
is
that
flagger
will
do
progressive
delivery
across
your
deployment.
So
if
I
check
flaggers
canaries,
I
think
it
hasn't
been
too
long.
A
They
should
still
be
in
progress.
So
on
our
back
end
service,
we
currently
have
five
percent
of
our
traffic
going
to
the
new
version
of
the
proxy
and
fi
95
of
our
traffic,
going
to
the
old
version,
and
it's
going
to
periodically
check
with
prometheus
and
ask:
is
the
service
still
healthy
shift
a
little
bit
more?
Is
the
service
still
healthy
shift
a
little
bit
more
and
it's
going
to
be
really
boring,
which
is
one
of
our
favorite
things
to
be
in
istio
yeah?
A
So,
let's
take
a
look
at
our
advantages
and
disadvantages.
Now
our
proxy
is
now
maintained
up
to
date
with
an
assembler
range.
We
can
automatically
apply
updates
if
we
want,
if
we
want
to
without,
like
patch
updates,
can
roll
out
without
a
pull
request,
or
they
could
be
stuck
with
a
pull
request.
It
just
depends
on
how
your
operational,
like
security
and
safety,
how
much
how
paranoid
you
want
to
be.
I
want
to
be
pretty
paranoid
personally.
A
Canaries
are
controlling
the
rollback
and
we
have
auto
or
out,
and
we
have
automated
rollback.
If
something
is
wrong
with
that
proxy,
the
113
for
proxy
will
stop
executing
in
the
cluster.
The
canary
will
fail
back
to
113.3
and
our
cluster
will
stay
up,
even
if
our
istio
upgrade
was
not
great.
We
still
haven't
fixed
that
revision
based
problem,
but
when
once
we
do,
this
is
what
the
new
workflow
should
look
like.
When
a
new
version
of
the
control
plane
becomes
available,
we
don't
replace
the
control
plane
yaml.
Instead,
we
create
a
new
revision.
A
This
is
what
you
all
should
be
doing.
If
you're
upgrading
istio
it's
much
more
secure,
it
means
that
they'll
run
side
by
side.
Two
control
planes,
two
different
versions:
we'll
go
through
the
same
pull
request
process,
and
then
we
will
move
tags.
Tags
are
like
sim
links
for
your
revisions,
so
you
might
have
like
a
first
wave
second
wave
third
wave.
The
first
wave
is
going
to
get
updates
first,
second
wave.
Second,
what
we're
going
to
do
is
we're
going
to
find
this.
The
first
wave
tag
and
say
first
wave
tag
used
to
be
one.
A
A
If
there's
any
takeaways,
if
there
is
one
thing
that
you
leave
this
room
with
today,
I
would
like
for
it
to
be-
and
I
said
istio
here-
I
think
we
can
just
replace
that
with
service
mesh
automate
your
service
mesh
upgrades,
please
please,
please
or
consider
paying
a
vendor
to
do
it.
If
this
is
overwhelming
to
you,
google
and
many
other
vendors
in
this
space
will
take
care
of
your
upgrades
on
your
behalf,
and
that
is
a
totally
fine
way
of
doing
things.
A
A
C
B
Can
answer
that
so
the
first
version
of
my
repository
was
using
history
operator
and
that
works
great,
because
flagger
and
flux
can
do
with
any
kind
of
custom
resource
right.
It
was
the
same
experience
but
yeah,
so
the
instio
team
said
switch
to
helen
charles,
because
the
operator
all
he
does
in
inside
are
doing
is
the
same
thing,
he's
running
a
helm,
upgrade
and
so
on
and
flux
has
a
dedicated
helm
controller,
which
does
all
things
helm
way
better
than
you
know,
wrapping
it
out
and
hiding
it.
B
So
you
have
the
helm
experience
you
can
debug
it
with
helm
and
so
on.
So
it's
it's
add.
More
adds
more
observability
to
the
whole
process,
like
you
do
flux
and
get
helm
releases,
and
you
can
actually
see.
Oh,
it's
stuck
at
the
crds
or
if
there
is
a
problem
with
the
gateway,
the
operator
will
just
hide
all
of
that
under
a
single
upgrade
operation.
A
So
that
being
said,
if,
if
you're
happy
with
the
operator,
stick
with
it-
and
you
can
look
in
the
repo
about
three
weeks
ago,
that
was
what
we
were
doing
in
the
repo
if
you're
looking,
if
you're
not
currently
upgrading
and
you're.
Looking
for
a
way
to
do
that,
probably
don't
look
at
the
operator.
We
would
recommend
that
new
users
go
to
helm,
charts,
we're
still
going
to
keep
maintaining
and
fixing
bugs
in
the
operator,
but
the
future
direction
is
definitely
helm.
E
Hello,
nice
talk.
Thank
you.
I
just
had
one
question.
Would
you
say
that
this
git
ops
approach
is
better
than
doing
it
on
infrastructure
as
code,
because
sometimes
for
cluster
wide
or
infrastructure-wide
components,
we
usually
put
these
installations
in
terraform,
but
what
would
you
say
is
an
advantage
of
having
it
here
in
githubs.
A
I
guess
I'll,
I
probably
have
a
little
bit
of
a
different
opinion
than
stefan
does
on
this
particular
topic.
For
my
part,
I
want
to
make
sure
that
you're
comfortable
doing
it.
The
way
that
you
do
other
upgrades
istio
should
not
be
something
that
is
unique
and
special
in
your
environment.
It
should
upgrade
the
same
way
that
every
other
piece
of
infrastructure
that
you
have
so
if
terraform
is
your
tool
of
choice
for
all
of
your
infrastructure.
A
B
The
main
difference
from
my
perspective
is
the
fact
that
terraform,
you
run
it
once,
but
flux
will
keep
monitoring
what's
happening
right.
So
it's
the
difference
between
githubs
and
just
doing
once
applying.
It
is
the
fact
that
github
does
a
reconciliation
all
the
time.
So
if
something
breaks
or
something
goes
on,
the
class
someone
goes
on,
the
cluster
does
a
cube
cattle
edit
of
the
of
some.
You
know
pilot
deployment,
and
that
goes
wrong.
Flux
will
just
undo
it
automatically
because
flux
does
not
allow
divergency
a
drift
inside
the
cluster.
B
I'm
guessing
you
don't
run
hella
terraform
apply
every
couple
of
seconds.
Well
flex
can
do
that
because
it's
right
there
in
the
cluster
and
can
monitor
all
the
changes
that
were
done
outside
of
git,
and
you
correct
that
change
and
will
let
you
know
hey
some
modified
issue,
I'm
putting
back
in
the
same
thing.
In
the
same
way,
it
was
described
there
in
the
in
the
git
repo.