►
From YouTube: Dynamic Configuration with ComponentConfig and the Control Loop - Chris Hein, Apple & Leigh Capili
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Dynamic Configuration with ComponentConfig and the Control Loop - Chris Hein, Apple & Leigh Capili, Weaveworks
ComponentConfig is a pattern that allows you to use Kubernetes API Machinery to load Configuration files that look and feel like Kubernetes resources. This talk will introduce the idea of combining ComponentConfig with Dynamic Configs through CRDs to allow your controllers to dynamically reconfigure themselves similar to the way the Dynamic Kubelet Configuration works. This pattern can be useful for any Controller or Operator builder to make updating and managing the controller much more straightforward for end-users. This pattern also introduces the ability to report customized status about how the controller is operating, reducing the cognitive load for users to debug misconfigured controllers.
https://sched.co/ZemW
A
A
But
he
can.
He
can
see
that
there's
alerts
happening,
and
so
he
starts
to
look
into
it.
So,
first
line
of
action,
as
of
of
course,
he
goes
and
sshs
into
the
server
and
notices
it's
on
fire
and
starts
debugging
a
little
bit
and
notices
that
it's
the
nginx
configuration
file.
So
somehow
that
this
this
configuration
file
was
set
up
wrong.
A
So
he
opens
up
a
vim
session
now,
unfortunately,
for
all
the
emacs
users
in
the
group,
vim
bob
is
a
hardcore,
vim
user
and
he's
gonna
he's
gonna
get
on
and
get
in.
There
start
changing
everything
that
he
needs
to
change
and
once
he
figures
everything
out,
he
saves
he
quits
the
file
and
then
he
goes
and
reloads
nginx
and
as
you'd
hopefully
expect
when
somebody
was
to
reload
nginx.
B
Yeah
now
we're
here
24
hours
later
and
at
acme,
I'd
like
to
introduce
you
to
somebody
named
carol
carol
is
a
dev
and
she's
been
working
on
an
awesome
feature,
and
this
feature
needs
to
be
deployed.
It
requires
a
feature
flag
to
be
configured
in
a
newer
version
of
the
config
and
so
she's
got
everything
code
reviewed,
she's
super
excited
about
it.
It's
going
to
add
value
for
the
experimental
user
base.
B
That's
just
a
subsection
of
all
of
the
people
that
acne
serves
and
she
hits
the
deploy
button
with
the
approval
to
start
the
imperative
bash
script
that
jenkins
uses
to
orchestrate
the
rolling
update
of
the
nginx
config
and
things
are
going
good
she's
watching
the
metrics
things
are
rolling
out
all
green
and
suddenly
the
patch
logic
inside
of
that
bash
script.
It's
just
not
having
it
with
the
newer
version
of
the
config.
B
That
was
changed
by
bob
in
his
incident
response
and
suddenly
carol's,
seeing
502s
and
400s
in
downstream
services,
because
the
feature
flag
is
not
enabled
globally
for
a
long
enough
period
of
time
for
the
feature
to
do
its
rollout
and
now
the
servers
are
in
this
inconsistent
state.
We
have
configuration
drift,
which
is
this
problem
that
was
supposed
to
be
solved
with
systems
like
puppet
and
chef.
B
A
Yeah,
absolutely
so,
you're,
probably
all
familiar
with
what
we're
looking
at
here,
the
deployment
yaml
it's
just
kubernetes
and
realistically,
because
at
this
day
and
age
we
should
all
just
call
ourselves
yaml
engineers.
This
should
be
very
familiar.
But
if
it's
not
we're
going
to
dive
into
the
the
nuts
and
bolts
of
how
this
thing
works
so.
A
Some
really
good
benefits.
So
if
we,
if
we
debug
this
a
little
bit,
we
can
see
that
the
group
in
this
instance
for
deployments,
is
actually
apps,
and
the
version
is
that
v1
that
v1
tag
and
then
underneath
each
one
of
those
versions
that
can
be
defined.
And
there
can
be
any
number
of
these
that
that
are
needed
to
to
get
to
a
stable
state.
We
have
kinds
represented,
and
so
under
apps
we
have
things
like
daemon
sets
as
well
and
deployments
when
you
start
to
talk
about
more
the
the
lower
level
core
objects.
A
A
So
if
I
have
a
cluster-
and
I
know
in
my
cluster,
I
have
version
one
and
v,
one
beta
one.
If
I
supply
v,
one
beta
one,
I
know
it's
gonna
work,
because
that
that
api
is
already
exposed-
and
I
can
expect
that
my
schema
for
that
that
I'm
using
that
is
a
valid
schema-
will
satisfy
what
kubernetes
expects
to
be
passed
into
it
and
what's
really
cool
here.
Is
we
have
these
structured
schemas
and
we're
able
to
take
these
and
really
start
to
think
about
them
from
a
machine
level
and
a
human
level?
A
I,
as
a
person,
can
go
and
write
this
manifest
file.
It
does
might
take
me
a
little
bit.
I
might
have
to
go
and
look
at
stack
overflow
to
grab
the
the
actual
schema,
because
I
don't
remember
them
always
off
the
top
of
my
my
head,
but
at
the
same
time
I
can
build
tools
to
actually
work
with
these
things.
A
So
you
see
this
in
the
common
tooling
that
we
have
today
that
everybody
works
with
whether
you're
just
creating
a
cluster
and
you're,
just
using
cube
cuddle
or,
if
you're
starting
to
be
a
little
bit
more
advanced
and
work
with
templating
and
packaging
of
your
applications
using
things
like
customize
or
even
further
beyond
that
into
the
whole
plethora
of
other
third-party
tools
that
work
with
this
things
like
helm
and
pollumi
and
cdk,
there's
so
many
tools
out
there.
These
days
that
can
use
this
version
schema
to
provide
kubernetes
a
standard
api.
B
Yeah,
this
is
a
a
really
great
benefit
when
we
get
to
actually
work
with
these
apis.
Right
so
say
you
have
kubernetes
or
use
you're
using
these
kubernetes
native
tools.
Now
you
can
provide
configuration
and
these
apis.
They
promise
you
stability,
they
promise
you
versioning,
you
get
the
benefits
of
validation,
defaulting
and
conversions
between
these
apis
using
common
tooling.
B
But
if
you're
just
running
your
own
workloads
or
unfortunately,
I
mean
we
have
to
run
a
lot
of
other
people's
workloads
they're,
just
these
unix
style
programs
right,
you
invoke
them
and
then
you
give
them
arbitrary
strings.
It
just
so
happens
that
we've
used
flags
for
so
long
that
every
major
programming
language
now
has
parsers
for
flags,
but
it
doesn't
have
to
be
that
way
like
the
fact
that
we
have
hyphen,
hyphen
versus
slash,
slash
or
backslash,
like
you
know,
is
common
in
windows
command
line
tools.
B
It's
arbitrary,
there's
actually
parsers-
that
need
to
be
built
into
your
component
into
your
programs
that
work
with
these
flags,
but
those
flags
are
config
right.
That's
how
you
indirect!
That's
how
you
change
the
way
that
your
programs
behave
and
say
you're,
running
a
lot
of
programs
right,
because
we've
got
these
custom
parsers
and
we
are
able
to
work
with
convenient
small
deployments
but
say
you're,
running
kubernetes
right.
This
is
an
example
of
a
smaller
kubernetes
cluster
there's.
You
know
five
scds
here.
B
There's
a
couple
of
kubelets
running
for
various
nodes
as
well
as
coupe
proxy
you've
got
the
api
server
in
a
redundant
deployment,
and
then
you
have
all
of
the
controller
managers
and
your
own
web
hooks
and
operators
to
support
your
configuration
of
the
cluster.
This
is
a
lot
of
different
components.
Even
if
you're
not
running
kubernetes,
say
you're
in
an
environment
like
bob
and
carol
where
you
were
managing
your
own
servers.
B
The
reality
is
that,
in
order
to
run
modern
businesses
these
days,
we
happen
to
have
gotten
ourselves
into
a
world
of
managing
multi-program
distributed
cloud
deployments
and
when
you're
doing
things
with
flags,
it
can
be
hard
to
make
promises
across
changes
to
your
program.
Let's
consider
the
kublet.
This
is
one
component,
one
instance
of
it
on
a
single
node
inside
of
that
kubernetes
cluster.
B
We
were
just
talking
about
there's
a
wall
of
configuration
here
in
the
form
of
flags,
because
the
kubelet
literally
has
like
150
options
or
something
like
that
and
each
one
of
those
flags.
Some
of
them
are
common,
deliberate,
some
of
them.
You
can
pass
multiple
versions
of
the
flags
and
it
builds
into
a
collection.
B
So
then,
if
you
want
custom
tooling
right
to
be
built
around
this
harder
configuration
problem,
it
becomes
quite
clumsy
to
work
with
what
really
is
a
public
api.
That's
represented
by
these
flags
right
tools,
understand
these
structures.
You've
got
to
what
import
the
parsers
from
the
kubelet
into
your
config
management
tool,
and
the
fact
of
the
matter
is
that
this
is
just
something
that's
very
brittle
and
it's
not
very
fun
to
deal
with
either.
B
This
is
not
easy
to
look
at
if
you're
an
operator
trying
to
understand
the
difference
between
some
node
pools
of
your
kubelet
deployments,
and
this
is
true
as
well
for
any
of
the
workloads
that
you
might
be
building
yourself,
and
so
the
kubernetes
solution
that
we've
been
playing
with
in
more
recent
versions
of
the
releases
is
to
support.
What's
called
component
config
here.
We
have
an
example
of
the
kubelet
configuration
just
the
very
beginning
of
that
api.
B
For
a
specific
period
of
time,
and
then
it
also
lets
people
pick
their
own
strategy
for
upgrading
and
so
we've
got
kubelet.
We've
got
component
config
right
the
fact
that
you
can
load
a
kubernetes
api
owned
by
the
component
that
needs
to
be
configured
directly
from
a
file,
but
then
we
want
to
get
out
of
this
world
of
where
components
are
out
of
sync
with
their
configs,
and
so
you
need
some
sort
of
controller
they're.
B
All
there
isn't
always
a
controller,
that's
taking
care
of
you,
and
so
in
the
kubelet
we've
been
running
an
experiment
called
dynamic
configuration.
What
dynamic
kubelet
config
will
do.
Is
you
tell
the
kubelet
hey
in
the
api
server
that
you
can
connect
to
there's
a
config
map?
That's
holding
your
config,
it's
holding
all
of
the
data
in
a
raw
format
that
you
can
then
load
in
and
validate
and
default
and
convert
yourself,
and
this
means
that
we
are
using
the
kubelet's
controller
to
also
reload
its
own
config
and
change
the
way
that
it
behaves.
B
So
this
is
a
pattern,
that's
quite
useful,
but
we
need
some
way
to
do
that.
Conversion
right.
This
is
this
kind
of
creates
a
problem.
So
if
the
api
server
isn't
what
owns
the
kubelet
config
api,
then
we
don't
get
the
benefits
of
all
the
machinery
that's
built
inside
of
the
api
server.
There's
conversions,
defaulting
and
validation.
That's
normally
provided
as
those
apis
are
defined
and
registered
inside
of
what
would
normally
be
hosted
in
kubernetes,
but
now
we
don't
get
that
luxury.
B
This
allows
us
to
support
the
alpha
and
beta
versions
of
the
kublet
configuration
concurrently
with
a
single
version
of
the
kubelet.
We
do
this
through
an
internal
hub
type.
So
at
the
top
of
this
diagram
you
see
this
is
apps
internal
deployment,
but
imagine
having
a
kublet
configuration,
be
the
internal
type,
and
then
you
have
these
bi-directional
function
pairs
each
one
of
these
lines
or
these
spokes
in
the
diagram
connecting
to
the
hub
allows
us
to
then
get
from
v1.
B
Alpha
1
to
the
internal
type
or
v1
alpha
one
through
the
hub
type
to
the
v1
beta1
version,
so
you
can
build
tooling
around
these
apis
by
importing
the
go
native
library,
and
now
it's
fantastic,
that
we
have
these
internal
types
that
we
can
use,
go
native
libraries
and
api
machinery
to
extend
our
tools,
our
actual
programs
that
need
to
run.
But
in
a
world
of
controllers
and
operators,
we
actually
have
the
luxury
of
the
api
server.
Chris,
would
you
tell
us
a
little
bit
about
crds
yeah.
A
So
as
lee's
calling
them
crds,
they're
also
represented
as
custom
resource
definitions
and
you've
probably
heard
this
term.
It's
gotten
a
lot
of
buzz
in
the
last
year,
or
so
it
originally
spawned
from
a
feature
called
tprs
or
third
party
resources,
and
it's
been
around
for
a
bit
in
kubernetes,
but
when
you
start
to
actually
dive
into
it,
it's
not
that
complex.
A
If
you
understand
what
a
kubernetes
style
api
is
and
so
in
in
this
example,
you
can
see
that
we
have
an
api
version
and
a
kind
and
that
api
version
being
api,
extensions,
kates,
dot,
io
v1.
So
it's
a
stable
api
and
it
has
a
kind
of
custom
resource
definition
and,
as
we
kind
of
like
dive
into
these
things,
you
start
to
realize
what
it
really
is
doing.
A
So
it
is
allowing
us
to
take
a
kubernetes
api
server,
just
the
raw
api
server,
no
extra
servers,
no
extra
pods
that
are
actually
having
to
serve
direct
request
and
it
allows
us
to
dynamically
create
apis
against
that
that
control
plane.
So
in
this
example,
we're
defining
out
in
the
under
the
spec
field,
where
you
define
a
group
of
github.go.hind.dev
and
then
under
it
we
actually
define
some
kinds.
So
we
have
the
kind
of
repository
we
defined
the
list
kind
so
that
kubernetes
knows
how
to
parse
these
these
attributes.
A
And
then
we
define
the
singular
and
plural
version
so
that
it
understands
the
the
the
way
that
these
things
are
structured
and
how
how
people
might
address
them.
And
then
the
last
thing
in
here
you're
defining
a
version,
and
so
this
is
taking
a
yaml
manifest
and
saying.
I
want
to
tell
kubernetes
that
I'm
going
to
give
it
this
new
object
and
it
again
doesn't
need
an
extra
server
doesn't
need
anything
special.
It's
just
a
yaml
manifest
and
then
kubernetes
starts
to
do
some
special
magic
with
it.
A
Absolutely,
which
is
what's
so
cool
about
these
things,
and
I
think
that
they're,
probably
one
of
my
favorite
features
realistically
in
kubernetes.
I
spend
way
too
much
time
doing
these.
As
you
can
see.
This
example
is
about
creating
github
repositories
like
who
thinks
of
who
who
wants
to
go
and
manage
github
using
kubernetes.
It's
it's
a
weird
experience
and
and
more
like
an
experiment
for
me
to
do,
but,
as
you
can
see
here,
we're
taking
cube,
cuddle
and
saying
get
with
an
an
output
of
yaml
and
we're
calling
against
repositories
this
resource.
A
That
is
not
in
every
kubernetes
cluster.
It's
non-standard,
it
kind
of
doesn't
make
any
sense,
but
it's
fun
to
play
with,
and
now
I
can
say
I'm
to
create
a
repository
sample
and
under
it.
When
you
really
look
at
this
object,
you
can
see
everything
that's
going
on
behind
the
scenes.
I
have
my
gvk
defined
that
we
defined
using
that
kubernetes
manifest
before
the
actual
crd
manifest,
and
so
it's
github.go.gov
v1
alpha1
again
from
that
last
slide,
when
we
should
show
the
version
getting
exposed
and
the
kind
is
now
repository.
A
So
I
can
define
these
things
create.
As
many
of
these
as
I
want,
cubecut
will
apply
them
and
kubernetes
is
going
to
go
and
dynamically
create
these
endpoints
exactly
what
lee
was
asking
about,
creating
these
endpoints
that
you
see
under
the
self
link
under
metadata.
So
it's
going
to
be
slash
apis,
the
group,
the
version,
the
actual
kind
of
resource
and
then
the
repository,
the
actual
name
of
the
resource
that
it's
fetching
and
so
it
dynamically,
creates
these
things
and
allows
kubernetes
to
store
those
resources
for
you.
A
Yeah,
the
kubernetes
community
has
thought
of
it.
All
I
mean
we've
got,
we've
got
ways
to
do
all
of
these
things.
So
at
the
actual
yamo
level
you
can
define
the
schema
per
version,
and
so,
as
you
can
see
here,
we're
using
open
api,
v3
schema
and
say,
for
instance,
under
this
we
have
a
spec
attribute
and
under
that
there's
a
it
expects
there
to
be
a
home
page,
and
I
I
don't
want
to
get
requests
that
are
that
are
rejected
from
from
github.
So
I
have
a
pattern
match
on
that
attribute.
A
I'm
always
going
to
make
sure
that
it
matches
a
a
true
url,
so
it's
going
to
have
to
have
www
or
it's
going
to
have
to
have
http
or
https,
and
then
whatever
else
follows
after
that.
So
I
know
that
I
can
pre-validate
in
essence
before
I
send
off
to
another
endpoint
and
even
beyond
that
I
can
do
things
like
matching
based
on
the
type.
So
I
can
expect
an
integer.
I
can
expect
a
string.
A
So
now
I
have
this
this
structure
where,
in
the
in
the
comments
there
is
actually
where
you
define
that
schema
that
we
just
talked
about,
and
so
now
I
have
three
different
versions
all
exposed
at
the
kubernetes
control
plane.
I
didn't
create
a
web
server.
I
didn't
create
anything.
I
didn't
set
any
up
any
proxy
or
anything
like
that.
It's
just
purely
exposed
and
anytime.
I
make
a
request
to
v1
it's
going
to
it's
going
to
try
and
actually
serve
that
request
and
fulfill
that
request.
Now.
A
This
is
where
we
get
into
the
weird
space.
Now
when,
when
lee
was
talking
about
api
machinery,
conversions
and
how
that
works
in
in
component
config,
we
have
something
similar
in
this
space.
But
we
do
it
a
little
bit
differently
because
now
we're
on
the
on
the
back
end.
So
we
can't
just
do
it
client-side,
so
we
actually
define
a
conversion
strategy
for
these,
and
so
you
in
this
example
we're
using
the
web
hook
conversion
strategy.
A
You
can
actually
you
define
the
versions
that
you
want
to
convert
between.
So
when
this
api
server
gets
a
v1
or
a
v1
beta
1,
it's
going
to
know
to
take
that
object
and
lob
it
over
to
this
web
server
that
we're
exposing
you
can
hit
configure
the
mutual
tls
between
the
api
server
and
that
and
that
web
hook
server.
A
You
define
the
path
you
want
to
request
upon
and
it
sends
you
a
a
just
a
standard
object
that
you
can
parse
validate
convert
into
whatever
you
need
and
then
pass
it
back
to
the
api
server.
Now
how
this
all
works
behind
the
scenes
is,
is
continuing
to
be
slightly
different,
but
still
the
same
notion
at
the
end
of
the
day,
if
you
notice
back
in
the
in
the
versions
slide,
I
talked
about
served
and
stored.
In
that
example,
I
was
using.
A
So
when
you
do
this,
you
define-
and
this
is
following
the
patterns
that
we've
developed
within
qbuilder
these
days,
and
so
this
is
a
form
of
a
form
of
those
api
machinery,
but
instead
you
define
a
convert
to
and
a
convert
from
instead
of
having
a
handful
of
auto-generated
functions
from
api
machinery.
So
this
is
saying
I
want
to
convert
my
v1
beta
1
to
my
v1,
alpha
1,
and
if
I
wanted
to
convert
back
because
I
expected
to
get
a
v1
beta1
object
back.
A
I
can
convert
that
back
through
before
passing,
that
back
to
the
user
and
same
goes
for
the
v1
alpha.
One
and
what's
kind
of
interesting
here
is
like
as
you're
starting
to
realize.
We
can
tell
kubernetes
that
we
want
to
store
any
type
of
object.
I
created
a
repository
here,
it's
nothing
to
do
with
kubernetes,
but
it
doesn't
really
do
me
any
good
when
you
really
start
to
just
realize
that
this
is
just
being
treated
as
a
data
store.
So
lee,
can
you
talk
a
little
bit
about
controllers
and
how
those
work.
B
Yeah,
I'd
love
to
so
controllers
are
the
exciting
idea
of
kubernetes.
In
my
opinion,
right
and
the
value
of
a
controller
succinctly
is
that
it
allows
us
to
take
that
declared
data
and
then
build
a
level
triggered
system
around
what
it
declares
so
that
we
can
get
reliable
convergence
behavior,
I
say
level
triggered,
which
is
different
than
if
you've
ever
heard,
of
an
edge
triggered
system
which
uses
events
or
changes
in
values
translated
as
events
to
cause
controllers
to
operate
on
those
changes
where,
in
a
level
triggered
system.
It's
different.
B
Is
that
the
kubernetes
api
server
exposes
this
thing
called
the
watch.
The
watch
is
enabled
by
the
consistent
data
store
of
xcd,
and
it's
got
some
machinery
in
there.
That
gives
us
a
stream
of
events
with
some
guarantees,
and
so
without
fcd
you
don't
kind
of
have
the
basic
mechanisms
to
be
able
to
build
edge,
triggered
reconciliation
in
a
level
triggered
system
right
and
so
because
of
the
machinery
and
the
mechanisms
available
in
the
api
server
combined
with
what
fcd
gives
us.
B
As
a
data
store,
we
can
get
much
more
rapid
reconciliation
from
our
reliable
controllers,
and
so
when
you
combine
that
technical
solution
of
a
controller
with
the
fact
that
we
can
extend
the
kubernetes
api
with
custom
resources,
this
allows
us
to
build
our
own
controllers
that
allow
us
to
model
declarative
control
of
what
would
normally
be
really
error-prone
and
require
people
to
be
involved.
And
I
think
that
that's
super
exciting.
So
controllers
they're
the
big
idea,
but
you
might
have
heard
the
hot
term
of
operators
right.
Chris.
Can
you
help
me
understand?
A
Yeah
well,
besides
being
an
awesome
buzzword
these
days,
it's
there's,
there's
some
meat
behind
this.
So
what
a
what
an
actual
operator
is
meant
to
be.
Is
it
something
that
takes
that
that
control
loop
theory
and
the
way
that
we're
building
kubernetes
and
it
tries
to
take
non-standard
tooling
to
bring
them
into
that
world?
A
You
need
that
config
to
be
managed,
and
you
also
need
to
make
sure
that
you're
not
leaking
the
abstractions
of,
what's
really
going
on
under
the
hood
and
making
sure
that
you
define
a
solid
api
that
that
follows
those
declarative
patterns,
and
this
is
where
we
bring
back
in
the
notion
of
okay.
These
probably
are
going
to
be
be
created
out
of
the
box
with
things
like
flags,
so
we're
back
to
this.
This.
B
World
that
seems
a
little
problematic
right.
We
talked
about
how
flags
you
know,
as
you're.
Trying
to
maintain
the
configuration
of
a
component
can
become
a
little
bit
difficult
to
maintain
guarantees
for
you
know,
especially
if
you're
writing
an
operator
and
you're
trying
to
distribute
that
to
other
teams
that
are
deploying
and
getting
value
out
of
the
new
apis
that
you
are
controlling.
B
You
want
to
be
able
to
give
somebody
a
guarantee
that
the
way
that
the
operator
is
configured
to
work
is
going
to
function
next
year,
right
and
so
maybe,
instead
of
using
flags,
we
could
adopt
the
kubernetes
native
solution
right.
What
we're
doing
in
the
actual
components
inside
of
kubernetes
can
we
use
component
config
inside
of
operators
right.
So
in
this
diagram
we
have
a
deployment.
A
deployment
is
facilitated
in
kubernetes
because
of
other
control
loops
that
are
running
in
the
cluster.
B
We
have
the
controller
manager
which
is
actually
operating
the
deployment
struct
and
then
creating
pods.
We
have
the
coupe
scheduler,
which
is
assigning
those
pods
to
nodes.
We
have
the
kubelet
watching
for
pods
to
end
up
on
its
node
and
then
actually
schedule
it
with
the
config
maps
and
secrets
and
volume
mounts
and
all
that
properly
configured,
so
the
pod
runs
in
the
way
that
we
specify
right.
So
we
have
these
control
loops,
taking
care
of
us.
B
A
dynamic
version
of
this
example
involves
a
little
bit
of
different
machinery
here.
What
we
would
do
is
we
would
modify
the
operator's
control
loop
to
actually
keep
a
watch
with
the
api
server
for
its
component
config
from
a
data
field.
This
could
also
be
driven
by
a
secret
instead
of
a
config
map,
and
so,
instead
of
relying
on
the
controllers
that
are
inside
of
the
cluster
say,
we
want
to
deploy
the
operator
externally
from
the
cluster,
but
still
get
this
dynamic
component
config
reload.
B
This
would
be
the
analogy
to
the
dynamic
config
experiment
that
we've
been
running
but
directly
built
into
the
operator's
controller
to
change
the
way
that
it
behaves,
and
so
this
is
how
we
can
use
component
config
and
the
existing
examples
and
kind
of
like
history.
That's
already
been
established
there
for
the
technical
implementation
of
using
kubernetes
apis
to
give
promises
for
configuration
for
our
controllers
themselves,
bootstrapping
them
actually
running
them
and
getting
them
to
connect
to
an
api
server.
B
Do
leader,
collection
between
themselves,
configure
auth
and
mutual
tls,
but
there
is
an
even
more
interesting
world
beyond
component
config
beyond
just
bootstrapping
the
controllers
and
getting
them
to
update
in
a
consistent
way,
and
that
has
to
do
with
then
using
custom
resources
in
the
control
loop
to
change
behavior.
Chris,
can
you
tell
us
about
it?.
A
Yeah,
absolutely
I'd
love
to
again
crds
are
like
are
my
bread
and
butter
these
days.
I
I
love
the
the
thoughts
that
you
can
do
with
these
things,
and
so
taking
taking
custom
resources
and
and
trying
to
fit
this
into
a
model
of
managing
configuration
and
dynamic
configuration
is
pretty
fantastic
because
you
have
informers
that
are
built
in
as
long
as
you're
using
the
client
go
libraries
and
all
that
you
have
informers
that
you
can
set
watches
on,
and
you
can
say
anytime,
this
config
cr
changes
or
anytime
a
new
config
cr
gets
added.
A
I
can
load
that
in
and
I
can
dynamically
reconfigure
my
my
controller
or
my
operator,
and
so
you
see
this
actually
being
done
in
a
lot
of
a
lot
of
places
already,
and
this
is
trying
to
really
bring
up
this
as
a
pattern
and
make
sure
that
people
know
that
we
can
get
some
really
good
benefits
out
of
it.
So,
for
example,
this
is
actually
currently
being
implemented
directly
into
controller
runtime
and
it'll
be
coming
out.
A
I
believe
in
the
next
release
in
the
next
couple
months
and
that
allows
us
to
automatically
have
those
version.
Schemas
directly
with
any
controller
runtime
based
project,
and
so
when
this
gets
released,
that'll
end
up
in
operator,
sdk
it'll
end
up
in
cube
builder
and
you'll
be
able
to
create
operators
that
are
already
using
these
or
create
controllers
that
are
already
using
these
configuration
files.
But
then
we
can.
We
can
munch
new
worlds
into
it
to
do
that,
that
run
time
level
configuration.
A
B
Yeah
cert
manager
and
contour
have
some
great
prior
art
on
this
they're.
Some
of
the
earliest
and
most
successful
controllers
and
operators
in
the
ecosystem
and
cert
manager
has
this
idea
of
an
acme
issuer,
and
so,
if
you
have
these
namespaced
issuers
or
cluster
issuers,
then
you
are
able
to
then
pair
those
up
with
the
certificate
objects
that
ultimately
need
to
be
certified
by
the
issuer.
B
That's
running,
and
this
is
an
example
of
cert
manager,
providing
a
multiple
instance
of
configuration
api
that
then
changes
the
way
that
the
control
loop
treats
other
resources
in
the
same
cluster
and
then
we
get
to
contour,
which
is
my
favorite.
They
invented
this
idea
of
namespace
delegation,
I'd
really
like
to
chat
with
who
came
up
with
this,
but
I
saw
a
great
talk
by
josh
russo
a
couple
years
ago
talking
about
this
idea.
B
We
also
used
this
idea
of
namespace
delegation
inside
of
flux's
helm
operator,
and
so
this
is
a
it's
a
very
powerful
pattern
that
allows
you
to
use
a
custom
resource
of
one
type
in
a
parents
or
administrative
name
space,
and
with
the
way
that
you
declare
that
custom
resource
in
that
administrative
namespace.
You
can
change
the
way
that
the
controller
reacts
to
and
behaves
on,
custom
resources
of
another
type
in
the
target,
namespace,
and
so
the
way
that
they
use
this
in
contour
is
with
their
old
ingress
route
api.
B
You
could
do
ingress
route
delegation
which
would
allow
you
to
administer
the
way
that
subdomains
were
treated
in
other
name
spaces
to
restrict
tenants
and
then
in
flux.
With
helm
operator,
we
have
a
helm
release
delegation
which
allows
you
to
declare
helm
releases
in
one
namespace
that
ultimately
end
up
deployed
in
another
namespace,
so
great
pattern
there.
B
You
also
see
this
with
ignore
annotations
and
that
kind
of
thing,
but
creating
formal
apis
around
that,
so
people
can
depend
on
the
version
so
that
you
know
the
you
always
have
a
reliable
way
to
turn
off
your
operator
for
the
years
to
come,
that
it's
supposed
to
be
delivering
value
to.
You
is
something
to
consider
when
building
your
own
controllers,
so
great
examples
from
the
real
world
and
ultimately
I
think
that
when
you
combine
the
mechanisms
with
controllers
in
the
kubernetes
world,
we
can
get
away
from
this
config
drift
problem
right.
Chris.
A
Absolutely-
and
you
can
kind
of
see
this-
I
mean
if
we,
if
we
really
start
to
look
at
it.
The
new
world
of
this
is
going
to
allow
dynamic
configuration
to
be
passed
directly
to
your
operators.
The
control
loop
is
gonna,
go
and
go
and
configure
everything
for
you,
and
so
this
world
of
a
a
server,
that's
out
of
sync
is
just
not
gonna
happen.
So
realistically,
if
we
look
at
carol's
example,
again,
she's
gonna,
deploy
it's
gonna
active,
actively,
reconcile
and
eventually
get
to
their
correct
state.