►
From YouTube: Kubernetes is Your Platform: Design Patterns For Extens... Rafael Fernández López & Fabrizio Pandini
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes is Your Platform: Design Patterns For Extensible Controllers - Rafael Fernández López, SUSE & Fabrizio Pandini, VMware
Developing Controllers -- as well as other Kubernetes native extensions like admission webhooks -- is quickly becoming a mainstream practice to solve problems in a Kubernetes native way; but while developing a simple controller is pretty straightforward, things become complex as soon as you have behavioral dependencies with other components. But don't worry, this talk will provide you with reusable design patterns derived from the concrete experience and the hard lessons learned by the maintainers of Cluster API and Kubewarden, two projects built around the idea of extensible controllers. How to develop a plug-in system for your controller/admission webhook? How to add to your controller the capability to do RPC calls to pluggable external components? How to orchestrate many controllers co-operating in solving complex tasks? Come to this talk, we got you covered!
A
So
welcome
good
morning
so
yeah,
it's
cool
that
this
started
so
welcome
to
kubecon,
and
today
we
are
going
to
talk
about
when
kimuras
is
your
platform
and
some
design
patterns
for
extensible
controllers.
So
my
name
is
rafael.
I
work
for
suse,
I've
been
working
with
with
kubernetes
for
five
years
and
a
half
already
and
yeah
I
have.
I
have
contributed
a
little
to
cube
admin,
for
example,
and
with
me
it's
fabrizio.
B
B
B
Developing
a
controller
is
becoming
a
mainstream
practice
to
solve
a
problem
in
a
cloud-native
way.
You
can
use
a
tool
like
kuber
builder
operator,
sdk,
to
create
a
kubernetes
controller
in
few
minutes
and,
and
the
idea
is
that
the
controller
can
watch
a
customer
resource
and
basically
reconcile
the
state
of
your
system
to
the
to
the
to
the
desired
state
that
you
define
in
the
spec.
B
B
Basically,
you
get
to
align
that
every
developer
must
cross
where
your
controller
alone
is
not
enough
to
solve
the
problem.
So
today,
presentation
is
about
sharing
some
pattern.
Some
lesson
learned
about
how
to
cross
this
line,
and
I
before
starting
I
I
will
just
to
point
out
that
what
we
are
showing
today
is
based
on
experience.
The
business
periods
of
rafael
working
on
projects
like
cuba,
warden,
is
basing
on
experience
that
we
learned
in
cluster
api,
and
there
are
also
some
idea
on
how
we
can
improve
this
in
the
future.
A
But
then
you
need
to
you
need
to
keep
track
of
of
everything
and
they
need
you
need
to
orchestrate
them
somehow
so
in
the
next
slides.
What
we
are
going
to
do
is
to
check
this
last
part
for
now
so
before,
starting,
let's
clarify
some
terms,
because
there
are
some
loaded
terms
in
in
this.
In
this
thing,
and
so,
for
example,
the
controller.
What
we
refer
as
a
controller
in
this
stock
and
in
general,
is
just
a
single
reconciled,
loop
of
a
resource.
A
A
custom
resource
is
an
instance
of
a
custom
resource
definition,
and
then
we
have
the
owners
in
concept,
which
is
a
controller,
owns
a
resource,
and
is
this
controller
they're
responsible
for
driving
the
observed
state
of
a
resource
to
the
desired
state
of
this
resource,
and
then
the
act
of
watching
is
being
being
be
interested
in
in
something
happening
in
the
cluster.
So
we
need
to
act
before
because
of
that
in
order
to
drive
our
resource
to
the
desired
state
of
what
we
observed
in
the
world.
B
Okay,
so,
finally,
it's
time
to
start
talking
about
design
pattern
and
for
warming
up,
let's
start
to
focus
on
our
first
family
of
design
pattern
and-
and
what
we
are
focusing
on,
is
how
to
orchestrate
many
controllers,
it
could
seem
counterintuitive
starting
from
many
controllers,
but
in
reality
we
have
this
kind
of
pattern
in
kubernetes
since
the
beginning.
So
we
start
from
this
set
of
pattern
because
we
hope
that
they
are
familiar
to
the
audience
and-
and
this
will
basically
help
to
build
up
for
the
rest
of
the
presentation.
A
Yeah,
so,
let's
start
with:
let's
start
talking
about
what's
in
about
the
controls,
what's
in
the
same
object,
so
idea
is
pretty
simple:
we
have
one
controller
that
is
responsible,
that
owns
a
custom
resource
and
is
responsible
for
driving
this
resource.
In
this
case,
for
example,
if
you
think
about
cluster
api,
we
can
say
that
machine
controller
owns
the
machine
custom
resource.
So
just
as
a
reminder,
let's,
let's
I'm
going
to
repeat
that
again.
A
So
this
would
mean
zoning
and
so
say
that
we
want
to
extend
this
this
controller
and
we
don't
want
to
introduce
more
logic
into
it.
So
one
straightforward
way
would
be
that
we
have
yet
another
controller.
In
this
case,
for
example,
we
want
to
create
backups,
so
we
have
the
volumes
backup
controller
and
it's
going
to
do
backups
from
time
to
time
when
we
want
it
to
do.
But,
for
example,
we
are
going
to
to
delete
a
machine
right
now
and
we
want
to
to
make
a
backup
right
before
this
machine
is
really
deleted.
A
So
what
we
can
do,
for
example,
instead
of
putting
that
logic
into
the
machine
controller
which
will
make
it
harder
to
maintain
harder
to
understand.
We
have
just
the
volumes
backup
controller
that
is
just
doing
the
backup
and
what
it's
going
to
do
is
something
is
going
to
add
a
finalizer
to
other
to
our
resource
and
this
volumes.
A
Backup
controller,
for
example,
is
going
to
to
watch
for
this
object
and
when
it
finds
a
deletion
time
stamp
on
it,
it's
going
to
do
all
the
triggering
of
the
backup
and
then
when
it's
done
it
will
remove
the
finalizer
and
so
kubernetes
will
garbage
collect
the
resource.
So,
in
the
end,
the
machine
customer
the
machine
custom
resource
will
get
garbage
collected
when
we
have
the
final
backup
of
the
machine
before
even
it's
completely
gone.
A
So
some
lessons
learned
here
use
when
you
want
to
make
progress
after
a
precondition
is
met.
In
this
case,
for
example,
the
delete
on
timestamp
of
the
controller
is
non-null,
and
I
want
to
do
something
like
I
want
to
do
a
backup
right
now.
The
pros
is
that
the
repos,
the
responsibility,
is
very
well
defined.
I'm
not
adding
complexity
to
the
controller
that
I
have
I
I
am
adding
this
on
another
controller.
That
is
doing
only
this.
It's
a
battle
tested
pattern.
A
We
know
it
very
well
in
kubernetes,
and
we
see
this
on
lots
of
controllers
that
are
very
widely
used
and,
and
some
cons,
for
example,
is
that
yeah
you
you
need
to
orchestrate
somehow.
So
in
this
case
you
could
add
annotations
or
you
could
have
something
to
to
to
sequence
your
operations
if
you
are
working
with
with
a
single
resource
and
then
there
is
behavioral
dependencies
between
controllers
that
are
not
very
well
documented,
we'll
go
through
that
later
on,
but
yeah
the
we.
A
B
Let's
move
to
a
slightly
more
complex
pattern,
and
also
this
is
existing
kubernetes
since
the
early
days,
so
we
are
using
an
example
from
kubernetes.
You
all
know.
You
all
know
that
there
is
a
deployment
controller
that
owns
the
deployment
and
when
the
primary
is
reconciled,
what
happens
is
that
it
creates
another
resource
which
is
the
replica
set.
B
So
let's
talk
for
a
second
on
on
what
is
happening
is
that
there
is
a
relation
between
two
customer
resources
in
this.
The
simplest
way
to
document
this
relation
is
saying
that
this
is
a
parent-child
relation.
Okay
and
second,
there
is
also
like
an
intimate
knowledge
between
those
pieces.
So
the
deployment
controller
know
exactly
what
a
replica
set
is
how
the
spec
is
done,
how
the
status
updates.
So
those
are
the
principle
of
this
pattern,
and
the
last
bit
is
that
there
is.
B
B
So
when
the
the
main
point
is
that
we
can
reuse
this
pattern
when
there
is
a
relation,
a
parent-child
semantic
relation
between
the
the
object
that
that
you
control.
What
are
the
pros
is
that
this
is
battle
tested.
It
exists
in
kubernetes
and
it
it
scaled
well.
You
can
have
different
layering
of
complexity,
machine
deployment,
replica
set
and
pod
which
are
controlled
by
the
kubelet.
B
Also,
we
we
all
now
know
very
well
that
it
could
become
pretty
sophisticated.
You
can
add
history
exchange
rollbacks
deployment
strategy,
so
it
is
a
pretty
powerful
pattern.
What
are
the
cons
is
that
we
have
two
resources:
two
customer
resources.
We
have
two
controller
and
whenever
the
progressing
of
your
process
gets
stuck
is
harder
to
debug,
you
don't
know
exactly
to
which
controller
locks
to
look,
etc,
etc.
B
This
we
are
using
an
example
for
cluster
api,
so
we
have
the
a
machine
controller
that
basically
owns
the
machine
and
we
all
know
that
a
machine
could
be
creating
on
different
type
of
infrastructure.
In
this
example
is
we
are
using
the
sphere
infrastructure,
but
what?
If
we
want
to
create
our
machine
on
a
different
infrastructure,
aws
gte,
whatever
we
don't
want
to
rewrite
a
different
machine
controller?
We
want
to
use
the
same,
and
what
is
the
key
to
make
this
possible?
B
We
assume
that
all
the
machine
at
the
end
has
to
have
a
field
that
is
called
machine
infrastructure,
ready
that
basically
feedback
to
the
to
the
machine
controller.
The
only
information
that
he
cares
about
my
machine
is
ready.
Okay,
this
field,
basically,
is
our
api
contract
and
by
using
this
api
contract,
what
we
can
do
we
can
create
a
controller
that
owns
the
machine
implementation,
but,
given
that
the
dependencies
between
this
layer
down
and
the
layer
up
is
a
well-known
field,
basically,
the
lower
level
is
swappable,
and
this
is
pretty
powerful
in
cluster
api.
B
We
are
using
these
extensively
so
lesser
learner,
using
this
part
when
we
can
use
it
when
there
is
this
semantic
relation
between
object
between
an
object
and
a
set
of
related
objects
in
the
key
word
in
this
case
is
set,
and
the
example
was
machine
and
all
the
machine
infrastructure,
implementation,
okay,
second
and
for
most
important
part,
is
that
we
can
use
this
pattern.
When
there
is
a
it
is
possible
to
identify
a
set
of
common
api
fields
between
all
the
machine
implementation.
B
This
makes
the
pattern
generic.
In
the
example
we
discussed
about
status
infrastructure
already,
the
pro
it
allows
whoppable
implementation.
It
is
powerful
in
class
api,
where
I
don't
know
more
than
30
implementation
of
machine
infrastructure,
so
it
works.
Well,
we
use
this
extensively.
What
are
the
cons
is
that,
in
order
to
agree
agree
to
it
to
this
contract
is
something
that
required
discussion
among
all
the
stakeholder
of
the
stakeholder.
B
So
it
requires
a
certain
degree
of
communication
and
I
think
the
classroom
api
community
to
make
this
possible
another
downside.
Also
the
contract
is
defined
most
probably
over
time.
You
have
to
evolve
it
and
these
again
require
coordination,
etc,
etc,
and
the
last
bit
of
information
is
that,
given
that
you
cannot,
basically,
when
you
implement
your
machine
controller,
you
don't
know
the
the
concrete
implementation
of
the
machine
infrastructure.
A
B
A
Then
we
saw
how
we
can
have
strictly
custom
resources
or
even
hierarchical,
custom
resources
managed,
and
then
we
saw
how
we
can
have
a
very
small
field
of
set
of
fields
that
we
can
put
as
a
contract.
Now
it's
time
to
shift
perspective.
So,
instead
of
working
on
on
this
space,
we
are
going
to
see
how
we
can
extend
a
single
controller,
and
so
the
first
one
is
is
the
simpler
one.
A
So
when
it
comes
to
extend
the
capabilities
of
a
single
controller,
the
first
thing
to
notice
is
that
this
approach
is
not
used
extensively
in
kubernetes.
So
we
can
see
that,
for
example,
you
can
plug
your
own
scheduler,
which
is
slightly
different,
but
you
can
you
can
do
that.
You
also
have
dynamic
admission
control,
but
this
is
not
really
a
controller,
so
you
could
say
that
you
already
have
some
kind
of
extensibility,
but,
for
example,
the
scheduler
example
that
I
put
is
just
replacing
the
scheduler
is
not
like
extending
it.
A
So,
despite
the
idea
of
extending
a
control
is
not
yet
used
in
core
kubernetes
extensively,
we
see
that
on
the
ecosystem,
this
is
getting
traction
more
and
more,
and
so
we
are
going
to
talk
about
two
main
designs
that
will
allow
allow
us
to
do
this.
So
it's
one
of
it
is
in
process
extensions
and
the
other
is
out
of
process
extensions.
A
A
A
So,
let's,
let's
look
into
into
the
lessons
learned
about
this
one.
So
it's
it's
it's
cool
when
you
can
execute
external
banners,
because
this
is
the
other
problem.
This
is
running
in
our
in.
In
the
same
name
space
as
our
controller,
can
we
trust
this
binary?
What
it's
going
to
do
for
for
the
pros?
This
is
similar
to
the
cni
and
the
valero
model.
We
already
know
it
works.
We
we
have
already
people
using
that
and
also
the
exp.
A
The
error
space
is
narrow
in
the
sense
that
we
don't
have
the
network
in
the
middle,
for
example.
So
we
just
have
a
binary,
we
call
it
and
we
wait
and
for
for
cons,
we
need
to
do
some
kind
of
binary
either
download
or
registration
ahead
of
time.
So
that's
that's
something
that
we
have
to
do
so.
It's
also
hard
to
prevent
side
effects,
for
example
produced
by
the
binary.
We
need
to
understand
what
the
binary
is
doing.
Is
it?
Can
you
run
it
again?
A
It's
it's
the
kind
of
problems
that
we
find
with
maybe
binaries
that
we
don't
control
completely
and,
of
course,
you
need
to
have
binary
compatibility.
This
is
usually
is
not
a
problem
maybe,
but
you
need
the
os
and
the
architecture
of
your
machine
to
match
the
controller
where
the
controller
is
running,
and
so
because
of
that
there
is
a
variant
of
that
one.
That
is,
we
see
also
this
coming
more
and
more
over
time.
A
I'm
not
going
to
put
a
lot
of
time
into
that,
but
this
is
just
the
variant
of
webassembly
instead
of
having
a
binary
binary
that
needs
to
match
our
os
and
our
architecture.
It's
a
webassembly
binary
and
our
controller
just
creates
a
runtime
for
webassembly
and
runs
it
and
a
good,
a
good
thing
about
this.
It's
basically
the
same.
A
B
B
B
In
the
example,
we
are
calling
webook
configuration
customer
source
that
basically
extract
or
define
where
your
plugin
is
so
it
gives
the
or
it
gives
the
ci
bundle
to
connect
it,
and
everything
is
necessary
to
for
your
system
to
talk
with
the
plugin.
It
acts
as
a
disk
discovery
mechanism.
Basically,
when
the
plugin
is
discovered,
the
next
step
is
you
also
inside
your
controller?
B
You
also
have
your
controller
car
logic,
so
the
plugin
get
registered
in
your
core
logic,
so
whenever
customer
resource
get
reconciled
and
whenever
something
has
to
be
done,
your
call
logic
at
this
point
is
capable
to
call
the
plugin
which
is
responsible
from
some
additional
behavior
or
or
that.
Basically,
it
extends
your
controller.
B
So
taking
a
a
little
bit
step
back
on
this
pattern,
few
things
are
worth
to
notice.
So
the
core
idea
of
this
pattern
is
to
run
plug-in
in
a
separate
process.
That's
the
main
goal,
everything
else.
The
discovery
mechanism
that
we
discussed
is
plumbing
is
something
that
helps
us
to
reach
the
final
goal,
that
is
to
run
the
plugin
it
can
be,
it
could
be
implemented
in
a
different
flavor.
B
We
showed
that
this
implementation,
because
it's
it
reminds
the
how
the
dynamic
extension
of
a
book
works,
so
maybe
that
some
of
you
are
familiar
to
the
web
mutation
web
configuration
and
stuff
like
that.
So
it
is
the
same.
Let
me
say
approach
second,
things
were
to
notice.
Is
that,
with
respect
to
the
previous
pattern,
there
is
a
lot
of
moving
parts.
B
This
is
something
that
that
we
have
to
keep
in
mind.
So
let's
talk
about
lesson
learned
so
when,
when
you
have
to
use
it
when
it
makes
sense,
basically
to
take
charge
of
all
these
components,
point
number
one
is
that
when
your
plugin
basically
require
different
security
context,
I
make
you
an
example
in
cluster
api.
B
Our
controller
are
have
a
pretty
powerful
airbag
permission.
Most
specifically,
they
can
access,
for
instance,
the
cloud
identity
of
stuff,
and
we
don't
want
that
bargain
to
be
run
with
the
same
identity,
because
it
could
be
dangerous
so
running
by
running
the
plugin
into
separate
processing
into
a
separate
port.
We
can
assign
the
plugin
a
dedicated
security
context
with
very
limited
airbag
permission.
B
B
The
last
point
is
that
there
are
also
some
organizational
implications
in
this
pattern,
because
you
can
have
basically
two
different
person
which
are
responsible
of
what
gets
into
into
the
system.
So
if
you
have
a
person
that
is
responsible
to
manage
the
web
book
configuration,
this
person
is,
is
basically
able
to
get
which
plugin
gets
used,
and
this
is
a
sort
of
security
mechanism
that
can
help
in
organizations.
A
Yeah,
I
I
won't
talk
too
much
about
this
one,
but
this
is
the
is
the
same
as
the
previous
one.
Only
that
the
service,
instead
of
being
outside
of
the
network,
is
inside
of
ours
of
our
same
pod.
So
it
serves
the
network,
the
network
namespace
of
the
kernel.
So
you
can
just
reach
your
your
service
through
the
localhost.
So
that's
that's
the
variant
of
this
one
so
yeah.
B
B
B
A
Yeah,
so
with
this
in
mind
that
there
is,
there
is
a
set
of
problems
to
get
solved.
There
is
a
lot.
This
list
is
very
long,
but,
for
example,
the
first
question
is
how
to
document
behavioral
dependencies,
so
we
we
know
how
to
document
apis.
We
have
api
documentation,
but
how
we
document
behavioral
dependencies
like
this
control,
is
going
to
be
that
and
then
this
one
is
going
to
pause.
A
Until
this
happens,
this
kind
of
thing
textual-
it's
not
so
good
right,
so
how
we
do
that,
how
to
make
it
easier
to
back
when
a
resource
is
not
is
not
progressing,
why
this
resource
is
still
on
pending.
What
is
going
on,
we
have
we
have.
We
have
ways
to
do
that
and
seek
instrumentation
is
going,
is
working
on.
That
is
providing
us
a
really
good
tools
to
do
that.
B
So
we
have
a
lot
of
questions,
so
the
next
point
is
how,
as
a
community,
we
can
react
to
this.
So
do
we
need
some
framework
in
cluster
api.
We
are
starting,
we
are
developing,
we
have
a
proposal
and
we
are
matching
some
code
with
relate
with
something
related
that
we
call
runtime
sdk,
which
is
at
the
end.
It
is
a
small
framework
that
is
designed
to
help
you
to
manage
out
of
processes
extension.
B
B
Basically,
we
can
start
working
together
to
a
set
of
library
that
will
make
the
life
of
the
next
person
implementing
a
controller
much
easier.
And
finally,
we
can
have
also
the
same
approach
to
to
promote
like
how
do
we
package
our
plugin?
How
do
we
distributed
them
do
do
we
have
to
reinvent
something
or
we
can
rely
on
standard
like
oci
or
something
else
that
exists
today?
B
So,
in
conclusion,
extensible
controller
is
a
very
interesting
topic
and,
despite
the
work
that
we
did
in
some
project,
we
we
agree
that
there
is
still
much
to
to
explore
in
this
space
and
and
the
only
way
that
we
know
is
by
try
and
even
some
time
failing.
So
it
is
a
challenge,
but
also
an
opportunity.