►
From YouTube: Writing Kubernetes Controllers for CRDs: Challenges, Approaches and Solutions - Alena Prokharchyk
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).
Writing Kubernetes Controllers for CRDs: Challenges, Approaches and Solutions - Alena Prokharchyk, Rancher Labs, Inc. (Intermediate Skill Level)
Kubernetes as a platform is at the stage where more and more features are being developed and deployed as external controllers. And writing an extension in most of the cases implies use of CRD - a Custom Resource Definition - to manage and represent the resources your application creates. During this talk we will go over the entire management cycle for CRD resource starting with defining types, schema and client, following by controlling the resource using informers and leveraging object's fields like: * Spec, Status, Metadata, Labels * Finalizers * Owner References * Conditions Each aspect of CRD management has its own challenges, and they will be highlighted along with the alternative solutions as a part of a live demo.
About Alena
Alena is a Principal Software Engineer at Rancher Labs, who's been working on building infrastructure services first for Virtual Machines, now for containers with main focus on Kubernetes. She enjoys helping others make sense of problems and explore solutions together. In her free time Alena enjoys rollerblading, reading books on totally random subjects and listening to other people's stories. Participated as a speaker in Kubecon 2016/2017 USA.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hello,
everyone
thank
you
for
coming
to
the
session
I'll
try
to
make
it
fun
we're
going
to
extend
kubernetes
by
writing
a
custom
resource
and
a
custom
controller
for
it,
and
a
little
bit
about
myself,
I'm
originally
from
Eastern
Europe
Belarus,
so
I
speak
Russian.
Ten
years
ago,
I
moved
to
United,
States
and
now
I
work
as
a
software
engineer,
rancha
labs,
we
work
on
the
open
source
project
that
extends
kubernetes.
So
we
write
a
bunch
of
controllers
and
a
bunch
of
custom
resources
and
I
hope
to
share
some
experience
with
you
and
I'm
curious.
A
How
many
of
you
actually
do
write
custom
controllers
for
kubernetes
all
right
and
how
many
of
you
are
thinking
about
writing
custom
controllers
for
darkness
all
right,
I,
like
it
already?
Okay.
So
when
do
you
actually
start
thinking
about
writing?
Custom
controllers?
It's
when
you
need
some
specific
functionality.
There
is
not
a
part
of
core
kubernetes
and
you
see
you
don't
think
it's
generic
enough
to
become
a
part
of
it
or
you
want
to
maintain
your
future
release
lifestyle
for
yourself.
A
That's
where
you
go
with
a
custom
resource
and
a
custom
controller
and
yeah
every
custom,
controller
or
custom
feature
implies
a
custom
resource
that
user
can
view
modifying
and
monitoring
and
the
controller
appraised
on
the
custom
resource
by
monitoring
the
resources
and
running
the
backend
logic
for
based
on
the
resource
definition
and
updates
the
object
state
of
the
resource,
and,
if
you
think,
about
the
kubernetes
ecosystem,
it
consists
of
the
custom
controllers,
like
clogging
a
learning
monitoring.
Everything
is
out
there.
There's
not
a
part
of
kubernetes
is
a
custom
controller.
A
Now
I
would
really
like
to
focus
on
the
custom
resource.
During
this
presentation
you
hear
a
lot
about
custom
controllers
and
the
best
practices,
but
how
do
like?
How
do
you
actually
build
your
customer
source
like
what
are
the
best
practices
for
that?
So
they're
like
the
little
intro
to
the
customer
source
to
your
user?
It
shouldn't
matter
from
the
client
perspective.
It
shouldn't
matter
whether
the
resource
is
the
part
of
the
kubernetes
core
project
or
whether
it's
a
some
external
resource.
He
wants
to
use
the
cube
CTL.
A
He
wants
to
use
the
UI
the
API
and
he
wants
to
lease
these
resources,
as
he
would
least
pause
for
example.
On
this
picture,
I
gave
an
example
of
logs
and
alerts.
We
shall
like
certain
resources
controlled
by
external
controllers,
and
he
can
just
user
can
just
say,
keeps
it
he'll
get
alert
and
he
would
get
the
customer
source
back
now.
What
actually
makes
a
customer
source?
What
are
the
best
practices
for
that
I've
listed
some
as
some
parts
of
the
customer
source
in
this
picture.
A
The
first
one
is
the
metadata
and
the
green
parts
on
it
is
United
standard
felts,
it's
going
to
be
like
a
name
namespace.
If
your
resources
namespace
is
going
to
be
labels,
annotations
finalizes
object
references
initializers
if
you
use
them
well,
we
don't
in
our
project,
but
it's
a
useful
feature,
and
then
there
is
a
spec.
Spec
is
a
desired
set
of
the
resource.
A
So
let's
build
something
from
scratch:
let's
build
a
kubernetes
cluster
management
tool
which
will
allow
user
to
create
kubernetes
cluster
view
them
using
the
cube,
CTL
and
will
also
give
user
and
a
chance
to
use
the
cube
config
and
to
connect
to
this
newly
generated
clustering.
Now,
what
are
we
going
to
showed
you
in
the
demo
we're
going
to
show
the
client
generation
from
scratch?
A
I'm
gonna
use
this
open
source.
Our
key
tool,
which
is
to
build
by
Rancher
but
like
any
tool,
can
be
used.
I
just
prefer
to
use
our
tools
like
I'm
sure
that
you're
going
to
run
into
issues
even
the
demo
and
I
was
like
through
for
this
issues
back
to
the
team,
and
then
here
is
a
demo
control
of
logic
that
can
be
found
here
on
the
github,
alright
demo
time.
A
So,
as
I
mentioned,
we
start
with
first.
What
do
you
start
with?
You
start
with
defining
your
resource,
so
our
resource
is
clustered
the
first
two
fields.
Type
meta,
an
object
matter
is
a
standard
kubernetes
fields,
the
next
two
fields,
cluster
spec
and
cluster
status-
is
our
our
specific
fields
and
also
define
them
in
the
same
file.
A
It's
better
I
would
like
to
really
keep
this
I
tried,
presentation
mode,
but
I
lose
the
structure
of
the
I
use.
This
structure
and
I
would
like
to
demo
it
yeah,
so
I
forgot
to
mention
one
thing:
the
code
generator
requires
you
to
have
a
specific,
specific
layout
for
type
definition.
So
it
all
starts
with
this
pick.
A
Okay,
here
you
go
so
yes,
the
quest
cluster
status.
This
condition
is
somewhat
the
generic
field
recommended
way
of
storing
the
resource
seed
and
it's
actually
an
array
of
the
condition,
because
you
can
have
more
than
one
controller
operating
the
same
resource
and
it's
better
that
one
condition
is
used
and
said
by
one
controller.
Actually,
any
other
controller
can
evaluate
it,
but
it's
better
that
it's
only
one
controller
that
updates
this
field
and
then
I
also
have
this
absurd
generation
field.
But
it's
actually.
There
are
some
issues
in
using
it.
A
A
A
The
health
checker
controller
will
actually
check
for
the
cluster
health
seed
and
will
update
the
appropriate
condition
and
there's
also
confusion
area
controller
that
is
going
to
generate
the
cube
config
based
on
the
cluster
once
it's
provisioned
and
there's
one
extra
controller
that
is
purely
for
the
annotations
of
these.
So
let's
look
at
the
provisionary
controller.
A
A
A
A
It's
pretty
simple.
It's
just
address
where
your
cluster
is
going
to
recite
the
the
credentials
and
the
role
for
which
node
for
for
this
particular
host.
So
the
controller
working
on
a
CD
of
the
cluster,
the
Fiona
provision,
all
gonna
reside
on
a
single
node.
So
and
now,
let's
start
our
controller
and
before
we
create
a
cluster
a
little
bit
about
my
setup,
I
run
our
sects
and
I
have
mini
cube
so
and
I
run
my
controller
here,
so
what
it
does.
A
A
Alright,
so
what's
going
on
here,
so
you
can
see
the
message
that
cluster
is
updated
provisioning.
So
the
actual
provisioning
is
going
to
happen
here
for
I
think
it's
it
will
take
a
while
because
we
download
the
images
the
kubernetes
cluster
gets
deployed
in
the
container.
We
download
the
images
here,
but
let's
just
focus
on
this
message.
It
says
cluster
is
updated
provisioning.
A
Does
it
mean
that
the
provisioning
code
is
going
to
be
invoked
on
every
update,
because
you
can
update
multiple
fields
on
your
object,
like
you
can
update
the
metadata,
you
can
update
labels,
you
can
update
or
you
can
update
the
spec.
So
what
we
really
want
to
happen
is
for
the
provisioning
logic
to
be
invoked
only
when
the
spec
field
gets
updated.
How
do
we
achieve
that
I
as
I
use,
kubernetes
1.8?
A
In
my
case,
what
I
do
I
just
store
the
apply
it
configure
that
config,
the
cluster
config
that
get
applied,
I,
get
storing
the
status
and
then
I
check
the
one
from
the
spec
with
the
one
from
the
status.
If
they're
identical.
There
is
nothing
that
I'm
gonna
do
I'm,
just
gonna
return
nil
and
do
nothing
now,
let's
check
what
happened?
A
A
A
Some
construct
that
allows
you
to
run
the
pre
deletion
hook
when
your
object
gets
removed.
So,
like
let's
say
you
decide
that
you're
going
to
remove
you're
going
to
remove
the
cluster,
and
you
want
some
logic
to
run
before
object
is
actually
gone
from
the
ECD
database.
In
this
case,
you
inject
the
finalizer
and
kubernetes
will
not
remove
this
objects
from
these
in
the
database
till
you
run
your
logic
until
you
remove
this
finalizer
okay,
I'm
gonna
check
on
our
guy
still
going.
A
A
A
Wonder
my
conditions
are
I,
think
I
I
think
I
broke
something
in
and
in
my
code,
I
just
I
just
made
this
very
last
community
before
the
demo
is
always
it's
always
always
a
bad
idea.
But
what
actually
should
happen
here
in
the
status?
You
should
see
the
condition
that
says
provisioned
true
and
then
you
should
see
another
condition
that
says
ready
true.
So
the
provision
means
that
the
cluster
provisioner
actually
successfully
provision
the
provision.
A
The
cluster
and
ready,
true
means
that
the
cluster
health
checker
actually
says
that
the
cluster
is
healthy
and
and
ready
to
be
used.
Now,
why
do
we
use
two
conditions
because,
like
as
I
mentioned,
it's
better
that
the
one
condition
is
modified
by
a
single
controller?
There
is
one
more
rule
that
we
try
to
apply
to
when
it
comes
to
conditions,
it's
better
that
conditions
are
consistent
in
a
way.
A
The
true
means,
good
and
false
means
bad,
because
sometimes
you
might
want
to
see
the
you
might
see
the
conditions
in
the
kinetise
world
when
the
actually
false
means
a
good
thing.
It's
like
condition
can
be,
can
be
named.
Like
I,
don't
know,
cluster
is
failing
and
then
forth,
and
it
means
you're
good,
it's
better
to
use
the
same
construct
across
your
conditions
so
later
on.
If
you
would
like
to
signal
to
your
user
that
cluster
is
good,
you
can
actually
fetch
all
the
conditions
and
based
on
their
you
know,
based
on
their
value.
A
A
A
A
Okay,
I
have
no
idea
why
the
chief
config
wasn't
created,
but
I've
kind
of
expected
it
it's
demo.
It's
meant
to
fail,
but
the
cube
config
is
the
resource
that
was
meant
to
be
created
as
a
child
resource
of
the
cluster
II.
So
what
does
it
mean
to
be
the
child
resource?
So
it
means
that
the
object
lifecycle
can
be
kind
of
like
manage
indirectly
by
the
parents
resource.
So
it's
like
when
you
create
a
parent
resource,
you
create
a
child
resource.
A
A
A
All
right
and
let's
see
if
the
cluster
was
actually
removed.
No,
it's
still
here,
but
let's
look
closer
at
the
cluster.
You
see
that
it,
although
it
wasn't
removed
there
was
some
deletion
time
stamp
that
was
set
for
it
and
if
I
start
my
controller
back,
it
should
look
at
the
deletion
timestamp
and
based
on
its
value.
It
should
decide
whether
it
needs
to
go
ahead
with
a
provisioning
or
whether
it
needs
to
go
ahead
with
the
removing
good
boy.
It's
removing.
A
A
A
If
you
could
run
a
test
version
for
your
charities,
if
you
could
run
a
DS
version,
is
one
that
ten
use
some
custom
logic
if
your
kubernetes
version
is
less
than
that,
and
also
kubernetes
one,
that
ten
offers
a
new
construct
that
I'm
yet
to
try.
They
actually
created
a
CR
D
for
the
status.
So
you
can,
you
can
actually
check.
You
can
actually
keep
your
status
separate
from
your
main
resource.
A
So
when
you
change
your
main
resource
only
then
you
get
that
updates
in
the
in
the
controller
logic,
when
you
change
the
status
that
does
not
happen,
yeah
I'm
gonna
try
it
after
the
cube
con
and
then
conditions.
So
each
condition
should
represent
a
single
functionality
state
for
multi
functionality.
Reflection.
You
should
consider
using
multiple
conditions
and
yes
avoid
updating
multiple
conditions
by
multiple
controllers
and
there's
a
rule.
Number
three
true
should
mean
like
a
good
thing
and
be
consistent
across
your
conditions.
Then
the
finalizer
use
the
finalizer
only
one.
A
A
You
can
just
go
with
the
regular
sync
logic
and
then
the
owner
references
always
try
to
find
out
whether
the
kubernetes
has
some
built-in
functionality
that
you
can
invoke
by
placing
a
certain
field
on
your
object,
like
use
on
the
reference
to
invoke
the
garbage
collection
for
your
child
resources.
Don't
the
cleanup
manually
and
thank
you
all
for
coming.
If
you
have
any
questions,
you
can
ask
them
now
or
you
can
reach
me
out
later
on.
Twitter
I
can
just
step
by
and
yeah
yeah
I.
A
B
B
B
B
A
B
C
A
A
A
It
has
to
be,
it
has
to
be
the
opponents
like.
Basically,
you
do
the
Rican
self
or
your
resource,
as
you
would
just
you
know,
do
it
in
any
like
regular
update
case,
you
don't
rely
on
some
previously
state
yeah
saved
information,
and
if
something
fails
you
just
we
just
Riku
the
object
and
then
the
update
will
come
again
and
we'll
rerun
the
logic
once
again,
yeah.
A
C
A
A
Actually,
a
very
good
question,
but
it's
also
like
whether
you
support
external
update,
your
resource
or
whether
you
only
support
the
update
when
it
comes
like.
Let's
say,
user
comes
to
cube
CTL
and
here
there's
the
cluster
spec.
While
your
controller
is
down,
your
controller
should
be
able
to
handle
cases
like
that,
but
cases
like
user
goes
to
AWS
and
maliciously,
like
removes
the
docker
container,
the
controller
won't
do
it
you
can
yet,
potentially
you
can.
A
A
A
D
A
For
this
one
I,
just
I
just
ran
it,
as
it
was
advised
like
a
wise
example
in
the
sample
controller
like
Brunetti
sample
controller,
but
we
actually
in
Rancher.
We
have
our
own
framework
for
defining
the
CRTs,
for
building
the
clients
for
them
and
for
yeah
for
defining
all
this
boilerplate
for
informers,
Lister's
and
yeah.
A
I
did
not
I
did
know
about
it
before
the
talk
yeah
as
I
said,
I.
Think
every
every
big
company
come
up
with
some
sort
of
the
framework
and
is
good
if
it's
open
source.
But
then
you
just
watch
the
adoption
like.
How
is
it
it
to
use
how
wide
ladies
adopted
and
then
you
can
talk
about
the
framework
popularity
and
thank.