►
Description
Building Kubernetes Operators in the Wild - Managing the Lifecycle of Your Application on the Cloud
Martin Hickey (IBM)
This OpenShift Commons Gathering was held on July 6th, 2022 live in London, England
https://commons.openshift.org
A
Hey
so
hello,
folks,
it's
great,
as
I
say
to
be
back
in
london.
My
name
is
martin
hickey,
and
what
I
want
to
talk
about
today
is
you
know:
there's
not
some
mentions
of
operators.
I
even
see
some
of
the
sponsors
and
partners
out
there
have
got
their
own
operators
and
stuff,
but
I
want
to
kind
of
look
at
it
from
a
different
angle
and
I'll
talk
about
that
in
a
few
minutes
yay.
A
A
A
So
this
was
first
and
foremost,
kubernetes.
Wasn't
designed
around
state
relapse
in
mind
straight
away
now
someone's
going
to
stay,
oh
yeah,
you
can
now
use
it,
stateful
sets
etc.
Yes,
I
know,
but
it's
more
kind
of
bolted
on,
so
it
was
always
built
around
the
idea.
Here
of
you
know
you
can
deploy
your
hats
really
easily.
You
can
manage
them.
A
You
know
that
you
that
hides
things
as
well,
but
for
the
id
here,
if
you
need
to
do
things
out
of
the
box,
if
you
need
to
do
things
that
the
human
operator
does
or
your
sre
does
at
three
o'clock
in
the
morning,
for
example,
you
know
if
your
your
application,
you
update
your
application
that
runs
kind
of
smooth,
but
then
what
about
the
database?
It's
connected
to?
You
know
what
about
the
schemas,
what
about
the
backups,
etc?
A
A
A
So
what
do
I
mean
by
the
crd?
So
probably
you've
probably
heard
custom
resource
definition,
and
this
is
really
where
you
extend
the
api
of
kubernetes,
so
kubernetes
api
is
built
around
its
different
objects,
its
pods,
its
deployment,
its
ingresses,
etc.
You
probably
all
know
know
these
things
when
you
write
your
manifest
or
whatever
to
deploy
or
your
helm
charts
to
deploy
these
objects
into
the
cluster
and
with
the
crd
gives
you
the
ability
to
add
your
own
object
and
put
your
own
logic
in
there
now,
like
kubernetes
objects.
A
A
Okay,
a
bit
of
a
shy
bonus
today,
I
feel
there's
a
few
more
people
that
that
are
deploying
apps.
They
may
have
gone
to
sleep
already
or
whatever
so
we'll
have
to.
You
know,
liven
it
up,
maybe
in
a
few
minutes,
but
as
I
said
earlier,
you
know
we
generally.
When
we
talk
about
it,
we
talk
about
those
cases
around
the
caches,
you
know
managing
databases
etc.
A
A
I
want
to
look
at
a
different
style
of
operator
that,
I
suppose
the
background
is
to
me
and
a
colleague
about
a
year
ago
we
were
talking
about
operators
and
we
had
okay,
we
use
operators,
we
use
operators
heavily,
we
use
it
for
our
cloud
packs
and
for
all
other
different
capabilities,
but
we
were
saying
right.
Okay,
operators
are
great.
A
So
what
about
the
idea
of
being
able
to
change
your
credentials?
So
I
suppose
credential
rotation
is
an
old
phrase,
but
you
get
the
id
like
the
id
here
of
you
know,
change
your
credentials.
You
know
either
period
or
on
a
period
or
else
you
know
what
I
mean
if
there's
an
issue,
and
you
need
to
do
it
on
the
flight
and
we
were
saying:
can
you
do
that
with
an
operator.
A
A
I've
got
two
hands:
yeah
three
hands.
Actually
I'm
getting
confused
now
because
I
don't
know
if
people
people
are
not
answering
or
just
are
just
trying
to
confuse
me
in
a
way
like
I'm
not
sure,
but
so
that's
generally
the
idea,
but
what
you
know
what,
if
you
can
do
other
things,
so
this
is
where
we
brought
this
idea
in.
So,
as
I
say
there
was
this
simple:
we
did
run
times
that
in
the
ibm
cloud.
So
why
am
I
using
the
ibm
code
well
for
solace?
A
Wasn't
working
ibm,
but
the
second
one
is
because
I
can
use
the
cloud
for
free.
So
there's
a
big
part
of
this
like
right.
I
can
use
it
for
free
so
and
then
I
can
deploy
you
know
I
can
deploy
open
shift
for
free
as
well.
So
that
was
a
great
great
thing
to
be
able.
I
just
apply
out.
You
know
cluster
there
I
work
away.
I
don't
know
who
pays
for
put
there.
We
go
so
I
look
this
and
it
says
right.
A
We
had
these
runtimes
they're
there
for
a
good
number
of
years,
back
going
back
to
even
clown
foundry
times
where
you
could
just
take
no
java,
whatever
you
want
to
play
it
out
and
it
puts
a
simple
app
out
there
and
I
love
it
because
I've
used
it
over
times
where,
if
I've
had
to
write
some
node
and
I
wanted
an
old
server
running
a
web
server
running,
I
can
apply
this
out
scaffolds
out
and
I
said
one
take
out
the
little
basic
code
in
it
and
put
in
my
own.
So
it's
nice
landing.
A
So
this
one
deploys
out
a
very
simple
app
that
literally
you
you
go
to
the
website
you
can
put
in
the
name
and
it
stores
in
the
back
in
database.
You
can
connect
the
back-end
database
in
this
case.
It's
cloudant,
okay,
and
this
works
really
really
simple,
where
how
you,
how
the
app
connects
to
the
cloud
and
db
is
through
the
concept
of
a
resource,
api
key
so
basically
credentials
and
that
gets
stored
in
a
secret
inside
in
the
cluster,
where
your
application
is
deployed,
really
really
simple.
The
app
starts
up.
A
A
A
So
we've
taken
a
very,
very
simple
app
here,
that's
connected
to
a
cloud
and
db
and
all
of
a
sudden
we
have
a
series
of
steps.
You
know
now
they're
simple
steps,
but
you
can
see
here
that
which
are
applications
out.
There
is
there's
work
you
do
like
this.
Maybe
you
call
it
housekeeping,
maybe
you
call
it,
you
know,
I
don't
know
janitor
work
whatever
you
want
a
nice
term.
You
want
to
put
on
it,
but
is
there
work
like
that?
You
have
to
do
from
time
to
time.
A
A
A
It's
just
now
my
day,
really
is
it
so
because
when
I
came
in
here
I
said
it
was
amazing.
I
got
off
my
flight,
I
got
the
heater
express
in.
I
went
right
down
to
the
to
my
right
platform.
I
got
the
tube
outside
here
and
I
walked
down
came
in.
I
said
this
day
is
going
amazing
and
since
then
things
haven't
gone
so
well,
but
anyway
there
we
go
so
that
got
me
thinking
right.
I
said
I
have
that
app
now
so
straight
away.
I
didn't
enter
in
use
case.
A
I
said
no,
I'm
going
to
write
my
app
my
operator
now,
I'm
also
one
of
those
developers.
I'm
a
hearthstone
courses
person.
I
ain't
going
to
write
code.
If
I
don't
need
it,
you
know
I
got
a
family.
I
got
a
life
when
the
day
is
over.
I
just
want
to
go
home
and
enjoy
myself.
So
I'm
gone
from
that
three
o'clock
in
the
morning
drinking
coke
keeping
going
those
days
are
gone.
So
I
said
right.
I
have
my
app.
Let's
look
at
the
operator
now.
A
So
I
use
go
version
of
this
and
it
scaffolds
out
a
really
nice
simple
piece
of
code,
for
you
gives
you
your
basic
controller
and
your
ability
to
create
your
crd
et
cetera.
So
then
I
started
filling
it
out.
So
then
I
put
the
logic
in
so
if
we
look
here
on
the
right
hand,
side,
what
we
have
now
is
we
are
building
out
the
logic
of
what
your
credential
rotation
is.
A
First
of
all,
you
have
your
custom
resource,
so
this
is
the
new
resource
object
and
I
call
it
credential
rotator
or
your
original
name.
So
and
then
you
can
see
here
in
the
middle.
I
have
the
controller.
So
when
I
deploy
my
operator
using
the
operator
sdk,
it's
going
to
call
kubernetes
it's
going
to
register
the
crd.
A
A
So
what
we've
done?
We've
shifted
our
mindset
here,
where
we're
now
not
necessarily
doing
everything
in
the
cluster.
We're
now
calling
out
an
external
system
which
this
time
is
a
cloud
db,
that's
running
on
the
ibm
cloud,
but
it
could
be
running
on
the
google
cloud
or
azure
or
wherever
you
want,
and
all
it's
using
is
the
api
of
that
particular
cloud
to
connect
to
that
service
and
say
give
me
new
credentials.
It's
going
to
take
them.
A
It's
going
to
come
back
storm
into
secret
and
then
basically
it's
going
to
restart
the
apps
and
that's
what
the
controller
does
once
the
request
comes
in,
as
always
in
a
manifest
file
or
home
chart
or
whatever,
with
the
new
custom
resource
instance,
and
whatever
information
is
needed
in
this
case.
The
information
that
I'm
using
the
crd
is
the
name
of
the
app
the
namespace
of
the
app
the
service
gui
of
the
cloudant
and
the
particular
url,
and
then
also
your
cloud
api
key
because
you
know
am
I
do.
A
I
have
authorization
to
go
into
that
particular
account
and
change
your
credentials
on
that
and
that's
what's
passed
through
now.
You're,
probably
going
to
say
to
me:
that's
not
very
safe.
I
know
it
is
not
very
safe,
but
you
know
we're
doing
we're
doing
a
simple
app
here
on
this.
So
you
know
we
just
want
to
get
the
concept
across.
So
all
that
happens
then
automatically
automatically
using
the
declarative
model
of
kubernetes,
and
you
get
the
value
of
that.
A
So,
just
recapping
here
very
quickly,
what's
happening
here
with
a
credential
rotator,
a
credentialed
rotator
is
automating
these
steps,
they're,
not
that
big
of
steps
and
they're
not
that
complicated.
But
you
can
imagine
if
you've
seriously
these
jobs
for
your
app,
how
they
can
grow
and
keep
them
in
one
place
could
be
difficult.
A
A
Okay,
if
you
has
anyone
ever
heard
the
analogy
of
kubernetes,
where
kubernetes
is
like
a
temperature
gauge
when
you
set
a
certain
temperature,
then
eventually
the
system
then
will
change
the
temperature
to
that
temperature
hands
up.
Has
anyone
ever
heard
that
a
few
more
hands?
Okay?
I
know
you're
tough
audience
folks.
You
know
who'd
like
to
be
a
comedian
up
here
down
to
make
you
laugh
and
stuff.
A
So
that's
really
the
control
logic
or
the
robotic
logic
of
what
a
kubernetes
does
you
ask
for
a
particular
state,
your
desired
state
and
then
kubernetes
will
change
the
current
state
of
its
cluster
to
do
that.
But
what
we've
done
is
we've
gone
outside
the
box
here,
but
we've
extended
a
little
bit
where
we've
decided
that
you
know
we
are
going
to
actually
connect
externally
to
a
system.
A
I
put
a
new
credential
in
there,
then
in
the
cluster,
we're
going
to
update
the
secret,
we're
going
to
update
the
the
we're
going
to
restart
the
apps
and
then
we'll
go
back
and
delete
the
the
credential.
So
this
is
the
first
request.
So
let's
say
you
decide,
you
were
going
to
create
an
instance
of
a
credentialed,
rotator
cr
and
we're
going
to
call
it
demo.
A
So
once
it
grabs
that
if
it
can't
grab
it,
then
it's
going
to
exit
out
of
the
loop
and
just
remember
this
is
a
loop.
No,
so
when
the
request
comes
in
it's
going
through
a
loop
now,
it's
very
important.
This
loop
is
because
it's
a
continuous
loop
and
requests
are
continuously
coming
in
when
you're
processing
you're
supposed
to
process
in
small
steps,
not
in
big
large
steps.
So
you
br,
you
break
your
test
down,
so
I'm
going
to
break
the
test
down
here.
A
A
So
in
this
situation,
then
we
want
to
create
the
we
want
to
create
the
credential.
So
we're
now
going
to
connect
out
to
the
external
system
which
at
this
stage
is
into
the
ibm
cloud
it
says.
Will
you
please
give
me
a
new
credential
for
that
and
for
that
particular
service
is
going
to
create
a
credential
if
it's
not
a
success,
we're
going
to
exit
out
because
there's
something
badly
wrong.
A
A
A
I
went
to
kind
of
a
cheap
way
around
this,
where,
if
you
just
change
the
field
in
the
deployment
object,
it'll
automatically
restart
all
the
instances
for
you,
probably.
If
you're
writing
this
you'd
go
around
with
looking
at
each
instance,
you
have
making
sure
each
one
starts
before
you
go
off
doing
the
other
one
deployment.
Does
it
slightly,
but
you
may
get
cut
out
in
between
and
then
and
then
you're
going
to
recue
again
and
then
finally
you're
going
to
delete
the
previous
credential.
A
No,
we
were
going
to
do
an
operator
dimmer
and
jenna's
up
there
and
I
think
she's
as
happy
as
can
be
because
she
said
you're
tired
on
time.
You
will
not
get
through
the
demo
and
whatever
demo
god
she
prayed
to
that's
it.
We
won't
be
doing
the
demo
because
my
laptop
would
not
connect
properly
so
sorry
about
that
we
won't
do
the
demo,
but
if
I
can
go
back
a
second,
you
know
in
top
right
hand
corner.
A
A
I
suppose
wanted
it.
First
of
all,
we
said:
are
you
probably
sitting
there
and
if
definitely,
if
there's
old
school,
assisted
men's
here
are
gone?
Well,
I
like
scripting,
what's
wrong
with
scripting,
we
all
like
scripting.
I
keep
coming
back
to
script
in
a
trope,
my
life
for
some
reason,
no
matter
what
technology
they're
going
to
bought
their
scripting
for
some
of
this,
even
something
as
small
as
this
takes
a
bit
of
time.
You
know
because
you're
going
to
be
calling
carl,
you
want
to
look
at
the
error
messages
coming
back
and
so
forth.
A
Like
that,
you
know
you
really
want
to
use.
Some
of
the
standard
interfaces
are
there
now?
I
use
go
here
and
this
because
you
know
it's
the
native
language
of
kubernetes
and
there's
a
lot
of
apis
out
there
also
going
into
the
the
ibm
cloud.
The
sdk
was
go
as
well,
so
that
made
that
easy
and
if
I'm
truthful,
I
struggle
with
that
more
than
I
struggle
with
anything
else
on
it,
but
you're
getting
so
much
for
free
here,
you're,
getting
the
error,
handling,
you're,
getting
the
recovery,
cord,
etc.
A
A
Yes,
you're
using
this,
but
you're
using
your
standard
api
you're
using
the
standard,
cli
and
the
api
behind
it
or
you
can
call
the
api
behind
it.
So
your
tools
you're
already
using
for
this,
you
can
use
again
tracking
the
progress
of
modern
why'd.
You
want
to
do
that
and
there
was
a
great
talk
at
commons,
dublin,
dublin,
dublin,
where
eric
had
to
talk
about
the
cost
of
all
the
logging
and
all
this
which
you
still
need
to
provide
trace
and
stuff
and
especially,
if
you're
doing
credential
rotation
and
you're
changing
credentials.
A
A
Are
we
overloading
with
operators
here,
yes,
maybe
not,
but
remember
you
can
have
specific
operators
and
generic,
and
this
one
could
be
generic,
because
you're
passing
the
app
name,
you're
passing
the
namespace.
So
what
would
stop
you
changing
criteria
rotations
for
any
particular
one,
and
also
then,
if
you've
tweaked
enough
you'd
have
different
back-ends
to
ie
different
clouds
that
you
can
connect
and
change
so
jenna
30
seconds
just
rolling
in
there
just
about
so
I've
thrown
up
this
big
mad
quote
and
you're,
probably
going.
A
A
A
Otherwise,
your
app
is
just
literally
you're,
installing
your
update
and
you're
deleting
but
you're
not
taking
care
of
all
the
other
things
that
you
should
be
so
that
to
me
was
the
itch
that
I
was
scratching
and
what
I
liked
about
about
the
operator
and
what
you
can
do
with
it
and
what
you
can
play
with
it
and
really,
if
you're,
into
hacking
with
a
code,
it's
a
great
way
to
go
out
there
and
just
hack
a
bit
of
code
and
have
a
bit
of
fun.
Okay.
A
So
thank
you
very
much
indeed,
there's
two
links
there
at
the
bottom.
One
is
called
github
you
can
deploy
from
there
into
any
vanilla
cloud.
You
want
any
vanilla,
kubernetes
you
want.
The
other
one
on
top
is
probably
is
a
lot
of
talk
around
it
and
everything
else,
and
also
it
shows
you
how
to
deploy
out
into
openshift.
So
thank
you
very
much
indeed,.