►
From YouTube: MetaController: Build K8s Controllers as Declarative Intents - Amit Kumar Das, MayaData
Description
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects - Learn more at https://kubecon.io
MetaController: Build K8s Controllers as Declarative Intents - Amit Kumar Das, MayaData
In the world of Kubernetes (K8s) it is natural to have heard about controllers. Controllers are a natural extension to K8s that help in building domain specific infrastructure logic. In case you have written any controller logic, you would have noticed a lot of learning curve to implement the same. There has been various attempts to provide scaffolding and code patterns to ease this development. While these guidelines have tried to provide nice abstractions, they fail to groove us into the reconciliation mindset which acts as a steep barrier to enter into Kubernetes paradigm. In this talk, Amit will showcase how MetaController (https://metacontroller.app/) help teams to think and design controllers with reconciliation as the first class citizen. They will also demo how K8s specific details are pushed to the background enabling teams to innovate on their lines of business.
https://sched.co/YVxf
A
A
A
Okay-
and
maybe
you
know,
apply
it
when
we
started
using
kubernetes.
So
a
bit
myself.
I
am
one
of
the
maintainer
of
project
open
ideas,
as
well
as
the
meta
controller,
the
one
that
we
are
going
to
talk
about.
I,
walk
for
my
data
and
I'm,
a
remote
engineer.
I
don't
have
to
navigate
the
traffic
in
Bangalore,
that's
what
and
I
will
I
will
actually
talk
about
my
new
home
and
you
will
get
to
know
about
that
in
a
few
more
slides.
So
this
is
about
my
data.
So
what
do
we
do?
Essentially?
A
Is
we
dig
deep
into
kubernetes?
We
try
to
extract
the
internals
of
how
it
works,
what
works,
what
doesn't
work
and
try
to
fit
that
you
know
that
paradigm
called
kubernetes
into
another
paradigm.
That
is
infrastructure,
of
course,
which
is
storage
the
data
and
we
mix
it
up
together.
You
get
the
best
pieces
out
and
we
deliver
it.
As
you
know,
the
project
open
EBS
again
open
EBS
is
completely
open
source.
It
is
as
well
as
as
well
as
a
CNC
F
sandbox
project.
A
You
can
find
more
about
it.
You
know
later
on,
or
maybe
you
can
talk
about
it.
We
can
talk
about
it,
probably
throughout
the
day
after
this
talk
all
right,
so
this
was
a
whom
I
was
talking
about
most
of
the
times.
You
can
find
me
here.
You
know
asking
questions
or
you
know,
trying
to
solve
the
questions
of
the
users,
a
community
who
has
been
using
meta
controller
or
in
general
controllers.
A
Okay.
So
we
have
an
agenda
here
and
there
is
only
one
agenda
that
I
would
talking
about
here
and
that
is
how
to
Bill
kubernetes
controllers
in
a
really
really
simple
way,
without
the
baggage,
without
the
steep
learning
curve
that
it
has
okay.
So,
in
order
to
understand
or
in
order
to
really
get
to
that
level,
where
we
can
make
use
of
the
controllers
where
we
can
build
controllers,
really
easy,
we
need
to
you
know,
dig
in
a
little
bit
deep
into
what
exactly
the
control
loops
are.
A
What
exactly
those
are
those
jargons
like
a
recon
solution
system,
the
control
loops.
You
know
the
desire
state,
the
observe
state
last
applied
state
has
so
many,
and-
and
it's
really
difficult
when
it
come
from-
you
know
the
other.
You
know
SATA
the
world
when
I
say
other
side
of
the
world
in
the
kind
of
applications
that
we
deal
with
typically
makes
use
of
AP
ice
right
in
the
crud
operations.
A
But
here
it's
a
little
bit
different
okay
and
that's
where-
and
there
is
an
mismatch
you
know
in
what
we
have
been
doing
so
far
in
our
applications
versus
what
we
need
to
do
in
kubernetes,
so
that
these
applications
work
or
gel.
Well
right.
They
work
together
all
right.
The
number
one
thing
here
are
and
again,
like
I
said
we
had
a
little
bit
dig
into
the
internals
of
kubernetes
and
I'll.
Try
to
make
it
very
simple.
A
The
first
one
is
a
client
go
which,
from
the
name
itself
says
it's
something
ready
to
go
and
something
related
to
climb.
So
what
exactly
is
a
kind
right?
So
it's
a
kubernetes
client
that
heads
the
go
developers
okay
to
consume.
The
api
is
our,
for
example.
The
api
is,
are
nothing
but
a
kubernetes
resources
which
can
be
native
okay
as
well
as
can
be
custom
native
resources
are
something
like
part
deployment
sts.
A
The
ones
which
are
you
know,
built
out
of
box
or
given
to
us
by
the
kubernetes
and
a
custom
resources
are
the
ones
that
we
develop
right.
That's
that's
the
best
way
to
exchange
kubernetes
and
again
a
little
bit
flexible
declined.
Oh,
we
have
to
agree
that
you
know
this
is
being
developed
by
some
of
the
community
members
who
have
been
active
in
the
kubernetes
for
a
long
long
time.
I
remember
when
I
first
started
programming
in
kubernetes
some
couple
of
years
ago.
A
I
guess
it
was
three
and
a
half
years
ago,
I
was
using
a
clang,
go
some
version,
I
I,
don't
remember.
Maybe
it's
1.5
or
a
no
5.0
and
I
have
never
ever
got
an
issue
while
upgrading
this
clang
go
libraries,
so
it
is
that
efficient
I
know
it's
it's
just
a
robust
yeah.
The
second
thing
that
we
need
to
you
know
really
understand
is
a
CRD
okay.
A
So
what
does
it
mean?
So
it
is
essentially
a
mechanism
that
kubernetes
provides
right
to
the
and
developers,
okay
to
define
their
own
intense
right
intent
right.
What
is
an
intent
here?
So
it's
a
very
nice
mechanism
where
one
a
developer
right
or
a
sorry
or
it
evolves.
I
mean
got
it.
You
know
you
name
it
you
give
a
person,
but
they
either
want
to
really
make
use
of
kubernetes
and-
and
you
define
something
missing
in
kubernetes,
which
of
course
they
will
find
too
so
that
it
can
adopt
or
adapt
to
their
own
requirements.
A
You
have
to
fit
in
with
some
scripts
some
little
bit
of
logic,
some
topping
some
rappers
right
and
that's
where
kubernetes
series
comes
into
place.
So
you
have
a
definition
right.
There
is
a
call
a
see.
Are
these
custom
resource
definition?
There
is
an
instance
of
the
definition
which
is
known
as
custom
resources.
A
If
you
try
to
draw
an
analogy
between
the
CR
D
and
the
C
R,
with
respect
to
the
programming
languages
that
we
are
most
familiar
with,
for
example,
let's
say
I
take
example
of
Java
right
it
has
the
concept
of
class
and
objects
right
for
object.
Is
an
instance
of
the
class
and
the
class
is
a
blueprint
right,
so
you
can
think
of
that
in
a
similar
way.
It's
not
exactly
similar,
but
something
very
similar
right.
Where
kubernetes
is
online
system,
you
replace
Java
with
kubernetes
you're.
The
definition
is
something
you
you
you.
A
You
define
a
schema,
some
amount
of
schema
and
communities
just
by
the
definition,
just
by
applying
the
CRD
to
the
kubernetes.
What
do
you
get
you
get
so
many
things
out
of
the
box?
Okay,
what
are
they
you
get?
A
lot
of
cool
API
is
out
of
the
box.
You
get
the
REST
API
endpoints
right
without
having
to
write
a
single
line
of
code.
You
have
the
cube.
Cuttle
understand
your
own
custom
resources.
A
You
can
do
a
cube,
cut'
get
XYZ,
blah
blah
blah
and
we're
blah
blah
blah
is
the
name
of
your
CR,
okay
and
unless
to
say
these
are
the
building
blocks
of
the
controllers
that
you
know
very
softly
will
be
going
ahead
or
will
be
talking
about
all
right
now
we
start
digging
okay.
So
before
we
go
ahead,
like
I
said
here
to
understand
how
do
you
know
build
controllers
in
a
really
simple
way?
We
need
to
understand.
You
know
the
painful
aspects
of
that
right
and
I.
A
Don't
know
how
many
of
us
are
developers
and
who
really
in
a
code
on
a
day-to-day
basis,
but
a
few
of
the
slides
I
heard
that
we
are
going
to
see
are
little
bit
of
out
coding,
and
don't
just
fear
about
that.
You
know
you
don't
even
need
to
look
at
the
slides.
You
can
actually
listen
to
what
I
am
trying
to
say
and
let's
hope
it
will
be
simple.
Okay,
so
we
we
need
to
have
some
kind
of
when
we
try
to
design
or
implement
a
controller.
A
There
is
obviously
you
can
divide
it
into
multiple
parts.
I
have
tried
to
divide
that
the
way
I
understand
it.
So
the
first
part
that
I
thought
would
be
beneficial
for
authors
will
be
the
startup
logic
right.
So
the
setup
logic
is
a
one
where
what
we
do
is
we
try
to
configure
the
conflict
of
API
server,
the
community's
API
server
once
that
configure
is
established.
We
have
a
way
to
create
the
client
set.
A
So
what
is
the
client
set
via
a
client
said
we
can
make
crude
API
is
we
can
invoke
write
we
can
we
can
consume
the
API.
Is
that
the
kubernetes
exposes,
for
example,
we
can
do
it
get
of
a
pot
or
a
list
of
a
pot
right,
and
similarly
the
moment
we
try
to
you
know,
apply
our
custom
resources.
The
same
Native
Client
won't
work.
We
have
to
have
our
own
client
and
here
again,
Klein
go
helps
us,
you
know,
via
some
code
generation
and
antwa
and
so
on
and
so
forth.
A
We
get
a
new
client
set.
The
client
says
that
is
dedicated
to
our
own
custom
resources.
So,
for
example,
we
can
do
a
programmatic
invocation
of
get
blah
blah
blah
where
blah
blah
blah.
If
you
remember,
is
the
name
of
the
resource,
the
customer
sauce
yeah,
let's
not
go
deep
into
this
code.
This
is
you
know,
kind
of
a
startup
code.
A
This
is
the
typical
constructor
code,
and
if
you
see
this
is
like
the
simplest
one
that
I
could
ever
make
out
and
I
have
actually
copied
that,
from
the
sample
controller,
the
repo
which
kubernetes
has
in
the
github,
you
can
actually
you
know,
google
it
and
you
can
find
it
out.
This
is
exactly
the
same.
A
A
Okay,
so
if
we
have
heard
about
the
reconciliation
systems,
are
the
control
loops
that
kubernetes
is
all
about
there
essentially
is
there
are
two
things
right,
two
states,
rather,
okay,
one
is
absorb
state,
the
other
one
is
a
desired
state
and
whose
state
are
we
talking
about.
We
are
talking
about
the
state
of
the
resources,
for
example
the
blah
blah
blah
resource
that
we,
you
know
try
to
frame
here
and
in
this
session,
that's
a
custom
resource
and
it
can
have
an
observed
state.
A
It
can
have
a
desire
stage
right
and
all
the
loops
right
if
you
are
attended,
you
know
the
keynote
session.
Now
what
Dennis
was
talking
about
that
circle,
right,
div,
observed
and
desired
that
circle.
That
is
where
you
know
essentially
boils
down
to
reading
the
controllers,
and
what
is
something
like
it
will
watch
your
resource.
Okay,
it
will
get.
The
latest
observed
state
of
the
resource
to
you
to
your
code
when
I
say
code
to
your
hook
or
to
your
logic
where
we,
where
you
and
I
we
try
to
you
know,
do
some.
A
You
know
money
relations,
do
some
customizations
and
try
to
give
back.
In
return,
the
desired
state
all
right,
so
this
is
again
this
is
now
now.
This
is
going
to
be
a
little
bit
interesting
as
well
as
little
bit.
You
know
heavy
when
we
say
we
had
queued
the
watch
now,
what
about
the
desired
state?
Why
do
we
queue
the
desired
state?
Yeah?
We
need
to
queue,
you
know.
Sometimes
what
happens
is
if
you
have
created
a
custom
resource
right.
A
You
want
to
have
that
custom
resource
generate
some
other
resources
which
may
be
native
as
well
as
custom.
A
point
in
example.
Here
is:
if
you
consider
the
deployment
right,
it
can
lead
to
creation
of
replicas
set
and
then
in
turn
the
replicas.
It
creates
one
or
more
parts
right.
So
you
have
like
observed
state
as
your
deployment
and
the
design
state
as
the
parts,
and
you
know
you
can
actually
build
one
or
more
controllers
to
manage
this-
is
design
state
and
reconcile
to
the
observed
state.
A
Well,
okay,
I
understand
it's
getting
really
a
little
bit
heavy
with
the
code
and
all,
but
then
this
is
the
basic
way
that
kubernetes
suggests
us
to.
You
know
to
understand
in
order
to
implement
a
controller
using
plain
vanilla,
client
go
right
and
then,
after
all,
once
we
set
up
the
queues,
we
have
the
startup
code
and
then
we
set
with
the
queues
now
comes
the
turn
of
the
worker,
where
we
actually,
you
know
there
are
some
terms
like
threads
and
all
we.
A
We
run
these
workers
in
a
continuous
loop
in
a
never-ending
loop
so
that
they
keep
on
watching
and
whenever
you
know
it
event
gets
triggered
due
to
probably
a
change
in
the
inner
state
of
the
resource.
It
captures
that
and
gives
it
to
your
hook,
and
you
try
to
do
something
and
you
give
it
back
as
a
desired
state
all
right.
A
So
so,
when
you
capture
that
you
have
to
get
it
from
the
queue
and
have
all
these
methods,
which
are
known
as
sync-
and
you
know,
the
sync
is
something
like
you
get
it
from
the
queue
you
you
unwrap
that
okay
and
you,
the
think,
is
actually
the
the
method
or
the
function
where
you
write
your
business
logic,
okay
and
the
business
logic
is,
should
be
all
about
creating
the
desired
state.
You
have
to
respond
your
desired
state
and
nothing
else.
A
All
right,
I
understand
that
we
have
been
dealing
with
a
lot
of
code
and
it
should
definitely
be
getting
really
heavy
and
boring.
But
are
we
done
what?
What
do
you
think?
Well,
we
had
not
done
yet
because
we
haven't
yet
written
our
business
logic.
It
was
all
about
dealing
with
the
kubernetes
the
way
the
kubernetes
works
right
all
about
under
Singh
hue,
the
startup.
You
know
all
those
sort
of
things
all
right
so
again,
like
I
said
this
is
the
the
sample
controller.
A
In
the
sample
controller,
we
have
a
resource
called
foo
right,
we
have
an
imaginary
source
called
foo
and
when
we
apply
the
resource
who
we
get
a
desired
state,
the
design
state
is
the
deployment
all
right.
So
when
we
handle
the
business
logic,
we
have
to
understand.
What
is
the
food
we
have
to?
You
know
derive
that
we
have
to
get
the
food
if
the
food
is
really
available
or
not.
A
It
may
happen
that
the
food
that
we
got
as
an
event
was
a
stale
event
which
in
which
in
turn
means
the
foo
might
have
been
deleted
already.
Who
knows
that?
So
all
those
kind
of
things
we
have
to
you
know
understand
and
where
to
code
it
and
every
controller
that
we
write
write
each
of
them
have
to
do
this
basic
set
of
things.
These
are
the
common
boiler
plates,
but
then,
yes,
you
have
to
do
it.
There
is
no
way
out
of
it.
A
Yeah
again,
you
had
a
code,
the
desire,
you
are
to
see
that
the
desired,
and
you
know
the
observed
that
is
the
actual
state
in
the
kubernetes
cluster
matches.
If
it
doesn't
match
you
do
something,
maybe
you
create
another
resource
and
delete
the
resource
or
update
the
resource
right
and,
like
I,
said
what
is
it
important
here?
So
just
try
to
understand
you
or
you
know
when
you
write
the
code
when
you
write
the
code
for
the
controller,
we
may
be
getting
events.
A
The
events,
the
reason
for
these
events,
the
root
cause
for
these
events
may
be,
you
know
different.
Sometimes
the
event
will
be
coming.
It
will
be
incoming
when
there
is
a
new
resource
that
gets
created
at
some
other
points
of
time.
The
event
will
be
coming
whenever
when
there
is
no
change
at
all
right,
because
these
are
all
periodic,
the
reconciliation
systems
are
continuously
it's
looping
around
and
around,
and
obviously
that
will
be.
You
know
even
that
will
come
up
when
there
are
some
updates
happening.
A
One
of
the
most
freakiest
case
here
is
that
the
even
scanner
even
happened
when
your
controller
goes
into
a
restart
right.
There
are
a
lot
of
cases
where
the
pod
can
get
into
a
fiction
mode.
The
controller
restarts,
and
you
get
a
dealer
of
all
the
events,
all
the
old
new
everything
you
get
it.
So
you
have
to
write
the
code
in
such
a
way
that
the
code
is
idempotent.
You
have
to
always.
You
have
to
make
sure
that
the
code
returns
only
the
desired
state.
A
Okay,
I'll
be
explaining
the
desired
state
a
little
bit
later,
when
we
go
to
that
logic,
but
I
will
but
wewe
need
to
essentially
understand
the
design.
State
is
very,
very
important.
Okay,
when
we
write
controllers
and
that
forms
the
basis
of
writing
item-
put
into
logic
all
right
so
to
make
it
very
simple-
the
design
state
here
in
the
sample
controller
is
we
want
that
particular
structure
of
the
deployment,
and
this
is
the
deployment
that
should
get
in
as
a
design
state
given
a
full
resource.
A
A
Okay,
we
have
been
talking
all
over
this
hard
ways
and
all
is
there
any
really?
You
know
easy
way
that
you
know
the
agenda
promised
and
is
it
really
possible?
Okay?
So
this
is
our
goal.
We
have
nothing
to
do
at
all.
When
we
try
to
you
know,
write
controllers,
that's
my
objective!
I'll!
Try
to
you
know!
You
know,
try
to
explain
that
in
as
as
much
time
I
have
left
with
me
now,
okay,
you
know:
how
can
we
do
that?
Is
it
even
possible?
That's
the
first
question
that
we
must
be
asking
ourselves
right.
A
Okay,
so
the
first
thing
in
meta
controller
is
is
a
CR
okay.
This
CR
is
not
the
one
that
your
which
is
equivalent
to
your
business
CR,
which
is
not
your
intent.
This
CR
is
something
that
meta
controller
understands
and
if
you
see
right
so
here
is
this,
is
this
looks
something
like
what
you
see
is
what
you
get
right.
There
is
a
parent.
Okay,
there
is
a
child,
the
parent
is
nothing
but
the
full
resource.
Now
this
full
resource
is
the
which
is
your
business
CR,
and
there
are
something
called
child
resources
right.
A
The
child
resources
are
something
called
the
desired.
What
do
you
want
when
you
apply
a
full
resource?
You
want
the
deployments,
so
that's
goes
the
deployments.
Okay
and
it's
constructed,
or
it's
represented
in
such
a
way
that
you
get
to
know
before.
Even
writing
your
code,
you
get
to
know
what
you
intend
to
do
with
your
controller
and
is
entire
thing
is
again,
you
know,
wrap
in
as
a
yeah
Mel,
how
which
is
a
CR
that
is
only
understood
by
matter
controller.
You
don't
need
to
write.
You
know
code
behind
for
this.
Okay,
all
right.
A
This
sounds
kind
of
okay,
but
then
what
about
the
business
logic
right?
Okay,
like
again
like
I,
said
yeah.
We
have
a
start
good
here,
but
just
see
the
start
code
here
and
if
you
remember
the
oldest
startup
code,
do
you
find
anything
that
talks
about
kubernetes
right?
Do
you
find
anything
called
in
for
most
Watchers,
even
broadcasters
and-
and
you
know,
cue,
the
thready
needs
and
so
on
and
so
forth?
I,
don't
think
there
is
anything
here.
What
you
see
is,
like
a
just.
A
You
know
some
kind
of
a
structure
which
looks
very
similar
to
JSON
you
grab
whatever
has
been
given
to
you
as
part
of
your
those
hooks
right
and
you
check.
If
that
particular
here
I
mean
what
you
grab
is
a
deployment
you
check.
If
the
deployment
really
belongs
to
the
Foo
resource,
right
or
or
it
was
just
a
standalone
deployment,
we
never
know
right.
So
if
it
really
belongs
to
the
food
employment,
and
then
you
use
you
get,
you
know
the
replicas
of
that
deployment
and
set
it
as
a
desired
status.
A
So
so
what
this
code
does
is
when
you
apply
a
full
resource.
Okay,
a
deployment
gets
created,
okay
and
the
replicas
of
the
deployment.
The
count
of
replicas
of
the
deployment
will
be
set
against
the
status
of
the
foo,
okay
and,
and
what
you
are
seeing
here
is
exactly
that
amount
of
code
that
you
need
to
write.
I
have
not
tried
to.
You
know,
hide
away
some
important
stuff
at
all.
What
has
fit
into
the
slidy
is
exactly
the
same
amount
of
code
that
you
need
to
write
in
your
code.
A
All
right-
and
this
is
the
design
state.
So
the
first
one
was
the
config.
The
second
one
was
the
startup
code.
The
third
one
is
the
desired
state
and
design
state
is
nothing
but
the
exactly
the
yanil
that
you
intend
that
you
want
right.
And
what
do
you
want?
We
want
the
deployment
and
that's
where
okay,
so
that's
yeah,
you
can
call
it
a
JSON
representation
and
so
on
and
okay
now
are
we
done?
Are
we
done
with
the
you
know?
The
building,
the
controllers,
with
whatever
made
a
controller,
is
supposed
to
do?
A
Yes,
the
answer
is
yes,
that
was
those
three
slides
was
all
you
needed
to
do
to
build
an
entire
controller
and
by
the
way,
this
controller
is
a
sample
controller
that
kubernetes
provides
in
a
gate
up
and
that,
if
you
want
to
do
it
using
matter
controller,
you
can
do
that
and
you
just
needed
three
slides
at
least
I
needed
three
slides
and,
of
course,
I
use.
Javascript
I
do
not
use
go.
No
now
am
I.
Saying
mara
control
is
using
matter
controller.
We
can
build
controllers
that
were
language
agnostic
right.
A
The
answer
is
yes
again
yeah.
You
can
use
any
languages
of
your
choice.
Even
there
are.
You
know
there
are
users
of
matter
controller
who
have
used
JSON
net
to
build
the
controller's.
Having
said
that,
you
know,
let's
not
just
jump
into
the
JSON
net
or
all
those
fancy
languages
right.
It's
it's
all
about
what
you
need
if
it's
really
really
simple
yeah,
of
course
you
can
go
ahead
with
the
juice
on
it.
Otherwise,
just
stick
to
your
competence.
A
If
it
is
Python
yeah,
why
not
use
a
Python
code
to
you
know
to
build
your
controllers?
You
don't
have
to
learn
kubernetes
for
that
right,
because
the
meta
controller
abstracts
away
the
kuben.
It
is
entirely
from
you
when
you
try
to
Bill
your
controllers,
okay,
so
that
was
about
like
what
meta
controller
can
achieve
in
a
language,
language,
agnostic,
fashion
right.
But
then
let
us
try
to
you
know,
look
a
bit
into
deployment
models
of
meta
controller,
or
one
of
the
models
that
we
can
use
is
a
shared
model.
A
Very
meta
controller
runs
as
a
pod,
a
single
part,
and
we
can
have
multiple
the
pods
which,
in
each
of
the
part
they
will
have
the
controller's
right.
These
spots
will
be
talking
onto
the
meta
controller.
All
right
and
each
of
these
parts
can,
you
know,
observe
or
can
watch
their
customer
sources.
That
is,
this
is
just
one
model.
The
second
model
is
shared-nothing
model.
Where
you
don't
share
anything
with
each
controller
is
isolated
right.
You
have
your
own
meta
controller
and
you
have
your
own
controller
and
then,
of
course
you
are
the
CR.
A
You
can
have
everything
boiled
down
to
or
a
wrap
in
one
container,
so
you
can
have
one
part
one
container,
okay
and
that
particular
container
will
be
taking
care
of
your
taking
care
of
the
controller
needs
of
your
customer
resource.
It
depends
on
you,
so
how
how
you
want
to
you
know,
build
it
alright,
so
you're
getting
to
the
end
of
the
station.
Now.
What
are
the
key
takeaways?
A
The
best
thing
is,
you
don't
have
to
deal
with.
Kubernetes
kubernetes
is
Delta
for
you
via
the
meta
controller,
so
no
informants,
no
Lister's,
no
Watchers,
no
crew
api's.
You
don't
need
to
do
any
client.
Api
calls
at
all
and
of
course,
idempotent
code
is
like
really
really
easy
for
you,
because
all
you
need
to
do
is
construct
the
final
state,
the
desired
state,
and
if
you
do
that
and
keep
on
sending
the
same
thing
again
and
again
in
your
response,
your
code
is
idempotent
again.
A
So
this
is
one
of
the
very
very
important
aspects
of
testing
right.
100%
test
isn't
test
coverage
is
always
possible.
Okay,
because
you
don't
have
to
you
know,
because
there
is
no
kubernetes
and
all
those
infamous
Watchers
and
all
those
stuff
you
don't
have
to
depend
on
the
fake
and
a
fake
libraries
and
all
you
don't
have
to
write
your
code,
your
business
logic
in
such
a
way
that
you
know
you
have
to
you
should
be
able
to
inject
the
mocks,
and
all
you
don't
have
to
you
can
just
rely
on
your
standard
idiomatic.
A
Your
own
language
unit,
testing
pattern
and
I
like
I
said,
like
we
talked
about
is
language
agnostic.
It
has
been
on
top
of
clinical.
You
can
even
for
those
golang
libraries
or
for
those
golang
authors
who
want
to
you
know,
implement
their
controllers.
You
using
golang,
you
can
even
input
meta
controller
as
a
library
you
just
don't
need
to.