►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
To Crd, or Not to Crd, That is the Question - Ed King & Sam Gunaratne, Pivotal
One of Kubernetes’ most promising features is the ability to extend it’s API via use of Custom Resource Definitions (“CRDs”). As such, it comes as no surprise that CRDs have recently seen a big rise in popularity. But what exactly makes CRDs so great? And, more importantly, when should you be thinking about using them? Whilst Kubernetes Operators are often cited as a main use case for CRDs, their use is evolving into something much more exciting. Now, not only are developers extending the API with customizations for their Kubernetes clusters, but we are starting to see them making use of CRDs to build their applications! This is something that both Sam and Ed have been thinking about for a while now. In this talk they present their learnings by discussing the relative merits and drawbacks of a CRD-based approach to application development compared to more traditional approaches.
https://sched.co/NrnG
A
All
right,
everyone,
let's
get
started,
so
thank
you
so
much
for
attending
our
talk
and
on
custom
resource
definitions,
also
known
as
CI
DS,
it's
great
to
be
here
in
Shanghai.
Let's
start
off
with
some
introductions,
so
my
name
is
Sam
gonna
Radner
I'm,
a
software
engineer
working
for
pivotal
and
from
London
my.
A
Great,
so
there
are
a
lot
of
talks
about
C
IDs
at
this
conference.
It's
sort
of
a
hot
topic
in
the
community
at
the
moment.
So
let's
just
set
the
scene
about
what
we'll
be
talking
about
in
this
talk
specifically,
so
ed
will
start
us
off
just
talking
about
our
motivations
for
giving
this
talk,
then,
since
this
is
a
beginners
talk,
we'll
just
cover
some
basics
about
the
kate's
api
customer
resource
definitions
and
operators.
B
The
rest
of
this
talk,
and
so
this
all
started
about
six
months
ago,
when
Sam
and
I
were
assigned
to
a
new
team,
and
we
were
tasked
with
building
a
brand
new
system
component
that
would
eventually
integrate
into
part
of
a
bigger
product
that
we
were
going
to
be
shipping.
And
so
we
went
off.
We
did
some
research,
we
spoke
to
some
users
and
we
eventually
came
up
with
an
idea
for
what
we
might
like
to
build
for
our
alpha,
and
the
question
then
became
okay.
So
what
technology
should
we
use
to
build
this
alpha?
B
B
However,
at
the
time
we
were
very
aware,
a
lot
of
noise
coming
out
of
the
kubernetes,
the
kubernetes
community,
around
the
kubernetes
api
and
specifically
with
regards
to
extending
the
api
by
a
use
of
these
things
called
c
RDS,
and
so
are
a
little
ears
pricked
up
and
we
thought
okay.
What
might
it
look
like
for
us
to
build
this
system
component
on
top
of
the
kubernetes
api
as
to
as
opposed
to
a
more
like
traditional
web
server
with
a
backing
relational
database?
B
A
Ok,
so
let's
take
a
quick
step
back
and
talk
about
the
kubernetes
api
from
a
high
level,
so
the
kubernetes
api
is
an
api
centric.
So
what
do
I
really
mean
by
this?
So
when
I
first
came
to
communities
and
I
came
from
thinking
about
wanting
to
deploy,
workloads,
won't
use
communities
as
an
Orchestrator
and
a
scheduler
for
my
containers,
but
the
more
I
came
to
understand
about
kubernetes.
A
The
more
I
realized
that
one
of
the
reasons
that
it
is
such
a
good
scheduler
is
because
it
has
this
API
with
a
set
of
rich
features
that
allows
for
the
kubernetes
ecosystem
to
evolve
and
four
features
and
api
s--
to
be
added
an
extreme
rate
from
companies
all
across
the
world
and
its
really
the
kubernetes
api
that
facilitates
this.
So
let's
have
a
little
look
at
some
of
these
high-level
properties.
Now
the
first
is
of
the
communities.
Api
is
declarative,
so
this
is
the
Watts
versus
how
approach
so
in
an
imperative
procedure
way.
A
You
might
say:
I
would
like
to
go
to
node
a
and
start
one
pod
and
then
I'd
like
to
go
to
node
B
start
another
pod
you're
telling
the
system
how
to
achieve
something
with
a
declarative
system.
You
say
what
you
want
to
happen,
so
you
say:
I
won
two
pods
and
by
the
way,
don't
have
those
pods
running
run.
On
the
same
note
and
there's
been
an
evolution
over
time
of
how
you
treat
infrastructure
moving
from
procedural
to
declarative.
A
So
you
might
write,
have
written
bash
scripts
ten
years
ago
to
manage
your
infrastructure
and
then
you
might
have
chosen
something
like
chef
or
ansible
to
manage
your
infrastructure
and
then
things
like
terraform
came
along,
which
is
a
purely
declarative
way
of
defining
infrastructure
and
cuba
dennis
takes
that
to
the
next
level
being
declarative,
but
also,
as
you'll,
see
constantly
reconciling
that
state
of
the
world,
though
next
is
the
the
cuban
a's
api.
Has
this
clear
separation
of
states?
A
A
Next
up
is
that
the
cuban
a's
api
is
level
based
rather
than
edged
based.
So
what
do
I
mean
by
that?
This
is
terminology
that
sort
of
comes
from
system,
programming
and
electrical
engineering,
and
it's
about
how
you
measure
the
changes
on
a
wire
over
time.
So
in
a
H
based
system,
you
notice
the
individual
changes
and
you
keep
a
track
record
of
those
changes
level
based.
A
So
next
up
is
that
the
CUNY's
api
is
transparent
by
default.
So
writing
resources
to
the
API.
You
have
a
single
control
plane
for
everything
and
permission,
allowing
all
of
your
components
can
access
all
the
other
resources
from
other
components.
So
this
allows
for
composability
between
teams,
controllers
can
watch
other
teams
resources
and
then
extend
the
functionality.
B
So
at
its
cool,
the
Kuban
Eddie's
API
is
a
restful
api,
backed
by
an
HDD
datastore,
and
there
are
a
bunch
of
resource
objects
that
are
available
on
that
API.
So
these
are
things
like
your
pod
node
deployment,
and
these
are
the
objects
that
users
of
the
API
are
able
to
interact
with
and
to
create,
update
and
delete,
and,
as
a
user
of
the
API,
it's
up
to
me
to
declare
my
desired
state
of
the
world.
B
As
Sam
said,
the
kubernetes
api
is
entirely
declarative
and
so
for
this
example,
I
might
say
something
like
I
want
one
pod
to
be
running
and
the
way
that
you
do
this
is
by
posting
gamal
into
the
API,
and
typically
this
is
done
with
a
cube.
Ctl
apply
eff.
Take
a
quick
look
at
that
Yama.
We
tried
really
hard
not
to
get
any
Amal
into
this
presentation,
but
that
is
a
really
hard
thing
to
do
like
when
you're
talking
about
the
kubernetes
api.
So
we're
try
to
keep
it
brief.
B
So
the
kubernetes
api
is
comprised
of
a
number
of
controllers
and
each
controller
is
watching
for
any
changes
to
certain
objects
of
a
certain
type.
So,
for
example,
this
controller
in
the
top
left
may
be
looking
for
any
changes
to
any
object
of
type
pod
and
when
it
notices
a
change
in
this
case
that
a
new
pod
has
been
created,
it
gets
notified
about
it
and
it
then
goes
through
a
process,
that's
known
as
reconciliation.
B
B
So
we've
got
our
resources.
We've
got.
Our
control
is
how
the
custom
resource
definitions
fit
into
all
of
this.
Well,
essentially,
custom
resource
definitions
or
see
IDs
for
short,
allow
users
of
the
API
to
define
their
own
custom
resource
types.
So
perhaps
I
don't
want
to
create
a
pod
or
a
deployment.
Perhaps
I
want
to
create
something
more
exciting.
C
B
Take
an
example
of
a
lovely
piece
of
cake:
everybody
loves
cake.
What,
if
I
wanted
to
create
update,
delete
cakes
on
my
kubernetes
api?
This
is:
what's
the
ideas
allow
us
to
do,
and
so
in
this
example,
we
might
say:
ok,
actually
I
want
a
kind
of
cake
right
and
in
the
specification
that
I'm
saying
that
I
want
a
Victoria
sponge
type
of
cake,
and
the
nice
thing
here
is
that
I
can
take
this.
B
B
I
will
know
how
to
reconcile
the
world
upon
any
changes
to
those
objects,
and
this
is
really
at
the
core
of
the
extensibility
of
the
kubernetes
api
using
crts
and
your
custom
controllers,
and
so
you
could
imagine
that
you
could
keep
going
building
up
your
own
custom
types.
Custom
control
is
such
that
it
looks
a
little
something
like
this.
A
Ok,
so
let's
have
a
brief
look
at
the
timeline
of
extensibility
in
the
communities
API,
so
around
I
think
communities,
1.2
third
party
resources
are
introduced,
and
this
is
the
first
time
that
you
had
a
way
to
add
new
things
to
the
kubernetes
api.
So
shortly
after
core
OS
introduced,
this
thing
called
the
operator
framework,
and
so
really
what
they
realized
is.
A
This
operator
will
ensure
the
cluster
is
alive
and
healthy
and
take
backups
and
that
sort
of
stuff-
and
this
has
proved
an
immensely
popular
pattern
in
the
committee's
community.
So
the
operator
framework
is
one
framework
of
actually
doing
this,
but
now
there
are
more
such
as
coop
builder,
so
in
2017
CRTs
were
introduced
as
a
replacement
for
third-party
resources
to
have
even
closer
alignment
to
the
kubernetes
native
resources
and
to
be
more
powerful
and
then,
after
that,
we
started
to
see
bigger
systems
being
built
on
top
of
see
IDs.
A
It's
a
good
example
of
this
is
Kay
native,
which
is
an
entire
new
platform
and
set
of
abstractions
on
top
of
kubernetes.
Another
great
example
is
sto,
which
allows
you
to
get
an
entire
service
mesh
on
top
of
kubernetes
and
I.
Think
sto
has
something
around
23
or
so
custom
resource
definitions
that
it's
made
up
of
so
next.
This
is
something
that
was
very
interesting.
It
was
the
AWS
released
an
operator,
and
this
was
special
because
it
didn't
act
on
resources
within
your
cluster.
A
Instead,
it
acted
on
resources
in
AWS
is
public
cloud,
so
as
a
developer,
I
created
a
custom
resource
asking
for
infrastructure,
and
the
operator
would
go
to
Amazon's
public
cloud
to
provision
that-
and
this
is
an
interesting
example
of
what
the
operator
pattern
being
used
to
not
manage
workloads
in
your
cluster,
but
managing
something
completely
different,
and
so
this
really
got
us
thinking
like
what
is
the
next
step
of
this
transition.
Percy
IDs,
so
just
in
the
near
term,
series
are
hoping
to
go
GA
in
the
next
couple
of
releases.
A
B
Cool,
so
let
me
paint
you
a
picture
so
for
the
next
part
of
this
talk,
we're
going
to
be
running
through
an
example
service
and
take
a
look
at
what
it
might
look
like
to
build
something
on
top
of
the
cube
kubernetes
api
and
the
example
that
we're
going
to
go
with
is
that
of
a
home
automation
service.
So
we're
going
to
imagine
that
Sam
and
I
have
a
hugely
successful
home
automation,
company
and
we've
got
a
really
great
home
automation
service.
B
That's
used
in
loved
by
thousands
of
people
all
around
the
world
and
before
taking
a
look
at
what
it
might
look
like
to
build
that
on
top
of
kubernetes.
Let's
just
imagine
what
it
may
look
like
today
right-
and
perhaps
it
looks
something
like
this.
Perhaps
we've
gone
with
a
classic
microservices
architecture,
whereby
some
user
of
the
system
posts
some
JSON
into
some
front
end
some
gateway
and
the
micro
services
go
off
and
do
whatever
they
need
to
do
so.
B
We
take
this
Jason,
we
post
it
in
the
micro
services,
go
off
and
do
that
thing
and
because
we're
a
hugely
successful
company,
we
have
hundreds
and
hundreds
of
developers
co-located
all
across
the
globe
right,
and
so
it's
quite
likely
that
each
service
is
going
to
have
a
corresponding
team
who
are
going
to
be
responsible
for
developing
that
service
and
really
each
team.
Their
main
focus
is
that
they
want
to
be
delivering
the
absolute
highest
quality,
best
lighting
service
that
they
possibly
can
right.
B
And
so,
let's,
let's
draw
some
team
boundaries
around
each
of
these,
so
that
it
looks
a
little
bit
like
this
and
then
we
can
also
imagine
that
each
service
is
going
to
need
some
data,
and
so
each
service
is
probably
going
to
need
a
backing
database
right.
And
so
it's
going
to
look
something
like
this
and
then
we're
going
to
imagine
that
we
really
care
about
a
high
availability
and
the
reliability
of
our
service
is
really
and
really
important.
B
If
this
goes
down,
users
are
going
to
be
locked
out
of
their
home
and
it's
going
to
be
a
disaster.
So,
let's
add
in
a
whole
bunch
of
high
availability
to
each
of
these
services,
but
we
know
that
nothing
is
ever
perfect
and
things
go
wrong,
and
so,
when
things
do
go
wrong,
we're
going
to
need
to
make
sure
that
each
of
the
services
has
a
whole
bunch
of
like
retry
logic
baked
in
and
then.
B
Finally,
we're
probably
also
going
to
need
some
sort
of
like
service
discovery
mechanism,
so
that
each
of
these
services
can
find
each
other
in
order
to
do
the
job
and
so
actually
like
I.
Look
at
that
and
I
think
that
there
there's
a
lot
going
on
there
right
and
importantly,
there's
a
lot
of
stuff
going
on
that
actually
is
distracting
each
of
the
teams
away
from
delivering
the
features
on
each
of
the
services,
and
that's
really
the
thing
that
they
want
to
be
doing
right.
B
So
there
are
a
lot
of
distractions.
Each
team
has
to
think
about
their
storage,
the
backing
databases
and
all
of
the
operational
overheads
that
come
with
that
the
high
availability,
the
reliability,
the
API
contracts
between
all
of
the
services.
All
of
this
is
just
taking
time
and
effort
away
from
the
main
focus
of
each
team
which
it's
developing
the
best
service
that
they
possibly
can,
and
so
maybe
maybe.
A
A
A
Top
of
the
kubernetes
api,
so
not
using
kubernetes
as
a
container
scheduler,
but
just
using
its
API
to
achieve
this
sort
of
business
problem
that
we
have
here.
So
let's
go
through
that
example.
Now
the
first
of
all,
each
of
the
team
wouldn't
be
responsible
for
creating
a
micro
service
like
a
restful
micro
service.
There
be
responsible
for
creating
a
controller
instead
and
we
wouldn't
have
a
gateway.
Instead,
we'd
have
our
controllers
watching
a
single
kubernetes
api,
and
so
before
our
teams
had
to
manage
their
in
database.
A
Now
they
can
rely
on
the
kubernetes
api
to
store
their
data
via
C
IDs.
So
in
this
world
the
user
might
express
intent
to
the
system.
Declaratively
saying
I
want
the
kitchen
lights
to
be
on,
and
so
they
could
post
this
into
the
kubernetes
api
and
they
might
have
a
resource
that
represents
a
room
for
example,
and
I
have
a
specification
for
what
I
want
the
desired
state
of
the
room
to
be
in
terms
of
the
lights
here.
So
you
can
see.
A
I
have
I
want
two
lights
to
be
on
with
different
brightnesses,
so
that
could
be
posted
in
and
we
would
have
a
controller,
the
team
that's
responsible
for
that,
watching
the
API
it
would
be
notified.
It
would
start
to
reconcile
that
request.
It
would
look
at
the
system
and
say:
hey,
there's
no
lights
on,
so
what
would
it
be?
It
would
create
some
light
resources
and
those
light
resources
would
represent
the
individual
actions
for
each
light
bulb
in
our
room
wherever
it
is
so.
A
The
light
resource
just
has
a
single
speck
requesting
the
brightness,
and
then
this
would
be
picked
up
by
our
lights
controller,
and
this
is
the
boundary
between
our
two
teams.
In
this
example,
the
room
team
just
creates
this
light
resource
event.
It's
another
team's
responsibility
through
its
controller
to
satisfy
that
request.
So
the
light
controller
then
would
go
and
look
at
the
actual
state
of
the
world
go
and
talk
to
the
LED
bulbs,
wherever
they
are
notice
that
they
are
not
on
turn.
C
A
A
But
then
our
team's
controller
would
be
notified
of
this
change,
because
it's
a
parent
of
these
light
resources.
It
created
them.
It
has
an
interest
of
what's
going
on
with
them
and
then
it
would
update
the
room
resource
with
the
status
of
the
lights,
and
this
is
important
because
the
user
or
whatever
system
component
created
the
room,
maybe
doesn't
care
so
much
about
the
internal
resources
used
to
accomplish
the
request.
A
It
cares
about
this
abstraction,
and
so
it
can
just
look
at
the
room
resource
to
see
the
status
of
the
request
and
obviously
in
this
world.
If
one
of
our
controllers
goes
down,
for
example,
then
it's
decoupled
from
the
API.
You
can
still
post
room
resources,
post
light
resources
and
then,
when
the
controller
comes
back
up
again,
it
can
just
reconcile
and
because
of
some
of
the
aspects
of
the
API
being
level
based
that
will
just
work.
B
Well,
maybe
I
am,
let's
see
if
you
are
thinking
about
doing
this
sort
of
thing.
There
are
going
to
be
some
pros
cons
and
considerations
right
and
so
for
the
rest
of
this
talk
we're
just
going
to
be
walking
through
some
of
the
main
things
that
popped
up
during
our
research
for
this
so
I'm
going
to
talk
through
a
few
of
the
more
purely
sort
of
technical.
C
B
Then
Sam's
going
to
talk
through
some
of
the
wider
considerations
around
this
as
well:
I'm
gonna,
kick
things
off
by
talking
about
storage
and
really
here
I'm.
Referring
to
this
idea
that,
if
you
are
developing
on
top
of
the
kubernetes
api,
you
essentially
get
the
backing
at
CD
data
store
like
for
free
right,
it's
just
there,
it's
part
of
that
API
control
plane,
and
it
means
that
it's
probably
owned
and
operated
by
whoever
is
responsible
for
keeping
the
convene,
Eddy's,
API
up
and
running,
and
so
as
a
developer
who's
developing.
On
top
of
that
API.
B
But,
of
course,
there's
like
there's
no
such
thing
as
a
free
lunch,
and
there
are
some
other
considerations
around
here.
It's
not
quite
that
simple
one
of
the
one
of
the
main
things
that
popped
up
while
we
were
sort
of
looking
at
this
is
that
HCD
is
not
a
relational
database,
which
means
that
some
of
the
relational
features
we've
come
to
know
and
love
unavailable,
and
the
example
that
we
tend
to
give
is
what,
if
you
needed
to
do
a
big
join
of
your
data
right,
it's
just
not
really
possible,
and
so
I
guess.
B
The
key.
The
key
thing
to
note
here
is
like
just
make
sure
that
your
data
model
is
suitable
for
running
in
this
new
model
recently
readily
related
to
the
storage.
Is
this
idea
of
high
availability,
so,
when
deployed
in
a
multi
master
configuration
the
capes
API
is
highly
available
and
again
that's
sort
of
highly
available
just
for
free
right,
again,
really
really
enticing
to
me,
because
it
means
I,
don't
have
to
worry
about
the
availability
of
my
API.
B
It's
just
highly
available
for
free
and
I,
then
have
more
time
to
focus
on
writing
my
services
and
shipping.
The
features
that
I
care
about
there
is
another
aspect
to
this.
This
idea
of
high
high
availability,
I
think
Sam
touched
on
it
earlier
when
we
were
talking
about
high
availability
of
the
controllers
themselves
and
so
there's
maybe
a
question
of
like
hey,
should
I
be
running.
My
controller
is
highly
available
and
I
guess
I
guess.
The
thing
to
note
there
again
is
that
in
this.
A
B
Level
based
system-
perhaps
it's
actually
okay,
if
your
controller
goes
down
for
some
period
of
time,
because
all
that's
going
to
happen
when
it
comes
back
online,
is
it's
going
to
read
the
desired
state
of
the
world
trigger
it's
reconcile,
function
and
away
you
go,
and
the
last
thing
I
want
to
talk
about
is
performance.
So
again,
the
performance
of
your
API
is
going
to
be
largely
dependent
on
the
underlying
EDD
datastore,
and
there
have
been
some
concerns
around
the
performance
of
EDD
in
the
past.
I
guess.
B
The
key
thing
here
is,
or
maybe
the
key
worry
is
that
if
you've
got
a
particularly
noisy
service,
that
then
could
potentially
have
a
very
negative
impact
on
the
performance
of
not
only
your
system
but
also
like
any
other
system.
That
is
also
running
on
that
kubernetes
api,
and
so
we've
got
to
be
a
little
bit
careful
I
guess
the
general
advice
is,
if
you,
if
you're
developing,
something
that's
going
to
be
slamming
the
database,
quite
a
lot
like
maybe
like
a
metrics
thing
or
if
you're
doing
something
like
dumping.
B
Your
logs
in
the
database,
probably
not
a
good
idea,
but
we
do
also
have
this.
This
idea
that
there's
a
there's
a
big
conversation
going
on
in
a
community
at
the
moment
as
to
whether
or
not
you
should
just
be
running
one
huge
kubernetes
cluster
or
if
perhaps
it's
better,
to
run
lots
of
little
ones
and
if
you're
off
the
mind
that
lots
of
little
ones
is
the
way
to
go,
then
perhaps
these
sorts
of
considerations
are
actually
not
so
bad.
B
It's
even
possible
that
you
may
just
have
a
separate,
dedicated
cluster
purely
for
these,
like
extension,
you
may
not
even
enable
the
scheduler
one
man,
for
example,
so
there's
lots
of
things
to
think
about.
The
final
thing
I
want
to
say
about
performance.
Is
the
API
machinery.
Folks
are
thinking
about
scaling
targets
for
CID
as
themselves
and
there's
a
survey
there
if
you're
interested,
please
go
put
that
out.
A
A
Take
advantage
of
all
of
the
benefits
of
kubernetes
being
declarative,
eventually
consistent
for
our
own
needs
of
our
own
service.
Then
we
need
to
put
in
some
work
to
actually
achieve
that.
One
of
those
things
that
we
need
to
do.
One
of
those
things
we
need
to
do
is
we
can't
just
take
our
micro
service
and
turn
it
into
a
controller
very
easily.
It's
a
completely
different
paradigm
of
writing
software.
A
Your
service,
it's
probably
like
a
restful
service,
and
it's
request
and
response
talking
to
a
database
may
be
talking
out
to
some
other
services
through
RPC
or
more
rest
calls.
However,
writing
a
controller
is
running
a
reconcile
function
that
is
watching
an
API
and
converging
state.
It's
a
very
different
way
of
thinking
and
writing
software
and,
as
I
just
said
saying,
not
everything
fits
like
the
cumulates.
A
Api
is
opinionated
and
for
good
reasons,
but
if
what
you're
trying
to
model
doesn't
fit
so
you
do
need
that
request
response,
synchronized
synchronicity,
you
can't
map
your
domain
as
resources
or
you
have
a
very
complex
domain
model.
Then
it's
going
to
be
an
uphill
battle
and
you're
going
to
be
fighting
the
API
to
model
it
in
kubernetes
next
up-
and
this
is
particularly
interesting
for
us-
that
what
we
really
want
to
do
is
collaborate
the
way
the
kate's
community
collaborates.
A
We
want
CRTs
to
be
the
standard
interface
between
our
teams.
In
the
micro
services
example,
we
should
saw
earlier.
Maybe
each
team
had
to
define
their
own
API,
and
maybe
you
have
company
standards
for
that
API.
We
also
had
to
have
a
service
discovery
system
so
that
we
could
discover
all
the
other
API
is
available.
So
in
kubernetes
the
CID
is
the
interface
between
our
team
and
what's
the
ideas
are
available
in
the
cluster?
A
Is
the
discovery
mechanism,
and
so
this
interest
EDA
leads
to
interesting
collaboration
patterns
where
we
can
have
different
teams
responsible
for
different,
see,
IDs
and
maybe
independently.
Another
team
can
monitor
the
status
of
one
team
and
extend
the
functionality
through
this
mechanism,
so
it
allows
for
real
loose
coupling
of
interactions
between
teams
and
finally,
we
don't
really
have
time
to
go
through
too
many
of
these,
but
there
are
a
whole
bunch
of
other
API
features
that
you
probably
want
to
consider
and
research.
A
If
you
are
thinking
about
doing
something
like
this
I
just
took
through
a
couple
so
binary
data,
for
example,
you
should
not
be
using
sed
to
store
to
store
binary
data.
If
you
do
need
this,
this
might
not
be
the
right
approach
for
you.
Alternatively,
you
could
upload
your
data
to
something
like
s3
and
then
just
refer
to
it
in
your
specification
in
your
custom
resource
next,
if
command
any
supports,
defining
relationships
between
resources
and
there's
sort
of
two
main
mechanisms
for
doing
this.
A
The
first
is
a
owner
reference
which
allows
you
to
set
up
child
parent
relationships
between
your
resources,
and
this
gives
you
properties
such
as.
If
you
delete
the
parent,
the
child
resources
will
be
cleaned
up.
The
other
common
one
is
adding
labels
to
your
resources.
However,
it
should
be
noted
that
managing
the
integrity
of
those
labels
is
up
to
you
as
the
controller
writer,
so
you
have
to
ensure
that
those
labels
make
sense
in
a
changing
system.
A
Next
versioning
is
interesting,
so
custom
resources
allow
you
to
have
multiple
versions
installed
at
the
same
time.
So
you
can
imagine
that
teams
are
writing
new
versions
of
their
custom
resource,
but
other
teams
are
relying
on
older
versions,
so
there's
some
backwards.
Compatibility
features
there
and
now
in
communities,
1.5
there's
a
feature
that
allows
you
to
migrate
from
one
custom
resource
version
to
another
custom
resource
version.
And
finally,
since
we
have
this
one
control
plane
the
kubernetes
api
and
everything
is
represented
as
c
ids.
A
We
can
take
advantage
of
any
tooling
that
exists
for
this
control
plane.
So
couple
will
work
any
you
eyes
that
are
developed.
That
will
work
the
way
you
list.
Everything
in
your
system
will
be
ubiquitous,
and
so
all
tooling
will
be
the
same.
So
I
guess
the
question
is
sort
of
remains.
Is
it
a
good
idea
to
use
C
IDs
outside
of
the
scheduler
so
to
see
I
do
not
to
see
ID.
B
And
you
should
go
all
in
and
it's
gonna
be
fantastic.
Others
are
more
concerned,
think
that
just
stay
away,
and
so
whenever
in
doubt
you
know
we'd
like
to
look
to
Twitter
to
see
what
people
are
thinking
there
was
actually
like.
There
was
a
good
tweet
the
other
day
there
kind
of
some
some
of
this
up,
which
was
this
one
here
and
I
guess
the
takeaway
is
like
yeah.
D
B
Is
a
new
technology
I
think
people
are
getting
excited
about
it
and
I
think
quite
rightly
so.
I
think
there
is
a
lot
of
potential
in
this
new
way
of
working
and
I'm
really
really
excited
to
see
where
it's
going
to
go,
but
at
the
end
of
the
day
we
need
to
remember
like
not,
everything
has
to
be
a
CRV
and
with
that
I
think
that
that's
all
we've
got
time
for
so.
Thank
you
very
much.
B
C
E
A
Guess
the
honest
answer
is
I'm
not
sure
so
there's
quotas,
which
have
been
introduced
in
a
new
version
of
cabinets
but
I'm,
not
sure
that
cost
of
white
coats
is
I,
don't
know
if
anybody
knows
I'm,
so
your
shakes
have
head
so
I.
Don't
think
so
think
so
you
can
set
quotas
within
a
namespace,
but
not
a
cluster
wide,
but
not
today.
B
B
Yeah,
so
actually,
like
your
controllers,
don't
even
need
to
be
running
in
kubernetes,
so,
for
example
like
when
we're
developing
this
control
is
one
pattern
we
see
is
that
you
run
your
controller
on
your
local
developer
machine
and
you
just
have
to
point
it
and
make
sure
that
you
have
access
to
the
master
API.
So
you
can
actually
run
these
controllers
from
anywhere
I.
Think
the
typical
pattern
is
that
you
would
typically
run
them
inside
the
cluster,
but
there's
there's
nothing
that
says
that
you
have
to
do
that.
B
D
So
I
have
a
kinda
change
about
the
mountain
version
and
we
all
see
IDs
I'm,
not
sure
in
your
case.
How
are
you
supporting
the
mountain
version
for
USAID,
for
example,
for
your
case
for
Team
a
he
wanted,
the
lightning
service
version,
but
example
version
one,
but
they
added
every
team.
C
wants
this
version
to
how
managing
the
relationship
between
the
mountain
versions
and
the
other
one
and
how
you
validate
your
say
at
least
gamer,
how
you
make
sure
all
the
same
object?
C
A
You
series
allow
you
to
have
multiple
versions
install
but
you're
still
going
to
have
the
problem
of
needing
to
know
to
some
degree
which
teams
are
consuming
that
version,
because
you
don't
want
to
keep
your
old
CI
DS
around
forever.
As
a
team,
you
want
to
know
when
you
can
deprecated
them
and
I,
don't
think,
there's
any
sort
of
technology
solution
for
that.
It's
sort
of
a
team
collaboration,
sort
of
problem
so.