►
From YouTube: Cloud-Init with Crossplane - Marques Johansson, Equinix
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Cloud-Init with Crossplane - Marques Johansson, Equinix
Baremetal and VM instances provisioned with Crossplane must rely on user-data fields or other forms of provisioners. It is only with these additional provisioning steps that it is possible to replicate managed services on bare metal or managed infrastructure. Cloud-init is a standard format for user-data that allows for numerous config files or scripts to be combined, all in different formats. Cloud-init data can be lengthy and cloud providers limit the size of user data. It is often necessary to combine and compress cloud-init scripts. We’ll explore ways to leverage cloud-init as user-data, demonstrating the higher-order integrations that are possible when Crossplane manages low-level resources.
A
Hi,
I'm
marcus
johansen,
I'm
going
to
talk
to
you
about
using
crossplane
to
provision
cloud
init
resources
which
is
effectively
text
which
we're
then
going
to
use
to
provision
bare
metal
resources
or
vm
instances
that
take
a
user
data
parameter.
The
user
data
parameter
is
a
blob
of
string
text,
but
cloud
init
is
a
special
format
for
user
data
that
allows
us
to
get
more
text
in
there
by
compressing
it
and
mime
encoding
it.
Let's
take
a
look.
A
What
do
we
know
about
cloud
init?
Well,
it's
a
service
that
runs
inside
the
operating
system.
That
knows
how
to
determine
what
cloud
it's
running
in
if
it's
running
in
a
cloud
or
if
it's
using
some
local
configuration
say
pulling
cloud
data
from
a
floppy
disk
from
cd-rom
or
from
the
metadata
of
your
cloud
provider.
A
What
is
inside
of
that
cloud
init?
What
is
that
cloud
net
service
processing
the
data
that
it's
fetching
from
that
metadata
service?
The
data?
That's
touching
from
that
floppy
or
cd-rom
drive?
Well,
what
it's
processing
is
a
number
of
different
possible
formats.
It
could
be
mime
encoded,
multi-part,
mime,
encoded
text.
It
could
be
gzipped
multi-part,
miming
coded
text.
A
So
what
does
it
look
like
to
use
cloud
net?
Well,
it
there's
not
an
api
that
is
out
there
called
cloudinit.com
where
you
do
anything
with
this.
Is
I'm
sure
that
domain
exists,
but
that's
not
what
we're
talking
about
so
the
way
that
you
typically
interact
with
cloud
in
it
is
through
your
cloud
provider
if
you're
spinning
up
a
vm
instance
or
a
bare
metal
instance,
you
need
to
provide
some
sort
of
way
to
provision
that
resource
and
usually
they
will
provide
a
field
called
a
user
data
field.
A
A
Within
that
metadata
service,
it
will
fetch
the
user
data,
some
when
I
say
metadata
service,
so
like
ec2
instances
they
have
this
address.
I
think
it's
once
extined
at
254.110
and
then
there's
a
common
url
path
to
get
to
that
user
data.
I
think
it
in
this
case.
It's
just
slash
user
data
or
it
might
be
slash
metadata
user
data.
A
In
any
case,
when
system
boots
up,
it
is
essentially
fetching
that
web
address
and
what
it
finds.
There
is
some
blob
of
text,
the
blobs
of
text
that
cloud
and
it
recognizes
and
knows
how
to
handle
typically
start
with
a
shebang,
and
if
they
don't
start
with
the
sherbeting,
then
it
also
knows
how
to
handle
the
multi-part
mime
format.
A
So
when
you
are
provisioning
the
device,
you
have
some
field
where
you're
specifying
the
user
data
that
you
want
to
boot
with
and
that's
it
the
like,
I
said
it's,
the
user
data
becomes
available
within
the
metadata
service,
the
api
that
you're
creating
the
device
through
or
the
instance
through,
may
let
you
modify
that
user
data
later
later,
or
look
at
that
user
data
later
why
you
might
think?
Why
would
that
be
a
useful
thing
to
do?
A
Well,
the
user
data
is
typically
used
at
startup,
but
other
formats
of
user
data
say
the
ignition
format
or
the
kickstart
format.
So
in
some
systems
this
data
can
be
used
on
subsequent
boots
or
if
the
api
allows
you
to
always
fetch
that
user
data,
then
you
could
sort
of
use
it
as
a
way
to
to
be
bad
and
send
messages
from
your
api
from
your
cloud
providers.
Api
to
your
metadata
service
that
your
instance
can
then
pick
up.
A
So
this
is
cross
playing
community
today.
Why
am
I
sitting
here
talking
about
cloud
net
and
provisioning
devices?
Well,
crossplane
is
known
for
was
known
for
managing
services
on
various
clouds.
It's
known
for
managing
storage,
in
some
cases
like
through
the
rook
provider
or
through
block
storage
services
on
various
clouds
and
all
kinds
of
sql
databases
and
kubernetes
manage
kubernetes
services
and
any
kind
of
managed
service
you
can
think
of,
but
wait.
There's
more
crossplane
also
knows
how
to
provision
vms
and
bare
metal
instances.
A
Now
I,
with
the
providers
that
are
out
there
today,
I
don't
know
how
strong
that
statement
is.
I
know
that
the
equinix
metal
provider
does
allow
you
to
provision
bare
metal
devices.
I
think
that
the
digitalocean
provider
lets
you
provision
droplets.
I
previously
worked
on
a
linux
provider
that
knew
how
to
provision
instances.
A
So
there
are,
there
are
at
least
a
handful
half
a
handful.
Maybe
I
think
ec2
instances
are
probably
supported
in
the
aws
provider
since
that's
generated
and
there's
some
like
60
plus
resources,
supported
point
being
that
with
crossplane.
There
are
a
number
of
ways
to
a
number
of
providers
that
allow
you
to
provision
resources
that
would
benefit
from
user
data.
A
A
A
Wait
a
second
crossplane
provider,
there's
no
provider
here
who
there's
you
said:
there's
no
cloudynet.com!
I
heard
you
and
there's
no
remote
api
right.
So
crossplane
has
crosstalk
providers,
have
this
concept
of
credentials
and
and
provider
configuration
and
usually
there's
some
sort
of
status.
When
the
resource
is
done,
you
don't
have
any
of
those
things
with
cloud
in
it
right,
so
why?
Why
are
we
going
to
make
a
provider
at
all?
A
Why
not?
This
doesn't
have
to
be
a
cross-plane
provider,
though
I
hear
you
saying
this
and
you're
right.
You
know
there.
You
could
just
make
a
custom
controller
that
save
rights
to
the
to
the
cross,
plain
resource,
writes
the
user
data
that
you
want,
or
maybe
it
will
do
something
similar
to
what
we're
going
to
do
with
this
provider.
Maybe
it'll
write
out
a
config
map,
then
we
can
read
in
that
config
map.
A
A
Another
reason
is
that
crossplane
providers
give
you
composability
now
you
could
say
well
anything's
composable,
but
at
least
in
the
past.
I
think
it's
still
the
case
today.
There
are
limitations
on
what
you
can
use
in
a
composition
and
those
things
that
you
use
in
a
composition
have
to
be
cross.
Plane
package
managed
resources.
A
So
if
we
want
to
compose
a
combination
of
a
device
with
a
cloud
in
it,
resource
they're
going
to
have
to
both
be
cross-plane,
managed
resources-
and
you
know
the
the
future
is
so
far
away
that
that
may
not
be
the
case
anymore.
I'll.
Take
the
comments
so
in
the
future,
we
might
also
want
to
make
these
different
providers
more
tied
together
and
crossplane
has
a
thing
called
referencers
which
allows
you
to
grab
one
field
from
one
resource
and
bring
it
into
another
resource.
A
Why
do
we
need
a
crossplan
provider?
We
have
xrd's
which
allow
you
to
make
these
compositions
that
make
it
so
that
you
don't
need
to
make
a
controller.
You
don't
need
to
make
a
cross
plane
provider
for
every
possible
thing.
You
could
imagine
just
create
a
composition
for
it.
Well,
I
would
really
like
to
see
that
be
the
option
here
and
the
way
out,
I
couldn't
figure
out
how
to
make
that
work.
Today
I
did
find
a.
A
I
was
referred
to
an
open
issue
that
deals
with
making
available
more
functions
that
x,
that
compositions
could
take
advantage
of
the
kind
of
functions
that
we
would
need
to
handle
cloud
in.
It
would
be
first
off
reading,
config
maps,
multiple
config
maps
or
secrets
and
then
concatenating
them
all
and
then
base64
encoding
that
data
and
gzip
compressing
it
and
then
base64
encoding
that
as
part
of
a
multi-part
mime
document.
So
it's
not
a
lot
of
complexity
there.
A
These
aren't
complex
functions,
but
we
don't
have
a
way
to
run
these
functions
right
now,
so
we
can't
do
it.
So
what
does
the
syntax
for
this
look
like?
Well,
we've
got
our
typical
preamble
of
api
version
in
kind.
We've
got
our
metadata
that
I've
omitted
here
with
a
name,
no
namespace,
because
this
is
a
cluster
scope
resource
as
all
providers
are.
A
A
So
there
is
the
definition
of
what
config
map
we
will
write
out
to,
and
can
I
just
call
that
cloudnet?
We
have
to
give
it
a
namespace
again,
because
this
is
not
a
namespace
resource
being
a
provider
resource
a
managed
provider
resource.
They
are
cluster
scope
today,
so
our
four
provider
field
again,
this
is
a
field
that
doesn't
really
make
sense
in
this
provider.
A
So
maybe
this
level
will
go
away
in
the
future,
but
the
arguments
that
are
available
here
are
a
boundary
we're
just
going
to
go
with
the
default
of
mind
boundary
and
the
only
reason
you
would
change.
That
is.
If
your
text
contained
the
word
mind
boundary
in
it
somewhere,
then
you
would
have
to
set
a
unique
boundary
part.
So
there's
multiple
parts.
This
is
actually
out
of
date.
This
should
be
parts
there's
a
config
map,
key
ref.
A
Okay
and
then
the
other
file-
that's
in
here
is
a
provider
config
interesting
thing
about
the
provider.
Config
is
that
it
won't
actually
load
right
now,
there's
no
spec,
because
there's
no
spec
to
provide-
and
we
don't
actually
need
the
provider
config,
because
this
config
this
cloud,
init
config
resource,
just,
doesn't
load
a
provider
config,
it
doesn't
have
a
provider
reference,
it
doesn't
need
one.
A
A
A
And
then
we
are
going
to
apply
the
cloudinet
resource
which
combines
the
food
resource
with
two
other
string
cloud,
config
parts
cloud,
init
parts
and
now
that's
reconciled.
So
let's
show
what
has
happened
here.
Let's
do
another
get,
and
this
time
we
can
see
that
we
have
both
the
the
cloud
resource
we've
created.
It
created
a
config
map
called
cloud
init.
We
have
our
foo
config
map
and
we
have
our
coordinate
config
map.
A
So
let's
take
a
look
at
that
cloud:
config
init
map
cloud
and
knit
config
map,
and
here
we
see
that
it
has
all
of
our
multi-part
mime
contents.
It
has
the
empty
shell
script
here.
It
has
the
hello
world
shell
script
here,
and
it
has
the
cloud
config,
which
is
all
of
this
business
here
that
gets
my
ssh
keys
in
place
all
right.
So
now
we
have
a
config
map
with
a
cloud
init
key
ready
to
be
used
by
any
cross
plane
managed
resource-
that
has
say
a
user
data
field
that
could
benefit
from
this.
A
So
now
that
we've
seen
the
demo,
I'm
sure
there
are
some
alternatives
that
you're
considering
like.
Why
did
you
bother
with
that?
I'm
sure
there's
better
ways
to
do
it
and
yeah
there
probably
are.
So
what
could
we
have
done?
We
could
have
maybe
gone
the
other
way
and
told
our
told
our
managed
resource
the
the
equinix
metal
device,
how
to
read
multiple
config
maps
as
user
data.
We
could
have
told
it
that
if
it's
going
to
do
that,
then
it
should
also
mime
encoded,
base64
encoded,
and
that
is
that's
probably
fair.
A
A
Another
reason
for
doing
so
is:
do
we
really
want
to
bake
all
of
these
things
into
each
provider?
No,
maybe
we
could
bake
all
of
these
things
bake
the
appending,
the
base64
encoding,
the
g-zipping.
Maybe
we
could
make
that
into
cross-plane
run
time
as
a
new
field
type.
That's
understood
like
it
knows
how
to
take
secrets
as
parameters.
There's
some
type
definitions
for
that.
So
maybe
it
knows
how
to
take
a
list
of
config
maps
and
do
this
thing
that
a
handful
of
providers
could
benefit
from.
A
Maybe
maybe
not
another
way
that
this
could
be
implemented
is
maybe
with
just
a
a
selector
field,
so
we'd,
say
config
map
selector
and
provide
some
labels
to
look
for
to
search
available,
config
maps
for
we'll
just
take
all
the
config
maps
that
match
that
selector
and
we
will
do
the
base
64
encoding,
we'll
do
the
mine
multiparting
we'll
do
the
gzipping
all
at
so
there
are
other
ways
that
we
could
do
this
and
in
the
future.
You
know
another
reason
why
I
think
that
this
is
better
to
have
as
an
independent
provider.
A
Now.
This
is
an
experience,
that's
similar
to
what
you
may
be
familiar
with
if
you've
used
terraforms
providers
for
cloudnit
or
terraforms
providers
for
templates,
so
they
have
an
example
right
on
one
of
these
pages,
where
they
show
you
having
a
template
file
applying
some
variables
to
it,
combining
it
all
into
one
cloud,
init
data
that
the
terraform
provider
knows
how
to
to
how
to
render
it
knows
how
to
take
multiple
parts.
A
It
knows
how
to
take
the
options
of
base64
encoding,
mime
encoding,
taking
even
a
mime
boundary
field,
so
that
in
case
you
need
a
custom
one
for
some
reason,
and
so
terraform
has
all
of
those
things
already.
That
was
one
of
the
examples
that
we
used
in
defining
this.
This
concept
of
a
provider
well,
crossplane
knows
how
to
ingest
terraform
providers.
So
why
didn't
we
just
make
a
cross
plane
provider
that
ingests
this
terraform
provider
and
get
this
for
free?
A
A
A
One
more
way
that
we
could
have
approached
this
is
to
use
the
composition
to
construct
our
user
data
and
that
that's
an
area
that
might
be
possible
now
might
be
possible
soon,
but
effectively.
We
would
need
a
composition
that
knows
how
to
read
in
a
config
map
that
may
or
may
not
exist
yet
take
the
data
from
that
config
map
and
drop
it
into
the
user
data
field
of
some
other
managed
resource.
A
A
It's
interesting
to
me
that
the
terraform
cloud
init
provider,
it
has
been
deprecated
in
favor
of
the
cloud
init
config
data
source
and
the
template
file
provider
is
seemingly
deprecated
in
favor
of
a
template
file
function,
which
is
an
hcl
level
function.
So
it
would
be
interesting
to
see
crossplane
develop
a
similar
function
to
serve
a
similar
function,
since
we
don't
have
paths
to
load
in
well,
I
suppose
we
do
have
pads
you.
Can
you
have
paths
that
are
volume
mounts
on
pods,
but
we're
not
dealing
with
that
with
cross
plane.
A
We
are
dealing
with
various
crd
cr
instances
there
there's
no
pod
here
to
for
a
path
to
be
relevant,
so
the
kind
of
path
thing
that
is
available
to
us
is
config
maps
and
secrets,
and
they
live
in
different
name
spaces
and
they
have
different
names
and
they
have
different
keys.
So
perhaps
some
way
that
cross
plains
function,
library
or
the
composition
library
gives
you
access
to
managing
mangling,
reading,
config
maps
and
secrets,
taking
those
variables
and
applying
additional
functions
on
them
and
then
putting
them
back
somewhere.
A
A
There
might
be
some
blurriness
here.
I
think
that
the
helm
provider
has
the
capability
of
connecting
to
a
remote
cluster,
but
a
lot
of
the
functions
that
it
had
made
available
to
it
really
suited
what
I
was
trying
to
do.
There
were
functions
in
there
to
read
in
config
map
and
secrets,
and
that's
something
that
I
needed
to
do
here.
A
One
of
the
big
differences,
though,
is
that
again
there's
no
api.
This
is
this
cloud
provider
provider
has
there's
no
api,
calls
I'm
considering
making
some
some
pretty
drastic.
I
think
compared
to
the
the
normal
provider
changes
to
this
ripping
out
the
provider
config,
because
we
don't
need
to
configure
a
provider
ripping
out
the
at
provider
and
the
four
provider
components
of
the
spec
and
status.
A
A
They
didn't
have
credentials,
they
didn't
have
this
ad
provider
for
provider
at
provider
and
for
provider
fields,
and
you
can
tell
that
that
was
part
of
the
thinking,
because
of
the
way
that
the
managed
resources
in
crossplane
runtime
are
named
they're
named
after
external
provider
external.
This
there's
no
external
client
needed
here.
So
have
we
lost
sight
of
what
it
looks
like
to
make
a
provider
that
doesn't
interact
with
an
api?
A
And
how
can
we
improve
that
experience?
Do
we
want
to
improve
that
experience
taking
terraform
for
an
example,
terraform
does
have
a
handful
of
these
providers
that
just
operate
without
an
api,
and
perhaps
you
know
we
need
more
of
those
in
cross
play.
Then
again
we
have
compositions
that
are
allowing
us
to
skirt
some
of
those
responsibilities.
As
long
as
we
can
take
advantage
of
functions
and
compositions,
we
might
be
able
to
avoid
the
need
for
api
less
providers
in
cross
plane.
A
I
understand
how
janky
this
must
appear
to
many
of
you,
and
so
I'm
willing
to
accept
questions
and
negative
feedback
on
the
github
provider.
Repo
display
slash
in
it
at
the
time
of
recording
this
in
the
future.
Who
knows
where
it
might
live?
Maybe
it'll
live
in
cross,
plain
contribs
and
if
you
want
to
get
a
hold
of
me
outside
of
now,
the
twitter
is
one
good
way
and
you
can
find
my
work
on
github
at
display.