►
Description
Zero Configuration Pattern of Kubernetes on Bare Metal [A] - Rob Hirschfeld, RackN
In recent releases, we've enabled node admission and configuration APIs that eliminate configuration requirements for Kubernetes workers. This allows cluster operators to add and remove nodes from clusters without a configuration management tool driving the process. This fully automate node management behavior allows physical data centers to be much more cloud-like and lights-out.
In this session, we'll run this process as a demo and decompose the various parts that must work together for success. We'll discuss the specific APIs and how to implement them in a coordinated way that ensures node security and minimizes workload disruption. We'll also discuss how to improve node security by using trusted platform modules (TPM). By the end of the session, operators will be able to duplicate the steps on their own to learn the process.
While we have a focus on bare metal infrastructure for this session, the lessons learned are equally usable on cloud infrastructure.
About Rob Hirschfeld
Rob Hirschfeld is CEO and co-founder of RackN. He co-chairs the ClusterOps SIG and served four years on the OpenStack Board. With over 15+ years of cloud and physical infrastructure automation experience, he brings a unique technology and process perspective to DevOps and SRE fields. He is also the co-founder of the open Digital Rebar project which focuses on underlay automation for physical infrastructure.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
So
I'll
let
the
room
feel
a
little
bit
more.
How
many
of
you
were
here
for
the
last
demo,
the
last
meeting
very
few
okay
good?
We
just
find
that
Google
was
just
presenting
hybrid
control
planes
and
in
some
ways
that's
the
the
peanut
butter
to
this
chocolate.
But
we
can
we
can
get
into
that
in
QA
if
we
have
time
and
I
refuse
to
do.
Kubb
control,
I'm
gonna,
be
a
good
cuddler
until
I
die.
Sorry
great,
so
I
apologize
for
that
in
advance.
A
Okay,
hello,
my
name
is
Rob
Hirschfeld
I
am
CEO
and
co-founder
of
a
company
called
rockin
I've
been
involved
in
the
kubernetes
community
since
the
very
very
early
days,
the
Oz
con
announcement
and
today
we're
going
to
talk
about
some
really
exciting
new
stuff
rack
and
focuses
on
operations
specifically
on-premises
physical
metal
operations.
Right,
that's
our
area.
We
do
a
lot
of
other
things
too,
but
that's
really
our
focus
area.
A
Today
we
want
to
talk
about
something
that
we
think
is
really
exciting,
which
is
immutable,
bootstrapping
and
then
we're
going
to
talk
about
some
other
places
where
operations
and
kubernetes
meet
in
interesting
ways,
which
is
noted,
mission
and
dynamic,
couplet
configuration
if
you
haven't
heard
of
those
things
we'll
get
there
if
you're
interested
in
community
side
I'm
involved
in
cluster
ops
sig.
So
we
like
to
try
and
roundup
operators.
A
Please
get
involved
in
that
and
then
a
lot
of
what
I'm
going
to
show
you
we're
gonna
we're
gonna
with
something
called
digital
rebar,
which
is
an
open
source
project
that
rack
and
maintains
for
doing
physical
provisioning.
So
the
bare
metal
underlay
side
of
this
this
whole
equation
and
before
I,
get
into
the
immutable
boot.
It's
it's
worth
doing
a
hat
tip
to
Kubb
spray.
A
We've
done
a
lot
of
work,
building
tube
spray
deployments
since
version
1.2
back
when
it
was
coupe
sprayed
the
first
time
and
then
cargo
and
then
back
to
coupe
spray,
and
it's
a
really
solid,
ansible
playbook.
It
has
great
features
like
a
CH,
a
and
plugins,
and
things
like
that.
So
I'm
we're
not
trying
to
endorse
one
installer
over
another.
We
actually
think
if
you
like
ansible
coupes,
raise
a
really
good
job,
but
this
this
presentation
has
a
lot
of
these
pregnant
butts
in
it.
A
We
wanted
to
notes
just
to
come
up
boot
and
be
discovered
that
would
be
really
nice
without
having
SSH
involved.
So
if
you're
doing
a
mutable
infrastructure
I'm
going
to
pause
and
not
assume
the
room
knows
what
I
mean
by
a
mutable
infrastructure
like
shell
defined
it
I
think
of
the
next
slide,
the
mutable
infrastructure,
and
then
we
just
think
it
should
be
much
faster.
Coop's
praise
is
really
cool,
but
it
takes
20
minutes
or
so
to
run,
we'd
like
to
get
that
down
in
the
sub.
A
Five
minutes
range
fast
enough
for
me
to
demo
in
a
presentation
window.
So
what
is
the
mutable
infrastructure?
How
many
people
know
what
a
mutable
infrastructure?
That
term
means
something
to
them?
Okay,
how
many
people
think
they
agree
with
my
definition
of
a
mutable
mutability?
That's
a
dark,
I
know:
I
know
you
don't
that's
a
lie,
so
immutability
has
has
different
meanings
for
different
people.
To
me,
the
fundamental
architectural
piece
of
it
is
this:
create
destroy,
repeat
pattern.
A
We
work
on
doing
that
on
physical
infrastructure,
so
we're
literally
turning
over
nodes
very
quickly.
I'll
show
you
two
ways
that
we
do
that
and
the
reason
we
want
to
do
this
pattern.
Is
it's
one.
Yeah
two
is
the
three
is
faster,
but
it's
a
very
repeatable
and
predictable
pattern.
So
that
means
I
can
say.
This
is
the
thing
I
want
to
deploy
and
you
deploy
it.
A
Configuration
management
tools
are
really
good,
but
they
can
be
finicky
or
especially
over
time,
configuration
drift
happens
and
it's
very
hard
to
maintain
nodes,
and
that
makes
everything
simpler
and
so
in
operations.
If
we
can
increase
simplicity
and
speed,
that's
a
big
win
and
that's
why
we
think
immutability
is
a
very
big
deal.
There's
a
side
effect
in
here
that
that
we
think
is
really
important,
which
is
people
know
how
to
do
it
in
the
cloud.
A
So
here's
what
we
want
to
show
this
is
the
zero
configuration.
What
we
want
to
be
able
to
do
is
do
a
three-step
cluster
initialization,
and
this
is
normal,
qu
Badman,
if
you,
if
you
haven't
done
it,
we
just
don't
want
to
have
to
touch
anything
to
make
this
happen.
So
we
want
to
initialize
the
master.
We
need
to
retrieve
that
token
from
the
initialization
and
then
we
need
to
join
nodes
with
that
token.
So
it's
this
is
normal.
A
If
you've
done
it,
you
log
into
one
node,
you
do
things
you
log
into
the
other
nodes
with
information.
We
want
to
eliminate
that
step
because
that
still
requires
some
coordination
and
synchronization
manually.
If
we
get
rid
of
that,
then
clusters
are
going
to
come
up
automatically.
If
you
look,
if
you
were
in
the
last
demo,
if
somebody
is
hosting
your
control
plan
already,
then
you
can
do
basically
the
skip
the
initialization
step
and
use
a
remote
control
plane.
A
A
Potentially
we
could
do
everything
in
memory,
but
it
makes
sense
to
have
some
persistence
for
the
docker
volumes
to
go
somewhere
instead
of
RAM,
and
then
you
have
to
have
docker
installed
right.
All
of
these.
These
are
your
prereqs
is
pretty
simple,
but
they
need
to
get
done
and
then
to
make
the
actual
bootstrapping
go.
You
need
to
have
some
control
mechanism
right,
so
it's
ansible
or
cloud
in
it
or
terraform,
or
what
we've
been
able
to
do
is
eliminate
those
those
pieces
and
just
use
the
boot
provision
infrastructure.
A
We
build
with
digital
rebar
and
it
looks
like
this,
so
we
have
four
nodes
in
the
system.
We
have
to
prep
them,
so
we
have
to
install
an
operating
system
and
install
docker
pretty
straightforward,
and
then
we
need
to
let
the
admin
node,
the
master
node
come
up,
so
it
has
to
be
a
Kubb
kuben
it
and
create
a
cluster
token
right.
So
that
token
then
is
going
to
be
used
to
join
the
other
system.
A
So
once
that
tokens
available,
all
the
other
systems
are
going
to
come
in
and
join
it,
and
then
at
that
point
we
can
just
reply.
Rinse
repeat:
we
can
grow
the
cluster
we
could
drain
and
remove
nodes
from
the
cluster.
Things
become
pretty
straightforward
at
that
point.
So
this
is
this
is
the
basic
bootstrapping
process?
What
we
want
to
be
able
to
do
what
we
want
to
be
able
to
do
there,
I'm
gonna,
hold
on
that.
What
we
want
to
be
able
to
do
is
synchronize
that
process.
A
So
we
want
the
notes
to
come
up
just
out
of
it
as
a
group.
The
first
one
up
should
be
able
to
grab
being
master,
create
the
token
and
then
let
the
other
notes
go
and
that's
what
we'll
show
you
in
the
demo
and
that'll
be
able
to
bootstrap
the
cluster
completely
by
itself,
but
I
do
want
to
before
I
demo.
I
do
want
to
give
a
PSA.
This
is
not
a
new
installer.
A
A
It's
it's
a
it's
just
using
the
tools,
the
way
they
are
just
like
when
we
use
coop
spray,
we
just
use
coop
spray
and
that's
our
goal
is
to
be
able
to
use
the
community
tools
the
way
the
community
tools
are
designed
to
be
used
all
right,
so
the
demo
all
right.
So
this
is
a
coop
admin
rebar,
which
is
digital,
rebar
and
immutable
bootstrapping
for
the
demo.
I'll
show
you
what
these
things
look
like.
This
is
the
the
overview
system.
A
So
this
is
a
digital
rebar
consoles,
where
we're
going
to
start
and
in
this
system,
I
have
put
eight
nodes,
because
we're
gonna
do
two
different
deployments
that
are
completely
independent,
and
my
job
here
is
not
to
show
you
digital
rebar.
If
you
want
to
do
that,
go
to
the
community
site
and
you
can
you
can
duplicate
all
these
steps
there.
What
I
want
to
do
is
focus
on
the
the
actual
bring
up,
and
so
for
that,
what
we've
done?
Can
people
see
this
in
the
back?
Okay,
I
can
that's
why
I
asked
there.
A
A
What
what
I've
done
here
is:
we've
created
two
clusters
in
advance
for,
for
the
demo,
one
is
a
ram
cluster
and
one
is
a
kubernetes
install
cluster,
and
so
in
this
cluster.
It
basically
just
tells
you
what
the
what
it
is:
a
kubernetes
cluster
or
crib,
my
naming,
not
everybody.
My
team
loves
that
it
works
for
me
and
I've
created
a
another
cluster
install
that
will
actually
do
it.
One
of
them
it's
going
to
use
our
in
memory
fast,
booting
infrastructure,
the
other
one
was
actually
gonna.
A
Do
a
full,
regular
CentOS
install
in
this
deployment
and
then
what
we
have
to
do
is
we
have
to
assign
these
these
profiles.
So
in
this
case,
my
first
four
nodes
I've
already
put
into
this
Ram
based
cluster
deployment,
so
they're
already
booted
they're
waiting
they're
available
for
me
and
what
I?
What
I
need
to
do
is
go
ahead
and
just
tell
it
to
start.
The
install
process
outweigh
ones
thing
to
show
you
first
and
then
somebody
can
start
a
clock
on
the
demo.
A
A
Do
the
crib
install
step
three
and
while
that
works,
I'll
show
you
a
little
bit
about
the
actual
scripts
that
are
going
on
behind
the
scenes
and
then
we're
done
we're
just
going
to
wait
and
then
I'll
run
kubernetes
for
you
so
to
launch
that
script.
I
just
have
to
start
the
workflow
to
the
mount
local
disks
phase,
which
I
can
do
for
are
those
four
nodes
pretty
easily
nodes
and
I
just
need
to
go
to
mount
local
disks.
So
this
is
the
beginning
of
that
workflow.
A
So
I've
applied
that
workflow,
you
can
see.
The
stages
are
mounted
here.
I'm
gonna
get
some
live
events
back
as
the
system
goes
through
that
process
and
that'll
that'll.
Let
us
see
what's
going
on
with
these
nodes,
so
the
first
thing
that's
going
to
happen
for
these
nodes.
Is
that
they're
going
to
start
going
through
the
process
of
building
the
actual
infrastructure
I'm?
Looking
at.
A
They're
gonna
go
through
the
actual
infrastructure
phase
and
start
running
running
the
the
install
process
for
the
disks,
and
then
we
should
be
able
to
start
moving
things
all
the
way
through
so
we're.
While
we
wait
for
that
to
kick
off
what
I'm
going
to
do
is
I'm
going
to
take
the
other
four
nodes
that
I
have
going
and
I'm
going
to
add
them
into
the
other
cluster
that
I
have
ready
to
go
and
for
them,
what
we
want
to
do
is
I
need
to
put
them
in
this
case,
install
cluster.
A
A
A
We're
gonna
come
back
and,
as
part
of
these
two
I'll
take
you
through
what
the
what
the
scripts
look
like.
That'll
help
explain
how
we're
using
cube
admin
for
this.
What
will
happen
is
is
that
as
the
cluster
comes
up,
the
first
node,
that's
elected,
is
going
to
identify
itself
as
the
master
and
will
inject
I
am
the
master
into
the
into
this
profile
and
that'll
become
a
shared
resource
for
all
the
other
clusters.
So
that's
how
we
coordinate
this
operation
of
a
single
node.
All
the
nodes
boots
simultaneously,
one
becomes
elected.
A
That
self
elects
is
the
cluster
right.
We
don't
predefined
that
it's
a
it's
basically
a
race,
one
node
wins
and
then
once
that
node
wins
the
cube
admin
initialization
step
will
pull
back
the
required
token
and
it's
gonna
then
pull
back
join
information
for
all
the
other
nodes
to
participate
in.
So
let
me
show
you
what
that
looks
like
so
inside
of
this
stage.
A
The
crib
install
stage
here
there's
a
single
task
in
it,
and
this
is
something
relatively
new
for
us.
A
lot
of
what
we
typically
do
is.
We
would
start
to
decompose
this
single
tasks
into
multiple
tasks
and
then
those
tasks
become
alerts
and
information
that
get
passed
into
the
system,
so
you
could
actually
track
the
install
progress
of
an
individual
node
and
then
notify
other
systems
of
what
was
going
on.
A
So
it's
going
to
go
in
and
then,
if
I
dig
through
in
here,
I
have
to
install
couplet
I
have
to
come
in
and
actually
run
somewhere
down
in
all
this
code
is
the
actual
Kubb
admin
init
phase
from
there
we're
gonna
grab
the
results
of
Kubb
admin
in
it
all
the
other
workers
are
gonna
back
off
and
then
they're
going
to
keep
going
through
the
infrastructure.
From
that
side,
let
me
see
where,
where
we
stand
with
the
nodes.
A
And
so
literally,
this
node
is
now
installing
docker
and
also,
although
I
can
actually
see
it
from
here.
All
of
these
nodes
are
in
the
docker
install
step.
One
of
these
four
I
still
don't
know
which
yet
will,
when
it
advances
to
the
next
phase
in
this
workflow
will
become
the
master
node
and
it
just
takes
a
little
bit
of
time
to
install
docker.
One
of
the
things
that
we're
doing
in
the
background
here
is
this.
This
demo,
obviously
I
don't
have
a
stack
of
servers
here
doing
the
demo
we're
using
something
called
packet
net.
A
If
you
haven't
tried
them,
we
actually
have
a
coupon
code,
rack
in
capital,
letters
100.
You
can
get
I,
think
a
30
dollar
credit
on
packet
to
try
out
these
same
types
of
things
that
are
a
bare-metal
hosting
company.
So
if
you're
practicing,
you
don't
have
a
lab
or
even
if
you
do
have
a
lab,
if
you
need
spare
spare
metal
servers,
we're
big
fans
of
what
packets
been
doing
and
we
have
a
lot
of
integrations
with
their
workflow.
A
So
in
this
case,
we're
just
waiting
for
the
docker
install
the
finish
and
then
we'll
get
kubernetes
install
happening
right
after
that.
Just
a
minute,
yeah
and
then
part
of
what
we're
able
to
do
in
the
background
to
do
is
actually
see
the
job
progress.
So
this
is
our
docker
install,
install
actually
running
and
then
giving
me
results
of
the
docker
bring
up
process.
A
A
Right
so
in
this
case,
node
fifty-six
one,
the
leader
election,
and
so
that
will
that
node
is
now
doing
the
work
of
doing
the
coup
Badman
in
it.
So
if
you've
installed
cube
admitted
it
that's
the
first
step
that
you're
going
to
do
and
then
the
information
that
gets
initialized
in
that
cluster
will
show
up
in
this
profile
dynamically,
when
we,
when
we
get
a
little
bit
further
in
the
stage,
okay,
so
I'm
right
on
the
edge
I'm
tempted
to
jump
back
to
the
slides,
but
we're
right
on
the
edge
of
doing
it.
Alright.
A
A
A
But
there's
a
little
bit
more
to
do
with
this.
Adding
nodes
requires
a
token
with
cube
admin.
We
could
create
a
long-lived
token,
but
that's
that's
not
as
cool
as
it
would
be
to
actually
inject
the
token
and
bring
it
back.
Our
couplets
still
requires
configuration.
So
if
we
want
to
make
things,
you
know
sort
of
different
or
interesting
or
you
have
site
variation,
we're
gonna
end
up
having
to
inject
configuration
into
the
couplet,
and
then
it
would
be
really
nice
to
be
able
to
orchestrate
this
upgrade
and
run
it
using
some
cluster
API.
A
So
there's
there's
plenty
more
bootstrapping
things
to
do
beyond
where
this
goes.
The
fact
that
I
can
bootstrap
without
having
any
you
know,
external
things
about
the
the
system
is
really
handy
right,
so
I
can
just
bring
up
a
admin
cluster
and
let
it
go
so
let
me
see
where,
where
we
are
with
the
kubernetes
install
at
this
point.
A
So,
in
this
case,
what
you'll
see
is-
and
this
has
been
really
good
because
I've
been
able
to
show
you,
sir,
the
intermediate
steps-
we
started
with
just
the
cluster,
and
now
we
actually
have
the
cluster
the
join
commands
here.
Those
join
commands
are
then
being
passed
in
automatically
to
the
rest
of
the
nodes
in
the
cluster
and
then
what
we
also
have
here
is
we
actually
have
the
full
admin
Kumpf
file
that
we
need
to
run
the
cluster,
and
this
is
where
the
demo
gets
really
fun
to
me.
A
There,
it
is
what
I'm
able
to
do
is
using
the
rebar
CLI
I
can
pull
that
file
out
of
the
configuration
values,
so
in
this
case
I'm
specifying
the
cluster
that
I
want
to
talk
to.
So
this
is
the
kubernetes
cluster,
that's
running
in
rat
Ram
and
then
I'm
just
pulling
out
the
admin
comp
file.
So
at
this
point
that
same
parameter
that
I
showed
you
is
now
in
the
is
now
in
my
local
system,
which
means
I
can
start
running,
and
this
is
on
my
local
system.
A
The
clusters
running
in
a
data
center
in
New
York
I
can
actually
go
in
as
and
use
that
kubernetes
control
admin
file
to
get
the
nodes
that
I
just
deployed
right.
So
in
the
course
of
this,
this
little
demo
I've
actually
managed
to
bring
up
a
full
kubernetes
cluster
I'm,
actually
bringing
up
two
of
them
in
parallel
and
then
what
gets
really
fun
from
there.
So
cou
Brunei's
is
up
what
we
actually
want
to
be
able
to
do,
and
even
more
importantly,
I
didn't
have
to
figure
out
what
my
credentials
were.
A
This
is
this
is
letting
me
into
my
cube,
admitted
admin,
dashboard,
okay,
so
five
minutes
cube
cuddle,
no
typing
no
logging
into
the
machines
no
ssh
required.
I
have
a
cluster
that
I
can
log
into
and
start
using.
It's
a
pretty
big
deal.
I
think
it's
a
big
deal,
but
I'm
excited
alright,
so
that
makes
sense.
That's
the
deaths
bootstrapping
demo,
if
you're
trying
to
deploy
this
type
of
thing
on
site
or,
if
you're,
trying
to
extend
on-premises
infrastructure
from
what
like
google
was
showing
you
with
a
hybrid
cloud,
you
could
eject
the
parameters.
A
A
Back
to
slides
so
there's
two
other
things
that
we
talked.
We
said
we
were
going
to
talk
about,
that.
We've
been
hearing
a
lot
of
interesting
things
for
for
can
we
make
this
faster?
Can
we
make
this
more
secure?
One
of
them
is
noted.
Mission
noted
mission
is
this
is
a
pattern
and
the
in
the
presentation,
if
you're
in
schedule,
you
can
get
a
link
back
to
these
docks
and
the
URLs.
A
Ideally,
what
we
would
get
with
noted
mission
is
that
when
we
booted
nodes
we
wouldn't
need
a
token.
The
node
would
boot
we'd,
say:
hey
I
like
this
node,
let
it
in
and
then
configure
it
that's
what
we
want
to
be
able
to
do
so.
We
don't
have
to
drive
any
configuration
infrastructure
should
make
auto
scaling
easier,
so
node
admission,
it
comes
up,
something
says
yes,
I
have
a
picture,
I'll
show
you
and
that
should
make
it
faster.
A
We
should
be
able
to
use
kubernetes
to
do
everything
right,
sort
of
a
kubernetes
conquers
the
world
paradigm,
and
it
would
look
like
this
using
this,
this
an
admission
service,
so
a
node
would
come
in
couplet
comes
in.
It
tries
to
create
itself
right,
tries
to
create
the
node
that
node
actually
create
node
action.
Using
the
admission
protocols
says:
I
can't
create
the
node
until
I
check
in
with
external
service,
whatever
web
hook
that
web
hook
says.
Yes,
I
trust.
What
this
that
you're,
creating
a
node
that
you're
allowed
to
do.
A
You
could
do
this
with
any
object
in
kubernetes,
create
validates
that
the
node
is
a
valid
node
and
says
yes,
kubernetes,
said
wow
what
a
relief
I
trust
this
node
it
sends
back.
The
it'll
creates
the
object
and
now
couplets
in
the
system.
Right,
that's
that's!
That's
object,
admission,
node
admission
protocol-
and
this
is
a
little
bit
scary.
So
this
is
this
is
the
pattern
that
somebody
described
to
me
and
they
started
to
get
very
excited
about.
A
Hsm
means
a
hardware
signing
module,
so
you
could
cryptographically
sign
the
information
that
you
put
in
that
node
and
you
could
start
the
node
with
a
token
from
a
trusted.
Third
party
sign
that
node
by
the
by
the
hardware
signing
module
on
that
note.
So
it's
unique
to
that
node
and
then
the
encrypted
token
is
what
gets
passed
up.
A
This
is
tricky
to
follow.
It
gets
the
encrypted
token
nautical,
not
the
clear
token.
It's
passed
up
to
the
API
server,
the
API
server
forwards
it
as
part
of
the
node
admission
request.
The
external
party
goes
back
to
the
PKI
to
decrypt
that
token,
using
the
public
key
and
that
that
system
says
wow.
This
matches
the
key
that
I
gave
you.
Nobody
else
knew
about
it
except
me
and
everybody's
happy
in
this
process,
and
it
sounds
really
cool.
The
challenge
is
that
in
bringing
up
the
node
this
way
you
already
have
to
inject
information.
A
That
is,
is
cryptographically,
relevant
or
unique
per
node
it,
the
level
of
extra
effort
to
do
this
really
doesn't
guarantee
you
that
the
node
is
any
more
secure.
So
while
it's
a
really
interesting
pattern,
we
iraq
and
think
that
this
really
doesn't
provide
any
additional
security
from
a
from
a
node
authorization.
Node
guarantee
space.
So
we
think
it's
interesting,
but
you've
already
got
to
be
pretty
sure,
you're
bringing
up
a
note
of
validating
what
the
node
is
by
putting
a
secure
token
on
it
an
admission
token,
so
marginal
marginal
benefits
for
a
lot
of
complexity.
A
The
other,
the
other
pattern
that
looks
like
it
should
be
super
interesting
for
zero
touch.
Configuration
is
to
use
Kubla
dynamic
configuration,
so
you
want
to
be
able
to
say:
hey,
couplet,
here's
a
new
node.
I
trust
it
right.
We
already
went
through
this
process
and
I
don't
know
anything
about
it.
You
tell
me
everything,
I
know
to
do
that,
configuration
and
that
seems
really
cool,
because
kubernetes
has
already
has
to
interact
with
the
node.
A
Why
not
let
it
configure
the
node
and
there's
a
there's,
a
line
of
thinking
along
these
lines,
which
I,
which
I
call
the
bootstrap
fallacy,
which
is
if
I,
have
a
tool.
I
should
have
the
tool
bootstrap
the
tool
we've
made.
We've
made
this
mistake
before
it
doesn't
always
it's
sort
of
tricky
I'm,
hinting
it
where
I'm
going
with
this,
but
you'll
see
it's
really.
It
should
look
like
this.
We
centrally
configure
the
system
we
install
kulit.
A
We
allow
Kubla
to
register
and
kubernetes
configures
the
couplet,
but
it
looks
like
this
in
practice
and
I
hope
we're
going
to
continue
to
see
investment.
This
is
still
an
early
feature,
so
might
get
more
like
slide.
One,
but
right
now
it's
like
slide
two,
you
install
the
couplet,
you
configure
the
couplet
it
registers
to
the
API
server.
You
inject
your
configuration
into
the
the
API
server
and
then
you
reconfigure
the
couplet
to
use
dynamic
configuration,
so
it
gets
configuration
from
the
server.
A
So
you
literally
have
to
configure
configure
reconfigure
configure
which,
which
isn't
really
that
productive,
considering
you
still
have
to
configure
this
is
the
bootstrap
bootstrapping
problem.
So
if
I'm
injecting
configuration
into
the
system
I
might
as
well
just
inject
all
the
configuration
into
the
system
and
do
it
once
and
so
from
our
perspective.
A
Since
we
already
have
to
inject
configuration
and
we
already
have
an
external
system-
that's
managing
that
configuration
or
at
least
aware
of
that
configuration
we're
not
really
saving
anything
by
taking
50%
or
75%
of
a
configuration
template
and
putting
it
somewhere
else
matter
of
fact,
we
might
actually
be
making
things
harder
and
we
start
having
multiple
sources
of
truth
and
things
like
that.
So
it's
interesting,
but
we
really
don't
see
that
we've
we've
accomplished
as
much
because
we
haven't
eliminated
the
configuration
stages
for
the
node.
A
So
it's
only
to
keep
on
your
radar
from
that
perspective,
but
overall
we're
really
making
great
progress
right.
The
fact
that
we're
able
to
use
coop
admin
to
bootstrap
a
cluster
this
easily
to
be
able
to
just
let
this
sort
of
three-step
process
automatically
configure
a
cluster
is
really
big
deal,
usability,
usability,
infrastructure
and,
and
it's
it's
the
right
direction
right
so
off
to
improve.
But
this
is
this
is
a
real
step
forward
in
how
we're
going
to
build
and
manage
clusters.
A
A
There
is
a
and
we
really
didn't
I
tried
to
not
talk
about
rebar
too
much.
Basically,
there's
a
Content
package
for
did
called
for
for
that
for
crib
and
the
content
package
is
available
in
the
catalog
for
digital
rebar,
so
I
don't
know
which
repo
we
ended
up,
putting
it
in,
but
yeah
it's
it's
totally.
You
can.
You
can
repeat
this
whole
infrastructure
we're
doing
a
longer
demo
of
this
with
a
longer
discussion
on
Thursday
as
a
webinar,
so
you're
happy
to
participate.
But
this
is
the
type
of
thing
we
really
do
want
to
see.
A
A
Great
question,
so
the
question
is
what
was
the
state
of
the
the
sledgehammer
machines
before
they
were
pixie
booted
into
an
in-memory
image
which
is
based
on
CentOS,
so
we
have
optimized,
in-memory,
boot,
environment
and
so
yeah.
So
it's
a
it
literally
the
default
process.
The
default
workflows
here,
workflow
so
in
this
workflow
discover,
is
a
pixie
boot
and
then
it
literally
just
pinks.
It
adds
SSH
keys
and
inventories
and
then
it
waits.
So
that's
the
minimal
state.
It
takes
it's
pretty.
It's
an
optimized
boot
process.
A
So
the
question
is:
what
do
we
think
a
linux
kit?
We
did
a
demo
with
linux
kit
awhile
ago
it
was
pretty
nice.
We
were
able
to
make
inject
some
images
and
make
mix,
links
kick
go
and
it's
still
floating
around
in
the
community.
I
think
somewhere
were
we're
just
not
seeing
a
lot
of
uptake
on
it
and
so
in
the
community.
It's
one
of
those
things
we'd
be
happy
to
see
more
people
trying
to
use
it
saying
the
same
is
true
with
core
OS
right.
A
What
we're
doing
it's
generic,
so
you
could
easily
be
running
core
OS
through
some
of
these
processes.
Although
core
us
is
it's
a
little
bit
different,
some
of
what
we
do
relies
on
our
post
provisioning
agent,
which
is
it's.
It
gets
dissolvable,
so
it
doesn't
stay
in
the
system
that
our
our
post
provisioning
agent
will
pull
down
work
out
of
the
system.
So
it's
a
question
of
how
you
inject
that
post
provisioning
step
into
those
immutable
images.
A
So
so
ignition
is
actually
built
into
the
or
components
I'm
looking
at
greggy's,
so
we're
actually
adding
ignition
into
the
post
config
into
the
sledgehammer
post
config.
So
you
could
actually
run
ignition
scripts
in
this
in
memory
boot
state
and
take
and
then
drive
directly
to
disk.
So
that's
that
we're
really
excited
about
all
that
all
those
things
we
think
it's
the
right
pattern
and
you
had
a
question.
A
So
the
question
is:
can
you
accelerate
this
by
by
baking
more
things
into
the
image?
Yes
hell?
Yes,
the
what
we
did-
and
this
is
a
very
deliberate
decision
sledgehammer,
which
is
our
sort
of
quick
start
boot
discovery
thing
is,
we
is
really
optimized
so
at
we
didn't
want
to
add
this
into
sledgehammer,
but
you
could
create-
and
this
was
a
subject
of
some
debate
before
I
was
just
told.
A
No,
we
the
way
things
work,
but
we
we
almost
made
a
case
ledge
hammer
or
a
kludge
hammer
or
whatever
google
hammer,
something
that
would
have
had
these
things
in
it.
But
the
reality
is
it's
so
fast
to
pull
these
images.
It's
really
not
that
much
overhead
for
docker
and
cube.
But
if
you
had
a
whole
bunch
of
other
pieces,
then
I
would
totally
do
that
makes
sense,
yeah
more
questions
just
about
out
of
time.
So
we're
happy
to
take
you
no
more
questions
up
here.