►
From YouTube: TGI Kubernetes 077: All your certificates have expired
Description
Tune in this week as Duffie works through how to rotate all the certificates in a Kubernetes Cluster. He will also describe what they are for and some of the characteristics of each one.
A
Debbie
so
I'll
be
hosting
this
one
today,
and
my
goal
today
is
actually
to
talk
about
what
happens
to
your
cluster.
Wouldn't
hold
your
certs
expire
and
there
is
a
lot
to
this
subject.
So
I'm
really
looking
forward
to
digging
in
this
week's
hack
MD
is
actually
going
to
be
put
in
the
channel
if
it
hasn't
already,
I
will
actually
pull
open.
The
notes
here
we
got
Timmy
checking
in
we
got
Joey
Tucker
from
Richmond
Virginia,
hello,
joy,
Martin
from
the
Netherlands
Christopher
from
Germany
Michael
from
Chile
got
a
van
or
from
Tunisia.
A
That's
awesome,
how's
it
going.
We
have
celebrating
fork
who
really
deserved
a
certificates
this
month,
OpenShift
admin
after
holding
CK
ack
ad
and
go
evan
or
that's
awesome.
That's
a
lot
of
work.
We
have
marcos
coming
or
marco
or
marcos
for
the
two
from
boulder
colorado
that'll.
Actually
I
have
a
lot
of
friends
in
boulder
colorado,
we'll
get
a
bunch
of
people
in
Colorado
as
part
of
our
team,
so
some
other
co-workers,
my
buddy
Steve
and
Ryan
here,
they're
actually
part
of
my
team.
Here
at
VMware
we
have
a
Robert
coming
from
Dublin.
A
That's
awesome,
Norman
from
London
I'm
actually
about
to
go.
Take
a
two
and
a
half
weeks,
vacation
to
London
and
I'm
very
much
looking
forward
to
it,
and
so
I'm
super
excited
about
that.
We
have
Marco
from
Milan
Italy,
well,
Chris,
it's
Chris,
hi
Chris
we
have
Lee
from
Tampa
and
for
Visio
I
was
actually
just
found
out
some
very
exciting
news
about
Fabrizio,
so
I'm
super
jazzed,
but
the
reason
is
actually
one
of
the
active
contributors
in
the
upstream
in
the
upstream
sync
cluster
life
cycle
stuff
and
we
also
have
live
Amir.
A
The
same
also
super
awesome
and
I'm
just
I'm
blessed
that
I
get
to
work
with
such
awesome
people,
so
I'm
very
glad
you
all
are
here.
These
two
have
actually
been
instrumental
in
some
of
the
work
that
I'm
actually
really
showing
off
in
this.
In
this
episode,
what
we're
gonna
dig
into
kind
of
like
rotation
and
how
that
works?
There's
actually
a
lot
of
really
exciting
stuff
that
is
coming
in
115
and
we're
gonna
kind
of
play
with
that
a
little
bit
as
part
of
this
whole
episode
so
yeah.
A
How
can
be
notes
are
up
and
let's
get
into
the
episode
here-
just
greeted
face
all
right,
so
here's
our
hack
and
B
for
this
week.
This
is
a
nod
toward
the
oh
all.
Your
should,
because
I
have
expired,
is
kind
of
a
nod
toward
Pokemon
I,
don't
know
if
y'all
play,
but
that
was
definitely
kind
of
where
I
was
headed
with
the
with
the
comment
and
we
have
cat
hi
cat
and
we
have
an
in
our
a
candy
this
week.
A
We
have
some
news,
you
know,
so
we
actually
just
released
114
and
won
13-6
those
words.
Those
were
released
just
this
last
week
and
don't
use
them
because
they
have
most
security
regression
in
them.
The
security
regression
is
actually
pretty
interesting.
Let's
pop
that
up
and
take
a
look
at
it.
So
it
took
me
it
took
me
a
couple
reads
to
understand
what
was
actually
happening
here
and
it's
in
this
regression.
A
First
of
all,
you
are
awesome
because
very
few
people
are
doing
that.
But
if
you
are
I
salute
you
and
I
think
that
more
people
should
do
that.
But
if
you
are
doing
that
and
you
deploy
your
application
to
kubernetes
the
first
time
it
starts
up,
it
will
work
as
you
expect
you'll
be
running
in
that
particular
user.
A
So
the
second
time
the
container
starts,
it
will
actually
come
up
and
be
initialized
as
you
in
the
in
the
root
context,
not
in
the
user
context,
which
is
kind
of
a
tricky
thing
to
find
I'm
really
impressed
with
some
sounds
so
quickly
because,
as
I
said
114
to
only
came
out
just
a
couple
days
ago-
and
you
know
I
guess
possibly
it's
related
to
the
community
just
being
so
on
top
of
it.
But
you
know,
like
somebody,
noticed
this
and
filed
it,
and
we
were
able
to
get
in
front
of
it
pretty
quickly.
A
You
can
potentially
downgrade
to
114
one
if
you
have
that
problem.
If
that
problem
is
actually
happening
inside
of
your
cluster
or
if
you
determine
that
you
are
about
viable
to
it,
understand
that,
like
the
downgrade
means
actually
just
changing
the
qat,
the
the
cubelet
binary
on
all
of
the
nodes,
you
don't
have
to
downgrade
the
database.
You
don't
have
to
downgrade
like
the
control
plane.
You
can
leave
the
API
server
controller
manager
scheduled
all
those
things
at
114.
You
just
need
to
basically
replace
the
cubed
binary
and
restart
that
cubed
binary.
A
So
it's
such
that
the
new
cubed
binary
is
used,
and
that
is
one
way
to
downgrade
all
of
the
things.
No
different
distributions
tend
to
list
differently
and
I'm
sure
you're
gonna,
probably
if
you're
using
the
distribution
of
communities
you're
going
to
see
some
really
interesting
for
you
matter
about
how
to
manage
that.
But
at
the
end
of
the
day
it's
released
about
the
cubelet
that's
running
on
all
those
nodes.
It
needs
to
be
a
non
vulnerable
version
for
you
to
actually
mitigate
this
particular
see
it
thumb,
TV,
really
interesting.
A
Stuff
I
had
a
super
impressive
that
actually
got
picked
up
so
quickly.
The
next
thing
I
put
a
link
in
the
hack
MD.
This
is
going
to
be
a
Oh
trove
of
videos,
I'm
going
to
click
through
to
it
real
quick.
There
is
a
scatter
of
content
and
I
was
actually
I
mean
shout
out
to
the
team
at
Q
con
Barcelona
this
this
year,
because
they
were
actually
posting
the
same
day.
In
some
cases
like
so
talk
would
end,
they
would
compile
the
video
and
kick
it
out.
A
Like
I
mean
they
were
there
with
adding
to
that
playlist
live
like
day
by
day,
which
was
great,
so
there
wasn't
really
a
lot
a
lot
only
time
to
actually
get
a
hold
of
those
things
which
I
thought
was
incredible.
I
have
a
number
of
friends
were
there
to
actually
speak
in
person,
I
ended
up
being
in
Australia
for
a
week
and
a
half
helping
kind
of
enable
some
of
our
other
folks
that
are
actually
in
the
in
Australia
I
had
a
great
time.
A
Australia
is
awesome,
but
but
I
you
know
how
it's
definitely
experiencing
the
FOMO
for
for
Kim
con
cube
Economist.
Actually
it
sounds
like
it
was
a
great
time.
The
hallway
track,
of
course,
always
the
best,
but,
as
I
said,
I
have
a
number
of
friends
who
actually
got
some
talks
in
including
handing
Jacob's
putting
together
a
pretty
decent
community
list
of
great
talks,
and
this
talk,
which
should
be
near
and
dear
to
all
of
your
hearts.
This
talk
is
actually
not
going
to
play
it.
A
This
is
a
talk
that
was
put
out
by
Joe
at
cube
con
talking
about
two
years
of
GG
I
K,
which
is
you
know,
as
you
can
tell
from
the
playlist.
If
you
go
to
our
playlist
and
so
they're
like
76
videos,
all
the
amazing
work
that
Joe
and
Chris
and
I've
done
over
the
last
two
years,
just
trying
to
get
content
out
there
to
help
you
kind
of
work
through
that
stuff,
and
so.
A
Got
to
talk
in
about
talking
about
two
years
of
Gigi,
okay-
and
that
has
been
you
know.
It's
a
great
talk,
so
go
check
that
out
and
that
link
again
is
in
the
hack.
Indeed,
we
also
have
a
also
saw
from
the
terraform
folks,
a
brand
new
0.12
terraform,
and
this
is
actually
a
big
one,
because
the
terraform,
you
know
I'm
just
gonna.
A
Let
you
read
through
this,
but
there's
a
ton
of
change
in
terraform,
0
1
of
0,
1,
2
and
they've
done
quite
a
lot
of
work
to
try
and
make
that
change
consumable
to
all
of
their
audience
right,
so
they're
trying
to
really
make
it
they're
trying
to
make
it
so
that,
like
you,
can
take
your
old
terraform
configuration
files
and
put
in
and
just
convert
them
to
the
new
terraform
configuration
files.
But
yeah
there's
a
lot
of
really
exciting
stuff
happening
in
the
new
terraform.
B
A
There's
a
lot
of
really
interactive,
released,
interesting
changes,
so
I
highly
recommend
checking
that
out.
If
you
end
up
using
terraform
for
your
infrastructure
automation
for
the
longest
time,
we've
seen,
we've
seen
a
lot
of
talk
about
how
movie
3,
but
now
we
can
actually
put
hands
on
the
Humvee
3.
So
there's
an
alpha
release
out
of
how
version
3
and
we
can
actually
start
playing
with
those
things
which
I
think
will
be
exciting.
A
And
lastly,
the
last
thing
I
wanted
to
show
you
was
our
back,
which
is
a
project
that
I've
seen
kind
of
floating
around
and
actually
has
been
kind
of
changing
pretty
quickly.
This
one
is
really
interesting
because
it's
a
visualizer
for
role
based
access
control,
it's
not
a
kookn.
It
is,
and
so
this
tool
effectively
lets
you
kind
of
a
numerate.
Our
back
as
it
sits
as
it
is
applied
to
your
cluster
in
a
visual
way.
A
This
put
up
I'm
like
how
michael
has
emboss
and
he's
been
working
with
a
couple
of
other
folks
in
the
community
to
really
get
you
know
those
things
matured.
There
was
actually
just
a
bunch
of
new
changes.
We're
trying
to
figure
out
who
was
the
new
Lincoln
said
it's
like
76
commits.
Maybe
it
was
well
I
can't
figure
out
what
it
is,
but
yeah
there's
been
some.
You
know
this
is
a
it's
a
quickly
changing
project
and
it's
a
really
interesting
one,
because
it
basically
lets
you
visualize
all
in
our
box
top.
A
Alrighty.
So
because,
there's
so
much
to
talk
about
in
this
particular
episode,
I've
made
myself
a
rough
outline
and
we're
gonna.
We're
gonna
work
our
way
through
it,
because
there's
just
so
much
so
much
content
to
actually
talk
through
the
first
thing.
I'm
gonna
do
is
going
to
talk
about
problems
set
up,
and
this
is
actually
a
pretty
fun
thing
so
before
I
get
too
far
into
that.
Let's
talk
about,
like
my
my
setup
in
general,.
A
Hold
on
what's
sick
again,
so
in
myself
in
general,
what
I've
got
is
I've
got
my
laptop,
which
is
a
we
know,
one
of
the
lenovo
x1
carbon.
So
then
I
also
have
an
Intel
skull,
Canyon
nook
that
is
sitting
off
to
the
side
and
the
reason
I'm
doing.
That
is
because
basically,
the
skull,
Canyon
nook
is
way.
A
You
know
from
Akoo
brininess
and
docker
perspective,
which
means
I'm
going
to
use
probably
quite
a
significant
amount
of
CPU
and
resources
and
stuff,
but
I
also
wanted
to
make
it
so
that
we
could
you
know
so
that
wasn't
affecting
the
video
that
I'm
serving
y'all,
and
so
that's
what
I've
got
built
here
and
I'm
gonna
show
you
and
I
think
it's
fun
to
show
that
stuff
off.
But
there
is
my
skull:
canyon,
nook,
I'm,
actually
gonna
flip
the
screen
to
just
face
its
you.
Do
it
so
there's
the
school
canyon.
B
A
And
there's
the
switch
underneath
it,
and
then
this
is
my
laptop
kind
of
a
terrible
picture
of
it,
but
and
the
way
that
those
things
are
all
wired
up
is
that
I
have
like
a
little
transit
network
between
my
laptop
and
my
skull
canyon
up
there,
each
connected
to
a
different
network.
This
laptop
is
connected
to
wireless
and
the
skull
canyon
isn't
connected
to
the
ethernet
and
then
I
have
like
a
little
30
network
or
one-to-one.
A
You
know
a
little
30/30
network
between
the
two
that
allows
me
to
access
that
I'm,
just
gonna
kind
enough
to
interact
with
it.
So
I
thought
that
was
pretty
interesting,
I'm,
not
sure
if
you'll
find
that
interesting
or
not,
but
I
find
it
pretty
handy
for
working
through
that
stuff.
So
that's
how
I've
got
my
physical
setup,
but
let's
talk
about
like
how
we're
going
to
reproduce
the
problem
of
kubernetes
clusters
or
Cabrini's
cluster
with
certificates
that
expire,
so
I'm
gonna
go
ahead
and
delete
my
cluster
yeah.
A
So
what
I
wanted
to
do
was
actually
I
wanted
to
make
it
so
that
I
had
a
version
of
cube
ATM
that
would
mint
certificates
that
were
short-lived
and
the
purpose
of
that
would
be.
Would
it
mean
would
be
to
make
it
so
that
the
cluster
itself
would
the
certificates
inside
of
the
cluster
would
expire
and
that
we
would
have
to
go
through
the
practice
of
actually
renewing
all
of
those
certificates
to
kind
of
show
how
that
practice
works
for
this
purpose,
I'm
I'm
still
going
to
use
kind,
absolutely
use.
A
A
/
kind
that
is
actually
the
project
directory
inside
of
github,
and
if
you
go
to
kinda
dot,
sync
ASIO,
you
can
see
the
project
itself
kind
has
gone
through
a
lot
of
changes.
Recently,
one
of
the
more
recent
changes
was
that
they
moved
instead
of
they
moved
off
of
docker
and
on
to
container
D
inside
of
the
cluster.
A
What
was
interesting
about
this
is
that
they
did
this
mainly
because
of
stability
like
most
of
the
time
when
accessing
when
using
docker
as
the
underlying
implementation
was
kind,
they
were
noticing.
There
was
a
bunch
of
instability
around
using
that
as
the
container
run
time
and
when
they
moved
to
container
D
a
lot
of
that
stabilized.
So
that
was
one
quick
win.
A
A
A
A
And
okay,
so
I
think
it's
probably
a
little
more
visible,
so
yeah,
so
I've
got
114
one
here
and
what
I
wanted
to
do
was
actually
go
into
the
command
directory,
because
all
of
cube
ADM
is
under
a
directory
inside
of
command,
so
the
entire
code
base
for
cubm
is
all
here.
This
is
where
it
resides
so
we're
gonna
make
a
patch
or
change
to
comedian.
You
want
to
change
that
functionality.
This
is
where
your
code
is
actually
going
to
merge,
underneath
the
app
directory
inside
of
kubernetes
communities
command
comedian
app.
A
A
So
what
I
did
to
actually
hack
cube
ATM
into
actually
having
short-lived
source
was
I
changed
that
Constance
from
timely
r
/
x,
24
x,
365
to
time
I
R
divided
by
4,
so
it
means
that
any
certificate
that
is
generated
using
the
standard
code
path
will
only
be
good
for
15
minutes.
Bonus
so
doesn't
mean
it's.
That
means
that
I
can
show
you
a
cluster
as
it
comes
up.
I
can
show
you
what
happens
specifically
when
the
certificates
expire
once
the
cluster
is
up.
So
you
know
before
we
get
too
much
farther
here.
A
A
Kinda
digging
more
into
like
how
this
all
this
works,
so
let's
go
into
get
kind.
Expired
inserts
and
I
want
to
talk
about
the
configuration
here,
real,
quick
but
I'm
going
to
start
the
cluster
first
so
that
we
have
that
that
clock
ticking
and
then
we'll
kind
of
go
back
and
talk
about
with
what
happens
there.
So
it
can
create
actually
I
think
you
probably
just
have
a
mystery.
A
A
A
A
A
This
is
actually
what
I
called
him
and
I
called
it:
Maui
lion-o,
and
if
I
do
that
build
command,
then
what
it'll
do
is
it'll
it'll.
Just
basically
do
a
docker
eyes,
build
of
all
the
pieces
necessary
to
bring
up
the
control,
plane
and
operate
communities,
and
then
it
will
load
all
of
the
dr.
container
images
that
are
necessary
and
also
all
of
the
are
not
docker
container
but
container
images
that
are
necessary
and
it
will
also
populate
the
under
that
that
note
image
with
all
of
the
binaries
necessary,
so
humane,
iam,
cubelet
and.
A
And
all
those
bits
will
Odin
will
be
loaded
directly
on
the
underlying
node.
So
that's
actually
pretty
cool
I
mean
it's
kind
of
an
incredible
project
because
it
allows
you
to
do
things
like
the
local
testing
of
your
kubernetes
code.
So
in
my
case,
I
just
patched
cube,
ATM
to
have
short-lived
certificates
and
then
I
built
a
new
node
image,
leveraging
that
patched
version
of
kubernetes
and
I
can
now
validate
that
it
works
the
way
I
expect.
A
So,
if
I
go
back
to
my
terminal
here,
I
can
see
the
cluster
has
come
up
and
if
I
do
keep
Kendall
actually
know
well,
if
I
do
so,
what
I'm
doing
here
is
I'm
going
to
copy
in
the
admin
comm
from
inside
the
cluster
out
doing
that,
because
we
want
to
actually
prove
the
whole
certificate
expiry
things
so
du/dr
CPE
kind,
control,
plane
and
what
I'm
doing
here
is
basically
using
a
docker
command
to
interact
with
a
particular
container.
That's
running
where.
A
We're
inside
of
which
I'm
running
Koopa,
it
is
and
I'm
copying
out
the
admin
Kampf,
which
is
actually
kind
of
a
high-value
circuit,
because
it's
actually
an
Administrative
a
static
administrative
credential
that
is
using
certificates
that
don't
expire
on
whatever
configuration.
Is
that
so
like
this
is
an
interesting
security
problem?
In
my
opinion,
admin.com
should
be
kind
of
like
something
that
you
really
consider
carefully
like
what
happens
with
it
but
interesting
stuff.
Nonetheless,
so
now
I
have
my
cluster.
It's
up.
A
B
A
A
I
had
to
make
it
so
that,
even
though
I'm
like
I'm
using
humain,
iam
mint
short-lived
service,
these
certs
are
only
good
for
15
minutes
if
I
use
queue
minion
to
renew
those
certs
and
I'm
using
the
same
queue
medium.
That
has
that
change,
then,
when
I
renew
those
certs,
the
new
search
will
only
be
good
for
15
minutes:
I,
don't
I'm,
not
it
I,
don't
necessarily
want
to
spend
the
next
hour
and
a
half
proving
how
we
get
out
of
that
situation.
A
A
So
what
I've
done
here
is
I've,
basically
just
copied
a
cue
medium
that
has
the
top
of
tree
and
I'm
doing
that,
because
I
want
to
show
off
some
cool
new
features
that
are
coming
in
115,
but
I'm,
going
to
use
that
new
cube
ADM
to
mint
or
to
rotate
the
certificates
of
those
sorts
of
things.
And
this
it
actually
gets
into
a
kind
of
an
interesting
point
that
I
wanted
to
talk
about
real,
quick,
it's
totally
viable
to
actually
leverage
a
newer
version
of
cue
ADM
to
manage
the
certificates
within
communities.
A
A
A
A
It's
pretty
neat
I
like
it
all
right,
so
you
can
use
any
QA
team
you
want,
like
you,
could
even
use
a
brand
new
one
if
all
you're
doing
is
like
something
modular
like
managing
certificates,
you're
stuck
with
changing
version
by
version.
If
you
want
to
manipulate
something
like
upgrade
so
if
I
wanted
to
upgrade
from
a
cluster
that
was
113
to
and
move
it
to
like
a
cluster
that
was
114
one
right
and
then
ideally
I
would
use
a
cube
ATM.
A
So
that's
enough
about
that,
let's,
let's
dig
in
a
little
bit
more!
So
what
the
first
cool
new
feature!
That's
coming
order
to
share
with
you
is
this
idea
of
cube
ATM
having
an
alpha
feature
that
shows
off
the
expiry
of
certificates
that
are
nestled
node,
so
to
run
this
I'm
going
to
do
doctor
exact
into
my
first
kind
of
control,
play
node
and
I'm
gonna
do
key
medium!
Alpha
SURS
check
expiration.
Now
this
command
only
exists
on
top
of
tree.
A
It
will
ship
as
part
of
115,
but
it's
not,
but
if
you're
not
like
grabbing
a
115
alpha
release
you're
not
going
to
see
this
command,
it
just
emerges
just
the
other
day,
but
what
power
like,
and
so
what
this
does?
Is
it
actually
will
evaluate
all
of
the
certificates
that
it
knows
about
locally
and
determine
what
the
expert
I'm
is
and
tell
you
how
much
time
you
have
left
until
that
axe
for
you,
it
will
also
tell
you
whether
it's
externally
managed
so
in
this
output.
A
We
have
a
number
of
really
interesting
for
the
bigots
that
we
can
actually
in
like
to
look
at
right.
First,
let's
take
a
look
at
the
the
ones
that
ended:
conf,
admin,
Kampf
controller
manager,
Kampf
and
scheduler
calm.
Now
earlier,
I
copied
admin
comp
out
and
let's
just
take
a
look
at
it,
real,
quick,
actually
yeah.
A
So
inside
of
my
admin
Kampf,
which
is
a
certificate
that
is
minted
by
Q
by
Q,
medium
on
the
mess
on
the
control
plane
nodes
as
part
of
it
as
part
of
minting
it
we're
going
to
generate
a
client
certificate,
we're
going
to
copy
in
the
CA
certificate,
authoritative,
G
data
and
we're
going
to
use
as
the
server
line.
One
of
the
can
that's
interesting
one
of
the
IP
address
in
this
case
of
the
load
balancer
most
of
the
time.
A
This
is
gonna,
be
like
a
fqdn
in
front
of
like
your
lb
or
one
of
those
things.
Yeah
Fabrizio
is
awesome.
She
is
really
kicking
out.
Some
amazing
work
in
in
queue
medium
and
the
same
for
you,
a
little
Mir
I
mean
the
both
of
you,
like
I.
Just
are
really
doing
a
great
job
in
the
community.
I
really
appreciate
both
of
you.
A
So
this
is
our
standard,
Cooper
config
and
here's
the
configuration
that's
minted
and
inside
of
here
we
can
see
that
we
have
a
base64
encoded
client
certificate.
So
I
want
to
copy
that
guy
and
I'm
gonna
echo
that
there's
kind
of
a
cool
trick.
If
you're
interested
in
this
sort
of
thing
took
these
64
minus
T
and
then
I'm
gonna
pass
that
output
to
X
file,
open
SSL
x.509
text
I'm
going
to
get
to
less.
A
A
Is
I'm
taken
from
my
cube
configure
or
the
admin
Kampf,
which
is
a
cube?
Config
I've
copied
the
actual
encoded
answer,
client
certificate
data
and
then
I'm
gonna
pipe
that
to
base64
to
you,
so
that
I
can
actually
see
the
certificate
pass,
that
to
open
SSL
x.509
text,
which
allows
me
to
inspect
the
the
the
the
metadata
associated
with
this
certificate
and,
if
I
enter.
A
This
is
what
I
get
I
can
see
that
this
was
issued
by
a
certificate
with
it,
with
a
seein
of
kubernetes
that
it
is
part
of
the
system
masters
group,
and
this
is
actually
how
we're
indicating
so
because
this
certificate
was
signed
by
that
CA
and
inside
of
the
subject.
I
have
an
O
you
of
system
masters.
This
is
actually
the
authorization
mechanism
that
is
being
used
when
I
use
this
cube
config
to
interact
with
kubernetes
since
minted
inside
of
the
certificate.
Is
that
oh,
you
system
masters.
A
That
means
that
give
it
that
means
that
tells
the
it
informs
the
API
server
that
I
want
to
authenticate
as
a
system
master
which
gives
me
like
full
access
to
everything
within
the
kubernetes
api.
So,
in
a
way,
if
you
think
about
this,
this
certificate
is
being
used
for
authorization
and
authentication.
A
What
authenticate
or
an
authorization
I
have
access
to,
and
so
in
this
case
the
certificate
is
being
authorized
as
system
masters,
and
so
this
is
authorization
and
authentication
both
of
them
in
one
certificate,
which
is
in
some
ways
good
and
in
some
ways,
scary
and
I.
Think
right.
I
think
that
my
friend
Rory
here
will
probably
have
any
number
of
things
to
say
about
that.
But
I
do
want
to
expose
that
information.
I.
Think
it's
interesting.
So
we
can
see
like
the
extended
key
usage
is
client
authentication,
so
I
couldn't
impersonate
anything
with
this.
A
Let's
go
back
to
our
command
previous,
so
we
talked
about
the
admin
confidence,
so
this
is
actually
interesting
because
the
the
check
expiration
tool
is
actually
introspecting,
not
just
the
certificates
that
are
securing
interfaces.
It's
also
able
to
inspect
the
certificate
that
is
embedded
inside
of
the
cube
configures
that
are
minted
that
are
being
used
to
bring
up
that
communities
cluster.
So,
like
your
admin,
kampf,
their
controller
manager,
configuration
kampf
and
your
schedule
kampf
these
are
each
actually.
A
Inspected
by
this
tool,
and
let
you
know
how
much
time
you
have
to
before
expert,
we
can
see
we're
on
a
short
clock.
We've
got
two
minutes
before
things
start
failing
in
interesting
ways,
so
I'm
looking
forward
to
that
I'm
sure
you
aren't
you
so
everything's
still
working
that
way.
Let's
talk
about
some
of
these
other
certificates
and
also
break
down
kind
of
the
CA
piece.
So
what
I
want
to
do
next
year.
A
A
I
want
to
talk
through
these
guys,
and
this
this
page
is
a
great
page
for
understanding
what
goes
into
all
of
the
certificates
within
current
ages,
and
there's
actually
been
some
recent
work
done
to
this,
but
I
highly
recommend
getting
into
this
stock.
If
you're
really
curious
about
like
why
we
do
what
we
do
and
how
it's
done
in
this
documentation.
A
Well,
the
reason
for
that
is
because,
if
you
have
a
server
certificate,
that's
servant
that
certificate
can
be
used
to
extend
trust
right
so,
for
example,
most
of
the
certificates
that
you
know
the
people
who
are
you
know
listening
to
me
right
now
are
probably
aware
of
our
serving
certificates
right.
I
go
to
my
local
I.
A
Go
to
my
local
certificate
provider,
like
you
know,
could
be
any
number
of
things.
It
could
be
a
let's
encrypt,
which
is
just
doing
an
amazing
job
and
helping
people
actually
encrypt.
All
the
things
and,
let's
encrypt,
will
actually
allow
me
to
mint
a
serving
certificate
for
my
web
server,
so
that
I
can
ensure
that
my
browser,
for
example,
actually
trusts
that
certificate.
A
But
if
I
use
that,
if
I
extend
the
capability
of
that
particular
certificate
service
server
certificate
to
also
do
client
authentication.
But
when
ends
up
happening,
is
that
I
end
up
using
that
same
certificate
for
both
client
and
server
authentication?
And
that
means
that
the
person
who
is
using
it
to
do
client
authentication
could
there's
no
reason
why
nothing
keeping
them
from
using
that
same
certificate
to
impersonate
the
serving
piece
which
is
a
subtle
distinction,
and
this
is
actually
part
of
why
you
know.
Obviously
there
aren't
two
different
there.
A
A
A
Right
we
can
see
the
x.509
extended
key
usage.
There
are
a
number
of
extended
key
usages
that
are
going
to
be
enforced
by
the
certificate
itself.
Right
in
this
case,
we're
saying
this
one
is
going
to
be.
This
can
only
be
used
for
web
client
authentication.
If
it
were
a
serving
certificate,
it
would
look
very
different.
So,
let's
log
it
let's
login
take
a
look
at
a
serving
certificate,
real,
quick,
just
to
kind
of
finish.
This
idea
off.
B
A
Okay,
so
here
is
our
certificate,
and
this
is
actually
going
to
be
the
certificate
that
is
in
front
of
the
API
server.
So
when
you're
using
cute
kettle
or
using
any
of
those
tools
to
actually
interact
with
the
API
server,
the
the
server-side
certificate
will
be
the
one
that
I'm
looking
at
here
and
we
can
see
that
it's
also
about
to
expire
as
we
mentioned
before.
But
if
we
look
at
down
here
at
the
extended
key
usage,
we
can
see
it's
very
different.
A
This
one
says
web
server
authentication
and
then
we
have
a
whole
different
section
in
the
same
xo5
x.509,
three
extensions
piece,
that's
called
subject:
alternative
name
and
that
subject
alternative
name
is
also
pretty
interesting
when
it
comes
down
to
the
certificate
piece.
So
a
bunch
of
the
things
that
you
see
in
this
output
in
this
particular
send
list
are
actually
kind
of
built
into
the
way
that
qadian
mid
certificates
and
are
things
that
you're
going
to
gonna
have
to
do
if
you're
doing
this
manually
as
well.
A
A
These
sands
are
generally
going
to
be
needed
to
actually
be
in
front
of
that
API
server,
because
the
API
server
is
expressed
in
a
number
of
interesting
ways.
It's
expressed
as
an
internal
IP
within
the
cluster.
That's
why
you
have
these
kubernetes
default
communities,
convenience
to
vault
services,
look
and
the
IP
address.
A
10960
one
is
actually
the
first
service
IP,
the
first
IP
and
the
service
signer
for
this
particular
bird
news
cluster
for
the
API
server
to
be
able
to
allow
authentication
for
things
like
the
controller
manager
or
the
scheduler
and
queue
proxy
and
all
the
other
bits
we
have
to
come
up.
We
have
to
have
these
built
in
you
know
if
we
also,
if
you
just
bring
up
a
kubernetes
pod
within
a
cluster,
and
you
look
at
the
environment
variables
that
are
defined
automatically
within
that
pod.
Some
of
those
are
going
to
include
these
IP
addresses.
A
10960
one
will
be,
in
this
case
the
kubernetes
service,
that's
defined
as
an
environment
variable.
So
since
these
are
all
approaches
to
the
API
server
right,
we
need
to
be
really
clear
that
all
of
these
approaches
are
part
of
the
subject.
Alternate
name
because
this
is
interfaces
being
secured
by
a
certificate,
any
path,
any
way
that
somebody
would
be
able
to
interact
with
the
cluster
needs
to
be
defined
inside
of
that
inside
of
that
San
list.
A
This
certificate
is
good
for
these
things,
but
not
for
the
one
that
you
actually
queried
beyond,
and
so
that's
actually
why
it's
so
important
that
this
San
list
actually
include
all
of
the
different
ways
that
we
could
interact
with
this
cluster,
where
the
interesting
stuff
all
right.
So,
let's
see
what
our
status
is
here
so
I'm
going
to
do.
My
key
medium
alpha,
certs
check,
expiration
bum-bum-bum,
looks
like
the
cluster
is
no
longer
looks
like
all
of
us
to
forget
that
make
up
the
control
plane.
A
For
this,
node
are
no
longer
viable,
so
I'm
gonna
exit
out
here
and
you
can
all
just
like
I-
would
expect
to
be
able
to
use
content
to
interact
with
the
cluster
right.
I
know
that
I
have
my
admin
kampf.
This
is
my
default.
Like
administrative
cluster,
it's
like
my
fallback
certificate.
I
need
to
be
able
to
use
that
interact
with
the
cluster,
so
I
come
to
work
one
day
and
I
new
cube,
kiddo
get
pods
all
names
bases.
A
And
it's
got
bad
news
for
me,
things
are
broken
I'm,
not
happy
with
my
output
here
so,
but
it
doesn't
really
necessarily
tell
me:
oh,
your
shirt's
expired
it's
kind
of
an
exercise
for
the
user,
but
so
cute
kid
won't
tell
you
that
you're
sure
it's
expired
because
it
doesn't
know
you're
using
the
search
to
authenticate
right.
It's
just
trying
to
actually
go
through
the
process
of
actually
figuring
out.
What's
actually
happening
here,
and
so
it's
telling
me
is
that
I
can't
connect
and
so
likely
what's
happening
is
my
load.
A
Balancer
specifically,
is
not
letting
me
in
so
for
the
purpose
of
this
exercise.
We're
going
to
do
is
I'm
going
to
go
back
into
the
done.
Control.
Flea
node
might
be
a
turn,
so
this
guy
is
IP
address
is
172
1708
and
we
know
that
it's
1
1308
was
actually
in
that
San
list.
So
what
I'm
gonna
do
here
is
I'm
going
to
actually
edit
my
admin
to
come
and
I'm
going
to
try
and
go
directly
to
that
server,
bypassing
my
load
balancer.
A
A
Interesting,
ok,
so
the
next
thing
I
would
do
is
I
would
try
and
go
to
one
of
the
control
plane
nodes
SSH
into
it,
and
take
a
look
around
see
what
see.
What's
what
right
so
I
can
do
instead
of
SSH
I'm
gonna
do
docker,
exec,
TI
kind,
control,
plane,
bash
and
now
I'm
in
my
system,
and
if
I
do
see
our
kettle
pots
I
can
see
the
pods
are
still
defined
by
this
year.I
kettle.
Ps
you're,
probably
gonna,
learn
a
lot
more
about
C
or
I
can
open
you've
ever
known
in
this
episode.
A
A
So
I'm
here,
on
the
right
hand,
column
we
have
this
attempt
column
and
instead
of
the
attempts
we
can
see
like
how
many
times
we've
tried
to
start
that
pod.
We
can
see
that
it
was
created
11
seconds
ago
seven
seconds
ago,
and
that's-
and
this
is
attempt
number
six
but
notice
the
thing.
But
you
can
probably
also
detect
the
thing.
That's
missing
right,
there's
no
cube
API
server
in
this
output,
and
so
like
I'm
kind.
A
Ps
minutes
a
I
can
actually
go
look
at
the
log
for
that
to
see
what
happened.
Let
me
do
see
your
ID
catalogs
and
it's
saying
you
can
get
to
the
backend.
My
registry
went
away,
saw
him
out.
He
saw
do
what
that
comes
back,
you
know.
Hopefully
they
don't
work
again,
and
so
it's
not
able
to
actually
back
it's
not
able
to
access
the
SD
cluster.
So
that's
probably
going
to
be
the
first
thing
we
need
to
fix
right.
A
A
A
This
is
the
one
with
the
expired,
sir,
so
I'm
gonna
mint
a
new
one,
so
that
I
actually
have
access
to
see.
If
there's
like
any
API
server
anywhere
that
it's
actually
working
now
I'm
gonna
make
a
special
note
that
whenever
using
cube
ATM
to
do
things,
it's
probably
in
your
best
interest
to
access
a
cube,
ATM
duck
the
configuration
that
you
used
to
stand
up.
The
cluster
in
my
case
kind
actually
puts
that
file
here,
underneath
kind
cube,
EDM,
calm
and
here's
the
configuration
that
was
used
to
instantiate
this
particular
communities
cluster.
A
So
up
here,
I
can
see
the
search
sands
that
were
added
I
can
see
the
control,
flea
and
end
point
which
actually
points
to
my
load.
Balancer
in
front,
which
is
a
little
H.
A
proxy
load
balancer
I
can
tell
I,
can
see
that
I'm
determinate
things
like
host
path,
provisionary
and
that
the
kubernetes
version
is
114
one,
and
it's
dirty
because
of
that
patch
that
we
talked
about
before
I'm
sitting,
pod
subnet,
I'm
minted
a
token
that
doesn't
expire
and
got
my
joint
configuration
defined
here
and
not
really
much
else
of
real
interest.
A
All
the
rest
of
them
are
pretty
default,
except
for
this
CR
eye
socket
piece.
That's
actually
why
these
pieces
exist,
so
the
CR
eye
socket
piece
basically
tells
Q
midium
that
what
instantiates
the
node
when
the
node
gets
when
the
node
registers
itself,
when
the
qubit
registers
itself
with
the
cluster,
it
tells
it
the
CI
socket
for
this,
for
the
for
the
cube
will
be
located
here.
A
So
that
helps
us
kind
of
configure
those
things
so
that
all
that
can
figure
or
that's
that's
where
Mike,
if
you
configuration,
is-
and
it's
important
that
I
use
that
configuration
because
of
this
line
right
here
right.
What
this
line
tells
me
is
that,
when
I'm,
going
to
mint
cube,
configs
like
the
admin,
kampf
I
need
to
use
this
file
to
inform
the
cube
ATM
code
to
use
that
control
plane
endpoint
as
the
server
in
the
admin
comp
so
to
show.
A
B
A
A
Then
what
this
does
it's
a
little
meant
a
new
admin
Kampf
just
like
it
does
when
it
statutes
the
cluster.
What
I'm
doing
is
basically
re
running
that
phase
that
minted
the
admin
Kampf
leveraging
kind
of
leveraging
cube,
ATMs
native-like
phase
model,
which
is
awesome
and
again
shout
out
to
Lumiere
and
the
entirety
and
for
me,
geo
and
everybody
else
involved.
It's
a
close
to
a
lifecycle
focused
on
cube.
Atm
really
makes
a
lot
of
this
stuff
easier.
A
A
What
is
the
I'm
gonna
conflict
like
now,
so
this
is
important
specifically
in
this
case,
because
it's
actually
going
to
populate
that
server
field.
But
it's
also
important
when
we
think
about
those
sans
right.
If
I
extend
the
sans
to
include
like
my
load,
balancer
name
or
any
of
those
things
and
I,
don't
use
that
queue,
medium
Kampf,
what
I'm
minting
a
new
certificate
or
rotating
certificates,
then
that's
it.
Then
then
we
might
mess
that
up
and
actually
make
it
so
that
those
certificates
that
we
rotate
don't
have
the
appropriate
sans
populated
in
them.
A
So
I'm
calling
the
south
is
super
important
anytime.
You
can
actually
make
use
of
that
cube,
ATM
comp.
You
should
do
so
in
a
single
in
a
single
master
cluster.
It
doesn't
really
matter
as
much
most
of
the
time,
but
if
you're
doing
multi
masters
I
usually
means
that
you
have
a
load
balancer
in
front
of
your
API
server
and
that's
where
things
like
this
actually
start
to
kind
of
shake
out
and
you've
really
got
to
make
sure
that
you're
using
that
cue
medium.
A
So
we
have
the
key
medium
kampf
locally
on
the
file
system,
which
is
awesome
because
if
we
were
in
a
state
right
now
or
shirts
have
expired
and
we
needed
to
get
into
the
cluster
and
be
able
to
access
the
stuff
we'd
be
in
kind
of
a
rough
spot,
because
we
can't
really
interact
with
the
club.
The
comedian
is
cluster
at
the
moment,
so
we
jump
back
out
and
I'm
gonna
copy
that
admin.com
outlet,
just
like
I,
did
before
so
kind
control
playing.
That's
the
cooperate
is
copy.
A
A
A
Can
see
now
that
I
got
a
year
right,
but
it
looks
much
the
same.
I
have
the
same
type
of
certificate
that
I
had
before
it
still
has.
Oh
you
masters
or
system
masters
from
the
v3.
It
still
says
web
client,
authentication
or
no
sans.
Obviously,
it's
a
client
certificate.
So
all
that
stuff
is
all
here
and
that's
all
great
okay.
So
now,
if
I
do
cube
kettle
get
pause,
all
namespaces
I'm
still
expecting
this
to
fail,
and
you
may
remember
why
or
you
might
not
so
I
do
for
I
in
the
control.
A
B
A
What
I've
done,
what
I've
just
done
here
is
I've,
run
ahead
and
run
come
in
and
on
each
of
my
control,
plane,
nodes
and
I
wanted
to
see
the
output
of
Sirach
LPS,
which
shows
me
they're,
running
containers
or
yeah
they're,
running
containers
on
each
node,
and
we
can
see
that
on
for
all
of
the
individual
master
or
control
a
knows.
I
have
no
API
servers
and
no
azzedine
know
it's
running.
A
Those
were
both
failed,
q-final
still
working
you
proxy
still
working
and
other
process
I
would
have
deployed
to
the
worker
notes
and
all
those
things
they're
all
still
running
and
active,
and
do
it
okay,
but
anything
that
would
rely
on
as
to
the
API
simmer
to
be
operational,
will
not
work
right.
All
of
those
things
will
have
failed.
A
A
But
first
thing:
I'm
gonna
do
is
I'm
just
going
to
take
down
the
control,
find
elements
and
then
figure
out
the
certificate
stuff
and
then
bring
up
a
control
plane
a
little
bit
one
after
the
other,
and
so
that
I
can
kind
of
rebuild
my
control
cluster
or
the
control
plane.
One
I
think
in
the
time
that
I
have
so.
B
A
Actually
held
or
the
the
bits
that
use
these
certificates
are
being
held
inside
of
this
directory
called
the
manifests
and
there's
my
NCD
node
definition
for
this
particular
node,
the
API
server
controller
manager
and
scheduler.
Those
are
most
of
the
things
that
actually
make
up
the
certificate
surface
for
the
control
plane.
Obviously
we
have
cubelets
as
well,
but
we're
going
to
get
to
that
here,
like
after
we
get
through
this
bit.
A
A
And
we'll
move
manifests
star
into
old,
and
so,
if
I
go
into
benefits
it's
empty.
So
if
I
do
see
our
kettle,
pods
I
should
see
that
those
pods
have
been
up
but
have
become
undefined
I,
don't
have
the
API
server
controller
or
scheduler.
Those
pods
are
now
gone.
Let's
go
ahead
and
do
that
for
the
next
for
the
next
two
nodes
as
well
and
so
mcdeere
I
see
it
is
oh.
A
Waiting
for
those
team
this
time
remember
sometimes
it
takes
a
minute
it
takes
too
long.
You
can
always
check
the
status
of
cubelet.
Don't
make
sure
that
it's
actually
working
the
way
we
expect
it
is
an
active
state.
I
never
go
so
they
did
that
removed.
So
sometimes
it
takes
a
second
for
cuba
to
actually
converge
on
that
change.
A
So
now,
we've
got
all
three
nodes:
we're
not
going
to
state
where
none
of
the
control
plane
elements
are
up.
So
now
we're
kind
of
in
a
clean
state
right.
We
can
go
back
and
clean
up
the
certificates
necessary
to
make
all
this
work.
No,
in
my
case,
all
of
my
certificates
were
minted
by
cube
ATM.
In
your
case,
you
might
have
used
any
other
mechanism
like
within
your
within
your
cluster,
but
I'm
going
to
use,
cube.
A
Atm
is
kind
of
like
a
way
to
describe
to
think
about
this
particular
problem,
and
so
just
because
I
think
it's
you
know
it.
Ships
with
kubernetes
itself
is
version
with
kubernetes.
There's
a
lot
of
really
great
things
about
queue,
medium
that
I
that
I
really
appreciate,
including
the
ability
to
actually
use
it,
to
do
things
like
recovery
like
that,
we're
talking
about
here.
So
what
I'm
going.
B
A
A
A
Some
kind
of
interesting
things
here
so
since
these
are
already
expired,
I'm
not
going
to
back
them
up,
but
you
may
want
to,
in
my
case,
I'm
just
going
to
wipe
them
out
for
the
purposes
of
this
exercise,
and
the
Burgin
doesn't
want
to
talk
to
you
like
how
how
they
getting
recreated
so
I'm
gonna
do
RM
API
server
star
I'm,
going
to
get
rid
of
the
front
proxy
client
star,
I'm
gonna
get
rid
of
a
CD
health
check.
That's
the
peer
and.
B
A
A
A
A
A
My
mistake
was
that
I
didn't
use
the
comedian
accomp,
and
so,
if
I
look
at
the
sands
that
were
used
for
the
API
server
serving
certificate,
I
don't
see
that
that
load,
balance
or
IP
in
the
eyepiece
I
saw.
That
was
my
mistake.
So
I'm
just
gonna
remove
the
that's.
The
breed
is
PID
I
server
star
again
I'm
gonna
go
back
and
do
my
init
face,
sir.
It's
all
config
kinda
can
see
video
calm.
A
And
so
we
could
see
the
difference
in
the
output
here
right
see
the
API
server
where
before
was
signed
with
10960,
1
and
172
1708,
and
now
we
see
that
it
was
issued
before
more
than
that,
it
was
issued
for
localhost
for
117
71
72
1706,
which
is
my
load
balancer,
IP
and
117
to
1708
and
10960
1.
So
this
is
what
I
was
talking
about
before,
like
if
I?
Actually,
if
I,
not
notice
that,
if
I
didn't
use
the
config,
then
what
ends
up
happening
is
that
I
might
actually
meant
a
certificate.
A
A
So
what
I'm
going
to
do
is
I
remove
the
old
manifests
back
into
the
manifest
directory
here
here
they
all
aren't
I.
So
now,
if
I
do
see,
you
can
see
our
kettle.
Pods
I
can
see
that
those
pods
have
been
redefined
again
by
the
cubelet.
These
static,
pods
that
are
defined
inside
of
the
SE
kubernetes
manifests,
are
actually
entirely
owned
and
operated
by
the
keyboard.
A
A
A
So
if
I
had
two
members
that
were
up
in
health,
even
as
it,
he
would
be
okay
with
me,
but
at
the
moment
I
only
have
one
member
that's
healthy,
and
so
it's
not
healthy
right,
because
it's
like
I
have
lost
quorum.
Things
are
bad
I
can't
move
forward
until
quorum
comes
back,
so
let's
go
ahead
and
fix
the
rest
of
the
nodes
to
get
that
to
get
corn
back
for
a
CD.
A
B
A
Oh,
look
at
that
I.
Don't
have
Q
medium
comp
on
here,
okay!
Well,
that
means
I
have
to
get
it
on
here.
Somehow
this
thing's
gonna
happen
to
you
like
in
your
indifferent
control
thing.
Sometimes
we
only
keep
the
commedia
minute
complement
one
that
we
actually
bring
up.
Initially,
we
don't
bring
it
up
on
off.
We
don't
put
it
on
all
of
them.
It
is
stored
as
a
config
map
within
the
cluster,
and
so,
if
you
actually
have
a
backup
of
that
comp,
you
can
actually
you
know
you
can
make
you
spit
it.
A
However,
your
mom
in
my
case
I,
don't
though
I
mean
I,
don't
have
access
to
the
cluster,
because
the
cluster
is
still
in
a
broken
state,
so
I
couldn't
use
as
you
need
to
get
access
to
that
file,
so
I
need
to
figure
out
another
way
to
get
it.
So,
since
it's
actually
on
disk
on
the
first
control
point,
node
I'm
gonna
copy
it
from
there
to
the
other
two
control
plane
notes
and
make
use
of
it
that
way.
I'm
gonna
do
docker
CP
again,
I
can
do
this
with
SCP
or
whatever
kind
of
control.
A
Kind
qadian
Kampf
would
you'd
see
that
all
those
search
that
we
deleted
are
gone,
but
you
know
other
interesting
stuff
that
happens
inside
of
this
output
is
actually
pretty
cool.
So
what
we've
done
is
we
basically
just
removed
a
bunch
of
certificates
from
the
directory?
That's
identified
by
surfer
ticket
dear
@cq
written
is
PKI,
and
then
we
can
see
that
in
some
cases
it's
you
only
use
an
existing.
It's
going
to
use
an
existing
resources
and
in
other
cases,
it's
generating
new
ones,
all
right,
so
the
CA
certificate
didn't
get
wiped
out.
A
We
want
to
use
the
same
CA
certificate,
but
we
are
going
to
generate
new
API
servers
using
that
our
new
API
sort
of
server
certificate
using
that
CA
we're
going
to
generate
a
new
cubelet
certificate,
we're
going
to
generate
we're
going
to
use
the
existing
that's
edca
and
then
what
we're
going
to
generate
a
new
pier
all
these
things
and
you
can
kind
of
see
how
that
all
works
out
right.
All
of
the
individual
certificates
that
we
need
to
generate
can
be
generated
using
the
existing
the
CAS,
because
we
didn't
wipe
them
out.
A
But
here
at
the
bottom
we
also
make
use
of
the
existing
si
key
we're
going
to
talk
about
like
what
that
is
a
keys
for
before
it's
all
over
as
well.
So
now
that
we've
generated
our
new
certificates,
let's
go
ahead
and
put
our
manifest
back.
So
I
think
you
really.
It
is
old
move
star
to
manifests
and
then
see
our
ikat.
All
pods
will
see
them
get
recreated
here.
There's
the
SE
one.
A
There's
the
rest,
so
now
we
have
to
as
the
codes
that
are
at
least
working
are
able
to
work,
but
we
need
to
actually
have
them
both
online
that
make
it
actually
functional.
So
we're
not
that
kind
of
chewing
on
something
permanent.
While
we
go
fix
the
third
control
plane.
Note:
three:
four:
okay,
so
same
thing
again,
as
you
can
renew
this
peak
in
our
API
server
star
front
proxy
client
star
at
CD,.
A
B
A
A
One
more
tool
I'm
actually
going
to
make
use
of
as
another
tool
that
I
wrote
to
actually
help
me
interact
with
GD
and
the
client
all
in
clusters
that
are
filled
with
qdm.
So
here's
how
I
want
to
do
that.
I'm
going
to
do
curl
so
I'm
inside
of
the
SE
kubernetes
manifest
directory
in
kernel,
hello,
ji.
A
B
A
So
what
this
pod
does
is
it's
a
statically
defined
pod,
it's
gonna
put,
it's
gonna
be
put
into
the
cube
system
namespace
and
the
commandments,
and
the
command
is
basically
just
going
to
sleep
for
forever.
You
know,
but
the
interesting
bit
actually
happens
further
down,
so
I'm
actually
gonna
base
on
the
same
SCD
image
that
the
cluster
may
use
it's
gonna,
be
using.
One
of
the
versions
of
Betsy
did
is
pretty
recent
called
version.
A
It's
a
tcp
CTL
API
version
3
the
third
version
of
api,
which
asked
me
to
interact
with
it
on
new
versions
of
kibriya
vet
CD,
I'm
going
to
specify
the
CA
cert
to
use
to
validate
that
the
serving
certificate
of
the
sv
cluster
is
signed
by
something
I
trust
as
a
client.
I'm
gonna
provide
a
cert
and
a
key
which
are
I'm
just
gonna
use
the
health
check.
Client.
Sir,
thank
you
for
that
and
then
I'm
going
to
specify
the
at
City
endpoints.
A
The
last
environment
variable
I
set
is
at
city
cluster,
and
this
one
is
an
interesting
one,
because
it
allows
me
to
learn
from
the
cluster
all
of
the
members.
So,
instead
of
actually
just
interacting
one,
two,
seven
zero
zero
one
I
can
do
things
like
check
performance
and
those
tools
against
the
entire
cluster.
So
let's
show
under
this
guy
and
take
a
look
at
how
this
works
see.
I
can.
A
So
again,
inside
of
here
we
can
see
all
the
environments
that
are
being
specified.
This
is
what
I
was
talking
about
for,
like
by
the
environment,
could
renew
this
port
443.
It
uses
the
IP
address,
so
an
application
that
makes
use
of
that
or
makes
you
some
Kuiti.
This
service
host
would
actually
be
trying
to
access
the
API
server
on
this
IP
address,
and
that's
why
it's
so
important
that
10960
one
is
inside
of
the
sand
for
this.
B
A
And
so
in
point
healthier,
because
I
have
that
cluster
config
environment
variable
it's
actually
able
to
actually
interact
with
each
of
the
members
in
the
cluster
and
see
if
that's
an
CFS
City
is
healthy.
So
what
this
allows
me
to
do
is
validate
that
those
certificates
are
all
in
a
healthy
state
right,
so
I'm
able
to
actually
interact
from
this
particular
entity
clinic
to
all
of
the
individual
pieces,
and
it's
all
working
good.
A
A
A
I
I've
fixed
a
CD
and
I
fixed
to
API
server,
but
there's
this
really
interesting
state
problem.
What
happens
around
ICD
like
if
you,
because
the
control,
because
of
the
way
the
controller
managers
work,
so
the
controller
managers
are
not
in
a
healthy
state
like
if
they
get
shut
down
or
they
lose
access
or
or
something
like
that
happens
then
some
happening.
Is
that
like
there's
nothing
to
actually
update
the
status
right?
So
if
I
were
to
do
cube
kettle
all
run
nginx.
A
A
So,
what's
interesting
here
is
that
when
I
do
can
I'll
get
pause,
all
namespaces
I
am
kind
of
looking
at
a
set
of
lies
there
right,
because
the
controller
managers
themselves
are
not
in
a
healthy
state
because
their
certificates
are
still
expired.
We
haven't
replaced
them
right,
so,
if
they're
in
a
state
where
they
can't
actually
authenticate
with
the
API
server,
and
how
do
we
get
to
the
place
where
the
API
controller
manager
and
the
scheduler,
for
example,
can
write,
because
neither
of
these
two
can
authenticate
at
the
moment.
Let's.
B
A
This
real,
quick
and
show
and
show
what
I'm
talking
about
so
I
did
get
pause
include
system
show
labels,
we
can
see
component
cube.
Scheduler
is
our
label
for
this,
so
we'll
do
cube
kettle.
Long's
income
system
ow
and
this
will
actually
pull
the
logs
for
the
entire
set
of
cube
schedulers,
and
we
can
see
that
from
the
OP
from
the
output
that
the
authentication
is
not
working
right.
A
Can't
actually
get
access
to
the
API
server
to
determine
if
there
is
a
lock,
so
all
of
them
kind
of
bailed
out.
So,
even
though
our
key
it'll
get
pods,
all
namespaces
shows
that
they're
in
a
running
State,
that's
a
lot.
I
mean
they
are
running
they're,
consuming
resources
but
they're
not
able
to
authenticate
right.
A
A
A
There's
an
interesting
thing
about
docker
containers
and
static
pod
manifests.
That
is
a
really
hard
hard
learned
lesson
frequently
when
we're
actually
bringing
up
a
container
and
we
copy,
and
we
associate
a
volume.
Sometimes
that
volume
represents
like
a
snapshot
of
the
content
of
that
of
that
data
rather
than
the
data
itself.
A
Ideally,
what
I
would
want
to
do
is
make
sure
that,
when
I'm
actually
making
a
change,
modifying
files
already
mounted
into
a
particular
container
that
I
take
the
container
down
and
restart
it
to
ensure
that
when
the
container
is
restarted,
it
has
access
to
those
new
files.
That's
one
way
to
think
about
the
problem.
A
The
other
way
to
think
about
the
problem
is
that
sometimes
the
controller
manager
and
the
scheduler
may
not
be
configured
to
watch
for
new
certificates,
and
so
if
they
are
not
configured
to
watch
for
new
certificates,
even
though
I
changed
that
file,
they
may
not
actually
converge.
Now,
in
our
case,
the
controller
manager
and
scheduler
are,
and
all
the
stuff
mounted
in
will
actually
eventually
converge.
It's
just
faster
to
move
it
this
way.
So
all
these
guys
back
into
manifests.
A
B
A
A
There
was
I,
basically
had
to
economic
force,
the
controller
manager
of
the
scheduler
to
be
restarted,
with
a
new
authentication
such
that
they
could
be
used
to
authenticate
to
the
new
API
server,
because
those
old
clan
sirs
had
expired
and
now
that's
all
fixed
and
so
now
I
can
actually
schedule
new
work
and
things
work
the
way
they
are
they're
expected
to.
So
that's
awesome.
So
now,
at
this
point,
I'm
back
to
a
working
control,
plane,
there's
one
more
thing:
I
wanted
to
cover
before
we're
completely
finished,
which
is
basically
that
cubelet
authentication
piece.
A
A
Within
any
Cooper
news
cluster
that
leverages
bootstrap
TLS,
there's
some
interesting
functionality.
If
I
do
cube,
can
I'll
get
CSR.
I
can
see
those
certificates
that
are
being
used
to
interact
with
it
with
it
with
the
cluster,
and
we
can
also
see
this
requester
piece
and
this
requester
search.
First,
let's
talk
about
what
gets
CSRs,
so
that's
ei
that
we
built
for
the
kubernetes
cluster
itself
is
actually
being
used,
internal
to
the
cluster,
to
mint
client
certificates
for
for
cubelet
nodes,
and
so,
if
I
have
docker
exact
backing
into
a
kind
of
fairy.
A
A
Obstacle
earlier
this
guy,
this
assuming
it
was
actually
issued
for
a
year,
so
we've
got
plenty
of
time
for
the
stack
actually
work
and
automatically
as
part
of
bootstrap
TLS.
The
certificate
will
be
rotated
in
a
word
to
expire
right.
So,
even
if
the
certificate
itself
we're
coming
up
on
ex
free,
the
it
would
actually
be
able
to
issue
a
new
CSR
and
automatically
in
the
that
CSR
could
be
automatically
approved.
A
So,
but
what?
If
I?
But
what?
If
I
needed
to
actually
refer
to
to
to
recycling
it?
What
if
it?
What
if
I
was
in
a
state
where
the
cubelet
itself
was
actually
like
down
for
bid
and
it
needed
to
actually
mint
a
new
certificate.
So,
let's
play
with
kind
of
worker
one
here
and
kind
of
going
through
that
exercise.
A
A
Of
the
Etsy
kubernetes
director
you're
going
to
see
a
couple
of
different
interesting
files
right,
the
first
one
being
this
bootstrap
cubelet
golf
and
inside
of
here
we
can
see
this
token,
that's
being
used
to
authenticate
right.
So
this
bootstrap
cubelet
configuration
has
a
copy
of
our
certificate
authority
data.
It
has
a
copy,
it
has
a
server
certificate
IP
and
then
it
has
this
token,
that's
being
used
if
I
go
back
to
to
here
in
a
month,
I'm
back
on
them
on
a
control
find
node
and
I
do
Hume
ATM
token
list.
A
A
B
A
Medium
to
join
the
kubernetes
cluster,
that's
part
of
the
Q&A
in
queue
medium
join
semantic,
but
part
of
that
part
of
the
mechanism
is
token,
is
basically
enabling
it
to
actually
issue
a
certificate.
Signing
request
to
the
cluster
CA
as
managed
by
the
by
the
csr
permanent
within
kourounis
api
and
then
allow
for
that
to
be
auto,
approved
and
then
the
cubelet
class,
the
cubelet
client
certificate
can
be
auto-generated.
So
let's
go
again:
let's
just
walk
through
this
again
sorry.
So
what
I'm
going
to
do?
A
This
will
tear
down
the
node
and
it
will
remove
all
resources
and
it'll,
basically
wipe
out
the
node
right
and
then
I'm
gonna
kind
of
go
back
through
I'm
gonna
go
back
to
my
control,
plane,
I'm
gonna
do
export,
who
config
equals?
That's
all
I'm
gonna
do
cube
kennel
get
nodes.
We
can
see
that
worker
is
still
defined
here.
Nothing's
deleted,
it's
also
kind
of
weird
that
it
says
ready
because,
obviously
it's
not.
It
will
eventually
notice
that
it's
not,
but
in
my
case
I'm,
just
gonna
go
ahead
and
wipe
it
out
so
delete.
B
A
A
B
A
Print
the
joint
command,
which
is
a
really
cool
command.
So
what
I
do
here
if
I
do
Q
medium
token,
create
print
joint
commander?
What
will
happen
is
that
QB
team
will
use
my
ethnic
Asian
taw
to
interact
with
the
cluster
it'll
emit
a
new
token
randomly
generated,
and
it
will
actually
print
the
joint
command,
which
should
be
everything
I
need
to
know
to
be
able
to
join
a
new
node
to
the
cluster.
A
A
And
this
cue
medium
join
command,
I'm,
actually
pointing
at
the
IP
address
of
the
load
balancer
as
learned
from
the
Cape
from
the
from
the
cluster
itself
in
part
of
it
as
part
of
that
uploaded
chameleon
kampf
I
generated
I'm
a
random
token,
which
is
the
identified
by
this
bit
of
text
here
and
then
the
other
interesting
thing
is
I've
just
said.
Discovery
took
in
see
a
certain
hash,
and
this
is
important
because
it
enables
us
to
ensure
that
when
I
interact
with
that
server,
once
every
two
1706
the
certificate
that
is
being
used.
B
B
A
Might
not
be
true
what
up
to
see
alright,
so
why
do
cubm
join?
Oh,
it's
gonna
tell
me,
you
know,
obviously
hey
you're
using
darker
and
darker
things
are
going
to
work.
The
way
you
expect
so
in
my
case,
I'm
just
gonna
go
ahead
and
do
ignore
pre-flight
errors,
because
it's
a
thing
in
your
case.
You
probably
want
to
understand
what
those
errors
are
specifically
for,
but
you
know
ain't.
Nobody
got
time
for
that
in
this
demo.
A
B
A
We're
waiting
for
we're
going
in
we've
downloaded
the
config
map,
we've
started,
cubelet
we've
configured,
the
cubelet
using
varlam
cubelet,
come
fig
da
mo
mo,
and
we've
also
specified
an
environment
flag,
we're
activating
leveraging
system
v
the
cubed
service
and
then
we're
doing
this
part,
which
is
the
interesting
bit
waiting
for
the
cubelet
to
performed
TLS
bootstrap.
And
then
it
says
the
cubelet
has
enjoined
the
cluster,
so
I
think
if
I
do
stee-rike
at
all,
pods
I
could
see
funnel
and
proxy
have
already
been
deployed
here.
A
They
started
32
seconds
ago
and
we're
a
member
of
the
cluster.
If
I
go
back
to
the
control
plane
again
and
I
named
futile
token
list,
I
can
sorry
cube
ATM
token
list.
I
can
see
that
now
this
other
token
that
was
generated
it's
this
one
here
that
DJ
LVD
h1
right
it
has
the
same
month.
On
occasion
it
expires
in
23
hours
because
it
was
only
just
minted
and
all
that
stuff,
my
new
mic.
You
can
don't
get
CSR
again.
A
And
we
could
see
that
this
was
actually
a
certificate
that
was
minted
for
kind
worker
and
that
the
token
that
it
used
to
authenticate
was
this
bootstrap
token
and
the
the
status
of
that
certificate
is
approved
and
issued.
And
so
all
those
things
kind
of
show
us
how
the
bootstrap
TLS
part
works.
But
what,
if
our
cubelet
and
so
what's
interesting
about
that,
is
that
the
keyless
certificate,
the
clients
doing
it
well,
will
Auto
renew,
let's
jump
back
to
our
cubelet
and
look
around
a
little
bit.
A
Inside
of
the
violin
cubelet
certificate
I
also
have
this
culet
cert
and
key,
and
these
are
the
certificates
that
are
used
to
secure
the
API
in
front
of
the
of
the
cubelet.
Let's
take
a
look
at
that
certificate.
Real
quick!
These
are.
This
is
a
certificate
that
is
created
whatever
Hewlett
starts.
I
do
open,
SSL.
A
Automatically,
but
in
our
case
we're
just
going
to
pretend
like
we
don't
have
that
working
for
us,
so
what
I'm
going
to
do
is
I'm
going
to
do
a
systemctl
stop
cheaply
and
now
that
this
particular
cubelet
doesn't
have
anything
running
on
it
at
the
moment,
it
won't
try
to
authenticate
with
the
API
server
and
I'm
gonna
wipe
out
everything
in
this
directory.
Our
analysts
are.
A
A
So
we
can
see
that
what
happened
there
was
we
went
back
through
the
bootstrap
TLS
process.
We
have
a
new
current
cubelet
client
certificate,
but
we
also
minted
new
cubelet
certain
clients,
the
serving
certificates,
those
are
also
brand
new.
So
if
all
the
stuff
was
expired,
that's
one
way
to
renew
it.
There's
one
more
important
bit
inside
of
Etsy
kubernetes
bootstrap
coop
cubelet.
A
We
need
to
make
sure
that
this
token
is
valid,
and
so,
if
this
token
has
expired,
if
we
look
at
a
control
plane-
and
we
see
that
the
token
is
expired-
we've
got
to
mint
a
new
one
and
then
we
can
just
replace
this
file
the
cubelet.
But
we
can
just
place
this
here
token
directly
or
we
can
do
something
like
you
may
DM
reset
like
we
did
earlier
and
we
can
force
it
to
use
a
new
token.
A
So
at
that
point
we
have
now
talked
through
the
entire
security
or
TLS
surface,
there's
so
many
certificates
and
so
much
interesting
stuff
about
them.
That's
what
all
of
them
kind
of
do
and
how
they
all
work.
I've
not
replaced
the
CA
certificates
in
the
cluster
cuz
they're
good.
For
ten
years
generally,
this
was
just
replacing
all
of
the
actual
client
and
serving
certificates
in
the
entire
set,
and
so
yeah.
B
A
B
B
A
Those
things
I
fixed
all
the
things
we
proved
that
they
were
all
fixed.
The
last
thing
I
want
to
talk
about
was
this
search
versus
token
thing
and
I
said:
I
was
going
to
come
back
to
that.
So
let's
talk
about
that
real,
quick
and
then
we'll
be
all
done
for
the
episode.
So
what
I
wanted
to
point
out
was
that
the.
A
Service
accounts
within
kubernetes
are
generally
don't
expire
now,
some
of
that's
changing
and
that
you'll
have
like
the
ability
to
define
when
a
service
account
will
expire,
but,
generally
speaking,
a
service
account
or
a
token
that
is
defined
within
communities,
clustered
doesn't
doesn't
expire,
and
the
other
interesting
thing
is
that
the
token
itself
is
not
a
certificate
right.
It
is
a
JW
and
and
the
way.
A
A
So,
in
all
of
his
expertise,
tuff
we've
talked
about,
we
haven't
had
to
actually
rotate
any
service
accounts,
because
those
service
accounts
themselves
haven't
expired.
They
were
signed
by
the
by
a
certificate
that
doesn't
expire,
so
let's
jump
into
a
control,
plane,
node
and
and
look
at
how
that
works.
So
we're
back
on
our
kind
control
plane,
one
or
the
the
first
kind
of
control,
plane
node.
If
I
move
into
seq
renews
PKI
here
is
the
SA
pub
right.
A
Build
that
it's
being
used
to
the
signing,
so
the
private
key
itself
is
being
used
to
do
the
signing
and
the
public
key
is
being
used
to
do
the
validation.
So
when
a
JWT
or
a
service
account
is
created,
we're
going
to
use
and-
and
you
use
that
service
account
token
to
authenticate
to
the
cluster.
We're
going
to
use
this
public
key
to
validate
that
the
the
JTP
to
the
WT
was
signed
by
a
private
key,
that
we
trust
and
it
can
be
a
stack
it
doesn't
have
to
be
just
one
I
say
public
key.
A
If
you
have
multiple
I
say
public
keys,
you
can
actually
provide
multiple
of
them
in
here
and
will
actually
validate
each,
and
so
this
is
pretty
interesting
stuff.
If
you
think
about
it.
This
is
a
way
of
maybe
of
extending,
if
you're
careful
about
it,
some
way
of
maybe
extending
trust
across
multiple
clusters,
one
one
of
the
month,
one
of
the
many
different
ways
that
you
could
actually
achieve
this
so.
A
B
B
A
A
A
And
so
what
we've
just
done
here
is
again
we've
kind
of
like
configured
that
we
pull
down
a
configuration
from
the
kubernetes
cluster
to
use
as
an
authentication
piece.
So
now,
if
I
do
export,
who
config
equals
cook
config
I
do
QK
don't
get
pause.
All
namespaces
I
will
get
that
I
can't
do
that,
because
if
you
remember
the
role
binding,
that
I
created
was
only
for
the
default
namespace.
A
A
B
A
Clicked
back
into
my
control
plan
for
a
second,
so
we
were
talking
about
how
like
how
do
we
get
access
to
this
file,
the
kind
of
ADM
kampf?
If
you
have
a
cluster
that
I
mean,
if
you
have
a
cluster,
that's
up
and
running,
you
can
actually
just
download
this
directly
right.
So
if
I
do
queue,
medium
configure
of
you.
A
A
Bits
are
here
right,
like
which
team
is
to
use
what
the
positiveness.
It
was
said
that
you
know
all
the
actual
configuration
editor
edit
as
it
is
rendered,
including
that
control,
plane,
endpoint
piece
and
the
controller
manager
has
drugs.
So
all
of
these
pieces
are
those.
So
this
is
the
only
thing
I
really
need
for
to
bring
up
the
cluster
itself,
and
so,
if
you're,
in
a
state
where
you
don't
have
that
cube,
ATM
comp
and
you
need
to
get
back
to
a
place
where
you
have
it.
This
is
one
way
to
do
that
right.
A
You
can
actually
just
download
it,
but
hopefully
you're
gonna
download
it
before
all
of
your
shirts
expire,
because
otherwise
you're
in
a
bad
way.
Anyway,
that
was
all
I
had
to
share
with
you
today.
I
hope,
I
was
informative.
I
know
that
this
is
an
important
subject,
shout
out
to
Sam
and
a
few
other
folks
inside
of
the
kubernetes
lac,
who
actually
asked
me
this
question
a
couple
of
times
in
the
last
few
weeks
and
have
helped
them
through
this
process.
A
Much
really
informative
and
that
you
get
a
lot
of
it.
Oh
one,
more
thing,
sorry,
it's
always
one
of
my
thing!
Isn't
there,
okay
in
115
in
115!
Thank
you,
Fabrizio,
there's!
Actually
a
new
code
going
in
that
will
allow
you
to
rotate
certificates
even
easier
than
the
one
even
easier
than
all
the
things
that
we've
already
done
right.
So
in
115,
you
can
now
in
as
an
alpha
command.
Do
something
like
awesome,
certs,
renew
all
remember
how
we
had
to
kind
of
like
go
through
and
delete
them
and
do
all
that
other
stuff.
A
A
A
A
B
A
Because
it's
expensive
but
I'm
like
Bing,
that's
amazing
anyway,
so
yeah.
Thank
you
all
for
your
time.
I
hope
you
enjoyed
it
and
I'll
see
you
next
time.
Actually,
I'm
gonna
be
up
for
vacation
for
two
and
a
half
weeks
right
after
this
next
week,
so
likely
it'll
be
it'll,
be
mr.
Joe
betta
back
with
you,
or
perhaps
somebody
else
from
my
team.
So
thanks
again,
except
this
is
a
this-
is
the
guy
I
was
working
with
to
actually
fix
his
cluster.