►
From YouTube: A Basic Kubernetes Debugging Kit: curl, jq, openssl, and Other Best Friends - Joe Thompson
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
A Basic Kubernetes Debugging Kit: curl, jq, openssl, and Other Best Friends - Joe Thompson, Mesosphere
Joe Thompson walks you through a set of essential command-line Linux tools – curl, jq, openssl, netcat, dig and others – and how to use them to debug network and other issues in your Kubernetes clusters. The basic function and use of each tool is covered, with examples for some common Kubernetes-related tasks like verifying if a certificate is correctly generated or checking if DNS resolution in your cluster is working properly. We’ll also look at some container-based debugging procedures, including how to use one container to debug another that’s not running properly or to debug the host OS when the tools you need aren’t installed there.
To Learn More: https://sched.co/GrYA
A
So
you
know
it's
a
basic
kubernetes,
debugging
kit
and
I
mean
a
couple
of
things
by
that
and
you'll
see
what
I
mean
as
we
go
on
I
put
a
few
of
the
usual
suspects
up
there,
but
we're
gonna
see
some
other
things
as
well.
Who
am
I.
Why
are
we
here?
Why
are
we
sitting
in
this
room?
Why
are
you
bothering
to
listen
to
me
and
what
is
the
ultimate
fate
that
we're
all
bound
for?
A
A
There's
work
for
those
places
before
I
started
with
kubernetes
in
early
2015,
so
I'm
at
least
as
new
as
most
of
you
are,
and
I've
been
working
with
it
full-time
since
late
2015
when
I
joined
core
OS
currently
work
at
mesosphere,
we
are
hiring
come
see
me
come
to
our
booth,
pronouns,
he
and
him
blood
type.
Caffeine,
positive,
that's
probably
the
most
important
thing
on
this
slide
and
I
have
some
contact
info
there
and
that
great
big,
ugly
QR
code
is
that
same
contact
info
plus
the
slide
URL.
A
So
I'm
gonna
leave
this
up
for
a
second.
If
anybody
wants
to
grab
that
because
I
apologize
in
advance,
some
of
the
command
line
text
on
these
slides
is
a
little
bit
hard
to
read
from
way
in
the
back
of
the
room,
because
they're
intended
to
be
read
and
kind
of
referred
to,
rather
than
just
be
pretty
slides.
A
A
Why
low-level
tools
still
matter
in
this
world,
where
we
have
all
these
nifty
new
cloud
native
tools
that
do
all
kinds
of
nifty
new
cloud
native
things?
Sometimes
your
new
tools
don't
go
in
place
smoothly.
Sometimes
they
break
that's
just
the
way.
Things
are
other
reasons
they
might
matter
in
your
day-to-day
operations.
They're
already
there.
You
already
know
how
to
use
them.
A
You
just
have
to
pick
them
up
and
put
them
to
work
and
you'll,
probably
get
further
with
tools
that
you're
experienced
with
all
other
things
being
equal,
then
something
that
you're
trying
to
learn
how
to
use
for
the
first
time
in
an
emergency.
I,
don't
know
about
anybody
else,
but
I
have
been
the
guy
in
the
data
center
and
somebody
says:
oh
just
run
the
debug
tool
that
we
wrote
for
that
and
I'm
like
okay.
How
what
do
I
need
to
do?
I
tried
to
run
it,
it
didn't
work.
A
What
arguments
do
I
need
so
we're
gonna
go
through.
This
is
probably
the
meat
of
the
talk.
This
is
the
basic
tools
that
we're
gonna
go
over
and
then
we're
gonna
see
some
applications
of
those
tools
and
we'll
talk
about.
You
know
some
of
the
ways
that
you
can
get
those
tools
in
where
you
need
them
to
be
in.
A
Everybody's
seen
coop
control,
I
hope
if
you
haven't
well
that's
why
you're
here
right,
so
you
can
get
familiar
with
that
kind
of
thing.
Some
of
the
lesser-known
cube
control
things
that
you
can
do.
You
can
do
a
coop
siku
control
attach
you
watch
the
standard
output
of
a
container
in
real
time.
If
you
read
the
documentation,
it
will
say
that
you
can
actually
send
signals
to
the
process.
I've,
never
gotten
that
to
work
and
I
was
talking
to
one
of
the
kubernetes
maintainer.
A
Yesterday,
a
contributor
summit
and
I
said,
and
he
was
one
of
the
CLI
people
and
I
said
hey
what
about
this?
I've
never
been
able
to
get
this
to
work.
Anyone,
oh
yeah,
I,
never
have
either
so
I
didn't
feel
so
bad
anymore,
but
I
mean
basically,
so
you
know
what
we're
looking
at
here
is
I
just
created
a
container
and
all
it
does
is.
It
continuously
writes
random
data
to
standard
out
and
also
tease
it
to
a
file.
A
So
speaking
of
things
going
to
a
file,
if
you
read
the
12
factor
manifesto,
it
says:
do
not
log
to
a
file
log
to
standard
out
and
let
the
environment
pick
it
up.
Guess
what
a
lot
of
applications
do
anyway.
So
if
you
want
to
get
that
log
file,
you
have
to
break
into
the
container
and
grab
it
somehow.
Well,
one
of
the
easy
ways
to
do
it
is
cube.
Ct,
Cube
control,
CP,
you
can
just
copy
a
file
out
of
the
container.
You
can
put
one
into
the
container
it'll.
A
Actually
let
you
do
some
some
more
interesting
things,
but
the
basic
usage
is
just
this.
It
looks
a
lot
like
copying
to
an
NFS
share
if
you've
ever
done
that
before
I'm
sure
some
of
you
have,
you
can
find
out
more
about
what's
going
on
than
you
ever
wanted
to
know.
By
throwing
a
couple
of
V
arguments
at
it,
you
can
just
give
it
a
Kubb
CTL
get
nodes
gives
you
the
nice
neat
little
table.
If
we
say
cube,
CTL
V
v6
get
nodes,
we
get
a
little
more
output.
A
If
we
say
cube,
CTL
V
99
get
nodes.
We
get
a
lot
more
output,
I
actually
clip
this
off,
because
it's
just
pages
and
pages
and
pages
of
config
loaded
from
cash
and
config
loaded
from
this
file
in
that
file,
and
then
it
gets
down
to
this
here,
and
this
is
the
part
that
I
really
wanted
to
show
you
is.
It
actually
shows
you
right
here.
This
is
the
curl
command
that
you
can
run.
They
would
do
the
same
thing,
but
it
just
did,
and
then
it
tells
you
here.
A
This
is
the
response
that
it
got.
These
are
the
response
headers.
This
is
the
response
body.
It's
basically
every
single
piece
of
data
that
you
could
have,
except
for
the
raw
TCP
dump
packet
capture
from
this
particular
transaction.
Now,
in
reality,
there
aren't
99,
separate
levels
of
verbosity
I,
think
it
stops
around
8
or
9,
but
the
99
is
just
kind
of
the
the
community
convention.
4,
give
me
everything.
You
got
there's
a
couple
things
you
can
do
to
get
help
on
the
fly.
A
A
So
you
know
it
might
be
helpful
if
you
could
have
your
utilities,
explain
things
to
you,
of
course,
if
you're
in
the
data
center,
you
know
you're
trying
to
read
things
on
your
phone.
This
might
be
helpful
there.
Just
on
the
off-chance,
you
can
say,
explain,
CRT
and
it
gives
you
a
description
of
a
CRD,
a
custom
resource
definition,
and
it
gives
you
all
the
fields
in
a
brief
description
of
the
fields
and
if
you
say
well,
wait
a
minute.
I
want
more
detail
on
that
field.
A
A
There's
a
couple
of
links
down
there.
There
are
some
other
things
cube,
see
tail
completion.
You
can
actually
have
cube
control
output,
shell
completion
logic
for
your
shell,
if
you're
using
bash
or
zsh
off
can
I
literally
just
you
say,
cube
cube
control
off,
can
I
create
plot,
and
it
says
yes
or
it
says
no,
and
then
API
resources
gives
you
a
summary
of
all
the
API
resources.
There
aren't
really
arguments
or
options
for
that.
It's
basically
just
a
quick
reference
table
of
all
the
things
you
can
do
on
that
server
go.
A
So
if
you
have
multiple
clusters
with
multiple
API
versions,
you
can
actually
ask
it.
What's
valid
on
this
cluster
and
it'll.
Tell
you
output
control,
oh
yeah
Mel
gives
you
a
nice
and
manifest.
It
actually
gives
you
some
extra
stuff.
You
probably
have
never
created
a
Yambol
manifest
with
things
like
generation
or
creation.
Timestamp,
probably
well.
You
might
have
created
some
with
annotations
and
then
it
has
a
status
field.
But
the
point
is
that
you
get
options
for
how
you
are
given
the
description
of
the
workload
that
you're
looking
at.
You
can
do.
A
Json
and
it'll,
give
you
it
in
JSON
format
and
we're
actually
gonna
be
looking
a
lot
at
JSON.
Very,
very
shortly,
oh,
and
also
look
at
JSON
path,
support
as
well.
That
allows
you
to
do
some
pre
filtering
of
that
JSON
right
there
from
your
cube
control
command.
You
can
give
it
a
filter,
expression
and
it'll.
Just
give
you
back
the
bits
of
JSON
that
you're
actually
interested
in
so
then
we
turn
to
curl.
A
We
saw
earlier
that
Cube
control
will
tell
us
how
to
curl
a
transaction
to
duplicate
the
functionality
of
the
cube
control
utility.
Well,
one
of
the
things
you
have
to
do
to
actually
do
that
is
you
have
to
throw
a
header
in
there
if
you're,
using
bearer
tokens
or
something
to
say
this
is
my
authorization
token,
and
that's
how
you
do
it.
A
You
just
say
header:
it's
not
rocket
science,
there's
some
other
things
that
you'll
need
to
potentially
do
with
curl
and
we'll
get
to
those
and
some
of
the
things
you
have
to
pay
attention
to
are
what
kind
of
requests
are
you
making
with
curl
if
you're,
just
getting
info
on
a
resource
or
if
you're,
watching
or
listing
resources?
That's
a
get!
You
don't
need
to
specify
a
method
with
that.
That's
the
default
method,
then
you,
if
you're
doing
other
things,
you
need
to
use
a
different
submission
method.
A
A
This
is
just
get
me
a
pod
list,
and
you
know
this
is
a
scrolling
thing,
so
you
can
see
it
goes
all
the
way
down
to
the
bottom
and
then
I
cut
it
off,
because
you
get
the
point
at
that
point
and
then
I
put
links
down
here.
There
are
actual
detailed
guides
in
the
API
reference
for
write
operations
and
read
operations
on
pods
and
everything
else
too.
For
a
lot
of
these
things.
If
you
go
in
there
and
you
look
at
the
API
spec
for
it,
you
click
on
it.
A
A
Watching
for
changes,
so
this
is,
you
could
take
this.
You
can
watch
for
changes
to
resources.
This
is
basically
the
core
of
every
custom
resource
control.
You've
ever
seen,
they
put
a
watch
on
something
and
then,
as
data
comes
through,
they
act
on
it
and
if
the
watch
terminates
for
some
reason
the
connections
broken
or
something
they
pick
up
where
they
left
off
and
they
start
the
watch
again
until
you
terminate
that
resource
controller.
A
A
So
you
can
watch
pretty
much
any
API,
endpoint,
there's
also
a
watch
part
of
the
API
it
seems
like
the
convention
is
to
use
question.
Mark
watch
equals
true
at
the
end
of
the
endpoint,
rather
than
using
the
JP
I
specifically
use
whichever
one
works
for
you.
I
haven't
found
any
cases
where
they're
different
at
all.
The
other
thing
I
want
to
call
your
attention
to
here.
Is
we
have
this
bit
here?
So
it
tells
us
a
type
because
we're
looking
at
pods,
it
tells
us
that
are
no
excuse
me
we're
looking
at
nodes
here.
A
We
were
looking
at
pods
before
pods
in
the
cluster
I.
Did
this
on
don't
change.
Much
I
threw
an
example
pot
in
there,
and
so
it
said:
hey
there
is
a
pod,
so
it's
as
added
because
then
your
controller
would
go.
Oh
there's
a
pot
I
need
to
watch
it.
Do
something
with
nodes.
Are
a
little
more
active
nodes
actually
check
in
every
few
seconds
and
they
constantly
update
themselves.
So
what
you
get
in
this
three
node
cluster
is
right.
A
When
you
do
the
watch,
you
get
the
info
on
the
nodes
at
their
current
state
as
an
added
type.
Then,
as
they
start
updating,
you
start
getting
these
modified
types
down
here,
because
now
they're
updating
their
status,
every
5
10
seconds,
whatever
the
interval
is,
and
so
you
can
actually
watch
what's
going
on
in
your
cluster
nodes.
If
you
filter
that
information
down
you
can
see,
it
goes
all
the
way
off
to
the
right
here.
But
you
know.
A
A
You
can
run
a
bash
while
read
if
anybody's
ever
used,
read
line
or
or
other
read
operations
in
here,
you
can
have
it
reading
everything
that
comes
through
all
of
these
added
and
modified
events
that
are
coming
through
here
can
be
read
and
acted
upon
and
in
this
case,
I'm
piping
them
to
this
whole
long.
Jq
thing
here.
A
This
is
all
a
JQ
pipeline
will
talk
more
about
JQ
pipelines
in
a
little
bit.
What
I'm
doing
is
I'm
filtering
out
some
info
about
that
node
and
then
I'm
putting
it
into
a
formatter
I
want-
and
this
is
what
I
get
it
down
at
the
bottom.
It
actually
says:
here's
a
node.
This
is
the
node
name.
That's
all
I'm
interested
in
that
there's
another
thing
you
can
do
with
Bosch.
It's
called
a
co
proc.
A
A
So
OpenSSL
everybody
here
loves
open,
SSL
right,
it's
it's
the
most
user-friendly
tool
in
your
arsenal:
right
hands
up
user
friendly.
Okay,
some
certificates
verify
live
in
this
case.
What
I
did
was
I
literally
just
extracted
the
CA
certificate.
For
my
cluster
and
said
this
is
a
trusted
CA,
so
I
told
open
SSL
here.
There's
your
trusted.
Ca
now
verify
for
me
whether
this
client
cert
is
valid
and
it
came
back
and
said
that
client
cert
is
okay.
You
trust
the
CA
that
generated
it.
A
If
I
take
that
out,
that's
enough,
it's
a
self
generated
CA.
So
of
course
it's
not
trusted
by
anything
in
the
world,
but
this
is
the
kind
of
thing
that
open
SSL
throws
out
at
you.
If
you
have
a
problem
with
your
certificates,
you
probably
will
see
that
all
the
time,
if
you're
browsing
to
the
dashboard
or
something
on
a
kubernetes
cluster
and
your
cluster
certificate
was
self
generated,
Firefox
complains
that
you
Chrome
complains
at
you
I
think
Chrome.
Won't!
Even
let
you
go
there
anymore.
A
See
so
working
with
a
live
server,
so
that's
great
if
you
have
certificate
files
already
on
disk
that
you
can
just
read.
But
what,
if
you
need
to
verify
with
a
particular
server,
whether
it's
valid
or
not,
you
can
do
that
with
open,
SSL,
S
client.
You
tell
it
to
connect
to
a
server
and
it
connects
to
the
server
and
says
no.
That
certificate
is
not
valid.
I,
don't
trust
the
CA
that
issued
that
certificate
I'm
unable
to
verify
that,
but
then
it'll
give
you
the
certificate
chain
anyway.
A
A
They
are
useful
for
debugging
your
network
traffic
and
your
network
interfaces.
You
can
break
into
a
container
and
run
a
dig
and
say
well,
ok,
dns
resolution
isn't
working.
Why
isn't
it
working?
You
can
actually
recurse
your
way
up
the
dns
chain
and
pinpoint
where
that
is
failing,
you
can
do
a
TCP
dump.
You
can
say
well,
ok,
I
know
traffic
is
going
out
of
this
container.
Is
it
coming
into
this
container?
No,
it's
not
or
yes,
it
is.
If
it's
coming
in,
maybe
something
else
is
happening
to
it.
A
Netcat
you
can
test
bandwidth
between
two
systems,
two
containers,
whatever
you
can
see.
If
something
is
slowing,
you
down,
there
shouldn't
be
and
SS
cysts
at
which
is
the
new.
Instead
of
netstat
to
look
at
what's
listening
on
open
ports,
you
should
be
using
SS
instead
of
netstat
these
days,
and
so
that's
one
of
the
things
that
I
wanted
to
note.
There
was
you
know:
some
of
these
tools
have
been.
A
The
traditional
tools
have
been
deprecated
like
I
have
config,
you
should
not
be
using
ifconfig
anymore,
not
even
for
old-style
debugging
of
regular
bare
metal
systems
should
be
using
IP
at
or
show
IP
route
show
that
kind
of
thing.
The
IP
route
to
tools
set
and
that
stat
has
been
replaced
by
SS.
The
other
thing
is
be
aware
of
dependencies
of
these
things.
A
lot
of
the
time
you
can't
just
copy
the
binary
and
expect
it
to
work.
There's
gonna
be
libraries.
A
It
depends
on
in
particular,
anything
that
involves
DNS
is
going
to
work
differently
in
a
muscle
based
container
that
doesn't
have
the
traditional
Lib
C
in
it.
So
if
you're
running
alpine
and
Digg
is
acting
funny,
that
may
be
why
it's
not
that
it's
broken
it
just
acts
differently.
It
is
very
strictly
within
the
spec
for
Lib
C.
A
So
now
we
come
to
JQ.
We've
already
seen
that
Jake,
you
can
filter
your
JSON
for
you,
but
you
know:
what
can
you
do
with
it?
Basic
usage,
you
pipe
some
JSON
content
at
it.
You
give
it
some
flags,
you
give
it
a
filter,
expression
and
it
spits
out
put
out
the
other
end,
easy
filters,
the
dot
filter,
you
say,
JQ
dot
and
it
doesn't
filter
it
at
all
asterisk.
Why
did
I
put
asterisk
there,
because
the
JSON
spec
actually
says
that
beyond
a
certain
precision,
you
are
not
required
to
faithfully
reproduce
numbers.
A
So
if
you
feed
something
in
that
JQ
recognizes
as
an
integer
that
is
of
a
higher
precision
than
it
can
handle,
it
will
smash
it
down
to
lower
precision
and
there
is
not
a
darn
thing
you
can
do
about
it
and,
yes,
the
bug
has
been
filed
and
the
maintainer
said
it's
allowed
by
the
spec,
so
I'm
not
going
to
change
it.
You
won't
probably
run
into
that
too
often
the
one
place
that
I've
run
into
it.
A
A
lot,
though,
is
if
you're
piping
out
put
out
of
etsy
D,
because
an
Etsy
denote
ID
is
at
U
n
64
and
that's
a
higher
precision
than
the
JSON
spec
handles,
and
so,
if
you
pipe
at
C
D
JSON
cluster
info
into
JQ,
the
note
IDs
that
you
get
out
are
not
the
actual
IDs
of
the
nodes
because
of
that
precision
issue.
But
you
can
see
here
what
I
did
was.
I
just
took
this
long
single
line,
JSON
expression,
which
is
very
hard
to
read
and
I
piped.
A
It
through
JQ
dot,
+
JQ,
gave
me
this
nice
little
key
tree
here,
so
you
can
see.
What's
going
on,
I
can
do
a
little
more
than
that
I
can
say
dot
key
and
then
what
it
says
is:
oh
he's
just
interested
in
this
here.
So
it
gives
me
just
the
value
of
key
in
that
tree.
I
can
say:
hey,
there's
an
array
there
I
want
the
second
item
in
that
array
there
zero
based
arrays,
so
if
I
say
dot
key
for
index
one.
A
A
The
other
thing
is
I
didn't
want
to.
I
can
sit
here
and
talk
all
day
about
JQ,
there's
a
JQ
cookbook
and
a
JQ
manual.
You
should
definitely
go
read
those
the
cookbook,
especially.
The
other
thing
is,
if
you're
having
trouble
with
something
and
you
feel
like
you're
almost
there,
but
you
can't
quite
get
it.
There's
a
Stack
Overflow
community
around
JQ.
A
They
do
ask
that
if
it's
a
problem
with
using
JQ
and
not
an
actual
bug
that
you
put
it
in
Stack,
Overflow
and
tag
it
rather
than
filing
a
github
issue,
scripting
beyond
one-liners,
you
can
create
a
script
file.
Just
like
you
can
create
a
bash
script
file
and
for
most
of
the
same
reasons,
it's
more
readable.
That
way
it
helps
mitigate
some
quoting
issues.
A
One
of
the
things
you
may
notice,
if
you
go
back
and
look
at
this
example-
is
I-
have
single
quotes
around
this
whole
filter
expression,
and
that's
because
you
know,
if
you
don't
put
single
quotes
around
it.
Sometimes
your
content
contains
like
dollar
signs
and
they
try
to
get
interpreted
variables
and
she'll
meta
characters
and
it
just
becomes
a
big
mess.
So
putting
your
JQ
filter
into
a
script
file
can
help
mitigate
that.
So
here,
I
just
created
a
very
simple
script
at
JQ.
That
was
essentially
the
exact
same
thing
that
I
had
here.
A
A
Let's
see
where
are
we
on
time?
I
think
we've
got
time.
Your
bash
prompt
is
actually
an
important
debugging
tool
and
can
help
keep
you
out
of
trouble.
So
one
of
the
things
that
we've
all
gotten
used
to
is
if
you're
running,
is
root.
Your
prompt
change
to
a
hash
mark
from
a
dollar
sign,
or
is
that
you
normally
use
so
a
visual
reminder
which
cluster
you're
working
in
and
what
context
you're
working
in
can
be
useful
and
so
I
actually
put
a
little
example
here
of
what
you
could
do
in
your
bash
profile.
A
This
is
a
command
I
just
found
out
about
a
few
months
ago.
You
can
define
a
prompt
set
command
and
then
it
will
actually
create.
You
know
you
can
put
whatever
you
want
in
there
and
then,
when
you
run
that
through
your
bash
profile,
when
your
bash
profile
gets
sourced
on
login
now,
anytime
I
have
coop
config
set
I
get
this
up
there.
It
tells
me
hey
you're
in
this
Cates
context,
admin
at
local,
so
that's
kind
of
a
warning
to
me:
I
could
even
put
like
colors
and
highlighting
and
bold
and
stuff
in
there.
A
A
If
you've
ever
done
this
with
docker,
you
can
easily
plug
one
container
into
another
containers
in
network
namespace
with
this
and
that
equals
container.
You
just
give
it
the
container
ID.
It's
basically
the
same
way
as
when
you
do
net
equals
host,
and
then
what
that
allows
you
to
do
is
in
this
case,
I
just
ran
Fedora
on
the
same
container
in
the
same
network
namespaces.
The
container
I
started
way
back
at
the
beginning
of
this
talk,
that's
just
outputting
random
data
and
then
because
it's
a
fedora
container
I
don't
have
IP
route.
A
Yet
so
I
install
that
stuff
and
now,
when
I
do
IPA
I
can
actually
see
my
container
network
interface
there.
So
what?
If
you're,
not
using
docker
containers?
What,
if
you're
one
of
the
one
of
the
fortunate
people
out
there
who's
using
something
else,
I
don't
want
to
bust
on
docker
too
bad.
They
just
they're
so
dominant
that
it
makes
me
happy
whenever
I
see
anybody
who
said
well,
we
run
rocket
cuz
I
used
to
work
at
core
OS.
A
You
can
still
get
into
that
container,
because
what
you
can
do
is,
in
this
case
I'm
saying
well.
Where
is
that
container
running?
What
node
is
it
running
on?
So
that's
the
note
it's
running
on
I
do
get
pods,
Oh
wine
and
it
told
me
the
node
now
I
go
on
that
No
and
I
say:
okay,
what's
the
process
ID?
Well,
this
is
the
process.
This
is
the
process
ID.
A
What
can
I
do
with
that
process?
Id
I
can
NS
enter
that
process.
Id
and
say
put
me
in
that
processes
namespace
and
if
I
don't
give
it
something
to
run,
it
just
runs
a
shell,
and
so
in
this
case
it
puts
me
in
a
shell
in
that
process.
Namespace
and
you
can
see
I'm
looking
at
the
same
container
interface
without
needing
to
do
a
docker
attached
to
it.
So.
A
If
you're
gonna
run
a
debug
container
like
that
which
one
should
you
run,
anybody
want
to
take
a
guess
which
container
I
recommend
you
run
come
on.
Somebody
somebody
throw
something
good.
Yes,
it's
really
up
to
you,
I,
don't
care
which
one
you
run
I
want
you
to
figure
out
which
one
you
should
run
and
it
needs
to
have
all
the
tools
you
need
in
it.
A
It
could
be
a
standard
distro
container
if
you've
ever
used
core
OSS
toolbox,
utility
on
container
or
Linux
all
toolbox
does
is
download
an
extractor
for
the
or
a
container
and
run
it
as
a
privileged
container.
And
that
way
you
can
yum
install
all
those
tools
or
actually
I
should
say,
DNF
install
all
those
tools
that
core
OS
doesn't
have
to
debug
your
core
OS
host
I,
remember:
I,
used
to
teach
classes
for
them
and
I
taught
a
class
once
and
I
had
a
guy
in
the
front
row
and
he
went
oh
my
god.
A
If
you
aren't
building
your
own
containers,
I
recommend
that
you
do
it
occasionally,
don't
just
say:
okay,
we're
gonna
use
a
district
container
and
that'd
be
the
end
of
the
conversation,
because
this
show
containers
tend
to
be
pretty
large
and
I.
Want
you
to
get
a
sense
of
do
we
really
need
to
be
pulling
in
600
Meg's
of
distro
container
every
time
we
need
to
debug
something?
Maybe
you
only
need,
let's
say
a
hundred
Meg's
or
50
Meg's.
A
What
should
be
in
your
debug
container,
all
your
standard
system
tools
that
you
need
to
debug
your
hosts,
and
that
includes
potentially
things
like
if
you
have
weird
networking
gear,
weird
storage
gear,
whatever
it
is
filtering
and
parsing
tool,
we
already
saw
JQ.
You
know
anything
else
that
you
feel
comfortable
with
threw
a
pearl
in
there.
If
you
like,
pearl,
it's
been
a
while
since
I
did
any
of
it
myself
but
yeah,
throw
it
in
there
any
tools
specific
to
your
apps.
A
Demos,
this
is
always
the
fun
part
right.
So
what
I'm
gonna
do
is
I'm
actually
gonna
play
you,
because
it's
all
command
line
based
anyway
and
there's
really
no
reason
to
do
it
live,
but
I
can.
If
you
really
want
to
see
it.
This
is
something
so.
The
story
of
this
demo
is
I
was
gonna
demonstrate
using
curl
to
access
the
kubernetes
api
and
it
wasn't
working.
I
was
roughing
it
out
on
play
with
Kate's
hands
up
everybody.
Who's
used
play
with
Kate's
in
here.
A
Okay,
I
see
a
couple
of
you
there
yeah,
so
I
was
roughing
it
down
and
play
with
Kate's,
and
it
wasn't
working
and
I
was
like
wow
I
extracted
the
certificate,
and
it's
not
going
what's
going
on
so
that
debugging
session
actually
became
the
demo.
So
we
begin
where
I
was
I
was
sitting
on
the
command
line.
I
was
rude,
I
mean
I
should
be
able
to
do
anything
right,
I,
reinstall,
open
SSL.
You
get
to
see
me
fumble
around
on
the
command
line
for
a
little
bit,
JQ.
A
You
know
this
is
this:
is
one
I
think
everything's
gonna
go
great
I'm
just
gonna.
Do
this,
it's
gonna
work,
I'm,
gonna
copy
and
paste
that
into
my
slides
and
and
we're
gonna
be
off
and
running,
okay,
that
I
expected
because
I
didn't
feed
any
certificate
or
token
or
anything
in
there.
So
the
kubernetes
api
server
shouldn't
tell
me
I'm,
gonna
authorised
so
I
said:
okay,
let's
extract
all
the
certificate
info
this
by
the
way
cube
control.
Config.
A
Do
you
minify
extracts
just
the
info
about
your
current
context
from
your
cube,
config
flattened
preps
it
to
be
actually
bundled
up
and
transport
it
somewhere
else,
so
you
can
see
it
put
client
key
data
and
all
that
stuff
that
it
normally
read
acts
there.
So
if
you
really
wanted
to,
you
could
sit
here
and
copy
all
this,
but
this
cluster
is
not
gonna
exist,
five
minutes
after
I
finished,
so
don't
waste
your
time,
okay,
so
pipe
it
through
JSON.
A
A
Yes,
it's
it's
what
it
should
be,
it's
been
issued
by
who
you
think
it's
issued.
It
says
that
you
are
system
masters
and
user
admin.
So
what's
going
on
so
I
said:
well,
let's
throw
a
curl
VV
in
there
and
that's
when
I
saw
this
and
I
say
like.
If
you
heard
a
weird
sound
about
a
week
ago,
it
was
my
forehead
slamming
the
table
in
my
office
repeatedly,
because
I
went
why.
A
Why
would
you
do
that
current
versions
of
curl
don't
do
this
by
the
way,
because
I
actually
went
to
file
the
bug
and
I
said
well,
let
me
reproduce
on
my
the
door
a
laptop
first
and
it
didn't
do
it
and
I
said
okay
now,
obviously
they
they
got
complaints
and
they
fixed
it.
It
didn't
by
the
way,
do
that
with
this
client
key.
It
didn't
do
it
with
the
CA
certificate.
It
only
did
this
weird
thing
with
the
client
certificate,
but
at
least
it
tells
me
what
to
do
there.
A
A
And
there
it
is
so
I
finally
got
authorized
to
the
API
server
and
I
got
my
node
names.
Second
demo.
If
I
have
time,
I
do
have
a
little
bit
of
time.
Second
demo
is
based
on
a
scenario,
so
we
have
a
cluster
we've
been
managing
for
a
while.
We've
got
an
employee
named
Jim,
hey
Jim's
in
here,
maybe
a
couple
that
don't
want
to
raise
their
hands
because
they
know
I'm
about
to
say
something.
A
Bad
we've
got
this
employee
Jim
and
he's
been
an
admin
on
our
cluster
for
a
while
and
Jim
just
kind
of
up
and
left
under
bad
circumstances.
So
we
need
to
figure
out
exactly
what
we
need
to
remove
Jim
from
because
we
haven't
been
doing
our
management
like
we
should.
So
how
do
we
extract
out
from
our
cluster
what
cluster
roles
is
Jim
bound
to?
So
we
can
do
a
get
cluster
role,
bindings,
Oh
JSON,
and
it
throws
this
big
blob
of
data
at
us.
I.
A
Don't
want
to
sit
here
and
read
all
this
I'm
gonna
filter
it
down
with
JQ,
and
so
you
can
see
what
I
did
there.
If
I
can
scroll
back
up
a
minute
where
they
go
well,
I
did
a
basic
filter
there,
we'll
just
take
that
as
read
okay,
so
you
can
kind
of
see
right
there
that
I
filtered
it
down
and
said.
Tell
me
first
of
all,
all
of
the
cluster
role
bindings
that
have
subjects
because
a
lot
of
them
don't
wear.
One
of
these
subjects.
A
So
with
that
final
thoughts,
I
always
like
to
throw
a
few
in
there,
don't
panic!
That's
the
thing.
I
always
tell
people
I
know
it
always
feels
stressful.
When
you're
in
there
and
something's
gone
wrong
and
and
somebody's
looking
at
even
going
find
out
the
answer
fix
it
fix
it
fix
it
find
out,
know
your
tools.
If
you
know
your
tools-
and
you
know,
what's
underneath
your
tools,
you
can
keep
that
panic
reaction
down.
We're
not
doing
anything
new
here.
A
Containers
are
just
Linux
I
think
somebody
actually
has
that
on
their
sponsor
booth
containers
are
Linux,
I
saw
a
guy
walking
around
with
that
t-shirt,
I
think
it
was
Red
Hat.
Thank
you
very
much.
This
is
a
QR
code
that
links
directly
to
the
slides.
That's
just
all
that
it
is,
is
the
slide
link.
Anybody
have
any
questions
before
we
have
to
get
out
of
here.