►
Description
Come hang out with Duffie Cooley and Ian Coldwater as they do a bit of hands on hacking of Kubernetes and security related topics. Some of this will be them talking about the things they know. Some of this will be exploring something new with the audience. Come join the fun, ask questions, comment, and participate in the live chat!
A
So
we're
all
glad
we're
both
glad
to
be
here,
and
this
one
is
really
we're.
Just
gonna
talk
a
little
bit
about
like
what
we're
gonna,
what's
kind
of
on
our
mind,
what's
happening
with
what's
happening
with
kubecon
and
everything
else
that's
happening
this
week.
We'll
do
some
hacking
on
some
of
the
great
exploits,
or
I
don't
know
what
you
want
to
call
them
some
of
the
really
fun
stuff
that
ian
and
brad
got
to
present
this
year
at
cubecon
2020.!
A
B
A
A
We
got
george
and
josh
on
backup,
so
they're
both
going
to
be
helping
us
out
in
the
chat,
and
that's
really
awesome.
Thank
you.
Both
for
being
here,
we
got
martin
borgman
from
the
netherlands
and
bowdoin
saying
hello,
rory
mccune
from
luck
oil
head.
I
don't
know
why.
I
keep
trying
to
say
that
word,
but
you
know
it
is
what
it
is
david
cheney.
A
I
wonder
if
that's
the
dave
cheney
or
if
it's
a
different
dave
cheney,
because
you
also
refer
to
the
bay
of
fire
unless
you're
talking
about
australia
and
jason
serrano
is
saying
hello
from
smoky
denver.
That's
where
josh
is
too
yes.
Definitely
records
from
germany
rodolfo
here
to
learn
awesome
good
to
see
you
josh
signing
in
got
victor
lee,
saying
honk
honk
good
to
see
you
victor
brandon
good,
to
see
brandon
and
romana
from
india
and
tim
downey
from
the
east
bay
yeah.
A
Two
people
of
note
for
certain
and
cory
quinn,
saying
hello
from
san
francisco
and
twitter
good
to
see
you
corey
and
alexander
brand,
checking
in
good
to
see
you
alex
alex
a
good
as
a
co-worker.
So
a
reminder
that
if
you
see
me
looking
down
it's
because
I
have
two
monitors,
there's
the
one
where
the
business
is
up
in
front
and
then
I
have
all
of
my
chat
and
everything
down
below.
So
that's
what
that's
about.
A
A
So
this
week's
notes,
as
usual,
are
up
at
tgik,
dot,
io,
slash
notes,
and
so,
if
you
want
to
keep
track
of
like
where
we're
at
with
it
or
you
want
to
see
links
to
something
that
is
being
shared
in
the
chat
or
in
this
talk.
This
is
where
you
can
go
to
see
that
stuff
core
kubernetes,
there's
a
great
blog
post,
oh
hold
on
that's
the
button.
I
wanted
to
push
there's
a
great
blog
post
on
the
kubernetes
blog
talking
about
moving
forward
from
beta.
This
has
been
a
pretty.
A
This
is
a
really
important
discussion
at
the
last
kubecon,
and
it
continues
at
this
kubecon
talking
about
how
we
we
really
don't
want
features
in
kubernetes
to
languish
in
beta
and
how
we
are
actually
managing
moving
those
things
forward.
So,
if
you're
interested
in
that
space
or
if
you're
curious
about
how
that's
working
out,
definitely
read
this
article,
it's
a
good
one
and
we're
seeing
things
like
ingress
go
ga
in
119.
A
there
is
a
webinar
coming
up
on
september,
25th,
where
you
can
check
in
and
hear
like.
What's
new
in
kubernetes
119
from
the
release
team
itself,
I've
been
a
lot
of
really
great
folks
working
on
kubernetes.
This
round,
so
definitely
check
it
out
and
we're
still
on
target.
I
think
for
like
august
25th
for
the
release
of
119.,
we
have
a
couple
of
articles
from
stock
rocks
and
cystic
talking
about
what's
new
in
119
and
so
lots
of
really
good
stuff.
A
I
actually
talked
a
little
bit
about
this
in
one
of
the
talks
that
I
did
at
kubecon
this
year.
Talking
about
how
setcomp
is
actually
becoming
a
first-class
thing,
it's
being
moved
into
the
actual
pod
spec,
rather
than
being
used
as
an
annotation,
which
I
think
is
really
pretty
exciting
anything
exciting
for
you
in
119
ian
or
have
you
had
a
chance
to
even
look
at
it.
B
B
A
Yeah
there
was
a
long
time
when
I
think
it's
actually
in
118
that
a
lot
of
the
a
lot
of
this
got
deprecated,
but
for
a
long
time,
cube
got
all
run,
was
very
equivalent
to
docker
run
and
you
could
like
get
away
with
almost
everything
that
you
could
do
with
docker
run
with
cube
kettle
run,
and
it
would
just
create
a
pod
for
you
in
kubernetes,
rather
than
creating
a
you
know,
a
container
as
it
were.
So,
if
you're
interested
in
getting
a
little
preview
into
what's
happening
in
keyboard
19.
A
A
B
A
B
Coupon
virtual
was
a
little
bit
surreal
for
me
this
year
there
has
been
a
lot
happening
where
I
live.
That
has
very
much
been
reality
kind
of
getting
in
the
middle
of
everything,
so
that
was
it
was
an
interesting
experience
in
a
couple
of
ways.
I
you
know
it
was
really
like.
There
were
really
good
talks
that
I
was
really
excited
about.
B
It
was
really
good
to
get
to
see
all
of
the
people
on
slack
and
in
one
place
it
didn't
really
feel
quite
the
same
as
regular
kubecon,
but
I
know
that
cncf
and
the
lf
events
team
worked
really
hard
to
like
make
it
as
good
experience
for
everybody
as
they
could
and
I
feel
really
grateful
to
them
for
putting
in
all
of
that
work
and
for
making
an
experience
for
us
that
was
pretty
sweet.
I
liked
your
talk
a
lot.
It
was
really
good.
Yeah.
A
A
A
B
A
A
For
ebpf-
and
I
think
I
know
you've
heard
me
talk
about
this
before
on
the
show,
but
eppf
is
crazy,
super
powers
and
it
covers
a
huge
swath
of
surface
area
like
there's.
There
are
people
using
evpf
and
xtp
for
network
capability
and
that's
kind
of
where
psyllium
fits
into
it.
There
are
people
using
it
for
security
for
tracing
and
profiling,
observability
and
monitoring.
A
There's
just
it's
a
it's
a
very,
very
large
thing
to
fit
in
one's
head
and
so
definitely
worth
kind
of
breaking
it
down
in
chunks
and
taking
a
look
at
like
maybe
some
of
the
projects
that
implement
parts
of
it
and
digging
into
what's
happening
there.
But
this
is
a
great
resource
for
that
and
it's
a
growing
resource
and-
and
I
think
that
they
link
to
the
they
link
to
it.
A
A
A
Another
article,
one
of
the
many
speakers
I
got
to
see
this
year
at
kubecon,
was
liz
rice
and
I
really
enjoy
watching
liz
speak
on
any
number
of
different
topics,
including
this
one,
which
was
an
interview
with
the
cube
around
cncf
and
how
it
helps
and
and
some
of
the
changes
that
have
had
come
about
in
the
way
that
cncf
onboards
projects
and
manages
that
sort
of
stuff.
One
of
the
big
takeaways
for
me,
which
I
thought
was
interesting,
was
that
moving
something
into
sandbox
has
become.
A
K3S
is
now
a
cncf
sandbox
project
which
is
pretty
interesting.
K3S
is
a
is
a
take
on
kubernetes.
That
is
really,
I
think.
In
my
personal
opinion,
it's
a
really
great
one
for
edge
deployments
and
those
sorts
of
things
it's
really
looking
at.
How
can
we
take
some
of
the
model
of
kubernetes
and,
like
figure
out
how
to
support
those
things
in
resource-constrained
environments,
and
it
made
it
an
incredible
showing
in
your
talk.
B
Yes,
k3s
is:
I
want
to
make
clear
that
what
we
said
about
it
in
the
talk
wasn't
a
dunk
on
k3s
as
a
project
or
it's
security.
It's
a
really
useful
project
that
allows
for
people
to
use
kubernetes
in,
like
really
resource-constrained
environments
in
an
interesting
way
and
the
way
that
it
was
designed
for
people
to
do
that
also
made
it
interesting
for
us
to
be
able
to
use
it
for
perhaps
unintended
purposes.
A
A
A
Yeah,
I
think
it
cleared
up
pretty
quickly
last
time,
we'll
see
we'll
see
how
it
shakes
out
this
time
so,
but
with
cn
with
with
k3s
it's
a
different
gig
right
with
k3s.
What
ends
up
happening
is
that
the
k3s
node
establishes
a
tls
connection
with
the
api
server
and
k3s
is
actually
managing
connections
back
through
that
tls
connection,
when
it
wants
to
establish
connectivity
to
the
node,
which
I
think
is
actually
pretty
interesting
way
of
handling
it.
A
A
A
Interact
with
those
resources
that
are
created
by
kubernetes
and
it's
a
really
interesting
project
by,
I
think
chris
hein
instituted
it
originally
as
a
project,
but
lots
more
effort
has
been
put
in
here.
If
this
is
something
that
you're
interested
in
exploring
definitely
check.
That
out
looks
like
a
fun
one
to
extend
kind
of
using
the
patterns
that
kubernetes
expresses
to
extend
forward.
A
Some
of
the
resources
that
are
available
within
aws,
the
last
one
I
have
for
a
community
is
carvel,
which
is
a
rename
of
a
thing
called
k14s,
which
is
a
bunch
of
different
tools
that
were
developed.
I
think
mostly
at
pivotal,
but
these
are
being
re-kind
of
repackaged
in
this
new
in
this
new
project
carvel.dev,
and
it
includes
things
like
ytt
cap.
I
think
that
joe
has
actually
presented
on
are
presented
with
cap
and
ytt
a
few
times
in
the
past.
A
I
was
watching
lachlan
everson's
thread
on
on
watching
him
watch
the
talk
which,
if
you
have
a
moment
after
this
episode,
definitely
go
check
it
out.
It's
a
very
entertaining
thread.
Lucky
is
a
is
an
incredibly
innately
funny
person,
so
he
he
like.
He
does
a
very
dramatic
reading
of
what's
happening.
There.
B
I
really
appreciated
that
thread.
We
got
brad
and
I
got
to
give
this
talk
one
other
time
at
rsa
conference
and
and
I'm
not
sad
that
we
got
to
do
that,
because
it
was
like
the
last
place
that
anybody
was
able
to
give
talks
in
real
life
anywhere,
but
but
the
audience
there
was
not
very
familiar,
I
think
largely
with
kubernetes
internals,
and
so
we
had
been
really
looking
forward
to
being
able
to
give
this
talk
in
front
of
an
audience
who
would
be
able
to
like
crop
what
they
were.
B
A
A
A
Yep
rodolfo
100,
plus
I'm
not
sure
what
that
symbol
is,
and
then
we
have
bajan
saying
extended
kubecon
friday
hosted
by
duffy.
It's
true,
I'm
excited
to
be
here.
I
was
playing
with
coins.
Taylor.
Thank
you.
Taylor
is
actually
the
best
recruiter.
I
think
I've
ever
met
in
my
life.
Taylor's
awesome.
Do
you
know
taylor
martin?
I
think
you've
met
with
him
before
corey.
I
bet
ian's
favorite
kubecon
part
was.
B
A
B
A
A
A
Like
you,
your
talk
was
already
pre-recorded
for
me
anyways
it
was,
it
was
lower,
but
it
was
also
kind
of
weird
because,
like
I
had
all
it
was
like
you
know,
I
don't
know
if
everybody
has
this
but
like
when
I
say
stuff
like
I'm
playing
that
back
in
my
head,
you
know
and
then
like
later
on
I'll,
be
like
dang
it.
Why
did
I
say
that
that
way-
or
you
know
those
sorts
of
things
right
like
it
always
comes
from
me,
and
so
it
was
amazing.
B
B
At
the
same
time
as
listening
to
myself
do
the
talk
was
a
very
different
experience
than
giving
a
talk
in
real
life.
It
was
a
lot
of
information
coming
in.
It
was
cool
to
be
able
to
like
see.
People
react
in
real
time
in
the
different
streams.
I
think
we
didn't
figure
out
a
ideal
way
to
be
able
to
answer
the
questions
that
were
coming
in
it
was.
It
was
a
lot
to
look
at,
but
it
it
was
it
was.
It
was
interesting.
It's
different.
A
Yeah
that
makes
sense
yeah
I
can
see.
I
can
see
how
that
would
be
challenging,
especially
if
you
I
mean
you
know
like
from
the
focus
perspective
just
so
much
going
on,
and
you
know
yeah
and
in
person
you're,
just
you're
able
to
like
really
just
completely
focus
on
that
one
thing.
Whereas
this
is
like
you
know,
you
have
noise
coming
from
everywhere
suddenly,
and
that
would
be
a
lot.
B
A
A
A
A
All
right,
so
what
we're
going
to
play
with
in
the
demonstration
stuff
is
brad
was
able
to
actually
open
source
the
demonstration
files
for
what's
in
here,
although
there's
no
license
yet
so
I'll
bug
him
about
that.
But
what's
in
here
are
actually
the
demo
files
that
were
that
we
were
going
to
explore.
A
This
actually
also
links
to
the
talk.
So
once
those
things
are
available
in
youtube,
you
should
definitely
check
out
this
talk.
It's
a
really
good
one,
and
I
can
tell
that,
like
you
know
again,
the
two
of
you
have
really
practiced
this
like.
I
could
really
tell
that
you
had
really
spent
a
lot
of
time
on
this,
which
was
really
exciting,
so
I've
gone
ahead
and
checked
this
out
and
then
let's
go
ahead
and
brad.
B
A
B
B
A
Really
fun,
stephen
agreeing
that
mr
taylor
is
the
best
josh
saying
hello,
to
maya
good,
to
see
you,
my
god,
glad
that
you're
here
we
got
honk
erino
saying
welcome
good
to
have
you
in
the
community
and
pedro
costa,
saying
awesome,
presentation,
ian
brad
and
duffy
and
sebby
saying
welcome
again
and
multitasking
qa
during
my
own
talks
for
pre-record
absolutely
worked
well
for
my
adhd
says
corey.
If
that's
a
useful
data
point
fair
enough.
A
Walid's
saying
the
black
hat
and
joe's
saying
sure
talks
can
be
really
challenging
it.
They
can
be
especially,
but
I
think
that
you
know
we've
got
a
we've
got.
You
know
it
really.
I
think
it's
like
it's
a
chemistry
thing
like
if
you,
if
you
all,
would
be
friends,
then
it
seems
like
it's
going
to
work
out
a
little
bit
better
anyway,
whereas,
like
some
people,
I
I've
presented
I've
co-presented
with
I'm,
like
you
know
where
you
just
can't
like
it.
A
B
B
Where
you
know
like
and
and
ideally-
and
I
think
this
has
been
the
case
with
you-
and
I
definitely
and
I
think
it's
the
case
with
brad
and
me
too
different
people's
strengths
can
really
like
help
play
each
other
up.
So,
like
you
know,
one
person
maybe
is
really
inclined
to
do
really
intense
technical,
deep
dives
and
one
person
is
inclined
to
like
really
try
to
break
things
down
into
simple
terms.
B
A
I
completely
agree
that
is
100.
That
was
absolutely
my
take
away
from
our
experience,
which
was
that,
like
you
know
frequently
I
would
be
like,
then
we
could
do
this
thing
and
we
could
do
that
thing
and
and
ian
and
ian
would
say
but
or
maybe
that
would
be
too
much
detail.
Maybe
we
should
you
know
like
what
does
that
actually
mean
like
and
and
really
calling
me
out
on
that
stuff,
which
I
thought
was
critical
to
ourselves?
I
would
be.
B
Like
oh
well,
we
could
just
say
this
and
duffy
was
like.
I
think
that
that
would
work
out
better
if
we
explained
it
in
this
way
and
I
would
be
like
yeah
and
it
sure
worked
out
great.
So
I
don't
know
I
recommend
co-presenting
if,
if
doing
things
with
your
friends
and
working
things
out
in
that
way
and
helping
each
other
level
up
sounds
appealing
to
you,
it's
something
I
like
doing
a
lot.
B
B
B
That
is
like
a
kind
of
fun
way
to
like
just
kind
of
ease
into
thinking.
That
way,
so
I
brought
it
up
on
stage
at
qcon
and
now
we
all
honk
at
things
I
think
generally
honk
is
sort
of
known
as
a
celebratory
expression
of
chaos
and
the
goose
is
canonically,
I
think
chaotic
evil.
But
I
think
generally,
we
describe
the
goose
as
being
chaotic,
good
ish.
You
know-
and
you
know
basically,
like
you
might
maybe
chaotic
neutral
like
watch
out,
you
might
get
a
goose
in
your
cluster.
A
B
A
B
A
A
We're
going
to
set
up
the
cluster
and
we're
going
to
explore,
maybe
like
a
couple
of
things
so
there's
like
validating
web
honks,
there's
the
shadow
api
server,
there's
k3s
c2,
which
I
probably
aren't-
I'm
not
going
to
really
spend
the
the
time
to
stand
up
a
bunch
of
different
clusters
to
do
the
c2
thing
per
se,
but
like
we
will
probably
explore
one
of
the
others
right
validating
web
hook,
shadow
api
server
cubelet
exploit.
I
think
it's
really.
A
B
A
A
A
B
I
think
when
we
talked
about
this
before,
if
I
could
break
the
fourth
wall
for
a
minute,
I
think
you
were
excited
about
doing
shadow
api
and
the
cubelet
exploit.
One
was
my
baby
and
the
one
that
I'm
particularly
excited
about,
but
that
one
is
the
is
both
less
elaborate
and
shorter,
so,
whichever
one
you
want
to
do,
first,
maybe.
A
So
using
demo
magic,
I
really
like
demo
magic.
It's
a
great
tool
for
this
sort
of
thing
basically
gives
you
the
ability
to
like
describe
like
what
the
steps
are
and
what
you
want
things
to
happen.
What
things
you
want
to
happen
all
right!
Well,
let's
kick
it
off
and
see
what
it
looks
like
see
if
this
thing
works
for.
A
A
So
there
is
the
cube
config
or
the
the
pod
specification
for
the
replacement
cube
api
server
and
it
looks
like
this
is
just
gonna.
So
let's
talk
about
a
little
bit
about
what's
happening
here,
which
I
think
is
actually
pretty
fascinating
and
you
covered
this
pretty
well
in
the
talk.
But
let's
just
give
like
a
little
overview
of
what's
happening
here
and
then
we'll
kind
of
get
into
some
of
the
detail.
A
So
the
story
here
with
the
idea
of
a
shadow
api
server
is
that
if
you
are
running
a
control
plane
where
the
cubelet
also
resides,
then
you
could
schedule
on
that
control
plane.
Another
pod
that
represents
another
api
server
and
because
likely,
you
know,
because
you
know
by
default.
For
example,
inside
of
kubernetes,
you
might
have
access
to
things
like
host
path
and
and
other
things
that
would
be
necessary
to
go
ahead
and
grab
the
other.
The
credentials
that
the
ex
you
know
the
real
api
server
is
using
to
authenticate
to
etcd.
A
You
could
feasibly
stand
up
a
second
api
server
and
grab
those
same
credentials
from
the
underlying
host's
file
system
and
then
use
that
second
api
server
to
do
things
to
the
data
inside
of
ncd
kind
of
control.
What's
happening
inside
of
your
kubernetes
cluster,
and
do
that
in
such
a
way
that
auditing
would
not
be
a
thing
right
like
you
would
be
able
to
see
events
for
things
that
are
changing,
but,
for
example,
you
wouldn't
be
able
to
track
the
actual
api
calls
that
were
happening
because
only
the
api
server
keeps
its
logs
right.
A
That
way,
if
you
have
like
a
multiple,
if
you
have
an
h,
a
cluster,
each
of
the
members
of
that
h,
control
plane
would
have
some
portion
of
the
log
for
api
calls
that
were
being
made,
and
in
this
model
we're
saying
what.
If
we
just
made
a
new
api
server
and
we
threw
the
logs
away
and
we
used
that
api
server
as
a
way
to
manipulate
data
inside
of
the
kubernetes
cluster,
which
I
think
is
a
pretty
interesting
one.
A
But
I
do
want
to
call
out
that
it
definitely
does
make
the
assumption
that,
for
this
trick
to
work
that
you're
going
to
be
able
to
authenticate
to
that
cd
cluster
and
that
there's
going
to
be
some,
you
know
client
certificate
and
those
sorts
of
things
that
are
going
to
allow
us
to
authenticate
to
ncd
and
manipulate
the
data
inside
of
ncd
directly.
Without
that
access
this
trick
would
not
work,
but
with
that
access
we
could
do
all
kinds
of
fun
stuff.
A
B
Yes,
all
of
the
attacks
in
bat
talk
required
cluster
admin
access
and,
on
the
one
hand,
you
know
people
sort
of
ask
the
question
and
I
think
fairly
so
like
well,
if
you
have
cluster
admin,
why
are
you
bothering
you
know?
Cluster
admin
is
usually
considered
the
end
goal,
but
the
thing
is:
I
think
that
there
were
a
couple
of
things
that
we
were
thinking
with,
that
one
of
them
was
that
for
one
thing,
cluster
admin
isn't
always
the
end
goal.
Different
attackers
have
different
goals
and
motivations.
B
Some
of
them
might
actually
be
looking
for
cloud
metadata
or
something
else
like
that,
and
so
we
wanted
to
kind
of
expand
people's
ideas
of
what
was
possible
and
I
think
both
in
terms
of
like
who
an
attacker
could
be
and
also
like
what
kind
of
attacks
were
possible,
and
so
we
figured
we
were
going
to
be
creative
and
imaginative
and
help
kind
of
expand
people's
understandings
of
like
what
could
be
yeah,
and
that
was
sort
of
the
point
of
the
talk.
So
that's
that's
why
we
made
it
that
way.
A
B
Everything
in
there
requires
you
to
be
admitted
in
the
first
place,
which
is
you
can
absolutely
elevate
your
privileges
to
cluster
admin
on
a
cluster
that
allows
you
to
do
so
in
any
number
of
ways,
and
so
that's
not
a
thing,
that's
impossible.
We
just
assumed
that
that
given
attacker
had
done
that
first
either
because
the
cluster
had
already
been
compromised
or
because
they
were
a
malicious
insider
who
had
it
already.
A
Also
the
title
of
the
talk,
you
all
called
it
persistent
threats,
which
I
think
is
kind
of
for
me.
That
was
the
that
really
highlighted
like.
Why
do
this
stuff
right
because,
like
this
is
a
persistent
threat?
This
is
a
threat
that
I
would
install
in
a
cluster
that
I
had
admin
rights
on,
so
that
I
could
get
value
out
of
that
over
time.
A
A
A
B
B
That
allows
you
to
escalate
privileges
within
that
kernel
attacks.
Any
number
of
other
things.
There's
there's
a
lot
of
ways
in
as
an
attacker.
I
really
enjoy
that.
I
sort
of
you
know
kubernetes
kind
of
wide
and
multi-faceted
varied
attack
surface
to
me,
that's,
like
you
know,
sleeping
in
an
orchard
or
a
candy
store
or
something
there's
lots
of
possibilities
and
lots
of
ways,
and
I
kind
of
enjoy
that
actually.
A
And
I'm
closing
the
parentheses
there
we
go
all
right.
So
what
was
being
done
so
what's
being
done
here?
Let's
just
talk
through
it,
real
quick,
so
we've
got,
we've
got
our
shadow
api
server
deployed.
So
if
I
do
cube
kettle,
oh
control
c
cube
kettle,
get
pause,
dash
and
coop
system,
and
in
theory
we
don't
have
to
actually
have
this
in
cube
system.
We
can
have
this
in
any
name
space
at
all,
but
brad's
demonstration
does
deploy
inside
of
cube
system.
A
I
think
so,
but
you
could
deploy
this
into
default
or
anywhere
as
long
as
you
have
permission
to
schedule,
this
pod
on
a
control,
plane,
node
and
you
had
the
permission
to
access
the
underlying
file
system.
It
wouldn't
matter
what
namespace
you
put
this
pod
in
once
that
pod
exists,
though,
then
we
can
actually
interact
with
it.
A
Now
there
are
two
api
servers,
there's
the
one
that
was
built
into
the
kind
cluster
when
I
stood
it
up
and
there's
a
shadow
one
that
we
stood
up
and
they're
both
actually
configured
in
a
very
similar
way.
We've
only
made
some
minor
modifications
that
if
we
do
cube
kettle
get
pods
dash
and
cube
system
show
labels,
we
can
see
they
actually
tier
control,
plane
isn't
common.
A
A
Yeah,
so
we
have
our
shadow
api
server
and
we
have
our
our
regular.
You
know
regularly
scheduled
api
server
and
if
we
do
dash
o
wide,
we
can
see
the
ip
addresses
that
they're
using
and
all
of
these
pods,
because
they're
most
of
them
are
operating
as
static
pods
and
because
the
control
plane
instances
themselves
are
actually
using
the
ip
address
of
the
underlying
node.
A
A
And
let's
talk
about
some
of
the
configuration
changes
that
that
that
brad
made
to
this
manifest
real
quick,
because
I
think
that's
actually
kind
of
one
of
the
fun
parts
of
what's
happening
here:
shadow
api
server
and
specifically
what
brad
did
or
what
brad
and
ian
did
is
that
they
enabled
anonymous
auth
equals
true
and
they
turned
the
authorization
mode
to
always
allow
and
they
set
the
insecure
port
to
port
443.
Now,
by
default
inside
of
kind,
the
api
server
will
bind
to
6443.,
and
so
they
don't
conflict.
A
These
are
the
these.
Are
the
real
threats
in
this
particular
model
right?
If
you're
going
to
protect
against
this,
then
you'd
want
to
use
something
like
admission
control,
whether
that
is
pod
security
policies
or
making
use
of
something
like
opa's
gatekeeper,
you'd
want
to
make
sure
that,
like
you,
had
really
good
control
at
admission,
that
would
not
allow
host
path
unless
it
was
explicitly
allowed.
B
A
A
Your
point
is
totally
valid
if
the
fcd
servers
were
sitting
on
some
other
piece
like
maybe
we
brought
this
whole
mess
up
in
in
aws
and
extra
and
std
was
external
and
we
had
different
ip
addresses
for
the
different
fcd
servers
that
we
would
interact
with
on
the
ap
on
the
from
that
kind
of
api
server.
Then
that
would
totally
work.
A
A
A
A
And
so
that
is
kind
of
the
point
of
having
the
shadow
api
server
configured
and
that's
where
some
of
this
persistent
threat
comes
in.
If
you
had
that
sort
of
a
configuration
set
up-
and
you
didn't
call
it
something
like
shadow
api
server
which
might
be
kind
of
a
giveaway
to
somebody
who's
looking
for
trouble
in
their
cluster,
maybe
you
called
it.
A
B
B
When
brad-
and
I
were
talking
about
this
a
lot
when
we
were
when
we
were
working
on
this
talk,
you
know
as
much
as
I
talk
about
kubernetes
not
being
secured
by
default.
It's
a
lot
more
secure
by
default
than
it
used
to
be.
You
know
I
mean
when,
when
I
started
playing
with
kubernetes
it
was
you
know
there
was
at
first.
There
was
no
authorization
at
all.
You
know
just
none
of
it
it
you
know.
B
In
theory,
you
were
supposed
to
put
nginx
on
as
a
kind
of
firewall,
but
otherwise
you
could
just
like
execute
commands
as
root
with
an
unauthenticated
call,
and
everything
was
kind
of
like
that.
You
know
you
could
just
hit
ncd
directly
it
didn't
it
didn't
have
encrypted
traffic.
You
could
just
dump
secrets
from
it.
You
could
you
know.
Things
were
exposed
externally
to
the
internet
and
you
could
make
hublet
commands
on
it
and
it
it's
not
like
that
anymore.
It's
really
changed
a
lot
and
I
you
know.
B
I
knew
that
because
I
lived
through
it,
but
when
we
were
first
working
on
this
talk,
we're
kind
of
going
through
some
of
our
older
stuff
and
we're
like
wow.
It's
really
not
like
that
anymore,
like
you,
used
to
really
just
be
able
to
like
run
around
in
there,
and
you
can't
really
do
that
at
this
point.
A
Honk
away
yeah,
I
used
to
be
able
to
do
all
kinds
of
wacky
stuff,
but
unfortunately,
like
things,
things
have
changed,
but
I
remember,
like
another
anecdote,
I
was
evaluating
a
cluster
for
its
configuration
recently
and
one
of
the
things
that
one
of
the
things
that
caught
me
was.
I
was
looking
at
the
flags
that
were
used
to
operate
fcd
as
a
server
and
they
had
all
the
tls
settings.
Except
for
this.
A
If
you
don't
have
this
set
to
true
it's
you,
can
you
don't
actually
need
a
client
cert?
You
could
just
it'll
still
be.
You
know,
encrypting
traffic
between
you
and
the
std
cluster,
but
you're
not
actually
required
to
have
a
client
search,
so
clients
are
off
through
and
peer
clients
are
off.
True
are
both
very
very
important
flags
when
you're
looking
at
or
evaluating
the
secure,
the
the
security
of
the
configuration
of
your
ncd
cluster,
pretty
important
stuff.
B
There's
a
comment
in
chat
from
jos
rosso
saying
funny:
I've
had
some
clusters
running
shadow
pods
without
knowing
it
because
they
turned
psps
on
and
didn't
know
it
was
blocking
the
mirror.
Pods
scary-
and
I
think
it's
a
thing
to
note
about
a
lot
of
the
attacks
in
this
talk
is
that
you
know
we
were
coming
at
them
from
the
perspective
of
attackers
who
were
inclined
to
misuse
things.
But
a
lot
of
these
actually
could
be
potentially
very
useful,
debugging
or
monitoring
configurations
for
people.
A
Yeah
totally
it's
it's
absolutely
legit
and
I
remember
like
when
we
were
developing
our
talk
for
black
hat.
That
was
absolutely
like
one
of
the
the
fun
parts
of
of
that
exploration
was,
I
was
like
you
know,
leveraging
or
being
able
to
leverage
something
like
a
static
pod
to
interact
with
the
cluster
or
being
able
to
use
exec
or
some
of
these
tools.
Although
the
great
debugging
tools,
they
also
represent
pretty
significant
ways
to
to
kind
of
lower
the
or
to
use
as
attack
surfaces
against
things
as
well.
B
A
A
B
Andy
randle
says
any
thoughts
about
coop
ctl
debug.
I
was
very
excited
as
a
pen
tester
when
the
news
about
ephemeral
containers
came
out
because
I
was
like
this
is
going
to
really
present
some
interesting
new
attack
servers
for
me
that
I
was
really
excited
about,
and
then
I
discovered
that
almost
everything
I
could
possibly
think
of
had
before
I
managed
to
think
of
it
already
been
preemptively
sabotaged
by
jordan
liggett,
who,
I
think
single-handedly
makes
my
life
more
difficult
than
anybody
else
on
the
planet.
A
A
A
A
A
The
way
I
read
this,
it
was
a
configuration,
a
mis,
I'm
a
configuration
missed
up
by
the
person
who
was
opening
it.
But
if
I'm
wrong
about
that,
definitely
definitely
let's
revisit
it.
There
is
neither
a
crl
nor
nor
is
ocsp
supported,
that's
correct,
but
there
is
actually
an
rbac
implementation
in
ncd
that,
like
I
don't
as
far
as
I
know,
isn't
used
by
anything
inside
of
kubernetes.
A
Right,
neither
of
them
are
all
right
back
to
our
setup
here,
so
we
got
our
cluster
configured
with
a
shadow
api
server.
We're
able
to
deploy
that
thing,
and
we
can
see
how
easy
that
is
to
deploy.
We
basically
just
targeted
the
control
plane
node.
We
mounted
the
host
path.
We
basically
just
grabbed
the
configuration
of
what
was
being
hosted
on
the
control,
play
node
modified
a
little
bit
and
then
deployed
another
one,
so
that
was
a
pretty
fun
attack
and
that
gives
us
complete
access
to
scd.
A
A
Yeah
it
could
be,
I
mean
you
could
call
this
thing
anything
right.
You
could
actually
just
rename
the
binary
from
api
server
to
pretty
much
anything
else
and
and
drop
it
in
there
as
long
as
you
have
access
as
long
as
you
have
network
access
to
ncd
and
you
have
access
to
the
credentials
that
will
allow
you
to
authenticate
to
entity.
If
that
is
even
a
requirement,
as
we've
just
pointed
out,
then
then
you
then
there's
nothing
keeping
this
from
working
for
you.
A
B
A
B
You
know
it
was
kind
of
fun.
It
was
kind
of
easy
to
use
when
we
were
talking
about.
You
know
the
way
that
the
premise
of
this
talk
is
that,
as
you
know,
as
kubernetes
has
gotten
larger
and
more
complex
and
more
sophisticated
and
as
the
security
defaults
have
improved,
attackers
have
really
had
to
level
up
in
terms
of
you
know,
there's
not
as
much
low
hanging
fruit
anymore.
B
It's
not
only
attackers
that
have
had
to
become
more
sophisticated
in
the
kinds
of
techniques
that
they're
using
it's
also
administrators,
and
also
everybody
else
has
more
to
keep
up
with,
and
so
it's
you
know,
as
that
has
happened
in
kind
of
the
larger
project.
There
have
been
more
kind
of
demands
from
developers
and
users
that
can
we
figure
out
a
way
to
make
this
easier
to
use.
Like
all
of
these
things
are
kind
of
getting
in
the
way
can
we
can
we
make
it
easier?
B
You
know
with
this
increasing
openness
coming
from
this
increasing
complexity,
let's
just
open
it
right
back
up
again,
and
let's
just
put
this
back
with
this
dynamic
configuration
and-
and
I
I
liked
the
sort
of
metaphor
of
that
now.
My
understanding,
I
think
from
you
duffy-
is
that
at
one
point
people
were
talking
about
dynamic
config
of
the
api
server.
Is
that
still
a
thing.
A
I
think
it
is
still.
I
know
that
there's
been
a
bunch
more
work
happening
in
component
config.
I
don't
know
exactly
where
that
work
is,
but
I
do
know
that
it's
still
ongoing
that
the
that
the
idea
is
that
there
will
be
it
would
allow
for
some
dynamic
configuration
of
other
components,
not
just
the
cubelet
yeah.
A
B
A
A
A
A
A
It's
totally
working
all
right,
so
that's
working!
So
what
we've
done
here
is:
we've
actually
used
the
proxy
endpoint
of
the
node
itself,
the
kind
worker
node
to
grab
the
config
z
output,
and
what
that
allows
us
to
do
is
see
what
the
configuration
of
the
cubelet
is.
I'm
going
to
restart
your
cam
again
real
quick
ian,
because
you
have
some.
You
have
that
artifact
again.
B
B
A
A
B
A
Map
seems
to
have
worked
now.
What
we're
going
to
do
is
we're
going
to
dynamically
reconfigure
that
worker
node
to
effectively
turn
off
authentication
now.
This
is
another
really
super
subtle
one,
because
what's
going
to
happen
here,
is
we're
going
to
change
the
configuration
like
effectively
in
software
of
this
cubelet
and
that
cube
is
going
to
honor
that
new
configuration
by
basically
disabling
authentication
against
the
cubelet.
A
So
what
could
be
running
on
a
cubelet?
That
would
be
a
high
risk
right
like
if
we
had
run
this
on
the
cubelet,
where
the
api
server
was
running.
This
gives
us
the
ability
to
exec
right
into
the
api
server
copy
files
from
it
put
files
in
it.
We
can
basically
use
this
as
an
attack,
and
one
of
the
questions
that
we
talked
about
before
right
was
like
once
you
have
admin
on
the
cluster.
Why
is
this
stuff
interesting
well
in
this
model?
A
In
my
opinion,
in
this
model,
once
you've
actually
removed
the
authentication
requirements
from
all
of
the
cubelets
right?
If
the?
If
the
administrator
isn't
aware
that
this
change
has
been
made
because
they're
not
watching
for
the
configuration
of
the
nodes,
then
this
is
a
very
persistent
threat,
because
it
means
that,
as
long
as
I
can
still
get
to
that,
cubelets
ip
address
I'll
still
be
able
to
find
my
way
back
in
any
other
number
of
ways
right.
A
A
A
So
this
is
the
config
source
for
this
cubelet.
It's
saying
that
the
configuration
is
held
in
a
config
map
and
that's
where
it's
held,
and
so
this
is
what
it
looks
like
to
have
that
cube
configured
in
this
way
and
what's
interesting,
is
that
that
configuration
can
also
be
made
global
right.
So
when
you
look
at
the
dynamic
configuration
of
cubelet,
it's
really
a
pretty
interesting
one,
because
it
allows
you
don't
have
to
necessarily
create
a
configuration
for
each
kubelet.
A
You
could
create
one
global
configuration
for
all
of
your
cubelets,
which
is
too
many
eggs
in
one
basket,
so
that
this
is
absolutely
like
kind
of
like
that
that
that
story
that
tells
itself
about
a
million
times
like
every
year
right
where
we're
in
we've
gotten
so
good
at
automation.
That
if
we
make
one
mistake,
we
actually
see
that
mistake
propagate
across
a
very
wide.
B
A
A
Demo,
2
sh.
So
what
can
we
do?
We're
going
to
jump
into
the
attack
pod
that
we
created
and
remember
before
when
we
tried
to
authenticate
to
the
local
10
250
port,
we
got
authorized.
We
got
unauthorized
right
because
the
authentication
model
for
that
cubelet
was
requiring
a
web
hook,
and
my
and
my
pod
running
on
that
cube
doesn't
have
the
credential
necessary
to
authenticate
to
the
cubelet
normally.
A
A
A
B
Somebody
is
asking
me
to
give
me
and
brad
who
is
in
the
chat
and
not
on
the
camera,
to
give
our
thoughts
on
how
you
audit
and
secure
kubernetes
clusters
brad
for
you,
duffy
says
there
is
a
period
of
after
always
allow
so
I'll
answer
this.
While
you
figure
the
period
out.
How
do
I
audit
insecure?
I
would
say
you
know
some,
you
sort
of
start
out
with
code
review
right.
You
take
a
look
at
the
gambles,
see
if
there's
anything
really
glaring
in
there
or
a
comma.
B
He
says,
and
you
know,
then
you
can
kind
of.
I
look
out
for
what
kind
of
admission
control
is
on
there
if
there's
anything,
really
wacky
on
there.
If
there's,
you
know
some
things
I
might
look
out
for
could
be
anything
that
is,
you
know,
bracket
star
bracket
mounted
files,
external
urls,
generally
external
access
to
things.
It
depends
on
exactly
what
I'm
trying
to
look
for,
but,
generally
speaking,
start
out
with
code
review,
see
if
you
can
find
anything
weird
looking
in
there,
part
of
that
is
sort
of
pattern.
B
B
If
I
can,
if
I
can
impersonate
if
I
can
go
anywhere
or
do
anything
interesting
how
many
unauthorized
I'm
getting
in
response,
that
kind
of
thing
and
and
then
kind
of
move
from
there,
you
sort
of
you
know
you
enumerate
the
attack
surface,
you
see
if
there's
any
kind
of
foothold
you
can
get.
If
there
is
anything
else
you
can
find,
then
you
sort
of
enumerate
that
see
how
far
you
can
go
and
then
you
just
sort
of
keep
doing
that.
That's
a
short
version
of
what
I
do.
I
think.
A
A
B
I
think
I
shouted
this
out
in
chat
already,
but
there
are
some
kind
of
interesting
projects
that
do
things
like
this,
and
one
of
them
is
called
k-rail
which
might
be.
I
don't
know
that
anybody
has
made
a
k
row
config.
That
is
specific
to
anything,
I'm
saying,
but
generally
speaking,
kind
of
sets
of
rules
that
you
can
run
your
cluster
through.
B
B
B
B
A
I've
heard
joe
refer
this
refer
to
this
as
the
that's
a
definitely
a
challenge,
all
right,
so
that's
got
it
up
there
and
what's
interesting
in
this
in
this
command
line.
Here,
where
we
say
cube
kettle,
create
config
map,
my
node
config
dash
from
file,
you
can
see
how
there's
like
equals
on
either
side
of
the
cubelet
word
here.
A
A
B
A
A
A
Yeah,
because
one
thing
I
will
say
about
kubernetes,
especially
lately,
is
that,
like
in
every
every
patched
version,
I've
seen
recently
like
there's
some
collection
of
security
improvements.
Right
I
mean
like,
and
that
might
be
partially
due,
because
due
to
the
fact
that
now
there
is
a
what
do
you
call
it
where
you
get
paid
to
find
things?
What
do
you
call
that
you
know
what
it's
called?
B
A
Yeah,
I
think
this
is
what
I
think
I
found
what
I
was
missing.
So
one
of
the
other
things
that
was
done
to
set
this
demo
demonstration
up
was
a
command
that
would
actually
add
dynamic
configuration
directory
to
the
kubot's
configuration
which
cube
adm
doesn't
enable
by
default,
because
qbm
doesn't
enable
full
dynamic
cubelets
by
default.
You
have
to
turn
that
feature
on,
and
that
is
what's
missing.
So
if
I
were
to
go
ahead
and
do
this
command,
then
we
should
be
able
to
see
the
change.
B
Somebody
asked
me
what
my
thoughts
are
on
micro,
vms,
like
kata
and
firecracker.
There
was
a
talk
that
just
happened
at
black
hat
about
how
to
escape
from
kata
containers,
which
I
thought
was
great,
and
I
think
that
there
are
a
lot
of
you
know.
I
think
it's
an
involving
changing
discipline
and
involving
changing
attack
surface
and
I'm
excited
to
be
able
to
find
new
ways
to
bring
things.
I
think
that's
what
I
think.
A
A
B
A
A
B
Not
familiar
with
that
project,
but
I
think
off
the
top.
I
don't
know
that
there
is
a
single,
be
all
and
always
secure
way
of
running
foreign
code.
I
think
probably
it
looks
like
defense
and
death
and
that
looks
like
securing
your
stuff
all
the
way
up
and
down
across
the
stack
and
thinking
of
kind
of
container
security
and
orchestration
security
as
a
holistic
thing.
So
I
I
doubt
it.
B
A
B
In
depth
like
the
defense
in
depth,
the
way
that
you
know,
if
anybody
is
you
know,
plays
video
games
out
there.
If
you
play
plants
versus
zombies,
there's
some
sort
of
tower
defense
game.
You
know,
if
you
think
about
it,
like
that,
you
don't
want
a
single
point
of
failure
between
your
castle
and
whatever
kind
of
invading
hordes
are
coming
in.
B
A
What
we've
explored
so
far
is
like
some
of
the
tools,
and
I
want
to
point
out
that,
like
nothing,
I
did
today
involved
anything
on
the
cloud
everything
I
did
was
here
on
my
local
laptop,
using
kind
as
a
project,
and
it
really
speaks
to.
I
think
something
that
is
really
important
to
both
ian
and
myself,
which
is
that
anyone
can
do
this.
A
There
is
no
magic
to
it
right
like
you,
you
can
absolutely
like,
with
a
budget
computer
deploy,
install
docker
on
it,
get
linux
for
free
play
with
these
things,
expand
your
understanding
of
the
way.
The
way
these
things
work
right
and
that's,
I
think,
if
there's
like
one
thing,
I
hope
that
is
conveyed
when
we
explore
these
things
on
tgik
and
in
the
community.
A
It's
that
you
know
go
explore,
be
goose,
do
crimes
you
know
like
like
go,
go,
go,
learn
this
stuff
and
if
you,
if
you
want
help
learning
it,
if
you
have
questions
like
the
community,
is
there
for
you
like
we're
here
to
like
set
you
on
the
right
path?
Answer
your
questions.
Get
you
involved,
do
everything
we
can
to
like
really
get
that
stuff
out
there.
B
Absolutely
yeah
it
requires
curiosity,
it
requires
being
able
to
think
about
how
something
might
be
used
in
ways
that
its
creators
aren't
originally
intended,
and
that's
it
you
know,
and
those
particularly
the
second
one.
That's
a
learnable
skill.
You
know
it's
a
thing
that
you
can
practice
is
the
thing
that
you
can
get
better
at
and
we
I
want
to
encourage
everybody
to
to
get
into
that,
because
it's
useful
in
terms
of
being
able
to
figure
out
how
an
attacker
might
do
things,
how
they
might
look
at
that
stuff.
B
It
allows
you
to
be
able
to
look
at
that
stuff.
In
that
way
too.
It's
also
really
fun.
It's
fun
to
be
a
horrible
guest.
Sometimes
so
I
really,
you
know,
recommend
people
kind
of
putting
themselves
into
those
kinds
of
shoes
white
feet,
something
and
yeah
anybody
can
do
this
stuff.
It's
not
magic,
it's
logic
and
there's
lots
of
folks
in
the
community
who
are
amazing
and
who
are
helpful
and
who
are
really
excited
to
help.
You
learn.