►
Description
Kubernetes is a powerful and highly configurable tool, but by default it's extremely insecure. In this live hacking session, I'll show how an attacker can expand the blast radius of an application exploit through to control of the entire cluster whilst demonstrating the misconfigurations which allow this to occur and showing how you can avoid these scenarios in the real world.
A
But
just
before
we
move
on
to
to
introducing
that
speaker,
we
do
have
a
number
of
sponsors.
Who've
made
this
event
possible,
and
so
I
want
to
give
a
shout
out
to
jet
stack
and
just
quickly
introduce
what
jet
stack
do
so
jet
stack
is
a
cloud
native
technology
and
services
provider
they've
been
active
since
the
very
beginning
of
kubernetes,
and
they
were
one
of
the
original
members
of
the
cncf.
A
Jetstack
has
become
particularly
well
known
for
its
work
in
open
source
development
and
is
the
original
creator
of
the
cert
manager
project,
which
has
grown
in
popularity
as
a
fully
open
source
solution.
As
a
cloud
native
ecosystem
has
grown,
jetstack
has
evolved
as
a
professional
services
company
and
has
deep
experience,
helping
enterprises
building
platform
infrastructure
using
kubernetes.
A
In
recent
months,
jetstack
has
launched
security
focused
products
to
give
both
enterprise
platform
and
security
teams,
full
visibility
and
control
of
their
machine
identities
to
protect
workloads
and
ensure
cluster
health
across
any
type
of
cloud-native
infrastructure
as
an
open
source
company,
their
culture
and
values
are
centered
around
community
they're,
also
growing,
really
fast.
So
they're
looking
to
talk
to
any
talented
engineers
out
there,
if
you're
interested,
please
check
out
their
website
to
see
what
opportunities
exist,
and
so
we
are
ready
for
our
final
speaker
for
track
two.
He
probably
needs
no
introduction.
B
B
So
our
starting
point
for
today's
demo
is
that
we
have
discovered
this
vulnerable
web
application
on
the
internet,
and
this
is
a
remote
command
execution
vulnerability.
It's
clearly
mocked
up
for
the
sake
of
this
demo,
but
there
are
lots
of
real
world
examples
of
this
as
one
almost
exactly
like
this
for
tomcat,
and
so
these
do
actually
exist
at
this
point.
B
So
let's
have
a
think
about
what
we
might
be
able
to
do
from
here.
So
the
first
thing
we
might
be
thinking
about
is
that
most
applications
now
run
in
containers
and
most
containers
are
configured
with
environment
variables.
So
looking
at
the
environment,
variables
of
our
application
might
tell
us
some
interesting
things,
and
so,
if
we
do
command
equals
n,
that
does
indeed
give
us
lots
of
interesting
information.
So
we've
got
multiple
references
to
kubernetes,
so
we
can
clearly
ascertain
that
we
are
running
in
a
kubernetes
cluster.
B
B
So
we
can
now
update
our
diagram
of
what
we
know
so
we've
discovered
the
kubernetes
cluster.
We
know
there's
an
internal
ip
address
to
the
api
server
and
we
know
that
that
we've
got
a
service
there.
So,
let's
see
what
else
we
can
find
out.
So
what
we
might
be
interested
in
is
our
the
networking
information,
so
let's
run
the
ip
command
there
and
we've
discovered
the
ip
address
of
the
pod,
and
this
is
going
to
be
useful
to
us
later.
B
So
we're
going
to
make
a
note
of
that
10.244.1.9
and
we'll
come
back
to
that
a
little
bit
later
and
we'll
see
why
that's
important.
So,
let's
see
what
else
we
can
do
by
default.
Every
pod
in
a
coupe
in
kubernetes
has
a
service
token
auto
mounted
in
it,
and
this
is
associated
with
the
service
account
that
was
used
to
create
the
pod.
You
can
control
whether
this
is
auto
mounted
on
a
service
account
or
a
pod
level.
B
B
So
let's
just
copy
that-
and
let's
run
that,
so
what
I've
done
here
is
to
to
cat,
where
I
think
the
token
might
be-
and
clearly
I
do
have
a
token
auto
mounted
here.
So
it
seems
that
our
permissions
for
this
particular
pod
will
let
us
access
the
token.
So
if
we
go
back
to
our
timeline
of
doom,
the
next
thing
that
we
can
add
to
our
exploit
scope
is
that
we've
discovered
a
pod
token.
B
So
let's
see
whether
we
can
use
that
token
to
get
some
stuff
from
the
internal
address
of
the
api.
So
again
we're
going
to
construct
a
url
here
to
run
this
remote
command
and
what
we're
going
to
do
is
test
whether
we've
got
curl.
We
clearly
have
because
we're
getting
information
returned
from
the
from
the
api
server
here
and
what
what
we've
done
is
see
if
we
can
access
this
endpoints
api
of
the
api
server,
and
this
has
actually
returned
us
a
bunch
of
data.
B
And
so,
if
we
go
back
into
our
to
our
our
diagram
here,
what
we've
done
is
go
from
our
pod
to
the
internal
ip
address
of
the
of
the
api
server
and
we've
we've
passed
it
the
token
and
it's
given
us
back
the
external
ip
address
of
the
api
server,
and
we
can
now
add
another
vulnerability
in
here
that
that
pod
token
has
allowed
us
access
to
the
endpoints
api.
So
it's
a
two
permissive
policy.
On
the
service
token,
that's
allowed
us
to
discover
additional
information
about
the
kubernetes
cluster.
B
So
now
we've
got
a
token.
Let's
see
if
we
can
configure
our
our
local
cube
couple
to
use
this
token,
which
will
make
us
a
bit
easier
for
us
to
to
continue
our
attack.
So
I've
got
a
little
script
here
to
set
up
my
my
coup
config
and
I'm
just
going
to
add
that
token
that
we've
just
found
and
my
address
of
the
kubernetes
api
server
is
going
to
be
that.
B
So,
let's
just
pull
this
out
a
bit.
So
we
can
see
what
we're
doing
here
so
and
let's
make
sure
that
we've
cube
config
equals
demo
config
right.
So
the
first
thing
that
we've
got,
what
we
might
won't
be
interested
in
is
to
see
what
pods
are
running.
So,
let's
just
do
couple
get
pods
in
the
default
name.
Space
and
what's
interesting
here-
is
that
we've
immediately
been
had
a
a
a
an
error
that
we're
not
allowed
to
run
to
get
pods
in
the
default
name
space.
B
But
what
the
so
that's
the
service
token
is
not
permitted
to
hasn't,
got
the
permission
to
get
hold
of
pods
and
objects,
but
it
has
given
us
some
more
information
here,
because
what
we
can
see
here
is
that
we
can
see
the
service
account
that
was
used,
that
that
token
is
related
to
and
we
can
see
the
service.
Cam
is
namespace
into
this
secure
namespace.
So
let's
try
and
get
pods
in
that
namespace
and
we
can
see
that
that
command
has
actually
has
actually
succeeded.
B
So
clearly,
this
token
in
this
secure
namespace
has
some
permissions.
So
let's
look
what
permissions
they
actually
has
and
there's
a
couple
of
different
ways.
We
can
go
about
this
in
kubernetes.
We
can
do
the
use
the
kubecuttle
auth
command.
Can
I
minus
minus
list?
Minus
minus
token
equals
and
then
our
token-
and
this
is
in
the
default
name
space-
I
didn't
name
space
that
command,
so
we
can
see,
we
don't
have
actually
many
permissions
here.
We've
got
that
endpoints
permission
there,
but
other
than
that
we
don't
have
very
much.
B
But
if
we
now
repeat
that
for
our
secure
namespace,
we
can
see
that
we've
got
a
whole
bunch
of
of
permissions
here
against
all
resources
in
this
namespace
and
there's
also
a
different
way.
We
can
visualize
this,
for
what
we
can
do
is
use
the
access
matrix
plugin,
which
gives
us
a
different
view.
This
is
a
crew
plugin
that
I've
got
installed.
B
So
if
we
do
that
in
the
secure
namespace
we
can
see,
we've
got
all
sorts
of
permissions
and
all
sorts
of
things.
So
this
is
a
fairly
common
kind
of
misconfiguration.
Here's
a
fairly
common
pattern
where
people
assume
that
name
spaces
are
actually
a
security
boundary.
So
I've
got
a
service
account.
It
only
exists
in
this
name
space.
I
can
let
it
do
what
it
wants
in
that
name
space
providing.
I
don't
allow
it
to
do
anything
else
in
in
other
name
spaces.
So
now
we
go
back
to
our
our
slides.
B
We
know
we've
got
our
external
ip
address
of
the
api
server
that
I
connected
to.
We
know
we
have
a
default
namespace,
we
don't
know.
What's
in
it.
We
know.
We've
got
this
secure
namespace
and
the
the
the
vulnerability
here
is
that
that
role
has
been
given
too
many
permissions.
So
it's
it's
a
service
account,
that's
been
given
too
many
permissions
in
that
name
space.
As
I
said,
it's
a
the
idea
being
that
that
somehow
a
name
space
is
a
security
boundary.
B
So
again
we'll
update
our
timeline
of
doom.
So
we've
got
this
role
with
too
many
permissions
in
the
name
space.
So
let's
try
and
get
a
a
a
shell
on
our
compromised
pod.
So
if
we
do
okay
get
pods
in
in
our
secure
namespace,
we've
only
got
one
there,
so
we
can
assume
that
is
the
one
that
we
compromised
externally.
B
So
now
I
could
do
k
exec
minus.
I
t
I've
just
got
a
coupe
kettle,
alias
to
be
k
there
if
anybody's
wondering
so
we
need
that
pod
in
the
secure
name
space
and
let's
try
and
run
bash,
and
so
this
gives
us
a
shell.
So
that's
interesting!
So
I
I
can
see
that
I
am
a
I'm,
not
the
root
user,
which
is
good,
so
I'm
running
as
this
web
admin
user.
B
So
the
first
thing
I'm
going
to
try
and
do
is
become
room,
so
I'm
going
to
do
sudo
su
and
the
sudo
command
doesn't
even
exist
in
this
in
this
command
in
this
container.
So
this
is
good
from
a
security
practice
perspective.
You
know
we're
running
as
an
unprivileged
user,
we're
not
allowing
sudo.
So
let
me
try
something
else:
I'm
gonna
try
and
create
a
file,
and
so
that
has
has
worked
so
I've
been
able
to
create
a
file
there,
which
is
a
another
vulnerability
here.
B
If
we've
got
a
read,
write
file
system
that
allows
attackers
to
download
software,
potentially
change
configuration-
I
I
I
know
I
have
curl
there
already,
so
it's
definitely
possible.
I
can
now
download
tools
from
other
places,
and
this
is
caused
by
not
setting
read.
Only
root
file
system
is
true
in
the
in
the
security
context
settings
for
my
container,
and
you
know
there
are
in
most
scenarios
you
really
don't
want
to
have
read,
write
file
systems
inside
containers.
B
You
know
we
should
be
we're
not
storing
stuff
in
there
and
it
only
gives
an
attacker
an
additional
additional
way
of
exploiting
your
containers.
So
I'm
going
to
come
out
of
that
for
a
second,
and
so
we
know
we
have
permissions
in
this
secure
name
space.
So
let's
try
and
spawn
a
root
pod
and
I've
got
some
some
yamls
here
and
if
I
look
at
the
root
podcamel.
So
this
is
just
gonna
spawn
a
alpine
container
which
I
know
runs
as
root.
B
B
So,
let's
have
a
look:
what's
actually
happened
there,
so
we've
got
this
create
container
config
error
so
that
looked
like
something
has
actually
stopped
us
from
running
the
root
pod
and
if
we
do,
google
describe
describe
the
pod
root
pod,
insecure
namespace,
and
this
error
here
container
has
run
as
non-root
and
image
will
run
as
root,
so
something
there
has
stopped
us
from
running
a
root
container.
This
is
probably
a
pod
security
policy,
but
at
this
point
we
don't
really
know
so.
B
Let's
try
and
run
a
privilege
pod
and
see
what
happens
so
demo
yamls,
I'm
gonna,
run
non-route,
but
I'm
gonna
try
and
spawn
a
privilege
for
so
I've
got
this.
This
container
image
of
mine,
that's
in
a
public
registry
and
I'm
going
to
spawn
it
privileged,
but
it's
running
as
a
normal
user,
and
I'm
also
going
to
try
and
do
a
host
path
mount.
So
I'm
going
to
try
and
mount
the
host
file
system
in
this
in
this
container.
B
So
there's
a
pod
security
policy
in
place
on
this
secure
namespace.
So
that's
kind
of
interesting.
You
know
that's
a
good
thing
right
from
a
security
perspective,
so
let's
try
and
do
something
else.
So
now
I'm
going
to
try
and
run
non-root
non-privileged
container
and
again,
I'm
gonna
use
my
super
sneaky
container
image.
That's
running
in
a
pub
that's
from
public
registry,
but
I'm
not
gonna
run
privileged.
I'm
gonna
run
as
a
normal
user.
B
So
let's
just
apply
that
and
I
don't
know
you
know
no
root,
not
improve
in
the
secure
namespace
and
something
seems
to
be
seems
to
be
happening
there.
So
let's
have
a
look
what's
happening,
so
my
sneaky
pod
is
up
and
running
so
again.
This
is
another
vulnerability
here,
because
what's
happened
is
this
cluster
has,
let
me
pull
arbitrary
container
images
from
public
registries
and
I
built
this
container
image,
so
I've
been
able
to
pack
it
full
of
all
sorts
of
hacking
tools
that
I
need
in
order
to
extend
my
exploit.
B
B
B
I'm
going
to
connect
to
sport
8080
and
I'm
running
a
remote
tty
on
this
container
image
so
on
this
pod,
so
I've
been
able
to
have
another
access
into
the
cluster
here
and
you
know
when
I
I
do
who
am
I
in
this
pod,
I'm
the
sneaky
user.
So
clearly
I'm
I'm
I'm
not
root,
but
what
I
can
do
is
that
and
I've
been
able
to
achieve
root
by
running
sudo,
and
the
problem
here
is
that
the
pod
security
policy
doesn't
include
allowed
privilege.
Escalation
equals
false.
B
So
we've
this
this
setting
here
allowed
privilege
escalation
equals
false,
and
many
people
think
that
that
is
redundant.
If
you
are
have
set
that
you
don't
allow
privileged
parts
and
you
don't
allow
running
as
root,
but
it's
not
redundant
and
I'll.
Show
you
why
so,
let's
come
out
of
the
out
of
the
root
user
again,
and
so
what
I'm
going
to
do
is
to
try
and
map
the
network
here.
B
B
10.244.1.9
you'll
remember
was
the
ip
address
of
our
original
vulnerable
application
that
was
running
on
portfol
to
5000,
but
what
the
scanner
has
shown
us
is
another
application,
also
running
on
port
5000.
Now,
that's
very
interesting
to
me.
That
could
well
be
another
instance
of
my
vulnerable
application
so
before
we
go
any
further
with
with
looking
at
that,
let's
add
our
pod
security
policy
didn't
disallow,
privilege
escalation.
That
allowed
me
to
sudo,
and
we
can
add
some
more
information.
We've
discovered
another
pod
running
on
port
5000.
B
We
don't
know
anything
about
it
at
the
minute.
We
don't
know
which
name
space
that
new
application's
in,
but
we
can
also
add
to
our
timeline
of
doom
that
we
have
no
network
controls
in
place.
So
we
had
nothing,
no
policies,
no
service
mesh
that
stopped
us
from
connecting
from
one
pod
into
another
pod,
potentially
escaping
our
name
space.
B
So
what
I'm
going
to
do
is
to
create
a
tunnel,
so
I'm
going
to
use,
use
socat
and
I'm
going
to
go
to
dot
10
10.244.1.10.5000,
I'm
going
to
represent
that
5001
on
this
pod
here.
So
I'm
going
to
create
that
tunnel
and
now
I'm
going
to
exit
out
of
the
port
forward
that
I
already
had
there
on
on
the
original
on
my
sneaky
pod,
and
I'm
now
going
to
reconnect
that
port
5001.
B
B
B
And
let's
try
kubrick
will
get
pause,
so
I'm
now
able,
with
this
new
token,
to
find
out
what
pods
are
running
in
the
default
namespace.
So
let's
have
a
look
on
the
chaos.
Can
I
command
what
this
new
token
has
given
us
and
we
can
see.
We've
now
got
lots
of
permissions
in
the
default
namespace
that
we
didn't
have
before,
so
we've
been
able
to
escape
that
secure
namespace
and
we've
got
new
permissions
in
the
default
namespace.
B
So
again
we
created
that
tunnel
from
5000
to
5001,
and
we
know
now
that
this
this
pod
is
running
in
the
default
namespace.
So
let's
try
and
and
do
something
else
here.
So
if
we
now
look
at
our
we're
going
to
try
and
run
a
privilege
pod
here,
as
I
said,
it's
going
to
run
that
same
container
image.
It's
going
to
try
and
mount
the
host
path
might
do
a
host
path
amount
of
the
root
file
system
of
the
host.
So
if
we
now
apply
that.
B
B
So
what
I
can
now
do
is
use
the
root
command
to
change
my
route
to
that
the
root
of
the
of
the
host
of
the
node,
and
so
now,
when
I
run
the
ps
command,
what
I'm
actually
seeing
there
is
all
the
processes
that
are
running
on
the
node
itself,
so
I've
basically
been
able
to
compromise
the
underlying
node.
We
can
see
the
container
d
processes.
We
could
see
some
of
the
control
planes
running
here.
B
We
could
potentially
see
other
other
container
images
that
were
running
on
the
node
and
if
we
update
our
our
scenario
like,
we
were
able
to
create
that
privileged
pod.
So
there
was
no
pod
security
policy
in
default,
we've
been
able
to
mount
mount
stuff
on
the
host
and
again
our
timeline
of
doom.
There
was
no
restrictions
in
the
default
namespace,
so
once
I've
got
the
hosts
file
system
here,
we
can
do
all
sorts
of
things.
B
I'm
going
to
run
the
shell
and
I
actually
get
forbidden
there,
because
the
kubelet
token
isn't
actually
allowed
to
start
pods
like
that,
but
it
does
say
it
can
create,
mirror
pods,
and
this
is
another
way
I
could
attack
this
node
in
the
et
cetera
kubernetes
manifest
directory
on
a
node.
If
we
put
yaml
in
there
that
will
get
automatically
started,
so
we
could
start
to
create
pods
in
the
coop
system
name
space.
B
We
could
make
other
attacks
against
the
kube
system
against
the
the
cluster
itself
by
using
mirror
pods,
but
because
we've
escaped
the
pod
security
policy
and
we
now
know
which
nodes
we
have.
We
can
actually
attack
on
a
different
vector
at
this
point,
so
we're
going
to
launch
a
pod
to
the
node
hosting
the
xcd
cluster
and
in
the
pod,
configuration
we're
going
to
actually
mount
the
xcd
configuration
from
the
node
so
that
we
should
be
able
to
connect
to
to
xcd
using
the
node
credentials.
B
So
if
we
have
a
look
at
our
demo,
yaml's
xcd
client,
we
can
see
what
we're
going
to
do
here
is
set
mount.
The
host
file
system
mount
where
the
xcd
cluster
configuration
is
and
then
we're
going
to
to
basically
pass
in
all
of
our
configuration
for
xcd
and
that
will
let
us
get
connect
to
the
xcd,
the
xcd
cluster
in
that's
running
inside
the
our
kubernetes
cluster.
B
Right
and
our
xtd
client
is
now
up
and
running,
so
what
we
can
do
is
we
want
to
exec
our
xcd
client
and
we're
going
to
run
xtd
control
member
list,
and
this
is
just
to
check
whether
we
can
connect
to
xcd
and
we
can
see
we
can
because
we've
been
returned.
The
members
of
our
xcd
cluster
and
ftd
contains
a
lot
of
interesting
information
about
the
cluster,
not
least
of
which
is
a
bunch
of
secrets.
B
B
C
C
B
B
Minus
minus
token
equals
token,
and
what
we
can
now
see
is
that
this
token
has
all
of
these
permissions
on
the
cluster
roles.
So
I
can
update
roles.
I
can
basically
do
anything.
I
want
to
any
user
account,
that's
on
the
cluster,
so
that's
really
the
end
of
the
so
just
to
go
back
over
what
we
just
did
we
created.
We
know
that
the
xcd
cluster
is
running
on
a
particular
host.
We
started
in
an
xtd
client
pod
to
connect
to
that
tv
cluster
and
finally,
we
got
cluster
admin
rights.
B
B
B
Before
you
get
to
production,
we
could
have
been
scanning
our
kubernetes
yaml
files
before
we
deployed
things,
and
so
most
scanners,
sneak
included,
will
tell
you
about
many
of
the
things
the
misconfigurations
that
I
just
talked
about
in
this
in
this
talk
and
finally
scanning
our
container
images
so
making
sure
that
our
container
images
don't
have
vulnerable
applications
in
them.
You
can
sign
up
for
a
free
account
and
try
all
that
out
with
sneak
dot,
sneak
dot
io.