►
From YouTube: CAP_NET_RAW And ARP Spoofing in Your Cluster: It's Going Downhill From Here - Liz Rice
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
CAP_NET_RAW And ARP Spoofing in Your Cluster: It's Going Downhill From Here - Liz Rice, Aqua Security
Did you know that by default, your applications running in Kubernetes can open raw network sockets? This talk demonstrates how, in the right circumstances, the CAP_NET_RAW capability that allows this can be abused by a compromised application. * ARP spoofing: pretending to represent the wrong IP address * If the app can ARP spoof the IP address of the DNS service, this potentially lets it spoof DNS addresses: pretending to represent the wrong domain name Sounds bad, doesn't it? These attacks, and their consequences, will be demonstrated live, along with preventative measures that you can take to ensure they aren't happening on your cluster. This talk explains CAP_NET_RAW and spoofing, but the audience is expected to be comfortable with Kubernetes concepts like pod specs and admission controllers.
https://sched.co/UaX0
A
Hi
everyone
thank
you
for
coming
to
my
talk
today,
so
just
to
level
set.
How
many
of
you
know
what
I
mean
when
I
say:
cabinet
raw,
okay,
pretty
knowledgeable
crowd
and
ARP
spoofing
yep,
pretty
good,
okay
right.
So
what
I
am
going
to
show
you
this
morning
is
a
an
exploit
that
a
chap
called
Daniel
siggy
who
was
working
in
my
team
over
the
summer.
A
He
discovered
this
and
we
were
sufficiently
sort
of
surprised
that
this
was
possible,
that
we
reported
it
to
the
kubernetes
security
team
and
they
said
yeah
it's
kind
of
working
as
designed.
But
if
you
wanted
to
raise
the
profile
of
like
it's
a
gating
against
this,
that
might
be
a
very
good
idea.
So
what
I'm
going
to
show
you
is
on
a
just
regular,
ordinary
kubernetes
cluster
with
default
settings,
a
layer
to
networking
plugin
and
the
one
thing
I
have
changed,
and
this
is
only
to
make
it
more
reliable.
A
As
a
demo
is
that
out
of
the
box,
you
would
get
two
instances
of
core
DNS
and
I've
deleted
it.
So
there's
only
one
now
before
I
actually
show
you
that
exploit
I
want
to
just
talk
through.
What's
going
to
happen,
so
I
have
a
victim
pod
and
the
victim
pod
wants
to
make
some
kind
of
network
request
to
well,
in
my
case,
to
the
outside
world
and
when
it
does,
let's
say
a
curl
request
to
example.com.
A
A
That's
all
well
and
good,
and
then,
when
we
have
the
exploit
and
the
same
thing
is
going
to
happen,
the
victim
is
going
to
be
trying
to
make
a
network
request,
to
example,
comm
it's
going
to
do
the
DNS
lookup,
but
this
time
my
hacker
I'm
gonna
have
a
hacker
pod
that
is
spoofing,
DNS
queries.
It's
gonna
receive
the
DNS
lookup
request
and
it's
gonna
redirect
that
request
to
another
pod
that
I'm
going
to
set
up
this
called
fake.
A
Alright,
let's
see
this
happening.
Hopefully,
this
is
going
to
be
big
enough,
oh
honey.
If
at
any
point,
I
forget
to
change
my
screens,
just
shout
at
me:
okay
and
I'm,
just
going
to
run
a
little
utility,
that's
going
to
trace
out
requests
for
capable
flags,
I'll
come
back
and
talk
about
that.
A
little
bit
later,
though,
right
I'm
gonna
start
my
three
pods,
which
are
in
its
directory.
A
A
A
Okay
and
I'm
also
going
to
set
this
up
so
that
it
can
start
responding
with
like
some
arbitrary
content.
Why
don't
I
say
hello,
you've
gone
and
it's
gonna,
basically
listen
on
port
80
and
respond
to
network
I.
Don't
like
the
look
at
that
something's
gone
wrong
with
my
typing.
Don't
think
the
I'm
going
to
just
remove
that
exclamation
mark.
That's
probably
bad
idea.
Okay,.
A
A
Right
and
now
I'm
going
to
run
a
little
exploit
script.
All
of
this
is
on
github
and
I'll.
Give
you
the
URL
for
that
later
and
after
we've
seen
it
I
will
explain.
What's
going
on,
it's
now
ready
to
take
over
DNS
requests
and
if
I
go
back
into
my
victim
pod
and
I
curl
example.com,
it's
a
little
bit
slower
than
the
request
to
the
normal
network,
but
we
start
seeing
the
spoof
DNS
response,
and
then
we
see
we've
been
able
to
apply
fake
traffic.
A
A
So
let's
have
a
talk
about
what's
actually
happening
and
first
of
all,
we'll
talk
a
little
bit
about
address
resolution
protocol
and
in
order
to
talk
about
that,
let's
just
briefly
remind
ourselves
about
the
seven
layers
of
networking.
We
don't
really
care
about
all
seven
layers.
For
this
talk,
we
really
only
care
about
three
layers:
the
application
layer,
where
we're
going
to
make
that
HTTP
request
the
network
layer
that
deals
in
IP
traffic
and
the
data
link
layer
which
is
dealing
with
Ethernet
packets.
A
So
when
our
victim
pod
makes
that
request
to
an
a
URL
DNS
is
what
translates
the
human
readable
name
into
an
IP
address
and
then
address
resolution
protocol
is
what
takes
that
IP
address
and
maps
it
to
a
MAC
address,
say:
there's
a
ARP
table
that
gets
built
up
that
has
these
mappings
between
IP
addresses
and
MAC
addresses
that
unknown
to
the
local
lots
of
the
node.
So
I
could
take
a
look
at
that.
A
A
So
it's
a
little
bit
because
it's
wrapped
it's
a
little
bit
confusing
to
see
but
odds,
are
we're
interested
in
there's
our
core
DNS
pod
and
that's
at
10:32
dot,
zero
dot,
two
which
is
well
the
second
line.
Second
highlighted
line
there
and
it's
got
the
being
able
to
fake
in
there
ARP
table.
It's
got
the
same
MAC
address
as
10.30
to
0.3,
which
is
yeah.
A
A
We
start
with
a
network
namespace
on
the
host
the
route
network
namespace,
and
it's
got
some
Ethernet
connection
to
the
outside
world
and
it's
also
got
a
bridge
and
the
bridge
is
what
holds
that
ARP
table.
Now
we
create
a
pot
and
the
pod
also
gets
a
network
name
space.
It
has
Ethernet
connection,
it's
a
virtual
connection,
it's
fake,
but
it
looks
like
we've
plugged
the
net
cable
in
from
our
pod
into
that
bridge.
From
a
networking
perspective,
it's
exactly
as
if
there
was
an
Ethernet
cable
between
the
two
and
on
the
bridge
side.
A
There's
this
virtual
Ethernet
connection
that
has
a
kind
of
a
MAC
address
associated
with
it.
We
create
another
part:
we're
going
to
get
a
situation.
We've
got
another
virtual
Ethernet
link
between
the
bridge
and
that
pod.
So,
if
Padre
wants
to
send
traffic
to
pod
B,
it's
got
an
IP
address
for
pod
B.
Well,
first
of
all,
the
packet
has
nowhere
else
to
go
other
than
out
of
f0
and
through
that
virtual
Ethernet
tunnel
to
the
bridge
and
the
bridge.
Does
the
art
look
up?
A
If
pod
a
wants
to
send
a
packet
to
some
other
destination
that
isn't
on
this
node,
then
the
packet
gets
routed
out
of
the
pod
in
exactly
the
same
way.
But
when
it
arrives
at
the
bridge
there
is
no
ARP
table
entries,
because
the
ARP
table
only
has
entries
for
connections
on
that
bridge,
and
that's
only
in
this
node.
So
we
can
only
have
ARP
table
lookups
for
pods
that
are
running
on
this
node.
A
A
So
if
we
want
to
do
I'm,
not
speaking,
basically,
all
we
need
to
do
is
pretend
that
an
IP
address
is
at
the
MAC
address
that
we're
actually
living
on.
So
if
I
was
pod,
B
I
could
pretend
that
the
MAC
address
of
certain
V
f1
is
an
IP
address,
but
it
isn't
really
but
I'm
going
to
pretend
it
is
and
that's
what
our
briefing
is
and
then
because
we've
told
the
bridge
yeah
I
own,
that
IP
address
it's
over
here
on
my
MAC
address.
A
A
That's
some
different
IP
address
from
what
it
actually
exists
that
so,
if
we're
making
a
DNS
request
on
the
victim
pod,
just
like
before
it
gets
routed
through
to
the
bridge
or
sent
through
to
the
bridge,
really
I
should
probably
shouldn't
say,
reaches
the
bridge
says:
do
I
have
an
entry
in
my
art
table
for
core
DNS
his
IP
address
and
the
hacker
pot
is
being
saying.
Yeah
I'm
over
here
with
core
DNS
is
an
IP
address.
A
Send
it
to
me
bridge
believes
it
sends
the
DNS
request
to
the
hacker
pod
and
the
hacker
pod
and
respond
with
any
kind
of
result.
It
wants
to
say:
DNS
requests
are
looking
up
the
IP
address
that
corresponds
for
it
to
it.
The
main
name.
You
send
that
DNS
request
to
the
IP
address
of
a
DNS
service.
If
you
are
co-located
on
the
same
node
as
the
DNS
service,
you
can
take
over
DNS
the.
A
Best
way
of
preventing
this
from
happening
is
to
stop
the
hacker
pod
from
being
able
to
populate
information
into
that
off
table
and
the
way
you
can
stop,
that
is
by
preventing
it
from
being
able
to
open
raw
network
sockets,
essentially
that
cat
net
raw
capability.
So
most
of
you
put
your
hands
up
when
we
talk
when
I
mentioned
cat
net
raw,
but
just
in
case
there
are
a
few
people
who
didn't
have
their
hands
up
happen
it
raw,
but
is
one
of
a
set
of
linux
capabilities.
A
Dozens
of
these
capabilities
actually
haven't
counted,
I'm
gonna,
guess
dozens
and
they
give
quite
fine-grained
privileges
inside
the
Linux
kernel
percent,
normal
IP
traffic.
You
don't
need
any
special
capabilities,
but
if
you
want
to
open
what's
called
a
raw
socket
and
if
you
want
to
send
packets
directly
at
that
Ethernet
level,
you
need
the
cat
network
privilege.
A
This
is
a
new
EBP
eff
based
open-source
project
that
we've
just
released
a
couple
of
weeks
ago.
It's
still
very
experimental,
it's
called
Tracy
and
it
lets
us
trace
out
kernel
events
and
wealth
system
calls
and
some
kernel
events,
one
of
which
is
capable,
which
is
the
event
that
happens
when
you're
checking
whether
you've
got
certain
set
of
capabilities.
A
A
A
Hey
and
if
I
run
the
exploit
this
time,
what
we're
going
to
see
is
it
will
fail,
because
it's
going
to
make
that
first
cat
net
rule
request
and
that's
going
to
be
failed
and
basically
the
exploit
code
doesn't
know
how
to
deal
with
that.
So
there
we
go.
There
is
the
cat
net
raw
check
that
failed,
so
you
might
be
thinking,
but
I'm,
not
gonna,
run
this
crazy,
like.
Why
would
I
run
this
crazy
hacker
pod
on
my
cluster?
That
would
be
madness.
Of
course,
it
would
be
madness.
A
I'm
not
suggesting
you're,
going
to
run
the
hacker
pod
I'm,
suggesting
that
somebody
might
compromise
a
pod
in
your
cluster
and
run
something
like
this
without
you
intending
them
to
you.
So
suppose
you
have
a
pod
that
has
some
kind
of
vulnerability
and
an
attacker
is
able
to
get
into
that
pod
through
that
vulnerability.
Maybe
they're
gonna
start
running
code
like
this
hacker
pod,
but
if
you
have
removed
the
cat
net
or
capability,
they
won't
be
able
to
do
this
particular
exploit.
A
Also
looked
at
the
gatekeeper
admission
controller.
This
is
anybody
here
using
gatekeeper,
okay,
very
small
number
of
hands,
so
gatekeeper
is
using
open
policy
agent,
which
we
heard
about
in
their
keynotes
this
morning,
and
it
has
a
variety
of
these
different
reg,
oh
and
plates,
making
it
easy
to
use
open
policy
agent
basically-
and
they
have
this,
get
what
with
this
repo
called
pod
security
policies,
it
sounds
really
encouraging.
It
says
it
contains
the
common
policies
needed
in
pod
security
policy,
but
implemented
as
constraints
and
constraint
templates
with
gatekeeper,
I
thought
yeah.
A
This
would
be
really
cool.
We
could
use
gatekeeper
to
make
sure
pods
don't
have
that
cat
net
rule
capability,
except
it
does
all
the
things
in
pod
security
policy
except
Linux
capabilities,
which
I
was
bit
disappointed
about.
They
may
be
at
some
point
that
will
be
added
to
gatekeeper,
but
right
now
that
is
not
a
solution
that
will
fix
this
particular
problem
for
us.
A
A
One
is
that
you
can
use
a
layer,
3,
networking,
plugin
and
if
again,
I
can
figure
out
how
to
sort
switch
screens.
I
have
another
single
node
cluster
running
this
one
leave,
as
can
okay
running.
So
let's
just
look
at
the
cube
system:
namespace,
yes,
so
this
is
using
Calico,
which
is
a
layer,
3,
networking,
plugin
and
if
I
try
to
I'm
running
the
exact
same
hacker
pod
here
and
if
I
go
into
that
one
and
I
try
running
exploits.
A
Its
going
to
tell
us
something
that
isn't
exactly
true:
it
thinks
that
it's
running
on
a
different
node
from
keep
DNS.
That's
not
true,
it
says,
keep
DNS.
It
means
the
DNS
service
for
kube.
It's
actually
called
ENS
and
there's
a
single,
no
cluster,
so
I
know
for
sure
they
are
running
on
the
same
node.
But
what
it
means
is
from
the
wallets
expectations.
Are
it's
not
on
the
same?
It's
not
on
the
same
bridge
say,
and
you
can
see
here,
there's
some
yeah.
This
is
not
a
MAC
address
that
you
could
then
packets
to.
A
So,
if
you
were
using
a
layer,
3
network
plug-in,
you
are
safe
from
this,
exploits
how
many
of
you
using
layer
2.
How
many
of
you
don't
know?
Ok,
all
right!
There
is
another
thing
that
you
can
do
to
make.
This
exploit
go
away,
which
is
you
can
configure
core
DNS
to
use
TLS,
that's
actually
a
pretty
simple
configuration
change
and
that
at
least
when
I
tried
it.
A
Apart
from
whether
or
not
it's
you
know,
you
might
know
whether
you're
running
layer,
2
or
layer
3
plug
it,
you
could
try
the
exploit
for
yourself,
because
I'm
gonna
give
you
the
URL,
so
you
can
try
running
it
yourself.
We've
also
built
a
test
for
this
in
to
keep
hunter
okay.
Can
we
get
a
hands
up
if
you've
heard
of
keep
hunter?
Okay,
maybe
about
half
of
you
so
cube.
A
Hunter
is
a
project
that
we
built,
that
does
penetration
testing
for
kubernetes
clusters,
and
it
has
a
whole
series
of
things
that
it
will
attempt
to
do
and
report
back
whether
it
found
anything,
that's
maybe
misconfigured
or
that
kind
of
leaks
information
or
is
an
outright
vulnerability.
And
we
added
a
test
for
this
ops.
Well,
two
tests,
one
for
the
ARP
spoofing
on
one
for
the
DNS,
spoofing
and
I
hope.
If
I
find
the
right
screen.
A
A
And
looks
a
bit
big:
let's
try
making
this,
so
you
can
see
that
it
was
able
to
it
tested
to
find
out
that
it
could
use
cap
net
raw
and
that
there's
a
possibility
of
ARP
spoofing
and
a
possibility
of
DNS
spoofing
so
to
run,
keep
hunter
to
bec
this
effectively.
You
actually
run
cube
hunter
as
a
job
inside
the
cluster.
You
can
also
run
keep
hunter
external
to
the
cluster
and
it
will
do
various
tests,
but
this
is
not
one
of
the
tests
that
it
can
effectively
test.
A
A
So
I've
shown
you
a
whole
load
of
things
there,
which
I
kind
of
hope.
At
least
some
of
you
will
go
away
and
try
out
for
yourselves.
So
the
three
links
there,
the
first
one
is
that
exploits
itself.
So
that's
under
Daniel
Sagi's
repo
and
it's
a
well.
You
saw
it's
a
Python
exploit
and
it
has
all
the
configuration
for
the
hacker
pod
and
the
victim
pod.
A
There's
cube
hunter
penetration,
testing
and
Tracy,
which
is
this
new
tool
for
tracing
out
events
using
EBP
F.
We
would
love
feedback
in
particular
on
Tracy,
because
we've
only
released
that
a
couple
of
weeks
ago,
just
yesterday,
I
had
conversations
with
three
different
people
who
had
three
really
interesting
ideas
about
things
we
could
do
with
Tracy.
So
I'm
really
really
interested
to
hear
what
your
ideas
are
around
that
as
well
and
I've
also
mentioned
that
tomorrow
morning,
myself
and
Michael
housing
blasts
are
going
to
be
signing
books.
A
B
A
D
A
So
what
I'm
free
of
the
bug
yeah?
The
reason
why
that
works
is
because
the
MAC
addresses
are
not
it's
not
like
a
single
lookup
from
Oh
from
the
root
bridge
to
it
from
it
looks
from
the
perspective
of
this
exploit,
as
if
your
pods
are
actually
on
different
nodes,
they're,
not,
but
the
way
it
goes
up
to
lay
up
three
and
back
again
in
calico
I.
E
So
a
very
much
a
cube
nuba
here
am
I
and
I'm
just
wondering
like
if
you
know,
or
if
anybody
knows
of
why
you
wouldn't
just
use
layer,
three
networking
all
the
time
it
seems
kind
of
foolish
to
even
have
a
broadcast
domain
inside
your
cluster.
Is
there
performance
reasons
to
prefer
having
layer,
2,
networking
over
layer,
three
for
some
workloads,
I'm.
B
F
A
G
A
And
so
the
only
thing
that
you
can't
do
if
you
drop
that
privilege
is
open
raw
sockets.
So
unless
you
have
a
reason
to
so
do
that
there
is
no
disadvantage.
Some
of
the
capabilities
are
a
bit
like
kind
of
tied
up
into
more
than
one
thing,
but
this
particular
one
is
very
straightforward.
So
unless
you
have
a
reason
to
write,
Ethernet
packets,
completely
fine
I.
A
C
I'm
also
really
new
to
karate,
so
I,
don't
know
the
networking
details
and
clearly
the
problem
is
a
shared
ARP
table.
That's
the
exploit,
but
is
the
ARP
table
actually
in
the
bridge
or
is
it
somewhere
else
because
normally
in
most
network
I,
don't
see
the
ARP
tables?
The
bridges
are
actually
associated
the
interfaces,
each
Universal
machine
model
they're
those
places
is
the
bridge
really
where
that
our
table
is
in
the
stack.
My.