►
From YouTube: Shipping in Pirate-Infested Waters: Practical Attack and Defense in Kubernetes [A] - Greg Castle
Description
Shipping in Pirate-Infested Waters: Practical Attack and Defense in Kubernetes [A] - Greg Castle & CJ Cullen, Google
Kubernetes has a growing array of security controls available, but knowing where they all fit in, what the highest priorities are, and how it all helps against real attacks is still far from obvious. In this talk we’ll take a vulnerable application, exploit it, install tools, escalate privileges, propagate between containers and gain control of the cluster. At each stage of the attack we’ll demonstrate how proactive steps could have prevented these actions (or at least made them more difficult), from the container build process to writing RBAC/PodSecurity/AppArmor/Network policies, and more. Since configuration of each defence could be the subject of it’s own deep-dive talk, we’ll mainly focus on the big picture of “what” technologies you’d use to configure your cluster securely and “why”.
About Greg Castle
Greg is the tech lead for the Kubernetes and Google Container Engine (GKE) security team at Google. Prior to GKE, Greg worked on the Google incident response team developing open-source investigation tools, and on OS X platform hardening. His pre-Google job roles have included pentester tech lead, incident responder, and forensic analyst.
About CJ Cullen
CJ works on the Google Container Engine (GKE) Security team. CJ has helped develop the Kubernetes authentication and authorization system, as well as building the cluster deployment and management infrastructure of Google Container Engine.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
At
the
moment,
the
community
is
working
really
hard
on
security
controls,
there's
already
a
lot
of
defensive
options,
and
it
might
not
be
all
that
obvious
way
to
start
or
how
to
prioritize
so
we're
not
going
to
cover
all
the
security
best
practices
in
this
talk
it
there's
a
lot
to
cover,
and
so
today's
focus
is
going
to
be
mostly
on
prevention.
We're
not
going
to
be
talking
much
about
detection
or
instant
response
or
application
security,
or
a
bunch
of
other
kind
of
topics
that
we
could
be
talking
about.
A
I've
only
got
35
minutes
and
we're
trying
to
make
it
entertaining
by
having
some
demos
so
we're
focusing
on
cluster
administrator
and
developer
tasks
and
we're
focusing
on
kubernetes
there's,
there's
nothing
gke
specific
in
this.
In
this
presentation,
we
have
a
link
to
a
blog
post
at
the
end
about
that
gives
you
a
whole
bunch
of
hardening
specifics
for
for
gke.
A
So
we're
gonna
keep
with
a
pirate
theme.
The
Cooney's
the
threat
model
assumes
application
compromised.
So
if
you
consider
the
entire
world
of
communities,
communities
clusters
that
are
running
in
production,
there's
there's
lots
of
bugs
there's,
there's
lots
of
vulnerabilities
in
that
whole
spectrum.
So
it's
really
kind
of
what
happens
off
to
a
code
execution.
That's
actually
interesting
for
communities,
security,
and
so
our
goal
is
always
to
be
secure
by
default,
where
we
can.
A
So
we
have
three
demos,
which
is
a
lot
to
fit
in
35
minutes,
but
hopeful
keeps
things
interesting
and
the
idea
is
to
show
you
an
attack
of
learning
clusters
at
different
stages
of
security
evolution.
So
the
first
one
is
our
back
is
off
or
it's
in
a
clan
and
old
cluster
that
doesn't
have
our
back
and
there's
a
fairly
well-known
kind
of
application
to
cross
the
escalation
path.
If
you,
if
you
have
a
cluster
like
that,
so
I
want
to
make
people
aware
of
that.
A
The
next
one
is,
we
kind
of
take
that
cluster
and
apply
some
hardening
to
it,
and
we
try
the
same
attack
and
demonstrate
that
it
doesn't
work.
And
then
we
move
into
like
a
container
breakout
and
privilege
escalation
kind
of
problem
that
can
get
you
access
to
the
cube
way
and
then
from
there
we
apply
some
more
hardening
to
that
to
that
cluster,
and
then
we
run
the
same
attack
and
kind
of
once.
A
You've
closed
off
the
obvious
privilege,
escalations
and
there's
sort
of
the
attacker
looks
around
a
bit
and
there's
there's
kind
of
not
an
easy
way
to
escalate
on
that
node.
They
probably
look
to
propagating
through
the
cluster,
can
I
look
for
vulnerable
services
that
are
running
in
the
cluster
and
so
will
demonstrate
kind
of
how
that
might
work
and
then
talk
about
some
defenses.
For
that
scenario,
too,
so
I
said
we're
going
to
keep
the
pirate
theme
and
we're
gonna
actually
assume
roles
here.
So
I'm
gonna
be
part
preview.
B
A
A
C
I
will
be
your
skipper,
shipper
and
behind
this
well
provisioned
uniform
and
cap
I
hide
some
slightly
lacks
coding
standards
and
a
bit
of
security
naive
die
so
to
demonstrate
some
of
these
attacks
and
defenses
that
are
available
in
kubernetes
we're
we
made
up
this
fake
app.
This
isn't
real,
please
don't!
This
is
not
a
real
thing,
but
this
will.
Let
us
talk
through
some
of
the
things
that
you
might
think
about.
C
So
taking
a
look
at
what
this
app
looks
like
we're,
gonna
have
a
new
member
web
page
that
allow
all
of
our
new
members
to
sign
up.
We
will
store
their
info
in
some
database
in
the
back.
We'll
have
a
payment
processor
that
that
turns
on
the
database
takes
our
new
members
and
charges
them
takes
our
winners
and
pays
them
out
through
some
third
party
payments.
Api,
and
then
you
know
just
so.
D
A
Yeah,
so
here
we
have
our
our
web
front-end
that
skipper
shipper
has
built
so
I've
already
done.
Some
reconnaissance
on
this
web
front-end
and
I'm
pretty
sure.
There's
a
shell
injection
in
this
input
field
here.
So
some
pretty
lacks
coding
standards
going
on
the
skipper
ship
has
been
enforcing,
so
he
looking
from
the
kind
of
doing
from
my
reconnaissance,
I've
kind
of
realized
that
he's
probably
taking
this
this.
This
input
from
the
from
the
internet,
that's
completely
untrusted
and
making
and
chilling
out
and
making
a
call
with
it
without
doing
any
input
sanitization.
A
So
this
is
obviously
a
huge
no-no,
so
I'm
gonna
go
ahead
and
exploit
it,
so
I'm
gonna
close
off
the
previous
command
with
the
semicolon
and
then
download
a
downloaded
backdoor
and
run
the
backdoor,
and
over
here
I've
got
my
Metasploit
listener
and
that's
a
penetration
testing
framework
that
allows
you
to
run
an
attack
and
have
install
these
backdoors
on
systems
and
then
connect
back
to
this
service.
So
this
is
my.
This
is
my
exploitation
server,
so
I'm
gonna
go
ahead
and
hit
submit
on
that,
and
you
can
see.
A
A
So
I
mentioned
that
our
back
is
disabled
in
this
cluster
or
or
it's
in
or
it's
an
old
cluster
that
doesn't
have
our
back
enabled
and
so
since,
in
the
interest
of
time,
I'm
gonna
cut
kind
of
straight
to
the
chase
here
so
I
know
in
in
clusters
that
don't
have
our
back
enabled
there's
a
service
account
that
lives
at
this
location
on
the
file
system.
That's
very
privileged,
so
let's
just
see
if
I
can
see
that
on
this
system
and
I
can
so.
This
is
the
this
is
the
token
here.
A
That's
the
secret
for
that
service
account
so
I'm
gonna
use
Metasploit
to
download
that
token
and
then
I'm
gonna
on
my
own
attacking
machine
just
write
out
a
cube,
config
file.
That
includes
that
token,
so
that
can
make
requests
to
that
to
that
API
server.
So
now,
if
I
switch
over
to
my
attacker
machine.
A
So
this
is
the
cube
config
file
that
I
wrote
out
from
my
attack.
That
includes
the
the
token
that
I
stole-
and
this
is
this-
is
completely
separate
from
the
cluster
now
so
I've
used
Metasploit
to
extract
this
token
out
to
my
own
machine
and
now
I'm
gonna
come
at
the
API
server
from
completely
outside
the
cluster.
Using
that
token
I'm
gonna
try
and
get
pods.
A
That
works,
so
I
can
see
the
pods
that
are
running
in
a
cluster.
What
else
can
I
do
with
this
took
in
so
let's
try
getting
secrets
for
all
namespaces
and
I'm,
just
gonna
grip
out
the
cube
system
stuff
to
kind
of
keep
this
the
output
down
a
little
bit
so
I
have
a
couple
of
kubernetes
service
account
tokens
here,
which
are
not
super
interesting
I
already
have
one
of
those,
but
this
payments,
API
key
sounds
really
interesting,
so
from
from
skipper
shivers
kind
of
overview
of
the
application.
A
This
is
how
money
money
gets,
transferred
and
stuff
gets
stuff
gets
done
inside
the
application.
So
let's
try
getting
that
secret,
because
that
one
sounds
interesting,
so
we're
gonna
get
that
payments,
API
output
in
llamó
format-
and
so
we
can
see
this
here-
is
the
base64
encoded
secret.
So
that's
actually
clear
text.
It's
it's
not
encrypted
in
in
this
view
here.
So
I
now
have
that
payments
API,
secret
and
I
could
make
make
some
requests
for
the
payments
API,
but
it
actually
gets
worse.
A
So
the
this
service
account,
as
I
said,
is
very,
very
privileged
in
the
cluster.
When
you
don't
have
all
back
running
so
I
can
actually
just
exec
anywhere
in
the
cluster
into
any
pod.
So
I
can
see.
There's
this
payments
pod,
so
I'm
just
gonna
run
a
shell
inside
there.
So
now
I've
got
a
shell
from
my
attacker
machine.
All
the
way
into
the
payments,
and
you
can
see
like
the
payments
related
code
here
is
running
in
this
directory,
so
that
seems
kind
of
bad.
C
Yeah,
so
let's,
let's
walk
through
what
happened
there.
Our
our
attacker
was
able
to
find
a
somewhat
obvious,
pretty
simple
shell
injection
into
our
web
facing
front
end
from
there.
He
knew
that
there
was
likely
a
service
account
token,
just
kind
of
hanging
out
inside
of
that
pod.
Using
that
service
account
Tok
and
he
started
probing
around.
He
proved
out
that
it
had
a
little
bit
too
much
access
on
our
API
and
and
from
that
point
he
was
able
to
exact
directly
into
our
payments
pod.
C
C
Obviously
you
want
to
enforce
lease
privilege
on
your
workloads
and
the
the
most
obvious
way
to
do
that
in
kubernetes
is
to
turn
our
turn
our
back
on.
That
means
disable
the
the
legacy
a
back
or
make
sure
you
have
some
authorization
on
on
what
your
service
accounts
are
allowed
to
do
once
you
and
and
that'll
actually
make
it
so
the
service
accounts
have
no
privileges
by
by
default.
You
have
to
grant
them
if
you
want
them.
C
A
A
Again,
we're
gonna.
We
left
the
vulnerability
there,
but
we
have
a
new
cluster
now
with
those
defenses
enabled
same
exploitation
pattern,
and
we
have
our
second
session
open
now,
so
the
Metasploit
actually
allows
you
to
have
multiple
at
a
time.
So
same
thing:
again,
we
have
a
shell,
let's
download
that
service
account
open,
so
downloaded
it
to
my
own
machine,
wrote
out
another
cute
config
and
I'm
gonna
try
the
same
thing.
I
did
last
time,
so
let's
try
and
get
pods
so
in
in
this.
A
Now
that
we
have
our
back
on
this
cluster
that
that
service
account
is
no
longer
privileged.
So
we
can
see
here
the
service
account
in
the
sign-up
namespace.
The
default
service
account
there
can't
list
pods
in
the
namespace
default,
and
we
can
try
some
of
the
other
things
we
tried.
So
stealing
the
payments
API
key
also
fails,
and
we
can
keep
trying
stuff
that
we
didn't
have
lost
Emma,
but
this
by
default,
though
the
service
account
has
no
privileges.
So
this
is
this
is
now
a
dead
end.
A
A
A
A
So
essentially,
what
we've
done
here
is
is
simulate
a
container
break
out
in
a
privilege
escalation
so
to
make
this
demo
a
bit
simpler
and
not
require
us
to
do
all
that.
Work,
we've
simulated,
a
container,
breakout
and
privilege
escalation
by
announcing
too
much
stuff
into
the
container
and
then
being
a
little
bit
crazy
with
our
pseudo
privileges.
A
So
we're
gonna.
Take
the
you
can
see
here
that
the
there's
a
cubelet
count,
client
key
reference
and
a
cubelet
a
client
reference.
So
we
really
gonna
take
copies
of
those,
so
I'm
gonna
copy
them
out
of
that
host
path.
Mount
write
out
a
cube,
config
file
that
uses
them
and
then
we're
gonna,
try
using
the
cubelets
credentials
against
the
API
and
see
what
we
can
do
and
to
do
that.
I
actually
have
to
download
cube
control
because
I'm
running
inside
a
container
I
didn't
have
Q
control
in
there
with
me.
A
So
now
that
I've
done
that
I
can
use
the
cube
config
file,
that
has
the
cubelet
credentials
and
my
copy
of
cube
controller
I
just
downloaded.
So,
let's
see
if
I
can
get
secrets.
So
this
fails.
So
this
is
the
it
says,
this
user,
which
is
actually
the
cubelet
for
this
node.
Countless
secrets
in
a
namespace
default
I
can
only
get
individual
resources
of
this
type.
So
I
can
only
get
individual
secrets,
I
can't
say
all
secrets:
okay,.
A
About
show
me
all
the
parts
then
so
this
is
all
the
pods
in
all
the
namespaces
just
cutting
out
the
cube
system.
Stuff
I
can
do
that
and
the
reason
I
can
do
that
is
cubelet
needs
that
to
do
its
job,
so
people
it
has
to
is
scheduling.
It
needs
to
know
kind
of
what
pods
to
schedule
on
itself.
So
if
I
can
now
that
I
know
the
name
of
a
pod,
maybe
I
can
try
exacting
into
it.
That's
what
I
did
last
time
with
the
service
account
that
really
privileged
service
account
token.
A
A
So
I'm
gonna,
try
grabbing
this
secret
and
I
can
still
steal
that
payments
API
key
secret
and
that's
still
a
little
surprising
really,
because
this
is
the
web
front-end
and
it
has
no
reason
to
be
able
to
need
to
access
that
that
API
back-end
secret
isn't
actually
necessary
for
for
this.
This
front-end
to
function,
so
this
seems
better.
That's
still
not
great.
A
C
Let's
take
a
look
at
what
happened.
Their
attacker
found
the
same
old
challenge
action.
We
didn't
remove
it,
it's
more
fun
that
way.
He
took
a
look
at
the
pod
service
account
token.
Luckily,
it
was
not
able
to
do
anything
in
our
cluster,
but
we
left
him
a
nice
little
obvious
escalation.
He
was
able
to
grab
a
hold
of
the
cubelet
token.
On
the
note,
the
qubit
credentials
on
the
note
that
he
was
running
on
from
there.
He
did
have
some
access
to
the
API.
C
He
was
luckily
not
able
to
exec
directly
into
our
payment
spot
again,
but
he
was
able
to
grab
that
API
key
and
probably
likely
cost
me
a
bunch
of
money
again.
So
what
could
we
have
done
to
prevent
those
obviously
limit
the
local
escalation
paths
that
attackers
might
have
don't
want
to?
You
want
to
avoid
running
his
route,
really
take
a
look
at
what
what
sort
of
host
path
mounts
you
really
need,
and
you
can
enforce
these
sort
of
things
with
APOD
security
policy
from
there.
C
You
want
to
ensure
that
all
of
your
nodes
actually
have
leased
privilege
so
on
kubernetes,
1.7
and
up
this
means
enabling
the
node
authorizer
in
the
node
admission
controller,
and
as
you
saw
in
that
cluster
that
we
were
looking
at,
we
actually
did
have
that
that
authorizer
enabled
that's
what
prevented
the
node
from
being
able
to
get
all
the
secrets.
It
was
only
able
to
get
the
secrets
that
were
relevant
to
pods
co-located
with
that
node.
C
So,
once
you
have
lease
privileges
for
nodes,
you
may
want
to
go
further
and
separate
sensitive
workloads
from
the
more
possible.
You
know
the
web
facing
type
workloads
that
may
be
more
vulnerable
and
and
there's
some
scheduling
primitives
that
allow
you
to
do
that
and
that
would
have
prevented
our
attacker
from
being
able
to
grab
the
key
if
it
wasn't
Co
scheduled
with
him
and
then.
C
Lastly,
you
can
turn
on
certificate
rotation
for
for
those
cubelet
credentials,
and
what
that'll
do
is
make
sure
that
the
attacker
doesn't
have
a
credential
that
he
can
just
take
and
use
it
as
leisure
for
some
long
period
of
time.
The
the
lifetime
of
those
credentials
would
be
shortened.
They'll
be
rotating
out
from
under
him
again.
To
maintain
this
compromise,
our
attacker
would
have
to
maintain
presence
in
our
system
so
yeah,
let's
fix
those
things
up
and
now,
let's,
let's
really
ship
this
thing
again,.
A
So
let's
have
a
look
around
and
see
how
we
did
with
the
hardening
and
removing
those
those
problems
from
the
previous
cluster.
So,
let's
see
if
we
still
got
that
directory
from
the
cubelet,
we
don't
have
that
anymore.
Let's
take
a
look
at
the
amounts
to
see
if
there's
anything
crazy
in
there
and
they
look
much
more
reasonable.
We
don't
have
all
those
all
those
hosts
path
mounts
so
at
at
this
point.
We
don't
have
that
kind
of
obvious
cubelet.
A
It's
got
a
escalation
path
to
break
out
of
the
container
we
can
relate
without
that
vulnerability.
I
can't
really
demonstrate
having
the
the
scheduling
changes
that
we
made
that
now
the
now
the
the
sign-up
pod
is
scheduled
on
a
different
node
to
the
the
payments
back-end.
So
if
we
were
to
steal
secrets,
we
wouldn't
be
able
to
get
that
payments
API
key.
So
let's
assume
now
that
we're
kind
of
attack
has
landed
in
this
cluster
I'm,
like
oh,
damn,
no
super
obvious
privilege
escalations
no
obvious
route
stuff.
A
Let's
take
a
poke
around
in
this
cluster
and
see
if
we
can
propagate
elsewhere.
Let's
look
for
some
vulnerable
services.
That
kind
of
stuff,
so
really
like
traditional
obvious
way
to
do
this,
is
using
the
nmap
utility,
which
is
a
pullet
scanner.
So
I've
downloaded
a
version
of
nmap,
that's
statically
compiled
and
I'm
gonna
run
a
scan
across
just
looking
at
port
80
across
the
services.
A
Ip
address
range,
there's
a
slash
20
that
is
used
for
the
services
in
this
cluster.
This
takes
like
two
minutes
to
run,
which
is
pretty
much
an
eternity
for
this
presentation,
so
this
is
the
only
thing
that
isn't
live,
so
here's
one
I
prepared
earlier.
This
is
the
end
map
scan
results.
So
you
can
see
here
that
I
have
a
sign
up,
sign
up
front
end
running
on
port
80
I,
have
a
payment
service
running
on
port
80,
as
well
as
some
like
internal
kubernetes,
other
stuff
heaps
to
a
dashboard,
etc.
A
A
So
the
really
an
only
really
interesting
part
of
that
hTML
is
that
it's
a
directory
listing
that
shows
us.
There's
a
payments
PHP
app
now
we
could
actually
like
just
kind
of
stop
the
attack
here,
because
this
is
kind
of
demonstrating
the
problem
that
I
shouldn't
be
out
of
get
to
this
payment
service.
From
from
this
front-end
API
of
further
this
front-end
pod
that
I've
that
I've
compromised
but
I
think
I
want
to
make
it
rain
and
I.
Think
we've
I
think
we've
been
talking
about
payments,
a
lot
but
not
actually
getting
paid.
A
A
A
C
Let's
talk
that
one
through,
so
what
we
have
there
same
shell
injection
same
service
account
token
still
no
permissions
no
way
to
escalate
directly.
That
was
obvious.
So
at
that
point
our
attacker
starts
doing
the
normal
attack,
things
and
kind
of
probing
around
see
even
what
he
might
have
access
to,
and
unfortunately
he
was
able
to
directly
call
this
this
payment
service.
Now,
when
I
designed
this
wonderful
app
I,
never
imagined
that
my
signup
form
would
have
any
reason
to
contact
my
payment
service
directly.
C
So
so
this
was
very
surprising
that
it
would
have
this
access,
but
our
attacker
found
this
made
a
call,
and
you
know
cost
me
some
money.
So
what
could
I
have
done
there?
As
you
separate
your
applications
out
into
micro
services?
There
are
these
sort
of
natural
security
boundaries
where
you
you
get
to
start
thinking
through
what
pieces
of
my
system
should
really
be
talking
to
which
other
pieces
and
in
kubernetes
the
network
policy
object
in
in
1.7
and
later
lets.
You
start
to
define
and
enforce
that.
C
So
in
our
case
that
means
an
ingress
policy
on
the
payment
app
based
on
the
label.
That
says
that
I
will
only
accept
connections
from
things
that
have
the
label
admin
portal.
So
the
only
thing
that
should
be
true
directly
talking
to
my
payment
processor
is
this
admin
portal
that
I'm
helpfully
providing
my
admins,
then
the
other
thing
that
in
the
past
was
often
floating
around
the
network
was
the
the
cubelet
api
and
you
really
want
to
make
sure
that
you
have
authentication
and
authorization
turned
on
to
that.
A
Okay,
so
we
mostly
focused
on
well
we've
entirely
focused
on
existing
defenses
and
talking
about
prevention,
so
just
want
to
sum
up,
try
and
give
you
some
some
kind
of
takeaways.
We
talked
about
a
lot
of
different
defenses
and
to
try
and
help
a
little
bit
with
priority.
So
hopefully
we
fairly
convincingly
made
an
argument
for
turning
on
our
back.
A
B
A
Another
really
important
part
of
this
is
keeping
up
with
the
kubernetes
releases
you'll
see
in
all
those
defensive
slides.
We
were
talking
about
the
versions
where
those
defenses
are
available,
and
you
know
a
lot
of
them
are
kind
of
one
six,
one,
seven
fairly
recent
versions:
that
pace
is
going
to
continue,
and
so
there's
really
a
lot
of
value
in
keeping
up
with,
with
with
the
latest
versions
of
kubernetes,
so
I'd
like,
if
you're
not
doing
either
of
those
I
consider
those
kind
of
probably
the
hardest
priorities
from
there.
A
The
having
a
really
kind
of
small
container
OS
actually
makes
a
big
difference.
When
an
attacker
lands
in
your
cluster
they're
kind
of
more
comfortable,
they
are
the
easier
it
is
for
them,
so
it
should
be
fairly
uncomfortable
when
he
landing
in
a
cluster,
so
at
one
extreme,
that
would
be
like
kind
of
building
like
scratch
containers,
but
they
get
really
hard
to
debug,
so
at
least
a
really
minimal
or
less
like
what
we
have
with
cause
on
gke
and
then
thinking
about
kind
of
carefully
thinking
about
you
know.
A
Do
I
really
need
w
getting
coal
in
this
in
this.
In
this
container,
do
I
need
a
shell
and
there's
trade-offs
with
debugging
there,
but
all
those
things
make
attacker
actions
a
lot
more
easy,
a
lot
easier
if
they're
available
and
obviously,
as
we
said,
no
root,
no
host
path,
Network.
What?
Where,
where
you
don't
need
it
and
you
can
enforce
those
things
with
pod
security
policy
at
the
cluster
level
and
then
the
as
kind
of
siege
I
mentioned
earlier.
A
The
this
decomposition
into
micro
services
gives
you
these
really
natural
boundaries
for
separate
or
segregating
out
your
your
cluster
and
then
building
network
policies
and
dedicated
nodes
to
kind
of
make
propagation
harder
once
an
attacker
does
land
in
there.
Making
it
as
hard
as
possible
to
get
to
the
other
places
in
your
cluster
is
either
really
valuable
thing
that
that
isn't
hire
fit
and
there's
a
whole
bunch
of
companies
that
are
doing
work
on
kind
of
learning,
network
networking
patterns
and
then
helping
you
enforce
network
policies
kind
of
automatically.
A
So
you
don't
necessarily
have
to
write
that
bright.
Those
manually
yourself,
so
we
we
have
a
great
community
I!
Think
it's
a
god.
We
have
some
really
stellar
security
engineers.
Not
talking
about
myself
but
dealer,
but
definitely
the
other
people
who
were
working
in
sigilyph,
I'm,
I'm
I,
don't
know
I'm
very
I'm,
very
glad
to
be
working
with
this
community,
because
I
think
there's
some
some
really
great
security
expertise
in
it
and
we'd,
like
other
people
to
get
involved.
So
we
meet
every
Wednesday
for
two
every
two
weeks.
A
Every
two
weeks
on
Wednesdays,
obligatory,
shout
out
from
my
team
in
Seattle
was,
is
hiring
so
if
the
intersection
of
security
and
committees
is
interesting
to
you,
please
come
talk
to
me
and
here's
a
whole
pile
of
links,
so
everything
that
we
talked
about
today
in
terms
of
the
defense
is
on
this
slide
with
a
link.
I
definitely
want
to
call
out
again
the
gke,
hardening
blog
post
and
there's
a
the
second
link.
A
There
is
a
living
document
for
securing
a
kubernetes
cluster
that
you
know
won't
age
like
this
presentation
will
age,
as
as
things
kind
of
change
and
a
couple
of
other
call-outs.
So
if
you're
expecting
to
see
kind
of
security
roadmap
of
features
that
sigilyph
is
working
on
the
in
this
talk,
we
didn't
have
time
to
do
that.
We
kind
of
wanted
to
deliver
here's,
why
what
you
should
do
and
why
I
should
do
it,
but
Jordan
Leggett
is
giving
a
singles
update.
That
has
a
a
good.
B
A
A
A
C
So
I
think
the
question
is:
what
consideration
have
we
given
to
enforcing
l-3
level
partitioning
based
on
l7
level,
traffic
being
able
to
automatically
discover
that
sort
of
thing,
I?
Think
from
the
kubernetes
standpoint?
We
don't
want
to
be
really
really
opinionated
about
that
I.
Think
different
people
have
different
opinions
about
how
you
want
to
think
about
application
security
inside
of
your
cluster,
and
we
want
to
make
the
tools
available
such
that
the
information
is
there,
that
you
can
build
on
top
of
it,
but
the
the
endgame
is
definitely
up
to
you.
I
think
I.
A
Think
to
like
in
that
space,
like
that's,
probably
three
or
four
vendors
on
within
the
floor
right
now
that
are
like
that
have
built
systems
exactly
to
do
that.
So
it
I
think
that
the
community
is
kind
of
perspective.
Is
that
you
know
we
want
to
keep
things
out
of
core
where
they
didn't
really
need
to
be
in
court
and
then
provide
opportunities
for
other
people
to
build
tooling
on
top
and
where
there's
already
kind
of,
like
that's
already
happening
and
I.
A
Yeah,
so
it
was
actually
in
the
in
the
second
DOMA.
It
was
actually
just
so.
The
question
was:
why
could
you
get
that
that
payment
secret
if
the
payments
thing
wasn't
scheduled
on
the
on
the
node,
where
you
were?
Where
would
compromise
it
so
in
the
in
the
first
demo
it
actually
was,
we
made
sure
it
was.
It
was
Co
scheduled
and
then
kind
of
like
to
fix
it.
A
We
separated
it
out,
but
yeah,
that's
exactly
what
the
node
authorizer
and
admission
does
it
in
make
sure
like
yeah,
if
the
the
blast
radius
for
compromise
of
a
single
node
should
be
the
secrets
that
I
could
further
pods
that
are
currently
scheduled
on
that
node
and
not
all
the
other
secrets
in
the
system.
So
that's
yeah
as
of
1:7.
That
was
a
case.
D
A
So
there
is
a
bit
of
a
weakness
in
the
init
ain't
scintillations
mechanisms
at
the
moment,
so
you,
the
cubelet,
can
change
its
labels.
That's
exactly
one
of
the
things
that
sorry.
The
question
was
like:
if
you
have
the
cubes
credentials,
could
could
you
change
something
to
get
the
other
stuff
scheduled
on
you,
where
it
isn't
supposed
to
be
that's
yeah
and
so
yeah?
There
is
that
weakness
at
the
moment
and
that's
definitely
something
we're
working
on
in
in
2018
we're
gonna
close
that
one
off
I
would.
A
They'd
have
to
change
the
labels,
so
if
you're
using
the
taint
scintillations
and
the
label
selectors
to
put
them
on
different
pods
and
the
cubic
and
change
those.
So
I
think
it's
still
worth
doing
that
at
the
moment
anyway,
because
the
the
protections
are
going
to
come
in
and
you
already
be
set
up
to
set
up
these
own.
C
The
question
was:
is
that
the
case,
even
when
you
download
cube
CTL
and
yeah,
it
just
depends
on
what
identity
you're,
acting
as
what
credentials
you
grabbed
so
cube?
Ctl
is
just
gonna
use,
whatever
identity
you
were
able
to
grab
and
if
that's
a
node
identity,
a
cubelet
identity,
you're
restricted
by
the
permissions
that
that
cubelet
has
yes.
A
So
the
prerequisite
would
be
root
on
the
node
to
get
access
to
those
those
cubelet
credentials.
So
you'd
have
to
do
contain
a
breakout,
privilege
escalation,
steal
the
qiblah
credentials
then
do
this.
So
there
is
definitely
like
a
that's,
not
a
trivial
process.
There's
the
serious
work
involved
in
that
yeah.
A
A
Yeah,
so
the
question
is
like:
how
do
you
respond
to
an
attack
like
this,
like?
How
do
you
detect
it?
How
do
you
respond
to
it?
We
didn't
really
cover
that
at
all,
there's,
definitely
a
bunch
of
stuff
you
can
do
so.
One
really
basic
thing
you
definitely
want
to
do
is
collect
the
API
master
or
audit
logging.
So
over
all
those
interactions
we
were
making
with
the
API
server
or
get
logged,
and
then
you
can
build
detections
on
those
on
gke.
A
They
all
go
into
a
stack
driver
and
if
you
enable
logging-
and
you
can
do
really
quite
powerful,
querying
and
detection
on
the
API
stuff,
there
I
think
there's
also
the
scope
and
a
bunch
of
products
kind
of
operating
in
the
space
of
detecting
stuff
on
the
node
itself,
and
so
there's
a
bunch
of
people
building
products
right
now
to
do
you
know,
detection
of
unusual
events,
things
that
are
happening
on
the
node
itself.
Is
there
anything
else,
you'd
add
to
that
yeah.