►
Description
Recently, 2 CVE disclosures (CVE-2020-8555, CVE-2020-8552) made by the community – the first affects the control plane and the latter is network security related that can be exploited to hijack host and other containers traffic.
In this webinar we will review a holistic preventative prescription to Kubernetes security and how it can be used to detect and prevent exploits of this kind and others.
Presenter:
Gadi Naor, CTO & Co-Founder @Alcide
A
Okay,
all
right
we'll
go
ahead
and
get
started.
Hi
I'd
like
to
thank
everyone
who
is
joining
us.
Welcome
to
today's
cncf
webinar
kubernetes
security
anatomy
and
the
recently
disclosed
cves,
I'm
christy
tan
I'll,
be
moderating.
Today's
webinar
we'd
like
to
welcome
our
presenter
today,
gaddy
noor,
cto
and
co-founder
of
outside
a
few
housekeeping
items
before
we
get
started
during
the
webinar.
You
are
not
able
to
talk
as
an
attendee.
There's
a
q,
a
box
at
the
bottom
of
your
screen.
A
Please
feel
free
to
drop
in
your
questions
in
there
and
we'll
get
to
as
many
as
can.
In
the
end,
this
is
an
official
webinar
of
the
cncf
and,
as
such
is
subject
to
the
cncf
code
of
conduct.
Please
do
not
add
anything
to
the
chat
or
questions
that
would
be
in
violation
of
that
code
of
conduct.
Basically,
please
be
respectful
of
all
your
fellow
participants
and
presenters.
A
B
While
I
submitted
the
original
slides
for
this
talk,
we
actually
had
only
two
cves
as
part
of
this
webinar
and
since
then
there
were
more
three
more
cves
disclosed,
which
means
that
this
is
a
great
area
for
anyone
that
is
kind
of
adopting
kubernetes
as
part
of
their
cloud
native
application
infrastructure.
B
By
way
of
me
hosting
you
guys
today,
as
far
as
where
I'm
from
I'm
happily
locked
down
or
grounded,
I
should
say
in
in
tel
aviv,
israel-
and
it
looks
like
it's
going
to
stay
like
that
in
the
foreseeable
future.
Unfortunately,
other
than
doing
cloud
native
fun
stuff,
I
try
to
enjoy
skateboarding,
which,
happily
I
I
managed
to
do
that
more
often
than
than
usual.
B
Because
of
the
new
reality
and
by
way
of
a
professional
experience,
I
spent
most
of
my
career
as
a
kernel
developer
in
in
the
past
few
years,
since
starting
alcide,
which
is
a
pure
play,
kubernetes
security
company,
which
is
focused
on
delivering
end-to-end
kubernetes
security.
I
am,
I
got
an
upgrade
to
deal
with,
only
the
fun
stuff
which
is
containers,
kubernetes
and
and
dealing
with
building
scalable
systems
and
securing
them.
B
So
we
are
going
to
start
with
an
overview
that
kind
of
takes
us
to
the
four
c's
and
and
which
is
cloud
cluster
containers,
which
is
somewhat
kind
of
extended
to
kubernetes
and
code
and
we'll
see
what
are
the
building
blocks
that
you
really
need
to
think
of
when
we
are
trying
to
build
a
practice
or
thesis
around
our
kubernetes
security,
hopefully
I'll
be
able
to
show
you
some
demos
that
are
relevant
to
the
topics
that
I
am
going
to
discuss
in
this
session,
as
well
as
discuss
the
recently
disclosed
cves.
B
As
I
mentioned
there,
there
were
five
of
them
and
also
talk
about
detections
and
preventions
in
the
context
of
those
cves,
so
without
further
ado
I'll
dive
right
in
so
when
we
are
really
trying
to
think
about
what
are
the
kubernetes
security
building
blocks.
This
is
a
relatively
complex
topic
and
it's
not
because
kubernetes
is
not
complex.
It's
because
security
by
itself
is
a
complex
topic,
and
when
we
really
try
to
think
about
what
do
we
want
to
achieve
with
improving
or
implementing
security
controls
around
our
environments?
B
We
really
need
to
think
about
how
we
build
our
clusters.
How
we
access
our
clusters,
how
we
operate
our
clusters
and
even
before
that,
what
exactly
are
we
doing
as
far
as
the
code
that
we
are
shipping
into
our
infrastructure?
So,
let's
start
with
the
easy
stuff
which
we
are
not
going
to
talk
very
much
in
depth
in
this
webinar.
B
So
naturally
we
have
the
traditional
practices
of
scanning
our
code
with
code,
analyzers,
there's
whole
religious,
around
security,
application,
testing
and
dynamic
application,
security,
testing
and
and
software
component
analysis,
which
pretty
much
kind
of
involves
the
large
challenges
or
the
new
challenges
that
we
have
with
adopting
and
bringing
in
open
source
components
as
part
of
our
applications
and
once
we
kind
of
feel
comfortable
around
the
practices
that
the
code
analysis
brings
us.
We
really
want
to
think
about
how
we
package
them
in
containers.
B
So
we
want
to
think
about
container
image
scanning
and
the
main
motion
that
we
see
with
the
security
building
blocks
that
we
want
to
apply
in
this
context
mainly
revolves
around
integrated
security
controls
to
our
development,
life
cycle
and
development
kind
of
practices,
as
well
as
our
ci.
But
once
we
move
from
this
point
and
then
think
about
the
kubernetes
environment
that
we
are
spinning
up,
so
we
really
need
to
think
about
two
more
c's.
The
first
one
is
the
cloud.
B
Right
so
I
am
for
the
sake
of
completeness.
This
is
the
entire
slide
and
right
now
I
am
at
the
two
additional
c.
So
we
talked
about
the
code
and
we
really
want
to
talk
about
the
cloud
and
the
cluster
so
essentially
when
we
are
deploying
our
kubernetes
or
provisioning
our
clusters
into
the
cloud,
we're
willing
to
think
about.
First
of
all,
how
we
secure
our
cloud
account.
So
this
is
an
entire
topic
by
itself
that
we
really
need
to
kind
of
figure
out.
How
do
we
provision
our
kubernetes
clusters?
B
How
do
we
segment
them
from
other
parts
of
our
cloud
infrastructure
or
services?
How
do
we
regulate
access
to
the
users
to
the
cloud
account,
because,
essentially
this
is
something
that
can
be
prerequisite
to
access
our
kubernetes
environment,
but
when
we
are
starting
to
think
about
the
kubernetes
cluster
itself,
so
we
want
to
talk
about
what
exactly
are
we
doing
to
secure
the
control
plane,
which
is
the
brain
of
our
cluster,
and
we
are
leveraging,
for
example,
managed
kubernetes
services
like
eks
or
aks?
B
Then
the
cloud
providers
actually
takes
a
lot
of
the
burden
of
from
us
in
terms
of
securing
the
control
plane
and
the
management
plane
of
the
cluster
and
and
we'll
see
that
slightly
in
depth
later
as
far
as
identity,
like
as
we
think
we
try
to
think
about
who
do
we
allow
access
to
the
cluster?
So
the
first
thing
we
want
to
make
sure
is
to
establish
the
identity
of
users
as
well
as
service
accounts
that
are
accessing
the
cluster.
B
So
here
we
want
to
make
sure
that
we
have
very
strong
identity
controls
in
terms
of
accessing
the
cluster.
So
we
want
to
have
something
like
oidc,
which
pretty
much
enables
us
kind
of
stronger
authentication
and
making
sure
that
credentials
are
expired
in
relatively
short
time
intervals.
So
we
can
make
sure
that
users
are
not
able
to
access
the
cluster
in
a
relatively
permissive
way.
B
If
you
try
to
think
about
a
little
bit
of
what
happened
with
twitter,
it's
I
mean
now
that
I'm
trying
to
be
kind
of
going
depth
there,
but
essentially
there
was
some
social
engineering
like
human
social
engineering
elements.
So
if
things
are
breaking
through
our
identity
systems,
which
is
what
happened
with
twitter,
think
about
the
blast
radius,
that
would
happen
in
terms
of
your
kubernetes
cluster.
B
If
your
highly
privileged
sre
is
performing
actions
against
the
cluster,
and
we
will
see
the
relevance
of
that
in
the
context
of
the
control
plane,
cves
that
were
disclosed
now,
once
we
move
from
the
identity
to
our
bach,
we
really
want
to
make
sure
that,
as
far
as
authorization
would
basically
regulate,
who
is
allowed
to
do
what
we
have
the
right
controls
in
place
and
those
controls
should
first
off
regulate
what
users
are
allowed
to
do,
as
well
as
what
service
accounts,
essentially,
operators
and
and
automated
processes
that
are
running
from
within
the
cluster
and
accessing
the
api
server,
validating
admission
controllers
and
mutating
admission
controls.
B
You
can
really
think
about
them
as
extensions
to
the
kubernetes
arbuck
policies
where,
whenever
the
arbuck
policies
are
not
not
granular
enough
or
you
really
want
to
do
some
sort
of
operations
that
extend
those
policies,
we
can
really
turn
into
leveraging
the
extensible
parts
of
kubernetes,
which
are
in
the
form
of
those
admission
controllers.
B
Once
we
move
from
our
infrastructure
to
our
workloads,
then
this
is
where
things
become
extremely
interesting,
so
we
first
want
to
make
sure
that
we
run
our
workloads,
our
application
workloads
and
those
can
be
also
our
infrastructure
workloads
in
the
best
secured
manner,
in
the
sense
that
we
are
not.
We
do
not
want
to
run
anything
privileged.
We
want
to
make
sure
that
we
don't
create
services
of
type
load
balancer
that
enable
access
into
the
cluster.
B
That
is
something
that
arbuck
by
itself
is
insufficient
port
security
policies
for
those
who
know
or
or
actually
miss,
that
will
not
see
the
day
of
life
in
terms
of
ga
and
presently
the
at
least
one
element
that
is
assumed
to
be
replacing
part
security
policies
is
opa,
open
policy,
agent
and
gatekeeper,
which
by
itself
has
some
challenges
because
part
security
policies
are
restrictive
by
nature,
which
means
that
if
you
deploy
them,
then
it
might
break
your
cluster
on
one
way.
B
But
on
the
other
hand,
you
will
be
running
in
accordance
to
very
strict
policy
profiles
in
terms
of
how
your
pods
and
containers
are
running.
But
on
the
other
hand,
if
you
go
the
opa
gatekeeper
path,
then
you
really
need
to
manage
pretty
much
discrete
features
and
making
sure
that
you
apply
those
controls
in
in
an
additive
manner,
which
is
quite
different
than
how
pot
security
policy
works.
But
at
the
end
of
the
day,
if
you
think
about
it,
our
workloads
or
clusters
never
run
with
workloads
that
are
not
privileged.
B
So
some
of
the
infrastructure
components,
for
example,
must
run
with
some
privileges
with
respect
to
the
host
itself.
With
some
privileges
to
with
respect
to
the
network,
with
some
privileges,
with
respect
to
the
api
center
and
at
the
end
of
the
day,
this
is
something
that
we
really
want
to
make
sure
that
we
have
well-established
guardrails
in
terms
of
both
understanding.
B
Once
we
move
into
things
like
secrets,
which
has
their
own
lifespan,
which
is
different
than
the
cluster
itself,
it
makes
up
becomes
extremely
challenging
challenging
and
we
want
to
revert
to
stuff
that
or
or
or
components
that
help
us
manage
secret
life
cycle
outside
the
cluster
itself.
B
So
things
like
hashicorp,
vault
or
dynamic
secret
injection
or
leveraging
things
like
pod
identity
injection
into
the
pod,
by
components
like,
for
example,
imrols
for
aws
and
eks
clusters
are
really
things
that
help
us
integrate
our
kubernetes
workloads
in
a
way
that
kind
of
natively
integrates
with
cloud
services
or
or
cloud
kind
of
offering
that
do
not
require
to
wire
secrets
in
in
in
a
weird
way
or
in
a
way
that
essentially
can
be
bypassed.
B
Network
policies
are
an
extremely
important
component.
We
will
see
that
shortly
when
it
comes
to
the
cves.
So
what
exactly
happened
as
far
as
those
security
building
blocks
and
the
recent
cves
that
were
discovered?
B
So
what
we
really
see
here
is
the
trust
boundaries
of
a
typical
kubernetes
environment,
where
we
really
want
to
make
sure
that
the
code
that
we
bake
into
containers
is
something
that
we
fully
understand
like,
which
are
the
source
container
image
registries
from
which
we
pull
images
into
the
cluster,
because
this
is
kind
of
a
thread
vector
that
inject
injecting
kind
of
malicious
images
to
our
container
image
registry
is
something
that
eventually
can
land
into
our
cloud
account
our
cluster
worker
nodes
and
then
from
there.
B
An
attacker
can
actually
start
to
manipulate
the
cluster
and
and
perform
those
lateral
movements
to
obtain
or
or
get
a
hold
of,
more
valuable
assets
and,
in
the
worst
case,
exfiltrate
data
from
the
cluster.
But
we
really
see
several
vulnerabilities
that
were
disclosed.
Two
of
them
are
relevant
to
the
control
plan,
to
the
management
of
kubernetes,
even
in
the
event
of
the
managed
kubernetes
services
and
three
more
vulnerabilities
that
are
related
to
the
working
node
themselves
and
let's
take
a
deeper
dive
on
that.
B
So,
just
to
summarize,
you
really
want
to
know
what
exactly
you
guys
are
trying
to
achieve
as
far
as
the
security
posture
that
you're
building,
as
well
as
understanding
your
clusters.
So,
for
example,
we
know
that
in
every
cluster
we
have
infrastructure
components
and
we
want
to
make
sure
that
we
account
for
them
from
a
security
perspective
in
the
sense
that
we
know
that
they
are
supposed
to
run
privilege.
B
We
know
that
they
are
supposed
to
be
running,
for
example
like
prometheus
or
security
agents,
so
we
want
to
have
a
mechanism
that
enables
us
to
kind
of
manage
those
exceptions
in
our
environment.
The
second
piece
is,
if
you
guys
are
running
with
multiple
clusters,
automating
and
regulating
security
across
multiple
clusters
requires
some
unification
of
the
security
infrastructure
that
we
want
to
implement.
It's
not
straightforward
to
do
that.
B
If
there
is
a
difference
between
application
that
there
are
element
that
are
application
specific,
but
if
there
are
shared,
for
example,
method
for
provisioning,
the
cluster
that
you
can
really
share,
the
security
building
blocks
and
policies
and
guard
wealth
that
you
implement
multi-cloud
becomes
extremely
challenging.
If
you
think
about
the
the
sea
that
cover
the
cloud
aspect,
clouds
can
be
quite
different
than
other
from
one
another,
especially
around
the
layers
that
configure
the
account
the
credentials.
B
The
networking,
even
though
kubernetes
is
appears
to
be
something
that
denormalize
or
abstracts
many
of
the
cloud.
The
underlying
cloud
elements.
There
are
still
cloud
specific
elements
to
kubernetes
itself.
So
again,
multi-cloud
becomes
extremely
challenging
as
far
as
the
security
life
cycle
of
components
that
are
running
inside
the
cluster,
for
example,
secrets
api
keys,
user
identity
certificates.
B
All
of
those
needs
to
be
well
thought
in
terms
of
how
we
bring
them
into
our
applications
and
when
it
comes
to
shifting
left
or
or
the
devsecops
culture
that
we're
trying
to
build
as
far
as
bringing
more
security
or
delivering
security
through
awareness
and
implementation.
That
is
already
done
in
the
development
cycle.
So
you
can
really
implement
that
in
ci
by
leveraging
open
tools
or
commercial
tools
which
are
quite
powerful
in
that
sense.
B
So,
let's
take
a
look
quickly.
Look
at
what
does
it
mean
to
scan
our
environment,
so
I
have
prepared
here-
and
this
is
something
that
you
can
kind
of
register
and
create
a
nice
dashboard
for
your
environment.
So
I
have
here
a
little
cluster
that
I
created
as
part
of
this
for
this
webinar.
B
It's
a
kubernetes
kind,
cluster,
it's
a
single
node,
pretty
simple,
doesn't
do
really
much,
but
this
is
something
that
is
based
on
kubernetes
version
18.2
and
one
of
the
things
that
you
can
achieve
with
a
seed
advisor,
for
example,
is
the
ability
to
define
scan
policies,
and
specifically,
I
define
a
policy
that
helps
me
identify
which
kubernetes
cves
the
kubernetes
infrastructure
cvs.
B
Now
vulnerable
doesn't
mean
exploitable,
but
it
really
means
that
presently,
the
kubernetes
software
version
that
I'm
running
in
this
specific
environment
is
vulnerable
to
those
following
cves
and
you
can
pretty
much
drill
down
into
the
individual
details
and
really
understand
like
what
exactly
is
happening
in
terms
of
what
this
cv
is
about,
whether
it's
relevant
to
the
control,
plane
or
the
nodes,
and
essentially,
when
you
try
to
think
about
vulnerabilities
that
are
relevant
to
your
cluster
nodes,
it's
relatively
harder
to
upgrade
and
maintain
the
nodes
themselves
pretty
much.
B
If
you
are
running
stateful
applications,
it
will
take
you
longer
to
upgrade
the
underlying
kubernetes
software
pretty
much,
because
you
need
to
account
for
the
data
that
is
running
there
and
make
sure
that
you
are
rotating
the
nodes
or
or
doing
kind
of
this
rolling
upgrade
in
a
way
that
preserve
the
data.
So
you
really
need
to
kind
of
build
your
cluster
in
a
way
that
helps
you
deal
with
those
maintenance
windows.
B
So
this
is
an
example
where
a
seed
advisor
our
cloud
sas
can
monitor
your
kubernetes
clusters
for
your
cves
and
naturally,
you
can
define
multiple
scan
profiles.
For
example,
those
that
enables
you
to
see,
for
example,
port
security,
analysis
of
your
environment
and
we'll
see
that
down
the
road
when
we
are
talking
about
cves
and
how
we
can
deal
with
that.
So
what
are
seriously
is
and
why
they
are
important
and
why
everybody
is
talking
about
vulnerability
scanning
and
what
does
it
mean
in
the
context
of
kubernetes?
B
B
So
in
the
four
six
weeks
since
kind
of
I
started
to
do
the
slides
for
this
talk,
there
were
five
cves
disclosed
the
interesting
part
here
and
I'm
taking
here
a
wild
guess
that
we
will
see
five
six,
so
eight
five,
five
five
six
disclose
at
some
point
in
times.
The
ideas
that
we
see
here
do
have
a
meaning
kind
of
their
sequential
in
a
way.
B
But
it
looks
like
something
is
going
to
be
released
about
it,
that's
kind
of
a
guess
I
don't
know
for
sure,
but
at
the
end
of
the
day,
we
had
a
couple
of
cves
that
talks
about
the
control
plane
and
the
basic
idea
around
it
is
we
want
to
make
sure
how
or
what
is
the
risk
or
the
blast,
radius
that
are
associated
with
each
and
every
vulnerability
or
weakness
that
we
have
in
our
environment,
and
we
want
to
make
sure
that
we
can
understand
in
terms
of
the
blast
radius.
B
What
are
the
components
inside
our
cluster
that
may
risk
our
environment
in
a
way
that
can
can
someone
can
really
exploit
those
vulnerabilities?
So,
for
example,
the
the
one
that
actually
was
disclosed
last
week,
talks
about
a
pod
that
is
able
to
inflict
a
denial
of
services
essentially
can
take
down
the
entire
node,
while
the
kubernetes
control
plane
is
not
aware
of
the
state
of
the
node
itself.
B
So
this
is
something
that
can
be
done
with
a
pod
that
can
mess
in
quotes
with
the
storage
of
the
node
itself
and
at
the
end
of
the
day,
if
a
pod
or
if
you're,
running
workloads
that
are
coming
from
an
untrusted
source,
for
example,
take
build
systems
and
if
those
build
systems
are
running
code
that
can
manipulate
the
node,
they
can
inflict
a
denial
of
service
on
the
node
and
potentially
on
the
entire
cloud
service.
That
may
may
be
hosting
that.
B
So
this
is
kind
of
one
example
of
how
we
try
to
think
about
it.
So
one
major
issue
that
was
disclosed
a
few
weeks
ago
was
really
talking
about
men
in
the
middle
attack,
where
pods
with
enough
privileges
could
have
been
send
malicious
traffic
or
actually
it's
not
malicious
traffic.
But
it's
traffic
that,
at
the
end
of
the
day,
is
ipv6
router
advertisement
traffic,
which
basically
confuses
the
forwarding
play
so
think
about
calico,
for
example,
which
was
vulnerable
to
this
attack,
or
even
a
default.
B
Kubernetes
cni,
which
was
vulnerable
to
this
attack,
confuses
the
underlying
forwarding
mechanism,
the
routing
mechanism
of
your
cni,
the
container
networking
interface,
in
a
way
that
the
pod
can
get
a
hold
of
a
traffic
that
is
destined
to
the
host
or
destined
to
neighboring
pods.
Now
the
prerequisite
for
the
attack
is
the
ability
to
craft
ipv6
traffic,
and
this
is
a
very
specific
process
level
capability
that
requires
an
attacker
to
be
able
to
mount
such
an
attack.
B
Mitigating
the
attack
would
kind
of
help
us
if
we
could
have
blocked
any
egress
traffic
from
pods
in
a
way
that
you
know
nobody
can
run
ipv6
or
inject
traffic
into
our
pods
or,
for
example,
if
we
disable
the
very
specific
os
kernel
feature
that
enables
the
support
of
ipv6
router
advertisement.
Now,
if
you
try
to
think
about
it,
why
ipv6
is
relevant
to
your
clusters
if
you
are
only
using
ipv4
and
ipv6
is
some
some
sort
of
a
legendary
thing
that
everybody
is
talking
about,
but
not
really
implementing?
B
Well,
the
answer
is
that
out
of
the
box,
linux
systems
are
equipped
with
a
working
stack
of
ipv6
and
the
fact
that
malicious
pod
or
an
attacker
on
the
host
is
able
essentially
to
mount
this
attack
and
get
a
hold
of
network
traffic
that
is
destined
to
neighboring
pods
by
manipulating
the
underlying
forwarding
plane.
So
essentially,
this
is
something
that
is
relevant
to
your
system.
Now,
let's
think
about
what
exactly
are
the
prerequisites.
B
So,
for
example,
if
we
have
a
port
that
is
running
on
the
host
namespace,
we
really
want
to
make
sure
that
we
do
not
allow
it
to
do
its
those
type
of
attacks
if
you're
running
a
privileged
poll.
This
is
something
that
we
really
want
to
make
sure
we
cannot
run
so
at
the
end
of
the
day,
we're
looking
into
something
that
is
really
required
by
having
the
ability
to
send
traffic
raw
traffic
into
the
host.
Another
good
example
a
very
recent
one
from
last
week,
bypassing
the
localhost
trust
boundary.
B
So
again,
this
is
a.
This
is
a
vulnerability
in
the
way
that
kubernetes
provisions,
the
hosts
the
nodes
themselves,
pretty
much
related
to
q,
proxy
and
cubelet,
which
essentially
an
attacker,
can
send
a
traffic
from
a
neighboring
pod
or
even
from
the
host
itself,
from
a
pod
to
the
localhost
address.
So
that's
127.0.0.1.
B
So,
even
if
you
are
sending
that
from
a
neighboring
node
by
manipulating
the
layer,
two
addresses
and
the
layer
three
says
you
can
really
gain
access
to
a
process
that
is
listening
on
a
port
on
the
local
host
from
a
remote
host.
So
this
is
something
that,
for
example,
an
application
can
make
an
assumption
that
no
one
can
access
a
local
or
process
that
binds
to
a
local
host,
and
this
vulnerability
actually
violates
that.
B
So,
if
we
look
about,
if
we,
if
you
look
about
this,
if
we,
if
we
look
into
this
vulnerability
on
a
host
on
a
node,
it
was
rated
as
5.4,
but
if
we
are
able
to
load
a
pod
on
the
control
plane,
which
is
something
that
is
possible
with
the
non-managed
kubernetes
services.
B
So,
if
you're,
not
using
eks
but
using
cops,
for
example,
then
loading
a
pod
on
the
node
and
and
exploiting
kind
of
the
api
server
running
with
insecure
port,
which
is
off
by
default,
but
sometimes
you
can
kind
of
change
the
default.
You
can
really
kind
of
take
the
entire
cluster,
so
this
is
something
that
is
rated
as
8.8.
So,
if
you
think
about
how
can
you
know
whether
you
are
vulnerable
or
not?
B
This
is
something
that
you
really
need
to
make
sure,
for
example,
that
you
don't
have
pods
that
are
able
to
send
traffic
into
the
network
and
if
you
have
pods
that
has
those
toleration
that
enables
them
to
run
on
the
control
plane.
That's
again,
something
that
you
want
to
avoid.
So,
at
the
end
of
the
day,
we
really
want
to.
We
really
want
to
understand
the
blast
radius.
So,
let's
take
another
example.
This
is
a
cve
that
is
related
to
the
kubernetes
control
plane.
So
the
interesting
part
before
we
dive.
B
We
are
diving
into
this.
The
details
of
that
this
is
something
that
is
relevant
to
some
extendable
extensible
parts
of
kubernetes,
specifically
around
storage
class
and
the
interesting
part
that
you
can
see
that,
with
definition
of
the
storage
class,
you
can
see
that
on
the
parameters,
you
have
a
parameter
that
says
rest
url,
which
kind
of
gives
us
a
hint
that
kubernetes
by
way
of
the
implementation,
is
actually
activating
this
url.
B
So
as
far
as
rating
goes
on
the
none
on
the
on
the
non-managed
kubernetes,
it
was
rated
as
6.3
effectively.
The
vulnerability
basically
enables
an
attacker,
so
someone
that
can
run
control
command
to
extract
data
from
the
api
server
and
specifically
the
vulnerable
component,
is
the
cloud
controller
manager.
B
So
one
point
to
actually
note
as
far
as
the
disclosure
timeline.
So
when
we
really
think
about
yeah,
those
cves
are
are
coming
and
going
pretty
much
quite
often,
it's
not
really
the
case
with
kubernetes
at
least
recently.
B
B
But
the
mind-blowing
fact
is
that
on
december
6
2019
microsoft
reach
research
center
center
was
actually
had
a
submission
around
this
specific
vulnerability.
B
So
that's
more
than
six
months
ago,
and
you
can
see
that
kubernetes
security
committee
requested
an
embargo
so
normally
kind
of
the
disclosure
process
you
can
see
that
from
discovery
actually
from
disclosure
to
a
fix
is
something
that
took
significant
amount
of
time
and,
if
you
think
about
it
from
a
public
disclosure
from
from
actually
kubernetes
releasing
the
patches,
it
also
takes
time
to
apply
those
patches
to
your
environment.
B
So,
at
the
end
of
the
day,
if
you
think
about
all
of
those
cves,
it
necessarily
means
that
your
systems
were
running
vulnerable
for
many
many
many
months
so
effectively.
Cves
are
almost
kind
of,
or
in
some
cases
can
be
those
postmodern
fixes
or
remediation,
that
we
apply
to
our
system
and
our
systems
may
have
been
exploited
or
abused
by
those
cves
if
they
are
not
kind
of
disclosed
or
packed.
B
So
there
is
a
very
wide
time
window
that
that
normally
cves
can
be
exploited
by
the
systems,
and
you
really
need
to
ask
yourselves
like
what
are
the
non
kubernetes
native
security
controls
that
help
you
deal
with
figuring
out,
for
example,
if
those
weaknesses
are
being
exploited
inside
your
environment.
B
So
again,
going
back
to
my
example
here.
B
I
can
really
see,
for
example,
like
which
capabilities
are
being
used
by
pods
that
are
running
inside
my
cluster,
so,
for
example,
this
is
the
permission
that
enables
me
to
mount
the
network
attack
so
bypassing
the
local
host
boundaries
or,
for
example,
messing
up
with
the
ipv6
router
advertisement.
So
this
is
an
example
where
I
can
explicitly
understand
risk
and
blast
radius,
and
if
I
feel
that
this
is
something
that
is
by
design
and
it's
not
exploitable,
I
can
really
say
okay.
This
is
something
I
will
understand.
B
I
can
exclude
that
from
my
scan
results
and
I
don't
need
to
further
dig
in
and
understand
kind
of
the
risk
associated
with
that.
So
again,
the
ability
to
analyze
our
configuration
helps
us
to
reduce
dramatically
the
blast
radius
of
our
clusters
and
environments
in
general,
so
going
back
to
the
slides.
B
So
what
really
happened
with
the
server
side
request
forgery.
So
let's
track
this,
because
this
is
something
that
actually
I
want
to
highlight
something
that
is
related
to
how
we
monitor
the
kubernetes
api
server
and
what
is
the
important
behind
kubernetes
audit
log.
So,
as
we
saw
earlier,
the
weakness
or
the
vulnerability
is
related
to
the
fact
that
once
we
deploy
storage
classes
from
a
very
specific
kind,
which
is
glusterfest
and
and
a
couple
of
more
whether
using
this
type
of
storage
or
not,
it's
not
really
important.
B
B
The
api
server
will
turn
this
request
to
the
cloud
controller
manager,
the
cloud
glue
layer
of
kubernetes
and
in
a
way,
if
I
place
in
the
rest
url
that
we
saw
earlier
an
attacker
server
that
I
can
control,
I
can
respond
with
http
redirect
which,
basically
because
of
a
weakness
that
happens
in
the
go
http
client
and
the
way
that
the
cloud
controller
manager
implements,
I
can
send
an
http
redirect,
which
will
make
the
cloud
controller
manager
make
a
call
of
my
discretion.
Specifically.
B
If
this
is
a
managed
kubernetes
service,
I
can
call
to
the
underlying
cloud
provider
perimeter
so
think
about
it.
You
can
perform
http
calls
inside
the
manage
kubernetes
environment
of
your
eks
gke
and
extract
stuff
from
the
metadata
api
server,
or
do
other
elements
and
the
data
that
returns
or
some
of
it
will
be
returned
to
the
user
that
actually
performed
those
events.
B
If
you
really
see
kind
of
the
action
that
we
see
here,
we
create
resource
of
certain
type.
We
get
events
and
in
order
to
mount
an
exploit,
we
really
need
to
kind
of
invoke
those
commands
in
a
quite
frequent
manner,
which
means
that
if
we
monitor
the
kubernetes
api
server
logs
or
the
audit
logs,
we
can
really
defer
from
those
logs
that
actually
something
anomalous
happens
in
the
system.
Which
brings
me
to
the
last
topic
I
have
for
this
webinar.
B
B
Every
api
invocation
made
by
human
users
like
cubecontrol
or
or
even
your
cloud
dashboard
is
logged,
and
it's
saved
in
normally
into
kind
of
persistent
storage,
that
external
systems
can
actually
stream
and
analyze
those
streams
and
defer
insights
about
the
system
in
terms
of
those
logs.
B
We
can
even
detect
our
bug
misconfiguration.
So,
for
example,
if
we
see
a
spike
or
an
increase
of
permission
denied
after
we
deploy
a
new
software
version,
one
of
the
root
causes
for
for
that
may
be
misconfigured
or
or
not
aligned,
or
our
back
policy
is
not
aligned
with
the
application
functionality.
B
Another
actually
good
example
that
one
can
leverage.
As
far
as
the
kubernetes
audit
log,
you
can
place
rules
that
says,
for
example,
if
someone
exec
into
a
pod
in
a
production
cluster,
we
really
we
can
really
send
an
alert
to
the
team.
So
it
really
gives
us
the
ability
to
both
plug
analysis
and
hunting
for
anomalous
behavior,
as
well
as
placing
policies
that
helps
us
kind
of
regulate
how
kubernetes
cluster
is
being
utilized.
B
So
when
we
kind
of
think
about
all
of
those
use
cases
having
something
that
is
more
of
kind
of
both
policy
driven
and
ai
driven
helps
us
leverage
the
kubernetes
audit
log
to
detect
the
cv
that
is
associated,
for
example,
with
storage
class.
So
let's
take
that
as
an
example,
if
an
attacker
starts
to
deploy
storage
class
and
create
persistent
volume,
claims
of
this
vulnerable
storage
class,
so
an
increase
or
or
an
unexpected
increase
of
those
api
invocations
is
something
that
threshold-based
approach
can
help
us.
B
But
the
problem
is
that
we
really
don't
know
beforehand,
which
are
the
vulnerable
apis
are
there
or
how
exploits
are
actually
being
exploited
or
how
vulnerabilities
and
weaknesses
are
being
exploited
by
an
attacker,
which
means
that
having
something
that
is
more
of
adaptive,
that
learns,
how
resources
and
users
and
service
accounts
activate
the
kubernetes
apis
is
something
that
can
help
us
detect
that.
So,
with
that
in
mind,
let
me
show
you
an
example
of
what
a
system
like
that
can
perform.
B
So,
while
the
system
loads,
we
can
really
see
here
an
example
of
a
timeline
that
tells
us
a
story,
so
I
can
zoom
in,
for
example,
on
a
couple
of
assets.
I
can
investigate
those
specific
users,
but,
more
importantly,
I
can,
for
example,
see
that
these
specific
users,
which
are
unauthenticated
users
in
the
environment,
actually
perform
api
access
to
the
kubernetes
api
server
and
got
permission
denied.
B
If
I
want
to
have
a
look
on,
for
example,
which
type
of
namespaces
they
were
trying
to
access,
we
can
see
that
those
assets
actually
were
able
to
access
or
trying
to
access
all
the
namespaces
in
the
environment.
If
you
want
to
see,
for
example,
the
origin
of
those
users,
I
can
pretty
much
visualize
those
elements
so,
at
the
end
of
the
day,
the
ability
to
understand,
for
example,
if
we're
taking
again
the
example
of
the
cloud
storage
vulnerability.
B
We
can,
for
example,
see
resource
type
and
we
can
see
which
resource
types
are
being
employed
inside
my
environment.
So
I
don't
see
here,
for
example,
storage
class.
So
this
is
an
indication
that
helps
you
understand
that
this
some
this
is.
This
specific
exploit
was
not
applied
into
the
system
based
on
the
fact
that
we
don't
ever
don't
have
a
record
of
those
storage
class
and
pvc
creation
attempts.
B
That
would
enable
you
to
perform
ci
can
understand,
for
example,
analyzing
your
r
buck,
or
you
want
to
take
our
cloud
service
for
a
spin
and
analyze
your
kubernetes
clusters
for
cds,
or
even
try
kubernetes
audit
log
analyzer,
which
is
also
audit,
hit
those
links
and
we'd
happy
to
kind
of
take
it
for
a
deeper
spin.
And
with
that
in
mind,
I
think
we
can
move
into
the
q
a
part.
A
Awesome
thanks
so
much
for
a
great
presentation.
It
looks
like
we
have
one
question
here:
it's
from
jindong
and
it
says
in
the
clusters.
Fs
rest
api
example.
Traffic
to
outside
kubernetes
should
be
controlled.
B
We
so,
and
then
this
is
in
the
context
of
the
control
plane,
vulnerability.
A
B
Yeah,
so
so,
essentially,
yes,
the
general
idea
is
that
the
control
plane,
even
in
the
managed
kubernetes
cloud
instance,
it
should
be
controlled.
But
going
back
to
the
beginning
of
this
webinar.
B
If
we
are
provisioning,
our
kubernetes
cluster,
with
a
publicly
facing
api
server.
So
the
when
you
provision
clusters,
you
can
actually
do
private
clusters
or
you
can
do
public
clusters.
So
if
you
are
using
the
managed
kubernetes
option
with
a
public
cluster,
the
control
plane
can
actually
connect
from
the
cluster
to
the
internet,
which
means
that
an
attacker
can
can
still
manipulate
a
server
that
is
running
on
the
outside
and
with
those
redirect
that
I
showed
earlier.
A
Vpc,
great
okay,
okay,
folks,
another
minute,
a
reminder,
if
you
have
a
question,
feel
free
to
drop
it
in
the
q.
A
box
at
the
bottom
of
your
screen
give
folks
just
a
few
moments
here:
okay!
Well,
I
think
that
is
all
the
questions
that
we
had
for
today.
Thank
you
again,
gaby
for
a
great
presentation
and
thanks
again
to
everyone
for
joining
us,
just
a
reminder
that
the
webinar,
recording
and
slides
will
be
posted
later
today
to
the
cncf
website.
A
We
look
forward
to
seeing
you
at
a
future.
Cncf
webinar
and
hopefully
cubecon
virtual
here
in
a
couple
weeks,
have
a
great
day
and
thanks
everyone.