►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
CSI Volume Attacks – The SRE Strikes Back - Hendrik Land, NetApp
Container Storage Interface (CSI) has made it easy for stateful workloads to consume storage - but does it protect your data from unauthorized access? The CSI standard only orchestrates the creation of volumes, snapshots or clones. How do you ensure that neither other workloads in the same Kubernetes cluster nor someone outside the cluster can access your data? You will learn the inherent security models provided by Kubernetes as well as additional configurations you can and should apply. Beyond concepts and architecture, a series of short demos will cover topics such as: - Security of Persistent Volume Claims and Persistent Volumes throughout their lifecycle - Pod Security Policies and volume types - File system permissions on your volumes - Securing common storage protocols such as iSCSI and NFS - Securing CSI drivers in your cluster
A
A
My
name
is
hendrick
I'm
a
solution
architect
at
netapp,
and
I
help
our
customers
with
storage
and
data
management
solutions
for
kubernetes,
both
in
public
cloud,
as
well
as
in
their
own
data.
Centers,
and
one
of
the
questions
I
get
a
lot
is
around
csi
the
container
storage
interface,
which
is
really
nice
because
it
orchestrates
the
equation
of
volumes,
snapshots
and
clones.
A
A
A
So
just
as
a
very
brief
recap,
how
does
dynamic
storage
provisioning
with
csi
work
for
the
purpose
of
this
session?
I'm
going
to
focus
on
the
part
level,
of
course,
in
a
real
world
environment
you
would
have
to
stay
for
set
if
it
is
a
stateful
workload
or
a
deployment
or
replica
set.
I'm
leaving
these
details
out
that
they
are
not
relevant
for
this
topic.
A
A
Each
solution
has
its
own
provisioner
that
takes
care
of
everything
that's
required
for
that
specific
solution,
but
for
the
user
or
application
all
of
these
complexities
are
stacked
away.
You
just
create
a
pvc
that
references,
a
storage
class.
Everything
else
happens
automatically
by
whatever
csi
provisioner
is
running
in
your
cluster.
A
A
The
volume
from
a
storage
solution
is
then
represented
inside
kubernetes
as
a
persistent
volume
or
short
pv,
and
that
pv
is
bound
to
the
volume
plane
and
finally
made
available
as
a
file
system
inside
your
container
so
much
for
the
basics.
So,
let's
think
about
how
an
attacker
might
try
to
get
access
to
our
data.
The
first
one
is
easy
assume
the
attacker
can
run
another
plot
in
the
cluster.
A
Now
an
obvious
target
are
our
kubernetes
resources,
the
pv
and
the
pvc.
If
someone
could
get
access
to
these,
they
would
be
able
to
mount
the
volume
into
their
own
pot
and
access
the
data.
Fortunately,
the
answer
to
this
problem
is
easy.
What
do
we
do
to
separate
resources
went
up
within
our
cluster
correct
namespaces?
A
A
They
are
not
within
the
specific
namespace
that
is
okay
for
the
storage
class,
as
it's
only
used
during
the
dynamic
provisioning
of
a
pv,
but
doesn't
give
any
access
to
the
volume
itself
we're
going
to
look
into
limiting
access
to
a
storage
class
later
on.
For
now,
just
remember
that
there
can
be
one
and
only
one
default
storage
class.
A
A
A
A
A
A
A
A
A
A
The
inherent
security
model
of
kubernetes
protects
the
pv,
even
though
it's
not
part
of
the
namespace,
and
it
also
protects
it
beyond
the
lifecycle
of
the
volume
plane.
So
protection
mechanism
number
one-
is
kind
of
obvious
make
use
of
namespaces
and
make
sure
that
no
one
has
access
rights
to
the
persistent
volume
resource.
A
A
Let's
take
a
look
at
how
that
works.
We
have
our
pot
that
has
the
volume
attached.
We
can
take
a
closer
look
at
that.
If
we
extract
into
the
pot
and
check
what
is
mounted
and
from
there
we
see
that
slash
data
is
backed
by
the
nfs
mod.
Simply
because
my
best
car
storage
class
uses
nfs
storage,
this
would
be
iscsi
or
anything
else.
We
would
see
some
more
details
about
that,
such
as
the
device
path
on
the
kubernetes
host.
A
The
actual
mount
is
performed
by
the
kubernetes
node,
not
by
the
pot,
but
we
see
the
mount
from
within
the
pot
as
well.
Let's
assume
that
someone
found
out
that
this
is
the
nfs
mount
contains.
The
uid
of
the
pv,
so
nothing
you
could
easily
guess,
but
there
are
tools
such
as
show
mount
to
discover
all
mounts
on
an
mfs
server
or
iscsi
discovery
to
find
all
iscsi
luns.
A
A
A
A
A
A
A
A
A
In
this
case
here
I
set
the
user
id
to
100
000
and
the
group
id
to
200
000..
In
addition
to
that,
I
can
set
the
apis
group
parameter,
and
this
specifies
the
group
permissions
on
the
volume.
So
when
the
container
starts,
kubernetes
actually
sets
the
group
of
all
files
and
folders
in
our
pvc
to
this
value,
and
the
container
is
started
with
this
group
in
my
example:
300
000
as
a
supplemental
group.
So
if
the
application
can
access
the
data
on
the
volume,
there
are
two
caveats
with
this.
A
A
A
A
The
second
caveat
with
fs
group
is
performance
by
default.
Kubernetes
will
do
a
recursive
modification
across
the
entire
file
system.
Every
time
the
pvc
is
mounted.
Usually
you
would
only
need
that
operation.
The
first
time
the
pvc
gets
mounted,
but
kubernetes
doesn't
have
any
way
to
figure
out.
If
this
is
the
first
time
the
volume
gets
used
or
not.
A
Kubernetes
will
then
check
the
permissions
of
the
volume
route
if
they
already
match
the
requested
efforts.
Group
kubernetes
assumes
that
the
rest
of
the
file
system
is
ok
as
well
and
skips
the
recursive
modification
of
permissions,
so
unless
you
or
your
application
mess
around
with
your
permission,
this
option
enables
kubernetes
to
only
apply
permissions
once
and
avoid
the
performance
penalty
on
every
mount.
A
A
A
A
A
A
A
A
Last
but
not
least,
there
are
a
few
things
outside
of
kubernetes
itself
that
you
can
do
to
improve
security.
First
on
the
prodigal
side,
many
csi
drivers
use
standard
protocols
such
as
nfs
or
iscardia.
In
the
background,
we
won't
have
time
today
to
go
over
all
the
best
practices
for
these
protocols,
but
here
are
a
few
things
you
should
consider
for
nfs.
The
first
question
is
about
protocol
version.
A
A
A
most
linux
systems
are
configured
to
try
version
4
first
and
then
fall
back
to
version
3
if
necessary,
but
you
can
also
enforce
a
specific
version
with
mount
options
specified
in
your
storage
class,
as
all
mounting
of
volumes
is
handled
by
your
csi
driver.
There's
no
need
for
the
show
mount
utility
that
reveals
the
list
of
all
exports
on
the
nfs
server.
As
we
saw
earlier,
the
mount
path
can
be
used
to
create
an
inline
mount
in
a
pod,
not
revealing
that
mount
path
in
the
first
place
is
a
good
additional
step
you
can
take.
A
So
if
your
nfs
server
allows
to
disable
show
mode
functionality
then
make
use
of
that
root.
Squash
in
nfs
allows
you
to
map
any
file
activity
of
the
root
user
to
someone
else,
typically,
the
user.
Nobody,
as
your
containers,
shouldn't
run
with
root
anyway,
enabling
root
squash
does
not
impact
your
applications,
but
it
makes
sure
that
no
one
can
access
files
with
root
permissions.
A
Most
servers
also
allow
you
to
configure
export
policies
that
limit
access
to
an
nfs,
export
to
specific
appear,
addresses
or
hostnames.
It
acts
a
little
bit
like
a
firewall
just
for
nfs
as
the
nodes
in
the
kubernetes
cluster
might
change
over
time.
Your
csi
driver
should
have
the
ability
to
dynamically
update
this
xbox
for
iscsi
as
a
protocol.
Two
things
are
relevant.
First
iscsi
allows
you
to
configure
authentication
between
client
and
server.
A
If
you
have
a
good
csi
driver
that
should
automatically
handle
all
chat
setup
on
the
nodes
in
your
cluster
and
then
similar
to
the
export
list
for
nfs
servers,
many
iscsi
servers
support
initiator
groups
as
a
way
to
limit
storage
access
to
specific
nodes
in
the
case
of
iscsi.
That
is
usually
not
based
on
ip
addresses,
but
on
the
iqn
identifier
of
the
client
and
again,
if
you
have
a
good
csi
driver
that
should
manage
this
automatically
for
you.
A
Let's
do
a
quick
recap:
make
use
of
namespaces
and
the
inherent
security
model
of
kubernetes
will
protect
the
pvc
in
the
namespace,
as
well
as
the
tv
that
lives
outside
of
the
configure
namespace
pod
security
policy
that
prevents
inline,
nfs
and
iscs
enable
always
access
control.
So
users
cannot
manually
create
a
pd.
Everything
should
be
based
on
dynamic
provision,
set
the
security
context
of
your
pot
to
control
the
file
system.
Permissions
of
the
volume.