►
From YouTube: Kubernetes Security Best Practices
Description
For many DevOps teams, Kubernetes has become an enterprise IT mandate, but like previous waves of infrastructure change, Kubernetes security best practices must be followed throughout the container life cycle. In this webinar, you will learn how to:
- stay on top of ongoing Kubernetes security hygiene
- harden your nodes
- implement RBAC best practices
- secure your production workloads
- thwart an attack, with a live demo
A
Hello,
everyone
and
thank
you
for
attending
today's
webinar
kubernetes
security,
best
practice
best
practices
before
we
get
started.
I
want
to
cover
a
few
housekeeping
items.
If
you
have
any
questions
during
the
webinar,
you
can
go
to
the
q
a
button
at
the
bottom
of
your
screen
and
submit
your
question
at
any
time.
During
the
session
we
will
have
a
q
a
session
at
the
end
of
the
webinar.
A
A
Lately
he
is
focused
on
helping
users,
understand
the
complete
risk
context
for
their
kubernetes
workloads
and
enabling
them
to
implement
effective
security
controls
to
mitigate
those
risks.
Previously
he
revamped
significant
parts
of
medallions
worldwide
data
center
and
application
deployment
architecture,
and
now
here's
connor.
B
Hey
everyone
thanks
for
joining
the
webinar
today
yeah,
I
think,
as
we
alluded
to
we're
talking
about
community
security,
best
practices
and
so
just
jump
into
it.
So
what
we'll
cover
today,
you
know
talk
about
general
kubernetes
hygiene,
talk
about
some
workload,
best
practices
I'll
go
through
a
demo
of
how
those
workload
best
practices
translate
into
actual
stopping
or
stopping
real
live
attacks,
and
then
I
can
take
any
questions
you
guys
have
there
at
the
end
cool.
So
what
are
we
doing
here?
You
know
kubernetes
is
coming.
B
I
think
a
large
portion
of
everyone's
organizations
are
moving
towards
it.
Some
of
you
are
probably
pushing
kubernetes
at
your
organization,
seeing
the
benefits,
and
some
of
you
are
probably
getting
kubernetes
pushed
under
you
right
and
wondering
how
you're
going
to
manage,
especially
the
security
of
kubernetes,
and
so
this
would
be
a
overview
today
and
and
of
course,
we've
got
a
lot
more
content
for
you
down
the
road,
so
starting
with
kubernetes
hygiene.
B
What's
the
the
first
thing
to
start
with
upgrading
you
know,
upgrades
are
never
super
fun,
but
in
general
the
kubernetes
maintainers
maintain
vulnerability
patches
for
the
last
three
releases.
So
at
this
point
that's
116,
115
and
114,
and
the
last
place
you
want
to
be
when
a
vulnerability
comes
out
is
trying
to
roll
out
new
infrastructure
and
roll-up
infrastructure.
B
You
know,
if
you
think
about
the
time
that
kubernetes
deprecated
at
cd2
in
lieu
of
scd3
right,
you
don't
want
to
be
rolling
out.
Large
data
plane
upgrades
for
a
vulnerability
because
you
usually
want
to
plan
those
things
out.
So
you
don't
want
to
be
surprised
by
a
vulnerability
or
not
having
support
for
something
so
always
be
sure
to
upgrade
one
of
the
challenges
with
that
is
knowing
when
to
upgrade.
How
do
you
know
that
there's
a
new
version
out?
I
personally
love
this
google
group.
B
It's
an
orchestrator
that
runs
on
top
of
what
I
would
consider
somewhat
traditional
infrastructure
of
vms
or
servers
right
is
make
sure
you're
controlling
the
network,
access
to
the
sensitive
ports
and
kubernetes,
specifically
in
this
case
the
ports
used
by
the
kubelet
right
and
then
make
sure
you
try
to
limit
the
access
to
the
kubernetes
ap
api
server
itself.
So
there
was
a
recent
cve
that
came
out
that
actually
allowed
unauthenticated
ddos
to
the
kubernetes
api
server.
B
You
know
just
this
is
a
general
infrastructure
best
practice,
but
make
sure
you
minimize
the
access
to
kubernetes
nodes
right.
If
people
have
root
on
nodes
right,
they
can
docker
exec.
They
can
look
at
the
secrets
that
might
be
mounted
or
the
volumes
that
are
mounted
right
and
they
pretty
much
have
access
to
your
infrastructure.
So
restricting
access
to
these
nodes
is
paramount
and
making
sure
the
people
who
do
have
access
really
really
need
that
access.
B
So
it's
an
alpha
now
something
to
look
at
moving
forward
and,
finally,
from
a
kubernetes
perspective,
what
we've
done
is
you've
restricted
the
access
to
the
api
server,
you've
restricted
the
access
to
the
nodes
themselves
right
and
then,
let's
restrict
access
to
the
developers
and
service
accounts
about
what
access
they
actually
have
within
that
api
server.
So
the
way
that
they've
created
this
is
basically
you
have
subjects
which
is
the
who
and
and
who
can
do
what
right.
B
But
generally
you
want
to
make
sure
we
set
these
up
so
as
you're
kind
of
going
through
your
kubernetes
journey.
A
lot
of
people
start
with
everyone
having
cluster
admin,
you're
trying
to
figure
out.
You
know
what
am
I
doing
with
kubernetes?
How
can
I
launch
resources,
you
know
and
make
sure
that
you
have
maximum
velocity
when
you're
doing
that,
but
once
you've
kind
of
passed
that
stage?
B
So
this
is
just
kind
of
a
quick
overview
of
you
know
the
kubernetes
orchestrator
itself
and
how
you
can
kind
of
secure
the
actual
physical
orchestrator,
but
we're
going
to
jump
into
securing
your
workloads,
which
I
think
is
a
huge
part
of
kubernetes
security.
Specifically
because
now
we
have
developers
launching
applications
on
top
of
kubernetes
and
and
how
do
they
secure
those?
B
And
how
can
you
help
your
developers
secure
those
so
a
pretty
simple
one,
pretty
straightforward
one
but
leveraged
name
spaces
namespaces
for
rbac,
for
example-
are
super
useful
where
you
can
segment
specific
teams.
So
just
imagine
you
have
a
payments
team,
an
operations
team
and
a
front-end
team,
and
you
really
want
to
ensure
that
they
don't
bleed
data
between
them
and
that
you
can
make
sure
that
they
can't
access
each
other's
resources
right
and
some
of
it
might
be
customer
sensitive.
B
It's
really
useful
for
multi-tenancy
to
make
sure
that
you
know
you
have
customer
one
professional
services
for
customer
one
and
professional
services
for
customer
two
and
that
they
can't
access
each
other's
instances
in
terms
of
network
policies
which
we'll
go
into
kind
of
at
the
end
there
towards
in
the
demo
portion.
But
you
can
really
see
that
you
can
segment
between
tenants,
so
you
can
really
ensure
for
compliance
purposes,
that
they
can't
access
each
other's
instances
or
databases.
B
I
love
it
for
resource
usage
tracking
to
see
which
team
is
the
most
of
your
money
and
most
of
your
resources
and
then,
finally,
I
think
it
really
makes
coop
cuddle
results
like
much
more
easy
to
read
and
much
more
sane
to
walk
through
where
you're
just
looking
at
hey
what
resources
exist
in
the
specific
name
space-
and
you
know
kind
of
just
it's
a
nice
segmentation,
as
opposed
to
one
big
chunk
of
resource
in
in
the
default
namespace.
So
namespaces
are
super
important,
they're,
really
nice
logical
segmentation
on
top
of
kubernetes.
B
Where
deployment
is
saying,
I
want
to
run
a
pod
with
x
number
of
replicas
a
demon
set
is
more
of
an
agent
approach
we
were
talking
about.
You
know.
I
want
to
run
this
this
part
on
every
single
node
right,
so
starting
with
the
image
you're
really
looking
at
the
contents
of
that
image.
What
are
the
vulnerabilities
that
exist
in
the
packages?
What
your
registries
are
from?
B
Is
it
a
trusted
registry
which
files
exist
that
could
be
used
by
attackers
or
for
debugging
purposes
right,
and
then
you
roll
that
to
container,
and
then
it's
okay?
Where
am
I
running
this
container?
Am
I
running
it
in
test
versus
prod?
Do
I
have
separate
clusters
for
these
things?
Are
they
one
cluster
all
right
which
businesses
are
launching
these
specific
clusters
and
using
these
for
their
ciecd
processes,
right
you're?
Looking
at
labels
and
annotations?
I
think
these
are
a
really
critical
and
awesome
part
of
kubernetes,
where,
basically,
you
can
annotate
your
resources.
B
I
love
using
it
for
the
owner
of
a
particular
resource,
so
you
know
who
the
point
of
contact
is.
It
could
be
an
email,
it
could
be
a
team
slack
a
web
hook,
and
that
would
just
help
security
and
ops
look
at
a
specific
resource
and
say:
okay.
I
know
it
belongs
to
this
team
and
here's
the
point
of
contact
for
that.
B
So,
if
there's
an
incident
either
performance
or
security,
you
automatically
loop
back
to
a
specific
person
and
a
concrete
action
that
you
can
take
right,
so
that
was
kind
of
metadata
around
pods
and
then,
as
you
jump
into
specific
privileges
and
access
right,
you
can
see
which
you
know
how
much
access
does
a
particular
pod
have
in
terms
of
the
api
that
will
be
given
by
a
service
account
on
the
host.
What
is
it
running?
B
As
does
it
have
access
to
any
host
mouse
right,
like
for
data,
that's
persisted,
you
know,
potentially
secrets
right,
so
you're,
looking
at
kind
of
the
risk
around
a
particular
pod
and
it's
configuration
you
jump
into
a
network
segment
where
you're
looking
at
okay.
Now
that
I
have
this
pod,
it
has
this
access.
What
can
it
talk
to?
Can
it
talk
to
the
internet
which
other
pods
can
I
talk
to
and
effectively
you're
kind
of,
defining
the
blast
radius
of
a
particular
pod,
so
this
pod
got
compromised?
B
How
far
like
I
like
to
think
of
it
as
like
how
many
degrees
from
kevin
bacon,
you
are
right,
it's
like
how
many
degrees
from
the
internet
or
access
is
your
critical
pots
and
then,
finally,
you
know
a
pod
and
a
container
and
an
image.
All
lead
to
one
thing
which
is
basically
just
running
an
application,
and
so
the
actual
behavior
of
that
application
is
really
important.
So
who
does
that?
What
does
the
application
actually
talk
to?
What
processes
are
actually
launched?
What
are
the
actual
executions?
B
So,
in
terms
of
like
concrete
configurations
to
explore,
I'm
definitely
going
to
highlight
the
top
three
today
a
read-only
root
file
system,
linux
capabilities
and
network
policies,
but
there's
many
many
more
knobs.
You
can
turn
on
containers
and
kubernetes
itself.
Looking
at
host
mounts,
you
can
not
mount
service
accounts
in
to
every
container,
we'll
get
environment,
environment
variables
and
resource
requirements.
B
So
attackers
make
use
of
this
and
we'll
drop
payloads
there
and
execute
them
and
in
a
container
world
it's
possible
to
not
make
it
read,
write
because
you
know
exactly
what
your
application
is
doing.
Instead
of
running
a
large
number
of
processes
inside
a
vm
you're
running
just
a
single
application.
Typically,
it's
a
process
or
two,
and
you
can
kind
of
understand
exactly
what
actions
it's
taking
so
I'll
jump
in
and
show
why
this
matters.
B
B
So
I've
got
it
running
now
and
I'll
go
ahead
and
run
the
exploit.
So
what
this
exploit
is
doing
is
it's
crafting
a
header
sending
it
to
the
strut
server
it
downloads,
a
minor
d
binary
from
the
internet,
untars
it
and
runs
it,
and
so
at
this
point
you
can
see
that
we're
running
miner
d
in
our
container,
you
know
not
the
most
malicious
thing
someone
could
do,
but
it's
definitely
stealing
our
resources
and
you
can
imagine
this
can
be
used
for
much
worse
purposes.
B
So
what
I
can
do
here
is
I'll
show
you
my
read-only
stretchy
amble,
that
I'm
about
to
apply
and
I've
just
set.
The
security
context
read
only
root
file
system
to
true
and
inside
the
docker
file.
I've
made
a
volume
at
user
local
tomcat,
because
tomcat
does
need
to
write
temporary
files.
So
it's
not
completely
a
root
file
system
and
I
will
replace
this
one.
B
B
Right
and
so
in
this
case
we
tried
to
download
the
minority
z
into
slash
and
even
slash
was
a
read,
only
file
system,
whereas
you
know,
if
they
had
known
that
user
local
tomcat
wasn't,
then
they
could
write
it
there.
But
you
know
a
lot
of
these
attackers
are
just
running
scripts
against
an
environment
and
checking
for
specific
vulnerabilities
and
so
they're,
not
really
that
complex
into
looking
into
hey.
Is
this
file
system
read
only
where
can
I
find
something?
That's
read
right.
B
So
in
this
case
we've
thwarted
an
attack
with
you
know
basically,
two
lines
of
yaml,
and
so
it's
really
important
and
it's
a
really
easy
way
to
reduce
the
attack
surface
of
your
container.
The
hard
part
of
this
is
figuring
out.
How
do
I
know
that
I
can
make
my
application
read
only
and
and
which
paths
do
need
to
have
volumes.
B
So
what
I've
done
here
is.
I
just
ran
an
nginx
right.
Obviously
I
didn't
write
nginx,
so
I
don't
know
where
it
writes
temporary
files
to,
and
I
ran
it
on
my
local
docker
for
desktop
and
I
just
ran
on
docker
diff,
so
docker
diff
will
look
at
which
files
have
changed
over
the
lifetime
of
a
container,
and
so
what
we
can
see
here
is
that
there's
kind
of
two
main
areas
of
look
the
run
secrets
kubernetes
is
handled
by
kubernetes.
You
don't
need
to
worry
about
those,
but
slash
run.
B
Nginx.Pid
is
definitely
something
to
worry
about
right,
so
we
should
make
that
read,
write
because
nginx
will
write
out
to
there
and
then
also
it
looks
like
in
this
case.
The
root
of
a
volume
here
is
var
cache
engine
x.
So
in
this
case,
if
I
use
two
volumes
and
made
var
cache,
nginx
read,
write
and
then
flash
run
read
write
as
well
for
the
nginx.pin,
then
nginx
itself
could
be
containing
a
read,
write
our
read-only
file
system,
excuse
me,
and-
and
we
could
easily
reduce
the
attack
surface
of
that.
B
So
this
is
just
one
easy
way.
If
you're
using
local
dev
that
you
could
look
at
specific
containers,
you're
running
and
seeing
which
files
are
actually
being
written
at
runtime,
you
know
there's
a
lot
other
ways
you
can
do
this.
You
can
look
at.
You
know
a
live
monitoring
system,
that's
looking
at
files,
but
this
is
kind
of
a
simple
way.
If
you're
just
checking
out
and
trying
to
reduce
the
attack
surface
of
some
of
your
apps.
B
Cool,
so
the
next
configuration
that
I
would
love
to
discuss
are
linux
capabilities,
and
linux
capabilities
are
one
of
those
knobs
that
you
can
turn
that's
super
powerful,
but
a
lot
of
people
don't
really
know
how
to
use
them
or
haven't
had
a
ton
of
experience
with
them,
and
so
what
linux
capabilities
are
is
like.
The
idea
was
that
basically
root
was
a
little
bit
too
powerful,
and
so
what
they
want
to
do
is
split
the
root
superpowers
into
a
series
of
capabilities
that
are
more
fine-grained.
B
B
B
So
theoretically,
if
we
drop
the
capabilities
right
these
capabilities
here,
then
the
tar
will
be
unable
to
run,
and
then
we
should
block
the
attack
that
downloads
the
minor
d
and
makes
it
executable
and
runs
it.
So
as
we
jump
into,
I
can
demo
that
there
so
I'll
just
go
ahead
and
cap
this
guy
and
what
I've
done
here
is
inside
the
security
context.
B
I've
dropped
all
capabilities
and
the
reason
I
can
drop
all
capabilities
is
because
tomcat
doesn't
use
any
capabilities,
but
also
the
port
that
we're
using
8080
is
an
unprivileged
port.
So
you
do
need
net
admin
if
you're
running
in
a
privileged
port.
So,
for
example,
if
you're
running
port
22,
you
wouldn't
need
net
admin,
though
don't
run
port
22
ssh
is
not
something
that
should
be
contained
in
containers,
so
I'll
go
ahead
and
modify
my
current
application.
That's
running
just
apply
this.
B
So
the
download
in
this
case
actually
ended
up
succeeding,
unlike
last
time
when
we
wrote
it
to
slash
minor
monitor.targeyz
and
then
it
failed
the
ownership
change.
So
in
this
case,
you
can
see
that
basically
just
dropping
these
linux
capabilities,
that
weren't
being
used
or
exercised
anyways
ends
up
blocking
an
attack
that
doesn't
know
that
these
capabilities
are
dropped
a
lot
of
times.
This
is
enough
for
an
attacker,
because
they're
just
running
a
script
right
and
if
their
script
fails
then
they'll
back
out
and
move
on
to
the
next.
B
So,
finally,
the
one
I
would
like
to
discuss
right
is
network
policies,
and
so
the
way
I
think
about
kind
of
an
attack
right
in
this
case,
we've
blocked
the
file
right.
The
payload
drop,
we've
blocked
the
changing
of
the
ownership
or
the
capabilities
that
we
don't
need,
and
then
finally,
it's
like.
Okay,
how
do
we
just
block
the
network?
B
So
in
this
case
this
particular
yaml
example
yellow
here
we
have
a
pod
selector,
it's
matching
the
labels
app
web,
so
any
pod
that
has
that
is
what
this
will
apply
to
and
then
it
allows
ingress
so
allows
connections
from
one
of
these
locations
so
from
a
name,
a
name
space,
selector
standpoint,
we're
selecting
a
specific
namespace,
we're
selecting
one:
that's
just
team
operations,
so
it's
probably
an
operations
name
space
and
then
the
pod
selector.
The
type
has
to
be
monitoring.
B
So
imagine:
you're,
running
prometheus
inside
a
name
space
that
has
the
label
team
operations
and
it's
scraping
endpoints
on
this
web
server.
We
will
allow
ingress
from
any
pod
that
has
monitoring
right,
so
we
will
label
prometheus
with
a
label
type
monitoring
and
that
will
have
access
to
this
web
server.
So,
in
this
case,
this
will
be
the
only
one
that
has
access
to
our
web
application.
B
You
can
see
that
we've
selected
our
api
server
right
and
we
specify
the
policy
type
egress.
Now
we
haven't
specified
any
rules
for
this
because
we're
basically
blocking
all
egress
our
web
server
on
the
internet
should
not
be
reaching
out
to
the
internet.
We
should
only
be
receiving
connections,
so
the
impact
of
this
is
when
I
run
the
exploit.
B
B
So
the
final
thing
to
remember
is
that
security
is
hard
right.
Some
of
these
capabilities
are
hard
to
use
or
or
can
be
a
little
bit
more
difficult,
but
any
attempt
and
an
effort
that
you
make
in
reducing
the
attack
surface
of
your
containers
is
definitely
worthwhile
and
can
have
a
big
impact
on
your
organizations
and
overall
security
posture
thanks.
I
think
we'll
we
can
take
questions
now.
A
B
Yeah,
so
I
think
a
large
variety
of
of
runtime
monitoring
is
really
effective
in
these
areas.
Right,
so
looking
at
the
files
that
are
being
read
are
like
written
inside.
A
container
right
can
really
help
like
the
read-only
file
system
situation,
or
you
know,
monitor
actively
monitoring
the
capabilities
that
are
being
used
right.
If
you
don't
want
to
use
this
kind
of
like
at
runtime,
what
you
can
do
is
you
can
give
tools
to
developers
to
start
doing
this
themselves
as
they're
kind
of
deploying
and
generating
these
animals.
B
B
Right
so
our
back
is,
is,
is
built
into
kubernetes
and
and
there's
kind
of
a
variety
of
different
ways
and
cloud
providers
provide
it
in
in
different
mechanisms
as
well,
but
you
can,
you
can
set
it
at
a
service
account
level
if
your
services
do
need
to
access
kubernetes
api.
If
not,
then
you
don't
need
to
do
it
there,
but
then
you
can
set
it
up
for.
B
Right
so
so
these
do
need
to
be
set
at
the
app
definition,
but
you
can
use
things
to
enforce
this,
like
pod
security
policies,
which
does
map
to
the
security
context.
So
if
you
want
to
require
people
to
drop
specific
capabilities
or
not
run
as
privileged,
the
the
pod
security
policies
can
be
used,
they're,
just
basically
an
emission
controller
or
there's
a
variety
of
tools
that
provide
admission,
controller
access,
and
then
you
can
create.
Policies
live
like
that.
So,
for
example,
stack
rocks
does
does
that.
B
Right
so
I
think
I
think
a
large
portion
here
is
looking
at
the
traffic
and
then
also
kind
of
like
working
your
way
through
these.
So
as
you're
migrating
them
right,
you
kind
of
have
a
unique
opportunity
of
seeing
what
what
needs
to
access
what
so
the
first
thing
to
do
for
a
namespace,
for
example,
is
you
can
set
a
default
deny?
And
so
then
you
have
to
poke
holes
to
make
sure
everything
works
right.
B
So,
as
you're
migrating,
these
services
you're
going
to
start
working
through
the
network
connectivity
and
you
can
see
kind
of
like
some
of
the
failure
modes.
Also,
there's
some
really
cool
tools.
Stack
rex
has.
This
is
really
awesome,
which
is
basically
you
can
look
at
the
live
traffic,
that's
running
between
specific
pods
and
then
you
can
start
scoping.
Then
the
network
policies
to
the
live
traffic
that
you're
seeing
right
there's
a
lot
of
distributed,
tracing
tools
that
provide
at
least
the
the
network
visibility.
B
B
I
think
network
policies
are
really
effective
there
in
terms
of
just
stopping
the
access
from
from
you
know,
potentially
jumping
through
pods
that
that
shouldn't
be
able
to
talk
to
each
other,
so
network
policies
can
really
help
in
that
situation,
and
then
you
know
making
sure
that
you're
not
really
mounting
host
volumes
and
that
the
isolation
is
is
really
kept
in
containers.
So
you
know
try
to
not
rely
on
on
things
that
exist
on
the
host.
B
Yeah,
so
if
you
guys
have
any
specific
topics
that
you
want
to
hear
about,
feel
free
to
put
them
in
the
the
q
a
and
we
can
address
them
in
future,
webinars.
We're
always
interested
in
making
sure
that
you
guys
are
getting
content
that
you
guys
want
to
hear
or
are
really
interested
in
and
want
to
know
more
about.