►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Using K8s Audit Logs to Secure Your Cluster - Mark Stemm, Sysdig
K8s Audit Logs are a new feature in K8s 1.11/1.13 which allow an operator to see a stream of events from the API server that show the changes being made to your cluster. In this talk, we’ll describe how auditing works and how to get it working it for popular K8s variants. Then we’ll dive into specific security-oriented use cases, showing how you can use audit logs to enforce security best practices, detect misuse, and fill the gap between what you think the cluster is running and what's actually running. Some specific use cases we’ll discuss include misuse of configmaps to hold sensitive data, overly loose permissions on pods/services, and abuse of cluster role bindings that grant too many (or the wrong) permissions. Attendees should come away with the ability to enable K8s Audit Support in their cluster and what to look for in their audit logs to ensure that their cluster is secure.
https://sched.co/MPcS
A
Alright,
everybody,
let's
get
started,
hope
you're
excited
for.
What's
the
last
talk
of
day,
one
I'm
mark
stem
and
we're
going
to
talk
about
how
to
use
kubernetes
audit
logs
to
secure
your
cluster.
So
a
little
bit
about
me,
my
work
at
Cystic
I'm,
a
senior
software
engineer.
There
I'm
also
one
of
the
maintainer
Zahn
Falco,
which
is
our
open
source
security
product
and
there's
my
github
alright.
So
what
we're
gonna
do
to
start
off
with
Lissa's.
Let's
talk
a
little
bit
about
kubernetes
audit
events
in
audit
support.
A
Here's
an
example,
audit
event,
I've,
trimmed
it
a
little
bit
because
there's
a
whole
lot
of
stuff
in
there,
but
I
kept
all
the
interesting
bits.
So
you
can
see
it's
just
a
you
know,
JSON
object,
and
you
know
it's
got
stuff
like
the
timestamp
of
when
the
person
did
their
operation,
what
action
they
were
taking.
That's
the
verb.
You
know
because
it's
all
like
restful
api
s,
it's
very
straightforward-
to
translate
the
HTTP
verbs
into
the
verbs
in
the
audit
event.
A
You
can
see
what
you
are
all
there
accessing,
which
is
often
the
resource
that
they're
grabbing
in
cases
where
you're
doing
something
specifically
to
a
referred
object
that
will
be
in
the
audit
event
as
well,
in
this
case
somebody's
trying
to
do
a
delete
of
a
namespace,
so
the
object
reference
is
in
there
and
you
can
say
like
hey,
it's
a
the
resources,
namespaces
and
then
here's
the
specific
namespace
that
they're
acting
on
you
can
see
user
information.
In
this
case
it
was
mini
cube.
A
You
can
see
our
back
information,
so
you
can
see
what
the
result
of
the
authorization
decision
and
you
can
see
like
the
HTTP
response
code
and
that
kind
of
stuff.
So
it's
a
really
rich
source
of
information
about
what's
going
on
for
each
of
these
events,
so
just
I
wanted
to
give
a
preview
of
like
some
of
the
you
know,
all
the
stuff
that
you
can
kind
of
get
out
of
these
audit
logs.
It's
a
pretty
rich
source
of
information.
A
A
If
you
create
service
accounts,
if
you
create
role
bindings
to
link
the
service
accounts
and
the
roles
together,
you're
gonna
see
all
of
that
stuff
in
the
audit
log
as
well.
So
it's
a
you
know
great
way
to
see.
What's
going
on
and
also
you
can
see,
you
know
viewing
stuff.
So
every
time
you
write
cube,
Caudill
get
pods,
that's
a
basically
a
get
request
on
the
resource.
Pods
and
you're
gonna
see
that
in
the
audit
log
all
of
the
above
includes
both
actions
by
individual
users.
So
me
typing,
mini
cube.
A
Are
you
using
cube
Caudill
to
manage
your
production
cluster,
but
it
also
includes
internal
kubernetes
services,
so,
for
example,
when
you
create
a
namespace,
it'll,
usually
create
a
service
account.
That's
working
with
that
namespace
and
you'll
see
that
in
the
audit
log,
even
though
you
didn't
create
the
service
account,
it
was
actually
something
like
cubes,
cubes
scheduler.
Creating
the
service
account.
So
you'll
see
that
as
well.
A
Now,
let's
talk
a
little
bit
about
how
you
actually
enable
and
work
with
audit
logging.
So
there's
a
couple
big
pieces.
The
first
is
the
audit
policy
that
controls
what
events
go
in
the
audit
stream.
So
in
there
you
know,
when
you
define
one
of
these
audit
policies.
It's
basically
just
a
ya,
know:
file
that
has
a
list
of
and
all
of
the
audit
events
that
are
coming
out
of
the
API
server,
get
matched
against
those
set
of
rules
and
if
it
finds
a
match
between
the
rule.
A
In
the
event,
the
event
goes
in
the
audit
lock.
So
you
can
do
things
there
saying
I
only
want
to
see
the
things
that
relate
to
pods
or
I
only
want
to
see
the
deletes
of
anything.
So
the
Paul
audit
policy
is
your
first
step
towards
filtering
out
the
audit
events
to
show
you
the
stuff
you
want
to
see.
The
other
piece
is
the
audit
backend
and
that
basically
controls
where
the
audit
events
go.
You
have
a
couple
choices.
A
It
can
go
to
log
files,
so
you
know
just
some
log
on
the
same
machine
where
the
API
server
is
running
and
you
can
also
go
to
web
hook.
So
you
can
configure
is
saying:
hey
when
you
get
a
bunch
of
audit
events,
go
to
an
HTTP
POST
to
this
URI,
and
then
somebody
else
will
take
care
of
it
from
there
and
that's
all
in
111
in
113.
One
thing
they
added
with
this
was
this
notion
of
dynamic
audit
backends.
So
there
it's
a
little
bit
like
on
the
web
book
and
the
audit
policy
combined.
A
So
now,
let's
talk
about
what
goes
in
the
audit
log
for
each
of
these
backends.
So
when
you
set
up
a
log
back-end,
it's
just
one
line
per
JSON
record
and
one
record
per
audit
event.
So
it's
just
the
JSON
object
that
we
saw
before
just
one
line
at
a
time.
So
pretty
simple,
you
have
controls
to
basically
say
like
where
the
log
file
goes,
how
to
rotate
it.
A
You
could
do
rotation
based
on
size
or
age
or
anything
like
that
stuff.
When
you
set
up
a
web
hook
back-end,
what
happens?
Is
it
does
a
little
bit
of
batching
so
it'll?
Take
a
small
number
of
events.
Turn
it
into
a
little
chunk,
make
a
giant
JSON
array
out
of
it
and
then
post
it
off
to
your
web
book.
A
Again,
you
have
controls
for
the
location,
so
you
can
say
like
what
host
name
and
so
forth
to
send
it
to
and
then
some
controls
about
how
the
batching
actually
happens.
You
can
say,
like
oh,
wait:
500
milliseconds
before
sending
me
some
events
or
wait
five
seconds
before
sending
me
some
events
and
also
some
controls
about
like
there's,
some,
like
rate
limiting
and
other
kinds
of
stuff
that
you
can
do
when
you're
setting
up
the
web
hook
back-end
when
you
set
up
the
dynamic
back-end,
it's
a
kind
of
a
combination
of
these
things.
A
So
the
process
for
enabling
audit
logging
is
a
little
different
between
111
and
113
in
111.
A
lot
of
these
components
are
static.
So
they're
generally
like
arguments
to
the
API
server
command-line
arguments,
so
you
know
you'll
see
a
command-line
argument.
This
is
audit
web
config
file
and
then
you'll
name
a
file,
and
then
that's
where
you're
gonna
specify
where
the
web
hook
goes
same
thing
for
like
the
log
path
and
where
the
policy
the
audit
policy
is
located
because
it's
all
I
can
lick
a
command
line.
A
You
you
know
generally
have
to
restart
the
API
server.
If
you
ever
want
to
change
one
of
these
things,
so
it's
not
as
flexible
as
the
mechanism
that
you'll
find
in
1.13,
and
it's
also
a
little
hard
to
see
what
any
of
these
contents
are.
Unless
you
can
get
on
to
the
API
server
and
so
for
like
manage
kubernetes
environments,
you're
not
gonna,
be
able
to
see
that
stuff
at
all
or
they're.
Gonna
have
a
different
visibility
way
to
show
it
to
you
in
113.
They
cleaned
up
a
lot
of
that
stuff.
A
So
now
it's
just
these
audit
sync
objects
that
you,
you
know
like
I
said
you
create
them
as
resources
and
can
act
on.
Like
anything
else,
you
generally
have
to
be
a
cluster
admin
to
do
it,
but
the
nice
thing
is,
you
can
create
multiple
of
them,
so
you
can
create
one
audit,
sync
that
has
a
policy
and
a
destination,
and
then
you
can
create
another
audit,
sync
that
has
a
different
policy
and
maybe
a
different
destination,
and
you
can
create
them
in
parallel
and
then
the
audit
events
will
go
to
both
locations.
A
Okay.
So
now,
hopefully
you
know
what
audit
events
are
now,
let's
talk
about
how
you're
gonna
get
data
out
of
them
and
make
sense
out
of
what's
in
the
audit
log?
Okay,
so
there's
two
ways
to
do
it,
there's
the
hard
way
and
the
easy
way,
I'm
a
falco
engineer.
So
as
a
little
hint,
the
easy
way
is
gonna
involve
Falco,
but
first
we'll
do
the
hard
way
so
on
a
really
handy
program
that
you
can
use
for.
This
is
called
JQ.
If
you
haven't
used
it,
it's
kind
of
a
streaming
json
processor.
A
What
happens
is
it
reads?
Json
objects
from
standard
input
does
some
transformations
and
filtering
and
so
forth,
and
then
just
writes
JSON
objects
to
standard
output
or
strings,
or
something
like
that.
So
it's
a
really
handy
way
to
act
on
these
audit
logs
to
extract
specific
information
out
of
it
just
some
examples
of
what
you
can
do
with
JQ,
so
you
can
do
selections.
So,
given
this
input
object
show
me
the
value
of
the
property
called
key
and
it'll
print
out
some
value.
A
If
you
want
to
do
filtering,
you
can
get
an
array
of
objects
and
say:
hey
only
give
me
the
objects
where
the
value
of
key
is
v1
and
it'll
just
give
you
that
object.
You
can
also
do
transformations.
You
can
say,
hey
give
me,
take
this
object
and
then
output
a
string,
that's
a
mix
of
like
plain
text
and
the
value
of
the
property
called
key
from
the
input,
object
and
it'll
print
out
an
output
string.
A
Okay,
so
once
we
have
JQ,
we
can
basically
build
up
a
bunch
of
filters
that
work
on
the
log
file
to
extract
interesting
information
out
of
the
audit
stream.
So
here's
just
some
examples
of
what
you
can
do
using
jq1.
You
know.
The
first
example
is:
let's
say:
I
want
to
find
out
anytime.
Somebody
creates
a
namespace,
so
the
first
line
is
what
we're
gonna
do.
A
The
selecting
on
we're
gonna
say
show
me
the
objects
where
the
ver,
the
value
of
the
verb
property
in
the
object,
is
create,
and
then
where
this
is
like
a
nested
search
where
object
referee
source
is
namespaces,
so
somebody's
doing
a
create
and
they're
doing
it
to
a
namespace.
The
namespaces
resource
when
I
do
that
the
string
I
want
to
print
you
want
you
to
print
out
is
the
second
line.
So
it's
just
a
plain
text
string,
but
hey
take
the
time
stamp,
put
some
brackets
around
it.
So
it
makes
look.
A
It
makes
it
look
a
little
bit
more
like
a
log
and
then
print
out
this
string,
saying
namespace
was
created
and
then
get
object.
Graph.
Okay,
deleting
a
namespace,
is
very
similar,
but
all
we're
doing
is
changing
what
the
verb
is
from
create
to
delete
this
third
one's
a
little
bit
more
fun.
Where
we're
basically
saying
like
hey,
show
me
the
cases
where
somebody
creates
a
config
map
and
they're
really
dummies
and
they
put
in
their
AWS
private
key
in
the
config
map.
A
Don't
do
that
do
secrets
instead,
but
if
you
did
do
that,
this
would
be
the
way
you
could
look
for
it.
So
the
first
line
is
very
similar.
It's
saying,
hey
show
me
the
objects
where
somebody's
doing
it
create
and
the
resource
that
they're
doing
is
config
maps
and
then
the
second
line
is
kind
of
a
filter.
Saying
okay,
when
I
take
the
request,
object
and
I,
convert
it
to
a
string
and
then
I
search
in
the
string.
Do
I
see
AWS
access,
key
ID.
A
A
I
didn't
write.
Just
these
three
I
wrote
a
whole
bunch
of
different
ones
and
you
can
find
them
all
on
github,
it's
a
little
just
there.
It's
just
called
JQ
filters
for
kubernetes
audit
events
and
I.
Think
I
got
like
20.
There's
then
look
for
different
use
cases
so
go
to
town
with
that.
Okay.
So
that's
the
hard
way.
If
you
love
Jake,
you
stick
with
Jake
you,
but
if
you
love
Falco
and
you
should
use
Falco
and
if
you
don't
know
what
Falco
is
I'm
gonna
make
you
love
Falco.
A
So
Falco
is
a
recent
CN
CF
project
cystic
it
owned
it
for
a
while.
We
basically
moved
it
over
to
the
CN
CF
and
it's
basically
a
run
time
activity
monitor
what
it
does
is
accepts
events
of
various
types
and
then
it
matches
the
events
against
a
bunch
of
rules
to
look
for
notable
or
suspicious
events
and
when
an
event
is
found,
it
basically
sends
information
about
the
event
to
one
of
a
number
of
outputs
back
in
December.
We
added
support
for
kubernetes
audit
events
to
Falco.
A
We
basically
added
a
little
embedded
web
server
into
the
Falco
binary
that
can
be
used
post
these
events
too,
and
then
we
wrote,
you
know,
30
odd,
new
rules
to
look
for
specific
use
cases.
They
kind
of
fell
into
three
different
classes
on
the
first
one
is
kind
of
suspicious
activity.
This
is
cases
where
the
rules
are
pretty
opinionated
and
they're.
Looking
for
specific
notable
things
or
bad
things,
the
second
class
is
just
changes
to
your
cluster.
So
this
isn't
quite
so
opinionated,
but
it
is
letting
you
know
anytime.
A
Somebody
tries
to
create
something
or
delete
something
or
modify
something,
and
the
final
class
is,
if
you
don't
get
what
you
want
out
of
the
anything
else,
we
do
have
one
rule
which
is
literally
print
every
audit
event.
We
get.
You
probably
don't
want
to
turn
that
on
in
production,
but
it's
a
good
thing
for
like
debugging
type
stuff,
so
here's
the
architectural
diagram
of
Falco,
just
because
you
can
kind
of
see
the
flow
of
the
messages
and
how
it
works.
Everything
in
the
dashed
blue
box
is
the
Falco
binary
itself.
A
It
kind
of
starts
with
these
rules.
The
Falco
rules
and
the
C
my
lower
right
and
also
your
lower
right
and
the
rules
get
loaded
up
into
a
rule
engine,
and
then
they
just
accept
events
from
a
variety
of
sources
to
see
if
the
events
match
one
of
the
rules,
traditionally
it
works
with
system
calls.
So
we
have
this
kernel.
Module
for
system
calls,
but
importantly
for
kubernetes
audit
support.
It
all
comes
from
this
embedded
web
server.
So
you
enable
kubernetes
audit
support
and
your
API
server
have
it
post
to
the
web
hook.
A
Our
web
server
picks
it
up,
takes
to
the
events,
matches
it
against
the
rules
and
then,
when
an
event
matches
one
of
the
rules,
we
send
it
to
one
of
the
outputs,
so
you
can
send
it
to
like
syslog
brought
it
to
a
log
file
post
it
to
a
slack
web
hook.
Send
an
email
make
pagers
go
off
if
everybody
still
has
a
pager,
so
that
kind
of
stuff.
A
A
So
there's
two
kinds
of
this
is
all
yam
all
there's
two
kinds
of
objects
here:
there's
macros
and
rules:
macros
are
kind
of
just
used
for
code,
reuse
in
structure
and
that
kind
of
stuff,
so
the
first
macro
is
called,
create
private
credentials
and
the
condition
field
is
basically
a
filter
that
a
little
filter
snippet
that
we're
using
to
match
against
the
events.
In
this
case
we're
saying:
hey
it's
a
like
a
logical
expression
combined
with
these
field
names
and
then
it
resorts
results
in
a
true
or
false
value.
A
A
Obviously,
if
you
put
your
password
in
your
config
map,
but
you
didn't
name
it
password,
we
won't
find
it.
But
if
you
did
that
you're-
probably
not
dumb
enough
to
put
your
password
in
the
config
map
in
the
first
place,
so
I'm
not
worried
about
those
guys
we're
worried
about
the
slightly
dumber
guys.
A
So
that's
the
first
macro.
The
next
macro
is
basically
just
called
the
config
map
and
it's
saying
is
the
target
resource
config
maps
again
for
each
of
these
fields,
we're
extracting
that
data
out
of
the
JSON
audit
event.
We
already
showed
you,
okay
and
the
third
macro
is
called
modifying.
We're
just
saying:
hey
is
the
verb
part
of
the
audit
event
either
create
or
update
or
patch.
Okay.
A
Now
we
have
a
rule.
Rules
have
names
in
this
case.
It's
create,
modify
config
map
with
private
credentials
on
they
have
a
description
which
is
nice
for
output
purposes,
and
then
they
have
the
condition
again,
where
we're
filtering
against
the
audit
events
to
see
if
something
matches
or
doesn't
match
and
we're
just
taking
all
the
macros
and
just
putting
them
together
with
ands.
A
So
it
was
a
config
map
and
it
was
a
modify
action
and
when
you
go
look
at
the
contents
of
the
config
map,
it
had
something
shady
and
it
like
a
password
when
the
event
matches,
we
have
to
decide
what
string
to
send
to
our
outputs
and
that's
controlled
by
this
output
property
on
the
object,
it's
kind
of
a
mix
of
plaintext
and
thus
printf
style
fields.
Again,
the
values
of
these
fields
come
from
the
audit
event
itself.
A
We
kind
of
run
it
through
a
glorified
version
of
sprint
up
and
then
that's
the
message
we're
going
to
send
off
to
the
emails
and
slack
and
so
forth.
Okay,
and
then
we
have
like
priority,
so
you
can
do
some.
You
know
severity
matching
and
only
you
know
really
can't
make
the
pages
go
off
or
the
Critical
stuff,
but
not
the
debug
stuff
and
and
you
could
have
tags,
so
you
can
group
rules
together,
okay,
so
what
I
thought
we'd
do
now
is:
let's
kind
of
jump
into
some
ideas
about
how
to
use
audit
logs.
A
A
So
the
first
thing
is:
maybe
you
just
want
to
know
what's
going
on
right,
you're,
not
particularly
opinionated,
to
look
for
bad
activity,
but
you
just
want
to
see
all
the
changes
that
are
happening
to
your
cluster.
So
we
have
a
bunch
of
rules
that
look
for
all
that
stuff,
so
kubernetes
deployment
created
or
deleted
service
created
or
deleted
all
that
kind
of
stuff,
both
the
main
resources
and
also
like
our
back
stuff.
So
when
you
create
service
accounts
or
roles
or
role
bindings,
we'll
see
all
that
stuff
as
well.
A
Okay,
so
getting
a
little
bit
more
opinionated,
let's
say
one
thing
you
want
to
do:
is
you
want
to
just
like
define?
What's
allowed,
like
only
let
certain
users
do
stuff
to
my
cluster
or
only
let
pods
run
if
the
image
is
in
a
set
of
images
that
I
say
are
okay
or
only
let
people
run
in
the
production
namespace
and
the
dev
namespace,
but
not
the
super
hacker
or
a
crypto
mining
namespace.
A
There's
somebody
created
that
you
didn't
find
out
about
until
two
months
later,
when
you
wonder
why
your
AWS
bill
is
10
grand
if
you're
lucky.
So
we
have
a
bunch
of
rules
that
look
for
that
kind
of
stuff,
and
each
of
these
rules
is
basically
associated
with
a
list
so,
along
with
disallowed
gates,
user,
there's
a
list
of
allowed
keith's
users,
and
then
it's
just
matching
the
user
against
the
contents
of
the
list.
I
know
someone
told
me
that
all
security
is
just
list
matching
and
but
yeah.
A
Now,
if
you
want
to
get
even
more
opinionated,
where
you're
looking
for
more
specific
use
cases,
we
we
can
do
like
another
set
of
class
of
rules,
so
maybe
you
want
to
be.
Maybe
you
want
to
limit
what
pods
can
access?
If
you
want
to
say,
okay,
good,
no,
don't
leave!
You
can't
go
to
the
keynote,
lock
the
doors.
A
Okay.
So
if
you
want
to
get
a
little
more
opinionated,
let's
say
you
want
to
limit
what
pods
can
do.
You
know,
following
the
principles
of
least
privilege,
let's
try
to
explicitly
enumerate
what
your
pod
really
needs
to
do
to
do
its
job
and
not
give
it
any
more
privileges
than
that.
So
we
have
one
rule.
That's
create
privileged
pod.
That
basically
just
looks
for
anybody
starting
a
pod
where
they
set
the
privilege,
true
flag.
A
You
know
maybe
there's
a
good
case
to
do
it,
but
generally
you
at
least
want
to
be
aware
of
it
and
might
want
to
take
stronger
actions.
Based
on
that,
so
that's
one
rule
that
we
have
another
one
is
create
sensitive
mount
pod.
What
that
does?
Is
we
basically
look
to
see
what
mounts
you
have
for
the
pod
when
you
create
it,
and
we
see
if
they're
mounting
paths
from
the
hosts
file
system
and
then
for
some
paths
we
consider
them
sensitive.
A
So,
for
example,
if
somebody
tried
to
mount
Etsy
shadow
from
the
host
into
your
pod-
probably
not
a
great
idea
or
if
they
tried
to
mount
proc
or
if
they
tried
to
mount
the
docker
socket
or
if
they
tried
to
mount
Etsy
crontab,
so
they
could
launch
other
attacks
directly
on
the
host.
We
have
a
fun
blog
post
about
that
in
crypto
mining.
So
we
have
a
rule
that
basically
looks
on
that.
A
We
declare
some
paths
on
the
host
to
be
sensitive
and
if
somebody
creates
a
pod
that
mounts
any
of
them,
that's
when
that
rule
kicks
in
another
one
is
creating
a
pod
with
host
networking
to
be
true
generally,
if
you
can
get
away
with
it,
try
to
keep
the
pods
in
their
own
network
namespace-
maybe
you
don't
have.
Maybe
you
have
a
good
reason
for
that,
but
at
least
it's
something
you
might
want
to
be
aware
of,
and
then
another
one
is
creating
a
pod
in
the
cube
any
of
the
cube
namespaces.
A
A
It's
a
great
Avenue
for
privilege
escalations,
so
something
you
want
to
really
be
aware
of.
So
we
have
a
rule
that
basically
looks
for
that
is
saying:
hey
somebody's
trying
to
exact
into
a
pod
and
we'll,
let
you
know
about
it:
we
have
a
bunch
of
rules
devoted
to
like
users
and
account
level
stuff.
So
we
have
a
rule
service
account
created
in
the
cube
namespace.
So
this
is
exactly
that
you're,
any
service
count
and
you're,
and
one
of
the
cube
namespaces
probably
shouldn't
be
over
there.
A
Another
one
system,
cluster
role
modified
or
deleted.
If
you're
doing
any
action
on
cluster
roles
and
the
name
of
the
role
start
a
system,
that's
kind
of
like
a
reserved
thing
for
vanilla,
kubernetes,
probably
shouldn't
be
doing
that
so
we'll.
Let
you
know
if
somebody
tries
these
view,
are
about
being
lazy
when
you're
defining
your
roles.
So
when
you're
defining
your
roles,
you
really
should
be
limited
to
only
explicitly
enumerate
the
things
you
want
and
don't
ask
for
anything
else,
don't
give
them
anything
else
than
what
they
really
need
to
run.
A
So,
if
you're
writing
one
of
these
roles
and
you're
new
to
our
back
and
you're
like
oh,
my
god,
what
does
all
this
yeah
mole?
But
you
saw
like
oh
there's
this
cluster
admin.
If
I
had
to
do
that,
I
can
just
do
the
things.
I
want
to
do.
That's
a
good
idea
if
you're
just
learning
out,
but
it's
probably
not
a
great
idea
later,
because
cluster
admin
can
do
all
kinds
of
stuff.
So
probably
not
a
rule
binding
that
you
want
to
grant,
and
so
we
ever
we
have
a
rule
for
that.
A
When
you're
defining
your
role,
you
might
be
a
little
bit
lazy
and
you
might
say
you
know
what
I'm
just
gonna.
Let
people
do
anything
to
pods.
So
for
the
for
the
verb,
I'll
just
say,
star
or
I'll,
say
you
know
what
I'm
gonna,
let
people
do
crates
or
maybe
lists
on
anything
so
I'll
explicitly
say,
get
or
yeah
view
or
list
only,
but
then
for
the
resources
I'll
just
put
star.
A
So
if
you
do
that,
probably
not
a
great
idea,
it'd
be
much
better
if
you
explicitly
laid
out
the
things
that
you
needed,
and
so
if
you
have
that
we
that's
what
this
rule
is
looking
for.
It's
looking
for
people
where
they're
trying
to
wild-card
either
the
resources
or
the
verbs
when
they're
defining
their
role
a
couple
either
a
couple
easier
ones
that
are
more
specific.
So
if
somebody
cries
tries
to
create
a
cluster
role
that
has
right,
privileges
will
let
you
know
about
it.
A
Obviously,
there's
great
great
use
cases
for
it,
but
there
might
be
cases
where
you
at
least
want
to
be
notified
when
somebody's
doing
it
and
another
one.
This
is
getting
back
to
this
idea
of
privilege
escalation
in
pods.
Even
if
you
didn't
let
anybody
actually
exec
into
in
Depaz,
but
you
left
the
our
backside
open.
They
could
just
basically
create
a
role
that
lets
them
executive
pods,
even
though
you
didn't
let
them
and
then
they'll
be
able
to
executive
pods.
So
it's
kind
of
like
a
second
order,
security
problem.
A
So
if
you've
tried
to
create
a
role
that
allows
the
pod
exec
privilege
we'll
let
you
know
about
it
and
then
a
few
I
couldn't
easily
categorize,
but
I
still
wanted
to
talk
about.
We
already
talked
about
this
one,
the
creating
the
config
map
with
the
secret
stuff
in
it.
There's
this
there's
this
one
where
it's
a
little
bit
legacy.
But
if
you
create
a
request-
and
you
don't
actually
have
authentication
configured
properly,
you
end
up
end
up
being
this
anonymous
user.
A
A
We'll
just
create
a
simple
deployment
based
on
nginx,
so
you
can
see
that
it
showed
up
in
both
both
things.
It
showed
up
a
little
quicker
in
the
audit
log
based
one
mostly
because
there
isn't
any
batching
there.
So
the
you
know
it's
just
reading
the
file,
it
just
says
the
second
it
gets
written
and
it's
not
waiting
to
batch
up
any
events
and
send
them
to
the
API
server.
So
it
shows
up
a
little
bit
quicker
there,
but
it
shows
up
easily
in
both
and
you
can
see
it's
like
hey
somebody
created
a
deployment.
A
A
A
A
Okay,
so
let's
do
some
stuff
for
some
more
stuff
related
to
config
Maps,
here's
a
sample
config
map.
This
is
getting
into
that
config
maps
with
passwords
in
it.
So
here's
a
simple
config
map
up
here.
We
got
the
contents
of
the
config
map
and
look
I'm.
A
dummy
I
put
my
AWS
access
key
and
the
kid
big
map.
So
not
a
good
idea,
so
we'll
just
go,
create
that
and
kind
of
see
what
happens.
A
A
A
Okay,
so
we
exactly
Intuit
running
a
shell,
and
now
we
have
all
the
all
the
powers
that
you've
granted
to
then
nginx
deployment,
but
because
we've
noted
the
pod
exec,
both
in
JQ
and
Falco.
We
will
let
you
know
that
somebody
tried
to
do
the
exec,
it's
basically
going
through
the
API
server
and
it's
just
a
get
on
the
pods
exec
resource.
A
You
can
you
we
can
do
that
with
Falco,
but
not
using
the
kubernetes
audit
events,
because
those
once
you
have
the
exec
session
open,
I'm.
Sorry,
the
question
was:
can
you
see
all
the
commands
that
the
person
is
now
typing
now
that
they
exact
you
can
do
that
if
you
use
the
system
call
based
stuff-
and
we
have
something-
that's
a
little
bit
like
that,
though
it's
pretty
false
puzzled
approach.
A
A
A
A
You
can
see
that
both
in
the
ghj
queue
based
output
and
the
falco
based
output,
because
we're
looking
for
the
config
map
we're
sorry
for
the
for
the
cluster
role,
looking
at
it,
seeing
that
there's
a
wild
card
in
there
and
we'll,
let
you
know
about
it:
okay,
another
one!
So
let's
look
at
the
one
where
you
try
to
grant
permissions
to
pod
exact,
even
though
you
weren't
able
to
pod
exec
before
so.
Here's
another
cluster
role
in
custom
role,
binding
where
we're
basically
grant
trying
to
grant
ourself
the
ability
to
exact
into
pods.
A
A
So
I'm
going
to
create
some
namespace
called
secret
namespace,
put
all
my
bad
stuff
in
there
and
all
my
crypto
mining
and
stuff.
Well,
you
do
that.
Well,
will
detect
that
to
you,
because
number
one
we
can
detect
when
the
namespace
code
created
and
then
also,
if
you
had
this
allowed
list,
not
allowed
list
we'll
check
it
against
the
list
and
let
you
know
it's
outside
of
the
list
of
things
that
we
expected
to
see.
A
All
righty
so
hopefully
you're
super
psyched
about
kubernetes
audit,
logs
or
hopefully
Falco,
because
we
love
Falco.
So
here
we've
got
all
the
links
for
you.
Falco
org
is
the
main
site
for
Falco.
We
have
a
public
slack
and
there's
a
Falco
Channel
on
there,
where
you
can
learn
more
about
Falco.
We
got
a
bunch
of
blog
post
where
we
talked
about
Falco
use
cases
and
that
kind
of
stuff,
it's
all
open
source.
So
you
can
just
get
the
source
code
on
github.
A
We
got
Doc's
there
and
you
know
we
published
docker
images
of
all
the
software,
so
you
can
learn
more
about
the
images
and
that
kind
of
stuff
and
I
think
I
was
supposed
to
say
sche
decom.
If
you
love
this
talk
and
love
Falco
go
review.
The
talk
there
is
that
right,
all
right
there,
you
go
all
right
thanks
a
lot.