►
From YouTube: Implementing Least Privilege Security and Networking with BPF on Kubernetes - Arvind Soni, Covalent
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Implementing Least Privilege Security and Networking with BPF on Kubernetes - Arvind Soni, Covalent
BPF is becoming the fastest growing technology in the Linux kernel and is revolutionizing networking, security, and tracing. At the same time, the rise of Kubernetes is creating demand for routing, load-balancing & security infrastructure that is highly scalable, application-aware, and resilient. Microservices architectures divvy up application functionality into services and expose them via APIs using protocols such as HTTP/REST, gRPC, or Kafka. This creates new challenges. What was previously traditional Layer 3-4 networking security (limited to the IP and port level) now exposes either the entire API surface or none of it. This is insufficient to implement least privilege security for microservices. This talk introduces the open source project Cilium - built on BPF to provide Linux native networking and least privilege security for microservices while integrating with Kubernetes.
To learn more: https://sched.co/GrWl
A
Good
afternoon
and
thank
you
for
joining,
my
name
is
Arvind.
I
work
with
the
ISO
valent
team
is
founding
company
behind
the
open-source
project
called
helium
and
we're
going
to
talk
about
as
I
mentioned,
the
powerful
new
technique
in
linux,
kernel
called
BPF
and
how
BPF
can
be
brought
to
a
kubernetes
cluster
near
you
right
so
to
speak
and
delivered
networking
and
security
for
kubernetes,
oh
yeah.
A
So
the
agenda
is
roughly
will
give
you
a
little
bit
of
overview
about
BPF,
then
how
Celia
Muses
BPF
for
kubernetes
we'll
try
our
hands
and
struggle
with
the
projection
and
things
like
that
with
the
demo
and
font
size
and
terminals
and
Q,
Khan
and
Q
cutoff,
and
think
like
that.
So
hopefully
that
will
go
well
and
then
we'll
take
a
look
at
some
of
the
cool
stuff
that
we
are
doing,
which
were
also
covered
in
a
couple
of
previous
talks.
So
well.
I'll
just
mention
that,
briefly,
towards
the
end.
A
Ok,
so
I
think
in
recent
times
there
have
not
that
many
aha
moments
in
Linux,
kernel
and
I
think
safe
to
say
that
in
for
more
than
a
decade
or
so,
BPF
is
probably
the
most
exciting
kernel
technology
to
come
out.
It's
a
pretty
powerful
technique
and
it
is
already
impacting
a
wide
range
of
areas
in
Linux
systems,
operations,
troubleshooting,
networking
performance
analysis,
DDoS
mitigation,
firewalls,
and
things
like
that.
A
As
far
as
we
know
and
the
the
gist
of
this,
this
technique
is
actually
it's
very
simple:
animation
like
a
overly-complicated,
Linux
kernel,
module
or
something
that
you
need
to
wrangle
with
install
and
it's
a
it's
a
built-in
technology,
and
if
you
have
used
any
of
you
have
used
TCP
dump
in
the
past.
This
was
the
technique
behind
it.
A
Hence
the
wonderful
name,
Berkeley
packet
filter
at
this
point-
it's
a
big
misnomer,
because
it's
not
just
for
packet
filtering.
You
can
attach
these
small
programs
to
many
different
types
of
system
calls
and
kernel
events.
You
can
attach
it
to
memory
requests
to
file
descriptors
opening.
You
can
attach
it
to
observe
things
for
performance.
You
can
attach
it
to
do
extra
processing
of
packets,
denying
them
forwarding
them
and
things
like
that.
So
it's
essentially
small
programs,
byte
code
programs.
A
You
write
them
in
C
and
Python
like
a
upper
level
language
it
gets
compiled
into
a
bytecode
before
it
is
loaded
and
executed
in
the
kernel
space.
It's
verified
so
that
there
are
no
loops.
There
is
no
incorrect
memory
access
and
things
like
that.
So
there
is
a
just-in-time
compiler
and
a
verifier.
The
JIT
makes
it
efficient.
So
there
is
no
interpretation
going
on
it's
not
like
each
and
every
bytecode
is
getting
interpreted.
It
gets
compiled
into
underlying
CPU
architecture.
A
It's
not
a
linux
kernel
module.
This
is
one
of
the
questions
that
we
got
very
frequently.
Is
it's
just
a
technique,
just
like
other
Linux
techniques
that
are
available
in
the
Linux
distribution
and,
at
this
point,
I
think
all
major
distributions
have
announced
support
for
this
I
think
the
last
was
rl8
and
they
are
working
on
back
porting
it
as
well
Oh
point
218,
AWS
Linux,
pretty
much
all
the
recent
instances,
gke
costs
I
mean
you
name
it
it.
As
you
say,
all
major
distributions
have
support
for
this.
A
At
this
point,
and
for
those
of
you
who
just
came
in
BPF
part
is
essentially
just
a
small
bytecode
program
that
you
can
attach
to
various
system
calls
okay
beyond
that,
I
don't
know,
so
it
is
pretty
powerful
in
in
what
you
can
do.
If
you
attach
programs
like,
if
you
think
about
it,
Linux
does
a
lot
of
things
from
packet,
processing,
handling
files,
handling
various
devices
and
things
like
that.
A
These
are
the
couple
of
use
cases
that
I
pulled
out,
because
this
is
kind
of
telling
of
where
we
want
to
go
with
kubernetes
and
BPF,
and
things
like
that.
One
of
the
popular
use
case
is
around
load
balancing.
So
when
the
packets
come
in,
you
know
the
source
and
destination
adults,
and
let's
say
you
just
want
to
forward
it
for
TLS
termination
or
further
processing.
Then
you
can
do
it
very
efficiently.
It
doesn't
have
to
go
up.
The
Linux
stack
right
packet
comes
in.
A
You
can
just
forward
it
somewhere
at
the
very
early
stage
and
that's
what
the
xdp
part
does.
So
you
can
see
that
the
kubernetes
also
has
a
lot
of
load
balancing
going
on
right
like
cluster
IP,
node
port.
They
all
have
the
underpinnings
of
taking
the
packet
forwarding
it
and
it
spraying
it
to
some
other
place.
So
the
load
balancing
is
one
obvious
use
case
for
taking
BPF
applying
to
kubernetes.
Another
good
area
is
around
dropping
the
packets
or
filtering
the
packets
right.
A
You
have
some
PCI
sensitive
pods
and
services,
and
you
don't
want
them
going
out
to
the
internet.
Here
you
can
just
look
at
the
destination
header
and
drop
them,
so
very
efficient
firewalling
can
be
built
very
efficient
load
balancing
can
be
built
using
VPS
and
that's
the
idea
behind
helium
project
to
take
this
powerful
technique
and
help
you
run
micro
services
at
a
large
scale,
with
higher
churn
much
more
efficiently.
Then
and
cerium
is
open
source.
It's
a
project.
A
It
was
founded,
but
slightly
over
two
years
ago,
and
this
is
a
good
artistic
rendition
of
our
ambitions.
We
want
to
tear
apart
the
shackles
of
monolithic
structure
that
are
there
in
the
Linux
kernel,
iptables
being
sort
of
one
of
the
representatives.
So
if
you
think
of
your
micro-services
as
a
rocket
ship
moving
fast
or
like
a
fast
moving
ship,
these
are
kind
of
the
anchors
pushing
you
down.
A
You
should
have
seen
the
artwork
sketch
work
that
we
sent
it
to
the
designer
and
the
guy
really
job
with
it,
explaining
and
capturing
the
intention
so
that
the
so,
let's
cover
a
little
bit
about
what
is
what
is
needed
from
a
least
privileged
network
security
perspective.
That's
a
mouthful
of
a
title.
It's
essentially
just
saying
what
parts
can
talk
to
which
other
parts
and
services
internally
externally
with
in
your
data
center
across
the
data
center,
and
things
like
that.
A
Just
to
illustrate
the
point,
let's
say:
there's
a
bunch
of
front-end
ng
and
X
and
a
pod.
There
are
some
memcache
services
which
are
expediting
the
key
lookup.
You
can
have
the
front
and
pod
insert
things
into
this
memcache,
but
giving
this
front-end
access
to
something
like
get
all
is
probably
disastrous.
At
this
point,
we
all
know
that
it's
like
a
very
expensive
operation
from
memcache
and
causes
widespread
disruptions
yeah,
but
that
firewalling
of
what
to
allow
and
what
not
to
allow
is
not
an
l3
l4
security
right.
It's
more
of
an
API
aware
security.
A
Yes,
what
front-end
can
talk
to
memcache?
But
what
exactly?
Should
it
be
able
to
do
it's
becoming
more
and
more
important
because
we
are
taking
this
monolithic
applications
which
used
to
nicely
talk
using
function,
calls
we
are
replacing
them
with
API
calls.
So
previous
security
of
nice
function
calls
inside
the
operating
system
has
gone
into
API
communications
over
the
network.
So
just
doing
an
l3
l4
is
insufficient.
You
have
to
go
into
enforcing
API,
aware
security,
and
you
can
see
the
movement
around
this
in
sto
and
MPLS
and
various
other
efforts
making
things
API
aware.
A
So
that's
the
part
which
is
inside
the
cluster.
That's
the
sort
of
the
east-west
security.
It
has
to
go
from
l3
l4
to
API
of
a
level
when
the
thing
when
things
are
going
out,
the
identity
outside
the
cluster
is
little
bit
difficult
to
establish.
But
at
least
DNS
aware.
Security
is
something
that
we
should
all
aspire
to
achieve
right,
because
DNS
is
a
good
identity
for
all
pretty
much
all
external
services.
So
we'll
talk
about
how
we
can
enable
DNS,
aware
security.
Last
but
not
least,
is
the
part
around
who
are
we
securing?
A
What
is
the
identity
of
this
part
in
the
good
old
V
M
days?
You
had
a
bunch
of
domain
controllers
and
you
knew
they
were
domain
controllers
and
they
had
a
nice
IP
and
you
could
say
well
those
are
domain
controller,
that's
the
identity
of
them
right,
but
now
these
parts
are
like
shapeshifters
and
if
they
are
running
with
one
IP
in
one
host
and
other
IP
in
another
hole,
they
are
alive
for
15
seconds
and
dead
after
that,
or
something
like
that,
so
this
is
very
hard
to
establish
identities.
A
For
these
ephemeral,
scale-out
hide
sharon
environments
at
least.
One
approach
is
to
take
the
meta
data
into
account.
What
labels
are
there
or
if
you
have
crypto
certificates
into
account,
to
establish
the
identity?
At
least
the
three
problem
domains
around
the
least
privileged
security
yard
is
what
I've
tried
to
capture
there's
the
identity
problem,
there's
the
enforcement
problem,
and
then
there
is
the
API
aware,
security
problem.
A
The
enforcement
suffers
with
one
other
problem
of
churn
these
parts
as
they
come
and
go
they're
eyepieces
they
come
and
goes
that
that's
where
the
monolithic
structures
like
IP
table,
start
hurting
and
that's
where
the
efficiencies
of
things
like
BPF
start
coming
in
I
promise
you
I
will
end
the
slides
very
soon.
The
cilium,
in
a
nutshell,
is
essentially
a
software
on
top
of
BPF.
It
listens
for
the
kubernetes
network.
A
Policies,
takes
those
network
policies
and
translates
it
into
BPI
programs
for
ingress
and
egress
rules
right
so
it
runs,
has
a
daemon
set
on
all
your
worker
nodes.
It's
listening
for
network
policy
objects.
When
the
policies
come
in
it
takes
it
listens
for
them.
It
takes
the
policy
input
and
just
attaches
BPF
programs
corresponding
to
that.
So
that's
the
pretty
easy
way
of
of
understanding
or
see
Liam
does
this?
Is
this
white
can
deliver
the
all
the
power
of
BPF
for
for
yo,
kubernetes
cluster
em?
A
A
So
that's
that's
one
of
the
more
harder
parts
how
to
iterate
on
the
firewall
firewalls
and
securities,
and
things
like
that,
and
it
has
same
constructs
like
endpoint
selector
ingress/egress,
a
white
blue
whitelist
rules
are
all
an
hour
of
each
other
okay.
So
let's
go
struggle
with
the
terminal,
and
hopefully
the
projections
will
be
fine,
but
this
is
what
I
wanted
to
go
over
rather
than
putting
amell's
and
trying
to
explain
what
each
of
them
are
doing
here.
A
Let's
just
run
them
in
the
terminal
in
a
smaller
environment
and
see
what's
the
impact
and
we
will
try
some
of
these
policies
right,
as
I
said,
if
you
have
policy
disabled,
it's
a
pretty
bad
security
posture,
any
part
can
do
anything
within
the
cluster
or
outside
the
cluster,
a
sort
of
a
good
starting
point.
You
can
start
with
something
like
default
deny,
then
nothing
is
happening
right
and
that's
one
posture,
but
a
decent
one
that
we
have
learnt
across
many
users.
Is
they
just
allow
within
the
namespace
to
begin
with?
A
So
all
the
services
within
the
namespace
are
able
to
talk
with
each
other,
but
you
have
to
allow
egress
to
cube
DNS,
we'll
see
how
you
can
do
that
securely
as
well,
then
the
other
sensitive
spot
is
around
the
Kuban.
It
is
API
server,
since
the
API
server,
the
back
endpoints
may
or
may
not
have
any
labels.
It
becomes
hard
how
to
establish
the
ciders
for
those
API
servers,
and
things
like
that.
So
there
is
a
construct
in
cilium
that
can
help
with
that
too,
but
kubernetes
api
server.
A
As
we
learned
from
the
recent
the
exploit
and
CVE
alert
that
it's
a
pretty
sensitive
means
holds
the
keys
to
the
entire
kingdom.
Essentially,
so
we
need
to
secure
the
kubernetes
api
server
for
sure,
and
then
we
will
look
into
more
granular
security.
This
is
where
the
API
aware
security
comes
in,
we'll
see
how
you
can
restrict
the
the
behavior
of
certain
services
when
they
are
accessing
other
services
from
at
an
APL
area
and
then
from
an
ingress
perspective.
A
A
So
Before
we
jump
into
the
terminal,
though
just
one
double
click
on
allow
within
namespace.
It
is
a
pretty
decent
I
think
what
we
have
learned
in
practice
from
other
customers
is.
This
is
a
good
good
starting
point.
It
protects
lot
of
this
sensitive
access
points,
API
server,
cubelet,
server,
metadata
services.
If
you
are
running
on
cloud
any
cross
tenant
breaches.
If
you
have
mapped
your
tenants
to
namespaces
and
unwanted
ingress
you
here,
so
it's
a
decent
starting
point
right
so
about
my
demo
right
around
10:45
a.m.
this
morning.
Everything
was
working.
A
Fine,
then
then
mini
2
blew
up.
My
backup
plan
was
coughs
cluster,
but
then
the
Wi-Fi
here,
just
any
queue
cuddle
commands,
were
just
not
going
through.
So
if
you
see
a
little
bit
of
paranoia
in
my
face
because
of
the
torturous
experience
for
the
past
one
and
a
half
hours
I
downgraded
to
mini
queue,
to
version
o
28
or
something,
and
then
it
worked
fine
but
30
seems
to
be
hanging
in
my
computer
at
least
so
in
this
mini
queue.
Cluster
I
have
a
monitoring
namespace.
A
Many
of
you
guys
would
have
the
same
monitoring
names,
except
probably
for
the
Empire
dog.
We
are
big
fans
of
Star
Wars.
They
were
our
early
customer
and
inspiration
so
that
Empire
dog
is
essentially
a
policy
dog
fooding
pod.
We
will
see
whether
it
can
access
things
and
things
like
that.
As
we
apply
the
policies
there,
other
namespace
aptly
named
Empire
and
it
has
just
one
service
called
Death.
Star
I
borrowed
it
from
Thomas's.
Talk,
didn't
want
to
create
any
new
service
in
an
empire.
A
A
A
Actually,
if
we
try
to
go
ahead
and
access,
the
cube
shuttle
looks
like
there
is
no
authentication
on
this
and
it
can
go
ahead
and
access
all
the
parts
that
are
running
there
now,
I'm,
not
a
very
good
hacker,
but
you
could
craft
some
really
nifty
commands
and
execute
and
get
access
to
running,
pods
and
copy
files,
and
things
like
that,
there's
just
no,
no,
no
purpose
of
random
part
having
cubelet
access
right
I
do
have
the
API
access
blocked.
So
if
you
look
at.
A
This
command,
which
is
actually
just
doing
the
curl
to
go,
get
that
one
I
do
have
authorization
there,
but
this
authorization
is
actually
also
very
weak.
It's
just
based
off
of
the
service
account
if
I
change
the
service
account
on
that
Empire
dog
to
cluster
admin,
I,
pretty
much
sure
it's
going
to
reveal
a
bunch
of
secrets
out
of
there.
Okay.
A
But
the
basic
point
is
this:
was
a
random
part
that
I
created
as
testing
it
just
has
a
bunch
of
access
without
any
particular
rhyme
yeah,
and
it
can
not
only
do
damage
by
accessing
this
sensitive
services.
It
can
also
go
across
namespaces
and
and
try
to
find
services
to
exploit
that
are
exposed.
So
this
is
our
favorite
inspired
from
Star
Wars
that
you
can
actually
go
blow
up
the
entire
Death
Star.
That's
what
actually
happened
in
the
movie.
They
had
a
sense
that
they
were
exhaust
port
open
for
HTTP,
put
access.
A
Okay.
So
how
do
we
rectify
this
situation?
I
mean
it
seems
like
a
pretty
it's
a
pretty
happy
situation.
From
from
the
application
perspective
you
can
see.
The
graph
anna
is
talking
to
prometheus.
Prometheus
is
pulling
the
helium
matrix
in
this
scale.
The
drop
count
and
end
pointers
is
so
silly.
Immed
expose
a
whole
bunch
of
Prometheus
matrix
from
an
app
perspective.
It's
a
pretty
happy
situation.
The
these
are.
These
things
are
node
port
services,
they
are
happily
accessible
everywhere
from
the
world
from
the
security
is
not
so
good.
Okay.
A
So,
let's
start
with
the
within
namespace,
as
I
had
mentioned
a
surprisingly
very
simple
step
can
go
along.
We
had
to
protect
the
service.
One
thing
to
highlight
here:
is
you
see
all
the
labels
are
empty,
but
every
CNP
is
every
network
policy
is
scoped
to
a
namespace.
So
when
you
push
this
policy,
which
says
all
endpoint
selectors
are
allowing
to
go
to
all
eagerness
and
immigrants
destinations,
it's
not
all
the
ports.
A
Talking
to
everybody
helium
will
insert
a
namespace
labor
because
it's
already
scoped
to
a
namespace,
so
all
parts
within
the
namespace
can
talk
to
each
other.
That's
what
this
policy
will
result
in.
You
don't
have
to
put
the
namespace
in
each
and
every
where,
where
you
want
to
apply
this
okay,
so
let's
apply
this
and
see
how
that
goes.
A
Well,
that
happens.
One
other
nifty
thing
to
launch
over
here
would
be
to
just
see
the
repercussions
of
these
policies,
whether
they
are
functioning
sort
of
a
TCP
dump.
If
you,
if
you
would
for
getting
insights
into
what
is
getting
dropped
and
things
like
that
yeah.
So
let
me
long
exact
into
the
cerium
pod
and
I
will
launch
a
tool
called
senior
monitor.
A
And
I
will
restrict
it
to
just
show
us
the
drops.
Okay,
it
looks
like
already
after
we
applied
this
within
namespace
policies.
A
whole
bunch
of
things
started
becoming
unhappy.
You'd
probably
get
calls
from
your
app
developer
that
hey
I
can't
access
my
graph
on
and
things
like
that.
The
calls
to
not
port
990,
even
the
cause
to
UDP,
are
getting
blocked.
Okay,
so
that's
the
first
thing
that
we
need
to
allow.
You
saw
that
there
was
no
egress
allowed
in
that
within
namespace
rule,
but
without
cube
DNS
X,
as
nothing
is
gonna
work.
A
So,
let's
rectify
that
situation
and
the
cube
DNS
to
allow
cube
DNS,
you
can
use
the
same
identity
and
label
based
system.
You
can
use
the
label
core
cube
DNS,
allow
specific
access
to
port
53.
So
this
is
a
good
example
of
a
combination
of
l3
and
l4.
It
can't
go
to
any
other
system
on
cubes
in
the
cube
system,
namespace
any
other
service
there.
So
it's
a
it's
a
good
limited
policy.
A
It
still
it's
very
liberal
in
that
it's
allowing
you
to
look
up
any
service
and
anything
that
is
running
internally
or
externally
and
towards
the
end
I
will
cover
how
we
can
restrict
even
the
lookups
to
specific
services
using
wildcards
and
things
like
that
and
enforce
that
us
there.
Okay.
So
let's
see
if
this
helps
the
cause,
let
me
stop
this
and
grep
for
port
53
right
there
for
UDP.
That
should
also
53.
A
A
A
Thank
you
all
right
thanks
for
that.
So
this
this
particular
vulnerability
had
nothing
to
do
with
the
DNS
with
the
DNS
looker,
it
is
directly
hitting
the
IP
right,
but
now
this
is
getting
blocked
because
there
is
no
ingress
and
egress
allowed
in
this
namespace.
So
it's
a
good
spot
to
start
with
and
we
can
test
the
API
is
and
and
the
Deathstar
explosion
as
well
as
it's
just
not
going
to
be
allowed.
I
will
just
show
you
the
test,
the
cross
namespace
one
so
that
we
make
sure
that
the
DNS
lookup
is
working
right.
A
See
the
it's
not
resulting
in
a
DNS
lookup
error,
it's
just
hanging
for
the
curl
to
timeout.
So
that's
the
example
of
a
policy
l3
denied,
okay,
there's
one
other
thing
is,
since
this
is
so
noisy.
There
is
one
other
option
in
see
Liam
which
will
help
reduce
it.
So
we
can
actually
look
at
what
all
endpoints
it
is
managing.
So
here
you
can
see
that
it
is
managing
cube,
DNS,
it's
managing
graph
fauna.
It's
managing
all
of
these
things,
and
this
is
like
the
our
Empire
dog
test
pod.
A
We
can
just
restrict
the
calls
to
this
particular
endpoint
and
say:
hey
just
show
me
all
the
drops
related
to
this.
Not
this
command
the
CDM
monitor
command
right.
So
that
way
it
will
become
less
noisy
and
if
I
showed
you
one
of
these
calls
here,
it
should
give
you
its
rizal
to
the
right.
Back-End
service
is
going
to
port
80,
but
it's
getting
policy
denied
right.
So
this
is
like
a
very
simple
starting
point
with
we
said:
okay
allow
within
namespace.
A
A
So
there's
the
graph
on
a
service
again
using
the
same
label
based
identifier,
abre
fauna,
you
allow
it
the
access
from
entities
called
world
now
the
notion
of
world
is
also
a
little
bit
challenging
for
see.
Liam
all
the
endpoints
outside
of
the
cluster
are
essentially
classified
as
world
or
for,
if
any
reason
it
can't
resolve
an
identity.
It
just
falls
back
as
a
world
as
well.
Okay,
but
there's
one
interesting
as
I
had
mentioned
earlier.
There
is
a
distinction
when
the
traffic
is
coming
from
node
port
versus
a
traffic
originating
from
the
host.
A
If
I
allow
all
from
the
world
yeah
all
of
those
traffic
will
go
through,
but
by
default
all
the
traffic
from
host
is
allowed.
So
how
do
we
distinguish
right?
How
do
you
differentiate
what
traffic
is
originating
from
the
host?
Like
a
health
check,
you
can
have
readiness,
probes
and
things
like
that
from
cubelet
going
into
the
pod.
So
what
we
do
is
when
a
traffic
is
originating
from
any
of
the
interfaces
inside
the
helium's
host.
A
We
put
a
small
marker
in
it
which
will
tell
us
that
hey,
it
is
a
traffic
originating
from
the
host,
using
that
we
can
distinguish
whether
it
is
coming
from
the
host
or
it
is
actually
bouncing
from
an
external
from
an
outside
node
port
kind
of
a
service
that
makes
sense.
The
Ravana
actually
has
a
HTTP
readiness
probe,
and
you
can
see
that
graph
hana
is
not
failing,
even
though
we
don't
have
any
ingress
allowed,
so
immigrants
from
host
is
always
allowed.
A
So
you,
even
though
you
have
specifications
or
a
service
definition,
you
can
still
enforce
controls.
So
this
this
prevents
like
accidentally.
If
somebody
puts
Kafka
as
a
node
port
instead
of
cluster
IP
and
all
of
a
sudden
exposed
to
the
world
which
it
has
no
business
being
accessible
from
outside
often
doesn't
have
any
are
back
or
TLS
or
anything
to
it
right.
So
you
can
have
this
safeguards
in
place,
which
will
prevent
accidental
Mis
configuration
from
exposing
a
whole
bunch
of
services
or
unnecessarily
to
your
internet.
All
those
fancy
graphs
went
away
by
now.
A
You
know
the
reason,
because
there
is
no
egress
allowed
for
Prometheus
to
escape
anything
right.
We
started
with
a
very
conservative
spot.
We
said
like
just
within
the
namespace,
but
Prometheus
needs
access
to
go
and
scrape
a
whole
bunch
of
other
endpoints
and
Prometheus
also
needs
access
to
the
kubernetes
api
server
to
go
scrape
those
endpoints.
Otherwise,
it
won't
know
what
is
the
what
HTTP
IP
address
it
should
go
to
so
first,
we
need
to
give
Prometheus
the
kubernetes
api
access.
A
So
let's
look
at
the
now.
Api
server
is
a
little
bit
of
a
tricky
situation
because
you
may
or
may
not
have
labels
for
the
for
the
backends
for
the
API
server.
It
may
be
deployed
as
a
part.
It
may
be
running
as
a
process
in
the
host,
but
what
we
have
here
for
those
kind
of
scenarios
where
the
back-end
can't
be
resolved
reliably
or
you
have
a
headless
service
kind
of
a
scenario
you
just
have
exposed
a
headless
service
inside
kubernetes.
There
is
a
construct
which
we
have
created.
A
It's
essentially
two
services
construct
this
part
here
and
you
can
specify
the
name
of
the
service
and
name
space
and
see
Liam
will
automatically
start
tracking.
What
are
the
backends
for
this
right?
So,
if
I
push
this
policy,
it
will
allow
Prometheus
to
have
kubernetes
api
access
that
empire
dog
guy
still
won't
have
any
access
to
the
api
server
yeah.
So
let's
do
that,
so
you
don't
have
to
worry
about
creating
a
cider
block
or
figuring
out
my
API
server.
A
A
A
Ok,
so
if
let's
say
Prometheus
pod
gets
compromised
and
you
don't
have
any
API
level
security,
you
have
just
given
Prometheus
access
to
all
of
your
endpoints,
potentially
even
port
80
endpoints
that
have
/
matrix
exposed
and
it
if
it
gets
compromised.
You
have
a
whole
bunch
of
post
and
deletes
and
random
actions
going
on
over
there
right,
just
piggybacking
off
of
a
really
allow
transaction.
So
for
that,
what
you
can
do
is
using
cilium.
You
can
specify
the
labels.
A
You
can
specify
the
port
that
hey
the
traffic,
can
go
to
any
port
1990
and
scrap
the
matrix
or
any
port
80,
but
you
can
restrict
it
to
just
doing
an
HTTP
GET.
Nothing
else
you
can't
do
put
I
can't
do
post
delete,
none
of
that
and
this
actually
leverages
envoy
so
for
in
order
for
to
parse
the
traffic
and
filter,
see
Liam
will
see.
Liam
has
an
envoy
built-in,
it
will
route
the
traffic
using
socket
redirections
and
such
and
efficient
key
parse
it
out
and
block
the
traffic
yeah.
A
A
A
A
A
That's
why
I
created
that
Empire
dog
howl
is
kind
of
hard
to
know
what
is
getting
dropped
so
now
we
have
our
application
functioning
back.
This
is
much
more
secure
than
the
previous
is
what
we
started
about
ten
minutes
ago.
Everything
is
allowed
to
talk
to
everything
here,
and
monitoring
is
a
very
good
application
to
relate
to,
but
you
have
many
different
services
which
are
probably
don't
have
any
policies
in
place.
A
This
way,
you
can
iterate
one
step
at
a
time
and
you
can
just
allow
the
services
that
you
want
the
key
question
that
arise
out
well,
ow
and
that's
at
least
you
see
the
the
direction
over
there
that
see.
Liam
exposes
a
socket
a
set
of
logs
and
and
communication
that
is
going
on,
that
information
can
be
used
to
build,
who
should
be
allowed,
who
is
getting
dropped
and
things
like
that
right.
A
One
other
policy
that
I
wanted
as
I
mentioned
earlier
was:
these
are
all
the
policies
that
we
applied
already
now
the
question
of
external
egress
access.
So
let's
say
our
our
customer,
the
Star
Wars
Empire
foundation
team.
They
want
to
backup
the
matrix
right,
so
they
want
to
put
an
agent
inside
from
the
the
Prometheus
and
they
want
to
take
a
copy
and
put
it
in
s3
or
something
now.
This
kind
of
a
backup
service
is
very
common,
but
again
this
backup
service
should
only
be
able
to
go
to
s3.
A
They
should
not
be
looking
up
at
crypto
mining
comm
or
something
like
that
right.
So,
let's
see
how
we
can
restrict
the
and
this
one
I'm
going
to
make
it
a
little
bit
bigger.
So
again,
the
test,
our
dogfooding
app
over
here
and
the
same
label
based
identification
for
cube
dns
same
port
53
axis,
but
we
have
added
one
more
condition
here,
which
says
that
you
can
only
do
dns
lookup
for
s3
and
amazon
AWS
like
star
dot,
s3
dot
any
region
inside
amazon
AWS.
You
can't
even
look
up
anything
else.
A
You
are
a
backup
service.
You
have
no
business
looking
up
anything
else.
This
is
really
tightening
the
knot
to
the
net,
like
it's
just,
truly
least
privileged
for
any
of
the
parts
and
services
that
you
can't
do
anything
more
than
what
you
are
required
to
do.
The
second
part
of
it
is
okay,
fine,
you
have
done
the
DNS
lookup.
Where
can
you
make
the
tcp/ip
connections
to?
A
And
that's
this
part
which
says
like
well,
those
are
the
domains
you
can
look
up
and
those
are
the
domains
that
you
can
connect
to
so
same
pattern
based
fqdn
rules
which
will
control
egress,
and
you
can
combine
them
with
the
four
four
three
now
one
of
the
challenges
with
the
external
service
ex
is
going
over
TLS
is
there
is
no
good
way
to
do.
Tls
visibility
unless
you
two
want
to
go,
try
out
trusted
man-in-the-middle
models,
and
that's
some
of
the
things
that
we
are
working
with
around
ke
TLS.
A
Once
that
is
there,
you
can
even
add
sections.
What
exactly
can
you
do?
This
service
should
just
be
able
to
put
and
that
state
it
should
not
be
able
to
get
all
the
buckets
and
things
like
that
because
you
may
have
other
buckets
there,
so
you
can
see
the
direction
of
where
this
is
going
right.
Now
it
controls
access
to
external
service
on
a
port
end
and
a
DNS
level.
A
Once
we
have
the
ke
TLS
and
those
things
sorted
out
in
the
kernel
and
an
enforcement,
then
we
can
you'll
see
the
HTTP
Clause
over
here
as
well.
Then
it
will
again
very
tight
security
around
this.
So,
let's
see
if
this
pattern
based
policy
works,
this
is
very
new,
so
I
pulled
in
the
nightly
build
from
couple
of
days
ago.
So,
let's
see
if
it
cooperates
right
so
and
I'm
going
to
go,
use
the
to
access
some
of
our.
A
Public
assets
that
we
have
now,
it
should
be
allowed
to
go
to
this
star
dot,
s3
US
west,
to
region
right
because
that
matches
the
pattern
it's
going
to
a
particular
bucket
in
a
particular
region
and
I
I,
don't
have
any
authentic
oaken's
enabled
so
the
l3
communication
was
allowed.
It
failed
at
an
access
denied,
l7
level,
okay,
so
the
the
pattern
based
enforcement
and
things
like
that.
Well
as
I
say,
if
it
can
go
to
google.com
or
Twilio
dot-com,
or
something
like
that,
this
should
not
even
allow
it
to
do
a
DNS
lookup.
A
We
should
get
a
resolved
host
error
or
something
like
that
right,
because
we
didn't
even
allow
it
to
do
DNS
lookups.
This
prevents
against
IP
spoofing
and
DNS
poisoning
in
those
kind
of
attacks.
Since
we
are
using
so
much
of
DNS
now
it's
only
a
matter
of
time
that
they
rear
their
head.
So
just
a
quick
recap:
this
is
all
the
policies
that
we
enforced.
A
We
started
with
allowed
within
namespace,
then
we
talked
about
pattern
based
DNS,
API,
aware,
and
why
that
is
needed.
We
used
special
construct
like
to
services
to
secure
the
kubernetes
api
access.
We
used
a
special
construct
reserved
world
to
control,
ingress,
egress
and
then
I
described
how
we
distinguish
between
the
host
in
the
world
traffic
yeah.
There's
a
lot
more
API
services
that
are
very
common,
so
we
have
added
support
for
them.
More
importantly,
we
have
created
a
extension
framework
for
on
voice,
so
there
is
a
golang
based
extension
framework.
A
You
can
write
your
own
custom
protocol
or,
if
you
want
to
contribute
to
one
of
the
popular
protocols,
that's
also
pretty
easy.
You
can
use,
go,
go
based
on
voyage
tension
to
do
that.
Yeah.
The
I've
added
this
here.
If
somebody
is
browsing
the
slides
later
on.
This
is
the
same
policies
that
I
showed
over
there,
and
this
is
some
of
the
newer
stuff
that
is
coming
in
now.
Securing
one
of
the
clusters
is
one
thing,
but
pretty
much.
A
Every
customer
that
we
talked
to
now
has
many
clusters
for
one
reason
or
the
other,
and
there
is
often
a
need
to
share
the
access
across
multiple
clusters
is
use
the
same
service
identity
across
different
clusters.
So
what
we
have
done
is
created
a
notion
of
cluster
mesh.
It's
a
very
easy
way
to
set
up
connectivity
across
multiple
clusters,
and
the
security
identities
also
are
across
all
the
clusters.
So
if
you
have
that
Deathstar
service
running
in
one
cluster
and
also
in
another
cluster,
it
will
get
the
same
security
identity,
and
this
is
available.
A
I
think
it's
already
posted
as
a
blog
in
our
one
dot
preview.
More
of
the
details
are
available
in
Thomas's
talk
on
how
we
actually
do
it
by
reading
the
HDD
across
each
each
of
the
cluster
and
understanding
the
identities.
But
on
your
part,
the
effort
is
really
around.
How
do
you
secure
the
connectivity
across
these
clusters
at
a
lower
sort
of
l2
fabric
of?
A
Do
you
need
to
create
IPSec
or
VPN
gateways,
and
things
like
that,
once
you
have
established
that
connectivity,
we
can
take
care
of
connecting
all
the
community
services
and
securing
them,
and
this
is
personally
my
super
favorite.
This
is
I
am
very
excited
about
the
opportunity
that
KT
LS
will
unlock
a
lot
of
the
sophisticated
compromise
that
we
see
now
are
actually
piggybacking
off
of
an
encrypted
channel
with
authorize
tokens
and
everything
and
the
fundamental
problem.
There
is
just
visibility.
A
You
don't
even
know
who
this
compromise
is,
what
they're
doing
just
they're
going
in
the
encrypted
channel.
So
K
TLS
is
a
really
nifty
concept.
The
asymmetric
part
of
the
exchange
happens
just
like
it.
It
happens
always
right
the
client
and
the
server
they
will
do
the
asymmetric
key
part.
The
once
the
handshake
over
there
is
done.
The
symmetric
key
is
offloaded
to
the
kernel.
So
then
the
kernel
will
do
the
encryption
for
you.
You
don't
have
to
worry
about
doing
that
encryption
in
the
app.
So
what
is
the
benefit
of
this?
A
Between
the
user
space
and
the
kernel?
The
packet
is
unencrypted
and
that's
where
you
can
attach
a
BPA
program
and
inspect
the
headers
and
things
like
that
or
redirect
it
to
a
proxy.
So
you
can
get
visibility
and
enforcement
in
a
TLS
communication
before
the
symmetric
encryption
happens
on
the
data
payload
right
and
the
biggest
benefit
of
this
is
no
route,
see
a
propagation.
None
of
that
multi
chain
certificate,
headaches
of
where
this,
where
all
do
I
need
to
propagate
the
root
CA
yeah.
So
this
is
personally
my
favorite
new
feature.
A
It's
going
to
be
a
little
bit
of
a
grind
to
get
it
out,
but
it's
definitely
very
powerful,
but
in
in
a
naturally
you
see
that
this
is
why
I
started
with
BPF
like
the
program's.
It
allows
you
flexibility
to
attach
this
programs
at
various
levels,
whether
for
TLS
visibility,
packet,
filtering
firewall
and
things
like
that
and
see.
Liam
is
just
making
it
easy
for
you
to
tap
into
this
power
and
and
run
it
in
the
communities
in
your
cluster.