►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
The Enemy Within: Running Untrusted Code with gVisor - Ian Lewis, Google
Containers are a great way to isolate application resources but they can fall short when it comes to security isolation. How do you improve the security of your workloads without giving up the properties of containers that you've come to love? There are many approaches to sandboxing containers, such as virtual machines and unikernels, but which is right for you? gVisor is a unique open-source sandbox runtime that allows you to run unmodified applications in containers with a higher level of isolation and low overhead. In this talk I will explore the container security model of gVisor and use cases for sandboxing containers. I will discuss various approaches and their tradeoffs before diving into the architecture of gVisor and how it differs from virtual machine based sandboxes. Finally, I will bring it all together with a demo of a minimal serverless platform using gVisor and Kubernetes.
https://sched.co/Nrou
A
So
when
thinking
about
kind
of
sandboxing,
like
the
like,
normal
containers,
are
kind
of
a
great
way
to
run
applications
but
aren't
always
a
great
way
to
kind
of
sandbox
them
to
run
untrusted
code
so
like
in
a
lot
of
cases
like
you
might
have
code.
That
is
that
you
want
to
run,
but
you
don't
want
to
actually
expose
that
your
infrastructure
to
that
code,
so
that
might
be
something
like
a
running
code
that
users
upload
to
your
service.
A
So
something
like
a
software
is
a
service
or
some
sort
of
like
serverless
situation,
a
running
third-party
code.
So,
like
you,
have
a
vendor
that
provides
you
with
a
binary,
and
you
just
need
to
run
that
in
your
cluster.
But,
like
you
don't
know,
what's
in
that
binary,
you
don't
know
like
you
know,
if
it's
safe
and
don't
know
like
what
sort
of
vulnerabilities
might
be
in
it.
I
know
there
is,
like
you
know,
running
kind
of
code
on
complex
user
inputs
of
things
that
you
know
these.
These
are
all
in
this
case.
A
A
So,
as
I
said,
some
of
the
use
cases
for
like
actual
like
kind
of
business
use
cases
might
be
something
like
SAS
or
cyrillus.
You
know
video
image,
transcoding
machine
learning,
there's
some
pretty
typical
use
cases,
but
you
know,
as
I
mentioned
like
he
could
be
just
running
third-party
vendor
code
or
that
sort
of
thing.
Oh.
A
Like
kind
of
to
break
down
like
what
like,
why
we
need
to
sandbox
and
like
what,
like
the
attack
surface,
looks
like
when
you're
running
something
in
a
container
like
so
here's
our
application
and
this
application,
like
is
not
trusted.
This
is
just
an
application
that
could
run
any
type
of
arbitrary
code,
so
it
can
and
and
may
run
code
that
we
don't
expect
it
to
run
or
that
it
may
you
know
it
could
be
completely
malicious
and
so
what
applications
typically
do
kind
of
as
they're
running
is
they?
A
A
They'll
typically
make
a
sis
call
to
the
Linux
kernel,
and
so
as
the
applications
running
on
the
CPU,
like
part
of
that,
it
hits
this
like
call
to
the
Linux
kernel,
which
then
does
a
context
switch
into
the
Linux
kernel.
You
know
right
some
some
code
or
some
data
to
memory
and
then
just
the
context
switch
to
the
kernel,
which
then
reads
that
memory
and
executes
some
sort
of
system
call,
and
so
in
this
case
we're
like
opening
a
file
right,
but
then,
like
typically
as
part
of
that
after
that
context
switch.
A
That
is
with
all
of
the
privileges
that
the
kernel
itself
has,
and
so
this
is
something
that
we
don't
really
want
to
do.
You
know
so
like
as
part
of
the
normal
running
of
the
application,
it's
actually
calling
into
the
Linux
kernel
and
running
some
some
code
as
the
Linux
kernel,
and
so
this
is
typically
the
code
that
is
kind
of
suspect
in
terms
of
like
being
privileged
and
that
you
know,
if
there's
a
bug
in
it
could
be
potentially
problematic
in
terms
of
like
the
security
of
the
overall
infrastructure.
A
A
The
goal
of
like
sandbox
isolation
is
typically
to
you
know,
reduce
the
attack,
the
attack,
surface
and
bite.
You
know
the
part
of
doing
that
is
like
reducing
the
the
execution
of
trusted
like
privileged
code
in
the
in
the
kernel
itself,
so
this
is
typically
done
through,
like
some
sort
of
virtualization
or
abstract
of
these
hosts
of
the
host
itself.
So
you
might
do
something,
like
you
know,
have
like
a
virtual
machine
that
abstracts
the
who
are
abstracts
away.
A
So,
like
one
of
the,
the
idea
here
is
that
we
don't
want
to
expose
the
the
infrastructure
to
any
one
single
bug,
and
this
is
typically
you
don't
want
to
have
like
a
single
bug,
be
how
this
thing
would
be
a
single
bug
away
from
having
be
able
to
get
it
things
like
user
data
or
something
that's
that's.
You
know
important
to
you
and
the
reason
for
this
is
just
that.
You
know,
if
you
have
it,
you
may
have
maybe
having
a
single
bug
is
not
all
of
that.
A
All
that
come
in,
but
it's
you
know
almost
virtually
unheard
of
to
have
like
two
bugs
that
that
work
at
the
exact
same
time,
in
order
to
you,
know
that
the
attacker
has
it
has
access
to
in
order
to
get
like
out
of
a
particular,
you
know
a
sandbox
or
a
runtime
environment.
This
just
like
makes
it.
You
know
by
having
by
requiring
that
they
have
like
two
layers
there,
or
they
get
past
two
layers
in
order
to
get
to
user
data
or
to
something
on
the
host.
A
The
sandbox,
like
we
there's
you,
know
a
lot
of
different
alternatives
to
building
sand
boxes.
So
one
of
those
is
you
know,
obviously
like
running
container.
Just
running
containers
normally
and
running
the
code
inside
of
a
container
is
an
option,
but
you
know,
as
we
said,
with
the
being
one
like
kind
of
kernel,
bug
away
from
people
breaking
out.
A
A
Another
idea
is
like
using
you
know,
kernels.
You
know
kernels
are
a
way
of
kind
of
creating
Sam
boxes,
and
typically
they
do
that
with
like
by
kind
of
linking
in
or
building
an
application
with
a
guest
OS
as
a
built
as
a
library
and
then
running
that
in
a
hypervisor
arena
in
a
locked-down
kernel
or
in
a
locked-down
container.
A
Well,
like
you
know,
containers
and
you
know,
kernels
are
interesting
but
like
they
have
a
lot
of
kind
of
downsides
in
the
sense
that
containers.
Obviously
like
aren't
good
for
security
isolation.
They
only
have
one
layer
of
isolation
and
it's
it's
hard
to
reduce
the
attack
surface
by
just
using
a
container
or
the
the
features
that
you're
allowed,
that
you're
they're
available
to
containers.
So
things
like
set
comp
or
app
app
armor.
A
But
Yuna
curls
our
way
of
helping
with
some
of
that,
but
they
can't
you
don't
you're
not
allowed
to
kind
of
like
bring
your
own
container
like
you
kind
of
have
to
build
a
specially
crafted
kind
of
container
that
contains
the
the
operating
system.
So
you
have
to
bring
a
lot
of
that
operating
system-level
like
support
with
you
in
as
part
of
the
container.
A
A
Virtual
machines
are
probably
the
the
most
well
understood
and
most
used
way
of
kind
of
sandboxing
things
or
sandboxing
applications.
Virtual
machines
give
you
kind
of
you
know
a
guest
OS.
Typically,
you
bring
the
whole
like
kind
of
VM
image,
so
a
guest
OS
and
the
application
together.
But
then
you
have
like
the
and
isolation
layer
at
the
hypervisor
label
layer
which
gives
you
like
a
pretty
strong
boundary
between
you
and
the
hosts.
A
You
know
typically,
like
you
know,
depending
on
whether
you're,
using
something
like
type,
1
or
type
2
virtual
machine.
You
know
something
that
has
hardware
support
or
something
that
or
a
hardware
yeah.
Basically,
a
hardware
support
for
the
hypervisor,
like
we're,
not
kind
of
depends
on
whether
how
you
have
like
you
know,
kind
of
one
or
two
layers
of
security,
but
the
typically.
This
gives
you
a
reasonably
good
level
of
security.
A
You
know
you
have
a
fairly
like
with
say,
like
type
two,
you
have
a
hypervisor,
that's
built
into
the
kernel
and
the
the
VM
is
run
as
a
as
a
as
a
regular
process
on
the
machine,
and
this
gives
you
a
pretty
decent.
Like
kind
of
single
layer
of
boundary,
but
you're
still
kind
of
like
one
bug
say
in
the
hypervisor
away
from
you
know,
potentially
having
a
full
host
compromised
with
type
ones
you
kind
of
get.
A
You
know
you
could
be
in
the
situation
where
you'd
have
to
get
like
kind
of
a
hypervisor
bug
plus
like
have
some
a
hardware
issue
in
order
to
break
out
like
if
you
had
like,
like
memory,
fencing
or
whatever
like
this
is
maybe
a
little
bit
better,
or
this
is
a
little
bit
better,
perhaps
like,
but
virtual
machines
have
a
lot
of
other
kind
of
downsides
assets
that,
like
they're,
it's
not
easy
to
have
like
flexible
resource
usage
with
a
with
a
VM
I.
So
you
don't
get
to
kind
of.
A
A
You
can
do
memory
ballooning
but,
like
typically
like
you
have
to
have
kind
of
cooperation
with
the
OS
inside
the
sandbox.
That
may
or
may
not
be
trusted
or
may
not
be
trusted,
and
so
you
don't
really
necessarily
want
to
have
to
do
that.
You
also
want
to
have
a
quick
startup
time
so,
depending
on
the
environment
like
if
you're
using
kind
of
one
of
the
micro
VM
things,
you
can
get
better
startup
time.
A
Another
thing
that
we
wanted
to
do
as
part
of
G
Weiser
was
to
like
have
something
that
would
integrate
well
with
containers
like
solutions,
so
something
that
was
very
container
like
but
would
actually
give
us
a
sandbox.
So
this
is
something
that
you
know
would
easily
integrate
in
something
like
kubernetes
and
give
people
a
very
container
like
experience
in
terms
of
like
how
they
assign
and
and
manage
the
resources
of
the
containers.
But.
A
Would
also
provide
kind
of
a
sandbox,
so
you
wouldn't
have
like
this,
like
totally
different
paradigm
for
your
sandboxes
versus.
Not
we
thought
about
like
a
lot
of
this
like
how.
How
would
we
kind
of
build
a
system
that
would
give
us
all
of
the
things
that
we
wanted?
So
you
know
with
virtual
machines.
You
have
like
kind
of
a
hardware
virtualization
layer
but
and,
as
I
said
like,
if
you
weenie
in
order
to
like
actually
do
a
to
build
us
a
sandbox.
Typically,
you
would
virtualize
the
the
system
layers
right.
A
So
here,
like
you're,
actually
virtualizing
the
hardware
and
providing
an
interface
at
that
layer
and
everything
above
that
layer
is,
is
untrusted,
but
with
with
G
visor
the
way
that
we
kind
of
figured
that
we
would
do
it
is
by
virtualizing
the
the
virtualizing
at
the
OS
level.
So
in
this
case
we're
actually
virtualizing
something
virtualizing
the
OS,
and
then
you
know,
basically
only
the
application.
Well,
like
I'll
talk
a
little
bit
about
that.
But,
like
you
know,
people
can
basically
bring
their
untrusted
application
and
we
can
run
that
in
the
sandbox.
A
So
the
virtualize
the
virtualized
iOS
is
like
is
trusted
but
like
at
least
up
until
the
point
that
the
application
is
actually
run.
I'll
talk
about
that
in
a
little
bit,
so
this
like
for
us
like
gives
us
the
two
layers
of
isolation
that
we
want,
because,
as
when
we
actually
boot
up
the
container,
we
have
control
of
the
OS.
A
A
Sis
calls
and
then
handling
those
as
part
of
the
user
space
kernel
and
then
that
that
entire,
that
entire
kind
of
package
or
sandbox
runs
inside
of
the
inside
of
user
space
and
then
needs
only
to
make
limited
limited
amount
of
sis
calls
in
order
to
implement
the
sandbox
to
the
actual
host.
And
so,
by
doing
that,
we
reduce
the
kind
of
the
amount
of
sis
calls,
and
you
know
specifically
the
type
of
we
get
to
control
a
little
bit
more.
The
actual
arguments-
and
things
like
that
that
are
sent
to
the
sis
calls.
A
Gives
us
the
choose
layers
of
isolation,
use
this
kind
of
the
same.
You
know
principle
of
virtualization
as
things
like
VMs,
but
it
is
doing
it
at
kind
of
a
different
level.
It's
doing
it
like
an
OS
level
rather
than
at
a
hardware
level,
and
this
reduces
the
hosts
attacks
or
risk
by
you
know.
Reducing
the
amount
of
sis
calls
that
are
handled
by
the
actual
host
kernel,
and
you
know
handling
like
these.
A
So
I'm
going
to
talk
a
little
bit
more
about
like
the
actual
architecture
of
the
of
the
sandbox
and
like
how
it
kind
of
starts
up
and
how
that
actually
makes
a
container
secure.
So
the
way
that
we
integrate
with
kübra
nice
is
we
use
container
D
and
container
D
provides
a
API
that
allows
you
to
run
something
called
a
shim
which
will
then
run
in
actual
container
and
container
D
implements
the
CRI
interface
that
kubernetes
needs
in
order
to
run
containers.
A
So
it's
very
similar
to
the
way
that
containers
run
like
we're
like
normal
containers
run
container
D
will
execute
the
container
D
shim
for
for
run
C
in
order
to
run
regular
containers,
and
this
runs
in
a
very
similar
way
to
that.
But
we
can
provide
what's
called
a
run
tank
handler
through
kubernetes
on
through
container
D,
in
order
to
tell
it
which
runtimes
to
actually
use.
So
in
this
case,
we
really
like
we're.
A
Gonna
use
run
SC
and
we're
going
to
run
a
container,
and
so
the
first
thing
that
the
duh
that
the
container
does
or
that
that
G
visor
will
do
is
it
will
set
up
our
kind
of
sandbox
environments,
and
we
call
this
part
the
sandbox.
This
part
that's
like
in
the
dashed
lines
here
and
that's
where
the
application
itself
is
going
to
run.
But
at
this
point
we
we
still
have.
We
have
kind
of
a
trusted
sandbox
in
the
sense
that
we're
not
running
any
untrusted
code.
A
Yet
we
have
a
century
that
we
know
is
like
is
the
actual
century
that
we
or
user
space
kernel
that
we
expect
it
to
be.
And
then
we
have
this.
We
set
up
something
called
a
gopher
which
is
going
to
do
a
lot
of
our
file.
I/O
I'll
talk
a
little
bit
about
that
layer,
and
then
we
are
going
to
use
either
p
trace
or
k
vm.
What
we
call
platforms
in
order
to
intercept
thesis,
call
and
then
once
we've
done,
that
we
like
put
both.
A
So
we
run
these
in
like
a
fairly
restricted
actual
container,
and
then
we
actually
run
the
application
itself,
and
so
the
application
itself,
once
it
starts
running,
can
start
making
sis
calls
to
the
century,
and
at
this
point
we
reconsider
the
entire
sandbox
on
trusted.
So
we
can
think
of
the
sandbox
itself
as
being
potentially
compromised
at
this
point,
because
we're
running
untrusted
code.
A
So
the
idea
here
is
to
handle
most
of
the
the
assist
calls
like
entirely
within
the
sandbox
and
in
the
cases
where
we
actually
do
need
to
use
host
resources.
We
can.
The
century
does.
Does
some
level
of
filtering
and
making
sure
that
the
the
SIS
calls
that
get
made
to
the
kernel
itself
are
not
malicious,
are
going
to
be
problematic
in
any
way,
and
then
we
can
do
things
like
you
know.
A
A
There's,
there's
different,
like
there's
slightly
specific
namespace
support
within
within
G
advisor,
but
essentially
their
names
based
kind
of
in
a
similar
way
to
what
you
would
expect
from
from
a
pod
and
then
so
you
get
a
very
similar
experience
to
the
way
that
containers
run.
You
can
assign
them
all
the
same
IP
address
you
can
they
can
talk
to
each
other
on
localhost
that
sort
of
thing
you
get
a
very
kind
of
similar
experience
to
a
normal
container.
A
Though
talking
about
some
of
the
design
principles
behind
G
Weiser,
one
is
that
we
have
these
two
security
layers
and
then
so
part
of
that
is
just
the
fact
that
we
have
the
Sentry
and
then
we
have
a
set
comp
sandbox.
So
to
give
us
two
layers
of
isolation
there
between
Linux
the
Linux
kernel
syscalls,
but
also
with
with
things
like
file
operations.
We
go
through
like
a
separate
application
outside
which
is
called
a
gopher
in
order
to
you
know,
separate
things
like
being
able
to
open
files
so
like.
A
If
you
you
want
to
open,
say
like
an
arbitrary
file,
you
would
need
to
break
through
the
century
and
then
find
another
bug
within
the
gopher
in
order
to
actually
start
opening
and
writing
to
files
that
you
wouldn't
normally
be
able
to
write
to.
So
that
provides
are
kind
of
two
layers
of
isolation
for
say,
like
filesystem,
writing
and
then.
A
You
know
some
other
design
principles
so
like
that
we
have
like
we
provide
minimal
access
to
the
host,
so
like
gnosis
calls
are
actually
passed
through.
So
it's
not
like
a
SATCOM
sandbox.
It's
actually
like
the
this
is
calls
that
are
made
to
the
host
or
actually
done
as
part
of
in
the
implementation
of
the
sandbox,
and
then
we
limit
the
number
of
Cisco's
allowed.
A
We
also
use.
We
also
implement
the
sandbox
or
the
we
implement
a
sentry
and
other
parts
of,
do
you
visor
and
go
and
the
reason
we
do
this
is
like
so
that
we
don't
have
like.
You
know
a
lot
of
the
the
problems
that
you
would
have
if
you
write
applications
and
see
or
kernels
and
see
in
terms
of
like
buffer
overruns
and,
like
you
know,
there's
you
know,
essentially
our
you
know,
array
length,
checking
and
things
like
that
built
into
the
language.
A
And
then
we
use
like
a
lot
of
the
you
know
the
built
in
like
host
features
like
see
groups
and
namespaces
and
chroot
or
pivot
route,
in
our
case
and
running
the
actual
sandbox
as
the
the
UI
UI
DNG
idea.
Nobody
so
that
we
don't
have
like
the
the
user,
doesn't
have
escalated,
privileges
or
privileges
that
it
doesn't
need
and
we
drop,
like
you
know,
pretty
much
all
the
capabilities
for
the
sandbox,
because
they
can
most
of
those
can
be
implemented
by
the
sandbox
itself.
A
A
We
encapsulate
that
in
namespaces,
so
this
is
very
similar
to
like
yeah.
The
sandbox
itself
is,
is,
you
know
very
similar
to
like
an
actual
container,
but
it's
a
very
restricted
container
in
the
sense
that
you
know
we,
we
are
fixed
to
running
as
a
UID
and
GID
of
nobody,
and
we
have
a
fairly
rich
frickin
SATCOM
profile
and.
A
A
Nc
so
like
the
sandbox
itself,
contains
like
almost
no
file
system
at
all
and
then
the
we
basically
drop
all
the
capabilities
and
set
the
user
ID
and
GID
to
nobody,
and
then
we
put
in
a
restricted
set
count
profile
that
only
allows
the
running
of
about
thirty-year
services,
calls
30
or
40
I.
Think,
and
then
we
have
four
file
operations,
because
the
sandbox
can't
really
do
any
file
operations
like
opening
a
file.
We
have
to
do
that.
We
do
that
through
a
separate
process
called
the
gopher.
A
The
gopher
has
a
similar
kind
of
experience,
except
for
it
actually
has
access
to
a
file.
The
file
system
that
the
d
container
that's
used
by
the
container.
So
this
is
also
running
a
C
group
and
has
reduced
set
of
namespaces,
is
too
routed
to
the
actual
or
pivot
root
into
the
actual
file
system
and
the
file
systems.
That
are
needed
for
the
container
are
mounted
in,
and
then
this
also
has
reduced
capabilities
and
has
like
a
fairly
restricted
set
comp
profile.
A
So
these
are
things
that
you
kind
of
have
to
keep
in
mind,
and
you
know
in
in
effect,
in
the
case
of
like
raw
sockets
were
actually
like
as
part
of
gee
visor,
not
really
allowing
you
to
have
access
to
raw
sockets,
but,
like
the
idea
is
that
you
know
the
who
vernie's
gives
you
a
lot
of
defaults
that
are
not
necessarily
meant
to
be
secure,
and
so
you
have
to
help.
Keep
that
in
mind.
When
you
actually
run
a
pod,
you
need
to
keep
things
like
you
know.
A
Memory
and
disk
limits
for
for
the
actual
application
run
see
will
set
up
that
set
that
up
for
a
sandbox
if
you
provide
it
to
OCI,
but
you
need
to
make
sure
that
those
are
there
in
case
the
sandbox
decides
to
try
to
use
up
all
the
memory
on
the
host,
for
instance,
other
things
like
Network
and
disk
isolation.
Like
you
know,
network
access
is
something
that
you
need
to
be
aware
of,
because
for
pods
yours
are
giving
an
IP
address
and
they're
on
the
pod
network.
So
you
need
to
be
able
to
use
things.
A
You
need
to
use
things
like
network
policy
or
some
other
net-like
mechanism
to
limit
what
the
the
sandbox
can
do.
A
network
things
like
arbitrary
packet
injection,
like
I,
said
like
raw
sockets.
This
essentially
provides
isolation
here
in
the
sense
that
it
runs
inside
of
this
sandbox,
but
the
sandbox
itself
has
access
to
right
right,
arbitrary
packets,
and
so,
if
somebody
is
able
to
take
control
of
the
sandbox
or
the
of
the
century
itself,
they
can
write
arbitrary
packets
on
the
network,
and
so
you
need
to
be
kind
of
aware
of
that.
A
Another
thing
is
file
rights
and
permissions.
Like
you
know,
if
people
can
like
write
to
the
root
file
system
on
your
container,
for
instance,
they
could
potentially
overwrite
files
or
executables
that
get
executed,
and
so
you
can
have
some
vulnerabilities
in
that
sense.
So,
like
you
know,
it's
ideal
to
use
things
like
read-only
file
systems
for
the
the
base
root
file
system
and
things
that
you
don't
expect
to
change
within
the
container.
A
I'm
gonna
try
to
wrap
up
here,
but
you
know
gee
visor
itself
is
you
know
something
that
is
very
container
like
and
we
we
are
kind
of
trying
to
make
it
as
close
to
the
kind
of
container
experience
as
you
as
possible
and
integrate
that
really
well
with
kubernetes.
So
currently
we
have
the
the
ability
to
use
runtime
class
in
kubernetes
to
specify
gee
visor
as
a
runtime
class
and
then
have
that
go
to
container
d
and
run
containers
as
part
SG
in
G
visor.
A
We
have
things
like
a
mini
cube
add-on
that
allows
you
to
time,
try
out
G
visor
and
play
with
it
a
little
bit
easier
than
having
to
set
up
your
whole
kubernetes
cluster
yourself
so
give
that
a
try.
We
also
have
things
like
you
know.
On
on
Google
cloud,
we
have
GK's
sandbox,
which
allows
you
to
use
TV
visors,
get
G's
visors
report
kind
of
out
of
the
box
there
and
as
part
of
all
this
or
in
order
to
support
all
this
we're
building
this
the
we're
working
on
the
G
visor
container,
D
sham.
A
A
A
So
that's
really
what
I
wanted
to
show
you
there
so
anyway,
I
think
I'll
finish
up
there
since
I'm
kind
of
at
a
time.
But
if
you
have
any
questions,
I'll
be
around
here
and
I
have
some
stickers.
If
you
want
stickers
so
yeah,
and
we
also
have
Kevin
from
the
development
team
here
to
help
answer
questions,
if
you
guys
have
any
so
thanks
a
lot.