►
From YouTube: Container Security & Multi-Tenancy Tales from Kata & Nabla - Ricardo Aravena & James Bottomley, IBM
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Container Security and Multi-Tenancy Tales from Kata and Nabla - Ricardo Aravena & James Bottomley, IBM
With the introduction of the Kubernetes CRI many different choices have emerged for users to run their various containerized workloads. There have been concerns about the complexity of making traditional containers more secure using Linux kernel facilities such as AppArmor, SELinux, and seccomp. In this talk, Ricardo will showcase how Kata and Nabla Containers can be used to isolate your tasks effortlessly. He'll describe the unique capabilities of each containerized approach along with their pros and cons, and how both of their communities are collaborating. He will also demonstrate how to make use of the Kubernetes RuntimeClass with both of these runtimes. By the end of this talk, the audience will be able to understand how to use Kata and Nabla Containers with Kubernetes and its new RuntimeClass to fully enable multi-tenancy with minimal risks in their infrastructure.
To learn more: https://sched.co/GrZH
A
Thank
you
for
coming
to
this
session.
Thank
you
for
sticking
around
for
the
last
session.
I
know
it's.
You
know
kind
of
late,
so
we're
excited
to
be
here
cube
con
and
to
tell
you
about
multi-tenancy
I
about
nabla
and
cata
containers
a
little
bit
about
ourselves.
So
I'm
Rico
and
I
work
branch,
metrics
doing
cloud
operations
and
my
cotta
containers,
contributor
and
I'm.
A
Great,
so
briefly,
what
we
do
a
branch
we
were
in
containers
everywhere
we
run
kubernetes
in
all
kinds
of
environments,
QA
development
and
production,
server
in
API
services
and
all
kinds
of
different
languages.
We
also
run
messes
some
of
our
data
pipelines,
run
on
top
of
messes
sparklink
and
druid
type
of
applications.
Our
main
runtime
is
doctor.
A
So
traditional
containers,
are
you
probably
seen
this
slide
a
few
times,
so
you
have
a
regular
kernel
on
top
of
your
hardware
and
you
may
have
a
virtual
machine
or
you
may
not
have
a
virtual
machine,
but
the
thing
about
traditional
containers
is
that
they
share
that
kernel.
So
if
you
run
a
container
and
it's
compromised
and
through
a
system
call,
then
your
other
containers
might
be
compromised.
So
your
host
kernel
gets
compromised
and
everything
and
on
that
particular
node,
might
get
compromised
and
then
so
you
can
see
it
in
this
diagram.
A
So
you
have
a
something
going
wrong
with
one
of
the
containers
and
then
the
compromises,
your
or
your
kernel.
So
if
you
can
see
that
Linux
kernel
has
over
300
system
calls,
so
every
system
call
makes
it
possible
for
a
point
of
attack
right.
So
it
common
sense
says:
if
you
have
less
of
these
system
calls
available,
you
have
less
potential
vulnerabilities.
A
Then,
if
you
look
at
the
darker
succumb
default
profile,
you
see
that
it
only
blocks
about
40
for
system
calls.
So
there's
still
about
280
system
calls
available
right
so
and
it's
really
hard
to
create
a
very
generic
set
comm
profile.
So
it's
loud
and
then
you
can
create
some
for
very
specific
applications,
but
it
becomes
really
difficult.
So,
as
you
can
see,
the
default
second
profile
has
an
inductor
has
a
1500
1778
line,
so
it
becomes.
You
know,
time,
tedious
and
difficult
to
figure
out.
A
For
me,
I
mean
alright,
if
you're
an
expert,
maybe
a
bit,
but
for
regular
people
it
might
be
difficult
and
then
there's
Linux
capabilities,
so
there's
39
of
them
and
as
you
can
see
each
one
of
them
has
this
unique
way
of
isolating
or
doing
different
things
of
the
Linux
kernel
level.
So
it
would
take
a
lot
of
time
for
you
to
understand,
or
you
can
try
jay-z
ourselves
contain
that
a
half-game
and
learn
a
little
bit
more
about
Linux
capabilities.
A
But
then
again,
it
will
take
you
time
to
figure
out
another
approach
that
a
lot
of
people
use
is
using
SC
Linux
to
contain
more
of
whatever
workloads
are
running
on
your
system,
but
then
you
have
to
create
a
base,
compile
a
base
module
and
it
becomes
kind
of
difficult
to
get
it
enable.
So
to
me,
this
is
kind
of
what
AC
unit
looks
like
you.
Have
all
these
different
options.
It's
like
a
galaxy
and
the
stars
are
all
the
kind
of
all
the
different
options.
B
A
A
So
you
see
that
the
difference
here
is
that
you
have
a
Linux
kernel
and
you
have
a
virtual
machine
and
that
virtual
machine
you
run
at
your
container.
So
the
virtual
machine
has
its
own
unique
kernel.
Color
containers
has
a
this
lightweight
kernel
and
I.
On
top
of
that,
you
have
your
application.
So
if
you
something
goes
wrong
with
that,
Linux
kernel,
the
other
host
Linux
kernel
doesn't
get
compromised
and
the
other
Linux
kernels
don't
get
compromised
either
on
for
the
other
containers.
A
But
if
you
have
regular
containers,
you
I
mean
if
you
have
a
second
profile,
it
slows
down
your
container
but
yeah,
but
you
know
the
way
most
people
run
regular
containers.
It's
not
us.
It's
actually
faster
than
running
cata
containers
and
it's
not
as
low
as
a
regular
virtual
machine,
because
it's
lightweight
lightweight
kernel
and
it's
some
lightweight
virtual
machine.
A
So
requirements
for
card
are
containers
either
very
metal
or
nested
virtualization,
obviously
running
on
Linux.
You
can
use
public
cloud
providers
like
GCP
or
Asher,
but
they
support
necessary
translation.
They
don't
have
bare
metal
offerings
right
now.
We
need
oh,
yes,
you
can
use
either
your
metal
instances
and
you
can
also
use
I
five
metal
instances
that
are
in
the
works
and
for
private
cloud.
You
can
run
it
with
anything
OpenStack
or
you
can
use
very
anything
bare
metal
or
the
supports
first
next
to
virtualization
and,
of
course,
for
platform.
A
Anything
kubernetes
with
to
orchestrate
your
workloads
with
would
cry
or
container
D,
or
you
can
also
use
plain
daughter.
When
you
first
install
cotta
containers,
you
can
just
run
this
utility
cut,
a
runtime
cut
a
check
and
then
it'll
tell
you
whether
you
can
run
counter
containers
in
that
host
or
not,
and
if
you
run
when
around
coordinate
is
you
can
use
this
cutout
deploy
utility
and
github
and
automatically
installed
counter
containers
on
all
of
your
nodes,
or
you
can
follow
the
cutter
installation
guide.
A
A
So
this
is
what
it
looks
like
with
containers
you
need
to
before
you
have
to
see.
If
you
have
a
lot
of
processes,
you
have
a
kind
of
container
diem
for
every
single
container
in
a
pod,
and
then
also
you
have
a
cottage
scene
for
every
single
container
in
a
pod
going
through
a
proxy,
and
then
you
you
see
that
you
need
a
shame
for
the
cada
agent
and
you
assume
for
every
single
container
in
your
pod.
A
B
B
So,
as
you
heard
one
of
the
ways
you
can
actually
improve
your
container
experience
in
terms
of
security
is
by
minimizing
the
amount
of
kernel
surface
you
actually
exposed
to
the
system
calls
so
part
of
what
nabla
containers
was
based
on
was
the
idea
somewhat
similar
to
what
G
visor
does
that?
If
you
can
take
parts
of
the
kernel
up
here,
I
suppose
I
should
do
it
over
there.
For
you,
people
isn't
as
well.
B
You
can
actually
move
some
parts
of
the
system
call
from
their
natural
place
and
the
Linux
kernel
up
into
this.
What's
called
an
a
blue
tender
which
is
basically
a
unicorn
alight
lib
operating
system
thing
and
do
emulation,
G
visor
is
doing
almost
exactly
the
same
thing.
The
only
real
difference
between
G
visor
and
Yoona
kernels
is
that
you
know
kernels
are
existing
technology,
hopefully,
with
more
of
the
bugs
worked
out
that
we
actually
repurpose
for
running
inside
a
container
as
a
library
operating
system.
B
B
We
actually
have
only
seven
system
calls
that
we
actually
project
into
the
Linux
kernel,
so
we've
reduced
the
system,
call
surface
from
the
huge
300
all
the
way
down
to
seven,
and
we
can
actually
experimentally
prove
that
this
gives
us
a
much
better.
What's
called
horizontal
attack
profile
of
the
Linux
kernel.
So
an
attack
profile
is
basically
the
number
of
lines
of
code.
You
traverse
multiplied
by
the
bug
density
of
that
code,
and
that
gives
you
a
number
that
corresponds
to
the
insecurities
of
your
process
of
your
per
thing.
B
Whatever
you're
running
the
higher
this
number,
the
more
insecure
you
are
horizontal
attack
profile
is
where
you
actually
hit
the
common
shared
components
of
the
system,
and
you
have
to
do
a
traversal
path
through
them
down
and
up
to
actually
get
whatever
effect
you're
trying
to
do
in
your
web
server
to
function
and
the
reason
horizontal
is
the
problematic
one
is
because
it's
a
shared
component.
If
the
user
can
come
down
and
hit
an
exploit
in
the
shared
component,
they
can
use
that
to
compromise
the
entire
physical
system.
B
Remember
Ricoh
told
you
that
the
kernel
is
actually
shared
amongst
all
containers
on
a
bare
metal
system.
If
that's
what
you
actually
do
so
the
thing
that
we
actually
measure
with
the
horizontal
attack
profile
is
the
portion
of
the
kernel
that
you're
actually
exploitable,
as
you
come
through
the
system
and
obviously
every
system
call
on
this
diagram
is
inside
the
kernel,
so
it
represents
a
potential
horizontal
exploit.
So
these
300
system
calls
in
an
ordinary
container,
contribute
fairly
enormous
Lee
to
your
horizontal
attack
profile.
B
So
the
way
it's
constructed,
as
I
said
is
so
if
you
went
to
just
in
Cormac
stalk,
he
told
you
that
all
code
has
to
be
modified
for
Yuna
kernels,
because
it's
linked
with
the
library
operating
system.
This
is
actually
untrue,
but
the
reason
that's
untrue
is
because
most
of
you
in
this
room
like
what
are
called
web
languages,
web
languages
run
on
top
of
an
interpreter.
B
So
the
way
we
get
your
web
languages
to
work
is
that
we
modify
the
interpreters
link
with
the
eunich
earn
'l,
but
your
web
language
stuff
goes
on
top
of
it,
so
we
can
run
nodejs
Redis
nginx
code.
All
of
that
can
actually
be
run
unmodified
inside
the
unit
kernel
tender.
But
if
you
wanted
to
run
something
like
a
vanilla,
C
program
that
would
have
to
be
modified,
and
fortunately
very
few
web
programmers
today
actually
write
C
code.
So
most
of
the
code
you
write
will
actually
run
unmodified
inside
nabla.
B
So
the
tender
converts
from
virtual
machine
into
the
Linux
system
calls-
and
this
gives
us
the
ability
to
run
this
thing
purely
inside
a
container
and
obviously
nablas
requirements
are
pretty
much
the
same
as
almost
any
other
environment.
So
we
can
run
and
we
ideally
would
run
on
regular
bare
metal.
Obviously,
the
operating
system
should
be
Linux,
but
one
of
the
interesting
things
about
nabla
is
that
actually,
because
we
only
use
seven
system
calls
and
they're
pretty
generic
system
calls
almost
any
POSIX
thing
can
actually
run
on
them.
B
And
the
answer
is
absolutely,
of
course
there's
no
reason
it
has
to
be
Linux
kernel
underneath
there
as
long
as
you
can
supply
these
seven
system
calls
it
can
actually
be
an
AIX
kernel
and
then
I
said.
Ax
is
still
a
thing,
and
he
said
oh
yeah,
because
he's
part
of
the
AIX
group,
but
the
point
is
that
we
can
use
this
narrowness
of
the
system,
call
interface
to
get
some
universality
here.
B
We
can
probably
run
on
more
things
than
Linux
and
obviously
then
we
can
run
on
any
public
cloud,
because
almost
every
public
cloud
is
Linux
or
provides
Linux
interfaces.
If
it's
Azure,
we
can
run
on
almost
all
the
private
clouds
and
we
can
run
on
almost
all
of
the
platforms
provided.
The
kubernetes
thing
has
been
modified
with
the
runtime
class,
which
Rico
will
actually
talk
about
later.
B
The
requirements
we
currently
have
for
nabla
are
this
thing
called
run
and
C,
because
every
different
runtime
currently
has
a
different
run
C,
so
we've
got
a
run
and
C
for
run.
Nabla
containers
divisor
has
run
SC
for
their
run,
secure,
containers
and
so
on,
but
once
we
actually
get
the
runtime
class,
we
can
actually
run
them
all
in
a
standard
way
and
obviously
we
can
run
them
with
docker
as
well
just
using
the
particular
runtime
specifier.
B
As
long
as
you've
got
the
docker
version
that
actually
accepts
run
C
so
side-by-side
comparison,
we
can
do
this
together.
These
are
some
of
the
metrics
that
we
actually
did.
This
is
somewhat
of
a
repetition
of
what
you
saw
in
the
previous
talk.
These
are
actually
throughput
metrics
and
the
red
at
the
top
is
pure
docker.
This
is
all
running
on
bare
metal,
so
it
didn't
look
at
any
virtualization
penalties
here,
which
is
the
most
interesting
use
case
and
docker.
B
One
is
the
bare
metal
one
that
we're
all
chasing
it's
the
high
result,
so
docker
on
bare
metal.
As
you
can
see,
the
2d
Weiser
results
are
actually
sitting
down
here
and
that
mirror
is
pretty
much.
What
was
seen
in
the
last
performance
talk
JEE
visors
performance,
particularly
on
benchmarks
like
Redis,
get
set,
Python
tornado,
node
Express
is
not
terribly
good.
We
actually
have
done
the
differential
between
this
is
actually
the
P
trace.
G
Visor,
and
this
is
the
KVM
G
visor.
B
So,
as
you
can
see,
if
you
have
a
significant
system
called
penalty,
P
traces
losses,
you
a
lot
more
performance
than
actually
trying
to
do.
The
KBM
interface
does
and
then
you
can
actually
see
cata
containers
here
in
all
of
these.
So
by
and
large,
the
difference
between
quatre
and
docker
is
not
that
high,
which
is
sort
of
pretty
much.
Also
what
the
previous
talks
told
you
and
then
for
nabla
containers,
nabla
raw
is
actually
running
in
the
KVM
alike.
B
Interface
babbler
container
is
actually
running
it
through
the
standard,
tenderized
container
interfaces
and
again,
the
performance
penalty
is
something
but
it's
not
as
much
as
sort
of
factors
of
two
or
three
it's
down
in
the
percentile
regions.
For
almost
all
of
these
throughput
tests,
so
I
described
a
bit.
What
horizontal
attack
profile
was,
but
basically
it's
an
attempt
to
measure
security
in
any
system.
So
this
this
measure
doesn't
just
work
for
nabla.
It
works
for
almost
any
system
that
claims
to
have
security.
B
It
was
done
by
IBM
research,
because
I
got
pretty
tired,
I've
been
in
containers
a
long
time
and
the
low
level
of
them
and
we've
had
endless
arguments
with
hypervisor
people
about
our
container
secure
our
containers
not
secure
in
the
early
days
IBM
bluemix,
which
is
the
old
IBM
cloud,
was
the
world's
first
actually
container
bare
metal
cloud,
which
meant
your
containers
were
not
running
on
a
virtual
machine.
They
were
actually
running
on
native
bare
metal.
It
was
actually
built
using
the
OpenStack
docker
Nova
interface,
which
is
something
that
really
doesn't
exist
in
our
days.
B
So
nowadays
we
were
all
about
kubernetes
in
the
cloud,
but
in
those
early
days
who
are
about
docker.
But
the
point
is
that
a
bare-metal
cloud
actually
gives
you
far
more
performance
and
control
than
almost
anything
else.
So,
as
Justin
was
saying
in
his
talk
previously
bare
metal
containers
are
things
we
all
want
to
aim
for
so
IBM
initially
aimed
at
them
first.
B
But
everybody
gets
frightened
by
the
fact
that
we
have
these
problems
with
horizontal
containment
and
forcing
customers
to
use
virtual
machines
as
one
of
the
ways
that
cloud
service
providers
can
take
this
from
being
our
problem
as
a
cloud
service
provider
to
your
problem
as
a
customer.
However,
if
you
think
about
the
trends
in
clouds
today,
the
idea
is
actually
to
reverse
that
we
should
be
taking
away
more
of
your
problems
than
actually
pushing
them
back
onto
you,
which
is
why
clouds
going
towards
bare
metal
would
actually
be
a
good
thing.
B
The
way
we
actually
measured
it
inside
the
Linux
kernel
is
just
to
use
F
trace.
F
traces
just
tells
you
as
a
trace
of
everything
that
comes
through
the
Linux
kernel.
How
many
functions
are
called,
and
you
just
multiply
that
by
the
bug
density
which
for
the
Lexx
kernel,
horizontal
piece,
we
assume
to
be
uniform.
It's
not
quite
true.
We
as
a
I'm.
Actually,
a
driver
developer
for
Linux
I
gotta
confess
that
about
80%
of
Linux
is
bugs,
do
tend
to
be
in
drivers.
B
The
the
rest
of
the
kernel
only
has
about
20%
of
the
bugs,
but
in
my
defense,
I
have
to
say
that
80%
of
the
Linux
kernel
is
drivers
so
on
that
measure,
potentially
perhaps
we're
all
uniform
and
obviously
the
runtimes
you
can
do
this
with
and
not
just
in
a
blur.
It
scatter
Docherty
visor
anything,
and
these
are
the
results.
B
So
this
is
looking
at
the
actual
footprint
inside
the
kernel
of
all
of
this,
and,
as
you
can
see,
the
interesting
thing
is
the
docker
is
not
actually
you'd
expect
it
to
have
the
highest
possible
footprint,
and
I
should
also
stress
that
this
is
docker
running
an
application,
so
the
docker
here
is
not
using
all
300
system
calls.
That's
why
it's
pretty
low.
B
What
this
is
showing
is
that
if
you
could
get
a
perfect
set,
calm
profile,
which
lots
of
people
will
claim
can't
be
done,
and
if
you
claim
that
I'll
set
Jessie
Frizzell
on
you
because
she
claims
it
can,
but
if
you
could
do
it,
this
is
what
the
banding
sort
of
horizontal
attack
profile
of
docker
would
be
right.
So
the
point
to
make
is
that
this
is
the
bounding
horizontal
attack
profile
of
cata
right,
and
these
two
are
again
not
very
different.
B
If
you
talk
to
hypervisor
people,
they
would
usually
tell
you
that
a
hypervisor,
because
it's
only
got
about
you-
know
ten
hyper
calls
and
a
container
has
300
system
calls
should
be
thirty
times
more
secure
than
a
container.
That's
not
what
these
figures
show,
primarily
because
this
docker
instance
is
not
using
all
of
the
system
calls,
and
there
are
actually
reasons
why
hypervisors
actually
use
much
more
of
the
kernel
interface
than
they
let
on
through
the
hyper
calls.
It's
why
any
vulnerability
and
KVM
is
almost
instantly
a
fatal
vulnerability
of
containment
and
hypervisor.
B
So
on
the
horizontal
attack
profile.
The
difference
here
is
only
around
20%
is
in
the
or
30%.
That's
not
not
significantly.
I
mean
on
some
of
the
tests
where
docker
uses
more
system
calls
I
mean
cata,
always
sort
of
hides
around
this
line.
Primarily,
it
just
depends
how
many
functions
inside
the
hyper
calls
your
exciting
docker,
actually
bounces
around
pretty
significantly,
depending
on
the
number
of
system
calls
it
uses
in
the
test.
The
interesting
thing
on
this
is
that
pretty
universally
G
visor
is
on
some
of
its
incarnations
higher
than
docker.
B
That
means
that
technically
G
visor
is
using
more
system
calls
than
docker
is
which
looks
like
a
paradoxical
result.
How
the
hell
is
it
doing
this
and
the
reason
is
G
visor
is
implemented
entirely
on
a
go
runtime
and
it
turns
out
that
the
go
runtime
is
pretty
profligate
in
the
way
the
user
system
calls.
So
as
its
translating
the
kernel
into
go,
it's
actually
making
more
system
calls
than
the
system
calls
are
translated.
B
It's
an
interesting
question
as
to
whether
the
system
calls
it
makes
their
exploitable
or
not
and
we'll
get
into
that,
but
right
right
at
the
moment.
Just
on
the
horizontal
attack
profile,
G
visor
is
actually
more
vulnerable
than
docker
and
obviously,
since
we
tuned
a
blur
against
this
horizontal
profile
measure,
so
we
have
a
lead
on
almost
everybody
doing
this,
because
we
invented
the
measure
published
it.
But
then
we
tuned
our
system
to
it.
B
Nabla
is
sitting
at
about
half
of
what
katidis
for
most
of
the
time,
and
the
point
is
this
is
a
pure
container
runtime
with
no
virtualization
that,
on
the
only
known
measure
of
security
profiles,
is
showing
itself
to
be
about
50%
more
secure
than
a
hypervisor,
which
is
the
achievement.
We
were
looking
for.
B
There's
also
one
interesting
thing
that
we
have
on
our
website,
which
is
a
while
back
using
the
idea
that
you
could
actually
look
at
what
interfaces
the
system
was
using
in
the
kernel.
One
of
the
things
I
set,
an
IBM
sort
of
in
term
was
okay,
so
just
saying
that
we
can
do
this
is
one
thing
can
you
prove
to
me?
We
can
exploit
it.
B
So,
look
at
the
way
that
quatre
is
using
the
interfaces
and
come
up
with
an
exploit
based
on
an
existing
CVE
point
of
basing
it
on
an
existing
CV
is
the
exploit.
I
show
you
only
works
and
older
kernels.
Everybody
has
patched
it
now,
so
it
wouldn't
work
on
the
public
cloud
you're
taking
it
to
so
I'm
trying
to
be
a
good
white
hat
citizen
instead
of
a
black
hat.
But
the
point
is
that
it
turns
out
the
9-p
interface
in
Katter
is
pretty
much
a
handoff
from
the
guest
of
the
host.
B
Any
file
system
exploit.
I
find
in
the
host
kernel
can
actually
be
exploited
over
the
9p
interface
and
kata,
and
so
this
exploit
actually
shows
somebody
firing
up
a
binary
running
inside
a
virtual
machine
and
cata
and
then
oops
Singh
the
host
kernel,
primarily
because
it
does
it
through
a
fast
an
exploit.
B
B
A
B
And
with
so
nabla
is,
some
binaries
cannot
be
run
unmodified
because
you
have
to
link
with
the
Unicode.
So
if
your
programming
and
a
low-level
language
like
C,
C,
double
plus,
you
have
to
modify
your
binary
before
it
will
run
inside
a
nabla
tender.
However,
if
you're
running
a
something
webby
that
sits
right
at
the
top
and
has
its
own
interpreter,
we
can
just
modify
the
interpreter
and
the
code
that
you
care
about
will
run
unmodified.
So
we
do
not
have
complete
university,
unlike
cata,
which
does
and.
B
We
just
don't
need
any
virtualization
of
tools,
so
we
run
equally
well
either
on
bare
metal
or
with
nested
virtualization.
Just
because
we
don't
use
any
of
the
virtualization
calls,
but
again,
as
I
said,
I
think
the
less
virtualization
you
have
the
better.
You
are
because
the
more
responsible
you're
making
your
cloud
service
provider,
which
is
a
good
thing
in
my
book,
Makara.
A
B
Where
I
mean
this
is
a
research
project
I'm
not
kidding
you
you've
been
to
many
talks
about
doing
this
in
production.
I,
probably
is
a
research
project.
I
should
be
able
to
tell
you
that,
obviously
it's
not
in
production,
however
I
know
one
person
who
is
actually
using
nabla
and
production.
Surprisingly
enough,
I
didn't
recommend
they
do
it
and
I
won't
name
them
and
shame
them
here,
but
if
one
person
is
actually
running
it
in
production,
but
obviously,
ideally
this
is
still
a
cutting-edge
research
project,
not
something
you
should
be
deploying
on
for
everyday
use.
A
B
Enabler
the
tender
is
based
on
net
bsd,
which
means
in
theory.
We
have
a
mismatch
between
linux
and
that
bsd,
but
the
job
of
this
thing
called
run.
Prong
that
you
saw
inside
the
delta
is
actually
to
shim
that
difference
so
effectively
or
any
program
that
runs
on
linux
can
be
linked
with
enabler
ten
with
the
nabla,
a
romper
on
runtime
run
on
the
tender,
and
it
should
all
work.
B
But
the
key
thing
here
is:
should
we
have
explored
a
lot
of
different
applications,
including
things
like
the
go:
runtime
the
Python
runtime
so
far,
it
all
seems
to
work,
but
because
we're
based
on
net
bsd
instead
of
linux.
At
some
point,
you
might
expect
to
find
a
problem
down
that
round.
There
cool.
A
Let's
talk
about
kubernetes
and
how
you
can
run
this
with
kubernetes,
although
both
of
these
runtimes,
so
you
can
run
it
with
either
a
crier
or
a
container
cry
or
support
it's
not
quite
there
yet,
but
it
will
be
there
at
some
point
and
you
can
use
the
runtime
class
in
kubernetes
that
became
available
and
we're
dead.
It's
112,
so
kind
of
requirements.
Again
it's
just.
We
talked
about
that
before
per
medal
on
extra
virtualization.
A
A
So,
let's
talk
about
the
runtime
class.
This
is
work.
What
I
call
the
ec2
or
kubernetes
just
because
in
just
because
in
easy
easy
to
you
have
all
kinds
of
different
instance
types.
So,
with
the
renter
class,
you
given
all
kinds
of
different
runtime
classes
right,
so
you
can
mix
and
match
and
the
types
of
workloads,
so
you
can
mix
in
your
cluster
depending
on
the
type
of
runtimes.
Are
you
using
so
what
he
said?
A
We
said
you
can
run
different,
runtimes
and
kubernetes
available
in
kubernetes
1:12,
and
you
need
to
enable
the
feature
gate
installed.
Customer
service
definition
it
covered
a
days.
You
need
to
configure
container
D
the
CRI,
and
then
you
need
to
specify
that
runtime
class
name
in
the
pod
when
you
instantiate
it.
So
this
is
what
it
looks
like
running:
a
runtime
class
specification.
A
This
is
a
single
string
and
you
need
that
runtime
handler,
which
is
another
string
that
gets
configure
at
the
CRI
level
in
container
the
or
cryo,
and
then,
when
you
want
to
run
your
part,
you
specify
a
string
in,
and
this
could
be
something
like
add
our
runtime
or
novel
at
runtime
and
then,
with
the
container
Li
configuration
to
specify
the
again
the
string
which
is
a
CRI
configuration
and
then
the
binary
of
the
runtime
that
you
want
to
use.
So
there's
a
feature
in
the
works
in
sick
note.
A
They
will
allow
you
to
identify
the
runtimes
automatically
on
the
nose.
So
as
soon
as
the
cubelet
boots
up
and
register
registers
with
the
cube
api
server,
it
will
tell
it
you
know
what
kind
of
runtimes
are
available
on
that
node
and
they
will
be
available
in
curators
by
the
time
you
issue
like
a
command
like
a
cube:
CTL
computer
it'll
get
runtime
command
and
then
you'll
get
the
list
of
runtimes.
A
So
briefly,
workloads
that
you
can
run
with
these
runtimes
anything
goes
so
user
going
to
a
load
balancer,
it's
just
an
example
sto
and
running
with
unbeli
proxy
in
kubernetes
other
types
of
workloads,
api's
credential,
store
databases
and
applications
for
big
data
and
analytics
example
of
running
micro
services.
This
is
micro
service,
with
a
back
in
and
use
of
the
database
and
kubernetes
using
a
kubernetes
ingress
credential
store.
A
This
is
an
opposite
of
what
type
of
workload
you
would
think
you
run
in
these
types
of
runtimes,
because
they're
not
necessarily
untrusted,
it's
more
like
a
trusted
workload,
but
it
also
goes
with
these.
If
you
want
to
keep
your
workloads
isolated,
where
you
want
I
have
a
user
come
through
TLS
and
then
in
the
backend.
A
Your
services
going
through
TLS
as
well
through
to
a
data
store
that
can
be
anything
like
Cassandra
did
datastore
at
CD
or
Amazon
s3
an
example
or
relational
databases
also
can
be
run
with
something
to
be
natives
and
any
of
these
runtimes
and
a
brief
word
on
multi-tenancy.
So
so
this
these
runtimes
will
allow
you
to
run
in
a
cluster
having
maybe
two
namespaces
for
different
tenants
and
in
an
isolated
way,
so
in
a
single
cluster.
So
allowing
you
to
have
that
isolation
across
the
board.
A
A
Five
minutes
or
something
so
I
have
two
definitions
here
for
a
power
in
Cara
and
it
specifies
the
runtime
class
cat
a
runtime
here
and
I
also
have
power
in
nabla
and
it
specifies
table
runtime
here.
So
these
are
pre-configured
ahead
of
time
to
run
in
a
in
the
CRI
in
our
in
a
kubernetes
cluster
running
in
DCP,
and
this
GCP
cluster
has
nestled
virtualization
enable
so
I'm
gonna
go
ahead
and
up
like
these.
A
A
To
show
you
that
nebula
is
running,
I
will
show
the
logs
and
you
can
see
the
solo
guru
actually
boot.
It
put
it
up
here
and
then
it's
actually
listening
on
port
8080
and
then
for
cotta
containers.
I
have
to
assess
age
I'm,
going
to
go
ahead
and
SSH
to
each
one
of
my
kubernetes
nodes
and
run
a
kata
runtime
list
command
and
then
and
one
of
the
notes
these
this
POTUS
it
was
running
and
it
will
show
you
that
the
containers
are
there
for
Okada.
A
So
that's
SS
aging,
so
they're
they're,
my
kind
of
runtime
pods,
so
yeah.
So
this
is
a
runtime
class
and
you
can
run
both
runtimes
together
on
a
single
cluster,
pretty
cool
stuff.
So
there's
talk
about
implementing
and
maybe
more
capabilities
in
the
runtime
class.
That
I
will
specify.
Maybe
nabla
has
some
certain
capabilities
and
Kara
has
some
other
certain
capabilities
and
so
that
people
can
choose
which
one
they
want
to
use
and
in
other
runtimes,
like
divisor
or
any
of
the
newer
runtimes
that
get
created.
B
The
nebula
containers,
obviously
the
thing
that
we
come
to
the
here
for
is
kubernetes
integration
run
ANSI
is
okay,
but
we,
it
could
be
better.
Obviously,
we
need
better
universality.
If
you
want
to
run
C
applications,
although
I'm
not
sure
we
can
get
there,
we
need
lots
of
public
cloud
support.
We
need
more
tooling
around
the
images.
B
If
you
want
to
create
your
own
rump
run
images
instead
of
using
our
preset
ones,
even
if
you're
building
things
like
Python
and
nodejs
sometime,
you
actually
want
to
tune
the
Python
and
nodejs
interpreters
for
the
runtime
environment
and
obviously
build
listen.
Server
lists
to
applications
were
looking
at
nabla
and
in
the
ax
use
case.
B
If
you
actually
look
at
what's
going
on,
is
that
we're
emulating
most
of
the
system
calls
so
effectively
we're
committing
the
hypervisor
fault,
because
any
system
call
we
emulate.
We've
pushed
up
into
the
layer
that
you
effectively
own
as
the
person
who's
renting
this
instance.
The
ideal
would
be
to
try
and
actually
push
more
back
to
the
cloud
service
provider
so
make
them
responsible
for
the
tender
and
so
low
five
and
everything
which
we
can
do,
but
one
of
the
things
we're
actually
going
to
be
looking
at
is
two
different
things.
B
One
is
the
idea
of
a
horizontal
attack
profile
where
the
actual
measurement
is
a
number
of
bugs
traversed.
A
number
of
lines
of
code
traversed
x.
Bug
density
is
not
quite
right,
because
if
you
look
at
the
CVE
sprinkling
throughout
linux,
some
system
calls
are
much
more
vulnerable
on
CVEs
than
others,
and
this
means
that
some
interfaces
are
actually
much
safer
than
others.
B
One
of
the
things
we
actually
need
to
do
to
refine
the
horizontal
attack
profile
is
to
multiply
by
safety
of
the
interface,
which
will
give
us
a
much
more
accurate
number
and
then,
from
my
point
of
view,
as
the
kernel
engine,
much
more
interesting
is
asking
the
question.
These
system
calls
that
sit
inside
the
Linux
kernel
right
at
the
moment,
they're
unsafe
on
a
horizontal
attack
profile,
because
the
kernel
runs
in
the
same
address
space
any
escape
from
that
system
call
compromises
the
entire
kernel
and
therefore
the
entire
system.
B
One
thing
that's
interesting
to
look
at
from
an
operating
system.
Point
of
view
is:
could
we
actually
run
the
Linux
kernel
and
multiple
address
spaces
for
multiple
different
things
that
were
using
it?
Because
if
I
were
actually
running
in
an
address
space
that
was
different
from
Buttle
ID
to
the
actual
process,
runtime
for
my
system,
cool
I
could
still
compromise.
That
system
call
breakout,
but
the
only
thing
I
get
access
to
is
the
thing
I've
already
compromised
anyway,
which
is
the
container
I,
can't
actually
break
horizontally
into
another
container
so
effectively.
B
It
allows
you
to
run
proper
Linux
kernel
system
calls
in
a
safe
environment,
and
this
is
one
of
the
things
we'll
actually
be
looking
at
as
future-looking
research
later
and
obviously
we
have
lots
of
future
workloads
for
everything
we're
running
out
of
time,
so
we'll
skip
over
them.
We
are
looking
at
doing
this
on
server
list,
for
both
of
us,
firecracker
is
the
sort
of
thing
for
him,
and
actually
firecracker
is
useful
for
me,
because
the
civilus
people,
a
year
ago
on
I,
showed
them
nabla
and
IBM
told
me
to
go
away
now.
B
B
So
the
question
is:
have
we
tried
running
docker
inside
and
a
below?
The
answer
is
yes,
and
it
doesn't
really
work
very
well.
The
reason
it
doesn't
work
very
well
is
because
the
go-go
run
time
was
not
very
that
the
problem
with
docker
is
that
it
claims
to
be
written
in
go
which
is
an
easy
language
to
put
a
tender
under,
but
the
big
problem
is,
it
makes
lots
of
system
calls
outside
of
the
go
environment
using
that
go
system,
cool
thing,
and
they
are
much
harder
to
poke
through
things
like
namespace
creation,
for
instance.
B
A
Question
is:
are
cars
using
key
you
and
what's
the
reason
why
it's
using
key
mu
I
believe
it
for
historical
reasons.
So
there
is
any
mu
support
in
one
point
for
which
is
a
stripped-down
key
you,
so
you
can
make
use
of
that
and
again,
like
I
mentioned,
we
were
working
with
the
Firecracker
community
that
will
allow
us
to
around.
You
know,
cut
out
with
firecrackers,
so
firecrackers
are
stripped
down
EMM
that
has
just
whatever
three
three
devices
yeah,
so
very
lightweight.
Okay,.