►
From YouTube: Beyond Orchestration: The Cloud Native Runtimes Ecosystem for Performance and... - Alexander Jung
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Beyond Orchestration: The Cloud Native Runtimes Ecosystem for Performance and Security - Alexander Jung, Unikraft
Speakers: Alexander Jung
The ecosystem of runtimes for container-based applications has been growing dramatically over the last few years with new implementations providing deeper integration between host and application with two primary goals in mind: performance and security.
Many of these systems, however, rely on manually adapting your application for the runtime provider for performance which negatively impacts adoption or uses existing virtualized, ad-hoc and traditional OSes/kernels for security which negatively impacts performance.
In this talk, we introduce Unikraft: an open-source library Operating System which enables the construction of ultra-lightweight VMs quickly, easily and without time-consuming developer effort. These VM images are tailored to the application itself and have high-performance, low resource usage (e.g. 3-5ms boot times, MBs of memory and disk usage, to name a few metrics) and a small attack surface (e.g.: no co-processes, no shell). We make comparisons across the current runtime landscape and demonstrate how you can use Unikraft with Kubernetes today.
A
So
the
themes
that
I've
noticed
over
the
last
few
days
here
at
kubecon,
going
down
booze
and
going
at
lots
of
different
talks,
is
that
there's
a
lot
of
cve
scanning
package
scanning,
there's
applications
that
are.
You
know
we
look
at
a
particular
item
and
then
we
have
some
system
to
introspect
it,
and
we
can
do
this
in
a
number
of
different
ways
and
so
there's
also
a
number
of
Open
Source
projects
that
are
doing
this.
A
But
there
are
also
a
number
of
you
know:
companies
that
are
offering
this
as
a
service,
and
this
is
sort
of
the
idea
here
is
we're
going
to
step
back
and
kind
of.
What's
the
problem
here
in
overall
right
in
the
sense
that
we
are
looking
at
a
traditional
system
and
the
other
one
is
performance
and
how
we
tackle
this
and
again
it's
a
lot
of
introspection.
A
It's
a
lot
of
you
know,
there's
some
different
themes
here,
like
we
have
ai
models
to
look
at
traffic
detection,
and
then
we
optimize
that
way
or
we
look
at
inside
and
we
change
the
libraries
we
compile
them
to
make
them
more
Specialized
or
we
have
like
slimmers
like
there's
darker
slim
slim
AI.
Out
of
this.
There
are
a
number
of
ways
that
we
can
reduce
what
we
have
already
to
make
it
smaller
and
therefore
more
performant,
and
so
overall
in
general.
A
What
we're
doing
is
we're
introspecting
and
we're
looking
at
something
and
we're
trying
to
find
out
ways
to
make
it
more
secure
and
ways
to
make
it
more
performant,
and
when
we
look
at
how
we
approach
the
runtime
of
things
in
the
cloud,
there
are
a
number
of
different
sort
of
solutions
that
are
lightweight.
If
you
look
at
some
projects:
K3
Os
from
Rancher,
Venture
RS,
of
course,
this
Linux
kit
and
this
from
VMware
photon
sort
of
a
container
runtime
platform
and
these
projects.
A
They
basically
still
rely
on
the
traditional
kernel
stack
and
they
still
run
on
top
of
a
hypervisor.
And
so
what
I'm
trying
to
get
at
here
is
that
both
the
security
and
the
performance
are
reliant
on
a
traditional
fashion.
That
is
the
large
kernel,
hypervisor
kernel,
the
large
Linux
kernel
right
and
there's
a
lot
of
problems
with
this
particular
stack
right.
A
And
if
we
look
at
the
container
runtime
container
D
there's
some
new
projects,
like
lip
k,
run,
run
J,
Etc
and
they're
still
sort
of
probing
underneath
it
the
operating
system.
So
as
much
as
you're
trying
to
tune
this
particular
model,
you're
still
relying
on
a
sort
of
flat
base
that
is
sort
of
homogeneous
across
the
industry,
this
is
the
living's
kernel
right.
A
The
problem
I'm
trying
to
get
at
ultimately
is
that
there's
lots
of
layers
of
indirection
lots
of
virtualization
involved
here
that
you
are
so
far
away
from
your
application,
your
application
so
far
away
from
the
metal
that
is
actually
running
on,
so
your
pod
is
usually
containing
an
application.
Sometimes
this
you
know
it's.
You
can't
even
put
two
applications
in
it.
You
like,
if
you
start
a
bash
script
and
it
spawns
multiple
processes.
A
This
is
then
separated
by
namespace
by
the
operating
systems.
Container
runtime,
the
container
runtime
is
doing
Communications
with
the
OS
and
then
finally,
it's
running
on
top
of
a
hypervisor
lots
of
indirection,
and
this
is
creating
the
problem
here.
This
is
the
security
that
we
have
to
look
at
now
across
many
different
layers
and
there's
the
performance
penalty
of
having
all
of
those
layers.
A
And,
of
course,
you
know
the
security
part
here,
your
Cube
system,
things
mingling
with
your
pods.
So
if
we
try
and
tackle
the
problem
sort
of
at
the
lowest
level,
we
try
and
reduce
Linux.
So
the
you
know
the
monolithic
kernel
that
we
have.
We
find
that
it's
actually
very
difficult,
and
so
we
did
a
study
where
we
looked
at
the
dependencies
between
different
modules.
Inside
of
the
Linux
kernel,
we
use
a
program
called
C
scope.
A
If
one
function
was
referenced
in
another
module,
then
we
counted
that
as
a
individual
reference,
and
then
you
can
see
this
sort
of
maze
of
graphs
between
major.
You
know
subsystems
within
Linux,
so
it's
very
difficult
actually
to
pull
apart
the
Linux
kernel,
despite
there
being
tools
to
optimize
it
despite
their
being
systems
to
try
and
reduce
it
further.
A
A
So
if
you
look
at
the
full
stack
application
here,
you
have
your
application,
of
course,
its
third
party
libraries
and
dependencies
that
you
need
for
it
to
run.
But
then,
of
course,
on
top
of
it
is
the
traditional
stack
that
is
the
kernel.
It's
the
operating
system.
This
might
be
a
distribution
for
example,
and
then
finally,
it's
sitting
on
top
of
platform
and
Hardware
stepping
back
from
the
30
years
of
research
and
development,
that's
gone
into
Linux
and
to
all
other
operating
systems
and
picking
apart
the
different
parts
that
we
need.
A
We
can
sort
of
think
about
it
as
a
library
of
os,
Colonel,
Os
or
kernel.
Libraries
right
and
you
can
pick
and
choose,
and
so
the
uni
kernel
model
is
where
you
select
the
necessary
libraries
for
it
to
become
a
kernel
and
then
go
through
a
process
of
building
it
and
constructing
it.
To
result
in
what
is
a
final
unicorn
or
binary
image.
A
A
A
It
has
a
single
sealed,
address
space,
so
there's
no
longer
a
separation
between
kernel,
space
and
user
space
they're,
all
one
of
the
same,
which
means
every
Cisco
is
no
longer
a
guard
between
a
permission
check.
It
is
just
a
it's
just
a
function
call
so
what
was,
for
example,
300
CPU
Cycles
to
make
that
check
is
now
only
a
few
CPU
Cycles,
no
syscals
right.
It
only
does
one
thing,
so
there
are
no
other
processes
that
are
running
at
a
single
process.
There's
no
SSH,
there's
no
system
D.
A
A
So
this
is
what
unicraft
does
unicraft
is
a
uni
kernel,
development
kit.
We
started
as
a
Zen
incubator
project
around
2017
and
we're
actually
Linux
Foundation,
but
we're
trying
to
break
in
sort
of
into
the
cloud
native
space
because
it
makes
quite
a
good
use
case,
for
example,
for
lightweight
serverless
functions,
lambdas
Etc,
so
there's
a
little
bit
about
our
community.
A
So
we
started
off
as
research.
We
have
a
lot
of
industrial
Partners,
a
lot
of
universities
that
we
work
with,
and
a
lot
of
contributors
that
are
still
with
us,
the
complexity
and
so
the
next
few
slides
are
just
a
little
bit
about
the
result
of
building
something
as
a
unicorn.
The
complexity
of
your
final
kernel
image
is
a
lot
simpler
and
makes
it
a
lot
easier
to
debug,
for
example.
A
So
if
you
remember
that
sort
of
mesh
graph
that
we
showed
before
for
Linux,
this
is
that
same
program
running
now
only
for
unicraft,
and
this
is
nginx
compiled
against
unique
craft.
So
it's
now
an
nginx
uni
kernel
and
there's
also
just
a
hello
world
that
just
simply
boots
and
prints
the
words
hello
world.
A
We
did
a
performance
comparison
against
unicraft
and
then
a
number
of
other
Technologies
and,
of
course,
Docker
running
on
container
D
and
then
Linux
bare
metal,
for
example.
This
is,
we
have
a
physical
machine.
We
installed
Linux,
we
ran
nginx
on
it.
We
pinned
nginx
to
a
core
and
then
in
comparison
here.
It
sort
of
really
demonstrates
that
the
cost
between
jumping
from
the
user
space
space
right,
if
you're
doing
like
a
read
or
a
right
or
you
know
doing
some.
I
o
path
into
kernel
space,
really
has
a
performance
penalty.
A
So
there's
a
82
percent
increase
in
comparison
compared
to
docker
memory
requirements,
of
course,
also
go
down
so
unicraft
here
on
the
far
left,
only
using
a
few
megabytes
of
memory
that
are
required
to
run
any
particular
application
here,
the
different
applications-
and
we
also
support
a
number
of
different
virtual
machine
monitors
platforms.
Basically,
so
we
can
run
on
firecracker-
and
here
you
can
see
that
you
can
boot
a
simple
hello
world
application
and
this
little
3.1
milliseconds.
This
was
as
a
virtual
machine
just
for
fun.
We
were
playing
around
with
Edge
devices.
A
It
makes
sense
it's
given
the
sort
of
low
resource
capabilities,
so
this
is
on
Raspberry
Pi,
the
amount
of
time
it
takes
to
boot
actually
and
get
something
to
work.
So
here
we
have
nginx,
for
example,
booting
in
only
92
milliseconds,
compared
to
when
we
installed
Alpine
Linux-
and
here
we
even
you
know
customized
the
Linux
kernel
so
that
it
had,
for
example,
security,
mitigations
turned
off
so
that
there
weren't
extra
guards
in
between
different
process
calls
a
little
bit
of
comparison
for
CPU
cycles
and
how
long
it
takes.
A
The
transport
is
obviously
now
a
lot
smaller,
because
you're
no
longer
having
to
ship
a
full
file
system
containing
shared
libraries
with
things
that
you
might
not
necessarily
actually
use
for
your
application.
So
if
you
looked
at
nginx
and
we
pulled
the
same
version
here-
it
contains
shared
libraries
that
are
not
statically
linked
against
the
nginx
binary,
so
they're
required,
and
they
contain
functionality
that
you
don't
necessarily
need.
But
with
unicraft
you
literally
compile
all
the
necessary
libraries
that
you
need
and
then
you
can
go
even
further.
A
You
can
actually
do
compile
time
specialization
strategies
such
as
link
time,
optimization
dead
code,
elimination,
Etc
as
a
result.
There's
a
reduced
attack
surface,
so
you
have
no
shell,
there's
no
way
to
penetrate
sort
of
externally.
If
there's
exposed
ports
there,
we
have
the
opportunity
to
specialize
the
the
memory,
so
you
can
do
aslr
every
time
you
rebuild
the
kernel,
you
can
make
it
a
unique
and
you
get
now
the
lowest
level
of
isolation
that
is
provided
by
to
you.
That
is
Hardware
right.
A
The
virtual
machine
is
still
the
standard
unit
of
security
in
the
cloud
and
you
still
provision
containers
inside
of
virtual
machines,
and
now
you
get
actually
this,
but
without
the
overhead
and
then
there's
lots
of
specific
CPU
security
features
that
you
can
enable
given
a
certain
use
case.
So
we
have
a
lot
of
arm
support.
A
So
the
idea
is
here
alongside
the
Library
operating
system,
is
that
you
have
like
a
tool
that
basically
lets
you
grab
all
the
libraries
that
you
do
need
and
also
you,
you
know,
you
add
a
simple
craft
file
to
your
repository.
It
looks
a
little
bit
like
this.
You
can
then
list
all
the
libraries
that
you
will
necessarily
need.
You
know
if
you
need
TCP
IP
stack
or
if
you
need
this
is
new
lib,
which
is
a
lib
C
which
provides
basic
functionality
at
your
application.
A
The
idea
is
that
you
didn't
go
fetch
those
libraries
with
this
tool
called
crafted
and
then
just
build
it
and
out
pops
a
unique
kernel.
You
can
then
also
use
the
same
tool
to
build
different
packages,
so
you
can
make
it
OCR
make
it
look
like
an
OCA
archive,
or
you
can
also
then
push
it
to
a
registry,
and
then
you
can
then
use
it
sort
of
in
production.
A
So
there's
a
lot
of
tools
out
there
that
already
allow
you
to
run
virtual
machines
or
manage
virtual
machines
through
kubernetes
Lima
is
a
new
addition
to
the
cncf
space
and
there's
also
cute
bird,
that's
kind
of
containers
and-
and
they
still
rely,
however,
on
the
idea
that
you
are
running
your
container
within
a
virtual
machine,
and
so
if
you
deep,
dig
deep
dive
into
the
source
code,
it
still
has
this
requirement
to
SSH
into
that
virtual
machine
and
launch
a
container
right.
So
it
looks
you
know
it
looks
like
a
part.
A
It
shows
up
as
a
pod.
It
has
all
the
right
sort
of
characteristics,
of
feeling
like
it's
Cloud
native
but
you're,
still
managing
this
big
bulky,
Linux
kernel
with
a
container
inside
of
it,
and
so
obviously
you're
still
going
to
have
the
negative
effects
of
performance
and
then
by
total
implicit,
because
SSH
is
required,
you're
now
exposing
an
additional
attack
vector.
A
So
maybe
you're
familiar
with
wonk
we're
making
something
called
run.
You
it's
a
very
high
level
right.
This
is
showing
how
we
sort
of
connect
between
a
controller,
node
and
physical
nodes
or
virtual.
You
know,
worker
nodes.
What's
important
here
is
the
interaction
between
the
container
runtime
and
the
host,
and
so
the
rest
of
it
is
just
exposing
the
right
apis,
so
that
contains
so
that
kubernetes
looks
and
sees
things
as
pods
or
services.
A
So
basically
a
binary
and
it
accepts
ttrbc
requests
or
it
could
also
be
grbc
requests,
and
this
binary
basically
invokes
a
run
something
program,
and
then
this
invokes
the
ultimate
like
system,
that's
creating
it
previously
a
container,
but
in
this
case
a
uni
kernel,
and
so
in
the
previous
case
or
in
the
normal
cases,
invoking
run
C
run.
C
then
starts
a
namespace.
A
You
know
it's
agreement,
it
does
truth,
creates
this
file
system
for
you
and
offloads
everything
into
the
file
system
from
the
Otay
archive
and
then
starts
the
process
with
the
given
environment.
But
this
is
where
we
inject
ourselves
in
the
whole
process.
We
create
a
run
new
shim.
The
run
new
shim
then
invokes
the
instantiation
of
a
kernel,
using
whatever
virtual
machine,
monitor
tool
that
you
would
like,
for
example,
for
example,
kmu
or
firecracker.
A
Then
all
you
have
to
do
is
build
a
binary.
This
is
so
we
built
a
binary
code,
run
you
it'll.
The
interface
is
basically
creating
sub
commands
and
these
are
the
sub
commands
necessary
to
start
an
instance
based
in
this
whole
life
cycle,
and
then
these
basically
map
to
the
instantiation
of
a
virtual
machine
so
to
then
make
the
uni
kernel
feel
like
it's
a
container
we
used
oci,
which
is
basically
a
turbo
and
inside
the
turbo.
A
A
It
unpacks
it
and
then
goes
through
actually
craft
kit
to
then
unbundle
it
and
have
the
file
system
where
the
Unicorn
is
available,
and
so
the
result
here
so
I
showed
you
previously
that
we
had
this
virtual
machine
and
inside
the
virtual
machine
with
the
container
here,
we
now
have
direct
access
to
the
hardware.
So
this
is
what
the
model
finally
looks
like.
A
So
we're
still
working
on
this
there's
a
lot
of
work
to
still
do,
there's
still
when
we're
having
regular
meetings
and
we're
sort
of
building
more
Integrations
with
more
Cloud
native
tools,
and
so
one
of
the
things
that
we
found
was
that
the
oci
archive
specification
was
very
limiting.
It
has
there's
a
lot
of
opportunity
to
put
more
metadata
into
the
oci
archive.
A
For
example,
if
you
look
at
the
specification
provided
by
the
runtime
spec,
the
runtime
spec
has
very
limited
information
about
what
the
host
is
capable
of
doing,
and
you
basically
just
specify,
for
example,
that
there
is
an
OS
and
what
OS
its
name
is,
and
that
basically
is
a
map
of
like
Linux,
FreeBSD,
Windows,
Mac
or
something
like
this.
And
but
this
is
very
a
closed
view
of,
what's
actually
capable
in
the
hardware.
You
could,
for
example,
expose
more
information
about
what
capabilities
the
hardware
have
or
what
drivers
are
available.
A
What
system
services
are
available,
for
example,
whether
you
have
a
particular
file
system
available
to
you
or
not,
or
how
much
of
that
file
system?
And,
of
course,
it
also
indicates
what
architecture
is
is,
but
it
doesn't
indicate,
for
example,
which
architecture
extension.
It
is,
for
example,
for
arm
so
there's
more
information,
there's
a
lack
of
information.
We
found
with
the
oci
specification
and
we'd
love
to
be
able
to
collaborate
and
find
Opportunities
to
make
this
more
rich,
of
course,
with
uni
kernels,
since
there
are
specialized
Services
and
their
compile
time.
A
Specialization
means
that
you
can
do
there's
opportunity
to
make
it
even
more
performant
or
even
more
secure.
If
you
really
know
what
is
happening
at
the
end,
so
unicorns
usually
find
a
good
place
in
the
edge
where
Hardware
resources
are
limited,
but
at
the
same
time,
if
you
are
understanding
what
the
scheduling
of
services
might
be,
for
example,
if
you're
co-locating
Services,
you
could
specialize
those
uni
kernels
to
be
better
co-located.
A
For
example,
you
could
do
shared
memory
Pages
by
compiling
a
special
library
that
then
allows
you
to
do
shared
memory
rings
so
being
able
to
tap
in.
For
example,
to
the
scheduling
or
the
scheduler
for
kubernetes,
for
example,
and
understand
how
it's
going
to
schedule
Services
could
benefit
unit
kernels,
but
there's
some
open
challenges
here,
like
you
have
to
pre-compile
it
or
you
know,
how
does
it?
How
what
does
the
deployment
model
look
like?
A
Do
you
put
this
into
the
oci
archive,
for
example,
and
then,
of
course,
the
same
problem
before
about
the
host,
like
there's
lots
of
opportunities
to
create
many
permutations
for
a
given
unique
kernel,
in
the
same
way
that
you
have
several
permutations
for
a
normal
OCR
archive
for
when
you
compile
it
against
several
different
architectures,
so
this
kind
of
is
the
result
of
the
talk.
Thank
you
so
much
for
listening.
A
B
A
A
It's
really
only
if
you
wanted
to
compile.
You
know,
debug
the
kernel
itself
and
then
for
runtime
languages
such
as
JavaScript
and
python.
We
have
pre-built
unique
kernel
images
for
those
runtimes,
so
you
would
just
Mount
those
files
into
a
file
system
and
then
they
would
start
executing
in
the
normal
way
that
you
invoke.
You
know
Python
and
then
your
program
name.
A
B
A
A
It
really
depends
on
what
you
want
to
see
and
when,
if
you
want
to
just
try
and
debug
it
because
you're
getting
like
a
program
error,
it
would
be
a
different
sort
of
model
here,
maybe
locally,
as
opposed
to
when
you're
doing
in
production.
Then
you
would
have
you
know
these
monitorings
exposed
and
as
for
example,
as
a
library.
A
Yep,
so
virtio
is
supported,
number
of
different
fertile
platform
drivers,
and
so,
if
you
know
what
your
target
is,
if
you
want
to,
for
example,
use
KVM
with
vertio,
you
would
then
use
those
particular
libraries
you
would
enable
them
we
have
similar.
Have
you
ever
seen
like
make
menuconfig
you
open
up
and
make
menu
config?
A
B
So
in
that
case,
given
normally
you
have
this
kernel
space
and
user
space
acceleration,
but
with
unique
kernel
assumes
you
don't
have
the
acceleration.
Is
there
any
that
actually
has
security
implication
now
with
all
these
different?
Well,
our
protocols,
potentially
the
application,
can
more
easily
to
explore
that
if
there's
any
bugs
in
these
protocols
right.
A
So
when
you
are
now
the
building
your
application
as
a
unicorn,
you
trust
more
of
the
code
base
because
you,
it
is
only
the
few
libraries
that
are
necessary
for
it
to
be
a
virtual
machine.
You
are
no
longer
reliant
on
the
full
Linux
kernel
that
has
its
vulnerabilities
or
it's.
You
know
it's
a
there's
ways
to
exploit
this,
for
example,
but
there
are
a
number
of
security
mechanisms
that
you
can
also
enable,
like
ubisan,
KSN,
there's.
A
A
A
But
you
can
build,
for
example,
your
golang
application
or
rust
application
in
like
normally
and
make
it
a
unicorn
at
the
end.
And
then
you
can
run
this
and
we
also
support
major
Cloud
vendors,
such
as
AWS
gcp
Etc.
So
you
can
deploy
your
uni
kernel
as
a
virtual
machine
there,
and
then
you
don't
have
to
use
as
much
resources
as
well
compared
to
running
it
as
a
full
virtual
machine.