►
From YouTube: What's Up With All The Different Container Runtimes? - Ricardo Aravena, Branch Metrics
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)
What's Up With All The Different Container Runtimes? - Ricardo Aravena, Branch Metrics (Intermediate Skill Level)
The Kubernetes Container Runtime Interface (CRI) recently went GA (or rc?) and during the lead up we saw an a jump in the number of container runtimes hit the market, each with its own pros and cons. In this talk Ricardo will dive deep into the variety of current and future container runtimes. What are their pros and cons depending on the type of workload and application. Why stick to old plain Docker? Why use Mesos containers? Why CRI-O? What about Kata containers? After this session the audience will have a better understanding of the container landscape of the Kubernetes CRI and why multiple container runtimes exist and how to leverage their unique capabilities.
About Ricardo
Ricardo currently works at Branch as a Sr Data Ops Engineer. He has been working in tech for more than 18 years and comes from a diverse professional background, having been in different roles at large companies such as Cisco and VMware as well as startups such as Hytrust, Exablox and SnapLogic. Most recently he was at Coupa Software where he spent three years working on automating their cloud infrastructure to handle over 500 customers and preparing for their IPO.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
The
motivation
behind
this
talk
is
this
article
that
I
came
across
last
year
after
Q
Khan,
and
it
talked
about
all
the
different
containers
runtimes
that
it
started
showing
up
in
the
market
due
to
the
container
runtime
interface,
when
Combinator's
and
also
the
there
were
some
comments
about
what
about
these
other
container
runtimes
like
mesos
and
open
DZ.
So
I'll
talk
about
those
as
well.
So
a
little
bit
of
background
I
work,
a
branch
we
run
containers
pretty
much
everywhere.
A
We
run
all
of
our
API
services
in
kubernetes
in
all
the
different
types
of
environments,
QA,
development
and
production,
lots
of
different
languages.
We
service
about
maybe
60
to
70,000
requests
per
second,
and
then
we
run
all
of
our
data
pipelines
and
flink
and
sparked
frameworks
using
on
top
of
messes,
also
and
all
the
types
of
different
types
of
environments.
And
then
we
ingest,
maybe
about
10
billion
over
10
million
records
a
day
and
and
over
10
billion
queries.
Real-Time
queries
today.
So
they
are
the
runtime
that
we
use.
A
So
this
claimer,
where
the
runtimes
actually
live.
So
a
lot
of
you
probably
already
know,
but
the
Linux
kernel
has
all
this
facility
is
to
make
containers
possible
like
to
instantiate
the
containers
to
run
the
containers.
So
those
facilities
are
namespaces,
see
groups
at
barmer,
SELinux,
SE,
comm,
latex
capabilities,
so
everything
that
the
containers
do
it
it's.
A
So
in
2002
kernel
namespaces
became
available
in
the
Linux
kernel
and
that's
the
first
feature
related
to
containers.
Then
in
2006,
open
BC
was
created,
so
you
could
run
containers,
but
you
needed
a
kernel
level
patch.
So
you
needed
to
create
your
your
own
kernel
image,
put
up
your
server
and
run
your
containers
with
that
specific
kernel
image
and
then
in
2007.
Another
feature
related
to
containers
became
available
in
the
kernel
and
Nazi
groups.
A
A
So
in
2014
they
said.
Ok,
Alexi
is
not
providing
all
these
different
things
that
we
want
to
do
with
that.
The
the
colonel
supports,
so
we're
gonna
write
our
own
library
called
lid
containers,
so
they
release
dr.
lip
container
on
top
of
lip
container
and
then,
in
the
same
year,
rocket
became
available
as
an
alternative
to
rocket
to
docker
the
2015.
The
people
from
or
the
communities
from,
docker
and
rocket
got
together
and
they
said
we
need
to
work
together
to
come
up
with
some
standards.
A
So
they
came
up
with
a
open
container
initiative
and
then
Russy,
which
led
up
to
OCI
is
the
first
OCI
compliant
runtime
started
working
2014
10
in
2016
kubernetes
or
the
kubernetes
community
came
out
with
CRI
container
runtime
interface
because
they
had
to
do
a
lot
of
modifications
for
to
make
rocket
work
on
kubernetes.
So
they
said,
if
there's
any
new
container
runtime
they're
gonna
have
to
implement
this
interface,
and
so
kubernetes
doesn't
have
to
care.
A
You
know
what
it
needs
to
you
know,
call
out
in
the
back
to
run,
containers
and
some
of
the
the
CRI
schemes
available
are
cry
or
rocket
for
rocket
frack
tea
for
VMs
and
CRI
container
D.
So
what
does
it
look
like
now
in
we've?
Seen
something
like
this
and
you
have
all
these
different
containers
going
all
over
the
place.
You
know
you
don't
know
where
it's
going.
You
know
everybody
is
creating
its
own
container
runtime.
So
it's
a
little
bit
spinning.
A
So
the
landscape
looks
like
this,
so
you
have
open
bc
by
itself
and
you
have
a
looksie
also
by
itself,
then
you
have
missiles
with
its
own
container
riser
running
on
top
of
ab
c
and
then
also
meso,
supporting
docker
and
possibly
supporting
LCI
in
the
future.
And
then
you
have
kubernetes
with
all
this
different
runtimes
and
shims.
You
know,
Rockland,
docker,
cryo
and
cryo
can
run,
run
CD
or
cotta
containers.
You
can
run
queries
to
the
doctor
same
and
run
other
OC
I
complain
run
time,
so
it's
a
little
bit
confusing.
A
A
A
So
let's
talk
about
message
so
messes,
first
implementation
of
c
group
since
+,
+,
+,
Caprio
performance,
it
supports
ocular,
app,
see
images
and
it
potentially
will
support
OC
I.
But
the
word
is
that
they'll
latch
on
to
whatever
your
doctor
does
so.
The
best
use
for
from
Essos
is
kind
of
like
spark
and
flink
frameworks
for
big
data
applications,
not
that
it
cannot
be
used
for
other
types
of
things,
but
different.
A
The
containers
are
particularly
particularly
good
for
those,
and
the
cons
is
that
you
can't
run
these
containers
outside
meso,
so
you
need
to
mess
with
framework
to
make
this
containers
run.
So
let's
talk
about
Alexei,
so
this
is
the
first
time
or
the
the
I
came
across
containers
myself,
and
this
is
what
the
page,
the
main
LXE
page,
looks
like
in
2012.
So
up
until
then,
everybody
was
talking
about
virtualization
and
there.
A
A
The
manages
your
containers
and
your
images
written
go
and
LX
fuse
which
matches
the
file
system.
Is
it
all
CI
compliant?
No,
it's
not
o
CI
compliant,
but
the
word
is:
it
is
out
there
that
they're
working
on
an
l,
XE
run
that
will
be
o
CI
compliant
I.
Couldn't
find
a
lot
of
information
and
LX
e
run,
but
I
was
able
to
take
something
happen.
That's
what
it
looks
like.
So
some
of
the
downsides
is
that
it's
got
its
own
image
image
standard
and
there's
not
a
lot
of
adoption.
A
A
Let's
talk
about
rocket
so
rocket
supports
two
types
of
images:
it's
just
a
single
process,
runtime
our
KT
rocket.
It's
a
dosing!
I
compliant!
No,
it's
not
o
CI
compliance
still
in
development.
They
plan
to
do
it
through
something
similar
to
run.
See
what's
interesting
is
that
the
people
who
worked
on
rocket
also
work
with
docker
to
create
run
C,
but
they
had
they
didn't
get
around
and
incomplete
or
CI
compliance
to
rocket,
and
it's
got
some
unique.
A
This
is
how
you
run
rocket
with
kubernetes
aka
rock
Nettie's,
so
you
would
just
start
the
cubelet
and
you
specify
you
contain
a
runtime
to
rocket
so
some
of
the
pros
it
works
out
the
pirate
box
with
kubernetes.
It's
got
this
pot
basin
metrics
very
similar
to
kubernetes
just
a
single
process.
It
supports
KVM
to
run
workloads,
so
it
supports
docker
FC,
maybe
OCR
in
the
future,
and
some
of
the
cons
is
the
core
OS.
A
A
company
behind
rotted
got
just
got
acquired
by
Red,
Hat
and
they're,
not
so
focused
on
runtimes
that
much
anymore
they're
more
focused
on
kubernetes
distributions
Rockland.
The
project
that
is
supposed
to
implement
the
CRI
for
rocket
is
still
under
development
and
there's
not
a
lot
of
adoption
in
production.
A
Some
people
are
using
it
to
like
galaxy,
but
but
it's
not
it's
not
massive
and
their
standard
image
format.
Fc!
It's
not
it's
no
longer
in
development,
so
they
decided.
The
OCI
is
the
future,
but
they
haven't
gotten
around
supporting
it.
So
let's
talk
about
docker,
so
the
first
name
was
just
docker
and
now
it's
all
these
different
things.
You
have
darker
demon,
Runcie,
container
D
and
the
CRI
container
d.
A
So
this
year,
I
continually
just
recently
a
couple
of
months
ago
became
just
a
CRI
plug-in
and
the
continuity
daemon
handles
your
story
to
imaging
your
network
in
and
it
takes
care
of
of
stenchy
8
in
your
containers.
So
this
is
what
it
looks
like
in
the
old
days.
This
is
from
the
main
container.
The
page
you
have
dr.
management,
just
search
the
imaging
and
your
network
in
and
then
on
the
new
container,
the
one
point
X
you
have,
those
functions
moved
into
the
container
D
demon
and
they
it
was
still
a
container.
A
These
still
instantiates
their
containers.
Ok,
so
let's
talk
about
C
all
right
now,
CRI
is
just
an
interface.
It's
not
this
container,
the
ECI
continuity
or
or
the
CRI
plug-in.
So
that's
a
little
bit
confusing,
but
you're
serious.
It's
just
the
interface
introduced
in
2016,
so
that
kubernetes
can
talk
to
multiple
runtimes
and
it's
just
a
G
RPC
interface
and
the
shims
are
available
or
cry
or
Auklet
in
development.
Frak
t
to
run
VMs
in
like
I
mentioned
before,
see
I
plug-in.
A
So
this
is
what
container
D
looked
like.
Maybe
two
months
ago
you
had
an
extra
daemon,
CRI
container
d.
The
cube
light
would
talk
to
it
using
G,
RPC
and
and
then
this
demon
will
have
an
image
service
and
a
runtime
service
and
then
would
talk
to
another
demon
called
container
D
through
G
RPC
and
that
demon
would
instantiate
all
the
parts
and
the
containers
a
little
bit
confusing.
So
this
is
what
it
looks
like
now
that
may
be
from
a
month
ago.
So
now
you
just
have
a
single
demon.
A
A
You
actually
don't
have
to
configure
anything
because
the
default
runtime
is
run
C.
But
if
you
want
an
untrusted
workload
runtime
you
can
configure
that
and
that
it
could
be
something
like
cotta
containers
or
clear
containers.
You
can
also
specify
another
runtime
instead
of
run,
see
if
you
don't
want
to
run
the
default.
This
is
how
you
start
container
these.
A
So
just
a
system
start
you
demon,
and
then
you
start
your
cubelet
with
saying
that
your
containing
the
runtime
is
remote
and
you
point
it
to
the
container
the
demon,
the
demon,
the
the
sorry
they
could.
You
point
it
to
the
socket
so
that
socket
it's
listening,
GRP
c
and
c
RI,
and
then
you
can
also
set
it
up
as
a
as
a
system
d
service
in
the
cubelet.
That
is
so
some
of
the
pros.
It's
very
widely
adopted
very
stable.
A
lot
of
companies
are
using
it
pretty
good
performances
we
didn't
go.
A
The
community
is
pretty
big,
the
the
there's,
a
lot
of
commercial
support
about
companies
behind
them,
running
in
thousands
and
millions
of
environments,
and
what
are
the
interesting
things
that
their
work
is
windows,
containers
of
a
Windows
and
Linux
containers
over
windows.
Some
of
the
cons
is
that
the
runtime
meaning
container
D
is
very
couple
with
the
the
daemon,
the
continuity
demon.
A
A
Let's
talk
about
cryo
cryer
is
just
a
shame.
It's
not
an
actual
runtime.
It's
just
a
shame
that
allows
coordinators
to
talk
to
your
right
to
another
container
runtime.
So
it's
nickname
is
cryo
like
cryogenics,
like
the
logo
and
main
components
is
cryo
daemon
that
it's
very,
very
similar
to
container
D.
It
handles
the
storage,
the
imaging
the
networking
and
the
cube
latakia
right
to
it.
It
has
this
other
component
called
the
common
demon
that
is
very
similar
to
the
darker
shim
that
keeps
your
container
up
and
running,
and
this
one
monitors
your
container.
A
This
happens
to
be
written
in
C
and
is
it
OC
I
complain?
Yes,
it
is
so
CI
compiling
soup
and
you
can
run
run
C
or
any
OC.
I
complain
on
runtime,
and
this
is
what
it
looks
like
at
a
high
level.
You
have
the
cube,
let
talk
your
proceed
to
the
cryo
demon
and
the
cryo
demon.
Instantiating,
your
parts
and
your
containers
and
each
container
be
monitored
by
this
common
demon
and
this
common
demon
is
it's
managed
by
system
D.
So
this
is
how
you
start
cryo.
A
You
would
cry
or
demon
and
then
very
similar
to
container
D.
You
specify
your
container
runtime
to
be
remote
and
you
point
it
to
the
cryo
socket
that
it's
listening,
ER
PC
CRI
and
some
of
the
pros
is
kubernetes
ready.
It's
got
great
performance.
The
it's
written
ago.
There's
three
demons
by
default
package
package
for
kubernetes,
and
this
common
demon
that
I
mentioned
is
written
in
C.
It's
actually
monitors
the
containers.
So
whenever
you
restart
cryo,
your
containers
don't
get
restarted
as
opposed
to
container
D
and
Dockers.
A
A
It's
demoing
this
tool,
and
so,
if
you're
interested
you
can
find
out
more
about
it,
it's
called
Putman
and
there's
no
default
for
image
management
as
well,
but
they
also
have
another
tool
in
the
worst
call
builder
or
you
can
also
use
third-party
tool
like
third-party
tool
like
Scorpio
or
emoji,
which
are
OC.
I
met
image
management
tools.
Let's
talk
about
sear
on
this
is
another
runtime.
So
when
did
it
start
started
in
last
year,
2017
by
Red
Hat,
it's
mainly
implemented
in
seed
and
it's
got
the
best
performance.
Is
it
OCI
compliant?
A
Yes,
it
is,
and
the
downsides
just
it's
just
pretty
new
there's,
not
a
lot
of
adoption.
So
it's
a
work
in
progress.
These
are
some
benchmarks
from
their
page.
So
if
you
run
a
hundred
times
been
true
with
no
new
network
namespace,
it's
about
40
persons.
Forty
point,
seven
percent
faster
than
run,
see
if
you
run
the
same
with
new
network
namespace,
is
about
sixteen
point.
Five
percent
faster.
Now,
let's
talk
about
kata
containers
when
they
start
start
started
last
year
by
the
OpenStack
foundation.
Its
first
release
is
slated
for
next
month.
A
So
what
is
it?
It's
two
projects
getting
together.
Rugby
and
clear
containers
so
run
V
from
hyper
data
sage
and
clear
containers
from
Intel,
and
basically
what
they're
trying
to
do
is
run
containers
in
Liam's.
The
remain
implementation,
listen
go.
The
security
is
supposed
to
be
the
best.
The
best
isolation,
you're
running
completely
isolated
in
a
VM,
and
they
said
Oh
see
I,
complain
yesterday,
so
CI
compliant.
This
is
what
it
looks
like.
You
have
run,
V
and
clear
containers.
A
So
Kara
is
supposed
to
support
both
of
these
types
of
environments,
where
you
run
frak
T
or
were
you
where
you
run
it
with
cryo
and
in
cryo
runs
another
continuing
runtime
and
which,
in
this
case
is
clear
containers,
but
it
will
be
Kara
containers,
and
this
is
Kara
with
docker
and
comparison
to
run
C.
So
you
see
that
it's
not
very
much
different.
So
it's
pretty
much
the
same.
A
You
have
your
run,
see
at
the
bottom
and
just
you
just
plug
in
your
cat,
a
runtime
at
the
bottom-
and
this
is
configuring
cutter
with
docker
in
pretty
simple
your
new
container
D
conflict.
That
tunnel,
you
say
the
runtime
is
Kara,
and
this
is
cutter
with
you
container
these,
your
I
plug
in
and
use
specified.
You
don't
trust
a
workload
runtime
to
be
cara,
cara
and
kubernetes.
So
you
see
or
OC
I
complain
runtimes.
A
So
you
can
see
you
can
use
any
CEO
right
sim,
like
so,
I
could
see
all
right,
plug-in
just
got
renamed
or
cry
or
or
you
guys,
frac
T
or
you
can
use
this
darker
sim
and
then
you
can
run
cut
out
in
the
back
and-
and
this
is
shows
cut
out,
running
with
cryo
and
alongside
and
a
trusted
workload
that
will
run
with
run
C
and
I.
Don't
trust
the
workload
that
would
run
with
cut
out
inside
of
the
app
cutter
and
cryo.
A
You
just
change
your
or
add
this
to
your
configuration
to
cryo
that
kampf.
You
point
it
to
the
cut
of
runtime
binary
and
you
specified
your
workload,
trust
to
be
untrusted,
and
this
is
Kara
with
frac
T.
So
you
you
start,
you
start
up
your
factory
demon.
Then
you
started
your
queue,
blade
and
just
say:
remote
container
run
time,
and
he
pointed
to
the
practice
socket
and
then
you
restart
this
hyper
hyper
d.
A
Demon
that
is
supposed
to
practice
was
to
talk
talk
to
to
to
banish
the
virtual
machines
and
then
so
the
cutters,
the
cutter,
the
the
pros
are
that
he
provides
great
isolation.
It's
got
the
best
security.
It's
got
good
performance,
not
the
fastest,
but
your
performance
it
does.
The
ends
have
been
around
for
quite
a
while,
so
they're
pretty
stable
and
you
could
potentially
use
like
stateful
type
of
applications
like
where
you
attach
your
your
storage
to
a
VM,
and
maybe
you
want
to
attach
that
on
an
encrypted
fashion.
A
Or
maybe
you
want
to
store
some
security
information.
Some
database
security
information
might
be
pretty
useful
for
those
type
of
applications
and
some
of
the
cons
you
need
a
bare
metal
machine.
So
a
lot
of
the
cloud
providers
have
VMs.
So
AWS
just
recently
announced
bare
metal
machines,
but
they're
still
in
preview,
so
like
we
have
an
account
and
we
haven't
been
able
to
create
to
to
have
our
metal
machines
on
AWS
and
I.
Try
to
run
this
and
see
five
instances,
but
I
wasn't
able
to
run
it.
A
The
C
files
has
the
new
KVM
hypervisor
supposed
to
that.
You
can
run
nested
virtualization,
but
I
wasn't
able
to
run
it
and
there's
some
issues
with
the
startup
time.
But
there's
a
couple
of
features
from
that.
The
catacomb
unity
is
working
on
like
VM
templating
and
allowing
your
VM
to
startup
with
no
firmware
and
for
now
it's
actually
slower
than
something
like
run,
C,
so
other
notable
runtimes.
A
So
there's
a
bunch
so
there's
an
NVIDIA,
runtime
and
just
a
specific
to
GPUs
and
it's
a
modified
version
of
run
C
and
it
talks
to
this
live
Nvidia
container
and
it's
written
in
C.
And
then
it
talks
directly
to
the
GPUs
there's
also
a
rail
car.
It's
a
no
CI
implementation
and
rust.
There's
pouch
alibaba's
runtime.
It's
just
a
shame
that
uses
run
C
in
the
back
their
system
and
spawn
system.
Dn
spawn
in
it's
a
very
lightweight
isolation
mechanism,
not
quite
like
a
runtime.
It
just
provides
some
namespace
isolation
and
there's.
A
Let
me
contain
that
for
you
from
Google
and
that
just
that
got
open
source
in
2015,
but
it's
not
longer
being
developed
on
because
they
have
excuse
me
they're,
focusing
more
on
OCIO,
CI
runtimes.
So
a
brief
water
and
unique
kernels.
So
a
lot
of
people
are
there,
there's
been
a
lot
of
buzz
about
unicorns
and
people
have
said.
Oh,
this
is
the
next
best
thing
after
containers.
So
it's
not
really
clear.
That's
gonna
be
the
case,
so
people
are
saying
that
once
you
instantiate
your
unit
kernel,
you
cannot
reduce
your
memory.
A
There
are
no
tools
to
debug
like
but
where's
PS,
eh
top
or
top,
and
you
have
to
learn
maybe
specific
languages
that
come
prepackaged
with
your
unit
kernel
and
the
ones
currently
available,
as
some
like
Haskell
Oh,
camel
and
airline.
So
you
have
to
learn
those
languages.
If
you
don't
know
them
so
I
mean
they're
sort
of
popular
but
they're
not
as
popular
as
the
mainstream
languages.
A
A
Maybe
the
CRI
plug-in
might
have
an
advantage
because
they
have
the
docker
container,
did
code
base
and
they've
been
around
longer.
If
you
want
some
good
performance,
you
can
use
something
like
kata
through
an
even
better
performance.
You
could
use
anything
with
run,
C
or
rocket.
Both
of
them
are
written
and
go
so
they're
pretty
fast,
and
if
you
want
the
best
performance,
you
can
just
see
run
or
you
might
have
to
wait
until
it
becomes
widely
available.
A
If
you
have
a
mixed,
kubernetes
and
meses
environment.
Just
like
we
do
a
branch
you
might
stick
to
docker
and
continuity,
because
it's
supported
by
both
environments
by
vessels
and
kubernetes.
If
you
have
a
lot
of
big
data
type
of
applications
like
spark
and
flink
you,
you
probably
want
to
use
messes
with
docker
container
here,
not
with
messes
containers
but
docker
container
d
kubernetes
just
release
a
spark
controller,
but
still
in
the
works.
It's
not
very
production
ready.
So,
but
if
you
want
to
use
production,
ready,
spark
or
fling,
you
might
stick
to
messes.
A
So
if
you
learn
good
security
and
isolation,
well,
anything
will
run
C
or
rocket
rocket
and
might
have
a
little
bit
of
an
advantage
there
with
trusted
platform
modules.
Tpms
but
run
C
also
provides
all
the
all
the
different
nitty
gritty
like
Linux
as
you
come
up,
armored
Linux
capabilities
and
so
forth.
And
if
you
want
the
best
security,
you
might
want
to
wait
for
the
first
release
of
kara
containers.
A
So
what
does
the
future
look
like,
so
I
see
more
and
more
of
these
runtimes
converging
and
conversion
into
o
CI,
and
not
only
o
CI,
but
all
the
other
standards
like
container
networking
interface
and
container
storage
interface.
As
far
as
your
is
concerned,
kubernetes
will
use
that
as
a
default
to
talk
to
all
the
different
container
runtimes,
then
docker
will
adopt
the
the
COI
plugin
for
kubernetes
to
talk
to
it
so
currently
with
the
dark
room.
The
current
dr
implementation
of
kubernetes
is
more
like
it
directly
talks
to
the
dr.