►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Let's Try Every CRI Runtime Available for Kubernetes. No, Really! - Phil Estes, IBM
The CRI (container runtime interface) in Kubernetes—designed to abstract the host’s runtime details from the kubelet codebase—has been around for a few years now. CRI implementations beyond the pre-existing Docker engine driver have been appearing for some time now. As of early 2019, Docker, Virtlet, containerd, cri-o and Sylab’s Singularity project all implement the CRI! If you add in Kata containers and AWS Firecracker, the unique number of CRI combinations is growing rapidly. How would you decide which CRI implementation is right for you? Clearly each have tradeoffs that are worth understanding whether you are a developer or operator. We'll live demo each CRI runtime and summarize the details and why a cluster might choose—or not choose—a particular runtime. It will be a fast-paced but hopefully informational talk for those looking to understand the CRI runtime landscape!
https://sched.co/MPdB
A
All
right
thanks
everybody
for
coming
out
after
lunch
on
the
last
day
of
coop,
con
selfie,
oh
wow,
actually,
maybe
at
the
end,
because
we've
got
a
lot
to
get
through
and
I,
don't
know
if
we
can
pull
it
off,
but
so
no
intro,
you
heard
who
I
am
what
I
want
to
do
is
give
you
a
very,
very
quick
whirlwind,
tour
of
why
we
are
where
we
are
who's
been
in
this
room
before
lunch
at
either
of
the
two
presentations
yeah.
Not
enough.
So
we
do.
A
We
have
to
go
through
this
because
actually,
both
prior
speakers
did
a
great
job
of
kind
of
walking
through
where
we
are,
but
really
the
OCI
open
container
initiative
has
brought
us
to
this
place
where
a
multiple
OCI
compliant
run
times.
We
have
compliant
registries.
The
distribution
spec
is
being
worked
on
this
year
in
the
OCI,
and
so
we
have
this.
This
set
of
runtimes,
which
is
both
great
in
one
sense,
but
also
can
be
confusing.
You
know
what
do
I
use.
Why
do
I
make
these
choices?
A
And
so
the
idea
for
this
talk
was
to
give
you
again
a
very,
very
fast-paced
kind
of
walk
through
all
your
options.
I
can't
tell
you
what
to
choose,
but
hopefully
maybe
through
the
the
work
I
did
to
try
and
get
this
all
set
up
and
working.
It
may
give
you
some
insights
as
far
as
you
know
why
why
various
runtimes
exist?
Why
they've
been
created
and
really
following
the
OCI
kubernetes
said
hey.
This
is
going
to
be
crazy,
trying
to
support
all
these
runtimes
in
the
codebase.
A
A
Obviously,
the
runtime
has
certain
responsibilities
that
really
are
around
the
pod
lifecycle
itself,
starting
pods,
stopping
pod
status.
Pulling
images
do
I.
Have
this
image
where
the
logs
give
me
a
you
know
exactly
a
process.
Everything
else
is
handed
by
kubernetes,
all
the
orchestration.
Of
course.
This
pictures,
even
you
know
not
quite
correct,
there's
other
little
sub
pizzas,
whether
we're
talking
to
CNI
or
CSI
for
storage.
But
again,
that's
all
outside
of
the
runtimes
purview.
A
A
A
You
know
if,
if
you
ever
searched
the
nailed
it
fails.
This
is
an
example.
You
know
you're
gonna,
take
this
amazing
picture.
You
saw
somewhere,
but
in
reality
it's
a
lot
Messier.
So
that's
what
you're
gonna
see
today.
I
have
multiple
clusters:
I've
got
gke
to
show
to
know
cluster
to
show
docker
again,
not
that
exciting
we've
been
using
docker
as
a
kubernetes
runtime
for
a
very
long
time,
but
again
just
to
just
to
fulfill
my
promise.
A
That's
where
I'll
show
cata
and
G
visor
I
have
another
single
node
VM,
with
with
creo
I
used,
cube
ATM
to
install
that,
and
then
someone
in
IBM
cloud
said:
hey
I've
been
messing
around
with
the
with
the
community
version
of
OpenShift,
so
I've
got
a
three
node
cluster,
so
I
also
have
that
with
creo.
So
these
are
the
pieces
I'll
be
showing
to
give
you
kind
of
that
that
quick
demo
of
each
each
runtime
and
then
at
the
end
of
each
I'll
kind
of
give.
A
You
there'll
be
a
slide
with
a
lot
of
words
on
it.
I'll
probably
try
not
to
read
all
the
words
I
will
share
it
online,
so
you
can
easily
reference
that
after
the
talk
so
first,
obviously,
as
I
said
docker
since
the
beginning
of
the
kubernetes
project
has
been
that
default.
Runtime
that
that
handles
this
pod
lifecycle
for
your
containers
that
get
created
and
started
again,
I
said
I
would
I
would
demo
all
of
them.
So
if
I
go
here
and
show
you
this
gke
cluster,
we
should
see
two
nodes.
A
As
I
said
at
the
very
end,
I
know
this
is
messy.
I
tried
to
make
the
fun
big
enough.
Is
this
going
to
work
for
those
in
the
back
partially
I
can
even
make
it
bigger.
So
at
the
very
end
of
this
raft
line,
which
is
a
little
messy,
you
see
docker
89.3
so
again,
gke
by
default.
When
you
create
a
kubernetes
cluster,
you
end
up
with
docker
engine
you
can
also
during
that
gke
process
of
creating
a
cluster.
A
You
can
pick
the
container
d
image
and
you
can
actually
use
container
d
and
I
don't
speak
for
google,
but
I
believe
they
will
be
maybe
moving
fully
to
container
d
in
the
near
future,
like
we
do
in
IKS.
At
this
point
we
can,
we
can
obviously
deploy
a
simple.
You
know:
engine
X,
Y
Amal
and
see
that
you
know
we
have
pods
running
running
engine
X.
We
know
that
running
of
a
docker
engine
I'm
not
going
to
access
the
worker
nodes
and
prove
that
but
I'm
sure
you
believe
me
on
that.
A
A
So
again,
docker
being
up
in
that
worker
node
makes
it
simpler
for
tools
that
may
be
run
outside
of
the
kubernetes
infrastructure
and
just
use.
You
know,
docker
commands
and
again,
as
people
have
migrated
away
from
docker
to
other
CRI
implementations.
That's
definitely
been
an
interesting
challenge
to
find
that
you
know.
Basically,
some
tuples
need
to
to
be
reworked
to
not
expect
the
docker
engine
or
the
docker
api
socket.
A
Definitely
in
some
sense
it's
more
than
enough
engine
for
kubernetes.
There
was
a
talk
here
just
before
lunch
about
a
company
migrating
to
container
d
and
making
the
comment
that
it's
really
more
than
than
what
we
needed
and
then
over
over
time,
and
what
really
brought
about
some
of
the
other
CRI
implementations
with
were
these
concerns
over
Dockers
lifecycle,
versioning
and
kubernetes,
and
kind
of
the
synchronization
between
that
and
you
know
which
engine
version
would
actually
work
with
which
kubernetes
version
and
obviously
because
the
docker
engine
has
more
capabilities
and
components.
A
Extra
memory
and
CPU
use
container
D.
So
again,
this
is
the
project
where
I'm
a
maintainer.
We
have
several
other
maintainer
z'
here
in
the
room
and
if
you're
interested
in
a
lot
more
detail
about
container
d,
the
deep-dive
session
is
this
afternoon:
I
think
355
in
room
e4,
but
it's
on
sked.
You
can
find
it
and
easily
go
to
that.
So
if
you
want
to
hear
a
lot
more
about
container
t's
architecture,
how
it's
used,
you
can
go
to
that
session
again.
I
will
demo.
A
A
We
can
deploy
our
same
exact
yamo
again.
The
value
of
the
OCI
here
is
that
I
don't
have
different
images
when
I
ran
on
GK's,
docker,
docker,
backed
cluster
and
so
I.
Don't
have
a
different
image
here,
and
so
obviously,
if
I
look
at
these
pods,
you
know
they're
up
and
running
they're
using
container
D
I
could
login
to
the
worker
nodes
and
prove
that
to
you
and
actually
for
the
next.
A
Several
when
we
look
at
the
sandbox,
VMs
or
VMs
and
other
isolation
techniques
we'll
actually
have
access
to
the
command
line
of
the
host
and
and
we
can
validate
what's
actually
running
but
again.
Suffice
it
to
say,
docker
container,
D,
very
straightforward
to
to
run
that
so
again,
container
DS
kind
of
current
status,
gke
you've
got
it
in
beta.
A
Interestingly,
every
docker
installation
also
includes
container
D,
so
every
in
a
sense,
every
public
cloud-
that's
using
docker,
is
also
using
container
D
and
by
nature
of
that
we
get
lots
and
lots
of
testing
the
platform.
You
know
even
those
millions
of
developers
who
install
docker
desktop
that's
also
using
container
D.
A
Obviously,
a
value
of
continuity
being
smaller
is
that
you
memory
CPU
usage
and
it's
also
got
a
really
nice
clean
API,
which
again
you
could
hear
more
about.
If
you
go
to
the
deep
dive,
it's
obviously
without
the
docker,
tooling,
an
API
socket.
You
have
this
migration
complexity
for
a
plugins
and
tools.
So
IBM
we've
worked
with
a
lot
of
our
our
partners
and
IBM
cloud,
so
you
may
have
seen
announcement
Cystic
sports
container
d
or
assisting
sports
cryo.
A
A
So,
let's
look
at
this
one
I
have
here:
I
have
a
VM
and
I
tweeted
this
the
other
day
just
to
to
be
goofy,
but
my
VM
was
originally
for
container
D,
and
so
it's
host
name
is
container
D,
but
it's
running
creo.
So
that's
just
your
your
container
runtime
joke
for
the
day.
So
again,
I'm
running
again,
a
single,
a
cluster
I
brought
up
with
cube
ATM
I
also
have
access
to
that
cluster
I
mentioned.
A
And
yeah
here
you
can
see
the
output
of
that
it's
running
again,
the
okd,
the
community
distribution
also
running
creo
1.11.
So
again
it's
a
little
bit
older
than
my
cube
ATM
up
cluster
here.
So
again
we
can.
We
can
do
the
same,
and
your
next
example
and
see
that
we
should
have
some
nginx
pods
again
running
here
now
this
I'm
actually
on
the
node.
A
For
this
single
node
cluster,
again
creo
is
used
in
OpenShift.
It's
sousei
uses
in
their
container
as
a
service
platform.
There
are
other
customers
and
uses
of
creo
I
believe
there's
an
end
user
in
the
CAF
community,
who
has
mentioned
that
they're
using
creo
as
well
again,
the
creo
team
has
promoted
as
all
the
runtime
queue
beneath
these
needs
and
nothing
more
and
has
this
idea
of
kind
of
concerns.
A
So,
instead
of
all
the
pieces
being
in
a
single
engine,
there's
pod
man
there's
build
a
scope
EO,
and
so
they
all
work
together
on
the
same
back
end
now
again,
because
it's
meant
primarily
for
kubernetes.
So
again
it
doesn't
have
necessarily
tools
of
its
own,
like
a
client,
so
you
use
the
other
tools
as
a
again
as
a
design
choice
I
found
that
you
know
use
an
installation
of
it.
Unlike
a
bun
toot,
it's
because
that
VM
was
a
bun
too,
can
be
complicated.
Again.
The
team
is
aware
of
that.
A
A
So
if
you
had
a
workload
that
you
wanted
a
higher
level
of
isolation
on,
you
could
basically
say
this
is
an
untrusted
workload
and
then
the
way
you
had
set
up
your
cluster
is
that
you
either
use
cata
or
one
of
the
other
sand
boxes
as
the
implementation
of
that
untrusted
runtime
runtime
class
brought
more
flexibility
and
that
you
could
add
multiple
runtimes
with
a
name
and
a
handler,
and
so
at
that
point
you
could
then
begin
to
to
basically
specify
in
a
pod.
This
pod
should
work
under
this
run
time.
A
You
use
this
runtime
here
or
to
create
and
start
this
pod,
and
so
that's
where
we're
going
to
show
here
in
a
minute
before
I
did
that
I
thought
this
picture
from
the
Cata
folks
was
helpful.
That
again,
why
we
added
the
v2
shim
API.
In
a
sense,
many
of
these
Sam
boxes
were
seen
as
just
run,
C
replacements.
A
A
De
she
meant
a
ssin
for
your
runtime,
which
again
in
this
case
cata,
could
then
have
that
direct
link
to
their
agent
to
talk
to
inside
the
VM
and
again
so
this
simplified
the
picture,
and
so
now
you'll
see
in
a
minute.
We
have
many
shims
available
for
container
D
to
run
the
various
sand
boxes,
so
the
first
one
I
will
show
is
cata.
A
A
set
of
shims
that
includes
a
DBS
firecracker.
It
includes
the
standard,
v1
and
v2,
run
C
shims
and
includes
run
SC,
which
is
for
G
Visor
and
also
the
the
cata
shim
as
well.
So
again
on
this
node,
where
I
have
a
single
node
cluster
container
D
is
the
runtime,
but
container
D
can
drive
these
other
shims
by
either
specifying
it
in
the
container,
D
client
or
by
using
runtime
class,
which
is
what
we're
going
to
do
so
again
to
just
show
you
what's
what
we
have
here
again.
A
First,
obviously,
it's
container
D,
so
if
I
don't
specify
our
runtime
class,
then
container
D
is
actually
going
to
run
that
container
and
we
should
be
able
to
see
that
if
I
use
the
CTR
I
won't
see
it
if
I
just
ask
container
D
for
a
list
of
containers
because
container
D
is
name
spaced
and
so
container.
D
on
this
particular
node
is
being
shared
by
docker
and
kubernetes
and
whatever
other
uses
I
might
want.
So
if
I
do
namespaces
LS,
you
can
see.
A
A
A
So
as
I
said,
there's
there's
many
things
running
because
although
I
do
have
the
nginx
pod
I
just
started
here,
there's
also
the
kubernetes
8
again
a
single
node
cluster
I've
got
the
API
server
running
core
DNS,
etc.
So
you
can
see
all
those
containers
at
this
point.
I
will
register
the
runtime
classes,
and
so
we
can
look
at
one
of
them,
so
the
runtime
class
4G
visor,
for
example,
very
simple,
very
simple,
yamo
I'm,
creating
a
runtime
class.
The
name
is
G
visor
and
the
handler-
and
this
is
in
container
DS
config.
A
A
A
A
A
A
Yep
sorry
I'm
trying
not
to
touch
so
if
I
do
D
message
in
the
container.
This
would
never
happen
if
we
were
using
simple
Linux
containerization,
but
we've
seen
we've
actually
booted
a
kernel,
and
so
this
is
hopefully
our
proof
that
we've
run
katha
again.
There
was
a
great
talk
in
here
the
beginning
of
this
session
on
cata
that
you
know
has
the
history
of
coming
from
Intel,
clear
containers
and
the
hyper
dot.
A
Sh
project
uses
k,
vm
q
mu
base
isolation,
they
do
work
with
docker,
creo
and
container
D
and
so
again,
they're
part
of
the
OpenStack
foundation.
It's
growing
project
and
recently
they've
actually
added
the
ability
to
use
katha,
but
also
to
choose
between
the
q,
mu
k,
vm
or
the
Firecracker
rust,
based
vm,
and
so
again
you
could
use
katha
to
drive
either
fire
cracker
or
the
cat
isolation
for
cata
base
isolation.
They
support
quite
a
few
architectures,
including
arm
including
ibm's,
p,
&,
z,
series
and
so
again
this
is
a
interesting
project.
A
A
A
A
It's
basically
specifying
the
snapshot
and
the
runtime,
and
so
here
I
am
at
a
prompt
and
again
I
see
that
I've
booted
an
actual
kernel
again,
because
firecracker
has
done
that
in
the
background.
I
have
my
snapshot
of
running
and
it's
set
up
the
busybox
root
filesystem
and
provided
that
to
to
firecracker
and
again
other
than
that
it
looks
like
a
standard
container.
This
is
busybox.
A
So
again
this
is
firecracker
and
again,
as
I
said,
it's
a
fairly
young
project.
It's
got
a
rust
based
of
vmm
for
that
originally
from
google's
crosa
vm
project.
This
was
open
source
by
Amazon
just
late
last
year.
It's
a
young
project.
It
tightens
down
both
kernel
access
with
its
jailer
and
then
again
integrates
with
container
D
via
the
shim
that
we
just
saw
getting
it
packaged
and
delivered
I.
This
was
fairly
complicated.
You
have
to
set
up
a
root
filesystem,
you
have
to
put
the
agent
inside
of
it.
A
So
there's
a
lot
of
kind
of
manual
steps,
so
I
think
is
this
project
matures
they'll
be
better
kind
of
cleaner
delivery
of
how
can
I
actually
use
this
AWS
has
intentions
to
use
it
under
their
their
lambda
service.
Again,
this
higher
level
of
isolation
for
untrusted
code
coming
from
everyone's
functions:
G
visor,
let's
see
how
we're
doing
on
time
yeah
this
is
gonna,
be
tight,
so
G
visor
so
again,
I
have.
A
A
Again,
a
lot
of
information
out
there
about
G
visor,
it's
using
this
Colonel
written
in
userspace
concept,
it's
written
in
golang.
If
you've
heard
of
Google
cloud
run,
it's
actually
using
G
visor
as
this
isolation
security
boundary.
For
that
you
can
use
it
stand
alone
as
an
OCR
replacement
and
again
they
have
the
shim
that
we
just
demonstrated
and
again
they
were
there.
Reducing
cysts
calls
that
are
using
the
real
host
kernel
and
your
application
is
being
sort
of
routed
through
their
user
space
kernel.
A
Some
applications
may
not
work
if
they
haven't
implemented
the
Cisco
yet
and
there's
a
lot
of
good
detail
online
from
their
team.
Again,
there
is
a
KVM
mode
that
may
win
back
some
of
the
performance
loss
of
capturing
your
sis
calls
and
getting
them
true,
G,
visor
I,
don't
think
I'm
gonna
have
time
to
to
demo
singularity
singularity
again
as
a
high
performance
computing
runtime.
That
initially
was
an
OC
I
compliant
in
order
to
have
a
CRI.
But
in
the
last
few
months,
they've
added
those
capabilities.
A
It's
not
demon
based
it
sort
of
has
again
it
fits
this
HPC
model
of
sort
of
end
user
workload,
scheduling
so
a
big
national
lab
scientists
submit
their
jobs
via
sort
of
a
login,
node
and
so
singularity
matches
that
model.
Well
and
now
they
have
a
CRI
again
with
interest
in
kubernetes
in
the
HPC
community
in
the
in
the
science
community.
So
if
you
were
here
earlier
in
the
week,
you
saw
folks
at
CERN
demonstrating
using
kubernetes
in
their
experiments.
A
A
Runtime
here
so
again,
it's
listening
on
a
UNIX
socket.
If
you
told
couplet,
go
talk
to
this
socket
and
then
deployed
a
pod.
I
actually
do
haven't,
set
up
with
cry
CTL,
and
so
you
know
I.
Could
you
know
we
do
have
the
ability,
through
crises,
yell
to
run
a
pod
so
again,
I'm
not
using
the
kubernetes
api
I'm,
just
using
the
CRI,
API
and
so
I
can
run
nginx
and
you'll
see
that
you
know
the
demon
got.
That
request.
Did
an
OC
I,
create
of
a
container.
A
Again
I
said:
I
wasn't
going
to
demo
nabla
we're
already
very
very
close
to
time,
but
this
is
another
sandbox
idea
using
a
unit
kernel,
so
IBM
research
created
this.
It
uses
a
very
highly
limited
set
comp
profile.
Just
a
few
sis
calls
are
allowed
and
then
the
unit
kernel
is
compiled
with
your
application.
A
One
of
the
drawbacks
is,
then
that
you
actually
have
to
use
their
docker
hub
based
base
images,
because
they're
already
linked
to
these
user
unit,
kernel
libraries
so
again,
IBM
researches
is
trying
to
remove
that
limitation
as
we
speak,
because,
obviously
that
would
be
one
that
we
couldn't
use
the
same
nginx
image
we've
used
in
all
these
other
demos.
So
again
just
finalizing
the
OCI
has
gotten
us
to
a
place
where
we
have
this
common
underpinning
that
allows
innovation,
but
yet
have
interoperability.
A
We
didn't
have
to
change
container
images
to
use
all
these
different
runtimes
CRI
has
given
us
that
same
kind
of
plug
ability
in
kubernetes,
so
we
can
use
all
kinds
of
different
runtimes
under
kubernetes
and
then
runtime
class
gives
us
additional
flexibility
to
within
the
same
cluster.
Have
you
know
lots
of
different
uses
of
different
sandboxes
and
default
Linux
based
isolation,
so
with
that
I'm
gonna
say
thank
you,
I
think
I've
barely
kept
with
at
the
time,
and
if
you
have
questions
I'm
around
the
rest
of
the
day
and
be
happy
to
answer
them.