►
From YouTube: Who's Running My Pods? A Deep Dive into the K8s Container Runtime Interface - Phil Estes
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Who's Running My Pods? A Deep Dive into the K8s Container Runtime Interface - Phil Estes, IBM
Kubernetes is far and away the most popular project around container orchestration today. One little known fact to some is that Kubernetes itself has no code to run or manage Linux or Windows containers. So, what code *is* running the containers within your Kubernetes pods? Since Kubernetes 1.5 a new API contract, the Container Runtime Interface (CRI), allows any container runtime to plug into the kubelet configuration and provide container runtime services for Kubernetes. In this talk we'll look at the CRI options that exist today, and then deep dive into how the CNCF containerd project and it's CRI-implementing plugin work in concert with Kubernetes to run your pods. This will lead into a deep-dive at the command line, showing via live demos how Kubernetes, the CRI, and CRI-supporting runtimes like containerd work together to handle the container lifecycle operations.
To learn more click here: https://sched.co/FuKH
A
If
you
don't
know
why
you're
here,
hopefully
you're
here,
because
you
want
to
hear
about
the
kubernetes
CRI
interface
so
how
it
talks
to
container
runtimes
and
specifically,
first
half
I'll
talk
about
the
CRI
and
and
runtimes
second
half
I'll
try
and
demonstrate
and
make
it
practical
by
showing
container
D
as
a
runtime
for
kubernetes.
The
fun
part
of
this
demo
is
we're
going
to
talk
to
my
kubernetes
cluster
back
in
dallas,
texas
and
IBM
cloud
over
my
phone's
4G
connection.
A
A
More
importantly,
to
this
topic,
I've
been
contributing
to
container
runtimes
for
about
four
years,
starting
with
docker
and
the
whole
docker
ecosystem,
and
then,
in
the
last
couple
years,
container
D
I'm
a
maintainer
in
container
D
as
well
as
a
we
have
a
few
others
here.
In
fact,
if
you're
unhappy
with
how
deep
I
get
into
container
D,
you
have
another
chance
today
at
3:05,
Derek
is
over
here
on
your
the
far
corner
of
the
room.
A
Derek
and
Mike
Brown
Mike
Brown
is
also
from
IBM
and
a
maintainer
in
the
CRI
plugin
they're,
going
to
take
you
on
a
deep
dive
of
the
whole
container
D
architecture.
So
again
you
have
a
couple
chances
to
hear
more
today
about
that
so
you're
at
cube
con
Shanghai,
the
very
first
cloud
native
and
cube
con
on
the
continent,
and
so
maybe
it's
silly
to
ask
you
know:
are
you
using
kubernetes,
maybe
you're
not
yet,
but
you're
interested
or
you're
learning
this
week
about
it?
A
And
the
important
thing
to
note
is
that
kubernetes
is
really
just
that
orchestration
platform
above
a
container
run
time
and
that's
important,
because
kubernetes
doesn't
actually
run
your
containers.
So
there's
no
code.
If
you
look
through
the
entire
github
repo
for
kubernetes,
you
will
not
find
any
code
that
knows
how
to
run
to
start
a
container
on
a
Linux
or
Windows
system
to
handle
the
lifecycle
commands
of
creating
starting
stopping
containers.
For
many
years.
That's
been
done.
That
role
has
been
played
by
the
docker
engine.
A
Again,
you
can
look
back.
This
URL
here
is
to
release
1.4
of
kubernetes,
so
you
know
ancient
history
if
you're
keeping
up
with
kubernetes
1.11
1.12,
but
you
know
pick
an
old
release.
You'll
find
a
package
called
docker
shim
in
kubernetes,
and
the
docker
shim
was
implemented
to
know
how
to
talk
to
the
to
the
docker
engine
and
have
it
do
commands
like
docker
run
docker
start
on
behalf
of
kubernetes
as
it
created
pods
as
it
created
your
put
your
application
orchestrated
across
a
cluster.
A
So
again,
that's
that's
kind
of
been
the
life
cycle
of
kubernetes
again,
I.
Don't
have
a
lot
of
time
to
go
into
how
docker
has
itself
has
grown
and
matured
over
the
last
few
years
to
really
be
a
stack
of
components
with
the
open
container
initiatives
run
see
at
the
bottom
of
that
stack,
container,
d.a,
CN
CF
project,
as
you
know,
and
docker
driving
those.
So
again,
that's
been
kind
of
the
way
things
are
for
a
good
while,
but
back
in
2016,
and
actually
we're
coming
up
on
two
years.
A
A
This
is
a
rough
differentiation
between
the
role
of
what
kubernetes
still
has
to
do,
and
then
the
CRI
interface
effectively
separating
out
what
the
container
runtimes
job
is
underneath
kubernetes,
this
picture
isn't
perfect.
I've,
gotten
feedback
from
a
few
other
talks
that
you
know
there's
obviously
ways
in
which
some
of
these
come.
Some
of
these
roles
are
handled
on
both
sides
of
the
CRI,
but
it
gives
you
a
general
idea
that
kubernetes
is
still
handling
the
kubernetes
api.
It's
handling
storage
concerns.
A
You've
picked
a
C&I
plugin
to
handle
your
networking
kubernetes,
all
the
API
machinery,
around
health
checking
and
placement
all
the
orchestration
parts
of
kubernetes,
all
that
is
still
handled
by
kubernetes
on
the
when
it
wants
to
place
a
container
on
a
node
via
the
couplet.
Obviously,
the
things
on
the
right
are
things
that
your
container
runtime
is
going
to
have
to
implement
that
are
part
of
the
container
runtime
interface
such
as
all
this.
All
the
lifecycle
commands
you'd
expect
start/stop
delete,
also
image
management,
so
your
container
runtime
is
going
to
have
to
talk
to
registries.
A
Pull
pull
images
actually
push
is
not
correct.
You
don't
push
images
from
a
pod,
get
the
status
of
an
image
you
know.
Is
it
pulled,
get
to
get
all
the
layers
pod
status
and
all
the
related
parts
of
knowing
the
state
of
your
pods
and
containers
within
them,
and
then
there's
a
bunch
of
things
you
can
do
from
the
kubernetes
interface
side
like
I,
want
an
exact
into
a
container
I
want
to
attack.
I
want
to
assign
a
port
I
want
to
get
logs.
A
All
those
interactions
have
to
be
implemented
by
your
container
runtime
via
CRI,
so
that
when
those
kubernetes
api
czar
called
your
container
runtime
can
handle
the
runtime
aspect
of
those
operations.
So
again
that
gives
you
a
basic
idea
of
who's.
Job
is
what,
in
the
kubernetes
world,
between
the
runtime
and
the
kubernetes
stack.
A
So
now
that
we
have
the
CRI,
you
might
think
well
who's
implemented
this
API.
You
know
what
runtimes
are
available
underneath
kubernetes.
Obviously,
docker
shim
has
been
repurposed
to
implement
the
CRI,
and
so
you
you
know
many
clusters.
Today.
Many
production
clusters
continue
to
use
docker
as
the
runtime,
obviously
with
container
D.
The
project
I'm
associated
with.
We
have
a
CRI
implementation
that
handles
the
CRI
API
and
then
calls
into
container
D
to
again
do
all
those
those
runtime
steps
for
the
CRI
who's
heard
of
creo
from
Red
Hat.
A
But
if
we
fast
forward
today,
a
lot
of
that
has
been
is
now
handled
in
the
merged
projects
of
cata
containers,
which
I
think
you've
probably
heard
about
a
little
bit
this
week
already,
and
not
only
is
cata
associated
with
frac
D,
but
they're
working
with
both
cryo
and
with
our
team
in
container
D,
so
that
you
can
actually
use
cata
containers
on
other
runtimes
as
well
again.
The
couplet
has
ways
to
specify
which
runtime
do
I
want
to
use.
A
Treating
containers
like
a
black
box
will
eventually
leave
you
in
the
dark
and
I
hacked
it
to
say.
Treating
container
runtimes
like
a
black
box
will
eventually
leave
you
in
the
dark.
I
haven't
officially
gotten
his
approval
that
the
the
sense
of
the
tweet
still
stands.
But
what
what
I
mean
by
this
is
that,
even
as
an
application
developer
or
someone
at
a
higher
layer
of
the
stack
knowing
what
container
runtime
is,
is
their
and
what
its
capabilities
are
and
how
to
interact
with
it
can
be
valuable
and
definitely
to
your
operational
team.
A
There
may
be
benefits
to
container
D
or
cry
or
docker,
depending
on
other
needs
of
your
environment.
So
for
you
know
just
just
to
hypothetically,
you
know
talk
about
what
would
you
need
from
a
container
runtime?
You
probably
are
going
to
care
about
performance.
You
know
you've
heard
in
the
keynotes
all
the
interesting
workloads
that
people
are
placing
on
top
of
kubernetes
everything
from
a
I.
You
know
genomic
sequencing.
They
were
talking
about
this
morning.
A
Maybe
you
have
security
concerns,
maybe
you're
running
untrusted
workloads,
and
so
the
hypervisor
isolation
of
Katta
is
very
interesting
to
you
or
maybe
you're
interested
in
G,
visor
or
other
interesting
isolation
capabilities.
So
maybe
the
runtime
you
choose,
you
want
to
make
sure
it
has
those
those
those
options.
A
Maybe
you
care
about
multi
architecture
support,
so
at
IBM
we
have
power
systems,
we
have
Z
systems
and
so
making
sure
that
my
container
runtime
operates
in
those
environments,
maybe
you're
in
the
IOT
space,
and
you
have
arm
devices
or
other
cpu
enablement
that
you
want
to
make
sure
your
container
runtime
has
those
capabilities.
So
again,
those
are
things
you
might
think
about.
If
you
were
deciding
what
runtime
do
I
want
my
kubernetes
cluster
to
use.
A
So
let
me
talk
a
little
bit
more
about
container
D.
I.
Don't
have
enough
time
in
the
in
the
session
to
really
dig
deep,
but
as
I
mentioned
at
the
beginning,
and
some
people
have
showed
up
so
I'll
mention
it
again
later
this
afternoon
at
3:05,
there's
a
deep
dive
on
container
D
on
its
architecture
that
will
go
into
more
depth
than
I
can
but
effectively
over
the
last
few
years,
docker
has
contributed
code
that
used
to
be
part
of
the
monolithic,
docker
codebase
to
various
open
source
foundations.
A
So
you
have
run
C,
which
is
that
basic
unit
of
simply
executing
containers,
mostly
thinking
in
a
Linux
kernel,
interface
environment.
So
how
are
the
namespaces
and
C
groups
and
everything's
set
up?
That's
what
run
C
is
doing
for
you.
Container
D
has
been
donated
to
the
cloud
native
competing
foundation
so
again,
you've
seen
container
D
as
a
component
of
the
many
projects
within
the
CNC
F.
It
sits
on
top
of
run
C
and
basically
manages
those
run.
C
processes
for
you
and
then
in
our
recent
releases
under
the
CNC
F
has
image
management.
A
So
we
can
push
and
pull
from
registries.
It
fully
supports
OCI.
It
has
a
great
client
API,
so
we're
finding
many
users
who
love
container
D
as
a
way
to
be
embedded
into
other
runtimes
or
other
software
systems,
and
so
alabama's
pouch
container
is
also
a
user
of
container
D,
and
so
on
top
of
that
you
can
obviously
build
other
other
software
stacks.
A
So
the
docker
engine,
docker
seee
docker
for
mac
docker
enterprise,
they
all
use,
container
D
and
run
c
kubernetes
via
the
CR
I,
can
use
the
same
stack,
docker
swarm
and
their
orchestration
can
use
the
same
stack,
and
so
we
see
container
D
is
really
that
core
container
runtime
for
the
cloud
again.
Some
of
the
benefits
these
are
just
highlights
not
really
a
deep
dive,
but
really
we
intended
for
container
D
to
have
broad
you
in
the
industry,
so
projects
like
Linux
kid
and
kubernetes
Cloud
Foundry,
some
of
the
other
use
cases
I
mentioned.
A
We
expect
the
container
D
because
of
a
good
client.
Api
stability
performance
can
be
used
by
many
many
projects
as
a
great
container.
Runtime.
We've
also
spent
a
lot
of
time
doing
stress,
testing,
making
sure
that
container
D
is
stable,
that
we
can
meet
its
performance
criteria
and
so
Michael
Crosby
one
of
the
key
contributors
and
maintainer
zhan
container
d
tweeted
earlier
this
year
that
we
have
a
24/7
system.
A
That's
just
constantly,
starting
stopping
exacting
all
the
lifecycle
operations
into
container
D
and
running
millions
of
containers
every
week,
just
to
continue
testing
and
making
sure
that
every
change
that
gets
checked
in
we
haven't
impacted
the
stability
or
performance
of
container
D
and
again.
Another
aspect
of
what
we've
done
with
our
commitment
to
stability
is
also
a
stable,
API
and
and
stable
releases
with
very
specific
support
guarantees,
and
so
we
do
bug-fix
releases
with
you
back
ports
and
bug
fixes
into
old
releases.
A
This
was
announced
yesterday
morning,
Liz
Rice
mentioned
that
between
Copenhagen
and
this
event
that
container
D
integration
into
kubernetes
was
announced
as
GA
in
May.
That
was
actually
around
the
timeframe
of
coop
con
in
Copenhagen,
but
again
just
a
validation
that
the
implementation
of
the
CRI
and
integration
into
container
D
is
viable
for
use
cases,
production,
use
cases
in
your
cloud
or
in
your
software
platform.
So,
based
on
that,
you
know
who
is
using
container
D
today.
A
Obviously
one
of
the
interesting
places
you
can
go
hear
a
lot
more
about
container
D
being
used
as
Kelsey
Hightower's
kubernetes
the
hard
way
uses
in
all
the
examples
container
D
is
the
runtime.
We
have
two
public
clouds
that
are
currently
using
container
D
in
sort
of
early
preview.
Ibm
clouds
kubernetes
service
has
container
D.
If
you
select
kubernetes,
1.11
or
1.12
as
your
cluster,
you
automatically
get
container
D
as
the
runtime
and
then
down
and
below
that.
That's
the
google
kubernetes
engine
logo
they
have
in
beta.
A
Today
you
can
select
container
D
as
the
runtime
in
a
gke
cluster.
Again,
every
version
of
docker
for
the
last
few
years
uses
container
D.
So
every
time
you
install
docker
you're
also
getting
container
D
Linux
kit,
which
is
a
project
that
builds
Linux
distributions
purpose.
Optimized
Linux
also
uses
container
D.
Our
Rancher
has
a
project
using
container
D
cata
has
been
a
big
proponent
of
container
D
and
our
new
v2
shim
that
maybe
you
can
hear
more
about.
A
If
you
go
to
the
deep
dive
today,
I
mentioned
alibaba's
pouch
container
and
there's
a
couple:
I
Oh
T
projects,
ballina
and
Elliott
that
are
using
container
D
and
the
Cloud
Foundry
container
runtime
was
using
run
C
and
there
now
switching
to
container
D
as
well.
So
again,
the
purpose
of
making
sure
the
container
D
was
viable
for
many
use
cases.
A
We
feel
like
we're
seeing
that
come
to
fruition
with
a
lot
of
users,
a
lot
of
people
building
on
top
of
container
D
and
finding
it
valuable
for
their
model
of
needing
a
container
run
time
so
effectively
the
rest
of
the
time.
So
we
have
just
about
15
minutes,
I'd
like
to
demonstrate
kind
of
these
layers
and
before
I
do
that
I
thought
it'd
be
useful.
A
So
really
now
that
we're
talking
about
kubernetes
having
the
CRI
and
a
container
runtime,
you
can
think
about
talking
about
the
workloads
on
your
node
from
three
aspects.
One
is
the
cube
cuddle
commands,
which
is
you
talking
to
the
kubernetes
api,
so
I'm
going
to
use
the
client
the
standard
client,
something
about
a
command
like
cube,
cuddle,
get
pods,
and
so
that
asks
the
kubernetes
api
list.
My
pods.
A
A
A
My
secret
little
meth
of
getting
onto
the
so
now
I.
You
can
see
I'm
on
w2,
which
is
work
or
two.
So
if
I
go
back
to
this
other
picture,
you
know
that
should
match
this.
The
middle
entry
here,
a
certain
public
IP,
that's
the
actual
note
on
sshd
into,
and
so
obviously,
if
I'm
right
about
this
cluster
running
container
D
I
should
see
that
PS
looking
for
a
docker
process,
there's
nothing
on
the
system
running
docker.
A
I'm
sorry
that
I
made
the
text
big.
So
hopefully
everyone
can
read
it,
but
that
makes
the
output
not
quite
as
pretty,
but
these
are
obviously
the
list
of
of
pods
that
are
running.
There
are
many
pods,
because
there's
obviously
many
namespaces,
so
IBM
and
cube
system
and
it's
I
have
sto
deployed
here.
A
You
know
it's
a
huge
list
of
pods,
but
because
in
this
other
window
you
can
see
I
have
this
my
nginx
running
and
there's
four
of
them
and
I
only
have
three
nodes,
so
I
should
at
least
have
one
pod
on
this
node,
and
so,
let's,
let's
see
if
I
can
find
that
here
if
I
cry
cuddle,
pods
and
grep,
my
nginx
sure
enough.
It's
down
here
at
the
bottom
I
have
the
TV
for
7q
pod.
A
So
if
we
go
back
and
look
at
the
pods,
that
looks
like
the
very
last
one
here
that
the
my
nginx,
so
that
pod
is
running
on
my
worker
two
node
and
cry
cuddle
I've
been
able
to
use
I've
seen
from
coop
cuddle.
It's
told
me
that
these
pods
exist
I've
gone
to
one
of
my
worker
nodes
and
now
I'm
talking
to
the
CRI
socket
and
I've
asked
it
to
tell
me
the
pods
I'm
sure
enough.
My
nginx
pod
is
here.
Well
now,
let's
go
down
to
container
D
and
see
if
I
can
find.
A
I
think
I
already
exported
so
another
thing
that
maybe
you'll
learn
about.
If
you
go
to
the
deep
dive,
actually
I
don't
have
it
set
here
so
container
D
supports
namespaces,
which
means
you
can
actually
use
container
D
by
multiple
kind
of
higher
layer
users,
including
actually,
if
I
go
to
my
laptop
here.
You
can
see
that
if
I
talk
to
the
CTR,
that's
so
my
laptop
is
running
docker
18:09
that
just
came
out
last
week.
If
I
ask
the
CTR
on
my
on
my
laptop,
you
know
what
namespaces
do
you
have
you'll
see
it?
A
The
you'll
see
ka
desa,
I,
oh
and
Moby.
So
now
that
docker
is
using
my
system
container
D,
it's
creating
containers
inside
the
mobi
namespace.
If
I
had
a
Kubla
running
on
my
laptop,
it
would
use
the
K
dot,
IO
namespace,
and
so,
if
I
go
back
to
this
worker
node
and
do
CTR
namespaces
LS,
you
can
see.
I
only
have
a
que,
ésta
IO
I,
don't
have
docker
running
here.
A
A
Name,
space
equals
ka
dust
io
and
now,
when
I
do
CTR,
CLS
I
get
a
huge
lung
again,
it's
kind
of
ugly
output
because
of
the
text
size.
But
again
these
are
all
the
containers
that
container
D
knows
about
because
again,
couplet
has
talked
to
the
CRI.
It
started
pods
which
started
containers,
and
so
this
is
representing
that
list,
and
so,
if
I.
A
You'll
see
that
here
at
the
bottom,
I
have
a
long
container
ID
and
you
can
see
that
the
image
being
used
is
my
little
nginx
demo
and
it's
using
the
container
DV
1,
Linux,
runtime
and
so
again,
we've
mapped
all
the
way
from
cube
cuddle
get
pods.
So
we
saw
that
there
were
a
bunch
of
nginx
pods.
We
asked
the
CRI
to
list
pods.
We
saw
that
one
of
those
pods
is
on
this
worker
node
and
now
I've
gotten.
All
the
way
down
to
a
container
in
container
D
is
actually
running
the
nginx
there.
A
So
it
starts
3,
9
6:05
if
I
copy
this
and
then
task
CTR.
To
tell
me
the
tasks
that
are
running,
which
map
to
actual
processes
on
my
system,
I
can
find
that
there's
a
pid'
two,
six,
nine
seven,
seven
that's
actually
running
and
this
matches
that
container,
which
matches
that
pod
and
so
unless
I've
got
something
disastrously
wrong.
If
I
ask
the
system
to
list.
A
Processes
and
graph
for
that
pid'
it
better
be
an
engine,
X
process
and
sure
enough.
Well,
it's
got
F
CGI
running
and
engine
X.
So
this
is
my
actual
containers
process
running
engine
X.
So
again,
we've
gone
all
the
way
from
cube,
kuddle
listing
pods
to
cry
cuddle
again
listing
what
pods
are
on
this
specific
node
to
a
container
to
a
task
to
an
actual
pid'
running
that
workload.
A
A
If
I
hit
I've
got
this
connected
to
s,
T
O's
ingress,
all
this
nginx
container
is
doing.
Is
it
returns
the
pod
name?
You
know
as
the
HTML
response
from
a
request
for
index.html,
and
so,
if
I
keep
refreshing
I
should
you
know
be
hitting
the
various
pods
and
every
once
in
a
while
you'll
see
it's
actually
hitting
the
pod
on
this
node
that
I'm
logged
into
the
TV
for
seven
queue.
A
A
A
You'll
see
that
it
started
anew
and
on
on
this
node
here
and
so
what
what
shouldn't
have
happened
is
we
should
still
have
a
pod
named
tv4
7q,
because
I
didn't
actually
kill
the
pot.
I
killed
the
container
and
it
can
recover
from
that
by
just
restarting
the
container.
But
now
so
again
we
can
step
back
up
to
CRI.
A
A
A
A
So
it's
interesting,
we
should
have
seen
in
coop
Cod.
Oh,
we
should
have
seen
a
failure
in
coop
kado,
get
pods
and
then
a
restart
I
think
we're
almost
at
time.
We
only
have
a
couple
minutes
so
I
think,
rather
than
investigate
why
I
didn't
get
to
show
you
that
properly,
let's
just
switch
back
to
the
charts
for
a
second
and
we'll
finish
up.
A
So
again,
there's
a
grit.
So
if
you're
gonna
use
CRI
and
then
CRI
run
times,
there's
a
great
users
guide
for
cry
cuddle
all
the
things
you
can
do.
I
showed
almost
nothing
of
all
the
possible
things
you
could
do
with
cry
cuddle.
You
can
even
create
pods.
You
can
have
a
no
CI
spec
and
start
and
stop
pods
from
cry
cuddle.
There's
a
great
talk
from
Kubb
khan
in
copenhagen
doing
some
more.
A
He
had
a
little
more
time,
so
you
could
do
interesting
things
like
actually
replace
the
runtime
underneath
the
coolant
you
could
actually
switch
from
docker
to
container
d
and
back
and
show
that
the
pods
kind
of
restart
and
continue
running.
We
have
lots
more
information
at
our
github
project.
Page
and
I
should
be
adding
in
our
website,
because
we
have
a
new
website
container
di
o.
That
also
has
a
lot
of
great
information.
A
So
with
that
I
say,
thank
you
and
say:
please
go
see,
Derek
and
Mike's
deep
dive
if
you're
interested
in
more
about
container
D
and
the
architecture
later
today,
and
you
can
find
me
online
or
send
me
an
email.
If
you
have
more
questions
and
I
think
we're
effectively
all
out
of
time,
I'll
pack
up
and
be
available
here,
if
you
want
to
chat
but
I'll
make
room
for
the
next
speaker.
So
thank
you
very
much.