►
From YouTube: containerd Introduction and Deep Dive - Phil Estes, Maksym Pavlenko, Derek McGowan, & Mike Brown
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
containerd Introduction and Deep Dive - Phil Estes, Amazon; Maksym Pavlenko & Derek McGowan, Apple; Mike Brown, IBM
Join containerd maintainers for an introduction and deep dive into the latest updates on containerd. This last year has seen tremendous growth in both project usage and contribution. From end user CLI to low level runtime implementations, there have been exciting developments and proposals toward making containerd more stable and shaping the next generation of container use cases. The maintainers will go over internal changes to containerd which help make the core project interfaces cleaner and easier to integrate with from different components and plugins. For Kubernetes use cases, we will cover related changes happening in containerd including updates in the CRI implementation. Finally, the maintainers will cover exciting new features and sub-projects such as nerdctl, lazy-pulling (stargz), shim plugability, and more.
A
So
we're
excited
to
see
everyone
again
and
give
you
an
update
on
the
project
if
you
were
at
kubecon
san
diego
or
you
watched
what
we
called
the
mini
summit
we've
given
other
updates
since
then,
obviously,
there's
been
some
virtual
kubecons
and
other
events
since
since
then,
but
I
thought
you
know
it's
our
first
time
back
together
in
person.
Let's
just
recap
the
last
two
years
for
those
who
may
have
seen
the
last
18
months
as
a
fog,
and
you
don't
really
remember
what
happened
in
all
that
time.
A
A
A
A
A
Also,
in
that
time
we
firmed
up
kind
of
our
disclosure
and
security
process
and
we've
gotten
to
test
it
with
three
cves
in
the
last
couple
years,
and
so
that
seems
fairly
well
streamlined
as
well,
and
we
even
have
a
new
role,
a
security
advisor
role
that
came
along
with
rethinking
our
process
and
there's
some
other
governance
updates.
You
can
find
that
in
our
container
d,
slash
project,
repo,
we've
added
new
reviewers
from
multiple
employers
and
there
have
been
over
370
unique
contributors
just
in
the
last
two
years
to
the
project.
A
10
of
those
are
not
just
you
know,
one
time
and
gone
actually
have
submitted
10
or
more
prs.
So
we've
got
a
great
community.
That's
been
continuing
to
grow
a
little
more
detail
on
that.
If
you
ever
use
dev
stats
that
the
cncf
have
put
together,
there's
lots
of
other
community
statistics
you
can
dig
into
you
can
see
that
67
unique
companies
are
participating.
A
We
now
have
13
committers
and
14
reviewers
from
again
a
good
cross-section
of
companies
and
we'd
like
to
see
that
as
well
as
the
community
comes
together
to
build
container
d,
one
of
the
other
things
if
you
were
at
that
mini
sub
summit
or
watched
the
video
or
have
followed
since
then,
we've
added
this
concept
of
non-core
sub-projects
of
container
d.
So
at
the
mini
summit
I
believe
kohai
was
there
and
presented
star
gz
snapshotter.
A
That's
now
part
of
the
container
d
repo
as
a
non-core
sub-project
image
encryption
was
being
worked
out
at
the
time.
That's
also
now
a
subproject
in
the
continuity
repo
and
then
something
that
didn't
exist
a
few
years
ago,
but
has
gotten
a
lot
of
community
activity
recently
is
nerd
ctl
the
akihiro
one
of
our
maintainers
put
together.
A
There's
been
interest
from
the
confidential
computing
consortium
and
adding
capabilities
into
container
d
around
that.
In
fact,
the
inclavar
project
is
already
using
container
d
and
there's
a
bunch
of
work
maxim
will
talk
about
around
sandbox
api
and
shim
and
a
bunch
of
rework
that's
going
on
even
right
now,
which
we
hope
to
get
into
the
1
6
release.
B
C
Are
we
good
okay?
So
I'm
going
to
talk
about
the
deep
dive
section
of
container
d,
I'm
going
to
kind
of
go
through
the
break
it
up
roughly
into
three
different
parts,
so
everyone
can
be
familiar
with
what
we're
talking
about
here.
So
in
container
d,
we
have
the
client
and
phil
mentioned
nerd
ctl.
This
is
one
of
the
clients
ctr.
C
Also,
our
cri
implementation
actually
uses
that
same
go
library
in
the
container
d
daemon,
which
you're
probably
all
familiar
with
running
on
your
hosts.
There
we
have
our
api
server,
this
actual
cri
plug-in,
and
then
we
have
all
our
resource
managers
that
do:
data
storage,
garbage
collection,
everything
around
shim
management,
which
actually
owns
the
containers
and
then
the
actual
container
d
shim
is
we
have
a
shim,
that's
per
container
or
pod
instance,
and
this
is
what
actually
manages
all
the
running
processes
so
to
break
that
out.
It
ends
up.
C
Looking
like
this
you'll
see
the
client
all
the
way
there
on
the
on
the
left
box,
you'll
see
the
daemons
actually
in
the
middle
and
then
the
shim
on
the
side,
so
in
the
client
here.
So
this
is
this
is
roughly
how
our
go.
Library
is
kind
of
laid
out.
All
image
distribution
is
actually
done
inside
of
our
go
library,
it's
not
inside
the
daemon
per
se
as
a
service.
Today,
everything
that's
related
to
setting
up
a
container
so
creating
your
oci
spec
setting.
C
All
your
container
options
are
actually
defined
inside
of
our
inside
of
our
go
library,
all
the
configuration
for
the
services
as
well
as
so
like.
We
have
the
ability
here
so
for
the
client.
You
can
use
these
service
proxies
which
talk
over
our
grpc
api,
but
we
actually
use
the
same
client
in
the
cri
plugin,
which
actually
doesn't
go
through
the
grpc
api.
It
actually
directly
uses
the
service
implementations
so
inside
the
daemon
we
have
our
api
servers,
we
have
our
cri
plugin
and
then
in
the
core.
C
We
actually
have
service
implementations
for
all
of
the
all
the
services
that
we
define,
so
everything
that
related
to
containers
related
to
accessing
content,
relax
accessing
the
actual
images
name,
spaces
and
snapshots
all
have
a
definition
here
and
the
api
gives
you
access
to
those
through
through
grpc.
So
if
you
know
the
container
d
api
today,
you
might
be
familiar
with
some
of
our
some
of
these
apis
they're
very
low
level.
C
If
you're
used
to
using
docker,
there's-
probably
a
lot
of
stuff
here-
that
you
wouldn't
necessarily
use
directly
you'd,
see
stuff
more,
that
you're
used
to
pull
push
inside
inside
the
actual
go
library,
and
then
here's
where
we
actually
do
our
data
storage.
So
in
our
metadata
layer,
this
is
where
we
store
everything
related
to
any
sort
of
label.
You
have
every
single
image.
What
it
stores
every
snapshot,
every
container
is
actually
stored
here,
and
we
have
a
garbage
collection
process
that
will
go
in
and
it
will
delete
stuff.
C
That's
that's
no
longer
no
longer
referenced
by
anything.
That's
so
that
we
can
have
our
approach
of
failing
clean.
So
if,
if
something
becomes
unreferenced
we'll
clean
it
up
the
back
end,
underneath
that
this
is
where
the
actual
snapshots
will
live.
The
actual
large
content
blobs
that
you
pull
down
from
the
registry,
the
actual
clients
that
that
talk
to
the
the
shims
are
at
this
layer
here
and
the
metadata
layer
will
actually
manage
everything.
That's
done
there,
so
the
shims.
D
C
Visor
kata
containers
run
c,
which
everybody
knows
firecracker
all
implement
this,
this
gem
interface,
and
what
this
allows
us
to
do
is
to
separate
the
underlying
container
implementation
from
our
actual
resource
management
of
that
container.
So,
from
a
container
d
perspective,
we
can
treat
these
shims
as
a
resource.
C
We
don't
have
to
worry
about
everything,
that's
being
managed
necessarily
underneath
it,
whether
you're
using
c
groups
or
you
have
a
vm
implementation.
All
that's
actually
related
to
how
the
shim
implements
that
and
we
can.
We
can
implement
it
a
little
bit
more
abstractly
I've
added
since
the
last
time.
We
did
this
talk.
Actually,
I
guess
we
didn't
have
this
two
years
ago
this
this
diagram,
but
at
least
since,
since
last
kubecon
we
have
the
sandbox
manager.
That's
a
new
interface
that
we're
adding
and
we'll
be
discussed
by.
C
C
So
when
you
do
a
poll,
the
first
thing
that's
going
to
be
done
is
we're
going
to
get
a
lease
and
a
leasing.
Container
d
is
basically
a
way
to
tell
the
garbage
collector
that
I'm
using
this
content
right
now
and
if
something
fails,
then
the
garbage
collector
will
eventually
clean
it
up
when
that,
once
that
lease
expires.
C
We
always
try
to
make
sure
we
never
leave
anything
around
so
after
that
the
pull
process
is
actually
going
to
resolve
your
content.
So
it's
going
to
resolve
your
tag
to
some
oci
registry.
It's
going
to
grab
that
that
data
from
the
oci
registry
and
it's
going
to
go
into
the
content
store
and
the
path
it's
going
to
make
from
the
client
is
through
grpc
through
our
service
layer
through
our
metadata
layer
and
then
actually
to
the
storage.
C
After
that,
every
poll
you're
going
to
unpack
that
into
disk.
So
the
poll
is
going
to
actually
create
the
snapshots
and
then
for
every
snapshot.
It
creates
it's
going
to
unpack
that
data
using
our
using
our
diff
service
and
all
the
disk
service
does.
Is
it
takes
these
layers
and
it
unpacks
them
into
into
the
snapshots?
C
And
then,
after
that,
we
can
just
create
an
image.
This
is
the
lightest
weight
part
it
just
links
to
the
content
that
we've
now
unpacked
on
disk.
Likewise,
let
me
go
through
a
run
flow,
real,
quick,
you'll
kind
of
see.
Unlike
poll,
we
don't
have
a.
We
don't
have
a
top
level
run
function,
so
nerd,
ctl
or
ctr
will
actually
like
orchestrate
these.
It's
going
to
do
the
same
thing.
First,
it's
going
to
grab
it's
going
to
grab
a
lease
and
then
it's
actually
going
to
go
resolve
your
image.
C
C
Oci
configuration
create
the
container
now,
in
this
case,
the
container
is
actually
a
metadata
object
that
we
use
in
container
d
to
keep
track
of
container
as
a
resource,
and
then
after
that
is
where
we
actually
start
the
tasks
and
the
tasks
here
doesn't
actually
have
data
that
we
store
in
container
d.
You
can
see
it
kind
of
skips
over
our
storage
layer
there
and
goes
directly
to
the
back
end.
C
C
So
you're
going
to
start
the
container
you're
going
to
wait
for
the
container
you're
going
to
exec
or
kill
or
whatever
you're
going
to
do,
is
actually
going
to
go
through
the
same
flow
from
the
client
all
the
way
back
to
the
shim,
and,
as
I
mentioned
before,
I
added
sandboxes
in
here.
This
is
something
that
we're
in
the
process
of
adding.
C
We
don't
have
a
flow
all
the
way
back
through
the
client
today,
because
the
sandbox
api
is
still
being
worked
on,
but
this
is
one
of
the
areas
that
we're
working
on
we're
working
on
today.
So
this
will
help
in
cases
where
you
have
a
vm,
sandbox
you'll
actually
be
able
to
manage,
manage
that
sandbox
through
the
container
d
api
in
the
life
cycle.
C
D
Hey
guys,
well,
I
said
deeper,
but
really
I'm
going
to
step
back
a
bit
and
talk
more
about
or
how
we
interact
with
kubernetes
itself.
Okay,
as
I'm
sure
a
lot
of
you
guys
are,
you
know,
kubernetes
users,
and
you
don't
really
know
what
goes
on
in
node,
except
that
you
configure
a
container
runtime.
So
I'll
go
ahead
and
talk
about
this
a
little
bit
the
the
concept.
The
idea
is
on
your
node.
D
You
have
kubelet
registering
with
api
server
to
host
all
the
pods
and
resource
types
that
pods
need
on
on
the
note
on
the
worker,
node
right
and
it's
the
job
of
kublet
to
run
its
managers
and
converse
with
the
container
runtime
allocate
volumes
and
attach
things
to
to
the
container.
Through
these
cry,
apis,
okay,
the
cry
apis
were
were
done
by
tim.
Some,
you
know
some
guys
in
google,
probably
about
five
years
ago,
as
a
way
to
to
modify
how
it
currently
we
were
integrating
with
docker
right
for
for
launching
containers.
D
Managing
we
thought
would
be
a
good
idea
to
have
a
cry
api
so
that
other
people
could
come
in
and
implement
without
having
to
fork.
Kubele,
which
is
basically
what
we
were
doing
originally
we
had,
there
was
a
rocking
at
ease
and
a
docker
fork,
and
and
currently
we
let's
go
ahead-
let's
see,
let's
go
in
the
next
short
down
arrow.
D
You
know
integration,
you
know
techniques
and
things
that
are
in
there
and
it
was
decided
that
all
that
stuff
needs
to
be
pushed
over
into
the
cry
so
that
the
container
run
times
that
are
running
using
only
the
cry
interface
would
have.
You
know
more
legitimate
api
to
work
with
it
just
felt
easier.
So
what
we're
going
to
do
and
what
has
been
happened-
is
we've
deprecated,
docker
shim,
that's
the
internal
version,
not
the
external
version.
The
external
version
of
docker
shim
is
still
being
worked
on
and
it'll
still
work
with
docker.
D
I
don't
know
if
you
know
about
it,
but
underneath
the
covers
docker
actually
uses
container
d.
Also,
that's
another
piece
of
information,
but
we've
got
some
we're
going
in
a
way
to
make
all
the
test
buckets
work
for
the
container
run
times,
either
cryo
and
or
container
d,
so
that
you
guys
will
be
able
to.
You
know
trust
that
everything
that
can
happen
with
docker
can
also
happen
with
these
other
container
runtime
types
that
you're
all
migrating
to
when
it
gets
completely
pulled
out.
I
guess
maybe
124
or
125
we'll
see
what
happens.
D
Okay
along
this
route,
we've
just
the
the
cry
api
was,
has
been
in
alpha
for
about
three
years.
I'm
sure
you
know
the
api
cycle
for
kubernetes
apis
can
be
very
long,
but
we
are
moving
to
a
version.
One
beta
status
and
sasha
has
a
pull
request
in
there
right
now
to
to
add
the
v1
api
is
an
optional
api
in
the
2-3
cycle.
D
D
So
when
kubelet
comes
up,
it'll
check
to
see
if
you've
got
v1
in
the
container
runtime
or
not,
if
you,
if
you
have
v1
it'll,
go
ahead
and
start
using
that
as
soon
as
this,
this
pr
gets
merged
in
in
two
three
of
kubernetes
and
kubelet
and
and
then
all
be
great
right
and
then
later
on.
If
container
d
reboots
when
and
you
have
to
reconnect
kubelet
to
container
d,
then
it'll
be
using
continually
using
the
b1
api.
D
If,
however,
you're
using
an
older
version
of
container
d
or
cryo,
and
it's
connected
initially
to
the
v1
alpha
api,
then
it'll
still
be
using
the
v1
alpha
api
unless
you
reboot
kubelet
and
then
it'll
recheck,
to
see
if
the
b1
is
there.
Just
give
you
an
idea
that
yes,
there's
going
to
be
a
migration
kind
of
issue,
but
we're
going
to
tackle
it
in
the
container
run
times
and
include
to
be
okay
with
either
one
migrating
up.
D
But
you'd
have
to
reboot
both
to
get
up
to
the
next
version
of
the
api,
and
I
imagine
we'll
do
the
same
thing
when
we
go
from
the
b1
beta
to
the
v1.
Ga,
okay,
so
that's
going
to
happen
confidential
computing,
very
exciting,
a
lot
of
interesting
stuff
going
on
in
multi-tenancy,
we've
run
into
some
issues.
D
The
one
of
the
oldest
issues
in
in
kubernetes
is
a
little
problem
with
if
a
pod,
a
pulls
an
image
down
to
down
to
the
node
and
pod
b,
wants
to
use
that
image
and
you're
only
you're
only
saying
pull.
If
not
present,
then
it'll
plot
b
will
get
to
use
the
image
from
the
first
poll,
whether
or
not
it
had
access
to
it.
So,
if
you're,
storing
an
image
as
a
private
image
with
it
can
only
get
access
with
a
password,
you
know
an
authentication,
then
it
may
already
be
on
the
node.
D
So
what
we
want
to
do
with
this
ensure
insert
secret
you
know,
images
is,
is
actually
check
and
keep
a
running
list
of
who
actually
pulled
it
and
whether
authors
require
it
or
not.
If
you
follow
these
links,
you'll
you'll
find
the
cap.
It's
been
approved
for
one
two
three,
and
and
it's
and
we've
got
the
code
for
it
as
well.
Just
just
a
heads
up.
You
know
this.
This
problem
should
be
fixed,
however.
The
the
workaround
right
now
for
multi-tenancy
is
to
force
a
cont.
D
D
C
D
We
can
answer
questions
on
this
later
on,
hoping
you
guys
are
interested
now
another
another
set
of
changes
that
we
we
would
want
to
do
if
you,
if
you're
looking
at
k,
native
or
istio
or
some
some
other
situations,
where
maybe
edge
type
situations
where
you
want
the
pods
to
be
really
really
fast.
Well,
there's
a
somewhat
a
couple
of
problems
with
the
way
kubernetes
is
currently
designed.
It
wasn't
designed
to
run
pods.
D
So
what
we'd
like
to
do
is
improve
how
we
handle
probes
to
move
them
from
a
per
second
kind
of
kind
of
level
to
milliseconds
and
we'd,
also
like
to
change
the
the
cube
process,
kubelik
process,
actually
for
for
how
it
determines
the
current
state
of
pods.
Today
it
determines
the
current
state
on
a
configurable
but
default.
I
believe
it
is
two
minute
pro
cycle
time,
so
it
it.
D
You
won't
know
at
the
scheduler
side
until
after
a
couple
of
minutes
whether
or
not
you
know
you
need
to
run
another
pot
and
that's
not
really
fast
enough,
for
you
know
for
a
k-native
kind
of
kind
of
scenario,
so
we'd
like
to
move
to
a
subscription
process,
it
sounds
it
sounds
obvious,
we'll
go
with
eventing.
We
already
have
inventing
support
for
tasks
inside
of
inside
the
runtime
container,
so
we'd
like
to
move
that
outside
and
that's
that's.
It.
B
B
B
Added
available
support
to
our
streams
in
1.6
beta.
They
use
the
same
plugin
system
as
continuity
daemon.
So
it's
easy
to
add.
New
services
to
the
runtime
depend
define
dependencies
between
these
plugins
and
so
on,
and
in
order
to
support
new
services
on
demand
side.
We
want
to
introduce
new
ocean
service
that
will
manage
sims
processes
independently
from
api
that
this
shims
implement
and
we
give
more
controls
to
clients
how
to
schedule
and
how
to
manage
workloads
through
containing
the
apis.
B
B
B
B
So
can
the
ldd
still
manages
shim
processes
life
cycle,
so
we
can
ask
continuity
to
start
a
new
stream
for
us
or
request
existing
one
by
id,
but
containing
daemon
is
not
aware
what
kind
of
services
particular
swim
implement
instead,
we'll
add
content
id
client
to
talk
to
shim
and
depending
depending
on
services,
runtime
offer
client
can
decide
proper
flow
of
calls.
So,
for
instance,
if
stream
implements
star
service,
we
launch
container
from
client
like
we
the
same
way.
B
We
do
it
today
if,
in
addition
to
that
stream
supports
inbox,
api
client
can
start
sandboxing
sandbox
instant
before
launching
containers.
If,
in
addition
to
that
stream
supports
confidential
containers,
apis
client
can
prepare
image
for
that
sandbox,
and
this
is
flexible
enough
to
enable
all
kind
of
service
combinations
and
support.
It's
easy
to
support
new
use
cases.
B
The
concept
behind
stream
service
is
really
simple.
It
can
create
and
it
can
delete
ships
to
be
precise.
We
already
do
that
today.
We
just
split
existing
touch
service
into
many
streams
and
does
tasks
independently
as
two
different
plugins
and
we'll
expose
its
api
to
clients.
So
clients
can
control
shins
containing
this
still
expects
to
track
a
shim
lifecycle.
So
whenever
shim
process
dies,
containid
will
clean
it
up.
B
This
service
is
going
to
be
a
foundation
for
high
level
services,
which
depends
on
specific
ship.
Implementations
task
is
the
only
service
we
require
to
implement
to
be
implemented
today,
so
continuity
can
launch
new
containers
by
first
calling
action
service
to
start
new
sim
instance
and
then
by
calling
the
service
api
from
that
stream,
and
this
way
we
keep
backward
compatibility
with
previous
versions
of
continuity.
B
B
The
goal
is
to
provide
the
well-defined
shim
interface
to
test
task
api
that
can
be
used
to
add
new
sandbox
implementations
and
runtime
level.
Behind
sandbox
concept
can
be
really
anything
depending
on
how
runtime
implements
it's
so,
for
instance,
it
can
be
post
container
micro,
vm
or
even
virtual
machine.