►
From YouTube: Introduction and Deep Dive into containerd - Michael Crosby & Derek McGowan, Phil Estes & Wei Fu
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Introduction and Deep Dive into containerd - Michael Crosby & Derek McGowan, Apple, Phil Estes, IBM, & Wei Fu, Alibaba
Join containerd maintainers in a combined introduction and deep dive discussion. This talk will include a brief introduction to the design and architecture of containerd along with the latest updates to the project. After that, maintainers will deep dive into new features including NRI (Node Resource Interface), new Sandbox API, the CRI (Container Runtime Interface) implementation move into containerd core, and improvements in image distribution from remote snapshotters to better proxy support. Come learn about how to get started with containerd as a user or contributor.
https://sched.co/ekGF
A
Welcome
to
the
continuity
introduction
and
deep
dive
session,
I'm
derek
mcgowan
maintainer
on
the
community
project
at
apple,
I'm
joined
today
by
fellow
maintainers
phil
estes
from
ibm,
michael
crosby,
from
apple
and
way
foo
from
alibaba.
We're
going
to
give
you
a
brief
introduction
to
community
today
and
deep
dive
into
various
parts
of
container
d.
First,
we're
going
to
discuss
what
container
d
is
and
why
to
use
it.
Then
we
will
give
an
overview
of
its
architecture.
We
will
deep
dive
into
the
new
node
resource
interface,
also
called
nri
and
continuity's
other
plug-in
systems.
A
B
But
the
way
container
d
is
designed.
It's
a
very
extensible
system,
and
so
a
lot
of
the
projects
that
we've
seen
use
or
grow
out
of
container
d
have
used
this
extensibility
to
great
advantage.
It's
also
a
tightly
scoped
project,
so
we
don't
change
the
scope
of
container
d.
Often
the
only
thing
we've
really
done
in
the
lifetime
of
the
last
four
years
is
bring
in
the
cri
plugin,
so
that
kubernetes
had
a
natural
plug
point
to
use
container
d
as
a
runtime.
B
There
are
a
lot
of
new
governed
sub-projects
within
the
container
d
repository
and
organization.
We
now
have
a
rust,
written,
ttrpc
implementation,
container
encryption
implementation
support
a
star
gz
snapshotter,
which
handles
things
like
lazy
image
poll
and
you're
going
to
be
hearing
about
nri
as
well
during
this
presentation.
So
that
gives
you
a
quick
snapshot
into
what
container
d
is
now.
B
During
that
time,
we
created
completely
new
interfaces
for
managing
containers
and
images,
and
then
we,
after
defining
those
in
grpc,
we
also
created
a
go
client
api.
That's
used
heavily
by
the
cri,
for
example,
as
it
implements
the
container
runtime
interface
for
kubernetes
and
many
other
projects,
use
that
clean
api
to
manage
containers
and
images
and
container
life
cycle.
B
B
Now
we
don't
have
a
lot
of
new
information
on
container
d
adoption
across
the
ecosystem,
since
we
just
shared
many
of
these
numbers
and
percentages
just
a
few
months
ago.
But
there
is
a
cncf
survey.
That's
now
out
capturing
some
of
the
20
20
changes
to
that
information
and
we're
hoping
to
see
very
soon
assistig's
latest
usage
report,
which
should
be
out
by
the
time
you
hear
this
or
maybe
in
the
next
month
now
the
cncf
survey
update
did
show
that
container
d
usage
grew
by
over
a
third.
B
Since
the
last
time
the
community
was
surveyed,
production
use
of
container
d
is
also
growing.
36
percent
of
respondents
noted
that
they
were
directly
using
containerity
in
production
and
so
again,
without
a
lot
of
new
data.
We
definitely
see
continued
growth
in
the
use
of
container
d,
and
we
also
see
in
our
community
project
a
lot
of
new
activity
from
people
utilizing
container
d,
maybe
in
scenarios
that
aren't
going
to
be
shared
publicly
but
obviously
show
a
growing
use
of
container
d
across
the
ecosystem.
B
B
B
Maybe
you've
heard
of
linux
kit,
but
amazon
recently
released
bottle
rocket
os
as
an
open
source
project
that
again
uses
container
d
for
a
containerized
linux.
Os
idea
that
they've
now
shared
with
the
broader
community.
Now
we've.
Given
you
a
lot
of
information
on
how
container
d
is
used,
who
might
be
using
it?
B
That's
an
area
we've
been
talking
to
the
cncf
about,
and
so
no
matter
what
skill
level
or
or
what
your
interest
is
we'd
love
to.
Have
you
join
us
in
the
container
d
community
and
reach
out,
if
you
can't
figure
out
where
to
get
involved,
definitely
reach
out
to
to
us
and
we
can
find
a
way
to
get
you
involved.
A
Thanks
phil
now,
let's
take
a
look
at
communities
architecture
in
this
diagram,
we
have
broken
down
container
d
into
three
main
components.
The
client
is
containerd's,
go
client,
which
contains
most
user-facing
container
logic
you
may
be
familiar
with
in
the
middle.
Is
the
container
de
daemon
broken
down
by
api,
core
backend
and
plugin?
This
is
where
container
d
service
interfaces
live
and
handles
all
data,
storage
and
resource
management.
A
This
gym
is
what
owns
the
container
processes
and
communicates
low
level
operations
to
and
from
container
d
the
shim
is
responsible
for
interacting
with
the
lower
level
container
run
time.
As
a
user
of
container
d,
the
client
is
the
first
place
you
will
interact
with
if
you
are
building
a
system
on
top
of
container
d.
The
client
is
what
you
will
be
working
with.
A
You
can
see
in
this
diagram
that
some
core
functionality
that
is
often
associated
with
container
runtimes
is
implemented
here
in
the
client,
such
as
container
management,
which
handles
creation
of
the
oci
runtime
spec,
preparing
the
container
snapshot
as
well
as
starting
individual
container
tasks.
Image
pulling
is
also
completely
implemented
inside
the
client.
Because
of
this
approach,
clients
are
mostly
limited
to
using
our
go
library,
even
though
python
java
or
another
language
could
talk
to
the
container
d
api
using
grpc
basic
functionality
would
need
to
be
re-implemented
to
use
the
full
runtime.
A
The
cri
interface
is
probably
much
easier
to
work
with.
If
you
are
not
implementing
your
project
and
go
one
of
the
first
things
you
will
come
across
in
the
client
is
the
options
pattern
with
continuity.
We
chose
to
keep
the
core
as
simple
as
possible
by
defining
clean,
simple
interfaces
in
the
core
that
meant
that
the
client
had
to
have
more
functionality
and
configurability
in
the
container
decor.
A
There
may
just
be
generic
variatic
options
on
a
core
type,
but
in
the
client
you
will
see
with
options
used
on
these
types,
implement
a
feature
you
will
commonly
see
this
used
for
labeling
and
filtering
on
types.
We
were
also
hoping
this
would
allow
clients
to
include
their
own
opinions
rather
than
be
forced
to
work
around
opinions
in
the
container
d
daemon.
The
core
of
container
d
contains
the
components
we
consider
most
critical
in
terms
of
stability.
A
A
The
core
has
a
main
implementation
of
these
interfaces,
and
these
interfaces
are
extended
and
used
all
the
way
to
the
client.
The
design
is
such
that
all
data
flows
through
the
core,
so
that
plugins
and
other
components
don't
need
to
store
data.
This
is
important
because
we
have
implemented
garbage
collection
inside
the
core
metadata
store.
So
if
data
is
being
stored
somewhere
else,
it
is
really
hard
to
track
that
data
and
make
sure
it
is
removed
when
it
is
no
longer
needed.
This
makes
containity
more
stable
by
keeping
storage
and
memory
usage
stable.
A
It
also
avoids
data,
inconsistency
and
crashes.
We
don't
expect
as
many
change
in
the
core
relative
to
other
parts
of
container
d
features
generally
aren't
implemented
inside
the
core.
However,
some
features
will
require
small
changes
inside
the
core.
So,
for
example,
we
recently
added
a
feature
to
support
remote
snapshotters.
The
high
level
feature
of
remote
snapshotters
is
not
something
that
the
core
knows
anything
about,
but
one
functionality
we
did
add
to
the
core
was
the
ability
for
the
snapshotters
themselves
to
report
that
they
already
know
about
a
snapshot
as
it
is
flowing
through
the
core.
A
So
when
you
create
a
new
snapshot,
you
can
pass
information.
That
says
this
is
the
target
snapshot
that
I'm
trying
to
create
and
it
will
pass
that
to
the
backend.
The
backend
now
can
actually
communicate
back
up
to
the
core
to
let
it
know
it
already.
Has
that
snapshot
that
allows
the
client
to
know
the
snapshot
is
already
available
as
a
remote
snapshot
when
it
sees
the
core
report,
the
snapshot
as
existing.
Meanwhile,
the
core
has
no
knowledge
of
what
a
remote
snapshot.
Is.
A
It's
really
important
for
us
to
keep
the
core
of
container
d
and
to
make
sure
we
don't
get
feature
creep
showing
up
in
the
continuity
daemon.
We
also
don't
want
opinions
showing
up
in
the
daemons
so
that
later
on,
new
features
or
requirements
are
limited
by
previous
opinions.
Added
to
the
core,
one
thing
you
may
notice
is
that
there
are
no
tasks
in
the
metadata
store,
even
though
tasks
are
considered
a
core
type
with
a
core
service.
The
data
associated
with
the
task
is
completely
ephemeral
and
only
related
to
the
actual
running
container.
A
This
data
may
be
something
like
process
information
or
unix
sockets.
When
a
node
is
restarted,
all
the
process
information
will
be
gone
as
expected.
However,
all
the
metadata
associated
with
the
container,
including
its
specification
and
snapshots,
are
persistent
in
the
metadata
store.
This
information
can
be
used
by
clients
to
recreate
the
container
and
tasks.
The
disk
service
is
another
core
service
without
any
state.
It
operates
on
snapshots
and
content,
but
does
not
store
its
own
information
notice
that
the
cri
plugin
sits
inside
of
the
container
daemon.
A
In
current
versions
of
container
d,
the
cri
plugin
exists
as
a
single
imported
plugin
in
containerity
1.5.
We
are
merging
the
cri
code
base
and
integrating
cri's
components
into
communities
core
and
backend.
Today,
cri
operates
by
using
the
container
d
client
package
directly
and
calling
out
to
separate
plugin
interfaces
such
as
cni
and
nri.
Let's
take
a
look
inside
the
cri
plugin
cri
consists
of
two
grpc
services
image
and
runtime.
Both
of
these
services
share
a
service
backend
and
the
plugin
today
and
call
into
the
container
d
client.
A
The
cri
plugin
is
responsible
for
defining
the
pod
by
sharing
the
name,
spaces
and
c
groups
between
all
containers
in
a
pod.
It
is
also
responsible
for
managing
the
pause
container,
which
is
just
a
container
that
is
responsible
for
holding
namespaces
open
in
the
future.
The
pause
container
will
be
able
to
go
away,
since
there
are
other
ways
this
could
be
accomplished
today.
The
cri
plugin
invokes
cni
for
setting
up
container
networking.
A
C
All
right,
I'm
going
to
show
you
some
of
the
work
we've
been
doing
on
the
node
resource
interface
for
container
d
and
kubernetes.
So
when
you
think
about
resource
management,
we
have
different
workload
requirements.
You
have
customers
that
that
are
running
batch
workloads
and
also
customers
that
have
latency
sensitive
workloads.
That
may
be
in
the
request
response
cycle
for
for
various
users,
interacting
with
the
service
and
both
of
these
type
of
workloads.
C
C
So
when
you
think
about
resource
management,
we
have
various
resources
on
a
compute
host.
You
have
cpus,
which
you
may
want
to
dedicate
cpus
or
restrict
access
to
cpus,
depending
on
the
workload
you
also
have
pneuma
on
multi-socket
systems,
and
then
we
can
expand
out
into
l3
cache
use
pages,
aligning
making
sure
your
workload
runs
on
the
same
pneuma
node
that
your
gpus
are
connected
to.
So
the
current
solutions,
cubelet,
does
have
a
cpu
manager,
there's
a
few
keps
improving
this
expanding
into
the
pneuma
support
case
and
like
pinning
as
cpus.
C
So
I
thought,
let's
make
cni
for
resources,
and
this
is
where
nri
comes
in,
because
cri
was
already
taken.
I
I
would
rather
it
be
named
the
container
resource
interface,
but
we'll
we'll
stick
with
node
resource
interface.
For
now
so
myself
and
some
others
in
the
community.
We
felt
like
the
cubelet's,
not
the
right
abstraction
for
things
like
this,
the
lines
between
the
cubelet
and
then
the
lower
level.
C
Cri
is
a
little
too
blurry
and
we
think
that
being
able
to
hook
into
the
life
cycle
of
containers
at
the
cri
level
is
the
right
way
to
do
resource
pinning
and
topology.
On
a
node
cri
implementations
like
container
d,
we
know
how
to
interface
with
the
underlying
host
systems.
We
can
get
the
information
for
what
devices
are
available,
and
this
is
where
things
like
cni
for
networking
also
hooks
in.
C
A
Thanks,
michael,
let's
go
back
to
the
diagram
for
the
underlying
run
time.
There's
a
component
like
continuity,
daemon
and
then
external
shim
for
managing
container
processes.
The
runtime
the
daemon
is
responsible
for
starting
up
new
runtime
shims.
It
will
pass
the
oci
runtime,
spec
and
commands
to
the
shim.
The
runtime
shims
are
the
boxes
on
the
right
side.
These
shims
are
what
actually
own
the
container
processes.
A
The
containers
are
parented
directly
to
the
shim.
The
container
daemon
will
talk
to
the
shims
using
the
lightweight
grpc
protocol
called
ttrpc.
We
use
this
lightweight
rpc
to
reduce
the
memory
footprint
of
the
shim.
If
container
d
gets
restarted,
it
will
connect
to
the
shim
in
order
to
send
commands
to
the
container.
There
are
many
different
runtime
shim
implementations,
the
run
c
implementation
is
the
most
common.
There
is
run
hcs
for
running
on
windows.
There
are
also
sandbox
shims,
such
as
firecracker
and
cotton
containers.
A
Since
the
shims
only
implement
this
rpc
interface,
anyone
can
implement
their
own
shim
to
be
used
by
containerity.
Pretty
easily
containerd
just
needs
to
know
about
the
shim
binary
in
order
to
use
it
to
run.
Containers
plugins
can
call
into
the
core
of
container
d
and
the
core
can
call
out
to
different
backends.
The
back
ends
themselves
are
also
pluggable.
For
example,
snapchatters
are
all
implemented
as
plugins.
Each
plugin
can
define
their
own
configuration,
which
gets
included
with
containity's
global
configuration
object.
A
If
you
have
a
grpc
service,
it
can
be
added
as
a
plugin
and
call
into
the
core
services
directly
in
the
client.
There
are
many
options
for
customization.
By
default,
the
container
d
client
will
use
the
grpc
services,
which
communicate
with
the
containity
daemon
through
the
grpc
api.
However,
a
client
can
be
instantiated
with
any
custom
implementation
of
a
service
even
allowing
clients
to
operate
completely
without
a
daemon.
You
can
also
add
your
own
resolvers
resolvers
are
what
are
used
to
push
or
pull
images
between
community
and
registry.
A
The
default
implementation
communicates
with
a
docker
registry
using
the
standard,
oci
distribution
api.
However,
you
can
implement
your
own
very
easily
if
you
have
a
different
or
faster
way
to
distribute
images,
including
your
own
snapchatter
plugin,
is
pretty
easy.
You
can
compile
in
the
snapchatter.
C
A
D
I
guess
we
all
learn
so
many
things
about
canadia's
architecture
from
derrick,
the
core
of
canada,
deconstructs,
the
projecting
mechanism
about
how
to
manage
osi
image
and
how
to
run
a
container
for
common
cases.
Kinetic
has
some
building
plugins
sd4
functionalities
in
linux
platform.
Granny
can
download
image
with
gz
format
and
unpack
it
as
the
bundle
on
overlay
fs
file
system,
and
then
we
can
run
it
by
run
c,
but
it
is
just
one
common
case
with
osi
perspective.
Ocs
back
outlines
how
to
run
and
unpack
file
system
bundle.
D
For
example,
the
fire
system
bundle
can
be
built
and
unpacked
in
various
ways.
An
osi
runtime
spec
describes
the
lifecycle
of
contender,
which
can
be
handled
in
gas
kernel
for
additional
isolation
for
different
scenarios.
There
are
still
so
many
combinations
here,
but
don't
worry
about
it.
You
don't
need
to
put
your
special
implementation
into
kinetic
coil
base.
D
D
D
D
D
D
D
It's
not
good
to
use
one
kinetic
stream
to
integrate
different
kinds
of
runtime
by
commander
line,
so
kinetic
stream,
which
provides
general
console
api
for
for
the
containers,
lifecycle,
hiding
the
implementation
of
running
manager,
runtime
manager,
no
matter
what
kind
of
runtime
it
is
contender
b
can
communicate
with
third
party
runtime
invitation
by
streaming
way
to
api
and
can
addition,
which
will
also
provide
binary
name
convention
to
locate
your
own
stream,
for
example,
run,
say:
v2
name,
we
translate
into
the
binary
name,
connection
run
c
with
you
in
release.
1.4
runs.
D
D
So
connect
d,
enhanced
the
container
log
by
ui
uri
schema
right
now.
Kenneth
d
can
support
a
file
redirection,
the
connectedness,
then
the
I
o
can
be
written
into
file
directly
and
then
the
schema
also
supports
the
binary
plugin.
You
can
set
up
a
back
a
background,
long
running
process
to
handle
the
I
o.
A
Thanks
now,
let's
do
a
recap
of
continuity's
architecture,
real
quick,
going
back
over
the
whole
container
d
diagram.
You
can
see
how
operations
flow
from
the
client
on
the
left
all
the
way
to
the
runtime
and
back
end
on
the
right.
The
components
on
the
left
tend
to
be
more
stateless.
The
components
on
the
right
have
more
state,
even
though
our
garbage
collector
is
in
the
middle.
It
tracks
resources
used
by
all
the
back
ends,
but
it
operates
completely
invisible
to
the
client.
A
Our
end
goal
is
to
track
any
resource
which
may
be
used
by
container
d
to
guarantee
the
long
term
stability
of
the
runtime
container
d.
1.4
is
the
most
recent
release
of
container
d
released
back
in
august.
It
includes
support
for
c
groups,
v2,
improved
sc
linux,
support,
support
for
remote
snapshotters
and
support
for
cri
on
windows.
A
We
are
about
to
start
the
beta
process
for
container
d
1.5.
A
significant
change
in
this
release
is
the
merging
of
the
cri
code
base
into
the
main
container
d
code
base.
This
is
not
a
significant
change
to
the
end
user,
but
allows
us
to
better
integrate
cri
components
into
container
d.
It
will
also
allow
us
to
introduce
new
apis
for
sandbox
and
resource
management,
which
directly
tie
into
containerity's
core
services.
A
A
This
release
will
continue
the
effort
of
integrating
cri
into
communities
core
and
improving
resource
management.
We're
also
looking
into
ways
to
improve
the
api
between
community
and
kubernetes
on
the
container
d
project,
we're
always
looking
for
new
contributors.
If
you
are
a
container
d
user,
please
file
issues.
We
are
also
looking
for
feedback
from
end
users
on
our
roadmap
and
documentation.
A
We
know
our
documentation
is
lacking
for
a
project
as
widely
used
as
container
d.
If
you
have
a
passion
for
documentation
on
the
user
journey,
we
would
like
your
help.
If
you
are
interested
in
being
a
maintainer,
we
encourage
you
to
get
involved
in
contributing
and
reviewing
code.
Community
has
a
high
quality
threshold
for
code
contribution
and
any
extra
help
reviewing
code
is
appreciated.
A
The
maintainers
also
recently
added
a
new
security
advisor
role.
This
role
is
intended
to
help
get
individuals
who
have
a
vested
interest
in
container
runtime
security
involved
in
our
security
release
process.
As
the
cloud
native
community
has
adopted
container
d,
our
users
have
a
greater
need
for
access
and
communication
within
the
community.
We
have
been
doing
the
best
we
can
as
maintainers,
but
we
would
really
appreciate
having
additional
support
in
managing
the
community.