►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Containerd Mini-Summit - Phil Estes, IBM; Lantao Liu, Google; Derek McGowan, Docker; & Yu-Ju Hong, Google
Join containerd maintainers in this mini-summit to discuss the latest work happening on containerd. They will provide an overview of the state of the project, deep dive into recent changes, and discuss ongoing work for Windows support in containerd and Kubernetes CRI. At the end, maintainers will participate in a panel discussion and Q&A with attendees. The moderated panel will discuss topics related to various uses of containerd, from isolated runtimes like Firecracker, high performance environments like serverless, and user focused workflows like build.
https://sched.co/UakD
A
All
right,
I
think
it's
225
and
you
are
in
for
90
minutes
of
container
D,
so
yeah.
This
is
a
container
d
mini
summit,
which
is
the
name
we
came
up
with,
and
the
conference
organizers
didn't
tell
us.
We
couldn't
call
ourselves
a
mini
summit,
so
it
feels
very
important,
but
it's
just.
Instead
of
having
an
intro
and
a
deep
dive
session,
we
thought
we
just
combined.
A
We
have
a
whole
host
of
our
maintainer
crew
here
for
core
container
D
for
the
CRI,
so
several
of
us
are
going
to
be
speaking
and
kind
of
combine
status,
some
deep
dives
and
then
today,
on
how
long?
When
did
we
get?
We
hope
to
save
a
good
amount
of
time
for
some
Q&A,
and
if
people
don't
have
questions,
we've
made
up
questions
so
that
we'll
have
something
to
talk
about.
So
that's
kind
of
the
flow
and
I
think
we
have
that
here.
So
I'll
start
with
a
little
bit
about
you
know.
A
You
know
this
came
up
yesterday
at
the
end
of
my
talk.
You
know
this
runtime
word
is
overloaded,
but
it's
pretty
easy
to
just
call
ourselves
a
container
runtime
kind
of
between
and
below.
You
know.
Things
like
kubernetes
and
docker
are
above
us,
but
we're
above
lower-level
runtime,
so
OCI
Runcie
is
our
default
to
actually,
you
know,
run
containers
at
the
OS
layer
and
then
obviously
you
have.
The
sandbox
is
like
cat
and
firecracker
and
G
visor,
so
that's
obviously
kind
of
generally.
What
we
are.
We
handle
the
resource
management.
A
So
when
Derek
takes
you
through
the
architecture
and
walks
through
some
of
those
flows,
you
know
we're
managing
the
container
process,
lifecycle,
the
image
artifacts
so
talking
to
registries,
resolving
image
names
to
manifests
and
configs
the
snapshot
system.
So
again
how
how
you
actually
build
up
the
copy-on-write
file
system
and
the
various
implementations
that
we
have
and
then
the
metadata
around
that
again
the
focus
of
container
Dees
and
since
the
project
started,
has
been
very
tightly
scoped.
A
So
we're
not
trying
to
take
the
place
of
the
full
docker
experience
or
you
know
other
sort
of
additional
way
you
might
think
of.
As
value
add,
you
know
pieces
of
a
more
full
runtime
ecosystem.
Our
governance
is
such
that
it
takes
full
agreement
of
all
the
maintainer
to
change
that
scope
and
the
only
thing
we've
really
done
since
joining
the
CNC
F
is
adding
the
CRI
team
as
a
component
of
the
project,
so
bringing
the
CRI
plug-in
into
container
D
as
a
built-in
component
is
really
the
only
thing.
A
We've
changed
from
that
original
statement
of
our
scope.
So
where
are
we
this
year?
Obviously
we
talked
about
this
at
Barcelona
because
it
had
happened
then,
but
we
we.
We
are
a
graduated
project,
the
fifth
one
and
the
CNC
F
we've
had
broad
support
and
tons
of
great
contribution
from
across
the
ecosystem.
We
now
passed
a
couple
hundred
contributors
representing
you
know.
If
you
look
at
the
dev
stats
that
the
CNC
F
maintains
there's
over
100
companies
who've
at
least
contributed
something
to
the
Container
D
project.
A
A
So
yeah
we've
got
a
great
great
set
of
maintained,
errs
and
reviewers
has
has
grown
in
the
last
year.
In
some
way,
all
the
major
cloud
providers
are
using
container
D.
Some
of
them
obviously
still
use
docker
as
part
of
their
managed
services.
We'll
talk
about
who's
using
it
directly.
We
have
good
linux
and
maturing
windows
support
multiple
architectures
are
supported.
B
A
Said
we
have
a
growing
number
of
kind
of
users
and
use
cases
that
includes
public
cloud,
so
both
gke
and
IBM's
managed
kubernetes
service
directly
offer
container
D
as
a
runtime
for
your
kubernetes
clusters,
but
there's
also
infrastructure
infrastructure
projects
like
kind
that
have
started
using
container
D
or
at
least
making
that
available.
You've
got
end
users,
so
Ticketmaster
came
in
and
talked
to
us
yesterday.
After
one
of
the
talks
they
switch
to
use
container
D
and
their
self
managed
clusters.
A
There
are
some
others
that
haven't
given
us
approval
to
share
widely,
but
definitely
there's
growing
adoption
there
DevOps
tools,
so
we've
works
ignite
or
fire
cube
they're
using
container
D
via
the
the
go
API
and
then
all
the
custom
sandbox
is
so
a
talk.
I
gave
both
Barcelona
yesterday
here,
you've
got
firecracker
G
Weiser
cata
all
have
shim
implementations
for
container
D
that
also
use
container
D
in
that
way,
so
maybe
a
good
springboard
from
just
saying:
here's
a
list
of
people
using
it.
A
It's
interesting
that
those
uses
fall
really
in
different
camps,
sort
of
how
they're
using
container
D
one
way
to
use
container
D
is
essentially
as
a
library.
So
maybe
you
don't
even
want
to
use
the
container
daemon
to
run
containers
that
you
like
the
Klein
API,
so
there's
things
like
arras
that
are
that
are
using
or
implementing
certain
aspects
of
the
API
like
the
content
store
using
the
resolver
and
and
that
API
you've
got
Bill
kit
that
can
drive
container
de
via
the
API
as
well.
A
I
mentioned,
we've
works,
we
works
ignite
IBM
cloud
functions
also
drives
container
D
via
the
go
API.
Then
you've
got
extensibility
points
within
container
D,
so
remote
snapshot
airs
and
and
resolvers.
So,
when
amazon
gave
their
talk
a
couple
of
days
ago,
they
showed
their
custom
ECR
resolver
asher
has
their
telemetry
project,
which
has
a
custom
snapshot
or
to
have
that
right.
Tell
oh,
my
goodness,
I
said
telemetry
yeah,
sorry
about
that.
This
is
not.
A
This
is
not
a
great
advertisement,
but
do
not
search
on
add
your
telemetry,
because
I'm
not
sure
what
you'll
find,
but
teleport
is
an
actual
project.
I
just
wanted
to
confuse
people
I'm
sorry,
and
then
you
know
just
simply
that
container
D
has
created
a
set
of
packages
in
our
github
repo.
Some
of
them
are
sub
projects
that
the
main
project,
lots
of
other
projects,
are
finding
those
useful,
and
so
you
know
if
you
go
look
at
who's
importing
like
container
DC
groups,
which
is
just
a
nice
generic
cgroups
implementation.
A
Now
even
the
the
creo
project
imports
some
of
those
packages
and
uses
them
so
there's
kind
of
the
library
aspect
of
container
D
there's,
obviously
the
ability
to
use
it
as
a
kubernetes,
CR
I
run
so
I
already
mentioned:
IBM's
IKS,
gke
Ticketmaster,
all
about
McLeod
micro,
Cates
kind
k3s.
All
these
are
using
container
Diaz
up
here,
just
Cooper
Nettie's
runtime
via
the
CRI
plug-in
in
the
non
kubernetes
use
case.
A
You
can
also
simply
just
use
the
daemon,
so
docker
relies
on
container
DS
G
RPC
api's
as
a
running
daemon,
similar
to
how
Bill
kit
you
can
point
it
at
a
container
D
running
daemon
and
use
that
for
building
via
build
kit.
So
that's
kind
of
you
know
general
ways,
different
ways
that
that
container
D
is
used.
A
Just
a
few
months
ago,
we
released
our
next
kind
of
major
series
container
D
1.3,
some
of
the
the
biggest
sort
of
new
features
that
were
in
that
release
window
support
via
the
shim
v2
API
device,
mapper
snapshot,
which
was
contributed
by
Amazon
and
the
Firecracker
team,
a
new
plug-in
interface
interface
for
processing
layers.
So
the
encrypted
container
work
is
using
that
so
you
can
actually
have
external
tools
which
can
deal
with
certain
media
types
and
process
that
that
byte
stream,
for
example,
decrypting
compressing
in
special
ways
decompressing.
A
A
There
is
remote
snapshot
or
work
for
sharing
snapshots
in
a
cluster.
So
there's
a
lot
of
interest
around
this.
Maybe
you've
heard
of
Google
CR
FS
CERN
has
been
interested
with
our
CVM
FS.
So
you,
you
know,
you've
got
a
clustered
architecture.
You
don't
want
to
pull
images
to
every
node
in
the
cluster,
because
you
have
a
shared
file
system
that
you
want
to
pre-populate.
A
So
this
is
a
there's
a
lot
of
interest
in
this.
There
are
several
issues
and
kind
of
proposals
open
and,
and
that
work
is
has
been
started,
see
groups
v2.
This
is
work.
That's
already
ongoing.
You
can
look
at
the
the
see
groups.
Sub
project
of
container
D
on
github
you'll
see
a
lot
of
PRS
in
the
last
few
weeks,
bringing
a
bringing
in
see
groups
v
to
support
the
CRI
team
is
working
on
more
complete
Windows
support
and
getting
that
into
CI.
There's
a
whole
aspect
of
better
management
for
mounts
and
resources.
A
That
container
D
would
give
you
ways
to
to
help
manage
those
life
cycles.
Derick
can
talk
a
bit
more
about
that
if
you're
interested
and
then
again
I've
mentioned
the
image
encryption
so
kind
of
finalizing
and
finishing
up
that
work
in
the
CRI,
so
there's
sort
of
that
full
life
cycle
through
kubernetes
to
deal
with
encrypted
image
layers.
C
C
Management
though,
but
we
can,
we
can
talk
about
that
at
the
we
can
talk
about
that
at
the
end,
all
right
so
I'm
gonna
go
through
the
architecture
of
container
D.
Some
we're
gonna
drill
into
the
different
parts
of
it.
I'm,
hoping
that
at
the
end
of
this
you'll
have
a
idea,
if
you're
a
user
of
container
D,
whether
you
know
it
or
not,
we're
spitting
in
your
stack
today
and
if
you
want
to
contribute
to
container
D.
C
Maybe
you'll
get
an
idea
of
where
you
might
fit
into
this
or
based
on
what
you
know
about
container
D
or
what
you're
interested
in
so
we're.
First
gonna
talk
about
container
D,
mostly
focused
on
the
the
Dayman
part,
which
is
going
to
be
the
the
center
part
of
this.
So
first
thing
we're
going
to
talk
about
the
API.
This
is
how
users
are
going
to
interact
with
container
D
itself,
so,
whether
that's
through
the
CRA
interface
or
through
container
DS
g
RPC,
or
just
getting
metrics
through
container
d.
C
So
for
the
CRI
implementation,
it's
it's
actually
just
a
plugin.
It's
a
g,
RPC
plug-in,
that's
built
in
so
there's
no
extra
hop
or
anything.
You
just
used
the
container
d
socket
that
everything
uses
and
point
the
cubelet
directly
at
container
D
in
it
will
it
will
be
able
to
use
that
interface.
So
cubelet
can
just
be
configured
to
use
container
D
directly
as
a
runtime
using
its
socket.
C
Also,
we
have
the
the
G
RPC
API,
which
is
primarily
used
by
the
go
clients.
So,
unlike
if
you're
familiar
with
the
CRI,
API
or
or
the
the
docker
API,
it's
a
little
bit
higher
level,
whereas
container
da's
api
is
pretty
low
level,
it
gives
you
access
to
roughly
the
components
that
container
DS
using
internally.
C
So
if
whether
it's
a
snapshot
so
it'll
give
you
direct
access
to
what
the
snapshot
or
API
is
content
store,
containers,
creating
containers,
creating
tasks,
events,
the
GRP
capi
mirrors,
what
the
underlying
services
actually
look
like
and
then
for
metrics
exposed
through
Prometheus
8
through
Prometheus
API.
This
will
actually
expose
metrics
for
the
container
D
process,
as
well
as
container
level
metrics.
C
C
So
these
are.
This
is
a.
This
is
roughly
what
the
components
and
services
that
we
have
in
container
D
are,
most
of
them
are
represented
here.
So
labo
makes
sense
of
image
service
we'll
have
a
snapshot
service
container
service
we'll
also
have
stuff
for
handling
the
namespaces.
That
container
D
does
we'll
have
a
lease
service
which
is
actually
used.
C
You
just
use
part
of
our
resource
management
so
that
it
helps
container
be
able
to
keep
track
of
what
objects
that
are
created,
who's
using
it
and
tracks
when
we're
actually
able
to
remove
those
our
metadata
store
and
container
D.
It's
it
handles
just
like
labeling
on
all
our
different
objects
that
we
have
all
the
resources
in
ensures
that
they're
all
referenced
by
top-level
resources,
such
as
like
an
image
or
or
a
container
when
any
of
these
resources
are
no
longer
used.
We
have
a
garbage
collector
that
will
actually
go
through
and
keep
track
of.
C
C
So
if
two
of
them
are
using
the
same
hole,
the
same
content
from
the
registry,
they
can
both
reference
that
same
content
and
metadata
store
will
keep
track
of
who's
actually
using
it
and
then
once
it's
no
longer
used,
it
will
be
able
to
garbage
collect
that
now
in
the
back
end
of
container
D.
This
is
where
you
would
get
more:
the
resource
management
so
stuff
like
storing
the
actual
snapshots
file
systems,
as
well
as
the
content.
C
These
are
all
handled
and
our
back
ends
as
well
as
the
actual
runtime
implementations
that
call
out
to
all
the
runtimes
I
had
mentioned
before
whether
it's
run
C
or
you
need
implementations.
So
for
the
content
store,
we
have
a
built-in
local
implementation.
So
this
is
pretty
it's
a
pretty
simple
implementation,
it's
just
as
a
file
system
and
the
content
is
named
by
attached.
That's
just
a
Content
address
file
system
layout
and
there's
the
ability
to
plug
stuff
in
right.
C
Now
you
can
only
use
one
at
a
time,
but
we
do
have
a
plug-in
point
there.
So
you
actually
swap
out
the
content
store.
Snap
shutters
are
a
little
different.
There's
there's
multiple
built-in
snap
shutters
and
the
clients
can
actually
select
which
now
chartered
to
use
the
snapshot
is
gonna
is
what
provides
the
copy-on-write
storage
analogous
to
Dockers
graph
drivers.
We
have
Union
file
system
implementations
such
as
overlay,
FS,
au
FS,
as
well
as
block
device
implementations.
So
we
just
added
device
mapper
and
use
better
FS
as
well
as
ZFS.
C
So
you
can
have
images
that
aren't
in
the
snapshot
or
you
can
have
images
that
actually
are
unpacked
into
multiple
snapshot
ORS
here
and
let
the
client
manage
all
of
those
now
I
mentioned
the
runtime.
So
it's
an
example
of
some
of
the
runtimes.
We
have
implementations
for
so
in
container
D
1.2.
We
stabilized
the
Ashim
interface.
C
So
this
this
API
it
uses
it
uses
a
lightweight
G
RPC
called
TT
RPC.
It's
like
teeny-tiny
RPC.
It
has
a
really
small
memory
footprint
and
each
of
the
each
of
the
shims
are
a
separate
binary.
They'll,
listen
on
a
little
listen
on
a
socket
using
TT
RPC
and
then
the
container
dv2
runtime
has
a
client
which
will
connect
to
those
runtimes
to
actually
manage
all
the
container
lifecycle
be
choose
now
default
of
v1
still
in
there.
But
it's
it's.
It's
deprecated.
C
So
I'm
going
to
go
through
a
few
flows
using
some
of
these
service
interfaces,
you
can
see
how
you
would
pull
an
image,
push
an
image
and
run
a
container
using
container
D.
So
first
we're
going
to
go
through
the
poll
flow,
so
some
of
the
actors
that
we're
going
to
go
through
is
we
have
the
DIF
service,
the
content,
service
snapshot
or
an
image
service.
It's
got
a
kind
of
got
an
idea
of
where
that
fits
into
the
core
of
container
D.
C
There
were
gonna,
have
a
client,
that's
actually
gonna,
be
driving
the
poll
operation
and
then,
since
in
this
case,
there's
going
to
be
a
registry
involved
implementation
of
the
distribution
API
v1,
whatever
some
day
but
using
the
Saros
I
guess
but
yeah
any-any,
OCI
registry
or
docker
registry.
Whatever
you
want
to
call,
it
will
will
work
here.
The
first
thing
that
happens
in
a
poll
in
this
case
we're
gonna,
get
the
manifest.
So,
unlike
in
docker,
where
it's
always
the
daemon
that's
going
to
be
connecting
to
the
registry,
it's
a
little
different
with
well.
E
C
A
little
different
with
container
D
and
that
it's
our
client
that
actually
does
all
the
poll
implementation
so
in
the
docker
case
it
still
docker
connecting
to
the
registry.
But
if
it's
the
CRI
plug-in
this
year,
I
plugin
connects.
But
if
you
have
a
client
such
as
build
kit
or
CTR
or
some
custom
tooling,
you
have
that's
gonna
pull
an
image.
It's
actually
your
client,
that's
importing
the
the
go
container
decline.
That's
actually
going
to
do
the
connection
to
the
registry
and
then
from
there.
It
just
looks
like
just
one
more
each
to
be
interaction.
C
It's
going
to
it's
going
to
request
the
manifests
going
to
do
some
authorization
and
then
it's
going
to
pull
down
the
resources
that
it
needs
for
that
manifest
once
it
has
those
resources,
it's
going
to
connect
to
container
DS
content
service
and
it's
going
to
store
that
manifest
and
then
for
each
layer.
It's
actually
going
to
go
back
to
the
registry.
It's
going
to
retrieve
each
of
those
layers,
which
is
just
one
request
to
go
in
request
that
layer
content
and
then
it's
going
to
put
each
of
those
layers
into
the
content
service.
C
Now,
for
each
of
those
layers
just
pulling
the
layers
isn't
necessarily
enough
to
run
an
image
so
to
unpack
those
for
each
the
layers.
First,
we're
going
to
prepare
a
snapshot.
That's
going
to
go
back
through
if
it's
from
the
container
engine
into-
let's
say
the
overlay
snapshot
or
it's
going
to
connect
through
or
g4c
interface
talk
to
the
the
service,
and
then
that's
going
to
go
back
to
the
backend,
then
from
there
the
we're
going
to
use
a
disservice
actually
do
the
unpacking.
C
So
in
this
case
the
the
client
will
just
send
to
the
diff
service
what
what
the
layer
that
it
wants
to
unpack,
as
well
as
the
mounts
that
it
got
from
preparing
the
snapshot.
The
dis
service
will
actually
get
the
content
directly
from
the
content
service,
so
the
data
is
not
going
kind
of
back
and
forth
to
the
from
the
content
service
back
to
the
client.
The
disservice
allows
doing
this
directly.
C
Then,
at
the
end,
after
that,
after
that
layer
has
been
unpacked
yeah,
it
will
be
committed
back
to
the
snapshot,
or
now
it
went
from
having
a
read/write
layer
where
it
could
do
this
unpacking
now,
I'll
have
a
read-only
layer
and
then
we'll
create
an
image
from
that
and
that's
gonna
be
the
end
of
the
pole
flow.
The
push
flow
is
pretty
simple
in
container
D.
We
just
have
an
image
service
and
a
Content
service
involved.
The
first
thing
container
D
is
going
to
do.
Is
it's
going
to
get
the
image
from
the
image
service?
C
C
So
it
will
read
the
manifest
from
the
manifest
will
read
all
the
layers
read
those
layers
send
them
all
up
to
the
registry,
send
up
the
manifest
and
that's
all
there
is
to
the
push.
So
if
you
were
to
compare
this
to
say
what
what's
done
in
docker
today,
there's
a
lot
more
involved
in
the
push,
because
it's
actually
generating
a
lot
of
this
this
content,
so
it's
actually
would
be
going
to
Dockers
of
equivalent
of
the
snapshot
or
to
actually
generate
these
layers
on
the
fly.
C
It
doesn't
actually
have
the
manifest,
so
it'll
actually
generate
a
new
manifest.
So
in
this
case,
container
D
assumes
that
you
either
have
some
builder,
that's
doing
it
or
you're
keeping
that
content
around.
If
you,
if
you
intend
to
reach
it,
okay,
so
let's
get
into
what
a
what
a
run
image
looks
like
so
likewise,
there's
still
going
to
be
the
snapshot
or
service
involved,
but
we're
also
going
to
be
using
the
container
service,
the
task
service
and
then,
as
well
as
a
runtime
implementation,
and
those
runtimes
will
talk
to
Shem.
C
So
first,
we're
gonna
prepare
a
snapshot.
So
that's
going
to
take.
Usually,
as
a
base,
some
some
read-only
snapshot,
so
that
could
be
whatever
your
image
snapshot.
Is
it's
going
to
prepare
a
writable
layer
on
top
of
it,
then
it's
going
to
create
a
new
container
using
using
that
snapshot
and
whatever
your
your
configuration
is
for
that
container
and
then
it's
going
to
call
into
the
test
service
create
a
new
task
so
that
task
service
is
going
to
go
down
into
the
runtime.
C
That
runtime
is
actually
going
to
exec
your
sim,
which
will
create
a
a
new
shim
process,
and
then
it's
going
to
tell
that
new
shim
process
to
create
create
this
container,
create
this
task
inside
that
shim.
The
runtime
is
going
to
return
that
task
back
to
the
task
service,
and
then
the
task
service
will
pass
oh
and
the
pid'
and
the
container
IDs
back
to
the
client.
C
At
this
point,
container
d
itself
doesn't
have
a
network
set
up.
The
client
can
do
this,
so
if
it's
the
CRI
implementation,
will
you
see
an
eye
to
do
all
the
network
setup?
But
at
this
point
you
have
the
process
ID
that
you
need
in
order
to
do
anything.
So
all
the
namespaces
are
created,
you
can
do
anything.
This
is.
This
is
normally
at
the
time
which
the
network
setup
is
gonna,
be
done
that
a
client.
C
C
Wait
on
that
container
and
then
it's
going
to
start
that
container.
So
the
start,
so
it's
going
to
go
down
to
the
shims
can
just
tell
it
to
start,
and
then
you
have
a
running
running
container
when
the
container
exits
that
exit
is
going
to
return
from
that
same
wait
and
then,
and
then
the
client
and
go
ahead
and
do
any
cleanup
that
it
wants
to
do
if
it
when
it
goes
and
actually
deletes
the
task.
C
C
You
can
build
them
in
as
well,
but
the
proxy
plugins
allow
basically
taking
a
container
deep
binary
and
you
just
you-
can
just
configure
it
without
recompiling
it
to
test
out
your
mostly
for
the
snapshot.
Ors
it's
the
most
useful,
but
it
allows
you
to
do
your
implementation
without
having
to
continuously
recompile.
C
This
is
how
the
device
mapper
snapshot
era,
was
originally
created
and
made
it
much
much
easier
to
implement.
Likewise
for
the
the
runtime
shim
since
they're,
just
separate
binaries,
there's
no
need
to
recompile
or
make
a
bunch
of
changes
to
container
D.
They
can
just
be
implemented
against
our
our
interface
for
the
clients.
We've
really
made
a
push
to
make
our
client
very
extensible,
so
we
want
basically
the
ability
to
customize
any
part
of
the
flow,
since
our
API
is
pretty
is
broken
down
into
low-level
components.
C
We
wanted
Bill
they'll
switch
out
any
of
those
components
do
any
sort
of
customization
or
or
plugins
that
you
want
at
in
in
the
client.
So
a
common
pattern
you'll
see
is
we
have
these
Shinzon
crating
when
you
create
the
when
you
actually
create
the
client
or
when
you
create
some
of
these
services
or
you
go
to
do
a
poll
so
we'll
have
these
like
service
options
where
you
can
actually
go
and
say,
configure
container
D
I
want
to
use
this
implementation
of
this
service.
C
You
can
just
override
one
service
and
keep
all
the
other
defaults
or
you
can.
You
can
have
a
complete
implementation
if
you
wanted
to
say,
have
a
client
that
didn't
talk
to
container
daemon
at
all.
You
can
do
that
here
as
well.
Just
like
I
think
good
example,
that
is
when
we
move
CR
I
used
to
be
out
of
process,
and
we
moved
it
in
process.
It
was
able
to
leverage
this
too
from
using
container
D
from
a
separate
as
a
separate
process
to
actually
a
plugin
in
that's
built
into
container
D.
C
It
was
able
to
just
basically
switch
these
options
to
say
now.
I
just
want
to
use
these
service
implementations
that
are
inside
of
the
process
rather
than
connecting
over
G
RPC.
Likewise,
for
we
have
these
remote
options
that
allow
you
to
basically
customize
by
replacing
the
resolver
you're,
basically
replacing
the
entire
I
pull
flow.
You
can.
You
can
not
even
use
a
registry
at
all
like
our
default.
C
So
what
we've
tried
to
do
internally
is
is
use
kind
of
the
same
plug-in
model
that
we
would
that
we
would
ask
kind
of
anyone.
That's
implement
a
plugin
to
do
so.
Like
we've,
we've
tried
to
make
all
our
internal
components
loosely
coupled
clear
boundaries
for
how
these
plugins
will
interact
with
each
other.
C
We
define
like
different
types
for
what
these
plugins
are
and
then
just
each
of
the
plugins
register
themselves.
So,
even
internally,
all
these
components
that
you
saw
they
all
just
register
themselves
as
plugins.
They
all
define
what
their
dependencies
are.
So
there's
no
place
in
container
d,
that's
basically
listing
now
or
defining
like
what
the
order
you
know
what
all
these
plugins
are.
The
plugins
will
actually
define
this.
C
We
will
create
a
benzene
dependency
graph
and
actually,
when
we
start
up
container
D,
it
will
resolve
that
graph
and
initialize
everything
in
the
in
the
correct
order.
So
then,
if
you
wanted
to
add
your
own
custom,
you
want
to
customize
container
D,
like
you.
Have
your
own
container,
D
binary
and
add
your
own
built
in
plugins.
It's
pretty
easy.
All
you
have
to
do
is
is
import
all
the
components
that
you
want.
You
can
import
all
of
container
DS
and
add
your
own.
C
You
can
swap
them
out,
and
then
you
just
build
your
own
main
function
and
the
main
function
is
pretty
small.
We
just
initialize
ya
and
run
it
so
I'm
gonna
go
through
a
quick
example
of
what
the
snapshot
our
practice
we
plugin
looks
like.
So
if
we
have
a,
if
you
look
at
what
our
snapshot
or
interface
looks
like
it's,
it's
pretty
small,
it's
what
you
might
expect
from
a
copy-on-write
interface.
You
can
prepare
something
you
can
commit
something,
remove
something
and
list
it's
pretty
simple.
We
use.
C
We
actually
return
mounts
instead
of
doing
the
operations
inside
the
snapshot,
so
the
snapshot
itself
doesn't
handle
any
data
coming
in
and
out
of
it.
It
can
just
give
you
melons
that
the
client
can
use
in
order
to
operate
on
those
file
systems.
So
you,
as
an
implementer
of
this,
just
need
to
implement
our
the
snapshot,
orgy,
RPC
API
and
in
the
backend
we'll
proxy
those
requests
when
it's
configured
to
use
that
and
then
to
to
compile
it
as
a
separate
binary.
We
we
have
some
helpers.
You
just
take
your
snapshot,
our
implementation.
C
Likewise,
for
the
runtime
plugins,
the
Shin
v2
API,
it's
it's
pretty
minimal
and
what
what
it's
designed
to
do
it
handles
the
execution
lifecycle
of
a
container
to
use
like
if
you
create
your
home
and
you
want
to
test
it
out,
you
can
pass
in
the
runtime
using
here's
an
example
where
you
would
use
IO
container
D
run
SC
v1,
and
this
will
actually
look
for
the
binary
container.
D
Tim
run
SC
v1,
and
it
will
use
that
when
you
specify
that
runtime
and
then
to
actually
do
the
implementation,
you
implement
the
the
task
service.
C
The
test
service
handles
the
crate
deleting
and
everything
you'd
expect
to
do
from
to
a
container
at
a
low
level,
including
getting
kind
of
the
metrics
or
getting
the
stats
on
it.
Shutting
it
down
all
that
kind
of
stuff,
and
we
have
I,
didn't,
have
a
slide
for
it,
but
it's
similar
to
the
snapshot.
Api.
We
have
helpers
where,
if
you
have
an
implementation
of
the
tasks
service
can
create
a
binary
pretty
easily
directly
from
from
the
implementation.
C
F
F
Okay,
hi
Lantau
from
Google
and
I'm,
both
oaken
dirty
and
kubernetes
maintainer
and
oh
by
the
way
I'm
using
a
different
template.
This
is
the
the
cube
contemplated
by
to
be
honest,
I
think
the
direct
Y
is
much
better
than
this
one,
but
yeah
let's
go
with
this.
I
don't
have
time
to
change
it
and
I'm
Lantau
and
I'm.
Going
today,
I'm
going
to
introduce,
give
some
status
update
of
the
some
business.
F
We
are
doing
to
make
kubernetes
work
on
Windows
and
using
canned
air
D
as
the
container
and
time
instead
of
talker
and
I'm
happy
that
actually
in
the
community
community,
we
always
we
keep
have
new
things
coming
up
so
that
I'm,
so
that
I
wake
up.
I
can
talk
about
something
new
and
fork.
You
can
Shanghai
talk
about
G,
Weiser
and
demo
day
and
now.
F
I
have
something
new
to
demo,
which
is
the
Windows
support,
so
I'm
going
to
briefly
introduce
it
and
demo
have
a
lab
demo,
so
the
content
I
will
first
briefly
introduce
what
is
Windows
container.
There
is
some
other
session
we'll
go
deeper
into
this.
Cube
has
already,
and
indeed
but
there
was
some
session
so
but
to
to
to
to
to
to
make
it
easier
to
understanding
the
following
things.
We
still
need
to
cover
a
briefly
cover.
G
F
Firstly,
Windows
container,
and
actually
there
are
two
flavors,
so
fondos
container
one
is
called
the
Windows
Server
container,
which
is
more
Linux
container
like.
Basically,
all
the
containers
on
the
node
will
share
the
same
Windows
kernel
and
those
containers
are
isolated
with
some
some
process:
isolation
technology
in
even
those.
So
that's
one
another
one
is
hyper-v
container
and
in
this
flavor,
basically,
each
container
is
just
a
hyper-v
VM
and
each
VM
has
its
own
kernel
inside
this
is
hyper-v
and
it
feel
compared
them.
F
You
must
have
it
and
Microsoft
provide
two
kinds
of
base
layers.
One
is
a
very
large
one
called
Windows
server
core.
It
has
the
best
compatibility,
but
it's
large
after
unconfessed
is
5
gigabytes,
so
it
takes
a
long
time
to
pull
it
down
and
and
another
one
is
called
windows
nano
server.
So
it's
much
smaller,
but
it
only
supports
a
fixed
out
of
framework
or
application.
So
you
may
want
to
choose
which
one
you
use
based
on
your
application
and
for
windows
server
container.
F
They
have
the
hard
common
that
your
this
space
layer
have
to
match
the
major/minor
build
version
of
the
host
OS.
So
let's
use
an
example
on
the
right
side.
So
that's
it.
Let's
say
that
today
you
have
a
container
which
which,
whose
base
layer
is
had
the
exactly
the
same
version
with
the
host
version
and
tomorrow
you'll
want
to
update
the
host.
If
you
are
only
updating
the
the
revision
number
that
last
digit,
that's
fine
everything
still
works,
but
if
you
are
updating
the
second-last
the
build
number,
then
you
can
hear
don't
run.
F
So
please
keep
this
in
mind
because
this
may
affect
you.
If
you're
using
Windows
server
container,
this
may
affect
your
CI
CD
workflow.
You
may
need
to
rebuild
your
image
whenever
you
upgrade
the
host
OS
version
and
another
interesting
thing
that
Windows
container
also
supported
linux
Canon
on
Windows,
so
they
have
the
special
term
double
cause
windows
around
can
windows.
Alcoholics
links
can
do
now
on
Windows
and
basically
before
to
support
Linux.
F
They
just
run
a
hyper-v
VM
and
when
running
the
container
and
Linux
kernel
inside
the
VM,
that's
another
thing
you
may
want
to
know
and
then
now,
let's
talk
about
the
content,
the
windows
support
thanks
to
the
cleaner,
the
nerdiest
will
defend
plug-in
interface.
It's
actually
Derek
has
covered
a
lot
of
them.
F
So
there
are
a
lot
of
plug-in
point
in
Kennedy
and
the
windows
police
I
did
basically
just
the
several
canor
debugging,
so
it
doesn't
touch
I
didn't
touch
the
kinetochore
at
all
and
for
the
image
part
way
needs
snaps
our
snapshot,
a
plugin
and,
if
plug-in,
when
we
needed
for
both
ll
cop
Linux
container
and
double
cut
windows
container
and
for
the
runtime
part
they
are.
We
need
a
shame
between
communication,
basically
a
canary
stream
that
understand
how
to
start
and
manage
windows,
container
and
microsoft
period
and
is
out
of
the
canary
repository.
F
So
for
the
image
part
I
think
the
work
Derek
has
already
covered
the
workflow.
Basically,
the
only
thing
we
need
to
change
is:
we
need
a
deep
plugin
to
apply
the
changes
for
each
layer
and
also
need
a
window
snapshot
her
to
manage
the
snapshots.
The
reason
that
on
linux
we
have
overlay
file
system
and
we
have
patio
file
system,
but
windows.
They
have
done
something
different,
it's
similar
with
a
union
file
system,
but
it's
just
different,
so
they
need
they
need
their
own
plugin
hi
that
logic.
F
So
that's
how
the
image
is
handled
and
for
the
runtime
part.
Basically,
we
just
need
to
tell
that
clearly
understand
that
for
Windows
container
you
need
to
use
this
shim
instead
of
the
standard
one
and
this
and
the
leadership
will
understand
how
to
talk
with
the
windows,
opening
system,
the
host
computer
service
to
start
and
manage
the
container
and
then
the
random
part
is
done.
So
it's
just
that.
That's
simple
and
here's
the
state
herself,
the
canary
suburban,
no
support.
F
So
we
support
both
windows,
server,
container
and
hyper-v,
and
we
support
both
table
car
and
alcohol,
and
everything
is
nothing.
This
is
feature
complete
in
1.3
but
because
of
last
I
mean
lack
of
test
coverage.
So
we
still
call
it
experimental
so
we're
working
on
improving
the
test
coverage
and
announce
table
as
soon
as
possible,
and
now,
let's
get
to
the
Windows
part,
how
a
house,
the
windows,
support,
sorry
kubernetes,
part
how's,
the
windows
holding
kubernetes
today,
so
here's
the
windows
part
look
like
in
kubernetes
today.
F
Basically,
each
pod
has
multiple
windows,
server
containers
and
all
those
containers
share
the
same
IP
and
they
share
volumes
means
that
they
can
access
the
same
volume,
but
on
windows
there
is
no
namespace
sharing
the
only
namespace
they
have
is
network
namespace.
So
for
them,
how
on
linux
you
can
have
PID
namespace
sharing
in
can
heroin.
You
can,
if
you
configure
it
correctly,
you
can
see
all
the
process
processes
in
compared
to
which
is
not
possible
today
and
also
there's
no
pod
level
resource
limit
on
Linux
actually
at
the
pod
level
level.
F
F
Here's
a
picture
of
after
we
have
hyper-v
support.
Things
will
be
looked
like
they,
especially
at
the
pod
arrival
we're
going
to
have
a
hyper-v
VM
and
instead
hyper-v
VM.
You
have
your
own
Windows
kernel
and
multiple
instance
windows
server
container.
So
that's
what
a
Windows
pod
look
like
in
kubernetes
today
and
for
the
scheduling
part
the
earth
samms
there's
some
some
issues.
Basically,
today,
even
you
only
want
to
run
windows
were
closed.
F
You
still
need
some
Linux
knows
in
your
cluster,
one
of
the
reason
that
actually
some
system
items
and
also
the
whole
windows,
the
whole
kubernetes
control
plane
only
support
Linux
today,
so
you
have
to
run
those
on
a
set
of
linux
node
and
then
create
another
set
of
windows,
know
to
run.
Your
windows
were
closed,
so
means
that
all
at
least
today,
all
the
windows
cluster
will
have
both
linux
nodes
and
windows
nodes.
F
To
make
sure
that
you,
your
Linux
container,
only
run
Linux
run
on
Linux
nose
and
with
no
windows
no
do
need
some
scheduling,
Magnum
and
well.
Today
we
are
using
the
node
label
and
tint
basically
for
all
the
windows
nodes.
We
were
going
to
apply
the
label
that
label
and
that
change
by
default
and
with
the
label
you
can
say
I
want
to
run.
My
Windows
part
knows
having
these
labels,
which
are
windows
nodes
and
with
the
10th.
F
It
means
that
by
default,
all
the
parts
are
rejected,
because
most
powers
are
Linux
parts,
so
they're
all
rejected.
If
you,
if
you'll,
have
a
Windows
workload,
you
won't
see
that
I
want
to
run
on
Windows.
You
need
to
add
some
toleration.
So
basically,
that's
how
you
run
windows
were
closed.
Today,
you
need
to
apply
the
no
distract
her
seeing
that
I
want
to
line
around
windows
and
also
the
Toleration
means
that
I'm
allowed
to
run
Windows.
F
F
I
have
a
support,
and
one
worth
noting
limitation
in
that
there's
no
privilege
or
host
name
space
support,
which
calls
some
challenge
for
demons
eyes
like
black
CSI,
because
CSF
needs
privilege
to
do
some
host
operation,
which
you
can
not
easily
do
on
Windows,
and
there
are
some
work
in
progress
proposal
or
solution
like
CSI
proxy.
If
you're
interested
you
can
go
and
look
at
the
proposal
in
covering
his
up
stream
and
for
other
limitations,
you
can
check
the
doc
I
have
linked
here
and
now.
F
Finally,
let's
get
to
our
recent
work,
which
is
make
everything
work
together
and,
firstly,
just
one
page
to
to
review
her
to
help
us
reveal
what
is
the
RI
plug-in?
Which,
because
for
can
you
run
him
to
work
with
kubernetes?
You
need
to
implement
the
kubernetes
random
interface
and
for
compare
d.
Our
implementation
is
the
native
plugin
of
kennedy.
It
was
introduced
in
the
1.1
and
GA
since
april
last
year,
and
here
is
how
everything
works
on
linux.
F
Basically,
communities
gave
us
a
CRI
container
config
for
linux,
CRI
container
config,
and
we
do
some
conversion.
We
have
some
logic
inside
the
plug-in
and
converted
to
a
OCI
containers
back,
of
course,
also
for
Linux
container,
and
we
we
create
a
pod
level
namespace
and
three
groups,
and
we
use
Linux
in
my
plug-in,
to
set
up
the
network
and
with
y'all
can
hurdy
that,
for
this
can
for
this
group
of
container,
please
use
the
standard
stream
instead
of
some
something
else.
F
So
so,
after
all
this,
we
have
a
pod
running
and
can
funny
inside
and
for
windows.
Things
are
very
similar,
basically
well.
The
the
config
we
got
from
kubernetes
is
windows
can
config
and
we
need
not
add
new
logic
to
convert
it
to
OC.
I
can
inspect
for
windows
and
windows,
windows
can
in
the
OCS
back,
and
we
need
to
setup
the
pod
level
network
namespace.
As
I
said,
they
only
support
network
namespace
and
we
for
each
part
we
need
a
single
IP.
F
So
we
need
to
start
up
this
at
the
power
level
and
Microsoft
also
provides
some
windows
singer
plug-in
like
iced,
tea
in
bridge
or
I
forgot,
the
other
one,
the
name
of
a
steam
overlay
something
so
we
need
to
use
those
thinit
having
to
set
up
the
network
and
windows
and
of
course
we
need
to
talk
and
air
D.
This
time
use
some
special
shape
for
Windows,
don't
use
the
regular
fancy
one
and
then,
as
I
said,
the
shim
knows
how
to
handle
the
canary
life
cycle.
After
that.
So
that's
it.
F
We
support
windows,
kubernetes
and
nearly
all
together,
that's
how
it
works.
And,
yes,
some
stickers
update.
Actually
everything
are
they
works
in
hat,
except
some
small
or
some
small
work-in-progress
atoms,
like
library,
saw
that
they
feel
cute
and
parody
after
it
comes
back.
There
is
some
problem
with
picking
up
old,
running
containers.
We
are
fixing
fixing
that,
and
also
it
doesn't
support
GMS
a
yet
and
there's
no
happy
we're
working
still
working
on
hyper-v
support
right
now
and
we
have
all
the
CRI
level
conformance
tests
passing.
F
F
Our
plans
to
have
alpha
supporting
nearly
1
point
for
rescue
Bernie
has
one
point
18
and
although
walking
provides
items
are,
should
be
done
at
that
time
except
the
hyper-v
stuff,
and
we
should
have
a
plan
to
have
a
quibble
and
has
coverage
with
docker
and
Windows
and
for
the
hyper-v
part
we
are
going
as
I
said,
with
the
old
plan
to
add
it
into
a
nerdy,
but
it
will
be
probably
a
separate
release.
We're
making
good
progress
on
it
right
now
and
now
it's
the
demo
time.
F
So
well,
I
won't
demo
the
castor
bring
up
because
it's
super
slow,
but
this
is
how
you
bring
up
a
caster
today.
There
is
a
pending
change
in
upstream
and
we
are
going
to
merge
it
very
soon.
So
after
it's
merged,
you
just
need
to
run
this
script
and
you
can
have
a
communication.
Those
castor
with
canary
are
all
the
knows,
other
things.
No,
the
windows,
node
and
here's
the
thing
I'm
going
to
demo
and
I
already
have
a.
F
White
yeah
well
master,
nose
to
Linux
nose
and
two
windows
nose,
and
you
can
see
that
container.
You
used
all
the
nose,
but
on
Windows
nose
their
version
is
unknown
because
I
built
it
from
Canada
hi
that
we
don't
have
a
semantic
version
yet,
but
it's
your
ingenuity
and
you
can
see
that
is
Windows
Server
data
center
and
if
you'll
take
a
look
at
the.
If
you
describe
the
node,
you
will
see
the
the
label
and
her
tenth
I
mentioned.
So.
F
Here's
the
label
we
market
as
core
any
windows
or
ass,
and
also
the
tends
to
avoid
other
parts
to
schedule
here.
Okay
and
now,
let's,
let's
create
a
simple
workload.
So
in
the
workload
I'm
going
to
have
a
service
is
this:
is
a
web
server
and
I'm
going
to
have
a
service
exposed
by
node
port
and
the
service?
The
service
itself
is
just
a
web
server
written
in
PowerShell
and
you
can
see,
is
Windows
image
and
to
make
it
scheduled
to
windows.
F
F
Okay,
I
think
it
okay,
it's
money.
Actually,
the
less
I
said
that
the
image
the
windows-based
image
is
super
big,
so
it
really
takes
minutes
to
download,
but
because
I
already
downloaded
on
the
note,
so
it's
very
fast
okay,
what
do
I
do
that's,
first,
exactly
into
the
the
part
to
make
sure
it
is
Windows
I'm,
not
cheating
you
you
guys.
So
that's
one
approach,
PowerShell!
F
F
F
F
Let
me
demo
something
else.
Basically,
most
of
the
things
you
know
should
work.
For
them
whole
we
can
even
put
forward.
If
you
don't
know,
port
forward
is
basically
forward
a
remote
container,
a
pod,
a
port
of
the
the
party
into
your
local
environment,
so
that
you
can
access
it
locally,
and
even
this
works,
let's
try
it
so
I
get
parts.
This
is
a
part.
F
This
and
that's
4
2
1,
1,
2,
3,
4,
5,
ok,
and
now
it
the
police
were
happy
they
being
forwarded
to
a
local
environment,
honey
whew.
Let's
see,
let's
try
access
localhost
one.
Two,
three
four
five
years
we'll
be
able
to
see
it.
Yeah
you
see
so
port
forward
also
works
and
also,
as
demo
also
demos
getting
the
status
of
the
container.
We
feel
no
cube
cut
off
top.
It
shows
you,
the
status
of
the
container.
You
can
see,
we
have
the
memory,
usage
and
scipio
is
zero
because
it's
idle.
F
G
F
Okay,
I
was
hired
later.
Oh
yeah
I
just
tried
it
before
before
the
the
hawk
issue,
however,
try
it
after
the
next
thing.
I
do
and
also
I
want.
Another
thing:
I
want
to
show
you
that
you
are
the
local
debug
to
also
work.
So,
let's
try
to
get
on
to
one
of
the
windows
node
this
one.
She
can't
computer.
F
So,
let's
go
down
to
the
windows:
node
you'll,
see
it's
windows,
I'm
more
familiar
with
PowerShell
is
use.
Powershell
and
oh
sorry,
I
need
to
export
one
thing,
so
4k
nerdy
the
local
debug
tool
and
actually
also
for
cryo
cryo.
The
local
debug
tool
we
use
is
Craig
Hutto
and
I
Hadi
install
in
the
custard
Terina,
castor
bootstrap,
where
I
really
started
on
the
node.
So
if
we
run
cuddle
pause,
you
can
see
that
all
the
parties,
when
you
can
see-
and
it
is
Windows
part
an
different
peers.
F
F
F
Yes,
car,
so
this
way
most
things
you
know,
I,
do
you
work
for
Windows
and
there
are
some
limitations
in
the
document.
You
can
go
and
take
a
look
at
least
if
you're
compared
to
nerdy
way
stalker.
There
shouldn't
be
too
much
difference
today,
besides
the
the
the
working
progress
part
and
we're
going
to
finish
done
in
community
1.4,
and
that's
it
thank
you.
Yeah
and
all
we
have
panel
discussion.
C
D
D
D
B
I
ask
about
the
recent
work
about
remote
snapshot.
There
yeah,
especially
like
a
yeah
and
a
Google
CLA
face,
and
it's
arm
via
email.
If,
if
and
I
also
add
your
terrible
deprivation,
teleportation
Airport,
could
you
could
you
give
me
no
maintenance
opinion
about
remote
snap
show
this
in
a
progress
on
me.
C
E
C
No
necessarily,
if
you
have
shared
storage
between
those
between
those
nodes,
so
the
idea
between
remote
snapchatters
is
that
when
you
pull
in
an
image,
you
can
share
those
snapshots
across
all
the
nodes.
So
the
way
container
D
is
architected.
We
have
kind
of.
We
have
this
resource
management
layer
that
keeps
track
of
all
the
snapshots
that
the
node
knows
about
and
then
underneath
it
we
have
snapshot
ORS
that
actually
do
the
storage.
So,
even
if
something
is
available
on
the
storage,
we
don't
necessarily
know
about
it
and
container
D
from
a
resource
management
perspective.
C
So
the
main
work
behind
the
remote
snapchatters
and
that
we're
gonna
have
in
the
next
release
is
updating
this
updating
that
middle
core
layer.
So
that
we
can
actually
make
use
of
the
snapshots
if
they're
available
on
the
back
end,
so
that
when
we're
actually
doing
the
poll
process,
we
don't
have
to
actually
go
through
those
steps
of
let's
pull
something
and
then
try
to
extract
it.
If
the
extraction
doesn't
need
to
take
place
at
all,
so
we
can
essentially
try
to
do
that.
C
Preparation
of
the
snapshots
before
we
even
get
to
the
point
of
pulling
it
so
that
we
never
have
to
open
up
that
connection
to
the
registry.
So
what
it
probably
would
look
like
comparing
the
flow
is
we'd
pull
us
not
or
when
we're
going
to
pull
an
image.
We
pulled
up
the
manifest
and
then
we
can
we
can.
We
know
at
that
point
what
that
image
is
going
to
look
like
and
if
we
can
go
ahead
and
create
that
image
based
on
the
snapshots
that
are
available,
we're
gonna,
we're
gonna
be
able
to
do
that.
C
So
that's
going
to
be
in
the
next
release.
We
think
it's
a
valuable
feature,
but
the
changes
that
summarizes
what
we
see
as
the
changes
to
the
core,
but
we're
not
necessarily
going
to
provide
an
implementation
of
the
cluster
snapshot
or
we
see
that's
more
of
a
community
ecosystem
thing,
but
we
want
to
make
sure
that
our
core
can
can
do
that.
Not
all
of
our
pull
tools
handle
that
correctly.
D
F
C
We'll
release
when,
when
there's
a
need
to
release-
or
usually
we
have
the
next
big
feature,
so
we
want
Windows
to
be
the
next
big
feature
in
the
release,
but
there's
kind
of
a
few
other
things
about
remote,
snapchatters
or
some
other
things
that
we're
looking
at.
But
usually
it's
the
first
big
feature
to
complete,
we'll
do
a
release.
F
C
D
J
Already
supported
except
cigarette
stuff,
so
quite
well,
you
can
run
our
purchase,
but
you
can't
limit
CPU
resources
and
memory
resources,
but
we
are
going
to
get
C
group
support
using
a
secret
program.
It's
already
supported
by
a
Padma,
it's
kind
of
a
competing
project
and
we
can
follow
a
similar
way.
Probably
it's
a
request
using
system
D
or
for
our
normal
user
to
configure
a
cigarette.
D
J
So
the
Rossi
support
what
secret
person
to
is
in
progress,
and
actually
we
need
more
reviewers
for
rusty
project,
but
actually
as
a
serum,
which
is
it
as
implementation
of
drug
plan
times,
pick
already
suppose,
a
single
person
to
always
reverse
mode
and
contrary
we
we
need
to
support
C
group
to
are
prosperous
and
the
world
for
most
at
first
and
for
route
good
rest
mode.
We
also
need
to
support
user
instance
of
system
D.
It's
a
run
for
each
of
user
and
with
system
of
the
API
we
contain
delegated
permission.
Corals
sea
group
present.
H
Security
thing
now,
but
so
I
was
trying
to
run
the
container
D
stuff
like
the
image
stuff
that
we
were
working
on
in
like
a
pipeline
like
Tecton,
so
like
get
to
run
it
in
the
container,
so
I
think
maybe
I
didn't
try
it
hard
enough,
but
I
don't
know
what
the
goodness
is.
Also
related
to.
Can
we
run
like
maybe
partial
part
of
the
demon
inside
the
container,
so
that
we
can
like
do
image
operations
and
stuff
like
that
I'll
run
the
okay
inside
I,
don't
know.
Maybe
I
didn't
try
hard
enough.
J
So
a
Buick,
each
user,
conscious
idea
as
a
rivalry
and
ability
already
supposed
to
do
rest
mode
and
also
we
have
taken
support
for
accessibility
and
you
can
run
Buddhist
ability
other
put
wizard
any
security
configuration.
But
at
least
you
need
to
disable.
Second,
because
we
have
some
system
calls
for
containers.
D
J
So
supports
in
two
different
modes:
the
first
mode
is
just
using
Canady
as
a
robbery,
so
you
don't
even
notice
that
your
quantity
is
used,
but
in
most
cases
I
think
it's
fine.
But
if
you
want
to
consider
D
as
a
demo
builder
kit
also
connect
to
the
quantity
demo,
but
in
most
cases,
I
think
just
using
quantity
as
a
library,
wizard
demo,
it's
fine
and
anyway
you
can
export
an
image.
Any
format
such
as
droga
format
or
OCI
home
at
and
early
can
imports
out
of
the
image
to
quantity
or
whatever.
D
F
F
I
F
For
divisor
so
first
on
the
open
Southside
to
integrate
basically
D,
just
like
all
the
other
runtimes,
we
need
a
shame
implementation
and
we
have
a
open
source.
We
possibly
called
Jeeva.
Secondarily,
shame
it's
under
Google.
If
you'll
go
and
search,
you
see
it
and
we're
using
with
that
who
can
actually
I
feel
setup
properly.
You
can
are
they
use
query
his
way,
Steve
either
in
open
source,
but
you
need
to
understand
that
set
it
up.
F
So
that's
the
open
source,
dangerous
and
actually
internally,
we're
also
using
that
shame
and
I
also
heard
that
and
financial
how
they
are
using
jevetta
they're,
also
using
their
shame
because
they
use
post,
kubernetes
and
G
wiser.
So
it
means
that
it
works
and
we
on
gke
we
provide
a
product,
a
feature
on
GK
cloud,
GK
sandbox,
and
that
that
one
is
also
using
that
shame
and
G
visor
to
provide
a
sandbox
in
for
your
part,
so
that
that
feature
is
beta
and
we
are
working
on
or
g8.
F
D
F
And
for
divider
itself,
we
I'm
not
part
of
the
to
achieve
either
team
I'm,
the
kubernetes
team
and
we'll
walk
with
divide
a
team
and
that
team
they
are
making
good
progress.
Improving
jivas
are
like
recently
they
did
love
things
like
I
had
some
IP
table
stuff
support
would
be
how
they
want
to
run
some,
not
some
something
like
the
sto
sidecar
inside
the
Chivas
are
so
they
are
working
on
that
and
also
they're
improving
the
file
system
performance,
because,
if
you'll
note
you
either
die
forge
even
older
for
security
reason,
Oh
auntie,
valya.
F
Oh,
the
file
system
operation
will
need
to
go
through
a
golfer
which
is
the
Golem
process
and
I'm
using
Nyepi
and
for
some
reason,
9p
I
mean
Carson
performance
issue
with
all
the
file
system,
operations
and
they're,
defining
a
new
protocol
to
to
make
it
more
efficient
and
I
think
they
they
have
a.
They
have
a
design
for
that
and
they're
working
on
that.
So
that's
their
progress,
and
maybe
they
also
have
some
other
fancy
stuff.
But
this
is
the.
D
K
So
this
whole
world
of
the
different
container
run
times
is
new
to
me.
Ish
I'm
interested
to
know
your
thoughts
and
I
want
to
trigger
any
political
things,
but
like
cRIO
or
cryo
and,
like
the
difference,
is
there
and
why
there's
a
shim
layer
needed
for
container
D,
and
why
would
why
would
somebody
use
one
over
the
other
is
one
just
had
more
historical
context,
I'm
just
interested
people's
thoughts
on
cryo.
D
A
Yeah
I
mean
there's
a
lot
more
kind
of
backstory
behind
all
that
I
guess
then
makes
sense
to
really
try
to
unpack,
but
both
in
Barcelona
than
here
yesterday,
I
kind
of
walked
through
you
know,
docker
container
te,
creo
and
kind
of
the
pros
cons.
Why
you
know
who
created
them?
Why
do
they
exist?
I,
guess
to
me
the
most
important
thing
is
like
more,
not
so
much
about
where
we
came
from,
but
future
and
so
talking
to
Vincent
batts.
He
was
here
in
the
session
earlier,
but
I
think
he
had
to
step
out.
A
You
know
I
think
right
now,
it's
not
so
much
that
there's
like
some
amazing
set
of
features
of
one
it's
not
in
the
other.
Their
design
choice
differences.
So
you
know,
creo
has
was
started
as
we're.
Gonna
make
a
simple
OCI,
compliant
runtime
for
kubernetes,
and
that's
all
it's
going
to
do
like
it's
not
going
to
be
extensible.
It's
not
gonna.
Have
you
know
the
snapshot
proxy
plugins
kind
of
kind
of
this
idea
of
extensibility
around
an
API,
so
you
know
that
that's
just
a
different
design
point:
it's!
A
You
know
I,
think
they're
kind
of
tagline
was
all
the
runtime,
the
kubernetes
needs
and
nothing
more.
You
know
that
was
their
design
point,
whereas
container
D
was
the
CRI
was
a
plugin.
That
was
one
use
case,
but
there
be
other
use
cases
for
container
D
and
therefore
all
the
all
the
plug
abilities
that
be
able
to
use
piece
parts.
A
You
know
you
can
use
the
diff
service
and
not
touch
anything
else,
because
you
can
talk
directly
to
that
service
through
its
G
RPC
API
or
with
the
go
kind
of
runtime
library,
but
back
to
kind
of
the
future.
I
think
you
know
Vince
and
I
were
talking
about.
You
know
it's
you
know.
Is
there
a
way
to
kind
of
solidify
what
we're
offering
into
something
that's
more
aligned?
So
it's
not
like.
Oh
well,
they
pick
creo
and
they
pick
container
D
and
these
must
be
competing.
D
D
Okay,
good
point,
thank
you
yeah!
So
architectural
II,
they
both
have
the
same
number
of
hops
when
kubernetes
couplet
requests
a
service
that
goes
over
the
CRI,
which
is
G
RPC
interface.
When,
when,
when
this,
when
the
request
comes
in
it
does
because
come
in,
you
know,
that's
one
hop
and
it
comes
into
a
container
runtime
service.
D
This
acting
as
the
client
rate
when,
when
a
when
any
container
is
executed
created,
there
is
a
processor
in
front
of
it
and
for
for
the
hosting
of
the
service
like
getting
the
return
exit
code,
you
know
whatever
you
need
to
do
to
host
that
that
container
on
a
long-running
process
so
that
you
can
reboot
container
D.
You
know
the
container
runtime
and
still
have
this.
D
D
Yeah,
so
this
question
is:
is
cryo
more
lightweight?
No,
not
really
I,
don't
think
so.
I
mean
we
can
run
performance
summers
that
late
make
container.
Do
you
look
better
and
they
can
make
them
the
I
think
what
the
main
one
of
the
main
differences
they
can.
They
can
run
headless,
they
could
they
can
run
without
a
daemon.
We
prefer
architecture,
we
can
reboot
ours,
they
just
have
decided
to
reboot
theirs.
On
every
instance,.
I
We
didn't
want
to
dictate
what
kind
of
runtimes
you're
going
to
use
so
when
we
designed
the
interface
at
that
same
time,
there
were
several
projects,
I
mean
to
support
this
interface
and
they
all
have
different
pros
and
cons.
I'm
sure
I
was
more
involved
with
the
development
in
the
early
days
now,
I'm
pretty
sure
they
all
have
features.
I,
don't
know
about.
I
F
Just
as
a
sign
for
them,
though,
if
you're
using
G,
K,
E
or
e
KS,
they
will
decide
that
decided
for
you.
You
don't
need
to
care
about
it
and,
if
you're,
using
setting
up
your
own
data
center
use,
set
up
governance
yourself,
at
least
you
will
have
an
operating
system
right
and
whatever
they
provided
is
what
you're
going
to
use,
because
when
you
choose
to
or
SDG
built
in,
you
already
made
that
decision.
F
Yeah
like
like
I,
remember
earlier
this
I'm
going
to
you,
don't
have
system,
do
you
have
what
app
up
start
or
something
and
where
you
have
system
D
and
how
to
make
the
decision?
What
to
use?
Basically,
your
choice,
OS
distribution,
use
whatever
provide
you
and
now
I
will
everything's
using
systemd.
Now
so,
who
knows
what
happens
after
five
years.
D
Yeah,
so
usually
is
a
one
of
the
gut
on
on
signal,
this
working
on
the
CRI
interfaces
and
they
get
requests
coming
from
both
directions.
They
get
requests
from
from
the
cryo
and
the
container
D
guys
and
other
teams
right
there.
Wanting
more
Intel's
all
over
us
for
additional
performance
features,
Microsoft
guys,
one
add
scene,
and
so
things
get
pushed
up
and
they
come
back
down
and
the
the
signal
team
meets
weekly
and
and
discuss
some
of
these
issues
and,
and
they
ask
the
cry
guys
to
come
in
and
help.
D
I
I
think
there
are
many
changes
in
the
pipeline.
Some
of
them
are
still
in
discussion
or
there's
a
proposal
out.
Some
of
them
are
closer
to
implantation
stage
like
we
have.
We've
introduced
a
new
feature
in
kubernetes
called
thermal
containers,
which
is
basically
support,
adding
container
to
a
running
pod
so
that
you
can
load
it
with
a
bunch
of
debugging
tools
and
do
a
live
debugging
without
having
to
destroy
the
path,
and
that
would
require,
for
example,
having
the
option
to
join
a
specific
namespace
of
an
individual
container
in
the
path.
I
So
that's
something
we
can
expose
in
CRI
and
there
are
also
other
things
like
resource
updates.
So
there's
a
proposal
for
adding
support
for
huge
pages
in
for
Linux.
Only
of
course,
we
don't
have
that
support
for
Windows
yet
and
there's
also
the
employees
resource
update
where
you
can
actually
modify
the
request
of
container
in
a
pod
without
having
to
kill,
and
there
are
other
things
more
on.
The
discussion
like
I,
think
image.
D
So
not
just
a
timeout
I
know:
I
noticed
that
in
in
the
docker
shim
there's
a
just
came
up
the
other
day
that
the
dr.
shim
had
a
and
one
little
switch
in
there
for
setting
a
fire,
a
toggle
for
a
five
minute
duration
for
if
the
image
doesn't
come,
you
know
just
abort
kind
of
thing,
but
we
don't
have
that
and
crying
today
we
wait
until
we
come
back
up,
so
progress
would
definitely
help
so
people
wouldn't
try
to
kill
their
their
pods
first
yeah.
D
D
We
only
tested
two
stacks,
that's
what
they
call
a
dual,
but
it's
it's
a
list
right,
so
you
should
build
using
the
any
IP
ranges
that
you
want
to
use,
configure
those
so
c9
and
just
request
that
one
okay,
we
are,
we
are
supporting
zero,
seven,
one
of
CNI
and
zero.
Seven.
Six
of
the
plugins
we
had
seen
idns
support
for
the
windows
team.
D
D
B
That's
what
second
time
Christian
and
about
about?
Yes
him
about
him
from
the
standards
perspective.
Oci
runtime
spec
is
standard
interface
for
OCA
runtimes,
but
in
real
world
in
the
real
world
theme,
shim
is
actually
every
OC.
A
runtime
has
every
each
seems
and
yeah
in
year.
What
same
API
seems
the
fact
standard
for
seven
times
so
so
do
you
have
any
plan
to
like
make
shim
APA
to
standard
world
such
kind
of
works.
D
So
this
is
a
great
question
right.
We
had
this
question
many
times
in
Phil
mentioned
that
we're
gonna
sit
down
with
Vince
and
sometime
in
the
first
quarter
and
get
together
the
all
the
whole
OCI
team.
Maybe
maybe
with
that
discussion
will
come
up
now.
I
can
use
this.
You
know
the
question
came
up
during
during
coop
Kai
right
yeah.
We
we
knew
from
the
start
that
runs.
D
He
wouldn't
be
enough
to
cover
everything,
but
it
was
just
enough
that
if
you
had
configured
a
config
JSON
right
that
you
would
be
able
to
run
one
okay
as
long
as
you've
got
your
already
got
your
mountable,
you
know
ready
ready
to
run.
You
know
run
time
and
you
can
fix
this
on
desk.
That
was
really
the
plan.
It
wasn't
meant
to
be
this.
This
replacement
API,
something
we
never
wanted,
run
seed
to
be
a
competition
for
docker
or
any
like
pod
man,
any
other
high-level
API
to
a
client
tool.
D
A
G
Thank
you.
So
this
is
a
really
specific
question,
so
I've
been
working
on
some
shrimp
to
using
community,
and
so
I
would
like
to
pull
down
the
image
using
a
different
user
other
than
the
user.
Invoking
the
continuity
process
should
I
do
er
plug
into
content
content.
I've
got
the
name
content
or
snapshot
/.
C
D
C
G
G
C
I'll
just
say
like
there's,
there's
two
ways
to
do
it.
The
first
one
is
like
yeah
you
could
you
could
plug
in.
It
would
be
a
difficult
in
custom
and
like
on
the
unpack.
So
we
have
like
an
apply
function
that
you
can
implement
where,
if
you
want
to
do
some
customization,
such
as
using
some
other
mapping
but
there's
also
like
the
the
operating
system
level,
mappings.
J
I
D
D
D
E
So
Max
and
I
here
work
at
Amazon
and
we
work
on
the
fire
cracker
container
deep
project,
which
is
a
way
to
use
the
fire
cracker
vmm
to
run
containers
with
hypervisor
isolation.
So
it's
yet
another
runtime
that
runs
under
container
D,
I,
just
I
guess
a
quick
update
is
we've
made
a
bunch
of
progress
on
it.
We
were
able
to
run
multiple
containers
inside
one
VM
now,
I,
don't
know:
we've
we've
yeah
we've
implemented
a
bunch
of
stuff
with
networking
yeah.
E
D
E
The
extensibility
was
like
we
built
that
first
as
a
gr
PC
proxy
plugin,
so
it
was
an
out
of
process
process
of
process
plug-in
that
ran
alongside
container
D
and
community
would
call
out
into,
and
then
I
think
it
was
fairly
easy
for
max
to
convert
that
to
an
end
process.
Thing
he's
nodding.
Yes,
so
yeah.