►
From YouTube: Tutorial: KataContainers the Hard Way: Kubernetes + containerd... - Lei Zhang & Xu Wang
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Tutorial: KataContainers the Hard Way: Kubernetes + containerd + KataContainers - Lei Zhang, Alibaba & Xu Wang, HyperHQ (Limited Seating Available - See Description for Details)
Are you ready for KataContainers in Kubernetes? This tutorial includes sub-topics: 1. Install Kubernetes with kubeadm + custom container runtime. 2. Install containerd with CRI support. 3. Plug-in KataContainers as containerd runtime with shimv2 4. Use RuntimeClass to handle multiple runtimes 5. Setup Node e2e tests for cluster validation 6. Install CNI networking for KataContainers 7. Persistent storage plugins for KataContainers. 8. Deep dive into KataContainers features and highlights. 9. Pros & Cons of this deployment.Audiences will learn about benefits of KataContainers, and, the best practices of using KataContainers to secure and enabling hard multi-tenancy in Kubernetes cluster, how to manage multiple container runtimes on same node, and learn about the latest updates of KataContainers from its maintainers.A pre-prepared small cluster (either VMs or baremetal) is expected.
To Learn More: https://sched.co/GrZN
A
So
hello,
everyone
and
welcome
to
our
tutorial
and
our
topic
today
is
cut
the
containers
the
hard
way,
and
we
will
basically
talk
about
kubernetes
with
container
d
and
cut
containers,
and
we
will
give
you
live
demo.
Your
basic
knowledge
about
how
this
kind
of
system
works
and
what's
the
benefit
of
date,
and
we
also
talked
a
little
bit
about
the
difference
between
the
different
kind
of
secure
containers.
So
I
am
lady.
B
I'm
sure
and
I'm
the
co-founder
and
CTO
of
hybridizes
and
also
ve.
We
are
well
the
initiator
of
the
color
containers
project,
and
so
so
today
we
will
have
a
first
we'll
have
a
brief
introduction
about
the
color
containers
yourself
and
then
they
will
go
through
the
CR
CRI
part.
And
then
we
will
video
began
to
connect,
connect,
the
catechins
nurse
and
the
kubernetes,
and
some
future
works
so
some
time
to
begin.
B
So,
firstly,
I
will
introduce
about
Karagounis
Karagounis,
so
my
year
old
project
and
last
last
year
in
in
the
in
the
Austin
Austin
cacao,
we
launched
this
project
and
last
week
we
celebrated
the
white
year's
birthday
of
color
containers,
but
at
our
booth
we
still
for
some
some
some
some
audience
still
not
know
well
about
what
kya
is
actually
correct:
container
and
hem,
it
Castle
a
currency
in
the
same
place
of
fluency
and
works
in
almost
everywhere.
Resi
works.
So
in
brief,
it's
a
container
and
ham,
and
it's
for
all
the
also
yet
OCI
specifications.
B
And
but
the
difference
is
its
it's
built
with
the
authorized
agent
technologies,
just
like
IBM.
So
from
the
from
the
hostage
by
Oren,
Kara
containers
you
can
you
can
you
may
find
there?
There
is
a
VM
in
your
house
and
from
the
applications.
Will
itself
is
a
container
and
the
protector
firstly
initiated
by
our
hybridization
and
the
inhale
and
hostility
of
a
stereo
foundation
and
also
have
other
contribute
kind
of
contribution
from
have
a
Google
and
Microsoft
and
Alibaba
a
sort?
Sorry
I,
don't
least
you
are,
and
yes.
A
B
Know
us
and
his
character
and
energies
is
virtualized
a
container
and
here's
a
brief
comparison
between
the
currency
and
Kara
containers.
And
but
this
part
currency
values
Rossi.
We
have
the
process
for
the
applications
and
they
are
protected
by
the
some
some
something
like
sick,
home
and
other
others
Institute
to
help
to
isolate
from
the
from
from
other
other
others
others
and
used
namespace
to
do
the
isolations
and
also
also
see
groups
and
but
oh
dear.
Oh,
the
container
process
is
run
on
the
same,
say,
Colonel
and
and
depend
depend
see.
B
We
have
tried
to
do
some
some
demo
about
this,
and
when
we,
when
we
have
some
some
hack
of
the
kernel,
we
have
some
past
version.
Kernel
such
as
to
to
30-footer
here-
and
there
is
a
there-
is
a
CV
and
we
used
s
to
just
break
the
Red
Sea
and
get
that
control
the
other
privileges
of
the
of
the
hosts
and
other
other
containers.
So
at
this
at
this
point,
if
you
want
to
have
to
have
a
mole
more
secure,
secure
I
saw
a
solution
for
your
containers.
B
B
We
used
virtual
visualization,
and
then
we
run
the
iris
and
box
every
pulse
and
box
inside,
instead
of
actually
a
virtual
machine
and
the
process
inside
the
cast
corollary.
So
instead
that
water
immersion
there
is
a
smoke
cast
kernel
and
even
even
if
the
they
can
do
the
exploit
in
the
cast
that
they
cannot
escape
to
the
outside.
So
so,
this
ensure
a
better
isolation
than
the
Razzie
container
here
here,
I'm,
not
saying
that
episode,
Loutre,
say
safe,
but
it's
a
better
title.
Isolation.
C
C
B
That's
III,
don't
know
whether
I
understand
your
question
correctly.
You
mean
we
are
open-source
projectors,
so
so
we
used
to
buy
if
you,
if
you
run
on
bare
metal
things,
your
containers
running
against
than
the
your
container.
The
position
of
your
container
is
just
like
an
instance
of
the
ec2
instance
I.
B
This
projector,
actually,
we
have
two
similar
protect
from
hepatitis
HIV
and
from
inhale.
The
career
containers.
The
tour
product
o
began,
begins
at
May
2015
so
and
we
launched
almost
in
the
same
in
the
same
month.
Actually
in
the
same
same
week,
so
before
that,
for
for
me,
I
I,
don't
I,
don't
know,
inhale,
has
such
a
project
and
for
inhale
I,
don't
know
whether
whether
they
notice
project
before
and
when
we
launch
the
we
we
we
realized,
the
to
protect
are
quite
similar
and
we
have
some
some
interaction.
B
Actually
just
in
Seattle
at
the
August
of
the
of
of
2015,
we
have
the
first
meeting
face
to
face
to
talk
about
a
collaborate
between
the
two
product
and
in
the
in
the
in
the
in
2016
and
2017.
We
have
more
and
more
collaborations,
and
finally,
we
decide
who
merged
to
protect
into
one
and
the
color
product
caricatures.
B
B
Right
now
we
have
about
six
weeks
release
schedule,
and
that
means
we
will
have
naturally
seen
about
the
end
of
this
this
this
month
that
will
fall
into
the
the
Christmas
holidays.
So
so,
maybe
maybe
in
maybe
after
the
new
year
in
at
the
beginning
of
January,
we
all
loved
b-real
released
1.5.0
in
the
past
half
a
year.
We
have
added
some
some
more
features,
such
as
hopper
hat
and
the
template
and
resource
support
and
the
multi
architecture,
such
as
the
the
arm
and
the
power
and
s/390
acts,
and
also
have
some
improved
devices.
B
Support
such
as
the
GPUs,
and
currently
we
have
the
the
highlight,
the
community
symbol,
to
support
that
this
pattern
have
already
been
murdered
in
to
the
master
and
will
release
two
in
the
point:
five,
zero
and
later
we
will
have
the
time
off
for
it,
and
also
we
have
the
the
Intel
new
mill
as
a
newbie
mm
to
the
support.
Actually
after
the
a
table
is
announced
at
the
fair
cracker
we
are.
We
are
talking
about
to
have
the
support
of
the
fair
crackers
and
that's
an
interesting
topic,
but
not
another
point
of
date.
B
B
We
prepare
some
some
postcard
a
little
a
little
bigger
postcard
and
there
is
some
for
some
short
instruction
for
for
roncada
track,
so
you
have
about
maybe
ten.
So
if
you
have
interest,
you
are
interesting
in
this
and
you
can.
You
can
have
this
card
after
the
session
and
also
we
have
a.
We
have
a
booth
in
the
in
the
exhibition.
B
How
the
boot
is
seventeen
I
think
so
so
you
can
read
it
there
and
sometime
I
will
be
there
and
there
are
other
upstream
developers
there
and
you
can
talk
with
them,
and
this
is
for
the
photo
bundle.
So
you
can
just
just
use
for
come
on
the
line
to
install
the
cara
cara
into
the
into
your
linux
cause
when
to.
D
B
C
B
Can
just
use
these
for
Pokemon
lines
to
just
get
the
the
salsa
from
the
photo
for
the
Tata
at
the
to
use
the
apt
to
e,
so
the
install
the
packages,
and
actually
there
is
maybe
even
simpler,
actually
I
prepare
the
base
tutorial
I
try
to
install
new
merch,
a
new
water
wash
into
for
the
demos,
always
I
found.
There's
some
amazing
things
when
we
saw
the
in
so
that
when
18
18,
though
1804
and
I
found
I
found,
there
is
a
page
for
the
some
server
snaps
they
use
tablet
there.
B
C
D
B
C
B
C
C
C
D
B
B
B
A
Okay,
thank
you
sure.
So
so
you
know
last
demo,
which
is
very
simple,
but
should
just
show
you
how
to
use
darker
to
start
a
container
by
using
by
specify
that
I
want
to
use
kata
as
a
runtime
right,
but
in
the
real
cases
we
don't
do
that
because
you
know
docker
is
basically
a
more
complicated
project,
but
what
we
want
to
see
is
we
just
want
to
use
a
secure
container
runtime.
A
So
we
don't
have
to
use
docker
and
we
all
know
that
kubernetes
is,
you
know
the
most
popular
orchestration
layer,
so
the
if
you
very
straightforward
was
to
think
if
whether
we
can
use
cut
containers
directly
as
another
alternative
container
run
times
for
kubernetes.
So
we
can
just
run
the
work
clothes
from
kubernetes
inside
the
cut
container,
and
then
we
can
use
these
kubernetes
to
build
a
very
secure
and
highly
isolated,
a
private
cloud
or
public
cloud
right.
A
So
we
don't
need
to
care
about
darker
stuff,
but
in
order
to
use
the
you
know
to
use
card
container
as
a
runtime
or
kubernetes,
the
first
thing
we
need
to
know
is
the
CRI
container
runtime
interface.
This
is
a
very
interesting
topic,
because
ci
is
actually
one
of
the
most
important
interface
of
kubernetes
project.
A
So
if
you
check
the
code
of
criminal
evan
today,
you
will
see
that
there's
smoke
big
package
which,
namely
rocket
which
is
actually
used
by
that
time
to
to
to
integrated
work
heat
container
in
kubernetes.
It
is
the
actual
agency
problem,
but
then
in
this
way
you
actually
have
to
maintain
later
element
and
two
different
runtimes
in
kubernetes
itself,
for
example
I'm
a
member
of
signal,
so
we
never
reshape
a
new
feature
which
is
related
to
Condor
and
how
we
have
to
think
that
okay,
this
solution
should
work
for
both
darker
and
rocket
right.
A
Some
kind
of
like
Keisler,
remote
control,
runtime,
so
corporate
will
intentionally
stand
a
remote
code
to
an
underlying
server,
and
this
server
will
be
responsible
to
create
containers
for
you
to
code
docker
or
to
Kolkata
or
to
core
anything
that
you
want
like
raw
heat
or
something.
So
this
is
actually
the
basic
idea
of
how
or
why
we
want
to
have
this
kind
of
CRI,
incriminated
and
another
another,
and
another
reason
is
that
you
know
I
think
you
guys
made
my
heart
bad.
A
My
heard
about
that
from
2015
that
docker
the
project
itself
is
becoming
more
and
more
complicated.
If
you
compare
the
darker
like
the
doctor
today
with
the
darker
project,
maybe
one
years
ago,
you
just
you
just
a
run,
thought
you
just
said
talked
her
runs
run.
Something
you
will
find
out
today.
Talk
her
wrong
ago
could
seem
a
lot
of
more
time
than
eight
works
before
this.
You
can
try
right
now.
It
is
choose
it,
because
that
darker
now
has
a
lot
of
unnecessary
part
built
into
the
the
project
itself.
A
So
the
kubernetes
community
also
notice
this
problem,
because
a
daughter
goes
to
this
way
become
more
complicated.
Then
kubernetes
had
to
rely
on
a
lot
of
features
of
a
darker
right.
This
is
the
this
is
the
this
is
a
very
realistic
problem
at
that
time,
so
the
is
like
for
burnett's
community
also
think
about
how
to
tekapo
kubernetes
from
darker,
for
example,
how
can
we
make
kubernetes
workers
only
a
very
few
API
of
darker,
like
credit
containers,
that
containers
top
container?
That's
all.
A
We
don't
need
to
care
about
this
forced
warm
or
networking
or
any
warning
related
API
abductor.
We
don't
even
have
to
install
a
talker,
maybe
so
based
on
these
two
facts
and
then
there's
another
sting
come
which
is
actually
initiated
by
hyper
to
a
stage
which
I
also
participate
of
this.
This
kind
of
this
kind
of
period
because
hyper
a
stage
just
created
another
runtime
which
named
the
wrong
way
based
on
hypervisor.
So
this
container
runtime
is
basically
a
smallish
machine,
a
light,
a
very
rich
machine.
A
How
can
we
make
this
container
a
container
on
time
works
with
kubernetes?
This
is
more
difficult
and
than
to
integrate.
Something
like
rock-eating
to
criminate
is
because
it's
essentially
different
way.
For
example,
how
can
you
handle
the
host
networking
by
using
cut
by
using
hyper
other
best
runtimes?
You
have
to
deal
with
that
from
a
very
different
approach,
which
is
totally
different
from
any
kind
of
elitist
containers.
So
these
olives
facts
here.
The
kubernetes
community
finally
proposed
something
which
named
at
that
time,
is
the
Chrysler
remote
container,
runtime
and
Izzy.
A
The
the
first
issue
is
actually
created
by
Brendan
burrs,
and
we
talk
a
lot
about
how
to
you
know,
make
this
cloud
server
mode
container
runtime
as
a
default
content
around
having
kubernetes
instead
of
darker
or
rocket
or
hyper
or
wrongly
or
clear
container.
So
that
is
actually
the
very
beginning
of
the
story
and
after
that,
the
leader
of
stick
node
of
acronyms
project,
who
name
is
dong
dong
Chan.
He
says
that
the
cupellation
of
vendor
around
humming
to
an
implementation.
That
means
she's
thinking
that
kubernetes
should
not
have
a
default
container.
A
Runtime
right,
so
Koopalings
does
not
need
to
care
about.
Can
the
runtime
it
just
need
to
care
some
kind
of
interface?
That's
enough,
and
that
is
why
the
developers
from
Google
koi
Shepherd
a
stage
began
to
you
know
they
Co
proposed
a
a
desire
which
name
is
container
runtime
interface.
So
this
CR
thing
is
actually
a
G
RPC
interface
and
this
interface,
the
couplet.
Whenever
Devon,
whenever
coolant
want
to
create
a
container,
it
just
sent
a
remote
juristic
Cole
to
the
underlying
runtime.
A
We
said
that
gr
PC
can
improve
the
performance
between
the
cloud
and
server
mode,
very
especially
compared
to
the
normal
HTTP
request.
That's
why
we
choose
the
RPC
interface
as
the
baseline
here
and
the
first,
and
then
they
began
to
write
a
code
based
on
the
desire
of
CI
and
the
first
implementation
was
the
I,
which
name
is
dr..
Shim
I
think
you
guys
might
heard
about
this
project.
It
is
actually
a
build
in
package,
a
build
in
container
runtime
in
crew,
bread.
A
You
can
find
this
package
in
corporate
and
this
package,
which
named
extreme',
is
exactly
the
implementation.
Obviously
our
interface
for
darker,
but
the
reason
we
we
building
this
server
inside
equivalent
instead
of
you
know
just
around
a
stepper
taco
stream
on
the
only
physical
machine
along
with
a
node,
is
that
at
that
time,
darker
is
the
only
choice
for
most
of
the
people.
So
we
don't
want
to
make
in
kubernetes
installation
very
complicated.
You
need
to
install
kubernetes
at
first
and
then
install
it
dr.,
shim
and
then
stalker
that
will
be
too
complicated.
A
So
we
choose
to
keep
dr.
shim
and
part
of
the
crew
blade,
but
today
we
began
to
talk
about
how
to
save
her
daugher
stream
from
crew
blade.
So,
in
that
case,
the
duction
will
become
just
like
as
the
the
normal
CI
implementations
and
and
soon
after
docking
was
implemented.
We
began
to
create
another
project
which,
namely,
frankly
and
frankly,
is
the
first
non
darker
CI
implementation,
which
firk
hers
mostly
on
the
hypervisor
best
container.
So
you
can
think
about
that.
A
So
I
will
give
you
a
little
bit
more
detail
about
how
CI
designer
works,
so
in
crew
blade
cooperate
in
kubernetes
is
actually
a
psycho
loop.
The
cycle
loop
means
it
is
a
infinity
loop.
It
just
continue
running
and
running
and
corporate
a
running
a
loop
to
do.
It
actually
watched
the
the
change
of
part
in
the
API
server,
a
ket,
say
D
and
whenever
there
is
a
new
part
is
scheduled.
Then
the
couplet
of
a
note
is
that
okay,
there
is
a
part
you
can
hear,
sir.
A
That
is
changed
because
we
never
could
read
his
schedule
part.
It
was
stated
in
node
name
of
that
part
to
a
specific
node
and
then
the
crew
Bratton
noticed.
Ok,
this
node
name
is
the
exactly
same
with
me.
So
this
crew
blade
will
finish
this
part
from
API
server
to
its
local
cache
and
then
began
to
create
all
the
containers
divided
in
this
part.
A
That
is
how
the
cycle
loop
works
and,
of
course,
this
cycle
up
should
never
been
blocked
because
it
need
to
continue
checking
and
checking
so
any
kind
of
a
time
consuming
operation
in
the
cycle.
Loop
will
be
deferred
to
another
go
team
or
we
use
some
instinct
ways
to
do
things
separately.
We
start
several
workers
or
like
that,
for
example
kubernetes.
Whenever
mount
volume
directly
in
this
step,
we
actually
mount
volumes
being
a
separate,
go
team
for
you
and
could
read
only
check
if
this
volume
is
ready
or
not
okay.
A
So
this
is
psycho
loop
and
whenever
psycho
loop
found
that
we
need
to
create
a
part,
then
it
will
actually
call
a
generic
runtime
and
Janica
runtime
Gina
Quran
tab
is
a
fake
container
runtime
inside
a
corporate,
so
kubernetes
do
not
know
what
they
do
not
know
which
contain.
Runtime
is
running
on
your
node,
so
it
just
called
general
runtime
and
a
general
grant
ago
transfer
your
part
EPA
object
into
the
real
CI
interface
call
a
remote
called
geophysical.
For
example,
there
will
be
a
API
which
is
like
credit
container,
that
is,
the
crew.
A
Blade,
will
call
this
the
CI
interface
to
create
container
for
you,
and
we
also
have
a
definition.
Like
Stannah
box,
a
standard
box
means
how
the
how
you
implement
a
part,
for
example,
if
you
are
in
darker,
you
actually
do
not
have
to
implement
a
part.
The
only
thing
the
only
thing
to
do
during
the
credits
tender
box
face,
is
you
just
creating
for
a
container
which
name
in
the
past
I
think
you
guys
might
heard
about
this
container.
It's
very
small.
A
It
is
the
old
way,
the
first
container
credit
event,
the
party,
the
credit
and
these
these
containers.
You
would
use
to
hold
it
in
network
namespace
of
the
whole
pot.
That's
all
so
when
you
have
the
CI,
when
you
have
the
interface
like
this,
that
it
is
totally
your
business
to
decide
how
to
implement
every
ERP
Seco.
Here,
that's
why
we
have
so
many
streams
right
now,
like
taka
stream,
like
rail,
by
Red
Hat,
like
Frankie,
like
a
CI
container,
D
and
everything
there.
Everything
here
is
actually
steer.
A
Shame
so
from
this
example,
you
can
actually
see
what
is
the
I,
so
CRI
is
actually
an
interface
that
describes
what
crew
blade
expects
from
container
runtimes.
So
that
means
kubernetes
only
care
about
this
very
limited
number
of
API
from
a
container
run
home.
No
matter
is
darker.
It's
it's!
It's
a
rocket
is
cryo
is
concerned.
Do
even
it's
a
virtual
machine
like
kata
criminals.
Do
not
care
about
that.
A
As
long
as
you
can
provide
this
limited
number
of
interfaces
to
crew
blade,
it
works
and
we
also
decide
to
implement
mostly
I,
decide
to
decide
CI
as
a
container
static
interface.
So
we
do
not
have
a
part.
The
concept
apart
in
the
CI
interface,
that
because
pod
is
a
coronaries,
related
concept.
We
cannot
expect
the
container
runtime
under
understand
disk.
This
concept,
like
darker
darker,
has
no
concept
of
pod,
so
we
cannot
say:
hey
dog,
her
just
create
a
pod.
For
me,
it's
impossible!
That's.
Why
is
the
I?
A
We
we
introduce
another
concept,
which
name
is
sandbox.
That
means
okay.
Whenever
kubernetes
credit,
pod
I
will
close
the
eye
to
create
a
sandbox,
but
how
this
CI
regime
or
the
implementation
created
this
sandbox.
It's
all
depends
on
you.
You
can
just
ignore
this.
If
Y
you
can
do
that
and
I
think.
The
key
word
of
CI
here
is
that
extensibility
we
want
to
make
sure
the
the
maintainer
of
container
runtimes
and
I
talked
her
like
inherently.
They
have
totally
freedom.
You
have
totally
freedom.
A
Ways
to
you
know,
implement
their
own
CR
streams
for
kubernetes,
and
even
if
you
are
just
normal
developer,
you
want
to
you
know,
implement
something
implement,
something
which
is
not,
which
is
currently
already
darker.
You
can
do
that
in
a
CI
implementation,
because
CR
implementation
will
then
code
darker.
So
in
this
in
this
period
you
can
do
a
lot
of
things.
You
can
wrap
your.
If
you
request
you
can
transfer
the
EPA
request.
You
can
do
audit,
you
can
do
everything
you
want
to
do
you're
in
the
CIA
okay.
A
So
so
here
the
question:
how
can
I
implement
the
CI
I
will
give
you
a
very
simple
example.
For
example,
for
instance,
what
we
want
to
do
is
create
a
part
which
name
in
the
floor
with
two
containers.
Installation
name
is
container
a
and
a
container
B.
So
whenever
you
use
kubernetes
to
credit
this
part,
then
what
CI
will
do
it
like
this?
A
The
first
step
is
the
crew
plate
will
call
the
wrong
parts
tan
box
flow
by
using
CI,
then
you'll
see
our
implementation
will
will
actually
call
the
under
lighting
container
runtime
like
darker
or
cotta
or
anything.
But
here
is
a
difference
because
I
said
it's
your
freedom
to
implement
the
CI,
then
you
can
make
the
season,
for
example,
this
rum
Potter
sandbox.
What
I
will
do
in
essentially
if
I
was
like
I
was
a
taco
maintainer.
What
I
actually
do
is
create
a
very
small
in
for
a
container
which
never
the
pause
animation.
A
So
that
is
basically
what
takashima
doing
today.
It
will
create
a
very
small
container
for
you,
so
this,
in
fact,
container
will
always
be
the
first
container
in
in
a
pod,
but
if
I
was
using,
if
I
am
using
car
containers,
what
I
will
do
is
total
different
in
this
step,
I
will
create
a
light
weighted
virtual
machine
which
is
actually
implemented
by
car
containers.
So
there
will
be
a
real
stana
box
which
named
the
floor
in
this
step.
A
If
your
even
darker,
then
you
will
actually
call
talk
API
to
create
a
container
a
and
container
B
for
you,
it's
very
simple,
but
if
you
are
in
card
containers,
then
you
don't
need
to
create
new
containers,
because
there
is
already
a
small
washer
machine
running
which
name
is
full
and
what
you
need
to
do
is
create
the
two
containers
inside
of
this
wash
machine.
But
these
two
containers
are
just
very
simple
containers
with
month
namespace.
So
it's
not
a
it's,
not
a
real
container,
just
the
separating
namespace
with
separate
route
FS.
A
So
so
this
step
of
it
is
very
different
from
darker,
but
again,
kubernetes
never
know
about
this
card
difference
and
the
cupola
doesn't
only
know
that.
Ok,
I
just
create
a
pod,
and
the
pod
runs
all
the
way
through
its
lifecycle.
That's
all
that's
why
we
need
to
that's
why
we
want
to
have
the
CIE
implementation
here
and
another
thing
need
to
think
about
sorry.
Yeah.
A
E
A
E
A
B
Let
me
answer
this
question
and
your
question
is
about
coherence,
warm
with
calendars
right,
so
actually
most
of
harlot.
That
works
because
the
cation
ham,
it's
always
a
gesture,
container
and
ham,
and
the
only
part
is
about
the
networking
part,
especially
about
the
Tina's
part,
and
there
are
some
some
workaround
for
that
for
that.
I
can't
say
that
the
implementation
is
clean,
but
yes,
we
can.
We
can
hack
it
to
support
that
yeah.
That's
that's
a
case,
and
and
now
we
will
go
on
on
the
CRI
apart
yeah.
A
So
so
the
so
I
just
mentioned
that
the
first
step
you
want
to
implement
see
is
about
how
to
implement
the
part
lifecycle.
The
second
part
you
need
to
implement
is
a
streaming
server
streaming
API.
What
is
Qi
media?
It's
like
logs,
execute
command
attach.
These
are
all
streaming
API,
so
streaming
API
implementation
is
very
different
in
kubernetes.
I'll.
Give
you
another
example
here
for
the
instance.
A
The
CI
a--
implementation
just
return
the
UL
of
the
Jimmy
server,
so
the
streaming
server
is
started
whenever
a
CI
implementation
is
started
because
kubernetes
provided
you've
is
a
standard
library
to
implement
a
streaming
server
to
response
to
any
streaming
API.
So
in
this
time
you
just
return
you,
the
UL
back
to
the
to
crew,
blade
an
equivalent
return.
All
the
way
back
to
the
EPS
over
and
under
which
triggered
a
redirect
requests
which
stand
to
the
real
streaming
server
running
on
you'll
note,
and
then
the
streaming
server
responds
to
your
streaming.
A
Api
like
logs
or
exe,
see
here
the
reason
we
decided
like
this,
because
we
found
that
crew
blade
will
be
actually
very
busy.
If,
if
there
are
a
lot
of
streaming,
API
is
like
a
lot
of
customers
or
a
lot
implies.
They
are
doing
logs
or
exe
season,
crew
later
actually
died
or
crashed.
So
we
separate
all
streaming
API
to
a
Stepford
streaming
server
here
and
in
Kiruna
days
they
can
an
increment
this.
A
B
Thank
you
that
we
will
go
back
to
the
to
connect
that
cotta
containers
with
the
kubernetes
first.
First
of
all,
we
will
think
about
the
talker
shim
and
the
talker
pad.
Maybe
we
could
just
put
the
color
color
in
him
to
the
to
the
talkers
position,
to
the
addresses
position
if
we
used
the
talker
and
to
call
the
caller
and
ham
and
the
Kaddish
in
and
to
do
the
connection
and
just
a
just
a
minute
minutes
before
we
have
sho
show
you
the
all
the
processes
of
the
of
the
color
of
the
of
this
case.
B
B
B
It
process
all
the
all
the
adult
agency
and
it
won't
pass
the
annotations
to
the
to
the
ran
temps
and
there
is
some
had
purchased,
submit
to
talk
to
movie
and
that's
the
opening
there
for
half
a
year,
so
so
the
from
the
dollar
there's
no
direct
way
to
support
that.
If
you
don't
apply
some
of
trip
of
trip
hatch
patches,
but
that
won't
be
a
big
problem
right
right
now,
the
you
can
use
the
community
itself.
Actually,
the
community.
B
Week
we
use
Kennedy
because
it's
already
supports
the
I.
The
the
CRI
have
plugin
have
been
part
of
the
visual
part
of
the
the
community
canary
release,
so
we
can
use
the
community
to
took
out
the
cover
and
ham
Conqueror
and
hammer,
and
the
Gothic
can
hear
the
first
thing.
A
committee
will
created
a
Kennedy
Rand
have
a
condition,
and
this
is
for
the
big
high
postings
and
Kennedy
symbiote
called
a
katharine
ham
and
was
color
and
katharine
ham
bo
created
the
sandbox
and
proxy.
B
But
not
so
arrogant,
actually
yeah,
you
have.
You
have
already
seen
the
the
processes
for
that.
If
you
run
that
ran
the
ran,
the
kubernetes
use
the
kinetic
community
and
kara,
you
will
have
the
condition
process
and
the
kardashian
process
and
the
proxy
process
and
the
senbatsu
process,
so
so
many
process
in
the
in
the
in
your
house,
though
it
works
and
I'm
most
amazed
about
the
proxy
here
here.
It's
surrounded
with
the
with
the
dash
lines.
Actually,
the
prophecy
is
not
not
really
not
that.
B
B
B
So
no
more
process,
and
so
that's
that
sounds
good
and
the
only
thing
is
we
can
try
to
make
make
these
things
become
a
plug-in
for
the
for
the
Kennedy
than
them
may
may
lack
the
structure
to
be
a
more
arrogant
but
way
there
is
still
a
problem
here.
If
you
want
to
support
more
randoms-
and
that's
that's-
may
not
not
work.
If
it
become
a
Kennedy
plugin,
then
if
you
want
to
use
the
other
plugin
to
run
the
rescue
container
in
the
same
hostel,
that's
not
easy
to
implemented.
B
That
may
in
need
some
architecture
change
for
the
photo
Kennedy
side.
So
that's
that
sound,
not
so
good
photo
for
the
community
and
fortunately
that
we
are.
We
are
not
the
only
wealth
fund.
A
this.
This
issue,
and
at
the
upstream
Kennedy
upstream,
is
bound
to
solve
this
based
problem
as
well,
and
here
we
can
state
the
problems
here.
B
Actually,
the
main
problem
here
is
about
a
shame
session.
There
is
a
shim
between
the
Kennedy
and
I
reckon
dinner,
actually,
even
with
Rossi,
there
is
a
condition
for
se.
The
Kennedy
is
self.
Esteems
itself
is
just
to
to
react
the
process
of
the
of
the
container
and
they
they
can
decouple
the
Kennedy
tema
with
the
running
with
the
running
process
of
the
of
the
containers.
So
if
you
upgrade
the
Kennedy
or
the
continuity
itself
is
crashed
and
you
don't
need
to
that,
that
won't
affect
and
it
running
running
containers.
B
B
This
is
a
good
point,
but
the
problem
here
is
the
condition,
is
serviced,
always
always
think
the
container
when
hammies
resi-
and
that
means
your
container,
your
Wrentham-
should
have
a
commander
line.
That's
compatible
with
renze
yeah
has
addressee
command
line.
That's
why
the
cattle
and
Hemis
try
to
use,
try
to
implement
what
resi
implement,
but
but
things
is
the
the
Renzi
command
line
itself
is
not
standardized.
B
There
is
a
OCI
specification.
You
know
the
OCI
is
try
to
be
an
open,
open
standard
for
the
photo
community
for
the
container
community,
and
it
has
two
parts
of
the
of
the
specification
is
about
around
ham
and
the
other
is
about
the
ebay
images,
but
for
the
random
parts,
this
back.
They
have
a
Jason,
Jason
form
aspects
and
some
some
different
nations,
but
no
command
line.
Specs
and
I.
Think
last
year
there
maybe
the
year
before
last
year,
peoples
are
talking
about
the
command
line.
B
Standardization,
but-
and
you
know
the
Kemal
is
not
standardizing
and
and
other
people's
try
to
give
up
these
efforts
because
we
have
a
better
candidate
and
more
work.
I
will
I
will
receive
container
is
a
process.
You
know
just
adjusted
because
that's
the
the
container
it's
a
process
and
may
be
a
process
group
C
Group
I
mean
if
it's
a
process,
then
it
must
have
a
PID
here.
So
the
process
ID
the
PID-
means
much.
At
least
you
can
queue
it
right.
B
B
Even
if
it's
not
a
process,
it
should
support
these
features.
Just
like
cutta
cutta
cutta.
We
we
use
the
3m
as
a
as
a
sandbox
for
containers.
That
means
the
sandbox
is
a
pod
and,
and
there
there
may
be
container
inside
the
pod
may
be
multiple
container
inside
the
pod
right.
If
we
have
three
container
inside
a
pod,
but
in
hostel
there
is
no
process
for
each
containers
right
and
if
it's
it
assumed,
as
a
Rancic
ran
SiC
container,
that
it
has
a
PID
for.
B
N
and
it
could
send
the
signal
to
the
to
the
container,
to
the
PID
and
to
queue
it,
but
here
we
cannot
know
which,
which
which
process
to
be
killed.
That's
the
case,
so
we
have
to
implement
another
another
process
called
cutter
shim
that
have
a
PID
when
it
gets
a
signal,
and
it
will
forward
that
the
signal
to
us
to
the
to
the
process
inside
the
the
sandbox
there's
the
only
the
only
the
only
responsibility
for
this
process.
It
looks
quite
tricky
here
and
actually
we
took
we
don't
like
this
implementation,
but
Banaras
you.
B
You
must
have
the
resi
compatibility.
You
have
to
implement
this
one.
So
so
here
is
a
stub
process
to
forward
it.
That's
a
shim!
You
know.
Firstly,
there
is
a
container
team
and
then
there's
the
cottage
ma2
team
for
one
container
and
not
one
sandbox
for
ranking
dinner.
So
once
you
have
a
have
a
sandbox
so
and
once
you
have
a
sandbox
and
the
several
containers
inside
the
sandbox
you
have,
maybe
you
have
an
empress.
One
seems
for
the
for
two
times
of
in
France
man
shims
for
the
foot
apart.
That's
a
bit
terrible
right.
B
B
Here
is
the
problem
too
many
directions
and
fortunately
the
Google
+
community
folks.
They
also
realized
that
this
this
problem
and
they
put
deep
this.
They
are
very
smart
to
propose
a
very,
very
small
change,
to
fix
this
issue,
a
small
change
and
it's
make
a
lot
of
difference.
The
trend
here
is:
they
guide
the
shim
stock,
its
URL
from
shim
new
occasion,
instead
of
to
pass
it
to
the
gym
before
the
version
of
the
shim.
B
When
the
when
the
community
occurs
the
cannon
regime,
it
will
pass
a
unique
local
socket
address
to
the
to
the
shame
and
the
shim
will
serve
on
this
address
vista
with
the
interface.
Then
that
means
every
I
reckon
dinner
should
have
a
independent
address
socket
and
if
you,
if
they
give
them
in
the
vehicle
case,
the
regime
could
my
Okada
shim
without
with
other
such
a
such
an
address.
B
The
shimmer
will
have
their
own
addressing
and
retain
the
address
back
to
the
to
the
to
the
Kennedy
that
the
trainer's,
the
consume
itself
will
decide
which
address
to
return
and
the
sale
regime
may
retain
the
symmetrize
back
to
the
to
the
canary.
In
this
case
we
could,
we
could
arrive
if
we
launched
our
containers
inside
a
single
sandbox,
and
then
we
could,
for
the
oldest
and
box
of,
could
could
return
the
same
address
to
the
back
to
the
Kennedy.
B
So
there
will
be
only
Rankin
Hashim
for
all
these
and
parks
and
no
no
other
process,
no
independent,
a
cottage
in
here.
So
instead
of
2
times
of
Empress
Van
shims,
we
have
only
one
team
for
rampart
and
the
scenes
could
be
even
better
because
the
PID
means
later
now.
That
means
they
don't
depends
on
the
the
PID
itself
to
the
to
the
Keogh
and
signal,
since
the
just
of
Kadir
interface,
provided
by
the
by
the
symbol
to
the
community
cost
code,
a
code,
this
socket
Avista
with
some
with
some
G
RPC
calls.
B
Instead
of
the
try
to
kill
the
PID
in
this
case,
even
with
Val
shim.
For
all
the
sandbox,
there
were
no
problem
for
that.
The
PID
is
only
an
ID
here,
not
another
process,
so
that
was
to
implement
a
new,
simple
shim
for
for
the
photo
kata.
That's
for
the
we
could
provide
one
fewer
indirection
layer
right.
That's
the
most
hard
things
in
computer
science.
B
C
B
We
we
do
to
manage
reviews
and
ethics
and
the
including
many,
including
some,
some
of
the
refactor
of
the
of
the
calicut
around
hams
and
it's
more
today
and
the
will
be
released,
and
there
is
some
small,
some
small
patches
on
the
way
to
be
merged
about
that
hosts
more
fixes,
and
we
will
get
them
to
be
more
to
soon,
and
with
this
case
you
can,
you
can
see,
we
have
a
pretty
simple
picture
now.
The
picture
is
too
small
compared
to
the
to
the
prior
events.
B
B
B
B
B
If
we
want
to
use
Vista
City,
are
you
can
just
just
with
the
simple
command
line
to
support
to
provide
at
random?
I/O
doubt
continue,
eat
that
one
da
da
da
da
we
and
that's
the
name
name
hater-
will
find
the
R
or
R
Kennedy
Shem
color
me
to
binary
that's
the
nickname
in
pattern
and
you
can
run
run
eight
pizza
box.
C
B
B
Yes,
oh
the
IV
area
to
something
something
more
and
before
that
I
will
I
will
tell
you
some
some
new
facts
and
yes,
but
what
we
could
get
from
the
shim
be
true,
and
fortunately
we
have.
We
have
much
fewer
processes
and
is
really
appreciated,
and
almost
everyone
like
this
decadence
feature.
No,
nobody
like
too
many
teams,
and
last
month
we
have.
We
have
forum
in
the
in
the
open
stance
amid
reporting,
and
there
is
a
forum
and
in
this
forum
I
said,
I
I,
hate
teams,
but
our
a
committee
member
Samuel.
B
He
said
he
loved
James
Pataki.
You
know
he
didn't
love
to
manage
teams,
so
this
one
we
post
like
this
one
and
only
one
team
for
for
when
sandbox
and
no
more
other
teams-
and
this
is
this-
is
the
part
for
the
photo
community.
Some
of
you
may
may
use
the
crowd
here
and
for
chorale
they
are
going
to
support
the
condition,
the
symbol
interface
as
well,
so
at
that
ham
from
the
Kennedys
from
the
chorale
side.
B
Other
others
have
success,
and
the
four
is
at
least
here-
and
this
for
all
depends
on
the
host
network.
Tammy's
put
the
put
Olli
put
the
sandbox
inside
the
the
network.
Name
space
with
the
host
I
mean
share
the
share
the
same
name,
names,
basis,
host
and
I
mean
make
it
as
a
system
as
a
container
in
the
hostel
system,
and
this
is
this
breaks,
the
the
isolation
for
the
photos
and
past
and
that's
not
supported
by
design.
It's
not
so
Palace
a
feature.
B
Here
is
the
configuration
of
the
of
the
continuity
and
in
Canadia
we
have
some
some
some
other
other
things,
and
the
most
important
thing
is
about
the
plugins
part,
and
here
we
have
the
CRI
platin
and
the
default
hem
of
the
CRI
rakia,
and
here
here
these
two
alliances
works,
and
now
the
the
current
configuration
is
the
V
van
version,
one
of
the
of
the
of
the
shim,
and
then
it
will
color
color
and
hem.
This
is
the
the
Ottawa
and
at
the
already
exist
exist.
One
and
I
will.
D
B
B
B
B
B
This
dam
is
about
to
create
us
the
sandbox
and
create
a
container
inside
the
sandbox.
So
that's
the
normal
case
for
a
single
container
pod
and
the
container
I
think
it
is
just
a
pizza
box
and
the
fasces
for
the
20
times
of
the
test.
The
fastest
one
is
about
one
second
and
eight
hundred
millisecond,
and
the
slowest
is
to
second
and
more
than
1,400
seconds.
At
the
mean,
time
is
almost
one
point
nine
seconds,
and
this
is
for
the
shimmy
through
a
shimmery
one.
B
B
Yes,
it
began
mr.
xie
me
too
and
like
that,
like
that
the
title
has
run,
we
were
sure
sure
that
the
other
slice
to
save
some
time.
Yes,
and
actually,
when
we
support
support,
achieve
a
tool.
We
will
have
much
much
shorter
time
for
the
for
the
container
boot
and
this
basis
for
the
para
mental
case-
and
this
is
for
the
there
is
thesis
for
the
for
the
nasty,
the
virtualization
tab
and
even
a
supporter
ization.
This
will
be
more.
B
B
One
thing
about
symbol,
because
the
actually
for
kata
in
the
in
the
production
usage,
the
pain
point-
is
always
be
the
storage.
If
you,
if
you
take
into
the
project
deeply,
you
will
find
this
and
there
are
many
critics.
Sound
9pf
is
support.
Everyone
I
really
want
to
put
it
into
production.
We
will
talk
to
about
the
hub.
What
do
you
think
about
the
storage
part?
B
It's
not
only
because
of
the
the
performance
part,
but
also
the
the
Navy
has
some
some
problems
on
the
POSIX
7070
compatibilities.
Sometimes
it
will
break
some
some
project
semantics
and
we
have
to
do
some
30
have
to
to
make
the
the
apt
and
other
things
Rael.
Also
because
90th
has
the
implementation
itself
is
not
so
good
and
in
the
in
the
queue.
So
sometimes
there
is
some
some
some
security
backs
about
the
Nyepi,
so
so
we
have
to
avoid
we
to
use
it.
B
For
the
production,
production
environment,
so
for
community
this
case
is
snapshot,
I
will
pass
them
the
mound
structure
to
the
task
plug-in.
Actually,
the
Linux
task,
provi
and
the
stream
will
mount
the
file
system
and
then
has
the
file
system
to
to
the
to
the
run
ham
and
for
the
women
case.
We
cannot
do
anything
with
just
the
access
we
can.
B
We
can
only
accept
that
the
file
systems
and
if
you
want
to
pass
the
proxy
block,
even
you-
visa,
with
the
palak
based
storage,
trust
that
snapshot
or
you
have
to
pass
the
acceptor
file
systems
and
you
have
it.
You
have
some
choice.
The
first
way
is
you
use
Nighy
to
pass
it
into
the
guest
and
then
the
second
way
is
you
try
to
amount
it
again.
B
That's
establish
the
regime
mounted
and
the
runtime
I'm
a
mounted
pastor,
Tudor
cast
that
looks
very
ugly
and
sometimes
the
the
duck
that
DeCosta
itself
as
assumed
the
regime
have
mounted
so
they
they
break
some
assumptions
and
maybe
maybe
lead
to
some
problems.
But
if
you
do,
if
you
don't
don't
emanate
it
I've
just
a
pastel
per
octave
ice.
If
you
think
that's
our
party
was,
then
the
poster
hosting
and
the
guest
I
have
the
right
access
to
the
to
the
file
systems,
and
this
may
break
the
super
block
of
the
file
system.
B
That's
the
worst
thing.
So
there
is
no
clean
and
good
implementation
for
this
case,
but
in
stream
with
who
now
we
we
can
her
daughter,
we
can
handle
the
storage
by
ourselves.
That
is
the
amount
past
whooshing
veto
and
she
met
who
can
decide
to
remand
it
or
not?
So
so
we
can.
If
it's
a
block,
we
can
just
pass
the
block
to
the
to
the
to
the
VMS.
Then
this
makes
things
better
yeah
and
let's
go
back
to
to
our
death.
B
D
B
No
difference:
okay,
okay,
bingo
I
got
it
okay,
okay,
almost
of
the
same
and
and
I
think,
there's
a
little
bit
by
her
and
I
think
at
that
time.
The
bell
for
the
photos,
even
though
the
worst
things
is
2.4
and
the
min
ham
is
about
about
two
nice,
seven
or
one
night
time,
yeah
yeah,
so
a
little
bit
better
and
there
is
always
always
things
always
happen.
Some
some
things
when
people
to
Timon
you
know-
and
this
time
is
not
not
very
bad.
At
least
days-
become
better
I.
B
Should
be
much
better,
but
but
for
them
for
demo,
you
know
this
many
things
you
can't,
you
can't
handle
it
and
that's
the
last
page
and
kind
of
ikana
is
as
a
fully
open
open
source
project
and
hosted
by
the
OpenStack
foundations,
and
we
are.
We
are
not
open,
stack
the
project,
but
also
we
are.
We
are
an
independent
project
and
the
only
code
and
the
documentation
is
various
and
the
kid
have
a
and
we
have
a.
B
We
have
the
inhale
and
then,
however,
have
the
usual
initial
contribution,
but
the
project
is,
and
there
are
a
party
license
a
parrot
to
license
and
we
have
an
open
governance
about
it
so
and
every
every
week
we
have
the
architecture
architecture
medium.
So
we
are
in
the
open,
open,
developing
stairs
and
any
income.
Any
contributors
are
welcome
and
or
select,
and
IRC
and
million
is
not
welcome
everybody
to
joy.
So
if
you
have
in
fear,
I
have
interesting-
and
here
some
some
tips
for
the
for
getting
started
and
also
you
can.