►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)
Improving your Kubernetes Workload Security with Hardware Virtualization - Fabian Deutsch, Red Hat & Samuel Ortiz, Intel (Intermediate Skill Level)
On any given node, all Kubernetes workloads are running through software-only isolation. The security concerns related to that level of isolation could be mitigated by using hardware virtualization for both pods and traditional (legacy?) workloads. This talk will present two complementary approaches for doing so: Kata Containers and KubeVirt. We'll be describing how both projects leverage CPU virtualization to implement a stronger security architecture for Kubernetes. When combining both approaches, one can run a wider range of workloads, from untrusted containers through Kata Containers to more traditional, lift and shift applications with KubeVirt.
About Samuel
He is a Principal Engineer at Intel.
About Fabian
Fabian Deutsch is working at Red Hat and used to be much more active in the Fedora community as he is today, worked on the oVirt project for a few years, and is now involved in KubeVirt.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
B
A
B
B
B
The
thing
that
you
can
run
with
kubernetes
will
all
share
the
same
kernel
so
Oh
on
the
same
host,
at
least
so
all
your
containers
will
have
to
share
the
same
kernel.
You
can't
have
a
dedicated
kernel
to
your
to
your
specific
workload.
So
if
you
work
with
these
very
specialized
and
need
something
specific
from
in
terms
of
kernel
features,
that's
probably
not
something
that
kubernetes
can
run
and,
finally,
all
workers
will
be
running
through
software
based
isolation.
B
So
we've
seen
what
kubernetes
can
run
we've
seen
what
communities
can
not
run,
and
the
point
of
our
talk
is
to
show
that
by
using
hardware
virtualization,
you
can
basically
expand
the
set
of
workloads
that
communities
can
run
from
basically
cloud
native
apps
cloud
native
native
apps
to
a
lot
more
workloads,
and
this
is
the
point
of
this
talk.
So
a
quick
question
who
can
sincerely
say
that
they
know
what
hardware
virtualization
is
in
this
room?
B
Okay,
a
handful
of
them,
that's
good
for
the
rest
of
you.
I'll
just
go
really
quickly
through
what
we
think
other
virtualization
is.
Sometimes
people
call
it
virtual
machines,
but
that
doesn't
really
help.
We
all
know
what
we've
all
seen,
what
what
virtual
machines
are
and
and
how
they've
run,
but
really
hardware
virtualization,
is
about
making
sure
that
your
resources,
your
CPU,
your
memory
and
your
devices
are
not
accessible
to
a
set
of
different
workloads.
So
if
your
workload
is
trying
to
run
a
privileged
instruction,
this
won't
run
with
with
a
hardware
virtualization.
B
If
it's
with
hardware
virtualization,
your
workload
won't
get
direct
access
to
your
physical
memory,
it
will
have
a
virtualized
virtualized
memory
same
for
devices
where
your
workload
is
not
going
to
be
able
to
set
DMA
ranges
and
DMA
address
space
for
specific
devices
and
be
able
to
mess
directly
with
your
physical
memory.
So
how
do
our
virtualization
is
really
about
this,
making
sure
that
a
workload
is
not
allowed
to
use,
run
or
touch
any
critical
or
privileged
instruction
devices
or
piece
of
memory?
That
was
very
short
introduction
to
hardware
virtualization
in
the
context
of
kubernetes?
B
A
really
important
thing
is
that
we
won't
hardware
virtualization
to
be
transparent.
We
don't
want
it
to
be
something
that
end-user
need
to
know
about.
It's
just
a
way
to
add
more
workloads
that
you
can
read
through
kubernetes
make
them
more
secure,
but
it
has
to
be
transparent
to
to
the
rest
of
the
cluster.
A
Sure
so
the
workload
we
support,
natively
or
kubernetes,
is
supporting
natively
our
cloud
native
applications
and
Thanks.
So
what
we
have
today
is
that
we
have
a
kubernetes
api
server
which
is
taking
care
of
the
entities
taking
cover
all
parts
of
our
compute
units
and
that's
getting
scheduled
to
two
nodes
in
order
to
run
them
a
number
of
containers
which
really
represent
the
the
processes
which
we
want
to
run
and
they
really
do
the
computing.
All
that
is
run
I
mean
here.
A
We
just
show
one
pod,
but
all
of
the
pods
are
run
on
a
single
kernel
per
node.
So
there
is
no
no
differentiation
between
kernels
per
pod
or
per
container
so
by
adding
a
new
isolation
layer.
That's
funny
because
by
adding
a
new
isolation
layer,
we're
actually
allowed
to
run
cloud
native
applications
with
a
stronger
isolation.
So
how
does
it
look?
A
Kubernetes
encounter
containers
introduced
now
the
heart,
the
virtualized
spot
up
there
of
the
green
box,
there's
no
vault
written
statement
about
heart
with
virtualized
pod,
and
there
you
see
that
the
linux
kernel,
which
is
used
by
the
containers,
is
now
actually
part
of
the
pod
and
not
of
the
node
anymore.
So
that
means
we
can
have
a
per
pod
Linux
kernel
with
different
features
with
different
kernel
modules,
as
you
as
you
like
it,
and.
B
You
can
run
anything
you
want
inside
your
pot,
but
if
you
want
to
touch
any
physical
resources
on
your
node,
this
is
now
completely
virtualized
inside
your
virtual
machine
and
you're
no
longer
allowed
to
do
this
and
as
fabian
highlighted
there's
also
a
major
difference
here
is
that
now
you're
running
on
top
of
your
exclusive
kernel,
you're,
not
sharing
your
kernel
across
different
pots.
You
have
one
Linux
kernel
for
per
pod
and
you
can
use
and
set
and
tune
your
kernels
as
you
wish.
So,
if
you
have
a
workload,
that's
needs
security.
B
If
you
have
a
workload
that
needs
very
specific
kernel
modules
or
kernel
settings
or
camera
tunings,
you
can
do
that
with
with
the
Hubbard
versus
Y
spots,
and
so
this
is.
This
is
already
extending
the
range
of
workloads
that
you
can
run
with
communities
just
by
adding
a
nice
relation
layer.
That's
based
on
hardware
virtualization.
A
Very
well,
ok,
so
that
was
the
one
way
of
how
to
introduce
heart
of
it,
realization
to
kubernetes.
The
other
way
is
by
adding
in
new
custom
resource.
So
what
does
it
mean?
We
add
the
custom
resource
in
order
to
run
isolated
legacy
applications.
So
here
the
purpose
of
d
virtualization,
the
hardware
virtualization,
is
totally
different.
We
introduced
the
virtualization
as
part
to
isolate
a
different
workload
factor
from
from
kubernetes
in
order
to
run
here.
That
sounds
a
little
bit
weird,
but
that's
the
case.
B
B
So
here
I'm
going
to
go
into
available
more
details
on
the
on
the
first
part,
which
is
adding
an
isolation
layers
for
at
the
pot
level
being
able
to
use
hardware
virtualization
to
isolate
your
pots
between
each
other's
and
the
this
project
is
Carrick
in
teens
and,
as
I
said,
I'm
one
of
the
maintainer
of
Carrick
containers.
So
this
is
this:
what
I'm
going
to
talk
about
now?
B
So
Keller
containers
is
really
about
running
your
pod
in
a
more
secure
way
by
simply
putting
them
inside
a
virtual
machine
more
or
less,
and
it's
a
full-blown
virtual
machine.
You're
gonna
run
your
your
container
on
top
of
their
own
dedicated
Linux
kernel
with
their
own
dedicated
middleware
applications.
But
one
really
important
thing
about
Kara
containers
is
that
it's
it's
no
CI
compatible
runtime
the
same
way
who
knows
run
Z
in
room.
Who
knows
what
one
sees
yeah?
B
All
you
can
take
kubernetes
workload
today
are
running
on
top
of
run
T,
so
you
should
know
about
run
Z
and
the
run.
C
is
a
no
CI
compatible
runtime
it's.
It
sticks
to
the
OCI
specification
so
that
you
can
translate
the
the
commands
that
you
get
from
from
kubernetes
from
the
CRI
into
something
that
runs
he
understand
through
OC
I.
B
So
Cala
containers
is
following
exactly
the
same
specification
as
run
Z
and,
as
a
matter
of
fact,
you
can
switch,
run
Z
or
replace
fancy
with
Kara
containers
so
that
you
can
just
run
your
containers
as
you
would
usually
run,
except
that
they
will
run
inside
a
virtual
machine.
So
you
get
this
additional
isolation
between
isolation,
layer
between
pods,
with
Kara
containers.
Just
with
that
color
containers
provides
something
else.
B
It
provides
a
set
of
native
CRI
api's,
so
the
the
the
OCI
compatibility
is
very
useful
if
you
want
to
integrate
with
projects
existing
project
like
continuity
or
cry
out,
but
if
you
want
to
implement
your
own
dedicated
super
efficient
and
kubernetes,
only
CRI
implementation
and
not
talk
to
yet
another
set
of
abstraction
layers.
You
can
use
Keller
containers
just
as
a
library
and
you
can
implement
you
see,
regime
and
that
will
call
into
Kara
containers
to
manage
all
your
container
life
cycles
as
virtual
machine.
All.
B
You
part
lifecycle,
sorry
as
virtual
machines
and,
as
I
said,
with
Kara
containers,
either
with
the
OCI
compatible
runtime
and
cryo
/
continuity,
or
by
calling
into
the
the
native
craap
ice.
You
will
end
up
with
your
pod
running
inside
its
own
virtual
machine
and
that's
fully
transparent
to
the
rest
of
the
of
the
kubernetes
tacky
to
get
into
a
little
more
details.
This
is
this
is
how
the
the
whole
stack
on
the
node
so
that
just
one
node
I'm
not
talking
about
the
master
node.
B
This
is
how
it
looks
like
so
Cuba
led
call
into
CRI.
This
is
that
doesn't
change
it?
Calling
to
cry
or
or
container
e
same
thing
as
what
what
you
usually
do
today,
but
instead
of
calling
into
run
see
you
will
call
into
Cal
runtime,
which
is
the
OCI
compatible
binary
that
you
get
when
you,
when
you
install
a
container
on
your
note
and
that
runtime
will
just
take
whatever
OCI
commands
and
specification.
It
gets
from
the
CRM
implementation,
cryo
or
continuity,
and
translate
that
into
a
virtual
machine
that
will
run
your
container
seamlessly.
B
So
you
just
install
that
you
get
your
own
Linux
kernel
and
and
your
containers
inside
this
pod
will
run
inside
the
virtual
machine,
so
the
boundary,
the
security
boundary,
is
no
longer
than
namespace
or
the
other
C
groups
or
the
addition
of
two
of
the.
But
it's
the
virtual
machine
and
the
hover
virtualization
implementation.
B
As
I
said,
you
can
also
use
Kara
containers
as
a
library,
and
this
is
what
Frankie
another
CI
shim
implementation
does.
So
you
you
basically
remove
a
few
abstraction
layers.
It
simplifies
things
together,
but
it's
it's
yeah.
It's
less
integrated
and
it's
very,
very
kubernetes
Pacific.
But
if
you
want
to
go
that
way
and
have
a
very
efficient
implementation,
you
can
use
cata
containers
as
a
library,
and
this
is
what
fractal
does.
The
whole
flow
is
identical,
except
that
we
don't
call
into
yet
another
binary,
which
would
be
the
OCI
compatible
binary.
B
What's
your
status
today,
kara
containers
is
almost
1.0.
If
you
want
to
talk
about
release,
dates
and
schedule
and
from
the
OpenStack
foundation
is
here,
it's
going
to
be
released.
Fingers
crossed
in
May
22nd
at
the
OpenStack
summit
in
in
Vancouver
BC.
So
it's
almost
there
I,
don't
know
if
any
of
you
guys
are
familiar
with
clear
containers,
which
was
the
the
previous
project
that
kara
containers
is
based
on
calico
containers.
1.0
today
is
surpassing
clay
containers
the
latest
version
of
clay
containers.
B
B
You
can
use
I,
think
Tim,
I,
don't
know
if
Tim
is
in
room.
Tim
will
clear
from
from
Google
he's
going
to
talk
on
Friday,
about
secure,
pods,
sakura-san
thoughts
and
secure
containers
and
how
these
new
concepts
is
making
it
into
the
community
CRI
implementation.
It's
basically
a
way
to
say
now.
We
want
to
be
able
to
define
the
isolation
layer
that
I've
been
talking
about
for
for
the
past
five
minutes.
We
want
to
be
able
to
define
this
at
the
specified.
B
The
community
specific
specification
level
and
say
I'm
gonna
run
this
part
with
this
specific
isolation
layer
right
now
this
is
a
specified
and
the
only
thing
you
can
do
is
hope
for
the
best
and
see
what
what
the
the
components
below
you
run:
Z
and
cry
or
do
for
you
with
Keller
containers.
You
can
implement
this
new
secure
containers,
implication
and
basically
use
hardware,
virtualization
natively
within
kubernetes,
so
I'm
tempted
to
say
that
with
Kara
containers
and
some
other
solutions,
hopefully
VMs
are
becoming
a
first-class
kubernetes
citizen.
B
Up
to
now,
VMS
were
just
a
way
to
create
your
comparison.
Note
and
with
Kara
continues
now.
Vms
are
just
yet
another
isolation
layer.
The
same
way,
namespace
is
an
isolation
layer
between
between
pots.
Vms
now
are
just
another
one
yeah
Fabia.
Maybe
you
disagree
with
the
fact
that
VM
is
our
first
class
community
citizen.
So
if
you
wanna
talk.
A
About
this,
I
have
to
disagree
right,
no
I
mean
what
we've
seen
is
that
so
far
are
in
cata
containers
VMS
are
totally
transparent,
so
they're
introduced
to
you
and
just
wrapped
around
the
port
in
order
to
bring
you
isolation.
What
Qbert
is
now
and
we're
getting
to
that
is
it's
strong
isolation
for
legacy
workloads
and
the
workload
type
is
really
that
property.
What
what
helps
you
to
differentiate
Cala
from
Qbert,
which
we
try
to
to
solve
that
confusion
around
kalanchoe
Qbert
in
this
talk,
so
legacy
workloads
before
we
dive
in
there?
A
Let's,
let's
take
a
look,
what
legacy
workloads
on
why
they
are
supporting
also
for
Cuba
Cuban
Ares
legacy.
Workloads
are
all
the
application
you've
been
taking
care
of
over
the
last
years,
like
10,
15
20
years,
and
you
know
you've
built
them
with
love
and
dedication.
You
know
you
know
them
very
well,
you
get
into
the
office
in
the
morning
at
you
know.
Oh
well,
that's
the
little
bubble.
It
popped
up,
Philippi
Emma's,
like
MS
office
I.
Can
you
have
to
click
the
way,
so
you
know
very
well.
A
A
However,
besides
the
pure
technical
content,
it
is
also
important
that
we
take
a
look
at
how
are
these
applications?
How
were
they
created?
What
is
the
developer
workflow
around
them
and
that
workflow
is
completely
different
to
or
I
wouldn't
say
completely,
but
it
is
different
to
how
we
how
we
build
containers
and
cloud
native
applications.
We
might
not
have
that
complete
CI
CD
pipeline.
We
have
four
or
four
kind
of
containers
today,
but
we'd
rather
have
I,
don't
know
visual
studio
or
we
have
Linux
and
build
rpms
and
deploy
them.
A
A
So
the
point
is
that
we
might
not
always
be
able
to
to
change
this
legacy
application,
because
it's
too
cost
intensive
to
change
it
to
make
it
work
on
kubernetes
or
because
we
can't
change
the
processes
quickly.
So
we
have
that
long
validation
pipeline
with
a
lot
of
manual
testing.
We
cannot
change
that
from
one
day
to
the
other.
That
is
what
we,
but,
on
the
other
end,
we
see
the
need
that
we
need
to
move
into
the
future.
A
We
want
to
run
our
application
on
kubernetes
tomorrow
so
and
we
need
to
get
there,
but
we
can't
leave
our
legacy
application
alone.
So
so
what
should
we
do?
The
question?
Actually,
we
need
to
then
ask
ourself
is
how
can
we
run
our
legacy
workload,
our
legacy
application
in
the
cloud
native
world,
and
that
is
actually
where?
Oh,
yes,
to
rephrase
it
legacy.
Workload
in
this
case
is
a
virtual
machine.
Why
virtual
machine?
A
Because
if
we
look,
for
example,
at
the
vnf
virtual
network
function
sector
or
in
it
our
OpenStack
environment,
over
the
overt
environment,
VMware
proxmox,
all
those
virtualization
management
systems,
they
run
VMs,
so
a
form
factor
of
legacy
applications
seem
to
be
virtual
machines.
So
the
question
really
is:
how
do
we
run
virtual
machines
in
the
cloud
native
world,
and
that
is
where
Q
word
comes
in
Qbert
is
the
virtualization
operator.
A
So
there
was
just
the
operator
framework
introduced
by
the
Kouros
people
yesterday
pretty
interesting,
so
we
provide
an
I
in
runtime
which
allows
you
to
run
VMs,
as
you
know
them
so
VMs
which
contain
your
legacy
applications.
How
does
it
work
so
I
said
early
on,
the
user
is
working
on
a
VM
custom
resource.
So
if
you
look
at
the
kubernetes
api,
we
use
CR
DS
to
extend
it,
and
then
you
get
the
virtual
machine
API.
So
you
can
define
a
VM
as
you
know
it
or
you
import
the
VM
actually
into
kubernetes.
A
Then
we
have
our
cubed
operator
right
at
the
bottom,
who
is
seeing
this
new
resource
and
then
the
next
step,
creating
a
pod
with
that
specific
VM.
For
you,
that's
pretty
simple.
The
cool
thing
is
that
it's
running
inside
a
pod
that
might
look
crazy.
But
let
me
before
you
flame
me
some
flame
wars
used
to
be
a
common
thing.
A
The
nice
thing
is
because
it's
running
in
a
pod,
it
inherits
a
lot
of
stuff
from
the
pod,
node
affinity,
anti
affinity
we
inherited
from
kubernetes.
We
don't
have
to
implement
it
storage
attachment
to
parts
to
VMS,
because
it's
running
inside
the
pod
we
can
inherit
that
so
there's
a
lot
of
infrastructure.
We
can
just
take
from
kubernetes
and
use
it
for
scheduling.
It's
the
same.
To
just
make
the
scheduling
decision
we
inherited
from
kubernetes.
A
So
that
is
how
how
Cubert
works,
because
it's
an
operator
you
can
just
use
it
as
as
you
use
any
other
cloud
native
application,
so
deployment
ADIZ
is
easy.
You
use
cube,
CTL,
apply
and
the
ML
file
to
deploy
it
on
your
on
your
kubernetes
cluster.
You
might
require
some
privileges
in
order
to
do
that.
That's
fine,
but
then
you
use
can
can
use
cube
Carol
in
order
to
manage
the
VM,
so
you
can
use
cube
CD
or
get
oopsy
he'll
create
apply,
limited
like
with
pods.
A
So
it's
pretty
native
how
you
how
you
manage
those
VMs
for
divert
specific
functionality.
We
have
a
special
tool
called
verts
ETL,
which
is
also
a
cube
color
plugin,
in
order,
for
example,
to
get
access
to
the
graphical
display
of
a
VM
or
maybe
in
future
again
to
live
migrated
to
a
different
node.
You
can
try
that
yourself,
using
our
mini
cube
based
demo.
So
where
does
cubed
stand
today
leading
zero
in
our
version,
because
we
make
great
progress
in
the
last
year,
but
we're
not
there
to
be
called
stable.
A
Yet
what
we
do
have
is
Native
community
start
digression,
so
we
can
use
plain
kubernetes
Peavey's,
which
is
great
because
we
inherit
any
storage
subsystem
that
is
supported
by
kubernetes.
We
have
native
kubernetes
network
integration
so
into
the
stockpot
networking,
not
multiple
NICs
or
so
so
you
can
actually
work
with
vm.
You
look
just
like
you
work
with
any
other
pot.
You
can
use
it
with
services
with
ingress
with
labels
yeah
so
that
that
classical
stuff
we've
got
the
verticity
our
client
utility.
A
It
works
on
vanilla,
kubernetes,
an
open
ship,
for
example,
and
we
have
a
few
a
little
bit.
Syntactic
sugar,
like
virtual
machine
preset
and
offline
virtual
machines,
presets,
allow
you
to
define
predefined
virtual
machine
configurations
for
certain
guest
types
like
for
Windows,
for
real
for
Ubuntu,
for
MINIX,
FreeBSD
or
whatever
you
want
in
future,
or
what
we're
working
on
is
actually
a
UI
we're
looking
at
and
modifying
the
openshift
console.
A
B
So
we've
just
seen
2
project
katakana,
taters
and
kubert's
addressing
2
different
use
cases
in
in
in
the
ecosystem.
The
first
one
is
being
able
to
secure
cloud
native
application
with
Kara
continues
and
the
other
one
is
to
be
able
to
run
legacy
workloads
through
virtual
machine
and
the
the
thing
that
we
want
to
highlight
here
is
that
by
adding
hardware
virtualization
support
either
through
either
or
or
through
Kara
containers
or
Cubert,
you
can
significantly
expand
the
ranges
of
workloads
that
you
can
run
through
through
kubernetes.
B
B
A
This
yeah
for
I
mean
for
cut
it's
pretty
native,
because
they're
working
with
pods
and
for
ports
we
have
the
state
will
set.
You
know
all
the
high
level
controllers
will
will
apply
to
to
to
cut
it,
because
it's
just
a
different
way
of
running
pots
for
Qbert
it's
different,
so
Cuba
is
running
traditional
games
and
so
far
you
cannot
use
an
existing
high
level
workload
controller
for
for
custom
resources.
So
what
do
I
want
to
say
is?
A
We
have,
for
example,
a
virtual
machine
replica
set
which
allows
you
to
scale
up
VMs
and
scale
them
down
or
scale
out
rather
to
scale
the
moutain
scale
them
down
again.
However,
we
also
see
that
an
upstream
there
are
also
requests
to
see
that
high
level
worker
controls
like
deployment
replicas
set.
The
state
will
set
get
support
for
also
templating
custom
resources,
which
would
benefit
which
would
be
benefit
to
keyboard.
A
So,
yes,
we
do
virtualized
network
interfaces
and
currently
we've
virtualize
exactly
one
network
interface
and
that
network
interface
is
then
connected
to
the
port
network.
So
what
we
do
is
we
actually
have,
and
we
are
a
version
with
0.4,
because
we
still
do
research.
Currently,
we
move
the
IP
from
the
pod
to
the
VM,
so
the
pod
effectively
loses
network
connectivity,
which
is
not
too
bad,
but
the
VM
is
getting
the
full
network
connectivity.
So
you
can
you
can
access
the
cluster
just
like
from
the
pod.
A
B
Sorry
so
for
Keller
containers
you
as
far
as
networking
is
concerned,
you
can
you
have
many
different
choices.
You
can
do
verts
on
it.
If
you
want
to
really
share
you
the
same
networking
interface
across
different
containers
or
you
can
do
direct
device
assignment
mix
with
the
Silv,
for
example,
if
you
want
to
scale
your
networking
interface
and
keep
the
same
level
of
performance
or
if
you
don't
have
a
survey
available
on
your
on
your
card,
you
can
do
direct
device
silent
and
have
one
can
take
one
pod
using
one
networking
interface
exclusively.
B
A
The
way
and
just
an
additional
port
Cubert,
so
what
we're
doing
is
we're
actually
working
with
upstream
big
network
and
think
the
plumbing
network
network
plumbing
working
group.
In
order
to
see
that
we
get
the
required
networking,
we
want
four
VMs
in
kubernetes
first
and
then
being
able
to
consume
it
in
in
Cubert.
So
what
we're
currently
doing
is
you
know?
How
can
we
formalize
to
get
SRV
devices
or
DPT
week'
capable
the
interfaces
into
a
port
so
that
then
we
can
consume
it
from
yemm's?
D
Two
questions
which
ever
depending
on
how
much
you
have
time,
how
do
the
either
of
these
convert
to,
but
let
proposal
from
mere
aunties
and
also
how
much
of
these
two,
how
much
common
technology
under
the
covers,
because
it
sounds
like
there
should
be
a
lot
of
common
nests
here,
yeah.
So
two
questions.
What
Layton.
A
I
can
take
the
vert,
let
one
sure
yeah
I
should
have
mentioned
fertile
before
so
fertile.
It
is
actually
pretty
much
in
between,
but
it's
rather
between
Cara
and
between
Q
Bert
Bert.
Let
is
using
the
plot
specification
to
run
a
VM,
so
it
is
using
the
specification
of
Cara
and
it
is
running
VM.
Just
like
Q
Bert
is
doing
it,
and
so
it
falls
really
in
between
yeah.
D
A
B
B
So
first
fabian
was
highlighting
the
fact
that
we
go
to
signet
work,
cig,
node
and
all
sort
of
six
to
highlight.
The
fact
that
currently,
a
lot
of
assumptions
about
kubernetes
is
that
it's
going
to
run
a
process
inside
a
container
so
a
lot
of
the
vm
problems
and
and
are
not
taking
into
account
by
by
by
kubernetes.
B
So
we're
trying
to
join
forces
together
and
and
go
and
talk
with
this
with
these
six
to
make
sure
that
whenever
something
new
comes
up
in
the
specification,
it
also
accommodates
for
virtual
machines
as
just
a
regular
way
of
isolating
were
closed.
But
we
use
a
lot
of
different
different
technology
in
common
I
mean
the
whole
virtualization
scheme
and
implementation
is
exactly
the
same.
So
we
use
k,
vm
q
mu
and,
I
guess
yeah
yeah.
E
B
I
those
who
are
not
familiar
with
with
what
Google
is
announced,
it's
called
G
Weiser
and
it's
a
it's
what
they
call
it
a
hypervisor
and
it's,
but
it's
really
trying
to
virtualize
at
the
process
level
now
that
the
machine
level.
So
it's
a
it's
I'm,
going
to
take
a
shortcut
and
say
this:
is
it's
a
Cisco
proxy
and
that
you
can
virtualize,
so
it
virtualizes
all
the
SIS
calls
and
the
the
way
we
see
it
is
that
it's
I
mean
it's
very.
It's
one
more
stone
added
to
our
to
our
construction.
B
There's
Cubert
as
Kara
containers.
There's
no
G
Weiser,
that's
entering
the
this
game,
but
all
of
us
are
basically
using
hardware
virtualization
to
provide
one
way
of
isolating
workloads
between
each
others.
I,
don't
think
the
overlap
between
divisor
and
and
Kara
containers
is
is
huge.
I
think
divisor
has
a
very
specific
set
of
workloads
that
they
can
efficient
efficiently.
B
Virtualize
for
this
very
specific
set,
they're,
probably
very
good
and
very
efficient
Kara
containers-
is
using
virtualization
one
layer
up
to
that
where
we
literally
virtualize
everything
devices
syscalls
everything,
whereas
divisor
is
more
focused
area
and
in
in
in
one
hand
you
have
Kara
containers.
On
the
other
hand,
we
have
a
much
more
focused
set
of
workloads
and
probably
more
efficient
for
that
specific
set
of
orbitals,
so
yeah.
F
A
F
So
if
it's,
the
answer
is
too
complicated,
we
may
discuss
it
afterwards,
but
anyway,
how
does
the
problem
of
multiple
runtimes
get
soft
I
mean
you
probably
need
to
runs
things
like
Q
proxy
pots
on
the
same
node
as
the
inputs,
and
how
does
it
is
it
handled?
Basically,
you
don't
want
to
run
to
have
Q
proxy
on
the
node
run
inside
the
VM
right.
So
how
is
this.
B
So
the
question
is
about
how
you
you
handle
multiple
runtimes,
because
for
those
who
not
really
from
here
with
that,
the
idea
is
to
be
able
to
run
karat
containers
alongside
regular
containers.
So
you
want
to
be
able
to
run.
You
run
C
containers
together
with
your
caret
containers
on
the
same
node,
and
you
need
a
way
to
specify
to
the
runtime
which
continue
around
time.
You're
going
to
use
you
need
to
tell
you
need
to
be
able
to
tell
see
a
cry
or
continuity
or
frak
C.
B
Okay,
I
want
to
run
this
workload
in
a
more
secure
way
or
in
a
more
isolated
way
and
and
I'm
gonna
use
this
specific
runtime.
So
it's
a
question.
It's
an
architectural
question
and
I
think
this
is
currently
being
discussed
in
in
signal
as
part
of
the
secure
container
specification
where
we
we
are
going
going
to
hit
that
problem,
and
we
want
to
solve
that
problem.
Collectively.
I,
don't
know
what
what's
going
to
be
the
solution,
because
it's
going
to
be
a
community
decision
but
yeah.
B
It's
part
of
the
Biggers,
a
secure
container
and
a
discussion
that
are
happening
right
now.
So
I
don't
have
an
answer
for
this
right
now
in
in
cryo
and
continuity,
we
use
kubernetes
specific
labels
to
basically
select
which
runtime
you're
going
to
use.
So
if
you,
if
you
want
to
run
a
pod
and
you
label
it
as
insecure
or
if
you
label
it
as
needing
a
more
isolated
pod,
it's
gonna
run
a
specific
runtime.
If
you
don't
label
it
this
way,
it's
gonna
burn
the
default
runtime,
which
is
basically
grungy,
and
you.
B
F
Basically,
yeah:
that's
how
we
also
do
it
with
use
this
CI
proxy
think
that's
also
works
with
docker
and
my
another
question
is
about
people
concerning
the
network
support
when
we
add
in
CNI
support
to
worklet.
We
ran
into
the
problems
that
CNI
spec
is
not
very
well
followed
by
different
CNI
plugins.
So,
for
instance,
in
case
of
calico,
we
look
at
the
very
hard
time
with
their
Gateway
and
so
on.
So
basically,
we
had
to
use
some
special
cases
for
different
see
a
nice.
So
my
question
is:
how
well
are
different
sin?
F
A
Let
me
ask
the
second
question.
Answer
the
second
question:
first,
yes,
you
can
just
write
it
V
M,
you
can
even
use
services.
Covery
fqdn,
based
service
discovery
with
with
the
VMS,
and
you
can
use
services
with
the
VMS.
You
can
use
network
policies
with
DBMS.
It's
just
like
a
pod
wait.
The
other
thing
is
is
more
difficult
to
answer.
My
answer
is
we
see
there
are
limits
with
calico
and
weave
and
flannel,
but
we
we
see
that
kubernetes
in
general,
has
has
the
desire
from
the
community
to
get
a
better
networking
support.
A
So
that's
where
we
want
to
how
we
want
to
fix
it,
not
by
looking
how
we
can
fix
it
for
flannel
for
Wii
for
calico,
but
rather
see
how
can
we
lift
the
networking
concepts
in
communities
to
our
higher
level
to
support
our
use
case?
So
that's
actually
network
services
from
Cisco
is
actually
something.
You
also
want
to
look
into.
I
guess:
okay,.