►
From YouTube: Introduction to Virtual Kubelet – Featuring Titus by Netflix - Ria Bhatia & Sargun Dhillon
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Introduction to Virtual Kubelet – Featuring Titus by Netflix - Ria Bhatia, Microsoft & Sargun Dhillon, Netflix
Virtual-kubelet can extend kubernetes in many interesting and unique ways. This talk will go through how providers are utilizing virtual-kubelet to extend Kubernetes either for their customers, or for the benefit of their platform. The talk will also go through how Netflix is using virtual-kubelet to aid in integration with their existing architecture. Virtual-kubelet is able to give them the best of both worlds. Netflix has been able to leverage the Kubernetes API Server and the controllers as a mechanism to accelerate their control plane, whilst being able to use their existing containerization and isolation technology that’s been in development under the guise of the Titus (https://medium.com/netflix-techblog/the-evolution-of-container-usage-at-netflix-3abfc096781b) project since December 2015. The flexibility of the project, has allowed them to introduce new southbound, and northbound concepts to their product, which is enabling greater efficiency, and scalability.
https://sched.co/Uai8
A
Okay,
cool
hi:
everyone
can
you
guys
hear
me
cuz
I
can
hear
myself
good,
so
I'm
really
excited
because
we
have
Netflix
on
stage
with
us
they're
an
end-user
for
virtual
kulit,
but
I'm
going
to
go
through
an
intro
first,
but
first
to
introduce
myself.
My
name
is
Rhea
Bhatia
I'm,
a
program
manager
at
Microsoft.
I
am
one
of
the
maintainer
zuv
virtual
couplet
and
here's
Sargon.
Sorry.
B
A
So
to
get
us
started
a
little
bit
of
history
about
virtual
ku,
but
it
started
off
as
a
small
project,
actually
a
just
an
experimental
project
by
Brendon
burns
at
Microsoft,
and
we
basically
were
just
playing
around
with
what
it
meant
to
make
kubernetes
more
flexible,
because
a
node
and
kubernetes
is
inherently
not
very
flexible.
So
that's
where
we
started
and
it
was
actually
called
the
ACI
connector
a
long
time
ago
and
before
koukaon
into
summer,
2017
I
want
to
say
we
decided
to
make
it
open
and
we
decided
to
make
it
pluggable.
A
It
actually
started
out
as
a
tight
script
project,
and
then
we
rewrote
everything
and
go
presumably
to
make
it
more
friendly
to
get
working
developers
in
the
project.
I
think
it
worked.
I
don't
know
if
you
guys
want
to
work
in
typescript,
but
we
could
probably
do
that
too
in
the
future,
probably
not
yeah,
but
that
was
the
room
that
we
all
worked
in.
We
hacked
on
it
and
there's
Eric
who's
over
here.
He
was
also
one
of
the
core
people
in
the
beginning
to
help
us
start
on
the
project.
A
Ok,
so
what
is
virtual
kulit,
so
rich
I'll
cool
it?
We
don't
replace
the
couplet,
but
what
we've
done
is
inherently
make
it
more
flexible
for
you.
So,
for
example,
one
use
case
enema
I'm
gonna
make
it
really
abstract.
So
it
makes
sense
if
you
wanted
to
order
pizza,
for
example,
and
you
wanted
to
create
a
provider
that
ordered
pizza,
you
could
do
that
with
virtual
couplet.
So
anytime,
you
in
a
pod
against
the
virtual
couplet
node
in
your
cluster.
It
would
spin
out
and
actually
call
Domino's
and
get
you
a
pizza.
A
So
that's
how
abstract
we've
made
the
note
if
you
know
how
to
fill
in
the
functions
of
creating
a
pod,
deleting
a
pot
and
updating
a
pod,
you
can
basically
create
a
provider
for
virtual
kulit
and
that
could
be
anything
from
ordering
a
pizza
to
turning
on
the
lights
in
this
room.
It's
really
really
flexible
in
that
case,
in
super-easy
gate
get
started,
and
these
are
the
things
that
we've
built
into
it.
So
you
can
get
pods,
you
can
get
pod
status.
A
You
can
update
the
pod,
you
can
create
a
pod
and
we
also
have
some
loosely
defined
capacity
like
restrictions
around
it,
because
we
don't
actually
need
to
make
a
restriction.
So
they're
restrictions
like
one
terabyte
of
memory
and
things
like
that.
But
that's
because
we're
not
an
actual
node
we're
just
sending
out
API
calls
to
whatever
is
in
the
backend,
so
whether
it's
Domino's,
whether
it's
a
sure
whether
it's
Amazon,
whatever
it
could
be
all
we're
doing,
is
just
transferring
information
from
kubernetes
to
that
actual
back-end
service.
A
Alright.
So
the
anatomy
of
the
node
today
and
I'm
gonna,
step
in
front
of
here
just
so
I,
don't
miss
anything.
So
the
node
inherently
comes
with
a
lot
of
constraints
and
that's
what
we
set
out
to
solve.
How
do
we
make
a
node
more
flexible?
How
do
you
have
an
unbounded
amount
of
resource
and
things
like
that?
And
so,
and
there's
a
lot
of
constructs
of
pods
actually
getting
scheduled
out
to
nodes
that
we
also
needed
to
figure
out
within
virtual
kulit.
A
So
when
you
schedule
a
pod
to
a
virtual
couplet
node,
we
actually
changed
the
way
that
you
have
to
interact
with
the
node
itself.
So
you
don't
lie
within
the
same
capacity
restraints
because
you're
not
actually
in
the
cluster.
So
a
lot
of
the
pods
that
we're
spitting
out
are
actually
backed
by
a
different
service
and
they're,
not
in
your
kubernetes
cluster
at
all.
A
Ok-
and
this
is
another
slide
to
show
you
what
the
node
does
today
we're
continuously
in
kubernetes
getting
there.
We
have
control
loops
to
basically
get
container
logs
and
get
status,
and
things
like
that.
This
is
something
that
we've
replicated
in
virtual
Kubla,
but
we
have
a
smaller
set
of
things
that
we've
replicated
in
virtual
kulit
itself
and
what
is
virtual
couplet,
so
virtual
couplet
creates
treats
the
concept
of
pods
and
nodes
in
the
abstract
just
like
what
we
talked
about
before
with
Domino's
it's
an
unbounded.
A
I
know
it
isn't
about
an
amount
of
resource,
but
virtual
qubit
itself
is
not.
We
have
a
lot
of
common
code
in
the
VK
in
virtual
couplet.
Well,
we
don't
have
a
lot
of
common
code,
but
that's
where
providers
come
in
so
you
actually
gets
to
define
how
networking
works.
You
define
how
creating
a
pod
works,
how
updating
a
pod!
We
just
give
you
a
library
which
you
fill
in
on
your
own
and
we
also
have
a
community
that
can
back
you
up
and
help
you
write.
A
Your
own
providers
and
virtual
Kumud
itself
can
actually
be
deployed
as
a
pod.
So
that's
what
we
do
in
a
jar
today.
We're
deploying
virtual
cool
it
as
a
pod,
but
it
looks
like
a
node
in
your
cluster.
But
if
you
go
into
your
coop
system,
name
is
face:
you'll,
be
able
to
deciever
to
cool
it
as
a
pod
cool.
A
So
why
virtual
couplet
and
we
have
a
bunch
of
different
use
cases,
one
of
them
being
I'm
the
bursting
use
case
of
being
able
to
burst
into
a
different
service.
So
you
don't
have
to
use
the
capacity
within
your
cluster.
You
can
burst
out
somewhere
else.
We
also
have
some
IOT
use
cases
if
you
want
to
basically
represent
a
virtual
couplet
pod
as
an
IOT
device.
A
I
mean
things
like
that
and
then
also
just
an
alternate
couplet
and
we'll
probably
get
into
that
a
little
bit
later,
and
this
is
a
shout-out
to
the
azure
folks
and
something
that
I
worked
on
in
the
beginning,
so
I
kind
of
wanted
to
show
off
the
virtual
couplet.
First,
we
can
use
case
where,
basically,
you
have
a
defined
pool
of
resources
within
your
VMs
and
you
use
virtual
kulit
to
spin
out
into
something
called
a
dream
container
instances.
B
So
I'm
we're
talking
about
changing
the
engines
out
and
fight
in
how
we
adopted
the
virtual
kulit
for
utilization
and
production.
The
system
I'm
going
to
be
talking
about
is
called
Titus.
Titus
is
the
primary
container
runtime
that's
used
at
Netflix.
It's
used
for
all
sorts
of
applications
at
Netflix
from
big
data
workloads
like
presto
to
our
online
streaming
service.
So,
if
you've
used
Netflix
on
your
phone,
you
probably
use
Titus
to
our
media
encoding
engine
and
the
project
was
started
in
2015
in
2015
the
container
run
time.
B
Landscape
was
pretty
significantly
different
for
one
kubernetes
was
only
about
a
year
old
and
it
hadn't
really
become
the
ubiquitous
cluster
manager
that
it
has
become
today
and
when
we
started
the
project,
we
weren't
really
sure
whether
this
was
gonna
work
or
not.
So
we
wanted
to
try
to
leverage
as
many
off-the-shelf
technologies
as
possible.
In
leveraging
those
technologies,
we
decided
to
use
docker
as
a
container
runtime
basis
as
our
cluster
management
and
resource
management,
layer
and
G
RPC
of
protobufs
for
API
definitions.
The
team
that
works
on
titus
is
relatively
tiny.
B
Today
we're
an
11
person
team
in
about
two
years
ago
we
were
a
six-person
team.
We
develop
operate
and
support
the
product,
we're
on
call
for
it.
We
have
to
provide
user
support
and,
given
that
we're
such
a
lean
team,
we
have
to
make
our
decisions
relatively
conservatively
and
in
that,
when
we
adopt
new
technologies,
we'd
like
to
move
iteratively
as
opposed
to
make
big
jumps
about
a
year
ago,
we
decided
to
open
source
office
code
and
what
I'm
gonna
be
talking
about
today
is
mostly
in
the
open
source.
B
Now
the
tightest
architecture
is
a
little
bit
different
than
what
you
have
in
traditional
kubernetes
environment.
The
tightest
architecture
works
with
the
tightest
API
fronting
fronting
it.
The
tightest
API
has
many
concepts
that
are
very
similar
to
AWS
concepts,
because
most
people
and
Netflix
are
used
to
AWS
concepts.
So
we
have
the
idea
of
a
job
which
is
very
similar
to
an
ASG
and
the
idea
of
a
task
which
is
very
similar
to
an
instance.
B
In
addition
to
this,
the
tightest
API
has
many
references
to
actual
AWS
constructs
like
a
Obi's
like
Auto
scale
and
groups
etcetera,
because
we
try
to
mirror
these
concepts
as
closely
as
possible
or
even
push
responsibilities
as
much
as
possible
onto
AWS.
Behind
that
tightest
API
we
have
the
tightest
master
and
the
tightest
master
is
responsible
for
orchestrating
the
cluster
and
it
has
the
source
of
truth
of
the
cluster
that
stores
inside
Cassandra
and
it
forces
down
all
of
the
state
of
cluster
in
tomatoes
and
maize.
B
So
this
is
then
responsible
for
farm
and
get
out
to
all
of
the
agents
and
getting
them
to
converge
on
the
intended
state
of
the
cluster
diving
a
little
bit
deeper
into
the
tightest
agent
architecture.
Today
we
use
a
previously
we've
used
meso
station
and
Mesa
cosa
agent
has
around
the
tightest
executors
and
the
tightest
executors
effectively
our
own.
You
can
think
of
it
as
a
provider
where
it
talks
a
docker,
it
spins
up
containers,
it
matters
that
container
lifecycle.
So
what's
missing
in
this
architecture,
it's
lasted
us
a
few
years.
B
Now
it's
in
production,
and
we
were
pretty
happy
with
it.
There's
a
couple
of
things
that
we
wanted:
what
are
these
was
a
northbound
API,
so
the
idea
that
pods
and
nodes
could
signal
upstream
weather
status
is
we've
actually
talked
about
this
in
our
blog
in
the
blog
post
of
using
predictive
CPU
scheduling
for
opportunistic
workloads.
These
nodes
know
more
about
their
state
than
the
cluster
of
control.
Plane
does,
for
example,
a
node
can
analyze
its
workload
and
say
I
have
for
free
CP
is
available
and
change
its
node
status.
To
say
this.
B
The
other
aspect
is
extensibility.
The
fact
that
the
downstream
API
can
include
anything
like
an
I
am
role
or
various
metadata
about
the
pod
allows
us
to
go
ahead
and
monitor
that
information
and
act
upon
that
information
intelligently
previously
with
meso.
The
answer
for
this
was
to
write
a
custom,
C++
module
and
do
a
bunch
of
awkward
stuff
to
make
it
work.
Whereas
a
you
know,
kubernetes
has
some
better
solutions
for
this,
and
at
the
end
of
the
day,
we
wanted
to
leverage
custom
data.
B
We
wanted
to
be
able
to
leverage
the
RDS
and
we
wanted
to
leverage
config
maps
in
other
objects
in
order
to
instrument
our
system
in
order
to
go
ahead
and
experiment
with
our
system
and,
lastly,
the
vibrant
community.
We
saw
what
the
kubernetes
community
was
doing
and
what
all
of
you
out
there
have
been
developing
and
we
were
kind
of
jealous
like
we
wanted
access
to
this,
so
that
left
us
kubernetes
curious
and
we
wanted
to
see
whether
we
could
start
to
leverage
parts
of
kubernetes
in
our
ecosystem
or
in
our
environment.
B
So
we
looked
at
the
typical
kubernetes
architecture
and
in
the
typical
kubernetes
architecture,
instead
of
having
a
separate,
API
and
control
plane
and
that
everything
is
built
around
the
API
server
and
all
of
your
clients
go
ahead
and
talk
to
the
API
server
and
whisper
in
your
objects
around
you
know,
replication
sets
or
staple
sets,
etc.
And
then
you
have
controllers
that
go
ahead
and
act
on
these
objects
to
go
ahead
and
create
the
pods
inside
of
the
API
server
that
then
the
kubja
is
responsible
for
converging
the
state
on.
B
If
we
look
at
the
typical
couplet,
the
couplet
does
have
a
level
of
modularity.
The
level
of
modularity
in
the
Kubla
is
built
upon
the
idea
of
using
the
CRI,
the
container
runtime
interface.
The
container
runtime
interface
allows
people
to
drop
in
replacement
container
runtimes
and
run
their
own
containers,
but
these
container
runtimes-
and
it
has
two
other
large
portions-
of
plug
ability,
CSI,
the
container
storage
interface
and
CNI
the
container
network
interface.
B
B
It
works
for
most
people,
but
Titus
has
grown
over
the
past
couple
years
to
become
a
fairly
large
portion
of
our
fleet,
and
in
that
we
have
a
lot
of
users
inside
of
Netflix
that
are
used
to
Titus
API
and
asking
those
hundreds
of
users
to
move
over
to
a
new
API
would
probably
be
a
little
bit
difficult.
The
other
aspect
is
that
we
wanted
to
use
as
many
components
as
we
had
today,
because
we
were
operationally
used
to
them
and
we
had
known
too
we
we
knew
how
to
run
them
at
scale.
B
So
the
other
approach
is
to
run
the
API
server
as
a
replacement
tomatoes.
This
is
nice
because
it's
a
small,
incremental
change.
It
involves
replacing
one
component.
The
other
aspect
is
that
it
doesn't
require
those
hundreds
of
callers
to
move
over
the
the
downside
of
this
is
it's
an
unproven
path.
It
not
always
in
unproven
paths.
The
couplet
itself
isn't
meant
to
absorb
pods
really
from
like
arbitrary
pod
providers,
or
have
a
lot
of
the
information
that
we
were
able
to
pass
in
to
the
containerization
platform.
B
Previously,
in
addition
to
that,
the
other
aspects
are
that
Titus
has
some
different
concerns
in
terms
of
security
and
in
terms
of
isolation
that
D
typical.
Our
architecture
did.
For
example,
we
care
a
lot
about
security
in
terms
of
container
breakout.
We
care
a
lot
about
isolation
in
terms
of
performance.
So
with
this
architecture,
we
decided
to
go
down
the
route
of
using
API
server
as
a
drop-in
replacement
from
asos
and
running
this
thing
that
we've
decide
to
call
titus
kubernetes
in
titus
kubernetes.
B
What
we
do
is
we
have
the
Titus
master
and
the
Titus
master.
All
it
does.
Is
it
pushes
pods
down
into
API
server
and
then
API
server
only
runs
a
subset
or
runs
with
a
subset
of
the
controllers
that
you
typically
have
make
kubernetes
architecture,
and
these
are
just
your
basic
things-
to
maintain
your
namespaces
to
maintain
policy
etc.
B
And
then
this
gets
pushed
down
to
the
agent
whose
job
it
is
to
eventually
converge
upon
the
state
that
the
Titus
control
plane
has
whispered
down
into
API
server,
and
these
controllers
are
things
that
we
all
largely
have
already
implemented.
So
we
already
have
implemented
things
like
resource
code.
Isn't
the
tightest
master
a
horizontal
pod?
B
Let's
go
in
the
ties,
master
the
idea
of
a
cloud
controller,
and
these
are
the
proven
components
that
I
was
talking
about
earlier,
not
only
this,
but
we
also
have
components
inside
of
the
rest
of
the
ecosystem,
like
spinnaker,
that
does
deployments,
and
that
does
config
management
for
us.
So
what
about
the
agent?
How
do
we
move
forward
with
the
agent
given
that
this
is
how
we
move
forward
with
the
control
plane?
B
So
we
ask
ourselves
what
if
we
just
implements
Eri
what,
if
we
just
implement
that
API,
that
I
was
talking
about
earlier,
that
anyone
can
go
ahead
and
drop
in
a
replacement
modular
interface
well,
first
year
to
understand
what
we
were
working
against
before
we
were
working
against
the
maysa
API
and
the
miso
API
is
a
much
smaller
footprint
in
a
much
less
opinion.
Api.
B
It
effectively
has
two
callbacks
launch
tasks
in
kill
tasks
and
in
that
it
basically
passes,
use
and
bytes
that
are
from
the
control
plane
and
you
get
to
decide
whatever
you
want
to
do
with
it.
It's
not
particularly
opinionated
on
what's
what's
in
that
it
can
be
a
pod,
it
can
be
a
container,
it
can
even
be
a
spark
job.
On
the
other
hand,
this
is
the
CRI
API,
and
this
is
only
a
subset
of
the
CRI
API.
B
In
addition
to
that,
we
had
built
up
our
own
networking.
Plugins
in
our
own
storage,
plugins
that
didn't
really
jive
in
the
world
of
CNI
in
CSI
and
the
cost
of
moving
these
over
was
unknown
at
the
time.
We
also
had
our
own
container
composition
model
that
wasn't
pods.
Instead,
we
relied
on
this
idea
of
system
services,
where
the
actual
executor
was
responsible
for
injecting
in
the
sidecars
that
ran
alongside
the
user's
code.
B
The
only
real
thing
that
we
were
missing,
that
the
Koopa
implemented
was
C
group
management
and
being
able
to
make
sure
that
the
container
is
torn
down
at
the
end
of
the
the
executors
lifecycle.
So
another
question
was
what,
if
we
just
implement
a
kubernetes
api
client,
what?
If
we
go
ahead
and
we
talk
directly
to
api
server
and
we
go
ahead
and
have
all
the
behavior
of
converging
the
state
of
the
node
ourselves.
B
Api
saying
this
is
what
you
should
be
running
to
pushing
that
down
to
an
imperative
API
and
where
you
can
implement
the
callbacks
to
say.
Let
me
actuate
your
system
say,
run
this
task
or
set
this
task
down.
In
addition
to
that,
virtual
couplet
makes
it
so
that
you
don't
have
to
do
a
lot
of
the
heavy
lifting
that
you
traditionally
have
to
do
when
you're
implementing
an
API
server
client
like,
for
example,
the
state
caching,
it
does
secret
caching
and
it
has
config
map
caching.
So
you
don't
have
to
implement
a
lot
of
this.
B
What
we
call
under
French
and
heavy
lifting
it
also
implements
a
reconciliation
model.
It's
been
proven
by
a
bunch
of
folks
and
been
operationalized
by
a
bunch
of
other
folks,
rather
than
having
to
have
us
figure
out
how
to
implement
this
in
step
on
all
the
potential
landmines
that
are
out
there
in
the
virtual
coop
of
the
API.
The
callbacks
that
we
I
was
talking
about
are
really
similar
to
the
Masons
callbacks.
B
B
B
We
didn't
have
to
retrain
everyone
on
how
the
the
system
runs
differently
on
kubernetes
versus
on
our
legacy
architecture
and
as
a
cluster.
What
we
can
do
is
we
can
run
an
instance
of
Titus
virtual
kuba
on
every
instance
inside
if
our
our
clusters-
and
then
these
can
go
ahead
and
spend
uptight
executors
underneath
them
and
we
can
reroll
these
in
flight.
So
did
the
the
virtual
couplet
library
has
features
to
be
able
to
do
in
place.
We
start
in
a
place
reconciliation.
B
So
what
about
the
transition?
We
obviously
couldn't
say:
hey
everyone
shut
down
your
pods
shut
down
your
clusters
and
move
over.
We
had
to
have
a
Federation
solution
and
that's
where
Titus
Federation
came
into
play
and
what
Titus
Federation
is
is
it's
a
mechanism
by
which
we
can
use
to
route
different
workloads
to
different
cells
in
each
region.
We
have
about
two
cells
today
to
the
three
cells
and
we
split
workloads
in
order
to
scale
tightest,
because
we
know
how
much
scale
a
given
cell
can
handle.
B
We
have
a
look,
a
side
table
that
says
based
on
some
routing
rules,
dispatch
this
workload
to
a
cell
and
we
were
able
to
leverage
the
same
capability
to
say:
hey
your
workload
is
a
little
bit
more
tolerant
to
things.
Therefore,
we
can
route
you
to
the
Cooper
knyghtes
cluster
earlier,
rather
than
a
very
high
risk
workload,
and
then
on
the
read
path.
It
can
take
the
view
from
our
kubernetes
cells
and
our
Mesa
cells
and
combine
these
into
one
global
view
of
the
system.
B
So
what
we
were
able
to
do
in
transition
was
run.
The
stack
with
maysa
and
run
the
staff
with
kubernetes
side
by
side
under
tightest
Federation
and
our
users
were
able
to
continue
to
speak
the
same
API
that
they
were
speaking
to
before,
where
they
had
these
models
that
were
very
similar
to
AWS
models,
and
this
is
something
that
we
been
able
to
prove
at
some
level
scale.
B
B
So
what
we
ended
up
doing
in
order
to
solve
this
problem
was
we
ran
the
couplet
and
we
simulated
a
bunch
of
state
transitions
and
we
figured
out
what
the
couplet
was
doing
in
made
the
virtual
koopa
act
in
the
same
way
and
that
kind
of
became
like
a
running
thing
where,
rather
than
having
any
kind
of
documentation
to
look
towards
the
solution,
was
to
look
towards
the
good
code
in
the
couplet.
As
a
formalization
of
the
specification,
so
what
about
the
future?
B
The
other
aspect
that
we've
considered
is
how
do
we
scale
out
API
server,
and
can
we
leverage
--vk
to
scale
at
the
API
server
of
saying,
have
one
v-k
that
registers
in
multiple
API
servers
and
combines
this
data
together?
We're,
alternatively,
even
having
a
VK
that
represents
multiple
API
servers
underneath
it
or
multiple
clusters
underneath
it.
B
Lastly,
the
flexibility
that
the
VK
gives
us
is
immense
and
we've
thought
about
adding
new
executors
to
the
virtual
kulit.
For
example,
the
ability
to
add
a
firecracker,
G
visor,
based
executors,
would
give
us
the
ability
to
run
containers
in
greater
isolation
rather
than
having
a
environment
where
we're
using
docker.
That
has
lesser
isolation,
and
that
is
the
tightest
virtual
couplet.
Thank
you.
B
A
C
So
you
have
said
that
we
can
use
the
pad
abstraction
to
express
pretty
much
anything
which
I
understand
that
this
does
not
need
to
be
necessarily
were
close,
as
we
know
them
with
some
kubernetes.
You
use
the
example
of
ordering
pizza.
The
question
is
kubernetes,
like
the
extreme
kubernetes
is
pretty
strict
about
the
schema
of
the
pod
template
of
the
pods
back.
So
how
do
you
create
your
own
custom
schema
for
a
pods
back
for.
C
A
Basically,
take
in
the
same
pod
sect
that
kubernetes
does
it's
still
kubernetes.
That's
delegating
the
workloads
to
the
nodes
specifically,
so
it's
still
Cuban
eyes.
That's
scheduling
it
out,
so
it
still
needs
to
understand
the
same
things
and
we
take
in
so
what
we've
done
on
the
provider
side
is
we'll
take
in
the
same
like
if
you're
having
a
CPU
constraint,
it's
up
to
the
provider
itself
to
write
that
in
and
take
it
in.
But
if
you
do
write
it
in
it's
not
gonna.
Stop
you
from
scheduling
out
to
the
virtual
couplet
node.
A
A
Yeah
and
that's
it's
really-
we
put
a
lot
of
that
work
and
I'm
talking
from
like
a
maintainer
virtual
kulit.
We
put
a
lot
of
that
onus
on
the
actual
providers
themselves,
so
you
will
implement
as
much
of
the
pods
back
as
you
want
to
and
you'll
take
in
you'll
figure
out
what
it
means
in
terms
of
consumption
and
in
terms
of
what
even
like
what
a
pod
image
would
mean
and
things
like
that,
but
for
curious
discovery.
Well,
it
does
need
to
have
some
sort
of
you
know,
aspect
of
a
pod.
D
Yeah
so
I
think
this
in
the
Netflix
use
cases.
Stephanie
is
good.
One
demonstrator
use
these
of
the
virtue,
coupe
net
buy
and
shooing,
and
how
to
adapt
existing
and
software
platform
right
using
other
technology
and
into
the
kubernetes
in
a
platform.
You
also
discussed
the
three
dolphin
have
between
different
options.
I
was
just
wondering
and
if
Jerez
speaking,
the
English
it'll
cases
if
I
build
something
from
scratch.
So
what's
the
good
user
case
is
Steven
useful
using
the
coop
net?
D
Also,
if
you
look
at
the
long
term,
rights
and
montañas
other
thing,
so
you
think
this
kind
of
the
solution
yeah
by
introducing
the
cubanelle.
What
you
could.
Let
us
do
a
good
solution
from
the
non-han
perspective
or
why
don't?
We
just
have
a
puny
and
kubernetes
and
the
native
solution.
Thank
you.
So.
B
For
us,
we
had
a
couple
things
to
consider.
We
had
the
immediate
world
to
consider.
Do
we
spend
a
bunch
of
time
transitioning
to
a
code
base
which,
although
people
run
at
a
relatively
large
scale,
don't
run
an
environment?
That's
like
exactly
like
ours,
and
the
other
aspect
was:
how
long
do
we
want
to
spend
migrating
our
code?
There
was
the
slide
earlier
that
I
had
that
showed
the
overlap
between
components
at
the
virtual
couplet
at
the
time
his
executors
implemented
and
the
Kouba
implemented.
A
Are
you
asking
for,
like
the
general
use
kit,
it's
a
general
use
case
of
virtual
couplet
I
mean
if
you're
trying
to
mask
different
infrastructure,
that's
not
easily
implemented
in
core
kubernetes.
Already,
that's
where
I
would?
The
couplet
would
come
in
or
if
you're,
trying
to
spin
out
to
a
different
infrastructure
or
a
different
service?
And
things
like
that,
if
couplet
bridge
to
a
couplet,
isn't
it's
the
solution
to
the
problem?
A
You
wouldn't
look
at
virtual
cool
at
first,
you
would
say:
hey
I,
have
this
problem
I
want
to
use
kubernetes
on
top
of
Titus
or
I
want
to
I
want
to
be
able
to
use
my
container
instances
infrastructure,
but
I
want
to
use
the
same.
Api
is
as
kubernetes
and
have
a
cluster
on
this
side,
or
a
hybrid
use
case
could
also
be
in
this
case,
where
I
have
on-prem
and
I
want
I'm
on
prime
clustered
nodes
or
something
that
I
also
have
a
cluster
in
the
cloud.
A
E
A
Using
nodes
in
your
cluster
and
a
virtual
couplet
node
right
yeah,
that's
one
of
our
primary
use
cases
in
Azure,
actually
in
a
lot
of
the
clouds.
Is
we
run
in
this
mixed
mode
where
we
use
Azure
container
instances
and
we
back
that
up
by
virtual
couplet
and
then
in
the
cluster
itself
in
your
Karina's
cluster,
wherever
it
is,
that's
the
nodes
that
are
in
the
cluster
and
then
anytime,
you
spin
out
to
the
virtual
Kubla
it
actually
spins
out
to
a
different
service.
A
E
E
A
So
I'm
a
one
thing
I
didn't
mention
is
we
have
a
lot
of
providers
in
virtual
couplet
and
so
past
Netflix
past
Microsoft?
We
also
have
been
working
with
Alibaba
VMware
and
those
are
provided
in
Hashi
corp,
and
these
are
providers
that
are
providing
services
to
people
in
terms
of
end-users
since
we're
all
like,
since
the
thing
is
virtual
Kubla,
it's
very
specific
in
the
terms
of
your
spitting
out
two
different
service
correct.
So
what
ends
up
happening
is
a
lot
of
us
have
a
lot
of
customers
that
we
can't
talk
about.
A
B
Runtime
class
is
like
a
CRI
implementation.
Yeah
I
think
that
any
time
where
you
have
a
composition
model,
that's
different
than
the
CRI
composition
model
or
the
composition
model
is
one
case.
So,
for
example,
if
you
want
to
run
virtual
machines,
they
don't
compose
the
same
way
that
particularly
composed
and
some
a
bit
darker
or
a
container
T,
and
in
the
other
cases
where
you
have
pods
that
might
not
really
be
VMs
or
containers
at
all.
For
example,
if
you
want
to
run
like
a
service
function
as
a
pod.
A
F
B
The
virtual
couplet
one
point
out
a
little
while
ago,
and
in
that
time
we
haven't
seen
any
breaking
API
changes,
but
we
would
have
to
go
ahead
and
mash
those
API
changes
from
the
provider
as
that
transition.
One
of
the
aspects
that
kinda
makes
us
more
complicated
is
that
not
all
people
who
are
creating
pods
for
consumption
by
the
virtual
crew
but
are
actually
conforming
to
the
way
that
kubernetes
behaves
itself
in
at
the
end
of
the
day
like
this
is
kind
of
the
problem.