►
From YouTube: Beyond Namespaces: Virtual Clusters are the Future of Multi-Tenancy - Lukas Gentele, Loft Labs
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Beyond Namespaces: Virtual Clusters are the Future of Multi-Tenancy - Lukas Gentele, Loft Labs
Multi-tenancy is a hot topic in the Kubernetes community. A lot of operators have started to think about lowering cost and consolidating workloads in large, multi-tenant clusters rather than creating 1000s of micro-managed, small clusters. Namespaces are a great way to separate tenants in shared clusters. But what if tenants need to install their own CRDs, run workloads across multiple namespaces or even require different versions of the Kubernetes API server? Virtual clusters are an exciting new approach that extends namespace-based multi-tenancy to address such advanced use cases. At its core, virtual clusters let you run Kubernetes clusters on top of other Kubernetes clusters by provisioning isolated Kubernetes control planes for each tenant (i.e. separate Kube API server, data store (etcd), controller manager etc). This talk will explain how virtual clusters work, show what implementations are available today, and demonstrate fascinating, real-world use cases for virtual clusters.
A
All
right
welcome
everyone
to
the
talk
beyond
namespaces
virtual
clusters
are
the
future
of
multi-tenancy,
I'm
lucas
and
I'm
the
ceo
of
loft
labs
in
I'm
based
in
san
francisco,
but
originally
from
germany.
So
you
know,
excuse
the
funny
accent,
together
with
my
team,
we're
working
on
some
really
exciting
things
in
the
kubernetes
space.
Our
commercial
product
loft
essentially
enables
large
companies.
A
Our
oldest
project
is
called
devspace,
it's
a
developer
tool
for
kubernetes
and
it's
essentially
a
replacement
for
docker
compose
that
is
designed
to
be.
You
know
for
kubernetes
that
you,
you
know,
can
you
know,
streamline
your
workflow
with
kubernetes
the
same
way
as
you
can
do
with
docker
compose
with
your
local
docker
daemon?
A
But
now
you
can
also
do
that
with
a
remote
kubernetes
cluster,
we're
also
working
on
kiosk,
which
is
a
multi-tenancy
extension
for
kubernetes
and
and
on
js
policy,
which
is
a
policy
engine
that
allows
you
to
write
policies
in
javascript
or
typescript,
making
it
much
easier.
You
know
to
write
these
policies
and
maintain
them
over
a
long
period
of
time.
A
Even
with
someone
you
know
in
the
company
leaves
that
you
know
was
the
ricoh
expert
writing
policies
beforehand
and
one
of
the
projects
that
we're
working
on
is
v-cluster
and
that's
what
we're
actually
going
to
talk
about
today.
It's
a
really
really
exciting
project
around
virtual
communities
clusters.
A
So,
let's
dive
in
beyond
namespaces
virtual
clusters
are
the
future
of
multi-tenancy
gosh.
That's
a
pretty
heavyweight
title
right.
I
don't
even
know
where
you
are.
You
know
where
you're
attending
this
session,
the
future
of
multi-tenancy
wow.
What
is
that
right?
Virtual
clusters
never
heard
of
that
you're,
probably
all
here,
because
the
first
part
right
beyond
namespaces
right.
You
know
namespaces
yeah.
I
got
this
right.
I
think
everybody
attending
kubecon
probably
knows
what
a
namespace
is.
But
do
you
really
know
what
a
namespace?
A
A
These
virtual
clusters
are
called
name
spaces,
so
namespaces
are
virtual
clusters.
Then
we
don't
even
need
this
talk
right,
it's
built
in
kubernetes,
you
all
know
about
it.
I
guess
we're
done
here
right.
If
you
ask
me,
this
is
pretty
outdated.
Namespaces
are
not
clusters,
so
they
are
not
virtual
clusters.
A
Namespaces
and
that's
also
part
of
the
documentation,
provide
a
scope
for
names
in
kubernetes,
right
for
your
pods,
etc.
Names
of
resources
need
to
be
unique
within
a
namespace,
but
not
unique
across
namespaces
right.
So
I
can
have
two
deployments
called
database
and
they
live
in
two
different
name
spaces
and
I
don't
have
any
naming
collisions
right.
That's
great!
That's
what
namespaces
do
and
that's
about
it
right.
They
don't
do
a
lot
more,
but,
as
we
all
know,
a
kubernetes
cluster
is
much
more
than
just
you
know,
naming
objects.
A
A
If
you
actually
want
to
check
out
which
resources
are
cluster-wide
and
which
ones
are
name-spaced,
you
can
actually
ask
qctl
right.
You
can
run
cubectl
api
resources
and
you
can
either
use
the
flag,
namespace,
true
or
namespace
fault,
to
get
the
respective
resources
and
kubernetes
will
tell
you
all
the
resources
in
your
cluster
that
are
available
hands
up
if
you're
a
kubernetes
contributor,
there's
a
couple
yeah
a
couple
more
so
this
slide
is
not
for
you
because
we
heard
earlier,
you
know
stephen
augustus
in
his
keynote
was
talking
about.
A
We
need
help
with
documentation
and
kubernetes
that
docs
page
that
I
was
showing
earlier.
In
particular,
you
know
that
sentence
about
virtual
clusters
is
over
five
years
old.
It
hasn't
been
touched
pretty
much
since
the
inception
of
kubernetes
right
and
we,
you
know
really
a
lot
of
people
need
to.
You
know,
get
involved
to.
Actually
you
know
maintain
this
vast
set
of
knowledge
that
we're
building
in
the
you
know
kubernetes
documentation.
A
So
after
this
talk,
everyone
should
know
the
differences
between
namespace
and
virtual
clusters
and
should
be
able
to
fix
this
and
whoever
is
a
first-time
contributor
to
the
documentation
fixes.
This
will
get
some
kind
of
a
little
surprise
present
for
me.
So
pay
close
attention
to
this
talk,
and,
hopefully
you
know,
there's
an
edit
page
button
directly
on
the
docs
page
right,
just
google
kubernetes
namespace
head
to
the
docs
open
that
pull
request
on
github.
A
Well,
let's
get
back
to
the
topic
multi-tenancy.
Why
are
you
even
here?
Why
should
you
care?
Well,
in
my
opinion,
spinning
up
thousands
of
single
tenant
kubernetes
clusters
is
a
really
terrible
idea,
because
it's
very
very
costly-
and
it's
very
very
hard
to
manage.
Just
imagine
you're
spinning
up
a
kubernetes
cluster
for
a
thousand
engineers
right
and
they're
spread
across,
like
you
know,
200
teams
you're
going
to
end
up
with
200
clusters
per
team
or
even
a
thousand
clusters.
A
A
That's
a
lot
of
work
and
it's
really
really
hard
to
keep
in
sync
and
what
happens
if
someone
breaks
a
cluster
throwing
away
a
cluster
recreating
that,
typically,
you
know
the
user.
The
actual
engineer
who
you
know
wants
to
just
connect
this
kubernetes
cluster
to
the
cicd
pipeline,
or
just
do
some.
You
know
pre-production
development
work.
There
doesn't
have
the
permission
to
destroy
a
cluster
and
recreate
it.
It's
pretty
tough.
A
So
what
about
namespaces
can't
we
use
multi-tenancy,
create
shared
clusters
and
essentially
separate
things
by
namespaces
right.
That's
the
alternative,
and
a
lot
of
people
are
doing
that
today.
If
we're
comparing
those
two
approaches,
we
see
that
you
know
name
spaces,
have
a
lot
of
advantages,
they're
much
cheaper.
It
is
much
easier
to
share
resources
like,
for
example,
you
know
shared
english
controller
for
all
our
tenants
in
this
shared
cluster,
but
a
lot
of
other
things
are
really
problematic.
A
A
We
need
to
really
lock
these
name
spaces
down,
so
our
tenants
can't
get
out
of
there
and
actually
you
know,
bring
the
cluster
down
or
harm
another
tenant
etc,
and
that's
a
lot
of
work
and
if
we're
doing
it
well,
then
we're
restricting
that
that
very
same
tenant.
That
now
cannot
do
certain
things.
Right
so,
let's
say:
there's
a
team
in
the
organization
that
needs
five
namespaces
and
needs
to
do.
You
know
things
across
five
namespaces,
but
you
set
up
network
policies
in
each
namespace
to
lock
the
traffic
down
to
that
particular
namespace.
A
A
That's
what
we're
trying
with
virtual
cluster
and
with
our
implementation.
V-Cluster
v-cluster
is
an
open
source
project.
You
can
access
it
on
vcluster.com.
You
can
find
it
on
github
and
it
tries
to
be
a
multi-tenancy
solution
for
kubernetes.
That
is,
you
know
beyond
namespaces.
It
is
more
extensive
than
what
a
namespace
offers,
but
at
the
same
time
it
is
as
lightweight
as
a
namespace.
A
A
There's
a
data
store
where
our
you
know
name
spaces,
pods
et
cetera,
are
stored
in
that's
typically
at
cd,
then
we
have
a
controller
manager
which
controls
our
replica
numbers.
If
I
create
a
deployment
with,
you
know,
five
replicas,
you
know
someone's
going
to
create
these
five
parts,
that's
what
the
controller
manager
does
and
then
there's
the
scheduler,
which
then
takes
these
parts
and
assigns
them
to
nodes
to.
Actually
you
know
where
the
people
it
starts,
these
containers
for
us,
and
then
we
have
the
workloads
on
top
of
these
control
planes.
A
And
typically
you
know
we
want
to
create
a
namespace.
We
want
to
create
some
pods
in
there.
We
want
to
create
a
couple
of
other
namespaces
and
then
what
we
always
do.
Is
we
talk
to
this
api
server
right?
That's
our
gateway
to
kubernetes.
That's
our!
You
know
single
source
of
proof
in
a
way
right
so
for
thinking
about
virtual
clusters
that
run
on
top
of
another
cluster,
and
we
have
the
capability
to
spin
up
workloads
inside
name
spaces.
A
Then
why
not
create
a
control
plane
that
runs
as
a
pod
inside
a
namespace
of
our
cluster?
That's
what
a
virtual
cluster
is.
That
means
we
have
an
api
server
and
potentially
a
couple
of
other
components,
and
it
really
depends
on
you
know
your.
You
know,
attitude
towards
virtual
clusters
and
your
you
know
how
you
technically
want
to
implement
this,
but
you're
certainly
going
to
need
some
kind
of
entry
point
that
you
talk
to
so
now
we're
not
talking
to
that
api
server
of
our
real
cluster
anymore.
A
We're
talking
to
this
api
server
up
here.
So
when
I
run
a
cube,
ctrl
command.
I
talk
to
a
pod
right.
That
runs
a
separate
api
server.
So
let's
take
a
look
at
how
this
actually
works.
I
recorded
this
demo
and
will
create
a
name
space
in
you
know
a
local,
let's
say:
mini
cube
cluster.
We
call
it
hostname
space
because
it's
supposed
to
host
our
virtual
cluster
and
then
we're
using
vcluster
cli,
which
is
a
very
lightweight
binary,
to
create
a
virtual
cluster.
A
We
call
it
vc1
and
we
tell
it
created
in
host
namespace
right
the
solar
name
space.
We
see
that
v-cluster
under
the
hood
executes
a
helm
command,
but
you
don't
even
need
that
you
could
just
run
cubesat
apply
or
create
a
customization
out
of
it
right,
but
for
ease
of
use.
You
know,
just
you
know,
running
helm
under
the
hood,
pulls
a
chart,
deploys
it
to
your
cluster
and
then
we
run
recluster,
connect,
vc1
and
specify
our
namespace
again.
A
And
what
this
does
and
you
see
that
here
in
the
output,
it
actually
starts
our
cube,
ctl
port
forward
to
now
connect
to
this
api
server
right
and
the
second
thing
that
it
does
and
that's
why
the
you
know
cli
is
pretty
handy
to
get
started
with
virtual
clusters
over
just
you
know,
using
cube
ctl.
It
also
creates
a
cube
config
file
in
our
current
working
directory.
A
A
We
can
also
list
parts
in
the
cube
system
name
space,
and
we
may
not
even
have
the
permission
to
do
that
in
the
underlying
cluster.
We're
not
admin
in
the
underlying
cluster.
We
just
have
access
to
a
namespace
and
now
we're
admin
in
this
virtual
cluster.
That
means
we
can
also
create
namespaces
in
that
virtual
cluster.
Solving
you
know
our
problem
with
network
policies.
A
I
can
now
do
things
across
10
namespaces,
although
I
just
have
access
to
one
namespace
in
the
underlying
cluster
and
of
course
I
can,
you
know,
schedule
workloads
and
that's
actually,
the
tricky
part
right.
Creating
a
namespace
is
just
an
entry
in
you
know
lcd
or
data
store,
but
what
happens
if
I,
you
know,
create
a
deployment
nginx.
In
this
case
I
set
the
replica
number,
the
two
and
I
schedule
that
in
our
newly
created
virtual
namespace
ns1,
if
we
list
the
parts
for
this
deployments
a
second
later,
we
see
that
they
actually
get
scheduled.
A
The
first
part
is
still
creating
and
the
second
one
is
already
running.
But
where
is
it
running
right?
That's
a
question
and
I'm
gonna
dive
a
little
bit
into
this.
We'll
see
you
know
both
of
them
are
running
now.
Our
virtual
cluster
seems
to
function
like
a
regular
communities
cluster
and
that's
really
the
goal
with
virtual
clusters.
A
So,
let's
recap
of
what
happened
here.
You
know
from
an
architecture
perspective.
We
created
this
hostname
space
in
our
regular
kubernetes
cluster,
and
then
we
deployed
a
virtual
cluster
with
the
v-cluster
cli
for
ease
of
use
right.
A
virtual
cluster
has
mainly
two
components:
it
has
a
stateful
set.
We
call
that
vc1,
just
like
the
virtual
cluster
we
created
and
it
has
a
service.
A
The
stateful
set,
essentially,
you
know,
creates
a
pod.
It
has
replica
number
one
in
this
case
and
inside
this
pod
is
really
where
our
virtual
cluster
lives.
Now
we
have
two
containers
in
this
virtual
cluster.
The
first
container
is
a
control
plane.
It
has
an
api
server,
as
we
talked
about
earlier.
That's
definitely.
What
we
need
to
you
know
do
any
kind
of
cube,
ctl
interaction
and
it
has
a
separate
controller
manager
and
it
has
a
separate
data
store
as
well.
A
So
it
has
a
separate
state
from
the
underlying
cluster
to
store
our
pods
namespaces
and
everything
we
do
inside
the
virtual
cluster
and
then
the
third
component
is
the
so-called
sinker,
and
that
is
what
is
replacing
the
scheduler.
You
may
have
noticed
you
know,
control
plane
is
incomplete.
The
schedule
is
missing.
A
Instead,
we're
going
to
have
the
syncer
and
we're
going
to
talk
about
this
in
a
second
next
part
of
the
demo
was
essentially
running
v-cluster
connect,
which
establishes
this
port
forwarding
connection
through
this
v-cluster
service,
so
that
we
can
now
talk
to
our
vcluster
api
server
and
this
api
server
is
ub.
Cluster
is
a
certified
kubernetes
distribution.
It
just
recently
became
a
certified
distro.
It's
a
very
new
project.
We
just
launched
it
in
q1
this
year
and
very
recently
became
certified.
That
means
this
api
server
that
we're
talking
to
is
a
hundred
percent.
A
You
know
compliant
with
all
the
you
know,
conformance
tests
that
we
have
in
kubernetes
and
it
behaves
one-to-one
like
a
regular
kubernetes
cluster.
You
won't
be
able
to
tell
the
difference
except
you
know
I
guess.
After
this
talk,
you
should
be
able,
you
know
to
know
some
tricks
on
how
to
find
out
if
it's
actually
a
virtual
cluster.
A
So
let's
get
rid
of
our
underlying
cluster
and
just
focus
on
the
virtual
cluster
for
a
second,
let's
get
rid
of
this
underlying
control
plane.
The
first
thing
I
ran
in
this
demo
was
creating
a
namespace
all
right
right.
That's
an
entry
in
our
data
store
by
default.
Vcluster
is
using
sqlite
because
it's
a
very,
very
lightweight
data
store
and
by
default,
we're
using
k3ss
api
server,
which
supports
sqlite
as
a
storage
backend,
but
you
can
also
use
you
know
postgres
or
even
lcd.
A
If
you
need,
you
know,
full-blown
virtual
cluster
and
you
know
we're
currently
working
with
k3s,
but
we're
also
working
on
you
know,
making
real
kubernetes
api
server
work
with
vcluster
as
well
is
one
of
our
next
steps,
so
we're
creating
a
namespace,
so
that
creates
an
entry
in
our
data
store
and
then
the
next
thing
we
saw
in
the
demo
was
creating
a
deployment
with
replica
number
two
right.
So
we
have
another
entry
in
our
data
store.
That's
it
right.
A
That's
all
we're
doing
with
the
api
server
and
then
the
controller
manager,
part
of
our
virtual
control
plane,
sees
that
new
entry
right
gets
notified.
It's
listening
on
these
objects
and
sees
okay.
Replica
number
was
two:
I
create
two
parts.
What
does
that
mean?
Well,
it
just
writes
two
more
objects
into
our
data
store,
so
we
have
four
objects
in
our
separate
virtual
cluster
state.
A
Now
we
have
a
namespace,
we
have
a
deployment
and
we
have
two
parts
that
you
know
are
owned
by
this
deployment,
and
you
know
in
the
beginning,
these
parts
are
just
you
know,
they're
waiting
to
be
scheduled
right,
they're,
nothing
more
than
just
an
entry.
In
a
data
store-
and
now
that's
part
of
the
scheduler
to
actually
schedule
these
pods
to
nodes,
but
in
a
virtual
cluster
instead
of
the
scheduler
with
the
syncer,
and
that's
actually,
what
makes
it
really
truly
virtual
what
this
thinker
does?
A
Is
it
copies
the
parts
from
within
that
data
store
that
we
have
inside
the
virtual
cluster
to
the
underlying
host
name
space
so
that
the
host
clusters
scheduler
can
now
schedule
these
parts
right?
That
means
all
the
restrictions
we
have
in
terms
of
network
policies.
What
we
have
in
terms
of
you
know
our
bank,
anything
we
have
in
terms
of
service
accounts
being
mounted
into
these
into
these.
You
know
parts
and
all
these
things
can
be
restricted
by
the
underlying
cluster.
You
can
have
regular
mission
control
right.
A
Everything
is
still
enforced,
but
only
at
the
pot
level
the
deployment
doesn't
exist
inside
the
underlying
cluster,
only
the
parts
and
because
we
can
obviously
create
multiple
namespaces
in
our
virtual
cluster.
You
know
what
about
naming
conflicts
right.
We
were
just
talking
about
that
earlier
namespace,
you
know
being
the
scope
for
names.
How
do
we
solve
that?
Well,
what
the
synchro
does,
when
it
sinks
from
multiple
name
spaces
down
to
a
single
namespace
in
the
in
the
host
cluster.
A
It
actually
renames
these
parts
and
it
adds
a
suffix
with
the
virtual
namespace
in
this
case
ns1
for
both
of
these
parts
and
even
with
the
name
of
the
virtual
cluster,
because
we
could
have
just
like
three
stateful
sets
with
a
virtual
cluster
inside
the
same
host
namespace
right.
So
we
need
those
two
suffixes
and
it's
really
important
to
understand
that
synchro
only
sings
a
minority
of
things
right
very,
very
few
things,
mainly
pods,
so
synchro
sings
only
low
level
resources
that
includes
pods,
plus
everything
that
parts
need
to
start
right.
A
A
We
added
that
because
there's
a
lot
of
need
for,
like
shared
ingress
controller
across
different
virtual
clusters
right
so
this
is
actually
by
default
on
in
vcluster,
but
you
can
disable
it
with
a
flag
pretty
easily
and
then
the
question
is
yeah.
What
are
the
nodes
inside
the
network
cluster
right?
What
if
I
run,
keep
cd?
I
get
nodes,
that's
actually
up
to
you
as
configurable,
because
synchro
syncs
in
both
directions.
A
It
doesn't
just
parts
down,
it
also
sinks
the
status
up.
That's
why
we
saw
earlier.
You
know
one
of
these
parts
was
container
creating
and
the
other
part
was
already
running
right
and
we
saw
that
inside
the
virtual
cluster,
although
that
state
actually
lives
in
the
underlying
cluster,
because
that's
where
these
parts
are
being
scheduled-
and
it's
really
important
to
understand
that
those
are
only
you
know
very,
very,
very
few
resources
in
our
communities,
cluster,
all
higher
level
objects
are
not
being
synchronized.
A
Only
very
basic
resources
are
being
synchronized,
and
high-level
resources
include
everything
that
is
replica,
controlled
right
deployments.
Stateful
sets
daemon
sets
all
these.
You
know
secrets
and
conflict
maps
which
have
nothing
to
do
with
pause
where
we
just
store.
Some
kind
of
state
service
accounts
jobs.
You
know
a
lot
of
these
things
are
not
synced
and,
most
importantly,
crds
and
the
custom
resources
that
we
create
from
them
and
that
actually
creates
a
really
interesting
possibility.
A
They
don't
need
any
crds
right.
They
may
need
to
add
an
ingress
controller
there
and
some
basic
monitoring
for
these
parts
that
we're
scheduling
and
obviously
some
network
policies
right,
but
we
don't
need
anything
specific
to
applications.
We
are
not
going
to
run
into
any
conflicts
with
crds
either
right.
Two
teams
can
have
two
versions
of
crds
and
run
side
by
side
in
the
same
host
cluster,
but
within
two
virtual
clusters.
Very
very
interesting.
So
the
vast
majority
of
resources
exists
only
in
the
scope
of
our
virtual
cluster.
A
So
what
about
part
networking
and
dns?
Well
part-to-part
communication
is
given
right.
Parts
are
actually
running
in
the
underlying
cluster,
so
they
can
communicate
vip,
just
as
they
can
in
a
normal
cluster
pod
to
service
communication.
As
we
heard
earlier,
we
by
default
sync
services,
and
we
do
you-
know
wire
them
up
to
the
actual
right
parts
right,
so
that
works
out
of
the
box
as
well,
because
the
sinker
takes
care
of
this
and
what
about
cluster
or
v-cluster
internal
dns?
A
Well,
as
you
may
have
seen
earlier,
when
we
listed
the
cube
system,
the
parts
in
the
cube
system
name
space
within
our
v
cluster.
There
was
one
part
already
running,
and
that
was
core
dns,
so
v-cluster
by
default
has
in
its
cube
system
name
space,
accordions
deployment
and
that
one
creates
a
part
that
part
is
being
synced
down
to
the
host
cluster
right.
That
means
we
have
a
part
and
also
a
service
for
dns
down
there
in
our
host
name
space,
which
now
is
managing
our
dns.
A
That
means
when
the
syncer
synchronizes
down
to
the
host
cluster,
the
only
thing
we
need
to
do
is
modify
this
the
part
spec
and
point
it
to
that
dns
service
instead
of
the
regular.
You
know,
cluster
wide
dns
service,
and
then
you
know,
dns
based
communication
and
resolution
name.
Resolution
works
as
well,
and
then,
as
I
mentioned
earlier,
you
know.
Ingress's
sync
is
pretty
interesting
as
well
to
even
you
know,
get
more
capabilities
out
of
this.
A
A
There
are
multiple
options
here,
the
first
option:
that's
the
default
option
is
fake
nodes
and
they're
dynamic.
That
means
what
we
do
is
we
let
the
schedule
of
the
underlying
cluster
schedule
our
parts
and
then
we
see
which
nodes
do
they
end
up
on
and
those
nodes?
We
add
in
our
data
store
because
a
node
is
just
an
entry
in
our
data
store
right
inside
the
virtual
cluster,
and
we
tell
this
virtual
cluster.
This
node
exists
and
then
we
fake
it
right
in
a
way
that
we
rewrite
the
name
we
rewrite.
A
A
A
A
What
else
is
possible
virtual
clusters?
It's
very,
very
exciting,
because
there's
a
lot
of
you
know
things
that
you
can
do
with
it.
One
thing
you
can
do
is
also
run
cube
ctl
inside
these
parts,
which
is
very
exciting
because
when
you're
thinking
about
it,
this
is
a
part
that
actually
runs
in
the
host
cluster
right.
So
if
I
run
cube
ct
out
there,
but
I
started
this
part
from
within
the
virtual
cluster,
so
it'd
be
odd.
A
If
I
run
cubesat-
and
I
talked
to
you
know-
with
a
service
account
being
mounted,
I
talked
to
the
host
cluster.
So
that's
why
we
actually
changed
this
environment
variable
in
kubernetes,
so
that
when
you
use
the
in-cluster
config
for
cube
ctl,
it
talks
to
the
virtual
clusters
api
server.
Instead
of
to
the
real
you
know
underlying
host
api
server.
So
we
get
the
true.
You
know
cluster
experience,
even
when
running
cubes
detail
inside
a
pod,
we
can
expose
virtual
cluster
api
servers.
A
It's
just
a
service
in
kubernetes
right,
so
we
can
add
a
load
balancer
to
it
type
load
balancer
or
we
can
put
an
ingress
controller
in
front
of
it
and
make
it
available
to
outside
of
the
cluster,
not
just
via
port
forwarding,
there's
a
couple
of
things
upcoming,
which
are
really
exciting
as
well.
Making
network
policies
work.
A
You
know
if
you're,
creating
a
network
policy
inside
of
your
virtual
namespaces
and
that
has
a
namespace
selector,
that's
a
little
tricky
right
because
you
may
not
have
the
same
namespace
in
the
underlying
cluster,
but
we
can
also
do
that
with
you
know,
changing
things
during
the
sync
process
and
we're
actually
working
on
that
right
now.
You
know
a
couple
of
weeks
away
from
making
this
work
pot
disruption.
Budgets
is
another
hot
topic
that
we're
working
on
and
non-root
bee
clusters.
A
Right
now,
k3s
runs
as
rude
and
they
have
the
option.
I
think
they
recently
released
the
option
to
run
k3s
as
non-root
and
that
will
be
possible
with
v-cluster
as
well.
It's
one
of
the
you
know
most
important
steps
in
making
v-cluster
very,
very
resilient
and
isolated,
and
a
nested
virtual
cluster
is
something
very
interesting
right.
You
could
now
talk
to
this
virtual
api
server
and
run
vcluster,
create
again
right,
create
a
stateful
set.
They
create
a
service
and
connect
to
this
virtual
right.
You
can
see
it.
A
You
can
go
pretty
crazy
about
this
and
rich
boris,
which
works
with
me
at
loft.
Labs
he's
also
attending
this
talk.
If
you
want
to
chat
with
him
later,
he
made
a
pretty
great
video
about.
You
know
it's
called
v
cluster
inception
nesting
virtual
clusters.
You
should
check
definitely
check
it
out.
There
are
a
lot
of
use
cases
for
virtual
clusters.
You
can
create
them
for
ephemerals
the
icd
environments.
Any
kind
of
you
know
integration,
end-to-end
acceptance
tests.
A
You
can
spin
up
preview
environments
per
pull
requests
and
get
an
entire
kubernetes
cluster.
You
can
even
have
different
versions
of
kubernetes
being
tested
in
ucicd
now
on
the
same
kubernetes
cluster
right,
because
that
api
server
you
can
enable
alpha
and
betaflex,
is
independent
of
your
underlying
cluster.
The
only
thing
your
underlying
cluster
needs
to
do
is
you
know,
creating
pods,
creating
services
right,
very
basic
things
right.
A
I
guess
if
the
pod
spec
changes
with
a
future
version
right
there,
maybe
some
income
compatibilities,
but
you
know
that
the
pot
spec
is
having
a
breaking
change
in
the
future.
You
know
that
probably
we
vape
more
problematic
on
other
edges
than
virtual
clusters.
You
can
spin
up
these
remote
development
environments.
Let
your
engineers
directly
work
with
kubernetes,
so,
instead
of
having
them
maintain
their
pet
mini
cube
clusters
or
docker
desktop,
have
them
just
spin
up
virtual
clusters
right
their
throwaway.
They
can
create
one.
A
They
create
another
one
for
different
project
right,
that's
actually
what
we
do
with
our
commercial
project,
so
with
loft
engineers
are
exactly
doing
that
right.
They're,
spinning
up
these
virtual
clusters
and
name
spaces
before
we
open
sourced
vcluster
virtual
cluster
was
for
about
six
months
already
part
of
our
commercial
offering.
A
So
we
already
have
companies
working
with
this
technology
and
essentially
providing
these
deaf
environments
to
the
engineers
and
dropping
the
local
host
environment
entirely,
and
what
we
do
as
well.
Is
we
put
those
environments
to
sleep
when
engineers
are
not
working
with
them
right
so
automatically
pausing
these
virtual
clusters,
purging
all
the
workloads
and
automatically
spinning
them
up
again
when
new
cube,
ctl
requests
come
in
so
that
you
know
when
your
engineers
leave
the
office
at
8
pm.
A
Everything
turns
off
30
minutes
later,
because
they're
not
working
again
or
they're,
shifting
between
their
virtual
clusters,
but
they're
only
really
working
with
one
at
a
time,
and
we
really
want
to
make
you
know
virtual
clusters,
cattle
in
a
way
and
make
them
disposable
right,
and
you
want
the
engineer
not
to
have
to
treat
their.
You
know
pet
localhost
cluster
as
they
do
it
today
we
see
a
lot
of
experimentation.
You
know
where
you
really
need
a
lot
of
compute
workload,
m-I-l-I
mlai
workloads.
A
We
see
cluster
simulations
as
well.
It's
a
very
interesting
concept,
adding
fake
notes
to
the
cluster
and
things
like
that
right.
There
are,
you
know,
so
many
things
that
you
can
do
like
have
like
10
virtual
nodes
mapped
to
the
same
underlying
physical
node
right.
All
of
these
things
are
possible
even
virtual
clusters,
spanning
across
different
kubernetes
clusters,
right
there's,
so
many
creative
ideas
of
what
you
can
do
with
this,
and
you
can
really
make
multi-tenancy
more
resilient
wink
ring.
A
You
know
where
that's
kind
of
like
the
name
of
the
conference,
and
you
can
do
things
like
sales,
demos
and
higher
level
things
as
well
with
these
virtual
clusters
so
how
to
get
started.
I
hope
you
all,
you
know,
got
excited
about
virtual
clusters
and
we
try
to
make
it
as
easy
as
possible
get
started
with
them.
You
can
essentially
just
download
the
you
know,
vcluster
cli
and
run
vcluster
create,
and
we
cluster
connect
in
any
pretty
much
any
kubernetes
cluster.
A
You
can
head
to
the
documentation
for
more
detailed,
getting
started.
Guide
source
code
is
available
on
github.
If
you
want
to
dive
a
little
deeper.
If
you
want
to
open
issues,
we
also
have
a
vcluster
website
with
some
information
you
can
follow
loftlabs
on
twitter.
We
regularly,
you
know,
post
blog
posts
about
virtualcluster
and
you
know
shoot
them
out
on
twitter
and
definitely,
if
you
have
any
questions,
reach
out
right,
that's
why
we're
all
going
to
a
conference.
We
want
this
interaction,
and
I'm
so
glad
that
we
have.
A
You
know
this,
this
personal,
hybrid
format
again
right
where
we
can
really
connect
to
each
other.
So
if
you
have
any
questions,
either
ask
now
or
join
us
on
slack
reach
out
via
twitter,
come
by
the
booth,
I'm
gonna
be
at
the
booth
right
after
this
talk
we're
when
you're
entering
the
exhibit
hall
to
the
right
close
to
the
project
maintainer
area,
both
numbers
su-49.
A
We
have
a
couple
of
pretty
nice
swag
and
I
specially
want
to
point
out.
We
have
cubecompose
cards,
they
look
like
this
and
you
know:
they're
really
meant
to
be
sent
out
to
your
colleagues
who
cannot
be
here
in
person,
so
feel
free
to
grab
one
and
just
you
know
send
it
out
to
whoever
cannot
be
here
in
person
and
since
we're
a
startup
and
we're
working
on
really
exciting
technology
and
a
lot
of
different.
A
You
know
open
source
projects
as
you've
seen
on
the
first
slide,
we're
also
hiring
so
if
you
want
to
work
with
my
really
brilliant
cto,
definitely
check
out
our
careers
page
or
you
know,
approach
any
member
of
our
team
reach
out
to
us,
and
you
know
see
if
we
can
work
together
and
I
think
we
have
about
like
two
minutes
left
for
questions.
So
if
you
have
any
questions
right
now,
just
raise
your
hand
happy
to
answer
them.