►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Deep Dive: Kubernetes SIG Multicluster - Quinton Hoole & Irfan Ur Rehman, Huawei
Multi-cluster Federation Deep Dive: Multi-cluster Federation is an API and control plane that actively manages multiple Kubernetes clusters and applications in those clusters. Federation v1's API mirrored the Kubernetes API directly, using annotations to achieve necessary multi-cluster extensions. Federation v2.0, now in Alpha, has it's own Multi-cluster aware API, and a more and a more customizable control plane composed of lower level re-usable building blocks. In this deep dive, we will discuss the existing API and control plane architecture, and it's primary target use cases. But most of all we would like to solicit input on what other use potential cases exist, and what additions or amendments might be required to make Multicluster Federation even more generally useful for your real-world stuff.
To learn more click here: https://sched.co/H1Vv
A
Right
good
afternoon,
everyone,
the
last
day
of
cube
con
China,
you're
fun.
My
colleague
and
I,
are
gonna.
Give
you
a
brief
introduction
to
who
the
multi
cluster
special
interest
group
in
the
kubernetes
world
is
and
and
then
spend
most
of
the
time
doing
a
deep
dive
into
the
Federation
system
that
has
been
built.
You
can
have
a
look
at
the
API,
as
you
can
have
a
look
at
the
architecture
and
how
you
can
use
it
for
various
different
things
and
then
we'll
have
a
Q&A
at
the
end.
A
So
this
is
roughly
what
we'll
cover
I.
Don't
think
I
need
to
go
into
too
much
more
detail
there.
Just
very
briefly,
the
the
mission
of
the
multi
cluster
special
interest
group
is
to
help
solve
all
of
the
common
challenges
that
relate
to
the
management
of
multiple
clusters
and
and
also
the
applications
that
live
in
them.
So
there's
a
bunch
of
things
we
do
api's
tools,
documentation,
etc.
A
One
of
our
big
projects
is
cluster
Federation,
which
is
an
entire
control
plane,
similar
in
size
and
scope
to
kubernetes
itself,
but
to
manage
multiple
clusters
instead
of
multiple
nodes.
But
then
we
also
have
some
other
sub
projects
and
we
anticipate
in
the
future
to
deal
with
several
other
things
related
to
mult
managing
multiple
clusters.
A
A
One
of
the
questions
that
has
comes
up
fairly
frequently
is
well.
Why
do
people
run
multiple
clusters
and
you
know
to
some
people?
That's
fairly
obvious
to
others,
lesser
and
in
fact
there
are
numerous
different
reasons
why
you
might
want
to
run
multiple
clusters,
the
obvious
one,
that
a
lot
of
people
are
aware
of
is
capacity
overflow,
so
cloud
bursting
is
another
name
that
goes
by.
So
if
you
have
an
on-premise
cluster,
the
classic
example,
you
ran
out
of
on-premise
capacity,
and
you
have
a
bunch
of
users
who
arrive
one
day.
A
How
do
you,
you
know,
make
use
of
public
cloud
automatically
and
overflow
your
capacity
into
the
public
cloud?
There
are
other
areas,
though
you
might
have.
You
might
want
to
use
public
cloud,
but
there
might
be
some
data
protection
reasons
or
other
reasons
why
you
have
certain
sensitive
workloads
that
need
to
run
in
one
place
and
then
other
workloads
that
need
to
run
in
another
place.
And
how
do
you
manage
that
dynamically
another
common
reason
why
people
are
interested
in
you
know
hybrid
or
multi
cloud
environments.
A
Multi
cluster
environments
is
to
avoid
you
know,
locking
into
a
single
vendor
as
much
as
some
of
the
particularly
the
public
cloud.
Vendors
have
very
good
offerings.
You
know.
Big
companies
in
particular
feel
very
hesitant
about
you
know
putting
everything
in
one
public
cloud
provider
and
never
being
able
to
move
out
again.
So
that's
another
common
reason
and
you
know
a
sub-sub
section
of
that
whole
area.
Is
you
know
if
you
have
very
big
companies
with
mergers
and
acquisitions?
A
Often
you
know
many
companies
merge
together
and
this
company
chose
vendor
a
or
public
cloud
a
and
this
other
company
Chad
chose
a
public
lobby
and
now
when
they
merge
together
that
they
have
different
clusters
in
different
public
cloud
providers
that
need
to
be
managed.
And
then
you
know,
arguably
one
of
the
strongest
motivations
is
high
availability.
So
if
you
do
have
an
entire
data
center
outage-
and
these
things
happen-
you
know
not
every
day,
but
they
they
happen,
often
enough
that
you
need
to
be
able
to
handle
them.
A
A
There's
a
scalability
aspect.
You
know
kubernetes
clusters
don't
scale
infinitely.
So
if
you
need
to
run
hundreds
and
hundreds
of
thousands
of
nodes,
you
probably
don't
want
to
build
a
cluster
that
big
you
probably
want
to
build.
You
know
ten
clusters
of
thousands
of
nodes
each
rather
than
hundred
thousand
node
clusters.
A
lot
of
the
impetus
behind
this
came
from
my
work
at
Google
running.
You
know
very
large
systems
that
ran
across
many
tens
of
clusters
and
some
of
the
lessons
that
we
learned
in
the
problems
with
that
and
how
do
how
to
do?
A
We
had
two
versions
of
the
Federation
concept,
so
what
we've
now
called
v1
was
mostly
a
proof-of-concept
exercise,
taking
the
kubernetes
api
and
trying
to
see
how
far
we
could
go
by
using
that
to
deploy
things
across
multiple
clusters.
So
you
can
imagine
if
you
have
a
deployment
or
a
replica
set
with
multiple
replicas
inside
it,
and
you
want
to
make
sure
that
that
gets
spread
across
three
different
clusters
so
that
if
any
one
goes
down,
you
only
lose.
You
know
one-third
of
your
capacity.
And
how
do
you
do
that
I
mean?
A
Obviously
you
can
do
that
manually.
So,
if
you
have
you
know
thirty
replicas,
you
could
say
right
manually,
put
ten
in
this
cluster
ten
in
that
cluster,
ten
in
another
cluster,
because
that's
how
many
you
know
how
much
capacity
I
need,
but
then
one
of
them
goes
away.
So
now
you've
only
got
20
replicas
left,
you
know,
do
you
have
to
wake
up
in
the
middle
of
the
night
and
going
to
add
five
to
each
of
the
other
clusters?
A
So
you
still
have
30
replicas
or
does
software
just
do
that
automatically
for
you,
and
so
you
know
the
latter
is
the
better
way
to
do
it.
We
then
moved
over.
So
we
we
sort
of
took
that
we
took
most
of
the
kubernetes
api
s
and
we
implemented
them
a
multi
cluster
variance
of
them
by
adding
annotations
to
the
kubernetes
api
to
say
you
know
which
clusters
do
you
want
to
go
this?
What
want
this,
whatever
it
is
resource
to
be
replicated
into?
What
are
the
relative
weights?
A
The
other
aspect
of
Federation
v2,
you
know
the
first
one-
was
kind
of
a
pretty
quickly
hacked
together
proof
of
concept,
but
it
wasn't
built
by
very
strong
software
engineering
standard,
so
it
was
fairly
monolithic.
We
had
a
big
controller
for
replica
sets
that
knew
how
to
spread
them
across
clusters.
We
had
another
controller,
for
you
know
other
things,
daemon
sets
and
stateful
sets,
etc.
But
we
decided
to
break
this
into
more
reusable
components
that
that
you
can,
then
you
know
plug
together
for
different
use
cases,
and
so
it's
based
on
CR
DS.
A
It
allows
Federation
of
any
kind
of
resource,
so
you
don't
actually
have
to
write
custom
code
per
resource
anymore
in
Federation
v2.
You
can
federate
your
own
CR
DS.
So
if
you
define
your
own
custom
resources
in
kubernetes
land,
you
can
you
can
just
configuration
to
the
Federation
control
plane
that
will
then
be
able
to
federate
those
across
multiple
clusters
and
I.
Think
around
about
this
point.
I'm
gonna
hand
over
to
my
esteemed
colleague,
Irfan
who's
been
spearheading
the
Federation
version,
2
implementation
and
he's
gonna.
Take
you
into
some
of
the
details
of
the
API.
B
C
C
Let's
come
back
to
it
later,
so
I
just
showed
your
games
to
it
and,
let's
come
back
to
it
later,
let
me
break
it
down
a
little.
So
Quinton
did
talk
about
that.
We
in
Federation
v2.
We
have
broken
down
the
API,
so,
for
example,
you
are
trying
to
federate
a
secret
I'm,
taking
an
example
of
a
simple
resource
which
is
secret.
It
does
not
does
not
have
many
fields
and
all
that
stuff
so
to
federate
a
secret.
What
does
it
take
so
I'm
showing
some
mammals
to
you?
C
So
if
you
see
here
at
the
bottom,
there
is
a
target
which
say
sky
in
and
version,
so
it
also
the
admin
can
also
specify
what
is
the
version
to
be
used
when
creating
or
updating
resources
into
the
particular
clusters,
so
that
is
specified
by
the
target,
which
is
a
roughly
the
sixth
line
from
the
bottom.
Then
couple
of
other
things,
because
we
have
broken
down
the
single
federated
resource
into
multiple
types.
C
So
you
have
a
placement
you
have
overrides
and
a
template
template
is
basically
nothing
but
the
target
specification
and
one
more
field
which
I
have
marked
as
blue,
is
of
particular
interest,
which
is
propagation,
so
admin
can
also
define
using
this
type
or
using
this
definition
that,
for
this
particular
target
type
propagation
should
be
enabled
or
disabled
propagation.
True
means.
The
sync
controller,
which
is
the
controller
associated
with
this.
Will
if
federated
type
secret
is
created
in
Federation
control
plane,
then
it
will
be
propagated
to
individual
clusters.
C
If
it
is
false,
it
will
not
be
so
it
will
just
be
defaulted
now,
let's
see
so.
We
can
also
mention
that
we
have
come
to
a
place
where
all
of
our
implementation
is
CRD
based.
So
when
we
talk
about
say
federated
type,
it
is
a
cre
installed.
So
this
is
so
cid
enables
you
to
define
your
types
and
the
spec
for
that
type.
So
I
the
spec
or
the
field
specification,
is
validated
using
a
validation
schema,
which
I
have
skipped
from
here,
because
it's
a
long
list.
C
I
have
tried
to
just
give
you
some
names
of
what
the
types
are.
So
the
three
core
of
primitive
types
are.
This
one
is
particularly
the
template
so
for
a
secret,
our
user
would
need
to
create
a
federated
secret,
which
is
which
will
wrap
around
a
secret
definition.
So
in
in
k
test,
you
would
have
a
sacred
secret
definition,
which
will
have
fields
like
say,
data
or
type
of
that
data
and
couple
of
other
fields,
so
that
will
be
part
of
the
template.
It
will
it
it's
the
same
type
which
is
used
in
caters.
C
C
Now
that
was
the
CID
definition.
This
is
actually
the
resource.
So
if
you
the,
for
example,
a
template,
I
am
not
showing
you
override
not
to
overwhelm
you
right
now,
so
this
is,
you
saw.
This
is
the
CID
definition,
and
this
is
what
actually
the
resource
will
look
like.
So
it
it
is
a
federated
secret
and
it
has
a
name.
I
have
given
it
a
test
secret
and
template
is
what
it
is
exactly
the
same
data
which
actually
acade
a
secret
has
placement
is
nothing
cryptic.
C
C
C
So
you
saw
here
this
is
this
is
the
template
which
a
user
is
going
to
create
in
federal
Federation
control,
plane
and
the
specified
data
over
here
is
a
some
some
token
and
the
type
is
opaque,
but
user
also
wants
that
that
same
data
should
not
be
propagated
to
cluster
two.
So
only
one
field,
which
is
data,
should
be
overridden
in
cluster
two.
So
when
a
secret
is
created
in
cluster
two,
the
field
data
will
be
populated
with
nil.
C
It
wouldn't
have
any
value,
so
what
it
is
giving
you
is,
it
is
giving
you
the
most
basic
level
primitives
using
which
you
can
create
the
resources
in
individual
clusters
with
the
values
you
want
for
those
resources
and
I
am
showing
this
with
an
example
of
a
secret.
But
it
is
possible
to
do
this
exactly
same
things
with
even
the
CRD
types
which
are
probably
the
most
advanced
or
the
more
common
or
most
complex
things
or
say
a
deployment
type
or
a
replica
set
type.
C
Now,
coming
back
to
that
same
diagram,
so
I
have
taken
only
this
much
consider
this
as
the
basic
primitives
layer,
which
has
a
template,
placement
and
override,
and
there
are
three
configurations
generally
done
by
admin.
So
there
is
a
cluster
configuration,
so
there
is
a
tool
which
we
have
written,
which
is
called
view
fed
to
and
join.
C
You
can
join
a
cluster
to
this
Federation.
After
you
have
deployed
the
Federation,
you
can
say
queue
fed
to
join
cluster,
one,
which
is
probably
the
name
of
the
context
which
you
have
in
your
queue
config
file
for
that
cluster.
So
then,
once
you
do
this,
the
Federation
knows
about
that
cluster.
You
can
join
as
many
clusters
as
you
want.
They
could
be
in
different
clouds.
They
could
be
on
Prem
or
given
that
the
clusters
are
accessible
on
the
network,
reachable
to
those
questions.
C
So
this
is
done
by
admin.
Then
the
federated
type
configure
a
source
is
what
also
created
by
the
admin,
and
it
defines
what
are
the
associated
type
for
say,
a
target
secret
in
a
particular
cluster
which
reserved
which
reside
in
the
federation
control
plane.
The
placement,
the
override
and
the
template,
and
then
there
is
an
Associated
controller.
That
controller
would
once
these
are
configured.
The
controller
would
know
about
the
clusters.
Controller
will
would
know
about
okay,
this
for
this
particular
type.
C
So
now
coming
back
to
that
full
diagram,
so
you
have
the
basic
types
and
using
which
you
get
all
the
primitives
which
you
can
use
to
create
specific
resources
or
with
over
aidable
types
or
with
different
fields
in
individual
clusters.
The
most
simple
type
would
be
that
you
just
create
a
template
and
there
is
no
placement
and
there
is
no
override,
so
the
same
resource
will
be
just
propagated
to
all
the
questions.
C
There
is
a
possibility
of
writing
higher-level,
more
complex
schedulers
or
more
complex
controllers.
Using
those
same
primitives
we
have
implemented
some.
We
have
implemented
cross
cluster
service
discovery
which
uses
similar
primitives
and
enables
the
discovery
of
services
across
the
questions,
and
we
have
implemented
some
scheduling
logic
also
using
which
replica
based
workloads
like
the
deployment
and
replica
sets,
can
be
shed.
C
You'll
de
cross
clusters
such
that
you,
the
user,
defines
the
total
replicas
which
needs
to
be
distributed
in
the
clusters
and
in
case
of
failures
or
problems
of
clustering
pleasure
going
out
and
coming
in
the
workload
total
replicas
are
still
maintained
across
the
questions
and
the
same
exactly
same.
Things
are
possible
for
CRT
types.
Also.
So
if
the
one
of
the
main
reasons
we
decompose
the
API
into
different
types,
was
this
to
enable
users
to
compose
functionality
using
some
primitives
and
writing
their
own
controllers.
If
there
are,
there
is
need.
C
C
Then
the
first
thing
I
told
you
about
the
type
config
which
is
defined
by
an
admin,
so
a
type
config
controller,
with
basically
watches
for
that
particular
type.
And
if
it
sees
that
there
is
a
type
config
created,
then
it
spawns
controllers,
which
we
call
the
which
I
mentioned
a
sink
controller.
So
it
spawns
the
same
sink
controller,
but
with
that
target
type.
So
the
code
base
is
all
same
for
syncing,
the
primitives,
it's
one
controller,
but
it
is
configured
with
all
the
different
target
types,
so
it
could
be
configured
with
a
CRT
type.
C
Also
it
could
be
configured
with
a
simple
type
which
is
already
existing
in
kaydessa.
So
as
of
now,
which
is
the
secret
and
that
controller
basically
starts
sync
loop,
which
looks
for
the
associated
federated
types,
which
will
be
the
Federated
secret,
federated
placement
and
federated
overrides
same
way
if
it
sees
I
mean
this
is
just
for
illustration.
If
it
sees
that
there
is
a
federated
type
conf
created
for
services,
it
will
start
the
same
same
controller
for
services.
C
In
case
the
type
config
says:
propagation
disabled
well,
I
showed
one
field
which
was
propagation
as
enabled
true
or
false.
If
it
sees
a
propagation
delay,
will
disabled
all
the
type
config
to
remove,
then
it
can
basically,
so
the
this
can
be
done
for
all
the
types
at
the
dimension,
so
different
type,
CRE
types
all
that
later,
if
it
sees
that
there
is
the
type
1
fig
delete
it
or
disable
it'll,
just
stop
the
sync
controller
for
that
particular
type,
and
that's
how
this
controller
logic
works.
C
Similarly,
the
scheduler
controllers
are
also
read
written,
so
there
is
a
scheduler
manager
and
different
schedulers.
The
scheduler
manager
watches
for
the
same
target
types,
so
the
scheduler
basically
works
on
a
target
type,
so
scheduler
can
be
associated
with
a
deployment
or
a
replica
set
which
needs
some
kind
of
scheduling
all
say
jobs
which
needs
some
kind
of
scheduling.
A
simple
type
like
secret
does
not
need
any
scheduling,
so
associated
scheduling
controller
would
be
started
if
the
scheduler
manager
sees
that
that
target
type
is
there
and
same
way.
C
C
This
is
about
the
grouping
of
the
API.
So
how
we
are
grouping
the
Federation
API
is
those
primitives.
What
we
call
the
three
basic
types
we
are
calling
them
primitives,
or
we
also
call
them
core
earlier
and
the
type
config
which
basically
defines
what
are
the
associations
between
different
types
and
there
is
a
propagated
version.
C
So
the
sync
controller
basically
uses
the
propagated
version
to
differentiate
between
what
is
the
resource
which
is
in
the
cluster
and
what
was
the
desired
resource,
which
is
in
the
Federation
control
plane
this
this
we
have
used
differently,
because
yesterday
somebody
asked
a
question
about
versions,
so
Federation
control,
plane,
mites,
might
have
a
template
versions.
Say
secrets
be
one
of
even
secrets,
whereas
the
target
cluster
can
handle
only
say,
v1
beta-1
secrets,
so,
but
the
API
when
it
is
submitted,
it
is
backward
compatible.
C
C
Scheduling
resources
are
our
scheduling,
types
are
put
into
scheduling
group
and
we
have
the
service
or
the
networking
types
we
have
put
them
in
multicast
DNS.
Each
of
them
have
a
much
more
complex
or
deeper.
You
know
explanation
which
can
be
given,
but
given
the
time
limitations,
I
can't
go
and
deeper
into
those
things.
C
Okay,
so
there
is
one
more
point
that
for
a
for
a
target
type,
we
are
creating
different
resources,
they
all
are
conveying
one
or
they
are
all,
are
converging
into
a
creation
of
one
resource
of
one
particular
type.
How
do
we
map
them
so
each
type
and
quotes
the
targets
name?
So
if
you
say
I'm
creating
a
taking
an
example
of
a
replica
site,
I
am
I
created
a
federated
replica
set
I
created
a
placement
for
that
I
created
an
override
for
that.
All
of
them
would
be
named
the
same.
C
So
that's
how
the
linking
happens
and
the
target
in
the
target
clusters
would
also
be
named
the
same.
So
my
RS,
if
that
is
what
is
the
name
use
the
target
cluster,
will
get
the
replica
set
with
that
name.
Scheduling
is
the
same.
So
if
a
scheduler
is
enabled
for
replica
sets
with
the
name
IRS,
then
the
scheduling
intent,
which
is
specified
in
the
replica
scheduling
preferences,
which
is
probably
the
total
number
of
replicas
and
the
weights
which
it
needs
to
give
to
individual
clusters,
would
also
be
named.
The
same
same
is
with
services.
C
C
When
you
want
to
install
it
as
a
CR
D.
Now
we
are
trying
to
even
move
one
step
further,
that
we
don't
have
to
write
any
code
for
that.
So,
if
say
you,
as
a
user
want
to
federate
a
particular
type,
a
cid
type
or
a
type
which
is
not
already
the
port
for
which
is
already
not
already
there.
In
today's
federation,
v2
repo,
you
can
just
simply
use
the
tool
and
it
will
auto
generate
the
cid
definitions
and
it
will
install
that
on
to
your
target
API
server,
so
queue
fit
federate.
C
Enable
HPA
is
that
that
command,
so
it
will
create
the
different
types
for
or
the
type
definitions
for,
the
associated
types
that
might
be
there.
So
HPA
I
chose,
because
that
is
something
which
is
not
there
in
the
Federation's
current
repo.
The
code
for
the
API
logic
is
not
there,
so
it
is
as
simple
as
Auto
generation
of
that
definitions
is
as
simple
as
federate
enable
HPA,
which
is
a
type
you
can
as
well
right,
horizontal
pod,
auto
scalars,
all
the
short
name
or
the
plural
name.
C
It
will
take
any
of
that
and
disable
is
as
simple
as
that.
Disable
only
disable
the
propagation
it
will
just
turn
the
propagation
to
false.
But
if
you
want
to
remove
the
CID
definitions
or
then
stall
definitions
from
the
API
server,
you
can
as
well
use
delete
from
API,
so
I'll
just
show
you
that
how
that
works.
C
C
C
So
you
see
a
horizontal
part:
autoscaler
is
available
from
the
existing
API
resource,
which
is,
in
the
cases
API.
What
we
are
looking
for
is
the
definition
of
the
same
to
be
installed,
which
is
the
Federated
counterpart
of
this
target
type,
which
will
be
the
Federated
horizontal
part
of
the
school
and
scalar
and
the
Federated
horizontal
part,
autoscaler
placement
and
overrides.
C
C
Hope
it's
legible,
so
you
see
it
is
it
created
type
config
which
I
showed
the
amyl
for
which
is
the
type
configuration
by
the
admin?
So
this
also
is
something
the
admin
I
really
would
do.
A
user
would
just
use
the
control
plane
to
create
his
resources,
so
there
is
a
type
config
created
and
then
it
will
tell
the
associated
target
and
the
kind
is
this
and
it
just
used
the
highest
available
version
from
the
target
API
server.
C
Then,
if
you
come
down,
it
has
created
the
definitions
for
the
associated
type,
so
all
of
those
are
CR
DS
and
if
you
see
the
type
of
this
particular
CI,
DS
federated
horizontal
part
autoscaler,
which
just
wraps
around
the
target
template.
So
when
a
user
creates
a
federated
horizontal
part
or
the
scalar
he'll
actually
be
giving
the
definition
of
the
quad
autoscaler,
which
he
wants
to
create
in
the
target
clusters.
C
So
now
let
me
do
the
same
thing.
So
you
see
these
types
are
now
available,
so
the
resources
are
now
available
against
the
same
API
server.
So
the
original
resource
which
was
available
was
the
horizontal
part
or
a
scaler.
We
have
now
the
placement
and
the
template
overwrites.
We
are
working
on
so
overwrites.
We
have
a
mechanism
where
we
can
override
particular
field,
but
there's
the
generic
way
where
we
we
should
be
able
to
override
any
field
in
any
of
the
target
resources.
Just
by
specifying
the
path,
for
example,
spec
dot.
C
C
So
the
next
step
could
be
once
this
is
done
by
the
admin
the
user
actually
goes
ahead
and
creates
the
resources,
the
resources
which
the
user
wants
to
create
or
the
application
which
user
was
to
deploy
into
individual
clusters.
It
could
be
a
set
of
yah
meals.
All
of
them
could
be
converted
using
this
and
then
just
created
into
individual
clusters.
So
you
have
five
clusters
or
ten
clusters
where
you
want
to
deploy
a
particular
application
and
you
don't
want
to
interact
with
each
of
the
clusters.
You
can
just
do
this
federate.
C
So
the
next
step
for
this
would
also
be
like
I
already
have
an
existing
application
with,
say
a
helm,
charts
kind
of
stuff
or
you
have
a
list
of
URLs
for
the
application,
which
comprises
of
same
say,
deployments.
Replica
sets
some
services
front-end
back-end,
all
that
stuff.
So
we
are
working
on
providing
a
mechanism
where
you
can
just
up
convert
all
of
that
into
the
federated
types.
So
if
you
use
this
is
something
in
pipeline
and
we
are
working
on
this,
so
you
say:
cuvette
federate
up,
convert
my
some
application.
C
Okay
and
you
should
just
be
able
to
so.
This
should
just
be
able
to
give
you
the
MLS
for
that
application,
or,
if
you
choose
so
it
should
just
be
created
in
Federation
control
plane,
so
the
application
gets
deployed
and
all
the
clusters,
wherever
you
want
and
which
obviously
overrides,
are
not
created
right
now.
If
you
want
customizations
in
each
of
the
clusters,
you
can
change.
The
overrides
fill
the
data
that
you
want
to
be
customizing
in
the
overrides
and
create
the
overrides,
and
the
customizations
would
be
applied
to
those
clusters.
C
You,
okay,
so
already
supported,
use
cases
which
are
available
with
the
working
group
is
we
can
so
simple
logic?
You
can
deploy
unified
workloads
across
multiple
clusters
with
active
reconciliation.
It's
not
like
you
have
deployed
and
forget
once
if
there
is
something
changed
in
the
cluster,
it'll
it'll
remain
changed,
a
reconciliation
will
happen
periodically
and
you
can
also
do
clusters
cluster
specific
optima
customizations.
C
We
are
have
implemented
a
cross
cluster
service
discovery,
which
is
also
a
very
good
networking
feature
in
which
case,
if
you
want
to
deploy
an
application
across
clusters
and
the
endpoint
reachable
across
clusters,
there
is
a
mechanism
to
do
that.
We
have
implemented
a
dynamic
rebalancing
of
replica
workloads.
C
We
have
also
implemented
namespace
Federation's
so
that
the
same
cluster
can
a
same
set
of
clusters
can
be
used
by
by
different
users.
So
one
user
deploys
the
Federation
in
one
namespace.
Another
user
can
use
the
same
base
cluster
to
deploy
flood
protection
in
another
namespace
same
way.
The
workloads
can
also
be
deployed
in
two
different
clusters
in
different
namespaces
by
the
same
users,
and
we
have
almost
completed
federating
Nek
this
API
resource
without
writing
any
code.
C
So
in
pipeline
we
are
working
on
overrides
for
arbitrary
fields,
any
fields
we
are
working
on
status
aggregation.
We
are
working
on
more
high-level
scheduling,
resources
and
tooling
to
ease
translation
of
existing
katas
resources
if
they
just
need
to
be
deployed
in
multiple
clusters
or
migrated.
There
is
another
use
case.
Where
is
such
that
you
have
one
cluster?
You
have
your
deployed
workload.
You
want
to
migrate,
so
you
can
use
Federation
for
that.
C
You
just
you
just
the
Fed
rate
or
up
convert
that
particular
workload
and
then
apply
the
placement
in
place
of
cluster
number
one.
You
can
put
it
in
cluster
number.
Two.
Okay,
I
think
I
almost
ran
over
time.
So
this
is
what
I
there
are
more
things.
I
can
explain,
but
we
are
running
out
of
time.
So
I
can
take
questions
as
of
now
if
they
right.
C
D
E
C
So
the
individual
clusters
are
also
accessible
directly
and
users
can
update
that
or
it's
a
a
controller
can
update
that.
But
I
mentioned
in
one
of
the
slice
that
there
is
an
active
reconciliation.
That's
the
whole
point
of
active
reconciliation
that
whatever
is
the
intent
which
is
specified
in
the
Federation
control
plane.
It
will
always
be
updated
eventually,
so
user
doesn't
update.
The
controller
from
here
will
update
it
again.
It'll
notice.
C
F
C
So
his
question
is,
he
wants
to
deploy,
say
100
replicas
across
many
clusters,
some
set
of
clusters.
So
what
does
it
take?
A
difficult
approach
would
be
you
define
that
these
many
replicas
in
each
cluster
say
5
in
first
10
in
second
and
20
in
third
and
that
kinds,
but
we
have
I
mentioned
about
the
replica
scheduling
preferences
controller.
That
is
exactly
for
this
use
case,
where
you
just
specify
the
total
number
of
replicas
and
in
the
template.
C
You
will
define
that
these
are
the
remaining
fields
like
the
parts,
the
specification,
the
container
that
you
want
to
create
and
the
replica
scheduling
controller
will
automatically
distribute
these
replicas.
You
can
distribute
them
by
weights.
For
example,
you
can
say
my
first
cluster
should
get
a
weight
ratio
of
2
and
second
cluster
should
get
a
weight
ratio
of
1,
so
replicas
will
be
distributed
in
two
is
to
one's
ratio,
and
you
can
specify
min
max
against
each
of
the
clusters.
So
this
is
already
available.
You
just
have
to
use
that
Percy
does.
C
The
capacities
considered
in
the
sense
that
if
a
cluster
when
say
replicas
by
weights
are
automatically
distributed,
one
cluster
gets
say
20
replicas,
but
the
capacity
of
that
cluster
is
only
10,
so
the
remaining
10
will
remain
unsaid
yield.
So
this
is
observed
by
the
scheduling
controller
and
can
move
those
replicas
somewhere
else
where
it
had
all
the
replicas
scheduled.
B
C
Okay,
so
the
traffic
or
the
users,
access
to
individual
clusters
is
a
networking
problem
which
we
have
kept
it
outside.
So
you
need
to
install
a
load
balancer.
You
can
discover,
you
can
discover
the
services
in
individual
clusters,
but
you
need
to
install
or
use
an
external
load
balancer
to
have
exactly
this.
This
particular
feature
use.