►
From YouTube: Antrea Multi-Cluster Service
Description
Come and join Lan Luo and Jiajing Hu to see how antrea makes multi-cluster service happen!! The multicluster stuff has been going on a while, with different approaches (i.e. https://cloud.google.com/kubernetes-engine/docs/concepts/multi-cluster-services ). Theres also an Upstream KEP on multiservice clusters (kubernetes/enhancements/keps/sig-multicluster/1645-multi-cluster-services-api/specification.md) that you can read to learn more about this stuff.
In this show, we'll learn about how antrea approaches this problem by creating a concept of a multi-cluster service object.
From the docs (https://antrea.io/docs/main/docs/multicluster/user-guide/)
Antrea Multi-cluster implements Multi-cluster Service API, which allows users to create multi-cluster Services that can be accessed cross clusters in a ClusterSet. Antrea Multi-cluster also supports Antrea ClusterNetworkPolicy replication. Multi-cluster admins can define ClusterNetworkPolicies to be replicated across the entire ClusterSet, and enforced in all member clusters. Antrea Multi-cluster is introduced in Antrea v1.5.0, and the ClusterNetworkPolicy replication feature is supported since Antrea v1.6.0.
A
A
B
A
A
A
So
so,
if
you
so
so,
for
example,
if
you
want
to
use
this
multi-cluster
feature,
you'll
need
something
called
entry,
multi
cluster
controller,
and
this
is
not
deployed
by
android
agent.
You
need
to
use
this
multi-cluster
controller.yelmo
to
create
this
controller
by
yourself
and
then,
after
you
have
this
one,
and
you
also
have
the
local
cluster
and
they
also
call
this
vocal
cluster
member
cluster.
A
If
I
remember
it
correctly
so
and
and
oh
by
the
way-
and
you
also
need
to
install
this
multi-cluster
controller
within
each
local
cluster
and
then
you
will
create
the
service
as
uru
as
and
if
you
want
to
export
this
service,
you
need
to
deploy
another
crd
that
is
defined
by
this
entry
multicultural
controller,
which
is
called
service
as
a
service
export.
I
think,
and
then
after
you
have
this
object
created.
A
This
controller
will
will
notice
that
this
one
exists
and
then
this
controller
creates
something
called
service
export
in
the
management
cluster
and
then
this
android
multicultural
controller
notice
that
notice
that
there's
like
a
resource
expert
created
and
it
will
create
another
thing,
called
resource
import.
So
all
this
all
the
objects
here
are
the
crt
defined
by
this
entry,
automatic
customer
controller
and
then,
after
you
have
that
there,
the
other.
A
So
I
I
only
draw
one
more
call
cluster
here,
but
actually
you
can
have
like
several
vocal,
closer
or
member
cluster,
which
is
the
official
name
that
called
in
andrea.
So
you
have
this
here.
Let
me
change
this
to
number
three
and
then
yeah,
so
both
of
also
all
the
controllers
in
the
member
cluster
will
watch
this
resources.
A
So
so,
when
they
notice
that
there's
well
there's
a
resource
import
created,
they
will
like
create
the
corresponding
service
import
here
and
the
service
the
real
service
here.
So
this
tool
will
be
created
by
this
controller
and
also
the
name
of
this.
Auto
created
controller
will
be
called
entry
dash
mc
dash,
the
service
name
and
the
corresponding
endpoint
will
be
also
created.
A
B
A
Okay
and
then
so
there,
so
if
we
want
to
take
a
closer
look
at
this
entry,
mlt
cluster
controller,
there
are
two
parts:
one
is
the
exporter
and
one
is
the
importer.
So
this
service
export
is
like,
I
think
it's
managed
by
this
exporter
and
when
you
want
to
create
a
service
input,
it's
managed
by
the
importer
in
this
controller.
So
there
are
two
parts
inside
this
controller.
B
A
B
B
B
B
B
D
A
D
No
worries
I
was,
I
was
just
explaining
something
real
quick.
Essentially
you
know
all
the
clusters
in
the
cluster
set,
you
know
were
were
treated
as
a
like.
I
wouldn't
say
you
know
the
same
cluster,
but
there
are
some
mutual
degree
of
understanding
between
you
know
what
one
namespace
in
the
cluster
and
the
other
namespace.
If
they
are
of
the
same
name
they're
essentially,
you
know
meaning
the
same
thing.
D
So
if
you
export,
you
know
one
service,
one
of
namespace,
one
in
one
cluster
and
if
you
export
service,
one
enamels
with
one
in
another
cluster,
those
two
will
be
considered
the
same
thing
and
when
you
do
the
service
import
and
export
those
the
end
points
of
both
services
in
different
clusters
will
be
merged
into
one.
So
essentially,
this
is
you
know
a
single
service,
but
across
spend
across
different
clusters.
So
now
you,
when
you
access
the
service,
the
the
back
end
of
it
might
be.
D
C
C
A
C
Okay
thanks
everyone
and
I'm
lolan
and
I'm
from
beijing
and
I
work
when
I
join
entry.
I
almost
start
with
and
much
cluster,
so
I'm
familiar
with
this
part,
so
I
will
introduce
this
part
today.
Give
a
give
you
an
overview
about
this,
and
let
me
know
if
you
have
any
questions
during
my
demo
on
the
introduction.
Thanks.
A
A
Let
me
find
that
yeah
here,
okay,
so
before
we
join,
we
go.
We
go
over
the
mirror
board
that
we
that
we
draw
in
the
rehearsal.
Remember
that
so
we
tell
us,
I
would
tell
the
audience
a
little
bit
about
the
high-level
picture
of
the,
how
yeah
management
cluster
vocal
cluster
and
how
this
like
controller
created.
A
D
C
Okay,
cool,
I
think,
in
the
upstream
we
have
independent
dock,
the
separate
part
dock
for
this
multi-cluster
and
all
of
this
smart
cluster.
You
can
see
here,
it's
you
know
it's
about
multi-cluster.
I
will
start
from
the
architecture.
So
let
me
go
back
and
then
give
that
give
us
an
overview
about
this.
You
know
that.
Let
me
know
if
I
should
is.
C
Okay,
cool
thanks.
You
know
that's
in
a
multi-cluster
environment,
so
we
usually
talk
about
more
than
one
cluster
and
in
a
traditional
way
to
use
a
comedies
or
when
the
developer
use
one
user,
kubernetes
development
management,
environment.
They
will
just
use
one
single
cluster,
but
the
one
which
goes
to
the
production
environments.
Usually
there
will
be
more
and
more
multi-class
and
more
and
more
kubernetes
cluster
and
different
workloads
running
in
different
clusters.
C
So
here
you
know
that
that's
it
will
be
a
problem
when
people
goes
to
math
cluster,
especially
some
of
the
surveys
you
you
know,
that
is
a
service
is
in
single
class
as
a
concept,
for
example
the
class
type
type
of
service
it
can
only
be
accessed
in
one
single
cluster
for
other
cluster.
It
cannot
access
this
kind
of
service
directly.
C
C
As
you
can
see
here,
we
have
a
few
new
crds
introduced
in
this
multi-cluster
feature
and
we
all
released
this
in
1.5,
and
you
know
that
for
the
class
claim,
it's
actually
used
by
the
admin
to
claim
one
cluster
itself
as
a
member
cluster
or
it's
a
later
customer.
It's
mainly
contains
class
id
and
the
name
and
the
assets.
C
It's
actually
contains
different
clusters,
information,
including
leader
and
the
member.
It's
a
little
slight
difference
in
member
cluster
and
the
leader
cluster
when
we
define
a
cluster
set.
But
I
will
show
you
the
difference
later,
when
I
demo
it
the
member
class
announcement,
announced
it's
actually
remember:
cluster
join
the
leader
cluster.
Actually
it's
a
talked
successfully
with
a
leader.
Then
we
have
a
member
class,
announce
cr
created
successfully
in
the
leader,
and
we
also
have
a
few.
There
are
actually
four
crd
here.
C
I
guess
danielle
already
introduced
this,
so
it
asks
for
import
rights.
I
maybe
I
can
skip
this
part
for
the
resource,
export
and
import
here
at
my
working
in
the
leader
cluster.
It's
actually
you
can
consider
it's
like
a
wrapper
in
the
resource
wrapper.
It's
a
it
can
have
a
different
kind
of
resource
information
in
this
cr
and
it
will
be
wrapped
and
encapsulated
and
also
the
resource
import.
This
kind
of
information-
it's
actually
sometimes
you
it's,
the
results
might
be
almost
the
same.
C
C
Okay,
as
I
mentioned,
the
class
claim
is
used
to
claim
a
cluster.
It's
just
a
simple
information
about
the
class
id
and
the
name.
Okay,
let
me
go
move
forward
and
you
know
that
matt
and
trump
cluster,
a
little
independent
to
the
entry-
and
you
can
totally
depart-
depends
this
standalone
deployment.
It
has
a
standalone
deployment
and
a
new
manifest
to
deploy
it
yeah.
B
C
Okay,
cool,
okay,
thanks
actually
in
release
1.5
we're
mainly
working
on
the
service,
the
resource.
You
know
we
can
have
a
different
types
of
results
to
export
input,
but
we
focus
on
service
export
imports
in
1.5.
So
here,
as
you
can
see,
there
is
our
member
two
member
cluster,
the
left
one.
Let's
talk.
C
Let's
say
that
so
we
have
one
service
here
named
foo,
oh
sorry,
full
in
the
names
in
the
name,
space
ns
and
it's
it's
already
running,
and
so
it's
working
and
the
admin
want
to
export
or
he
he
or
her
want
to
let
other
member
cluster
to
know.
He
has
this
kind
of
service,
and
I
would
like
to
let
others
to
access
this
service.
Then
the
admin
can
create
this
service
export
cr
and
just
include
the
service
name
and
the
name
space,
and
to
tear
our
multi-class
controller.
C
To
say
that
hey.
I
want
to
export
this
service.
Then
our
exporter
here,
it's
actually
part
of
our
cluster
control,
multi-class
controller,
but
to
be
as
enjoyed
as
an
independent,
charting
diagram
here.
But
it's
just
one
process:
you
can
think
about
that
and
it
will
watch
the
creation
of
the
service
export,
just
like
I
mentioned
before.
It
will
create
and
wrap
those
service
information
like
service
and
endpoint
information
to
the
leader
cluster,
and
it
will
become
the
resource
export.
C
Once
we
create
a
resource
export
into
the
leader
cluster.
A
leader
clusters
controller
will
watch
this
kind
of
creation,
even
then
the
resource
in
its
then
it
will
do
a
necessary
computation
or
just
converge
them
into
an
almost
the
same
resource
import
in
other
member
cluster,
for
example,
in
the
right
member
cluster.
The
imports
or
our
multi-class
controller
will
watch
this
kind
of
events
in
the
leader
cluster.
Then
it
can
create
a
corresponding
service
and
endpoints
locally.
C
Then
you
can
just
access
this
kind
of
service
almost
exactly
like
other
service.
The
only
difference
is
that
we,
we
actually
add
an
prefix.
Let's
say
the
entry
mc
dash
mc.
So
if
you
like
back
the
service,
then
you
need
to
write
prefix
that
you
should
be
able
to
occur.
Sorry,
you
should
be
able
to
access
the
service,
but
you
know
that's
it's
a
little
limitation
in
our
first
release
about
service.
C
It
depends
on
the
online
network.
We
need
to
yes,
sorry
yeah.
As
you
can
see
here,
we
have
a
few
little
limitations
in
first
release,
but
first
we
support
the
class
type
ip
type
and
the
underlay
network.
We
require
spot
ips,
reachable
between
different
member
clusters
and
also
no
ip
overlapping
and,
of
course,
in
the
future.
We
may
support
more
feature
to
maybe
allow
users
to
do
the
access
directly
with
the
data
exchange
and
it's
also
in
our
plan
yeah.
A
I
think
we're
good,
I
I
I
don't
see
any
questions
in
comments
and
by
the
way
for
all
on
the
for
all
the
audience
you're.
If
you
are
the
first
time
watching
our
show,
you
can
sign
into
your
youtube
youtube
account
and
leave
us
and
leave
the
comments.
So
we
can
like
answer
any
questions.
Do
you
have
or
show
your
comments
on
the
screen?
A
C
B
C
C
As
you
can
see,
we
actually
have
a
two
mult
cluster
controller
running.
One
is
in
this
name
space,
the
extreme
mcsns,
that's
our
you
can
treat
it
as
our
leader
cluster
or
the
leader
name,
space
working
as
a
leader
controller,
and
this
is
another
one.
It's
working
in
the
kubernetes
system
and
this
one
is
played
as
a
member
controller,
so
this
class
actually
acts
as
a
both
leader
and
the
member
and
okay.
Let
me
go
and
see,
as
I
mentioned
before,
we
will
have
a
class
set
and
a
class
claim.
C
C
As
you
can
see,
we
have
a
name
and
namespace,
and
this
is
the
id,
and
this
is
our
predefined
name.
Then
it
means
this
is
just
the
id
of
the
class
member
cluster
or
leader
cluster.
Here's
a
value,
it's
actually
the
id
of
the
leader,
and
you
may
see
that
sorry,
let's
see
another
plus
the
claim,
let's
actually
claim
the
class
id.
C
B
C
B
A
C
Actually,
yeah:
let's
go
back,
you
can
see
that
the
value
here
is
the
leader,
but
it's
actually,
the
usual
user
can
define
it's
a
totally
provided
by
user,
so
it
can
be
test
cluster
without
leader,
it's
just
a
value,
but
the
user
or
the
admin
need
to
guarantee
that
the
id
is
unique,
actually
we're
working
on
the
id
uniqueness
validation
in
first
release.
We
have
in
this
part,
which
means
you
may
provide
to
duplicate
or
the
same
name
for
the
two-member
cluster.
C
C
Let's
go
and
then
see,
this
is
the
status
that
we
can
check
later,
and
this
is
actually
just
back,
including
both
leader
and
the
member.
This
one
here,
you
can
see
this
leader
class
id,
which
you
just
we
defined
it
and
use
the
class
claim
to
claim
it
itself
as
a
leader
actually
as
a
part
of
the
class
set.
You
can
consider
that,
let's
just
use
the
class
claim
to
define
that
hey.
C
A
C
C
Here
it
is,
let's
see
here,
it's
actually
the
name.
You
can
consider
that
this
kind
of
a
claim
is
when
that,
what's
the
class
set
id,
I
will
I
will
join,
or
if
you
see
this
value
as
id,
then
it's
actually
claimed
that
which
class
set
I
will
join
see.
This
is
the
class
set
id,
because
here
tell
you
that
this
class
set
dot
combinations,
dot
io
here
and
but
you
see
that
let's
go
back
and
see
here
is
another
name:
it's
a
yeah.
I
think
it's
a
little
confused
here.
C
B
B
B
C
Yeah,
okay,
okay,
let
me
go
to
let
me
go
ahead,
so
we
have
a
class
set
defined
in
the
member
cluster
is
actually
almost
the
same
class
claim
and
a
class
set,
but
there
are
slights
different
because
you
know
the
in
the
leader
you
will
assign
the
service
account.
You
will
point
out
which
service
accounts
in
your
leader.
It
will
be
used
by
the
member
right,
but
in
the
member
cluster
it
it
will
be.
Another
information
is
about
the
secret
which
secret
it
will
be
used
to
access
leaders.
C
B
C
C
Okay,
as
you
can
see
here,
let's
see
here,
you
can
see
in
the
member.
It's
actually
just
a
you
know.
It's
just
concern
about
which
one
is
his
its
leader
right,
which
one
I
should
access.
So
it
contains
a
basic
server
access
information.
The
address
and
also
another
important
is
the
information
is
a
secret.
This
secret
is
actually
created
manually
by
user
so
far
and
the
usc
this
us
and
then
you
can
see
the
member.
Actually,
you
didn't
see
the
full
member
list
right
because
in
mem
your
member
cluster,
it's
all
linked.
C
C
As
you
can
see
here,
I
just
defined
a
source
export
and
this
is
a
name,
and
this
is
the
namespace.
It's
actually
this
point
to
this
service.
Let's
just
tell
the
mccontroller
to
say:
hey:
this
is
a
service
I
like
to
export.
To
tell
other
members
I
like
to
share
to
others
to
access
right,
so
let
me
apply
it.
C
A
C
C
Yeah,
you
can
see
that
we
have
a
new
service
and
endpoint
created
a
few
seconds
ago,
and
you
know
once
we
have
a
service
exported,
then
the
member
cluster
will
watch
this
creation
event
and
and
got
those
service
and
end
upon
information
and
the
writing
to
the
leader
cluster.
Here
it's
eventually
become
a
resource
export
in
the
leader.
Here
you
can
see
this
namespace
place
a
play
as
a
leader,
so
we
have
these
two
services
because
a
service
in
1.5
we
use
endpoint
as
a
part
ips
and
the
point
of
multicast
service.
C
C
C
Yeah,
that's
right
because
we
have
a
corresponding
endpoint
endpoint
for
the
service
right
and
in
our
current
implementation,
because
we
are
using
the
part
ip
as
endpoints
so
for
the
mod
cluster
service.
So
we
need
this
kind
of
information
for
other
members
in
the
future
we
will
just
use
class
type.
Then
you
may
just
see
one
resource
explorer
here.
C
As
you
can
see
here,
it's
almost
the
service
definition
and
the
the
most
important
thing
is
about
the
port
for
the
class
ip.
You
can
ignore
it
so
far
because
it's
actually
not
taking
any
effect
in
our
service
access,
and
you
know
that,
as
deanna
mentioned
the
same
names,
we
need
to
make
sure
that
all
exported
service
have
the
same
parts.
Definition
as
you
can
see
here.
If
you
have
the
same
name
space
and
the
service
like
we
all
call
it
ngx
and
as
a
default
namespace,
but
one
injector.
C
C
Okay,
let
me
go
to
another
member
cluster.
Let
me
go
to
see
the
service.
Okay,
as
you
can
see
here,
we
have
a
new
service
created
here
and
it's
with
a
predefined
prefix.
Here
we
have
a
new
service
engix,
but
it
has
our
own
prefix
here
and
the
class
ip
is
the
local
class
ip
and
let's
go
and
see
the
endpoint.
C
C
C
Okay,
we
have
a
new
new
endpoint
right,
it's
the
same
as
the
endpoint
in
the
in
this
member
cluster.
So
which
means
that's
why
we
need
to
the
the
limitation.
Is
that
part?
I
p
need
to
be
reachable
between
member
clusters
because
we're
using
the
same
endpoint
ip
right.
It
means
we
need
to
communicate
in
each
other's
cluster.
C
C
C
C
Yeah,
you
will
see
we
have
a
new
service
and
endpoint
type
of
resource
export
import
export.
This
one
is
from
the
north
and
this
is
his
default
namespace
and
the
name
service,
and
this
is
endpoint
type
and
this
service
type.
So
let's
go
to
this
member
cluster
to
see
what
we
have
now
for
the
endpoint.
C
You
will
see
that
there
are
almost
no
difference
between
the
service,
so
the
service
won't
be
updated,
but
to
the
endpoints,
because
we
have
a
new
member
class
exported
this,
then
you
will
see
we
have
a
new
sorry
new
endpoint
here
right
there
are
two
endpoints:
let's
go
back
to
check
the
the
loss
cluster.
Let's
just
say
that
ep.
C
C
C
C
C
C
C
You
will
see
this
actually
contains
all
the
other
available
addresses,
which
you
actually
see
that
they
are
reflecting
here
right,
because
the
local
member
class
controller
is
watching
these
resource
imports.
If
it's
a
new
created
one,
the
it
will
create
a
local
new
service.
If
it's
being
updated
like
this
one,
we
have
a
new
update
addresses.
Then
it
will
be
also
reflected
into
local
if
it
has
new
one.
So
let
me
go
back
to
delete
this
export.
C
C
C
C
A
Okay,
thanks
swan,
it's
like
a
really
clear
introduction
of
this
multi
cluster
service
and
I
I
learned
a
lot.
It's
really
cool.
It's
a
really
cool
feature
thanks
for
bringing
this
to
the
entry,
and
so
we
also
have-
we
probably
don't
have
enough
time,
but
maybe
we
can
have
like
a
a
quick
short
introduction
of
like
the
entry
cli
right.
So
we
have
like
judging
here,
to
join
us
to
tell
us
a
little
bit
about
the
cell
eye.
So
we
can
like
give
a
give
a
short
introduction.
A
A
Because
this
this
is
also
judging's
first
time
to
be
here
with
us.
So
if
you
like
to
introduce
yourself
to
the
audience,
we
welcome
that
and
also
we
probably
don't
have
enough
time
to
finish
all
the
and
and
curl
things
yeah.
So
we
can
like
do
a
short
beginning,
and
then
we
have
like
a
follow-up
episode
about
this.
E
Oh
okay,
I'm
judging
I'm
working
when
I
was
with
lola
and
I
am
working
on
the
future
about
the
market
cluster.
Maybe
my
cartoon
multicast
feature
is
a
is
a
feature
that
helps,
helps
us
to
make
it
easier
to
deploy
and
create
and
maybe
get
career
resources
up
in
the
mud
cluster
once
you
use
this
feature,
you
may
not
need
to
add
turns
some
mm
files
and
other
other
configures
in
the
in
the
local
just
long
some
commands
it
is.
E
A
Okay,
thanks
for
that,
and
also
in
in
the
follow
up
episode,
we
will
also
have
young
dean
with
us
to
tell
us
a
little
bit
about
to
tell
us
what's
happening
in
this
yeah
at
entry
policy.
It's
like
has
like
a
similar
business,
has
like
a
similar
behavior
like
this
service.
It
also
like
create
a
network
policy
in
all
the
member
clusters,
like
that,
so
we
will
we'll,
of
course,
invite
him
next
time
for
that,
and
also
I
really
really
want
to
thank
thanks.
B
I
have
still
have
five
minutes
right,
oh
yeah,
so
if
that
is
okay,
I
wanna
just
give
a
very
quick
summary
of
this
multicast
feature
we
are
doing.
I
think
lan
just
showed
us
the
the
monaco
service
money
current
service.
We
support
in
the
first
version
in
ontario
1.5,
actually
we're
actively
working
on
more
features.
B
As
you
said,
probably
last
time,
young
will
show
us
other
cars,
like
the
policy
that
we
are
implemented
in
until
1.6.
It's
already
released
with
that,
it's
very
similar
to
multicultural
service.
Basically
from
leader
cluster,
you
can
export
a
global
network
policy
and
the
policy
will
be
imported
by
the
member
cluster
and
applied
on
the
I
remember
across
to
the
local
traffic
of
the
member
cluster
and
the
police
are
less
less
less
the
major
feature
after
monochrome
service
and
then
on.
B
Child
107
polyline
also
briefly
mentioned
that
we're
working
on
connectivity
between
clusters,
so
that
will
remove
the
major
hard
limit
we
have
in
the
version
one
that
will
require.
Other
ports
are
directly
routable
of
course,
clusters.
It's
typically
a
lot
of
the
case
right,
because
posts
are
typically
in
some
overlay
network.
B
They
have
private
status
over
every
member
cluster,
so
they
are
not
directly
resultable
across
clusters,
so
with
1.7.
Hopefully
we
can
enable
the
connectivity
across
constants
with
the
tunnels
between
constants.
We
will
have
gateway
nodes
and
then
between
gateway.
We
establish
tunnels
with
gdp,
western
or
gi
tunnels,
and
finally,
we
can
also
have
ibsec
for
encryption
just
in
case
you
want
to
secure
communication
between
clusters
and
once
we
have
that,
then
we
will
no
more
require
the
port.
B
Iop
can
be
directly
routable
between
clusters,
but
the
water
traffic
cost
clusters,
for
example,
with
multicast
service
success,
can
go
through
the
tunnels
between
our
gateway
nodes.
I
think
at
that
time
probably
we
can
see.
This
feature
is
more
useful
in
the
typical
multicast
scenario
and
then
the
final
one
for
the
young
rosa
talk
in
his
epic
later
after
this
connectivity
being
cast
us.
We
also
want
to
enable
we
call
this
stretched
net
policy.