►
From YouTube: Keynote: Sailing with K8s Armada: Multi-Cluster Management with Massive Amounts of Nodes– Yifan Shen
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
与K8s舰队一起航行，海量节点的多集群管理 | Sailing with K8s Armada: Multi-Cluster Management with Massive Amounts of Nodes – Yifan Shen, PaaS Cloud Platform Architect, ICBC & Kevin Wang, Lead of Cloud Native Open Source Team, Huawei
A
Hello:
everyone,
I'm
kevin
wang
leader
of
the
huawei
cloud
native,
open
source
team,
so
glad
to
be
here
to
share
with
you
I'll
start
with
introducing
our
speakers.
For
me,
I
focus
on
developing
open
source
projects
and
maintaining
the
community.
I'm
a
cncf
ambassador
and
the
co-founder
of
several
cncf
projects,
including
comeda.
A
A
First,
let
me
start
from
the
construction
of
icbc's
cloud
platform.
Icbc
cloud
services
support
the
running
of
many
applications,
including
online
activities
on
holidays,
core
service
applications,
mysql
redis
and
other
technical
support
applications
and
the
ones
in
blockchain,
ai
and
some
new
technical
fields.
A
Icbc
has
been
developing
a
customized
cloud
platform
based
on
mainstream
open
source
projects,
ensuring
overall
independence
and
controllability.
It
is
also
the
largest
container
cloud
in
the
industry,
with
more
than
280
000
containers
deployed,
icbc
services
run
with
specific
requirements
and
on
a
large
scale,
cloud-native
infrastructure.
A
A
A
First,
limited
availability,
a
kubernetes
cluster
is
also
a
fault
domain,
so
automatic
recovery
across
fault
domains
is
required.
Second,
limited
resources,
only
application,
scheduling
and
auto
scaling
are
available
only
to
single
clusters.
Third,
non-transparent
clusters
with
heterogeneous
resources,
fault
domains
and
other
attributes
configured
for
clusters.
The
service
team
has
to
distinguish
all
the
underlying
clusters
to
find
out
the
target
cluster
where
they
want
to
run
services.
A
Upper
layer.
Applications
are
not
aware
of
cluster
differences.
Fourth
duplicate
configuration:
although
our
services
are
configured
on
the
cloud
management
platform
in
a
unified
manner,
the
specific
configuration
needs
to
be
delivered
to
each
cluster,
which
must
be
synchronized
to
address
these
challenges.
We
formulate
the
following
objectives
for
multi-cluster
management.
A
For
multi-cluster
management,
we
want
to
manage
all
clusters
in
one
place
and
their
entire
life
cycle
and
use
unified
standard
apis
for
resource
management.
We
need
to
support
multi-version
and
comprehensive
kubernetes
resources
and
multi-dimensional
resource
override
for
cross-cluster,
auto
scheduling.
It
should
be
realized
based
on
the
fault
domain
and
resource
margin,
partnering
with
cross-cluster
auto-scaling
for
disaster
recovery,
cross-cluster
resources
should
be
able
to
auto-recover,
and
the
control
plane
and
service
clusters
should
be
decoupled
for
compatibility.
A
A
First,
commercial
products
bring
in
vendor
lock-in
which
do
not
meet
our
requirements
on
independence
and
controllability
so
pass
other
open
source
software
can
be
a
consideration
such
as
kuberfed.
According
to
the
survey
on
kuberfed,
it
uses
non-native
kubernetes
apis,
which
makes
it
difficult
to
migrate
existing
clusters.
In
addition,
its
community
is
becoming
less
active.
It
doesn't
become
the
de
facto
implementation
standard
in
the
industry,
so
we
decided
to
develop
camera
to
satisfy
our
needs
as
we
are
building
the
icbc
financial
cloud
based
on
open
source
software.
A
Some
of
you
may
wonder
why
don't
we
choose
to
go
further
with
kuberfed
but
initiate
a
new
project?
Actually,
we
did
develop
the
kubernetes
federation
version
3
at
the
early
stage
and
quickly
completed
the
prototype
development.
However,
after
communicating
with
many
community
users,
we
found
that
the
federation
itself
cannot
fully
satisfy
what's
expected.
A
In
terms
of
the
core
architecture,
development
of
camera,
we
have
learned
from
the
experience
of
multiple
co-sponsors
in
multi-cloud
and
multi-cluster
management.
We
focus
on
the
native
api
support
of
kubernetes
and
the
scalability
of
the
core
architecture.
The
architecture
of
camera
is
similar
to
that
of
the
kubernetes
single
cluster.
In
many
ways,
both
of
them
have
a
control
plane,
an
api
server,
a
scheduler
and
a
group
of
controllers.
A
In
addition,
the
scheduler
framework
has
a
pluggable
design
so
that
users
can
customize
and
extend
scheduling
policies
in
terms
of
member
cluster.
Synchronization
camera
implements
agent-based
pull
mode.
In
this
way,
the
working
pressure
on
the
control
plane
can
be
effectively
reduced
and
users
can
easily
manage
large-scale
multi-cluster
resource
pools.
A
A
In
this
way,
users
can
use
the
yaml
or
apis
in
the
original
single
cluster
to
create
multi-cluster
applications
without
any
modification.
The
service
platforms
developed
based
on
kubernetes
apis
do
not
need
to
be
modified
either.
They
can
be
directly
transformed
from
the
single
cluster
architecture
to
the
multi-cloud
and
multi-cluster
architecture,
by
interconnecting
with
camera.
A
A
A
A
A
During
comada's
positive
cycle
consisting
of
design,
r
d
practices,
redesign
and
r
d
icbc
has
summarized
some
of
its
advantages
and
lessons
learned
in
the
following
four
aspects:
resource
scheduling,
disaster,
recovery,
cluster
management
and
resource
management.
I
think
the
following
three
points
deserve
special
attention
and
are
especially
prominent
in
the
actual
implementation
process.
A
A
A
After
talking
about
the
current
status
of
our
project,
let's
take
a
look
at
the
follow-up
plans
in
terms
of
large-scale
production.
We
hope
that
the
container
cloud
platform
will
serve
as
a
user-oriented
platform
and
the
underlying
layer
will
manage
and
schedule
resources
of
multiple
clusters
in
a
unified
manner
based
on
cometa.
In
this
way,
more
than
100
existing
kubernetes
clusters,
including
heterogeneous
clusters,
can
be
managed.
We
also
hope
to
make
continuous
contributions
to
the
community.
A
A
Of
course,
the
camera
project
also
has
many
other
exciting
functions.
Welcome
to
the
github
of
the
karma
project,
to
view
our
release,
notes
and
community
documents
to
learn
more
about
the
functions
and
details.
If
you
have
any
suggestions
or
feedback
when
using
camera,
please
join
our
wechat
group
and
slack
channel
to
reach
out
to
us.
That's
all
our
sharing
for
today.
Thank
you
for
your
time.