►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building the Multi-Cluster Data Layer - Chirag Narang, Yugabyte
Getting the data layer right is critical to building a cloud-native application. The global accessibility of persistent data across multi-cluster Kubernetes environments is the biggest challenge for building stateful workloads. As organizations rush to scale their infrastructure on multi-cluster Kubernetes, they often struggle with database deployments. Many traditional databases have not worked at the scale needed; either they are not horizontally scalable or highly available and do not provide strong consistency across multiple clusters. We’re here to show you that it’s easy to deploy a distributed database across multiple clusters. In this talk, join Chirag Narang, Product Manager from YugabyteDB to learn: Cross cluster deployment challenges in distributed databases How practitioners can adopt a service mesh to run a distributed SQL database across clusters How to solve global consistency, scalability, high availability, and geo-distribution challenges with YugabyteDB
A
Hi
kubecon
and
welcome
to
building
the
multi-cluster
data
layer
talk.
My
name
is
chirag
narang,
I'm
a
product
lead
at
egobyte.
In
today's
session.
I'm
going
to
show
you
how
to
build
a
multi-cluster
data
layer.
The
easy
way
we'll
go
over
some
key
requirements
for
deploying
a
database
across
multiple
kubernetes
clusters
running
in
different
regions
and
how
to
use
a
service
mesh
like
sto
to
solve
for
networking
challenges.
A
So
with
that,
let's
start
with
the
ego
by
db
project
you
could
buy
db
is
a
fully
open
source
distributed
database
which
is
built
for
the
cloud
native
world.
It
can
be
deployed
across
private
and
public
clouds,
including
kubernetes.
It
reuses
the
query
layer
of
postgres
sql,
so
it
offers
advanced
features
like
triggers
stored
procedures,
partial
indexes.
A
A
A
This
ensures
that
you
can
handle
peak
traffic
during
black
friday
and
cyber
monday.
So
once
the
holiday
season
is
over,
you
can
always
scale
back
eh.
You
don't
need
maintenance
maintenance
window
for
these
operations
anymore.
All
these
operations
can
happen.
While
your
database
is
online,
you
can
distribute
data
across.
B
A
So
very
quickly,
let's
look
at
the
database
design.
The
guiding
principle
for
gigabyte
is
a
layered
approach.
We
built
a
plugable
query
engine
which
preserves
the
top
half
query
layer
of
postgres
and
cassandra,
but
at
the
storage
layer
is
changed
to
use
docdb,
which
is
common
across
both
apis.
It
is
built
using
a
custom
integration
of
rough
replication
distributed
asset
transaction
and
the
roxdb
storage
engine
which
all
are
based
on
the
google
spanner
design.
A
So
now,
next,
let's
see
what
a
few
benefits
of
deploying
your
database
and
kubernetes
one
of
the
key
values
of
containers
over
deploying
your
applications
simply
on
vms,
is
that
you
get
to
package
all
of
the
system
dependencies
that
are
required
for
the
application
to
work
in
your
container.
That
includes
your
database
right.
Everything
can
travel
together,
so
that
makes
it
much
easier
for
you
to
ensure
that
your
application,
alongside
database
move
from
development
to
test
to
all
the
way
into
production.
A
Nothing
really
changes
except,
except
maybe
some
of
the
secrets
you
are
using
to
communicate
with
the
databases.
This
really
gives
you
operation
efficiency.
Next,
it
eliminates
the
single
point
of
failure.
Now,
since
databases
are
running
in
containers,
all
single
point
of
failure
can
be
eliminated
completely.
Database
containers
can
be
as
highly
available
as
any
other
container.
In
your
ecosystem,
you
can
set
up
multiple
kubernetes,
multiple
redundancies,
use
load,
balancing
to
bridge
multiple
containers
and
maintain
performance
at
all
times.
A
It
gives
you
better
resource
utilization.
When
databases
are
deployed
as
containers
they
automatically
become
on
demand
as
per
the
rest
of
the
application.
There
is
no
need
to
maintain
a
monolith
database
instance
to
maintain
your
data.
Instead,
applications
can
utilize
their
own
databases
as
needed,
and
only
when
they
are
needed.
A
The
next
one
is
declarative
conflict.
You
can
use
a
declarative
conflict
to
specify
your
database
resources.
So
if
the
content
of
that
file
changes,
kubernetes
can
automatically
reconfigure
a
database
to
match
that
config.
This
really
allows
easy
scale
on
demand
during
peak
traffic
and
the
last
one
is.
It
gives
you
the
freedom
to
run
anywhere.
It
gives
you
easy
portability
between
different
cloud
and
on-premise,
with
that,
let's
see
next,
how
we
modeled
yoga
by
db
as
a
workload
on
kubernetes.
A
The
database
has
two
distributed
services.
The
first
one
is
yb
master,
which
is
actually
responsible
for
keeping
your
system
metadata.
Coordinating
system-wide
operations
like
create,
alter
and
drop
table
commands
and
maintaining
your
operations
such
as
load
balancing
the
second
one
is
the
ybt
server.
A
A
One
of
the
one
of
these
services
is
actually
a
headless
service
that
enables
the
discovery
of
underlying
stateful
parts
and
other
one
is
a
load
balancer
which
is
needed
for
accessing
the
admin
ui,
whereas
the
ybt
server
needs
just
one
stateful
set
and
one
headless
service.
The
stateful
set,
as
as
I
mentioned
before,
is
actually
a
data
node,
so
you
can
easily
scale
up
your
cluster
up
and
down
by
just
changing
the
replica
account
and
it
will
trigger
a
rolling
upgrade
without
any
maintenance
on
time.
A
So
very
quickly.
Let
me
show
you
the
basics
of
installing
the
database
on
kubernetes,
then
we'll
switch
over
to
requirements
and
challenges
in
a
multi-cluster
deployment
model
deploying
on
kubernetes
is
pretty
straightforward.
We
have
a
helm
repo
for
you
to
get
sorted.
You
can
come
up
with
your
own
set
of
values.
What
kind
of
database
version
you
want
resource
allocation,
number
of
replicas
for
the
master
and
the
data
node,
and
and
if
you
want
to
change
the
storage
class.
A
So,
let's,
let's
see
behind
the
scene,
what's
happening
in
the
cluster
and
how
is
networking
set
up
using
kealy
at
this
point,
let
me
switch
over
to
the
terminal
all
right,
so
let's
first
check
the
safely
set
and
services
that
are
running
inside
a
single
cluster
deployment
of
cable
tv.
As
you
can
see
here,
I
have
both
t
server
and
master
stateful
sets
deployed
and
there
is
one
headless
service
for
yb
masters
and
one
headless
service
for
ybt
servers.
B
A
A
A
A
A
A
A
A
A
The
steps
actually
require
you
to
install
istio
control
plane
all
the
clusters
that
are
going
to
participate
in
a
multi-cluster
setup,
and
you
have
to
mark
each
cluster
as
a
primary
cluster
with
the
with
the
issue,
one
dot
with
with
1.8
release
of
istio.
They
introduce
a
concept
of
dns
proxy,
while
kubernetes
provides
dns
resolution
for
kubernetes
services
out
of
the
box,
but
any
custom
service
entry
is
not
recognized
with
the
latest
release
in
1.8.
A
The
gateway
in
each
cluster
must
be
reachable
from
from
from
the
other
cluster.
So
again,
you
would
need
to
repeat
this
step
of
setting
setting
up,
especially
for
every
cluster
that
that
will
be
participating
in
this
conflict.
Another
important
point
to
actually
keep
in
mind
here
is
that
gateway
will
be
public
on
intel
on
the
internet
by
default.
So
for,
if
you're
setting
up
anything
in
production,
you
may
need
to
make
some
additional
adjustments
on
your
firewall
rules
to
not
allow
public
access.
A
A
A
So
if
you
have
three
clusters
participating
in
this
configuration,
you
will
have
to
repeat
this
step
for
every
pair.
So
with
this
I
have
hto,
which
is
installed
in
all
my
clusters.
I
have
set
up
my
service
discovery
and
I
have
also
set
up
remote
secrets,
so
I
would
really
expect
this
cross
cluster
access
for
my
master
and
t
server
workloads
to
work.
But
there
is
a
slight
problem
which
brings
us
to
your
next
step,
which
is
to
expose
services.
A
What
this
means
is
really
that
there'll
be
one
master
deployment
per
cluster
right,
one
in
each
cluster,
the
master
part
and
the
east
cluster
is
connecting
to
the
master
pod
in
the
west
cluster
and
both
of
them
both
of
them
are
running
in
different
networks.
In
this
case,
everything
is
working
fine,
because
the
master
server
is
resolved
to
a
single
bot
ip,
because
there
is
just
one
port.
But
what
happens
in
case
of
a
in
case
of
t
server,
which
has
multiple
pods,
scheduled
on
the
same
cluster.
A
The
first
problem
that
we
faced
with
the
setup
was
a
single
t-server
headless
service,
which
is
what
we
actually
use
in
a
single
cluster
setup.
If
we
recall,
I
showed
you
when
we
were
talking
about
single
cluster
setup,
it
was
doing
round
robin
between
multiple
t
server
parts,
so
there
was
no
consistent
ip
address
to
reach
all
these
separate
parts.
So
if
the
master
from
west
was
trying
to
connect
to
t
server
in
east,
it
was
not
able
to
get
ips
of
all
the
t,
servers
that
were
actually
running
in
that
cluster.
A
A
A
A
A
A
A
B
B
A
A
Okay,
so
that
now
next
is
the
most
interesting
section
of
this
talk
where
we're
going
to
have
a
product
demo
of
fault
tolerance,
I'm
going
to
deploy.
I
have
deployed
a
three
node
three
node
ego
by
cluster
across
three
different
regions
and
we're
gonna
demonstrate
fault
tolerance
by
by
taking
down
the
entire
region.
A
This
is
my
cluster
spec.
I've
different
the
databases
deployed
across
three
gke
clusters
and
usbs
west
and
central
I'm
using
the
default
synchronous
replication
between
between
our
three
different
nodes
and
data
is
secured,
with
both
encryption
at
rest
and
in
flight
for
the
purpose
of
demo,
I'm
using
a
classic
risk
micro
services,
application,
which
is
which
is
also
running
in
kubernetes,
the
ui
app,
is
actually
outside
of
the
cluster.
A
A
B
B
A
B
B
A
B
B
A
Okay,
yeah,
so
data
seems
to
be
persisted
across
across
different
clusters,
so
this
verifies
that
single
database
database
stressed
across
multiple
kubernetes
is
working
fine.
With
with
data
consistency,
I
mean
I
don't
have
to
do
the
test
for
central,
we'll,
we'll
look
at
and
in
the
next
in
the
failover,
by
accessing
the
data
actually
through
central.
A
A
A
B
A
B
A
A
A
All
nodes
of
eco
by
db
are
eligible
to
have
short
leaders,
but
what,
if
you
want
to
localize
your
shard
leaders
closer
to
your
client
application,
which
might
be
deployed
in
a
particular
region,
for
example,
if
the
client
app
is
deployed
in
u.s
west?
So
what
if
you
also
wanted
to
have
a
show
leader
in
that
particular
region?
Or
imagine
another
use
case
that
your
application
reads,
including
multi-role
joins,
are
coming
from
a
single
region,
then
a
three
region
distributed
cluster
can
be
configured
to
have
short
leaders
pinned
to
that
single
region.
A
In
that
case,
you
can
set
a
preferred
zone
which
will
reduce
the
number
of
hops
for
database
to
write
transactions,
and
it
will
give
you
an
immediate
performance
improvement.
As
you
can
see
here,
you
can
use
yb
admin
command,
which
takes
all
master,
addresses
and
preferred
zone
as
as
an
as
input
variables.
So
try
this
use
case
out
and
let
us
know
how
does
it
improve
your
application
performance.
A
So
now
that
we
have
gone
through
getting
multi-cluster
setup
the
easy
way
and
that's
pretty
much
it-
we
have
shown
what
you
need
to
do
to
get
service
service
discovery
so
that
you
can
access
access
services
from
one
cluster
and
another.
We
have
outlined
what
what
it
takes
to
get
cross
cluster
access,
so
this
really
opens
up
new
possibilities
on
how
you
can
deploy
a
distributed,
sql
database
to
build
fault,
tolerant
cloud
native
applications.
A
A
The
second
one
is
what
we
talked
about
today
within
a
single
cloud
now
you're
stretching
your
single
database
across
multiple
regions.
This
gives
you
region
level,
failover,
resiliency
and
the
third
one
is
multi-cloud
multi-region,
which
is
really
now.
You
can
stretch
your
database
across
different
clouds
right.
The
same
constructs
the
primitives
that
we
talked
in
a
single
cloud.
Multi
region
now
can
be
applied
to
this
setup
as
well.
So
now
you
get
cloud
level
failover
resiliency
for
your
application
as
well.
A
Finally,
some
parting
thoughts.
We
are
a
fast
growing
project,
but
we
really
like
to
get
the
community
involved.
We
would
love
to
know
more
about
your
use
cases
so
jump
onto
onto
those
links
if
you
are
interested
in
becoming
a
cloud
native
database
expert
and
if
there
is
anything
that
you
need
or
help
with
come,
join
us
slack
community
with
that.