►
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
TiKV: A Cloud Native Key-Value Database - Dongxu Huang & Nick Cameron, PingCAP
TiKV is a distributed Key-Value database that features in geo-replication, horizontal scalability, consistent distributed transactions, and coprocessor support. As a CNCF incubating project, TiKV has been widely used in production by over 500 companies. An adopter has even stored over 1.3 trillion rows of in TiKV. In this talk, Dongxu Huang and Nick Cameron will share the story of how they built TiKV from scratch, including how they decided the technical solutions, how they interacted with other CNCF projects, and also how to apply Chaos Engineering on TiKV to guarantee system safety and robustness. In the end, Dongxu Huang and Nick Cameron will show the approach to a self-driving database in TiKV 4.0, which leverages Kubernetes to make TiKV more elastic and scalable.
https://sched.co/a3vq
A
Hello,
this
is
doshu
today
nick
and
I
will
give
give
a
brief
introduction
of
thai
kv,
which
is
the
cloud
native
key
valley
database.
You
know,
tegami
is
a
cncf
project
and
now
it
is
incubating
level
project,
but
take
away
right
now
is
on
the
graduation
process.
So
maybe
next
month
takeaway
will
become
the
graduated.
B
A
A
B
A
Okay,
today's
talk
will
be
divided
into
two
parts.
First,
I
will
give
a
brief
background
of
background
introduction
of
thai
kv
project
and
share
some
stories
and
a
little
bit
of
history
of
this
project,
to
explain
why
we
are
doing
this
in
the
second.
Half
nick
will
share
some
technical
details,
benchmarks.
A
A
A
That
paper
described
an
implementation
of
a
distributed
transactional
database,
which
is
widely
used
inside
google,
and
also
I
read
another
paper
called
google
f1,
which
was
the
sql
layer
on
top
of
spanner
at
that
time
spanner
is
more
like
a
nosql
database,
so
this
paper
said
spanner
plus
f1
replace
their
sql
cluster
behind
google
ads,
and
so
we
we
were
so
excited
about
that.
So.
B
A
A
A
A
We
think
that
the
sql
layer
is
closer
to
the
application
layer
closer
to
the
to
the
developers
and
there's
a
better.
You
know
it
better,
expresses
our
ideas
in
the
early
days
and,
more
importantly,
I
said
I
just
mentioned
that
thai
db
speaks
mysql
particle.
We
decided
to
speak
mysql
particle
so
for
the
quality
qualitative
purpose
we
want
to.
You,
reuse
the
test
cases
from
mysql
and
mysql
community.
For
example,
we
can
borrow
some
test
cases
from
wordpress,
because
wordpress
is
using
mysql
as
its.
A
A
A
A
A
A
It
has
to
go
from
the
bicycle
client,
to
tidy
b
sql
layer,
sequel,
parser,
sequel
optimizer
and
goes
through
the
the
the
hbase
co-processor
to
run.
The
two
phrase
commit
particle
and
the
requests
go
to
the
hbase
hbase
part
to
the
region
master,
and
then
the
data
is
written
to
to
the
hdfs
data
node
and
the
performance
is
really
really
bad
and,
to
be
honest,
you
know,
maintaining
the
hadoop
stack
is
really
painful
is
I
think
it
is?
Even
you
know
more
painful
than
maintaining
the
mysql
cluster.
A
A
A
Second,
the
performance
is,
you
know,
performance
matter
matters
yeah.
I
will
not
explain
more
about
this.
It's
very
straightforward
right
now.
Third,
it
should
have
a
building
transaction
support
right
because
you
know
this
spanner
have
the
the
the
transaction.
It
is
the
biggest
selling
point
of
that
system.
Right
and
and
next
it
is,
you
know,
two
thousand
and
it
was
2016.
A
B
A
B
A
A
Performance
perspective
and
we
found
that
such
a
system-
if
we
have
this,
we
have
build
such
a
system.
This
system
has
the
potential
to
be
the
building
block
for
other
these
distributed
systems.
So
imagine
that
you
are
making
the
another
distributed
file
system.
For
example,
if
you
want
to
build
a
new
hdfs,
you
may
need
a
you
know:
metadata
store
right,
not
just
like
hdfs
name
node.
You
won't
want
it
to
to
be.
You
know
the
the
scale
scalable
scale
out.
A
A
A
A
B
A
B
B
You
could
call
it
a
nosql
database,
but
typically
nosql
databases,
don't
offer
transactions
and
the
consistency
properties
that
thai
kv
does
so
a
tai
kv
cluster.
It
looks
like
this.
The
thai
kv
nodes
are
in
the
bottom
right.
You
can
have
as
many
of
them
as
you
like.
Three
is
the
usual
minimum
to
the
left.
B
We
have
the
thai
kv
client
in
this
case
we're
using
tie
db,
which
is
an
sql
layer
which
can
sit
on
on
top
of
tai
kv
and
on
the
top
right.
We
have
placement
driver
nodes,
and
these
have
two
jobs.
One
is
to
supply
monotonic
and
unique
timestamps
to
the
clients
for
use
in
transactions,
and
the
other
is
to
manage
the
taikv
nodes.
B
What
a
thai
kv
cluster
looks
like
next,
let's
have
a
look
at
the
what
tai
kv
internal
implementation
looks
like,
so
it
has
a
very
strongly
layered
architecture
at
the
bottom
of
the
of
the
stack.
We
have
a
low
level
local
key
value
store
for
which
we
use
roxdb,
and
this
is
the
way
that
we
persist
data
to
disk.
B
Off
to
the
right,
we've
got
the
the
co-processor,
which
is
not
really
doesn't
really
fit
into
that
layered
architecture.
So
much,
but
is
a
piece
of
software
which
takes
intermediate
representation
of
some,
probably
sql,
query,
fragments
and
executes
that
close
to
the
data
itself.
For
the
sake
of
performance.
B
So
taika
v
is
transactional
in
terms
of
the
the
acid
breakdown
all
our
transactions
are
atomic
and
durable,
and
we
offer
the
snapshot.
Isolation,
consistency,
property
snapshot,
isolation
means
that
you
will
never
read
uncommitted
data
and
you
will
never
lose
a
committed
update
and
if
you
read
the
same
key
multiple
times
in
the
same
transaction,
you'll
always
get
back
the
same
value.
Taikevi
transactions
are
also
linearizable.
B
In
terms
of
implementation,
our
transactions
are
a
two-phase
commit
protocol
and
that's
based
on
the
percolator
algorithm
from
google
and,
as
I
said
before,
we
implement
that
internally
using
a
multiversion
concurrency
control
scheme,
which
means
that,
rather
than
saving
a
single
value
for
each
key,
we
save
a
history
of
values
for
each
key.
And
so
when
you
once
when
you
run
a
transaction,
it
sees
a
snapshot
of
the
data,
a
consistent
snapshot
of
the
data
at
a
particular
point
in
in
the
history
of
each
key.
B
B
B
So
instead
we
paid
pretty
much
the
same
amount
of
money
for
for
the
two
setups
and
here's
some
charts
of
the
the
results
of
that
benchmarking.
The
top
row
is
throughput,
where
bigger
is
better.
The
bottom
row
is
latency,
where
smaller
is
better
and
spanner
is
in
blue
on
the
left
and
tidy
b
is
on
in
green
on
the
right.
B
Version
4
of
taikov
was
released
in
june
this
year
and
we're
planning
to
release
version
5
early
in
2021,
there's
a
bunch
of
really
exciting
things
planned.
So
I
want
to
go
over
a
few
of
those
in
the
I
mentioned
earlier
in
the
talk,
how
we
use
roxdb
as
a
local
key
value
store.
So
one
of
the
things
we're
working
on
is
allowing
you
to
bring
your
own
key
value,
store
and
use
whichever
one
you
like.
So
you
can
choose
the
performance
characteristics
and
other
properties
that
that
you
want
another
major
change.
B
I
I
talked
about
transactions
in
thai
kv.
That's
an
api
between
the
server
and
client
there's
another
api
which
lets
you
just
deal
with
raw
bytes
with
a
simple
get
set
kind
of
interface,
rather
than
the
the
transactional
api
and
we're
adding
a
third
api
which
gives
you
manual
control
over
the
multiversion
concurrency.
A
B
In
this
final
section,
I
want
to
talk
about
taikovs
community,
because
taika
v
is
an
open
source
project
and
it
just
wouldn't
be
where
it
is
today
without
its
community.
Our
core
repository
has
over
100
contributors
and
that
community
is
active
and
growing,
using
taikov
filing
bug,
reports
and
writing
code.
B
These
are
not
just
groups
for
developers,
but
for
anybody,
who's
interested
in
the
the
kind
of
things
I've
been
talking
about.
So,
if
that's
you
please
come
and
join
us.
We
develop
on
github
and
we
chat
on
slack
in
the
tai
kvwg,
which
you
can
find
by
following
either
the
qr
code
or
the
url
on
this
page.
B
I'm
nrc
in
both
places.
So
please
come
say
hi
and
let
me
know
if
there's
anything
I
can
do
to
help
you
get
involved
all
right.
That's
all
we've
got
for
today.
Thank
you
for
listening
and
I
hope
you
enjoyed
it
and
the
online
format
we're
looking
forward
to
answering
your
questions
live
when
this
talk
is
broadcast
later
this
month,
cheers.