►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Using a Distributed Key-Value Store - Nick Cameron & Andy Lok, PingCAP
Key-value stores are well-known and the concept is simple, but there are myriad differences in the details of different stores. How you choose and use a key-value store in your projects is dictated by these details. TiKV is a distributed and transactional key-value store, which makes it massively scalable and a great choice if you have huge volumes of data and need low-latency access with low tolerance for inconsistency. In this talk, Nick Cameron and Di’an Luo will walk through building an application around TiKV using a language-native client. They will delve into the features, architecture, and limitations of TiKV, and the design principles and APIs of the clients. The talk focuses on the practical aspects of building an application using TiKV, so that you can easily leverage this technology in your own projects. TiKV and its clients are open source and have an open and active community; TiKV is a graduated CNCF project.
A
Using
a
distributed
key
value
store-
and
the
word
I
really
want
to
emphasize
in
that
title
is
using,
because
our
talk
today
is
about
how
you
would
actually
use
a
distributed.
Key
value
store
in
your
application,
we're
nick
and
andy
we're
engineers
at
pincap,
where
we
both
work
on
thai
kv,
which
is
an
example
of
a
key
value
store
as
well.
We'll
tell
you
a
bit
later
in
the
talk.
A
The
talk
is
going
to
be
in
three
parts,
so,
first
of
all,
we're
going
to
talk
very
quickly
about
what
a
key
value
store
actually
is,
and
some
example
kv
stores
and
about
their
features
and
how
you
would
choose
one
of
them
for
your
application
after
that.
Well,
we
have
to
pick
one
of
those
to
actually
talk
about
the
rest
of
the
the
talk
and
we're
going
to
pick
taikovi
now.
A
B
B
This
example
in
python
is
a
dictionary
that
maps
the
key
k1
to
value
v1
and
tk2
to
value
v2
kvstores
often
allows
you
to
look
up
the
value
by
the
corresponding
key,
for
instance,
during
this
dictionary
with
gk1,
the
returns
is
valid
v1
this
operation
now
import
get
except
get
the
kv,
store,
also
have
method,
put
and
delete
for
modified
data
in
the
real
world.
A
kv
store
often
refers
to
a
database
that
persists
gigabytes
to
terabytes
of
data
in
the
form
of
key
value.
B
Instead
of
this
more
dictionary
impulses,
but
once
you
search
for
qr
stores
in
google
and
then
you
will
be
drawn
by
too
many
cube
stores,
which
may
only
have
subtle
differences,
and
you
must
struggle
to
figure
out
which
one
best
fits
your
needs.
So
I'll
pick
some
of
the
most
common
ones
and
talk
about
their
pros
and
cons
and
different
kinds
of
scenarios.
They
are
best
for.
B
B
B
B
B
B
B
B
B
B
B
A
So
there
so
thai
kb
is
distributed,
which
means,
rather
than
running
on
a
single
server.
It
runs
across
a
cluster
of
servers
or
nodes
that
makes
its
and
because
of
that,
and
because
it's
the
way
it's
designed
it's
scalable.
If
you
need
to
store
more
data
or
you
need
to
access
that
data
more
quickly,
you
can
just
add
add
nodes
to
the
cluster
and
you'll
be
able
to
store
more
and
access
more.
A
A
A
Kind
of
guarantee,
whereas
tai
kv,
supports
explicit
transactions
and
gives
you
the
kind
of
acid
guarantees
that
make
working
with
transactional
databases
so
much
nicer
and
it
has
a
somewhat
rich
key
value
api.
What
I
mean
by
that
is,
it
doesn't
just
offer
kind
of
the
the
dictionary
hashmap
get
set
kind
of
api.
We
there's
a
whole
bunch
of
operations
on
ranges
of
values,
batches
of
operations,
various
kinds
of
scans
and
and
so
forth.
A
A
And
but
without
the
the
sql
layer
which
is
provided
elsewhere,
so
tag
av
by
itself
or
as
part
of
tai
db,
is
used
by
a
fairly
large
community
of
of
users.
Some
of
those
are
huge
so,
for
example,
jd
cloud
shoppie,
maitlan,
jihoo,
so
jihu
are
happy
for
us
to
talk
about
some
numbers.
The
last
time
we
got
some
numbers
from
them.
A
A
Cluster
and
the
the
biggest
cost
that
we
know
about
is
around
1600
nodes.
So
when
we
say
that
tycho
v
scales
horizontally,
it
can
scale
pretty
big
tankavi
is
being
actively
developed
at
the
moment,
so
we're
working
on
performance
in
particular.
So
recently
we
made
some
fairly
major
changes
to
our
transaction
protocol
that
have
resulted
as
big
improvements
to
read
and
write,
latency
we're
implementing
new
features.
A
So
the
co-processor
is
a
component
of
tank
av
which
lets
us
run
computation
on
what
would
otherwise
just
be
data
storage
nodes.
So
this
came
from
tai
db.
Where,
for
things
like
aggregation
operations,
you
can
get
a
huge
performance
win
by
running
part
of
those
operations
on
the
nodes
where
the
data
is
actually
stored.
A
A
A
synonym
for
for
shard
in
this
case,
so
you
have
the
the
data,
is
sharded
into
regions
and
then
the
regions
are
spread
across
nodes
using
raft
which
ensures
kind
of
reliable
replication
of
the
data,
and
the
the
client
is
a
somewhat
active
participants
in
in
interactions
with
with
attacker
v
clusters.
I'll
I'll
show
on
your
next
slide
and
the
client
communicates
with
the
the
tag
av
nodes
via
grpc.
A
B
A
So
here
is
that
transaction
protocol
and
actually
takeover
supports
various
flavors
of
transaction
protocols.
This
is
just
one
example
just
to
give
you
an
idea
of.
What's
going
on
now
when
I
say
that
the
client
is
kind
of
an
active
participant.
That's
because
the
the
transaction
protocol
is
a
collaborative
protocol
so
that
the
client
is
responsible
for
its
end
of
the
protocol,
and
it's
only
if
both
the
client
and
the
the
tyco
v
nodes
all
kind
of
do
what
they're
meant
to
that.
A
A
It
starts
the
pre-write
phase
and
in
the
pre-write
phase
the
the
client
contacts,
all
the
the
tag
av
nodes
that
it
wants
to
write
to
and
make
sure
that
the
the
transaction
can
be
can
be
written.
And
at
the
end
of
this
phase
it's
basically
got
a
guarantee
from
every
node
that
if
it
commits
then
that
commit
will
succeed.
And
so
at
this
point
once
it's
got
all
those
pre-write
responses.
A
Then
the
the
client
knows
that
the
the
commit
is
guaranteed
to
succeed
at
that
point
has
to
get
another
time
stamp
from
pd
and
then
it
can
finally
send
its
commit
message,
and
it
only
has
to
send
that
to
one
node,
the
the
primary
node
of
the
transaction,
and
then
it
can
return
success
to
the
user
and
later
on.
In
its
own
time,
it's
going
to
commit
all
the
all
the
other
nodes
in
the
in
the
transaction.
A
A
This
kind
of
detail
here
is
just
so
you
can
see
that
the
the
interaction
between
kind
of
the
client
and
the
thai
kv
server
is
non-trivial.
Okay.
This
is
not
just
like
the
interface
to
a
dictionary,
like
andy,
showed
at
the
start
of
the
talk,
it's
a
bit
more
complicated
than
that
because
we're
in
this
distributed
situation
and
just
to
really
kind
of
make
this
point.
This
is
the
protocol
buffer
specification
for
a
pre-write
request,
which
is
something
the
client
would
send
to
a
thai
kv.
A
Node
and
there's,
as
you
can
see,
there's
a
lot
of
detail
there,
and
so
in
the
in
the
next
section.
We're
gonna
see
how
the
the
the
tag
ib
client
kind
of
abstracts,
all
all
that
away
for
you
and
just
gives
you
a
a
much
nicer
api
so
that
you
can
concentrate
in
your
application
on
the
business
logic
and
avoid
all
this
lower
level,
logic
of
transaction
protocols
and
stamps
and
regions,
and
so
on
and
so
forth.
So
I'm
gonna
hand
over
to
andy
now.
B
B
B
Different
interface
row
kv
and
transaction,
they
cannot
be
used
together,
so
you
need
to
choose
one
of
them
in
the
beginning,
no
kv
is
usually
fast.
Has
less
overhead
than
transaction
therefore
performs
in
lower
latency
still
raw
kb
guarantees.
Drone
consistency,
but
only
supports
single
raw
transaction
transaction
interface.
Is
an
extra
interior
in
transaction
interface,
adds
an
extra
interior
mvcc
layer
on
top
of
rkv.
B
B
Then
we
can
use
game
methods
with
k1,
then
use
gate
method
with
k1
will
return
value
v1,
which
we
insert
before
scan
method
returns
the
list
of
key
value
pairs
by
the
key
range
from
k1.
To
the
end,
the
scan
limit
is
required
to
be
explicitly
dated
to
avoid
flooding
the
titan
server
with
an
unexpectedly
large
result.
B
In
transaction
modes,
the
methods
in
the
client
class
is
moved
into
the
new
transaction
class
by
calling
between
optimistic
or
becoming
pessimistic
on
the
current,
a
new
transaction
will
be
created,
transaction
has
actual
methods
like
get
volume,
update
and
lost
keys,
which
is
the
features
for
pessimistic
transaction,
and
it
is
a
little
bit
out
of
the
topic
of
the
days.
So
you
can
find
a
description
in
the
document
of
ros
client.
A
That's
the
the
end
of
the
talk,
so
thanks
for
for
your
attention
just
to
quickly
go
over
what
we
talked
about.
We
we
talked
about
how
to
choose
a
key
value
store.
We
covered
roxdb
readers
mongodb
and
then
we
dived
a
tiny
bit
deeper
into
into
tai
kv,
which
is
a
distributed
and
transactional
key
value
store,
and
then
we
talked
about
how
your
application
would
communicate
with
takeov
using
the
either
the
the
raw
or
the
transactional
interfaces
of
taikv
via
the
python
client.