Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: TiFlash: Make TiKV 10x Faster and HTAP-able - Xiaoyu Ma & Liquan Pei, PingCAP

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

TiFlash: Make TiKV 10x Faster and HTAP-able - Xiaoyu Ma & Liquan Pei, PingCAP

HTAP is a term introduced by Gartner, describing the capability processing both transactional and analytical workload. It is hard to deal with both workload seamlessly in one platform since the storage format of the two is totally different and workload interference is a big issue. This session introduces a new component for TiKV project called TiFlash. It uses a novel way to solve the problem stated above: raft based HTAP architecture. This design makes TiKV 10x faster on analytical workload and provided strong workload isolation strategy.

https://sched.co/ekFK

A

Hello: everyone, today's topic is make thai tv 10 times faster and established. This topic is brought by saryama and lee tranpay, who are from pinkham.

A

So why to make thai kb edge tappable? Consider if you want to analyze online data in real time.

A

Usually we use different types of database for transactional processing and analytical processing workloads, because these workloads are totally different so for the database the design goal is totally different.

A

That means we use two different composite different type of the system, different type of database, and we need to move data from tpu to ap constantly by etl.

A

The data movement itself is very expensive, slow and hard to maintain. So usually, people set chrome tab, job constantly, maybe like one hour or like a day.

A

And the job is responsible for moving data between different systems, and this is not very simple. You need to constantly watch these watch these job to preventing failures or you to check data if it is consistent during the movie and that becomes especially hard if the data volume is very big.

A

So it's not a question, only a question of complexity.

A

That also means when you get a report that report might be based on data. From yesterday, you lost the data freshness.

A

So that means, if you want to read the fresh data or you want to eat a fresh fruit, the best way is to directly consume it from the source.

A

So why system is separated? Why we cannot combine two systems into one? Then we can dilate the moving data processing and directly read the data from the very beginning.

A

So that's because of different database for different goals that use different design, especially the storage formats. So for transaction processing system we usually use a row format. So what is a row format? Row format means when you store a single each row.

A

You store the column side by side.

A

Consider this kind of query select star from emp, where id equals to 7658 for a row format system. You can directly seek to the start of the row and do a sequential read and you get all the row all the row. Data from that row.

A

So that means row formats is very good for this kind of workloads. You read very few rows and you process them and store it back. So that's the typical pattern for a transactional processing system.

A

Let's consider another example: select average age from emp, so that looks like a reporting query. You want to read: they want to analyze the average age for your employee.

A

For that kind of query, the column format is more suitable than a row format. So what is a column format? A column format is a way to store your data that for each individual column, you store it tightly side by side instead of in a row, bias.

A

So that means, when we answer that query for average age you just need to directly seek to the start of the age column and do a sequential reads. Then you get all the data for age.

A

Then you do a aggregation and answer the query. You don't need to touch any data from the columns that you don't need now. You can see that's very efficient way to read our data than the row format.

A

But if you want to answer a query, like a point, query like where id equals to 7658, you just need to pick up a single row for a column format. You need to seek three times and to find each individual column rows of each individual column, data for that a single row and to combine it together. Then you can have the row.

A

So that means the column. Format is not very suitable for the transaction processing.

A

The other problem need to consider is the workload interference for tp workloads. They might need to be very stable, stable, low, latency and very high very high transactional rate rates, both for a reporting query. The qps might be very low, but they consume a large amount of their compute resource.

A

So that means, if you process those two kinds of workloads in a single single system, the reporting or the analytical workloads might largely interfere. Your transactional workloads and the transaction workloads is very fragile. You don't want this happen.

A

So how to make thai tv edge typical.

A

In the new version 4.0, we introduced a new component named typeflash into thai kv system. So what.

B

Is a tie, flash.

A

Type flash is a real-time updatable column, storage engine, so the code base is partially based on click house. The click house is a very famous open source project.

A

It's been built from the yandex and we think data as a learner role from thai tv by raft, as some of you might know, that we use raft as a consensus algorithm to replicate the data in thai kv system.

A

So in thai kb we have different replications for each piece of data and these replications are maintained by raft and some of the replication are leader, replication and some are follower replication and for tie flash replication. It's a learner role. That means the tie, flash replication will not vote and that we're preventing time flash to interfere. The stableness of tank tv system.

A

So the two-story engine together make thai db system a htap database and we can also access data via cpo. That means the tidbit optimizer is a cost-based optimizer and it can choose between a column formats or row formats from the based on the statistics, information.

A

It will choose a path that which can provide better performance or lower lower cost or lower resource consumption.

B

Thank you xiaoyu. Next, I'm going to talk about the architecture of idb.

B

This is architectural tidbit and we have tidbit as a computation layer and attack away as a storage layer.

B

The data in take away are divided into regions, and each region contains a contiguous key ranges.

B

Each region has multiple replicas and they are keeping consistent with the rough replication protocol.

B

What we add here is the latter part, which is a tie, flash cluster and there are multiple nodes inside the flash cluster.

B

You can see that here we add a dashed line between the teflon nodes and take away nodes, which means that the data replication in take away will not be affected. If one or more flash nodes are done.

B

Next I'll talk about how we do real-time updatable cloud, columnar storage, we designed the columnar storage named delta tree, and the key idea is that we split the data by primary keys and into blocks the design goal of the dirt tray is to avoid multi-way merge when we scan in batch.

B

When new data arrives, the data is append to the dirt space to to optimize for the read performance. The dirt space is sorted and indexed periodically.

B

The dirt space is compacted into the stable states for better reader performance.

B

In this picture, we want to compare and show that why the third third dirt tray can offer better read performance compared with the lsm trees.

B

In the left-hand side, when you want to return range from lsm tree, you need to read data from all the levels in the lsm3 and do a multiple way merge and this operation is really very heavyweight and the read amplification is high in dirt 3. You only need to merge data from the stable space and the third space.

B

On top of that, we use a b plus 3 to index the segment information. In this case, if you want to return data for a certain reach, you only need to read data from a few segments.

B

During the read, the third space and the stable space are merged compared with the lsm3. This merge is a two-way merge which has a lower overhead compared with the multi-way merge.

B

Besides, the stable stage is starting columnified, which offers much better read performance during scans.

B

Next, I'm going to talk about how do we achieve the rough based edge type?

B

We introduce a learner role to the rafter protocol. All the tech flash nodes are learner rules by learning rules. We mean that those nodes will not participate in the router leader reduction. No, they become part of the current during the data right.

B

As a result, the rising tech away does not need to wait for tech flush and take away works normally, even if tesla knows die, the replication between tacky away nodes and teflon nodes are direct replication and there's no intermediate channel between them.

B

This replication is very efficient and usually the latency between the is in milliseconds.

B

We also also rely on the automatic load balance and the fault tolerance features built in tidbit to make the data replication highly available.

B

This is also an illustration of what I just talked about in the last slides and compared the replication in kkv and tie flash with the etl. This is this is very efficient here already during etl.

B

We need to copy data to a staging area and then copy the data to the data warehouse which causes a lot of data to be copied.

B

Another thing people usually ask is: how do we achieve consistent rate.

B

Because replication from takeaway to tesla is asynchronous, we need to guarantee the consistency during the real time. How do we achieve this?

B

The idea is pretty simple and we use a learner rate algorithm and consult leader on the replication progress before we return the data to the client and this actually guarantees we can achieve a strongly consistent rate, for example, consider the case that at time t 0 there is a write to the tech every node and later at the time t 1 gives a rate to the test node.

B

How do we keep? How do we make sure that the data read at t1 is strongly consistent and it can return the data written at t0 during this?

B

The learner will first talk to the leader and ask the progress of the replication in the leader, and in this case it knows that the leader is ready at index four.

B

Before the learner returned to the client, it needs to wait until it replicates to index 4, and then it returns data to the client.

B

This, combined with the timestamp and vcc, you can actually achieve snapshot isolation in ter flash as well.

B

Next, I'm going to talk about performance here is an example of the benchmark result on the on-time data.

B

So on-time data is a data collected since the 1980s about the airplanes it considers it contains the data of the flight number and the start time and the landing time you can see from the result that the ty tachydb plus tesla offers a very good performance compared with the other solutions.

B

Some of them are built for analytics and big data processing. These queries are multi-dimensional. Analytic queries, whereas teddy b and teflash is good fit.

B

If you want to know more about the queries, look like you can look at read about the blog.

B

Here is an example of the performance improvement we see from an early adopter of tech flash it's an internet company called the xiaohungshu, it is a thaidb user and they have tidbit running production.

B

They selected like around 400 queries from their production system and then migrated them to typeflash.

B

From this graph. We can see there's a three to ten times. Speedup.

B

And we can see that for some long running queries, the speed up can be up to 20 times.

B

Also. I want to call out that this result is based on older version of tesla, and since then we made a lot of progresses and improvements on teleflash and you can. We can expect much better performance with the newer version of typeflash.

B