Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2018 (Shanghai), 21 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Running Vitess on Kubernetes at Massive Scale: JD.com Case Study - Jiten Vaidya & Xin Lv

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Running Vitess on Kubernetes at Massive Scale: JD.com Case Study - Jiten Vaidya, PlanetScale & Xin Lv, JD.com

Vitess was one of the first cloud-ready relational database systems. It served massive qps at Youtube on Borg - Google’s internal cloud orchestration software, and the precursor to Kubernetes. It has been 8 years since Vitess first served traffic at Youtube, and now many companies around the world depend on Vitess to serve production data at scale. Vitess is used on premise, in the cloud, on bare-metal or on orchestration frameworks like Kubernetes. PlanetScale’s co-founder & CEO, Jiten Vaidya, will present the Vitess features that make it a natural for cloud orchestration environments such as Kubernetes. Xin Lv from JD.com will present JD’s work on using and enhancing Vitess on JD's e-commerce platform. Specifically, he will show how JD uses Vitess to manage large volumes of complex transitional data to serve massive qps on one of the largest Kubernetes platforms in the world.

To learn more click here: https://sched.co/FuKe

A

Okay, it's 2:20. Let's start my name is jet Nvidia and thank you for coming here this lovely afternoon, my first time here in Shanghai everything I had heard about this city. It surpasses it it's a fantastic city, thanks for hosting us. Today we are going to talk about running Vitesse on kubernetes at massive scale. Half the talk would be by me and half the talk. Talk would be by shin from JD dot-com, so without further ado, let's jump into it.

A

First, a very little about planet scale, which is the company that I am the CEO of. It was founded early this year by me and subbu sukumaran, who is the wittich Vitesse community leader to support wit s. We do four things. We support open source, we test. We do some custom wit as development. If you need it, we have a kubernetes based deployment manager, which is a which is based on the operator SDK, and we are building a cross cloud database as a service set up on on top of wit s.

A

So we have a booth come by ask questions. If you have any.

A

Let's jump into the witness architecture, but before we start talking about MIT s, maybe I'll give you a little bit of a background about me. I worked at YouTube from 2007 to 2012 I joined Google in 2005, when Google acquired, YouTube sugu came into YouTube, with with that acquisition and sugu and I knew have known each other. For about 25 years, we used to work together at a company called Informix software. Some of you might know of it. It was a relational database company that had products on early unix systems, so we work together there.

A

He knew me I was at Google. He said hey, why? Don't you come in and work at YouTube? So I was one of the first two engineers should transfer from google to youtube and.

A

Sugu and I have faced the database challenges that youtube had together in around 2010 sugu and another engineer at YouTube, Mike Solomon. He was actually engineer number one at YouTube. They both decided that we were having too many database problems and rather than just mopping up the floor, they were gonna fix the leak they were going to come up with design principles that would allow them us to leapfrog. My sequel scaling problems that YouTube was having and come up with a system that will scale as YouTube's traffic scales and that's how virtus was born now.

A

Witness actually has been open source right from its inception, because both Mike and Sue had faced similar problems at PayPal. They had worked at PayPal before and they were facing them now at YouTube and they thought that five 10 years down the line they were going to face at other places and other places will must be facing similar problems. So it would be good if what we are developing is open source. So, right from the beginning, right from its inception, Vitesse has always been open source. It was actually all the bug fixes feature.

A

Implementations were done in open source. That code was brought back into YouTube, compiled and deployed because of that witness has some really nice properties for an open source system to run well in Google infrastructure, for example, Google has a lock server called chubby, so we needed to support that inside inside YouTube and that's why it was developed behind interfaces and that's how we can support sue keeper, its city or council in open source outside right.

A

So that's a little bit of a background about what Vitas is for those of you who absolutely don't know what witness is it's basically a sharding middleware that sits on top of your my sequel databases and creates a gives a view to your app that it's talking to one monolithic databases database without the application being aware of any shouting logic, how does witness achieve that? So, let's jump into the architecture and let's talk about the Vitesse architecture, a little bit.

A

Everything on the left hand, side is the app server and everything on the right hand, side is the other witness components of the Greenline right.

A

Vt gate is the stateless proxy that has a full. My sequel parcel built into it your your app server can talk either using the my sequel protocol to it or using G RPC protocol. Initially, we used to support only G RPC, but now we fully support the my sequel binary protocol, so you, as far as your app servers, know it's talking to a my sequel, D, a my sequel server. How does VT gate figure out where the queries go? How does the VT gate give your application the view that is talking to one large database?

A

What it does is that it parses your query. It looks at the where clause it sees what value there is in the where clause and, depending on the actual value of that column, it figures out which char the query should go to. So all that happens in weeding it, and then the queries go to shards.

A

Now, let's go before we look at the actual where the data is stored, which would be inside inside a shard before we go to the components there. Let's talk a little bit about VTC, TLD, vtc TLD is the control control panel. It exports a web UI and you can execute administrative functions of Vitesse through web UI on vtc TLD. It also exposes a gr pc-based api endpoint and there are command-line tools that wit is toolkit.

A

Has that talk to et cetera over G RPC, and you can perform the same functions that you would perform using the UI through et cetera, also through command line? Actually, that's. What is supported to the command line is a superset of what is supported through the web UI. Then we come to the lock server component. The reason that we have called it lock server is because we support its CD. We support Council and we support zookeeper as I originally mentioned, but basically that's just the metadata store you.

A

The lock server is not used during the query path at all, but only time you need to use the lock server is, if you add, a key space, if you add a shark to the key space or if you add a tablet to a shard right so anytime, your topology changes, you need to use the lock server anytime. You are adding new sharding information, you need. You know you need lock server, but for your query path, the lock server is not used at all more on that as we as we go forward.

A

Now we come to the component where the data is stored. So what is represented here is a typical, my sequel, cluster, with one master and pour replicas.

A

That master is replicating to those replicas using my sequel, binary logs, witness doesn't get you to the middle of normal replication right. So that's a your familiar, my sequel cluster there. But what is happening is that every my security gets a minder process called BT tablet. I mean and VT tablet is in charge of sort of two things: it's in charge of figuring out, replication and figuring out, resetting master-slave, replication and so on, and it's also responsible for sending the queries that it gets from VT gate to the my security.

A

There is also a full parser built in VT tablet and VT tablet does a lot of interesting things to keep your my security running safely. As you know, it's pretty easy for somebody to write a bad query which does a full table scan using a column that is not indexed and it's really bad in a world EP transaction LTP database. If somebody does that, so VT tablet actually adds it's on all of your queries.

A

It also puts timeouts on your transaction as well as your selects, so any long-running query is killed either a transaction auto select so anything that a conscientious DBA would do to keep the mice equality running well. Rita tablet, basically, does it for you every time there was a YouTube outage, a new feature was added to video tablet. It's pretty solid it.

A

We, my sequel, really runs a lot better underneath Vitesse than it runs by itself, and there are some companies who actually use it as not for its charging capability, but just for running running witness running your bicycle is better, so we take it since queries to get a tablet. / GL, PC VT tablet typically runs on the same node as a my security. They talk to each other using a UNIX socket, no need for them to run. On the same machine, they can also talk to each other using tcp/ip sockets.

A

My security actually does the job of query serving the query back to video tablet. Weighted-Average sends it to VD gate in case your query did not have a where clause at all Veta gate is capable of doing scatter gather and sending the results back to the back back to the app. So that is the Vitesse architecture. In short, the rest of the talk. What I am going to do is I am going to take each of these components and I am going to show how they're in under kubernetes.

A

So let's start with VT gate, so we ticket as I said, is the stateless proxy accepts connection to the my sequel compatible server. It also has a GRCC endpoint and a web UI for debug for debugging endpoints are for observability, so every server in the Vitesse system has export sir HTTP endpoint for observability.

A

As I said it computes target shards. It sends queries to VT tablets for the target headshots receives collates and serves the response back to the application. As I said, this is a stateless proxy.

A

So on the right, so each of these slide has this format in which, on the left-hand side, I describe what that component does. On the right-hand side, we I I, explain how it runs in kubernetes right, so we tickets can be created. You know what you can create more instances of we tickets as the load increases.

A

You can start nvt gates as a replica set basically and for co-located workloads. You can start one VT gate per node and expose it with a clip cluster IP. So that's our recommendation for running VT gate.

A

Let's go to the next component, which is the beauty Cpl d VT city early is the is the control plane as I explained, and you serve the web UI you can use up, you know operational commands, it also shows status and it also has a topology browser.

A

So if you want to find out which particular IP address and port a particular a master or a replica for a particular channel is serving, you can go to the topology browser and you can you can you can see that it also serves an API over G RPC used by VT CTL client to and most importantly, it also supports recharging workflows. So if you, when you want to start a recharging workflow, that's where you will come and run it, we suggest that you start 1 or 2 VT catalytic processes per cell.

A

Just because again, this is not in the query in the query serving part at all. So not, but you should start one or two so that if you, if you need to do maintenance, there is another one. You start them as replica sets or deployments basically, and you expose them behind a service.

A

Let's go to the lock server, which typically in kubernetes is HCD, and this is this lets the Vitesse cluster. Together, it's the backing store for metadata. It's used for service discovery for topology and for storing V schema. Now we schema is the information that allows you to figure out how how the shouting works for a particular table.

A

It's not useful query serving it's needed for any change in topology, add key space, add shard, add tablet and change master. We recommend that you have one global cluster and one cluster per cell cell is a failure domain. You probably you would want one in u.s. East and one in u.s. West and one global cluster straddling both so Vitesse is very cognizant of multi data center topologies and supports it natively very well.

A

At YouTube, we used to run about 80 to 120 replicas per Master across 20 data centers all around the world, because YouTube is very write, read intensive traffic again, you expose the H citty cluster behind the service.

A

Each tablet is a beauty tablet and my sequel, D combination.

A

It requires a unique ID in the Vitesse cluster and they can be of type masters replicas and already only tablets of type replicas can be appropriate, promoted to master, and they typically should have low replicas lag.

A

We suggest that you start both containers in the same, and you start them as two containers in the same pod, they communicate with each other over a unique socket. It should be clear to the you know: you both container should mount a shared volume and that's where you create the socket and we used. We suggest that you use local persistent volumes for data. We have found that has pretty good properties and.

A

We also suggest that you use one master and two replicas with semi sync replication turned on for high availability and use anti affinity to make sure that replicas are not Kolok located with our other members of the Sharma okay, so I am at 235, so I'm going to really run through these slides quickly. This is a quick description of how.what certificates and authentication secrets, etc. Are needed for running a fully secure, vertes deployment and we use kubernetes secrets to implement them and round them in parts for high availability.

A

Witness supports planned free parent out of the box, it's coordinated via the lock. Several existing transactions are allowed to complete and so on point being that your application doesn't even know that a reparent has happened. Application does not see any errors we to get buffers your new transactions, while the repellent is going on for unplanned reparent.

A

We use Orchestrator a shin, we'll talk more about the limitations that they face at their humungous scale with this, and we do resulting by consuming the binary logs and splitting the binary log, applying the sharing logic to it and splitting it so that we write to multiple masters. We also support multiple sales, I said, and we also support with witness cluster spanning multiple kubernetes clusters. So that's my bit and I will let Shin come and talk to us about how they're deploying with us in jelly.

B

The interpreter can does not cannot listen to him.

C

As matter of fact, JT comm is one of the biggest retailer in China, both online and offline. We have around 300 million active users and we are ranked 181 in global focus. We have one of the largest logistic infrastructure within China, and 90% of all orders can be shipped out within two days and basically, we have our competitive partners, including T, sent Walmart and Google. Before we use fighter.

C

We have some issues within our database. That includes seeing a few layers first of all its application, because if we database is too big, we need application to reroute and it needs to bank on the application to do query for robustness. Actually, we have no anti affinity.

C

As for the resource, yet we have these pre allocated resources and that the resource usage is low. We can maintain that, as for OPI to QBs we're not we're not able to increase the serviceability of our DB. The reason why we use VTS I believe the previous speaker has talked about this. That is because of its advantages and including online split.

C

So you'll, if you have a massive amount of service or QBs, is too high in your device and your front end cannot support it. Then we are able to.

C

Increase our container efficiency, while not impacting our service provider and the other function, is routing or rerouting and to secure our service. It's stable and it's compatible with my sequel and it supports basically all the sequel, language and GB shriver.

C

In the last point. The reason why JD comm is able to deploy Vitesse in such a big skill. That is because many years ago, JG comm is start running container and we have have been using kubernetes for years. That is the reason why we can integrate container cuba, natives and feeties.

C

We started to use fittest in the beginning of the 2017 and before that we used a proxy. That is a self engineered and developed sequence, but there are some limiters, including we cannot increase our containing proficiency.

C

We have all so many of our kernel system is running above VTS, including our older system. It's all running paste on VTS and we have. There are five layers. There are five areas in JD comm. We have realized that deployment and spreading all across China and everything is supported online today, including Indonesia and Thailand. All the DB, oh, the DB of JD dot-com, is running above all, based on BTS.

A

C

Probably are running the most complicated in a larger scale of V, T's and based on classifications. We cannot offer too much details. We have more than a thousand key space. If you understand it is that one key space equals one business unit application in the front end, and we have these eight data centers, that's in Beijing, Guangzhou and suchan. We have these five largest data centers. As for shots, we have about 10,000 shots, so you can make a brief calculation that JD that counts, shot.

C

It is basically made for increase the usability and the efficiency of our resources.

C

We have millions of charts, the largest chart is 128, the largest key space had 128 charts and that the data amount is around 200, something TV and we have trillion and we have billions data flows.

C

So this we have this b2b price shipping logistics services is all based on the bank and Vitas, and our increase rate is around 10 per week and that's 10, something T.

B

C

Talked about the VG's.

C

If we only offers, we've only offers VG's, it cannot be deploy within the most scenarios for companies, because you need to have this front and usage of apps and Vitti's is just for building a psychology system that is after deploying Vitas in a company. We need to think about. How can our customers very easily be connected within our system and then TB when DB my grass VTS?

C

We are not going to disconnect with the traditional database, so the migrating issue is very important, while not stopping the online service, an operation and that's the upper stream and the downstream is that it's all based on the B lock of my sequel, including analysis and warning system.

C

So we need to based on the B, lock managed by the Vitas, and we have developed this cluster service it based on it's based on affinities,.

C

If we have a master flow over, we can dynamically do a cross over of the original resource and to update it instantly, and we can realize a overall collecting service of B log so that we can have a combination, better combination, and we have a better migration and integration between VTS. My sequel and be locked.

C

Within the past year, cheetahs and sugu steam is has always been helping us to optimize our feeties problem within our system, and we have given some response, including bug fix, including improving the grammar, including the grammars of multi Curie, begin camaro rock back support.

C

Also, there are some other grammar errors as well, and we have done some job on the compatibility of the my sequel.

C

Some sometimes are in my sequel protocols and javis gbt. They have the compatibility problem and there a lot of apps are written by PHP. So the compatibility and we've made a lot of effort on improving that for the performance for increasing the performance.

C

Because we have this large amount of data within JD comm, one example is that in one shot we have more than one TBS of data amount. So when you are scheduling the shot, you need a lot of time to do that. So, for this kind of scenario, we have done some optimization, including the copying of beep, B, lock and also we have responsed and we have take this to sugu, and so, when they're developing the new function of TP, we will want them to fix this to us.

C

Also through tuning GC, we can have a better performance.

C

Full read-only uses.

C

We have also made some optimizations when doing queries, including streaming on the back. We have all done some optimization for that.

C

So we have these four different ecological system on BET's, including the JD transfer plan, link, data, access, subnet and managed system.

C

So we have based on KITT- and we have this audit plug-ins for TB for the sensitive dps and user data, as we will secure it down, and these data will be sent to the GN q of GG comm. We can do a later audit or hurry to that including jail. Comm has made a management system is called DPS based on Vitas.

C

It can offer TB service.

C

We have a highly requirements for our resource usage rate, so we have made a lot of customized optimization, including sugu, has mentioned one part fold to container and my security.

C

But if you have this, you have enough resource. You can run this but JD comm. We have this isolation of resource, so the resource usage rate will not be too high. So the we need to put my sequel, my CD and tablets. We need to put that into one container.

C

So that all the Vitesse container we have this overclocked of CPUs about 5 times overclocked.

C

So one shot has one master, one replication and one read-only.

B

C

For elastic scaling, we have this local instant capacity expansion spit with one action, because we have a very complicated online environment. So sometimes this field and if it fails, we go back to our original solution and we were offering this v2 service. We need to consider its downside that if we put one thing in one rack or one eye, we put that into one rack or one IDC, to increase its compatibility.

C

The official suggestion of VTS is that is suggested to use in a BD, but in our company we have this backup database and requires high usage of resource, so it supports and rocks DB and the other database when using D B's. We have facing some multiple engines and splittings.

D

Has mentioned the database run by Gd JD. We have actually a large amount of data, so we need to do a lot of optimization because we don't want the burden of our database to be overloaded and also to lead. Also, we don't end want any interruptions and we want to control the size of our database and also to make sure that it's stable and have these splitter solution.

D

Splitting so I think a lot of people here in China note that Teemo and the Jade or JD and re that we have a lot of promotions activities they're both on the 18th of June and the 11th of November. You know we have actually a large amount of transactions on those two dates and we need to respond quickly and to scale up our database, maybe even like tenfold or 20 times their size of our usual capacity.

D

So our developers have had these online scaling up of our CPU with just clicking one button that we can realize these function.

D

So also the metadata part there, because we, when we use the actually, these metadata, will be stored in one database and because we have a large load of video CD. And so you need to have reload to these, like consequent, like constantly so like. We even have om at some extremes, but these were not interrupt the current services. But now, if we do regarding or over data case, that those will have, we have will encounter some problems and how to address those issues, because we have a lot of transactions, and so we don't need.

D

We didn't have much time. So we didn't risk a risk information on the database and once it's changed to order a detour, the watched and and once they got the URL and you can they can download. Then that made a data, and this is how we solved the the problem and the recursive this framework and the visca ma the information. So we used to space as a key and all those can be split up.

D

No, so the orchestrator, because we have five UCS and more than twenty cells and each one of the cell has are straighter. So why do we need a single Autotrader to brown, the cell so because Orchestrator, when they brown the when they will use that because it's not--, we have a large amount of load and all those of you will be in service, and we at that time we use orchestration model when the load is too high and the service capacity will lower down.

D

So at that time we were strictly control the size of the cell and the instinct of the size and the restrict way to you, Orchestrator so controlled within five thousand. So if we have something like it's more than five thousand, we have a new cell for and a new oxidizer and also in the future. We will work close where the Vitis to address three issues.

D

The first is the resetting isolation, because now we're resetting is based on Devi, witness witness, and now, if we have like several hundreds of rerouting so maybe SEC several hundred stuff of them are ok, then we only need to do a 10 or 20 of the reassuring. So we need to find these a solution for our resetting and also V schema, because now this schema, the content is currently stored in one value.

D

So we will have this restructure of our V schemer and in the future we will work with vintners and also in terms of the auto balance, will have this misako, auto monitoring system. So when we have overload- and these will be automatically transferred to another database, so we have this lines, so you can visit our exhibition hall to see the detail of our automated thesis or I have to say. Thank you.