Apache Cassandra Apache Cassandra World Party | July 2022, 20 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Improved Snapshots in Cassandra 4.1 | Apache Cassandra World Party 2022

Description

Presentation from the second Cassandra World Party on July 20, 2022. Learn more about Apache Cassandra at https://cassandra.apache.org/

Paulo Motta of Bloomberg presents “Improved Snapshots in Cassandra 4.1” - In this lightning talk, Apache Cassandra committer Paulo Motta will explore the recent improvements to snapshots in Cassandra 4.1, like auto snapshot cleanup and the refactoring of the legacy snapshot module.

About the Presenter
Paulo Motta is an Apache Cassandra committer and senior software engineer at Bloomberg.

Apache Cassandra is an open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Linear scalability and proven fault tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data. Get started: https://cassandra.apache.org/quickstart/

The Apache Cassandra World Party 2022 was a virtual, one-day celebration on Wednesday, July 20, 2022 in preparation for the launch of 4.0, the most stable database release ever shipped.

Join the Cassandra community: https://cassandra.apache.org/community/

#ApacheCassandra #database #NoSQL

A

Yep so I'll be talking about snapshots in cassandra today. uh This is the work I've done with other contributors in the past year that will be featured in cassandra 4.1 uh next slide. Please.

A

That's me back in cassandra summit, 2014 good old days today, I'll be talking about the snapshots. Snapshots are basically data, backups that you can take in cassandra. So when you issue a node to a snapshot command, you take a copy of your access tables and they are saved to to the disk via a hard link, and this is very useful command to take backups, but not only that and so uh yeah. In addition to taking manual backups there is this very underrated feature which is auto snapshot.

A

True and basically, whenever you drop a table or truncate a table by default, cassandra will take a snapshot of the data files.

A

This is very. This has saved many lives before like. If you do a command by mistake, you you drop a table in in the in production thinking it was dev, then cassandra will magically save the data files for you, so you can recover it. uh One problem with this is that these uh these snapshots, they might accumulate on disk, if you don't clear them up, and and so you need to clear them periodically by hand or create a tool to to clear those snapshots.

A

So I propose that adding a new configuration to cassandra, which is the auto snapshot ttl and that you you can set this configuration in this example- is 30 days and your your snapshots will be automatically cleared after this given detail.

A

This will allow clients to um the database will handle this instead of like handing handling via external tools. The way these work is that the db periodically checks for inspiredness snapshots and deletes them.

A

It's also possible to set a ttl via node 2 snapshots via the ttl flag, and this work was contributed uh via google summer of code, uh the student aboli palagashi mentored by myself and stefan miklozevich. So this was his work uh in in the last year that we did together and it's available available on cassandra 4.1.

A

So when testing the autosnop snapshot feature, I noticed that snapshots of dropped tables they were not showing up in the know to list snapshots, and this is a problem because I mean they they will be taking up disk space, but the operator will not be aware that the snapshots of dropped tables exist.

A

So we fixed this issue in in this ticket and now in cassandra 4.1.

A

When you list snapshots, it will also show snapshots of drop it table, so you can inspect them or clear them if you need, even though this looks like a simple fix, it required a relatively big refactoring to to decouple the snapshot module from the live, sstable data and another issue that we found when working on this snapshot was that the snapshot true size was not properly calculated.

A

So when you create the snapshot, they don't take up disk space when they are created because hard links are created to the live access tables. So a snapshot recently created will take no space because the the snapshot will reference the existing data files and when these data files expire or are deleted, then the snapshot will remain there.

A

The files and then the true size when the snapshot is created should be zero, but in this case it was wrongly calculated and now uh when in cassandra 4.1, when you release the snapshots, it will actually show that the true size that the snapshot is taking so size on disk is the is the size of all access tables uh of this of a given snapshot and the true size is the actual disks extra disk space that the snapshot is taking.

A

So in this case, as the snapshot was recently created, it will take zero bytes and as the live stats, the ss tables are deleted. Then this the true size should become equal to the size on disk.

A

And yeah this was a glimpse of of these fixes and improvements to snapshots in cassandra 4.1.

A

We hope you like it and that's it. Thank you. Everybody I want to thank, also, stefan and and a bully that worked with me on this effort.

A

Yeah, this is a really cool feature and I can't wait for 4.1 to come out. Yeah paulo. I appreciate it. Thank you very much. um This is this is a good preview and we already got a blog post from you. Didn't we.