Delta Lake Last Week in a Byte, 6 Jun 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Last Week in a Byte (2023-06-06)

Description

In this edition of Last Week in a Byte, we'll hear how Kubit uses Delta Sharing to power their product analytics platform, hear about an exciting new contribution to the Dask community, and 2 new releases from the Delta Lake and Delta Sharing projects!

A

Hello, everyone and welcome back to another edition of last week in a bite, your latest news and events for all things: Delta Lake. We have quite a few things that happened this past week. So a lot of Delta sharing we're gonna even hear from the Das Community this week. So let's jump into things.

A

First things: first, there was a really excellent article published on the Delta Lake blog this week by qubit, and, if you don't know, Cubit they're, basically a product analytics platform that was built to leverage data sharing Technologies and what I really like about this blog is that it shares how qubit uses Delta sharing to provide its customers with easy, fast and affordable access to massive amounts of product data and the tuning tips at the end were a really nice bonus. Things like using serverless, compute partition strategy or even optimizing.

A

The data files and I can hardly believe it, but I'm really excited to announce the release of Delta Lake 2.4 I feel like it was just a short time ago. I was getting to know all the features in 2.3, and here we are. We already have another release and just to mention a couple of the cool features in this release include support for Apache spark, 3.4 support for using magical deletion, vectors and write operations, a new handy, Purge command for removing dilution, vectors and adding a when not matched by Source clause. In your merge operations.

A

Another cool development this week comes this time from the Das Community. We're really excited to announce that the project Das Delta table was officially donated to the Das contributors organization and in case you missed it.

A

Matt Powers talked about this cool project during the Das demo day as a way to speed up queries from Das by reading data from Delta tables and, last but not least, we're excited to announce the minor release of Delta sharing 0.6.7, which adds a new Max, retry duration limit and socket timeout to the retry logic as well as refactor some of those Spark configs.

A

Well, that's all we have for this week. Folks, thanks for tuning in and don't forget to go out to go.delta.io forward, slash last week to subscribe to the Weekly Newsletter check us out in GitHub for the latest news and product releases and ping us out slack or LinkedIn with any of your questions or comments, see you all next week.

A