Delta Lake Last Week in a Byte, 14 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Last Week in a Byte (2023-03-14) - Pi day!

Description

Learn about the latest #deltalake news a week late! This quick byte includes:
- Our #deltalake contributor of the month is Gerhard Breukl!

- delta-rs 0.7.0 was released, which includes various enhancements, fixed bugs, and merged PRs. Check out New features in the Python deltalake 0.7.0 release of delta-rs for more information on the #python release cc #rustlang

- Brayan Jules Jacques published HIO: a library that provides elegant functions to manage HDFS filesystem and cloud object stores

- D3L2: Massive Data Processing in Adobe Experience Platform Using Delta Lake with Yeshwanth Vijayakumar, Senior Engineering Manager and Architect at Adobe discuss how the #data #lakehouse architecture at Adobe Experience Platform combines with the #realtime Customer Profile architecture to increase their #apachespark batch workload throughputs and reduce costs while maintaining functionality - with #deltalake.

- D3L2: The Journey Unifying Data Lake and Data Warehouse with Robert Kossendey at Claimsforce on their journey to unifying #datalakes and #datawarehouses cc Robert Kossendey, claimsforce
We recently had the fun London meetup Building reliable lakehouses with Delta Lake Primer and AMA with Simon Whiteley and Denny Lee

- Vítor Teixeira published Delta Lake — Automatic Schema Evolution and contributed PR 1645.

- Vítor Teixeira published Delta Lake: Keeping It Fast and Clean on how to improve your Delta tables’ performance.

- Brayan Jules Jacques published Jodie - Append Without Duplication which discusses Jodie (an open-source library to perform common #deltalake operations using #apachespark and #scala) focusing on deduplication functions.

- Bryan Cafferky published Understanding Delta File Logs - The Heart of #DeltaLake.

- TR Raveendra published a seven-part #DeltaLake tutorial series!

- Explore the full range of the merge command with Delta Lake Merge

- Want to know more on how to contribute to delta-spark? Check out Getting started contributing to Delta Lake Spark.

- Register for PyData Seattle 2023

A

A

Welcome to last week in a bite, so I want to start off by celebrating Happy pie day and in celebration. We want to do the foodie shout out to a local Seattle joint called Moto Seattle, where they have some great amazing and quirky Detroit pizzas. So if you want some more foodie suggestions, let me know and we'll keep on adding to it.

A

So the contributor of the month is Gerard. Brookel, he's a lead data engineer and Cloud solution architect at Pi, Cal he's been instrumental for the power bi and deltalay connector, so Gerard. Thank you very much for all of your amazing contributions.

A

We did want to call out the releases of Delta RS 0.7.0 and check out the new features in the python Delta Lake 0.70, release written by Will, Jones and Matthew powers as well. We want to call out to Braylon Jules Jacks for his publishing of the hio library that provides functions for managing hdfs and Cloud object stores.

A

Some recent events, including two D3 L2 sessions, one with the Adobe experience platform and another one with the journey from unifying data Lake and data warehouse with claims Force. So please check out these two sessions.

A

The latest Community blogs and videos include two by Vitor he published Delta Lake automatic schema Evolution and which includes a contribution of PR, 1645 and Delta Lake. Keep it fast and clean to improve your Delta Lakes table's performance, brilliant, who we just talked about also published Jody to append without duplication, I.E deduplication of Delta light data, uh Brian kafferke, published understanding, Delta file logs uh the heart of Delta Lake and TR, published an amazing seven part.

A

Delta Lake tutorial series that you really have to check out two Delta Lake guides by Nick karpov, one on Delta, Lake, merge and another one about getting started to contributing to Delta Lake spark. So if you want to contribute check out these guides today, it's last but certainly not least, a shout out to Pi data Seattle on April 26 to 28th, where Delta lake is a proud sponsor of the event. It's at the Microsoft center with a lot of the cool Pi data sessions.

A

So come join us there and we'll definitely join us Delta lake maintainers at the blueprint, Consulting Booth, where actually we're working together and that's it for today's session of last week. Invite hope to see you next week.

A

Thank you, foreign yeah.