Delta Lake Last Week in a Byte, 28 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Last Week in a Byte (2023-03-28)

Description

Learn about the latest #deltalake news a week late! This quick byte includes:
- We’re excited about the #DeltaLake 2.3.0 Preview on #ApacheSpark 3.3, For more information, check out https://github.com/delta-io/delta/releases/tag/v2.3.0rc1

- Matthew Powers, CFA and Miguel Angel Diaz Rodriguez, MLOps Director at AB InBev presented Model Serving with MLflow and Delta Lake main features at Apache Spark Bogata meetup

- We recently published Delta Lake D3L2 vidcast Implementing a Data Lakehouse for Improved Data Science and Analytics at T-Mobile with Robert Thompson and Geoff Freeman, Members of Technical Staff at T-Mobile

- If you're in the Seattle area in April, there are two #pydata events:
PyData 2023 Preconference Meetup PARTY!: https://www.meetup.com/seattle-spark-meetup/events/291903857/

PyData Seattle 2023
https://pydata.org/seattle2023/

- Jacek Laskowski's is adding more to his Internals of Delta Lake book, calling out #DeltaLake 2.2 metadata operations like LIMIT

- Nick Karpov published Z-Order: Visualization and Implementation, exploring the Delta Lake Spark connector’s Z-Order command through visualization and implementation. #zorder #apachespark

- Denny Lee's first blog as part of the “Ask Delta?” blog series is Why does altering a Delta Lake table schema not show up in the Spark DataFrame? #schema #apachespark

- Matthew Powers, CFA published How to Convert from CSV to Delta Lake #csv

- Jim Hibbard published Running ML Workflows with Delta Lake and Ray (Part 1) #rayml

A

A

Welcome to the March 28th edition of last week in a bite for your latest news on Delta Lake.

A

We're excited for the Delta Lake 2.3.0 preview on Apache spark 3.3 with key features, including zero copy convert to Delta from Iceberg tables, support shallow clones for Delta, parquet and Iceberg tables; item code, rights for DML operations, support reading change, data feeds in SQL queries and support, reading, Delta tables with deletion, vectors and much much more.

A

We want to color some really cool events that happened last week, Matthew powers and Miguel Angel, Diaz Rodriguez had went ahead and presented a model serving with mlflow and Delta Lake main features at the Apache spark Bogota Meetup.

A

Meanwhile, we recently published the Delta Lake D3 to L2 vidcast, implementing a data lake house for improved data science and analytics at TMO with Robert Thompson and Jeff Freeman members of technol staff at T-Mobile, which, including some interesting learnings of how the lake house allowed them to go from 100 Towers a month to 1500 a week when Teen Wolf started building their 5G Network.

A

If you're in the Seattle area in April, there are two awesome: Pi data events on April 25th, there's the Seattle spark and AI Meetup featuring uh introduction to spark connect with changing and jokjin, and also there's a spark Ray mlflow and Delta Lake panel, with eloisa trend from PI data, Jules damgy from any scale and myself on April 26 to 28th.

A

There is the pi data Seattle 2023 conference, featuring The Following sessions, building reliable open lake houses with Delta lake with Jim Hibbert and Nick karpov and ml Ops deployment patterns with Delta, Lake and ammo flow with Mary, Grace, mesta and finally, but certainly not the least.

A

We have some awesome recent Publications from from yasuk lakowski, who uh updated his internals of Delta Lake book, uh which included Delta, Lake, 2.2 and metadata metadata operations like limit karpov, published his Z order, visualization implementation, which explores how Delta Lake Sparks connector for the Z order command how it operates both both through visualizations and how it's implemented.

A

I also published my own blog on the first blog of the ask Delta blog series: why does altering a Delta lake table schema does not show up in the spark data frame mad Powers published how to convert from CSV to Delta, Lake and Jim Hebert published writing ml workflows with Delta, Lake and Rey part one, and with that thanks very much. uh We hope to see you next week.