Delta Lake Last Week in a Byte, 11 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Last Week in a Byte (2023-04-11)

Description

Learn about the latest #deltalake news a week late! This quick byte includes:
- We're excited about the release of Delta Lake 2.3 with all of its amazing features, check out https://github.com/delta-io/delta/releases/tag/v2.3.0
#deltalake #apachespark

- A shout out to Will Girten for his great #deltasharing tips including how you can use Delta Sharing for #streaming https://www.linkedin.com/posts/willgirten_deltasharing-opensource-activity-7051192801446744064-8dYH

- A shout to #finos #legend project where Delta Lake has been included as an integration: https://legend.finos.org/docs/community/external-integrations

- Like Delta Users #slack but wish we had an archive and it's searchable? Check out https://linen.delta.io

- A great blog by Nick Karpov on Support for Delta Lake Tables in #AWS #Lambda https://delta.io/blog/2023-04-06-deltalake-aws-lambda-wrangler-pandas/

A

Thank you. Welcome to the April 11th edition of last week in a bite. We've got a pack session, so we're going to start it off really happy for the release of Delta spark 2.3, there's a lot of really good features, including zero copy convert to delt from to Delta from Iceberg.

A

If I convert to Delta shallow clone for Delta, parquet and Iceberg tables, I didn't put them right for DML operations, improving merge when you're using the when not match by Source reading change data, feed, CDF in SQL queries, improve, read and write and performance in S3 uh recording vacuum operations in the lace transaction log, which is a very popular ask, and a really cool one, one of my favorites in fact, reading Delta tables with deletion, vectors, which seriously speeds up the performance of your Delta tables.

A

In addition to all that, we actually have a great call up by Will, gurton I'd love, you all to follow him on LinkedIn. He often posts some really good information on the latest information for Delta sharing and so in this case he's showing a really quick way for you to Showcase streaming operations from Delta sharing so check it out really happy to have him constantly advocating for the community.

A

Now one of the cool things that I like talking about is that Delta lake is part of the Linux foundation and it really helps for us to continue building community. In fact, if you join us at the open source, Summit North, America and Vancouver in uh mid-may uh there's a session that Carly ackerly and myself are going to uh to talk about Delta Lake and how Linux Foundation really helps us build the communities. One other cool project is Linux foundation's Finos, Financial OS.

A

The legend project Legend now also call has a shout out for Delta Lake as being part of one of its Integrations and, of course, we're going to be doing the same as well.

A

Another really cool thing that we just added is a linen.dev. It is a basically an archive for our slack.

A

So that way, all of our slack messages are now Google searchable and archivable, so you can always refer to them, and so it's really helpful just go to linen.o to check out our slack messages of the archive that is and finally last but certainly not least, I wanted to do a shout out to Nick karpov he's written a wonderful blog, called Introducing support for Delta Lake tables in adios Lambda, the little code snippet you see there, of course, that's directly covering his blog showcases how to go ahead and use adios lambdas to read and write from Delta tables.

A

So that's it for today's April 11th edition of last week in a bite. Hope you have a great week and see you next week.