►
From YouTube: 2021-10-26 delta-rs open development meeting
Description
Discussing recent progress made in the project and some production use-cases
B
Oh
okay,
no!
No!
I
I
hadn't!
I
haven't
clicked
on
youtube,
yet
so,
okay,
just
to
try
to
delay
it
a
little
bit
that
was
also
yeah,
so
we
actually
have
an
agenda
for
today's
delta
rush
meeting.
I
know:
isn't.
D
C
C
C
So,
let's
start
on
the
the
new
crate
release.
I
I've
got
to
make.
C
Quick
because
I've
actually
had
a
conflicting
meeting
the
last
time
we
did
a
release.
We
basically
we've
got
a
release
branch,
I
think
still,
but
we
basically
cut
out
the
azure
stuff
qp.
I
guess
do
you
just
want
to
do
the
same
thing
again
and
just
like
I
can
prep
something
yeah.
C
I
think
I
do
have.
I
won't
commit
them
to
this,
but
I
do
have
a
somebody
may
have
whispered
that
they're,
pushing
the
team
responsible
for
the
azure
sdk
to
to
actually
cut
a
release
of
that
crate,
which
would
mean
we
would
get
an
actual
at
some
point.
Hopefully,
before
the
end
of
the
year,
maybe
fingers
crossed,
we
would
be
able
to
cut
a
delta
crate
with
with
azure
support.
A
C
But
I
know
that
there's
there's
people
poking
people
poking
whoever's
responsible
for
that
that
that
azure
sdk
for
rust,
which
is
promising
so
qp
we've
got-
we've
got
a
rust,
v4
0.4
x
branch.
Do
you
just
want
me
to
take?
C
You
know
just
take
what
we
have
in
main
and
just
create
a
new
branch
and
just
basically
replay
the
or
replay
quote,
unquote
replay
the
commits
that
we
did
to
comment
out
the
azure
stuff.
A
Yeah,
I
think
we
could
just
merge
to
the
release
branch.
We
shouldn't
have
any
conflict.
A
Right,
sorry,
I
mean
like
create
a
new
branch
based
off
that
it's
like
merged.
C
C
Yeah
so
I'll
go
ahead
and
get
a
ticket
found
zero
five,
that's
great,
and
then
I
might
enlist
your
help
and
maybe
misha's
help
to
get
a
change
log
sorted
for
that
yep,
because
I
think
we
should
actually
write
some
change
like
this
yeah.
A
One
question
I
have
is,
I
think,
danny
I
asked
about
this
before
as
well
it
we
should.
We
should
be
able
to
write
a
blog
post
about
this
on
on
the
official
data
website
right.
B
It
helps
when
I
unmute
my
cell
phone
and
to
answer
your
questions.
It
really
does
help
that
process.
I
was
trying
to
say
absolutely
so
as
a
quick
update,
we're
currently
re-updating
the
delta.io
website
itself,
actually,
and
so
with
that
it'll
actually
go
ahead
and
be
in
better
shape
for
us
to
go
ahead
and
put
official
blogs
directly
onto
the
site
itself
too,
also
yeah,
so
we're
currently
working
on
that
right
now.
B
I
should
have
an
update
for
all
of
us
by
the
end
like
near
near
thanksgiving
and
as
a
quick
call
out
that
the
it's
using
gatsby
framework
and
we
can
basically
post
everything
using
markdown
expressions,
so
it'll
be
just
a
nice
way
for
us
to
write
the
blogs,
pretty
cool,
yeah,
okay,
occasionally
occasionally,
I
do
something
that's
worthwhile,
so
this
is
one
one
one
that
few
times
that
I'm
actually,
I
think
I
might
have
done
something
decent,
so
yeah.
C
E
B
C
Let's,
let's
go
look
at
this
pull
request.
Qp
you
wanted
to
talk
about
it.
A
Yeah,
so
just
calling
some,
I
draw
some
attention
to
this
pull
request.
It's
a
going
to
be
a
major
optimization
for
delta
rs.
One
of
the
our
contributors
reported
that
we
are
actually
performing
worse
in
terms
in
terms
of
memory
usage
compared
to
the
spark.
A
So,
and
there
is
opportunity
for
us
to
reduce
the
overhead
by
10
200
x
if
we
structure
the
data
structure
correctly,
so
this
pr
is
the
first
step.
Well,
first,
try
on
restructuring
internals
of
delta
rs
to
go
into
a
more
column
format,
so
that
we
can
reduce
memory
usage
and
improve
the
performance
by
order
of
magnitude.
B
Excellent
thanks
very
much.
I
had
just
two
small
call
outs.
One
is
that
florian
had
just
pushed
in
this
third
one,
this
basically
delta
lake,
to
work
with
aws
athena
as
part
of
aws
labs.
So
I
just
wanted
to
call
that
out
that,
like
that
that
pr
just
went
in
it's
still
like
it
hasn't
been
accepted,
yet
it
hasn't
been
merged
yet,
but
the
call-out
was
like
the
the
process
has
started.
So
this
is
pretty
cool.
That
is
right.
D
B
Yeah
and
then
yeah
small
little
thing
I'll
go
ahead
and
create
a
pr
and
then
create
an
issue
about
the
go
long
references
since
we
we
probably
could
use
some
help
from
the
community
to
to
go,
build
that
so
yeah.
That
was
just
another
small
call
out.
So.
B
C
I
think
this
is
everything
I
wanted
to
discuss,
since
we
have
you
on
record
denny.
When
do
we
get
our
t-shirts.
B
Actually,
I'm
currently
working
with
the
vendor
to
send
them
out.
The
problem
is
that
there's
about
630
t-shirts,
I
need
to
send
out
so
you're
gonna
need
to
give
me
a
couple
weeks
to
get
this
organized.
B
Yeah,
fair
enough,
fair
enough,
yeah
fair
enough,
so
we
got
t-shirts
plus
screen
cleaner
balls
and
stickers
coming
up.
B
A
D
E
B
C
Will
you
share
that
link
to
the
athena
thing
in
the
in
the
slack
channel?
I
want
to
take
a
look
at
that.
C
And
I'll
mention
this
to
the
rest
of
y'all,
I'm
chatting
a
little
bit
with
denny
later
today.
I've
got
some
some
upcoming,
crazy
ideas
for
things
that
we
can
do
with
with
delta
rs,
but
I'll
keep
everybody
posted.
If
we
get
something
fun
that
we
can
open
source
here,.