►
Description
T-Mobile’s mission to build the nation’s best 5G network drastically increased the number of monthly network projects planned and directly impacted the enterprise procurement and supply chain organizations. Like many enterprises, data at T-Mobile was spread between disparate, unintegrated and complex systems.
In this session, we will discuss the how and why we migrated from databases and data lakes to a data lakehouse on Delta Lake. Our lakehouse architecture allows reading and writing of data without blocking and scales out linearly. Business partners can easily adopt advanced analytics and derive new insights. These new insights promote innovation across disparate workstreams and solidify the decentralized approach to analytics taken by T-Mobile.
Quick links:
https://delta.io/
https://go.delta.io/slack
https://github.com/delta-io/delta/releases
https://groups.google.com/g/delta-users
B
A
A
A
D
A
A
We're
live
everywhere,
okay,
so
hi
everybody!
You
may
see
me
here
or
here
I'm
not
sure
where
you're
seeing
right
right
now,
but
my
name
is
denny.
We're
just
gonna
wait
for
about
three
more
minutes
for
people
to
log
into
linkedin
youtube
and
the
webex
and
we're
here
in
surprisingly,
sunny
seattle.
Well,
the
east
side
of
seattle
bellevue,
but
surprised
at
sunny
seattle.
So
give
yourself
a
couple
comments:
take
a
vile
break,
get
some
coffee
and
then
we're
gonna
be
starting
our
show
in
about
three
minutes
all
right.
A
In
the
meantime,
you
can
always
chime
in
on
webex
youtube
and
linkedin.
Tell
us
where
you're
based
out
of
we're
gonna,
take
a
look
at
in
a
couple
minutes
right
now,
I'm
presenting
my
screen,
so
I
can't
actually
see
it.
But
as
soon
as
we
go
ahead
and
go
live,
then
we'll
shift
over
we'll
do
some
introductions
and
we'll
dive
right
into
the
details.
A
And
then,
if
you
have
any
questions
that
are
related
to
delta
lake
lake
house
t-mobile,
why
we
dress
the
way
we
dress
things
of
that
nature
by
all
means,
go
ahead
and
jump
in
your
questions
here
and
we'll
do
our
best
to
answer
them,
provided
that
they're
not
too
gross.
That's
about
it
really.
So
should.
B
C
A
C
A
A
You
should
be
able
to
see
the
the
splash
screen
of
d3l2
we're
going
to
turn
it
off
in
a
second.
When
we
start
the
show
so
then
you
should
be
able
to
see
us
and
we're
gonna
make
right.
Now.
You
probably
should
see
either
a
splash
screen
or
you
can
potentially
see
the
room
that
three
of
us
are
in
in
sunny
bellevue,
I'm
just
out
of
seattle.
So.
C
A
A
A
Everybody
all
right,
it
is
9am,
so
I
think
it's
a
good
time
for
us
to
start
the
show.
My
name
is
denny
lee.
This
is
a
fun
session
with
d3l2
or
delta
lake
discussions
with
danny
lee.
No
I'm
not
not
narcissistic,
I'm
just
very
love
alliteration!
That's
all
this
is
and
so
anyways
I'm
really
happy
to
go
ahead
and
start
off
this
session
with
some
friends
here
from
t-mobile,
but
we
actually
go
from
much
farther
in
the
past
and
so
we're
probably
gonna
dive
into
that
little
context.
A
C
D
And
my
name
is
jeff
freeman,
I'm
also
a
member
of
technical
staff
solutions
architect
at
t-mobile.
I
also
spent
many
years
at
working
for
the
big
blue,
the
other
big
blue.
I
guess-
and
you
know,
worked
at
a
couple
other
places
like
gm
cruise,
but
I've
been
at
t-mobile
for
about
two
years
now,
but
yeah
so
denny.
D
B
A
C
A
D
D
Wow,
so
so
that's
way
back
in
the
day,
so
I
was
working
for
a
guy
named
sasha
burger,
who
that
was
one
of
the
original
writers
of
analysis
services
and
you
were
helping
to
get
the
the
ad
centers
bi
on
track.
It
was
challenged,
but
about
the
same
time
I
met
robert.
So
I
was
in
2010
yeah.
You
had
gone
off
and
were
doing
other
things.
You
were
working
with.
He.
A
A
Remember
I
was
first
in
being
with
you
guys
right
and
then
I
ran
off
to
sql
cat
because-
and
I
ran
off
to
single
cap
because
we
were
busy
trying
to
build
cosmos
at
the
time
yeah
and
I
was
going
no.
No,
no,
we
could
solve
everything
with
sql
server
everything
all
data
problems
required,
that
is
a
nail
and
all
data
pro
and
all
of
them
required
the
hammer
of
sql
server.
That
was
where
I
was
coming
from
when
I,
how
times
have
changed
absolutely.
D
A
A
Interested
in
analysis
services-
basically,
it
was
very
good
at
producing
a
heck
of
a
lot
of
random.
I
o,
and
so
the
problem
with
random
iops
was
basically.
If
you
had
spin
discs,
it
would
basically
make
the
ability
for
you
to
query
the
data
off
the
disk
slow
to
a
massive
crawl.
Basically,
that's
what
it
boiled
down
to,
and
so
the
whole
context
was
that.
Could
we
go
ahead
and
have
something
like
analysis,
services,
which
was
inherently
very,
very
rare,
random
iops
and
would
basically
saturate
memory
sap
trade,
the
bus
saturate,
the
cpu?
C
A
B
A
A
A
A
I
I
didn't
design
that
don't
worry,
but
by
the
way
there
is
actually
a
point
to
us
providing
the
context
of
this
history
basically
like.
I
think
the
whole
reason
we're
starting
off
like
this.
Is
we
want
to
give
you
some
everybody
who's
listening
here,
some
context
that
we
we
come
from
a
very
data,
warehousing,
bi
background
and
we've
done
this
for
a
while.
So
so,
for
example,
robert
there's,
a
question
from
fraz
on.
A
A
D
D
A
A
D
A
A
A
C
When
I
started
here,
we
were
trying
to
increase
the
build
rate
of
t-mobile's
network,
they
just
purchased
a
lot
of
spectrum
and
they
wanted
to
build
a
big
5g
network,
and
so
we
started
trying
to
solve
problems
mostly.
You
know
it
came
out
of
iteration
we
took
and
I
you
know
there
was
someone
that
had
a
build
plan
on
a
spreadsheet
and
I
was
like
well
that
should
be
done
on
the
computer
and
started.
B
C
Way
so
let's
put
this
in
a
table
and
see
if
we
can
figure
out
how
to
automate
this
and
the
more
that
that
the
more
things
that
I
brought
into
this
central
location,
it
was
an
mpp
that
we
had
built
and
just
brought
data
from
all
all
different
systems.
It
became
an
integration
point
and
just
took
off
from
there
and
then
the
more
people
that
used
it,
the
more
it
got
bogged
down
and
then
one
day
I
called
jeff
and
said:
hey,
you
want
a
job.
C
B
A
A
C
A
B
C
A
D
On
this
pile
yeah,
so
I
mean
basically,
the
important
thing
to
understand
is
that
we
are
not
the
data
warehousing
team
for
all
of
t-mobile
or
even
just
for
all
of
procurement.
We
are
a
data
science
and
analytics
org,
and
essentially
our
role
is
to
fill
in
the
gaps
for
stuff
that
has
not
already
been
properly
data
warehoused.
D
D
An
official
data
warehouse
where
all
of
the
stuff
is
supposed
to
be
integrated,
but
we
all
know
that
data
warehousing
is
an
intensive
effort.
It
takes
a
lot
of
time
to
model
things
properly
and
get
things
built
properly
and
there's
never
enough
resources
that
are
assigned.
So
the
original
impetus
for
our
team
was:
how
do
we
build
reports
off
these
systems
that
should
be
in
the
data
warehouse
but
aren't.
B
D
The
first
thing
that
has
to
happen
is
you
have
to
pull
all
the
data
into
one
place
and
no,
no
other
teams
have
done
that
on
the
on
the
procurement
side,
so
that
that
systems
that
hadn't
been
properly
modeled
into
the
data
warehouse
people
could
start
working
with.
So
when
robert
did
that
and
started
pulling
all
this
data
in
and
then
was
building
the
etl
for
it,
so
that
people
so
that
it
would
implement
this
one
particular
set
of
business
rules.
D
Essentially,
it's
not
properly
modeled
right,
it's
not
fully
where
we
would
put
it
into
a
business
warehouse,
but
we're
able
to
take
their
business
logic
and
then
identify
the
important
parts
and
then
model
those
pieces.
Okay,
got
it
right
right,
so
we've
we've
kind
of
become
a
staging
warehouse.
Okay
for.
D
C
A
D
A
right
now,
it's
accepted
that
we're
scaling
out
for
most
of
our
problems,
but
especially
for
enterprises,
especially
even
at
2020,
and
this
is
this
happened
during
that
decade
of
between
2010
and
2020..
A
lot
of
enterprises
were
like
no,
no.
We
can
scale
up
all
of
our
problems.
We
can
just
toss
more
memory,
the
there's
no
issues
with
pneuma
memory
management
at
all.
I
can
go
on
for
hours
and
you
don't
want
me
to
about
crossbands
between
memory
cards.
A
So
that
way,
in
terms
of
that's,
why
I
would
actually
slow
down
query
performance
because
they
would
actually
have
to
hit
the
crossbridge
okay.
I
could
go
on
for
hours
about
disk
utilization,
spin
disks,
rotations,
random,
iops
versus
serial
eye,
ops
got
it
got
it.
Thank
you
so,
but
the
con,
the
context
is
basically
every
at
the
time,
especially
enterprises
and
t-mobile
is
no
different.
Here.
A
A
About
the
the
memory
in
physics,
that's
just
because
of
an
ardent
geek,
though
that's
true,
it's
lit
the
part
of
the
reason
we're
bringing
I'm
bringing.
This
up
is
because,
literally
just
to
emphasize
the
point
that
we
were
literally
hitting
the
laws
of
the
wall
of
physics
at
the
side,
like
literally,
we
could
not
physically
go
any
faster
than
what
the
hardware
could
possibly
be
provided.
C
A
They
understand
spark
and
they're
pretty
happy
with
spark,
but
why
don't
you
paint
the
picture
in
terms
of
how
what
was
the
migration
path
in
order
to
get
people
who
had
who
are
typically
coming
from
that
edw
smp
type
systems
to
saying
that?
Actually
we
think
it
makes
sense
to
go
ahead
and
use
to
number
one
scale
out
and
number
two
scale
out
on
spark
and
at
the
time
you're
scaling
on
spark.
It
was
not.
It
wasn't.
A
C
So,
as
jeff
alluded
to
earlier,
we
we're
in
a
data
science
work
right.
So
most
of
the
people
that
we
interacted
with
were
already
clamoring
to
get
their
hands
on
spark,
and
when
we
decided
to
make
this
this
jump,
there
was
no
push
back
for
us
that
we
were
just
like
hey
we're,
gonna
move
this
stuff
to
spark
all
of
our
transformation.
Logic
will
be
in
spark
and
we're
going
to
try
to
figure
out
a
way
to
make
this
a
generic
system
that
you
can
just.
B
D
Well,
I
mean
there
was
some
pushback.
The
biggest
pushback
that
we
had
was
was
end
users
who,
who
thought
I
don't
want
to
learn
a
new,
a
a
new
tool
set
in
order
to
consume
data
right.
So
we
had
particularly
like
business,
analysts
and
and
report
builders
who
were
used
to
using
excel
or
sql
server
and
or
power
bi,
and
they
would
say,
I'm
not
going
to
be
like.
If
am
I
going
to
be
able
to
consume
this
data
from
your
lake
house?
Am
I
going
to
be
able
to?
D
C
Once
we
figured
out
that
we
could
do
pass-through
authentication
with
a
serverless
endpoint
that
became
a
non-issue,
though
I
like
it,
and
that
was
one
of
the
keys
to
our
design.
Was
we
wanted
to
lock
our
data
down
at
the
data
right,
so
we
put
axles
on
the
delta
tables
themselves
that
that
you
know,
if
you
have
access,
you
have
access.
If
you
don't
you
don't
right
right,
and
so
that's
once
we
had
passed
through
authentication
on
our
serverless
endpoint,
we
were
good
to
go
and
we
just
started
going
to
town.
C
D
D
B
D
The
initial
project
that
that
robert
built,
where
he
integrated
all
of
this
stuff
into
a
centralized
location,
was
to
how
do
we
accelerate
the
the
pace
at
which
t-mobile
is
building
towers.
So
t-mobile
had
made
a
commitment
that
they
were
going
to
instead
of
building
like
they
previously
mostly
leased
their
tower
space.
And
now
we
we.
B
D
Change
from
those
models
they
had
to
go
from
building
like
a
hundred
towers
a
month
to
building
1500
towers
a
week,
yeah,
bang,
yeah,
okay,
yeah,
so
a
huge
increase
so
that
had
that
comes
with
tons
of
problems
in
in
inventory
and
supply
chain,
but
also
the
tower
sites
themselves
have
an
incredible
list
of
things
that
have
to
happen.
In
order
to
build
a
cell
tower,
you
have
to
have
leases
and
permits,
and
and
like
land
holder
agreements,
and
it's
it's
a
huge
checklist
that
has
to
happen,
and
so
just.
A
C
All
of
those
check
boxes
right
well,
not
all
of
them,
but
they're
like
so
here's
a
system
that
has
some
of
the
check
boxes.
Here's
a
system
that
has
some
of
the
check
boxes.
Here's
the
system
that
has
some
of
the
check
boxes,
here's
a
system,
so
there
was
no
one
system
with
all
the
check
boxes
right
right.
So
the
system
that
robert
built
is
the
first.
D
A
Had
done
that
before
right,
that's
freaky!
So
in
other
words
prior
to
this
sorry
to
interrupt,
but
prior
to
this,
what
you're
telling
me
is
that,
and
if
I
wanted
to
go
ahead
and
build
a
tower,
I'd
actually
have
to
go
to
multiple
systems.
Hope
the
heck.
There
was
a
reconciliation
process
that
the
same
id
actually
meant
the
same
thing,
and
it
wasn't
by
accident
the
oh
actually
you're,
referring
to
an
id,
that's
in
a
completely
different
state
right
that
you
could
actually
have
some
form
of
reconciliation.
A
It's
like
oh,
that
id
that
document
that,
for
the
permit,
is
associated
with
that
particular
supply
chain.
You
actually
had
to
go
to
multiple
different
systems,
hope
the
heck,
those
were
the
they
actually,
the
the
data
definition
was
correct,
and
so
what
you
did
was
to
start
that
process
and
say:
hey,
let's
normalize
centralize.
D
Of
a
change
so
so,
once
all
of
that
data
was
in
the
same
place
and
he
started
building
this
report.
This
report
is
available
a
lot
faster.
It's
a
lot
more
accurate
and
all
of
a
sudden
we're
able
to
increase
velocity
of
build
right
and
all
these
other
business
users
are
like.
Oh
there's
a
ton
of
value
in
having
this
data
all
in
the
same
place,
so
they
all
came
to
access
it
and
that's
where
we've
got
our
workload
contention
yeah.
D
D
Thing
that
that
he
had
me
do
when
he
brought
me
on
is
extract
that
into
basically
a
lake
house
paradigm,
where
we
were
all
of
the
data
was
already
being
staged
in
the
data
lake.
Okay.
So
as
it's
being
loaded
into
our
centralized
system,
it
was
already
being
staged
in
the
data
lake,
so
I
built
delta
tables
off
of
what
was
in
the
data
lake
and
then
built
a
spark
process
that
would
do
the
etl
and
spit
out
the
report
got
it
and
and
ben
was
like.
D
A
Gotcha,
okay,
so
let's,
let's
shift
over
to
that
now?
Okay,
talking
about
the
data
lake
so
for
before
we
even
talk
about
delta.
Let's
talk
about
spark
and
data
like
so.
In
other
words,
you
initially
had
built
the
system
as
data
lake,
a
bunch
of
park,
k
files,
or
did
you
jump
right
into
delta
right
away?
I'm
just
curious!
No,
it
was
staged
as
part
k,
so
every
extraction
saved.
A
From
the
park
a
itself
so
cool,
so
what
were
some
of
the
pres
prior
to
shifting
over
to
delta?
What
were
some
of
the
issues?
Because
right
now
remember
from
a
contact
for
everyone?
That's
listening!
The
context
is
that,
right
now
sure
we've
ended
up
with
a
lake
house,
but
it
you
don't
just
jump
directly
to
the
lake
house
right,
there's
a
at
least
in
this
case
there
was
a
journey.
Yes,.
B
A
There
absolutely
was
a
journey
now,
obviously
we're
hoping
anybody.
That's
listening
to
us
that
you
learn
from
this.
So
that
way
you
can
jump
directly
to
the
lake
house,
but
so,
while
we
talk
about
like
that
that
process,
okay,
so
you've
got
a
bunch
of
databases,
you've
managed
to
centralize
normalize.
So
now,
there's
a
reconciliation
process.
Everybody's
like
going
oh
cool.
I
can
actually
understand
my
data
perfect,
so
now
you're
getting
these
larger
and
larger
data
lakes
with
more
and
more
parquet
files.
A
D
A
A
A
About
spark
with
a
bunch
of
parquet
tables
at
the
time,
what
were
some
of
the
like?
Ultimately,
just
like
jeff,
you
called
out
like
you
know,
you
were
able
to
avoid
some
of
the
contention
issues,
but
that's
because
so.
But
how
did
you?
What
were
the
issues
that
you
ran
into
as
you
were
transitioning
to
this
spark
of
data
like
parquet?
C
D
A
B
D
A
D
Yeah,
so
so
one
of
the
first
things
we
did
on
that
pilot
program
was
disentangle
all
of
the
views
that
we
were
consuming
from,
okay
and
and
rebuild
them
in
spark,
okay
and
and
that
allowed
us
to
considerably
streamline
and
break
it
and
and
rebuild
things.
But
the
the
key
lesson
that
we
took
from
that
is
that
as
we
as
we
mapped
out,
what
our
migration
was
going
to
be
from
the
existing
system
to
our
current
lake
house
is,
is
what
robert
was
saying
earlier.
D
D
A
Of
the
things
I
often
talk
about,
when
I
don't
get
deep
enough
into
it,
and
so
I
really
should
do
another
session
just
about
this
concept
is
that
the
whole
concept
of
data
engineering
is
what
you
guys
are
doing
is
actually
getting
at
the
time
when
we
started
data
engineering,
especially
back
in
the
back
in
the
day
data
engineering,
software
engineering
often
had
nothing
to
do
with
each
other
right,
yeah.
Basically,.
B
A
C
D
D
D
D
Where
we
would
mirror
the
tables
from
the
source
system
and
then
just
maintain
them
as
delta
tables,
okay
and
then
build
the
gold
tables
off
of
those
right
and
the
the
notebook
that
we
wrote.
So
we
wrote
a
generic
notebook
to
maintain
the
bronze
silver
tables
and
that's
almost
all
python.
Okay,
the
the
notebooks
that
we
wrote.
So
we
wrote
a
generic
wrapper
to
go
around
any
of
the
gold
tables
and
those
are
so.
B
D
B
C
B
A
A
A
Let's
see,
okay,
we
got
a
bunch
of
questions.
Okay,
we
got
a
youtube
question
for
greg,
so
I'm
guessing
it's
greg,
kramer
our
old
buddy
here,
so
so
so,
y'all
up
your
game.
What
is
being
demanded
of
old
biz
excel
folk
to
play
with
delta,
more
sql
more
python.
So
basically,
I
think
the
context
is
more
about
like
because
of
your
analysts
right,
which
were
using
excel
and
sql
things
of
that
nature.
What
like.
D
A
Did
they?
How
did
they
shift
over
to
be
able
to
utilize
spark
in
this
case,
like
I
mean
great,
that
you
talk
about
the
endpoints
and
the
authentication
that
covers
a
lot
of
the
issues,
but
then
the
idea
is
from
a
usability
ux
perspective.
How
did
what?
What
did
it
take
to
basically
transition
the
analysts
that
were
using
those
reports
that
were
so
used
to
using
power,
bi
excel
and
sql
sql
server
for
their
queries.
C
C
I
mean
the
serverless
endpoint
has
been
key
to
that,
so
being
able
to
just
connect
their
reports
to
the
serverless
endpoint.
They
didn't
know
any
difference,
so
they
they
don't
know.
So.
We've
put
our
delta
tables
in
creative
views
that
that
will
just
look
like
sql
tables
right
right,
so
they
don't.
B
B
D
Cool,
so
we
have
regular
training
sessions
that
are
quarterly,
and
then
we
have
weekly
office
hours
and
where
we
help
people
who
are
are
making
those
transitions.
But
the
thing
that's
been
driving
people
to
adopt
the
spark
endpoints
that
we
provide
is
the
actual
business
results
that
they
see
they've.
Seen
all
the
benefits
of
of.
D
I
know
that
my
report's
always
going
to
succeed
because
I
don't
have
to
worry
about
workload.
Contention
they've
seen
that
they
get
it's
really
easy
for
them
to
share
their
data
with
other
teams,
because
they
don't
have
to
go
to
a
sql
server
administrator
in
order-
and
you
know
open
a
firewall
port
or
something
along
those
lines.
They
just
add
somebody
to
an
active
directory
group.
D
A
C
A
D
A
C
A
C
D
One
of
the
so
this
was
still
a
glimmer
in
the
eye,
I
believe,
but
one
of
the
early
things
that
that
we
were
told
was
about.
You
know
the
change
data
feed,
yes,
oh
yeah,
yeah,
and
so
one
of
the
key
things
that
we
had,
I
mean.
Actually
one
of
the
things
that
robert
had
told
me
about
was
his
early
efforts
to
try
to
take
these
systems
that
hadn't
been
built
for
identifying
deltas,
okay
and
and
figure.
D
D
D
B
D
C
D
D
One
of
my
least
favorite
is
its
lack
of
modularity
like
it's
it's.
It
has
a
limited
depth
that
you
can
go
with
things
like
like.
You
can
only
embed.
You
can't
embed
a
four
loop
inside
a
four
loop
there's
a
lot
of
things
that
you
can't
do
right
right,
because
it's
built
on
logic,
apps.
I
I
think
that
I
think
that
one
of
the
things
that
we
would
probably
do
is
we
would.
We
would
use
a
more
modular
system,
something
like
maybe
dbt.
C
D
A
Gotcha
gotcha,
that's
actually
really
cool,
so
so
all
right.
So
now
this
journey
that
you've
went
through.
That's
we've
got
about
like
maybe
oh
10
minutes
left,
so
geez
yeah
wow
time
flies
yeah.
All
right,
yeah
wait,
wait
so
so
so
to
we've
migrated
from
these
multiple
systems
in
which
again
this
part
I
didn't
know
which
is
pretty
cool.
The
fact
that
from
a
procurement
perspective,
you
had
these
different
source
systems,
you
you
centralized,
it
were
able
to
you,
make
use
of
spark
making.
It
live
use
of
delta
lake.
A
People
start
realizing
that
they
can
not
just
do
for
this,
your
p3
project,
but
for
other
systems,
and
so
I'm
just
I'm
curious.
Do.
Are
you
seeing
this
paradigm
of
lake
house
then
start
being
like
usually
absorbed
or
understood
by
the
other
teams
within
t-mobile
or
any
of
the
folks
that
you're
currently
working?
Because
you
noted
the
the
the
monthly
sessions
the
the
office
hours?
Things
of
that?
C
C
D
D
There
are
other
teams
inside
t-mobile
that
that
we've
been
working
with
to
to
advance
or
promote
the
idea
or
that
have
already
like
come
to
it
on
their
own.
So
there's
a
team
called
razer
that
does
that
they're
focused
on
the
on
the
consumer
side
of
the
business
sure
so
we're
all
on
the
procurement
which
is
the
like.
You
know
the
the
network
side
and
right
razer
is
basically
the
customer
care
they're,
the
they're,
the
team.
That's
doing
what
we
do
on
the
customer
cares.
D
A
D
C
A
Well,
yeah,
I
I
used
to
remember
when
back
in
the
sequel
several
days
when
we
talked
about
like
the
step,
one
of
dba,
but
they
would
do
like
the
dba
to
table
ratio.
Like
you
know,
it's
like
okay,
once
you
go
past
one
to
fifty
or
something
like
that
or
then
basically,
you
were
host
or
something
I
forgot.
B
D
So
so
you
know
like
I
said,
those
results
seem
to
speak
for
themselves
and
the
fact
that
people
are
able
to
you
know
that
the
bottleneck
to
more
people
getting
access
to
the
integrated
data
that
that
robert
had
brought
together
for
that.
First
time
used
to
be
there's
only
so
many
resources
on
this
centralized
location
right
right
and
that's
no
longer
a
bottleneck
right
at
all
gotcha.
So.
B
A
A
From
a
bunch
of
databases
and
we're
all
data
warehousing
guys
right
and
we've
shifted
over
to
a
lake
house,
and
so
I'm
just
saying
like
for
anybody
that
is
in
the
same
shoes
as
you
like
who
actually
is
in
another
company,
where
you
also
have
again
prototypical
of
enterprises
where
they
there's
multiple
different.
It's
a
very
heterogeneous
environment.
Lots
of
different
data
systems
like
what's
some
advice
that
you
can
provide,
as
in
terms
of
doing
that,
transition,
firewell.
A
A
A
Out
about
this,
so
my
bad
too,
in
github,
yes,
there
we
are
absolutely
planning
to
open
source
delta
clones
for
everybody.
So,
yes,
it
was,
it
was
a.
It
was
purely
a.
I
forgot
to
add
it:
okay,
I'm
calling
myself
out
here,
okay,
so
no
more!
So!
But
yes,
so
obviously
we
can't
we
so
while
we
can
clone
delta
tables,
we
can't
clone
people.
So
yes,
so
continue!
Your
thoughts
are
in
my
heart,
so.
D
So
I
actually
want
to
go
back
to
what
I
assume
that's
98
had
asked,
okay
about
like
things
that
we
would
change,
observability,
capabilities
and
stuff
like
that.
The
there
are
systems
today
that
that
do
much
better
jobs
of
lineage,
tracking
and
and
making
it
data
discoverable
and
understandable
for
consumers.
D
You
should
check
your
code
into
source
control.
You
should
you
should
build
tests,
you
should
you
should
modularize
as
much
as
possible.
If
you
find
yourself
doing
the
same
thing
every
time
you
have
a
new
table,
then
you're
doing
it
wrong.
Yeah
right,
you
should.
You
should
build
stuff
in
components
in
such
a
way
that
when
you
decide
to
add
an
observability
module,
you
just
add
it
in
one
place
and
it
shows
up
everywhere
right
right.
So
that's
that's
the
the
number.