►
Description
Shirshanka Das and Maggie Hays from Acryl Data review recent improvements to the DataHub Looker connector.
B
All
right
that
was
great
and
loved
the
problems
that
you
called
out
around,
both
being
able
to
represent
multiple
tenants
well
inside
the
catalog.
I
think
that's
actually
something
that's
top
of
mind
for
us
as
well,
so
we
are
going
to
be
modeling
domains
way
better,
and
so
you
should
be
able
to
create
multiple
namespace
metadata
graphs
very
soon
and
in
terms
of
extensibility
of
the
ui
and
being
able
to
customize
it.
B
That
also
is
very
top
of
mind
for
us,
so
I
think
it's
a
very
right
time
to
get
involved
in
that
effort
as
well.
There's
quite
a
few
people
in
the
community
who
have
that
requirement,
including
the
linkedin
team,
who
runs
a
very
complex
version
of
data
hub
inside.
So
it's
a
great
time
to
actually
collaborate.
B
All
right.
Moving
on
the
local
connector
had
has
had
quite
a
bit
of
improvements.
Some
of
it
might
be
a
bit
disruptive.
So
it's
important
to
kind
of
pay
attention
to
this
quick
intro
to
looker
itself.
Looker
is
a
complex
beast.
It
is
an
etl
tool,
as
well
as
a
bi
tool
and
everything
in
between
which
is
the
slippery
slope
that
all
bi
tools
fall
into
right.
So
here
are
the
concepts
looker
connects
to
your
database
or
it
could
be
multiple
databases
through
something
called
a
connection.
B
B
What's
in
a
project?
Well,
there
are
model
files,
that's
kind
of
like
a
namespace
and
model
files
include
view
files,
so
you
can
have
views
inside
a
model.
Think
of
it
like
views
in
the
database
or
views
in
a
schema.
I
guess-
and
they
also
include
other
files
like
your
dashboard
files,
and
things
like
that.
B
B
So
that's
really
the
ecosystem
and
the
previous
connector
before
we
made
changes
had
a
few
problems,
one
the
we
have
two
connectors
actually
two
sources,
one
that
goes
after
just
the
charts
and
the
dashboards
called
the
looker
connector,
and
one
that
just
parses
the
files
which
are
in
the
git
repo
called
the
look
ml
connector
and
the
two
of
them
were
actually
non-overlapping.
The
looker
connector
went
after
dashboards
and
charts
and
tried
to
guess,
lineage
to
views
and
the
luca
ml
connector
would
get
the
views
and
tried
to
get
lineage
from
the
database.
B
Firstly,
views
were
being
named
as
global
names,
which
looks
great
until
it
breaks,
because
you
might
have
the
same
name
view
in
a
different
project
and
then
very
soon,
you've
got
name
collisions
happening
on
views.
There
are
some
challenges
with
currently
correctly
resolving
partially
specified
table
names
and
view
definitions.
When
you
write
sql,
you
know
you
can
write
the
full
definition
like
a.b.c
database,
dot,
schema.table
or
you
can
just
say,
table
name
and
looker
kind
of
does
defaults.
B
B
B
Eight
fourteen
two
and
you
know
we'll
we're
actually
making
quick
improvements
to
it
even
now
as
we're
getting
feedback,
so
there
might
be
a
dot
three
and
a
dot
four
coming
out
soon,
but
a
few
decisions
that
we
made
that
you
can
control
is
how
views
should
be
named,
so
we
decided
to
name
them
based
on
the
project
name.
So
this
is
the
project
under
which
the
views
live.
This
is
typically
your
github
project
name
and
we're
naming
views
as
project
name,
dot
views,
dot
view
name.
B
We
also
have
better
specification
for
these
db
connections,
because
those
db
connections
are
actually
maintained
on
the
looker
side
and
we
now
support
resolving
those
database
connections
through
the
api
as
well,
but
you
can
also
specify
them
offline
and
we're
also
extracting
explorers
and
owners.
On
the
looker
side,
there
are
no
really
required
changes
for
looker
ingestion,
but
the
look
aml
config
now
supports
api
credentials,
and
if
you
do
that
it
will
do
a
lot
of
resolution
on
its
own
and
you
don't
have
to
provide
it.
B
So
much
help
and
if
you're
not
using
the
api,
it
requires
the
project
name
so
net
net.
It's
all
good
one
big
caveat.
If
you're
upgrading
to
the
latest
connector
from
a
previous
connector
and
you've
got
data
or
metadata
ingested
from
the
past,
you
will
see
views
get
named
differently
and
your
old
views
are
going
to
stay
unless
you
nuke
them.
B
So
be
aware
that
that
could
happen.
So
you
want
to
roll
back
that
ingestion.
If
you
want
to
remove
those
views
or
if
you
really
don't
want
to
touch
your
view,
naming
you
can
actually
change
that
conflict
parameter
to
name
views
the
same
old
way
that
they
were
being
named,
the
docs
actually
go
into
that
in
a
lot
of
detail.
A
A
So,
let's
see
here,
for
example,
we
have
you
know,
data
set
property
or
sorry
we
have.
Where
did
my
thing
go?
Oh
there
we
go
data
set
properties,
so
this
is
where
chicago
is
talking
about
the
concept
of
a
view.
A
view
is
then
pulled
into
an
explorer,
and
this
this
explorer
lives
within
this
model
right.
So
these
are
just
like
very
basic
looker
notions.
A
So
we
have
it
broken
down
by
environment
by
our
platform
type
by
the
schema,
and
then
just
the
data
sets
as
well
and
then
down
here
we
have
a
beautiful
sand,
key
I'm
a
sucker
for
a
sand
key
chart
that
shows
you.
The
flow
of
the
data
sets
between
all
those
different
environments
and
ultimately,
this
terminal
node
is
how
many
have
descriptions
on
here
so
cool.
Now
we
have
this
this
dashboard.
A
So
if
we
go
into
our
data
hub
demo-
and
we
look
for
data
hub
health-
we'll
now
see
this
dashboard
in
here,
and
this
is
owned
by
me.
That's
great!
Within
this.
We
have
these
various
charts,
which
are
also
owned
by
me,
and
that's
also
great,
and
then,
if
we
wanted
to
look
at
the
lineage
of
it,
we
can
start
to
look
at.
A
A
This
explorer
is
built
off
of
a
handful
of
view
files
here,
so
oops
no
zoom
in
zoom
in
so
now
user
basic
info,
for
example.
We
have
we're
qualifying
that
as
a
view
within
looker
that
sits
on
top
of
a
snowflake
table
which
is
ingested
using
an
airflow
job,
etc
cetera.
So
we
start
to
see
the
full
lineage
of
data
transformation
then
presented.
A
I
think
there
was
one
other
thing.
Oh,
the
the
other
thing
is
when
in
in
the
actual
browse
path,
if
you're
looking
at
looker
data
sets,
we
used.
We
used
to
just
kind
of
lump
all
these
together
as
views
we're
now.
Actually
separating
out
these
entities
between
explorers
and
views
so
we're
getting
it
closer
to
we're
mimicking
the
same
kind
of
like
name,
space
and
structure
of
looker.
So
it's
more
intuitive
and
we're
not
you
don't
have
to
kind
of
do
the
mental
gymnastics
of.
Is
it
a
data
set.
A
B
Another
announcement
that
things
that
will
come
to
the
ui
very
soon
is
you'll
start
seeing
not
only
right
now.
When
you
look
at
the
search
previews,
it
says:
looker
data
set
so
that'll
start
having
to
say
look
review,
it'll,
start
saying:
look,
review
and
looker
explorer,
so
it'll
make
it
even
more
clear
whether
you're
interacting.