►
From YouTube: dagster + airbyte + dbt + snowflake - parte 4
Description
Practica de Data engineer usando tecnologias open source
A
A
A
We
are
going
to
copy
this
and
we're
going
to
go
to
our
repository
right
and
the
one
we're
going
to
be
using
right
now
is
the
module
we've
been
working
on
and
we're
going
to
copy
here
the
line
that
comes
up
there
in
case
it
shows
an
error
here.
It's
because
maybe
They
do
not
have
this
library
installed
So.
This
library
must
be
installed
when
installing
it,
they
can
put
python
here
and
the
version
will
come
out,.
Then
they
copy
this
clipboard
and
always
in
their
area
in
their
virtual
python
environment,.
A
I
should
have
it
installed
here,
I.
Have
it
that's
why
it's
not
giving
me
an
error.
So
I
have
installed.
Let's
see
what
else
comes
up
in
the
example
here,
it
says
to
write
something
about
this
So.
We
are
going
to
copy
it
here
and
here
it
is
not
asking
for
the
Hosts
in
this
case
when
it
has
this
variable.
It
is
because
it
expects
to
take
them
out
of
the
environment
variables.
We.
A
Are
going
to
burn
it
right
now,
but
normally
they
should
come
from
environment
variables
from
here
we
are
going
to
do
what
else
We
are
going
to
see
how
we
get
it
from
the
environment
variable,
in
theory,,
the
environment
variables,
that
is,,
when
doctor
compost
has
to
pass
the
environment
variable,.
As
we
have
said,
here,
these
data
are
passed.,
Well,
I
think
we
can
pass
it,.
Let's
see,
then
here
in
our
variety
of
environment.
To
do
this
example.
Well,
we
are
going
to
put.
A
A
So
all
this
are
only
the
services.
These
names
are
the
ones
that
are
used
to
use
these
names
to
communicate
with
the
services
So.
In
this
case,
it
would
be
no
in
this
case
it
would
be
container
nor
the
name
of
the
container.
True
true
I,
have
everything
right
here
and
also
this
is
not
asking
for
the
port,
so
we
can
put
a
variable
around
it
called
airbag.
A
A
So
it
wants,
we
are
already
passing
it
up
to
the
environment,
our
user
code
here
and
in
theory.
This
should
already
be
able
to
take
them
there.
What
else
can
we
do
like
this?
We
keep
looking
at
the
example.
It
tells
me
that
we
have
to
create
an
operation
and
a
Job
if
we
are
going
to
have
to
copy
all
this
I
am
practically
using
an
operation
that
is
removing
the
library
and
it
is
here.
It
is
not
asking
for
the
connection
that
we
are
going
to
use.
A
See
if
it
is
running,
we
have
errors
here
for
now.
I
am
going
to
see
those
errors
later,
but
what
I
want
to
see
is
that
I
gave
the
airbag
connection.
You
can
create
a
connection
And.
This
is
what
I
was
saying
about
the
power
of
airlight.
In
this
case,
I
made
an
example.
Look
at
the
internet
there
you
can
see
it
in
airbag.
A
This
is
personal
I,
create
it
today
for
this
test,
I
am
going
to
put
it
here.
This
is
my
snowflake
account
So
how
Come
here
I
don't
have
a
table
at
all,
so
what
we
are
going
to
do
is
we
are
going
to
download
the
content
that
this
app
is
not
returning
and
we
are
going
there.
We
are
going
to
create
a
table
in
snowflake,
so
here
I
can
normalize
data.
That
depends
on
you,
but
it
does
not
matter.
I
am
not
going
to
normalize,
and
here
I
have
configurations
nothing
more
true,
So.
A
A
Action
then
what
is
happening
here
that
it
is
not
loading
us,
we
have
to
find
out,
then
normally,
if
you
can
open
your
docker
client
right
and
see
the
service
that
is
not
being
done
to
it.
In
this
case,
These
are
all
the
containers
that
are
being
created.
As
you
can
see,
the
doctor
example
does
not
know
it
is
doing.
We
focus
you,
we
can
see
the
errors,
it
is
not
being
done
in
this
case.
It
is
saying
that
the
daxter
airbag
module
does
not
exist
And.
A
A
A
In
another
video
we
will
see
how
Or,
maybe
in
this
How,
to
make
this
a
little
more
flexible
with
a
recuayerment
so
as
not
to
have
to
put
this
and
be
able
to
have
our
own
and
have
specified
the
versions
of
the
libraries
that
are
going
to
be
used
that
now
we
are
going
to
give
it
now.
It
has
worked,
we
go
to
our
here
and
it
is
already
working
now.
A
As
you
can
see,
this
is
where
we
practically
configure
daxter
so
that
it
says
baxter
Connect
to
this
Host
through
this
Port
and
so
and
using
this
connection
ID,
it
executes
the
connection.
So
the
logic
behind
this
is
that
this
connection
is
going
to
execute.
This
connection
is
the
one
to
call
lapi
and
then
move
that
data
to
snowflake
That
is
the
logic
So.
We
are
going
to
run
it
right
now
to
see
what
happens
to
run.
A
A
For
reasons
that
I
do
not
understand,
they
do
not
tell
me
working
the
environment
variables,
So
I'm,
going
to
burn
them
in
this
tutorial,
I
'm
going
to
leave
How,
they
would
be
used
And,
you
guys,
Well,.
You
already
know
that
you
have
to
use
environment
variables,
I,
don't
know
why
it's
not
working
for
me.
A
A
Remember
not
I
have
no
table
here
in
snowflake,
so
this
is
our
our
operation.
That
says,
and
we
launch-
and
here
we
should
see
when
the
operation
is
being
launched,
and
here
we
have
the
following
error.
That
says
the
Host
is
on
port
80
more
and
because
it
cannot
access
And
that
it
is
because
the
container
the
airbag
contents
cannot
communicate
with
our
containers
because
our
containers
are
in
this
network.
A
So
what
we
have
to
do
is
put
it
in
the
same
Network,
because
only
the
containers
of
the
same
network
can
communicate.
In
that
case,
there
is
a
link
that
I
am
investigating
from
official
docker
to
use
a
resistant
network,
so
we
can
copy
this
and
paste
it.
We
open
our
compost,
we
wait
for
it
here
at
the
end
and
we
put
the
network
that
we
are
using
here.
That
would
be
doctor
example
Network.
A
A
A
Then
here
it
is
not,,
it
tells
us.
What
are
all
the
configurations
of
this
network
and
not
important
property
are
the
containers
that
are
within
that
network.
Here
we
can
see
there,
they
are
all
there
and
if
not,
we
can
realize
if
the
two
that
are
in
the
network
and
can
be
let
us
know
now
we
are
going
to
launch
this.
This
is
how
it
works.
A
A
A
A
Everything
you
can
do,
move
from
snoffin
to
snowflake,
desserts.
I,
don't
know
about
this
and
I'll
take
away
all
the
tutorials.
You
can
do
from
Amazon
strike,
among
all
this
they
can
do
and
if
you
configure
it
in
airlight,
you
can
practically
execute
this
any
connection
with
daxter
and
the
interesting
thing
is
that
daxter
is
going
to
let
you
chain
several
operations
or
jobs.
A
A
Can't
see
this
but
I
think
they
got
the
idea
of
.
What
it
is,
They
can
do
it
with
their
imagination
to
their
limit,.
So
here
it
is.
This.
Next
video
is
going
to
see
a
bit
of
dvt
and
how
to
integrate
it,.
It
must
be
practically
low,.
It
is
going
to
grab
the
data
that
is
in
the
Warehouse,,
which
is
snowflake,
and
it
is
going
to
transform-
and
it
will
put
it
back
So
that
and
with
that
we
would
already
have
a
super
incredible
data
transformation
platform.