►
From YouTube: dagster + airbyte + dbt + snowflake - parte 8
Description
Practica de Data engineer usando tecnologias open source
A
Well
we
got
to
the
video
where
we
have
the
part
where
we
have
to
integrate
these
DVD
things
with
daxter
But.
Before
that
we
are
going
to
do
something
interesting.
We
are
going
to
do
as
you
can
see
here
in
the
profile.
I
have
the
values
of
the
burned
profile
So.
We
are
going
to
do
that,
come
from
the
environment
variable,
so
here
an
environment
variable
we
are
going
to
put
a
new
variable.
A
A
On
the
DVD,
so
for
this
advt
we
have
to
pass
it
the
environment,
variables
So.
We
are
going
to
call
the
touch
of
example,
dvt
and
here
below
image.
We
are
going
to
put
environment
and
they
remove
the
call
for
our
new
environment
variables
that
we
are
going
to
pass
to
it,
and
these
have
to
be
with
the
same
name
that
we
put
here.
A
A
With
a
syntax
that
is
called
in
bar,
for
example,
here
it
would
be
in
then
what
it
is
going
to
do
is
that
in
the
container
and
that
this
file
is
going
to
be
copied,,
he
is
going
to
take
out
the
environment
variables
that
were
created
for
that
content,.
So
these
variables
have
to
exist
in
the
container
And.
How
will
they
exist?
Because?
Here?
We
are
passing
it
to
the
container,.
A
Equal
to,
and
there
we
have
the
environment
variables
ready
with
the
profile
to
see
when
to
write
down
interesting
is
that
they
have
to
do
this
in
the,
as
is
an
integer
and
in
the
bar
to
get
as
String
So.
If
they
don't
do
this
casting,
they
will
to
give
an
error
Ok.
So,
let's
see
if
this
is
working.
A
A
Then,
once
we
have
it,
We
only
have
to
run
to
test
it
to
see
what
works
then
To
test
it.
We
are
going
to
execute
the
ROM
Command
in
our
DVD
container
So.
That
is
going
to
be
that
the
models
that
we
have
here
are
run
remember
using
the
environment
variables.
Everything
goes
well,
it
will
work
as
in
the
video
previous.
A
A
So,
it
can't
work
as
command
line
it
It's
super
interesting,
I
could
make
it
work
if
you
put
the
two
DVD
and
Daxter
folders
in
the
same
content,
and
it
would
work
fine,,
but
in
this
video
we're
not
doing
it
that
way.
So,
what
option
do
we
have?
The
other
option
we
have
is
that
it
must
have
a
link,.
It
has
a
option
to
be
used
as
an
rpc
server.
Keep
in
mind
that
it
says
here
that
this
plugin
is
going
to
no
longer
be
available
at
the
end
of
2022
So.
A
If
you
are
watching
this
video
in
2023,
possibly
this
way,
It
will
change.
Maybe
I
guess
they.
They
are
going
to
switch
to
a
graphic
recedo
mode
as
daxter
and
airbag
do
so.
We
have
to
see
what
way
they
are
going
to
implement
by
then.
For
now
we
are
going
to
use
dbt
rrpc
for
the
questions
of
this
video
So.
A
A
A
A
A
A
A
A
A
A
A
A
A
Like
this,
in
this
way,
maybe
we
can
get
a
response,
then
Remember
that
in
which
compost
we
can
access,
the
services
are
available
through
the
as
they
are
available
through
the
same
network.
We
can
access
through
the
name
of
the
service
and
in
docker,
interesting
because
it
should
have.
We
are
going
to
see
here
that,
at
the
end
of
the
server
in
port,
85,
80
I
think
we
have
not
done
that.
I
think
we.
A
A
I
usually
have
a
file
called
request
that
a
plugin
that
I
use
in
this
case
I
have
already
copied
and
pasted.
It
responds
that
the
body
is
here,
so
we
make
a
request.
I
use
this
button
to
make
a
request
to
this
URL.
With
this
as
a
parameter,
they
are
only
getting
a
radius
Ok.
So
this
is
telling
us
that
we
can
now
access
from
my
machine
on
port
85
80.
We
can
reach
the
rc
rc
server.
A
And
it's
working
Okay,
but
it's
super
interesting
So
now
just
lack
of
in
daxter
How.
Can
we
access
all
that
then?
Normally,
we
are
going
to
see
something
interesting,
remember
that
we
cannot
access
it
as
a
command
line,
so
in
daxter
there
is
something
called
here:
I
have
the
link
called
a
has
a
library
called
daxter
dvd
that
we
need
to
install
it
And
that
gives
us
these
resources
to
be
able
to
use
daxter.
A
We
understand
the
resource
of
btrp
PC
resorts
and
be
very
careful
that
there
is
an
rpc
baby
without
Resort
this
asynchronous,
and
this
5
is
not,
as
it
says,
Wait
you
remember
it
is
completed.
This
is
super
important
so
that
they
don't
give
them.
Maybe
they
don't
think
that
something
is
failing.
If,
as
you
can
see,
you
just
have
to
import
it
and
configure
the
Host
and
the
port
So
we
are
going
to
do
it
now
we
can
close
It's,
not
enough
Well.
A
We
can
close
this
and
this
too,
but
the
first
thing
we
are
doing
is
install
the
daxter
DVD
in
our
container
as
Come.
Here
we
have
all
the
libraries
that
we
have
to
use
so
for
this
we
are
going
to
install
daxter
from
bt
and
we
are
going
to
paste
it
here
and
with
this
we
are
going
to
have
duster
in
our
container.
We
are
also
going
to
install
it
in
here
in
the
on
my
machine
in
my
python
virtual
environment
Because.
A
While
you
are
writing
code,
if
I
don't
have
it
in
the
virtual
environment,
it
will
be
giving
me
an
error.
So
I
install
it
just
so
that
it
doesn't
give
me
an
error,
but
that
will
prevent
it
when
it
is
importing.
The
library
gives
me
errors
So.
If
we
open
our
repository
Here,
we
are
going
to
create
Oops
I,
think
it
was
the
Oops.
So
here
we
have
the
airbag
one
that
we
have
used
previously,
but
now
we
are
going
to
use
the
bts
daxter
then
from
how
to
say,
Front.
A
According
to
this
we
have
to
create
a
costume,
Host
rcp.
We
are
going
to
put
it
down
here
right
and
this
is
Not
using
the
synchro,,
but
we
are
going
to
use
the
synthrom,
either
of
the
two
is
valid,
and
here
we
have
our
asking
for
the
Host
and
the
port,,
which
is
the
same
as
what
we
had
here
in
the
airbag
one,.
So
for
this
we
can
go
to
the
constants,
and
here
we
can
create
another
of
this
identical
that
is
called
dbt
config
and
we
are
going
to
remove
the
DVD
environment
variable.
A
A
A
A
A
This
Port,,
this
Port
parameter,
is
in
number.
So.
We
are
going
to
do
a
casting
so
that
it
does
not
give
us
an
error,
because
by
default
the
variables
are
brought
as
chain,
then,
once
we
have
that
we
can
already
in
our
UPS
and
we
can
bring
the
constants
that
we
have
done
previously.
In
this
case,
we
can
make
constant
point
of
Go
away
conflict,
and
with
that
we
have
a
resource,
represents
that
we
can
use
in
an
operation.
A
A
That's
burning
Something
super
weird
here
is
the
we
have.
Then
you
must
So
now
I
have
a
new
Shop
that
the
Job
rpc
here
for
this
to
work
It
expects
a
resource
called
dbt,
and
this
resource
is
the
rpc
that
we
created
earlier,
which
is
that
it
connects
to
daxta
and
remember.
This
was
the
resource
of
bts
with
the
configuration
So.
If
we
publish
this
Job
and
run
all
our
codes,
we
should
see
that
new
shop
in
our
text.
A
A
Let's
hope
it
works
well.
Local,
post-
and
here
is
my
new
shock
called
my
DVD
rpc
Shop
So.
If
everything
goes
well,
This
me
when
I
run
it,,
it
will
be
connected
to
my
DVD
container
and
the
Job
will
run,.
It
must
be
small
in
this
case.
They
are
the
models
that
I
have,.
We
are
going
to
run,
it.,
I
click
on
roll
and
it
is
being
processed,
and
it
seems
that
it
is
working
perfectly
in
theory.
When.
This
is
done.
A
What
else
can
we
do?
Ok,
we
can
do
another
example
because
to
see
a
little
of
the
interesting
thing
about
daxter,
we
can
chain
operations
so
that
they
happen
one
after
the
other.
We
are
going
to
do
a
simple
example.
For
example,
I
have
my
airbag
connection,
Remember
that
these
converter
is
connecting
to
an
iPad.
It
is
downloading
the
content,
it
is
saving
it
in
snowflake
and
then
We
are
going
to
suppose
that,
after
that
content
is
already
put
in
my
Warehouse
snowflake.
A
A
We
are
going
to
create
a
new
shock
that
we
are
going
to
see.
It
depends
if
Air
by
DVD-
and
here
as
first
I,
am
going
to
execute
the
connection,
the
byte
and
then
I
am
going
to
hide
the
dvt.
This
one
needs
the
resource
and
this
one
needs
the
DVD
resources.
That
is
why
now
he
is
getting
both
of
them
in
one
in
this
Shop
and
to
remain
dependent
on
duster
the
way
in
which
dependencies
are
created
in
these.
A
A
A
A
The
synchronization
Job
is
first
executed,,
which
is
brought
from
a
source.
In.
This
case,,
the
Pokémon
API,
will
put
it
in
our
snowflake
Warehouse
and
then
we
run
a
bt
transformation
and
everything
will
happen
in
order,
so
that
It
wasn't
super
flexible,
that
these
tools
can
be,
and
you
can't
imagine
a
lot
of
things
that
can
be
chained
together
and
create
this
very
complex
pipeline
flow
and
always
running
one
after
the
other,.
The
airbite
one
is
over
and
now
comes
the
one
for
Correct
DVD,
and
this
is
how
it
works.