►
Description
In this series of demos, Elementl engineers Shalabh Chaturvedi and Johann Miller provide an overview of three recent improvements to Dagster Cloud Serverless:
1) Fast Deploys: By using prebuilt Docker images, Dagster Cloud now deploys changes radically faster than it used to by bypassing the image rebuild. Dependencies get managed seamlessly.
2) Secrets Managements: Users can now set environment variables straight in the Dagit UI in Dagster Serverless.
3) Non-isolated runs: By bypassing the isolation of runs, developers can speed up their development and testing cycles.
A
Hi
everyone
I'm
shalab
and
I'm,
an
engineer,
I
work
in
Daxter
cloud
and
I'm
going
to
talk
about
serverless
deployments.
A
few
months
ago
we
launched
serverless
and
called
it
early
access.
Today
we
are
ready
to
call
it
1.0.
Over
the
last
few
months,
we've
made
a
number
of
reliability
improvements,
bug,
fixes
and
added
new
features,
some
of
which
we're
going
to
show
you
today,
foreign.
A
A
Typically,
this
could
take
anywhere
from
three
minutes
up
for
five
minutes,
sometimes
of
time
that
you
would
spend
waiting
to
see
the
effect
of
your
change
in
Dexter
Cloud.
We
build
a
new
Docker
image,
upload
it
to
the
extra
Cloud
where
a
new
container
is
provisioned
and
the
new
code
is
run
from
which
we
extract
the
new
asset
graph.
A
A
A
A
We
should
we
see
that
the
actual
the
new
asset
graphs
available
I
can
actually
just
refresh
this
page
and
see
my
new
acid
right
there.
This
is
ready
to
run,
and
meanwhile,
if
I
go
back,
Docker
based
push,
it
is
still
building
the
docker
image
after
which
it
will
upload
it,
and
then
it
takes
a
lot
longer.
So,
let's
not
wait
for
that.
Instead,
I'm
going
to
show
you
I'm
going
to
talk
a
little
bit
about
how
this
works,
so
how
do
fast
deploys
work?
A
The
key
idea
behind
this
optimization
is
that
when
you
make
a
code
change
instead
of
rebuilding
and
deploying
an
entire
Docker
image,
we
can
rebuild
and
deploy
only
a
smaller
slice
that
contains
all
of
the
code
that
has
changed.
For
this.
We
use
something
called
PEX
is
a
tool
that
given
a
set
of
python
code
or
python
packages,
will
bundle
them
into
a
single
file
called
the
PEX
file.
We
actually
use
two
PEX
files,
one
for
your
source
and
one
for
your
dependencies.
A
Once
these
PEX
files
are
uploaded
onto
diagster
Cloud,
we
run
them
on.
We
still
run
them
on
top
of
Docker
on
a
standard
base
image.
We
have
a
set
of
four
standard
base
images
for
python
versions,
3.7
through
3.10.
It
is
possible
for
you
to
build
your
own
Docker
image
and
upload
it,
and
it
can
be
used
as
a
base
image
to
run
these
PEX
files,
so
fast
deploys
are
Now
the
default
for
all
new
signups.
If
you
already
have
a
setup
with
Dexter
serverless,
we
give
you.
A
A
A
A
So
the
job's
finished
running
and
here
I
can
see
that
it
did
in
fact
get
the
secret
value
out.
These
values
are
stored
encrypted,
on
other
end,
using
a
organization
specific
encryption
key.
So
you
can
use
them
for
your
secrets.
You
connect
to
your
Warehouse
or
other
environment
variables
and
with
that
I'm
going
to
hand
it
off
to
Johan
who
will
show
you
another
serverless
deployment
feature.
B
B
This
run
will
execute
on
a
standing
container,
that's
allocated
for
each
code,
location
in
your
deployment.
The
Run
starts
immediately
rather
than
waiting
on
container
startup,
along
with
the
other
features
that
shalab
discussed.
Non-Isolated
runs
really
reduce
the
time
to
go
from
writing
code
to
executing
it
with
diagster
cloud.