Dagster Dagster Community Day - Dec 2022, 8 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Non-isolated Runs - Dagster Community Day - Dec 2022

Description

In this demo, Elementl engineer Johann Miller provide an overview of three recent improvements to Dagster Cloud Serverless: Non-isolated runs.
By bypassing the isolation of runs, developers can speed up their development and testing cycles.

A

A

I'm going to share another feature that we're adding to diagster Cloud to let you launch runs much faster than before. Previously every run in dijster Cloud took place in its own container. Let's launch one of these now.

A

Using separate containers gives the benefit of isolation between runs, but it also adds overhead where you need to wait for a container to be provisioned for every run that you launch. We can see this happening here.

A

To improve this, we're adding the option to launch non-isolated runs. Let's take a look.

A

I'll leave this new toggle, unchecked and click launch. This run will execute on a standing container, that's allocated for each code, location in your deployment.

A

The rung starts immediately rather than waiting on container startup, along with the other features that shalab discussed. Non-Isolated runs really reduce the time to go from writing code to executing it with diagster cloud.