Dagster / Dagster Demos

Integrate Cube with Dagster to seamlessly connect your data transformation pipelines and orchestration tools with your semantic layer. Handle all of your rough-grained transformation tasks with Dagster and land data in your warehouse and then perform your business logic and metric definition in Cube and provide consistent analytics throughout the organization. Use event-based integrations to trigger your pre-aggregation builds efficiently and easily, saving time and compute resources.

The Modern Data Stack gives us a wide range of free technology for data ingestion, data storage, data transformation, data orchestration, and data visualization.

Pedram Navid, Head of Data Engineering and DevRel at Dagster, walks us through one data pipeline design using a number of open-source solutions, including Dagster, dbt, PopSQL, and the Mastodon API, DuckDB, dbt-duckdbt, Evidence, Sling and Steampipe.

The pipeline is used to analyze bird observation data from the Cornell Lab.

Github Repo: https://github.com/dagster-io/mdsfest-opensource-mds
Source Data: https://feederwatch.org/explore/raw-dataset-requests/

Join the Dagster team for a special showcase of new dbt functionality included in Dagster 1.4. which makes Dagster a strong alternative to dbt Cloud. This session will benefit data professionals who work with dbt models and are looking to manage these as part of a larger pipeline. If you are looking to migrate off dbt Cloud or simply exploring dbt alternatives, this is a session you will not want to miss.

Last year we wrote an article on the Dagster blog called "Building a Poor Man’s Data Lake with DuckDB". In it, we showed that you could use two excellent open-source technologies - DuckDB and Dagster - and build a powerful, cheap, and useful data platform.

Since then, MotherDuck has emerged from stealth to build a high-powered cloud version of DuckDB that competes with other cloud data warehouses like Snowflake.

In this video, Pete Hunt (CEO) shows you how to migrate the Poor Man’s Data Lake away from S3 and Parquet files into a single system: MotherDuck. As you’ll see in this blog post, it’s very straightforward due to the elegant design of both DuckDB, MotherDuck, and Dagster, and we can realize all of the benefits of MotherDuck without even touching our business logic.

No description provided.

Note that a higher resolution of this same event can be found at https://dagster.io/events/dagster-airflow-event-feb-2023 and we recommend viewing the event there.

Join the Elementl team along with data practitioners from Dagsters Open Source and Cloud userbase, including some of Dagster's implementation partners, as we discuss the best ways to make the transition from Airflow to Dagster.

Many teams want to migrate off legacy orchestrators and onto newer best-of-breed orchestration platforms, but the idea can be daunting.

Well, it's now easier to make the switch. Odette Harary shares details on the new dagster-airflow library and perspectives on how to successfully migrate off Airflow by converting Airflow Ops to Dagster Jobs.