Orchestration

Dig into materialization metadata, inline visualizations, and best practices for asset observability using Dagster.

In this post, we build asset runs that respond to new data - using sensors, config-driven logic, and DuckDB-powered ad hoc reports.

Learn how to make your pipelines smarter by slicing them into manageable, date-based partitions and handling backfills like a pro.

Dagster finally earns its “orchestrator” title - this part dives into jobs, asset selection, cron expressions, and how to wire everything into automated schedules.

A deeper look at how Dagster handles reusable components like API clients and cloud connectors through resources - with best practices and cookie metaphors baked in.

How Dagster uses Definitions and Code Locations to scale cleanly across teams, assets, and environments.

Learning to wire assets together, load data into DuckDB, and build heatmaps with real NYC taxi data.

A student-hacker’s perspective on learning data engineering with Dagster: asset-centric thinking, orchestrator basics, and setting up the mental model.