Niklas Heringer
Niklas Heringer
Cybersecurity & Math.
⌘K
Subscribe Sign in
Niklas Heringer

dagster

Dagster in practice: assets, partitions, schedules, sensors and deployment for data pipelines that stay maintainable.

Data Engineering With Dagster Part Eight: Metadata
dagster

Data Engineering With Dagster Part Eight: Metadata

Dig into materialization metadata, inline visualizations, and best practices for asset observability using Dagster.
Data Engineering With Dagster Part Seven – Event-Driven Pipelines with Sensors
dagster

Data Engineering With Dagster Part Seven – Event-Driven Pipelines with Sensors

From Passive Pipelines to Reactive Workflows So far, we’ve scheduled jobs based on time: “Run this every Monday” or
Data Engineering With Dagster Part Six – Partitioning & Backfills
dagster

Data Engineering With Dagster Part Six – Partitioning & Backfills

Learn how to make your pipelines smarter by slicing them into manageable, date-based partitions and handling backfills like a pro.
Data Engineering With Dagster Part Five – Automating With Schedules
dagster

Data Engineering With Dagster Part Five – Automating With Schedules

Dagster finally earns its “orchestrator” title - this part dives into jobs, asset selection, cron expressions, and how to wire everything into automated schedules.
Data Engineering With Dagster – Part Four: Resources, DRY Pipelines, and ETL in Practice
dagster

Data Engineering With Dagster – Part Four: Resources, DRY Pipelines, and ETL in Practice

A deeper look at how Dagster handles reusable components like API clients and cloud connectors through resources - with best practices and cookie metaphors baked in.
Data Engineering With Dagster - Part Three: Definitions and Code Locations
dagster

Data Engineering With Dagster - Part Three: Definitions and Code Locations

How Dagster uses Definitions and Code Locations to scale cleanly across teams, assets, and environments.
Data Engineering With Dagster - Part Two: Dependencies, DuckDB, and Geo Heatmaps
dagster

Data Engineering With Dagster - Part Two: Dependencies, DuckDB, and Geo Heatmaps

Learning to wire assets together, load data into DuckDB, and build heatmaps with real NYC taxi data.
Data Engineering With Dagster - Part One: A Fresh Take on Orchestration
dagster

Data Engineering With Dagster - Part One: A Fresh Take on Orchestration

A student-hacker’s perspective on learning data engineering with Dagster: asset-centric thinking, orchestrator basics, and setting up the mental model.