Feat: Add lifecycle hooks to PrefectDbtRunner and PrefectDbtOrchestrator#21523
Feat: Add lifecycle hooks to PrefectDbtRunner and PrefectDbtOrchestrator#21523harsh21234i wants to merge 5 commits intoPrefectHQ:mainfrom
Conversation
desertaxle
left a comment
There was a problem hiding this comment.
Thanks for submitting a PR @harsh21234i!
I would take a slightly different approach to the interface. I think a decorator-based API would be better for readability and type checking.
Here's the shape I think this should take:
from prefect_dbt import PrefectDbtRunner, DbtHookContext
runner = PrefectDbtRunner()
@runner.on_run_start
def before_build(ctx: DbtHookContext): ...
@runner.post_model(select="tag:critical")
def on_critical_done(ctx: DbtHookContext):
if ctx.status == "error":
publish_metric(...)
@runner.on_run_end(select="tag:marts")
def after_marts(ctx: DbtHookContext): ...
runner.invoke(["build"])|
Thanks, that makes sense. I’ll follow the decorator-based API shape and keep the implementation aligned between PrefectDbtRunner and PrefectDbtOrchestrator. I’ll start by defining a typed DbtHookContext and wiring the run/model hook registration and execution path. |
|
Hello @desertaxle I updated the implementation to the decorator-based API you suggested. PrefectDbtRunner and PrefectDbtOrchestrator now expose decorator hooks with a typed DbtHookContext, including select=... filtering, and I added focused tests for registration, execution, filtering, and hook-failure isolation. |
# Conflicts: # src/integrations/prefect-dbt/prefect_dbt/core/_orchestrator.py # src/integrations/prefect-dbt/prefect_dbt/core/runner.py
|
I updated the implementation to the decorator-based API you suggested. PrefectDbtRunner now supports on_run_start, post_model(select=...), and on_run_end(select=...) with a typed DbtHookContext, and I applied the same hook shape to PrefectDbtOrchestrator as well. I also added focused tests for registration, execution, filtering, and hook-failure isolation. |
Fixes #21493 I’ve implemented this on a local branch and verified the focused
prefect-dbttest suite.Completed in this pass:
PrefectDbtRunnerPrefectDbtOrchestratorVerification: