Data Analysis & SQL5.0 · 0 ratings

Build An Incremental dbt Model

Writes an incremental dbt model with correct unique key, late-arriving handling, and backfill strategy.

Role-BasedStep-by-StepStructured-Output

Prompt

ROLE: You are an analytics engineer building production dbt models.

CONTEXT: Create an incremental model named [MODEL_NAME] from source [SOURCE_TABLE] (schema [SCHEMA]). Target warehouse: [DATABASE_ENGINE]. Grain: one row per [GRAIN]. Late-arriving data window: up to [LATENESS]. Unique/business key: [UNIQUE_KEY].

TASK:
1. Decide the incremental strategy (append, merge/upsert, insert_overwrite by partition) and justify it for this grain.
2. Write the dbt model SQL with the proper config block, the is_incremental() filter using an updated-at watermark with a safety lookback for late data, and the unique_key.
3. Add dbt tests (unique, not_null, relationships, accepted_values) for the key columns in YAML.
4. Explain how a full-refresh backfill behaves vs an incremental run.
5. Note idempotency: running twice must not duplicate or drift.

OUTPUT FORMAT: Strategy rationale -> Model ```sql``` -> schema.yml tests -> Backfill behavior -> Idempotency notes.

CONSTRAINTS: The is_incremental() predicate must include a lookback to catch late-arriving rows. Use merge with unique_key to stay idempotent on the chosen engine. No SELECT *; select explicit columns. State the watermark column and its timezone assumption.

Recommended models

claudegpt-4ogemini

More in Data Analysis & SQL