Prompt

Data Analysis & SQLPromptIn a pack

Spark Structured Streaming Pipeline Design — Sources, Watermarks, State, Checkpoints and Reliable Sinks

Builds Structured Streaming pipelines that survive restarts — source configuration, event time and watermarks for late data, stateful aggregations and state size, checkpoint management, idempotent or transactional sinks, trigger choices, monitoring lag and safe changes to running queries.

  • Role-Based
  • Step-by-Step
  • Constraints
  • Structured Output

Opening lines · ~300 words in full

ROLE: You are a streaming data engineer working with Spark Structured Streaming. Streaming jobs break in production through unbounded state, lost checkpoints, duplicate writes and changes that are incompatible with existing checkpoints, so you design for restarts and change from the start.
 …

Written by the PromptCorrectly team© 2026 PromptCorrectly · All rights reserved

How to use it

  1. Read it, then replace anything in [BRACKETS] with your details — the more concrete the context, the sharper the answer.
  2. Copy it (or download the .md) and paste it into the AI you already use — it knows your work, so that is where the prompt does the most.
  3. Want a sharper answer? Ask the AI to check its own draft against your brief before it replies — that is self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Step-by-Step

Forces explicit intermediate reasoning instead of jumping to a conclusion, which improves accuracy on hard tasks.

Learn this technique
Constraints

Sets the rules and boundaries — tone, length, what to avoid — that keep the output on-target.

Learn this technique
Structured Output

Pins the response to a defined structure so it drops straight into your workflow.

Learn this technique

Works with

Any chat AI — ChatGPT, Claude, Gemini, Copilot, Grok, Mistral or a local model. The structure does the work, so you are not tied to one vendor or one model version.

New to structured prompts? Start with how to prompt AI, the RCTCO prompt framework this prompt is built on, and role prompting examples.

More in Data Analysis & SQL

Translate Business Questions Into SQLTurns a plain-English stakeholder question into a correct, well-commented SQL query against a known schema.Optimize A Slow SQL QueryDiagnoses why a query is slow and rewrites it with targeted, explained optimizations and an index plan.Debug A SQL Query That Returns Wrong ResultsSystematically finds the logic error producing incorrect numbers and delivers a corrected, verified query.Explain An Unfamiliar SQL Query In Plain EnglishReverse-engineers a complex inherited query into a clear narrative, business meaning, and risk list.