Prompt chaining: breaking one big prompt into three small ones

A common pattern in early prompt design: one prompt asked to extract information from unstructured input, reason about what that information means, and format the result into a specific structure, all in a single call. It usually works, in the sense that it produces output. It's also usually worse at each of those three sub-tasks than three separate, focused prompts would be at the same tasks individually.

Why one prompt doing three jobs underperforms

Each instruction in a combined prompt competes for the model's attention with every other instruction in the same prompt — extraction instructions, reasoning instructions, and formatting instructions all sit in the same context, and the model has to juggle satisfying all three simultaneously. In practice this shows up as a specific, recognizable failure: formatting instructions get followed reliably (they're usually the most concrete, mechanical part of the prompt) while the reasoning quality degrades, because reasoning is the part of the task most sensitive to how much of the model's attention is actually available for it, and formatting is quietly eating some of that budget.

Splitting the chain

# One prompt trying to do three jobs
result = llm.complete(f"""
Extract the key facts from this document, reason about
what they imply for the customer's eligibility, and
format your answer as JSON with fields: eligible, reason.

Document: {document}
""")

# Chained: three prompts, each with one job
facts = llm.complete(f"Extract the key facts from this document as a bullet list.\n\n{document}")
reasoning = llm.complete(f"Given these facts, is the customer eligible? Explain why.\n\nFacts: {facts}")
formatted = llm.complete(f"Convert this into JSON with fields eligible (bool) and reason (string).\n\n{reasoning}")

Each step in the chain has one job and the full context budget to do it — the extraction step isn't competing with formatting instructions for attention, the reasoning step gets clean, already-extracted facts to reason over instead of raw unstructured text, and the formatting step is a comparatively mechanical transformation of already-correct reasoning into a fixed shape, which is exactly the kind of narrow task a model handles most reliably.

The cost this isn't free

Three calls instead of one means three round trips instead of one, which is real added latency, and three sets of input/output tokens instead of one, which is real added cost. This isn't a strictly-better technique — it's a trade, and it should be evaluated as one: better quality per step, at the cost of more latency and more spend, and that trade needs to actually be worth it for the specific task at hand.

Deciding when it's worth it

The deciding question is whether the combined prompt's failure mode is actually costing something that matters. For a low-stakes internal tool where a slightly-off answer is a minor annoyance, one combined prompt is probably the right trade — faster and cheaper, and "probably close enough" is genuinely close enough. For a task where the reasoning step's quality directly matters — a decision that affects a real outcome, output that feeds into something downstream and compounds errors — the latency and cost of chaining is usually worth paying, because the quality gap between a combined prompt and a chained one shows up specifically in the step (reasoning, usually) that matters most.

The general shape worth remembering: chaining doesn't make a task easier, it makes each individual step of a task easier by giving it the model's full attention instead of a shared slice of it. Whether that's worth the added latency and cost is a per-task judgment, not a default to reach for everywhere.