Prompt chaining connects several processing steps so that one result becomes input to the next. A chain can contain model calls, database queries, deterministic rules and validation. Decomposition is the decision about where those boundaries belong.
A chain is useful when a task contains distinct stages with different evidence or checks. It is less useful when it divides a simple request into extra calls without improving the result.
Split work at a verifiable boundary
Consider an order support workflow:
- Extract the order reference from the message.
- Fetch the order through an authenticated service.
- Apply the applicable cancellation rule.
- Draft an explanation from the verified decision.
The lookup does not require a language model. If the cancellation rule is precise, it can also run in code. The drafting step receives the decision and evidence rather than being asked to infer them again.
Each boundary should have an explicit contract. A missing order reference must become a clarification outcome before a lookup begins. A tool timeout must remain distinguishable from an order that does not exist.
Decide what each step is allowed to change
Intermediate results should carry their source and validation status. If step one extracts a customer requested amount, a later step should not relabel it as an approved amount.
Limit the input of each step to what it needs, but preserve essential constraints. A drafting call may not need the entire account history; it does need the approved decision, relevant policy and any restrictions on what can be promised.
This design makes errors easier to locate. It does not ensure that every stage is correct, so evaluate both the stages and the final outcome.
Understand error compounding
If every stage must succeed, overall success depends on the probability of each stage succeeding given the preceding stages. Under an illustrative assumption that each conditional success rate is 95%, the product is:
| Required stages | Calculated success |
|---|---|
| 1 | 95.0% |
| 3 | 85.7% |
| 5 | 77.4% |
| 8 | 66.3% |
Multiplying separately measured marginal success rates requires an independence assumption that may not hold. Real workflows can also detect errors, recover or tolerate a failed optional step.
Use the arithmetic to question unnecessary dependencies, then measure the complete workflow. Five stages with 99% conditional success give about 95.1%, showing why stage quality matters as much as stage count.
Parallelise only independent work
Suppose three document summaries each take 2.4 seconds and a final merge takes 1.8 seconds. Sequential execution takes 9 seconds. With sufficient capacity and no extra overhead, parallel summaries reduce the elapsed time to about 4.2 seconds.
The model work has not disappeared. Input and output use may remain similar, and concurrency can increase rate limit pressure. The merge still waits for all required summaries.
If the second step depends on the first result, parallelising them changes the logic. Do not generate an account specific decision before the account lookup is complete.
Choose between a chain and an agent
In a workflow, application code defines the available transitions. An agent uses model outputs to choose actions during execution. Hybrid designs can let a model choose among a small set of controlled branches.
Anthropic's workflow and agent guidance discusses this distinction and recommends matching complexity to the task. A bounded workflow often provides enough flexibility for a known business process.
An agent can also have a finite step limit and restricted actions. The difference is who chooses the next step, not whether the system is mathematically infinite.
Evaluate the complete chain
Measure final correctness, stage failure rates, recoveries, total cost and end to end latency. Include cases with missing inputs and unavailable dependencies.
Before adding a step, state what error it should reduce and how that reduction will be checked. After adding it, confirm that the improvement survives downstream processing. A better intermediate summary has little value if the final answer remains wrong.