Structured output makes an LLM response usable by software by giving it a defined format, such as a JSON object with known fields. It is useful for extraction, classification, routing and any workflow that passes generated data into another system.
There are several ways to request structure. Their guarantees differ, and function calling is a separate design choice from whether the output is constrained.
Compare the available output mechanisms
| Mechanism | What it provides | Remaining checks |
|---|---|---|
| Prompt asking for JSON | A formatting instruction | Syntax, schema, values and completion |
| JSON mode | Valid JSON under the interface's documented conditions | Required fields, types, values and completion |
| Schema constrained output | Adherence to supported schema constraints under documented conditions | Business meaning, evidence and exceptional outcomes |
| Function calling | A request to invoke a named tool with arguments | Argument validation, permissions and execution status |
Function arguments may themselves use strict schema constraints. Conversely, a model can return a structured answer without requesting a tool. Check both capabilities when selecting an interface.
For a provider example, Claude's llm structured output documentation distinguishes structured JSON responses from strict tool inputs and documents supported constraints and limitations.
What constrained decoding changes
A constrained decoder restricts possible continuations so generation follows a supported grammar or schema. It may rule out an unexpected field name or a string where an integer is required.
The restriction concerns the representation. If the source names order A-4471, a schema allowing any string can still accept A-4417. Even a pattern describing the order number format cannot tell which number appeared in the source.
Schema support also varies. Do not assume that every JSON Schema keyword is enforced during generation. Some SDKs transform a schema for the provider and apply further checks after receiving the result.
Design a response that can express missing information
An extraction schema should represent absence explicitly. If a message omits an amount, forcing a non null amount encourages an unsupported value.
{
"order_id": "A-4471",
"amount_minor": null,
"currency": null,
"evidence": ["Please check order A-4471."]
}
The example is an output object, not a complete schema. Its schema should define the allowed types, required fields and treatment of extra properties. The application should separately verify that the evidence appears in the input.
Use meaningful field names and descriptions. “amount_minor” should state the currency unit convention and whether the field means an amount requested, quoted or approved. Those are different business facts.
Handle refusals and incomplete responses before parsing
A provider may return a refusal or other status outside the requested object. An output limit may interrupt generation. A network connection can fail before the full response arrives.
First inspect the response envelope and completion status. Then parse and validate the completed content. Do not silently convert a refusal into empty success data or execute an action from a partial object.
These cases belong in the response contract. Keeping them explicit makes failures observable and prevents endless formatting retries.
Test structure and task quality together
Measure both schema compliance and factual task success. A change that eliminates malformed JSON but increases extraction errors has improved only one part of the system.
For difficult tasks, separating evidence gathering from final formatting may help. It can also introduce handoff errors and additional cost. Compare that design with a single call baseline instead of assuming that a second formatting call restores accuracy.
Tool use and final answer constraints can interact differently across runtimes. Verify that the selected configuration supports the required tool protocol. If tools are not being called, inspect response events and configuration before concluding that the model chose not to use them.
Once the object is complete, validate its meaning before allowing it to affect application state.