Domain 3 of 6

Getting Usable Output

Turning generated text into something a program can depend on. The three ways to ask for structure and how reliable each one actually is, schemas and what to do when validation fails, refusals and edge cases, splitting one prompt into several steps, and what all of it costs in tokens and in time.

Concepts

~14%

Of the exam

Practice questions

Concepts in this domain

01Structured outputThe three ways to make a model return a document a program can parse, what each one actually guarantees, and why a grammar compiled from a schema is the only one of the three that cannot emit a broken document.02Schemas and validationDesigning a schema a model can actually fill, the three layers of checking that sit at the boundary, what a repair loop costs in tokens, and the measured finding that an output constraint can switch tool calling off.03Refusals and edge casesWhat a feature does when the model declines, when retrieval returns nothing usable, when the question has two readings and when the honest answer is that nobody knows, and how to give each of those four an explicit path.04Chaining and decompositionSplitting one prompt into several steps a developer controls, the four reasons a chain beats one long prompt, the arithmetic of errors compounding across steps, and what every extra step costs in latency.05Cost and latencyTokens in and tokens out as the unit of cost, the mechanism that makes an output token the expensive one, a worked monthly bill that reconciles, and what prompt caching needs before it hits.

Try a question from this domain

A grocery wholesaler extracts order details from supplier emails using a prompt that ends by asking for JSON. The parser raises a decode error at line 1 column 1, because the model returned the sentence "Sure, here is the extracted data", then a markdown code fence holding a well formed object, then an offer of further help. The team needs a route that cannot hand the parser a document it rejects. Which one qualifies?

AA stronger prompt naming the container, forbidding any text before or after it, forbidding a code fence and giving one filled example of the shape.
BConstrained decoding, where a grammar compiled from the schema masks every token that would break the document before the sampler runs.
CFunction calling in its original form, since the provider checks the emitted arguments against the schema before the application ever sees them.
DThe same prompt at temperature 0, since a deterministic sampler cannot choose the tokens that opened the greeting.

13 questions on this domain.

One per page, with a worked explanation.

Start the set