AI Engineering
About this book
Huyen's book on building applications on top of foundation models rather than training them. Covers evaluation methodology in depth, prompt engineering, retrieval augmented generation and context construction, finetuning and when it is worth the cost, inference optimisation, and the architecture of a production AI application. Current with the practice as it stands and specific about the failure modes and their measurement.
Description via Product Digest.
Topics
Related concepts
- How AI Agent Loops Work: ReAct, Planning and Stop ConditionsUnderstand the AI agent loop, compare ReAct with plan and execute designs and set explicit limits on steps, tokens, time and tool actions.
- Foundation models and how they workTokens, embeddings and transformers explained for somebody who has to make decisions about a model, with nothing in it about building one.
- Customising a foundation modelThe five ways to close the gap between a general model and a specific problem, ordered by cost, and the question that decides which one a problem needs.
- What Is Prompt Engineering? Techniques and ExamplesUnderstand prompt engineering with examples of zero shot and few shot prompting, prompt templates, chain of thought, evaluation and common security risks.
- The foundation model lifecycleThe seven stages that carry a foundation model from raw data to a system in use, and which five of them a team building on somebody else's model actually owns.
- Google Cloud's AI platformThe AI first argument, what enterprise ready means, the hardware underneath the platform and the open approach Google describes.
- LLM Application Security: Guardrails, Permissions and ValidationSecure LLM applications with access controls, output validation, scoped tools and approval checks. Test what happens when model instructions fail.
- What Are AI Agents? Tools, Workflows and PermissionsUnderstand how AI agents use tools, how agents differ from fixed workflows and why read access, write permissions and execution controls matter.
- Building with Agent PlatformGemini Enterprise Agent Platform, formerly Vertex AI, with Model Garden, the RAG offerings, custom agents and the rename the exam now expects.
- What generative AI is good and bad atThe capabilities worth building on, the failure modes that are properties of the technology and will outlast any model, and how to tell a suitable problem from an unsuitable one.
- LLM Grounding Explained: Sources, Citations and HallucinationsLearn how LLM grounding connects answers to evidence, how it differs from RAG and how to check citations, source quality and unsupported claims.
- Google's foundation modelsGemini, Gemma, Imagen and Veo, what each is for, and how an organisation chooses between an open model it hosts and a managed one it calls.
- The five layers of the gen AI landscapeInfrastructure, models, platforms, agents and applications, and why the layer an organisation buys at decides what it is responsible for.
- Scalable oversightThe problem of supervising a system on work a person cannot check, the research directions aimed at it, and the control available in the meantime.
- Red teaming an AI systemDeliberately trying to make a system misbehave before somebody else does, how the exercise is run, what it cannot establish and what regulators now expect of it.
- Evaluating a foundation modelROUGE, BLEU and BERTScore, what each one can and cannot see, why a public benchmark is a filter and no decision, and why human evaluation stays the reference the automated scores are checked against.
- Deceptive alignment and situational awarenessWhat it would mean for a system to behave well only while it believes it is being watched, what the evidence for that actually shows, and why the possibility changes how evaluations are run.
- Scaling and emergenceWhy capability improves predictably with size while individual abilities appear abruptly, and what that does to the job of testing a model.
- Agents and autonomy riskWhat changes when a model is given tools, memory and several steps to itself, and why the danger is in the permissions, whatever the intelligence.
- Guardrails and output handlingThe controls that sit around a model, why layering them never produces a boundary, and the one practitioners get wrong, which is what the application does with the response.
- Open weights and release decisionsThe spectrum from a closed hosted model to a fully open release, the arguments on both sides, and what changes when a team builds on weights it can download.
- The OWASP LLM Top 10The industry's shared list of what goes wrong in applications built on large language models, what the 2026 edition reordered and why, and what the list cannot say about a particular system.
- How to Build LLM Evaluations: Cases, Graders and Release CriteriaBuild an LLM evaluation around real task requirements, with test cases, deterministic checks, human rubrics and explicit release criteria.
- The AI supply chainWhat a team is actually depending on when it ships an AI feature, how those dependencies fail, and the provenance work that has to happen while it is still boring.
- Hidden context exposureEverything an application quietly holds in the context window, why a system prompt was never a security boundary, and why leaked context is usually the step before the attack.
- Prompt Injection Explained: Direct and Indirect AttacksUnderstand direct and indirect prompt injection, how untrusted content redirects LLM applications and how to test the boundaries around data and tools.
- The alignment problemTwo questions hiding inside one word, whether the objective specified was the right one and whether the trained system actually acquired it.
- Capability evaluationsWhat an evaluation can establish about a model, why the method used to test a model decides the answer as much as the model does, and what a dangerous capability evaluation is for.
- How models are trained to behavePre training, supervised fine tuning, reinforcement learning from human feedback and constitutional methods, and why a helpful model is a manufactured artefact that can be unmade the same way.
- How to Evaluate AI Agents: Outcomes, Trajectories and ReliabilityEvaluate AI agents using verified task outcomes, permitted action paths, repeated trials and resource budgets, without requiring one exact tool sequence.
- Cost and consumption controlsWhy unbounded consumption is a safety problem before it is a finance one, where the money in an agent run actually goes, and the limits that decide what the worst case looks like before somebody else decides it.
- Excessive agency and tool permissionsA system able to do more than its feature requires, the three forms it takes, and why permission design is the one control that still works after everything else has failed.
- Choosing and configuring a foundation modelThe criteria that actually separate one model from another, and what temperature, top p and output length do to the answers that come back.
- How AI features failThe ways an AI feature goes wrong in production, ordered by how often a product team actually meets them, with the cause behind each and the control that has a chance against it.
- Corrigibility and controlThe property of a system that lets people correct, interrupt or shut it down, why optimisation works against it, and what keeping it costs in a product.
- What Is RAG? Retrieval-Augmented Generation ExplainedLearn how RAG works, from chunking and embeddings to hybrid search, reranking and evaluation, with clear distinctions between recall and hit rate.
- How to Reduce LLM Cost and Latency: Tokens and CachingCalculate LLM request costs and evaluate prompt caching, shorter outputs, model routing, streaming and parallel calls without hiding quality trade offs.
- LLM Evaluation Datasets: Sampling, Labels and Test-Set LeakageBuild representative LLM evaluation datasets from real and designed cases, prevent leakage and interpret small sample results with appropriate uncertainty.
- LLM Prompt Chaining: Workflow Design, Reliability and LatencyLearn when to split an LLM task into a prompt chain, how to validate intermediate steps and how dependencies affect reliability, cost and latency.
- Prompt and Context Engineering Practice: A Production PlaybookBuild a repeatable prompt and context engineering workflow with versioned artefacts, evaluation ownership, trace review and a practical capstone project.
- How to Handle LLM Refusals, Missing Evidence and Edge CasesDesign explicit outcomes for LLM refusals, ambiguous requests, missing evidence and incomplete output, with routing and evaluation guidance.
- Prompt Regression Testing: Compare Versions Before ReleaseBuild prompt regression tests that compare baseline and candidate versions, expose segment level failures and enforce quality, cost and latency gates.
- LLM Prompt Patterns: System Prompts and Reasoning TechniquesChoose effective system prompts, role prompts, decomposition and self consistency techniques, with examples and clear limits for each pattern.
- AI Agent Memory: Short-Term Context, State and Long-Term StorageDesign AI agent memory with clear ownership, source records, expiry and retrieval. Separate conversation summaries from authoritative application state.
- How LLMs Process Prompts: Tokens, Attention and GenerationLearn how LLMs turn prompts into tokens, use attention and generate responses, with practical implications for token budgets and reliable output.
- LLM-as-a-Judge Evaluation: Rubrics, Bias and CalibrationUse an LLM judge to score open ended responses with a clear rubric, then measure agreement, false passes and position bias against human labels.
- LLM Production Monitoring: Quality, Cost, Latency and DriftMonitor LLM applications for answer quality, retrieval failures, tool errors and drift, with meaningful samples, denominators and actionable alerts.
- LLM Function Calling: Tool Design, Validation and ExecutionLearn how LLM function calling works and design tool names, schemas, permissions and results that an application can validate before execution.
- LLM Model Migration: Evaluation, Compatibility and RolloutPlan an LLM model migration with API compatibility checks, controlled evaluations, prompt adaptation, canary rollout and a viable fallback.
- How to Validate LLM Output with JSON Schema and Business RulesDesign LLM output schemas, represent missing values and validate syntax, structure, source evidence and business rules before using generated data.
- How to Write LLM Prompts: Instructions, Examples and FormatWrite clearer LLM prompts with explicit rules, few shot examples and output requirements. Learn how to handle ambiguity and test prompt structure.
- LLM Structured Output: JSON Mode, Schemas and Function CallingCompare JSON prompting, JSON mode, schema constrained output and function calling. Learn what each guarantees and how to handle incomplete responses.
- LLM Temperature, Top-p and Deterministic Output ExplainedUnderstand temperature, top p, top k and seeds in LLM generation, including why temperature zero does not guarantee reproducible responses.
- Multi-Agent Systems: Context Isolation, Delegation and Trade-OffsLearn when multiple AI agents help, how to design subtask contracts and how to measure handoff quality, shared budgets and coordination costs.