Designing Data-Intensive Applications
About this book
Kleppmann covers the foundations beneath data systems. Part one covers reliability, scalability and maintainability, data models and query languages, storage engines and encoding. Part two covers replication, partitioning, transactions, the problems of distributed systems, and consistency and consensus. Part three covers batch and stream processing. The reason product people read it is that it explains what your engineers mean by a tradeoff.
Description via Product Digest.
Topics
Related concepts
- Context Engineering Strategies: Write, Select, Compress, IsolateFour context management strategies for LLM applications, which are external storage, relevant selection, careful compression and isolated task contexts.
- LLM Observability: Traces, Tool Calls and ReplayTrace LLM requests across retrieval, model calls, validation and tools. Learn what to record, how to protect data and where replay stops being reproducible.
- LLM Prompt Chaining: Workflow Design, Reliability and LatencyLearn when to split an LLM task into a prompt chain, how to validate intermediate steps and how dependencies affect reliability, cost and latency.
- Multi-Agent Systems: Context Isolation, Delegation and Trade-OffsLearn when multiple AI agents help, how to design subtask contracts and how to measure handoff quality, shared budgets and coordination costs.
- AI Agent Memory: Short-Term Context, State and Long-Term StorageDesign AI agent memory with clear ownership, source records, expiry and retrieval. Separate conversation summaries from authoritative application state.
- AI Agent Failure Modes: Loops, Duplicate Actions and Plan DriftDiagnose common AI agent failures and control retries, duplicate actions, invalid tool calls, plan drift and ambiguous tool results.