A foundation model knows what was in its training data. It does not know your pricing, your policies or anything written last week. Retrieval Augmented Generation closes that gap by finding relevant material at question time and putting it in the prompt.
What actually happens
Ahead of time, your documents are chunked into passages, each passage is turned into an embedding, and those vectors are stored in a vector database alongside the text.
At question time the question is embedded with the same model, the store is searched for the nearest passages, those passages are placed in the prompt along with the question, and the model answers from what it was given.
The model is never retrained and nothing about it changes. It is simply being asked a better question, with the material it needs already in front of it.
Why it is usually the right first move
Facts stay current. Updating an answer means editing a document and reindexing, not retraining.
Answers can be cited. You know which passages were supplied, so the system can show its sources, and a reader can check. That single property is often what makes an internal tool acceptable to the people who have to trust it.
Access control survives. Retrieval can be filtered by who is asking, so a user sees only what they are entitled to. Baking documents into weights loses that distinction permanently.
It is cheap to try. Days rather than a training run.
Where it goes wrong
Retrieval quality sets the ceiling. If the passage containing the answer is never returned, the model cannot answer, and it will often produce something anyway.
Chunking decides a lot of this. Chunks too small lose the context that made the passage meaningful. Too large and they dilute the embedding and waste context window. Cutting through a table or an argument damages both.
Embedding models are not interchangeable. Question and documents must be embedded by the same model, and changing it means reindexing everything.
Stale indexes lie confidently. A document deleted from the source and left in the index will be quoted as current.
The first thing to measure is retrieval, separately from generation. Ask whether the correct passage was in the returned set. Teams that skip this spend weeks tuning prompts against a retrieval failure.
On AWS
Knowledge Bases for Amazon Bedrock is the managed path, handling chunking, embedding, storage and retrieval so the pipeline is configuration rather than code.
For the vector store itself the exam names Amazon OpenSearch Service, Amazon Aurora, Amazon Neptune, Amazon DocumentDB with MongoDB compatibility, and Amazon RDS for PostgreSQL. The point being tested is recognising that several ordinary AWS databases can hold embeddings, so a vector store is rarely a new piece of infrastructure.
Agents for Amazon Bedrock extends the pattern to multi step work, where the model plans a sequence, calls tools or APIs, and uses the results. Retrieval answers from documents. An agent can also act, which raises the stakes on every security question, because prompt injection now reaches something that does more than talk.