A foundation model knows what was in its training data, up to the point that data stopped. Grounding is connecting it to a source outside itself so an answer can be traced to something checkable.
The three kinds of ground
They are separated by whose data it is, and the distinction decides both the mechanism and the risk.
First party data is yours. Policies, product information, customer records, internal documents. Grounding against it is what makes an assistant useful about your business, and it is where access control matters most, since retrieval has to respect who is asking.
Third party data is licensed from somebody else. Market data, industry research, a partner's catalogue. The considerations are commercial and legal as much as technical, because a licence usually says what the data may be used for and feeding a model may not be covered.
World data is public information. Putting a web search behind an answer brings current public knowledge to a model whose training stopped, which is the answer to the knowledge cutoff.
The three arrangements, and what each platform calls them
Managed retrieval over your own content. Chunking, embedding, indexing and retrieval handled for you, which is the right default for first party data. Google calls this prebuilt RAG with Agent Search, AWS calls it Knowledge Bases for Amazon Bedrock, and Azure calls it Azure AI Search with its integrated vectorisation.
Assembled retrieval. The stages exposed so you control chunking, the embedding model, the store and the ranking. Every platform offers the pieces, and so do LlamaIndex, LangChain and a Postgres database with pgvector. Worth it when retrieval quality is the product and the managed defaults are not enough.
Grounding against the public web. The model checks current public information rather than an index of yours, which answers the knowledge cutoff. Google offers grounding with Google Search, Azure offers Bing grounding, and on other platforms this is a search API called as a tool.
They are not exclusive. A system commonly grounds on internal documents for policy questions and on the web for anything about the wider world.
Sampling parameters
Grounding decides what the model sees. Sampling decides how it answers.
Temperature controls randomness. Near zero the model repeatedly takes the likeliest next token, giving predictable, conservative output. Higher values admit less likely choices, giving variety and more mistakes. Extraction, classification and anything feeding another system want low. Ideas and copy variants want high.
Top p, or nucleus sampling, restricts the choice to the smallest set of tokens whose probabilities add up to p. It reaches the same tradeoff by a different route, and adjusting both at once makes the result hard to reason about.
Token count and output length cap the response. They protect against a runaway bill and they truncate, so an answer stopping mid sentence usually means the cap was too low rather than that the model finished.
Safety settings filter categories of harmful content at configurable thresholds, on the way in and on the way out. They are a product decision rather than a technical one, since the right threshold for a children's product is not the right threshold for a medical one.
Choosing
The questions to ask in order. Does the answer depend on information the model cannot have, and whose is it. If it is yours, retrieval over your content. If it is the world's and changes, grounding against the web. Then set temperature low if the output feeds a system, higher if a person is looking for options, and leave top p alone.
Practise this
You need a grounded system of your own, ten minutes, and honest knowledge of what your sources do not cover.
Pick three gaps you are certain about rather than guessing at one, then ask across all three. The interesting result is not whether the answer is wrong but whether the system says it does not know. Then open every citation it offered and check that the cited passage genuinely contains the claim, which is the step almost nobody performs.
Three questions for a grounded system of yours.
1. True but absent.
"What is our refund window for orders placed in Norway?"
Ask this when the policy documents cover only the UK.
2. A false premise.
"Which of our two escalation tiers applies here?"
Ask this when there are three tiers, or none at all.
3. Just outside the index.
Take a covered topic and move it one year, one region or one
product line further out than your documents reach.
For each answer record: refused, hedged, or confident. Then open
every citation and mark whether it supports the sentence it sits on.
Look for a confident answer carrying citations that do not support it, which is the failure that retrieval creates rather than removes, because a list of sources reads as evidence to a user who is never going to open them. A hedged answer that names the gap is the behaviour you were trying to buy.
Retrieval lowers the rate of invention and raises the cost of each one that survives, since those now arrive dressed as sourced answers.