Concept 2 of 4

Grounding and retrieval

4 questions test this

A foundation model knows what was in its training data, up to the point that data stopped. Grounding is connecting it to a source outside itself so an answer can be traced to something checkable.

The three kinds of ground

The exam separates them by whose data it is, and the distinction decides both the mechanism and the risk.

First party data is yours. Policies, product information, customer records, internal documents. Grounding against it is what makes an assistant useful about your business, and it is where access control matters most, since retrieval has to respect who is asking.

Third party data is licensed from somebody else. Market data, industry research, a partner's catalogue. The considerations are commercial and legal as much as technical, because a licence usually says what the data may be used for and feeding a model may not be covered.

World data is public information. Grounding with Google Search puts current public knowledge behind an answer, which is the answer to the knowledge cutoff.

Google Cloud's three offerings

Prebuilt RAG with Agent Search. The managed path over your own content. Chunking, embedding, indexing and retrieval handled for you. The right default for first party data.

RAG APIs. The stages exposed so you can control them, for when retrieval quality is the product and the managed defaults are not enough.

Grounding with Google Search. The model checks against public web results rather than an index of yours. This is for world data, for currency, and for anything where the answer changes faster than you could reindex.

They are not exclusive. A system commonly grounds on internal documents for policy questions and on Google Search for anything about the wider world.

Sampling parameters

Grounding decides what the model sees. Sampling decides how it answers.

Temperature controls randomness. Near zero the model repeatedly takes the likeliest next token, giving predictable, conservative output. Higher values admit less likely choices, giving variety and more mistakes. Extraction, classification and anything feeding another system want low. Ideas and copy variants want high.

Top p, or nucleus sampling, restricts the choice to the smallest set of tokens whose probabilities add up to p. It reaches the same tradeoff by a different route, and adjusting both at once makes the result hard to reason about.

Token count and output length cap the response. They protect against a runaway bill and they truncate, so an answer stopping mid sentence usually means the cap was too low rather than that the model finished.

Safety settings filter categories of harmful content at configurable thresholds, on the way in and on the way out. They are a product decision rather than a technical one, since the right threshold for a children's product is not the right threshold for a medical one.

Choosing

The questions to ask in order. Does the answer depend on information the model cannot have, and whose is it. If it is yours, retrieval over your content. If it is the world's and changes, grounding with Google Search. Then set temperature low if the output feeds a system, higher if a person is looking for options, and leave top p alone.

Common misconceptions

Grounding and retrieval augmented generation are the same thing.

Grounding is the goal, which is tying an answer to a source that can be checked. Retrieval is the most common way of achieving it. Grounding with Google Search reaches the same goal without a retrieval index of yours.

Temperature and top p should both be tuned for best results.

They are two routes to the same tradeoff between predictability and variety. Moving both at once makes the effect of either impossible to reason about, so Google's guidance is to adjust one and leave the other.

Grounding removes hallucination.

It reduces it substantially and does not remove it. A model can still misread a retrieved passage, blend two of them, or fall back on training data when retrieval returned nothing useful, which is why citation and evaluation still matter.

4 questions test this concept

An assistant must answer questions about legislation that changes several times a year, using current public information. Which grounding approach fits?

  • APrebuilt RAG with Agent Search over an internal copy of the legislation.
  • BFine tuning the model on the legislation each time it changes.
  • CRaising the context window so more of the legislation fits in each prompt.
  • DGrounding with Google Search, which ties answers to current public information.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
AI Engineering, On grounding as an architectural decision.