Domain 3 of 5

Applications of Foundation Models

The heaviest domain. Choosing a model and setting its inference parameters, prompt engineering and the attacks against it, Retrieval Augmented Generation, the four ways to customise a model, and how to evaluate the result.

5
Concepts
~28%
Of the exam
18
Practice questions
Concepts in this domain
01Choosing and configuring a foundation modelThe criteria that actually separate one model from another, and what temperature, top p and output length do to the answers you get back.02Prompt engineeringThe parts a prompt is made of, the shot based techniques and chain of thought, and the four attacks the syllabus expects you to name.03Retrieval Augmented GenerationHow a model answers from your documents without being trained on them, what the pipeline actually does, and why it is the default answer to a model that does not know your business.04Customising a foundation modelThe four ways to make a general model fit your problem, ordered by cost, and the question that tells you which one you need.05Evaluating a foundation modelROUGE, BLEU and BERTScore, what each one can and cannot see, and why human evaluation stays the reference the automated scores are checked against.
Try a question from this domain

A service extracts the invoice number, date and total from uploaded documents and passes the result to an accounting system. The output must be consistent, because a downstream parser depends on it. Which inference setting is most appropriate?

  • AA temperature at or near zero, so the model consistently picks the likeliest continuation.
  • BA high temperature, so the model considers more possible readings of the document.
  • CA high top p value, to widen the set of tokens the model may choose from.
  • DA raised maximum output length, so the model has room to correct itself.
18 questions on this domain.

One per page, with a worked explanation.

Start the set