Concept 5 of 5

Evaluating a foundation model

3 questions test this

Evaluating a generative model is harder than evaluating a classifier, because there is usually no single right answer. Several outputs can be correct and differently worded, and the metrics available are approximations of a judgement only a person can make.

The automated metrics

ROUGE, Recall Oriented Understudy for Gisting Evaluation, measures how much of a reference text appears in the generated one. It is recall leaning and it is the standard for summarisation, where the question is whether the important material survived.

BLEU, Bilingual Evaluation Understudy, measures how much of the generated text appears in the reference. It is precision leaning and it comes from machine translation, where the question is whether what you produced was warranted.

Both count overlapping word sequences, which is their shared weakness. Say the same thing in different words and you score badly. Copy the reference wording while being wrong and you score well. They are useful for detecting that a change made things worse, not for deciding whether output is any good.

BERTScore compares meaning rather than words, by embedding both texts and measuring similarity. A correct paraphrase scores well, which is the thing the other two get wrong. It costs more to compute and it inherits whatever the embedding model misunderstands.

Perplexity measures how surprised a model is by a piece of text. It is a property of the model rather than of a task, useful when comparing base models and largely irrelevant to whether your application works.

Benchmarks

Published datasets with published scores, covering general knowledge, reasoning, coding and safety. They make models comparable and they are contaminated by scale, since a benchmark on the public internet is plausibly in the training data of everything that came after it.

Treat them as a filter rather than a decision. They tell you which models are worth testing.

Human evaluation

The reference the rest is checked against. People rate output, or choose between two, against criteria that match your case, such as accuracy, tone, completeness or safety.

It is slow and expensive, and it is the only method that sees the things that matter most. Whether an answer would embarrass you. Whether it is subtly wrong in a way a customer would notice. Whether it is useful.

The practical arrangement is a small evaluation set of your own cases, thirty to a hundred, with agreed answers or agreed criteria, scored by people once to establish a baseline. After that automated metrics can watch for regression between versions, with periodic human review to confirm the proxy still tracks.

Business measures

The syllabus is explicit that model scores are not the end of it, and asks whether the model meets business objectives. Productivity, user engagement and task completion.

That is the right final question. A model scoring well on ROUGE and producing summaries nobody reads has failed, and no amount of metric explains why. The useful comparison is always against what people were doing before.

On AWS, Amazon Bedrock provides model evaluation with both automatic metrics and human review workflows, and Amazon Augmented AI routes low confidence outputs to people, which is the mechanism for keeping a person in the path once the system is live.

Common misconceptions

A high benchmark score means the model will do well on your task.

Benchmarks measure general capability and correlate weakly with narrow applied work. A model topping a leaderboard can lose to a smaller one on your documents, which is why an evaluation set of your own cases is worth more than any published table.

BLEU and ROUGE measure whether the answer is correct.

They measure overlap with a reference text. A correct answer worded differently scores badly and a fluent wrong answer reusing the reference wording scores well, so neither is a truth check.

Automated metrics can replace human evaluation.

They are cheap proxies calibrated against human judgement. They are useful for regression testing between versions, and something has to establish what good looks like in the first place.

3 questions test this concept

A team is evaluating a summarisation feature and needs an automated metric to detect whether a new model version has made summaries worse. Which metric is the conventional choice for summarisation, and what is its limitation?

  • ABLEU, and it cannot handle summaries longer than the reference.
  • BPerplexity, and it only applies to models trained on the same corpus.
  • CAccuracy, and it requires a labelled test set that most teams do not have.
  • DROUGE, and it measures overlap with a reference rather than whether the summary is true.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
AI Engineering, On building an evaluation set before shipping.
Template
Experiment brief, The belief being tested, a hypothesis with a threshold and a date, how long the test runs and on how much traffic, and a table saying in advance what you will do with each possible result.