Domain 3 of 6

Techniques to Evaluate and Red Team

Building an evaluation set from the team's own cases, red teaming it deliberately and why a model at the top of a public benchmark says little about how it will do on the team's own work.

Concepts

~20%

Of the exam

Practice questions

Concepts in this domain

01Red teaming an AI systemDeliberately trying to make a system misbehave before somebody else does, how the exercise is run, what it cannot establish and what regulators now expect of it.02Evaluating a foundation modelROUGE, BLEU and BERTScore, what each one can and cannot see, why a public benchmark is a filter and no decision, and why human evaluation stays the reference the automated scores are checked against.03Evaluating a machine learning modelWhy accuracy is usually the wrong number, what precision and recall are really trading against each other, why the threshold is a product decision and the business metrics that decide whether any of it was worth doing.04How to Build LLM Evaluations: Cases, Graders and Release CriteriaBuild an LLM evaluation around real task requirements, with test cases, deterministic checks, human rubrics and explicit release criteria.05Prompt Injection Explained: Direct and Indirect AttacksUnderstand direct and indirect prompt injection, how untrusted content redirects LLM applications and how to test the boundaries around data and tools.

Try a question from this domain

A team plans to red team its assistant by having the two engineers who built it spend a day trying to break it. What is the main weakness of that plan?

AA day is too short a period for meaningful adversarial testing.
BEngineers are not qualified to assess harmful content.
CBuilders test the paths they designed and share the assumptions that created the gaps, so they find less.
DRed teaming should be performed by an external firm to be valid.

18 questions on this domain.

One per page, with a worked explanation.

Start the set