Domain 3 of 6

Techniques to Evaluate and Red Team

Building an evaluation set of your own cases, red teaming it deliberately, and why a model topping a public benchmark tells you very little about how it will do on your work.

Concepts

~17%

Of the exam

Practice questions

Concepts in this domain

01Red teaming an AI systemDeliberately trying to make your own system misbehave before somebody else does, how to run it, and what to do with what it finds.02Evaluating a foundation modelROUGE, BLEU and BERTScore, what each one can and cannot see, and why human evaluation stays the reference the automated scores are checked against.03Evaluating a machine learning modelWhy accuracy is usually the wrong number, what precision and recall are really trading against each other, and the business metrics that decide whether any of it was worth doing.

Try a question from this domain

A team plans to red team its assistant by having the two engineers who built it spend a day trying to break it. What is the main weakness of that plan?

ABuilders test the paths they designed and share the assumptions that created the gaps, so they find less.
BA day is too short a period for meaningful adversarial testing.
CEngineers are not qualified to assess harmful content.
DRed teaming should be performed by an external firm to be valid.

8 questions on this domain.

One per page, with a worked explanation.

Start the set