Domain 1 of 7

Fundamentals of AI Safety

The field separates risk into misuse, accidents and structural harm. Why each kind needs a different remedy, and where the disagreements between researchers lie.

Concepts

~16%

Of the exam

Practice questions

Concepts in this domain

01What AI safety meansThe three kinds of risk the field separates, why each of them requires a different remedy, and where the disagreements inside the field actually lie.02How AI features failThe ways an AI feature goes wrong in production, ordered by how often a product team actually meets them, with the cause behind each and the control that has a chance against it.03What generative AI is good and bad atThe capabilities worth building on, the failure modes that are properties of the technology and will outlast any model, and how to tell a suitable problem from an unsuitable one.04Catastrophic risk arguments and their critiquesThe case that rising capability could produce harm of a different order, the serious objections to it, and why both camps ask for much the same work.

Try a question from this domain

A company finds that its recruitment model has been used by a manager to screen candidates in a way the company never authorised, using a workflow the manager built themselves. The model works exactly as designed. Which category of risk is this?

AMisuse, because the system worked as intended and somebody used it to cause harm.
BAn accident, because the model was applied to a task it was not evaluated for.
CStructural harm, because it reflects competitive pressure inside the company.
DMisalignment, because the model pursued an objective its designers did not intend.

12 questions on this domain.

One per page, with a worked explanation.

Start the set