Product Digest

Product Digest AI Safety Practitioner Certification

Our own certification, for people who ship AI features rather than research them. Six modules and an assessment of 48 questions.

Practice questions (48)Free. Sign in to answer, so your result is kept and reported by module.
At a glance
Cost
Free No exam fee, no course fee, and nothing to upgrade to.
Format
Six modules and one assessment of 48 questions covering all of them. The reading is open to anyone. The assessment needs an account.
Pass mark
70 per cent, or 34 of 48, across the whole assessment rather than per module.
Prerequisites
None. It assumes you work on products rather than models. The Foundation certification covers how models are trained, where that goes wrong, and what evaluation and interpretability can establish, and it is worth taking first if none of that is familiar.
Renewal
None. The material will date as the law and the failure modes move, so a result records what you knew when you took it.
Based on
The our own syllabus, written from the published curricula of BlueDot Impact, the Center for AI Safety, DeepMind and Stanford
What it covers

What it covers. This is ours. Product Digest issues it, no external body accredits it, there is no proctor and nobody checks who is at the keyboard. It is worth what the reading is worth.

Foundation and Practitioner divide the subject rather than repeat it. Foundation explains how a model is made to behave and where that goes wrong. This one covers what to do about it when you ship something.

So the six modules follow a decision rather than a discipline. What goes wrong, whether to build it, how to test it, what to put around the model, what to watch afterwards, and what the EU AI Act and the NIST framework require.

Every question is a scenario. None asks for a definition, because knowing what a hallucination is was never the hard part.

The syllabus6 domains · free

Product Digest’s own domains. Each opens its own page listing the concepts beneath it, and each concept has a page of its own.

01Failure Modes in Deployed SystemsThe failure modes a product team meets rather than the ones papers are written about. Hallucination, bias, injection, misuse and the quiet one, which is people trusting output further than it deserves.How AI features fail · Over reliance and automation bias · What generative AI is good and bad at · Responsible AI in practice4 concepts
~18%02Suitability and Human OversightThe judgement that prevents more harm than any control does. What a wrong answer costs, whether a person has to be in the path, and the cases where the honest answer is that this should not be an AI feature.When not to ship an AI feature · Human oversight · Adopting generative AI in a business · Choosing and configuring a foundation model4 concepts
~17%03Techniques to Evaluate and Red TeamBuilding an evaluation set of your own cases, red teaming it deliberately, and why a model topping a public benchmark tells you very little about how it will do on your work.Red teaming an AI system · Evaluating a foundation model · Evaluating a machine learning model3 concepts
~17%04Controls and Safeguards Around a ModelDefence in depth for something that cannot be made reliable on its own. Filtering both directions, grounding answers in sources that can be checked, and least privilege for anything an agent is allowed to touch.Securing an AI system · Prompt engineering · Agents and their tools · Grounding and retrieval4 concepts
~18%05Monitoring and Incident ResponseThe stage teams skip. Drift, monitoring that watches quality rather than uptime, what a rollback criterion is for, and how to handle the day it produces something that reaches the press.AI incident response · The machine learning lifecycle · Transparency and explainability3 concepts
~15%06Regulatory Requirements and GovernanceThe EU AI Act's risk tiers and what each obliges, the NIST framework's four functions, ISO 42001, and the documentation you will wish you had started keeping earlier.The EU AI Act · The NIST AI Risk Management Framework · AI governance and compliance · Secure AI and the Secure AI Framework4 concepts
~15%
Try a question

A support assistant has been live for four months. Usage is steady, no incidents have been raised, and nobody has measured answer quality since launch. The team reports the feature as healthy. What is the strongest objection?

  • AFour months is too short a period to draw any conclusion about quality.
  • BThe absence of complaints is weak evidence, because most people who receive a poor answer stop using the feature rather than reporting it.
  • CUsage should be falling if the feature is working, since it resolves issues.
  • DQuality cannot be measured after launch without a control group.
48 questions across the 6 domains.

One per page, with a worked explanation.

Start the set