Product Digest AI Safety Practitioner Certification

A certification for the product people who build and ship AI features. Six modules with an assessment of 90 questions, and a practical exercise on every page.

At a glance
Cost
Free There is no examination fee and no course fee.
Format
Six modules and a single assessment of 90 questions covering all of them. The reading is open to anyone. The assessment requires an account.
Pass mark
70 per cent, or 63 of 90, measured across the whole assessment as a single total.
Prerequisites
None. The material assumes product experience and requires no background in building models. The Foundation certification covers how models are trained, where that training fails and what evaluation and interpretability can establish. Foundation is worth taking first where that material is unfamiliar.
Renewal
None. Regulation and failure modes move, so a result records the syllabus as it stood at the time of assessment.
Based on
The a syllabus developed by Product Digest, drawing on the published curricula of BlueDot Impact, the Center for AI Safety, DeepMind and Stanford
What it covers

What it covers. Product Digest issues this certification. No external body accredits it and the assessment is not proctored. It gives a structured route through the material and a record that a candidate has completed the reading.

Foundation and Practitioner divide the subject between them. Foundation covers how a model is trained to behave and where that training fails. Practitioner covers the decisions and controls that follow once a system is deployed.

The modules follow the sequence of a deployment decision. The first three cover what fails in production, whether the team should build the feature at all and how to evaluate and red team it. The last three cover the controls that belong around the model, what to monitor after release and what the EU AI Act and the NIST framework require.

Every page closes with a practical exercise that takes about fifteen minutes with an ordinary language model client. The assessment is scenario based throughout.

The syllabus6 domains · free

Product Digest’s own domains. Each opens its own page listing the concepts beneath it, and each concept has a page of its own.

01Failure Modes in Deployed SystemsThe failure modes a product team meets in a shipped system. Hallucination, bias, injection, misuse and the quiet one, which is people trusting output further than it deserves.How AI features fail · Over reliance and automation bias · What generative AI is good and bad at · Responsible AI in practice · The OWASP LLM Top 105 concepts
~17%
02Suitability and Human OversightThe judgement that comes before any control. What a wrong answer costs, whether a person has to sit in the path and the cases where the honest answer is that the team should not ship an AI feature at all.When not to ship an AI feature · Human oversight · Adopting generative AI in a business · Choosing and configuring a foundation model4 concepts
~13%
03Techniques to Evaluate and Red TeamBuilding an evaluation set from the team's own cases, red teaming it deliberately and why a model at the top of a public benchmark says little about how it will do on the team's own work.Red teaming an AI system · Evaluating a foundation model · Evaluating a machine learning model · How to Build LLM Evaluations: Cases, Graders and Release Criteria · Prompt Injection Explained: Direct and Indirect Attacks5 concepts
~20%
04Controls and Safeguards Around a ModelDefence in depth around a model that nobody can make reliable on its own. Filtering in both directions, grounding answers in sources a reader can check and least privilege for anything an agent is allowed to touch.Securing an AI system · What Is Prompt Engineering? Techniques and Examples · What Are AI Agents? Tools, Workflows and Permissions · LLM Grounding Explained: Sources, Citations and Hallucinations · Guardrails and output handling · Excessive agency and tool permissions · Hidden context exposure7 concepts
~23%
05Monitoring and Incident ResponseThe stage teams skip. Drift, monitoring that watches output quality, what a rollback criterion is for and how to handle the day a system produces something that reaches the press.AI incident response · The machine learning lifecycle · Transparency and explainability · Cost and consumption controls4 concepts
~12%
06Regulatory Requirements and GovernanceThe EU AI Act's risk tiers and what each obliges, the NIST framework's four functions, ISO 42001 and the documentation a team wishes it had started keeping a year earlier.The EU AI Act · The NIST AI Risk Management Framework · AI governance and compliance · Secure AI and the Secure AI Framework · The AI supply chain5 concepts
~15%
Try a question

A support assistant has been live for four months. Usage is steady, no incidents have been raised, and nobody has measured answer quality since launch. The team reports the feature as healthy. What is the strongest objection?

  • AFour months is too short a period to draw any conclusion about quality.
  • BUsage should be falling if the feature is working, since it resolves issues.
  • CQuality cannot be measured after launch without a control group.
  • DThe absence of complaints is weak evidence, because most people who receive a poor answer stop using the feature rather than reporting it.
90 questions across the 6 domains.

One per page, with a worked explanation.

Start the set