Product Digest

Product Digest Product Metrics and Analytics Certification

A certification for product managers who have to choose what to measure, instrument it, read what comes back and run an experiment that somebody can trust. Six modules with an assessment of 90 questions.

Practice questions (90)Free. Sign in to answer, so your result is kept and reported by module.
At a glance
Cost
Free There is no examination fee and no course fee.
Format
Six modules and a single assessment of 90 questions covering all of them. The reading is open to anyone. The assessment requires an account.
Pass mark
70 per cent, or 63 of 90, measured across the whole assessment rather than per module.
Prerequisites
None. The material assumes a few years of product experience and no statistics beyond arithmetic. Every statistical idea the assessment needs is built up on the page that uses it.
Renewal
None. A result records the syllabus as it stood at the time of assessment.
Based on
The a syllabus developed by Product Digest, drawing on the published experimentation literature and on how product organisations actually instrument and read their products
What it covers

What it covers. Product Digest issues this certification. No external body accredits it and the assessment is not proctored. It gives a structured route through the material and a record that a candidate has completed the reading.

The six modules follow the order the work happens in. Choosing what to measure, instrumenting it, reading behaviour, designing an experiment, reading the result honestly, and turning the answer into a decision. Each module opens on the one before it, so the reading rewards being taken in order.

Two modules cover ground most analytics material skips. Instrumentation asks where the numbers come from and why they are wrong, which decides whether anything downstream can be trusted. Reading a result honestly takes the five ways an experiment misleads a team that ran it correctly.

The statistics are built up rather than assumed. Significance, power, sample size and the multiple comparisons problem each appear where a product decision needs them, with the arithmetic worked through.

The syllabus6 domains · free

Product Digest’s own domains. Each opens its own page listing the concepts beneath it, and each concept has a page of its own.

01Choosing What to MeasureThe first decision, made before any tooling. What product analytics is for, the kinds of number a product produces, the north star metric and the frameworks around it, and how a measure is derived from a strategy rather than from whatever the tool reports by default.Product analytics · Product metrics · The north star metric · Product metric frameworks · Choosing a product metric5 concepts
~15%02Instrumenting a ProductWhere the numbers come from. Events and their properties, the tracking plan that keeps a taxonomy readable as a team grows, how one person is recognised across devices and sessions, what consent law allows a product to collect and what refusing it does to the figures, and the data quality failures that quietly invalidate everything downstream.Events and properties · The tracking plan · Identity and sessions · Privacy and consent in analytics · Data quality in analytics5 concepts
~12%03Reading Product BehaviourWhat the instrumented product is saying. Funnels and where people leave, cohorts that separate a change from a trend, retention curves and whether they flatten, churn measured in customers against churn measured in revenue, the activation moment that predicts staying, how engagement is counted, the segments hiding inside an average, and why a generative feature needs measures of its own.Funnels and conversion · Cohort analysis · Retention curves · Churn and revenue retention · Activation · Engagement and frequency · Segmentation in analytics · Measuring an AI feature8 concepts
~22%04Designing an ExperimentTurning a question into a test that can answer it. What randomisation buys, the hypothesis and the criterion a result is judged against, how many users are needed and for how long, the guardrails that catch damage a winning metric hides, and what to do when an experiment is impossible.Controlled experiments · Hypotheses and the overall evaluation criterion · Sample size and statistical power · Guardrail metrics · Experiment duration and exposure · Alternatives to a controlled experiment6 concepts
~18%05Reading a Result HonestlyThe five ways a correctly run experiment still misleads the team that ran it. What significance does and does not claim, why checking early inflates the false positive rate, the split that proves the plumbing is broken, effects that fade once novelty wears off, and what happens when enough metrics are tested at once.Statistical significance · The peeking problem · Sample ratio mismatch · Novelty and primacy effects · Multiple comparisons5 concepts
~16%06Turning Analysis into DecisionsThe part that changes the product. Deciding to ship, iterate or abandon on the evidence available, what happens to a measure once a team is judged against it, the questions behavioural data cannot answer at all, writing a finding somebody who was absent can act on, and building a practice that accumulates knowledge rather than dashboards.Reading an experiment result · Goodhart's law and metric gaming · What analytics cannot answer · Communicating analytics findings · Building an experimentation practice5 concepts
~17%
Try a question

A train operator's finance team reports that season ticket revenue fell four per cent in March. The product team is asked what to do about it. What can product analytics contribute that the revenue figure cannot?

  • AAn account from season ticket holders of why they stopped renewing, gathered from the people who left.
  • BA more accurate revenue figure, broken down by route and by ticket type.
  • CA count of what people did inside the product, such as the share of renewal reminders that led to a renewal being started in the app, and the date that share moved.
  • DA forecast of what season ticket revenue will do over the rest of the year.
90 questions across the 6 domains.

One per page, with a worked explanation.

Start the set