The vocabulary the exams and the daily work both assume you already have. Every entry gives the definition, the source it is drawn from, and the misreadings that follow from getting it wrong.
39 concepts in Metrics, page 1 of 2
The early action that separates people who stay from people who leave, how a candidate action is found by comparing retained and churned cohorts, the two properties a usable definition needs, and the correlation at the centre of every activation finding.
Metrics
The changes that cannot be randomised into two groups, the four designs available when a split is impossible, what each one assumes and how much smaller the claim becomes once the randomisation has gone.
Metrics
What an organisation needs to run tests as routine work, how many tests can share the same traffic, the record that stops a question being asked twice, the arithmetic of experiment velocity and what a one in three success rate means for the way a programme is built.
Metrics
The three charts teams use to track progress, what each one shows that the others hide, and why Scrum names all three without requiring any of them.
Metrics
Deriving a measure from the claim a product strategy makes, the goal, driver and guardrail roles a measure can play, the five tests a candidate has to pass before a team commits a quarter to it, and the two failure modes that leave a team steering by a number it cannot use.
Metrics
Churn counted in customers set beside churn counted in money, the formulas for gross and net revenue retention, two worked cases where the two counts point in opposite directions, and what net revenue retention above one hundred per cent does to the growth a company has to buy.
Metrics
Grouping people by when they arrived or by what they did and following each group forward, the difference between an acquisition cohort and a behavioural cohort, a worked case where a blended figure falls while no cohort gets worse, and the three directions a cohort table is read in.
Metrics
What turns an analysis into a finding somebody who was absent can act on, the four parts of a written result, why a chart with no claim attached settles nothing and what belongs in the record after the decision is made.
Metrics
Random assignment as the only practical route from correlation to cause, what the control and treatment groups are for, how the assignment unit is chosen and where it goes wrong, and the question an experiment answers that no amount of observation can.
Metrics
The six failures that quietly stop collected data matching what happened, from an event that never fires to a field renamed in a release, the checks that catch each one, reconciliation against a system of record, and the honest way to report a loss that cannot be recovered.
Metrics
How often people come back, daily over monthly active users as a ratio with a worked case where two very different products report the same 20 per cent, counting active days in a period, and the natural frequency that sets the ceiling for any product.
Metrics
The event model a product analytics practice is built on, the three kinds of property and why the difference between them decides what a team can ask later, an object action naming convention, and the choice between many specific events and few general ones carrying properties.
Metrics
Scrum.org's framework for measuring value, built on four Key Value Areas and a goal structure that turns strategy into a sequence of experiments.
Metrics
The three things that set how long a test runs, why the answer comes in whole weeks, the exposure point that decides who is counted, and the arithmetic showing that a one per cent ramp needs about twenty five times the duration of an even split.
Metrics
A funnel as a sequence of steps with a count at each one, the three choices that decide what it measures, a worked conversion calculation from forty thousand visitors down to four hundred and eighty customers, and why the denominator has to be stated before any rate means anything.
Metrics
Charles Goodhart's 1975 observation, the three ways a product measure gets gamed once a team is judged on it, why most gaming is done by people acting in good faith, and the defences that keep a measure honest under pressure.
Metrics
Measures watched for damage rather than for improvement, the two kinds Kohavi, Tang and Xu separate, a table of common guardrails with what each one protects, and the arithmetic showing how a two per cent win can be a loss once latency is priced.
Metrics
The hypothesis as a prediction specific enough to fail, the four parts it needs, the overall evaluation criterion set out by Kohavi, Tang and Xu in 2020, and a worked composite where the weights agreed in advance decide whether the same result reads as a win or a loss.
Metrics
How an analytics tool decides that two events came from one person, what stitching an anonymous identifier to a known account can and cannot recover, the browser storage limits that make a returning visitor look new, and the session as a timeout rule that changes every measure counted per visit.
Metrics
Acceptance rate and the unit it is counted in, edit and override rate, the error rate a team has to build because nothing emits an event for a wrong answer, the arithmetic of the trade between latency and quality, and why ordinary engagement measures point the wrong way for a generative feature.
Metrics