Activation

Activation is the earliest action in a person's life with a product that separates the people who go on to stay from the people who leave.

A flattening retention curve says that a stable share of every intake keeps coming back, and it says nothing about what that share did differently. Activation is the search for that difference, carried out in the first days of a person's life with the product.

The vocabulary arrived from two directions. Dave McClure put activation second in the five stages of his 2007 startup metrics framework, meaning the point where somebody has a first good experience of the product. The phrase aha moment came from growth teams describing the same point from the customer's side, which is the moment the product stops being a form to fill in and starts being useful.

A product manager meets this question as soon as anybody asks what to build into onboarding. Onboarding can hold perhaps three steps before people abandon it, so the choice of which three is a real choice with a real cost. Activation analysis is how that choice stops being a matter of taste.

The sections below give the method for finding a candidate action from data a team already has. One worked comparison follows, with figures that reconcile. The correlation at the centre of the finding is then set out in full, and the last section covers the two properties a definition has to hold.

Finding a candidate by comparing retained and churned cohorts

The method starts from a cohort old enough to have separated. A signup cohort from at least sixty days ago splits into people still active now and people who are gone, and every action either group took in their first week is already recorded.

For each candidate action, a team calculates two figures. The first is retention among the people who took the action. The second is retention among everybody who did not. A wide gap between those two makes the action a candidate, and a narrow gap rules it out.

Ranking by the gap alone produces a shortlist with an obvious flaw on it, which is that some actions almost nobody takes. An action performed by two per cent of a cohort may separate that two per cent perfectly and still be worthless as an onboarding target, because the other ninety eight per cent of the population is untouched by it. Coverage and separation are therefore read together.

One comparison worked through

The worked example is a cohort of four thousand signups at a bookkeeping product, measured sixty days after they arrived. Nine hundred of the four thousand are still active, which is an overall retention of 22.5 per cent. Three candidate actions were recorded in the first seven days.

The last row is the control on the exercise. Changing the logo splits the cohort into 26.6 per cent and 22.1 per cent, which is close enough to the overall 22.5 per cent to say that the action carries almost no information. Any method that cannot rule this row out is finding patterns in noise.

The first two rows both separate strongly and they do not rank the same way on both measures. Inviting a colleague gives the higher retention among the people who did it, at 61.1 per cent against 55.5 per cent. Connecting a bank feed reaches far more of the cohort, at 1,394 people against 957, which is 35 per cent of the population against 24 per cent.

A team choosing between them is trading depth against reach. Moving the bank feed rate from 35 per cent to 50 per cent touches six hundred more people in this cohort, and moving the invite rate by the same fifteen points touches the same six hundred at a slightly better rate. The honest answer usually comes from which one the product can actually influence, since a customer with no colleague cannot invite one at any level of persuasion.

The correlation at the centre of the finding

Every number in the table above is a correlation, and this is the point on which most activation work goes wrong. Three explanations fit the bank feed result equally well and they imply completely different work.

1. Connecting a bank feed makes the product useful, so people who connect one stay. The action causes the retention.
2. Some customers had already decided to move their bookkeeping across, and connecting a bank feed early is part of what moving involves. The intent causes both the action and the retention.
3. Connecting a feed is possible only at a supported bank, and the supported banks are the larger ones used by better funded businesses. A third factor causes both.

Under the first explanation, pushing more people to connect a feed raises retention. Under the second, it raises the connection rate and leaves retention where it was, because the people newly pushed through the step never had the intent that made it matter. Under the third, the useful work is supporting more banks and has nothing to do with onboarding at all.

Observational data cannot choose between those three. The cohort table shows what happened to people who behaved differently for their own reasons, and no amount of extra slicing turns that into a claim about what would happen if the product changed. Settling it requires a deliberate change applied to a randomly chosen half of new signups, which is the instrument the next module of this course is about.

Seven friends in ten days and what the story leaves out

The gap between a correlation and a test is visible in the most repeated activation story in the industry. Facebook's growth team compared engaged users against users who drifted away, and the comparison produced a threshold of roughly seven friends in the first ten days. Chamath Palihapitiya, who led that team, described the finding publicly at a growth conference in 2012, and the team organised its work around moving people over that line.

Two things about the story are usually dropped in the retelling. The number itself is a round figure taken from a range, so six friends in twelve days would have described the same pattern about as well, and treating seven as a precise constant misreads what the analysis produced. More importantly, the team did not stop at the correlation. They built and tested a long sequence of changes aimed at the friend count, and the tests are where the causal claim came from.

A team quoting the story as licence to pick a number and chase it has taken the first half of the work and left the second half behind.

Two properties a usable activation definition needs

Both halves of that work rest on a definition precise enough to build on. Two properties decide whether a candidate definition can do the job.

It has to sit early enough to act on. An action that most people take in their fifth week describes loyalty and arrives too late to design onboarding around. The useful definitions sit inside the first session or the first week, because that is the window in which a product still has somebody's attention.

It has to name a behaviour with a count and a time limit. Understanding the value of the product is not a definition, since nothing emits an event for it. Connecting a bank feed within seven days is a definition, because a query returns a number and two people running the query get the same answer.

A definition holding both properties gives a team one more thing worth having, which is a measure that moves within weeks. Retention at day sixty answers in two months. Activation rate answers in seven days, and a team that has established the link between them can use the fast measure as a stand in for the slow one.

Staying is one thing and coming back often is another. A cohort can retain at 22 per cent with those people opening the product daily or twice a month, and the difference between those two products is the subject of the next page.