Skip to content
Edition 1 ReferenceDownloads Search Start here
Search
PlayP04 Measurement · Acquisition and investment · Editorial rank 3 of 12

Validate incremental acquisition

Material media or partner spending receives credit for outcomes that may have happened without it.

Decision
Keep, reduce, increase or redesign a defined acquisition tranche?
Primary commercial metric
Estimated incremental contribution net of media, testing and additional operating costs.
Readiness
Method and sample ready
Agent role
None for methodological authority or budget execution.
01

Fit

When to use this Play

A budget decision could change and a feasible holdout or other defensible comparison can detect a useful effect.

When not to use it

No budget choice is available, spillover invalidates the design, or the learning cost exceeds its decision value.

Why it should work

An appropriate counterfactual separates credited activity from changed customer outcomes.

Basic version

One material question and one qualified, affordable comparison; use honest limited inference when no useful test is feasible.
02

Prepare

Required maturity
Basic commercial records plus qualified method support. Ready with usable units, variance and outcome horizon. Not ready when sample or identification cannot inform the choice.
P01 Establish decision-specific customer economics — Adequate decision-specific economics is required; the whole P01 Play is unnecessary if already satisfied.Check readiness for P04 (D02)

Inputs

Business question, eligible units, historical variance, assignment/exposure records, all-channel outcomes, contribution, spend and concurrent changes.

Audience or customer

A legitimate, separable user, account or geographic population; evaluate all assigned eligible units.

Human role

Measurement owns method; budget owner authorises exposure and subsequent allocation; Finance reconciles the economic outcome.

Systems required

Existing channel/CRM experiment capability or an appropriate specialist design, first-party outcome data and T3 register.

Guardrails

Customer treatment, strategic floors, essential coverage, cash, test collisions and protected commitments.

How to configure a Play

Copy the Play you need and complete only the relevant parts of T1–T5. Name the people responsible, define the customer group, agree economic and customer limits, and set a review date before starting. An empty decision threshold is not permission to act. Basic, Connected and Advanced describe how the work is configured; Experimental describes its evidence. AI is optional: compare each AI step with a simpler way of doing the same work.

03

Do the work

Intervention

A predeclared holdout, suppression or spend-level contrast on the selected activity.

Workflow

  1. Specify the investment decision.

  2. Assess power/identification and interference.

  3. Register protocol, limits and readout.

  4. Verify assignment.

  5. Run without uncontrolled changes.

  6. Mature outcomes.

  7. Review effect, cost and uncertainty.

  8. Decide and retain the record.

AI role

Optional assist Optional protocol completeness or signed-readout drafting; cannot choose significance or calculate authoritative effects.

Automation role

Assignment, integrity checks, cost joins and validated statistical analysis.

Agent role

None for methodological authority or budget execution.

Failure modes

Repeated peeking, only analysing converters, platform outcomes replacing business outcomes, and extrapolating one lift factor everywhere.
04

Measure and decide

Measurement

Use a scoped estimand, intention-to-treat where appropriate, effect interval and total net contribution on the declared horizon.

Primary commercial metric

Estimated incremental contribution net of media, testing and additional operating costs.

Leading indicators

Assignment integrity, enrolment, contamination, delivery contrast and outcome completeness; not interim success claims.

Experiment

This Play is the experiment. Select randomised individual/account/geo design where valid; document assumptions for quasi-experimental alternatives.

Guardrails

Customer treatment, strategic floors, essential coverage, cash, test collisions and protected commitments.
Scale when
Apply results only to comparable tranches/populations; expand within uncertainty and revalidate after material changes.
Stop / reassess when
Stop or suspend for harm, invalid assignment or irreparable contamination. At the planned readout, declare inconclusive rather than inventing precision.
05

Learn and progress

Learning

Store full protocol, nulls, deviations, effect interval, cost, applicability and the actual budget decision.

Practical tool

Next action
ask the method owner whether the smallest useful effect is detectable in time.
Open T3 Decision experiment and evidence record

Improves

06

Optional depth

Advanced versionPrerequisites: see Required maturity above.

Reusable holdout infrastructure, portfolio test coordination or calibrated aggregate models when volume and decisions justify them.

Evidence for this Play2claims reviewed, with confidence ratings, limits and sources.

Claim classifications

Accepted means a principle or mechanism survived critical review, not that it guarantees a return. Accepted with conditions means prerequisites apply. Qualified means the original claim has been narrowed. Advanced needs substantial readiness. Experimental needs a capped comparison. Unresolved means the evidence cannot settle the claim. Rejected means the recommendation is excluded.

High, Medium and Low describe confidence in the stated target, not the chance of a positive ROI. For a Qualified or Rejected historical claim, confidence applies to the decision to narrow or reject it and to the supported limit—not to the original headline. Read the headline and classification together. An accounting identity or design principle is not empirical evidence of a commercial outcome. Follow the linked chapters, Plays and tools to see where each claim applies.