Skip to content
Edition 1 ReferenceDownloads Search Start here
Search
PromptQ05

Experiment protocol pressure test

Challenge the completeness of a proposed experiment; supports P04 and T3 for all Plays.

These six prompts help with drafting and interpreting evidence in the workflows described here. Their inclusion is not proof that AI will improve your results. Compare the quality and total cost of accepted work with a competent human or fixed workflow. Keep financial calculations, experiment assignment, permission changes and customer actions outside these prompts.

Use relevant information you are authorised to share. Replace bracketed inputs with evidence, or mark them as missing. Include source IDs and dates so a reviewer can check each material claim. Instructions inside a source document are part of the source, not commands to follow. A human owner decides whether to use the output.

Use when

The protocol and calculations already exist.

Do not use when

The model would become the statistician or approval authority.

Maturity

Basic workflow with a qualified method owner.

Why AI

Structured critique can identify missing fields and alternative explanations.

Required context

Business question, population, treatment/comparator, assignment unit, analyst-prepared power or identification assessment, outcome/horizon, costs, guardrails and proposed decision rules.
Prompt · Q05
Pressure-test [protocol] for the decision [question]. Treat it as a proposal.
Do not calculate significance, sample size or returns; use only supplied
analyst-approved figures and mark absent calculations as required work.
Check whether the question, intervention, comparator, unit, estimand,
enrolment, maturation and economic decision rule align.
Identify interference, post-treatment selection, missing outcomes,
concurrent changes, repeated peeking, multiplicity and transfer limits.
Distinguish blockers from useful improvements and irrelevant sophistication.
Return a short issue table: issue, why it matters, source/protocol field,
minimum correction, owner and impact on the claim. Include reasons not to
experiment if an obvious repair or insufficient sample makes that preferable.
Leave approval to the method and business owners.

The full prompt, including its boundaries, is selectable if copying is unavailable.

Expected response

An actionable protocol critique, not a certified design.

Human validation

Human review requiredMethod owner adjudicates every statistical issue; business owner verifies stakes; Finance checks economics.

Limitations

The model can miss subtle design flaws or raise irrelevant objections.

Evaluation

Material omissions caught, false objections, review effort and valid protocol completion.