Message Testing Survey Template
Two documents and three sheets that split every response by segment and hold each variant to a response-count floor before naming a winner.
Free download · No account needed
Response Analysis
Before a Single Response Comes In
Every variant-by-segment cell tracked against a 30-response floor before any ranking gets trusted.
| Variant | |
|---|---|
| Segment | |
| Top-2-box % | |
| Clears 30-response floor? |
The variant that wins the blended sample and the variant that wins the segment that approves the purchase are not always the same one.
Every message testing template on page one asks the same question, which variant scored highest, and ships that one. Thornfield, a fictional accounts-payable vendor, ran that exact test on 240 respondents across four message territories and got a clean answer: the compliance angle won at 65.1% top-2-box purchase intent, twelve points ahead of the runner-up. Split the same 240 responses by role instead of reading them as one blended number, and compliance falls to last place among the 108 economic buyers who actually approve the purchase, at 33.3%.
The reversal is real, and so is the reason the blended number missed it. Compliance's economic-buyer cell held only 9 responses, under a third of the 30-response floor message-testing guidance built on a large participant panel sets for a directional read in any single cell. This pack tests every variant with a monadic design, one respondent judging one message alone rather than several side by side. It then splits Response Analysis by the economic buyer and the end user separately, flagging every cell that falls under the floor before its ranking gets trusted.
Decision Log then records which variant ships where and why, rather than only the chat message everyone forgets by the next planning cycle. Thornfield's actual call: cost control shipped to the buyer-facing homepage and sales deck because it cleared the floor and won among economic buyers. Compliance moved to onboarding and in-product copy instead of being discarded, since its end-user result was real and well sampled. Pair this pack with the messaging framework the variants are drawn from, then test a homepage rewrite against real responses before it ships everywhere unchecked.
What's in the pack
Test Design
The monadic method, the two segments recruited by role, and the 30-response floor applied to every variant-by-segment cell instead of each variant's total.
Variant Register
Three to five message territories with the hypothesis behind each one, so a wording tweak never gets mistaken for a genuinely different argument. Pair it with the positioning canvas when the territories themselves need re-deriving from the alternatives customers actually consider.
Response Analysis
Every variant-by-segment cell's respondent count, top-2-box score, and whether it clears the floor, with the blended ranking and the per-segment rankings placed side by side so a reversal is visible immediately.
Result Readout
The worked Thornfield example in full, including the blended ranking that crowned the wrong variant and the segment split that caught it before the homepage rewrite shipped.
Decision Log
What shipped to which surface, what got held instead of discarded, what got retired, and the specific retest trigger that would reopen each call. Once a variant ships to reps, the sales enablement kit checks whether real calls actually say it.
The segment-floor rule
The rule against crowning a winner below 30 responses in its own segment cell, with the worked example's own reversal as the reason it exists rather than being a formality.
How to use it
- 1
Send your message variants
Or take it blank and replace the worked example, four territories tested on a fictional accounts-payable vendor, with your own variants and who you plan to survey.
- 2
Recruit both segments separately
River computes the top-2-box score for every variant-by-segment cell as responses arrive, and flags any cell that falls under the 30-response floor before it gets read as a result.
- 3
Compare the blended and segment rankings
Two rankings, never one. When the blended ranking and the economic-buyer ranking disagree, as they do in the worked example, the segment ranking governs which variant ships to the buyer-facing surfaces.
- 4
Log the decision and the retest trigger
Decision Log records which variant shipped where, what got held instead of discarded, and the specific condition that would reopen the call next quarter.
Frequently asked questions
Is this template free, and what format are the downloaded files?
Free, and the download is not cut down. The zip holds all five files, two documents in Word and three sheets in CSV, no signup and no card required. Edit with AI is the optional half: send River your own message variants and survey responses, and it runs the same segment split and floor check on your numbers. Every pack sits in the template library.
What does Edit with AI actually do?
It installs this pack as a private Space and asks for your message variants plus who you plan to survey. Once responses arrive, it computes the blended top-2-box score and the score inside each segment separately, flags any variant-by-segment cell under the 30-response floor, and helps you decide which variant ships to which surface.
Why not just read the blended score across all respondents?
Because a blended score can be carried almost entirely by whichever segment was easiest to recruit in volume. On this pack's own worked example, the compliance variant's 65.1% blended score, the highest of four, came mostly from 132 end-user responses while its own economic-buyer cell held only 9. Read only the blended row and the wrong message ships to the audience that actually signs.
How many responses do I need before trusting a result?
At least 30 completed responses in that specific variant-by-segment cell, not 30 for the variant's total. A variant can clear 30 overall by leaning on one easy-to-recruit segment while its other segment's cell sits at single digits, and Response Analysis flags that cell as unusable regardless of how the blended number looks.
What happens when a variant wins one segment but fails the floor in the other?
It does not get crowned an overall winner. Decision Log routes it to the surfaces where its result is well sampled, onboarding or in-product copy for an end-user win. It leaves a retest trigger for the segment that failed the floor rather than discarding the result or averaging it away.
What if I can only recruit one segment for now?
Run the test anyway, and say so in Decision Log rather than treating the one segment you reached as representative of both. A result that only covers the end user still tells you something real about onboarding copy, but it cannot decide what ships on the buyer-facing homepage.
Find out which message actually wins with the segment that signs
Send your message variants and who you plan to survey. River splits every response by segment and flags any cell too thin to trust before a winner gets named.
Edit with AI