A simulation is a well-grounded estimate of how a group would behave. It is not a measurement of how they did, and it is not a forecast of events.
It is not a prediction of the future
A date in the future does not give twins foresight. It means they reason from what is known to be true at that point — announced prices, scheduled changes, established trends — and from whatever the environment records as an assumption, marked as one.
So a simulation set next spring tells you how this group would behave under these conditions. If the conditions turn out differently, the answer was not wrong; it was answering a different question than the one reality asked.
It is not a substitute for the real experiment
If you can run the actual test — a real price change in a real market, a real email to real people — the real test wins. Always. Use simulations to decide which experiments are worth running, to rule out options before you spend on them, and to understand results you already have. Not to replace the one experiment you can afford.
The exact percentages are approximate
Confidence is a statement about direction, ranking and rough magnitude. Two options whose ranges overlap have not been separated, however different their headline numbers look. Quoting "63%" as a precise figure will eventually embarrass someone; quoting "roughly two thirds, and clearly ahead of the alternative" will not.
Twins are built from evidence, and evidence is uneven
Some twins are built from records about individuals and their own words. Others are drawn from measured distributions to make a population's composition correct. Both are grounded in real data; they are grounded to different depths, and groundedness tells you the mix.
For a population with a low individually-grounded share, trust the aggregate and treat any single twin as illustrative rather than as evidence about a person.
Where attributes have been combined across different studies, they are right at the level of the pattern and looser at the level of the individual. A twin's spending is right for someone like them; the link between their spending and their attitudes is a little weaker than it would be in a real person.
Populations skew where their sources skew
A population built mostly from what people publish online reflects who publishes online. A population built from your customer records reflects your customers, including whatever selection got them there. Neither is a defect — it is the source telling you it describes a particular group of people — but it is a reason to read the source mix before reading the answer.
Small groups say less than they appear to
A subgroup breakdown showing 41% among under-25s is only as good as the number of under-25s in the sample. Groups too small to say anything about are suppressed, and groups that are just large enough carry wide ranges. Do not build a strategy on a cell you have not checked the size of.
It answers the question you asked
A missing option in the action space redistributes itself across the options you did list, invisibly. A scenario that hints at a preferred answer moves the result. A question that names a consideration makes it salient. None of these produce an error — they produce a clean-looking result to a question you did not mean to ask.
The setup is where accuracy is won or lost, which is why so much of this documentation is about the five fields rather than about the engine.
What it is unusually good at
For balance, the things it does that are hard to get any other way:
- Ranking options and sizing the gap between them, quickly and cheaply.
- Attributing a difference to one change, because everything else can be held exactly constant in a way no field test allows. See Comparison.
- Surfacing reasoning you had not considered, at a scale where a rationale held by 12% of a market is visible rather than lost in a focus group of eight.
- Asking follow-up questions of the specific people who gave a specific answer, immediately, as many times as you like.