---
title: What a synthetic panel cannot tell you
description: A synthetic panel is not a cheap sample. Four questions it cannot answer, and why the honest output is an abstention rather than a number.
---

# What a synthetic panel cannot tell you

Published 2026-09-24

The fastest way to make synthetic research untrustworthy is to ask it the questions a field
answers. A synthetic panel built on public survey data carries the structure of that data. It does
not carry a fresh probability sample, and it cannot create one.

Four questions sit outside what it can answer.

## 1. Incidence of a behaviour

How many people did a thing last month? Incidence requires a sample drawn from the population, with
known inclusion probabilities. A panel of personas is drawn from a distribution, and the distribution
is the answer you would be trying to measure. Asking it for incidence returns the assumption of the
model, dressed as a measurement.

## 2. Market share

Market share is incidence with brands attached, plus a share-of-wallet question. It needs a sample
of buyers, not a sample of profiles. The personas can tell you which arguments a segment finds
credible. They cannot tell you how much of the market a brand holds.

## 3. Margin of error

A margin of error is a statement about sampling. It has no meaning for a synthetic panel, because
there is no sampling frame and no randomness in the draw that a confidence interval would describe.
Reporting one is not optimistic; it is a category error.

## 4. Prevalence of a rare behaviour

Rare behaviours are rare in the source data too. If a behaviour appears in 1.5 percent of the
calibration sample, a panel that reflects that sample contains too few cases to say anything
segment by segment. The usual workaround is to oversample the rare group, which then breaks the
distribution it was supposed to represent.

## What it can do instead

The defensible uses are about the instrument and the structure of opinion:

- **Item diagnosis.** Does this item measure the dimension it claims, and does its wording move the
  answer?
- **Scale behaviour.** Does the scale survive a reversal, and do the points separate?
- **Segment structure.** Which groups differ, and by how much relative to the spread within groups?
- **Sensitivity.** Which item is the one whose wording changes the conclusion?

Each of those is a statement about the questionnaire and about relative structure, not about level
in a population.

## Abstention is an output

When the calibration data leaves a segment ambiguous, the persona abstains. An abstention is a
usable result: it marks the boundary of what the data supports. A fabricated answer is not, because
it enters the report indistinguishable from a measured one.

The rule is simple and worth stating plainly: a synthetic panel is a pretesting instrument. It tells
you whether the question can be answered. It does not answer it for the population.