How many prompts make a real AI visibility score?

Ask ChatGPT the same question twice and you get two different lists. That is not a bug, it is how generative models work. It also means most AI visibility reports are quoting a single roll of the dice as if it were a survey result.

The variance problem

In our runs, a single non-branded prompt on a single engine names a given store somewhere between 40% and 100% of the time across ten repetitions, depending on the market. The order of the stores changes more often than that. Run once and you get one sample. Publish it and you have told a dealer he is fourth when he might be second.

What a defensible run looks like

Prompt set. At least four prompts per bucket (Find, Buy, Service), so a single odd phrasing cannot decide the result. Twelve prompts per market minimum.

Locations. Each anchor city separately, because a store's visibility in Alpharetta and in Marietta are different numbers.

Engines. Each reported on its own.

Repetitions. Three runs per prompt, per engine, per city, on three different days. For a seven-city market on five engines that is 12 × 7 × 5 × 3, or 1,260 answers. That is the floor.

Reported spread. Alongside every mean, the range across runs. If store A is 23% and store B is 21% and the spread is five points, they are tied and the report should say so.

What to ask a vendor

How many prompts? How many repetitions? Which days? Can I see the raw answers? If a report has a single visibility number with no run count, ask for the count. If the answer is one, the report is a screenshot with a chart on it.

Why we publish the spread

Because dealers make decisions on these numbers. A spread makes the number smaller and less exciting. It also makes it true. Our methodology commits to it, and every client run stores the raw answers so the spread can be recomputed by anyone.

Does more repetitions always mean a better report?

Up to a point. Three runs on three days catches most of the variance. Ten runs tightens it further. One run tells you almost nothing about the second run.

Why do engines give different answers each time?

Sampling. Models generate probabilistically, and search-grounded engines may retrieve different pages on different requests. Both add variance.