How much an AI visibility number moves on its own
Before celebrating a rise or panicking over a drop, it helps to know how much the number moves without anyone having touched anything.
An AI visibility number does not sit still
It moves for three different reasons, and only one of them has anything to do with you.
- The model is stochastic. The same question, the same day, to the same engine, gives different answers. We measured it: of 269 questions asked three times in a row, in 12 of the 27 cases where the brand appeared it did not appear all three times.
- The engine changes without warning. Between 24 and 31 August 2026, Google AI Mode went from citing 12.22 sources per answer to 4.08, and eleven brands lost a third of their visibility in the same week without having changed anything.
- Engines are not alike. Two search engines answering the same question share between 4% and 13% of their sources.
A time series mixes all three. Separating them is half the work of measuring.
What margin of error the figure carries
It depends only on sample size, and it is far larger than almost anyone publishes.
Every percentage is published with its 95% Wilson confidence interval and with the number of observations behind it. The canonical example: over 90 questions, a 10% presence actually means somewhere between 5% and 18%. This is not a small-print caveat. It means the number you are shown could be nearly double or nearly half.
There is a specific trap on top of that: repetitions of the same question are not independent of each other. If you repeat 90 questions three times, for margin purposes you do not have 270 observations, you have something closer to 90. That is why the interval is computed on the effective sample and not on the total number of runs, which is the cheap way to make an interval look narrower than it deserves. The detail is in the methodology.
A zero does not always mean you are invisible
With zero appearances in n attempts, the real rate can still be as high as 3/n.
| Runs | The real rate could still be | Can you say "I do not appear"? |
|---|---|---|
| 3 | ~63% | No. You have measured nothing. |
| 10 | ~30% | No. |
| 30 | ~10% | Starting to be defensible. |
| 60 | ~5% | Yes. |
This cuts both ways. A zero over three runs is not a diagnosis of invisibility, and it is not proof that a tool "detects nothing" either. An honest zero always comes with its upper bound and with how many more runs would be needed.
How to tell whether a rise is real or noise
By comparing the change against the margin of your own sample: if it is smaller than the margin, nothing happened.
The practical rule is that any movement smaller than your sample interval is noise. To contrast two periods you use a two-proportion test, and the verdict is stated out loud: real drop, real rise, noise, or insufficient data. Two examples from the same corpus, to see the difference:
- This one is real: the panel Share of Answer fell from 23.4% to 15.9% over 990 weekly questions, and the following week it was still at 16.7%. Large sample, large movement, and it did not bounce back. The full finding.
- This one is not: going from 10% to 13% over 90 questions. The interval on that 10% already ran from 5% to 18%, so the 13% fits entirely inside the previous margin.
A report showing you a green arrow without telling you the sample size is asking you to celebrate noise.
Why two tools give different figures for the same brand
Because there are at least four legitimate ways to count, and they produce very different numbers over exactly the same data.
| Counting rule | What it requires | Effect on the figure |
|---|---|---|
| Mention in any run | Being named at least once | The highest number and the least useful |
| Selection in any run | Being recommended at least once | Noise-sensitive: one lucky run counts like a pattern |
| Majority selection | Being recommended in 2 of 3 runs of the same question | The strictest |
| Proportion of runs | The percentage of individual runs with a selection | The one we publish, with its interval and its n |
And that is only the arithmetic. On top of it sit four decisions that change the result just as much: which questions are asked, in what language and from what country, which engines go into the average, and whether questions that already name the brand have been excluded. A question containing your name yields almost 100% presence by construction; putting it in the headline is measuring yourself in a mirror.
Practical consequence: comparing two tools' figures means nothing unless you know which rule each one uses. If you are choosing one, the criteria are in how to choose an AI visibility tool.
What to demand from an AI visibility report
Six things, and all six are asked for before signing, not after.
| What to ask for | Why it matters |
|---|---|
| The n behind every percentage | Without a sample, a percentage cannot be read or compared |
| The interval on every figure | It is what separates a rise from a coincidence |
| The raw answers | Without them you cannot check how anything was classified |
| The counting rule, in writing | Four different rules give four different figures |
| The questions, with language and country | Change the questions and you change the result |
| What the data does not prove | A report that only asserts has not looked at itself |
Apply it to ours too. If any vilnux report fails this table, discard it.
Two of our own figures that came out wrong
We publish them because a method that has never been wrong in front of anyone is not a method, it is a brochure.
The first: our accuracy measurement returned 99.5%. It was false. Of the 107 checks producing that figure, 106 could not fail by construction. Once we required every data point to be contradictable by the answer, the real figure came out at 64.2% over 53 checks. It is all in this finding.
The second: until July 2026 our detector marked a brand as recommended if it appeared in the text and some recommendation word appeared anywhere, without checking who that word referred to. Re-analysing 2,031 real answers with proximity attribution, the "recommendations" went from 79 to 31. 62% were false positives, and in fourteen of them the recommendation explicitly went to a competitor. The correction took our own reported rate from 3.9% down to 1.5%.
That is the bias you would expect from a metric nobody audits: it inflates in the direction the client likes.
What this page does not solve The margin of error can be calculated; representativeness cannot. The set of questions is a human choice, not a census of your market's real demand: change the questions and you change the result, and no confidence interval protects you from that. It does not turn correlation into causation either. Outside experiments with a control group, everything measured here is observational.
Frequently asked questions
What margin of error does measuring a brand in AI carry?
It depends on sample size, and it is larger than usually published. Over 90 questions, a 10% presence actually means somewhere between 5% and 18%, with a 95% Wilson interval. Repetitions of the same question are also not independent, so the interval is computed on the effective sample rather than on the total number of runs.
How do you know whether a rise in AI visibility is real or noise?
By comparing the change against the margin of your own sample: if the movement fits inside the previous interval, it is noise. Going from 10% to 13% over 90 questions is not a rise, because the interval on that 10% already ran from 5% to 18%. A drop like the panel one in August 2026, from 23.4% to 15.9% over 990 weekly questions and sustained the following week, is.
What sample size do you need to measure visibility in AI search engines?
To claim you do not appear, around 60 runs. With zero appearances in n attempts the real rate can still be as high as 3/n: with 3 runs that is 63%, with 30 it is 10% and with 60 it is 5%. Below 30 runs, a zero only tells you that not enough has been measured.
Why do two GEO tools give different figures for the same brand?
Because there are at least four legitimate ways to count and they produce very different numbers over the same data: mention in any run, selection in any run, majority selection out of three, and proportion of runs. On top of that sit the questions chosen, the language and country, which engines go into the average, and whether questions naming the brand are excluded.
Are the tools that claim to measure visibility in ChatGPT reliable?
The ones that publish how they count and on what sample are, and only to the extent that they publish it. The risk is not theoretical: our own detector reported a 3.9% recommendation rate, and re-analysing 2,031 answers with proximity attribution it came out at 1.5%, with 62% false positives. A metric nobody audits inflates in the direction the client likes.
What should you demand from an AI visibility report?
Six things: the n behind every percentage, its confidence interval, the raw answers, the counting rule in writing, the questions with their language and country, and what the data does not prove. If any of the six is missing, the figure cannot be checked or compared against any other.
See your figure with its interval
The free audit returns every percentage with its sample and its confidence interval, and shows you the full answers it came from. No signup, no card.
Run my free audit →Home · Plans & pricing · Methodology · Research · Perplexity · Metrics · How to choose · Use cases · Changelog · Agencies · Alternatives · Diagnosis · Check if AI cites you · Four-step method · Optimizing for agents · X · Product Hunt