Why AI answers change when nothing else did

The experiment we did not mean to run

On 2 September 2026 we scanned a customer’s question bank twice by accident. The first scan finished at 06:07:17 UTC. The second started at 06:07:20 — three seconds later, same questions, same engines, same brand, same market. Nothing about the world had changed in between.

Of the 49 question-and-engine verdicts both scans scored, 4 disagreed. Three of them were Gemini, one Perplexity. That is 8.2% of the measurement flipping with no cause upstream of it.

The part worth sitting with: both scans found the brand cited exactly 18 times. The headline was identical. Underneath it, four verdicts had swapped places. A report that showed only the total would have looked perfectly stable while 8% of its evidence churned.

So we measured it properly

That pair was one accident, so we went back through every scan we have stored. For each pair of consecutive scans of the same brand, we compared the recorded verdict for every question and engine that both scans scored. Verdicts are taken after the majority vote, the same way a report collapses repeated runs, and an engine that returned no grounded answer is left out rather than counted as a miss.

That gives 740 comparable verdicts across 34 scan pairs and 7 brands, between July and September 2026. 100 of them flipped: 13.5%.

Then we split those pairs by how far apart the two scans ran, because a gap of three weeks can contain a genuine change and a gap of two days effectively cannot:

Read that twice. Results measured two days apart moved slightly more than results measured weeks apart. Whatever is driving the churn, it is not the passage of time, and it is not your marketing. It is the engines.

It is not the same across engines

Restricting to the short-gap pairs, where a real change is implausible:

Claude is four to five times steadier than Gemini. That is not explained by how often we ask: Gemini and Claude are both asked three times per question on Growth and up, and Perplexity and ChatGPT once. Repetition is not what separates them — the engines simply behave differently. If one number on this page changes a decision you make, let it be this one: a move on Gemini needs more evidence behind it than the same move on Claude.

Why it happens

Four mechanisms, none of them a fault in anyone’s product:

• The answer is generated, not retrieved. A model samples its output. The same input can produce different text, and a brand that was worth naming in one pass gets left out of the next for no reason you could point at.
• The search underneath it is live. Grounded answers run a real web search first. Indexes shift, results reorder, and a source that made the cut at 06:07:17 can miss it at 06:07:20.
• Whether the engine searches at all varies. Sometimes it answers from training knowledge and never looks anything up. We record that as no grounded answer rather than a miss, but it means the set of questions actually measured differs run to run.
• Prominence is unstable even when presence is not. You can be cited in both runs and move from second source to eleventh, which changes the score without changing whether you were mentioned.

Why this does not contradict our methodology page

How we measure reports that repeating a call within one scan left Perplexity’s verdicts unchanged. This page reports that 17% of Perplexity verdicts flipped between scans days apart. Both are true, and the gap between them is the finding: an engine can be stable across three calls in the same minute and unstable across the same call on two different days. Repeating a question inside one scan buys you less than it looks like it should.

What to do with this

• Do not act on one week’s movement. On the numbers above, a week-over-week change of roughly one verdict in six is inside the range these engines produce on their own. It is not a result.
• Read direction over several weeks, not levels between two. Four readings trending one way is evidence. One reading against the one before it is a coin landing.
• Weight the engines differently. A Claude move is worth more than a Gemini move of the same size.
• Look at which questions moved, not just the score. The back-to-back pair above had an identical total and four swapped verdicts. If the same three questions flip every week, that is noise with a pattern. If a question that never used to name you starts naming you and stays, that is the thing you were looking for.
• Be suspicious of a supplier whose chart only ever goes up. Ours does not, and any honest measurement of this will sometimes report a drop it cannot explain.

How to check this yourself

Everything above comes from two tables, geo_scans and geo_results, and one comparison: group the stored rows by question and engine within each scan, take the majority verdict, then diff consecutive scans of the same brand. We are describing the method rather than only the result so that a customer on Growth or above can run it against their own history and get their own number. If yours disagrees with ours, we would like to see it: hello@mediadesk.asia.

Background: what generative engine optimisation is, and how we measure it.