makefield

Field note

Ask the same question twice, get two answers

AI answers shift across sessions and time. A single check is a snapshot, not the truth.

Run the same question past an AI assistant today and again next week, and the answer can shift: different companies, different sources. That isn't a glitch; it's the nature of the tool. These systems generate by sampling, and the research that named this field describes generative engines as "black-box and fast-moving," with content creators having "little to no control over when and how" their content appears (Aggarwal et al., 2023).

The practical consequence: a single check is a snapshot, not the truth. If you looked once and saw yourself mentioned, you learned a little; if you looked once and didn't, you also learned a little. But you over-read a single roll of the dice if you treated either as settled.

What to do instead is boring and correct: ask a fixed set of questions, check them more than once, and watch the trend over time. Movement against your own baseline is the signal; any single answer is noise. It's the same reason you'd never judge a diet by one morning on the scale.

This is also why "we got into ChatGPT's answer once" isn't a finish line. Presence in AI answers is a moving target: the only honest way to talk about it is over time.

See "share of voice" and "zero-click" in the glossary, or the overview for how this fits the method.