{
  "@context": "https://schema.org",
  "@id": "https://denescsaszar.com/drift-2026-08",
  "identifier": "MF-2026-001",
  "@type": "ScholarlyArticle",
  "name": "The floor test, scope 08284347",
  "url": "https://denescsaszar.com/drift-2026-08",
  "creativeWorkStatus": "exploratory, results read 2026-08-29 (scope 08284347)",
  "author": {
    "@id": "https://makefield.io/#denes",
    "name": "Denes Csaszar"
  },
  "publisher": {
    "@id": "https://makefield.io/#org"
  },
  "citation_text": "Csaszar, D. (2026). The floor test, scope 08284347. Makefield study MF-2026-001. https://denescsaszar.com/drift-2026-08",
  "abstract": "Answer engines rarely word an answer the same way twice. This study asked eight answer engines eighteen questions about one company, Makefield, in seven capture waves between 4 July and 25 August 2026. Inside each wave, the repeat answers to one question from one engine set that cell's own noise floor: how much wording the repeats share. A cell counts as moved only when its answers across two waves share less wording than that floor. Of the 103 question-by-engine cells that have a floor, 85 moved. The study is exploratory: the analysis was specified after the data existed, the measure compares wording and not facts, and each wave is also one sitting, so the result cannot separate change over time from a difference between sittings.",
  "keyFindings": [
    "85 of the 103 cells with a noise floor (82.5 per cent) were less alike across waves than inside one wave. Under a permutation null, 40.9 per cent would be expected by chance (from the published summary).",
    "Repeat answers inside one wave already share little wording: the median within-wave similarity is 0.266, the median between waves 0.183 (Jaccard over content-word sets).",
    "Inside the same cell, the longest gap between waves gave less alike answers than the shortest in 87 of 108 cells; median difference minus 0.047."
  ],
  "question": "Do answers to the same question from the same engine move between capture waves by more than they move inside one wave?",
  "subject": "One company, Makefield (makefield.io), chosen as a subject with little published about it. Its site was not public during the study.",
  "engines": [
    "ChatGPT",
    "Claude",
    "Copilot",
    "Gemini",
    "Google AI Mode",
    "Meta AI",
    "Mistral",
    "Perplexity"
  ],
  "waves": "7 capture waves, 4 July to 25 August 2026",
  "sample": "18 questions; 111 question-by-engine cells with draws in two waves were compared, 103 of them with a noise floor; 12 cells excluded because the question text changed between waves; 2,952 records folded into the cells.",
  "method": "Per cell, the noise floor is the median Jaccard similarity over content-word sets between repeat answers inside one wave. A cell counts as moved when its between-wave similarity is strictly below its own floor. The count of moved cells is tested against a permutation null (5,000 replicates, seed 0); the median difference has a bootstrap interval clustered by question (2,000 replicates).",
  "methodVersion": "scope 08284347",
  "results": "Median per-cell difference (between-wave minus within-wave similarity): minus 0.065, 95 per cent interval minus 0.092 to minus 0.030, clustered by question.",
  "limitations": [
    "Exploratory: the analysis was specified after the data existed.",
    "Similarity compares wording, not facts. It is a floor on likeness, not a reading of meaning.",
    "Each wave is also one sitting, so the result is net of engine and question, not of period.",
    "One subject, chosen because little has been published about it; nothing transfers to a company the engines already know.",
    "Nothing here shows why anything changed. No intervention was controlled.",
    "The questions are not published.",
    "The eight engines are run by seven companies, and the captures were made in a way six of those seven companies' terms prohibit. The study page names the document and clause for each provider."
  ],
  "data": [
    {
      "name": "floor-test-cells-08284347.csv",
      "url": "https://denescsaszar.com/data/floor-test-cells-08284347.csv",
      "sha256": "691a5a7c59fcb3644806b2ec81610dca5c588a841f859c3fc057baf46c8f018b"
    },
    {
      "name": "floor-test-gap-pairs-08284347.csv",
      "url": "https://denescsaszar.com/data/floor-test-gap-pairs-08284347.csv",
      "sha256": "909fc37cbbc49c4fa1198b4af5c89f30e01c8cd5072a894e4a6d390d52e3bd2c"
    },
    {
      "name": "floor-test-summary-08284347.json",
      "url": "https://denescsaszar.com/data/floor-test-summary-08284347.json",
      "sha256": "cf21d8a97360246bd52f9cfc6a8be5f00f9d7b7e08e0592a91c97856c1d5394b"
    }
  ],
  "license": "CC BY 4.0 (data files)",
  "fingerprint": "SHA-256 of each data file, as above; the same values are in SHA256SUMS.txt next to the files.",
  "invented": false,
  "recordPage": "https://makefield.io/studies/floor-test/"
}