A URL is not a citation

Peer review was built for a world where a human made the analytic decisions and could reconstruct them. Ask how a number was produced and someone could tell you, because someone had decided.

That assumption no longer holds. When an agentic pipeline runs an analysis, the decisions still happen, dozens of them, but they happen inside the system. What gets reviewed is the output. The reasoning chain stays in the pipeline.

The uncomfortable part is that we already know how much those decisions matter.

The decisions are the result

Give the same dataset and the same hypothesis to enough independent teams and they do not converge. Seventy neuroimaging teams analysed one dataset against nine pre-specified hypotheses, and no two teams chose identical workflows. Their statistical maps were highly correlated at intermediate stages and their hypothesis tests still diverged.¹ In ecology, 174 teams working two datasets produced effect sizes ranging from -1.55 to 0.38 on one and -4.47 to 0.39 on the other, with heterogeneity attributable to analytic choices of 97.6 and 98.6 percent.²

The finding that should stop you is from a study of 161 researchers in 73 teams testing one hypothesis on one dataset. Their results split roughly a quarter negative, over half null, and a sixth positive. The authors then coded every decision each team had made, and reported: "More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow."³

Read that twice. They had the teams, the data, the outputs and the decision logs, and they still could not explain the spread. Reconstruction after the fact does not recover the path. If the path is not recorded as it happens, it is gone.

The citation is not the evidence

The same problem sits underneath every sourced claim, and it long predates any of this.

A meta-analysis of 46 studies covering 32,074 quotations found an inaccuracy rate of 16.9 percent, of which 8.0 percent were major errors. It also found no improvement over time: the slope across recent years was flat.⁴ A survey of 2,648 corresponding authors, asked to judge citations to their own work, found a quotation error rate of 16.6 percent with no significant variation by discipline. Only 11.3 percent of them had ever done anything about it.⁵

Two different methods, one measuring the literature and one asking the people best placed to know, landing within a third of a percentage point of each other.

The mechanism is not mystery. Reviewing the studies, one analysis found that of content errors, 64.8 percent were major, meaning the source failed to substantiate or contradicted the assertion, and concluded that this "might suggest that authors are not reading or possibly comprehending prior research."⁶ A 2026 study of three anaesthesia journals is sharper still: 98 percent of references retrieved successfully, and 16 percent of quotations were wrong, 68 percent of those majorly.⁷

Nearly every link resolved. One in six did not say what it was cited for.

The retraction data closes the loop. Across 13,252 citation contexts to papers that had already been retracted, 5.4 percent acknowledged the retraction. The authors put it plainly: retraction "did not change the way the retracted papers were cited."⁸ A separate corpus of 3,149 retracted publications found roughly a third of their 89,056 citations arrived after retraction, with about 4 percent of citing papers mentioning it.⁹

A citation can be live, formatted, resolvable, recent and wrong. Nothing about its appearance distinguishes the one somebody read from the one somebody found.

Regulated industries settled this decades ago

Here is what makes the current position strange. The answer is not novel. It is written down, in force, and old.

United States federal law has required, since 1997, "secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records."¹⁰ Two words carry it. Computer-generated, so the record is not an account written afterwards. Independently, so it is not the operator's account of themselves.

European GMP guidance says the same thing in plainer language, requiring consideration of "building into the system the creation of a record of all GMP-relevant changes and deletions," and adds that for a change the reason must be documented, not merely the fact of it.¹¹ British data integrity guidance goes furthest, defining the original record as the source capture "and all subsequent data required to fully reconstruct the conduct of the GXP activity."¹² The reconstruction material is not something produced on request. It is constitutive of the record.

And the AI Act restates it for AI systems: high-risk systems "shall technically allow for the automatic recording of events (logs) over the lifetime of the system."¹³ The obligation attaches to the architecture, not to the diligence of whoever operates it. High-risk application dates have since been deferred to December 2027 and August 2028.¹⁴ Deferred is not repealed.

Meanwhile the publishing standards resolve accountability by assigning it to a person. Authors must agree "to be accountable for all aspects of the work," and must remain responsible for AI-assisted material, because AI "can generate authoritative-sounding output that can be incorrect, incomplete, or biased."¹⁵ That is correct and insufficient. It places the duty on a human without requiring the system to produce what the human would need to discharge it.

What a sourced claim actually requires

Four things, and none of them are satisfied by a working link.

That the document was read, and the record says so. Fetching is retrieval. Retrieval is not comprehension, and the 16 percent gap between resolving and supporting is exactly the space between them.

That the claim is tied to the passage, not the document. A citation to a forty-page guidance is not a citation. The article, the clause, the date.

That the decisions are logged as they are made. Not summarised, not reconstructed. The 95 percent unexplained variance is what reconstruction gets you.

That the record is an output of the system, not a report about it. Computer generated, contemporaneous, independent of the party it vouches for.

"Verified" without a verification record is a claim about a feeling.

This applies to us. Anyone reading should be able to take those four tests to any provider of sourced analysis, including this one, and get a straight answer about which are met and which are not. A standard that only its author can pass is not a standard. It is marketing.