Detection Is Not Provenance
Why an AI detector score is evidence about production, not a record of the production process
Disclosure: FERZ, Inc. builds deterministic governance infrastructure for AI systems. This article concerns the evidentiary status of AI text detection and does not describe a FERZ product.
This argument does not require AI text detectors to be inaccurate. It requires only precision about what they measure.
When an institution treats detector output as a finding about how a document was produced, it assigns evidentiary weight to a model-generated estimate. That estimate may be useful evidence about production. It is not a contemporaneous record of the production process.
A note on the word. This article uses provenance to mean an event-linked recorded history of how a document was created and modified, not a generic indication of origin.
1. What a detector produces
Detection methods vary, and watermarking differs in kind. Supervised classifiers, likelihood-based scoring and zero-shot techniques estimate from finished text. A watermarking system instead inserts a signal during generation and later tests the finished text for that signal, so part of the evidence is created at the time of generation rather than inferred entirely afterward.
That distinction is real, and it does not rescue detection as provenance. A detected watermark may be evidence that a compatible generation process produced or preserved a signal. It ordinarily does not record the production history, establish authorship, capture transformations applied later, or bind the signal to an institution's policy rule. A watermark surviving in a document that a person wrote and a model lightly edited shows that the model touched it. It does not show how much, or whether that was allowed.
For the estimating methods, take one documented system rather than generalizing.
Pangram describes an initial training dataset of approximately one million documents, uses "mirror prompts" to match AI-generated examples to human examples on style, tone and content, then mines false positives and retrains on the resulting hard cases [1]. The system converts a document to embeddings and returns a label with a confidence score.
Note what that output is a function of. Not the document alone: the document, the detector, the model version, the training distribution, the decision threshold and the calibration. Change the threshold and the same document can change class. That dependence matters when the result is assigned evidentiary weight.
2. Two kinds of evidence
The comparison below concerns the estimating methods, not watermarking.
| Text-based detector estimate | Provenance record |
|---|---|
| Applies a model to finished text | Records assertions about creation and modification events |
| Produces a score or label | Produces an event-linked history or attestation |
| Depends on model, version, threshold, reference data | Depends on capture process, binding, signer, trust model |
| May support an inference about production | Provides contemporaneous evidence of asserted production history |
| Can be accurate or inaccurate | Can be valid, incomplete, misleading, or untrusted |
| Does not become an event record through greater accuracy | Integrity verification does not establish that the assertions are true |
The concession is deliberate. Detection can be evidence about provenance. Improvements are real, and better detectors can produce stronger evidence of that kind. Improved performance does not change the evidence type.
Improved performance strengthens a detector's inferential value without changing when or how the evidence was created. A version history may preserve a platform-mediated record of editing events as they occur. A detector score is an estimate made afterward, from the finished text alone.
3. Where the deployed instruments stand
The tempting version of this argument is that production leaves no recoverable trace in finished text. Recent Pangram-affiliated research directly contradicts that proposition.
EditLens [2], from Pangram Labs with a co-author at the University of Maryland, predicts the magnitude of AI editing in a document on a continuous scale, from the edited text alone. The authors state that "only the edited document y is available at inference time" and that "the model never conditions on x at inference." On its evaluated datasets, EditLens shows that a model can estimate a similarity-derived measure of AI edit magnitude without access to the source.
The authors also limit the claim: "Our objective is not to attribute authorship, but to predict the magnitude of change." Where a human draft has been reworked by a model, they describe authorship as "latent and entangled within the editing process," and treat word- or sentence-level binary authorship labels as ill-posed.
Then the numbers that bear on institutional practice. On a three-way task separating human, AI-generated and AI-edited text:
| System class | Type | Macro-F1 | AI-generated F1 | AI-edited F1 |
|---|---|---|---|---|
| Best-performing binary detector in the benchmark | Binary | 69.5 | 89.0 | 43.2 |
| Best-performing EditLens configuration | Regression (Cosine) | 90.4 | 94.1 | 86.8 |
Source: EditLens, Table 2. "Best-performing binary" refers to macro-F1 under thresholds calibrated on the authors' validation set. The source identifies every evaluated system.
For the best-performing binary system, F1 fell from 89.0 on fully AI-generated text to 43.2 on AI-edited text. Across all three binary systems, AI-edited F1 did not exceed 61.0. Under this evaluation, endpoint performance did not transfer to the mixed category that a policy permitting limited assistance would need to distinguish.
The shape of the output points the same way. The paper measures correlation against similarity-based proxies for edit magnitude, noting there are no direct or exact labels for the quantity. EditLens correlated at r = -0.606 with semantic similarity, 0.799 with Levenshtein distance and 0.781 with Jaccard distance. The best-performing binary detector reached -0.491, 0.615 and 0.556 on the same measures. That system does track edit magnitude to a degree. It tracks it less closely, and on this benchmark its scores clustered near zero or one rather than following the graded polish levels.
An independent study reaches a compatible result. Saha and Feizi evaluated twelve detectors across 15,000 samples of AI-polished text and report that "most detectors not only flag a large portion of minor-polished texts but also struggle to differentiate between the degrees of AI involvement" [3]. Their illustration is stark: RoBERTa-large classified 47.69% of minor-polished texts as AI-generated and 51.98% of major-polished texts, a difference of about four points across the study's minor- and major-polish conditions. In one extremely minor-edit condition, the three commercial systems evaluated in the study classified between 32.31% and 64.71% of samples as AI-generated.
One concrete illustration of what gets collapsed. Running nine default Grammarly instructions over human-written text, EditLens places "Fix any mistakes" at the bottom of the edit-magnitude distribution and "Summarize it" and "Make it more detailed" at the top. Those are different acts, and a binary label has no vocabulary for the difference.
Editing, rather than generation, is a large share of real usage. In an analysis of consumer ChatGPT messages, Writing was the most common work-related use case at roughly 40% of work messages, and "about two-thirds of all Writing messages ask ChatGPT to modify user text (editing, critiquing, translating, etc.) rather than creating new text from scratch" [4]. That study covers consumer plans only and excludes Education plans, so it does not describe student usage directly. It establishes that editing workflows are not a marginal use case.
4. The gap between the label and the question
Where an institution permits limited AI assistance but prohibits more extensive generation or rewriting, the governing question is not merely whether a model was involved. It is whether the particular assistance violated the declared rule.
A detector label does not by itself establish a policy violation. Where the rule distinguishes permitted assistance from prohibited assistance, the label does not reach the distinction, because the permission lives in policy and the detector was trained on text. Where a rule prohibits assistance of any kind, the definitional gap closes and the evidentiary one does not: reliability, calibration and fit to the screened population still stand between a score and a finding. The EditLens authors make the corresponding point: estimating degrees of involvement could support policies that permit some forms of assistance while prohibiting others.
The evidentiary problem follows. An aggregate accuracy figure alone does not supply the relevant base rate, distributional fit, threshold-specific performance, or calibration. Nor is distributional fit a theoretical concern: EditLens, the strongest model in its own comparison, drops from 0.904 macro-F1 to 0.866 on a held-out domain and 0.850 on a held-out model. Those results show that performance changed under two specified forms of distribution shift.
Turnitin's guidance and the EditLens authors each state a version of the caution. Turnitin says its model "may not always be accurate (it may misidentify human-written, AI-generated, and AI-paraphrased text)" and "should not be used as the sole basis for adverse actions against a student," and reports elevated false positives when the reported percentage falls between 0 and 19 [5]. The EditLens ethics statement is blunter: "Inaccurate AI detection software can cause harm as false accusations of AI misconduct can result in serious consequences, including emotional trauma, reputation damage, and undue punishments for academic misconduct. We acknowledge that our model has a non-zero error rate and its errors may result in such harms."
The caveats exist. The open questions are whether they reach the people making decisions, and what those people are given to use instead.
5. Records tied to the production process
What evidence is tied to the production process itself?
Contemporaneous capture, bound to the artifact. Some authoring environments preserve revision histories, including Google Docs and cloud-hosted Word documents. These records may provide process evidence, although they can be incomplete and do not necessarily reveal whether an external tool was used. Content provenance infrastructure does a more formal version: C2PA defines Content Credentials covering images, video, audio and documents, binding signed assertions about creation and modification to the asset and allowing recorded actions performed by AI or ML systems to be identified through digitalSourceType [6].
C2PA is candid about the limits. Its explainer states that "provenance information alone cannot tell you whether the digital content is true, accurate or factual." Provenance may be incomplete where an asset is modified by a tool that does not update it. Credentials can be removed, which is why the specification introduces durable Content Credentials. The explainer also rejects an automatic adverse inference from absence: it is "not the intention... to create a two-tier media ecosystem where assets without Content Credentials are universally less trusted."
That last limit deserves emphasis. Where provenance capture was optional or unavailable, the absence of a record should not be treated as evidence of AI use.
Version histories carry a different problem. They are not self-authenticating and not necessarily complete. An institution relying on them should declare in advance how they will be captured, verified and weighted. That is a governance gap, and closing it requires no advance in detection at all.
Definitional note: "provenance" here means the recorded history of a document's creation and modification. Under the Five Tests Standard, Provenance is a normative test concerning the established origin of the inputs grounding a verdict [7]. Both use provenance in the sense of origin rather than truth. They apply it to different objects.
6. The structural caution
There is a parallel to governance work more broadly. It should be stated narrowly, because the problems are not the same.
A text detector estimates characteristics of a completed artifact. An authorization artifact binds a pre-execution verdict to the proposed action and the policy and authority state used to produce it at a runtime authorization boundary. The objects, timing and failure modes are different.
The shared error is substitution of downstream observation for event-bound evidence. In text detection, the substitution is a detector score for a provenance record. In authorization, it is an execution log for a pre-execution authorization artifact. Better observation may sharpen the account of what happened, and still leave the question of what was authorized beforehand untouched.
The relevant question is what was bound at the time of the event, by whom, and to which artifact or action. A conclusion reconstructed afterward may still be useful evidence. It is not the record it is being asked to replace.
References
- Pangram, "How AI Detection Works." https://www.pangram.com/research/how-it-works
- K. Thai, B. Emi, E. Masrour, M. Iyyer, "EditLens: Quantifying the Extent of AI Editing in Text," arXiv:2510.03154, October 2025. https://arxiv.org/abs/2510.03154
- S. Saha, S. Feizi, "Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing," Findings of the ACL 2025. https://aclanthology.org/2025.findings-acl.1303
- A. Chatterji, T. Cunningham, D. J. Deming, Z. Hitzig, C. Ong, C. Y. Shan, K. Wadman, "How People Use ChatGPT," NBER Working Paper 34255, September 2025. https://www.nber.org/papers/w34255
- Turnitin, "Using the AI Writing Report." https://guides.turnitin.com/hc/en-us/articles/22774058814093-Using-the-AI-Writing-Report
- C2PA, "C2PA and Content Credentials Explainer," version 2.4. https://spec.c2pa.org/specifications/specifications/2.4/explainer/Explainer.html
- E. Meyman, "The Five Tests Standard (5TS)," Zenodo. Concept DOI: https://doi.org/10.5281/zenodo.21040295. Implementation repository: https://github.com/edmeyman/4ts-standard