Which frameworks require pre-specified criteria — and what do they accept as proof?

If an evaluation is supposed to be judged against criteria fixed in advance, someone eventually has to show that they were. This page reports what five regulatory documents actually say, measured the same way: the primary text of each was read in full and searched for the mechanisms it defines — for the dataset, for the model, and for the acceptance criteria themselves.

What each framework requires

• EU GMP Annex 22 (EudraLex Volume 4, consultation draft, July 2025 — not in force; the text in force is Annex 11). §4.2: acceptance criteria "should be documented and approved before the start of acceptance testing"; §7.2 requires an approved test plan containing "the pre-defined metrics and acceptance criteria" before the test is initiated. For the test data it is far more specific: §6 requires access control, audit trail logging accesses and changes, no copies outside the repository, staff separation and a four-eyes principle. §10.2 requires measures to "detect any unauthorised change" to the model. For the criteria: no such mechanism.
• ITU, Designing Regulatory Experimentation (June 2026). Methodological Note 3: "Every experimentation space must define regulatory hypotheses from the design stage (ex ante) and contrast them with observed results at the close (ex post), guaranteeing traceability between evidence and the normative decision." The clearest ex ante requirement in the set. The document contains no technical mechanism at all: no hash, no timestamp, no audit trail, no integrity.
• ETSI TS 104 008 (CABCA, January 2026). The one that goes furthest. Thresholds live in an Operationalization Specification that is "provided to Stakeholders for a formal Initial Review and Approvement"; artefacts are "version-controlled" and reports "Versioned and timestamped"; §8.1.4.2 states that "the acceptance threshold shall be justified by intended use and deployment context". What is missing is not a mechanism but an independent one: the words hash and cryptographic do not appear, and the version and timestamp are produced inside the system of the party being audited or of its auditor.
• NIST AI 300-1 ipd (July 2026, initial public draft). The dataset profile offers field 1.5.2 "Cryptographic Signature — cryptographically signed manifest containing one or more cryptographic hashes for the dataset instances", and the model profile offers the same at 1.6.2 alongside a version identifier. Both optional. The evaluation field 6.5(*) asks only for "Evaluation Metrics and Results" and "timing and frequency of evaluations"; its subfields cover intended use, robustness and evaluation reliability. The word threshold does not appear anywhere in the document.
• NAIC AI Risk Evaluation Supplement v5.0 (exposure draft, comments closed 29 September 2026). Exhibit C field 11 asks the insurer to "discuss testing model outputs … and how the model was validated prior to being deployed". Free narrative. The document contains no hash, no audit trail, no version control requirement.

The same lens, five documents

Each primary text was downloaded and searched with stem matching. Counts are of the whole document.

What the pattern is

Three of these frameworks know how to protect the integrity of an object and do it. NIST offers a cryptographic signature field for a dataset and for a model. Annex 22 puts access control, a copy prohibition and a four-eyes principle around test data, and requires unauthorised-change detection for the model. ETSI version-controls and timestamps its audit artefacts.

None of the five defines an equivalent for the acceptance criteria. Where the criteria are addressed at all, the requirement is procedural — documented, approved, justified, discussed — and the evidence that they existed at the stated time is a record held by the party whose work is being assessed. A reader who was not present still has to take that party's word for the ordering.

The clearest single illustration is NIST AI 300-1: a document that offers cryptographic signatures for datasets and models, in seventeen thousand words, never uses the word threshold.

A commitment to record changes is not the same as a checkable record

A fair objection to everything above: plenty of organisations do keep a record, and some now promise to publish one. That is true, and the promise is worth stating precisely, because it shows where the remaining gap actually is.

The clearest current example comes from the party being regulated rather than the regulator. OpenAI's Frontier Governance Framework (28 May 2026), written to map its practices onto the EU AI Act's Code of Practice for General-Purpose AI and California's Transparency in Frontier AI Act, commits in §7.1 that material updates will be presented to the board's Safety and Security Committee "with changes and justifications for material updates documented in a changelog and published within 30 days of the update", and in §7.2 to a framework assessment at least every twelve months. Of the documents surveyed here and their industry counterparts, it is the first to commit to recording the change itself and the reason for it.

That commitment closes the question what changed. It does not close the question who can check. The changelog is published by the organisation whose safeguards it describes; the document contains no hash, no timestamp requirement and no version identifier of its own. A reader who wants to know whether an entry is complete, whether the thirty days were kept, or what the previous text said, has one source for all three answers: the same organisation. The commitment is only as checkable as the party making it.

This matters because the fix is small and does not touch the substance of the promise. Publishing the digest of a changelog entry, and anchoring that digest to a time the publisher does not control, turns "we recorded it" into something a third party can verify without extending any trust. The commitment stays exactly as it is; only its verifiability changes.

The same reasoning applies to every row in the table above. The gap these documents share is not an absence of records. It is that the record which would settle the ordering is held, dated and attested by the party being assessed — which is close to the thing the requirement exists to rule out.

What this page is not claiming

• This is a five-document measurement, not a survey of regulation. It says nothing about frameworks not read here.
• The NIST fields are optional, and the Annex 22 draft binds no one — it is a consultation text whose scope excludes generative models and LLMs from critical GMP applications altogether.
• ETSI is not missing a mechanism; it is missing an independent one. The distinction matters and collapsing it would misrepresent the document.
• None of these bodies has endorsed, adopted or mentioned any particular tool, including ours. What is reported here is what the texts say.
• Nobody here is accused of hiding anything. The Frontier Governance Framework commits to publishing changes and their justifications; that is more than any of the five regulatory documents asks for. The point of that section is the narrower one: a published record still relies on the publisher.

How a provider or an examiner can close the gap now

1. Write the criteria down as one file before validation runs: the metric, the comparison rule, the threshold, the evaluation dataset (name and a hash of the exact files) and, if randomness is involved, the seed. Keep it short; it is a record, not a report.
2. Hash the file. sha256sum criteria.yaml. The 64-character digest identifies that exact set of bytes; change one character and the digest changes.
3. Get a time that you did not assign yourself. A file date, a git commit date or a signed memo is provider-attested: the party being examined chose the clock. Independent options that cost nothing: an RFC 3161 timestamp token from a public time-stamping authority over the digest, or an entry in a public transparency log. Either lets a third party check the time without trusting you.
4. Bind the record to what was tested. The criteria file should name the model version identifier and the dataset version it applies to, so the chronology cannot be reattached to a different run later.
5. Keep the token with the validation report. When an examiner asks, you hand over three things: the criteria file, its digest, and the independent time evidence. They can recompute the digest and verify the token themselves.
6. If the criteria changed after results were seen, say so. A second record with its own time evidence, and one sentence on why. A revised bar with a visible history is stronger evidence than a bar that was never written down.

Where PRML fits, if you want a standard form. The steps above need no particular tool. PRML is an open, vendor-neutral record format for exactly this object (metric, comparator, threshold, dataset identity and hash, seed) with a canonical byte encoding, a registered media type (application/vnd.prml+yaml) and free verifiers. It is evidence, not authority: the governance process still decides; the record independently proves which criteria object existed at the relevant time. What is PRML?

Sources

• EU GMP Annex 22 — European Commission, Annex 22: Artificial Intelligence, EudraLex Volume 4 consultation draft, published 7 July 2025, consultation closed 7 October 2025: consultation page.
• ITU — Designing Regulatory Experimentation: A Practical Guide for the Public Sector, Telecommunication Development Bureau, ISBN 978-92-61-42681-1: itu.int.
• ETSI — TS 104 008 V1.1.1 (2026-01), Continuous Auditing Based Conformity Assessment for AI-enabled systems: etsi.org.
• NIST — AI 300-1 ipd, Guidance and Templates for Public-Facing AI Documentation, July 2026, doi 10.6028/NIST.AI.300-1.ipd: nvlpubs.nist.gov.
• NAIC — AI Risk Evaluation Supplement v5.0 exposure draft, Big Data and Artificial Intelligence (H) Working Group.

Measured 13 September 2026 against the primary text of each document. Counts use stem matching; a boundary match misses plurals such as hashes.