No locked claim, no entry.
The submission gate: require a pre-dated public receipt for every evaluation claim that wants in.
Self-imposed pre-registration only constrains the honest. When the venue requires the receipt, the constraint binds everyone who wants in. This page is the copy-paste version of that requirement — a CI gate that refuses any submission whose evaluation bar was not publicly locked before the run.
01What the gate enforces
Every submission must carry a PRML manifest — 9 fields: metric, comparator, threshold, dataset hash, seed — whose SHA-256 was publicly receipted before the results existed. The receipt is Ed25519-signed, countersigned by an RFC 3161 timestamp authority, and mirrored to the Rekor transparency log: the submitter cannot backdate it, and neither can we. Each permalink states which anchors that particular record carries; records committed before 11 July 2026 were stored without their canonical bytes and so cannot be entered into the log retrospectively. Edit the manifest after locking and verification exits 3 (TAMPERED).
02The whole gate — two real steps
- name: Verify the locked claim
id: prml
uses: studio-11-co/prml-verify-action@v2
with:
mode: manifest
manifest-path: ${{ steps.find.outputs.manifest }}
- name: Require a public receipt
run: |
curl -sf "https://registry.falsify.dev/${{ steps.prml.outputs.hash }}" > /dev/null \
|| { echo "::error::No public receipt - lock the claim before running"; exit 1; }
Complete workflow, submission-requirements template and rejection copy live in examples/require-receipt.
03What to tell submitters
One paragraph in your rules is enough: “Write your PRML manifest before you run; lock it at registry.falsify.dev; submit manifest and results together. Entries whose bar has no pre-dated public receipt are rejected automatically.” Submitters need no account and pay nothing; locking takes under a minute in the browser or one curl.
04What this proves — and does not
It proves the success criteria existed, in exactly this form, before the results did. It does not prove the score is correct — score verification stays with you, the venue. PRML proves the bar was locked, never the result.
Put precisely: a published evaluation result can be wrong in two independent ways, and a reader of the record usually cannot tell which one happened.
The bar moved. The criterion was chosen, or adjusted, after the results were seen. This is what the gate closes: a receipt dated before the run means the threshold in it is the threshold that was committed to.
The run was subverted. The number came from an execution that did not do what it claimed — a contaminated dataset, an unsound grader, or an evaluation environment that was gamed. A receipt says nothing about this. Something that reaches the answers produces a genuine-looking pass against a genuinely pre-registered bar, and the manifest hash still verifies, because the manifest never changed.
We say this plainly because a gate believed to cover both is worse than one that covers neither. Requiring a receipt narrows what a submitter can do; it does not make a claim true. So ask two questions, not one:
- Was the bar recorded before the results existed? This gate. A PRML receipt answers it, and anyone can check it.
- What ties the reported number to a specific execution? Not this gate. Ask for what identifies the run: the artefacts, their versions, and anything a third party can check them against.
On the second question, prml-linkage/0 evidences the order lock → run → result at three stated strength tiers: a start record carrying the locked manifest hash is committed before the run finishes, and the final record chains to it. That is an ordering claim. It is not a claim about the run’s integrity. Without trusted execution hardware, a runner’s start time is process evidence, not cryptographic proof, and the specification says so in its own text rather than in a footnote.
05Where this is already required
The gate above is not an invention of ours, and it is worth knowing that before you ask anyone to pass through it. Four independent bodies already require that evaluation criteria be fixed before the results are known. None of them says how a reader outside the room would establish that they were.
The ITU, in Designing Regulatory Experimentation: A Practical Guide for the Public Sector (Telecommunication Development Bureau, 2026, ISBN 978-92-61-42681-1), states that every experimentation space "must define regulatory hypotheses from the design stage (ex ante) and contrast them with observed results at the close", and that "an indicator without a threshold becomes descriptive; a threshold without a decision becomes decorative". The guide is written for regulatory sandboxes in telecommunications, and it is design advice rather than binding law. It specifies no mechanism by which a reader could establish that the hypotheses predate the results.
The European Commission, in its draft implementing regulation on AI regulatory sandboxes (Ares(2025)10569703, not yet adopted), has the competent authority evaluate a completed project "in accordance with the objectives in the sandbox plan" — a plan agreed before the exercise begins. The same draft then allows that plan to "be updated during the participation" by mutual agreement, with no versioning requirement attached. Authorities must record activities and updates securely under Article 4(1); what the published exit report need not say is which version of the plan the result was judged against. Every EU Member State must have a national AI regulatory sandbox operational by 2 August 2027 (AI Act Article 57(1), as amended by Regulation (EU) 2026/1744).
A forty-eight-author proposal for frontier AI auditing (arXiv:2601.11699) recommends "pre-registration of evaluation protocols", on the ground that "specifying the questions to be answered, the metrics to be used, and the criteria for success before results are known" is what separates a confirmatory test from "post hoc rationalizations shaped by preliminary findings". The same paper proposes cryptographic commitment elsewhere — to the identity of the system, for change detection. It does not join the two.
A reporting framework for AI evaluations (arXiv:2606.09809, June 2026) goes further and names the mechanism outright, asking whether a report records "any GUID/canary usage, or other verifiable cryptographic commitment mechanisms (e.g. hashes of non-public preregistration)".
Then there is what actually reaches the record. Across the 29 case studies published through Singapore's Global AI Assurance Sandbox — a programme that asks participants in writing to publish their interpretation thresholds — a reader can recover the rule separating a good result from a bad one in three (per-document ledger, so any classification in it can be disputed). Independently, the reporting framework above measured the population rate of its own pre-registration field at 0.0% across 635 benchmarks, against 100% for the fields carrying the score.
So the requirement is written in four places and measured close to zero in two. That is the gap this gate closes, and the reason it has to be a condition of entry rather than a request: where the criterion is not required in the record, it is almost never in the record.
06Buying, not hosting? The procurement clause
The same requirement travels as one contract clause. Paste it into an RFP annex, a vendor due-diligence questionnaire, or a renewal condition — it names properties, not a product, so it survives legal review:
Verifiable evaluation evidence. For each quantitative evaluation claim relied upon in this engagement, the supplier shall: (a) commit the criteria that decide the claim — metric, comparator, threshold, dataset identity by content hash, random seed, and the identity of the claimant — to a SHA-256 digest over a canonical serialisation, before the evaluation is run; (b) obtain an independent timestamp of that commitment (RFC 3161 or an append-only transparency log), so that the time of commitment does not rest on the supplier’s own records; (c) deliver, with the reported result, the manifest and its commitment in a form the buyer can verify offline using only the manifest text and a standard SHA-256 implementation. A result whose recomputed digest does not match its commitment shall be treated as unsupported.
Conformance with PRML v0.1 (openly specified, CC BY 4.0) satisfies (a)–(c); any equivalent openly specified mechanism is likewise acceptable.
Why a buyer wants this: every number a vendor shows was produced under criteria the vendor chose — and could have adjusted after seeing the result. No report reveals that. This clause costs the vendor a hash and a timestamp, and turns “trust our methodology” into a check your team runs in under a minute. It is CC BY 4.0 — copy it, adapt it, strip the attribution.
07Running a venue and want it wired in?
The gate above is free and self-serve forever. If you want it adapted to your submission flow — custom fields, private registry, evidence packs for your reviewers — that is a fixed-scope written engagement: pricing · [email protected].