NIST AI 300-1, field 6.5: what should a model profile record about evaluation criteria?

NIST's AI 300-1 initial public draft (July 2026) is a template for public-facing AI documentation. Its default model profile records evaluation metrics and results, the intended use evaluated, robustness under shift, and the stability of the results. It has no field for the criteria the results were judged against or for when those criteria were fixed.

What the draft asks

6.5-(*) Evaluation Metrics and Results — Information about TEVV results, including methods and metrics for measuring or otherwise assessing the model's effectiveness for the intended use, and including the model's performance against those metrics, particularly on established benchmarks. Designation: Recommended.
Table A.3.1, default AI model profile

6.5-(*).3 Evaluation Reliability Metrics and Results — Information about the stability and reproducibility of evaluation results. […] This information enables interested parties to determine whether performance claims are stable enough to support deployment decisions.
Table A.3.1

Evaluation — […] This may include descriptions of the evaluation protocols and datasets, documented analyses of risks and potential impacts, quantitative evaluation results, and qualitative analysis.
Clause 5.3, model documentation template, field 6

The draft already provides version identifiers and cryptographic manifests for datasets (Table A.2.1, fields 1.5.1–1.5.2) and version identifiers and signatures for models (Table A.3.1, fields 1.6.1–1.6.2). It also sets an interoperability goal (4.2.6) and a freshness goal (4.2.5). What is missing is the object those identifiers should be bound to when a performance claim is made: the criteria.

Why stability is not the same as pre-specification

Field 6.5-(*).3 answers the reader's first uncertainty: would this number move if the evaluation were rerun? A second uncertainty is whether the bar was set before the number. Confidence intervals cannot answer it. An artifact can be fresh in the sense of 4.2.5 while the acceptance threshold silently changed between versions, because no field records the threshold's history.

How to document it now, without waiting for the template

1. Write the criteria down as one file before validation runs: the metric, the comparison rule, the threshold, the evaluation dataset (name and a hash of the exact files) and, if randomness is involved, the seed. Keep it short; it is a record, not a report.
2. Hash the file. sha256sum criteria.yaml. The 64-character digest identifies that exact set of bytes; change one character and the digest changes.
3. Get a time that you did not assign yourself. A file date, a git commit date or a signed memo is provider-attested: the party being examined chose the clock. Independent options that cost nothing: an RFC 3161 timestamp token from a public time-stamping authority over the digest, or an entry in a public transparency log. Either lets a third party check the time without trusting you.
4. Bind the record to what was tested. The criteria file should name the model version identifier and the dataset version it applies to, so the chronology cannot be reattached to a different run later.
5. Keep the token with the validation report. When an examiner asks, you hand over three things: the criteria file, its digest, and the independent time evidence. They can recompute the digest and verify the token themselves.
6. If the criteria changed after results were seen, say so. A second record with its own time evidence, and one sentence on why. A revised bar with a visible history is stronger evidence than a bar that was never written down.

In a model profile, that becomes one optional line under 6.5: Pre-specified evaluation criteria: [criteria object digest], fixed [date], independent time evidence [token / log reference], applies to dataset [1.5.1] and model version [1.6.1]; timing independently verifiable / provider-attested.

Where PRML fits, if you want a standard form. The steps above need no particular tool. PRML is an open, vendor-neutral record format for exactly this object (metric, comparator, threshold, dataset identity and hash, seed) with a canonical byte encoding, a registered media type (application/vnd.prml+yaml) and free verifiers. It is evidence, not authority: the governance process still decides; the record independently proves which criteria object existed at the relevant time. What is PRML?

What we proposed to NIST

Falsify submitted four comments on 9 September 2026 proposing an optional subfield 6.5-(*).4 "Pre-specified Evaluation Criteria" with replacement text and three consequential edits. The submission, verbatim. NIST considers input received by 16 September 2026 for the next revision; the Zero Draft is intended to feed ISO/IEC JTC 1/SC 42 through INCITS.

Sources

• NIST AI 300-1 ipd (PDF)
• NIST AI Standards Zero Drafts pilot