DSA Article 37 audits: where do the audit criteria come from, and when are they fixed?

Every very large online platform and search engine in the EU is audited independently at least once a year, at its own expense, on its systemic-risk obligations. On 31 August 2026 the Commission designated ChatGPT under the same regime. The audit criteria are not handed down by the Commission: the auditor derives them, in part, from benchmarks the provider itself supplies.

What the regime says

• DSA Article 37(1): providers of very large online platforms and search engines are subject, at their own expense, to independent audits at least once a year, covering their obligations under Chapter III, including the risk assessment and mitigation measures.
• Delegated Regulation (EU) 2024/436, Article 5(1)(a): the audited provider transmits to the auditor the benchmarks it uses to assert or monitor compliance.
• Article 10(2)(a): the auditor sets audit criteria based on the information submitted by the audited provider as regards the benchmarks the provider uses.
• Article 2(13): the audit is conducted to a reasonable level of assurance, defined as a high but not absolute level of assurance; conclusions are positive, positive with comments, or negative (Article 8(1)).
• Publication: the provider publishes the audit report within three months of its completion; the Commission and Digital Services Coordinators may take it into account in supervision.

For ChatGPT, designated on 31 August 2026 (Commission press release IP/26/1772), the additional obligations apply four months after designation and the first audit falls due within twelve months of that. Nothing on this page is legal advice; it is a reading of the text with the relevant articles named so you can check it.

What the regime does not ask

The auditor independently verifies the provider's compliance against criteria that are, in part, the provider's own benchmarks. The regulation asks whether the benchmarks were applied and whether the evidence supports the provider's assertions. It does not ask when the benchmark's pass level was chosen relative to the figures now reported against it. A benchmark whose threshold was tuned after the year's numbers were known can still be transmitted under Article 5, still be turned into an audit criterion under Article 10, and still support a positive conclusion. That is not a defect in the auditor's work; it is a question the framework never poses.

Why it matters more for AI systems

For a recommender or a general-purpose chatbot, the benchmarks in question are evaluation results: refusal rates, error rates on a test set, mitigation effectiveness scores. Those are exactly the kind of number whose meaning depends on whether the bar was set before the run. The same gap exists in the EU AI Act's Article 9(8) testing provisions and in NIST's documentation templates; the DSA regime is the one already in force, with named auditors and a public report.

How a provider or auditor can close the gap now

1. Write the criteria down as one file before validation runs: the metric, the comparison rule, the threshold, the evaluation dataset (name and a hash of the exact files) and, if randomness is involved, the seed. Keep it short; it is a record, not a report.
2. Hash the file. sha256sum criteria.yaml. The 64-character digest identifies that exact set of bytes; change one character and the digest changes.
3. Get a time that you did not assign yourself. A file date, a git commit date or a signed memo is provider-attested: the party being examined chose the clock. Independent options that cost nothing: an RFC 3161 timestamp token from a public time-stamping authority over the digest, or an entry in a public transparency log. Either lets a third party check the time without trusting you.
4. Bind the record to what was tested. The criteria file should name the model version identifier and the dataset version it applies to, so the chronology cannot be reattached to a different run later.
5. Keep the token with the validation report. When an examiner asks, you hand over three things: the criteria file, its digest, and the independent time evidence. They can recompute the digest and verify the token themselves.
6. If the criteria changed after results were seen, say so. A second record with its own time evidence, and one sentence on why. A revised bar with a visible history is stronger evidence than a bar that was never written down.

Confidential benchmarks are the common case here, and the record does not have to disclose them: a sealed commitment publishes the hash and the independent time while the criteria stay private until the audit needs them. How sealed commitments work.

Where PRML fits, if you want a standard form. The steps above need no particular tool. PRML is an open, vendor-neutral record format for exactly this object (metric, comparator, threshold, dataset identity and hash, seed) with a canonical byte encoding, a registered media type (application/vnd.prml+yaml) and free verifiers. It is evidence, not authority: the governance process still decides; the record independently proves which criteria object existed at the relevant time. What is PRML?

Sources

• Regulation (EU) 2022/2065 (Digital Services Act), Article 37
• Commission Delegated Regulation (EU) 2024/436 of 20 October 2023 on the performance of audits (applies since 22 February 2024)
• Commission list of designated VLOPs and VLOSEs