EU General-Purpose AI Code of Practice
EU-GPAI-COP-2025 · EU
Operational bridge between EU AIA Arts. 53-55 (general-purpose AI obligations) and provider compliance. Art. 56(8) AIA gives adherent providers a presumption of compliance with the substantive obligations — distinct from industry self-pledges (Anthropic RSP, OpenAI Preparedness, DeepMind FSF) and from intergovernmental voluntary codes (Seoul, G7 Hiroshima). Chapter 1 (Transparency) operationalises Art. 53(1)(a)-(c) model documentation + training-data summary obligations; Chapter 2 (Copyright) operationalises Art. 53(1)(c) opt-out compliance + Art. 53(1)(d) text-and-data-mining respect; Chapter 3 (Safety & Security) operationalises Art. 55 systemic-risk-tier obligations including capability evaluations + serious-incident reporting + cybersecurity protections + model-weight access controls. AI Office monitors implementation; Article 65 binding-decision procedure routes cross-DPA disputes. Not a binding regulation in itself — providers may choose alternative means to demonstrate compliance — but the Code is the AI Office's canonical reference and the operational rulebook national-competent-authorities consult during inspections. Currency (2026-06-21): The European AI Office published the FINAL Code on 10 July 2025 (superseding the 'third draft' described above), endorsed by the Commission and AI Board as an adequate voluntary compliance tool; 23+ providers have signed (Anthropic, OpenAI, Google, Microsoft, Amazon, IBM, Mistral, Aleph Alpha), Meta declined, and xAI signed only the Safety & Security chapter — GPAI obligations apply from 2 Aug 2025 with Commission enforcement beginning 2 Aug 2026 (source: https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai).
Background & scope
EU General-Purpose AI Code of Practice addresses 4 contested AI-governance topics explicitly, 1 via general principles.
Provisions & coverage
- governsFoundation Models / GPAIChapter 3 (Safety & Security) operationalises Art. 55 systemic-risk-tier obligations for GPAI providers[11]
- governsTransparency ObligationsChapter 1 (Transparency) — 13 commitments + ~40 measures operationalising Art. 53(1)(a)-(c) model documentation + training-data summary[11]
- governsTraining-Data RightsChapter 2 (Copyright) — Art. 53(1)(c) training-data summary obligations + Art. 53(1)(d) text-and-data-mining opt-out compliance[11]
- governsCatastrophic & Existential RiskChapter 3 systemic-risk-tier capability evaluations + serious-incident reporting + model-weight access controls (Art. 55 substrate)[11]
- implicitSynthetic Content ProvenanceChapter 1 transparency commitments brush against Art. 50(2) deployer marking + Art. 53(1)(a) provider documentation[11]
What the Code Commits Adherents To
Drafted by the European AI Office under Article 56 of Regulation (EU) 2024/1689, the final Code (published 10 July 2025) translates the AI Act's general-purpose-AI obligations into 13 commitments and roughly 40 measures across three chapters. Chapter 1 (Transparency) operationalises the Art. 53(1)(a)-(c) model-documentation and training-data-summary duties; Chapter 2 (Copyright) implements Art. 53(1)(c) reservation-of-rights respect and Art. 53(1)(d) text-and-data-mining opt-out compliance; Chapter 3 (Safety & Security) gives operational form to the Art. 55 systemic-risk-tier duties. As 1 situates it, the Code is the institutional hinge of a risk-based regime - yet the terminology it inherits is unstable: 2 document how 'GPAI' and 'foundation model' shifted across Act drafts, leaving the Code's core categories contested.
Standing Relative to Binding Law
The Code is a voluntary instrument, not a regulation: providers may demonstrate compliance by alternative means. Its leverage flows from Art. 56(8) of Regulation (EU) 2024/1689, under which adherence yields a presumption of conformity with the underlying Art. 53 and Art. 55 obligations - a rebuttable evidentiary shortcut, not a safe harbour. This distinguishes it from firm self-pledges (Anthropic's RSP, OpenAI's Preparedness Framework) and intergovernmental codes (Seoul, G7 Hiroshima), which carry no statutory presumption. The AI Office monitors implementation; the Art. 65 binding-decision procedure routes cross-authority disputes. The binding floor remains the Act: GPAI obligations apply from 2 August 2025, enforcement from 2 August 2026 (Regulation (EU) 2024/1689). Whether binding floors suffice is contested - 3 urge a compute-threshold treaty with audit powers beyond a voluntary code.
Critiques and Compliance Gaps
Three fault-lines recur. First, the copyright chapter rests on a TDM opt-out whose enforceability is contested: 4 catalogues post-LAION obstacles - robots.txt ambiguity, machine-readability, memorisation - concluding the exceptions 'seem workable in theory' but strain in practice, while 5 argues the Art. 3 CDSM research exception grants rightsholders no control at all; 6 reads opt-in/opt-out as a 'missed opportunity'. Second, the transparency chapter only implicitly touches synthetic-content provenance; 7 finds just 38% of image generators watermark adequately - a gap living under Art. 50, not the Code. Third, the safety chapter's systemic-risk evaluations are challenged by 8, whose 'accumulative' x-risk class evades discrete capability thresholds.
Adoption Trajectory and Outlook
Adoption is partial but front-loaded among frontier labs: 23-plus providers have signed, including Anthropic, OpenAI, Google, Microsoft, Amazon, IBM, Mistral and Aleph Alpha. The signature pattern is itself diagnostic - Meta declined outright, and xAI subscribed only to the Safety & Security chapter, suggesting copyright and transparency may draw sharper resistance. Because the Code is the AI Office's canonical reference and the rulebook national competent authorities consult during inspections, chapter-selective adherence will shape how the Art. 55 systemic-risk tier is enforced from 2 August 2026. The deeper question - whether voluntary capability evaluations suffice for catastrophic and dual-use threats - stays open: 9 maps AI-biosecurity pathways exceeding what a presumption-of-compliance regime can police, and 10 contends international law may oblige states to go further.
Enforcement & impact
Cross-jurisdiction comparison
How peer instruments treat the topics EU General-Purpose AI Code of Practice governs.
| Topic | EU-AIA-2024 | US-EO-14110 | US-EO-14179 | UK-WHITEPAPER-2023 | CN-GENAI-2023 | G7-HIROSHIMA | OECD-AI-PRIN | COE-AI-CONV | UN-RES-2024 | NIST-AI-RMF | BLETCHLEY-2023 | SEOUL-2024 | NIST-AI-RMF-GENAI | CA-SB-1047 | IN-DPDP-2023 | BR-AIBILL-2024 | ASEAN-AI-GUIDE-2024 | AU-AI-STRATEGY-2024 | ANTHROPIC-RSP-2024° | OPENAI-PREPAREDNESS-2023° | DEEPMIND-FSF-2024° | META-FRONTIER-2024° | UK-US-AISI-MOU-2024 | WH-VOLUNTARY-2023 | SG-MODEL-AI-2024 | JP-METI-AI-2024 | EU-GDPR-2016 | OMB-M-24-10 | GSA-AI-GUIDE-2024 | DOD-RAI-2022 | FEDRAMP-AI-2024 | DFARS-252-204 | CA-SB-53 | CA-SB-243 | CA-SB-942 | EU-PLD-2024 | UNESCO-AI-ETHICS-2021 | EU-PWD-2024 | CN-DEEPSYN-2022 | NY-RAISE-2025 | US-TAKEITDOWN-2025 | IT-AILAW-2025 | JP-AIPROMO-2025 | UN-GDC-2024 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Foundation Models / GPAI | governs | governs | silent | implicit | governs | governs | implicit | implicit | silent | governs | governs | governs | governs | governs | implicit | governs | implicit | silent | governs | governs | governs | governs | governs | governs | governs | governs | silent | implicit | governs | implicit | implicit | implicit | governs | silent | implicit | silent | silent | silent | silent | governs | silent | silent | implicit | implicit |
| Transparency Obligations | governs | implicit | silent | implicit | conflicts | governs | governs | governs | implicit | governs | implicit | governs | governs | implicit | implicit | governs | governs | silent | governs | implicit | implicit | governs | implicit | governs | governs | governs | governs | governs | governs | governs | governs | silent | governs | governs | governs | implicit | governs | governs | governs | governs | silent | governs | governs | governs |
| Training-Data Rights | implicit | silent | silent | silent | governs | silent | silent | implicit | silent | implicit | silent | silent | governs | silent | governs | implicit | silent | implicit | silent | silent | silent | implicit | silent | silent | silent | implicit | governs | silent | implicit | silent | implicit | governs | silent | silent | silent | silent | governs | silent | governs | silent | silent | governs | implicit | implicit |
| Catastrophic & Existential Risk | implicit | governs | silent | implicit | silent | governs | silent | silent | implicit | implicit | governs | governs | governs | governs | silent | governs | silent | silent | governs | governs | governs | governs | implicit | implicit | silent | silent | silent | silent | silent | implicit | silent | silent | governs | silent | silent | silent | silent | silent | silent | governs | silent | silent | silent | implicit |
°= industry self-imposed voluntary framework. Comparing a voluntary code's "governs" tint with a binding regulation's "governs" tint flattens the legal-force distinction; use the instrument-page banner for the operative status of each.
See also
Per-audience views
- Provisions →Article-by-article obligation breakdown for procurement + RFP authors.
- Disclosure form →Vendor-disclosure questionnaire derived from this instrument's operative obligations.
- Harm narratives →Documented harms relevant to this instrument's topics, for civil-society advocacy.
- Briefing pack →Journalist-ready summary with quotes + dates + primary-source links.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here
Further reading
93 academic & grey-literature sources on the topics this instrument addresses (not commentary on the instrument itself) — catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- Missing the Mark: Adoption of Watermarking for Generative AI Systems in Practice and Implications Under the New EU AI Act Peer-reviewed✦ AIEmpirical audit finds only 38% of AI image generators implement adequate watermarking and 18% deepfake labelling, exposing a compliance gap under EU AI Act Article 50.
- Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways Peer-reviewed✦ AIReviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats.
- Open Foundation Models and TDM Exceptions to Copyright – Building Blocks for an AI Ecosystem Peer-reviewed✦ AIArgues Art. 3 CDSM Directive's scientific-research TDM exception 'does not grant rightsholders any control' and can be a 'safe harbor' for training openly released foundation models without licensing data.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- Navigating China's regulatory approach to generative artificial intelligence and large language models Peer-reviewed✦ AIAnalyses China's 2022 deep-synthesis and 2023 generative-AI rules, including mandatory labelling/watermarking of synthetic content as a provenance-governance model.
- 'Sora is incredible and scary': public perceptions and governance challenges of text-to-video generative AI models Peer-reviewed✦ AIQualitative analysis of public commentary on Sora finds blurred real/fake boundaries drive demand for law-enforced AI-content labelling and provenance.
- Two types of AI existential risk: decisive and accumulative Peer-reviewed✦ AIDistinguishes 'decisive' (sudden takeover) from 'accumulative' AI existential risk, arguing governance must address gradual societal erosion as well as abrupt scenarios.
- Confronting Catastrophic Risk: The International Obligation to Regulate Artificial Intelligence Peer-reviewed✦ AIArgues international law imposes a precautionary-principle obligation on states to regulate AI to mitigate the threat of human extinction.
- Artificial Intelligence and Nuclear Weapons Proliferation: The Technological Arms Race for (In)visibility Peer-reviewed✦ AIAnalyzes how AI-driven detection/concealment in nuclear arsenals reshapes strategic stability and proliferation risk, with governance implications.
- International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty Preprint✦ AIProposes a conditional AI safety treaty with a compute threshold triggering mandatory audits by an international network of AI Safety Institutes empowered to halt development if risks are unacceptable.
+ 81 more across this instrument's topics — see the literature index.
References
Sources cited inline in the analysis (linked from the superscript markers), then the primary instrument sources behind the classifications.
- Martina Hulok (2025) The EU model of AI governance: regulating artificial intelligence through law and policy, ERA Forum. 10.1007/s12027-025-00869-1 — Analyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'. ↩
- David Fernández-Llorca, Emilia Gómez, Ignacio Sánchez, Gabriele Mazzini (2025) An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI, Artificial Intelligence and Law. 10.1007/s10506-024-09412-y — Traces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime. ↩
- Rebecca Scholefield, Samuel Martin, Otto Barten (2025) International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty, arXiv (cs.CY). arXiv:2503.18956 — Proposes a conditional AI safety treaty with a compute threshold triggering mandatory audits by an international network of AI Safety Institutes empowered to halt development if risks are unacceptable. ↩
- Stepanka Havlikova (2025) Technical Challenges of Rightsholders' Opt-out From Gen AI Training after Robert Kneschke v. LAION, JIPITEC – Journal of Intellectual Property, Information Tech. source — Examines post-LAION practical obstacles to the EU TDM opt-out (robots.txt, machine-readability, memorisation): 'While the TDM exceptions may seem workable in theory, implementing them in practice presents a variety of practical… ↩
- Arne Radeisen (2026) Open Foundation Models and TDM Exceptions to Copyright – Building Blocks for an AI Ecosystem, GRUR International. 10.1093/grurint/ikag002 — Argues Art. 3 CDSM Directive's scientific-research TDM exception 'does not grant rightsholders any control' and can be a 'safe harbor' for training openly released foundation models without licensing data. ↩
- Martin Kretschmer, Bartolomeo Meletti, Lionel Bently, Gabriele Cifrodelli, Magali Eben, Kristofer Erickson, Aline Iramina, Zihao Li, Luke McDonagh, Emma Perot, Luis Porangaba, Amy Thomas (2025) Copyright and AI in the UK: Opting-In or Opting-Out?, GRUR International. 10.1093/grurint/ikaf093 — Contends the UK opt-in/opt-out framing is a 'missed opportunity'; a broadened research exception plus market-entry transparency and creator remuneration would better serve both innovation and rightsholders. ↩
- Bram Rijsbosch, Gijs van Dijck, and Konrad Kollnig (2026) Missing the Mark: Adoption of Watermarking for Generative AI Systems in Practice and Implications Under the New EU AI Act, Policy & Internet. 10.1002/poi3.70041 — Empirical audit finds only 38% of AI image generators implement adequate watermarking and 18% deepfake labelling, exposing a compliance gap under EU AI Act Article 50. ↩
- Atoosa Kasirzadeh (2025) Two types of AI existential risk: decisive and accumulative, Philosophical Studies. 10.1007/s11098-025-02301-3 — Distinguishes 'decisive' (sudden takeover) from 'accumulative' AI existential risk, arguing governance must address gradual societal erosion as well as abrupt scenarios. ↩
- Kirolos Eskandar (2026) Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways, AI and Ethics (Springer). 10.1007/s43681-025-00872-9 — Reviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats. ↩
- Bryan Druzin, Anatole Boute, Michael Ramsden (2025) Confronting Catastrophic Risk: The International Obligation to Regulate Artificial Intelligence, Michigan Journal of International Law. source — Argues international law imposes a precautionary-principle obligation on states to regulate AI to mitigate the threat of human extinction. ↩
- General-Purpose AI Code of Practice, drafted by the European AI Office under Article 56 of Regulation (EU) 2024/1689 (EU AI Act); co-drafted by ~1000 stakeholders across providers, civil society, academia, and regulators; three chapters (Transparency, Copyright, Safety & Security) covering 13 commitments + ~40 measures.
- Chapter 3 (Safety & Security) operationalises Art. 55 systemic-risk-tier obligations for GPAI providers
- Chapter 1 (Transparency) — 13 commitments + ~40 measures operationalising Art. 53(1)(a)-(c) model documentation + training-data summary
- Chapter 2 (Copyright) — Art. 53(1)(c) training-data summary obligations + Art. 53(1)(d) text-and-data-mining opt-out compliance
- Chapter 3 systemic-risk-tier capability evaluations + serious-incident reporting + model-weight access controls (Art. 55 substrate)
- Chapter 1 transparency commitments brush against Art. 50(2) deployer marking + Art. 53(1)(a) provider documentation
How to cite this article
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/gpai-code-of-practice — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Does this instrument’s approach work? — the social-science evidence
Aggregated over the 5 topics this instrument governs: whether each harm is empirically real, and whether the peer-reviewed evidence shows governance reduces it. The badge is the epistemic status of the evidence— “thin”/“absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.
Of the 5 governed topics with a social-science evidence review, evidence that governance reduces the harm is established for 0, contested for 0, thin for 1, and absent for 4 — for most, no replicated study yet shows this instrument's approach works (the "second silence").
Catastrophic & Existential Risk
The catastrophic-uplift premise is genuinely contested: the empirical uplift studies that exist find current frontier models add little. RAND's red-team study found no statistically significant difference in the viability of bioweapon-attack plans produced with vs. without LLMs (Mouton, Lucas & Guest 2024), and OpenAI's 100-participant trial found GPT-4 gave at most a mild, non-significant accuracy uplift (mean +0.88 out of 10 for PhD experts, +0.25 for students; Patwardhan et al. 2024). Honest caveat: the harm is forward-looking, not yet observed — expert opinion on the catastrophic tail is sharply split (median AI researcher puts ~5% on extremely-bad/extinction outcomes, mean ~9-16% across differently-framed questions, n=2,778; Grace et al. 2024), and forecasters underestimated how fast risk-relevant capabilities (e.g. virology troubleshooting) actually arrived (Forecasting Research Institute 2025), so the relevant capabilities are a moving target rather than a settled magnitude.
Sources: Mouton, Lucas & Guest 2024 (RAND RR-A2977-2, Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study); Patwardhan et al. 2024 (OpenAI, Building an Early Warning System for LLM-aided Biological Threat Creation); Grace et al. 2024 (Thousands of AI Authors on the Future of AI, arXiv:2401.02843); Forecasting Research Institute 2025 (Forecasting LLM-enabled Biorisk and the Efficacy of Safeguards)
There is essentially no impact evidence that catastrophic-risk governance reduces catastrophic risk, and structurally there cannot yet be: the harm is a low-probability civilisational tail event, so no controlled trial or before/after evaluation of a realised catastrophe is possible. The dominant instruments are recent, voluntary developer frameworks (Anthropic's Responsible Scaling Policy 2023; OpenAI's Preparedness Framework 2023) built on if-then capability thresholds the developers themselves describe as speculative and qualitative rather than validated risk thresholds. The closest evidence is adjacent and indirect: trained-in deceptive behaviours can persist through standard safety training (Hubinger et al. 2024) — a demonstration that current mitigation may be insufficient, not that any governance regime works — and Anthropic's documented loosening of earlier commitments (RSP 2025 dropped the original pledge to define higher-tier ASL evaluations before developing the corresponding models) illustrates that even the strongest voluntary regimes lack external enforcement or measured efficacy.
Sources: Anthropic 2023 (Responsible Scaling Policy); OpenAI 2023 (Preparedness Framework); Hubinger et al. 2024 (Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, arXiv:2401.05566); Hendrycks, Mazeika & Woodside 2023 (An Overview of Catastrophic AI Risks, arXiv:2306.12001)
Foundation Models / GPAI
Whether the foundation-model category maps to a coherent capability/risk tier is genuinely contested. The original case rests on scale-driven 'emergent abilities' that appear unpredictably above a size threshold (Wei et al. 2022; Ganguli et al. 2022 documented capabilities that are smoothly predictable in aggregate loss yet locally surprising), but Schaeffer, Miranda & Koyejo (2023, a NeurIPS Outstanding Paper) showed many 'emergent' jumps are artefacts of discontinuous metrics and dissolve under linear/continuous scoring — implying capability scales more smoothly than a sharp tier would suggest. Honest caveat: this is a live empirical disagreement about measurement, not a settled finding either way, and compute (the regulatory proxy) is an imperfect stand-in for capability or risk regardless of which side is right.
Sources: Wei et al. 2022 (Emergent Abilities of Large Language Models, TMLR; arXiv:2206.07682); Schaeffer, Miranda & Koyejo 2023 (Are Emergent Abilities of Large Language Models a Mirage?, NeurIPS 2023, Outstanding Paper; arXiv:2304.15004); Ganguli et al. 2022 (Predictability and Surprise in Large Generative Models, ACM FAccT; DOI 10.1145/3531146.3533229)
There is no impact evaluation showing that GPAI/foundation-model governance reduces harm — the rules are too new (EU AI Act GPAI obligations and the 10^25-FLOP systemic-risk presumption only began binding on 2 August 2025) and the central regulatory lever is itself contested: Hooker (2024) argues compute thresholds are a shortsighted proxy because compute does not reliably track capability or risk, and the thresholds already diverge across jurisdictions (EU 10^25 vs. the now-rescinded US EO 14110's 10^26 operations, rescinded 20 January 2025). The mandated mitigation methods also lack validated efficacy: model evaluation and red-teaming face well-documented coverage limits and an 'audit gap' in the survey/position literature (behavioural testing cannot establish the absence of untested failure modes), and adversarial red-teaming repeatedly defeats deployed safeguards — the UK AI Safety Institute reports finding universal jailbreaks for every frontier system it has tested, and a large public agent-injection competition elicited policy violations across all 22 frontier models tested from ~1.8M attacks (Zou et al. 2025). Even compliant evaluation therefore cannot yet certify the safety the rules demand. (Caveat: this is an absence-of-evidence claim — no efficacy study has been done — not evidence the rules are ineffective.)
Sources: Hooker 2024 (On the Limitations of Compute Thresholds as a Governance Strategy, arXiv:2407.05694); EU AI Act Arts. 51 & 55 (GPAI systemic-risk presumption, 10^25 FLOP; binding 2 Aug 2025); US EO 14110 (10^26-operation reporting threshold, rescinded 20 Jan 2025 by EO 14148); Zou et al. 2025 (Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition / Gray Swan Arena, arXiv:2507.20526 — 22 frontier agents, ~1.8M attacks); UK AI Safety/Security Institute, Frontier AI Trends Report (universal jailbreaks for every system tested); METR, Common Elements of Frontier AI Safety Policies (2024)
Synthetic Content Provenance
The harm provenance targets is real but concentrated, and the technical premise that the mandated signal survives is itself empirically shaky. Synthetic-media harm is well documented in two domains: non-consensual intimate imagery (Ajder et al.'s 2019 Deeptrace audit found 96% of deepfake videos were pornographic and effectively 100% targeted women) and impersonation fraud (the Arup case, ~US$25.6M / HK$200M lost via a deepfake video call). The honest caveat is twofold: a feared broad political-misinformation harm is not yet demonstrated at scale, and CS work shows invisible watermarks are removable in practice (Jiang, Zhang & Gong 2023, WEvade, evade detection via adversarial perturbation; Zhao et al. 2024 prove pixel-level watermarks are provably removable via regeneration attacks), so the provenance signal a rule would mandate is itself contested.
Sources: Ajder, Patrini, Cavalli & Cullen 2019 (Deeptrace, 'The State of Deepfakes: Landscape, Threats, and Impact'); Jiang, Zhang & Gong 2023 ('Evading Watermark based Detection of AI-Generated Content', ACM CCS 2023); Zhao et al. 2024 (NeurIPS, 'Invisible Image Watermarks Are Provably Removable Using Generative AI'); Arup deepfake fraud (CNN Business, 2024-05-16, US$25.6M)
There is no impact evaluation showing that mandated provenance/labeling reduces synthetic-media harm; the major mandates (China's GenAI labeling Measures, effective 2025-09-01; EU AIA Art. 50, machine-readable marking) are too new and unevaluated, and the delivery layer is leaky: the C2PA spec's own Security Considerations document the strip-and-repost threat, and platform audits report C2PA/Content-Credentials metadata is stripped by essentially all major social platforms on upload (consistent with Imatag's 2018 finding that ~80% of uploaded images lose metadata, only ~15% retaining it). The closest analogue evaluation literature — Pennycook, Bear, Collins & Rand (2020), the 'implied truth effect' — gives reason for caution rather than confidence: labeling only some content can make unlabeled false content seem more credible, so a partial-coverage provenance regime could backfire.
Sources: Pennycook, Bear, Collins & Rand 2020 (Management Science 66(11):4944-4957, 'The Implied Truth Effect'); China Measures for Labeling AI-Generated Synthetic Content (eff. 2025-09-01); EU AI Act Art. 50; Imatag 2018 metadata-stripping study (~80%); C2PA Security Considerations (spec.c2pa.org) on manifest removal
Training-Data Rights
That foundation models ingest copyrighted and personal works without consent is undisputed; whether that ingestion produces legally cognizable reproduction harm is genuinely contested. The CS evidence that models can memorize and emit verbatim training text is robust and replicated — Carlini et al. (2021) extracted hundreds of verbatim sequences (including PII) from GPT-2, and follow-up work (Carlini et al., Quantifying Memorization, ICLR 2023) showed extraction scales log-linearly with model size and with example duplication. Honest caveat: verbatim reproduction is the exception, not the norm — the UK High Court held that Stable Diffusion's model weights never stored copies of the training images (defeating the secondary-infringement theory), and Getty abandoned its primary training-infringement claim at trial for lack of evidence, so whether the empirical phenomenon amounts to actionable harm (rather than transient, non-expressive use) remains the open question driving NYT v. OpenAI and parallel regimes.
Sources: Carlini, Tramèr, Wallace, Jagielski, Herbert-Voss, Lee, Roberts, Brown, Song, Erlingsson, Oprea & Raffel 2021 (Extracting Training Data from Large Language Models, 30th USENIX Security Symposium); Carlini, Ippolito, Jagielski, Lee, Tramèr & Zhang 2023 (Quantifying Memorization Across Neural Language Models, ICLR 2023; arXiv:2202.07646); Getty Images (US) Inc & ors v Stability AI Ltd [2025] EWHC 2863 (Ch) (UK High Court, 4 Nov 2025 — no secondary infringement; primary training claim abandoned at trial); The New York Times Co. v. Microsoft Corp. & OpenAI (S.D.N.Y., No. 1:23-cv-11195; consolidated In re OpenAI Copyright Infringement Litigation, Apr. 2025; ongoing 2025-2026)
There is no impact evaluation showing that the CDSM Directive Article 4 TDM exception plus its Article 4(3) opt-out reservation regime actually reduces unlicensed ingestion or channels compensation to rightsholders — the evidence that the rule works as designed is itself missing. The only available evidence is early case law and doctrinal scholarship, which document the mechanism's contested operation rather than its success: in Kneschke v. LAION the Hamburg Higher Regional Court (on appeal, 10 Dec 2025) held that a rights reservation in natural language did NOT satisfy Article 4(3)'s machine-readability requirement, invalidating the opt-out (note: the first-instance Regional Court had left the Article 4 question largely open and the case ultimately turned on the Article 3 scientific-research exception, so this machine-readability holding is appellate and not yet settled — a further appeal to the Federal Court of Justice was permitted). Legal scholars characterize the Article 4 opt-out as practically difficult and unharmonized, with no observed market in TDM licences or systematic enforcement to evaluate.
Sources: Kneschke v. LAION (Hamburg Regional Court, 27 Sept 2024, 310 O 227/23; on appeal Hamburg Higher Regional Court, 10 Dec 2025, 5 U 104/24 — opt-out held not machine-readable; further appeal to BGH permitted); Margoni & Kretschmer 2022 (A Deeper Look into the EU Text and Data Mining Exceptions, GRUR International 71(8):685-701); Quintais 2025 (Generative AI, Copyright and the AI Act, Computer Law & Security Review 56:106107)
Transparency Obligations
Documentation artifacts (model cards, datasheets) are well-specified as proposals and are genuinely adopted, but the empirical premise that mandated disclosure produces meaningful transparency is contested. Selbst & Barocas (2018) argue inscrutability and non-intuitiveness are distinct problems and that disclosing rules does not resolve the latter, and large-scale audits find documentation is sparsely and unevenly completed: a systematic analysis of 32,111 Hugging Face model cards (Liang et al. 2024) found environmental-impact, limitations and evaluation sections least often filled, and Bhat et al. (2023, 45 practitioners) found a substantial gap between the documentation proposal and actual practice. Honest caveat: the documentation frameworks themselves are real and adopted, so the dispute is about whether disclosure conveys decision-relevant information, not whether the artifacts exist.
Sources: Selbst & Barocas 2018 (Fordham Law Review 87:1085-1139); Liang et al. 2024 (Nature Machine Intelligence, s42256-024-00857-z, 'Systematic analysis of 32,111 AI model cards'); Bhat et al. 2023 (CHI '23, 'Aspirations and Practice of ML Model Documentation', DOI 10.1145/3544548.3581518); Mitchell et al. 2019 (FAccT, Model Cards for Model Reporting); Gebru et al. 2021 (CACM 64(12):86-92, Datasheets for Datasets)
There is no rigorous impact evaluation showing that AI transparency mandates (model cards, training-data summaries) measurably reduce bias, misuse or accidents — the central regulatory assumption is empirically untested, partly because flagship mandates like EU AI Act Art. 53(1)(d) GPAI training-data summaries are only subject to AI Office enforcement/verification from 2 August 2026 (the obligation itself began 2 August 2025 for new models). The closest analogue, mandated consumer disclosure, shows small and context-dependent effects: Bollinger, Leslie & Sorensen (2011) found mandatory calorie posting cut average calories per transaction by about 6%, while Loewenstein, Sunstein & Golman (2014) review evidence that disclosure effects are frequently diminished or even reversed by limited attention and often change provider rather than recipient behavior. These are analogues, not AI studies; no study demonstrates that AI transparency disclosure achieves its stated downstream safety aims.
Sources: Bollinger, Leslie & Sorensen 2011 (AEJ: Economic Policy 3(1):91-128); Loewenstein, Sunstein & Golman 2014 (Annual Review of Economics 6:391-419, 'Disclosure: Psychology Changes Everything'); EU AI Act Art. 53(1)(d) GPAI training-data summary (obligation from 2 Aug 2025; AI Office enforcement from 2 Aug 2026)