Synthetic Content Provenance

Regulatory approaches

The binding instruments converge on the same goal — make AI-origin detectable — but split on WHO must act and by what modality. The EU AI Act sets a two-sided architecture: Article 50(2) places a machine-readable marking duty on PROVIDERS of generative systems (outputs must be "marked in a machine-readable format and detectable as artificially generated or manipulated", using solutions that are "effective, interoperable, robust and reliable as far as this is technically feasible"), while Article 50(4) places a human-facing DISCLOSURE duty on DEPLOYERS of systems that produce deepfakes or public-interest AI text (EU AI Act Art. 50(2), 50(4); applicable 2 August 2026). China's Measures for Labeling AI-Generated Synthetic Content instead impose a dual-track duty on service providers themselves: an EXPLICIT label visible to users PLUS an IMPLICIT label embedded in file metadata, across text, image, audio, video and virtual scenes (CAC et al., effective 1 September 2025) — situated by analysts within China's layered 2022 deep-synthesis and 2023 generative-AI regime that treats labelling and watermarking as a provenance-governance model ¹. Voluntary regimes specify mechanism without mandate: the G7 Hiroshima Process Code of Conduct asks developers to deploy "reliable content authentication and provenance mechanisms... such as watermarking" carrying a model/service identifier (G7 Hiroshima Code of Conduct, 30 Oct 2023, commitment 6); some scholars go further, arguing legislation should require developers to demonstrate a reliable detection mechanism as a precondition of public release ². The C2PA standard supplies the cryptographic substrate these rules lean on — signed provenance manifests (Coalition for Content Provenance and Authenticity). The structural divide is provider-side technical marking (EU 50(2), China implicit) versus deployer/user-facing disclosure (EU 50(4), China explicit).

Key fault lines

Three governance-design disputes run beneath the convergent rhetoric. FIRST, locus of the duty: the EU splits provider marking from deployer disclosure (EU AI Act Art. 50(2) vs 50(4)), whereas China loads both the visible and embedded signal onto the generating service (China Labeling Measures, eff. 2025-09-01) — a divergence over whether platforms that REDISTRIBUTE content also owe detection duties, which California is now testing by requiring large online platforms to detect embedded provenance from 1 January 2027 (California AB 853, signed 13 Oct 2025). SECOND, durability of the signal: the dominant standard concedes that cryptographic manifests are "attached to" rather than embedded in assets and "can easily be stripped", with platforms re-encoding on upload (C2PA, durable Content Credentials rationale) — and the deeper computer-science result is that robust marking has hard limits: a formal systematization sets out the robustness/security goals and bounds for AI watermarking ³, recursive paraphrasing can drive text-detection rates down while barely degrading quality ⁴, and under natural assumptions strong watermarking is proven impossible ⁵, meaning the marking the law presumes is itself contestable. THIRD, scope of exemptions: Article 50 carves out assistive editing that does not "substantially alter" inputs, law-enforcement use, and "evidently artistic, creative, satirical, fictional" works, plus human-reviewed text under editorial responsibility (EU AI Act Art. 50(2), 50(4)) — boundaries that determine how much real-world content the regime actually reaches.

Trajectory — what's changing

Provenance obligations are entering force on a compressed 2025-2026 timeline, even as evidence questions whether the mechanisms work. China moved first: the Labeling Measures and the accompanying mandatory national standard took effect 1 September 2025, making the dual explicit/implicit label legally operative for generation and synthesis services (CAC et al., effective 2025-09-01; TC260 content-labeling standard) (Loeb & Loeb 2025). The EU's Article 50 transparency obligations become legally effective 2 August 2026; to operationalise the otherwise abstract "state of the art" standard, the Commission ran a public consultation from September 2025, published draft transparency guidelines in May 2026, and finalised a voluntary Code of Practice on marking and labelling of AI-generated content on 10 June 2026 — voluntary to sign but evidencing compliance with the binding Article 50 duties (European Commission, Code of Practice; EU AI Act Art. 50). Yet an empirical audit finds only 38% of AI image generators implement adequate watermarking and 18% deepfake labelling, exposing a live compliance gap against Article 50 ⁶; user studies likewise complicate the premise, finding provenance information often lowered trust in deceptive media but could also reduce trust in truthful media ⁷. The United States is converging through state law rather than federal mandate: California's AI Transparency Act (SB 942) was delayed by AB 853 from 1 January 2026 to 2 August 2026 to align with the EU date, and AB 853 layers in staged duties — large-platform provenance DETECTION from 1 January 2027 and capture-device latent-disclosure options from 1 January 2028 (California SB 942; AB 853, signed 13 Oct 2025). On the technical layer, C2PA's shift to durable Content Credentials (soft-binding watermarks, C2PA 2.x, 2025) signals the underlying standard is still maturing as the legal deadlines arrive. The federal layer is not wholly absent of soft law: the White House Voluntary AI Commitments ask signatories to "develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated, including robust provenance, watermarking, or both" (Voluntary commitment #5). The NIST AI RMF Generative AI Profile reinforces this federal posture by naming Information Integrity, covering synthetic-content labelling and provenance, as one of twelve GenAI risk categories (NIST AI 600-1).

Historical primacy & cross-jurisdiction tension

First addressed by Provisions on the Administration of Deep Synthesis of Internet Information Services on 2022-11-25 (governs). Subsequent regimes have either codified, diverged from, or remained silent on this baseline.

• Forum-shoppingEU AI Act↔Executive Order 14179 — Removing Barriers to American Leadership in AI
• Forum-shoppingExecutive Order 14110 on Safe, Secure, Trustworthy AI↔UK Pro-Innovation Approach to AI Regulation (White Paper)
• Forum-shoppingInterim Measures for Generative AI Service Management↔OECD AI Principles (Recommendation)

Silent regimes — gap signal

Instruments that do not address Synthetic Content Provenance — candidates for future policy work.

• Executive Order 14179 — Removing Barriers to American Leadership in AIUS
• UK Pro-Innovation Approach to AI Regulation (White Paper)UK
• OECD AI Principles (Recommendation)OECD
• Council of Europe Framework Convention on AIcouncil_of_europe
• Bletchley Declaration on AI Safetyglobal
• Seoul Declaration on Safe, Innovative and Inclusive AIglobal
• California SB-1047: Safe and Secure Innovation for Frontier AI Models ActUS
• India Digital Personal Data Protection Act + AI Advisory (MEITY)IN
• ASEAN Guide on AI Governance and EthicsASEAN
• African Union Continental AI StrategyAfrican_Union
• OpenAI Preparedness FrameworkUS
• Google DeepMind Frontier Safety FrameworkUS
• Meta Frontier AI FrameworkUS
• UK-US AI Safety Institute Memorandum of Understandingglobal
• General Data Protection Regulation (GDPR)EU
• OMB Memorandum M-24-10 (Advancing Governance, Innovation, and Risk Management for Agency Use of AI)US
• GSA Generative AI and Specialized Computing Infrastructure Acquisition Resource GuideUS
• DoD Responsible AI Strategy and Implementation PathwayUS
• FedRAMP AI Cloud Procurement GuidanceUS
• DFARS Subpart 252.204 (Safeguarding Covered Defense Information and Cyber Incident Reporting)US
• California SB-53: Transparency in Frontier Artificial Intelligence Act (TFAIA)US
• California SB 243: Companion ChatbotsUS
• Revised Product Liability Directive (Directive (EU) 2024/2853)EU
• UNESCO Recommendation on the Ethics of Artificial IntelligenceUNESCO
• Directive (EU) 2024/2831 on improving working conditions in platform workEU
• New York RAISE Act: Responsible AI Safety and Education ActUS
• TAKE IT DOWN Act (Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Networks Act)US
• Japan AI Promotion Act (Act on the Promotion of Research, Development and Utilization of AI-Related Technologies)JP

Does governance work? — the social-science evidence

What the peer-reviewed social science shows: whether the harm this topic addresses is empirically real, and whether governance of it works. The badge is the epistemic status of the evidence(not the policy debate) — “thin” or “absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.

• Is the harm real?evidence: contested

  The harm provenance targets is real but concentrated, and the technical premise that the mandated signal survives is itself empirically shaky. Synthetic-media harm is well documented in two domains: non-consensual intimate imagery (Ajder et al.'s 2019 Deeptrace audit found 96% of deepfake videos were pornographic and effectively 100% targeted women) and impersonation fraud (the Arup case, ~US$25.6M / HK$200M lost via a deepfake video call). The honest caveat is twofold: a feared broad political-misinformation harm is not yet demonstrated at scale, and CS work shows invisible watermarks are removable in practice (Jiang, Zhang & Gong 2023, WEvade, evade detection via adversarial perturbation; Zhao et al. 2024 prove pixel-level watermarks are provably removable via regeneration attacks), so the provenance signal a rule would mandate is itself contested.

  Sources: Ajder, Patrini, Cavalli & Cullen 2019 (Deeptrace, 'The State of Deepfakes: Landscape, Threats, and Impact'); Jiang, Zhang & Gong 2023 ('Evading Watermark based Detection of AI-Generated Content', ACM CCS 2023); Zhao et al. 2024 (NeurIPS, 'Invisible Image Watermarks Are Provably Removable Using Generative AI'); Arup deepfake fraud (CNN Business, 2024-05-16, US$25.6M)

• Does governance work?evidence: thin

  There is no impact evaluation showing that mandated provenance/labeling reduces synthetic-media harm; the major mandates (China's GenAI labeling Measures, effective 2025-09-01; EU AIA Art. 50, machine-readable marking) are too new and unevaluated, and the delivery layer is leaky: the C2PA spec's own Security Considerations document the strip-and-repost threat, and platform audits report C2PA/Content-Credentials metadata is stripped by essentially all major social platforms on upload (consistent with Imatag's 2018 finding that ~80% of uploaded images lose metadata, only ~15% retaining it). The closest analogue evaluation literature — Pennycook, Bear, Collins & Rand (2020), the 'implied truth effect' — gives reason for caution rather than confidence: labeling only some content can make unlabeled false content seem more credible, so a partial-coverage provenance regime could backfire.

  Sources: Pennycook, Bear, Collins & Rand 2020 (Management Science 66(11):4944-4957, 'The Implied Truth Effect'); China Measures for Labeling AI-Generated Synthetic Content (eff. 2025-09-01); EU AI Act Art. 50; Imatag 2018 metadata-stripping study (~80%); C2PA Security Considerations (spec.c2pa.org) on manifest removal

Related topics

Topics with similar coverage patterns across the tracked instruments.

• Deepfakes / Synthetic Content
• Catastrophic & Existential Risk