?asOf= parameter to see the current catalog state.Training-Data Attribution
training-data-attribution · Frontier safety
Technical methods that identify which training examples most influenced a specific AI model output, enabling provenance claims about generated content and supporting copyright / consent / accountability disputes downstream.
Definition & scope
Training-data attribution (TDA) is the inverse of training: given an output, recover the training examples that caused it. The technical lineage runs from influence functions (Koh & Liang 2017, 'Understanding Black-box Predictions via Influence Functions,' ICML) through gradient-based methods (Pruthi et al. 2020, TracIn) to recent scalable approximations for foundation models (Grosse et al. 2023, Anthropic, 'Studying Large Language Model Generalization with Influence Functions'; Park et al. 2023 TRAK). Adjacent methods include training-data extraction (Carlini et al. 2021, 'Extracting Training Data from Large Language Models') which surfaces verbatim memorisation rather than influence. Governance relevance is now legally acute. The NYT v. OpenAI complaint (Dec 2023) used training-data extraction to show verbatim NYT articles in GPT-4 outputs; ongoing US copyright suits (Authors Guild v. OpenAI, Getty v. Stability AI, Tremblay v. OpenAI) turn partly on whether attribution methods can demonstrate substantial similarity at training-corpus scale. EU AI Act Art. 53(1)(c) requires GPAI providers to publish a 'sufficiently detailed summary' of training-data content — a disclosure obligation that is the regulatory analogue of attribution. China's GenAI Measures Art. 7 requires legal sourcing of training data. Brazil's PL 2338/2023 includes an explicit author-compensation provision. India's DPDPA does not yet address training-data rights directly, but the 2024 MEITY advisories signal forthcoming guidance. Methodologically, TDA at frontier-model scale remains contested: influence-function approximations require restrictive assumptions (locally-linear loss surface) that don't hold for over-parameterised LLMs, and verbatim-extraction methods undercount the (likely larger) population of paraphrased or compositionally-derived outputs.
Use in governance
How instruments operationalise this concept
| Instrument | Jurisdiction | Status |
|---|---|---|
| EU AI Act | EU | in force |
| Interim Measures for Generative AI Service Management | CN | in force |
| Brazil AI Bill (PL 2338/2023) | BR | proposed |
Appears in topic articles
Editorial note
Distinguish TDA (which training examples *caused* this output, by influence) from training-data extraction (which examples are verbatim recoverable from the model). Both are policy-relevant but for different claims: influence supports causal-contribution arguments, extraction supports memorisation arguments.
See also
Further reading
Sources on the broader topics this concept relates to — complementing, not standing in for, the primary sources cited inline above. 58 academic & grey-literature sources; catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- Open Foundation Models and TDM Exceptions to Copyright – Building Blocks for an AI Ecosystem Peer-reviewed✦ AIArgues Art. 3 CDSM Directive's scientific-research TDM exception 'does not grant rightsholders any control' and can be a 'safe harbor' for training openly released foundation models without licensing data.
- Identifying Algorithmic Decision Subjects' Needs for Meaningful Contestability Peer-reviewed✦ AIEmpirically elicits what decision subjects need for contestation to be 'meaningful', informing the design of effective remedies and appeal mechanisms for ADM.
- Two Means to an End Goal: Connecting Explainability and Contestability in the Regulation of Public Sector AI Preprint✦ AIInterview study with 14 regulation experts distinguishes judicial vs non-judicial and individual vs collective contestation channels for public-sector AI remedies.
- Copyright and AI in the UK: Opting-In or Opting-Out? Peer-reviewed✦ AIContends the UK opt-in/opt-out framing is a 'missed opportunity'; a broadened research exception plus market-entry transparency and creator remuneration would better serve both innovation and rightsholders.
- Technical Challenges of Rightsholders' Opt-out From Gen AI Training after Robert Kneschke v. LAION Peer-reviewed✦ AIExamines post-LAION practical obstacles to the EU TDM opt-out (robots.txt, machine-readability, memorisation): 'While the TDM exceptions may seem workable in theory, implementing them in practice presents a variety of practical…
- Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity Peer-reviewed✦ AIExamines how the EU AI Act, liability regimes, GDPR, copyright and cybersecurity rules apply to generative AI, identifying gaps and proposing targeted regulatory refinements.
- A large-scale audit of dataset licensing and attribution in AI Peer-reviewed✦ AIAudit of 1,800+ AI training datasets finds "licence omission rates of more than 70% and error rates of more than 50%" on popular hosting sites.
- The Right to Transparency in Public Governance: Freedom of Information and the Use of Artificial Intelligence by Public Agencies Peer-reviewed✦ AIFinds freedom-of-information regimes "generally only grant access to existing documents" and that with "no mature standard for documenting AI models," public-sector AI transparency is limited.
- On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives Peer-reviewed✦ AISynthesises interdisciplinary evidence to argue that legally mandated human oversight of AI is often ineffective ('rubber-stamp') unless effectiveness conditions are explicitly designed for.
- Law and the Emerging Political Economy of Algorithmic Audits Peer-reviewed✦ AIAnalyses how AI-audit mandates create a new political economy of auditing, warning that audit markets can entrench rather than constrain power without underlying governance.
- Understanding Contestability on the Margins: Implications for the Design of Algorithmic Decision-making in Public Services Peer-reviewed✦ AIField study shows marginalized public-service users need intermediaries and informal channels for contestation, challenging individualistic right-to-contest designs.
- Lawfulness of the mass processing of publicly accessible online data to train large language models Peer-reviewed✦ AIArgues LLM training on scraped web data should be assessed under Art. 9 GDPR (sensitive data), and that consent and the 'manifestly made public' route leave only a 'limited amount of personal data' lawfully usable.
+ 46 more across this concept's topics — see the literature index.
References
The primary instrument sources behind the article's classifications.
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/training-data-attribution — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here