?asOf= parameter to see the current catalog state.Inference-Time Compute
inference-time-compute · Compute governance
The scaling regime in which model capability is increased by spending more compute at inference time (multiple samples, search, longer reasoning chains, tool-using iteration) rather than by training a larger model — disrupting the training-compute-as-capability-proxy assumption underlying most current AI governance.
Definition & scope
The dominant assumption underlying compute-threshold regulation (EU AIA Art. 51, US EO 14110 §4.2(a)) is that training compute correlates with deployment capability. Inference-time-compute scaling complicates this: a model trained at compute level C can be deployed with inference-time compute K·C per response, producing capability properties intermediate between the base model and a model trained at K·C. OpenAI's o1 (Sep 2024) and o3 (Dec 2024) series, Anthropic's extended-thinking modes, DeepMind's AlphaCode-2 / AlphaProof, and DeepSeek-R1 (Jan 2025) demonstrate the regime empirically. Snell et al. (2024, 'Scaling LLM Test-Time Compute Optimally') and Brown et al. (2024) provide the empirical scaling laws. Governance implications are direct. (a) Compute thresholds based on training-FLOPs alone (EU AIA 10²⁵, US EO 10²⁶) understate the deployed capability of inference-scaled models. (b) DeepSeek-R1 demonstrated frontier-tier reasoning at training-compute well below 10²⁵ FLOPs, weakening the threshold's empirical defensibility. (c) Capability evaluations must specify the inference-compute budget under which the model was tested, since a model can be safe at K=1 and dangerous at K=100. (d) The mitigation surface for inference-time-scaled capabilities is different — restricting access to high-compute deployment APIs is policy-tractable in a way that restricting model-weight distribution is not. The Seoul Declaration + Frontier AI Safety Commitments (May 2024) gesture at this with 'pre-deployment evaluation under realistic conditions,' but no regulator has yet formalised inference-compute-aware thresholds.
Use in governance
How instruments operationalise this concept
| Instrument | Jurisdiction | Status |
|---|---|---|
| Seoul Declaration on Safe, Innovative and Inclusive AI | global | in force |
Appears in topic articles
Editorial note
When citing 'compute' in AI-governance contexts post-2024, specify whether the claim is about training-time or inference-time compute. Conflating the two is the most common analytical error in 2025-2026 policy writing on compute thresholds.
See also
Further reading
Sources on the broader topics this concept relates to — complementing, not standing in for, the primary sources cited inline above. 67 academic & grey-literature sources; catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- European ambitions captured by American clouds: digital sovereignty through Gaia-X? Peer-reviewed✦ AIShows Gaia-X paradoxically incorporates dominant US cloud providers, undermining the very European digital sovereignty it was meant to advance.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- Defending Compute Thresholds Against Legal Loopholes Preprint✦ AIIdentifies 'enhancement techniques that are capable of decreasing training compute usage while preserving... model capabilities', exposing loopholes in compute-reporting thresholds.
- GPTs are GPTs: Labor market impact potential of LLMs Peer-reviewed✦ AIFinds around 80% of the U.S. workforce "could have at least 10% of their work tasks affected" by LLMs, which exhibit "traits of general-purpose technologies".
- Computing Power and the Governance of Artificial Intelligence Preprint✦ AIArgues compute is a uniquely governable lever because it is "detectable, excludable, and quantifiable, and is produced via an extremely concentrated supply chain".
- Training Compute Thresholds: Features and Functions in AI Regulation Preprint✦ AIFinds "training compute currently is the most suitable metric to identify GPAI models", but thresholds should only trigger further scrutiny, not determine risk measures alone.
- Compute North vs. Compute South: The Uneven Possibilities of Compute-based AI Governance Around the Globe Peer-reviewed✦ AICensus of hyperscale cloud regions shows a divide between "Compute North" states hosting training-relevant compute and a Compute South, shaping who can wield compute-based governance.
- Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity Peer-reviewed✦ AIExamines how the EU AI Act, liability regimes, GDPR, copyright and cybersecurity rules apply to generative AI, identifying gaps and proposing targeted regulatory refinements.
- European Dreams of the Cloud: Imagining Innovation and Political Control Peer-reviewed✦ AIAnalysis of GAIA-X, Bundescloud and Microsoft's EU cloud reveals 'a performative coupling of innovation and political ideas of control, territoriality and sovereignty'.
- Evaluating Frontier Models for Dangerous Capabilities Preprint✦ AIPilots dangerous-capability evaluations (persuasion, cyber, self-proliferation) on frontier models, finding 'early warning signs' but no strong present danger — grounding evaluation-based gating.
+ 55 more across this concept's topics — see the literature index.
References
The primary instrument sources behind the article's classifications.
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/inference-time-compute — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here