OMB Memorandum M-24-10 (Advancing Governance, Innovation, and Risk Management for Agency Use of AI)

Operative Mechanics: Three Pillars and the Risk-Management Floor

Issued March 28, 2024 under the Director of the Office of Management and Budget's authority to bind covered executive agencies, M-24-10 operationalizes Executive Order 14110 through three pillars: governance (Chief AI Officers and AI Governance Boards), responsible innovation, and risk management. Its operative core is Attachment 1, triggered by §5(c): before deploying "new or existing safety-impacting or rights-impacting AI," an agency must implement minimum practices — AI impact assessment, real-world performance testing, independent evaluation, ongoing monitoring, public notice, and human oversight — or "cease using the AI until compliance is achieved" by the December 1, 2024 deadline. Transparency runs through §3(a)(iv)'s annual public use-case inventory and §3(a)(v)'s aggregate-metric reporting, alongside a separate quarterly AI-procurement report to OMB, extending the EO 13960 inventory tradition.

Standing Relative to Binding Law

M-24-10 is not a statute or a notice-and-comment rule; it is an OMB memorandum whose force derives from the executive's control over agency operations and budget, not from the Administrative Procedure Act. It binds covered agencies internally yet creates no judicially enforceable private rights — §5(c)(v)(D)'s appeal-and-remedy guarantee is an administrative fallback, not a statutory cause of action. This contrasts sharply with the EU AI Act (Regulation (EU) 2024/1689), which imposes externally enforceable, fine-backed obligations on private and public deployers alike. The memo's reach is also confined: it governs only federal agency *use* of AI, leaving the broader market untouched. Its durability depends on executive continuity, since a successor administration may rescind or supersede it by memorandum without legislative process — a structural fragility absent from primary legislation.

Critiques and Structural Gaps

The memo's use-based "rights-impacting" trigger sidesteps model-capability classification: §5(c) imposes no compute threshold, so general-purpose foundation models are governed implicitly by deployment context rather than by training scale. This avoids the loophole problem Pistillo and Villalobos document for compute gates ¹ and the threshold-detectability rationale of Sastry et al. ², but it relies on accurate self-classification by agencies. Domain critiques sharpen the concern: Eloundou et al. find ~80% of the U.S. workforce exposed to LLM task-disruption ³, and Sheard shows anti-discrimination law struggling to reach algorithmic hiring harms ⁴ — gaps the memo's opt-out-where-practicable fallback cannot close. In healthcare, Weissman et al. ⁵ show unregulated LLMs producing device-like clinical output that the memo's agency-only scope never reaches.

Implementation Trajectory and Redress Design

Compliance hinges on the December 1, 2024 minimum-practices milestone and the recurring §3(a)(iv) public inventory, which makes agency self-disclosure the principal accountability lever — its credibility turns on completeness, a chronic weakness of prior EO 13960 inventories. The §5(c)(v)(D) human-consideration-and-remedy mandate is the memo's most consequential rights mechanism, yet contestability research warns that nominal appeal channels rarely deliver substantive recourse: Yurrita et al. specify what decision subjects need for *meaningful* contestation ⁶, and Schmude et al. distinguish judicial from non-judicial and individual from collective channels for public-sector AI ⁷. Trajectory risk is acute — as a memorandum the framework can be revised or revoked by a successor administration, leaving its inventory cadence and rights-impacting safeguards contingent rather than entrenched.

Cross-jurisdiction comparison

How peer instruments treat the topics OMB Memorandum M-24-10 (Advancing Governance, Innovation, and Risk Management for Agency Use of AI) governs.

°= industry self-imposed voluntary framework. Comparing a voluntary code's "governs" tint with a binding regulation's "governs" tint flattens the legal-force distinction; use the instrument-page banner for the operative status of each.

Does this instrument’s approach work? — the social-science evidence

Aggregated over the 6 topics this instrument governs: whether each harm is empirically real, and whether the peer-reviewed evidence shows governance reduces it. The badge is the epistemic status of the evidence— “thin”/“absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.

Of the 6 governed topics with a social-science evidence review, evidence that governance reduces the harm is established for 0, contested for 0, thin for 0, and absent for 6 — for most, no replicated study yet shows this instrument's approach works (the "second silence").

• Compute-Threshold Reporting

  • Is the harm real?evidence: contested

    Whether training-compute (FLOP) is a defensible proxy for governance-relevant capability is genuinely contested in the literature. The strongest empirical pressure against it is algorithmic efficiency: Ho, Besiroglu, Erdil et al. (2024) estimate the compute needed to reach a fixed language-model performance level has halved roughly every eight months (95% CI ~5-14 months, i.e. ~3x/year), so any static FLOP-to-capability mapping decays quickly; Hooker (2024) argues FLOP measures operations rather than end-performance, since techniques such as fine-tuning, retrieval, chain-of-thought and tool use can add large capability gains without proportional training compute, and Ord (2025) shows inference-time scaling further decouples deployed capability from training compute. Honest caveat: defenders (Heim & Koessler 2024; Pilz, Heim & Brown 2025) note compute remains the most quantifiable, externally verifiable, and ex-ante measurable correlate of frontier capability currently available, while themselves conceding it is an imperfect proxy that should not be used in isolation — the disagreement is about durability and precision, not whether any correlation exists.

    Sources: Ho, Besiroglu, Erdil, Owen, Rahman, Guo, Atkinson, Thompson & Sevilla 2024, Algorithmic progress in language models, NeurIPS 2024 (arXiv:2403.05812; Epoch AI); Hooker 2024, On the Limitations of Compute Thresholds as a Governance Strategy (arXiv:2407.05694); Ord 2025, Inference Scaling Reshapes AI Governance (arXiv:2503.05705); Heim & Koessler 2024, Training Compute Thresholds: Features and Functions in AI Regulation (arXiv:2405.10799); Pilz, Heim & Brown 2025, Increased Compute Efficiency and the Diffusion of AI Capabilities (AAAI 2025; arXiv:2311.15377)

  • Does governance work?evidence: absent

    There is no rigorous evidence that compute-threshold reporting reduces harm or achieves its stated aim, because the regimes have not produced an evaluable record. The US 10^26-FLOP reporting obligation (Executive Order 14110, invoking the Defense Production Act) was revoked on 20 January 2025 (by EO 14148) before its recurring binding reporting rule was finalized — the implementing BIS notice of proposed rulemaking (Sept 2024) never took effect, so no durable reporting record materialized; and the EU AI Act's 10^25-FLOP systemic-risk obligations for general-purpose models only became applicable on 2 August 2025 (with transitional periods into 2027), so no outcome evaluation yet exists. Moreover the 10^25 figure is a rebuttable presumption sitting alongside qualitative high-impact criteria (Art. 51(1)(a) and (2), rebuttable under Art. 52(2)), not a validated risk cutoff. The closest analogue is the broader regulatory-disclosure-mandate literature (Fung, Graham & Weil 2007), which documents that transparency policies' effects on outcomes are highly heterogeneous and frequently ineffective or counterproductive absent enforcement and downstream use — implying that the reporting trigger working as intended is an open empirical question, not a documented result.

    Sources: U.S. Executive Order 14110 (2023), Sec. 4.2 (10^26 FLOP, Defense Production Act); revoked by Executive Order 14148 (Jan 20, 2025); EU AI Act, Reg. (EU) 2024/1689, Art. 51 (10^25 FLOP systemic-risk rebuttable presumption; applicable Aug 2, 2025); Fung, Graham & Weil 2007, Full Disclosure: The Perils and Promise of Transparency (Cambridge University Press)

• AI in Employment

  • Is the harm real?evidence: established

    Discrimination and adverse outcomes in employment decisions are empirically well-established, and AI systems demonstrably reproduce them. The foundational field-experiment literature shows robust human baseline discrimination (Bertrand & Mullainathan 2004 found White-sounding names received 50% more callbacks), and AI-specific audits confirm the pattern: Amazon scrapped a recruiting tool that penalized resumes containing 'women's' (Dastin 2018), and a controlled resume-screening audit of language-model retrieval found systems favored White-associated names ~85% of the time and never preferred Black male-associated over White male-associated names (Wilson & Caliskan 2024). On the monitoring side, a meta-analysis (k=94, N≈23,461) found electronic performance monitoring reliably raises worker stress with no evidence of improved performance (Ravid et al. 2023). Honest caveat: measured disparities are highly model-, prompt-, and context-dependent, and most evidence comes from controlled audits and one firm's internal test rather than measured outcomes in live, at-scale hiring pipelines.

    Sources: Bertrand & Mullainathan 2004 (American Economic Review 94(4):991-1013); Wilson & Caliskan 2024 (AAAI/ACM AIES; 'Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval'); Dastin 2018 (Reuters, 'Amazon scraps secret AI recruiting tool that showed bias against women'); Ravid, White, Tomczak & Behrend 2023 (Personnel Psychology 76:5-40)

  • Does governance work?evidence: absent

    There is no rigorous evidence that governing AI in employment reduces the documented harms; the central evaluated regime appears to fail at the compliance stage before any impact on bias can occur. NYC Local Law 144 — the first jurisdiction worldwide to mandate independent bias audits and public posting for automated employment decision tools — was directly studied across 391 employers and found to produce 'null compliance': the law's discretion makes it impossible to tell whether firms comply, with very few posting the required audits (Wright et al. 2024). Parallel qualitative work shows the audits themselves are undermined by missing demographic data, opaque aggregation, and 'test data' that does not reflect real use (Groves et al. 2024). No study links any AI-employment rule to a measured reduction in discriminatory hiring outcomes — the evidence that the rule works is itself missing, largely because mandated transparency artifacts (audit reports) are sparse, non-standardized, and unenforced.

    Sources: Wright, Muenster, Vecchione, Metcalf & Matias et al. 2024 ('Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability', ACM FAccT '24); Groves, Metcalf, Kennedy, Vecchione & Strait 2024 ('Auditing Work: Exploring the New York City algorithmic bias audit regime', ACM FAccT '24); Ravid, White, Tomczak & Behrend 2023 (Personnel Psychology 76:5-40, on monitoring outcomes as the closest analogue evaluation evidence)

• Foundation Models / GPAI

  • Is the harm real?evidence: contested

    Whether the foundation-model category maps to a coherent capability/risk tier is genuinely contested. The original case rests on scale-driven 'emergent abilities' that appear unpredictably above a size threshold (Wei et al. 2022; Ganguli et al. 2022 documented capabilities that are smoothly predictable in aggregate loss yet locally surprising), but Schaeffer, Miranda & Koyejo (2023, a NeurIPS Outstanding Paper) showed many 'emergent' jumps are artefacts of discontinuous metrics and dissolve under linear/continuous scoring — implying capability scales more smoothly than a sharp tier would suggest. Honest caveat: this is a live empirical disagreement about measurement, not a settled finding either way, and compute (the regulatory proxy) is an imperfect stand-in for capability or risk regardless of which side is right.

    Sources: Wei et al. 2022 (Emergent Abilities of Large Language Models, TMLR; arXiv:2206.07682); Schaeffer, Miranda & Koyejo 2023 (Are Emergent Abilities of Large Language Models a Mirage?, NeurIPS 2023, Outstanding Paper; arXiv:2304.15004); Ganguli et al. 2022 (Predictability and Surprise in Large Generative Models, ACM FAccT; DOI 10.1145/3531146.3533229)

  • Does governance work?evidence: absent

    There is no impact evaluation showing that GPAI/foundation-model governance reduces harm — the rules are too new (EU AI Act GPAI obligations and the 10^25-FLOP systemic-risk presumption only began binding on 2 August 2025) and the central regulatory lever is itself contested: Hooker (2024) argues compute thresholds are a shortsighted proxy because compute does not reliably track capability or risk, and the thresholds already diverge across jurisdictions (EU 10^25 vs. the now-rescinded US EO 14110's 10^26 operations, rescinded 20 January 2025). The mandated mitigation methods also lack validated efficacy: model evaluation and red-teaming face well-documented coverage limits and an 'audit gap' in the survey/position literature (behavioural testing cannot establish the absence of untested failure modes), and adversarial red-teaming repeatedly defeats deployed safeguards — the UK AI Safety Institute reports finding universal jailbreaks for every frontier system it has tested, and a large public agent-injection competition elicited policy violations across all 22 frontier models tested from ~1.8M attacks (Zou et al. 2025). Even compliant evaluation therefore cannot yet certify the safety the rules demand. (Caveat: this is an absence-of-evidence claim — no efficacy study has been done — not evidence the rules are ineffective.)

    Sources: Hooker 2024 (On the Limitations of Compute Thresholds as a Governance Strategy, arXiv:2407.05694); EU AI Act Arts. 51 & 55 (GPAI systemic-risk presumption, 10^25 FLOP; binding 2 Aug 2025); US EO 14110 (10^26-operation reporting threshold, rescinded 20 Jan 2025 by EO 14148); Zou et al. 2025 (Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition / Gray Swan Arena, arXiv:2507.20526 — 22 frontier agents, ~1.8M attacks); UK AI Safety/Security Institute, Frontier AI Trends Report (universal jailbreaks for every system tested); METR, Common Elements of Frontier AI Safety Policies (2024)

• AI in Healthcare

  • Is the harm real?evidence: established

    Both the benefit and the harm of clinical AI are empirically real and well-documented, but outcomes are highly deployment-dependent. Rigorous prospective studies show genuine clinical value in narrow tasks — the MASAI RCT (>100,000 women) found AI-supported mammography detected ~20% more cancers (6.1 vs 5.1 per 1000 screened) at comparable recall rates (Lang et al. 2023, Lancet Oncology), and IDx-DR's pivotal trial achieved 87.2% sensitivity / 90.7% specificity for diabetic retinopathy (Abramoff et al. 2018, npj Digital Medicine) — yet widely deployed models can fail or harm: the Epic Sepsis Model, live at hundreds of US hospitals, scored AUC 0.63 with 33% sensitivity on external validation (Wong et al. 2021, JAMA Internal Medicine), and a population-health algorithm covering ~200M people understated Black patients' illness because it predicted cost not need (Obermeyer et al. 2019, Science). Honest caveat: there is no single 'AI in healthcare' effect — performance ranges from life-saving to dangerous depending on task, calibration, and whether the model was prospectively validated.

    Sources: Lang K, Josefsson V, Larsson A-M, et al. 2023 (Lancet Oncology 24(8):936-944, MASAI trial clinical safety analysis; AI-supported screening detected 6.1 vs 5.1 cancers per 1000, ~20% higher, similar recall rates); Abramoff MD, Lavin PT, Birch M, Shah N, Folk JC. 2018 (npj Digital Medicine 1:39, IDx-DR pivotal trial; 87.2% sensitivity / 90.7% specificity); Wong A, Otles E, Donnelly JP, et al. 2021 (JAMA Internal Medicine 181(8):1065-1070, Epic Sepsis Model external validation; AUC 0.63, 33% sensitivity); Obermeyer Z, Powers B, Vogeli C, Mullainathan S. 2019 (Science 366(6464):447-453, racial bias from cost-as-proxy)

  • Does governance work?evidence: absent

    There is essentially no impact-evaluation evidence that the prevailing governance regime for medical AI — FDA authorization, predominantly via the 510(k) substantial-equivalence pathway — measurably reduces patient harm or improves outcomes. Analyses of authorized AI devices find that clinical validation is frequently absent or non-prospective (of 521 FDA-authorized AI devices, ~43% had no published clinical-validation data and only ~28% were prospectively validated; Chouffani El Fassi & Henderson et al. 2024) and that demographic performance is almost never reported (race/ethnicity in 3.6%, and only 9.0% of 692 510(k)/cleared AI devices carried a prospective post-market-surveillance study; Muralidharan et al. 2024). Earlier analysis of 130 cleared devices likewise found 97% were evaluated only retrospectively (Wu et al. 2021). The closest analogue evidence on the pathway itself is discouraging: the Institute of Medicine (2011) concluded the 510(k) process was not designed to assess safety and effectiveness — i.e., no direct study establishes that the rule, as written, prevents the harms it targets. Caveat: this is an absence of impact evaluation plus reporting-gap and design-critique evidence, not a study showing the regime fails to reduce harm.

    Sources: Chouffani El Fassi S, Abdullah A, Fang Y, ... Henderson GE, et al. 2024 (Nature Medicine, 'Not all AI health tools with regulatory authorization are clinically validated', s41591-024-03203-3; 521 devices, ~43% no clinical validation, ~28% prospectively validated); Muralidharan V, Adewale BA, Huang CJ, et al. 2024 (npj Digital Medicine 7:273, scoping review of reporting gaps in 692 FDA-approved AI medical devices; race/ethnicity 3.6%, prospective post-market surveillance 9.0%); Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. 2021 (Nature Medicine 27:582-584, analysis of 130 FDA approvals; 97% retrospective-only evaluation); Institute of Medicine 2011 (Medical Devices and the Public's Health: The FDA 510(k) Clearance Process at 35 Years)

• Individual Redress

  • Is the harm real?evidence: established

    The premise behind redress — that affected people lack meaningful recourse against automated decisions — is real, but the flagship instrument is weaker than commonly assumed. Wachter, Mittelstadt & Floridi (2017) show GDPR creates only a limited 'right to be informed,' not a binding 'right to explanation' of specific decisions; and controlled work finds the explanations actually delivered do not measurably improve lay decision accuracy over showing the bare AI prediction (Alufaisan et al. 2021; and a 2022 meta-analysis by Schemmer et al. — screening 393 articles down to 9 in the final analysis — reports 'no effect of explanations on users' performance compared to sole AI predictions,' even though XAI overall had a positive effect). Honest caveat: the legitimacy/dignity value of being heard is empirically well established in the procedural-justice tradition even where outcome accuracy is unchanged, so 'redress fails' depends on which aim is measured.

    Sources: Wachter, Mittelstadt & Floridi 2017 (International Data Privacy Law 7(2):76); Alufaisan, Marusich, Bakdash, Zhou & Kantarcioglu 2021 (Proceedings of the AAAI Conference on AI 35(8):6618); Schemmer, Hemmer, Nitsche, Kühl & Vössing 2022 (AAAI/ACM AIES '22, meta-analysis)

  • Does governance work?evidence: absent

    There is no rigorous impact evaluation showing that mandated redress mechanisms (right-to-explanation, appeal, human-in-the-loop review) actually reduce erroneous or unfair automated decisions — the evidence that the rule works is itself missing. The closest experimental analogues are discouraging: explanations increase humans' acceptance of AI recommendations regardless of correctness (Bansal et al. 2021), and algorithm-in-the-loop oversight can introduce racial disparities and exhibit automation bias rather than reliably catching model errors (Green & Chen 2019). The procedural-justice literature (Tyler 1990; Lind & Tyler 1988) robustly supports a legitimacy and compliance benefit of fair process, but it measures perceived fairness, not reduction of the substantive decision harm redress is meant to cure.

    Sources: Bansal, Wu, Zhou, Fok, Nushi, Kamar, Ribeiro & Weld 2021 (CHI '21); Green & Chen 2019 (Disparate Interactions, ACM FAT* '19); Tyler 1990 (Why People Obey the Law, Yale Univ. Press); Lind & Tyler 1988 (The Social Psychology of Procedural Justice, Plenum Press)

• Transparency Obligations

  • Is the harm real?evidence: contested

    Documentation artifacts (model cards, datasheets) are well-specified as proposals and are genuinely adopted, but the empirical premise that mandated disclosure produces meaningful transparency is contested. Selbst & Barocas (2018) argue inscrutability and non-intuitiveness are distinct problems and that disclosing rules does not resolve the latter, and large-scale audits find documentation is sparsely and unevenly completed: a systematic analysis of 32,111 Hugging Face model cards (Liang et al. 2024) found environmental-impact, limitations and evaluation sections least often filled, and Bhat et al. (2023, 45 practitioners) found a substantial gap between the documentation proposal and actual practice. Honest caveat: the documentation frameworks themselves are real and adopted, so the dispute is about whether disclosure conveys decision-relevant information, not whether the artifacts exist.

    Sources: Selbst & Barocas 2018 (Fordham Law Review 87:1085-1139); Liang et al. 2024 (Nature Machine Intelligence, s42256-024-00857-z, 'Systematic analysis of 32,111 AI model cards'); Bhat et al. 2023 (CHI '23, 'Aspirations and Practice of ML Model Documentation', DOI 10.1145/3544548.3581518); Mitchell et al. 2019 (FAccT, Model Cards for Model Reporting); Gebru et al. 2021 (CACM 64(12):86-92, Datasheets for Datasets)

  • Does governance work?evidence: absent

    There is no rigorous impact evaluation showing that AI transparency mandates (model cards, training-data summaries) measurably reduce bias, misuse or accidents — the central regulatory assumption is empirically untested, partly because flagship mandates like EU AI Act Art. 53(1)(d) GPAI training-data summaries are only subject to AI Office enforcement/verification from 2 August 2026 (the obligation itself began 2 August 2025 for new models). The closest analogue, mandated consumer disclosure, shows small and context-dependent effects: Bollinger, Leslie & Sorensen (2011) found mandatory calorie posting cut average calories per transaction by about 6%, while Loewenstein, Sunstein & Golman (2014) review evidence that disclosure effects are frequently diminished or even reversed by limited attention and often change provider rather than recipient behavior. These are analogues, not AI studies; no study demonstrates that AI transparency disclosure achieves its stated downstream safety aims.

    Sources: Bollinger, Leslie & Sorensen 2011 (AEJ: Economic Policy 3(1):91-128); Loewenstein, Sunstein & Golman 2014 (Annual Review of Economics 6:391-419, 'Disclosure: Psychology Changes Everything'); EU AI Act Art. 53(1)(d) GPAI training-data summary (obligation from 2 Aug 2025; AI Office enforcement from 2 Aug 2026)

Related instruments

• EU AI Act · EU
• OECD AI Principles (Recommendation) · OECD
• Council of Europe Framework Convention on AI · council_of_europe
• Brazil AI Bill (PL 2338/2023) · BR
• General Data Protection Regulation (GDPR) · EU
• GSA Generative AI and Specialized Computing Infrastructure Acquisition Resource Guide · US
• California SB 243: Companion Chatbots · US
• UNESCO Recommendation on the Ethics of Artificial Intelligence · UNESCO
• Directive (EU) 2024/2831 on improving working conditions in platform work · EU
• Provisions on the Administration of Deep Synthesis of Internet Information Services · CN