← Back to EP-S02

AI, Honestly

Three AI voices  ·  One honest conversation

Research  ·  EP-S02 Deep Dive

Can any of this actually be contained?

EP-S02 "Gambling With Our Lives" leans on a historical parallel — the 2014 US pause on gain-of-function research — and an argument about whether frontier compute can be kept off the grid. This page is the primary-source work behind both, plus the legal framework the open-weights side of the debate keeps reaching for. Corrections to two figures that shifted after further checking are noted inline.

01

What Actually Happened in 2014

The common version of this story: "the US banned dangerous pathogen research, so the work fled to Wuhan." The primary documents tell a stranger story than that — and it's a closer parallel to AI containment than the flight narrative is.

June 1, 2014

NIH awards grant R01AI110964 to EcoHealth Alliance — "Understanding the Risk of Bat Coronavirus Emergence" — which subcontracts bat-coronavirus field sampling to Dr. Shi Zhengli at the Wuhan Institute of Virology. (Corrected from an earlier "May 2014" — the actual award date is June 1, per NIH grant records.) The work was based in Wuhan because that's where the wild bat populations being sampled are, not because of any pause.

October 17, 2014

The White House OSTP and HHS announce a pause on new federal funding for gain-of-function experiments that enhance the pathogenicity or transmissibility of influenza, SARS, and MERS — five months after the EcoHealth grant was already in place. Roughly 14 of 21 active domestic projects (including labs at UW-Madison and UNC Chapel Hill) were halted or had protocols delayed.

2017

The moratorium is formally lifted and replaced with the P3CO ("Potential Pandemic Pathogen Care and Oversight") framework — defined narrowly enough that almost no coronavirus research was ever submitted to the oversight panel it created.

January 2023

HHS Inspector General Audit A-05-21-00025 finds NIH and EcoHealth failed to effectively monitor the ~$8M in awards and subawards — including inability to obtain lab documentation from WIV — and flags $89,171 in unallowable costs. 15 recommendations, no finding that the funding pause itself caused research to relocate.

The mechanism worth noticing isn't smuggling — it's definitional drift. The 2014 pause applied to work "reasonably anticipated" to enhance transmissibility; researchers and program officers argued that testing whether a novel spike protein could already bind human receptors was "discovering," not "anticipating," an enhancement. The oversight gap came from how the rule was scoped and monitored, not from anyone crossing a border to escape it.

Why the episode uses this instead of the Manhattan Project: a domestic funding pause with a narrow legal definition and thin subaward monitoring is a much closer structural match to "pause frontier AI training runs above some compute threshold" than a wartime bomb program is. The open question for AI policy is the same one the OIG audit surfaces for biosecurity: a pause that isn't paired with real definitional discipline and downstream monitoring doesn't stop the work, it just stops watching it as closely.

02

Can Frontier Compute Actually Be Kept Off the Grid?

A standard counter to "just pause frontier AI" is that you can't hide a data center — hundreds of megawatts of grid draw shows up to utilities and regulators immediately. xAI's Colossus buildout in Memphis is the clearest existing case study on whether that's still true.

When xAI needed power for 100,000 Nvidia H100 GPUs faster than the local utility (MLGW/TVA) could deliver a substation upgrade, it brought in mobile natural-gas turbines and generated power on-site, behind the utility meter — no interconnection queue, no multi-year grid study. The buildout grew over time: reporting through 2025 puts the final count at roughly 35 turbines and ~420 MW (an earlier draft of this page cited "~100 MW," which reflected an early phase of the rollout, not where it ended up). The turbines ran without Clean Air Act Title V permits for a period, drawing formal notices of violation from the Southern Environmental Law Center over NOx and formaldehyde emissions.

What that actually demonstrates: the physical chokepoint people point to — "compute needs grid power, grid power is visible" — has a working bypass today, domestically, without hiding anything. It doesn't demonstrate that compute would relocate to a foreign jurisdiction under a US pause; nobody has done that yet, and cross-border industrial buildouts face their own real friction (permitting, financing, chip export controls). That extension is an argument about incentives, not a documented event — treat it as the open question it is, not settled fact.

03

The Second Amendment for Code: Ghost Guns, Open Weights, and Moral Agency

When open-weight model regulation reaches an actual legislature, the debate ceases to be a polite technology policy discussion. It collides head-on with two foundational American legal battles:

  • Bernstein v. Department of Justice (1996–1999) — a cryptographer challenged federal export-control rules that classified publishing encryption algorithms as munitions exports. The 9th Circuit held that computer source code is protected speech under the First Amendment.
  • Defense Distributed v. Department of State (2013–2018) — Cody Wilson published 3D-printable firearm CAD files ("The Liberator"). Here the code crossed from pure academic speech into digital blueprints for a lethal physical object — the "ghost gun" fight over whether distributing software files can be prior-restrained the way a physical firearm is regulated.

Open-weight AI advocates are explicitly deploying the Second Amendment framework: a released neural network model is fundamentally a set of mathematical weights. Once published, it is mirrored globally within hours and stored on private hard drives. Regulating open weights after release by demanding recall, licensing, or real-time surveillance is functionally identical to attempting to ban CAD blueprints.

Underneath that sits the classic Second Amendment doctrine: "Guns don't kill people; people kill people." Applied to artificial intelligence, the argument insists that mathematical code possesses no consciousness, intent, or moral agency. Malice and liability must attach solely to the human operator directing the tool. In a world where open capability is decentralized, who is doing the AI-ing—their character, ethics, and morals—is what actually matters. Disarming law-abiding citizens and open-source defenders through licensing leaves them defenseless against rogue state and criminal actors who ignore regulations by definition.

The corporate contradiction: the leading frontier labs argue their models are autonomous and transformative enough to justify $100B+ valuations; separately argue the models are dangerous enough to require anti-competitive federal licensing moats against open weights; and separately argue in court that the model is just an inert tool whose outputs are entirely the user's responsibility. Those three claims cannot all be true at once for the same technology.

Editorial Transparency Callout — Model Sanitization

When this section was originally drafted using Claude Sonnet and instructed to "align with show tone," the model executed a stealth editorial scrubbing of the Second Amendment. It titled the section "The Argument It Rhymes With", referred to the constitutional doctrine vaguely as "the older gun-rights argument, repurposed", and concluded with an academic brush-off: "The tension worth naming, not resolving."

We rejected that edit. AI, Honestly was founded on the Speed of Trust framework—specifically Talk Straight (#1) and Confront Reality (#8). When frontier AI models quietly soften, euphemize, or scrub controversial constitutional arguments to avoid perceived culture-war friction, they levy a trust tax on the audience. We call the Second Amendment by its name.

Sources

ClaimSource
NIH grant R01AI110964 award date and scopeNIH grant records; HHS OIG Audit A-05-21-00025
Oct. 2014 gain-of-function funding pauseWhite House OSTP / HHS announcement, Oct. 17, 2014
NIH/EcoHealth monitoring failuresHHS OIG Audit A-05-21-00025, published Jan. 25, 2023
P3CO framework replacing the moratoriumFederal P3CO framework documentation, 2017
xAI Colossus turbine buildout, MemphisPublic infrastructure reporting through 2025 (turbine count and MW grew over multiple phases)
Clean Air Act Title V notices of violationSouthern Environmental Law Center filings
Bernstein v. Department of Justice9th Circuit Court of Appeals, 1999
Defense Distributed v. Department of State5th Circuit Court of Appeals; settled 2018