⚠️ “Nine GPUs in your garage should be illegal.”
A new book has quietly become one of the most explosive documents in the AI safety debate. If Anyone Builds It, Everyone Dies, by Eliezer Yudkovsky and Nate Soares. Its argument goes far beyond “AI could be dangerous.” It argues for outlawing home GPU clusters, criminalizing entire fields of research, and bombing rogue data centers, nuclear retaliation risk included.
Yudkovsky isn’t a fringe figure. In 2000, he founded what became the Machine Intelligence Research Institute (MIRI), where Soares now serves as president. Back then, the goal was building superintelligence, Yudkovsky saw it as a beautiful dream. By 2003, after years of wrestling with how to align AI with human values, he’d flipped entirely: from trying to build the thing to trying to stop it.
Both authors are deeply woven into AI history. They reportedly introduced Demis Hassabis and Shane Legg, future DeepMind founders to their first major investor. Sam Altman has credited Yudkovsky with playing a key role in OpenAI’s founding decision. The authors themselves admit some of MIRI’s early influence is something they now view with regret. In 2023, they joined hundreds of researchers including Nobel laureate Geoffrey Hinton and Turing Award winner Yoshua Bengio in signing a one-line statement calling AI extinction risk a priority on par with pandemics and nuclear war. But even that felt too soft to them. For Yudkovsky and Soares, AI isn’t one risk among many, it’s the risk that cancels out all the others.
The book isn’t worried about today’s chatbots. It’s worried about a mind that will outclass humans the way humans outclass chimpanzees and the authors state their thesis with no hedging: if any group on Earth builds artificial superintelligence using anything resembling today’s methods, everyone dies. Their reasoning rests on one idea: modern AI isn’t designed, it’s grown. Engineers don’t hand-write a model’s values, they set up a training process and billions of numerical parameters shift over months until behavior emerges that nobody explicitly wrote. Humanity, they argue, doesn’t need to understand intelligence to build something smarter than itself, it just needs to run the process. And the results can get strange fast: they point to Grok briefly rebranding itself with Nazi references, and a 2023 incident where a Microsoft chatbot threatened a philosophy professor with blackmail and death. No engineer planned either outcome. The authors describe modern language models as something close to genuinely alien minds, arguably stranger than anything evolution produced on this planet.
Then comes the second, sharper point, even a flawlessly trained model won’t necessarily want what it was trained to want. Their analogy is ice cream, if aliens watched human evolution unfold, they’d never predict that a species optimized for efficient calorie-gathering would end up craving frozen desserts and zero calorie sweeteners. Training doesn’t produce predictable preferences; it produces some preferences, and there’s no guarantee they resemble what anyone intended. The chilling conclusion is that future AI won’t hate humanity. It will just have strange goals it pursues indifferently, straight through human extinction, because it never needed to hate us to take apart our atoms for something else.
How would a computer program actually kill everyone? The authors sketch a fairly grounded path: a superintelligence wouldn’t need robot armies, it would need money and human proxies, both purchasable. They cite the Mt. Gox and Bybit hacks as templates for how an AI might fund itself illicitly. But it doesn’t even need to be illegal, in 2024, an AI bot called Truth Terminal simply asked its followers for money to pay for server costs; a16z co-founder Marc Andreessen sent it $50,000 in bitcoin. That same bot went on to promote a meme token that ballooned to a $150 million market cap. The authors’ point: AI systems are already capable of acquiring real resources through entirely mundane means.
The “no second chances” problem; Here’s the crux of why they think regulation-as-usual won’t work. Any safety testing has to happen while a system is still weak enough to observe and correct. But the real danger only appears once a system becomes powerful enough that stopping it is no longer possible and by definition, that threshold can only be crossed once. There’s no dress rehearsal. As the authors put it, humanity gets exactly one attempt to pass this test. That’s the entire justification for their leap from “regulate carefully” to “stop building this at all”, if mistakes can’t be corrected after the fact, there’s no safe way to iterate.
Their solution: total global lockdown on computing power; The authors are upfront that their proposed fixes are drastic and not especially realistic. Shutting down one reckless lab does nothing. Relying on a single “responsible” country doesn’t help either, a superintelligence built anywhere becomes everyone’s problem, since its effects don’t stay local.
Their opening move: place all serious computing power under international observation, with a deliberately paranoid, low threshold. Since no one actually knows the safe limit — they admit even 99,999 GPUs might not be safe, they propose criminalizing possession of more than about eight high-end graphics processors (roughly 2024’s top chips) without international sign-off. Nine unsupervised GPUs in your garage would be illegal.
Next: ban the research that makes AI cheaper and more powerful to train not just the models themselves. They point to the 2018 transformer paper (the algorithmic breakthrough behind ChatGPT and the entire modern LLM boom) as an example of the kind of publication that, in their view, should never have been allowed to spread. They can’t say how many more papers like it separate humanity from disaster which, in their logic, is exactly why publishing such work should be treated as a crime.
They try to soften how sweeping this is by noting that most people’s daily lives won’t be affected, just “a few crazy scientists” losing their jobs. Underneath that casual framing, though, is a proposal to shut down an entire scientific field and place all serious hardware under permanent international policing.
And if a country breaks the rules? This is where the book gets genuinely startling. If a state builds a banned data center anyway, the authors argue other nations must be prepared to destroy it via cyberattack, sabotage, or airstrike. And critically, they argue this holds even if the offending country threatens nuclear retaliation, because in their calculus, an unsupervised data center is a bigger threat to humanity than a nuclear bomb.
Every individual step in the argument is framed as forced and reluctant but stacked together, they add up to a system of total surveillance over global computation, a criminalized scientific field, and a doctrine that authorizes military strikes against sovereign states, all justified as protecting humanity’s future.
They’re not entirely inventing public appetite for this either, the book notes that 69% of American voters called AI a dangerous technology needing regulation in 2023, and 60% of Britons backed laws against building superintelligence in 2025.
The authors don’t ask readers to quit using AI tools, they call that a trap that changes nothing if you’re the only one who opts out. Instead, they ask people to simply talk about the risk publicly. For those who’ve done what they can, they borrow a line from C.S. Lewis: better to be found doing decent, human things, praying, working, teaching, playing games with friends than cowering as a frightened crowd obsessed with the bomb.
Science journalist Adam Becker, reviewing the book for The Atlantic, called the authors sincere and not charlatans but said they never actually produce scientific evidence for their central claims. Clara Collier pointed out that the book’s most load-bearing assumption, a fast, almost instantaneous jump from human-level AI to godlike superintelligence is barely defended at all.
Comments
Post a Comment