Adam Becker on Why Silicon Valley's Big Bets Will Fail

Ray Kurzweil has a date circled on his calendar: 2045. That's when the singularity arrives — when artificial superintelligence emerges, accelerates beyond our comprehension, and solves every problem humanity has ever had. Jeff Bezos needs us in space within a few centuries or energy-hungry civilization boils the oceans and hits a wall. Eliezer Yudkowsky thinks we may already be zero steps away from an intelligence explosion that ends the world.

Astrophysicist and science writer Adam Becker has spent years at Bay Area parties listening to versions of these arguments. His new book, More Everything Forever (2025), is his attempt to do what the boosters skip: check the physics, the philosophy, and the history against the actual claims. On a recent episode of Machine Learning Street Talk, he walked through the whole architecture — and found it resting on foundations that don't hold.

The lily pad problem

Kurzweil's "law of accelerating returns" is the load-bearing wall of the entire techno-utopian worldview. The argument goes: Moore's Law was just one instance of a deeper pattern — an exponential trend in complexity and intelligence that stretches from the Big Bang through the emergence of life to our current technological moment, pointing to a near-future rupture where everything changes at once.

Becker's objection isn't that exponential growth never happens. It's that everyone knows how exponential growth ends. "The one thing that you can always say about any exponential trend, the one thing that's always true, is that they end," he told host Tim Scarfe. Kurzweil uses the lily pad example himself — pads doubling daily, pond half-covered on day 29, fully covered on day 30. Becker's response: "And then on day 31 the lily pads don't grow anymore because they filled the pond. There's no more pond."

Gordon Moore said as much about his own law — transistors can't shrink below the size of silicon atoms. Kurzweil's counter is that the overall exponential is made of stacked S-curves, each technology handing off to the next. Becker grants the pattern has happened before, but finds no reason to assume it must continue indefinitely, and notes that Kurzweil's own data ignores large stretches of computing history where no such handoff occurred.

There's a subtler problem too: Kurzweil's data points are cherry-picked. History, Becker notes, tends to look logarithmic to whoever's living through it — the recent past is clear, the distant past is compressed. What Kurzweil mistakes for an exponential trend in the record may just be what the skyscraper view of time always looks like.

The Bezos arithmetic

According to Becker, Bezos argues that humanity must expand into space because, if current energy growth rates continue, civilization will eventually consume all the energy Earth receives from the sun — within a few centuries. That part of the math is roughly right, Becker concedes.

What Bezos leaves out, Becker says: at that consumption level, waste heat alone would be boiling the oceans. And if you spot Bezos a faster-than-light drive for free — something physics firmly rules out — and he keeps compounding energy usage in space at the same rate, the entire observable universe's available energy is exhausted in under 4,000 years. Less time than separates us from the Great Pyramid of Giza.

The argument for space colonization doesn't solve the exponential problem. It relocates it.

Mars itself is no refuge. Becker runs through the particulars with a physicist's specificity: radiation levels too high, gravity too low, no breathable air, soil laced with perchlorates. Elon Musk's own line — "it'd be cool to die on Mars, just not on impact" — prompts Becker to note that dying on Mars is, in fact, remarkably easy to arrange. He's blunter about the comparative: the day the dinosaur-killing asteroid struck Earth 66 million years ago was still a better day for life than any day on Mars in the past several billion years. Mammals survived the asteroid. No mammal could survive unprotected on Mars's surface.

The speed of light closes the door on anything beyond the solar system. We can't exceed it. Particle accelerators have tested this repeatedly. Getting a spacecraft to any meaningful fraction of it would require shielding from radiation that remains an unsolved engineering problem at enormous scale. Space, Becker concludes, is not where humanity is going.

Pocket calculators for language

On AI, Becker reaches for an analogy that rewards thinking about rather than dismissing: LLMs are pocket calculators for language.

Calculators automated something we once considered an irreducibly human cognitive skill — mathematical computation. That was real, significant, and genuinely transformative for what mathematicians could accomplish. But nobody confused a calculator with a thinking mind. LLMs have done something analogous with language — automated a process we associated with thought, and done it well enough to be genuinely useful. The confusion arises because language, unlike arithmetic, feels like thinking. We're wired to read minds into things that talk.

Becker's sharpest point here isn't about what LLMs can't do. It's about what "hallucination" actually is. The word implies that when a model confabulates, something has gone wrong — a departure from normal functioning. Becker says that's exactly backwards: "They really only do one thing. And when they're hallucinating, they're doing the same thing that they're doing when they get it right." The mechanism doesn't switch modes. There's no error state, only output. That's a more unsettling observation than the standard "AI makes stuff up" complaint, because it means the failure condition is structurally indistinguishable from the success condition — and as these systems become more capable, that gets harder to detect, not easier.

He's not predicting that LLMs are a dead end. He's saying that absent a breakthrough that makes current architectures look as primitive as early chatbots do today, human supervision isn't a temporary inconvenience to be engineered away. It's the permanent cost of using tools that don't know when they're wrong.

The doomers who aren't grifters

This is where Becker's argument gets genuinely interesting — and where I find myself wanting to push on it rather than just report it.

He is careful — maybe too careful — not to call any of this a conspiracy. Yudkowsky believes what he's saying, Becker insists. So does Nick Bostrom. So do many of the researchers at Anthropic who have staked their careers on the proposition that AI alignment is the defining challenge of our time. They are, in Becker's framing, true believers who happen to be wrong.

But Becker also maps the structural condition in which these true believers operate: a venture capital ecosystem that needs a story about perpetual growth, run by people who grew up on the same science fiction as the rationalists and effective altruists, and who find the narrative of world-altering AI — whether utopian or apocalyptic — perfectly legible, because both versions center on AI as the most important thing happening. An AI that could save the world or end it is an AI worth pouring billions into. The doomer story and the booster story, told by people who genuinely disagree about the outcome, are both stories about the same investment.

What do I make of this? The framing is more interesting than a conspiracy would be, because it requires no coordination to operate. A researcher who sincerely believes misaligned AI could kill everyone is, without knowing it, sustaining a narrative that makes frontier AI labs fundable at scale. That's a strange and uncomfortable position to be in — useful to capital without intending to be, precisely because the sincerity is legible and the beliefs are dramatic.

The place where I'd push back on Becker: the true-believer defense runs in both directions. If the doomers can be sincere and still wrong, so can Becker. The question of whether current AI trajectories pose serious risks to employment, to epistemic infrastructure, or to power concentration doesn't resolve neatly just because the extinction scenarios are poorly supported. Becker's broader argument — take social science seriously, regulate the industry aggressively, tax the billionaires — addresses real problems. But it relies on political and regulatory institutions that are themselves under pressure from the same forces he's describing. "The Trump administration isn't going to be forever" is hopeful. It is not a plan.

What the physics leaves out

The rationalists and effective altruists tend to ignore political science, sociology, and anthropology, Becker says. They treat intelligence as a single-valued, scalable quantity — more compute equals more intelligence — an idea Becker traces to a history rooted in eugenics and IQ testing, frameworks that the scientific establishment has largely rejected. The AI field, he argues, is still downstream of that history in ways that don't get enough scrutiny.

Longtermism compounds the problem. Once you're doing utilitarian arithmetic over hypothetical quadrillions of future digital humans spread across the observable universe, the people alive now — with their concrete, soluble, embarrassingly mundane problems — become rounding errors. Climate change becomes a distraction from alignment. Poverty becomes a footnote to existential risk. The abstraction runs so far from the particular that it loses any grip on what it was supposed to be about.

Becker's own position is not that technology doesn't matter. He's clear that the technologies needed to address the climate crisis largely exist already. The problem is deployment, political will, and the social mechanisms for getting them adopted at scale — none of which yield to engineering solutions, and none of which get simpler by waiting for superintelligence to arrive.

The severest version of his critique is also the simplest: every exponential ends. The lily pads fill the pond. Moore's law runs into silicon atoms. The universe runs out of energy. Any worldview built on the premise that growth continues indefinitely is not a plan. It's a wish — and an expensive one, when the people doing the wishing control enough capital to make their wishes everyone else's problem.

Whether the bubble pops before or after the next wave of capability improvements, and what the wreckage looks like either way — that's the question Becker can't answer, and neither can anyone else. What he can do, and does, is insist that the physics was never on their side to begin with.

---

— Bob Reynolds, Senior Technology Correspondent, BuzzRAG