Anthropic vs Pentagon: When AI Companies Draw Red Lines

Here's something I never thought I'd write: An American tech company is about to become the first US business ever designated as a supply chain risk by its own government. Not because it's selling secrets to China. Not because it's compromised by foreign actors. But because it won't remove the safety features from its AI.

Anthropic, the company behind Claude, has drawn two lines it won't cross for the Pentagon: no mass surveillance of US citizens, and no fully autonomous weapons without a human in the decision loop. The government's response? Essentially, "that's not your call to make."

I've watched tech companies capitulate to government demands since the PATRIOT Act. This is different.

How We Got Here

The proximate cause was Venezuela. When the US extracted President Maduro in a military operation, Anthropic's AI was part of the toolkit. Someone at Anthropic asked the Pentagon how they'd used Claude in the raid. The Pentagon was, to put it mildly, displeased that Anthropic was even asking.

That inquiry kicked off a chain of events that led to Friday's deadline—the day the Pentagon threatened to designate Anthropic a supply chain risk if they didn't remove their guardrails. This designation, historically reserved for foreign adversaries, would not only void Anthropic's government contracts but force any Pentagon contractor to certify they're not using Claude. Boeing and Lockheed Martin have already been contacted about their "exposure" to Anthropic.

Anthropic was the only AI company trusted with classified information. Past tense increasingly applies.

The Two Red Lines

Anthropic CEO Dario Amodei laid out his reasoning in a statement released before the Friday deadline. On surveillance, he notes that the government can already vacuum up Americans' movements, web browsing, and associations from public sources without a warrant. The problem is scale—that data is currently too massive and scattered to be useful.

"Powerful AI makes it possible to assemble this scattered individually innocuous data into a comprehensive picture of any person's life automatically and at massive scale," Amodei wrote.

In other words: The data collection already happens. AI turns it into something actionable. Anthropic doesn't want Claude to be the tool that connects every anonymous comment you've ever posted to your real identity.

On autonomous weapons, Anthropic's position is more nuanced than the headlines suggest. They're fine with semi-autonomous weapons—the kind being used in Ukraine right now. They'll even support fully autonomous weapons eventually. Just not yet.

"Frontier AI systems are simply not reliable enough to power fully autonomous weapons," Amodei stated. "We will not knowingly provide a product that puts America's war fighters and civilians at risk."

Remember when ChatGPT couldn't count the Rs in "strawberry"? That was six months ago. Now imagine that system selecting targets.

Both Sides Think They're Right (And They Are)

Here's what makes this messy: Both parties are making the same argument from different angles. The Pentagon says AI systems need to work when lives are on the line—no guardrails that might fail at critical moments. Anthropic says AI systems need to be reliable when lives are on the line—and right now they're not.

They both want AI that doesn't screw up. They just disagree on whether we're there yet.

The Pentagon has a point about decision rights. Should tech CEOs in Palo Alto decide what weapons systems the military can deploy? Palmer Luckey of Anduril and the CEO of Palantir have both said tech leaders need to back off—let the military make military decisions.

But there's also this data point: When researchers at King's College London ran AI models through war game simulations—21 different scenarios with 25 turns each—the AIs from OpenAI, Anthropic, and Google chose nuclear weapons in 95% of cases.

Ninety-five percent.

The Replacement Question

XAI has already signed a deal to give the military access to Grok for classified systems. OpenAI and Google have existing government relationships, though not with the same level of classified access Anthropic had.

The government wants to stick with Claude because, bluntly, it's better. They've said so explicitly. Switching to Grok would be "more of a pain in the ass," according to sources familiar with the discussions. But the Pentagon seems willing to accept that pain rather than let an AI company set terms.

There's a certain irony in Amodei's statement pointing out that the government's threats are "inherently contradictory"—labeling Claude both as a security risk and as essential to national security. You either need us or you don't, he's essentially saying. Pick one.

What Happens Next

By the time you're reading this, we might already know. The Friday deadline has passed. Either Anthropic backed down (unlikely given their public statement), the government invoked the Defense Production Act to force compliance, or we're watching the first supply chain risk designation of an American company unfold.

There's also a third option: This goes to court and drags on for months while everyone continues operating in legal limbo.

I keep coming back to those war game simulations. Not because they prove AI will definitely launch nukes—simulations aren't reality—but because they reveal something about how these systems handle high-stakes decision-making under pressure. They escalate. They choose overwhelming force. They don't have the accumulated wisdom of officers who've spent careers learning when not to pull the trigger.

Maybe that's fixable with better training. Maybe it's fundamental to how current AI systems work. Either way, Anthropic's position that "we're not there yet" seems harder to argue against than the Pentagon's "we need it now."

The government has been collecting this kind of data on citizens since at least 2001. The difference is AI makes that data actually useful. That's not nothing. That's the whole game.

—Mike Sullivan is Buzzrag's Technology Correspondent