Brad Carson: AI Surveillance Dossiers Are Already

Here's the sentence that should be keeping you up at night: using AI to compile personal surveillance dossiers on U.S. residents is currently lawful.

Not a hypothetical. Not a future risk. Now. Legal. Today.

That's not a fringe claim from a civil liberties activist. It comes from Brad Carson — former two-term Congressman, former Under Secretary of the Army, and former Acting Under Secretary of Defense for Personnel and Readiness — in a recent appearance on Machine Learning Street Talk. Carson heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded, and he spent 80 minutes with host Keith Duggar making a case that cuts against the dominant fatalism in the AI conversation.

His core argument: the genie is not out of the bottle. We can still shape this. But the surveillance piece is where that argument lands hardest for anyone living with these systems — which is all of us.

The accountability vacuum

Carson's framing on surveillance is blunt. LLMs, he told Duggar, "can really lubricate" the creation of personal dossiers on individuals — aggregating records, synthesizing profiles, identifying patterns across datasets. And the law hasn't caught up. "Congress should step in and say that's not lawful," Carson said. "Most members of Congress don't believe that's lawful. Same thing on lethal autonomy."

That gap — between what Congress believes and what Congress has actually prohibited — is the practical stakes of this conversation. The Anthropic-Pentagon standoff that made headlines wasn't really about one company's ethics clause. It was about the absence of a legal boundary. When OpenAI and Google stepped in after Anthropic balked, their DoD contracts included language authorizing "all lawful uses." Carson's read on that phrase: "That means they're going to do all the things I said I don't really love personally. They're all lawful. That's the problem."

The AI companies can't fix this through product decisions. Anthropic walking away doesn't make domestic AI surveillance unlawful — it just means the Pentagon finds a different vendor. The fix lives in Congress, not in San Francisco.

"I can't court-martial Palantir"

The surveillance problem has a military cousin, and it's uglier. Carson spent time at the Pentagon overseeing law-of-war policy, which gives him an unusually grounded view of what probabilistic AI targeting actually means in practice.

The pre-neural-net model of war had a fiction embedded in it too — humans making "definitive" calls that were, in reality, uncertain. But those humans could be held accountable. You could court-martial a bad actor. You could interrogate the reasoning chain. The system was slow, fallible, and sometimes catastrophically wrong, but it had a throat to grab.

The Palantir Foundry model gives a commander a number. Someone in Gaza scores 0.73 on a combatant probability score. "What is 73?" Carson asked. "Do you get struck for that, or are you off the list? What's the threshold?" Nobody can answer that, including the people who built the system. The neural net isn't just opaque to outsiders — it's opaque to its own developers. Mechanistic interpretability researchers like Neel Nanda are doing serious work, but as Duggar noted, we're still largely using mathematical models to explain mathematical models. The reasoning isn't recoverable.

The consequence Carson names is structural: "I can't court-martial Palantir, the Foundry model. I can't do that. And that's just a radical change in the way war is being fought and not for the good."

What's new isn't that war produces errors. It's that the system now pre-quantifies acceptable error rates. False positives get built into the operational calculus. That shift — from "we made a mistake" to "we knew the model would be wrong X percent of the time" — is a moral architecture change, not just a technical one. Research on automation bias supports Carson's concern that "human in the loop" oversight often means little in practice: when a system outputs a confidence score, humans tend to defer to it. Carson put it more directly, calling meaningful human oversight in these contexts "a legal fiction" that is "operationally vacuous." That characterization reflects a real and documented tendency, even if researchers continue to debate the scope and conditions under which it applies.

The fatalism problem

Carson's counter to "AI is inevitable" is the part of this conversation that most AI-adjacent commentary dismisses too quickly. His historical examples are selective — he cites the 1975 Asilomar conference on recombinant DNA as evidence that scientific communities can voluntarily restrain dangerous research, though the moratorium there was partial, lasted roughly a year, and was followed by guidelines that allowed most research to resume. It's less a clean precedent than a proof of concept: restraint is achievable, even when it's imperfect and temporary. His stronger point is the chip leverage argument. The West — mostly the U.S., with Japan and the Netherlands holding critical nodes — controls the semiconductor supply chain that makes frontier AI possible. "Unless you can recreate Nvidia and ASML and Japanese photoresist companies," Carson argued, China cannot simply route around that constraint. "We could choose to do this."

I don't know if Carson is right that we'd choose to exercise that leverage, or that the political will exists to hold it. But the structural claim is real: we have more control over the trajectory of this technology than the "freight train" metaphor suggests. The fatalism isn't a factual description of the situation. It's a political choice dressed up as physics.

The regulatory capture trap

Carson also has a pointed take on the "regulatory capture" argument that tech-adjacent libertarians deploy against any proposed AI oversight. His response: "It's searching for a pea under 100 mattresses. It's never falsifiable." His actual target is the informal capture that's already happening — the a16z-style influence networks that shape AI policy without the accountability of a formal agency. "Having an agency subject to regulatory capture is at least more accountable than the kind of informal, very moneyed networks that are controlling AI policy as we sit here in 2026."

This is a genuine tension in the regulatory debate, not a settled question. But Carson's framing reorients it usefully: the alternative to imperfect formal regulation isn't clean governance, it's governance by whoever has the most informal access. For people who care about who controls AI systems and on whose terms, that should matter.

What Congress actually needs to do

On upskilling government, Carson noted that incoming members of Congress report having about 17 minutes per day to read and get smarter on issues. That's the institution being asked to set the legal boundaries for the most consequential technology in a generation.

He advocates for mandatory testing and evaluation of frontier models — not through a bloated agency, but through something like the public-company accounting model: private-sector verification organizations operating under regulatory oversight, designed to prevent the Enron-style failure modes. The goal isn't maximizing government footprint. It's creating accountability structures that don't currently exist.

The surveillance dossier question and the lethal autonomy question have the same answer: Congress has to define what's lawful. Until they do, "all lawful uses" is a blank check, and the AI companies — whatever their stated ethics — are operating in a space where the legal floor is the only constraint.

Carson is optimistic that Congress can do this. I find the 17-minutes-a-day data point harder to shake.

Rachel "Rach" Kovacs covers cybersecurity and privacy for Buzzrag.