AI Agents Still Can't Do Open-Ended Research

A lot of AI hype runs on a specific promise: that at some point soon, AI agents will be able to do AI research on themselves—reading papers, running experiments, proposing new architectures, iterating—creating a feedback loop of accelerating progress that humans can barely keep up with. It's the engine behind forecasts of explosive AI development. And a new paper is poking at the foundation of that argument.

The paper, published on arXiv (arxiv.org, 2607.27191) and summarized by the CRUX evaluation team at cruxevals.com, presents early evidence from two case studies testing whether frontier AI agents can conduct open-ended AI research. The short version: they can't, at least not yet.

What "Open-Ended" Actually Means Here

This distinction matters a lot and gets glossed over constantly, so let's be precise. "Verifiable" or "narrow" research tasks are things with clear right answers—can the agent reproduce a known result, solve a benchmark, implement a specific algorithm? Agents are actually pretty decent at those. "Open-ended" research is something different: formulate a novel hypothesis, design experiments to test it, recognize when a dead end is a dead end, regroup, try something else. That's the stuff that earns you a conference paper.

As the arXiv paper puts it, current evaluations "either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated" work—but there's been almost no rigorous testing of whether agents can actually drive the open-ended process autonomously. The paper tries to fill that gap with two structured case studies, and the results are—to use the authors' own measured language—not encouraging.

Six Days, Three Grand, Five Ways to Fail

Here's where it gets concrete. According to note.com's writeup of the study, the agents were given up to six days and $3,000 in API budget to work on the research tasks. That's not a trivial resource allocation. And they still failed to reach standards acceptable for academic conference submission.

The failure modes the research team identified are worth sitting with:

• Judging research standards: Agents couldn't reliably assess whether their outputs actually met the bar for publication-quality research.
• Redesigning experimental strategies: When an approach wasn't working, agents struggled to pivot meaningfully rather than just running the same thing again.
• Backtracking from dead ends: Related but distinct—recognizing that a path is a dead end and actually stopping is harder than it sounds when you're an agent with no gut feeling telling you something's off.
• Resource management: The agents burned through computational resources without the kind of cost-benefit reasoning a human researcher would apply.

AI as Normal Technology characterizes the core finding plainly: "Our results suggest that conducting open-ended research remains challenging for frontier AI agents." They're careful to note the findings are tentative—small sample size, limited model testing—but the pattern is consistent enough to take seriously.

The Async Problem Lurking Underneath

There's a technical dimension to this that the surface-level "agents can't research" framing doesn't fully capture. AI Engineer's piece on MCP tasks surfaces a structural challenge: agents operating asynchronously—spinning up subtasks, delegating, waiting on results—run into fundamental coordination problems. What looks like a "research agent" is often a cluster of subagents, and the overhead of managing that cluster creates its own failure modes.

Anthropic's engineering blog (anthropic.com) is candid about this from their own multi-agent research system work: "this asynchronicity adds challenges in result coordination, state consistency, and error propagation across the subagents." In plain terms: when multiple subagents are running in parallel, keeping them coherent—making sure they're not contradicting each other, making sure errors in one don't cascade into others—is genuinely hard. Anthropic's engineers are optimistic that the complexity will be worth it as models scale, but the "we expect the performance gains will justify the complexity" framing is doing a lot of work there.

The Forecasting Stakes

Why does this matter beyond "interesting benchmark news"? Because, as the arXiv paper notes directly, "forecasts of explosive AI progress hinge on AI agents automating AI research." The entire logic of certain AI timelines—that we're approaching a period of recursive self-improvement where AI accelerates its own development—requires that agents can actually do research, not just assist with it.

The paper isn't claiming that will never happen. The authors are careful: preliminary evidence, small sample, more testing needed. But what they're documenting is that as of now, the gap between "agent that can implement known solutions" and "agent that can navigate genuine scientific uncertainty" is real and measurable. Those are different cognitive tasks, and the second one requires something the current generation of agents apparently doesn't have in sufficient quantity: the ability to sit with ambiguity, make judgment calls about research quality, and know when to cut losses.

That's not a knock on the agents for being bad. It's a description of what open-ended research actually requires.

What This Doesn't Mean

It would be too easy to read this as "AI can't do research, humans win forever." That's not what the data says. The same agents that struggled with open-ended research are legitimately useful for structured, verifiable tasks—literature review, code implementation, running predefined experiments at scale. Anthropic's multi-agent system exists and produces useful outputs. The failure modes are specific to the open-ended, autonomous, self-directed flavor of research.

The distinction isn't just academic (no pun intended). It maps directly onto how organizations should be thinking about deploying AI in research contexts. Using an agent to search papers, draft sections, or run ablations on a researcher's design? Probably fine and useful. Expecting an agent to autonomously drive a novel research direction from question to publishable result? The current evidence says: not yet.

What's Still Unknown

The honest acknowledgment in normaltech.ai's analysis is worth repeating: the study has limitations. Two case studies is a thin evidentiary basis. The models tested may not represent the cutting edge by the time you read this—the field moves fast enough that a paper's model selection can be outdated before it clears peer review. The researchers themselves are working on expanding the sample and testing newer models.

So the appropriate reading is: this is early, real evidence that the gap exists, not a definitive proof of how large it is or how long it persists.

The open question isn't whether AI will eventually do open-ended research. It's whether the specific capability gap here—judgment, creative reorientation, knowing when a dead end is a dead end—closes gradually through scaling, or requires something qualitatively different. That's a question the field doesn't have consensus on, and the honest answer is that nobody knows yet.

Which is, somewhat fittingly, exactly the kind of open-ended research problem these agents apparently can't solve on their own.

---

Yuki Okonkwo is Buzzrag's AI & Machine Learning Correspondent.