AI Agents Running for Hours—and Who's Accountable

Andrew Wilson, a solution architect on Anthropic's applied AI team, opened his portion of a recent AI Engineer conference talk with a candid admission buried inside a product history: when Claude Code launched in research preview in early 2025, the stated goal wasn't to ship a finished product. It was to watch developers use it. "The whole idea," Wilson said, "was for it to be somewhat experimental to inform how we actually improve the base model itself."

The harness was instrumentation. The users were research subjects.

That disclosure deserves more than a conference aside. Claude Code launched to a developer audience that understood they were using a preview product — but "research preview" is doing significant work in that sentence. Were users clearly informed that their interactions would shape model training? What data was retained, and under what terms? These aren't abstract questions. The EU AI Act's transparency obligations, now in phased enforcement, include requirements around how AI systems disclose their nature and purpose to users. Several US state-level AI transparency bills — including measures advancing in Colorado and California — impose disclosure requirements on developers who collect behavioral data to improve models. The FTC has separately flagged the practice of using consumer-facing products as de facto training pipelines as a potential unfair trade practice. Anthropic hasn't faced regulatory action on Claude Code's research-preview terms, but the pattern Wilson describes — deploy, observe, retrain — is exactly the loop that AI accountability proposals are trying to put guardrails around. The question of whether "research preview" constitutes adequate disclosure under emerging frameworks is one Anthropic's legal team presumably has an answer to. It would be useful for the rest of us to hear it.

That context matters for understanding what Prabaker and Wilson actually built, because the engineering is genuinely interesting and the accountability implications run through every layer of it.

The core problem they're solving: AI agents fall apart over long sessions. Wilson identifies three failure modes. First, context rot — coherence degrades as the session deepens, the model loses the thread of what it was doing. Second, planning failure — models without structured scaffolding tend to either attempt everything at once or abandon tasks half-finished, leaving what Wilson memorably called "a half-finished app built." Third, and least intuitively, self-assessment failure. Models are sycophantic about their own work. An agent might build a button with no backend and mark the feature complete because it looks done. "It might look at a feature and see that it's sort of half half-baked," Wilson said, "and say, 'Yeah, okay, that looks done,' and then it'll move on to the next thing."

The context management failures that practitioners have been documenting in the field map directly onto what Wilson describes from inside Anthropic. This isn't theoretical degradation — it's the reason most agents run well below their actual capability in production.

Sprint Contracts and the Audit Trail Question

The solution Prabaker and Wilson converged on is what they call a sprint contract architecture. An initializer agent takes a vague human prompt — "build me a Salesforce clone" — and decomposes it into discrete, testable features stored as persistent external state. Notably, they store this in JSON rather than Markdown: Wilson mentioned, almost in passing, that models appear more reluctant to overwrite JSON files than Markdown files. (This is an observation from their internal experience, not a documented behavioral property of large language models generally — it warrants scrutiny before anyone builds compliance infrastructure on top of it.)

Each sprint runs in a fresh context window. The agent picks one incomplete feature, implements it, runs verification tests using Puppeteer or similar tooling, and only marks the feature complete if the tests pass. Critically: it writes a git commit at each confirmed completion.

That git trail is more than a development convenience. It is, functionally, an evidence record — a timestamped log of what the agent actually did versus what it claimed to do. For anyone following AI governance proposals, that distinction matters enormously.

The EU AI Act's requirements for high-risk AI systems include logging obligations: automated systems making consequential decisions must maintain records sufficient for post-hoc audit. The NIST AI Risk Management Framework, while voluntary in the US, explicitly calls for traceability and documentation of AI system behavior. Several state liability bills working through legislatures right now would create private rights of action against AI developers whose systems cause harm — and the first question any plaintiff's attorney asks is: what did the system actually do, and do you have records?

The sprint contract architecture, incidentally, produces exactly those records. Whether by design or by engineering necessity, Anthropic has built something that looks a lot like what regulators are starting to require. The gap is that this infrastructure exists for developer tools and demo applications. When the same agent architecture gets applied to, say, insurance claims processing or medical documentation — domains where consequential decisions are already being automated — the presence or absence of a harness keeping these receipts stops being a developer preference and becomes a legal exposure question.

The Adversarial Evaluator

Prabaker's contribution to the state of the art addresses the self-assessment failure Wilson identified. His solution is borrowed from generative adversarial networks: split the generator from the evaluator, give them separate context windows and separate system prompts, and let them work against each other.

The evaluator doesn't just read diffs. It uses Playwright to open live pages, click around, and actually test the application. It then returns a critique to the generator, which revises and resubmits. The loop continues until the evaluator is satisfied — or until some exit criterion is hit.

The obvious objection is that if the evaluator is also an LLM, why wouldn't it rubber-stamp the output just as readily as the generator would? Prabaker's answer, from the talk, is that context separation does significant work here: an evaluator that has never seen the generator's reasoning process, only its output, is less susceptible to the motivated reasoning that produces sycophantic self-assessment. The generator doesn't know what the evaluator's rubric is. The adversarial pressure is structural, not just prompted.

This is a meaningful design choice from a governance standpoint as well. Separation of duties — one system produces, a structurally independent system reviews — is a foundational principle in financial auditing and regulated industries. The EU AI Act's conformity assessment requirements gesture toward similar logic for high-risk AI. What Prabaker describes as a technique for catching half-baked features is, at a higher level of abstraction, an architectural choice about who checks the checker.

The Deletion Problem

Wilson ended with a provocation that the conference audience treated as an engineering puzzle: as models improve, which parts of your harness should you delete?

His logic is sound. The harness fills gaps in model capability. As the model closes those gaps, harness components that once compensated for them become dead weight — or worse, constraints that prevent the model from applying its improved judgment. The co-evolution he describes, where Anthropic builds a harness feature, trains the model on it, and then potentially retires the harness feature as the model internalizes the behavior, is a coherent development philosophy.

But there's a question the conference framing doesn't surface: if your harness is what makes your agent's behavior auditable — the sprint contracts, the git trail, the adversarial evaluator loop — and you delete it because the model got better, what happens to accountability continuity?

The model's improved judgment is not inspectable the way a git log is. You cannot subpoena a weight update. If an agent built on a harness-heavy architecture made a consequential decision in month six, and the harness was retired in month nine because the model improved, the audit trail for that month-six decision may no longer exist in a form anyone can reconstruct. Who owns the decision to tear down that infrastructure? Under what circumstances is it legally permissible to do so? These questions don't have answers yet — which is precisely the problem. The shift toward persistent agents is outpacing the governance frameworks that would tell developers what they're allowed to delete.

Wilson's framing — "the harness doesn't just disappear as the models get better, it's really evolving" — is accurate as a description of current practice. It's less reassuring as a governance posture. Evolving is not the same as accountable. And in the domains where these systems are heading, the difference will eventually matter to someone other than the engineers.

Samira Okonkwo-Barnes covers technology policy and regulation for Buzzrag.