The Gauntlet Loop Works—Until It Doesn't

The demo that kicked this off was a first-person shooter. Matt Shumer posted it to X: Claude had built it from a single prompt, no existing assets. People were surprised not because AI had written game code before, but because a complete, playable result had come out of one instruction. A few days later, Shumer published the method and gave it a name: the gauntlet loop.

The prompt is three lines. That's worth sitting with for a second, because the gap between "three lines" and "complete playable game" is where most of the interesting questions live.

What the loop actually does

The AI LABS team breaks the mechanics down clearly. Line one defines what you're building and sets a quality target. Line two tells the main agent to decompose the work and hand each piece to a separate sub-agent—each one operating in its own memory, with no view of what the others are doing. Line three sets the benchmark: in Shumer's original, that benchmark was Call of Duty.

That last part is doing more work than it might seem. Each sub-agent has a paired "critic" whose only job is to evaluate the output and send it back if it falls short. The critic starts fresh every time—no memory of prior attempts, no loyalty to whoever built the thing. It just grades. And critically, the critic needs something to grade against. Shumer's solution was to give it a real, shipped product and instruct it to compare the two blindly, picking whichever is better without knowing which one Claude made.

The keyword "ultracode" at the end of the prompt activates what the team calls a dynamic workflow—a fleet of sub-agents running in parallel rather than sequentially. The structure this draws is what AI LABS calls a diamond graph: one task fans out to multiple concurrent agents, then narrows back to a single agent that integrates the results. It's loop engineering at scale, not just a fancier version of asking Claude to try again.

The results have been striking enough to attract serious attention. Andrej Karpathy, one of OpenAI's founding researchers, commented that outputs at this quality level matter precisely because the economics had never made sense before—the work was never worth what it cost to produce. A model sidesteps that constraint.

Where it falls apart

Here's the problem nobody in the hype cycle is flagging: the gauntlet loop has two structural weaknesses, and together they define a hard ceiling on what the method can actually do.

The first is that the main agent writes its own checking instructions. You hand it a reference product and a note to "be a really harsh critic"—everything else, including how it spins up the critics and what judgment criteria it passes them, is the agent's decision. As the AI LABS team puts it: "checking is something you should set more concretely rather than letting the agent verify on its own." When something goes wrong at scale, you won't be able to trace which decision caused it.

The second problem is more fundamental. The quality bar in the original prompt is an existing product. That works for games, landing pages, and 3D environments because there's always something comparable to point to. But most real software isn't a game. If you're building the billing logic your own company runs on, there's no finished app to hand the critic as a reference. So the critic invents a standard, builds against it, and passes work that meets criteria you never agreed to. By the time you see the output, you've spent significant time and compute on features shaped by assumptions you never made.

"The gauntlet loop works when there's something close enough to copy," the team notes, "and it breaks the moment there isn't."

That's not a niche edge case. It's the normal situation for anyone building internal tools, proprietary workflows, or anything genuinely new. The demos that went viral—games, 3D worlds, portfolio sites—happen to sit in the category where the loop works by design.

Wayfinder as the reference anchor

The fix comes from a different direction than you'd expect. Matt Pocock, a developer who has published a set of installable Claude skills, built one called Wayfinder specifically for the planning problem. Its core insight is that AI planning has a fog problem: when an agent doesn't know something, it doesn't stop. It fills the gap with an assumption and continues as though the question had been settled. What you get back looks like a complete plan. Parts of it are invented.

Wayfinder surfaces those invented parts. It works through every undecided question methodically, splitting them into two groups: the ones that can be resolved now because their dependencies are already settled, and the ones still in fog because something upstream hasn't been determined yet. When it hits fog, it doesn't guess—it dispatches to research, to a rough prototype, or to a real-world action needed to form an opinion. Each answer opens up whatever was waiting on it. You work through the map until no fog remains.

The output is a spec: a single document recording every decision made, with the reasoning behind each one. That spec becomes the answer key—the thing the critic checks against, playing the same role Call of Duty played in Shumer's original, but for software that has never existed before.

The AI LABS team didn't use Wayfinder off the shelf. They found the original too exhaustive for their purposes—it's designed to carry a project all the way to a finished spec, generating a separate requirements file for every decision. They wanted only the planning clarity, not the full scaffolding. So they had Claude rewrite the skill to produce just two files: a map of every decision and its reasoning, and an answer key structured as a list of checks that each return pass or fail.

They then ran the gauntlet loop on an HR system their team actually needed. The planning phase involved 34 questions. Once those were answered and the answer key was written, they substituted the .wayfinder folder for Call of Duty as the source of truth, fed the modified gauntlet loop prompt to Claude, and let it run.

The build took one hour and thirty-three minutes. The team reports that every check in the answer key passed and every feature they'd planned was working. They note the design "only came out okay"—a candid admission that functional correctness and visual polish are different problems, and this approach solves the former more reliably than the latter.

On API pricing, they estimated the equivalent cost at around $116 for a build of that size—a figure the team flags as significant, though the actual cost will vary depending on model, session structure, and what "roughly 40% of a session" maps to in a given context.

What this reveals about agentic building

The gauntlet loop is genuinely interesting, and the game demos have shown what's possible when the conditions are right. But the conditions matter. The pattern works when there's a reference artifact the critic can use as ground truth. Strip that away—which is what happens the moment you build something that doesn't already exist—and the loop's self-assessment becomes circular. The agent sets the standard, meets the standard, and reports success.

Wayfinder's contribution is reframing the planning phase as the place where quality gets defined, not assumed. The spec isn't documentation after the fact; it's the answer key the loop runs against from the start. That's a different relationship between planning and execution than most AI-assisted development workflows assume.

The open question is whether the 34-question interrogation session is a feature or a friction point. For a business-critical internal tool, front-loading that rigor probably pays off. For rapid prototyping or creative experimentation, it might be too much ceremony before you've even confirmed the idea is worth pursuing. The gauntlet loop without Wayfinder is fast and unreliable for novel work. With it, you trade some of that speed for a spec that actually holds.

Whether that trade is worth making depends entirely on what you're building—and whether you can afford to discover what the agent assumed about your requirements after it's already built them.

---

Rachel "Rach" Kovacs is Buzzrag's cybersecurity and privacy correspondent.