How AI Is Actually Tested for Human-Level

No AI model in the world can reliably tell the difference between "inside" a shape and "outside" it. That is not a metaphor for some deeper philosophical limitation. It is a literal description of where the current frontier sits — and it matters more than anything you'll read in a press release this week.

The tests designed to probe that gap are worth understanding, because they reveal something the industry's marketing apparatus works hard to obscure: the distance between impressive performance and actual understanding. Ben Doyle, writing for the Half as Interesting channel, recently laid out the landscape of AI evaluation methodologies in plain terms, and the picture that emerges is more complicated — and more honest — than most of what passes for AI coverage.

Knowledge Is Not Intelligence

The flagship test that generated the most noise in early 2024 was Humanity's Last Exam — 2,500 graduate-level questions spanning mathematics, biology, the social sciences, and a range of specialist fields, each question submitted and verified by subject-matter experts. The ambition is genuine: if a system can answer what the best-trained human minds across every discipline know, hasn't it cleared some meaningful bar?

Doyle's answer is no, and his reasoning is correct. Trivia, even very hard trivia, is not intelligence. It's retrieval. The distinction matters enormously.

Consider the demonstration he offers: a question posed to a leading frontier model that asks whether a dead cat — explicitly stated to be dead at the outset — is alive or dead after sitting in a box with a poison vial. The cat is dead. It was dead in the first sentence. Any person who reads English with normal comprehension gets this instantly. The model, trained on countless iterations of the Schrödinger's cat thought experiment, pattern-matches to that famous setup and loses the thread of what the question actually says.

"A real meat person with a real meat brain," Doyle observes, "can simply read the question and think about what each individual word actually means." Models don't do that. They do something that resembles it closely enough to fool most observers most of the time, which is precisely why the distinction is easy to miss and important not to.

The same systems that stumble on this can perform graduate-level calculus. They can parse dense academic prose and generate plausible legal analysis. The juxtaposition — extraordinary capability sitting directly beside elementary failure — is not a bug that will get patched in the next release. It reflects something structural about how these systems work.

The Puzzle That Breaks Them

This is where ARC AGI becomes the more revealing test. Rather than asking what a model knows, it asks whether the model can figure out a rule it has never seen before, using only the examples in front of it.

The mechanics are simple: you're shown a series of input-output pairs that follow a hidden rule, and you have to identify the rule and apply it to a new case. Doyle describes one example where the rule amounts to erasing everything above a line and filling in everything below it — the kind of spatial reasoning a child masters before they can reliably tie their shoes.

About half the models that attempted that puzzle failed it. The underlying concepts — above, below, separated by a boundary — are not obscure. They're foundational to how humans navigate physical space. We learn them so early and apply them so constantly that we don't experience them as concepts at all. For a language model, they're statistical patterns in text, and not particularly well-represented ones.

"For an AI, this is all just statistical data," Doyle notes, "and not a lot of it."

ARC AGI 2 raises the difficulty — shapes with "open" versus "closed" properties, rules that depend on spatial containment — and the ARC AGI 3 results make the picture even starker. Humans solve it at a 100% rate. Frontier models can barely engage with it.

That third tier of the test looks nothing like the first two. Rather than static puzzles, it presents a series of minimal video games with no instructions — the player must figure out not just how to win but what winning even means in each case. One game involves moving boxes to a designated zone, with a helper character appearing in later levels whose assistance becomes necessary. Another involves routing a liquid substance. Another is a maze.

The cognitive demand is the kind of thing humans find trivially easy: observe, hypothesize, revise. Watch what happens when you try something. Notice the helper. Understand that the pink stuff flows like water without being told it's water. This is reasoning from observation, not retrieval from training. At the time Doyle recorded his account of these results, the best-performing models had barely made a dent in even the first game.

What the Tests Are Actually Measuring

The frame that makes sense of all of this is the distinction between knowledge and flexibility. Current AI systems have accumulated knowledge at a scale no human could approach. On Humanity's Last Exam, they've moved from scores that Doyle describes as "relatable and cute" to scores that outpace any individual human expert across the full range of questions. That progress is real and should not be minimized.

But intelligence — the kind that AGI researchers are actually chasing — is not primarily about the stock of knowledge. It's about what you can do when you encounter something genuinely new. It's the ability to apply a concept learned in one context to a problem framed entirely differently. To notice that the orange square in a game is helpful. To understand that a closed shape has an inside and an outside.

Descartes thought about this problem in the 17th century. Turing formalized it in the 20th. Doyle puts it plainly: "people have been developing tests for it since the 17th century... nerds have been doing this forever." The persistence of the question should tell you something about how hard the underlying problem actually is.

The hype cycle around AI tends to collapse these distinctions. A system that passes Humanity's Last Exam gets described as "superhuman." In a narrow, specific sense, that's accurate — it knows more facts than any human does. But it cannot reliably tell you whether a cat that was dead when the story started is still dead. These are not contradictory findings. They're a precise description of what these systems are.

The Architecture Question Nobody Wants to Ask

Here is the thing worth sitting with: it's not obvious that the current approach — training large models on vast text corpora — is the path to passing ARC AGI 3. The test is specifically designed to require reasoning that cannot be done by pattern-matching against prior exposure, because every puzzle is novel by construction. If a model can only apply rules it has seen variants of before, and the test is explicitly built to exclude that, then scaling the existing architecture might not help.

This doesn't mean AGI is impossible. It means the people claiming it's imminent owe the rest of us a more specific account of what architectural shift they think will bridge this particular gap. So far, that account is not forthcoming.

The tests are doing their job — revealing exactly where the ceiling is. The harder question is whether the industry building toward that ceiling has a plan for what comes after it hits.

Bob Reynolds is Senior Technology Correspondent at Buzzrag.