How AI Is Actually Tested for Human-Level Intelligence
Bob Reynolds4 days ago
AI benchmarks
26 stories tagged AI benchmarks.
AI. Bob Reynolds4 days ago
How AI Is Actually Tested for Human-Level Intelligence
ARC AGI 3 tasks AI with figuring out video games from scratch — and current models can barely start them. Here's what that reveals about the gap between AI and human reasoning.
Tencent HY3 Reviewed: Free, Open Source, and Uneven
Dev Kapoor2 weeks ago
AI. Dev Kapoor2 weeks ago
Tencent HY3 Reviewed: Free, Open Source, and Uneven
Tencent's HY3 is a free, 295B open-source model with real agentic strengths—but benchmark scores and real-world output quality tell different stories.
GPT 5.6 Sol vs Fable 5: Early Numbers, Real Tradeoffs
Yuki Okonkwo2 weeks ago
AI. Yuki Okonkwo2 weeks ago
GPT 5.6 Sol vs Fable 5: Early Numbers, Real Tradeoffs
GPT 5.6 Sol is half the price of Fable 5 — but is it half as good? Early benchmark comparisons, alignment regressions, and the politics reshaping who gets access.
AI Leaderboards Are Lying to You About State-of-the-Art
Yuki Okonkwo2 months ago
AI. Yuki Okonkwo2 months ago
AI Leaderboards Are Lying to You About State-of-the-Art
Bertrand Charpentier of Pruna AI makes the case that 'state-of-the-art' is a broken concept—and that efficiency belongs in the same sentence as quality.
AI Benchmark Scores Are Broken. Here's Who's Fixing Them.
Rachel "Rach" Kovacs2 months ago
AI. Rachel "Rach" Kovacs2 months ago
AI Benchmark Scores Are Broken. Here's Who's Fixing Them.
AI benchmark scores are less trustworthy than they look. Google DeepMind's Kaggle team is building open infrastructure to fix that—here's what you need to know.
GPT-5.5 Is Great, But You Might Not Notice—Here's Why
Yuki Okonkwo3 months ago
AI. Yuki Okonkwo3 months ago
GPT-5.5 Is Great, But You Might Not Notice—Here's Why
OpenAI's GPT-5.5 dominates benchmarks and handles complex coding tasks, but many users won't feel the upgrade. We dig into the paradox.
GPT 5.5 vs DeepSeek V4: The Benchmarks Tell a Jagged Story
Mike Sullivan3 months ago
AI. Mike Sullivan3 months ago
GPT 5.5 vs DeepSeek V4: The Benchmarks Tell a Jagged Story
OpenAI and DeepSeek released flagship models within 20 hours. The benchmark results reveal something more interesting than who's winning.
The Benchmark Paradox: What Qwen 3.6's Numbers Actually Mean
Zara Chen3 months ago
AI. Zara Chen3 months ago
The Benchmark Paradox: What Qwen 3.6's Numbers Actually Mean
Qwen's new 27B model is beating models 10x its size—on paper. Here's what those benchmarks aren't telling you about AI performance.
OpenAI's GPT-5.5 Claims Speed Crown—But Costs 20% More
Tyler Nakamura3 months ago
AI. Tyler Nakamura3 months ago
OpenAI's GPT-5.5 Claims Speed Crown—But Costs 20% More
GPT-5.5 promises faster AI coding with fewer tokens, but WorldofAI's tests reveal where it excels—and where it disappoints at premium pricing.
OpenAI's GPT-5.5: When the Benchmarks Don't Tell the Whole Story
Dev Kapoor3 months ago
AI. Dev Kapoor3 months ago
OpenAI's GPT-5.5: When the Benchmarks Don't Tell the Whole Story
GPT-5.5 arrives with impressive real-world benchmarks and doubled pricing. But the coding results reveal tensions in how we measure AI capability.
Anthropic's Opus 4.7: The Enterprise Model You Can't Afford
Mike Sullivan3 months ago
AI. Mike Sullivan3 months ago
Anthropic's Opus 4.7: The Enterprise Model You Can't Afford
Anthropic's Opus 4.7 excels at enterprise tasks but costs 35% more due to tokenizer changes. The upgrade everyone's complaining about, explained.
Three AI Models Just Dropped—Here's What Actually Matters
Tyler Nakamura3 months ago
AI. Tyler Nakamura3 months ago
Three AI Models Just Dropped—Here's What Actually Matters
Meta's Muse Spark, Z.ai's GLM 5.1, and Anthropic's Managed Agents all launched this week. Here's what they're good at—and what they're not.
Anthropic's Mythos Launch: Security Theater or IPO Theater?
Samira Barnes3 months ago
AI. Samira Barnes3 months ago
Anthropic's Mythos Launch: Security Theater or IPO Theater?
Anthropic's Project Glasswing positions Mythos as too dangerous to release. The timing before a $380B IPO raises questions about the narrative's purpose.
AI Benchmarks Are Breaking. Here's Why That Matters.
Zara Chen4 months ago
AI. Zara Chen4 months ago
AI Benchmarks Are Breaking. Here's Why That Matters.
New ARC-AGI-3 benchmark exposes how AI models memorize rather than learn. Humans score 100%, frontier AI models score less than 1%. The gap reveals everything.
GPT-5.4 Pro Costs $180 Per Million Tokens—And Beats Google at Its Game
Bob Reynolds4 months ago
AI. Bob Reynolds4 months ago
GPT-5.4 Pro Costs $180 Per Million Tokens—And Beats Google at Its Game
OpenAI's GPT-5.4 Pro outperforms competitors on new benchmarks, but at a steep price. What the latest AI model tells us about the real race.
AI Agents Are Accelerating—But Nobody Agrees What That Means
Dev Kapoor5 months ago
AI. Dev Kapoor5 months ago
AI Agents Are Accelerating—But Nobody Agrees What That Means
New benchmarks show AI coding agents tripling capabilities in months. Researchers urge caution. Investors price in economic collapse. Welcome to 2026.
Google's Gemini 3.1 Pro: When Benchmark Wins Stop Mattering
Bob Reynolds5 months ago
AI. Bob Reynolds5 months ago
Google's Gemini 3.1 Pro: When Benchmark Wins Stop Mattering
Gemini 3.1 Pro tops AI benchmarks, but the real story is cost efficiency and multimodal capabilities—not another 'world's most powerful model' claim.
Sam Altman Says AGI Arrives in 2 Years. Here's the Data.
Tyler Nakamura5 months ago
AI. Tyler Nakamura5 months ago
Sam Altman Says AGI Arrives in 2 Years. Here's the Data.
OpenAI's Sam Altman just compressed the AGI timeline to 2028. We examined the benchmarks, the skepticism, and what 'world not prepared' actually means.
Google's Gemini 3.1 Pro: Genius on Paper, Disaster in Practice
Zara Chen5 months ago
AI. Zara Chen5 months ago
Google's Gemini 3.1 Pro: Genius on Paper, Disaster in Practice
Gemini 3.1 Pro crushes benchmarks but fails at basic tasks. Developer Theo tests Google's 'smartest model ever' and finds a genius that can't follow instructions.
Why AI Benchmarks Are Breaking (And What That Means for You)
Tyler Nakamura5 months ago
AI. Tyler Nakamura5 months ago
Why AI Benchmarks Are Breaking (And What That Means for You)
Google's Gemini 3.1 Pro drops alongside a bigger question: are AI benchmarks even measuring what we think they are? The answer affects your buying decisions.