Tencent HY3 Reviewed: Free, Open Source, and

Every few weeks, another model drops out of China's AI labs wearing two tags that reliably generate attention: free and open source. Tencent's HY3 arrived this week with both, plus benchmark numbers that look competitive against Claude and GLM 5.2 on some agentic evaluations. The question the community is now sorting out—with the unsatisfying patience that real evaluation requires—is whether the numbers translate.

Julian Goldie, who runs the GoldieBench testing framework and has been building agent workflows across a wide range of open-source models, spent several hours this week putting HY3 through its paces. His verdict is neither a dismissal nor an endorsement. It's something more useful: a map of where the model actually lives.

What HY3 Is, Technically

Let's start with the architecture, because it shapes everything else. HY3 is a 295-billion parameter mixture-of-experts model, meaning its full weight is enormous but only about 21 billion parameters are active on any given inference. That design choice—common to models like Mixtral and DeepSeek—is what makes it practical to run via API without the inference costs of a dense 295B model. It's also why there's legitimate interest in whether it can eventually run locally; prior Tencent models were reportedly small enough to fit in 8GB, which would make local deployment genuinely accessible.

For now, HY3 is available free through OpenRouter (until July 21st, at which point pricing details remain unclear), and it can be plugged into tools like Hermes Agent, News Research integrations, and Kilo Code. Tencent's own documentation positions it explicitly around agent capabilities—browsing, tool use, multi-step task execution—rather than pure generative output quality.

That positioning is important context for what the testing actually found.

The Benchmark-Reality Gap

This is where the story gets genuinely interesting, and it's a tension that extends well beyond HY3.

On paper, HY3's benchmark numbers on MCP Atlas, Browse, and Claude Eval agentic workflows sit close to—and in some cases ahead of—GLM 5.2. That's a credible showing. Goldie's reaction to those numbers after actually running the model is worth quoting directly: "When I've tested it out myself, it doesn't seem to feel like that at all. Doesn't seem to be on the same level."

This is not a new problem in AI evaluation. Benchmarks are constructed tasks. They measure what they measure, and the models that score well on them have often been trained in ways that optimize for exactly those tasks. The practical question—does the model produce outputs I actually want to use—is a different question, and it doesn't always correlate.

For Goldie's specific testing methodology, this gap was stark. Using the same prompts across models for game-like creative coding tasks, HY3 produced outputs that were functional but thin: an open-world game that was "glitchy," a city driving demo with "super basic" geometry, a parachute game that was visually rough. After three to four hours of iteration, the outputs still weren't landing at the quality he was hitting with Fusion, Fable 5, or GLM 5.2.

"It's awesome that it's free, awesome that it's open source. Is it the best model or one of the best models I've ever used? No."

The honest framing there is important. He's not saying HY3 is bad. He's saying it doesn't compete at the frontier level, which is a different claim—and one that's entirely consistent with what Tencent actually designed it for.

Where It Actually Performs

Strip away the creative coding comparisons, and a more coherent picture emerges of what HY3 does well.

Loaded into Hermes as an agent backbone, it handles browsing tasks, skill execution, and tool calls with reasonable reliability and—notably—speed. For workflows that are less about generative quality and more about orchestration—open this page, learn this skill, run this tool, output this result—HY3 holds up. Goldie's description of using it in a Hermes profile to run Google searches and parse web pages is the kind of use case where a model doesn't need frontier-level flair; it needs to execute instructions consistently.

The HY3 Coder workflow is highlighted as a genuine value-add: a free API through OpenRouter, prompts on the left, live preview on the right, saved outputs below. That's a functional development loop that costs nothing, which matters for developers who want to experiment without committing to paid API credits.

The Remotion video generation demo is the most surprising highlight. By giving HY3 access to the Remotion JavaScript library—which programmatically generates animated videos—it produced animated video content that Goldie described as "certainly better than a lot of open source projects I've created." That's a narrow use case, but it's a genuinely novel one, and it points toward where the model's agent-oriented design actually shines: not generating directly from a prompt, but orchestrating tools that do the heavy lifting.

The Open Source Landscape It's Entering

It's worth placing HY3 in the broader context of Chinese open-source AI releases, because the ecosystem is crowded and the signals are mixed.

Goldie's own hierarchy, stated plainly: he'd reach for GLM 5.2 first, Qwen 3.7 second, and HY3 as a free-tier option when cost matters more than output ceiling. DeepSeek, despite its visibility in Western developer communities, doesn't even make his regular rotation.

That ranking reflects something real about how open-source models get adopted in practice. The headline release gets the attention; the model that's actually integrated into daily workflows six months later is usually the one that proved reliable under sustained use, not the one with the best launch benchmarks. GLM 5.2 has apparently built that trust in Goldie's testing environment. HY3 is newer, and three to four hours of evaluation—however thorough—is still an early read.

What's harder to evaluate from the outside is the governance and sustainability story, which is different for a model released by Tencent than for community-maintained projects. Corporate open-source AI releases come with implicit questions: How long does the free API stay free? What happens to the weights if Tencent's priorities shift? What's the licensing situation for commercial use? Those aren't criticisms of HY3 specifically—they're structural questions that apply equally to models from Google, Meta, and Mistral. But they're worth keeping in mind when you're building agent workflows around a model that's currently free because a large tech company decided it should be.

The Benchmark Problem Is the Real Story

Here's the thread worth pulling on, and it's bigger than HY3.

The open-source AI space has a benchmark inflation problem that the community knows about and mostly hasn't solved. Models get evaluated on curated tasks. Developers integrate them based on those evaluations. Then someone like Goldie spends four hours iterating on real outputs and discovers the gap. GoldieBench—his own testing framework with 45 tasks—exists precisely because he doesn't trust leaderboard numbers to predict real-world behavior.

That's not a bug in his methodology; that's a reasonable response to a measurement system that's broken in specific and well-documented ways. The question the community still hasn't fully answered is how to build evaluation frameworks that are both reproducible enough to be useful and realistic enough to predict the outputs developers actually care about.

HY3's benchmark performance looked competitive. Its creative coding output looked rough. Both of those things can be true simultaneously, and the gap between them is where the most interesting questions about AI evaluation currently live.

For developers deciding whether to experiment with HY3: it's free, the agentic workflow integration is real, and the Remotion-based video generation is worth trying. For anyone expecting frontier creative coding output—manage expectations before you spend four hours finding out the hard way.

By Dev Kapoor, Open Source & Developer Communities Correspondent, Buzzrag