Mastra Trace Intelligence Turns Agent Logs Into

Here's a problem nobody talks about enough: you deploy an AI agent, you turn on tracing because everyone said you should, and then you have... traces. Millions of them. Each one a dense log of every tool call, every step, every token your agent burned through. Congratulations, you now own a data warehouse and zero understanding of what your agent is actually doing in the wild.

Mastra thinks they have a fix. Their new feature, Trace Intelligence — currently in private beta — is designed to collapse that mountain of logs into something a human brain can actually work with. In a live workshop this week, engineers Daniel and Eric walked through both the UI and the technical machinery underneath it. The architecture is interesting enough that it's worth pulling apart properly.

Why Standard Observability Breaks Down at Scale

Before getting into how Trace Intelligence works, it's worth understanding why it needs to exist — and the answer starts with a decision Mastra made about their tracing stack that most people overlook.

They don't use OpenTelemetry (OTel) natively. That's a significant architectural choice. As Eric explained in the workshop, the standard OTel model requires you to wait until a span is complete before shipping it. For short-lived requests, fine. For agents that run for 30 seconds, several minutes, or — increasingly — indefinitely as long-running systems, that constraint is basically a non-starter. So Mastra built their own tracing stack that ships information as it occurs, in real time.

That custom stack exports to OTel if you want it to. It also exports to Braintrust, Langfuse, Arize, Grafana, and around eleven other destinations — simultaneously, if that's your setup. You can push to Mastra Platform and Langfuse at the same time without choosing. That flexibility matters in practice, especially for teams who've already built tooling around another observability provider.

The reason this is worth knowing before anything else: Trace Intelligence is built on top of that custom tracing foundation. The data quality and real-time completeness are prerequisites for what comes next.

A Megabyte of Data, Squeezed Into Four Sentences

The scale problem is starker than you might expect. During the workshop, Eric described production cases where a single trace — one agent run — clocks in at a megabyte of data. Scale that to even a hundred traces and you have a corpus that's genuinely impossible for a human to review, let alone pattern-match across. The Mastra team ran into this themselves when they first launched: heavy traces were breaking their own system because they were trying to pull everything into memory at once.

The solution they landed on is to extract a compressed signal from each trace, not the full trace itself. Here's the pipeline as Daniel and Eric described it:

First, deterministic extraction — code-driven, not model-driven. The system looks at the shape of the trace: which tools were called, whether calls succeeded, how token usage compares to similar traces, how user sentiment shifted over the course of a conversation. Tags get applied based on these structural features.

Then, model-driven signal generation. For each trace, the system generates four compact signals using a small, cheap language model. (The workshop confirmed they're currently using OpenAI's GPT-4o Nano for this step — deliberately light, given the volume.) Each signal maps to one dimension: goal (what was the user trying to do?), outcome (did they succeed?), behavior (what did the agent actually do during execution?), and sentiment (how did the interaction feel?). Different inputs get routed to different signal generators — not everything goes to all four.

The outputs are deliberately tiny. As Daniel put it: "You're getting like we talked before like it could be a megabyte of data more than a megabyte of data and you're basically squishing it down into just a few words." Four bullet points per trace. One per signal.

From Bullet Points to Behavioral Map

Those compact signals then get embedded — converted into high-dimensional numerical vectors that capture semantic meaning — and reduced into a lower-dimensional clustering space. According to Eric's explanation in the workshop, the team ran experiments to find a dimensionality that clustered well without losing signal or introducing too much noise, and landed on five dimensions for the projection step.

The clustering algorithm they use is HDBSCAN (Eric referred to it as "HTB scan" in the workshop). The key property of HDBSCAN is that you don't have to tell it in advance how many clusters to expect, and it can identify points that don't fit neatly into any cluster at all. For a system where you genuinely don't know what behavioral patterns will emerge from your users, that's the right tool.

The resulting clusters become themes — named groupings like "urgent order issue assistance" or "password reset" or "billing inquiry." Each theme represents a recurring pattern across many traces. And these themes aren't static: they grow, shrink, split, merge, and occasionally die as your agent's usage patterns evolve. If billing questions suddenly spike and start fragmenting into distinct subtypes, the system will start splitting that theme automatically. You can track theme trajectories over time through what the workshop called "snapshots" — point-in-time views of your trace distribution that let you compare, say, how your agent looked on July 29th versus August 19th.

The Sankey Chart and the Drill-Down

The main visualization is a Sankey chart — that's the flow diagram where the width of a band represents the proportion of traces moving through it. If you haven't used one before, the mental model is: each column represents a signal dimension, each band represents traces flowing from one theme to another across dimensions. Cross-referencing becomes visual. You can see immediately that most of your "billing" goal traces are ending in "frustrated sentiment," or that "password reset" traces almost always succeed.

From there, you can drill into specific themes, inspect representative individual traces, see how each signal scored for that trace, and pull up the full raw trace if you need to go deeper. Daniel walked through this during the demo, showing how you'd move from a theme-level view down to a single trace and its observations — the same observational summary format Mastra uses elsewhere in their memory stack.

There's also an API. Trace Intelligence exposes its data programmatically, which means you can query it through the Mastra CLI or hand it off to an agent directly. The practical implication: your debugging workflow doesn't have to live in the UI. You can ask an agent to pull themes for the past seven days, identify the biggest failure patterns, grab representative traces, and start reasoning about fixes — all in one loop.

Themes as a Map to Better Evals

The part of the workshop that I find most interesting isn't the visualization — it's the connection to evaluation and improvement. Right now, most teams who do evals pick their test cases somewhat arbitrarily: things that felt important at the time, edge cases someone thought of, maybe some early failure logs. Trace Intelligence gives you a principled way to select traces instead: find the theme that corresponds to a failure mode you care about, pull the most representative traces from that theme, and build a dataset that actually reflects what your production users are doing.

That closes a loop that's genuinely hard to close otherwise. "They're just traces," as one of the engineers put it in the workshop. "This is like your map to find the most relevant traces to a scenario be that a failure mode or a use case you care about and then you can create data sets you can run experiments." Mastra is already building the next step — automated dataset creation from trace themes — which will be covered in a follow-up workshop on September 3rd.

The honest caveat the workshop video itself flags: association is not causation. A theme correlating with frustrated sentiment doesn't tell you why users are frustrated, or whether fixing the agent behavior in that theme will actually help. That interpretive step still requires a human. Trace Intelligence gets you to the right questions faster; it doesn't answer them for you.

I think that caveat matters more than it might seem. There's a real temptation, once you have automated feedback signals, to optimize against them directly — to treat theme sentiment scores as the metric and hill-climb on it. That's a reasonable thing to try. It's also a reliable way to end up with an agent that games your measurement system while the underlying problem stays broken. The tool is sound. The discipline to use it without mistaking the map for the territory is on you.

Trace Intelligence is currently available in private beta for Mastra Platform users. Self-hosting support is in progress; today, the platform is the only path to using it.

Yuki Okonkwo is an AI & Machine Learning Correspondent for Buzzrag.