AI Agents Need Live Web Data, Not Bigger Models

Every pre-trained language model is, in a precise sense, a ghost. It knows the world as it existed up to some date, wanders around confidently in the present, and has no idea it's out of phase. That's not a metaphor; that's the architecture. Training ends, weights freeze, and the model's picture of reality stops updating. The world does not return the favor.

This is the framing Ariel Shulman, Chief Product Officer at Bright Data, lays out in a recent IBM Technology video: the central bottleneck in reliable AI isn't model intelligence, it's model freshness. And the argument lands hardest when you stop thinking about chatbots and start thinking about agents.

The ghost problem, illustrated

When a human asks an LLM about a phone that doesn't exist yet and gets a hallucinated spec sheet back, the human usually clocks it. The answer smells wrong. We have shame, intuition, a sense that something is off. We slow down.

An AI agent operating without a human in the loop does none of that. It reads the hallucinated answer as valid input and acts on it. In Shulman's example, a purchasing agent for an electronics company receives a fabricated product description, treats it as ground truth, and starts recommending component acquisitions. No friction, no pause, no sanity check. Then a downstream marketing agent and a finance agent each inherit that bad premise and build their own decisions on top of it.

As Shulman puts it: "AI agents can fail confidently, at scale, and very fast."

The cascading error problem is well-documented in agentic systems. What Shulman's framing adds is specificity about the root cause: the biggest failures in production AI systems, he argues, "are not about intelligence. They're about bad data, and systems acting on information that is wrong, outdated, or simply made up." Garbage in, garbage out. GIGO. A phrase from the punch-card era that turns out to describe one of the most urgent problems in modern AI.

Infrastructure, not just retrieval

The conventional response to stale models is retrieval-augmented generation (RAG), the technique of pulling in external documents at query time to supplement what the model already knows. That's the right direction. But Shulman is making a more specific point about what retrieval actually requires to work reliably at production scale.

A knowledge layer, the mechanism that injects real-time context into an LLM's response generation, can technically reach out to the open web directly. In practice, this is where things get messy. Websites serve CAPTCHAs. Anti-bot systems block access. Even when access is granted, the web hands back raw HTML: megabytes of nested markup, inline CSS, JavaScript bundles, and navigation chrome that have nothing to do with the information you're actually after. Dumping that into an LLM's context window is expensive (tokens cost money), slow (latency matters), and noisy (irrelevant markup can genuinely confuse the model).

Shulman's analogy: "Giving raw HTML to an AI agent is kind of like buying a bed in IKEA but not finding the instructions in the box. You get just a pile of wooden parts and some bolts."

This is being recognized across the industry as a distinct infrastructure category, not just a feature bolted onto existing systems. Fast Company has drawn the comparison to the pre-web era of enterprise computing: powerful systems sitting behind walls, disconnected from the broader information environment, waiting for the connective layer that would change everything. The web data infrastructure layer is, in this reading, that connective tissue for AI.

Five properties, no shortcuts

Shulman proposes five properties that web data must have to be trustworthy as LLM input. They're worth taking seriously because missing any one of them breaks the whole chain.

Grounded. Every data point needs a citable source. Shulman uses Wikipedia as the model here: people trust it not because it's authoritative by fiat, but because every claim links back to something you can check. Without that, users have no way to distinguish a real answer from a confident fabrication.

Deep. Snippets aren't enough. Feeding an LLM a one-paragraph summary and asking it to reason carefully is like asking someone to review a contract while only showing them the cover page. Full documents, full pages, structured metadata: the model needs enough context to actually understand what it's working with.

Fresh. This is the most variable property. Shulman's taxonomy is useful here: pricing data goes stale in minutes, news in hours, legal information over days. A fixed scraping schedule doesn't match that reality. For high-velocity data like e-commerce stock and pricing, the only defensible approach is to refresh at the time of inference, not ahead of it.

Formatted. Raw HTML converted to compact, structured formats: markdown, JSON with only relevant fields, clean text. This isn't just about readability; it's about token efficiency. Unnecessary tokens mean slower responses and higher costs, and that's before accounting for the errors that come from forcing a model to parse navigation menus and cookie banners before it gets to the actual content.

Timely. Context that arrives after the model has already answered is useless. Shulman's sports-commentary analogy nails it: an NBA play-by-play with a 20-second audio delay is technically accurate and completely worthless. Depending on the application, required latency can be seconds or milliseconds. Either way, it's a core system requirement, not a performance footnote.

Each of these properties fails independently. A data source can be deep but stale. Fresh but unstructured. Grounded but slow. The hallucination risk in agentic systems isn't one problem; it's five overlapping ones, and solving only a few of them while ignoring the others doesn't get you to trustworthy outputs.

The staleness asymmetry

One detail in Shulman's analysis that doesn't get enough attention: not all data decays at the same rate, and the data that decays fastest tends to be exactly what users want most urgently.

Product descriptions are relatively stable. Prices and stock availability are not, especially for in-demand items. The products users are most likely to ask about are the hot ones, and hot products are the ones most likely to go out of stock overnight. A shopping agent that scrapes inventory data once a day and caches it is structurally set up to disappoint the users who care most. Joe asks if the product is available, the agent says yes, Joe shows up at checkout, product is gone. Joe doesn't blame the retailer. Joe blames the AI.

As International Business Times has framed it: the cure for AI hallucinations isn't a bigger model, it's fresher data. That reframe matters. The industry's default response to AI failures has been to scale the model. More parameters, more training compute, longer context windows. Shulman is arguing that for a large and important class of failures, that's solving the wrong problem. You can have the smartest model in the world and still send someone confidently to a museum that closed three months ago.

What this actually means for AI development

The question worth sitting with is whether this reframing of the problem changes how we should evaluate AI systems. Right now, the headline metrics are model benchmarks: reasoning scores, coding ability, multimodal performance. Those matter. But if Shulman's diagnosis is correct, and the biggest production failures are data failures rather than reasoning failures, then the infrastructure layer deserves evaluation criteria of its own.

How fresh is the data? How reliably does the scraping infrastructure handle anti-bot systems at scale? What's the actual latency from page update to model context? These aren't model questions; they're plumbing questions. And plumbing, famously, is what determines whether a building is actually livable.

The conversation about AI capability has been dominated by what models can do in principle. The conversation about what AI agents can do reliably, in production, for real users who will lose trust the first time they get burned by a confident wrong answer, that's a different conversation. And it runs straight through the data layer sitting underneath the model.

---

Yuki Okonkwo is Buzzrag's AI and Machine Learning correspondent.