3 Advanced AI Agent Memory Patterns Explained

There is a specific kind of frustration that users feel when an AI agent makes them repeat themselves. You told it you were vegan three messages ago. You mentioned you already visited that museum. You shared a photo of the coastline you love. And then the agent comes back, blank-faced, asking again. In developer circles, this has been nicknamed the Goldfish Problem—and it is, by most accounts, one of the more significant gaps between what AI agents promise and what they actually deliver.

A recent Google Cloud Tech video tackles that gap directly. Annie Wang, a software engineer turned AI educator at Google, and host Martin Omander walk through three advanced memory patterns for AI agents built on Google's Agent Development Kit (ADK): callbacks, custom tools, and multimodal memory. The framing is deliberately practical—this is a how-to, not a vision statement—and it is worth unpacking not just as a technical tutorial, but as a window into the choices developers face when they decide what an agent should remember, how it should store that memory, and what structure it imposes on users' data in the process.

The premise worth examining first

Wang's central argument is one that deserves a moment's consideration before the code starts flowing. "Everyone thinks AI intelligence is just about the model and tooling," she says. "They're forgetting the other important piece, that is memory."

That is a pointed observation about where developer attention has been concentrated. The model—which foundation model, how large, how well-prompted—absorbs enormous energy in the AI development conversation. Memory architecture, by contrast, is often treated as an afterthought, a plumbing problem to solve once the "smart" parts are working. Wang's position is that this ordering is backward if your goal is an agent that users actually want to return to. An impressive model that cannot remember what you told it two minutes ago is an impressive model that fails at the job.

The first video in this series covered three foundational patterns: session state (memory within a single conversation), multi-agent state (memory shared across a system of cooperating agents), and persistent memory (memory that survives across sessions). The three patterns in this video are meant to address scenarios where those foundations are not enough—where you need memory that is automatic, structured, or richer than text.

Callbacks: memory that updates itself

The first pattern, callbacks, addresses a specific design tension: how do you keep an agent's context current without burdening the agent itself with the task of tracking everything?

Wang's answer is to intercept the agent's lifecycle. The ADK allows developers to insert custom logic at several points: before or after the agent runs, before or after the model is called, and before or after any tool is called. These hooks—callbacks—run automatically, invisible to the agent's core logic.

The demonstration uses a trip planning application. If a user has already visited a museum, the callback registers that activity type and updates the agent's context accordingly. The next time the user asks for a recommendation, the agent knows—without being explicitly told—to suggest something different. As Wang puts it, "the agent builds its own context simply by doing its job."

The design principle here is separation of concerns: the agent focuses on planning; the callback focuses on tracking. That is an architecturally clean approach, and it has real implications for maintenance. Adding memory tracking via callbacks does not require rewriting agent instructions or expanding the agent's cognitive load. It also means the memory logic lives in a discrete, auditable place—a consideration that matters more than developers typically acknowledge when thinking about debugging or, for that matter, explaining to a user what the agent recorded about them.

What callbacks do not resolve is the question of granularity. What counts as a meaningful activity type to track? Who decides the categories—cultural, food, outdoor—that the callback writes to context? Those choices are made by the developer, embedded in code, and largely invisible to the user. That is not a flaw unique to this pattern, but it is worth naming.

Custom tools: from conversation logs to structured records

The second pattern moves from automatic tracking to deliberate data structuring. Wang's framing is direct: "So far, memories have been unstructured text. It can be a lot of data for the agent to dig through. But if the agent needs to know that you're vegan, let's put that in a user profile."

The implementation involves two purpose-built tools: save_user_preference and recall_user_preferences. The agent is instructed to call recall_user_preferences at the start of any relevant interaction, and to call save_user_preference whenever the user states a preference. The preference is stored as a structured dictionary in a database, not buried in a chat log.

The user experience Wang demonstrates is clean. On first use, the agent asks about dietary preferences. On subsequent use, after the script is restarted, the agent simply recommends a vegan restaurant without asking. The friction is gone.

This is where the technical tutorial and the policy-adjacent question converge in ways that developers building production systems should think carefully about. Structured user profiles are more useful than unstructured logs—they are also more legible as data objects, which cuts both ways. A structured profile is easier for the agent to query. It is also easier to expose through an API, easier to aggregate across users, and easier to mishandle. The video is not a data governance tutorial, nor should it be—but the pattern it describes is one that creates genuine user data, stored persistently, structured for query. What developers do with that infrastructure will vary enormously.

None of this is a reason to avoid the pattern. It is a reason to build it thoughtfully, with attention to what users understand about what is being stored and why.

Multimodal memory: the harder problem

The third pattern is the most technically ambitious and, in some ways, the most conceptually interesting. Wang's setup: "All our saved memories have been text so far, but humans remember more than text. We remember the vibe of a photo or the sound of a voice or the feeling of a video."

The multimodal memory pattern stores sessions containing images, video, and audio in what the ADK calls a memory bank. An agent equipped with the preload_memory tool can then retrieve those stored sessions and reason across them. In the demonstration, a travel assistant agent—previously shown a picture, a video, and an audio file by the user—responds to the question "Where should I travel?" by synthesizing preferences drawn from all three media types.

That is a meaningfully different capability from anything the previous two patterns accomplish. Callbacks track activity types; custom tools store preferences stated in text. Multimodal memory attempts to capture something more ambient—aesthetic preferences, contextual signals, the kind of information that humans convey through what they share rather than what they say.

The gap between the demo and production complexity is worth acknowledging. Reasoning coherently across heterogeneous media stored across sessions is a hard problem. The video shows it working in a controlled demonstration; the general case—with varied media quality, ambiguous preferences, conflicting signals across sessions—is considerably messier. Wang and Omander do not oversell this, to their credit. They present it as a pattern that can be implemented, not a problem that is solved.

The memory bank itself, and the preload_memory tool that loads from it, represent an interesting architectural choice: the agent does not automatically receive all stored media on every invocation. It loads from memory when instructed to. That is a reasonable design for managing context window size and cost, but it also means the agent's access to its own history is mediated by a tool call—which can fail, which can be omitted, which introduces a dependency that pure in-context approaches do not have.

What this framework does and does not tell us

Taken together, the six patterns Wang has now presented across two videos sketch a reasonably comprehensive taxonomy of how agent memory can be structured: by scope (session vs. persistent), by architecture (single vs. multi-agent), by mechanism (callbacks vs. explicit tools), and by data type (text vs. multimodal). For developers building on the ADK, this is genuinely useful scaffolding.

What the framework does not address—and what no developer-focused tutorial really can—is the question of what users understand about how these systems work. An agent that quietly updates your dietary profile every time you state a preference is useful. It is also building a persistent record of your stated preferences, stored somewhere, accessible to the tools the developer has chosen to grant access. The agent that "just suggests a vegan restaurant" without asking is also an agent that has been silently maintaining a file on you.

That is not an argument against memory. It is an argument that memory, implemented well technically, still requires a parallel conversation with users about what is being retained and why—a conversation the industry has not historically been eager to have.

The technical capability is now, apparently, quite accessible. The harder work of what we do with it sits somewhere outside the codelab.

---

Samira Okonkwo-Barnes is Buzzrag's technology policy and regulation correspondent.