ADK vs RAG: When Your AI Should Act vs. Remember

The hardware store metaphor is useful until you realize it [obscures something important: most people building AI systems don't know which aisle they need until they're already halfway down the wrong one.

Katie McDonald from IBM Technology is trying to fix that with a decision framework that's refreshingly direct. In a new video explaining Agent Development Kit (ADK) versus Retrieval Augmented Generation (RAG) architectures, she presents the core question every AI builder faces: Does your system need to do things or know things?

The distinction sounds obvious until you start building. Then it gets messy fast.

The Action Side: ADK

ADK systems are procedural engines. They execute workflows, call tools, follow instructions, make decisions based on logic you've defined. McDonald describes them as systems where "the value comes from reasoning, not the memory."

Think onboarding assistants that guide new employees through paperwork, or workflow automation that routes support tickets based on content and priority. The AI isn't trying to remember facts—it's following a decision tree, applying rules, coordinating tasks.

The appeal is consistency. "AI agents are going to follow the same logic every time," McDonald explains, "which then makes evaluation simple." When you're automating HR processes or IT triage, that predictability matters more than creativity. You want the system to behave the same way on Tuesday as it did on Monday.

ADK shines in environments where the problem is well-defined and the solution is a series of steps. Content transformation, form completion, task coordination—scenarios where the AI needs to perform rather than inform.

What ADK doesn't do well: handle questions it's never seen before, or work with information that changes faster than you can update its logic.

The Recall Side: RAG

RAG systems are fundamentally different. They're built to retrieve information from your documents before generating a response. The model isn't relying on what it learned during training—it's pulling from your PDFs, technical docs, policy manuals, knowledge bases.

"Use RAG when accuracy must come directly from documents," McDonald says, "not from the model's internal guesswork that we've programmed."

This matters in domains where being wrong has consequences. Legal document lookup, medical research assistance, technical support grounded in product manuals—anywhere the answer needs to be verifiable, not plausible.

RAG excels when questions vary widely and unpredictably. "Where is this topic mentioned? What does this report say? Summarize this relevant section." The system doesn't need pre-programmed logic for every possible query. It just needs access to the right documents and the ability to extract relevant information.

The tradeoff: RAG systems are only as good as your document collection and retrieval pipeline. Garbage in, garbage out—except now the garbage is dressed up in fluent prose.

The Uncomfortable Middle

McDonald's framework is clean, but real-world AI projects rarely are. Most production systems need both action and recall, which is why hybrid architectures are becoming standard for anything complex.

Consider a legal co-pilot. It needs RAG to pull relevant case law and contract clauses from a massive document repository. But it also needs ADK-style reasoning to structure a legal argument, coordinate multi-step research workflows, or draft documents that follow firm-specific templates.

Healthcare assistants face similar demands. They need accurate retrieval from medical literature and patient records (RAG territory), but they also need to guide clinical workflows, coordinate care plans, and make protocol-based decisions (ADK territory).

"In hybrid systems, ADK handles the task flow, the logic, the steps, and the decision-making," McDonald explains. "RAG brings in accurate information from your documents. And this gives you a system that is both intelligent and well-informed."

Which sounds great until you're the one managing the complexity of making two different architectural paradigms work together smoothly.

What the Framework Doesn't Tell You

McDonald's decision model—act versus recall—is a helpful starting point. But it sidesteps some harder questions that developers actually wrestle with.

How do you handle the gray area where reasoning requires dynamic knowledge? An IT assistant might need ADK-style workflow coordination, but if it's working with constantly updated infrastructure documentation, it also needs RAG-style retrieval. The boundaries blur.

Then there's the evaluation problem. McDonald notes that ADK systems are easier to evaluate because they follow consistent logic. True enough. But RAG systems introduce different evaluation challenges: How do you measure retrieval quality? How do you know if the system is pulling the right documents, not just relevant ones?

And hybrid systems? You're now evaluating both the reasoning chain and the retrieval quality and how well they integrate. The complexity compounds.

There's also a question the framework doesn't explicitly address: governance. In ADK systems, the logic is yours—you defined the rules, the workflows, the decision points. In RAG systems, you're trusting the model to correctly interpret and extract from your documents. That's a different kind of control, with different failure modes.

The Real Decision

The hardware store metaphor works because it simplifies the choice: tools or reference guides. But most people building AI systems aren't in a hardware store. They're in a warehouse where the aisles keep rearranging themselves and half the labels are in languages they're still learning.

McDonald's framework—"Do you need your AI to act, to know, or to do both?"—is the right starting question. It forces clarity about what problem you're actually solving. But the answer reveals how much complexity you're signing up for, not how to avoid it.

For simple use cases, the choice is straightforward. Content generation with minimal retrieval? ADK. Document search with no task coordination? RAG.

Everything else? You're probably building a hybrid system, whether you planned to or not.

—Dev Kapoor