Karpathy's Obsidian Setup Challenges RAG Orthodoxy

Andrej Karpathy just dropped a Twitter thread that's going to make a lot of developers rethink their infrastructure choices. The former Tesla AI director and OpenAI researcher shared his personal knowledge management system—and it's aggressively simple. No vector databases, no embeddings, no retrieval pipelines. Just Obsidian, markdown files, and Claude.

The timing is pointed. Over the past year, RAG (retrieval-augmented generation) has become the default answer to "how do I feed my LLM more context?" Frameworks like LlamaIndex and LangChain have entire ecosystems built around it. But Karpathy's approach suggests that for many use cases, we've been over-engineering the problem.

The Architecture of Simplicity

Karpathy's system has three components: a "raw" folder for incoming documents, a "wiki" folder for processed content, and Claude Code to navigate it all. That's it.

Here's how it works: Documents—articles, papers, repositories—get dumped into the raw folder, which acts as a staging area. You can do this manually with Obsidian's Web Clipper extension, or let Claude do its own research and pull material directly. From there, Claude generates wiki entries organized by topic, maintaining a master index that tracks all existing wikis and their relationships.

The key insight is in the file structure. Everything lives in plain markdown with clear hierarchies. When you ask Claude a question, it doesn't need to compute embeddings or query a vector database—it just traverses a well-organized folder system. The LLM maintains index files and summaries automatically, creating what Karpathy describes as a self-organizing knowledge base.

"The large language model has been pretty good about auto-maintaining index files and brief summaries of all the documents it reads," Karpathy noted in his post.

This isn't just theoretical. Chase AI, a developer who walked through the implementation in a recent tutorial, demonstrated the system handling multiple wikis across different topics—AI agents, RAG systems, content creation—all navigable through simple markdown links.

What You're Actually Trading

The Obsidian approach makes different tradeoffs than traditional RAG, and understanding them matters.

Traditional RAG excels at scale. When you're dealing with millions of documents, vector similarity search is legitimately faster and cheaper than having an LLM read through even well-organized folder structures. The upfront cost of generating embeddings pays off in retrieval speed.

Karpathy's system optimizes for a different constraint: developer overhead. There's no infrastructure to maintain, no vector database to keep in sync with your source documents, no embedding model to version and monitor. You're trading computational efficiency at massive scale for operational simplicity at smaller scale.

The system also maintains human legibility in a way RAG often doesn't. "It isn't sort of abstracted away in a black box like it is in a RAG system," Chase AI explains in the tutorial. You can open Obsidian and see exactly what the system knows, how it's organized, and where information lives. This transparency has practical value—debugging is easier, and so is manually correcting or supplementing the knowledge base.

But there's a token cost consideration. Every query involves the LLM reading through files and indexes. At high query volumes, that adds up. Traditional RAG systems retrieve only relevant chunks, meaning less context per query and lower per-request costs.

The Scale Question

"Are we trying to scale to millions of documents or are we not?" That's the question Chase AI poses, and it's the right one.

For solo developers, small teams, or even mid-sized companies, the answer is often "not really." A few thousand documents is plenty to support most knowledge bases, research collections, or internal wikis. At that scale, the operational simplicity of markdown files wins.

But the beauty of starting with Obsidian is that you're not locked in. The knowledge is already in markdown—highly portable, easily parseable. If you hit a scale where traditional RAG makes sense, you're migrating structured text files, not rewriting your entire system.

This is pragmatism, not dogma. "Why wouldn't you just start with something like Obsidian?" Chase AI asks. "And if it's clear your scale goes well beyond the bounds of what this thing can handle, then just move into RAG."

What This Says About Tool Selection

Karpathy's setup is a reminder that the newest, most sophisticated tool isn't always the right tool. RAG systems are genuinely useful technology—but they solve problems that many developers don't actually have.

The tech industry has a tendency to optimize for theoretical future scale rather than present reality. We build systems that could handle millions of users when we have dozens. We implement enterprise-grade solutions for side projects. Karpathy's approach suggests that maybe we should be more willing to use simple tools until they break.

There's also something here about legibility and control. When you can see your entire knowledge base as files in a folder, when you can edit a wiki entry in plain text, when the system's behavior is fully transparent—you have a different relationship with your tools. You're using technology rather than trusting it.

The question isn't whether RAG is better than markdown files. It's whether the problems RAG solves are problems you actually have. For more people than the current discourse suggests, the answer might be no.

Marcus Chen-Ramirez is a senior technology correspondent for Buzzrag.