Building Search Engines From Scratch Still

Here's a question that feels increasingly relevant: why would anyone build a search engine from scratch when you can just dump everything into ChatGPT's context window?

Isaac Flath, who has over a decade in machine learning, has a pretty compelling answer. His new eight-hour course on Retrieval Augmented Generation (RAG) isn't about nostalgia for pre-LLM technology. It's about understanding what actually makes AI systems work—and why the people building production AI applications still need to know how search works at a fundamental level.

"This is what separates the prompt engineers from the actual AI engineers," Flath says in the course intro. And honestly? He might be onto something.

The thing about keyword search that nobody tells you

The course starts by completely ignoring LLMs. Instead, Flath takes students back to the foundations: text preprocessing, TF-IDF (term frequency-inverse document frequency), and inverted indexes. You know, the technology that powered Google back when "Googling" was still a weird verb.

This feels counterintuitive until you consider what keyword search is actually good at. Flath uses a medical example: if a doctor searches for "COVID-19" because they have a patient with COVID-19, they don't want results about "respiratory viruses in general" or "pandemic diseases." They want exact matches. Semantic similarity isn't helpful here—precision is.

"Nothing beats keyword search" when you need exact terms, Flath explains. "On the other hand, if you're searching for 'Alien movie,' you would want something that says 'Invaders from Mars' even though it doesn't specifically say the word alien in it. So you need both."

That "both" is the entire thesis of modern search: hybrid systems that combine old-school keyword matching with new-school semantic understanding.

What the course actually teaches

The curriculum is structured around building a real project—a Netflix-like streaming service called Hoopla. Students implement the search bar from scratch, which means writing actual Python code for everything from tokenization to multimodal embeddings.

The progression is deliberate. You start with basic keyword search using algorithms like BM25, then move to semantic search using embeddings and vector operations. The course covers chunking strategies (how you slice data matters more than people think), hybrid search methods that blend keyword and vector scores, and re-ranking techniques to surface the best results.

Then comes the RAG piece: combining retrieved context with LLM APIs to generate grounded, accurate answers. The course uses Gemini's API, mostly on the free tier, though Flath notes that Google's been changing their billing structure enough that you might need a credit card for a dollar or two of credits.

The advanced modules get into agentic workflows—where autonomous agents refine their own queries—and multimodal RAG, extending search to understand images alongside text.

The pedagogical choice that matters

What's interesting is how Flath structures the learning. The course isn't meant to be binge-watched. He explicitly recommends implementing each concept yourself before watching his solution.

"Make a boot.dev account," he suggests. "It's free. And actually go ahead of Isaac. Implement the search logic yourself and only then watch him go through that same lesson."

This matters because understanding search algorithms isn't about memorizing formulas—it's about building intuition for why different approaches work in different contexts. When would you use cosine similarity versus dot product? Why does document length normalization improve results? You can't really internalize these trade-offs without wrestling with the implementation yourself.

The course includes automated testing through Boot.dev's CLI, which addresses a real problem in AI development: "When you're working with things in the AI realm, it's very easy to get a number that looks plausible," Flath notes. "How would you know if a very large decimal is the right decimal or not?"

Why this matters beyond the course itself

There's a broader tension here about what AI engineering actually means in 2025. We're in this weird moment where "AI engineer" can describe someone who chains together API calls or someone who understands vector spaces and embedding models at a mathematical level.

Flath's course assumes the gap between these definitions matters. Companies building production AI systems need people who understand retrieval at a deep level because RAG is everywhere—in coding agents, customer service bots, enterprise search, document analysis tools.

Cursor, the AI coding assistant, uses semantic search for large-scale codebases. That's not a ChatGPT wrapper—it's sophisticated retrieval infrastructure that someone had to build and tune.

The question is whether understanding these fundamentals will remain valuable as AI systems get more capable. Maybe future LLMs will handle context so well that retrieval becomes unnecessary. Or maybe—more likely—the complexity just shifts to different problems: optimizing retrieval for specific domains, balancing latency with accuracy, handling multi-turn conversations where context accumulates.

The open question

Flath's course represents a particular bet: that understanding how search works from first principles will remain valuable even as the technology evolves. The course teaches TF-IDF alongside transformer embeddings, BM25 alongside neural re-rankers.

Whether that bet pays off depends on what kind of AI engineering work actually needs humans in five years. If it's mostly integration and prompt design, maybe the fundamentals don't matter. But if it's building systems that are reliable, explainable, and tailored to specific use cases—the kind of work that actually ships and generates revenue—then yeah, you probably need to know why your search is returning weird results at 3am when everything breaks.

The course is free to watch, with paid features for interactive assignments and certificates. Eight hours is a significant commitment for something that might feel like learning assembly language in the era of high-level frameworks. But then again, the people who understand assembly language are often the ones who can debug what everyone else just accepts as mysterious.