Boost.URL: C++ URL Parsing Done Right

I wrote my first CGI script in 1996. The URL encoding was my problem to solve, because in 1996 it was everyone's problem to solve — RFC 3986 wouldn't be finalized until 2005. You figured it out by reading the source of whatever working script you could find, copying the percent-encoding logic, and then discovering six months later that you'd missed the edge case where someone put a + sign in a query parameter and your form processor was silently eating spaces. The standard didn't exist yet. The bugs were load-bearing.

Which is the long way of saying: I have opinions about URL parsing libraries. Richard Thomson's recent Utah C++ Programmers talk on Boost.URL hit a nerve.

Stop Writing the Same Bug

Thomson's setup is blunt and accurate: "If you try to write string operations or regexes to try and extract out the different pieces of a URL, it just gets tedious and you always end up encountering new edge conditions and new edge cases as you proceed along."

That's not a library sales pitch. That's a description of what every C++ developer has done, repeatedly, across careers. The URL looks like a string. You treat it like a string. You write a little split function. You handle the scheme. You handle the host. You forget that the query string is technically freeform text and the key-value pairs are a convention, not a standard. Then someone sends you a URL with a fragment that contains encoded characters and your little parser quietly produces garbage. You fix it. Three months later it happens again with a different edge case.

Boost.URL's pitch is that it implements RFC 3986 correctly so you don't have to. The library is C++11, which matters because a significant portion of the embedded world — the IoT devices, the microcontrollers, the firmware that powers the thing your HVAC tech just replaced — is not running a C++20 toolchain. Some of that hardware is constrained enough that heap allocation is a genuine concern; on certain bare-metal microcontroller targets, there may be no heap at all, though this varies considerably across embedded environments. Thomson's point is that the library was designed with this in mind from the start, not retrofitted.

What the Standard Actually Says (And What It Doesn't)

The library's visual reference card — which Thomson spends some time on and is genuinely worth a look — maps every component of a URL to the methods that access or modify it. Scheme, user info, host, port, path, query, fragment. The thing that stuck with me is Thomson's note about the query string: "The internet standard for a URL just says that this entire thing is a query string... the fact that it's often seen as a set of named value pairs, that's just a convention. There's nothing in the standard that mandates that there be named parameters here."

I know this. You probably know this. But it's the kind of thing that gets forgotten every single time someone writes a URL parser from scratch, because every URL you encounter in practice uses the key=value&key=value convention. Boost.URL supports iterating the query as named parameters, but it doesn't pretend that's a requirement of the format. That's the kind of design fidelity that separates a library built on the spec from one built on "what I've seen URLs look like."

The url_view / url split mirrors the standard library's string_view / string relationship: a view is immutable and borrows storage it doesn't own; a url object is a container you can modify. The obvious trap Thomson flags — passing a string temporary to something that returns a view — is exactly the kind of lifetime bug that looks harmless in a demo and bites you in production at 2 a.m. The library can't prevent that class of mistake, but at least the mental model is familiar to anyone who's internalized the string/string_view distinction.

Where I'd want more information before committing to this in a production codebase: how does the library behave with malformed URLs that are, nonetheless, what the real world hands you? RFC 3986 compliance is the floor, not the ceiling. Web clients have been parsing technically-invalid URLs for thirty years in the name of compatibility. Thomson doesn't address this, which is fair for a tutorial talk — but it's the question I'd be researching before I dropped this into anything that has to process user-supplied URLs from the open internet.

The AI Workflow, Which Is the More Interesting Story

Thomson used ChatGPT to figure out what to present and what examples to build — a reasonable use of the tool. Then he used what he refers to in the talk as "Codeex" to actually generate the sample code. A note here: OpenAI deprecated the Codex API in March 2023, so Thomson may have been using a different product under a similar name, or referring to one of the code-generation features in ChatGPT itself. The transcript isn't specific enough to be certain, and it's worth flagging because the distinction matters when you're evaluating AI coding tools.

What Thomson describes, though, is instructive regardless of which tool he was using. The AI's first pass "just dumped everything inside main" with no unit tests. Thomson then had to tell it to move the business logic into a library and add tests. He characterizes the final result as "not any different than what I've written by hand."

Let me sit with that for a second.

The AI produced a working first draft that required a senior engineer to identify the structural problems, specify the correct architecture, and supervise the refactoring. Thomson is experienced enough to know that business logic in main with no tests is not acceptable code — so he fixed it. What would have happened if a less experienced developer had taken that first dump from the AI and shipped it? Nothing good. The code would have "worked," which is the most dangerous kind of wrong.

This isn't an argument against AI coding tools. It's an observation about what Thomson's workflow actually demonstrates: the AI is a fast typist. The engineering judgment — what belongs in a library, what needs a test harness, what the architecture should look like — still came from Thomson. The AI didn't know that. It needed to be told, explicitly, twice. The tool is useful in proportion to the expertise of the person directing it. That's not a revelation, but it keeps being the thing the AI hype train rolls past without stopping.

Thomson, to his credit, seems clear-eyed about this. He's not claiming Codex replaced anything — he's describing a workflow where he remained the adult in the room and used the AI to handle keystrokes he would have written anyway. That's a genuinely reasonable use. It's just not the thing the breathless proclamations are promising.

The Problem This Library Actually Solves

Here's where I land after watching this talk: Boost.URL is a well-designed library that solves a real problem. RFC 3986 compliance, allocation control, C++11 support, and the url_view/url split are all legitimate design decisions that reflect experience with where URL parsing actually goes wrong. Thomson's proxy-rewriting example — path-prefix matching that preserves query strings and fragments through the rewrite — is a concrete demonstration of the kind of work the library handles cleanly.

But the library solves the problem correctly only if you're disciplined enough to reach for it. The developers who have been writing bespoke URL parsers for years aren't doing it because they enjoy it — they're doing it because they started with std::string and kept adding cases until the thing mostly worked, and now it's woven into the codebase and nobody wants to touch it. Boost.URL doesn't fix that. It's a better path for new code, or for projects where someone has the mandate and the motivation to refactor.

I've seen enough "we should have used a proper library for this" conversations at Microsoft and Amazon to know that the gap between "this exists and is good" and "we are actually using this" is where most technical debt lives. Boost.URL is good. Getting it into the codebases that need it most is the problem the library can't solve for you.

Richard Thomson's example code is available on GitHub. The Utah C++ Programmers group meets monthly; their archive of past topics is worth browsing if you're a C++ developer who likes this level of depth.

Mike Sullivan is a technology correspondent for Buzzrag. Former Microsoft, former Amazon, current designated adult in the room.