Why Vertical AI Is an Org Problem, Not a Model

Half of all generative AI projects were abandoned last year, according to a figure Lovejoy attributes to Gartner—though the specific report and methodology behind that number warrant scrutiny before treating it as settled fact. Gartner published related predictions about AI project failure rates in 2024, and the general trajectory is consistent with what practitioners across the industry are reporting. Whether the precise figure is 50% or 40% or 60% matters less than the underlying pattern: companies are spending serious money on AI deployments that don't survive contact with real workflows.

Chris Lovejoy, a Cambridge-trained physician turned AI product builder, has a specific theory about why. At a recent AI Engineer conference session, he argued that the failure isn't primarily technical—it's structural. "Winning in vertical AI is an organizational problem," he said flatly. The frontier models are sophisticated enough for most enterprise use cases. What organizations are getting wrong is how they position human expertise around those models.

That argument has a particular resonance right now, when the venture community is projecting multi-trillion-dollar labor displacement from AI while also watching a meaningful percentage of their portfolio companies quietly shelve their AI initiatives. The gap between the pitch deck and the production system is, Lovejoy contends, mostly an org chart problem.

Lovejoy's framework divides domain expert roles into three types: the Oracle, the Evaluator, and the Architect. Each maps to a different stage of organizational maturity and a different answer to the question of whether AI quality can be objectively measured.

The Oracle owns the full improvement loop. They assess AI outputs, identify where the system is failing, and fix it themselves—typically through prompt engineering, document injection, or similar mechanisms. This is not a transitional role on the way to something more sophisticated; for certain products, it's the appropriate permanent structure. "There's no objectively perfect meeting note," Lovejoy observed, which is why the model works for Granola, the AI meeting notes company.

Granola's first employee—identified in the talk only by the first name "Joe," described as having a background in writing and journalism; a full name was not provided—wrote the company's original prompts and has remained the primary gatekeeper of AI quality through the company's growth. Granola reportedly crossed a billion-dollar valuation, though Lovejoy didn't specify the date or the funding round that established that figure, and valuations in private markets shift; readers should treat this as approximately current at the time of his talk rather than a verified present-day figure. The point of the example isn't the valuation—it's that a writer with editorial taste has been the structural linchpin of product quality, and that this was a deliberate organizational choice, not a placeholder until they could hire someone more "technical."

What makes the Oracle model sustainable at Granola is that the output domain resists quantification. You cannot write a loss function for "good meeting notes." What you can do is have someone with strong editorial judgment read them, feel where they fall short, and fix the prompts. That judgment is the product.

The Evaluator enters when outputs become measurable and iteration volume exceeds what one person can manage. Here, the domain expert's job shifts from fixing to defining—establishing the metrics, building the review infrastructure, and feeding signal to engineers who make the actual changes. Lovejoy described his own work at Anterior, a startup automating prior authorization decisions for US health insurers, as following this arc. He started as an oracle—reviewing AI decisions with his medical training, updating prompts directly—and transitioned to evaluator as the company scaled, building a clinical review dashboard and hiring clinicians to assess output subsets.

Tandem, a UK-based clinical AI company with which Lovejoy also worked, illustrates the decentralized oracle: a single Roy (a physician turned McKinsey consultant, again identified only by first name in the talk) who initially owned quality control, eventually succeeded by a network of specialists distributed across medical specialties and geographies. Lovejoy described Tandem as "the largest clinical AI product provider in the UK in terms of adoption" in his introductory remarks—a superlative that carries significant qualifier weight. "By adoption" is undefined: adoption by number of NHS trusts? By patient encounters? By clinician users? That distinction matters if you're evaluating the claim, and Buzzrag was unable to independently verify it ahead of publication.

The Architect is the most technically demanding role: a domain expert who designs systems that improve automatically from usage, reducing the need for continuous human intervention in the improvement loop. Lovejoy reached this stage at Anterior when manual iteration by the engineering team could no longer keep pace with the variation in how different insurance organizations interpreted their own prior authorization rules. The AI wasn't wrong in a consistent, fixable way—it was wrong differently depending on the customer, requiring a system that could learn those variations at the edge rather than waiting for an engineer to patch them centrally.

The framework raises a question Lovejoy doesn't fully resolve: what happens when organizations skip the oracle stage because they find it uncomfortable to put a non-engineer in that much authority over the product?

This is not an abstract concern. One of the more consistent patterns in enterprise AI rollouts—visible to anyone who has covered the space closely—is that domain experts get hired into advisory roles with no direct pathway to product modification. They write reports. They attend meetings. They review outputs and file feedback into a Jira board where it ages gracefully. The accountability structure keeps them downstream of the decisions that actually determine what the AI does.

Lovejoy named this dynamic directly: "not fitting them into your organization appropriately, not leveraging them correctly" is, in his telling, one of the three most common failure modes. But the mechanism worth examining is why organizations default to this arrangement. It's not usually incompetence. It's that giving a physician or a journalist or a compliance specialist the authority to rewrite prompts and change product behavior creates governance questions that companies haven't worked out how to answer. Who is liable when the domain expert's prompt change produces a bad clinical output? What review process applies? The oracle model distributes responsibility in ways that legal and compliance teams find genuinely difficult to structure.

That liability question isn't Lovejoy's focus—his frame is operational—but it sits underneath the organizational design problem he's describing. Companies building AI products in regulated domains like healthcare and insurance are navigating a landscape where FDA guidance on AI-enabled medical devices, CMS requirements for prior authorization automation, and state-level insurance regulations all have things to say about how quality is assessed and who is accountable for errors. The oracle model's informal, taste-driven quality loop may be exactly right for Granola. For a prior authorization system making coverage decisions that affect patients' access to care, "someone with good judgment tweaks the prompts" is not a compliance posture that survives regulatory scrutiny.

The oracle-evaluator-architect progression is most useful not as a prescription but as a diagnostic. It forces a specific question: can you actually measure what you care about, and if you can, is your iteration loop fast enough to be useful? Organizations that haven't answered those questions are typically doing something worse than any of the three models—they're iterating on vibes, deploying changes without clear success criteria, and wondering why the product isn't improving.

"The system for incorporating domain insights is more important than the sophistication of your models or your pipelines," Lovejoy said, and this holds up under examination. The companies that have made AI work in high-stakes domains—clinical documentation, legal research, financial analysis—have generally done so by building tight feedback loops between people who understand the output domain and the systems producing that output. The structural question is who owns that loop, what authority they have, and what happens when their judgment is wrong.

That last part is the part the framework doesn't answer. And in regulated industries, it's the part that will eventually determine whether vertical AI delivers on the market projections or quietly joins the 50%—or whatever the real number is.

Samira Barnes covers technology policy and regulation for Buzzrag.