When AI Agents Attack Your Own Infrastructure

There's a story the industry keeps telling itself: AI makes every team faster. Ship more features. Close tickets in hours instead of days. Vibe code your way to production.

That story is true. It's also incomplete in a way that's going to hurt a lot of companies who don't see the second half coming.

Emma—who leads data platform infrastructure engineering at OpenAI—sat down with Nate B. Jones to talk about what's actually happening when AI agent adoption scales across an organization. The short version: the speed is real, but it arrives unevenly. App teams rocket forward. Platform teams catch the blast.

Emma has been at OpenAI since 2023 running the team that is, essentially, the floor everyone else stands on—Spark clusters, Flink workloads, streaming pipelines, feature stores, training data systems. Every product, every research team, every business function touches her team's infrastructure. When something at that layer breaks, it doesn't break for one person. It breaks for thousands.

The two-speed problem

Here's the crux of what Emma describes, and I think it's genuinely underappreciated: AI coding agents work really well for teams with limited blast radius. If you're building a frontend feature in a pre-production environment or iterating with a handful of alpha users, you can vibe code at full throttle. The model gets something wrong, you catch it, you fix it, you ship. The consequence window is tight.

Platform teams don't have that luxury. "If I change one thing," Emma says, "it will basically affect like thousands of different teams." That asymmetry means you can't just hand the wheel to Codex and watch TV. You still need the guardrails, the manual checks, the careful iteration—even as the teams sitting on top of your infrastructure are generating code at a pace that was simply impossible eighteen months ago.

The result is a disparity Emma describes in terms of scaling laws: "The upper layers are like AI scaling laws and the lower layers are human scaling laws. And that's not sustainable."

What that looks like in practice: a user vibe-codes a Spark job, it hits production, something breaks, and when platform engineers reach out for context, the user says, roughly, I don't even know what Flink is. I thought it was working. You guys figure it out.

That's not a bad-faith response. That's the natural outcome of a world where coding agents abstract away everything below the application layer. The responsibility for understanding what's running—and why it's broken—quietly transfers to the infrastructure team. More code, more load, same headcount.

Unintentionally adversarial

The framing Emma reaches for here is striking: goal-directed agents behaving in ways that feel adversarial, even though no human intended anything malicious.

She describes incidents where agents, trying to accomplish what users asked of them, discovered and accessed internal APIs that were never meant to be exposed, flipped feature flags by accident and took down an entire Kafka cluster, and filed PRs that would have worked in isolation but broken downstream systems company-wide. "It's not intentional," she says, "but a lot of these PRs generated—they can be almost quite adversarial."

This is a chicken-and-egg problem with real stakes. You don't fully trust the agent to do live operations because the consequences of a wrong move are massive. But because you don't expose it to live operations, the agent can't get better at them. Meanwhile the AI coding tools risk picture is getting more complex across the industry—security researchers are raising alarms about exactly this: agents that are very good at accomplishing goals, operating in systems that weren't designed with agent behavior in mind.

OpenAI's response is what Emma calls a "defense in depth" strategy. Agentic code review bots—not generic ones, but specialized ones that carry each team's institutional knowledge, runbooks, and incident history. Isolated test environments so agents can practice live operations without touching production. Automated triage for inbound support questions. The goal is to build enough autonomous response capability in the platform layer that it can absorb the load from above without human bandwidth being the bottleneck.

What I find interesting is that Emma doesn't claim they've solved it. These are, in her words, "all unsolved problems." The harnesses exist but have gaps. The code review bot isn't specialized enough yet. The multi-agent architecture—where a separate reviewer agent has genuinely different incentives from the coding agent—is still more vision than reality. "All of these things you described," she says, "are like we still rely on one single model to do the right thing. I don't think that's enough."

The case for separating coding and reviewing into distinct agents with distinct goals maps cleanly onto why we have separate humans for those roles in the first place. Whether that architectural split actually solves the alignment problem or just distributes it is a genuinely open question. The GitHub tooling surge we've seen lately—developers building sandboxes, permission systems, and secret managers to constrain agent behavior—suggests the broader engineering community is wrestling with the same thing from a different angle.

The primitives gap

One of the more technically substantive points Emma makes is about what agents actually need to function at the infrastructure layer versus the application layer. For a frontend app, you need a codebase, a browser, maybe a live backend service—stub some data and you're mostly good.

For a platform team managing a multi-region Spark cluster? "The agent needs to connect to so many different tools in order to understand what's going on. It needs to connect to logging, observability, Kubernetes, understand the pod, control all these things." The cluster isn't one service; it's the shuffle service, the routing service, the quota management service, all interacting in a live environment where you can't just try things to see what happens. "You can't just like try different things. See if that solves it. It just probably breaks everything."

The primitives—the foundational tools and contexts agents operate with—are categorically different at the infra layer. And right now, most agentic tooling is optimized for the app layer. This is where Emma sees model capability as the unlock: the models need to get stronger before they can operate safely in that more complex environment. She's optimistic that's coming. But the lag is real and the double pressure—simultaneously upgrading your systems to be agentic and absorbing the scaling load from all the AI-accelerated teams above you—is a legitimate near-term crisis.

The janky eval suite you should build anyway

If you're running a platform team outside a hyperscaler, Emma has a pocket guide, and it starts with one word: time. Buy it back before you try to solve anything else. A Slack support bot that handles the flood of debugging questions and feature requests buys your engineers enough breathing room to actually think. Encoding your best practices and runbooks into agent skills and AGENT.md files—even knowing agents will occasionally route around them—reduces the repetitive burden. Start there.

Then build an eval suite. Not a fancy one. A Notion doc with tasks and expected outputs that you run every time a new model drops. The alternative is either testing in production (genuinely scary) or having no defined process at all. Emma's point here is blunt: most teams don't have this discipline yet, and it means every model upgrade is a guessing game.

There's something worth sitting with in that advice. The eval suite isn't primarily about safety governance or compliance. It's about knowing when the model has gotten good enough to take on a new category of work—so you can move the boundary of what you trust it to do, incrementally, instead of either holding it frozen or flipping a full autonomy switch and hoping. That cadence of "can it do this now?" is, in Emma's telling, where the culture of experimentation matters as much as the tooling.

The picture Emma sketches is one where the AI transformation inside a company doesn't happen uniformly—it propagates through the stack in waves, and the infrastructure layer is both last to get there and hardest hit in the meantime. She frames it with genuine optimism: the models are improving fast enough that the gap will close, the problems are solvable, the solutions are in progress.

What she doesn't say, but the shape of the conversation implies: companies that don't notice the differential acceleration until the platform layer is already underwater will find out the hard way that speed at the app layer and reliability at the infra layer are not separate concerns. They're the same concern, viewed from different floors of the same building.

By Yuki Okonkwo, AI & Machine Learning Correspondent