When Agents Generate Their Own UI: The Three

I've watched interface paradigms come and go since the desktop metaphor. Most claimed to revolutionize how we interact with software. Most didn't. But when Tyler Slaton from CopilotKit talks about agents generating their own interfaces, he's describing something that sidesteps the usual hype cycle question—not "will this change everything?" but "which flavor do you need?"

Slaton presented at a TypeScript AI demo day in San Francisco, walking through three distinct approaches to generative UI. His company ships the AG-UI protocol and powers 15 million agent interactions monthly for clients that include 10% of the Fortune 500. That's enough production experience to have opinions about what works.

The fundamental problem he's addressing: agentic applications break the request-response paradigm we've relied on since the web went mainstream. These things are long-running, they stream, they delegate to sub-agents, and increasingly they need to show dynamic interfaces that nobody pre-designed. "Agentic applications are really complex," Slaton notes. "They break the kind of traditional request and response paradigm that we're used to."

CopilotKit's answer is AG-UI—the Agent User Interaction protocol that sits alongside MCP (Model Context Protocol) and A2A (Agent-to-Agent). Think of it as completing a trilogy: MCP handles tools and context, A2A enables agent meshes, and AG-UI connects agentic backends to where users actually are. It's a streaming protocol, sending deltas instead of complete payloads, with events for everything from text messages to tool calls to UI updates.

But the interesting part isn't the plumbing. It's the spectrum of control-versus-flexibility that Slaton maps out.

Controlled Generative UI: Your Components, Agent's Props

On the controlled end, you're handing your agent a menu of React components from your existing design system. The agent picks which component to use and fills in the props. Slaton demonstrated this with a pie chart—the agent selected the component, populated it with CSV data, and streamed it into the interface in real time.

The code is straightforward: define a Mastra agent, give it a tool that fetches data, create a component with a Zod schema for parameters, and the agent handles the rest. "It's really simple to write," Slaton says. "You write a component, you give it to your agent, and your agent can show some UI based off of your data."

Advantages: pixel-perfect accuracy, happy designers, consistency with your brand. It's ideal for common paths in your application where you want predictable behavior.

Disadvantages: tight coupling between backend and frontend, linear code growth as you add use cases. Give the agent 25 components and you've got 25 tools polluting the context window. That's not a philosophical problem—it's a token budget problem.

Declarative Generative UI: Schemas and Renderers

The middle ground uses Google's A2UI specification. Here, the agent returns a schema that maps to a catalog of renderers on your frontend. Slaton showed flight booking cards—the agent composed logos and flight data into interactive UI elements, all from a declarative spec.

The key difference: lower coupling. You define surfaces (essentially component templates) and the agent generates schemas that hydrate them. One tool can produce many different UIs, rather than the one-tool-per-component model.

"You can give it a suite of components, and then the agent is going to delegate to some sub agents to go generate that schema, and now you only have one tool to generate UIs as opposed to 20," Slaton explains.

The tradeoff: the LLM now controls layout. That flight booking card will look slightly different every time the agent generates it. Not wildly different—we're talking about variations in arrangement, not design chaos—but enough that your pixel-perfect designers might get twitchy. It's extensible to any rendering framework since it's just JSON, but that flexibility means accepting some visual non-determinism.

Open Generative UI: The Wild West

This is where things get interesting in a "I'm not sure if this is brilliant or terrifying" way. Open generative UI lets the agent write raw HTML, sandboxed in a double-iframe for security, and render it directly in your application.

Slaton demoed a calculator that the agent generated on the fly. Every time he ran it, it looked different. Sometimes neo-brutalist styling worked, sometimes it didn't. But it consistently functioned. "This is where the agent is basically saying, 'I'm going to give you whatever you want,'" he says.

The coupling here is minimal—the backend gets one tool: generate HTML. The agent can create disposable interfaces grounded in your data without you defining anything in advance. Need a one-off visualization for a specific query? The agent builds it.

The obvious concerns: unpredictable styling, difficulty maintaining brand consistency, and the need for iframe sandboxing to prevent session hijacking. This isn't for your core product surfaces. It's for the long tail of user interactions where building a custom component doesn't make economic sense.

Agent State: The Missing Piece

The pattern that makes all three flavors work is shared state between agent and user. Slaton demonstrated with Mastra's working memory concept—a to-do list that both the agent and user could read and write. The agent added items, the user checked them off, and the agent could see those changes.

"That state can be generated by user or it can be generated by an agent," Slaton notes. "It's bidirectional." This is what enables canvas-style applications where the interface becomes a collaborative workspace rather than a command-response terminal.

It's also what enables something Slaton mentioned at the end: self-improving agents trained on human-in-the-loop steering. Every time a user nudges an agent mid-run—"no, not that way"—you're generating training data you didn't have to pay a labeling service for. That's the kind of feedback loop that actually improves over time rather than just accumulating technical debt.

The question isn't whether generative UI will replace traditional interfaces. It's which parts of your application benefit from which level of control. Pixel-perfect for your core workflows. Declarative for common but varied interactions. Open for the weird edge cases that would take more engineering time to pre-build than they're worth.

Slaton's mapping the terrain, not selling a destination. After 25 years of watching interface paradigms promise revolution, that's the kind of technical honesty I can work with.

— Mike Sullivan