How Google's VP Builds Products at Billion-User Scale

Google's AI Mode started with 500 people—friends, family, trusted testers. Now it has 75 million daily active users. The gap between those numbers contains everything worth knowing about building products at scale.

Robby Stein, VP of Product for Google Search, spent two years shepherding this transition. He's seen the pattern before. At Instagram, he built Stories and Reels. Before that, he founded two startups and watched both succeed and fail in instructive ways. The arc of his career traces a useful question: how do you know when something works?

The answer, it turns out, has almost nothing to do with launch metrics or press coverage. It starts with complaints turning into compliments.

The Signal in the Hallway

"The first moment is when people went from complaining to me to telling me how useful it was," Stein says. "Most of the people who were talking to me in the hallway or sending me bugs were like, 'Wow, I just realized I'm using AI mode all the time and it's completely natural now.'"

This isn't sentiment. It's data of a particular kind—the only kind that matters at the beginning. Stein describes it as solving a need "really clearly" for a specific group. If you can't get ten people to like something, you won't get a billion. The math doesn't work that way.

Google Search serves billions of users and generates over half of Alphabet's $400 billion in annual revenue. You might think building new features at this scale requires different rules. Stein's experience suggests otherwise. The same principles apply whether you're a founder in a garage or a VP at Mountain View. They just manifest differently.

After the trusted tester phase came Labs—a small US launch where people could opt in without heavy promotion. Thousands, then hundreds of thousands, then millions. Still small relative to Google's total user base. This is where Stein looks for what he calls "the distinct signature" of product-market fit.

The J-Curve Nobody Talks About

Most products lose users immediately after launch. Some percentage tries it once and never returns. Others check back a week later. A smaller group uses it every other day. The retention curve typically drops and then, if you're lucky, flattens.

"This is called J-curve retention or flat retention," Stein explains. "What that shows is that there's some probability per day that the user wants to come back and use it. And then if the product gets better over time, it's actually an intensifying probability."

The curve flattens when the natural churn rate meets the natural return rate. It starts climbing when the product improves faster than users lose interest. That's the moment—not launch, not press coverage, not even growth. Just that quiet inflection where the math changes direction.

Stein saw this pattern at Instagram with Stories. The format solved a specific problem: people wanted to post casually without the performance pressure of permanent grid posts. Stories disappeared after 24 hours. No likes to obsess over. Just ephemeral sharing that stacked neatly above the main feed.

"It was an awesome format that aligned naturally to the use case and job of what people wanted Instagram to be," Stein notes. The principle transfers directly to AI Mode. People have complex questions—multi-part queries about trips with friends who have allergies and dogs and want outdoor seating near their hotel. Traditional search wasn't built for that. AI Mode is.

The Question of Defaults

AI Mode remains a toggle. Users can switch between traditional search and conversational AI. This raises an obvious question: will AI Mode eventually become the default?

Stein's answer reveals something about how Google thinks. Search handles everything from tax forms to carpet images. For many queries, AI adds nothing—it might even slow things down or introduce unnecessary complexity. The system learns when AI helps and when it doesn't.

"You don't need to know like, oh, should I use AI mode? Should I put this in search?" Stein says. "You can just put anything you want right into the search box. You get this little AI preview, and if you click on it, you can have a follow-up and a back and forth in AI mode."

The interface adapts to the query, not the other way around. Simple searches get simple results. Complex questions trigger AI. Users don't need to understand the distinction. The product makes that determination.

This approach reflects a larger pattern in how technology shifts happen. Stein points out that Stories and Reels weren't first-to-market innovations. Snapchat pioneered disappearing content. TikTok popularized short-form video. But formats become standards—like feeds, which now appear everywhere from LinkedIn to DoorDash.

"It becomes less do you have a feed or not?" Stein observes. "In the world now it's like well do you have AI or not? Well kind of everything's going to have some level of AI. It's going to be well how does AI make your product amazing."

What Actually Counts

When measuring success, Stein focuses on depth of engagement over breadth. Instagram could count impressions—each scroll generated a new one. Search requires deliberate action. Someone has to type a question, send it to Google, wait for a response. That's a higher commitment threshold.

"Someone needs to like go proactively, it's like sending a message to a friend and like send a message to Google basically to get information," he says. "It's a pretty high bar actually relative to a consumption product or a consumption app."

The metric that matters: are people using it more intensely over time? Not more people using it a little, but the same people using it more. That pattern—small but intense, then gradually broader—appeared at Instagram too. It's appearing now with AI Mode.

Stein describes one early moment that crystallized his conviction. He asked AI Mode a couple of genuinely hard questions, and it nailed them. "It's kind of like when you hit a golf ball and you just hit a perfect golf shot," he says. "It all comes together and then you don't do that again for a while, but you're kind of like, I know what is possible."

That perfect shot matters more than consistent mediocrity. It proves the system can work, even if it doesn't work reliably yet. The gap between occasionally perfect and consistently good becomes your development roadmap.

The practical lesson isn't complicated. Start small. Watch for complaints turning to compliments. Look for flat retention curves. Protect your core business while testing new things. Move fast by working in details, not just approving from above. These aren't revolutionary insights. They're just rare to see implemented at this scale.

Google's AI Mode went from 500 testers to 75 million daily users. The distance between those numbers isn't about marketing or distribution advantages—though those help. It's about finding that moment when ten people genuinely need what you built, then figuring out if a million more might need it too. The metrics just make that visible.

—Bob Reynolds