Claude Can Now Control Your Computer. Here's What

Your AI assistant can now physically operate your computer. Not through APIs or integrations—like, actually moving your cursor around, clicking buttons, taking screenshots to see what it's doing. Anthropic just shipped Computer Use for Claude Code, and automation YouTuber Nate Herk put it through its paces.

The feature does exactly what it sounds like: Claude can control your mouse and keyboard to interact with any application on your desktop. It's currently Mac-only (Windows coming "in just a few weeks") and lives in research preview, which is tech company speak for "this is real but we're still figuring it out."

What makes this different from existing automation tools is that Claude doesn't need special permissions or APIs for every app. If you can click it, Claude can click it. The AI takes screenshots to understand interfaces, identifies the right buttons to press, and executes actions like a very patient intern who reads every tooltip.

How It Actually Works (and Looks)

Herk demonstrated the feature by having Claude literally start his video recording. He told Claude to "open up OBS and start recording," and the AI navigated his desktop, opened the app, took a screenshot to locate the record button, and clicked it. The video you're reading about? Started by an AI.

The more practical demo involved file management. Herk asked Claude to find a specific PDF in his downloads folder and send it via ClickUp as a direct message. What followed was fascinating if you care about how AI interprets visual interfaces (I do, obviously).

Claude first had to get permission to access Finder and ClickUp—a one-time-per-session security check. Then it navigated ClickUp's interface by repeatedly taking screenshots and identifying UI elements. "You can see what's going on on the right hand side is it's continuously taking screenshots and then it's going to press different buttons and figure out what to do," Herk explained, narrating the AI's journey through the app like nature documentary.

The AI even zoomed in on specific interface elements to be absolutely sure it was clicking the right thing. Found the attachment button. Searched for the correct file. Attached it. Asked for confirmation before sending (good instinct, Claude). Sent the message.

It's not fast—definitely slower than doing it yourself if you're already at your computer. But that's not really the point.

The Remote Control Piece

The feature gets genuinely interesting when paired with Claude's Dispatch functionality, which lets you control Claude from your phone. This combination means you can text your computer and have it do things while you're nowhere near it.

Herk's demo: texting Claude from his phone to calculate a specific multiplication problem using the Calculator app, then store the result in Notes. From his phone, he watched Claude open the calculator on his Mac, clear the existing numbers, input the new calculation, open Notes, and save the result.

"As long as my computer is awake, which we can actually turn on natively with Claude dispatch, then I can be anywhere from my phone and I can use Claude dispatch to start a new session, use computer use, and do whatever I need," Herk said.

The use case he's describing—being able to reach into your local machine from anywhere to grab files, run reports, or trigger workflows—is legitimately useful for certain work patterns. It's also the kind of capability that makes security professionals nervous, but we'll get to that.

What Doesn't Work (Yet)

The biggest limitation is intentional: Claude can't control web browsers for security reasons. It can open Safari or Chrome, but it has read-only access. No clicking, no typing, no form filling.

When Herk asked Claude to search for cute dog images in Safari and save them, the AI opened the browser but then basically shrugged: "Safari is restricted to read-only access on the system. However, I can use Google Chrome extension to do this."

For actual browser automation, you'd still need to use Chrome extensions or tools like Playwright. Which is probably the right call—giving an AI carte blanche to click around the web on your behalf opens up some obvious attack vectors.

The other limitations are more about polish than design: it's Mac-only for now, it can feel slow or buggy (research preview!), and it's only available in the desktop app for Claude Code and Claude Co-work. Not available on Teams or Enterprise plans yet.

You also have to be explicit in your prompting. If you're not clear that you want to use "computer use" specifically, Claude might try to accomplish tasks through other methods, like browser extensions.

The Interesting Questions This Raises

What Herk's demos show—and what he doesn't really dig into—is how this changes the economics of certain kinds of work. Legacy apps that never built APIs? Claude can still use them. Manual QA testing? Claude can click through your app looking for bugs. Weekly reports that require pulling data from three different desktop apps? Claude can do that while you're asleep.

The scheduled tasks angle is particularly interesting. You could theoretically have Claude check your email every morning, scan for certain types of messages, and take actions based on what it finds. Or pull reports every Friday at 5pm. Or test a development build every time you push code.

Herk frames this as "pretty much going to be as if you have a real human working on your laptop," which is both the promise and the thing that feels slightly off about this framing. It's not really like having a human on your laptop. It's like having a very literal, very patient assistant who can only see through screenshots and can only interact through the same interface you use.

Which raises questions about what happens when the interface changes. Or when something goes slightly wrong. Or when Claude needs to make a judgment call about ambiguous UI.

The research preview status matters here. Anthropic is clearly testing this in controlled conditions before wider release. The feature racked up 4 million views on its announcement post in just two hours, according to Herk—there's huge interest in this capability.

But interest and readiness aren't the same thing. The fact that browser control is explicitly locked down suggests Anthropic is thinking hard about the security implications. The fact that it asks for confirmation before sending messages suggests they're thinking about accidental actions.

What we're watching is Anthropic stress-testing a feature that could fundamentally change how people interact with their computers—or could end up as a neat party trick that's too slow and buggy for real work. The pace of shipping is notable though. As Herk points out, "Anthropic is shipping a new feature like every day," which signals both confidence and competitive pressure.

The question isn't really whether AI agents controlling our computers is technically possible—Herk's demos prove it works right now, even in preview. The question is whether the interface (screenshots + clicking) is the right abstraction, whether the security model holds up at scale, and whether people actually want this versus more structured automation tools.

We won't know the answer until more people get their hands on it. But the fact that you can already text your computer from across town and have it do work for you? That's not a demo anymore. That's just... available. If you have a Mac and Claude Pro.

—Yuki Okonkwo