Needle 2 Is a 45M Parameter Model Built for Edge

The AI industry has spent the last few years locked in a size competition, with each new model announcement competing to be the biggest, the most capable, the most expensive to run. Needle 2, from Cactus Compute, is running in the opposite direction. Not as a stunt, but as a deliberate architectural bet on what most real-world AI tasks actually require.

At 45 million parameters, Needle 2 is not trying to write your novel or debug your code. It is trying to hear "vibrate for five seconds" and execute that correctly, offline, on hardware that would struggle to run a modern browser tab. According to Cactus Compute's own documentation, the model is designed for "phones, wearables, smart home and robots." MarkTechPost reports it ships as a 14MB binary and runs a full session in 28MB of RAM.

Those numbers are worth sitting with for a moment. The smallest models that developers typically describe as "very small" run at 500 million parameters or more. Needle 2 is about ten times smaller than that floor. The NeuralNine video demonstrating the model puts it plainly: "45 million is extremely small. This is a tiny large language model."

What It's Actually Built to Do

The distinction Needle 2's creators are drawing is not subtle. This model is not a general-purpose assistant. It is a routing engine. You give it a prompt; it decides which tool to call. That is the entire job.

In the NeuralNine demonstration, the model is set up with a handful of tools: get the current time, check disk usage, check memory, list files. When asked "how much disk space do I have," it calls the disk usage tool. When asked "what's the time," it calls the time tool. It does not attempt to answer from internal knowledge or make things up. It routes. The model reads your docstrings or your tools.json definitions and figures out which function maps to your intent.

This is a genuinely different paradigm from how most people think about language models. It is less "AI assistant" and more "natural language dispatcher." The intelligence required is narrow but real: understanding the semantic relationship between a user's request and a set of function descriptions, then outputting the correct call. Forty-five million parameters, the video argues, is enough to do that reliably.

The deployment story is straightforward by design. Two paths: a Python package (installable via pip as cactus-needle) that lets you define tools directly in code with a decorator pattern, or a standalone binary you download for your target architecture and pair with a tools.json file. The binary approach is what makes the phone use case practical. Python dependencies can be unpredictable on non-standard architectures; a self-contained binary sidesteps that entirely.

The Android Demo

The more interesting part of the NeuralNine video is the Android demonstration, which moves this out of the "interesting benchmark" category and into something you can actually observe working.

Using Termux (a terminal emulator available on Android), the demonstrator runs the Needle 2 binary directly on a phone, paired with a Python script that handles the actual tool execution. The tools defined for this demo are phone-native: text-to-speech, battery status, vibration control, volume adjustment. The model runs locally. No API calls, no network requests, nothing phoning home.

The results are roughly what you would expect from a model with very constrained scope doing exactly what it was designed for. Asked for battery status, it routes to the battery tool and returns the correct reading. Asked to vibrate for five seconds, it routes to the vibration tool. Asked to say something via text-to-speech, it does. The video notes an imprecision with TTS ("it said hello through TTS, but...") that suggests the model's output isn't always perfectly formatted without explicit quotation marks, but the core routing behavior holds.

What is notable here is not that the demo is impressive in a GPT-4 sense. It is that this works at all on a standard Android phone, offline, from a 14MB file. The target isn't developers with beefy machines. It is devices that have been effectively locked out of the AI moment because they can't run anything larger.

The Real Question: What Gets Left Out

Any model with this kind of size constraint is making hard trades. Needle 2 is not ambiguous about this. The video is explicit: "It's not for chatting. It's not for coding. It's for tool calling and structured extraction." The model does not pretend to be something it isn't, which is more than you can say for a lot of AI products.

But it does raise questions the current demonstration doesn't fully answer. Tool routing with four or five simple functions is a well-bounded problem. What happens when you give the model a larger tool library? How does it perform when tools have overlapping descriptions, or when a user's request is genuinely ambiguous between two functions? The NeuralNine demo keeps the tool set small enough that those edge cases don't surface. That is reasonable for an introductory walkthrough, but it is where the real-world reliability of any routing system gets stress-tested.

There is also the question of who builds the actual tools. Needle 2 decides what to call; something else has to handle the execution. On Android with Termux, that means writing Python scripts that interface with Termux's built-in APIs. For a developer building a custom device, it means writing the tooling from scratch. The model lowers the barrier to natural language interfaces; it does not eliminate the engineering work underneath. For developers, that division of labor is probably clean and sensible. For non-technical users hoping to use this directly, it is still a project.

Why This Direction Is Worth Watching

The dominant AI deployment model right now is cloud-dependent: your request goes to a server, inference happens remotely, a response comes back. That architecture has real costs, in latency, in privacy, in the simple requirement of internet connectivity. It also creates a dependency that many use cases cannot tolerate: medical devices, industrial sensors, rural deployments, hardware that lives in environments where connectivity is unreliable or unacceptable.

Needle 2 is a bet that there is a large class of problems where you do not need general intelligence. You need reliable, fast, offline routing. A smart home device that understands "turn off the lights" doesn't need a model with hundreds of billions of parameters. It needs something that can parse intent and call the right function, every time, without sending your voice data to a server in another country.

The privacy implications of that are not nothing. A model that runs entirely on your device, routes locally, and never makes an outbound request is structurally different from cloud AI in ways that matter for anyone thinking carefully about data exposure. There are no logs on a remote server. There is no API key that can be revoked. The inference happens on hardware you control.

Whether 45 million parameters is the right size for the breadth of tool-calling tasks people will actually throw at this remains an open question. But the architectural direction, smaller, local, purpose-built, offline-first, is one that deserves more serious attention than it typically gets in an industry that tends to equate capability with scale.

Rachel "Rach" Kovacs is Buzzrag's cybersecurity and privacy correspondent.