Needle: A compact AI model for tool calling on consumer devices
A 26-million-parameter AI model delivers real-time function calling at thousands of tokens per second on smartphones and wearables. Its creators argue that massive LLMs are unnecessary for basic tool use.