Liquid AI recently introduced LFM2-24B-A2B, a specialized model designed for efficient local tool dispatch with minimal latency. In addition, they launched LocalCowork, a desktop agent application available on their Liquid4All GitHub Cookbook, enabling the execution of enterprise workflows entirely on-device to enhance privacy in sensitive environments.
The architecture and serving configuration of LFM2-24B-A2B focus on achieving low-latency execution on consumer hardware by employing a Sparse Mixture-of-Experts (MoE) design. Despite containing 24 billion parameters, the model only activates around 2 billion parameters per token during inference, reducing computational overhead significantly.
The model was rigorously tested using Apple M4 Max hardware with 36 GB unified memory and 32 GPU cores. It utilizes the llama-server for serving with flash attention enabled, quantized in Q4_K_M GGUF format, and has a memory footprint of approximately 14.5 GB RAM. Hyperparameters such as temperature, top_p, and max_tokens are optimized for deterministic outputs.
LocalCowork, an offline desktop AI agent, leverages the Model Context Protocol (MCP) to execute pre-built tools without relying on cloud APIs, ensuring data privacy by logging every action to a local audit trail. It offers a range of tools across MCP servers for tasks like filesystem operations, OCR, and security scanning, with a focus on reliability and accuracy.
The performance benchmarks of the model showcase an average latency of around 385 ms per tool-selection response, making it ideal for interactive applications. The accuracy of single-step executions is reported at 80%, while multi-step chains achieve a 26% end-to-end completion rate.
Key takeaways from the release include the emphasis on privacy-first local execution, efficient MoE architecture, sub-second latency on consumer hardware, standardized MCP tool integration, and the balance between single-step accuracy and multi-step limitations.
For more details and technical information, visit the repository and explore the provided resources. Stay updated by following Liquid AI on Twitter, joining their ML SubReddit, and subscribing to their newsletter. Additionally, consider joining their Telegram channel for more updates and discussions.
Be the first to comment