Exo has released a utility enabling the distributed execution of large language models across various devices. This tool optimally allocates resources from computers, smartphones, and single-board devices, including Raspberry Pi, using a peer-to-peer network to dynamically distribute model layers based on available memory and processing power. Select models like LLaMA, Mistral, LlaVA, Qwen, and DeepSeek are supported.
System Requirements and Compatibility
Exo is compatible with devices running Linux, macOS, Android, and iOS, with a version for Windows currently unavailable. The application requires Python 3.12.0 and additional components for machines equipped with Nvidia graphics on Linux. A model needing 16 GB of RAM can be deployed on two laptops with 8 GB each, while DeepSeek R1, requiring 1.3 TB of memory, can theoretically distribute work across a cluster of 170 Raspberry Pi 5 units boasting 8 GB each.
Network Performance and Security Considerations
Network speed and latency impact the overall performance of Exo, and the presence of weaker devices can slow the system. Joint execution of workloads increases security risks. Still, Exo's developers view it as a promising alternative to traditional cloud resources.
