Four Mac Studios combined into a cluster with 1.5 TB of memory – launched an AI that doesn't fit into a single graphics card

Apple provided the author with a cluster of four Mac Studio computers for testing RDMA over Thunderbolt 5, a new feature in macOS 26.2. The machines were connected using Exo 1.0, an open-source tool for distributing AI tasks over local networks. The total unified memory of the cluster is 1.5 TB, costing about $40,000, but was loaned by Apple. The author compared the M3 Ultra chip in Mac Studio to competitors like Nvidia DGX Spark and AMD AI Max+ 395, showing superior performance in Geekbench, FP64 HPL, and inference tasks. The cluster achieved 3.7 teraflops in HPL, scaling well despite the memory imbalance between nodes. Using Exo with RDMA, the author successfully ran massive models like Kimi K2 Thinking (600+ GB) and DeepSeek V3.1, achieving up to 32 tokens per second with Qwen3 235B. The author noted that while the cluster is powerful, managing macOS clusters is more cumbersome than Linux, and there are challenges with cable management and lack of Thunderbolt 5 switches. The entire system consumes less than 250 watts and runs quietly.