Sand.ai Open-Sources World's First 100B-Parameter MoE Video Generation Model: 114B Parameters, 6B Activated

Sand.ai, a Tsinghua-affiliated team, has open-sourced MAGI-2-preview, the first open-source video generation model with a hundred-billion parameters using Mixture-of-Experts (MoE). The model has 114B total parameters but activates only about 6B per inference, drastically reducing costs. According to current 8-card H100 pricing, generating a 10-second 1080p video costs only 0.5 yuan, about one-tenth of mainstream models. MAGI-2-preview ranks sixth on the AA video generation benchmark, with performance close to top-tier models despite only 6B activated parameters. The model uses a single-stream architecture where text, video, and audio tokens are processed together in a unified Transformer, enabling better audio-visual synchronization. It employs Multi-Head LatentMoE, splitting the 3072-dimensional hidden representation into 12 heads of 256 dimensions each, with independent routing per head, allowing each token to activate 72 small experts across 36 layers, totaling 3072 experts per layer. Sand.ai developed a custom MagiMoE kernel library and Head Parallel strategy to optimize communication and computation for long sequences, using Muon for matrix parameters and AdamW for expert parameters. The data strategy focuses on organizing diverse data rather than filtering it, and training is split into pre-training for broad coverage and post-training for alignment and safety. Open-sourcing this model changes the industry by providing researchers with a public object to study, offering enterprises a private deployment option, and intensifying open-closed competition. Sand.ai, founded by Cao Yue, co-author of Swin Transformer, continues its unconventional path, and its work has been praised by Kai-Fu Lee as the 'DeepSeek of AI video generation'.