28 trillion tokens is not the limit: Chinese AI model Kimi K4 aims higher, GPUs are the bottleneck
Moonshot AI
Moonshot AI is developing a new model, Kimi K4, expected to surpass its predecessor Kimi K3 which was trained on 28.8 trillion tokens. The company's CEO indicated that the model's scale is limited by the availability of GPUs, highlighting hardware constraints as a bottleneck.
Moonshot AI, the developer of the Kimi series of large language models, has announced its next-generation model, Kimi K4. The company's founder and CEO stated that Kimi K4's scale will surpass that of Kimi K3, which was trained on 28.8 trillion tokens. He emphasized that the primary limitation on the model's size is not technological ambition but the availability of graphics processing units (GPUs), pointing to hardware constraints as the key bottleneck.
- Сокращения
- GPU = Graphics Processing Unit — графический процессор
Source: DeepSeek (GNews) —
original