Moonshot AI

Latest AI news, models and releases from Moonshot AI. ['GLM 5.2', 'GLM-5.2', 'Hermes', 'K2', 'K2.6', 'K3', 'K3-mini', 'K3-MoE', 'Kimi', 'Kimi 2.6', 'Kimi-2.6', 'Kimi 3', 'Kimi 3T', 'Kimi Hosted Agent', 'Kimi k1.5', 'Kimi K1.5', 'kimi-k2', 'Kimi K2', 'kimi-k2-0711-preview', 'Kimi K2.5', 'Kimi-K2.5-1T-A32B', 'Kimi K2.6', 'Kimi K2.7', 'Kimi K2.7 Code', 'Kimi K2 Thinking', 'Kimi K3', 'Kimi-K3', 'Kimi K3.5', 'Kimi K4', 'Kimi K5', 'Kimi Linear', 'Kivine', 'Krea 2', 'Krea 2 Turbo', 'Mamba', 'Mamba-2', 'Mamba-3', 'MiniCPM-RobotManip', 'MiniCPM-RobotTrack', 'Minimax']

Research 🇷🇺

Mamba: The Architecture That Set Out to Kill Transformers

The Mamba architecture, based on selective state spaces (SSM), was introduced as an alternative to transformers, promising linear complexity instead of quadratic. In practice, however, Mamba did not replace transformers but entered hybrid models, where it is combined with attention layers: roughly one attention layer per seven Mamba layers.

Moonshot AIMoonshot AI
Habr — хаб NLP27.07 · 03:03
Fresh news