Mamba: The Architecture That Set Out to Kill Transformers
The Mamba architecture, based on selective state spaces (SSM), was introduced as an alternative to transformers, promising linear complexity instead of quadratic. In practice, however, Mamba did not replace transformers but entered hybrid models, where it is combined with attention layers: roughly one attention layer per seven Mamba layers.
Moonshot AI






