Mamba, a new State Space Model, offers a linear-time alternative to Transformers, promising comparable performance with up to 5x faster inference and handling of million-token sequences. It removes the quadratic bottleneck of attention using a selective state-space mechanism.
Princeton University (Sengupta Lab)
Researchers from Stanford University, Princeton University, and Adobe Research have developed a new video world model architecture called LSSVWM that uses State-Space Models to extend temporal memory efficiently. The model addresses the quadratic computational cost of attention mechanisms by employing block-wise SSM scanning and dense local attention, enabling long-term memory for tasks like spatial retrieval and reasoning in Minecraft and Memory Maze environments.
Princeton University (Sengupta Lab)