xAI — AI news & models | AI Digest DNB66

Research 🇺🇸

Beyond Standard LLMs: Linear Attention Hybrids, Text Diffusion, Code World Models, and Small Recursive Transformers

The article by Sebastian Raschka explores alternatives to standard autoregressive transformer LLMs, including linear attention hybrids (e.g., MiniMax-M1, Qwen3-Next, DeepSeek V3.2, Kimi Linear), text diffusion models, code world models, and small recursive transformers. It discusses the revival of linear attention mechanisms and challenges, such as MiniMax reverting to standard attention in its M2 model due to poor performance on reasoning tasks.

Moonshot AI

MiniMax

Alibaba/Qwen

DeepSeek

Google/DeepMind

Mistral

Grok 4.6: Musk's Two-Week Shake-Up in AI

xAI has released an updated model, Grok 4.6. The release marks another step in the company's strategy to accelerate development and rethink approaches to AI training.

xAI

xAI Grok (GNews)27.07 · 14:03