Google/DeepMind

Latest AI news, models and releases from Google/DeepMind. ['A2UI v0.9', 'AI Co-Scientist', 'AI Evaluator', 'AI Mode', 'AI Overviews', 'Alexa', 'AlphaEvolve', 'AlphaFold', 'AlphaGenome', 'AlphaQubit', 'Antigravity', 'Auto frame', 'BERT', 'BioBERT', 'Chinchilla', 'ClinicalBERT', 'Computational Discovery', 'Confidential GKE Nodes', 'Co-Scientist', 'Empirical Research Assistance', 'Era', 'ERA', 'Executive LLM', 'Farmscapes 2020', 'Flood Hub', 'frontier AI', 'Frozen v2', 'FunctionGemma', 'Gemini', 'Gemini 1.5 Flash', 'Gemini-1.5-pro', 'Gemini 1.5 Pro', 'Gemini 2.0 Flash', 'Gemini 2.5', 'Gemini 2.5 Flash', 'Gemini-2.5 Flash', 'Gemini-2.5-Flash', 'gemini-2.5-flash-image', 'Gemini 2.5 Flash Lite', 'Gemini 2.5 Pro']

Research 🇺🇸

A Visual Guide to Attention Variants in Modern LLMs

This article provides a visual overview of attention mechanisms in modern large language models, covering from standard multi-head attention (MHA) to grouped-query attention (GQA), multi-head latent attention (MLA), sparse attention, and hybrid architectures. It is accompanied by an LLM architecture gallery with 45 entries and poster versions.

OpenAIOpenAI Allen Institute for AIAllen Institute for AI MetaMeta Alibaba/QwenAlibaba/Qwen Google/DeepMindGoogle/DeepMind MistralMistral Hugging FaceHugging Face CohereCohere
Sebastian Raschka28.07 · 15:05
Fresh news