Google DeepMind

Latest AI news, models and releases from Google DeepMind. ['3.5 Flash', '3.6 Flash', 'Aletheia', 'AlphaEarth Foundations', 'AlphaEvolve', 'AlphaFold', 'AlphaFold 3', 'AlphaGenome', 'AlphaGo', 'AlphaProof', 'AlphaZero', 'Antigravity 2.0', 'Auto frame', 'Co-Scientist', 'DiffusionGemma', 'ERA', 'Gemini', 'Gemini 2.5 computer use', 'Gemini3', 'Gemini 3.1 Flash-Lite', 'Gemini 3.5', 'Gemini 3.5 Flash', 'Gemini 3.5 Flash Cyber', 'Gemini 3.5 Flash-Lite', 'Gemini 3.5 Pro', 'Gemini 3.6 Flash', 'Gemini 4', 'Gemini Deep Research', 'Gemini Deep Think', 'Gemini for Education', 'Gemini for Science', 'Gemini Omni', 'Gemini Omni Flash', 'Gemini Spark', 'Gemma', 'Gemma 4', 'Gemma 4 12B', 'Gemma 4 26B', 'Gemma 4 26B MoE', 'gemma4:31b']

Research 🇷🇺

6 Lessons for Creating Benchmarks from NeurIPS Tutorial

The authors of SWE-bench and GPQA at NeurIPS discussed key mistakes in creating benchmarks: tyranny of metrics, subjectivity of crowdsourcing, danger of using LLMs as evaluators, and the need for dynamic tests. Main advice — avoid saturated benchmarks, check multimodality, and publish results with confidence intervals.

Google DeepMindGoogle DeepMind
Хабр — Data Mining27.07 · 13:06
Fresh news