Hugging Face

Latest AI news, models and releases from Hugging Face. ['Diffusers', 'GLM 5.2', 'LeRobot v0.6.0', 'Nunchaku Lite', 'qwen3-32b', 'Qwen/Qwen3-4B', 'Qwen/Qwen3.5-122B-A10B', 'Qwen/Qwen3.5-4B', 'SmolLM', 'SmolLM2-1.7B', 'SmolLM3', 'SmolLM3 3B', 'SmolLM3-3B', 'SmolVLM-256M', 'TinyLlama', 'tokenizers', 'Transformers v4', 'Transformers v5', 'TRL', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', 'Xenova/whisper-tiny.en']

Official site ↗

Open Source 🇺🇸

NVIDIA NeMo AutoModel Accelerates MoE Fine-Tuning 3.7x with Expert Parallelism and DeepEP

NVIDIA has introduced NeMo AutoModel, an open library that builds on Hugging Face Transformers v5, delivering 3.4-3.7x higher training throughput and 29-32% less GPU memory for MoE models via Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels—using the same from_pretrained() API with only an import change.

NVIDIA

Hugging Face

Moonshot AI

DeepSeek

Hugging Face blog27.07 · 10:04

Research 🇷🇺

Token Cramming: Ideal Text Compression Does Not Preserve Meaning – AIRI Study

Researchers at AIRI demonstrated that the token cramming method, which compresses up to 1568 tokens into a single embedding, does not guarantee meaning preservation: under greedy decoding, accuracy drops from 99.96% to 40%. A new method, progressive cramming, was proposed to achieve 100% reconstruction, but it was found that even a perfectly compressed embedding impairs the model's reasoning ability, and the cause lies in the early layers of the transformer.