Loss convergence does not mean learning: Tencent Hunyuan ACL 2026 dissects 15.3% "false learning" samples in SFT training
Tencent
Tencent Hunyuan has published a study at ACL 2026 showing that during supervised fine-tuning (SFT), loss convergence does not guarantee that the model has truly learned. They identified that 15.3% of training samples are "false learning" cases where loss drops but the model does not actually acquire the intended knowledge. The research provides a method to detect these samples and suggests improvements for SFT.
Tencent Hunyuan presented research at ACL 2026 analyzing the phenomenon of "false learning" in supervised fine-tuning (SFT). The study reveals that even when the training loss converges, the model may not have truly learned the target knowledge. They found that 15.3% of samples in SFT exhibit this false learning, where loss decreases but the model fails to generalize. The team developed a detection method to identify such samples and proposed techniques to mitigate false learning, improving the reliability of SFT.
- Сокращения
- SFT = Supervised Fine-Tuning — обучение с учителем (дообучение)
- ACL = Association for Computational Linguistics — Ассоциация по вычислительной лингвистике
Source: Tencent Hunyuan (GNews) —
original