MIT Introduces SEAL: A New Step Toward Self-Improving AI
MIT
Meta
Alibaba/Qwen
OpenAI
DeepMind
Researchers at MIT have proposed SEAL (Self-Adapting LLMs), a framework enabling large language models to update their own weights via reinforcement learning. The method generates self-edits to produce training data and improves performance on downstream tasks. Experimental results on few-shot learning and knowledge integration show significant gains over baselines.
MIT researchers published a paper titled 'Self-Adapting Language Models,' introducing SEAL, a framework that allows large language models (LLMs) to update their own weights. The model generates synthetic training data through self-editing, which is learned via reinforcement learning. The reward is based on the updated model's downstream performance. Experiments on few-shot learning with Llama-3.2-1B-Instruct achieved a 72.5% adaptation success rate, far above baselines. In knowledge integration with Qwen2.5-7B, SEAL consistently outperformed baselines, sometimes surpassing GPT-4.1 generated data after two iterations. The paper notes limitations including catastrophic forgetting and computational overhead. The timing coincides with growing interest in self-evolving AI, including related work from Sakana AI/UBC, CMU, SJTU, and CUHK/vivo, as well as comments from OpenAI CEO Sam Altman.
- Сокращения
- LLM = Large Language Model — Большая языковая модель
- RL = Reinforcement Learning — Обучение с подкреплением
- GRPO = Group Relative Policy Optimization — Групповая относительная оптимизация политики
- PPO = Proximal Policy Optimization — Проксимальная оптимизация политики
- SFT = Supervised Fine-Tuning — Контролируемая тонкая настройка
- EM = Expectation-Maximization — Ожидание-максимизация
- SQuAD = Stanford Question Answering Dataset — Стэнфордский набор данных вопрос-ответ
Source: Synced —
original