ABBEL: Training LLMs to Update Beliefs for Efficient Long-Term Interaction

ABBEL (Autoencoder Belief Bottleneck for Efficient Learning) addresses the problem of LLMs struggling with long interactions where full context history cannot fit in the context window. Instead of traditional recursive summarization, ABBEL formulates summaries as belief states that the model updates periodically. Belief grading adds an auxiliary RL task that rewards beliefs based on how well they can reconstruct recent observations. In the CollabBench collaborative coding environment, ABBEL with reconstruction-based belief grading achieved a test pass rate of 0.48 (vs. 0.52 for full context) and success rate of 0.36 (vs. 0.39), while using 6.01 peak tokens vs 14.08 for full context, and requiring only 50 training steps vs 100. In the Combination Lock task, domain-knowledge belief grading allowed ABBEL to approach or exceed full-context performance. In multi-objective QA, a Peak Belief Length Penalty reduced memory usage with minimal performance loss. The paper was authored by Lidayan et al. from Berkeley AI.