Zidong Taichu Introduces GMC Coreset Pruning Method: Retains Full Multimodal Capability with 80% Fewer Tokens

The Zidong Taichu team at the Chinese Academy of Sciences has introduced GMC (Grounded Message Coreset Pruning), a novel coreset pruning method for vision-language models. Visual tokens are often redundant, causing high GPU memory usage and slow inference. Traditional Top-K token filtering retains only high-scoring tokens, which are concentrated in salient regions, leading to information loss and visual hallucinations. GMC uses a dual-stage architecture: complementary evidence adaptive selection and population transport. The first stage selects key tokens based on their contribution to covering uncovered evidence, considering semantic, visual, and spatial aspects. The second stage aggregates hidden states of deleted tokens into representative ones, preserving information without training. GMC is training-free, requires no task labels, OCR models, or external tools, and works with Qwen2.5-VL and LLaVA-1.5 models. Experiments show that with 80.2% token reduction (from 1296 to 256), GMC retains 97.78% of the full model's average performance on benchmarks like TextVQA, ChartQA, and MME. With 90.1% reduction, it retains 99.11%. On LLaVA-1.5, with 128 and 64 tokens, it retains 99.76% and 99.82% respectively. GMC also reduces visual hallucinations on POPE, AMBER, and HallusionBench. In long-document QA, it achieves 1.258x speedup, reduces KV Cache by 73.94%, and maintains 98.87% answer quality. The paper is available on arXiv.