Qwen2.5-LLM: Expanding the Boundaries of Language Models
Alibaba/Qwen
Alibaba's Qwen team has released the Qwen2.5 series of language models, including seven open-source models ranging from 0.5B to 72B parameters. The models feature a larger pre-training dataset (up to 18 trillion tokens), enhanced knowledge, coding, and math capabilities, and support context lengths up to 128K tokens.
Alibaba's Qwen team announced the Qwen2.5 series language models, which are decoder-only dense models. Seven models are open-sourced: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B parameters. The pre-training dataset expanded from 7 trillion to 18 trillion tokens. Compared to Qwen2, Qwen2.5 shows improvements in MMLU, coding benchmarks (LiveCodeBench, MultiPL-E, MBPP), and math benchmarks (MATH). Qwen2.5-72B outperforms Llama-3-405B on several tasks while using one-fifth the parameters. Additional models Qwen-Plus and Qwen-Turbo are available via Alibaba Cloud Model Studio API. Most models are licensed under Apache 2.0, except Qwen2.5-3B and Qwen2.5-72B which have separate licenses.
Source: Alibaba Qwen —
original