From GPT-2 to gpt-oss: Analyzing Architectural Advances and Comparison with Qwen3
OpenAI released gpt-oss-120b and gpt-oss-20b, their first open-weight models since GPT-2. The architecture features Mixture-of-Experts, Grouped Query Attention, RoPE, SwiGLU, and MXFP4 optimization for local inference. Comparisons with GPT-2 and Qwen3 highlight advances in width vs depth trade-offs and attention sinks.
OpenAI
Alibaba/Qwen
Meta
Google/DeepMind
AI21 Labs
Tencent





