First Desktop Operation Agent to Break 90% Success Rate: From Model Stacking to Engineering, Shiai Agent Tops OSWorld
The Shiai Agent, developed by a Chinese team, achieved a 90.2% success rate on the OSWorld benchmark, ranking first overall and in the Agentic Framework category. This breaks the global record for Computer-Use Agent evaluation, making it the first agent to surpass 90% accuracy.
On July 27, OSWorld updated its leaderboard. Shiai Agent achieved a 90.2% overall success rate with 325.59 points, topping both the overall and Agentic Framework sub-rankings. It set a new global record for Computer-Use Agent public evaluation, becoming the first desktop operation agent to exceed 90% success. OSWorld is a benchmark that tests agents in real Ubuntu environments across 361 tasks, evaluating perception, long-horizon planning, multi-software coordination, and system operations. Previously, GPT, Gemini, and Claude models were tested on this benchmark. Industry progress rose from 12.2% in 2024 to 72.6% in 2025, and 83.6% in May 2026. Shiai Agent's 90.2% breakthrough overcame technical bottlenecks. The agent excels in cross-app coordination (93 tasks), GIMP image processing (92.3% success on 26 tasks), and system-level operations (24 tasks with zero errors). The team attributes success to engineering over model tuning. Shiai Intelligent, founded in 2018, built a multimodal Harness system that uses APIs when available and GUI operations otherwise, trained on enterprise-scale data. The company plans to improve reliability to 99.99%, enhance dynamic environment adaptation, and deploy edge-cloud hybrid scheduling for production use.
- Сокращения
- OSWorld = Operating System World — операционная система World
- API = Application Programming Interface — программный интерфейс приложения
- GUI = Graphical User Interface — графический интерфейс пользователя
Source: InfoQ 中国 —
original