ScarfBench: A Benchmark for Evaluating AI Agents in Enterprise Java Framework Migration
ScarfBench is an open benchmark for evaluating AI agents on cross-framework migration tasks in Enterprise Java, covering Spring, Jakarta EE, and Quarkus. It tests whether migrated applications build, deploy, and preserve behavior. Current agents show overconfidence and struggle with environment issues, with configuration dominating migration effort.
IBM
Anthropic




