Testing Rails on Autopilot: An AI Agent That Writes Tests Instead of Developers

Mistral AI developed an autonomous agent that generates and improves RSpec tests for Ruby on Rails applications. The agent runs on top of Vibe, Mistral's open-source coding assistant, and operates on five main file types: models, serializers, controllers, mailers, and helpers. Key techniques include context engineering via an AGENTS.md file that provides a step-by-step execution plan and a forced self-review step, specialized SKILLS files for each file type, and custom tools for linting (RuboCop) and coverage checking (SimpleCov). The agent is designed to run in parallel, processing multiple files simultaneously. In an experiment on a repository with 275 source files (half without tests), the agent achieved 100% test pass rate and 100% line coverage, with zero RuboCop violations. The LLM-as-a-judge aggregate score for files that already had tests improved from 0.49 to 0.74. Models scored highest (0.81) and controllers lowest (0.67). The most impactful decision was bundling SimpleCov with RSpec execution as the final step, forcing the agent to run every test it wrote; initially only one third of tests passed on first execution, but the agent self-corrected all failures.