How I Verified That My AI Agent Fleet Is Truly Autonomous
Anthropic
An operator of a fleet of AI agents describes how they built a verification harness to prove their agents' autonomy. They defined four invariants as pure functions over event logs and ran them on real data. The results showed that while tier-2 risks were properly gated by human approval, independent verification was almost never performed, revealing a gap between declared discipline and reality.
An AI agent fleet operator, not a developer by background, set out to prove that their fleet of four machines running negotiations over six weeks was truly autonomous. They defined four invariants (INV-1 to INV-4) as pure functions over event logs to verify behavioral rules such as human approval before committing tier-2 actions and independent verification before commits. Running these checks over 64 real proposals and 317 events, they found INV-1 (human gate for tier-2) passed 100%, but INV-2 (independent verify) only 7.7%, indicating that VERIFY was advisory and not enforced. INV-3 caught a duplicate event storm bug where a hub repeated ACCEPT 17 times, and INV-4 revealed five escalations that were committed without waiting for human resolution. The operator published these results transparently, including a real trace showing both a proper human approval flow and the same trace failing INV-2 because verification was done by the committer. They also discussed limitations, errors, and the open-source code for the evaluation harness.
- Abbreviations
- JSONL = JSON Lines — JSON Lines
- VPS = Virtual Private Server — виртуальный выделенный сервер
- CI = Continuous Integration — непрерывная интеграция
- OS = Operating System — операционная система
Source: Habr — хаб ИИ —
original