How to reduce LLM costs by 70 times: experience migrating from Claude Sonnet to Qwen 3.5 Flash
Anthropic
Alibaba/Qwen
DeepSeek
JobPath migrated from expensive Claude Sonnet to cheap Qwen 3.5 Flash for job card generation, cutting monthly costs from ~$2000 to ~$30 while quality dropped slightly from 4.48 to 4.06 on a 1-5 scale. The article details selection via evaluation with a blind judge, issues with cheap models (empty responses, type mismatches, field typos), and error-handling lessons including a billing mistake that wasted 16,000 calls in one night.
JobPath initially used Claude Sonnet to structure raw job listings into structured cards, costing about $2000/month. To cut costs, they evaluated alternatives using an eval set of 50 real vacancies, judged blindly by Opus. They chose Qwen 3.5 Flash at $0.0007 per vacancy ($30/month), with a quality drop from 4.48 to 4.06. Cheap models showed discipline issues: empty tool calls with HTTP 200, wrong field types, typos in field names, and salary confusion. They added a protection layer: retries, type coercion, salary cap checks, and partial card fallback. A critical lesson was that billing errors should stop the entire batch immediately; once a dead key caused 16,000 wasted calls overnight. For critical user actions, they use fail-closed with fallback providers. Interactive and flagship features remain on premium models. The average monthly cost dropped to ~$30 with no user complaints.
- Сокращения
- LLM = Large Language Model — большая языковая модель
- API = Application Programming Interface — программный интерфейс приложения
- HTTP = Hypertext Transfer Protocol — протокол передачи гипертекста
Source: Habr — хаб NLP —
original