Prompt Proved More Important Than Model: Experiment with Cover Letter Generation

A developer created an app that generates cover letters for job vacancies using a local model (via Ollama) or a cloud API, and decided to test how much the text quality depends on the model. He conducted a blind evaluation with a friend who was looking for a job in IT. The first run with three models (qwen2.5:3b, qwen2.5:7b, llama-3.3-70b) resulted in a sendability of only 20–30%, and the cloud model did not outperform the local one. Analysis revealed common defects: unfilled placeholders, fabricated experience, and Chinese characters. It turned out that the problem was in the prompt itself—it was written in English and lacked restrictions. After rewriting the prompt in Russian, mechanical errors dropped from 25% to 100% cleanliness, but human evaluation remained at zero due to boredom. The friend pointed out that cover letters are scanned diagonally, so the density of hooks at the start of phrases is important. After ten iterations, the prompt explained to the model how letters are read, required a fact-based first line, a short greeting, and a separate paragraph addressing the job fit. With this prompt, local models (gemma-3-4b, qwen-3.5-4b) achieved 100% sendability on seven out of eight letters, while Claude Opus 4.8 and GigaChat-2-Pro achieved 100% with a livelier style. In a Turing test, the friend identified 6 out of 8 generated letters as human-written. Conclusion: the prompt sets the floor, the model raises the ceiling; on weak hardware, a local 4B model already produces decent letters.