Cheap to Generate, Expensive to Verify: The Hidden Cost of AI Outputs

The article discusses the challenge of verifying AI-generated content, emphasizing the asymmetry between low generation cost and high verification cost. Researchers at National Taiwan University found that language models can combine verifiable facts about different people into a single biography of a nonexistent person; standard factuality metrics missed this, and after adjusting for entity ambiguity, scores for four open models dropped by over 10%. A study called Verify with Caution tested five factuality metrics on eleven datasets, finding inconsistencies and errors, especially when supporting text was far from the claim. In human experiments, AI assistance sped up verification but led to over-trust when the model gave convincing but wrong explanations; showing pros and cons mitigated this but offered no significant advantage over search. The article compares verification across domains: code has compilers and tests, mathematics uses formal proof verification (e.g., AlphaProof and AlphaGeometry 2 solved four problems at the 2024 International Mathematical Olympiad), but analytics and strategy are harder to verify. Suggestions include presenting sources first, as in Attribute First, then Generate, and using process rewards (ThinkPRM) to check reasoning steps. The article also mentions NIST's risk-based approach and the European AI Act, which require more documentation and oversight for high-risk systems.