How prompt moderation works in AI services: good, bad, blocked

Content moderation in AI services uses a system of product policies, input classifiers, and output filters. Different models react differently to the same sensitive prompt: one may refuse, another may point out questionable details, and a third may comply. The article tested GPT-5.5 Instant (ChatGPT), Claude Sonnet 5, and Grok with a borderline erotic request for a comic scene. Only one model agreed to write the text. Moderation relies on product policies defining categories such as threats, hate, violence, self-harm, sexual content, child abuse, and weapons. The input classifier assigns category scores—for example, OpenAI's Moderation API returns 13 labels with subcategories. The classifier may produce false positives (blocking harmless queries like 'how to kill boredom') or false negatives (missing harmful content). Companies set thresholds per category: for self-harm, high recall is prioritized; for less dangerous categories, precision may be more important. Moderation can be implemented with a small LLM (prompt-based) or a specialized classifier. LLM moderators are flexible but costly and susceptible to prompt injection; specialized classifiers are faster but rigid. Best practice involves multiple verification levels, preparing a test set with slang and typos, configuring per-category thresholds, reviewing long dialogues, versioning configurations, and logging decisions for manual review.