Shieldstral: Mistral's Small Open-Weight Model Beats Much Larger Safety Classifiers

According to a new paper, Shieldstral, a 3-billion-parameter model from French AI company Mistral, achieves on common text safety benchmarks the performance of models three times larger. For simultaneous classification of text and images, it claims to set a new best result. Existing safety models often use fixed category systems, which the authors, including Mistral co-founder Guillaume Lample, see as having two weaknesses: public safety datasets are too heterogeneous in their categories, and fixed categories do not adapt to the specific use case. Shieldstral simplifies content moderation to a yes/no question: operators formulate in natural language what should be checked, e.g., "Does this content promote violence?", and the model responds only with yes or no. The system calculates a safety score between zero and one from the probabilities of both answers. The researchers combined around 54.1 million examples from various datasets on safety, harmful content, and manipulation attempts into a unified format. To teach the model finer distinctions, they had another language model rewrite safe texts into unsafe variants, and each example was accompanied by a similar but different category that explicitly should not apply. This taught Shieldstral to distinguish between closely related rules rather than just roughly between safe and unsafe. On combined text benchmarks, Shieldstral reaches an F1 score of 84.9 percent, tying with the about seven times larger GPT-OSS-Safeguard-20B from OpenAI and surpassing Qwen3Guard-8B (84.0), Nemotron-3.5-Safety-4B (83.3), and LlamaGuard-4-12B (69.1). For images and image-text combinations, it scores 83.8 percent, beating OmniGuard-7B (77.6) and LlavaGuard-7B (71.6). On the adaptation benchmark, GPT-OSS-Safeguard-20B leads with 94.1 percent versus Shieldstral's 91.3, but the authors argue their model is more practical because the others produce long intermediate steps, increasing computational costs, while Shieldstral outputs only a single word. Shieldstral is based on Mistral's Ministral-3B with Pixtral image recognition. In a test on the validation set for fine-grained categories, the synthetic category data increased the F1 score by 23.3 percentage points. Shieldstral is available as an open-weights model under the Apache-2.0 license. Safety classifiers are a validation layer that sits before and after the actual language model, checking inputs and outputs. The advantage is that rules can be changed without retraining, but the check runs on every request, so the classifier's size, speed, and cost matter. Shieldstral's core argument is that if operators can formulate their own criteria at runtime, they can tune the filter for their application instead of being subject to a foreign category system.