Mistral AI launches Moderation API for content management

mistral ai

Mistral AI is launching a new API for content moderation. It allows users to detect unwanted content in texts according to relevant policies.

French AI specialist Mistral AI introduces the Moderation API. This tool allows users to evaluate text content and identify unwanted elements based on different policy criteria. The API, already deployed in chatbot Le Chat, is designed to give customers the flexibility to customize the tool to their own security standards.

The Moderation API is a language model trained to classify text content within nine specific categories of undesirable content. These range from offensive language to the dissemination of personally identifiable information.

The API provides two endpoints: one for analysis of raw text and one for conversational content. This approach allows the model to better assess the context of conversations and analyze only the last message in the dialogue, allowing for more targeted customization.

Multilingual

The API is multilingual and explicitly supports English, Spanish, German, Chinese, Russian, Arabic, French, Italian, Japanese, Korean and Portuguese. Other languages such as Dutch also work in principle, but the system was not specifically trained on these.

read also

Mistral AI launches Moderation API for content management

With this new API, Mistral AI aims to contribute to more secure AI systems and collaboration with the broader research community. The company strives to collaborate with customers on scalable, lightweight content management solutions while working toward improved security in AI use. As a creator of AI models, it is important for Mistral to proactively address abuse so that generative AI continues to exist in a positive light. Previously, Mistral launched an API for developers to refine and test AI models.