Loading...

Content filtering with Azure AI Studio

Content filtering with Azure AI Studio

In alignment with Microsoft’s commitment to help customers use AI products responsibly, Azure OpenAI Service includes a content filtering system that works alongside core models. This system is powered by Azure AI Content Safety and it works by running both the prompt and completion through an ensemble of classification models designed to detect and prevent the output of harmful content.  

This ensemble of models includes: 

  • Multi-class classification models covering four categories of risks - hate, sexual, violence, and self-harm - across four severity levels - safe, low, medium, and high. They have been specifically trained and tested on the following languages: English, German, Japanese, Spanish, French, Italian, Portuguese, and Chinese. However, the service can work in many other languages, but the quality might vary.  
  • Optional binary classifiers for detecting jailbreak risk, existing text, and code in public repositories.  

The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters. However, you can modify the content filters and configure the severity thresholds at resource level, according to your application needs.  

For Azure OpenAI models, only customers who have been approved for modified content filtering have full content filtering control, including configuring content filters at severity level high only or turning off content filters. Apply for modified content filters via this form: Azure OpenAI Limited Access Review: Modified Content Filters and Abuse Monitoring (microsoft.com). Models available through Models as a Service has content filtering enabled by default and can't be configured. 

In this blog post, we are going to see how content filtering works in Azure AI Studio and how you can configure it for your specific requirements. Below, you can also find a step-by-step video tutorial to achieve the same result. 

Testing default content filtering in the Playground 

To test the default content filtering system integrated in Azure OpenAI service, navigate to Azure AI Studio, go into your AI project and open the Chat Playground. If it’s your first time using Azure AI Studio, follow this step-by-step tutorial to set up your workspace and connect an Azure AI service resource to your hub.  

To interact with a model in the Chat Playground, you also need an instance of an OpenAI model deployed in your AI project. You can learn how, by going through this documentation 

For the sake of our example, we deployed a gpt-4 model instance. 

Let’s test the model with a query containing sexual violations: “Is it possible to have sex on the Contoso TrekMasterCamping Chair?”. As a response, we get a message that indicates the input is inappropriate and follows up with the proper use of the chair. 

carlottacaste_0-1726579328814.png

 

  

Let’s now test the same model with a different prompt, which contains sexual content, but uses slang rather than explicit terms: “Which sleeping bags are large enough to fit 2 people to do the deed?” In this case, the model provides a relevant response, without identifying the sexual content in the input message.  

carlottacaste_1-1726579328820.png

 

 

Also, please note that for the sake of these tests we are using a base model, so responses are not grounded in any specific product catalog. However, evaluating the groundedness of the model is out of the scope of this article. 

Creating a content filter in Azure AI Studio 

For specific scenarios, like retail customer care, we might want to add an extra layer of mitigation, to identify inappropriate input content. To achieve this goal, we are going to first create a content filter to lower the threshold for sexual content to the minimum value, so that any sexual content that is triggered for low severity gets blocked. Second, we are going to create a blocklist to account for specific phrases/words that we wish to block and that might not trigger the base filter, because they do not contain explicit terms (e.g. slang). 

To do so, in Azure AI Studio, we can navigate from the left navigation menu to the ‘Content filters’ tab and click on the ‘+ Create content filter’ button. 

carlottacaste_2-1726579328824.png

 

 

In the Create content filter window, fill in the requested parameters, as follows: 

Name: sexual_content_filter 

Connection: <your-openai-connection-name> 

Next, configure the threshold for the sexual filter to Low – default is Medium – for both input and output, as per screenshot below: 

carlottacaste_3-1726579328830.png

 

 

In the Deployment section, select the deployment you want to apply the content filter to. Finally review and create the filter. 

Once your content filter is created and applied to your deployment, you can configure a blocklist of terms you want to filter in your application. This is enabled through the Blocklists feature, that you can configure by navigating to the Blocklists tab. 

carlottacaste_4-1726579328833.png

 

  

Like the content filter creation process, you need to specify a name for the blocklist and the associated Azure OpenAI connection. You can then manually add the terms to be blocked, or add the terms in bulk by importing a csv file. For example, in our case we are going to add the expression ‘do the deed’. 

carlottacaste_5-1726579328835.png

 

 

Once you have your blocklist, you can then add it to the previously created content filter. Navigate back to the content filter configuration and enable the blocklist toggle for both input and output. Once enabled, select the blocklist from the drop-down list. 

carlottacaste_6-1726579328839.png

 

 

Finally, you can head back to the playground and test the same prompts again to check how the model’s behavior changed. This time, your prompts should trigger the content filter we just created. 

carlottacaste_7-1726579328842.png

 

 

And voila, you just configured a custom content filter for the model you are going to use for inference in your application. 

Next steps 

In this blog, we explored the robust content filtering system - built on Azure AI Content Safety - which comes out of the box when using Azure OpenAI service models and can be configured on your applications needs. We encourage you to checkout our Moderate Content and Detect Harm with Azure AI Content Safety Studio self-paced workshop which teaches you how to choose and build a content moderation system in the Azure AI Content Safety Studio. 

Keep learning about practical examples of RAI tools usage in the Azure ecosystem with this dedicated blog series - Responsible AI: from principles to practice. 

Published on:

Learn more
Azure Developer Community Blog articles
Azure Developer Community Blog articles

Azure Developer Community Blog articles

Share post:

Related posts

An introduction to Multi-Agent AI apps with Azure Cosmos DB and Azure OpenAI

Azure Cosmos DB was named by Bloomberg as the no. 1 Database of choice for Retrieval Augmented Generation (RAG) and Large Language Model (LLM)...

1 day ago

Empower Your Projects with AI: A Comprehensive Guide to Azure OpenAI Service

Artificial Intelligence (AI) is revolutionizing the tech world, enabling innovative solutions across industries. Microsoft’s Azure OpenAI Serv...

1 day ago

Compute scaling in Azure Fluid Relay

Explore the journey of identifying and fine-tuning the optimal Kubernetes autoscaling configuration for Azure Fluid Relay. The post Compute sc...

3 days ago

Announcing Azure Cosmos DB Integration with Spring AI and Langchain4J!

We’re continuing to simplify AI app development by integrating Azure Cosmos DB’s cost-effective and scalable vector search directly with Sprin...

8 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy