General availability of Azure WAF Bot Manager1.1 Ruleset

General availability of Azure WAF Bot Manager1.1 Ruleset

Today, we are launching the general availability of Bot Manager1.1 ruleset in Azure WAF integrated with Azure Front Door.

Bot Manager1.1 extends all the rules in the existing Bot Manager1.0 ruleset and adds multiple new rules to provide comprehensive bot management capabilities to web applications. The new capabilities introduced in this ruleset include new Goodbots rules and a new Badbots rule.


The main value prop of the new ruleset is to reduce false positives in good bot detections and increase true positives in malicious bot detections.


Benefits of the new rules in the Goodbots rule group:


  1. Improving SEO rankings due to good bots crawling websites and reducing FP (false positive) seen by customers.

Customer websites are crawled by good bots which results in increased SEO (search engine optimization) rankings. With Bot Manager 1.1 ruleset, a comprehensive set of rules are added to the Goodbots rule group which allows a larger set of legitimate published bots. Examples of such Goodbots include Googlebot, Bingbot etc.

As a real-life scenario, we encountered an issue with the Bot Manager1.0 ruleset where certain Goodbots were absent, leading to blocked requests to web applications. For example, a valid Google crawler bot was getting blocked by the Bot Manager1.0 100200 rule, which resulted in lower SEO rankings for the customer and eventually disappearing from the SEO rankings. As a workaround, the customer disabled rule 100200 which brought their SEO rankings up but resulted in lowered protection from true malicious bots that have falsified their identities. Prior to implementing the Bot Manager1.1 ruleset, the only other alternative to allow legitimate crawlers was to add custom rules to allowlist their IP addresses. However, this approach posed challenges due to the dynamic nature of crawler IPs, which change frequently.

With the new updates to Bot Manager1.1, a comprehensive list of good bot IPs is added to the existing rule 200100 which results in lower false positive detections by the Bot Manager ruleset. The 200100 rule from Bot Manager1.0 ruleset is now revamped to only include good bots in the search engine crawler category.

  1. Bringing clarity to the Goodbots rule group

With Bot Manager 1.1 ruleset, many new verified good bot rules have been added that target different categories of good bots. These new rules include the link checker, social media, content fetchers, feed fetcher and advertising bots. Additional bots that don’t fit into any particular category are added to 200200 as verified miscellaneous bots.  This empowers customers to have granular control over their WAF policy. For example, if a customer does not wish to have social media bots crawling their sites, they can achieve this by changing the action associated with the social media rule.



Benefits of the new rule in the Badbots rule group:

Today customers see malicious bots perpetuating many malicious attacks. Examples includes:

  • Scraping websites and spreading dis-information, executing targeted phishing attacks and social engineering attacks.
  • Spamming customer websites with form submission pages.
  • Manipulating rankings of content tooling websites’ analytics pages.
  • Launching denial-of-inventory attacks.
  • and many others.


The new Bot Manager1.1 ruleset incorporates a novel rule, Bot100300, complemented by the existing rules in the Badbots rule group rules, effectively mitigates malicious bot attacks.



Let’s take a closer look at the Bot Manager1.1 ruleset:

Goodbots rule group

The following screenshot describes the new good bot rules added to the new ruleset



The details of all the good bot rules are given below:

Good bot rule ID






Verified search engine crawlers

Search engine bots- Google, Yahoo, Bing etc.



Verified misc bots

All verified good bots that do not fit into any specific good bot category



Verified link checkers

Link checker bots give information about a link or a domain name. It returns the screenshot or metadata about the link that the user trying to get to.



Verified social media bots

Social media bots – Facebookbot, LinkedInbot etc.



Verified content fetchers

Content fetcher bots retrieve content for websites on desktop, in-app browsers, mobile apps etc.



Verified feed fetchers

Feed fetcher bots periodically refresh feeds like the RSS feeds requested by users.



Verified Advertising bots

Advertising bots – GoogleAds, BingAds etc.


The default action for all the new good bot rules is ‘allow’ by default but it is possible to change them to any of the supported actions.



Badbots rule group

The following screenshot describes the new bad bot rule, rule Bot100300.



The bots detected by the Bot100300 rule includes risky IPs that are based on their high-risk score detected by threat intelligence. These IPs differ from the Bot100100 rule, which identifies verified malicious IPs detected by Microsoft Threat Intelligence and are subject to a different set of criteria, including their Tactics, Techniques, and Procedures (TTPs), any related lateral threat activity seen by the IP, and other indicators of compromise.


The default action for Bot100300 is ‘block’ by default but it is possible to change it to any of the supported actions.



JavaScript(JS) challenge mitigation in Bot Manager1.1 ruleset


The ruleset the newly released JS challenge on AFD WAF as an action to any of the Bot Manager rules. The JS challenge is an addition to existing actions and provides all the feature benefits of the invisible web challenge to protect web applications.



JS challenge action is available in Bot Manager 1.0 as well for backward compatibility.



How to enable Bot Manager1.1 ruleset 

The Bot Manager1.1 ruleset can be assigned through the drop-down “Assign” option as part of the managed rulesets.



The new Bot Manager1.1 ruleset expands the bot management capabilities to provide comprehensive protection against malicious bots while allowing verified good bots to go through Azure WAF.


You can obtain more details about this feature on MS Learn at What is Azure Web Application Firewall on Azure Front Door? | Microsoft Learn and Azure Web Application Firewall DRS rule groups and rules | Microsoft Learn


Sowmya Mahadevaiah

Principal Product Manager, Azure Networking

Published on:

Learn more
Azure Network Security Blog articles
Azure Network Security Blog articles

Azure Network Security Blog articles

Share post:

Related posts

Increasing Security for SQL Server Enabled by Azure Arc

Back in November 2023, the least privileges deployment model was introduced as a public preview. After thorough testing, we are excited to ann...

1 hour ago

Govern your Azure Firewall configuration with Azure Policies

Introduction:  In the rapidly evolving digital landscape, securing cloud environments is more critical than ever. Azure Firewall emerges ...

6 hours ago

Azure Verified Modules - Monthly Update [June]

AVM Module Summary The AVM team are excited that our community have been busy building AVM Modules. As of June 17th, the AVM Footprint curren...

12 hours ago

General Availability Announcement: Azure VM Regional to Zonal Move

Today, we announce the general availability of the capability to convert regional VMs to a zonal configuration within the same region. Th...

1 day ago

Azure WAF Public Preview: JavaScript Challenge

Microsoft has recently released JavaScript challenge in public preview for Azure WAF on Application Gateway and Azure Front Door.   Appro...

1 day ago

Public Preview Announcement: Azure Policy Built-in Versioning

Welcome to a new era of policy management, where policy definitions are more agile, adaptable, and accessible than ever before! We are thrille...

2 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy