Cloudflare’s bot products differ in available controls and exception support. Choose settings that fit both your subscription and your legitimate automated traffic instead of assuming every bot-labelled request is unwanted.
IN THIS ARTICLE
01 Information to have ready
Inventory search crawlers, uptime monitors, API clients, payment callbacks and deployment tools. Check the bot product actually enabled for the zone and record its current configuration.
02 Choose Cloudflare bot controls that fit the site and plan
- Open the zone’s Security Settings and filter for bot-related controls. Review the product and options available on your plan.
- Enable or adjust the intended protection while considering browser challenges and JavaScript requirements.
- Test legitimate automation and normal browser visits, including mobile clients and integrations that cannot complete an interactive challenge.
- Inspect Security Analytics and Events to identify the exact bot service taking action.
- Where exceptions are needed, use a product that supports the required scoped exceptions and monitor their effect. Disable only the incompatible control during a controlled diagnosis.
03 Confirm the expected result
Unwanted automation should be reduced while authorised services continue. Use actual event evidence and integration tests instead of only a headline blocked-request count.
04 Troubleshoot unexpected results
Ordinary Bot Fight Mode cannot be skipped with WAF custom-rule Skip actions. Super Bot Fight Mode and other products have different controls. Do not repeatedly add ineffective WAF exceptions to a separate evaluation pipeline.
05 Example: reduce abusive traffic without blocking customers
Suppose a login page receives repeated automated requests while ordinary product pages remain unaffected. Start with that path and the evidence in security events. A broad challenge across the whole website may interrupt genuine visitors, payment callbacks and monitoring systems.
- Identify the affected hostname, path, time period and rule action in the available events.
- Separate interactive browser traffic from machine-to-machine endpoints such as payment webhooks.
- Apply a narrow rule or supported bot-control setting appropriate to your Cloudflare plan.
- Test a signed-out visitor journey and the relevant integrations, then review the resulting events.
Keep a record of the previous configuration and the reason for the change. Do not permanently allow an entire country or a large shared network merely because one legitimate visitor was challenged.
What should I send us when a legitimate action is blocked?
Provide the public URL, time, Cloudflare Ray ID where shown and the action you were trying to complete. Avoid sharing session cookies or full request bodies containing customer information. The rule owner can then examine the specific event rather than disabling all protection to make one request pass.
Will blocking bots automatically block all AI crawlers?
Review the specific AI-bot control and the site’s policy separately. Product settings, recognised crawler categories and enforcement behaviour are distinct from a general robots.txt preference.