Cloudflare Mandates AI Bot Separation by September 15, Targets Google Gemini

According to Cloudflare, the infrastructure giant recently announced that all AI companies must separate search engine crawlers from AI training crawlers by September 15, 2026, or face complete blocking. Cloudflare, which manages approximately 20% of global web traffic, cited data showing that automated bot traffic has already exceeded human traffic in the first half of 2026—a turning point industry analysts predicted would occur one year later.

Google faces the most significant impact due to its 90% search market share, as it currently uses Googlebot for both indexing and training its Gemini model. The company must choose between losing access to 20% of indexed pages or separating its crawlers and risking AI training data shortages. By contrast, OpenAI has already split GPTBot for training data collection and ChatGPT-User for real-time web queries.