Cloudflare splits AI bots into Search, Agent and Training classes
- Defaults from 2026-09-15
Cloudflare splits AI bots into Search, Agent and Training classes
Cloudflare let all customers manage AI traffic by three use cases: Search, Agent and Training crawlers. From September 15, 2026, new domains block Training and Agent bots by default on pages that display ads, while Search stays allowed.
Related essays and timeline events
Google makes Merchant Center AI performance insights generally available
Google made AI performance insights in Merchant Center generally available as part of its holiday agentic commerce updates.
Search Console generative AI performance report reaches all sites
Google rolled out the generative AI performance report in Search Console to all websites worldwide.
Similarweb: 56% of AI-influenced visits arrive through search
Similarweb measured where visits come from after a brand appears in an AI answer.
Adobe: AI traffic to US retail sites up 393% as product pages score 66% readable
Adobe Digital Insights reported that traffic from AI sources to US retail sites grew 393% year over year in January through March 2026.
ACES study finds AI shopping agents show position bias and penalize sponsored tags
Researchers posted a study to arXiv that uses ACES, a provider-agnostic framework, to audit how AI agents choose products in a simulated marketplace.
Get the next one by email
Original essays on Hard-to-Keyword demand and agentic commerce, plus key protocol milestones. No spam.