Cloudflare is trying to solve one of the internet’s most valuable fights: who gets to use web content to train artificial intelligence, and on what terms. The company’s new Disallow AI Training feature lets website owners block their material from AI model training without sacrificing visibility in search, a change that could help reset the economics of the open web for publishers, brands and anyone whose content is being scraped by machine-learning systems.
Cloudflare Launches Disallow AI Training Feature
Artificial Intelligence
That matters because the current system forces too many owners into a bad trade-off. If they want to keep their work out of AI training datasets, they can risk losing search traffic. Cloudflare says that is no longer necessary with the new controls, which separate search indexing from AI training and give site operators a cleaner way to set rules once and apply them across supported crawlers. For businesses that depend on discovery, that’s a meaningful shift: they can preserve reach while still protecting the long-term value of their content.
Sentiment Indicators
It also lands at a time when AI companies are under pressure to prove they are using web data more responsibly. Cloudflare said bot traffic that uses one crawl for both search and AI training now accounts for 36.6% of all verified crawler traffic on its network. That is a big number, and it helps explain why this issue has moved from a technical nuisance to an economic bargaining point. If AI labs are pulling value from publishers’ sites at scale, owners will demand more control, more transparency and, eventually, better compensation.
For investors, the story goes beyond one feature launch. Cloudflare sits at the intersection of security, web infrastructure and the AI economy, and this gives it a more central role in how the internet may govern content access going forward. If the company can become the default control layer for crawl preferences, it deepens the stickiness of its platform and adds another reason customers may stay. That is especially important for a business whose stock has been extremely active and whose technical indicators show strong momentum, with the shares recently trading well above both the 50-day and 200-day moving averages.
The company’s decision to label Apple, Google and Microsoft as “Responsible Parties” also shows where the power is concentrated. These are the firms that can set the standards, shape crawler behavior and decide whether AI search becomes a cooperative ecosystem or a permissionless scrape-and-train race. Cloudflare is effectively trying to build a middle layer that lets the web keep working for search while making AI training more accountable.
The broader investment takeaway is straightforward: content rights are becoming a real operating issue in AI, not just a legal one. If Cloudflare’s controls gain traction, they could become part of a wider shift toward negotiated access, better attribution and more explicit rules for how digital content is used. That would be good for web owners, useful for compliant AI builders and potentially durable for Cloudflare as a platform. For long-term investors, it is worth watching closely and keeping on the watchlist.
| Entity | Gains | Losses |
|---|---|---|
| Cloudflare | ▲More platform relevance | ▼None immediate |
| Website owners | ▲Search visibility, content control | ▼Less leverage if adoption lags |
| AI labs | ▲Clearer crawl rules | ▼Easier access to free training data |
| Search engines | ▲Continued indexing access | ▼More compliance pressure |
