According to HotHardware, Cloudflare CEO Matthew Prince revealed his company blocked a staggering 416 billion unwanted AI bot requests in just five months. He slammed Google’s practices in an interview with Wired, stating that websites cannot opt out of having their content used for AI training without also opting out of Google Search indexing. Prince called this bundling “crazy” and accused Google of using its “monopoly position of yesterday” to dominate the AI market of tomorrow. He argued this creates an unfair playing field, preventing businesses from flourishing. The CEO directly labeled Google as “the problem” that is holding back the internet’s progress.
The Core of the Fight
Here’s the thing: this isn’t about stopping Google from indexing the web for search. Everyone wants to be found. The fight is over the new, voracious appetite of AI. For decades, letting Googlebot crawl your site was the cost of doing business on the internet. It was a trade—your data for your discoverability. But now, that same access is being used to fuel a competitor. AI models ingest that content to learn, to generate answers, and potentially to create services that compete with the very sites they’re scraping from. And the site gets nothing. No traffic, no licensing fee, often not even a citation. Prince says blocking these AI crawlers has had positive results for publishers. So the question becomes: why should they have to choose between being visible on the web and having their work used to train their potential replacements?
A Monopoly Lever
Prince’s “monopoly” argument is the real kicker. Google Search is the front door to the internet for most people. Opting out is commercial suicide for most businesses. So by tying AI scraping to search indexing, Google is basically saying, “If you want to exist in our ecosystem, you must also feed our AI.” There’s no real choice. It’s a take-it-or-leave-it deal where leaving it means vanishing from the largest discovery platform on Earth. That’s an immense amount of leverage. And as Prince points out, it’s using dominance in one established market (search) to bulldoze a path to dominance in a new one (AI). That’s the classic playbook that gets antitrust regulators interested.
What Happens Next?
So where does this go? Cloudflare can keep building better bots to block the scrapers, but it’s an arms race. The real pressure will come from a few places. First, legal and regulatory. We’re already seeing lawsuits from publishers and content creators. Second, from competitors. If another search engine gained meaningful traction by offering a “search-only, no AI training” crawl option, it could force Google’s hand. But let’s be real—that’s a huge “if.” Third, and maybe most likely, is a slow, negotiated compromise. Google might eventually offer a more granular robots.txt standard or some form of licensing for high-value content. But they have zero incentive to move quickly. They have the data, the models, and the distribution. For now, the internet’s gatekeeper is also its biggest AI data harvester. And that puts everyone else in a tough spot.
mjbnlb