Patreon Stops Asking AI Bots Nicely and Just Starts Blocking Them

For the past couple of years, websites that wanted to stop AI companies from scraping their content had one main tool available: the robots.txt file. This is a plain text document that sits on a web server and tells crawlers what they are and are not allowed to access. It is the digital equivalent of a polite sign on your front door reading 'no salespeople'. The problem is that several major AI companies, including OpenAI and Perplexity, have been caught ignoring robots.txt restrictions entirely. The polite sign was not working.

Patreon has apparently had enough of polite. According to TechCrunch, the creator monetisation platform has moved from asking AI bots not to scrape its content to actively blocking them. This is a meaningful escalation, and one that reflects a growing frustration across the web with the mismatch between what the robots.txt convention promises and what AI scrapers actually do.

What Patreon Stands to Lose

Patreon's content is particularly sensitive in this context. The platform hosts original creative work from more than 250,000 active creators, including writing, art, audio, and video, much of it behind paywalls that creators depend on for their income. When AI scrapers harvest that content, the consequences are not abstract. They potentially include that work appearing in AI training datasets, which could mean future AI models generating content similar enough to undermine those creators' livelihoods.

This is precisely the kind of scenario that has animated the ongoing legal battles between content creators, publishers, and AI companies over training data. The New York Times' lawsuit against OpenAI, filed in late 2023, argued that the company used millions of articles without permission or compensation. Several other publishers have followed with similar actions. For individual creators on Patreon, the legal route is not practical. Active blocking is the realistic alternative.

How Active Blocking Actually Works

Where robots.txt is a convention that scrapers can choose to honour or ignore, active blocking involves technical enforcement. This can include analysing the user-agent strings that bots declare themselves to be, rate-limiting suspicious request patterns that resemble automated crawling behaviour, deploying CAPTCHAs or JavaScript challenges that bots struggle with, and using commercial bot-detection services like Cloudflare's Bot Management or similar tools.

None of these methods are foolproof. Sophisticated scrapers can rotate IP addresses, fake user-agent strings, and deploy browser automation that mimics human behaviour. It becomes an ongoing technical arms race rather than a one-time fix. But it raises the cost of scraping significantly, which matters. If accessing Patreon's content requires substantial engineering effort to circumvent defences, smaller AI operations simply will not bother, and even larger ones face meaningful friction.

The broader trend here is important. We are watching the web shift from a relatively open information environment, where the implicit social contract was that crawlers index but do not reproduce, to a contested space where content owners are actively fortifying against AI ingestion. Patreon is part of a growing cohort that includes Reddit (which began charging for API access in 2023 partly to control AI training access), and several major news publishers who have partnered with AI companies through licensing deals rather than letting their content be taken without agreement.

A Question of Who the Web Is For

There is a philosophical dimension to this that the technical conversation sometimes obscures. The early web was built on ideals of openness and information sharing. Crawlers that indexed content were seen as benign or even beneficial, because they helped people find things. AI scrapers that harvest content to train commercial models that then compete with the original creators feel different in kind, not just degree. They extract value from creative labour without consent or compensation.

Patreon blocking AI scrapers is, in that sense, a creator platform doing what it exists to do: protecting the economic interests of the people who depend on it. The broader question of whether the web's openness norms need fundamental renegotiation in the AI era is one that regulators in the EU and UK are beginning to take seriously, but legislation moves slowly. In the meantime, technical countermeasures are the most immediate form of agency content creators have.