69% of DACH Media Block GPTBot. Only 18% Block ChatGPT's Search Bot.

I checked the robots.txt files of 50 of the largest media sites in Germany, Austria and Switzerland on September 17, 2026, and evaluated them for 17 AI bots plus Googlebot and Bingbot. The short version: 69% block OpenAI's training bot GPTBot, but only 18% block OAI-SearchBot, the bot behind ChatGPT's search results. Google and Bing get in everywhere.

The core numbers

Of the 49 sites whose robots.txt could be read (puls24.com didn't respond), this many block each bot, either for the whole site or for their article pages:

• Training bots: GPTBot 34 (69%), CCBot 31 (63%), ClaudeBot 30 (61%), Bytespider 29 (59%), Applebot-Extended 28 (57%), Google-Extended 26 (53%), Meta-ExternalAgent 25 (51%), anthropic-ai 21 (43%), cohere-ai 21 (43%)
• Search index bots: PerplexityBot 16 (33%), OAI-SearchBot 9 (18%), Claude-SearchBot 7 (14%)
• User-triggered bots: ChatGPT-User 15 (31%), MistralAI-User 11 (22%), Perplexity-User 7 (14%), Claude-User 5 (10%), Google-Agent 2 (4%)
• Classic search: Googlebot 0, Bingbot 0

The grouping matters, because these bots do different jobs. Training bots collect content for future models. Search index bots build the index that ChatGPT, Claude or Perplexity search when they answer with sources. User-triggered bots fetch a single page because someone asked for it in a chat.

Training blocked, search allowed

The clearest pattern in the data: most publishers have already separated the two. 25 of the 34 sites that block GPTBot let OAI-SearchBot through. For Anthropic it's 23 sites that block ClaudeBot but allow Claude-SearchBot. They say no to training and yes to being found in AI search.

That's a more deliberate stance than "block ChatGPT". A site that blocks GPTBot but allows OAI-SearchBot can still show up as a source in ChatGPT's search answers. Perplexity gets less of that goodwill: PerplexityBot is blocked about twice as often as the search bots of OpenAI and Anthropic.

Blocking user-triggered bots mostly does nothing

15 sites block ChatGPT-User in robots.txt. According to OpenAI's documentation, those rules "may not apply" because a user started the fetch. Perplexity says the same about Perplexity-User, and Google says its user-triggered fetchers, Google-Agent included, generally ignore robots.txt. Only Anthropic documents that Claude-User follows it.

So for three of those five bots, the robots.txt lines are a statement of intent rather than a barrier. Publishers who actually want to keep them out need server-side rules, and those come with the risk of also blocking readers who use an AI assistant to open an article.

Who blocks the most, and who blocks nothing

Counting the 17 AI bots above, the sites that block the most:

1. heise.de: 16 of 17
2. faz.net: 15
3. falter.at: 15
4. nzz.ch: 14
5. spiegel.de and manager-magazin.de: 13 each
6. nachrichten.at: 12

At the other end, 13 sites block none of the 17: Beobachter, Blick, Heute, Kleine Zeitung, Krone, Kurier, News, oe24, Profil, ServusTV, Trend, Watson and ZDF. Twelve of them are Austrian or Swiss.

Some sites only block articles, not the homepage. Golem allows AI training bots on its start page and a few sections but disallows everything else, and 20 Minuten disallows every URL containing "story", which covers its articles. Checking only the root path would count both as open.

Public broadcasters

The German ARD broadcasters in the sample all land on the same number: Tagesschau, NDR, WDR and BR each block 7 of the 17 AI bots. ORF blocks 8, SRF 7. ZDF blocks none.

Same funding model, opposite decisions. The robots.txt files don't say why, so I won't guess. What they do show is that there's no shared public-broadcaster position on AI crawlers, even within Germany.

Germany, Austria, Switzerland

Average number of the 17 AI bots blocked per site:

• Germany: 8.0 (25 sites), GPTBot blocked by 22
• Switzerland: 6.1 (7 sites), GPTBot blocked by 4
• Austria: 4.4 (17 sites), GPTBot blocked by 8

German sites block the most. The Swiss sample is small, and Austria is split: Falter, OÖN, Der Standard and Die Presse block a lot, while most tabloids and magazines block nothing.

By type of outlet

Average of the 17 AI bots blocked, by category. Several groups have only two to six sites, so read these as a tendency, not a ranking:

• Weekly (Die Zeit, Falter): 12.5
• Tech (Heise, t3n, Golem): 10.3
• Business (5 sites): 8.0
• Daily (14 sites): 7.8
• Regional (4 sites): 6.3
• Public broadcasters (7 sites): 6.1
• Magazines (6 sites): 3.8
• Tabloids (6 sites): 3.0
• TV (n-tv, ServusTV): 1.5

Tabloids and magazines are the most open. 20 Minuten is the exception among tabloids, blocking 11 bots from its articles.

An opt-out written as a comment

Krone blocks no AI bot in its robots.txt rules. Instead, the file opens with comment lines that prohibit text and data mining, AI and LLM development, and name the relevant EU directive and Austrian copyright law. Crawlers skip comment lines by design, so technically nothing is blocked.

Whether a written reservation like this counts as a machine-readable opt-out under EU copyright law is legally unsettled. It's a reminder that "0 bots blocked" in this study doesn't always mean a publisher agrees to AI use.

What this means for GEO

For publishers, the data shows a workable middle path that most large outlets already use: block training bots, allow search index bots. That keeps a site eligible as a cited source in AI search without handing content to model training.

For everyone else who wants to be cited in AI answers, the takeaway is simpler. Check that OAI-SearchBot, Claude-SearchBot and PerplexityBot aren't blocked by an old blanket rule, because those are the bots that decide whether your pages can appear as sources. The Crawler Access Checker shows it for any URL.

Methodology

• Sample: 50 high-reach DACH media sites (25 DE, 18 AT, 7 CH), the same list as the April study.
• Date: robots.txt fetched on September 17, 2026. puls24.com didn't respond, so the results cover 49 sites.
• Parsing: RFC 9309. A bot's own user-agent group applies, the * group only as a fallback, the longest matching rule wins and Allow wins ties.
• Blocked means: robots.txt disallows the bot for the root path or for an article URL. The article URL was taken from each site's homepage; for 10 sites without a usable link, a typical article path was used instead.
• Not measured: how crawlers actually behave, server or firewall rules, and licensing agreements between publishers and AI companies. robots.txt shows a site's stated policy, nothing more.
• Bots: 9 training bots, 3 search index bots, 5 user-triggered bots, plus Googlebot and Bingbot. Tokens without public documentation from their operator, such as xAI-Web-Crawler and DeepSeekBot, aren't counted.

FAQ