Tag
#robots-txt
1 post
Anthropic's 3-bot identity problem: blocking ClaudeBot doesn't block Claude
Anthropic runs ClaudeBot, Claude-User and Claude-SearchBot as three robots.txt identities. Block only ClaudeBot and the other two still read your site.
#ai-crawlers#anthropic#robots-txt
Paul ·