Interactive demo
Generate an AI bot consent policy in seconds.
Choose how different crawler categories should interact with a website, then preview the hosted policy, robots.txt output, llms.txt guidance, and dashboard activity.
Build a consent policy
Enter a domain, choose bot preferences, and preview the generated files.
Search indexing bots
Traditional search engines and discovery crawlers.
AI assistant retrieval bots
Agents that retrieve pages to answer user-initiated prompts.
AI training crawlers
Systems collecting content for model training or fine-tuning.
Commercial scrapers
Price, content, and market-intelligence scrapers.
Archive/research bots
Public-interest archiving, academic, and research crawlers.
Generated output
https://botconsent.com/policies/example.com
{
"domain": "example.com",
"policy_url": "https://botconsent.com/policies/example.com",
"version": "2026-05-06",
"categories": {
"search_indexing": "allow",
"ai_assistant_retrieval": "allow",
"ai_training_crawlers": "block",
"commercial_scrapers": "review",
"archive_research_bots": "allow"
},
"contact": "policy@example.com",
"attribution_required": false
}Dashboard preview
Mock activity for example.com
284Crawler events
92Blocked training
1.8kPolicy views
GPTBotAI training crawlerBlocked
PerplexityBotAssistant retrievalAllow
GooglebotSearch indexingAllow