MVP dashboard
View public demoManage AI bot consent across every site.
This polished prototype shows the core BotConsent workflow: verified sites, policy building, generated integration output, and crawler activity.
Consent policy builder
Category-level controls with bot-specific overrides coming next.
Search indexing bots
Traditional search engines and discovery crawlers.
AI assistant retrieval bots
Agents that retrieve pages to answer user-initiated prompts.
AI training crawlers
Systems collecting content for model training or fine-tuning.
Commercial scrapers
Price, content, and market-intelligence scrapers.
Archive/research bots
Public-interest archiving, academic, and research crawlers.
Policy preview
https://botconsent.com/policies/botconsent.com
{
"domain": "botconsent.com",
"status": "published",
"site_verification": "Verified",
"policy_url": "https://botconsent.com/policies/botconsent.com",
"preferences": {
"search_indexing_bots": "allow",
"ai_assistant_retrieval_bots": "attribution",
"ai_training_crawlers": "block",
"commercial_scrapers": "review",
"archive_research_bots": "allow"
}
}Integration
Generated embed instructions
<link rel="botconsent" href="https://botconsent.com/policies/botconsent.com" />curl https://botconsent.com/api/policy/botconsent.comActivity log
| Time | Agent | Category | Action | Path |
|---|---|---|---|---|
| 2 min ago | GPTBot | AI training | Blocked | /resources |
| 11 min ago | Googlebot | Search | Allowed | /docs |
| 24 min ago | ClaudeBot | Assistant retrieval | Attribution | /product |
| 48 min ago | Unknown scraper | Commercial | Review | /pricing |