dossier_ai_crawlers
shallowme.drwho/tools · Verify this server
Core dossier check: Report a domain's AI-crawler policy — whether robots.txt allows, blocks, or is silent on the six major AI agents (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, meta-externalagent). Use to answer "does this site let AI models train on / retrieve its content?" in a posture audit. Fetches https://<domain>/robots.txt, follows redirects (off-site redirect = no policy), 10s timeout; parses robots groups. Returns a CheckResult; on success { hasRobots, agents: { <agent>: 'allowed' | 'blocked' | 'unspecified' } }. A missing robots.txt is data, not an error (every agent unspecified).
1 trials · measured 2 days ago
dossier_ai_crawlers scores 100.0/100 on Vouch's measured behaviour index, from 1 real invocation trials against me.drwho/tools, measured 31 Aug 2026 under methodology v0.2.0. Every measured component scored 100.
Component breakdown
| Component | Weight | Value |
|---|---|---|
| Reliability | 35% | not applicable |
| Schema integrity | 25% | 100.0 |
| Failure behaviour | 15% | not applicable |
| Latency | 15% | not applicable |
| Concurrency | 10% | not applicable |
Tool details
- Transport
- remote
- Credential class
- self-provisionable
- Category
- Developer infrastructure
- Input schema
- not declared
- Output schema
- not declared
- Side-effect classification
- unclassified
Score history
| Day | Score | Tier | Methodology |
|---|---|---|---|
| 2026-08-31 | 100.0 | shallow | v0.2.0 |
Probe evidence
| Probe | Outcomes |
|---|---|
| schema_integrity | pass: 1 |
Raw request/response logs are not archived yet — the outcome counts above are drawn directly from every recorded trial.
Embed this score
Available for every tool, scored or not — not a verification perk. Always links back to this page.
[](https://vouch.tools/tools/8e42c765-14ed-4f99-82bb-4a248246e221)