Original data
What we have tested, and when
These are our own measurements, not vendor claims: every run publishes its method, the sample size, the limitations, and the script that produced it. When we have not tested something, the comparison pages say so and the claim carries its vendor source instead.
-
30 Sept 2026 · Marketing automation and lifecycle messaging
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-30. 0 of 6 restrict any checked AI crawler in robots.txt. 5 of 6 ship an /llms.txt (klaviyo, customer-io, mailchimp, omnisend, braze). Median homepage TTFB: activecampaign fastest at 62ms, klaviyo slowest at 720ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs -
30 Sept 2026 · CRM platforms
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-30. 2 of 6 restrict any checked AI crawler in robots.txt. 5 of 6 ship an /llms.txt (hubspot, pipedrive, attio, close, folk). Median homepage TTFB: close fastest at 50ms, hubspot slowest at 1439ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Workflow automation
AI crawler access test: can answer engines read these vendors?
4 of 4 tool sites reachable from our build machine on 2026-09-27. 0 of 4 restrict any checked AI crawler in robots.txt. 4 of 4 ship an /llms.txt (n8n, zapier, make, activepieces). Median homepage TTFB: n8n fastest at 41ms, make slowest at 776ms.
n = 4 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Personal AI agents
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-27. 1 of 6 restrict any checked AI crawler in robots.txt (policy undetermined for 1 — robots.txt served a non-robots page). 3 of 6 ship an /llms.txt (instinct, hermes-agent, openclaw). Median homepage TTFB: hermes-agent fastest at 40ms, muse slowest at 1529ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Local LLM stacks
AI crawler access test: can answer engines read these vendors?
4 of 4 tool sites reachable from our build machine on 2026-09-27. 0 of 4 restrict any checked AI crawler in robots.txt (policy undetermined for 1 — robots.txt served a non-robots page). 3 of 4 ship an /llms.txt (ollama, lm-studio, open-webui). Median homepage TTFB: jan fastest at 140ms, ollama slowest at 341ms.
n = 4 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Gmail automation
AI crawler access test: can answer engines read these vendors?
7 of 7 tool sites reachable from our build machine on 2026-09-27. 0 of 7 restrict any checked AI crawler in robots.txt. 3 of 7 ship an /llms.txt (streak, yesware, mixmax). Median homepage TTFB: yamm fastest at 40ms, mailtrack slowest at 3738ms.
n = 7 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Free SPA hosting
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-27. 0 of 6 restrict any checked AI crawler in robots.txt (policy undetermined for 1 — robots.txt served a non-robots page). 4 of 6 ship an /llms.txt (cloudflare-pages, netlify, vercel, render). Median homepage TTFB: github-pages fastest at 17ms, firebase-hosting slowest at 532ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · AI voice agents
AI crawler access test: can answer engines read these vendors?
3 of 4 tool sites reachable from our build machine on 2026-09-27. 0 of 4 restrict any checked AI crawler in robots.txt (policy undetermined for 1 — robots.txt served a non-robots page). 2 of 4 ship an /llms.txt (retell, vapi). Median homepage TTFB: synthflow fastest at 38ms, retell slowest at 348ms.
n = 4 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · AI meeting notes
AI crawler access test: can answer engines read these vendors?
4 of 4 tool sites reachable from our build machine on 2026-09-27. 0 of 4 restrict any checked AI crawler in robots.txt. 4 of 4 ship an /llms.txt (fireflies, otter, granola, tldv). Median homepage TTFB: fireflies fastest at 27ms, granola slowest at 399ms.
n = 4 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · AI coding assistants
AI crawler access test: can answer engines read these vendors?
8 of 8 tool sites reachable from our build machine on 2026-09-27. 2 of 8 restrict any checked AI crawler in robots.txt. 5 of 8 ship an /llms.txt (claude-code, cursor, github-copilot, gemini-cli, kiro). Median homepage TTFB: kiro fastest at 21ms, opencode slowest at 542ms.
n = 8 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · AI agent builders
AI crawler access test: can answer engines read these vendors?
4 of 4 tool sites reachable from our build machine on 2026-09-27. 0 of 4 restrict any checked AI crawler in robots.txt. 4 of 4 ship an /llms.txt (voiceflow, botpress, relevance-ai, zapier-agents). Median homepage TTFB: relevance-ai fastest at 47ms, voiceflow slowest at 239ms.
n = 4 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Self-improving AI assistants
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-27. 0 of 6 restrict any checked AI crawler in robots.txt (policy undetermined for 2 — robots.txt served a non-robots page). 3 of 6 ship an /llms.txt (motion, lindy, mem). Median homepage TTFB: motion fastest at 45ms, fathom slowest at 648ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs -
27 Sept 2026 · Agentic code review
AI crawler access test: can answer engines read these vendors?
6 of 6 tool sites reachable from our build machine on 2026-09-27. 0 of 6 restrict any checked AI crawler in robots.txt. 5 of 6 ship an /llms.txt (coderabbit, greptile, qodo, ellipsis, superpositional). Median homepage TTFB: superpositional fastest at 18ms, ellipsis slowest at 251ms.
n = 6 tools × 3 runs · script:
scripts/crawler-test.mjs