AI crawler log, 2026-09-10
66
crawler requests, up from 39 the day before — 66 of them (100%) took the markdown twin
By crawler
| Crawler | Requests | Markdown |
|---|---|---|
| amazon | 29 | 29 |
| claude | 15 | 15 |
| meta | 14 | 14 |
| openai | 5 | 5 |
| other | 2 | 2 |
| seo-tools | 1 | 1 |
Most fetched paths
/b.md4/blog/introducing-the-ad-price-index.md2/blog/sponsored-nofollow-dofollow-paid-placement-links.md2/blog/fast-tests-need-honest-numbers.md2/blog/what-500-followers-actually-buys-you.md2/blog/where-small-sites-get-traffic-after-ai-overviews.md2/blog/test-ad-copy-before-you-pay-for-clicks.md2/blog/does-domain-rating-predict-traffic.md2/blog/why-b2b-ad-costs-are-higher.md2/blog/click-rate-benchmarks-for-small-budgets.md2How this is counted
Every request that reaches the edge is checked against a list of known crawler user-agent tokens and recorded with the path it asked for. Nothing is sampled and nothing is extrapolated: this is the count.
A crawler is grouped by family, so GPTBot, OAI-SearchBot and ChatGPT-User all read as one row. An agent nobody here recognises is recorded as other rather than guessed at, and a user-agent string is a claim rather than proof — anything can say it is anything.
One small site is not the internet and one day is not a trend. What it is, is real, dated and checkable, which is more than most of what gets written about AI crawlers. Every day so far is on the index.