← every day
AI crawler log, 2026-09-09
39
crawler requests — 39 of them (100%) took the markdown twin
By crawler
| Crawler | Requests | Markdown |
|---|---|---|
| amazon | 14 | 14 |
| claude | 8 | 8 |
| openai | 8 | 8 |
| meta | 7 | 7 |
| seo-tools | 2 | 2 |
Most fetched paths
/blog/what-is-a-click-worth-to-you.md3/blog/are-startup-directories-worth-submitting-to.md3/blog/what-time-of-day-should-your-ad-run.md3/blog/advertising-an-ai-product-when-ai-is-everywhere.md2/blog/the-first-hundred-brands-on-the-board.md2/b.md2/blog/how-many-clicks-before-a-conversion-rate-means-anything.md2/blog/introducing-outbid-guard.md2/blog/backlinks-that-never-sent-a-visitor.md2/blog/introducing-the-ad-price-index.md2How this is counted
Every request that reaches the edge is checked against a list of known crawler user-agent tokens and recorded with the path it asked for. Nothing is sampled and nothing is extrapolated: this is the count.
A crawler is grouped by family, so GPTBot, OAI-SearchBot and ChatGPT-User all read as one row. An agent nobody here recognises is recorded as other rather than guessed at, and a user-agent string is a claim rather than proof — anything can say it is anything.
One small site is not the internet and one day is not a trend. What it is, is real, dated and checkable, which is more than most of what gets written about AI crawlers. Every day so far is on the index.