AI crawler log, 2026-09-13
21
crawler requests, down from 32 the day before — 21 of them (100%) took the markdown twin
By crawler
| Crawler | Requests | Markdown |
|---|---|---|
| claude | 13 | 13 |
| meta | 4 | 4 |
| amazon | 4 | 4 |
Most fetched paths
/blog/introducing-the-ad-price-index.md1/blog/advertising-an-ai-product-when-ai-is-everywhere.md1/blog/one-message-or-many.md1/blog/hourly-ad-slots.md1/blog/bidsurvivor-vs-ethicalads.md1/blog/bidsurvivor-vs-carbon-ads.md1/blog/best-advertising-platforms-for-startups.md1/blog/best-alternatives-to-google-ads.md1/blog/best-free-marketing-tools-for-bootstrapped-startups.md1/blog/advertise-saas-product-cheaply.md1How this is counted
Every request that reaches the edge is checked against a list of known crawler user-agent tokens and recorded with the path it asked for. Nothing is sampled and nothing is extrapolated: this is the count.
A crawler is grouped by family, so GPTBot, OAI-SearchBot and ChatGPT-User all read as one row. An agent nobody here recognises is recorded as other rather than guessed at, and a user-agent string is a claim rather than proof — anything can say it is anything.
One small site is not the internet and one day is not a trend. What it is, is real, dated and checkable, which is more than most of what gets written about AI crawlers. Every day so far is on the index.