AI crawler log, 2026-09-12
32
crawler requests, down from 60 the day before — 32 of them (100%) took the markdown twin
By crawler
| Crawler | Requests | Markdown |
|---|---|---|
| amazon | 26 | 26 |
| meta | 3 | 3 |
| openai | 2 | 2 |
| apple | 1 | 1 |
Most fetched paths
/blog/sponsorship-marketplace-comparison.md1/blog/zero-click-search-and-your-launch.md1/blog/market-a-side-project-with-no-budget.md1/blog/carbon-ads-alternatives-for-indie-makers.md1/blog/one-line-pitch-examples.md1/blog/google-ads-alternatives-for-small-budgets.md1/blog/advertise-a-bootstrapped-business.md1/blog/where-to-advertise-a-startup-with-200-dollars.md1/blog/ad-platforms-that-publish-their-numbers.md1/blog/google-ai-max-migration-what-you-lost.md1How this is counted
Every request that reaches the edge is checked against a list of known crawler user-agent tokens and recorded with the path it asked for. Nothing is sampled and nothing is extrapolated: this is the count.
A crawler is grouped by family, so GPTBot, OAI-SearchBot and ChatGPT-User all read as one row. An agent nobody here recognises is recorded as other rather than guessed at, and a user-agent string is a claim rather than proof — anything can say it is anything.
One small site is not the internet and one day is not a trend. What it is, is real, dated and checkable, which is more than most of what gets written about AI crawlers. Every day so far is on the index.