Sign inGet started
Dataset · Hugging Face

DeusHorizon/agent-web-index

Agent Web Index — how much of the web can AI assistants actually read? 50,415 domains measured live. 26% of them cannot be read by at least one of ChatGPT, Claude, Perplexity or Gemini. Updated daily. Live index: https://shop.lumnika.com/ai-readiness/ Every row here is the result of real HTTP requests, not an estimate and not a re-publication of someone else's crawl: each domain's homepage is requested once as a browser and once as each of the published AI crawler user-agents… See the full description on the dataset page: https://huggingface.co/datasets/DeusHorizon/agent-web-index.

Author / organizationDeusHorizon
License (as declared)cc-by-4.0
Downloads742 (as reported by Hugging Face)
Likes0
Created19 September 2026
Last updated at source17 October 2026
webcrawlersrobots-txtai-agentsllmmeasurement
Embed this entry
<iframe src="https://silytics.si/embed/item/a27f3bee-e921-4875-8916-85c75c42ba0f" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Popularity, last 2 days

742 now (+22 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.

Recent changes

  • 17 Oct 2026 Description: Agent Web Index — how much of the web can AI assistants actually read? 50,419 domains measured live. 26% of them cannot be read by at least one of ChatGPT, Claude, Perplexity or Gemini. Updated daily. Live index: https://shop.lumnika.com/ai → Agent Web Index — how much of the web can AI assistants actually read? 50,415 domains measured live. 26% of them cannot be read by at least one of ChatGPT, Claude, Perplexity or Gemini. Updated daily. Live index: https://shop.lumnika.com/ai

Changes detected by Silytics between refreshes of the source.

More from DeusHorizon

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.