{
  "edition": "2026-09",
  "snapshot": "data/census-2026-09-03.json",
  "universe": {
    "source": "Tranco",
    "listSize": 100000,
    "listDate": "2026-09-01"
  },
  "collectedAt": "2026-09-02",
  "publishedAt": "2026-09-03",
  "correctedAt": null,
  "methodologyVersion": "2.0",
  "coverage": {
    "domainsChecked": 99999,
    "denominator": 100000
  },
  "exclusions": [
    "Subdominios fuera de alcance (solo apex domains)",
    "Catch-alls excluidos con canary en DOS directorios: la raíz y /.well-known/ (v2.0)",
    "Respuesta HTML a rutas de texto contada como ausente",
    "Manifiestos (ai-catalog.json, ard.json) solo cuentan con un array `entries` no vacío (v2.0)",
    "Dominios propios fuera del Tranco top-100K no contados"
  ],
  "ownDomainsInScope": [],
  "notes": [
    "Methodology 2.0. The canary that discards catch-all servers now probes two directories, the root and /.well-known/, instead of the root alone. Servers that 404 a nonsense path at the root while answering 200 to anything under /.well-known/ were previously counted as publishers of every well-known surface. This is a measurement change, not a change in the web: the catch-all rates below are not comparable with editions 2026-07 and 2026-08, which under-detected them.",
    "Methodology 2.0. A manifest surface (ai-catalog.json, ard.json) now requires a non-empty `entries` array to count. A 200 carrying JSON without entries is a server answering, not a domain publishing. This was already required for ai-catalog.json and now applies to every manifest surface.",
    "Three surfaces appear in the census for the first time: agent-card.json and mcp.json (measured since 2026-08-19) and ard.json (since 2026-08-27, one day after ARD v0.91 moved the canonical path). A zero in an earlier edition means NOT MEASURED, not 'nobody published'.",
    "The four ard.json publishers were each verified by hand against the live path on 2026-09-03, with a canary probe per domain: brandfetch.com, railway.com, idescat.cat and bizzabo.com.",
    "The pass ran across a deployment: domains 0-39,999 were crawled with one build and 40,000-99,999 with the next. The two builds are behaviourally equivalent for every surface measured here — the changes add metadata and a two-domain opt-out list, neither of which affects present/absent. Recorded in the snapshot's .meta.json."
  ],
  "corrections": [],
  "domainsChecked": 99999,
  "tiers": [
    {
      "tier": "top 1K",
      "checked": 1000,
      "ai_catalog": 0,
      "llms_txt": 75,
      "agents_md": 4,
      "agent_card": 2,
      "mcp_well_known": 3,
      "ard_json": 0
    },
    {
      "tier": "1K–10K",
      "checked": 9000,
      "ai_catalog": 6,
      "llms_txt": 555,
      "agents_md": 18,
      "agent_card": 6,
      "mcp_well_known": 9,
      "ard_json": 0
    },
    {
      "tier": "10K–100K",
      "checked": 89999,
      "ai_catalog": 31,
      "llms_txt": 5724,
      "agents_md": 949,
      "agent_card": 45,
      "mcp_well_known": 72,
      "ard_json": 4
    }
  ],
  "aiCatalogQuality": {
    "present": 50,
    "invalid_json": 10,
    "with_entries": 37
  },
  "catchAll": [
    {
      "kind": "robots_ai",
      "responding": 53468,
      "real": 53468,
      "catch_all": 0
    },
    {
      "kind": "llms_txt",
      "responding": 7855,
      "real": 6354,
      "catch_all": 1501
    },
    {
      "kind": "agents_md",
      "responding": 1620,
      "real": 971,
      "catch_all": 649
    },
    {
      "kind": "mcp_well_known",
      "responding": 794,
      "real": 84,
      "catch_all": 710
    },
    {
      "kind": "agent_card",
      "responding": 754,
      "real": 53,
      "catch_all": 701
    },
    {
      "kind": "ai_catalog",
      "responding": 763,
      "real": 50,
      "catch_all": 713
    },
    {
      "kind": "dns_aid",
      "responding": 24,
      "real": 24,
      "catch_all": 0
    },
    {
      "kind": "ard_json",
      "responding": 727,
      "real": 22,
      "catch_all": 705
    }
  ],
  "manifestQuality": [
    {
      "kind": "ai_catalog",
      "present": 50,
      "invalid_json": 10,
      "with_entries": 37
    },
    {
      "kind": "ard_json",
      "present": 22,
      "invalid_json": 16,
      "with_entries": 4
    }
  ],
  "aiBotDirectives": [
    {
      "bot": "GPTBot",
      "named": 10540,
      "fully_blocked": 8307
    },
    {
      "bot": "ClaudeBot",
      "named": 9527,
      "fully_blocked": 7508
    },
    {
      "bot": "CCBot",
      "named": 9180,
      "fully_blocked": 8101
    },
    {
      "bot": "Google-Extended",
      "named": 8660,
      "fully_blocked": 6914
    },
    {
      "bot": "Bytespider",
      "named": 8474,
      "fully_blocked": 7814
    },
    {
      "bot": "meta-externalagent",
      "named": 7817,
      "fully_blocked": 6818
    },
    {
      "bot": "Applebot-Extended",
      "named": 7527,
      "fully_blocked": 6646
    },
    {
      "bot": "PerplexityBot",
      "named": 4291,
      "fully_blocked": 2157
    },
    {
      "bot": "ChatGPT-User",
      "named": 4193,
      "fully_blocked": 2147
    },
    {
      "bot": "anthropic-ai",
      "named": 3442,
      "fully_blocked": 2489
    },
    {
      "bot": "cohere-ai",
      "named": 2691,
      "fully_blocked": 2094
    },
    {
      "bot": "Claude-Web",
      "named": 2649,
      "fully_blocked": 1980
    },
    {
      "bot": "Perplexity-User",
      "named": 1579,
      "fully_blocked": 775
    },
    {
      "bot": "Claude-User",
      "named": 1502,
      "fully_blocked": 747
    },
    {
      "bot": "DuckAssistBot",
      "named": 1297,
      "fully_blocked": 922
    },
    {
      "bot": "meta-externalfetcher",
      "named": 1297,
      "fully_blocked": 947
    },
    {
      "bot": "MistralAI-User",
      "named": 843,
      "fully_blocked": 621
    },
    {
      "bot": "Gemini-Deep-Research",
      "named": 443,
      "fully_blocked": 322
    },
    {
      "bot": "NovaAct",
      "named": 364,
      "fully_blocked": 334
    },
    {
      "bot": "Google-NotebookLM",
      "named": 340,
      "fully_blocked": 247
    },
    {
      "bot": "Devin",
      "named": 325,
      "fully_blocked": 309
    },
    {
      "bot": "GoogleAgent-Mariner",
      "named": 292,
      "fully_blocked": 252
    },
    {
      "bot": "AmazonBuyForMe",
      "named": 249,
      "fully_blocked": 241
    },
    {
      "bot": "Manus-User",
      "named": 230,
      "fully_blocked": 218
    },
    {
      "bot": "TwinAgent",
      "named": 205,
      "fully_blocked": 201
    },
    {
      "bot": "Google-Agent",
      "named": 186,
      "fully_blocked": 148
    },
    {
      "bot": "Kagi-Fetcher",
      "named": 78,
      "fully_blocked": 74
    },
    {
      "bot": "Claude-Code",
      "named": 62,
      "fully_blocked": 48
    },
    {
      "bot": "Trae",
      "named": 49,
      "fully_blocked": 48
    },
    {
      "bot": "OpenCode",
      "named": 49,
      "fully_blocked": 47
    },
    {
      "bot": "Kimi-User",
      "named": 46,
      "fully_blocked": 42
    },
    {
      "bot": "GoogleAgent-URLContext",
      "named": 40,
      "fully_blocked": 37
    },
    {
      "bot": "Google-Gemini-CLI",
      "named": 35,
      "fully_blocked": 32
    },
    {
      "bot": "Cursor",
      "named": 22,
      "fully_blocked": 20
    },
    {
      "bot": "ChatGPT-Agent",
      "named": 8,
      "fully_blocked": 4
    }
  ],
  "dnsAid": {
    "probed": 98330,
    "present": 24
  },
  "notable": {
    "ai_catalog": [
      {
        "host": "huggingface.co",
        "rank": 1176
      },
      {
        "host": "hostinger.com",
        "rank": 1504
      },
      {
        "host": "padlet.com",
        "rank": 2759
      },
      {
        "host": "zapier.com",
        "rank": 2906
      },
      {
        "host": "airtable.com",
        "rank": 2949
      },
      {
        "host": "rudderstack.com",
        "rank": 3057
      },
      {
        "host": "nextjs.org",
        "rank": 10756
      },
      {
        "host": "gtmetrix.com",
        "rank": 11507
      },
      {
        "host": "speedof.me",
        "rank": 13035
      },
      {
        "host": "apify.com",
        "rank": 13443
      },
      {
        "host": "bestprice.gr",
        "rank": 15352
      },
      {
        "host": "railway.com",
        "rank": 18185
      },
      {
        "host": "mentimeter.com",
        "rank": 20841
      },
      {
        "host": "bird.com",
        "rank": 22245
      },
      {
        "host": "bizzabo.com",
        "rank": 24690
      }
    ],
    "llms_txt": [
      {
        "host": "cloudflare.com",
        "rank": 2
      },
      {
        "host": "github.com",
        "rank": 29
      },
      {
        "host": "wordpress.org",
        "rank": 47
      },
      {
        "host": "adobe.com",
        "rank": 68
      },
      {
        "host": "opera.com",
        "rank": 92
      },
      {
        "host": "samsung.com",
        "rank": 94
      },
      {
        "host": "wordpress.com",
        "rank": 98
      },
      {
        "host": "capgemini.com",
        "rank": 101
      },
      {
        "host": "kaspersky.com",
        "rank": 140
      },
      {
        "host": "dropbox.com",
        "rank": 142
      },
      {
        "host": "gravatar.com",
        "rank": 157
      },
      {
        "host": "paypal.com",
        "rank": 166
      },
      {
        "host": "shopify.com",
        "rank": 175
      },
      {
        "host": "webex.com",
        "rank": 177
      },
      {
        "host": "rubiconproject.com",
        "rank": 195
      }
    ],
    "agents_md": [
      {
        "host": "shopify.com",
        "rank": 175
      },
      {
        "host": "checkpoint.com",
        "rank": 350
      },
      {
        "host": "paloaltonetworks.com",
        "rank": 410
      },
      {
        "host": "shop.app",
        "rank": 637
      },
      {
        "host": "crazyegg.com",
        "rank": 2683
      },
      {
        "host": "razorpay.com",
        "rank": 3285
      },
      {
        "host": "nazwa.pl",
        "rank": 4223
      },
      {
        "host": "openweathermap.org",
        "rank": 4241
      },
      {
        "host": "nanit.com",
        "rank": 4386
      },
      {
        "host": "eufylife.com",
        "rank": 4898
      },
      {
        "host": "moovitapp.com",
        "rank": 5093
      },
      {
        "host": "laravel-news.com",
        "rank": 5176
      },
      {
        "host": "wyze.com",
        "rank": 6578
      },
      {
        "host": "gbnews.com",
        "rank": 6586
      },
      {
        "host": "jbhifi.com.au",
        "rank": 7185
      }
    ],
    "ard_json": [
      {
        "host": "railway.com",
        "rank": 18185
      },
      {
        "host": "bizzabo.com",
        "rank": 24690
      },
      {
        "host": "idescat.cat",
        "rank": 64770
      },
      {
        "host": "brandfetch.com",
        "rank": 66676
      }
    ]
  },
  "disclaimer": "Adoption and access signals do not demonstrate citations, recommendations or visibility inside any AI system."
}