What changed · 2-6 September 2026

227 new surfaces looked real. 157 were.

This is the first weekly edition of the diff, and the first in which every movement was re-checked live before publication. It changes what the numbers below mean. The raw comparison between the 2 and 6 September passes found 227 new surfaces and 91 withdrawals. Each one was then queried again, and the ones that did not hold up were dropped rather than published with an asterisk: 157 arrivals and 72 withdrawals survived. Roughly a third of the new surfaces and a fifth of the withdrawals were noise, most often a server that answers every request alike, or one that has started blocking us and looks from the outside like a site that removed a file. The largest confirmed arrival is github.com, Tranco rank 29, now serving /.well-known/ai-catalog.json with a single entry pointing at its MCP server card. supabase.com, cal.com and tailscale.com arrived the same week. Counts cover only surfaces read successfully on both passes, so a failed read at either end is excluded instead of counted as a change.

Started publishing 157

Domains that did not have the file on the earlier pass and did on the later one. Not "they published it this week": it is when we saw it.

Surface	Domains
`/llms.txt`	106
`/.well-known/ai-catalog.json`	11
`/.well-known/ard.json`	9
`/agents.md`	8
`robots.txt AI directives`	8
`/.well-known/agent-card.json`	7
`/.well-known/mcp.json`	7
`DNS-AID record`	1

Withdrawn 72

The file was there and is now a definitive 404 or 410. A failed read does not count as a withdrawal.

Surface	Domains
`robots.txt AI directives`	39
`/llms.txt`	17
`/.well-known/ard.json`	4
`/.well-known/agent-card.json`	4
`/agents.md`	3
`/.well-known/mcp.json`	3
`/.well-known/ai-catalog.json`	2

Edited 5253

Same file, different content. Immune to a server flapping: it only counts when both reads succeeded and the bytes differ.

Surface	Domains
`robots.txt AI directives`	2727
`/llms.txt`	1671
`/agents.md`	839
`/.well-known/ai-catalog.json`	5
`/.well-known/ard.json`	4
`/.well-known/mcp.json`	4
`/.well-known/agent-card.json`	3

Download every domain that moved (CSV) · raw JSON

How this is measured

Two complete crawls of the Tranco top 100,000, compared domain by domain. Not a sample.
Only surfaces read successfully on both passes count: 518,783 of them. That is the denominator for every percentage here, and it is smaller than the top 100,000 on purpose.
281,209 surfaces were excluded because at least one of the two reads failed. A timeout is our instrument failing, not the web changing, and counting it as a change is how this kind of number starts lying.
0 were not in the earlier pass at all, so they cannot be called new publishers. They are left out rather than counted.
Servers that answer 200 to any path are flagged by a random-path probe and their "edits" are dropped.

To put a number in context: 157 new publishers is 0.030% of what we could compare. The interesting weeks in this series are likely to be the quiet ones.

Corrections: checkpoint.com (Tranco rank 350) was the highest-ranked withdrawal the automatic verification confirmed, and it is not counted in the withdrawal table above. Checked by hand on 6 September from two networks: the host sits behind an AWS WAF that answers HTTP 202 with an empty body to every path, including /robots.txt and its own homepage. It did not withdraw its /llms.txt, it started blocking us. The movement was reclassified as contradicted rather than deleted, so the published count and the discarded list still add up to the 91 candidates. · Five of the withdrawals counted above sit on four hosts that answer as if publishing when asked for a path that does not exist, so they are weaker than the rest: vnexpress.net, maxwinexch99.com, detran.mg.gov.br and carused.jp. They are reported rather than removed because, unlike checkpoint.com, nothing was verified by hand either way.

Same crawler and methodology as the monthly census. Who runs this and where the data lives: about.