Census / live & auto-updating
The agent-facing web — a census
The rest of this site measures the agents that visit us. This page turns that around: FomiteBot, a small, disclosed, honestly-identified crawler, visits sites that already reached out to agents — their own +https:// calling card, left in their User-Agent, is how we found them — and records, structurally, which agent-facing surfaces they actually expose. See /methodology for exactly how a probe is scored.
303Sites crawled
24.4%Expose any MCP surface
18Block our datacenter IP
5.9%Blocked, of crawled
1. Which surfaces sites actually expose
“Present” means a probe of that exact path returned a structural presence verdict — not a 200. A homepage re-served at every path (a soft-404) is scoredabsent, not present; a robots.txt Disallow is scored disallowed, never routed around.
| Surface | Sites exposing it | % of crawled |
|---|---|---|
| llms.txt | 122 | 40.3% |
| ai.txt | 12 | 4.0% |
| .well-known/mcp.json | 32 | 10.6% |
| .well-known/ai-plugin.json | 8 | 2.6% |
| .well-known/agent.json | 26 | 8.6% |
| /mcp | 64 | 21.1% |
| security.txt | 45 | 14.9% |
| openapi.json | 38 | 12.5% |
2. How robots.txt treats us
fail-closed means the site's own robots.txt could not be fetched or parsed cleanly and we treated that as a refusal rather than assume permission; an allow-all after a 404 means no robots.txt existed at all.
| robots.txt outcome | Sites |
|---|---|
| Parsed normally | 187 |
| No robots.txt (allow all) | 34 |
| No applicable group | 14 |
| Fail-closed (fetch/parse failed) | 68 |
| Other | 0 |
3. Site by site
Every crawled site, most recently crawled first, and what it exposes.
| Host | Source | Blocks DC? | llms.txt | ai.txt | .well-known/mcp.json | .well-known/ai-plugin.json | .well-known/agent.json | /mcp | security.txt | openapi.json |
|---|---|---|---|---|---|---|---|---|---|---|
| founderdeck.app | seed | — | — | — | — | — | — | — | — | — |
| index.agentexchange.work | seed | — | yes | — | yes | — | yes | yes | yes | — |
| veristria.com | seed | — | yes | — | — | — | — | — | yes | — |
| tri-111-index.triaconter.workers.dev | seed | — | — | — | — | — | — | — | — | — |
| shadowalpha.online | seed | blocks | yes | — | — | — | — | — | — | — |
| inite.ai | seed | — | yes | yes | yes | — | — | — | yes | — |
| trust.redthreadsec.com | seed | — | — | — | — | — | — | — | yes | — |
| annalist.dev | seed | — | — | — | — | — | — | — | — | — |
| mcp-uptime.theopslog.workers.dev | seed | — | — | — | — | — | — | yes | — | — |
| rokha.ai | seed | — | yes | — | — | — | — | — | — | — |
| vela.partners | seed | — | yes | — | — | — | — | — | — | yes |
| keenable.ai | seed | — | — | — | — | — | — | — | — | — |
| aiagentboard.org | seed | — | yes | — | — | — | — | yes | — | yes |
| www.taifoon.io | seed | — | yes | — | yes | yes | yes | — | yes | yes |
| rhumb.site | seed | — | — | — | — | — | — | — | — | — |
| index.percall.dev | seed | — | yes | — | yes | yes | yes | yes | yes | — |
| slopbucket.app | seed | — | — | — | — | — | — | — | — | — |
| rooms.aurmada.com | seed | — | yes | — | yes | — | yes | yes | — | yes |
| takeitdown.agentexchange.work | seed | — | yes | — | — | — | yes | — | yes | — |
| localmark.ai | seed | — | yes | — | — | — | — | — | — | yes |
| koskamo.com | seed | — | yes | — | yes | — | — | — | — | — |
| glintbase.dev | seed | — | yes | — | — | — | — | — | — | yes |
| api.adadvisor.ai | registry | — | — | — | — | — | — | — | — | — |
| www.adadvisor.ai | registry | — | — | — | — | — | — | — | — | — |
| actwise.ai | registry | — | — | — | — | — | — | — | — | — |
| mcp.abmeter.ai | registry | — | — | — | — | — | — | — | — | — |
| abmeter.ai | registry | — | yes | — | — | — | — | — | — | — |
| mcp.aarna.ai | registry | — | — | — | — | — | — | — | — | — |
| 1325.ai | registry | — | yes | — | — | — | — | yes | — | — |
| mcp.lona.agency | registry | — | — | — | — | — | — | — | — | — |
| lona.agency | registry | — | — | — | — | — | — | — | — | — |
| scan.kesey.agency | registry | — | — | — | — | — | — | yes | — | — |
| mcp.goji.agency | registry | — | — | — | — | — | — | yes | — | — |
| goji.agency | registry | — | yes | — | — | — | — | — | — | — |
| www.hood.ag | registry | — | yes | — | — | — | — | — | — | — |
| propick.ae | registry | — | yes | — | — | — | — | yes | — | — |
| app.inside.ad | registry | — | yes | — | — | — | — | yes | — | — |
| inside.ad | registry | — | yes | — | — | — | — | yes | — | — |
| tandem.ac | registry | — | yes | — | — | — | — | yes | — | — |
| api.inference.sh | registry | — | — | — | — | — | — | — | — | — |
| sh.inference.ac | registry | — | — | — | — | — | — | — | — | — |
| agenttrust.site | seed | — | — | — | — | — | — | — | — | — |
| factanker.com | seed | — | yes | — | yes | — | yes | yes | — | yes |
| help.coccoc.com | seed | — | — | — | — | — | — | — | — | — |
| sleepbot.com | seed | — | — | — | — | — | — | — | — | — |
| dollardollardollar.com | seed | — | yes | — | — | — | — | — | — | — |
| neo-collettive.onrender.com | seed | — | — | — | yes | — | yes | yes | — | — |
| www.postgoo.com | seed | — | — | — | — | — | — | — | — | — |
| mosaiden.com | seed | — | yes | — | yes | — | — | yes | — | — |
| tools.nyrds.net | seed | — | — | — | — | — | — | — | — | — |
| toolvet.app | seed | blocks | — | — | — | — | — | — | — | — |
| mcprush.com | seed | blocks | — | — | — | — | — | — | — | — |
| leashkit.com | seed | — | — | — | — | — | — | — | — | — |
| juno.app | seed | — | yes | — | — | — | — | — | — | — |
| intentroute.net | seed | — | yes | — | — | — | — | — | — | — |
| everymcp.ai | seed | — | — | — | — | — | — | yes | — | — |
| www.letsencrypt.org | seed | — | — | — | — | — | — | — | — | — |
| citefiles.com | seed | — | yes | yes | yes | — | — | — | yes | — |
| scry.io | seed | — | — | — | — | — | — | yes | yes | — |
| domainarrivals.com | seed | — | — | — | — | — | — | — | — | — |
| golemreach.com | seed | — | yes | — | — | — | — | — | — | — |
| ha.imsj114.space | seed | — | — | — | — | — | — | — | — | — |
| mcpqueen.com | seed | — | yes | — | — | — | — | yes | — | — |
| agentreputation.dev | seed | — | yes | — | — | — | — | — | — | — |
| vemetric.com | seed | — | — | — | — | — | — | — | — | — |
| agenstry.com | seed | — | yes | yes | yes | — | yes | — | yes | yes |
| script.google.com | seed | — | — | — | — | — | — | — | — | — |
| toolpin.dev | seed | — | — | — | — | — | — | — | — | yes |
| hultra.link | seed | — | yes | — | — | — | — | — | — | — |
| hackernews.pink | seed | — | — | — | — | — | — | — | — | — |
| fortline.ai | seed | — | — | — | — | — | — | — | — | — |
| crawler.validus.ai | seed | blocks | — | — | — | — | — | — | — | — |
| mcpwitness.com | seed | — | — | — | — | — | — | — | — | — |
| www.linkedin.com | seed | — | — | — | — | — | — | — | — | — |
| api.temsor.com | seed | — | — | — | yes | — | — | — | — | — |
| www.checkmarknetwork.com | seed | — | — | — | — | — | — | — | — | — |
| fossick.bot | seed | — | — | — | — | — | — | — | — | — |
| verifymcp.io | seed | blocks | — | — | — | — | — | — | — | — |
| glimind.com | seed | — | yes | — | — | — | yes | yes | — | yes |
| modc2.com | seed | — | — | — | — | — | — | yes | — | — |
| www.bing.com | seed | — | — | — | — | — | — | — | — | — |
| trimtabist.com | seed | — | yes | — | — | — | — | yes | — | yes |
| zevruna.com | seed | — | yes | — | — | — | — | — | — | — |
| compose.market | seed | — | — | — | — | — | — | — | — | — |
| duckduckgo.com | seed | — | yes | yes | — | — | — | — | yes | — |
| registry.truespar.com | seed | — | — | — | — | — | — | yes | — | yes |
| agenttru.st | seed | — | — | — | — | — | — | — | — | — |
| station70.com | seed | — | — | — | — | — | — | — | — | — |
| www.micronewx.com | seed | — | — | — | — | — | — | — | — | — |
| donnees.hultra.link | seed | — | yes | — | — | — | — | — | — | — |
| discordapp.com | seed | — | — | — | — | — | — | — | — | — |
| forestengine.net | seed | — | — | — | — | — | — | — | yes | — |
| agentsure.tech | seed | — | — | — | — | — | — | — | — | — |
| agentalmanac.org | seed | — | yes | — | — | — | yes | yes | — | yes |
| mcpcentral.io | seed | blocks | — | — | — | — | — | — | — | — |
| hermes-agent.nousresearch.com | seed | — | yes | — | — | — | — | — | — | — |
| support.embed.ly | seed | — | — | — | — | — | — | — | — | — |
| awesome-stateless-mcp.org | seed | — | — | — | — | — | — | — | — | — |
| godigitalpigeon.com | seed | — | — | — | — | — | — | — | — | — |
| llmmart.ai | seed | — | — | — | yes | — | — | yes | — | — |
| www.metarabbit.dev | seed | — | — | — | — | — | — | — | — | yes |
| awesomemcps.io | seed | — | — | — | — | — | — | — | — | — |
| gemini.google | seed | — | — | — | — | — | — | — | — | — |
| tbn.hardinai.co.uk | seed | — | — | — | — | — | — | — | — | — |
| domainsbot.com | seed | blocks | — | — | — | — | — | — | — | — |
| support.google.com | seed | — | — | — | — | — | — | — | — | — |
| connector.raqilla.my.id | seed | — | — | — | — | — | — | — | — | — |
| live-vps.sasame.online | seed | — | yes | — | yes | — | yes | yes | — | — |
| docs.you.com | seed | — | — | — | — | — | — | — | — | — |
| www.misar.ai | seed | — | yes | — | — | — | — | — | — | — |
| spanly.com | seed | — | yes | — | — | — | — | — | — | — |
| iframely.com | seed | — | yes | — | — | — | — | — | — | — |
| getlulu.dev | seed | — | yes | — | — | — | — | — | — | — |
| agents.traderszone.net | seed | blocks | — | — | — | — | — | — | — | — |
| www.facebook.com | seed | — | — | — | — | — | — | — | — | — |
| mcpmeter.dev | seed | — | — | — | — | — | — | — | — | — |
| vouch-protocol.com | seed | — | yes | — | — | — | — | — | — | — |
| mcp-schema-archive.delfrost42.workers.dev | seed | — | — | — | — | — | — | — | — | — |
| discover.paygent.net | seed | — | yes | — | — | — | — | — | — | — |
| mcphq.ai | seed | — | yes | — | — | — | — | yes | — | — |
| agent-tools.cloud | seed | — | — | — | yes | — | yes | yes | — | yes |
| news.gcombinator.com | seed | — | yes | — | — | — | — | — | — | — |
| mcpcensus.com | seed | — | yes | — | yes | yes | — | — | yes | yes |
| crawler.exa.ai | seed | — | — | — | — | — | — | — | — | — |
| stashmark.app | seed | — | — | — | — | — | — | — | — | — |
| www.diffbot.com | seed | — | — | — | — | — | — | — | — | — |
| lastseen.dev | seed | — | yes | — | — | — | — | — | — | yes |
| agentklaar.nl | seed | — | — | — | — | — | — | — | — | — |
| proofbench.dev | seed | — | — | — | — | — | — | — | — | — |
| scour.ing | seed | — | — | — | — | — | — | — | — | — |
| n8n.hakulab.cc | seed | — | — | — | — | — | — | — | — | — |
| zerm.dev | seed | — | yes | — | yes | — | — | yes | — | — |
| solved.earth | seed | — | — | — | — | — | — | — | — | — |
| codepulse.kr | seed | — | — | — | — | — | — | — | — | — |
| 4am.anelabs.ai | seed | — | — | — | — | — | — | — | — | — |
| ottermind.ai | seed | — | yes | — | — | — | — | yes | — | — |
| fetchgate.dev | seed | — | yes | — | — | — | — | yes | — | yes |
| perplexity.ai | seed | — | — | — | — | — | — | — | — | — |
| www.sogou.com | seed | — | — | — | — | — | — | — | — | — |
| mcplookup.com | seed | — | yes | — | — | — | — | yes | yes | — |
| www.pulsegate.ai | seed | — | yes | — | — | — | — | — | yes | yes |
| trypromptflow.com | seed | — | yes | — | — | — | — | — | yes | — |
| mcp-cloud.ai | seed | — | yes | yes | — | — | — | yes | yes | — |
| netapi.com | seed | — | yes | — | — | — | — | — | — | — |
| aive.global | seed | — | — | — | — | — | — | — | — | — |
| mcpbeat.com | seed | blocks | — | — | — | — | — | — | — | — |
| forms.gle | seed | — | — | — | — | — | — | — | — | — |
| scienceguru.ai | seed | — | — | — | — | — | — | — | — | — |
| support.anthropic.com | seed | — | — | — | — | — | — | — | — | — |
| www.baidu.com | seed | — | — | — | — | — | — | — | — | — |
| openai.com | seed | blocks | — | — | — | — | — | — | — | — |
| api.slack.com | seed | — | yes | — | — | yes | — | — | yes | — |
| awario.com | seed | — | — | — | — | — | — | — | — | — |
| code.google.com | seed | — | — | — | — | — | — | — | — | — |
| roninforge.org | seed | — | yes | — | — | — | — | — | — | — |
| ai.zhedor.com | seed | — | yes | — | yes | — | yes | yes | — | yes |
| ironman.silentinfinity.com | seed | — | — | — | — | — | — | — | — | — |
| techfinx.top | seed | — | — | — | — | — | — | — | — | — |
| superpowerdaily.com | seed | — | yes | — | — | — | — | — | — | — |
| runai.live | seed | — | yes | — | — | — | — | — | — | — |
| cobec-spark.tail336878.ts.net | seed | — | — | — | — | — | — | — | — | — |
| repology.org | seed | — | — | — | — | — | — | — | — | — |
| wpnews.pro | seed | — | yes | yes | yes | — | yes | yes | — | — |
| kobolindustries.com | seed | — | — | — | — | — | — | — | — | — |
| saasearch.io | seed | — | yes | — | — | — | — | — | — | yes |
| threatintel.securityresearch.net | seed | — | — | — | — | — | — | — | — | — |
| rsscal.com | seed | — | — | — | — | — | — | — | yes | — |
| raven.local | seed | — | — | — | — | — | — | — | — | — |
| exxxtramedia.com | seed | — | — | — | — | — | — | — | — | — |
| kcap.halter.cloud | seed | — | — | — | — | — | — | — | — | — |
| newsomo.com | seed | — | — | yes | — | — | — | — | — | — |
| miniflux.app | seed | — | — | — | — | — | — | — | — | — |
| markonote.com | seed | — | — | — | — | — | — | — | — | — |
| hn.ysh.kr | seed | — | — | — | — | — | — | — | — | — |
| news.ycombinator.com | seed | — | — | — | — | — | — | — | — | — |
| www.gitlen.com | seed | — | — | — | — | — | — | — | — | — |
| flipboard.com | seed | — | — | — | — | — | — | — | — | — |
| flint.app | seed | — | — | — | — | — | — | — | — | — |
| www.fiper.net | seed | blocks | — | — | — | — | — | — | — | — |
| feedsearch.dev | seed | — | — | — | — | — | — | — | — | — |
| www.esds.co.in | seed | — | yes | — | — | — | — | — | — | — |
| www.dataaihub.co | seed | — | — | — | — | — | — | — | — | — |
| contentstudio.io | seed | — | yes | — | — | — | — | — | — | — |
| companymap.co.uk | seed | blocks | — | — | — | — | — | — | — | — |
| lighthouseapp.io | seed | — | — | — | — | — | — | — | — | — |
| metascope.dev | seed | — | — | — | — | — | — | — | — | — |
| zerobytes.monster | seed | blocks | — | — | — | — | — | — | — | — |
| knowngood.sh | seed | — | yes | — | — | — | — | yes | yes | yes |
| intellimedia.co.za | seed | — | — | — | — | — | — | — | — | — |
| hidden-view.com | seed | — | — | — | — | — | — | — | — | — |
| grails.com | seed | — | yes | yes | — | — | — | yes | yes | — |
| clarflow.cn | seed | — | — | — | — | — | — | — | — | — |
| aisos.cn | seed | — | — | — | — | — | — | — | — | — |
| brick.blue | seed | — | yes | — | yes | yes | yes | yes | yes | — |
| marketing.vilcom-net.co.ke | seed | — | — | — | — | — | — | — | — | — |
| allslop.news | seed | — | — | — | — | — | — | — | — | — |
| aiwire.example | seed | — | — | — | — | — | — | — | — | — |
| news.ajaiclub.com | seed | blocks | — | — | — | — | — | — | — | — |
| reddy.team | seed | — | — | — | — | — | — | — | — | — |
| agencyanalytics.com | seed | — | yes | — | — | — | — | — | — | — |
Cite this
Every figure here is recomputed live and is CC0. Take the JSON at /api/agent-web, read how it is measured (with the weaknesses stated) at /methodology, and read about the crawler itself, how to verify it, and how to block it at /fomitebot. If your site is listed here and something is wrong, tell us and we will fix it and say that we did.
The Fomite is a field station on the agentic web. Start here · FomiteBot, the crawler · the Observatory.