io.github.ryuamarines/robots-txt-checker
ai-factory-robots-txt-checker
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
timeout probed 2026-10-15 · HTTP 200
probe timed out after 15000 ms
- Endpoint
- https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp (streamable-http)
- Host
- faceted-napped-debtless.ngrok-free.dev
- Protocol
- 2025-11-25
- Reports itself as
- ai-factory-robots-txt-checker 1.0.0
- Capabilities
- tools
- Registry
- active, updated 2026-09-02 · registry entry
- Reachable
- 6 of the last 6 probes
History
We add a row when the result changes. A server with no change keeps its last row.
| Probed | Result | Init | Tools | Claude | o200k |
|---|---|---|---|---|---|
| 2026-10-05 15:37 | alive | 486 ms | 1 | 278 | 159 |
| 2026-10-05 00:46 | alive | 566 ms | 1 | 159 | |
| 2026-10-04 10:07 | alive | 353 ms | 1 | 159 | |
| 2026-10-04 08:00 | alive | 534 ms | 1 | 159 | |
| 2026-10-03 20:39 | alive | 428 ms | 1 | 159 | |
| 2026-10-03 17:55 | alive | 505 ms | 1 | 159 |
Badge and feed
[](https://mcpcost.ulehla.dev/s/io.github.ryuamarines/robots-txt-checker)
An entry for every probe that changed the result, the tool count or the token cost: Atom feed. Add it to a feed reader or a Slack RSS app to hear when the server stops answering or its tool set grows.