io.github.Srikarmk/evalpair
evalpair
Is the evalpair MCP server down? No. It answered initialize and tools/list in the probe of 2026-11-06.
Run LLM eval suites on local or hosted models; compare runs with intervals and a noise floor.
alive probed 2026-11-14 · unchanged since 2026-11-06
- Run it
- uvx evalpair@0.1.1 (stdio, probed in a container with the network off)
- Host
- pypi.org
- Reports itself as
- evalpair 0.1.1
- Capabilities
- prompts, resources, tools
- Instructions
- 641 characters
- Of a 1M window
- 0.2 % of a 1M context window
- Count
- 1.5k tokens in Claude Opus 5.5, exact, measured 2026-11-05. With the Claude Code prefix mcp__…__ on each tool name: 1.6k. The o200k_base estimate is 882. Counted again when the tool definitions change. Method.
- Repository
- https://github.com/Srikarmk/evalpair
- Registry
- active, updated 2026-11-05 · registry entry
- Changes
- 2 recorded, reachable in 2 of them. A row is added when the result, the tool count or the cost changes.
Most expensive tools
The Claude column scales the o200k estimate of each tool by this server's ratio of exact to estimate, 1.72×.
| Tool | ≈ Claude tokens | o200k estimate | Share of this server |
|---|---|---|---|
| run_eval | 762 | 443 | 50 % |
| compare_runs | 268 | 156 | 18 % |
| show_run | 210 | 122 | 14 % |
| list_runs | 194 | 113 | 13 % |
| list_suites | 83 | 48 | 5 % |
Compare this server with others.
More servers
Other registry entries on pypi.org, the ones next to this name; all of them.
- SpecLeft io.github.SpecLeft/specleft · crashed on start
- SportAPI io.github.SportApi-net/sportapi-mcp · alive
- Boltwork io.github.Squidboy30/boltwork-mcp · crashed on start
- bostonmove io.github.Srikarmk/bostonmove · alive
- whygit io.github.Srikarmk/whygit · alive
- repo2graph io.github.Srinivasan-78/repo2graph · needs args
- Fleetpost io.github.StanimirTenev/fleetpost-mcp · alive
- Fireweed io.github.Starksood/fireweed-mcp · alive
Servers that mention intervals or compare.
- AirShelf Catalog ai.airshelf/catalog · alive
- FastMetal ai.fastmetal/fastmetal · needs auth
- FindIP ai.findip/patent-search · needs auth
- Gondola Award Travel Search ai.gondola/gondola · alive
- INITE Visibility ai.inite/inite-visibility · alive
- La Luer — AI Skincare Commerce ai.searchshop.www/la-luer · alive
- Apostle — AI Men's Skincare Commerce ai.searchshop/apostleman · alive
- ai.smithery/JMoak-chrono-mcp · not found
History
We add a row when the result changes. A server with no change keeps its last row.
| Probed | Result | Init | Tools | Claude | o200k |
|---|---|---|---|---|---|
| 2026-11-06 08:40 | alive | 1.7 s | 5 | 1.5k | 882 |
| 2026-11-05 08:40 | alive | 1.9 s | 5 | 1.5k | 882 |
Badge for the README
The result of the latest probe, the tool count and the token cost. The feed has an entry for every probe that changed them.
A mail when this server changes
Stops answering, comes back, tool set or cost changes. One confirmation mail first, a stop link in every mail.
For the publisher
- The server answers initialize and tools/list. Nothing to fix.