io.github.psinglap/websearch-api-eval

Web Search API Eval

Is the Web Search API Eval MCP server down? No. It answered initialize and tools/list in the probe of 2026-10-27.

Web search API benchmark for AI agents: You.com, Tavily, Exa, Parallel. Results and charts.

alive probed 2026-11-14 · unchanged since 2026-10-27 · HTTP 200

11tools

2.0ktokens in Claude Opus 5.5

1.0 %of Claude's 200k window

0.8×the median server

59 msto initialize

39 msto list tools

Endpoint: https://websearch-api-eval.psberkeleymba.workers.dev/mcp (streamable-http)
Host: websearch-api-eval.psberkeleymba.workers.dev
Protocol: 2025-11-25
Reports itself as: websearch-api-eval 1.1.0
Capabilities: tools
Instructions: 568 characters
Of a 1M window: 0.2 % of a 1M context window
Count: 2.0k tokens in Claude Opus 5.5, exact, measured 2026-10-27. With the Claude Code prefix mcp__…__ on each tool name: 2.2k. The o200k_base estimate is 1.2k. Counted again when the tool definitions change. Method.
Repository: https://github.com/psinglap/websearch-api-eval
Website: https://psinglap.github.io/websearch-api-eval/
Registry: active, updated 2026-10-27 · registry entry

Most expensive tools

The Claude column scales the o200k estimate of each tool by this server's ratio of exact to estimate, 1.73×.

Tool	≈ Claude tokens	o200k estimate	Share of this server
compare_providers	340	196	17 %
get_charts	246	142	12 %
get_scorecard	227	131	11 %
recommend_provider	203	117	10 %
get_question	194	112	10 %

Servers that mention websearch.

The result of the latest probe, the tool count and the token cost. The feed has an entry for every probe that changed them.

Stops answering, comes back, tool set or cost changes. One confirmation mail first, a stop link in every mail.

Fixed something? Probe it now, with the same steps as the daily run. This page updates with the next daily run.