Submitted on October 16, GatewayBench studies a layer less visible than the model: the gateway exposing MCP tools to an agent. Its benchmark contains 524 tasks across six domains and a pool of 11,118 tools from 251 servers. The authors inject task-irrelevant tools to measure the effects of a growing action surface.
The paper contrasts flattened gateways, which present tools directly, with hierarchical gateways that add discovery mechanisms. Experiments across seven gateways examine success, efficiency and resilience under adversarial tasks. The authors find strong interactions with model and harness: a gateway cannot be assessed independently of the agent using it.
Hierarchy is not a free win. With few irrelevant tools, flattened designs can reduce interaction turns and latency; as catalogues grow, context-processing costs can reverse that advantage. The useful contribution is a comparison protocol under load and attack, not a security certification for a real MCP server. These are preprint findings and were not independently reproduced here.