Hermes Agent Ships Tool Search for MCP: Anthropic Evals Show 49% to 74% Accuracy Gain on Opus 4

Hermes Agent dramatically boosts AI tool searches within Anthropic’s models, promising significant accuracy gains but raising questions about reliance on proprietary BM25 indexing.

Nous Research’s Hermes Agent is tackling the persistent problem of “MCP context bloat” – the tendency of large language models like Anthropic’s Opus 4 to overwhelm themselves with irrelevant information – by introducing a tool search system. This system, built around a progressive schema disclosure using BM25, allows the model to actively query external tools for specific data, dramatically improving the quality of responses. Anthropic’s internal evaluations revealed a stunning 49% to 74% accuracy improvement on Opus 4, suggesting a substantial leap forward in how these models handle complex queries.

The Real Impact on Users

Developed by Nous Research, Hermes Agent leverages BM25, a well-established information retrieval technique, but adapts it for an agent-based system. Essentially, when a user poses a question, the agent doesn’t just rely on the model's internal knowledge; it proactively searches for relevant tools – likely incorporating a knowledge graph or external databases – to refine the response. This process, detailed in a recent MarkTechPost article, utilizes a “progressive schema disclosure” approach, meaning the model initially asks broad questions about the topic and then narrows down its search based on the responses it receives, optimizing efficiency and accuracy. This work was released in May 29th, 2026.

So, what does this mean for users, developers, and businesses? For users, it translates to more accurate, relevant, and concise answers from sophisticated AI models, especially when tackling complex tasks. Developers building on Anthropic’s platform will find a powerful new tool to integrate, potentially unlocking entirely new applications for the Opus 4 model. Businesses leveraging these models for customer service, research, or content creation could see a significant return on investment due to improved output quality and reduced need for human intervention.

This development aligns perfectly with the broader trend of “agentic AI,” where LLMs are evolving beyond simple chatbots to become proactive problem-solvers. We’re seeing a shift from models passively responding to prompts to actively seeking out information and coordinating actions – a movement heavily driven by the need to manage the ever-increasing complexity of data and the limitations of raw model knowledge. The focus on efficient indexing and retrieval, like Hermes Agent demonstrates, is critical to scaling these agentic systems effectively.

What Happens Next

However, this success also raises critical concerns. Dependence on BM25, a proprietary indexing method, concentrates power within Nous Research and Anthropic, potentially limiting innovation and creating vendor lock-in. Furthermore, the accuracy gains, while impressive, rely on the quality of the tools being searched – if those tools themselves are flawed or biased, the entire system suffers. It’s imperative to rigorously examine the underlying data sources and algorithms powering Hermes Agent, and to consider the potential for unintended consequences, particularly regarding information bias and the potential for manipulation.