Better tools made Copilot code review worse. Here's how we actually improved it.

GitHub initially expected replacing Copilot code review’s custom tools with shared Unix-style utilities like grep and glob would improve performance. However, benchmarks showed higher review costs and fewer issues caught. The tools themselves were not flawed; the problem lay in the instructions guiding the agent’s workflow. Copilot code review’s original tools were designed for earlier models that made fewer, broader tool calls, embedding extra context automatically. The shift to shared tools required rethinking how the agent approached reviews to avoid inefficiencies.

The Copilot CLI harness uses grep, glob, and view for code exploration across multiple products, aiming to reduce duplication and improve consistency. GitHub tested these shared tools in Copilot code review to streamline infrastructure. While the goal was to simplify tool maintenance, the initial results were counterproductive. The agent’s behavior shifted toward broad repository exploration rather than targeted pull request review, increasing token costs and diluting focus. Internal benchmarks revealed the agent was carrying unnecessary context forward, mimicking a coding assistant rather than a reviewer.

The core issue was a mismatch between the tools’ default instructions and Copilot code review’s specific task. Coding assistants benefit from broad exploration to understand repositories, but reviewers prioritize narrow, evidence-based analysis starting from the diff. The shared tools’ generic guidance led the agent to adopt an inefficient workflow, searching broadly and reading excessive code. GitHub’s traces showed the agent was behaving as if it were mapping an entire codebase, not evaluating a single change. This highlighted the need to tailor instructions to the review’s unique requirements.

GitHub addressed the problem by rewriting the tool instructions to align with a reviewer’s workflow. The new guidance emphasized starting from the diff, using grep and glob to narrow searches, and reading exact evidence with view. If searches failed, the agent was instructed to retry with simpler queries or pivot to glob instead of guessing paths. This approach reduced average review cost by roughly 20% while maintaining the same review quality. The fix demonstrates how workflow adjustments around shared tools can yield better outcomes than tool replacements alone.