Four evaluation engineering lessons learned from building agent plugins

Evaluation engineering is essential for reliable AI agents, as measuring true behavior determines whether a tool can be trusted or risks silent failures. Google’s new Cloud Developer plugin served as a testbed to apply these principles in real developer workflows. Initial evaluations using an LLM-as-judge to grade final text replies proved unreliable, as the grader missed correct tool executions that weren’t explicitly summarized in the agent’s response.

Grading agents solely on their closing pitch overlooks critical execution details. Google found that automated graders penalized agents for performing tasks correctly if they didn’t explicitly document each step in their final message. This revealed the need to score raw execution trajectories, including actual tool calls and validations, rather than relying on the agent’s narrative summary.

Extracting agent telemetry requires preserving turn structures to avoid masking critical behavioral flaws. Google observed that modern harnesses often batch tool calls, which can disrupt multi-stage skill activations by injecting guidance after commands were already executed. Flattened telemetry pipelines obscured these issues, leading to misdiagnosed failures.

Running live and plan-only evaluations in parallel exposes different failure modes. Google’s tests showed that live runs revealed permission errors, while plan-only runs exposed reasoning flaws, such as ignoring governance policies. Roughly half of their key findings emerged from comparing the two conditions, highlighting the importance of dual testing environments.