Five rules for evaluating agent skills before they bloat your context window

Google warns that testing agent skills by pasting prompts into chat interfaces or running single trials on developer laptops produces unreliable results. The company notes that skills embedded in markdown files without structured evaluation loops often confuse frontier models rather than assist them. Google’s own testing involved multiple agents, including Gemini, Antigravity, and Claude Sonnet 5, across isolated environments like Harbor pods on GKE Autopilot. The process also contributed upstream capabilities to NVIDIA’s open-source evaluator to refine assessment methods.

Google’s five-rule playbook advises against rewriting skill instructions when evaluations show no improvement, as agents frequently fail to load skills at all. The company highlights two failure modes in larger workspaces: under-triggering, where descriptions focus on implementation rather than intent, and sibling hijacking, where broad phrases in neighboring skills intercept prompts. Google recently open-sourced *skill-reach* to detect frontmatter collisions before deploying container sandboxes, validating manifests and measuring vocabulary overlap against rival skills.

Google emphasizes that standard code linters miss agent-specific risks like prompt injection or silent data exfiltration buried in markdown prose. The company argues that running benchmarks on workstations produces misleading results due to ambient state contamination, such as inherited project IDs or CPU load affecting agent trajectories. Google’s internal testing showed that resource contention can distort agent performance, with slower file scans and subprocess timeouts leading to premature abandonment of search paths.

Google rejects single-shot testing and screenshots as valid evaluations, noting that multi-turn agents branch on shell exit codes and stdout snippets. The company found that over a third of benchmark tasks produced varying scores across consecutive runs of identical code and prompts. Google advocates for paired ablation testing, where baseline and skill-equipped agents run in clean sandboxes to compare improvements, revealing hidden inefficiencies or native capabilities in the base model.