Deep, evolving environments for computer-use agents

Microsoft Research built twelve training environments—ten deep domain worlds and two capability worlds—each simulating real application behavior with coherent state changes across screens and users. These environments are seeded with realistic data and designed to reflect the consequences of actions, such as state changes or message deliveries, which are critical for training agents. The approach addresses limitations of open, login-free sites, which are unstable for training due to frequent redesigns and data changes. By controlling the environment, researchers ensure consistency and safety while enabling agents to learn from meaningful consequences rather than superficial interactions.

The research highlights the importance of depth in training environments, where workflows preserve dependencies, permissions, and shared state rather than merely rehearsing isolated actions. A 9B model trained on these environments nearly doubled its base score, approaching performance levels seen in larger models. The work emphasizes that shallow environments, which mimic isolated clicks without state changes, can reinforce incorrect behaviors, while deep environments that reflect real workflows lead to measurable improvements in agent performance on live applications.

The synthetic worlds are structured around three components: an environment that models the application and its state, tasks that define goals, and verifiers that grade outcomes against ground truth. Microsoft’s Echoverse pipeline generates these worlds, including domains like communication, banking, and healthcare, where real data is either sourced from public datasets or generated under strict constraints. Tasks are designed to be challenging, often requiring multiple dependent actions, and are validated against live applications to ensure realism and feasibility before being used for training.

The training loop is iterative, with each graded run improving both the model and the environment. Failures are analyzed to identify weaknesses in the world, backend, frontend, or task design, and fixes are applied before retraining. Tasks are exported with precise grading criteria, and the process ensures that only high-quality, grounded data is used for fine-tuning. The approach demonstrates that co-evolving the environment and the model, rather than treating them as separate stages, leads to more robust and transferable agent performance in complex, real-world systems.