Run Scale AI’s AgentEnv evals on Vercel Sandbox

Scale AI’s AgentEnv, a framework designed to evaluate agents by testing them against tasks and scoring outcomes, now supports execution on Vercel Sandbox. The integration is available starting with AgentEnv version 0.9.1334, which includes a Vercel provider to run the agent and its task environment in isolated sandboxes. This setup allows developers to assess agent performance in a controlled environment while maintaining separation between the agent and the task being evaluated.

To begin using AgentEnv on Vercel Sandbox, users can follow the provided setup guide to configure their environment. The process involves cloning the AgentEnv repository and running a specific command that leverages an existing Vercel CLI login. This command initiates the evaluation process, ensuring the agent operates within a sandboxed instance of the specified Vercel project.

The evaluation results include both the task’s score and a detailed log of the agent’s recorded actions, offering transparency into performance metrics. Users can inspect these outputs to analyze how the agent interacts with tasks and identify areas for improvement. The sandboxed execution helps isolate variables, making it easier to attribute performance outcomes to specific agent behaviors.

The integration requires users to replace placeholders in the command with their Vercel team slug and an existing project name. This ensures the evaluation runs within the correct sandboxed environment, aligning with the user’s existing Vercel infrastructure. The setup is designed to be compatible with standard Vercel CLI workflows, simplifying the adoption process for developers already using Vercel’s platform.