Building a software factory for AI SDK

The AI SDK, a widely used open-source project with over 20 million weekly npm downloads, faced an unsustainable backlog of over 1,000 open issues and 800 pull requests by June. Maintainers recognized that scaling human effort alone was insufficient, so Vercel developed an AI-driven software factory to automate parts of the workflow. The system now handles 25-35% of merged pull requests and resolves 70-80% of issues autonomously, reducing the manual workload while preserving human oversight for critical decisions.

The factory operates on a spectrum of automation, balancing efficiency with risk management. It prioritizes human accountability, ensuring that every change undergoes review before merging. Agents perform specific tasks such as bug reproduction, feature implementation, and backporting, while a centralized roadmap and risk assessments guide their work. Higher-risk changes trigger deeper human scrutiny, with agents providing documented evidence to streamline the review process.

Security was a core focus during development, as the factory processes untrusted input from public repositories. Vercel implemented isolated sandboxes for each agent, restricting access to only necessary resources and secrets. A shielding layer further blocks network paths that could be exploited for data exfiltration or supply chain attacks. These measures ensure that even if an agent is compromised, the impact remains contained.

The system was built incrementally, starting with issue classification before expanding to bug reproduction and fixes. Early prototypes used a local CLI for rapid iteration, but the final version runs on managed infrastructure. Human reviewers retain final control, merging only changes approved by the AI SDK team. This approach aims to sustain the project’s growth without sacrificing quality or security.