Building the Modern AI Infrastructure Stack with Cortex AI Gateway

The enterprise AI stack has grown fragmented, with vector databases, monitoring systems, and agents connecting to models through inconsistent protocols, creating governance gaps and security risks. Few organizations can track which models are used, who accesses them, or the associated costs, leading to restrictive policies that push employees toward unsanctioned tools. Cortex AI Gateway aims to resolve this by serving as a centralized control plane between AI clients and enterprise systems, offering unified visibility, governance, and oversight. It supports major model protocols like Chat Completions and Messages APIs, enabling existing clients to route traffic through a single endpoint without code changes.

Cortex AI Gateway provides platform teams with a single interface to manage model access, control spending, and monitor AI activity across the organization. It enforces role-based access control (RBAC) to restrict models to approved options and simplifies setup for coding agents via the Snowflake CLI, which automates credential injection and routing. The gateway also supports dynamic model routing, which selects the most cost-effective model for each task based on complexity, reducing token usage while maintaining quality. Internal tests showed up to 3x greater token efficiency in workloads and 25% fewer tokens in coding tests compared to frontier-model-only approaches.

Dynamic model routing in Cortex AI Gateway is governed by the same policies as manual selection, ensuring compliance with data residency and logging every decision for audit trails. Updates to routing logic can be applied centrally without requiring changes to agent configurations, adapting to shifts in model pricing and performance. The gateway also integrates open-source models like DeepSeek-V4-Flash, which outperformed leading proprietary models on ADE-bench, and GLM-5.3, which offers high efficiency with a low token footprint. These models are served within Snowflake’s secure perimeter, aligning inference with governed data access.

Cortex AI Gateway introduces a tool governance layer to address risks from unmanaged MCP servers, which can act as shadow IT with access to production systems. The tool catalog includes over 100 curated MCP servers with OAuth handled automatically, allowing administrators to enable or restrict tools centrally. Tool calls are logged and traced alongside inference, creating a full audit trail that mitigates risks like tool poisoning or shadowing. This layer ensures that only authorized agents can perform specific tool calls, reducing exposure to runtime authorization failures.