New AI agent observability and governance across BigQuery and Looker
Google Cloud introduces unified AI agent observability and evaluation tools for BigQuery and Looker, enabling enterprises to monitor performance, control costs, and audit agent behavior across first-party and third-party agents.
Enterprises deploying AI agents face operational challenges beyond deployment, including tracking reasoning, data access, and costs. Google Cloud’s new observability and evaluation capabilities aim to address these issues by providing unified visibility into agents built on first-party data in BigQuery and Looker, as well as custom third-party agents developed with frameworks like ADK or LangGraph. The solution focuses on three key areas: tracing multi-step reasoning while safeguarding sensitive data, managing token consumption and query expenses, and validating answer accuracy before changes reach users.
Google’s unified telemetry framework captures logging data from all agents and routes it into BigQuery tables owned by the customer, enabling analysis using existing tools. First-party agents, such as Conversational Analytics agents, automatically emit telemetry without setup, while custom agents built with ADK or LangGraph stream structured logs via a single-line SDK instrumentation. Conversation logging in BigQuery is now generally available, with managed logs including prompts and responses stored in customer-controlled tables, governed by default with audit logs and data residency controls.
Administrators gain prebuilt dashboards in BigQuery Studio and Looker to monitor usage, active sessions, knowledge sources, latency, token consumption, and user feedback. Operational telemetry is separated from conversation content to balance diagnostics with data privacy, with aggregate metrics available via Cloud Monitoring and step-by-step spans via Cloud Trace. Sensitive data such as user prompts and model responses are excluded from operational metrics, while deeper auditing is available through opt-in conversation logging that adheres to row-level security and policy controls.
The new tools include granular token and latency tracking, turnkey observability dashboards, and active spend controls like per-query byte scan caps and daily token quotas. Evaluation benchmarks allow teams to test prompt, model, or schema changes for SQL accuracy and answer quality before deployment. Runtime observability and systematic evaluation work together to ensure agents operate efficiently and reliably in production environments.