Cross-cloud conversational analytics made simple with open standards

Organizations often struggle with data scattered across multiple clouds and operational systems, forcing a choice between complex extraction pipelines or fragmented reporting. Traditional approaches rely on scheduled pipelines that copy remote datasets into a central warehouse, introducing synchronization delays, storage costs, and network egress fees. When upstream schemas change, these pipelines frequently break, requiring teams to spend significant time reconciling inconsistent dashboards. The new solution addresses these challenges by allowing compute and AI workloads to operate directly on remote data without moving it.

The announcement introduces a federated approach using the open Apache Iceberg REST catalog specification, enabling BigQuery to query data in place across Databricks, Snowflake, and AWS Glue. Metadata synchronization is handled by the Lakehouse runtime catalog, which periodically refreshes schemas, snapshot pointers, and table definitions based on a user-defined schedule. During query execution, compatible engines read Parquet files directly from remote object storage, such as Amazon S3, using temporary credentials issued by the remote platform. All processing occurs within Google Cloud compute resources like BigQuery or Managed Service for Apache Spark.

To establish the connection, administrators must create a federated catalog in Google Cloud, inspect its configuration to retrieve a generated service account, and grant access permissions within the remote platform. The process requires enabling external data access in Databricks Unity Catalog and associating the target schema with an external location. A Databricks Service Principal with a specific UUID must be created, and the EXTERNAL USE SCHEMA privilege granted to it. Federation policies are then configured on the service principal to complete the setup.

Once configured, users can run a single SQL query joining native sales transactions with customer records in Snowflake, product lines in Databricks, and store locations in AWS Glue. BigQuery processes these joins in place, accommodating differences in identifier casing across platforms. Federated Iceberg tables behave like native tables within BigQuery, allowing downstream features such as conversational analytics to operate directly on the consolidated data using natural language prompts.