What’s new with Google Data Cloud

Pub/Sub now supports AI inference for models hosted in the Gemini Enterprise Agent Platform, appending predictions to event streams for downstream processing in systems like BigQuery or BigTable. This generally available feature aims to streamline anomaly detection workflows by integrating real-time model outputs directly into event data pipelines. The approach reduces the need for separate inference layers, enabling faster response times in operational analytics environments.

Dataflow’s pause-on-failure feature, now generally available, allows users to preserve batch job state before failure, enabling resumption without data loss. The capability addresses temporary outages or capacity constraints by pausing pipelines externally, reducing resource waste and improving job reliability. This is particularly useful for long-running batch processes where interruptions can lead to significant reprocessing costs.

The legacy BigQuery insertAll streaming API has been rebranded as the BigQuery Storage Write API (REST), offering a stateless JSON-over-HTTPS endpoint for lightweight streaming workflows. Existing users require no code changes due to full backward compatibility, while the gRPC version remains recommended for high-throughput pipelines. The shift simplifies integration for serverless applications, IoT telemetry, and AI logging without altering existing implementations.

A new synthetic data generator tool for Managed Service for Kafka is now generally available, enabling users to start streaming mock data to a cluster in under two minutes with minimal setup. The tool eliminates the need to modify client applications or provision additional infrastructure, making it ideal for testing new clusters or validating features. It supports rapid iteration for developers who require immediate data flow without the overhead of production-grade data generation.