Amazon Redshift introduces long-term system table retention with Amazon S3 Tables integration

Amazon Redshift has introduced a feature that enables long-term retention of system table data by automatically writing it to Amazon S3 Tables in Apache Iceberg format. This addresses the previous 7-day retention limit, allowing customers to meet compliance, auditing, and observability needs without manual intervention. The integration eliminates the need for custom ETL pipelines, reducing operational overhead and resource contention with production workloads.

Previously, extending system table retention required building and maintaining custom pipelines to copy data, adding complexity and effort. Customers managing multiple data warehouses also faced challenges consolidating system table data for cross-warehouse analysis. The new feature automates replication, enabling centralized storage and analysis of system table data across warehouses without additional setup.

System table data stored in Apache Iceberg format can be queried using Redshift, Amazon Athena, or any Iceberg-compatible engine. This flexibility allows customers to build observability dashboards using AWS services or third-party tools without extra operational steps. The AWS Agent Toolkit further supports querying system table data to provide performance insights and optimization recommendations.

The feature is available for Amazon Redshift Provisioned RG and RA3 instances, as well as Amazon Redshift Serverless, across multiple AWS Regions. Customers can enable this capability to enhance compliance, auditing, and operational visibility while reducing the complexity associated with manual data retention processes.