Announcing Native BM25 Ranking in AlloyDB and Cloud SQL

Vector search excels at understanding conceptual meaning but struggles with exact identifiers like product SKUs, necessitating hybrid search that combines semantic and keyword-based retrieval. Until now, achieving this required maintaining a separate full-text search backend, introducing operational complexity and data synchronization challenges. Google Cloud’s new native BM25 index in AlloyDB and Cloud SQL for PostgreSQL 17+ eliminates this need by integrating full-text search directly within the database, leveraging the open-source pg_textsearch extension developed by TigerData.

The update delivers industry-standard keyword ranking through TigerData’s C-optimized BM25 scoring, which addresses limitations of PostgreSQL’s built-in ts_rank by incorporating inverse document frequency, term-frequency saturation, and document length normalization. This ensures rarer terms carry more weight and repetition does not dominate rankings, improving relevance for exact-match queries. Users can now perform hybrid searches by combining vector and keyword indexes on the same table, with AlloyDB offering an out-of-the-box hybrid search user-defined function (UDF) that merges results using the Reciprocal Rank Fusion (RRF) algorithm.

AlloyDB further enhances performance with up to 6x and 10x faster vector search queries compared to standard PostgreSQL, thanks to ScaNN and HNSW index types. This allows applications to prioritize both conceptual and exact-match results seamlessly, such as surfacing ‘California Sycamore’ for a query combining ‘trees that grow taller than houses’ and ‘California’. Cloud SQL users can achieve similar hybrid search functionality using Common Table Expressions (CTEs) and coalescing RRF scores, ensuring consistent performance across both database services.

To implement BM25, users enable the pg_textsearch extension and create an index on text columns like product_description. Queries use a special operator to execute BM25 full-text searches, with results ranked by relevance scores. Hybrid search setups in AlloyDB and Cloud SQL streamline development by unifying vector and keyword retrieval in a single system, reducing infrastructure overhead and improving data consistency for AI and search applications.