Google Cloud’s AlloyDB AI has introduced major improvements to its ScaNN index, enhancing vector search performance for both structured and unstructured data
The efficiency of filtered vector search depends on the order of applying vector search and SQL filters, which is determined by filter selectivity
High selectivity filters (rare values) use pre-filtering: the SQL filter is applied first, reducing the dataset before running the vector search, ensuring 100% recall
Low selectivity filters (common values) use post-filtering: the vector search is performed first, and the SQL filter is applied to the top candidates afterward
Medium selectivity filters enable inline filtering: AlloyDB combines vector search and filtering in a single pass using bitmap indexes for optimal efficiency
AlloyDB’s query planner intelligently chooses between pre-filtering, post-filtering, and inline filtering based on real-time filter selectivity
Adaptive filtering is a new feature that allows AlloyDB to learn and adjust filter selectivity at query time, using live statistics to optimize execution plans