Benchmarking Column Statistics for Analytical Query Pruning

Min/max indexes are the most widely used statistics for partition pruning in analytical systems, but they miss pruning opportunities when partitions contain outliers or wide value ranges. Richer statistics, including multiple min/max indexes per partition, Bloom filters, and dictionaries, can improve pruning effectiveness at the cost of additional metadata, yet no study quantifies when this overhead is justified. We present a benchmark for evaluating column statistics for partition pruning, implemented in DuckDB, and assess pruning effectiveness and query runtimes across metadata budgets of 10 × to 1000 × the size of standard min/max indexes. Our results show that richer statistics yield gains for unsorted data, data with outliers, and low-cardinality columns. We derive initial guidelines for selecting column statistics based on data characteristics at load time.