Probabilistic Kernel Function for Fast Angle Testing
Kejing Lu, Chuan Xiao, Yoshiharu Ishikawa
We propose two probabilistic kernel functions for angle testing, which can be used to accelerate vector-based similarity search.
Abstract
In this paper, we study the angle testing problem in the context of similarity search in high-dimensional Euclidean spaces and propose two projection-based probabilistic kernel functions, one designed for angle comparison and the other for angle thresholding. Unlike existing approaches that rely on random projection vectors drawn from Gaussian distributions, our approach leverages reference angles and adopts a deterministic structure for the projection vectors. Notably, our kernel functions do not require asymptotic assumptions, such as the number of projection vectors tending to infinity, and can be theoretically and experimentally shown to outperform Gaussian-distribution-based kernel functions. We apply the proposed kernel function to Approximate Nearest Neighbor Search (ANNS) and demonstrate that our approach achieves a 2.5x--3x higher query-per-second (QPS) throughput compared to the widely-used graph-based search algorithm HNSW. Our code and data are available at https://github.com/KejingLu-810/KS.
Proposes probabilistic kernel functions for angle testing enabling efficient approximate nearest neighbor search.
- Two projection-based probabilistic kernel functions for angle comparison and thresholding
- Leverages reference angles and deterministic projection vector structure without asymptotic assumptions
- Achieves 2.5-3x higher query-per-second throughput compared to HNSW algorithm
- Probabilistic kernel functions
- Angle testing
- Projection-based methods
- Approximate nearest neighbor search
- Graph-based search
- SIFT
- Word2Vec
- GIST
Authors did not state explicit limitations.
Authors did not state explicit future directions.
Author keywords
- Randomized algorithm
- Locality Sensitive Hashing
- Directional statistics
Related orals
TileLang: Bridge Programmability and Performance in Modern Neural Kernels
TileLang enables hardware-aware fused kernel programming with tile inference and recommendation achieving 5-6x speedup.
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
Generates minute-long high-resolution videos efficiently with linear attention and constant-memory KV cache for block autoregression.
Efficient Resource-Constrained Training of Transformers via Subspace Optimization
WASI applies subspace-based training to transformer models reducing memory by 62x and FLOPs by 2x while maintaining accuracy on edge devices.
Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
Analyzes low-precision flash attention training failure caused by low-rank representations and biased BF16 rounding errors.
Speculative Actions: A Lossless Framework for Faster AI Agents
Speculative Actions accelerates agent systems by predicting and executing likely future actions in parallel.