Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs

@article{Malkov2020EfficientAR,
  title={Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs},
  author={Yu A. Malkov and Dmitry A. Yashunin},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  volume={42},
  pages={824-836}
}
  • Yu A. Malkov, D. Yashunin
  • Published 30 March 2016
  • Computer Science
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
We present a new approach for the approximate K-nearest neighbor search based on navigable small world graphs with controllable hierarchy (Hierarchical NSW, HNSW. [] Key Method The maximum layer in which an element is present is selected randomly with an exponentially decaying probability distribution. This allows producing graphs similar to the previously studied Navigable Small World (NSW) structures while additionally having the links separated by their characteristic distance scales.

A Comparative Study on Hierarchical Navigable Small World Graphs

TLDR
It is found that the hierarchical structure in HNSW could not achieve "a much better logarithmic complexity scaling" as it was claimed, particularly on high dimensional data, and similar high search speed efficiency as H NSW could be achieved with the support of flat k-NN graph after graph diversification.

Multiattribute approximate nearest neighbor search based on navigable small world graph

TLDR
A novel approach for multiattribute ANNS based on navigable small world (NSW) graph, called MA‐NSW, which guarantees efficiency and it is defined in terms of arbitrary metric spaces (eg, Euclidean distance and cosine similarity).

Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graph

TLDR
A novel graph structure called Monotonic Relative Neighborhood Graph (MRNG) is proposed which guarantees very low search complexity (close to logarithmic time) and is proposed to further lower the indexing complexity and make it practical for billion-node ANNS problems.

Learning to Prune: General and Efficient Approximate Nearest Neighbor Search with Direction Navigating Graph

TLDR
This paper proposes the concept of Direction Navigating Graph (DNG) to offer an online pruning strategy combined with a heuristic search algorithm and develops a neural network that project queries and vertices into a low-dimensional direction vector space.

Learning to Route in Similarity Graphs

TLDR
This paper augment the vertices of a given graph with additional representations that are learned to provide the optimal routing from the start vertex to the query nearest neighbor, and demonstrates that the proposed learnable routing successfully diminishes the local minima problem and significantly improves the overall search performance.

Understanding and Generalizing Monotonic Proximity Graphs for Approximate Nearest Neighbor Search

TLDR
This paper studies a fundamental model of proximity graphs used in graph-based ANN search, called Monotonic Relative Neighborhood Graph (MRNG), from a theoretical perspective, and uses mathematical proofs to explain why proximity graphs that are built based on MRNG tend to have good searching performance.

GraSP: Optimizing Graph-based Nearest Neighbor Search with Subgraph Sampling and Pruning

TLDR
A graph optimization algorithm GraSP is introduced that improves the search efficiency of similarity graphs by learning to prune redundant edges and performs a novel subgraph sampling and iterative refinement optimization to explicitly maximize search efficiency when removing a subset of edges in expectation over a graph for a large set of training queries.

Hierarchical Satellite System Graph for Approximate Nearest Neighbor Search on Big Data

TLDR
The theoretical analysis reveals that HSSG decreases the search steps and reduces the computational cost and reducing the search time by searching on the hierarchical structure with a similar indexing time compared with SSG, hence reaches a better search efficiency.

Satellite System Graph: Towards the Efficiency Up-Boundary of Graph-Based Approximate Nearest Neighbor Search

TLDR
Inspired by the message transfer mechanism of the communication satellite system, a new family of MSNETs are found, namely the Satellite System Graphs (SSG), which inherits the superior ANNS properties from the MSNET and tries to ensure the angles between the edges to be no smaller than a given value.

A scalable solution to the nearest neighbor search problem through local-search methods on neighbor graphs

TLDR
This paper introduces an algorithm to solve a nearest-neighbor query q by minimizing a kernel function defined by the distance from q to each object in the database, and provides two approaches to select edges in the graph's construction stage that limit memory footprint and reduce the number of free parameters simultaneously.
...

References

SHOWING 1-10 OF 75 REFERENCES

Scalable Distributed Algorithm for Approximate Nearest Neighbor Search Problem in High Dimensional General Metric Spaces

TLDR
The performed simulation for data in the Euclidian space shows that the structure built using the proposed algorithm has navigable small world properties with logarithmic search complexity at fixed accuracy and has weak (power law) scalability with the dimensionality of the stored data.

Navigating K-Nearest Neighbor Graphs to Solve Nearest Neighbor Searches

TLDR
A quasi-proximity graph induced by the all-k-nearest neighbor graph is introduced, which is very good for many proximity searching applications, reviewing just a tiny portion of the database.

Query-Based Improvement Procedure and Self-Adaptive Graph Construction Algorithm for Approximate Nearest Neighbor Search

TLDR
A new algorithm for building graphs with navigable small world properties is proposed that is free from input parameters and has an ability to adapt on the fly to any changes in the distribution of data.

Fast Neighborhood Graph Search Using Cartesian Concatenation

TLDR
Experimental results on searching over large scale datasets (SIFT, GISTand HOG) show that the proposed new data structure for approximate nearest neighbor search outperforms state-of-the-art ANN search algorithms in terms of efficiency and accuracy.

Fast approximate similarity search based on degree-reduced neighborhood graphs

TLDR
The experimental results on a real data set verify the approximation scheme and high search performance of the proposed method and demonstrate that it is superior to E2LSH in terms of the expected search cost.

Approximate Nearest Neighbor Search Small World Approach

TLDR
A novel approach to solving the nearest neighbor search problem by building a non-hierarchical structure with possibility of local minimums which are circumvented by performing a series of searches starting from arbitrary elements of the structure.

FANNG: Fast Approximate Nearest Neighbour Graphs

  • Ben HarwoodT. Drummond
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
TLDR
A new method for approximate nearest neighbour search on large datasets of high dimensional feature vectors, such as SIFT or GIST descriptors, is presented, which is significantly more efficient than existing state of the art methods.

Efficient k-nearest neighbor graph construction for generic similarity measures

TLDR
N-Descent is presented, a simple yet efficient algorithm for approximate K-NNG construction with arbitrary similarity measures that typically converges to above 90% recall with each point comparing only to several percent of the whole dataset on average.

Finding Near Neighbors Through Local Search

TLDR
Three searching algorithms generalizing to local search other than greedy are introduced, and it is experimentally proved that this approach improves significantly the state of the art.
...