Fast hash table lookup using extended bloom filter: an aid to network processing

@inproceedings{Song2005FastHT,
  title={Fast hash table lookup using extended bloom filter: an aid to network processing},
  author={Haoyu Song and Sarang Dharmapurikar and Jonathan S. Turner and John W. Lockwood},
  booktitle={SIGCOMM '05},
  year={2005}
}
Hash tables are fundamental components of several network processing algorithms and applications, including route lookup, packet classification, per-flow state management and network monitoring. These applications, which typically occur in the data-path of high-speed routers, must process and forward packets with little or no buffer, making it important to maintain wire-speed throughout. A poorly designed hash table can critically affect the worst-case throughput of an application, since the… 

Figures and Tables from this paper

Fast Hash Table Lookup Using Extended Bloom Filter
TLDR
An overview article summarizes a novel hash table data structure and lookup algorithm which improves the performance over a naive hash table by reducing the number of memory accesses needed for the most time-consuming lookups.
Deterministic and Efficient Hash Table Lookup Using Discriminated Vectors
TLDR
In DEHT, a novel data structure on on-chip memory is built, with the help of which the off- chip memory access can be decreased to a single one at most per lookup even when the load of the hash table is very high.
Optimized Hash Lookup for Bloom Filter Based Packet Routing
  • Dagang Li, Pei Chen
  • Computer Science
    2013 16th International Conference on Network-Based Information Systems
  • 2013
TLDR
An optimized hash lookup scheme is proposed to utilize the preceding Bloom filter for better forwarding performance and usability, with the help of a 'summary vector', which achieves improved space efficiency, less calculation overhead, more deterministic lookup performance, and simpler update procedures.
Summary-aided bloom filter for high-speed named data forwarding
  • Dagang Li, Pei Chen
  • Computer Science
    2013 IEEE 14th International Conference on High Performance Switching and Routing (HPSR)
  • 2013
TLDR
This paper proposes to add a `summary vector' to the on-chip Bloom filter that can help in constructing an efficient off-chip hash table for better storage and lookup performance: a dynamic collision-free hash table that only needs to read into only one routing record for any lookup queries.
An overview of Fast Hash Table Lookup Using Extended Bloom Filter
TLDR
An overview of an article that proposed a novel idea for improving the worst case performance of the Bloom Filter is given and a way to extend and optimize the Bloom filter is described.
Cuckoo++ hash tables: high-performance hash tables for networking applications
TLDR
This paper proposes algorithmic improvements to cuckoo hash tables to eliminate unnecessary memory accesses, without altering the properties of the original cuckoos, so that all existing theoretical analysis remain applicable.
A hash-based scalable IP lookup using Bloom and fingerprint filters
TLDR
A novel hash architecture to address issues in the IP lookup architecture by using pipelined Bloom and fingerprint filters for a binary searching in keys and offers 4.5 and 50.1 times memory and power efficiencies than other contemporary hash and TCAM schemes, respectively.
Fast and deterministic hash table lookup using discriminative bloom filters
Fast routing table lookup based on deterministic multi-hashing
TLDR
The first deterministic multi-hashing scheme with small indexing overhead is designed, which evenly distributes address prefixes to hash buckets for routing-information storage and can maintain a constant lookup rate of over 250 million packets per second with today's commodity SRAM, which is much faster than the existing hashing schemes.
CHAP: Enabling Efficient Hardware-Based Multiple Hash Schemes for IP Lookup
TLDR
This paper presents a new simple hash probing scheme called CHAP (Content-based HAsh Probing) that tackles the hash overflow problem and shows through experimenting with real IP tables how CHAP can effectively deal with the overflow.
...
...

References

SHOWING 1-10 OF 32 REFERENCES
Scalable high speed IP routing lookups
TLDR
This paper describes a new algorithm for best matching prefix using binary search on hash tables organized by prefix lengths that scales very well as address and routing table sizes increase and introduces Mutating Binary Search and other optimizations that considerably reduce the average number of hashes to less than 2.
Fast address lookups using controlled prefix expansion
TLDR
The main technique, controlled prefix expansion, transforms a set of prefixes into an equivalent set with fewer prefix lengths, and optimization techniques based on dynamic programming, and local transformations of data structures to improve cache behavior are used.
Packet classification using tuple space search
TLDR
The Pruned Tuple Space search is the only scheme known to us that allows fast updates and fast search times, and an optimal algorithm is described, called Rectangle Search, for two-dimensional filters.
Longest prefix matching using bloom filters
TLDR
This work introduces the first algorithm that is aware of to employ Bloom filters for longest prefix matching (LPM), and shows that use of this algorithm for Internet Protocol (IP) routing lookups results in a search engine providing better performance and scalability than TCAM-based approaches.
Using multiple hash functions to improve IP lookups
  • A. Broder, M. Mitzenmacher
  • Computer Science
    Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213)
  • 2001
TLDR
This work describes an approach for obtaining good hash tables based on using multiple hashes of each input key (which is an IP address), which proves extremely suitable in instances where the goal is to have one hash bucket fit into a cache line.
Searching very large routing tables in wide embedded memory
TLDR
A novel scheme named BARTS (balanced routing table search) is proposed for searching large routing tables in wide embedded memory at OC-192 and OC-768 speeds, while also supporting fast incremental updates.
Deep packet inspection using parallel bloom filters
TLDR
This work describes a hardware-based technique using Bloom filters, which can detect strings in streaming data without degrading network throughput and queries a database of strings to check for the membership of a particular string.
Deep packet inspection using parallel Bloom filters
TLDR
This work describes a technique based on Bloom filters for detecting predefined signatures (a string of bytes) in the packet payload and uses hardware Bloom filters to isolate all packets that potentially containpredefined signatures.
Summary cache: a scalable wide-area web cache sharing protocol
TLDR
This paper demonstrates the benefits of cache sharing, measures the overhead of the existing protocols, and proposes a new protocol called "summary cache", which reduces the number of intercache protocol messages, reduces the bandwidth consumption, and eliminates 30% to 95% of the protocol CPU overhead, all while maintaining almost the same cache hit ratios as ICP.
Multilevel adaptive hashing
TLDR
According to the analysis, with the Multilevel Hashing Algorithm it is possible to build a dictionary that is able to answer 20 parallel searches among 64,000 entries in less than 5 ps using relatively inexpensive hardware.
...
...