High-throughput and memory-efficient multimatch packet classification based on distributed and pipelined hash tables

Yang Xu, Zhaobo Liu, Zhuoyuan Zhang, H. Jonathan Chao

Research output: Contribution to journalArticle

Abstract

The emergence of new network applications, such as the network intrusion detection system and packet-level accounting, requires packet classification to report all matched rules instead of only the best matched rule. Although several schemes have been proposed recently to address the multimatch packet classification problem, most of them require either huge memory or expensive ternary content addressable memory (TCAM) to store the intermediate data structure, or they suffer from steep performance degradation under certain types of classifiers. In this paper, we decompose the operation of multimatch packet classification from the complicated multidimensional search to several single-dimensional searches, and present an asynchronous pipeline architecture based on a signature tree structure to combine the intermediate results returned from single-dimensional searches. By spreading edges of the signature tree across multiple hash tables at different stages, the pipeline can achieve a high throughput via the interstage parallel access to hash tables. To exploit further intrastage parallelism, two edge-grouping algorithms are designed to evenly divide the edges associated with each stage into multiple work-conserving hash tables. To avoid collisions involved in hash table lookup, a hybrid perfect hash table construction scheme is proposed. Extensive simulation using realistic classifiers and traffic traces shows that the proposed pipeline architecture outperforms HyperCuts and B2PC schemes in classification speed by at least one order of magnitude, while having a similar storage requirement. Particularly, with different types of classifiers of 4K rules, the proposed pipeline architecture is able to achieve a throughput between 26.8 and 93.1 Gb/s using perfect hash tables.

Original languageEnglish (US)
Article number6565409
Pages (from-to)982-995
Number of pages14
JournalIEEE/ACM Transactions on Networking
Volume22
Issue number3
DOIs
StatePublished - 2014

Fingerprint

Pipelines
Throughput
Data storage equipment
Classifiers
Associative storage
Table lookup
Intrusion detection
Data structures
Degradation

Keywords

  • Hash table
  • packet classification
  • signature tree
  • ternary content addressable memory (TCAM)

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Software
  • Computer Science Applications
  • Computer Networks and Communications

Cite this

High-throughput and memory-efficient multimatch packet classification based on distributed and pipelined hash tables. / Xu, Yang; Liu, Zhaobo; Zhang, Zhuoyuan; Chao, H. Jonathan.

In: IEEE/ACM Transactions on Networking, Vol. 22, No. 3, 6565409, 2014, p. 982-995.

Research output: Contribution to journalArticle

@article{83f6ef7e3c2d4cd9af939114dffd6d68,
title = "High-throughput and memory-efficient multimatch packet classification based on distributed and pipelined hash tables",
abstract = "The emergence of new network applications, such as the network intrusion detection system and packet-level accounting, requires packet classification to report all matched rules instead of only the best matched rule. Although several schemes have been proposed recently to address the multimatch packet classification problem, most of them require either huge memory or expensive ternary content addressable memory (TCAM) to store the intermediate data structure, or they suffer from steep performance degradation under certain types of classifiers. In this paper, we decompose the operation of multimatch packet classification from the complicated multidimensional search to several single-dimensional searches, and present an asynchronous pipeline architecture based on a signature tree structure to combine the intermediate results returned from single-dimensional searches. By spreading edges of the signature tree across multiple hash tables at different stages, the pipeline can achieve a high throughput via the interstage parallel access to hash tables. To exploit further intrastage parallelism, two edge-grouping algorithms are designed to evenly divide the edges associated with each stage into multiple work-conserving hash tables. To avoid collisions involved in hash table lookup, a hybrid perfect hash table construction scheme is proposed. Extensive simulation using realistic classifiers and traffic traces shows that the proposed pipeline architecture outperforms HyperCuts and B2PC schemes in classification speed by at least one order of magnitude, while having a similar storage requirement. Particularly, with different types of classifiers of 4K rules, the proposed pipeline architecture is able to achieve a throughput between 26.8 and 93.1 Gb/s using perfect hash tables.",
keywords = "Hash table, packet classification, signature tree, ternary content addressable memory (TCAM)",
author = "Yang Xu and Zhaobo Liu and Zhuoyuan Zhang and Chao, {H. Jonathan}",
year = "2014",
doi = "10.1109/TNET.2013.2270441",
language = "English (US)",
volume = "22",
pages = "982--995",
journal = "IEEE/ACM Transactions on Networking",
issn = "1063-6692",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

TY - JOUR

T1 - High-throughput and memory-efficient multimatch packet classification based on distributed and pipelined hash tables

AU - Xu, Yang

AU - Liu, Zhaobo

AU - Zhang, Zhuoyuan

AU - Chao, H. Jonathan

PY - 2014

Y1 - 2014

N2 - The emergence of new network applications, such as the network intrusion detection system and packet-level accounting, requires packet classification to report all matched rules instead of only the best matched rule. Although several schemes have been proposed recently to address the multimatch packet classification problem, most of them require either huge memory or expensive ternary content addressable memory (TCAM) to store the intermediate data structure, or they suffer from steep performance degradation under certain types of classifiers. In this paper, we decompose the operation of multimatch packet classification from the complicated multidimensional search to several single-dimensional searches, and present an asynchronous pipeline architecture based on a signature tree structure to combine the intermediate results returned from single-dimensional searches. By spreading edges of the signature tree across multiple hash tables at different stages, the pipeline can achieve a high throughput via the interstage parallel access to hash tables. To exploit further intrastage parallelism, two edge-grouping algorithms are designed to evenly divide the edges associated with each stage into multiple work-conserving hash tables. To avoid collisions involved in hash table lookup, a hybrid perfect hash table construction scheme is proposed. Extensive simulation using realistic classifiers and traffic traces shows that the proposed pipeline architecture outperforms HyperCuts and B2PC schemes in classification speed by at least one order of magnitude, while having a similar storage requirement. Particularly, with different types of classifiers of 4K rules, the proposed pipeline architecture is able to achieve a throughput between 26.8 and 93.1 Gb/s using perfect hash tables.

AB - The emergence of new network applications, such as the network intrusion detection system and packet-level accounting, requires packet classification to report all matched rules instead of only the best matched rule. Although several schemes have been proposed recently to address the multimatch packet classification problem, most of them require either huge memory or expensive ternary content addressable memory (TCAM) to store the intermediate data structure, or they suffer from steep performance degradation under certain types of classifiers. In this paper, we decompose the operation of multimatch packet classification from the complicated multidimensional search to several single-dimensional searches, and present an asynchronous pipeline architecture based on a signature tree structure to combine the intermediate results returned from single-dimensional searches. By spreading edges of the signature tree across multiple hash tables at different stages, the pipeline can achieve a high throughput via the interstage parallel access to hash tables. To exploit further intrastage parallelism, two edge-grouping algorithms are designed to evenly divide the edges associated with each stage into multiple work-conserving hash tables. To avoid collisions involved in hash table lookup, a hybrid perfect hash table construction scheme is proposed. Extensive simulation using realistic classifiers and traffic traces shows that the proposed pipeline architecture outperforms HyperCuts and B2PC schemes in classification speed by at least one order of magnitude, while having a similar storage requirement. Particularly, with different types of classifiers of 4K rules, the proposed pipeline architecture is able to achieve a throughput between 26.8 and 93.1 Gb/s using perfect hash tables.

KW - Hash table

KW - packet classification

KW - signature tree

KW - ternary content addressable memory (TCAM)

UR - http://www.scopus.com/inward/record.url?scp=84903307243&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84903307243&partnerID=8YFLogxK

U2 - 10.1109/TNET.2013.2270441

DO - 10.1109/TNET.2013.2270441

M3 - Article

AN - SCOPUS:84903307243

VL - 22

SP - 982

EP - 995

JO - IEEE/ACM Transactions on Networking

JF - IEEE/ACM Transactions on Networking

SN - 1063-6692

IS - 3

M1 - 6565409

ER -