Training genetic programming on half a million patterns: an example from anomaly detection

@article{Song2005TrainingGP,
  title={Training genetic programming on half a million patterns: an example from anomaly detection},
  author={Dong Song and Malcolm I. Heywood and Ayse Nur Zincir-Heywood},
  journal={IEEE Transactions on Evolutionary Computation},
  year={2005},
  volume={9},
  pages={225-239}
}
The hierarchical RSS-DSS algorithm is introduced for dynamically filtering large datasets based on the concepts of training pattern age and difficulty, while utilizing a data structure to facilitate the efficient use of memory hierarchies. Such a scheme provides the basis for training genetic programming (GP) on a data set of half a million patterns in 15 min. The method is generic, thus, not specific to a particular GP structure, computing platform, or application context. The method is… 

Tables from this paper

Evolving High-Speed, Easy-to-Understand Network Intrusion Detection Rules with Genetic Programming
TLDR
This paper shows that GP can offer at least two advantages over other classical mechanisms: it can produce very lightweight detection rules (something of extreme importance for high-speed networks or resource-constrained applications) and the simplicity of the patterns generated allows to easily understand the semantics of the underlying attack.
Genetic Programming Approach for Multi-Category Pattern Classification Applied to Network Intrusions Detection
TLDR
The results demonstrate that the proposed genetic approach outperforms the existing GP-classification methods, and provides improved results compared to other existing techniques.
An ensemble-based evolutionary framework for coping with distributed intrusion detection
TLDR
Experiments on the KDD Cup 1999 Data show the capability of genetic programming in successfully dealing with the problem of intrusion detection on distributed data.
Evaluating the performance of a differential evolution algorithm in anomaly detection
TLDR
The results show the effectiveness of using differential evolution in detecting anomalies by achieving an average true positive rate of 100%, while the average false positive rate is only 0.582%.
Application of Partial-Connected Dynamic and GA-Optimized Neural Networks to Misuse Detection Using Categorized and Ranked Input Features
TLDR
Empirical results show that PCDNN with selected input features and categorized input connections offers better detection rate (DR) among the investigated models and the mentioned NN also performs better in terms of cost per example (CPE) when compared to other proposed models in this study.
SECURING NETWORK TRAFFIC USING GENETICALLY EVOLVED TRANSFORMATIONS
TLDR
The paper describes a new approach of classification using genetic programming that outperforms the existing GP-classification methods, and gives accepted results compared to other existing techniques.
Adaptive Framework for Network Intrusion Detection by Using Genetic-Based Machine Learning Algorithm
TLDR
Different techniques into classifier system to detect and classify intrusion from normal network packet are incorporated and steady state genetic-based machine leaning algorithm (SSGBML) and Zeroth Level Classifier system (ZCS) are investigated.
PERFORMANCE COMPARISON OF INTRUSION DETECTION SYSTEM USING VARIOUS TECHNIQUES – A REVIEW
TLDR
The aim of this paper is to improve the detection rate and performance of the proposed Intrusion Detection System.
Performance Analysis on Competitive, Roulette Wheel and Pseudo-Random Rules for Intrusion Detection
TLDR
The suggested roulette wheel selection rule and pseudo-random rule intrusion detection criteria integrated into BPN are superior to other schemes with only 11 features used further reducing complexity and computation time.
An Intelligent Intrusion Detection System Using Genetic Algorithms and Features Selection
TLDR
A new intelligent intrusion detection system has been proposed using genetic algorithms that can be used in real-time mode and showed a detection rate up till to 92.94%.
...
...

References

SHOWING 1-10 OF 40 REFERENCES
A Linear Genetic Programming Approach to Intrusion Detection
TLDR
Page-based Linear Genetic Programming is proposed and implemented with two-layer Subset Selection with careful adjustment of the relationship between subset layers to address a two-class intrusion detection classification problem as defined by the KDD-99 benchmark dataset.
A data mining framework for building intrusion detection models
  • Wenke Lee, S. Stolfo, K. Mok
  • Computer Science
    Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344)
  • 1999
TLDR
A data mining framework for adaptively building Intrusion Detection (ID) models is described, to utilize auditing programs to extract an extensive set of features that describe each network connection or host session, and apply data mining programs to learn rules that accurately capture the behavior of intrusions and normal activities.
A framework for constructing features and models for intrusion detection systems
TLDR
A novel framework, MADAM ID, for Mining Audit Data for Automated Models for Instrusion Detection, which uses data mining algorithms to compute activity patterns from system audit data and extracts predictive features from the patterns.
Adapting the Fitness Function in GP for Data Mining
TLDR
The results indicate that extending the gp with the saw-ing feature increases its performance when different types of misclassifications are not weighted differently, but leads to worse results when they are.
Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation
TLDR
An intrusion detection evaluation test bed was developed which generated normal traffic similar to that on a government site containing 100's of users on 1000's of hosts and the best systems failed to detect roughly half these new attacks which included damaging access to root-level privileges by remote users.
A Database of Computer Attacks for the Evaluation of Intrusion Detection Systems
TLDR
The focus of this thesis is the attacks that were developed for use in the 1998 DARPA intrusion detection evaluation and included older, well-known attacks, newer attacks that have recently been released to publicly available forums, and some novel attacks developed specifically for this evaluation.
Host-based intrusion detection using self-organizing maps
TLDR
Hierarchical SOMs are applied to the problem of host based intrusion detection on computer networks and specific recommendations are made regarding the representation of time, network parameters and SOM architecture.
Statistical traffic modeling for network intrusion detection
  • J. Cabrera, B. Ravichandran, R. Mehra
  • Computer Science
    Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728)
  • 2000
TLDR
It is verified that denial-of-service and probing attacks leave traces on simple network activity models, with rates of false alarm which are comparable to the false alarm rates obtained by the participants of the 1998 DARPA evaluation, in which much more complex detection schemes were utilized.
Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory
TLDR
The purpose of this article is to attempt to identify the shortcomings of the Lincoln Lab effort in the hope that future efforts of this kind will be placed on a sounder footing.
A decision-theoretic generalization of on-line learning and an application to boosting
TLDR
The model studied can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting, and it is shown that the multiplicative weight-update Littlestone?Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems.
...
...