# Rule extraction from linear support vector machines

@inproceedings{Fung2005RuleEF, title={Rule extraction from linear support vector machines}, author={Glenn Fung and Sathyakama Sandilya and R. Bharat Rao}, booktitle={KDD '05}, year={2005} }

We describe an algorithm for converting linear support vector machines and any other arbitrary hyperplane-based linear classifiers into a set of non-overlapping rules that, unlike the original classifier, can be easily interpreted by humans. Each iteration of the rule extraction algorithm is formulated as a constrained optimization problem that is computationally inexpensive to solve. We discuss various properties of the algorithm and provide proof of convergence for two different optimization…

## 155 Citations

Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines

- 2006

In recent years, Support Vector Machines (SVMs) were successfully applied to a wide range of applications. Their good performance is achieved by an implicit non-linear transformation of the original…

Rule Extraction from Support Vector Machines: A Sequential Covering Approach

- Computer ScienceIEEE Transactions on Knowledge and Data Engineering
- 2007

Results are presented that show the rules produced by SQRex-SVM exhibit both improved generalization performance and smaller more comprehensible rule sets compared to both other SVM rule extraction techniques and direct rule learning techniques.

Extraction of Interpretable Rules from Piecewise-Linear Approximation of a Nonlinear Classifier Using Clustering-Based Decomposition

- Computer Science
- 2011

This paper presents a novel approach for converting a widely acknowledged rule extraction algorithm for Linear Support Vector Machine into a number of constrained linear programming (LP) problems and claims that proposed approach helps to extract better rules from linearly non-separable cases and could be effectively employed even for non-homogeneous target classes with high inner variance.

Convex hull-based support vector machine rule extraction

- Mathematics, Computer Science2012 9th International Conference on Fuzzy Systems and Knowledge Discovery
- 2012

Results showed that the proposed rule extraction algorithm can improve the accuracy of rule covering and fidelity, and can open the black-box of support vector machine.

Polytope Classifier: A Symbolic Knowledge Extraction from Piecewise-Linear Support Vector Machine

- Computer ScienceKES
- 2011

It is claimed that the proposed polytope classifier achieves classification rates comparable to a nonlinear SVM and corresponding rule extraction approach helps to extract better rules from linearly non-separable cases in comparison with decision trees and C4.5 rule extraction algorithm.

Rule extraction from support vector machines based on consistent region covering reduction

- Computer ScienceKnowl. Based Syst.
- 2013

A Multiple Kernel Support Vector Machine Scheme for Simultaneous Feature Selection and Rule-Based Classification

- Mathematics, Computer SciencePAKDD
- 2007

This paper presents a novel feature selection and rule extraction method which is based on multiple kernel support vector machine (MK-SVM), which makes the feature selection problem in the context of SVM transformed into an ordinary multiple parameters learning problem.

Rule extraction from support vector machines by genetic algorithms

- Computer ScienceNeural Computing and Applications
- 2012

A rule extraction algorithm to extract the comprehensible rule from SVMs and enhance their explanation capability is proposed and results indicate that the proposed method performs at least as well as with the most successful direct rule learners.

Fuzzy rules extraction from support vector machines for multi-class classification

- Computer Science, MathematicsNeural Computing and Applications
- 2012

A new method for fuzzy rule extraction from trained support vector machines (SVMs) for multi-class problems, named FREx_SVM, which is suited for classification in multi- class problems and includes a wrapper feature selection algorithm.

Comprehensible credit scoring models using rule extraction from support vector machines

- Computer ScienceEur. J. Oper. Res.
- 2007

## References

SHOWING 1-10 OF 29 REFERENCES

Proximal support vector machine classifiers

- Mathematics, Computer ScienceKDD '01
- 2001

Computational results on publicly available datasets indicate that the proposed proximal SVM classifier has comparable test set correctness to that of standard S VM classifiers, but with considerably faster computational time that can be an order of magnitude faster.

Knowledge-Based Support Vector Machine Classifiers

- Computer ScienceNIPS
- 2002

Numerical results show improvement in test set accuracy after the incorporation of prior knowledge into ordinary, data-based linear support vector machine classifiers, and one experiment shows that a linear classifier, based solely on prior knowledge, far outperforms the direct application of priorknowledge rules to classify data.

Breast Cancer Diagnosis and Prognosis Via Linear Programming

- Medicine, Computer ScienceOper. Res.
- 1995

Two medical applications of linear programming are described in this paper. Specifically, linear programming-based machine learning techniques are used to increase the accuracy and objectivity of…

Feature Selection via Concave Minimization and Support Vector Machines

- Computer ScienceICML
- 1998

Numerical tests on 6 public data sets show that classi ers trained by the concave minimization approach and those trained by a support vector machine have comparable 10fold cross-validation correctness.

Generalized Support Vector Machines

- Computer Science
- 1998

By setting apart the two functions of a support vector machine: separation of points by a nonlinear surface in the original space of patterns, and maximizing the distance between separating planes in…

Survey and critique of techniques for extracting rules from trained artificial neural networks

- Computer ScienceKnowl. Based Syst.
- 1995

Least Squares Support Vector Machine Classifiers

- Computer Science, MathematicsNeural Processing Letters
- 2004

A least squares version for support vector machine (SVM) classifiers that follows from solving a set of linear equations, instead of quadratic programming for classical SVM's.

Extracting Refined Rules from Knowledge-Based Neural Networks

- Computer ScienceMachine Learning
- 2004

This article proposes and empirically evaluates a method for the final, and possibly most difficult, step of the refinement of existing knowledge and demonstrates that neural networks can be used to effectively refine symbolic knowledge.

Neural Networks and Structured Knowledge: Rule Extraction and Applications

- Computer ScienceApplied Intelligence
- 2004

The contributions collected here concentrate on the extraction of knowledge, particularly in the form of rules, from neural networks, and on applications relying on the representation and processing of structured knowledge by neural networks.

Projected Newton methods for optimization problems with simple constraints

- Mathematics1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes
- 1981

We consider the problem min {f(x)|x ¿ 0} and algorithms of the form xk+1 = [xk - ¿k Dk¿f(xk)]+ where [¿]+ denotes projection on the positive orthant, ¿k is a stepsize chosen by an Armijolike rule,…