Rule extraction from linear support vector machines

  title={Rule extraction from linear support vector machines},
  author={Glenn Fung and Sathyakama Sandilya and R. Bharat Rao},
  booktitle={KDD '05},
We describe an algorithm for converting linear support vector machines and any other arbitrary hyperplane-based linear classifiers into a set of non-overlapping rules that, unlike the original classifier, can be easily interpreted by humans. Each iteration of the rule extraction algorithm is formulated as a constrained optimization problem that is computationally inexpensive to solve. We discuss various properties of the algorithm and provide proof of convergence for two different optimization… 
Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines
In recent years, Support Vector Machines (SVMs) were successfully applied to a wide range of applications. Their good performance is achieved by an implicit non-linear transformation of the original
Rule Extraction from Support Vector Machines: A Sequential Covering Approach
Results are presented that show the rules produced by SQRex-SVM exhibit both improved generalization performance and smaller more comprehensible rule sets compared to both other SVM rule extraction techniques and direct rule learning techniques.
Extraction of Interpretable Rules from Piecewise-Linear Approximation of a Nonlinear Classifier Using Clustering-Based Decomposition
This paper presents a novel approach for converting a widely acknowledged rule extraction algorithm for Linear Support Vector Machine into a number of constrained linear programming (LP) problems and claims that proposed approach helps to extract better rules from linearly non-separable cases and could be effectively employed even for non-homogeneous target classes with high inner variance.
Convex hull-based support vector machine rule extraction
Results showed that the proposed rule extraction algorithm can improve the accuracy of rule covering and fidelity, and can open the black-box of support vector machine.
Polytope Classifier: A Symbolic Knowledge Extraction from Piecewise-Linear Support Vector Machine
It is claimed that the proposed polytope classifier achieves classification rates comparable to a nonlinear SVM and corresponding rule extraction approach helps to extract better rules from linearly non-separable cases in comparison with decision trees and C4.5 rule extraction algorithm.
A Multiple Kernel Support Vector Machine Scheme for Simultaneous Feature Selection and Rule-Based Classification
This paper presents a novel feature selection and rule extraction method which is based on multiple kernel support vector machine (MK-SVM), which makes the feature selection problem in the context of SVM transformed into an ordinary multiple parameters learning problem.
Rule extraction from support vector machines by genetic algorithms
A rule extraction algorithm to extract the comprehensible rule from SVMs and enhance their explanation capability is proposed and results indicate that the proposed method performs at least as well as with the most successful direct rule learners.
Fuzzy rules extraction from support vector machines for multi-class classification
A new method for fuzzy rule extraction from trained support vector machines (SVMs) for multi-class problems, named FREx_SVM, which is suited for classification in multi- class problems and includes a wrapper feature selection algorithm.


Proximal support vector machine classifiers
Computational results on publicly available datasets indicate that the proposed proximal SVM classifier has comparable test set correctness to that of standard S VM classifiers, but with considerably faster computational time that can be an order of magnitude faster.
Knowledge-Based Support Vector Machine Classifiers
Numerical results show improvement in test set accuracy after the incorporation of prior knowledge into ordinary, data-based linear support vector machine classifiers, and one experiment shows that a linear classifier, based solely on prior knowledge, far outperforms the direct application of priorknowledge rules to classify data.
Breast Cancer Diagnosis and Prognosis Via Linear Programming
Two medical applications of linear programming are described in this paper. Specifically, linear programming-based machine learning techniques are used to increase the accuracy and objectivity of
Feature Selection via Concave Minimization and Support Vector Machines
Numerical tests on 6 public data sets show that classi ers trained by the concave minimization approach and those trained by a support vector machine have comparable 10fold cross-validation correctness.
Generalized Support Vector Machines
By setting apart the two functions of a support vector machine: separation of points by a nonlinear surface in the original space of patterns, and maximizing the distance between separating planes in
Least Squares Support Vector Machine Classifiers
A least squares version for support vector machine (SVM) classifiers that follows from solving a set of linear equations, instead of quadratic programming for classical SVM's.
Extracting Refined Rules from Knowledge-Based Neural Networks
This article proposes and empirically evaluates a method for the final, and possibly most difficult, step of the refinement of existing knowledge and demonstrates that neural networks can be used to effectively refine symbolic knowledge.
Neural Networks and Structured Knowledge: Rule Extraction and Applications
  • F. Kurfess
  • Computer Science
    Applied Intelligence
  • 2004
The contributions collected here concentrate on the extraction of knowledge, particularly in the form of rules, from neural networks, and on applications relying on the representation and processing of structured knowledge by neural networks.
Projected Newton methods for optimization problems with simple constraints
  • D. Bertsekas
  • Mathematics
    1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes
  • 1981
We consider the problem min {f(x)|x ¿ 0} and algorithms of the form xk+1 = [xk - ¿k Dk¿f(xk)]+ where [¿]+ denotes projection on the positive orthant, ¿k is a stepsize chosen by an Armijolike rule,