A method for finding similarity between multi-layer perceptrons by Forward Bipartite Alignment

Abstract

We present Forward Bipartite Alignment (FBA), a method that aligns the topological structures of two neural networks. Neural networks are considered to be a black box, because neural networks have a complex model surface determined by their weights that combine attributes non-linearly. Two networks that make similar predictions on training data may still generalize differently. FBA enables a diversity of applications, including visualization and canonicalization of neural networks, ensembles, and cross-over between unrelated neural networks in evolutionary optimization. We describe the FBA algorithm, and describe implementations for three applications: genetic algorithms, visualization, and ensembles. We demonstrate FBA's usefulness by comparing a bag of neural networks to a bag of FBA-aligned neural networks. We also show that aligning, and then combining two neural networks has no appreciable loss in accuracy which means that Forward Bipartite Alignment aligns neural networks in a meaningful way.

DOI: 10.1109/IJCNN.2015.7280769

5 Figures and Tables

Cite this paper

@article{Ashmore2015AMF, title={A method for finding similarity between multi-layer perceptrons by Forward Bipartite Alignment}, author={Stephen C. Ashmore and Michael S. Gashler}, journal={2015 International Joint Conference on Neural Networks (IJCNN)}, year={2015}, pages={1-7} }