Training neural networks via simplified hybrid algorithm mixing Nelder-Mead and particle swarm optimization methods

Abstract

In this paper, a new and simplified hybrid algorithm mixing the simplex method of Nelder and Mead (NM) and particle swarm optimization algorithm (PSO), abbreviated as SNM-PSO, is proposed for the training of the parameters of the Artificial Neural Network (ANN). Our method differs from other hybrid PSO methods in that, n+1 particles, where n is the dimension of the search space, are randomly selected (without sorting), at each iteration of the proposed algorithm for use as the initial vertices of the NM algorithm, and each such particle is replaced by the corresponding final vertex after executing the NM algorithm. All the particles are then updated using the standard PSO algorithm. Our proposed method is simpler than other similar hybrid PSO methods and places more emphasis on the exploration of the search space. Some simulation problems will be provided to compare the performances of the proposed method with PSO and other similar hybrid PSO methods in training an ANN. These simulations show that the proposed method outperforms the other compared methods. Communicated by T.-P. Hong. S.-H. Liao (B) · J.-Y. Chang, · C.-T. Lin Department of Electrical and Control Engineering, National Chiao Tung University, 1001 University Road, Hsinchu, 300 Taiwan, ROC e-mail: liao622@gmail.com J.-Y. Chang e-mail: jychang@mail.nctu.edu.tw C.-T. Lin e-mail: ctlin@mail.nctu.edu.tw J.-G. Hsieh Department of Electrical Engineering, I-Shou University, Kaohsiung County, Taiwan e-mail: jghsieh@gmail.com

DOI: 10.1007/s00500-014-1292-y

14 Figures and Tables

Cite this paper

@article{Liao2015TrainingNN, title={Training neural networks via simplified hybrid algorithm mixing Nelder-Mead and particle swarm optimization methods}, author={Shih-Hui Liao and Jer-Guang Hsieh and Jyh-Yeong Chang and Chin-Teng Lin}, journal={Soft Comput.}, year={2015}, volume={19}, pages={679-689} }