In this paper, a swarm intelligence technique, better known as Particle swarm optimization, has been used in solving the fractional differential equations. The approximate mathematical modeling has been done by employing feed-forward artificial neural networks by defining the unsupervised error. The learning of weights for such errors has been carried out by using particle swarm optimization hybridized with simulating annealing algorithms for efficient local search. The design scheme has been successfully applied to solve different problems associated with linear and nonlinear ordinary differential equations of fractional order. The results were compared with available exact solutions, analytic solutions and standard numerical techniques including both deterministic and stochastic approaches. In case of linear ordinary fractional differential equations, relatively more precise solutions were obtained than those of the deterministic numerical methods. Moreover, for complex non-linear fractional differential equations, the technique is still applicable, but with reduced accuracy. The advantages of the proposed scheme are easy implementation, simplicity of concept and broad scope of applications.