Hidden factor such as gender characteristic plays an important role on the performance of Bangla (widely used as Bengali) automatic speech recognition (ASR). If there is a suppression process that represses the decrease of differences in acoustic-likelihood among categories resulted from gender factors, a robust ASR system can be realized. In our previous paper, we proposed a technique of gender effects suppression that composed of two hidden Markov model (HMM)-based classifiers that focused on a gender factor. In the proposed study, we have designed a new ASR for Bangla by suppressing the gender effects, which embeds three HMM-based classifiers for corresponding male, female and geneder-independent (GI) characteristics. In an experiment on Bangla speech database prepared by us, the proposed system that incorporates GI-classifier has achieved a significant improvement of word correct rate, word accuracy and sentence correct rate in comparison with our previous method that did not incorporate GI-classifier.