In many real-world tasks of image classification, limited amounts of labeled data are available to train automatic classifiers. Consequently, extensive human expert involvement is required for instance labeling. Detecting Egeria densa in digital imagery is one such real-world classification task. It presents an additional challenge due to subtle spectral changes in Egeria, which makes it difficult to find a single accurate classifier. A novel solution is proposed to employ an ensemble of classifiers for each class (class-specific ensembles), combined with an active learning scheme. The class-specific ensembles are implicitly diverse. Diversity is required to increase the overall accuracy when combining predictions. The combined predictions of the ensembles can be used to reduce the uncertainty in detecting Egeria. Iterative active learning is then suggested to adapt the ensembles to the new images, unseen to the active learner. A novel solution to build compact ensembles is also presented, which are needed to expedite the re-training of the active learner. The combined results are accurate and compact ensembles, which require significantly less expert involvement for image region classification.