In this paper, we develop data driven registration algorithms, relying on pixel similarity metrics, that enable an accurate (subpixel) rigid registration of dissimilar single or multimodal 2D/3D images. Gross dissimilarities are handled by considering similarity measures related to robust M-estimators. In particular, a novel (robust) similarity metric is proposed for comparing multimodal images. The proposed robust similarity metrics are compared to the most popular standard similarity metrics, on synthetic as well as on real-world image pairs showing gross dissimilarities. Three case studies are considered: the registration of single modal and multimodal 3D medical images, the matching of multispectral remotely sensed images, and the registration of intensity and range images. The proposed robust similarity measures compare favourably with the standard (non-robust) techniques. ( 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.