Face images captured by surveillance cameras usually have poor quality, particularly low resolution (LR), which affects the performance of face recognition seriously. In this paper, we develop a novel approach to address the problem of matching a LR face image against a gallery of relatively high resolution (HR) face images. Existing methods deal with such cross-resolution face recognition problem either by importing the information of HR images to help synthesize HR images from LR images or by applying the discrimination of HR images to help search for a unified feature space. Instead, we treat the discrimination information of HR and LR face images equally to boost the performance. The proposed approach learns resolution invariant features aiming to: (1) classify the identity of both LR and HR face images accurately, and (2) preserve the discriminative information among subjects across different resolutions. We conduct experiments on databases of uncontrolled scenarios, i.e., SCface and COX, and results show that the proposed approach significantly outperforms state-of-the-art methods.