Face detection and the numerous applications it leads to are now a part of our everyday lives and can be found in any electronic device. Detecting a face and its facial features in uncontrolled environments, however, come along with two main problems that we address in this paper: handling large variations in pose and facial occlusions. The Active appearance model (AAM) is a very efficient method to model and describe the face, but it is not robust against neither of these problems. In this paper we propose to use different AAMs for different pose configurations and introduce a switching method that makes use of the Fast Robust PCA reconstruction technique to decide on the final model to use. The presented method is a very simple and efficient one that is also robust against occlusions. We give results of experiments performed on a database of artificially occluded images to prove that the method is highly effective in detecting the pose of faces even in presence of occlusions.