Some novel vision-based dazzling avoidance systems, such as the ShadeVision system, can intelligently cast a shadow on driver's eye region to protect the eyes from the dazzling effect that are caused by the strong external light sources. However, during this protective state (eyes covered by shadow), the system still needs to detect the driver's face to realize further monitoring, e.g., the intention of the driver as well as the dazzling effect from another light source. The protective shadow will make it difficult to apply the state-of-the-art algorithms to detect the driver's face, as the eye information is of great importance to vision-based face detection. This paper presents a series of robust algorithms for the face detection with protective shadow cast on driver's eyes, among which the Partially Masked Training, the Consecutive Sub-block Training and the Overlapped Sub-block Training are proposed for the first time for this kind of task. The on-road experimental results verify the effectiveness of the designed algorithms.