With the growth of the digital advertising market, it has become more important than ever to target the desired audiences. Among various demographic traits, gender information plays a key role in precisely targeting the potential consumers in online advertising and ecommerce. However, such personal information is generally unavailable to digital media sellers. In this paper, we propose a novel task-specific multi-task learning algorithm to predict users' gender information from their video viewing behaviors. To detect as many desired users as possible, while controlling the Type I error rate at a user-specified level, we further propose Bayes testing and decision procedures to efficiently identify male and female users, respectively. Comprehensive experiments have justified the effectiveness and reliability of our framework.