AutoFAS: Automatic Feature and Architecture Selection for Pre-Ranking System

  title={AutoFAS: Automatic Feature and Architecture Selection for Pre-Ranking System},
  author={Xiang Li and Xiaojiang Zhou and Yao Xiao and Peihao Huang and Dayao Chen and Sheng Chen and Yunsen Xian},
  journal={Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  • Xiang Li, Xiaojiang Zhou, Yunsen Xian
  • Published 19 May 2022
  • Computer Science
  • Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Industrial search and recommendation systems mostly follow the classic multi-stage information retrieval paradigm: matching, pre-ranking, ranking, and re-ranking stages. To account for system efficiency, simple vector-product based models are commonly deployed in the pre-ranking stage. Recent works consider distilling the high knowledge of large ranking models to small pre-ranking models for better effectiveness. However, two major challenges in pre-ranking system still exist: (i) without… 

Figures and Tables from this paper



COLD: Towards the Next Generation of Pre-Ranking System

This paper designs a new pre- ranking system by joint optimization of both the pre-ranking model and the computing power it costs, and names it COLD (Computing power cost-aware Online and Lightweight Deep Pre-ranking system).

Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach

A novel pre-ranking approach is proposed which supports complicated models with interaction-focused architecture and achieves a better tradeoff between effectiveness and efficiency by utilizing the proposed learnable Feature Selection method based on feature Complexity and variational Dropout.

Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System

A novel way to train ranking models, such as recommender systems, that are both effective and efficient is proposed, and a smaller student model is trained to learn to rank documents/items from both the training data and the supervision of a larger teacher model.

AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

This work proposes a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS), which can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence.

AutoIAS: Automatic Integrated Architecture Searcher for Click-Trough Rate Prediction

This paper presents Automatic Integrated Architecture Searcher (AutoIAS), a framework that provides a practical and general method to find optimal CTR prediction model architecture in an automatic manner and designs an architecture generator network that explicitly models the architecture dependencies among components and generates conditioned architectures distribution for each component.

CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models

This paper presents CTR-BERT, a novel lightweight cache-friendly factorized model for CTR prediction that consists of twin-structured BERT-like encoders for text with a mechanism for late fusion for text and tabular features and significantly outperforms a traditional CTR baseline.

Learning Tree-based Deep Model for Recommender Systems

A novel tree-based method which can provide logarithmic complexity w.r.t. corpus size even with more expressive models such as deep neural networks is proposed and can be jointly learnt towards better compatibility with users' interest distribution and hence facilitate both training and prediction.

DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems

This work proposes an improved framework DCN-V2, which is simple, can be easily adopted as building blocks, and has delivered significant offline accuracy and online business metrics gains across many web-scale learning to rank systems at Google.

Privileged Features Distillation at Taobao Recommendations

By distilling the interacted features that are prohibited during serving for CTR and the post-event features for CVR, this work achieves significant improvements over their strong baselines.

AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System

The experiment over various scenarios reveals that AMEIR outperforms competitive baselines of elaborate manual design and leading algorithmic complex NAS methods with lower model complexity and comparable time cost, indicating efficacy, efficiency, and robustness of the proposed method.