Deep Learning for Click-Through Rate Estimation

  title={Deep Learning for Click-Through Rate Estimation},
  author={Weinan Zhang and Jiarui Qin and Wei Guo and Ruiming Tang and Xiuqiang He},
Click-through rate (CTR) estimation plays as a core function module in various personalized online services, including online advertising, recommender systems, and web search etc. From 2015, the success of deep learning started to benefit CTR estimation performance and now deep CTR models have been widely applied in many industrial platforms. In this survey, we provide a comprehensive review of deep learning models for CTR estimation tasks. First, we take a review of the transfer from shallow… 

Figures from this paper

Boosting Deep CTR Prediction with a Plug-and-Play Pre-trainer for News Recommendation

This work proposes a plug-and-play pre-trainer, namely PREC, to learn both user and news encoders through multi-task pre-training and focuses on how to use the derived user and item representations to boost the performance of conventional lightweight models for click-through-rate prediction.

DCAF-BERT: A Distilled Cachable Adaptable Factorized Model For Improved Ads CTR Prediction

The proposed model is DCAF-BERT, a novel lightweight cache-friendly factorized model that consists of twin-structured BERT-like encoders for text with a mechanism for late fusion for tabular and numeric features that allows for compartmentalised retraining which enables the model to easily adapt to distribution shifts.

Click-through rate prediction in online advertising: A literature review

Se-xDeepFEFM: Combining Low-Order Feature Refinement and Interaction Intensity Evaluation for Click-Through Rate Prediction

A novel model called Senet and extreme deep field-embedded factorization machine (Se-xDFEFM) is proposed for more effective CTR prediction and achieves competitive recommendation performance, demonstrating their scalability.

IntTower: The Next Generation of Two-Tower Model for Pre-Ranking System

It is shown it is possible to design a two-tower model that emphasizes both information interactions and inference efficiency and the effectiveness of IntTower on a large-scale advertisement pre-ranking system is verified.

Numerical Feature Representation with Hybrid N-ary Encoding

NaryDis first leverages hybrid n-ary encoding as an automatic discretization module to generate hybrid-grained discretized results (multiple encoded sequences), which are further constrained by a self-supervised regularization module.

Dynamic Explicit Embedding Representation for Numerical Features in Deep CTR Prediction

  • Yuan Cheng
  • Computer Science
    Proceedings of the 31st ACM International Conference on Information & Knowledge Management
  • 2022
The proposed Dynamic Explicit Embedding Representation (DEER) framework is able to discretize numerical features automatically and dynamically, which can overcome the discontinuity problem in the representation of numeric information.

LCD: Adaptive Label Correction for Denoising Music Recommendation

  • Quanyu DaiYalei Lv Ruiming Tang
  • Computer Science
    Proceedings of the 31st ACM International Conference on Information & Knowledge Management
  • 2022
This work first identifies serious label noise issues in an industrial music App, and then proposes an adaptive Label Correction method for Denoising (LCD) music recommendation by ensembling the noisy labels and the model outputs to encourage a consensus prediction.

Personalized Inter-Task Contrastive Learning for CTR&CVR Joint Estimation

The proposed PICO framework can effectively model the inter-task relationship and is utilized to jointly estimate the click-through rate (CTR) and post-click conversion rate (CVR) in advertising systems and introduces an aux- iliary network to capture theInter-task relevance at instance-level and transform it into personalized temperature param- eters for contrastive learning.



Neural Input Search for Large Scale Recommendation Models

This work proposes a novel type of embedding, namely Multi-size Embedding (ME), which allows the embedding dimension to vary for different values of the feature, and argues that the traditional Single-size embedding (SE) suffers from inefficient usage of model capacity and training data.

AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

An effective and efficient method called the AutoInt to automatically learn the high-order feature interactions of input features and map both the numerical and categorical features into the same low-dimensional space is proposed.

DARTS: Differentiable Architecture Search

The proposed algorithm excels in discovering high-performance convolutional architectures for image classification and recurrent architectures for language modeling, while being orders of magnitude faster than state-of-the-art non-differentiable techniques.

Deep & Cross Network for Ad Click Predictions

This paper proposes the Deep & Cross Network (DCN), which keeps the benefits of a DNN model, and beyond that, it introduces a novel cross network that is more efficient in learning certain bounded-degree feature interactions.

Journal of Machine Learning Research

  • 11(4),
  • 2010

Retrieval & Interaction Machine for Tabular Data Prediction

This work proposes a general learning framework named Retrieval & Interaction Machine (RIM) that fully exploits both cross-row and cross-column patterns among tabular data and achieves significant improvements over the state-of-the-art and various baselines.

volume 33

  • pages 5941–5948,
  • 2019

In RecSys

  • pages 43–50,
  • 2016


  • pages 2307–2316,
  • 2020

AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction

An end-to-end model, AutoGroup, is proposed, which casts the selection of feature interactions as a structural optimization problem and performs both dimensionality reduction and feature selection which are not seen in previous models.