Tuning Word2vec for Large Scale Recommendation Systems

  title={Tuning Word2vec for Large Scale Recommendation Systems},
  author={Benjamin Paul Chamberlain and Emanuele Rossi and Dan Shiebler and Suvash Sedhain and Michael M. Bronstein},
  journal={Proceedings of the 14th ACM Conference on Recommender Systems},
Word2vec is a powerful machine learning tool that emerged from Natural Language Processing (NLP) and is now applied in multiple domains, including recommender systems, forecasting, and network analysis. As Word2vec is often used off the shelf, we address the question of whether the default hyperparameters are suitable for recommender systems. The answer is emphatically no. In this paper, we first elucidate the importance of hyperparameter optimization and show that unconstrained optimization… 

Figures and Tables from this paper

Multi-objective Hyper-parameter Optimization of Behavioral Song Embeddings

This work studies the hyperparameter optimization of behavioral song embeddings based on Word2Vec on a selection of downstream tasks, namely next-song recommendation, false neighbor rejection, and artist and genre clustering, and presents new optimization objectives and metrics.

Graph Neural Networks for Link Prediction with Subgraph Sketching

A novel full-graph GNN called ELPH (Efficient Link Prediction with Hashing) that passes subgraph sketches as messages to approximate the key components of SGNNs without explicit subgraph construction is proposed, which is provably more expressive than Message Passing GNNs (MPNNs).

Improving word embeddings in Portuguese: increasing accuracy while reducing the size of the corpus

This work uses a word embedding technique based on the Word2Vec neural network parameterized and trained using a new dataset built from online newspapers to enhance available descriptions by adding semantically related tags in Portuguese.

DSER: Deep-Sequential Embedding for single domain Recommendation

TwHIN: Embedding the Twitter Heterogeneous Information Network for Personalized Recommendation

This work investigates knowledge-graph embeddings for entities in the Twitter HIN (TwHIN) and shows that these pretrained representations yield significant offline and online improvement for a diverse range of downstream recommendation and classification tasks: personalized ads rankings, account follow-recommendation, offensive content detection, and search ranking.


This research aims to study hyperparameters' impact on the downstream task of classification and shows that CNN with a specific combination of word embeddinghyperparameters can achieve the highest accuracy in the classification downstream task.

Predicting Feature-based Similarity in the News Domain Using Human Judgments

This study assessed different similarity functions for news item retrieval, by comparing them to human judgments of similarity, revealing that similarity functions in news are less representative of user judgments than those in movies and recipes.



Word2vec applied to recommendation: hyperparameters matter

This work investigates the marginal importance of each hyperparameters in a recommendation setting through large hyperparameter grid searches on various datasets, and finds that optimal hyper-parameters configurations for Natural Language Processing tasks and Recommendation tasks are noticeably different.

Distributed Representations of Words and Phrases and their Compositionality

This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.

WTF: the who to follow service at Twitter

An architectural overview of the architecture of WTF is provided and a few graph recommendation algorithms implemented in Cassovary are described and evaluated, including a novel approach based on a combination of random walks and SALSA.

Privacy-Aware Recommender Systems Challenge on Twitter's Home Timeline.

The key challenges faced by researchers and professionals striving to predict user engagements are touched on, and the key aspects of the RecSys 2020 Challenge that was organized by ACM RecSys in partnership with Twitter using this dataset are described.

Privacy-Preserving Recommender Systems Challenge on Twitter's Home Timeline

The key challenges faced by researchers and professionals striving to predict user engagements are touched on, and the key aspects of the RecSys 2020 Challenge that was organized by ACM RecSys in partnership with Twitter using this dataset are described.

An economic method of computing LPτ-sequences

Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining

Many small online retailers and new entrants to the online retail sector are keen to practice data mining and consumer-centric marketing in their businesses yet technically lack the necessary

Real-time Personalization using Embeddings for Search Ranking at Airbnb

The embedding models were specifically tailored for Airbnb marketplace, and are able to capture guest's short-term and long-term interests, delivering effective home listing recommendations.

Learning and Transferring IDs Representation in E-commerce

This paper proposes an embedding based framework to learn and transfer the representation of IDs, and deploy and evaluate the proposed approach in Hema App and the results validate its effectiveness.

Session-Based Fraud Detection in Online E-Commerce Transactions Using Recurrent Neural Networks

CLUE captures detailed information on users’ click actions using neural-network based embedding, and models sequences of such clicks using the recurrent neural network and provides application-specific design optimizations including imbalanced learning, real-time detection, and incremental model update.