Skip to search formSkip to main contentSkip to account menu

Cross-Document Language Modeling

The crossdocument language model (CD-LM) improves masked language modeling for multi-document NLP tasks with two key ideas, including pretraining with multiple related documents in a single input, via cross-document masking, which encourages the model to learn cross- document and long-range relationships.

Paraphrasing vs Coreferring: Two Sides of the Same Coin

This work used annotations from an event coreference dataset as distant supervision to re-score heuristically-extracted predicate paraphrases, and used the same re-ranking features as additional inputs to a state-of-the-art eventcoreference resolution model, which yielded modest but consistent improvements to the model’s performance.
View on ACL (opens in a new tab)

Unsupervised Linear and Nonlinear Channel Equalization and Decoding Using Variational Autoencoders

A new approach for blind channel equalization and decoding, variational inference, and variational autoencoders (VAEs) in particular, is introduced and significant and consistent improvements in the error rate of the reconstructed symbols are demonstrated, compared to existing blind equalization methods, thus enabling faster channel acquisition.
View on IEEE (opens in a new tab)

CDLM: Cross-Document Language Modeling

This work introduces a new pretraining approach geared for multi-document language modeling, incorporating two key ideas into the masked language modeling self-supervised objective: pretrain over sets of multiple related documents, encouraging the model to learn cross-document relationships.
View PDF on arXiv (opens in a new tab)

Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding

Distilled Sentence Embedding is introduced - a model that is based on knowledge distillation from cross-attentive models, focusing on sentence-pair tasks that significantly outperforms several ELMO variants and other sentence embedding methods, while accelerating computation of the query-candidate sentence-pairs similarities.
View PDF on arXiv (opens in a new tab)

Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space

This work reverse-engineering the operation of the feed-forward network layers, one of the building blocks of transformer models, shows that each update can be decomposed to sub-updates corresponding to single FFN parameter vectors, each promoting concepts that are often human-interpretable.
View on ACL (opens in a new tab)

Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps

A novel gradient-based method that analyzes self-attention units and identifies the input elements that explain the model's prediction the best, and obtains significant improvements over state-of-the-art alternatives.
View on ACM (opens in a new tab)

Blind Channel Equalization Using Variational Autoencoders

A new maximum likelihood estimation approach for blind channel equalization, using variational autoencoders (VAEs), is introduced. Significant and consistent improvements in the error rate of the…
View on IEEE (opens in a new tab)

RecoBERT: A Catalog Language Model for Text-Based Recommendations

This work introduces RecoBERT, a BERT-based approach for learning catalog-specialized language models for text-based item recommendations, and suggests novel training and inference procedures for scoring similarities between pairs of items, that don't require item similarity labels.
View PDF on arXiv (opens in a new tab)

Attentive Item2vec: Neural Attentive User Representations

This work presents Attentive Item2vec (AI2V) - a novel attentive version of Item 2vec that employs a context-target attention mechanism in order to learn and capture different characteristics of user historical behavior with respect to a potential recommended item (target).
View on IEEE (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)