• Publications
  • Influence
Revealing the Dark Secrets of BERT
TLDR
It is shown that manually disabling attention in certain heads leads to a performance improvement over the regular fine-tuned BERT models, indicating the overall model overparametrization.
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
TLDR
A large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives, and the impact on occupation classification of including explicit gender indicators in different semantic representations of online biographies.
Lessons from Natural Language Inference in the Clinical Domain
TLDR
This work introduces MedNLI - a dataset annotated by doctors, performing a natural language inference task (NLI), grounded in the medical history of patients, and presents strategies to leverage transfer learning using datasets from the open domain and incorporate domain knowledge from external data and lexical sources.
RuSentiment: An Enriched Sentiment Analysis Dataset for Social Media in Russian
TLDR
RuSentiment, a new dataset for sentiment analysis of social media posts in Russian, and a new set of comprehensive annotation guidelines that are extensible to other languages are presented.
GhostWriter: Using an LSTM for Automatic Rap Lyric Generation
TLDR
This paper demonstrates the effectiveness of a Long Short-Term Memory language model in the initial efforts to generate unconstrained rap lyrics, which produces better “ghostwritten” lyrics than a baseline model.
Here’s My Point: Joint Pointer Architecture for Argument Mining
TLDR
This work presents the first neural network-based approach to link extraction in argument mining, and proposes a novel architecture that applies Pointer Network sequence-to-sequence attention modeling to structural prediction in discourse parsing tasks and develops a joint model that achieves state-of-the-art results.
SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor
TLDR
A new shared task for humor understanding that attempts to eschew the ubiquitous binary approach to humor detection and focus on comparative humor ranking instead, based on a new dataset of funny tweets posted in response to shared hashtags collected from the ‘Hashtag Wars’ segment of the TV show @midnight.
What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes
TLDR
This work proposes a method for discouraging correlation between the predicted probability of an individual’s true occupation and a word embedding of their name, which leverages the societal biases that are encoded in word embeddings, eliminating the need for access to protected attributes.
Adversarial Decomposition of Text Representation
TLDR
The proposed method for adversarial decomposition of text representation uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition.
Combining Network and Language Indicators for Tracking Conflict Intensity
TLDR
A random-walk based measure of graph polarization, text-based sentiment analysis, and the corresponding shift in word meaning and use by the opposing sides of social or political conflict are looked at.
...
...