• Corpus ID: 237142369

An NLP approach to quantify dynamic salience of predefined topics in a text corpus

@article{Bock2021AnNA,
  title={An NLP approach to quantify dynamic salience of predefined topics in a text corpus},
  author={Alexander Asp Bock and Anthony Palladino and Skaidra Smith-Heisters and Ian Boardman and Ester Pellegrini and Elisa Jayne Bienenstock and Andrew Valenti},
  journal={ArXiv},
  year={2021},
  volume={abs/2108.07345}
}

• A. Bock, A. Palladino, +4 authors Andrew Valenti
• Published 16 August 2021
• Computer Science
• ArXiv

The proliferation of news media available online simultaneously presents a valuable resource and significant challenge to analysts aiming to profile and understand social and cultural trends in a geographic location of interest. While an abundance of news reports documenting significant events, trends, and responses provides a more democratized picture of the social characteristics of a location, making sense of an entire corpus to extract significant trends is a steep challenge for any one analyst or team. Here, we present an approach using natural language processing techniques that seeks to quantify how a set of predefined topics of interest change over time across a large corpus of text. We found that, given a predefined topic, we can identify and rank sets of terms, or n-grams, that map to those topics and have usage patterns that deviate from a normal baseline. Emergence, disappearance, or significant variations in n-gram usage present a ground-up picture of a topic’s dynamic salience within a corpus of interest.