Analysis Methods in Neural Language Processing: A Survey

  title={Analysis Methods in Neural Language Processing: A Survey},
  author={Yonatan Belinkov and James R. Glass},
  journal={Transactions of the Association for Computational Linguistics},
  • Yonatan Belinkov, James R. Glass
  • Published 2019
  • Computer Science
  • Transactions of the Association for Computational Linguistics
  • The field of natural language processing has seen impressive progress in recent years, with neural network models replacing many of the traditional systems. A plethora of new models have been proposed, many of which are thought to be opaque compared to their feature-rich counterparts. This has led researchers to analyze, interpret, and evaluate neural networks in novel and more fine-grained ways. In this survey paper, we review analysis methods in neural language processing, categorize them… CONTINUE READING
    Analyzing the Structure of Attention in a Transformer Language Model
    • 37
    • PDF
    Evaluating Recurrent Neural Network Explanations
    • 19
    • PDF
    Linguistic Profiling of a Neural Language Model
    oLMpics - On what Language Model Pre-training Captures
    • 40
    • PDF


    Publications referenced by this paper.
    What's Going On in Neural Constituency Parsers? An Analysis
    • 36
    • PDF
    Exploring the Limits of Language Modeling
    • 740
    • PDF
    Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference
    • 36
    • PDF
    What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models
    • 40
    • PDF
    Understanding Neural Networks through Representation Erasure
    • 220
    • PDF
    Visualizing and Understanding Neural Models in NLP
    • 373
    • PDF
    Representation of Linguistic Form and Function in Recurrent Neural Networks
    • 97
    • PDF
    Visualizing and Understanding Recurrent Networks
    • 724
    • PDF
    The Importance of Being Recurrent for Modeling Hierarchical Structure
    • 75
    • PDF