Pathologies of Neural Models Make Interpretation Difficult

@inproceedings{Feng2018PathologiesON,
  title={Pathologies of Neural Models Make Interpretation Difficult},
  author={Shi Feng and Eric Wallace and Alvin Grissom and Mohit Iyyer and Pedro Rodriguez and Jordan L. Boyd-Graber},
  booktitle={EMNLP},
  year={2018}
}
  • Shi Feng, Eric Wallace, +3 authors Jordan L. Boyd-Graber
  • Published in EMNLP 2018
  • Computer Science
  • One way to interpret neural model predictions is to highlight the most important input features---for example, a heatmap visualization over the words in an input sentence. [...] Key Method To understand the limitations of these methods, we use input reduction, which iteratively removes the least important word from the input. This exposes pathological behaviors of neural models: the remaining words appear nonsensical to humans and are not the ones determined as important by interpretation methods. As we confirm…Expand Abstract

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 71 CITATIONS

    AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models

    VIEW 7 EXCERPTS
    CITES BACKGROUND & METHODS
    HIGHLY INFLUENCED

    ERASER: A Benchmark to Evaluate Rationalized NLP Models

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    Universal Adversarial Triggers for NLP

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    Evaluating NLP Models via Contrast Sets

    VIEW 3 EXCERPTS
    CITES METHODS, RESULTS & BACKGROUND

    Interpreting Neural Networks With Nearest Neighbors

    VIEW 7 EXCERPTS
    CITES BACKGROUND

    Undersensitivity in Neural Reading Comprehension

    VIEW 5 EXCERPTS
    CITES BACKGROUND, RESULTS & METHODS
    HIGHLY INFLUENCED

    FILTER CITATIONS BY YEAR

    2018
    2020

    CITATION STATISTICS

    • 11 Highly Influenced Citations

    • Averaged 24 Citations per year from 2018 through 2020

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 50 REFERENCES

    Interpretation of Neural Networks is Fragile

    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Visualizing and Understanding Neural Models in NLP

    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    Understanding Neural Networks through Representation Erasure

    VIEW 9 EXCERPTS
    HIGHLY INFLUENTIAL