Corpus ID: 234777911

Effective Attention Sheds Light On Interpretability

@inproceedings{Sun2021EffectiveAS,
  title={Effective Attention Sheds Light On Interpretability},
  author={Kai Sun and Ana Marasovi{\'c}},
  booktitle={FINDINGS},
  year={2021}
}
An attention matrix of a transformer selfattention sublayer can provably be decomposed into two components and only one of them (effective attention) contributes to the model output. This leads us to ask whether visualizing effective attention gives different conclusions than interpretation of standard attention. Using a subset of the GLUE tasks and BERT, we carry out an analysis to compare the two attention matrices, and show that their interpretations differ. Effective attention is less… Expand
1 Citations

Figures and Tables from this paper

References

SHOWING 1-10 OF 26 REFERENCES
Analyzing the Structure of Attention in a Transformer Language Model
Attention is not Explanation
Interrogating the Explanatory Power of Attention in Neural Machine Translation
Attention is not not Explanation
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
...
1
2
3
...