How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations

@article{Aken2019HowDB,
  title={How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations},
  author={Betty van Aken and Benjamin Winter and Alexander L{\"o}ser and Felix A. Gers},
  journal={Proceedings of the 28th ACM International Conference on Information and Knowledge Management},
  year={2019}
}
  • Betty van Aken, Benjamin Winter, +1 author Felix A. Gers
  • Published 2019
  • Computer Science
  • Proceedings of the 28th ACM International Conference on Information and Knowledge Management
  • Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory. In order to better understand BERT and other Transformer-based models, we present a layer-wise analysis of BERT's hidden states. Unlike previous research, which mainly focuses on explaining Transformer models by their attention weights, we argue that hidden… CONTINUE READING

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 11 CITATIONS

    Cross-lingual Zero Pronoun Resolution

    VIEW 1 EXCERPT
    CITES BACKGROUND

    References

    Publications referenced by this paper.
    SHOWING 1-3 OF 3 REFERENCES