Llion Jones

Publications
Influence

Ashish Vaswani, Noam Shazeer, +5 authors Illia Polosukhin
Computer Science
NIPS
12 June 2017

We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Expand

15,924
3898
PDF

Natural Questions: A Benchmark for Question Answering Research

T. Kwiatkowski, J. Palomaki, +15 authors Slav Petrov
Computer Science
Transactions of the Association for Computational…
2 August 2019

We present the Natural Questions corpus, a question answering data set. Expand

Tensor2Tensor for Neural Machine Translation

Ashish Vaswani, S. Bengio, +10 authors Jakob Uszkoreit
Computer Science, Mathematics
AMTA
16 March 2018

Tensor2Tensor is a library for deep learning models well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model. Expand

Character-Level Language Modeling with Deeper Self-Attention

Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, Llion Jones
Computer Science, Mathematics
AAAI
9 August 2018

In this paper, we show that a deep (64-layer) transformer model with fixed context outperforms RNN variants by a large margin, achieving state of the art on two popular benchmarks. Expand

WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia

D. Hewlett, Alexandre Lacoste, +5 authors David Berthelot
Computer Science
ACL
1 August 2016

We present WIKIREADING, a large-scale natural language understanding task and publicly-available dataset with 18 million instances. Expand

The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation

M. Chen, Orhan Firat, +9 authors Macduff Hughes
Computer Science
ACL
26 April 2018

The past year has witnessed rapid advances in sequence-to-sequence (seq2seq) modeling for Machine Translation (MT). Expand

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Jonathan Shen, P. Nguyen, +56 authors E. al.
Computer Science, Mathematics
ArXiv
21 February 2019

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Expand

One Model To Learn Them All

L. Kaiser, Aidan N. Gomez, +4 authors Jakob Uszkoreit
Computer Science, Mathematics
ArXiv
16 June 2017

We present a single model that yields good results on a number of problems spanning multiple domains that can simultaneously learn multiple tasks from various domains. Expand

ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing

A. Elnaggar, Michael Heinzinger, +9 authors B. Rost
Computer Science, Mathematics
ArXiv
12 July 2020

We trained two auto-regressive language models (Transformer-XL, XLNet) on data from UniRef and BFD containing up to 393 billion amino acids (words) from 2.1 billion protein sequences. Expand

Accurate Supervised and Semi-Supervised Machine Reading for Long Documents

Izzeddin Gur, D. Hewlett, Alexandre Lacoste, Llion Jones
Computer Science
EMNLP
1 September 2017

We introduce a hierarchical architecture for machine reading capable of extracting precise information from long documents. Expand

Recommended Authors