Corpus ID: 214693121

Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

  title={Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)},
  author={Joelle Pineau and Philippe Vincent-Lamarre and Koustuv Sinha and V. Larivi{\`e}re and A. Beygelzimer and Florence d'Alch{\'e}-Buc and Emily Fox and H. Larochelle},
  • Joelle Pineau, Philippe Vincent-Lamarre, +5 authors H. Larochelle
  • Published 2020
  • Computer Science, Mathematics
  • ArXiv
  • One of the challenges in machine learning research is to ensure that presented and published results are sound and reliable. Reproducibility, that is obtaining similar results as presented in a paper or talk, using the same code and data (when available), is a necessary step to verify the reliability of research findings. Reproducibility is also an important step to promote open and accessible research, thereby allowing the scientific community to quickly integrate new findings and convert… CONTINUE READING
    16 Citations

    Figures and Tables from this paper

    Ideas for Improving the Field of Machine Learning: Summarizing Discussion from the NeurIPS 2019 Retrospectives Workshop
    • Highly Influenced
    • PDF
    On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
    • 9
    • PDF
    Acme: A Research Framework for Distributed Reinforcement Learning
    • 19
    • PDF
    Between Subjectivity and Imposition
    Exploring Reproducibility in Visualization
    Neural Temporal Point Processes For Modelling Electronic Health Records
    • Highly Influenced
    • PDF


    The Scientific Method in the Science of Machine Learning
    • 17
    • PDF
    A Step Toward Quantifying Independently Reproducible Machine Learning Research
    • 12
    • Highly Influential
    • PDF
    Unreproducible Research is Reproducible
    • 19
    • PDF
    Deep Reinforcement Learning that Matters
    • 722
    • PDF
    On the State of the Art of Evaluation in Neural Language Models
    • 329
    • PDF
    Reporting of artificial intelligence prediction models
    • 101
    Are GANs Created Equal? A Large-Scale Study
    • 478
    • PDF
    A manifesto for reproducible science
    • 1,199
    • PDF
    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    • 13,754
    • PDF