An LP-based hyperparameter optimization model for language modeling

  title={An LP-based hyperparameter optimization model for language modeling},
  author={Amir Hossein Akhavan Rahnama and M. Toloo and N. Zaidenberg},
  journal={The Journal of Supercomputing},
  • Amir Hossein Akhavan Rahnama, M. Toloo, N. Zaidenberg
  • Published 2018
  • Computer Science, Mathematics
  • The Journal of Supercomputing
  • In order to find hyperparameters for a machine learning model, algorithms such as grid search or random search are used over the space of possible values of the models’ hyperparameters. These search algorithms opt the solution that minimizes a specific cost function. In language models, perplexity is one of the most popular cost functions. In this study, we propose a fractional nonlinear programming model that finds the optimal perplexity value. The special structure of the model allows us to… CONTINUE READING
    2 Citations


    Algorithms for Hyper-Parameter Optimization
    • 1,645
    • PDF
    Random Search for Hyper-Parameter Optimization
    • 3,994
    • PDF
    A Gaussian Prior for Smoothing Maximum Entropy Models
    • 384
    • PDF
    A Neural Probabilistic Language Model
    • 4,682
    • PDF
    A bit of progress in language modeling
    • J. Goodman
    • Mathematics, Computer Science
    • Comput. Speech Lang.
    • 2001
    • 510
    • PDF
    Recurrent neural network based language model
    • 4,185
    • PDF
    Some Studies in Machine Learning Using the Game of Checkers
    • 2,183
    • Highly Influential
    • PDF
    Large Language Models in Machine Translation
    • 490
    • PDF
    The Unreasonable Effectiveness of Data
    • 986
    • PDF
    Class-Based n-gram Models of Natural Language
    • 3,070
    • PDF