• Corpus ID: 162168726

Time-Smoothed Gradients for Online Forecasting

  title={Time-Smoothed Gradients for Online Forecasting},
  author={Tianhao Zhu and Serg{\"u}l Ayd{\"o}re},
Here, we study different update rules in stochastic gradient descent (SGD) for online forecasting problems. The selection of the learning rate parameter is critical in SGD. However, it may not be feasible to tune this parameter in online learning. Therefore, it is necessary to have an update rule that is not sensitive to the selection of the learning parameter. Inspired by the local regret metric that we introduced previously, we propose to use time-smoothed gradients within SGD update. Using… 

Figures from this paper



GEFCOM 2014 - Probabilistic Electricity Price Forecasting

This paper compares a well-known and established method, ARMA with exogenous variables with a relatively new technique Gradient Boosting Regression, and reveals that a multi-model approach is significantly better performing in terms of error metrics.

A Local Regret in Nonconvex Online Learning

This work introduces another definition of regret, inspired by the concept of calibration and a local gradient based regret, and discusses why this definition is more interpretable for forecasting problems.

Long Short-Term Memory

A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.

Efficient Regret Minimization in Non-Convex Games

A natural notion of regret is defined which permits efficient optimization and generalizes offline guarantees for convergence to an approximate local optimum and gradient-based methods that achieve optimal regret are given.

Python for Scientific Computing

  • T. Oliphant
  • Computer Science
    Computing in Science & Engineering
  • 2007
Python is an excellent "steering" language for scientific codes written in other languages. However, with additional basic tools, Python transforms into a high-level language suited for scientific