Online Optimization with Predictions and Non-convex Losses

  title={Online Optimization with Predictions and Non-convex Losses},
  author={Yiheng Lin and Gautam Goel and Adam Wierman},
  journal={Proceedings of the ACM on Measurement and Analysis of Computing Systems},
  pages={1 - 32}
We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask:under what general conditions is it possible for an online learner to leverage predictions of future cost functions in order to achieve near-optimal costs? Prior work has provided near-optimal online algorithms for specific combinations of assumptions about hitting and switching costs… Expand
Leveraging Predictions in Smoothed Online Convex Optimization via Gradient-based Algorithms
A gradient-based online algorithm, Receding Horizon Inexact Gradient (RHIG), is introduced, and its performance by dynamic regrets in terms of the temporal variation of the environment and the prediction errors is analyzed. Expand
Combining Regularization with Look-Ahead for Competitive Online Convex Optimization
This paper proposes a new algorithm, called Regularization with Look-Ahead (RLA), that can get the best of both AFHC and the regularization method, and provides a matching lower bound for the competitive ratios of all online algorithms with look-ahead. Expand
Beyond No-Regret: Competitive Control via Online Optimization with Memory
A novel reduction from online control of a class of controllable systems to online convex optimization with memory is provided and a new algorithm is designed that has a constant, dimension-free competitive ratio, leading to a new constant-competitive approach for online control. Expand
Information Aggregation for Constrained Online Control
This paper uses a form of feasibility aggregation based on entropic maximization in combination with a novel online algorithm, named Penalized Predictive Control (PPC), and demonstrates that aggregated information can be efficiently learned using reinforcement learning algorithms. Expand
Dimension-Free Bounds on Chasing Convex Functions
The problem of chasing convex functions, where functions arrive over time, is considered, and an algorithm is given that achieves an $O(\sqrt \kappa)$-competitiveness, when the functions are supported on $k$-dimensional affine subspaces. Expand
Optimization Algorithms as Robust Feedback Controllers
This article reviews several research streams that have been pursued in this direction, including extremum seeking and pertinent methods from model predictive and process control, and focuses on recent methods under the name of “feedback-based optimization”, which directly implement optimization algorithms in closed loop with physical systems. Expand
Algorithms for Right-Sizing Heterogeneous Data Centers
An online algorithm based on a work function approach which achieves a competitive ratio of 2d + 1 + ε for any ε > 0.5 is developed, which is nearly optimal for time-independent operating cost functions. Expand
Competitive Control with Delayed Imperfect Information
A greedy, myopic policy is introduced that yields a constant competitive ratio against the offline optimal policy with delayed feedback and inexact predictions, and the fundamental limits of online control with limited information are analyzed. Expand
Online Optimization with Memory and Competitive Control
The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio and shows a connection between online optimization with memory and online control with adversarial disturbances. Expand
The Power of Predictions in Online Control
This analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adversarial settings, and requires only $O(\log T)$ predictions to reach dynamic regret, which matches the lower bound on the required prediction horizon for constant regret. Expand


Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent
OBD is the first algorithm to achieve a dimension-free competitive ratio, 3 + O(1/\alpha)$, for locally polyhedral costs, where $\alpha$ measures the "steepness" of the costs. Expand
Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization
A new lower bound is proved on the competitive ratio of any online algorithm in the setting where the costs are $m-strongly convex and the movement costs are the squared $\ell_2$ norm, showing that no algorithm can achieve a competitive ratio that is $o(m^{-1/2})$ as $m$ tends to zero. Expand
On the Value of Look-Ahead in Competitive Online Convex Optimization
This paper designs an Averaging Regularized Moving Horizon Control algorithm that is the first to attain a low competitive ratio that is independent of either the coefficients of the switching costs and service costs, or the upper and lower bounds of the inputs, and develops a WRMHC algorithm that carefully weights the decisions inside the look-ahead window based on the accuracy of theLook-ahead information. Expand
An Online Algorithm for Smoothed Regression and LQR Control
The generality of the OBD framework can be used to construct competitive algorithms for a variety of online problems across learning and control, including online variants of ridge regression, logistic regression, maximum likelihood estimation, and LQR control. Expand
An Optimal Algorithm for Online Non-Convex Learning
The Online Recursive Weighting algorithm with regret of $O(\sqrtT )$, matching the tight regret lower bound for the øco problem, and fills the regret gap between the state-of-the-art results in the online convex and non-convex optimization problems. Expand
Using Predictions in Online Optimization: Looking Forward with an Eye on the Past
This paper introduces a new class of policies, Committed Horizon Control (CHC), that generalizes both RHC and AFHC and provides average-case analysis and concentration results for CHC policies, yielding the first analysis of RHC for OCO problems with noisy predictions. Expand
Online Optimization With Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit
A fundamental lower bound on the dynamic regret for general online algorithms with a finite prediction window is provided, meaning that the performance can not improve significantly even with more computation. Expand
Using Predictions in Online Optimization with Switching Costs: A Fast Algorithm and A Fundamental Limit
A computationally efficient algorithm, Receding Horizon Gradient Descent (RHGD), which only requires a finite number of gradient evaluations at each time is proposed, and it is shown that both the dynamic regret and the competitive ratio of the algorithm decay exponentially fast with the length of the prediction window. Expand
A Tight Lower Bound for Online Convex Optimization with Switching Costs
This work investigates online convex optimization with switching costs (OCO), a natural online problem arising when rightsizing data centers, and designs competitive algorithms based on the sum of costs of all steps. Expand
Online convex optimization with ramp constraints
It is proved that AFHC achieves the asymptotically optimal achievable competitive difference within a general class of “forward looking” online algorithms. Expand