# Online Optimization with Predictions and Non-convex Losses

@article{Lin2020OnlineOW,
title={Online Optimization with Predictions and Non-convex Losses},
author={Yiheng Lin and Gautam Goel and Adam Wierman},
journal={Proceedings of the ACM on Measurement and Analysis of Computing Systems},
year={2020},
volume={4},
pages={1 - 32}
}
• Published 2020
• Computer Science, Mathematics
• Proceedings of the ACM on Measurement and Analysis of Computing Systems
We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask:under what general conditions is it possible for an online learner to leverage predictions of future cost functions in order to achieve near-optimal costs? Prior work has provided near-optimal online algorithms for specific combinations of assumptions about hitting and switching costs… Expand
10 Citations

#### Figures and Topics from this paper

Leveraging Predictions in Smoothed Online Convex Optimization via Gradient-based Algorithms
• Computer Science, Engineering
• NeurIPS
• 2020
A gradient-based online algorithm, Receding Horizon Inexact Gradient (RHIG), is introduced, and its performance by dynamic regrets in terms of the temporal variation of the environment and the prediction errors is analyzed. Expand
Combining Regularization with Look-Ahead for Competitive Online Convex Optimization
• Computer Science
• IEEE INFOCOM 2021 - IEEE Conference on Computer Communications
• 2021
This paper proposes a new algorithm, called Regularization with Look-Ahead (RLA), that can get the best of both AFHC and the regularization method, and provides a matching lower bound for the competitive ratios of all online algorithms with look-ahead. Expand
Beyond No-Regret: Competitive Control via Online Optimization with Memory
• Computer Science, Engineering
• ArXiv
• 2020
A novel reduction from online control of a class of controllable systems to online convex optimization with memory is provided and a new algorithm is designed that has a constant, dimension-free competitive ratio, leading to a new constant-competitive approach for online control. Expand
Information Aggregation for Constrained Online Control
• Tongxin Li, Bo Sun
• Computer Science
• Proc. ACM Meas. Anal. Comput. Syst.
• 2021
This paper uses a form of feasibility aggregation based on entropic maximization in combination with a novel online algorithm, named Penalized Predictive Control (PPC), and demonstrates that aggregated information can be efficiently learned using reinforcement learning algorithms. Expand
Dimension-Free Bounds on Chasing Convex Functions
• Computer Science, Mathematics
• COLT
• 2020
The problem of chasing convex functions, where functions arrive over time, is considered, and an algorithm is given that achieves an $O(\sqrt \kappa)$-competitiveness, when the functions are supported on $k$-dimensional affine subspaces. Expand
Optimization Algorithms as Robust Feedback Controllers
• Computer Science, Mathematics
• ArXiv
• 2021
This article reviews several research streams that have been pursued in this direction, including extremum seeking and pertinent methods from model predictive and process control, and focuses on recent methods under the name of “feedback-based optimization”, which directly implement optimization algorithms in closed loop with physical systems. Expand
Algorithms for Right-Sizing Heterogeneous Data Centers
• Computer Science
• SPAA
• 2021
An online algorithm based on a work function approach which achieves a competitive ratio of 2d + 1 + ε for any ε > 0.5 is developed, which is nearly optimal for time-independent operating cost functions. Expand
Competitive Control with Delayed Imperfect Information
• Computer Science, Mathematics
• ArXiv
• 2020
A greedy, myopic policy is introduced that yields a constant competitive ratio against the offline optimal policy with delayed feedback and inexact predictions, and the fundamental limits of online control with limited information are analyzed. Expand
Online Optimization with Memory and Competitive Control
• Computer Science
• NeurIPS
• 2020
The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio and shows a connection between online optimization with memory and online control with adversarial disturbances. Expand
The Power of Predictions in Online Control
• Computer Science, Mathematics
• NeurIPS
• 2020
This analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adversarial settings, and requires only $O(\log T)$ predictions to reach dynamic regret, which matches the lower bound on the required prediction horizon for constant regret. Expand

#### References

SHOWING 1-10 OF 62 REFERENCES
Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent
• Computer Science, Mathematics
• COLT
• 2018
OBD is the first algorithm to achieve a dimension-free competitive ratio, 3 + O(1/\alpha)$, for locally polyhedral costs, where$\alpha$measures the "steepness" of the costs. Expand Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization • Computer Science, Mathematics • NeurIPS • 2019 A new lower bound is proved on the competitive ratio of any online algorithm in the setting where the costs are$m-strongly convex and the movement costs are the squared $\ell_2$ norm, showing that no algorithm can achieve a competitive ratio that is $o(m^{-1/2})$ as $m$ tends to zero. Expand
On the Value of Look-Ahead in Competitive Online Convex Optimization
• Computer Science
• Proc. ACM Meas. Anal. Comput. Syst.
• 2019
This paper designs an Averaging Regularized Moving Horizon Control algorithm that is the first to attain a low competitive ratio that is independent of either the coefficients of the switching costs and service costs, or the upper and lower bounds of the inputs, and develops a WRMHC algorithm that carefully weights the decisions inside the look-ahead window based on the accuracy of theLook-ahead information. Expand
An Online Algorithm for Smoothed Regression and LQR Control
• Computer Science
• AISTATS
• 2019
The generality of the OBD framework can be used to construct competitive algorithms for a variety of online problems across learning and control, including online variants of ridge regression, logistic regression, maximum likelihood estimation, and LQR control. Expand
An Optimal Algorithm for Online Non-Convex Learning
• Computer Science
• Proc. ACM Meas. Anal. Comput. Syst.
• 2018
The Online Recursive Weighting algorithm with regret of $O(\sqrtT )$, matching the tight regret lower bound for the øco problem, and fills the regret gap between the state-of-the-art results in the online convex and non-convex optimization problems. Expand
Using Predictions in Online Optimization: Looking Forward with an Eye on the Past
• Computer Science
• SIGMETRICS 2016
• 2016
This paper introduces a new class of policies, Committed Horizon Control (CHC), that generalizes both RHC and AFHC and provides average-case analysis and concentration results for CHC policies, yielding the first analysis of RHC for OCO problems with noisy predictions. Expand
Online Optimization With Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit
• Computer Science, Mathematics
• IEEE Transactions on Automatic Control
• 2021
A fundamental lower bound on the dynamic regret for general online algorithms with a finite prediction window is provided, meaning that the performance can not improve significantly even with more computation. Expand
Using Predictions in Online Optimization with Switching Costs: A Fast Algorithm and A Fundamental Limit
• Computer Science
• 2018 Annual American Control Conference (ACC)
• 2018
A computationally efficient algorithm, Receding Horizon Gradient Descent (RHGD), which only requires a finite number of gradient evaluations at each time is proposed, and it is shown that both the dynamic regret and the competitive ratio of the algorithm decay exponentially fast with the length of the prediction window. Expand
A Tight Lower Bound for Online Convex Optimization with Switching Costs
• Mathematics, Computer Science
• WAOA
• 2017
This work investigates online convex optimization with switching costs (OCO), a natural online problem arising when rightsizing data centers, and designs competitive algorithms based on the sum of costs of all steps. Expand
Online convex optimization with ramp constraints
• Mathematics, Computer Science
• 2015 54th IEEE Conference on Decision and Control (CDC)
• 2015
It is proved that AFHC achieves the asymptotically optimal achievable competitive difference within a general class of “forward looking” online algorithms. Expand