Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories

@article{Parsonson2022ReinforcementLF,
  title={Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories},
  author={Christopher W. F. Parsonson and Alexandre Laterre and Thomas D. Barrett},
  journal={ArXiv},
  year={2022},
  volume={abs/2205.14345}
}
Combinatorial optimisation problems framed as mixed integer linear programmes (MILPs) are ubiquitous across a range of real-world applications. The canonical branch-and-bound al- gorithm seeks to exactly solve MILPs by constructing a search tree of increasingly constrained sub-problems. In practice, its solving time performance is dependent on heuristics, such as the choice of the next variable to constrain (‘branching’). Recently, machine learning (ML) has emerged as a promising paradigm for… 

Intelligent Variable Selection for Branch & Bound Methods

Combinatorial optimisation (Wolsey and Nemhauser [22]) is applied to a wide vari-ety of real-world problems like job scheduling, capacity planning and supply-chain management. These problems are

Lookback for Learning to Branch

To imitate the target behavior more closely by incorporating the lookback phenomenon in GNNs, two methods are proposed: (a) target smoothing for the standard cross-entropy loss function, and (b) adding a Parent-as-Target (PAT) Lookback regularizer term.

Network Aware Compute and Memory Allocation in Optically Composable Data Centres with Deep Reinforcement Learning and Graph Neural Networks

It is demonstrated that deep reinforcement learning, where the policy is modelled by a graph neural network can be used to learn effective network-aware and topologically-scalable allocation policies end-to-end, and can achieve the same acceptance ratio as the best performing heuristic.

Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks

  • Christopher W. F. ParsonsonZacharaya ShabkaAlessandro OttinoGeorgios Zervas
  • Computer Science
  • 2023
It is shown that maximum parallelisation is sub-optimal in relation to user-critical metrics such as throughput and blocking rate, and a proposed PAC-ML (partitioning for asynchronous computing with machine learning) is proposed.

References

SHOWING 1-10 OF 61 REFERENCES

Reinforcement Learning for Variable Selection in a Branch and Bound Algorithm

This paper leverage patterns in real-world instances to learn from scratch a new branching strategy optimised for a given problem and compare it with a commercial solver, and proposes FMSTS, a novel Reinforcement Learning approach specifically designed for this task.

Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies

A novel imitation learning framework is proposed, and new input features and architectures to represent branching are introduced, and the resulting policies significantly outperform the current state-of-the-art method for "learning to branch" by effectively allowing generalization to generic unseen instances.

Learning to Branch in Mixed Integer Programming

This work proposes a machine learning (ML) framework for variable branching in MIP, and observes the decisions made by Strong Branching, a time-consuming strategy that produces small search trees, collecting features that characterize the candidate branching variables at each node of the tree.

First return then explore

The substantial performance gains from Go-Explore suggest that the simple principles of remembering states, returning to them, and exploring from them are a powerful and general approach to exploration-an insight that may prove critical to the creation of truly intelligent learning agents.

NovelD: A Simple yet Effective Exploration Criterion

This paper proposes a simple but effective criterion called NovelD, which solves all the static procedurally-generated tasks in Mini-Grid with just 120M environment steps, without any curriculum learning and shows comparable performance or even outperforms multiple SOTA exploration methods in many hard exploration tasks.

Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents

This paper shows that algorithms that have been invented to promote directed exploration in small-scale evolved neural networks via populations of exploring agents, specifically novelty search and quality diversity algorithms, can be hybridized with ES to improve its performance on sparse or deceptive deep RL tasks, while retaining scalability.

Learning to Search in Branch and Bound Algorithms

This work addresses the key challenge of learning an adaptive node searching order for any class of problem solvable by branch-and-bound by applying its algorithm to linear programming based branch- and-bound for solving mixed integer programs (MIP).

Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon

Hybrid Models for Learning to Branch

This work addresses the first question in the negative, and addresses the second question by proposing a new hybrid architecture for efficient branching on CPU machines, which combines the expressive power of GNNs with computationally inexpensive multi-layer perceptrons (MLP) for branching.

Solving Mixed Integer Programs Using Neural Networks

This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one.
...