• Corpus ID: 246473259

Multi-Task Learning as a Bargaining Game

@article{Navon2022MultiTaskLA,
  title={Multi-Task Learning as a Bargaining Game},
  author={Aviv Navon and Aviv Shamsian and Idan Achituve and Haggai Maron and Kenji Kawaguchi and Gal Chechik and Ethan Fetaya},
  journal={ArXiv},
  year={2022},
  volume={abs/2202.01017}
}
In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency; however, since the gradients of these different tasks may conflict, training a joint model for MTL often yields lower performance than its corresponding single-task counterparts. A common method for alleviating this issue is to combine per-task gradients into a joint update direction using a particular heuristic. In… 

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach

A stochastic M ulti-objective gradient C orrection ( MoCo) method for multi- objective optimization that can guarantee convergence without increasing the batch size even in the nonconvex setting and can outperforms state-of-the-art MTL algorithms in challenging MTL benchmarks.

In Defense of the Unitary Scalarization for Deep Multi-Task Learning

It is shown that unitary scalarization, coupled with standard regularization and stabilization techniques from single-task learning, matches or improves upon the performance of complex multi-task optimizers in popular supervised and reinforcement learning settings.

Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models

Pareto Manifold Learning is proposed, an ensembling method in weight space that is able to discover such a parameterization and produces a continuous Pareto Front in a single training run, allowing practitioners to modulate the performance on each task during inference on the fly.

Auto- λ : Disentangling Dynamic Task Relationships

This work proposes an automated weighting framework, named Auto- λ, that explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training.

Auxiliary Learning as an Asymmetric Bargaining Game

This work proposes a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power and describes ancient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoret-ical guarantees for its convergence.

Improving Multi-task Learning via Seeking Task-based Flat Regions

A novel MTL training methodology is presented, encouraging the model to cooperate with task-based minima for coherently improving its generalization capability on all tasks, which can enhance model generalization ability on single-task learning.

ForkMerge: Overcoming Negative Transfer in Multi-Task Learning

ForkMerge is introduced, which periodically forks the model into multiple branches with different task weights, and merges dynamically to carry out detrimental parameter updates to avoid negative transfer.

Efficient Trust Region-Based Safe Reinforcement Learning with Low-Bias Distributional Actor-Critic

A safe distributional RL method based on the trust region method, which can satisfy constraints consistently and shows minimal constraint violations while achiev-ing high returns compared to existing safe RL methods.

Auto-Lambda: Disentangling Dynamic Task Relationships

This work proposes a gradient-based meta learning framework, Auto- λ, which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formula-tion of a meta-loss; where the validation loss automatically influences task weightings throughout training.

References

SHOWING 1-10 OF 61 REFERENCES

Multi-Task Reinforcement Learning with Soft Modularization

This work introduces an explicit modularization technique on policy representation to alleviate the optimization issue of training multiple tasks with unclear parameters, and designs a routing network which estimates different routing strategies to reconfigure the base network for each task.

Which Tasks Should Be Learned Together in Multi-task Learning?

This framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.

Gradient Surgery for Multi-Task Learning

This work identifies a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develops a simple yet general approach for avoiding such interference between task gradients.

Rotograd: Dynamic Gradient Homogenization for Multi-Task Learning

This paper proposes a novel approach for MTL, Rotograd, which homogenizes the gradient directions across all tasks by rotating their shared representation, formalized as a Stackelberg game, which allows it to provide stability guarantees.

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

An open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic manipulation tasks is proposed to make it possible to develop algorithms that generalize to accelerate the acquisition of entirely new, held-out tasks.

Multi-Task Reinforcement Learning with Context-based Representations

This work posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks, and proposes a framework for using metadata to help to learn interpretable representations and provide the context to inform which representations to compose and how to compose them.

Multi-Task Learning with Deep Neural Networks: A Survey

An overview of multi-task learning methods for deep neural networks is given, with the aim of summarizing both the well-established and most recent directions within the field.

Fair Exploration via Axiomatic Bargaining

This work derives policies that yield the Nash bargaining solution relative to the set of incremental utilities possible under any policy, and shows that on the one hand, the ‘price of fairness’ under such policies is limited, while on the other hand, regret optimal policies are arbitrarily unfair under generic conditions.

Multitask Learning

Prior work on MTL is reviewed, new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals is presented, and new results for MTL with k-nearest neighbor and kernel regression are presented.

A Cooperation Strategy Based on Nash Bargaining Solution in Cooperative Relay Networks

Simulation results demonstrate that when cooperation takes place, users benefit from the proposed strategy in terms of utility, and those with longer distance to the AP should spend more bandwidth to cooperate with others.
...