Collective Intelligence, Data Routing and Braess' Paradox

  title={Collective Intelligence, Data Routing and Braess' Paradox},
  author={David H. Wolpert and Kagan Tumer},
We consider the problem of designing the the utility functions of the utility-maximizing agents in a multi-agent system so that they work synergistically to maximize a global utility. The particular problem domain we explore is the control of network routing by placing agents on all the routers in the network. Conventional approaches to this task have the agents all use the Ideal Shortest Path routing Algorithm (ISPA). We demonstrate that in many cases, due to the side-effects of one agent's… 

Exploring Collective Intelligence for Network Routing

This paper introduces COllective INtelligence (COIN) as a new approach to address the challenges of complex systems and demonstrates the potential of the COIN theory by investigating a packet routing problem.

Modelling Cooperation in Network Games with Spatio-Temporal Complexity

This work uses multi-agent deep reinforcement learning to simulate an agent society for a variety of plausible mechanisms, finding clear transitions between different equilibria over time and defines analytic tools inspired by related literatures to measure the social outcomes, and uses these to draw conclusions about the efficacy of different environmental interventions.

Theory of Collective Intelligence

In this chapter an analysis of the behavior of an arbitrary (perhaps massive) collective of computational processes in terms of an associated “world” utility function is presented We concentrate on

On Braess's paradox and routing algorithms

This paper shows the novel issue of how the addition of resources can lead to a decrease in data network performance while using certain routing algorithm (ie, shortest widest path—used for quality‐of‐service routing, quickest path, geographic routing, quality‐ of‐service energy‐aware routing), without even considering congestion.

Advances in Distributed Optimization Using Probability Collectives

A set of new techniques for updating mixed strategies of agents in an iterated game are presented and an extension of the approach to include (in)equality constraints over the underlying variables is presented.

Partitioning agents and shaping their evaluation functions in air traffic problems with hard constraints

This work combines the noise reduction of fitness function shaping, the robustness of cooperative coevolutionary algorithms, and agent partitioning to perform hard constraint optimization on the congestion and reduce the delay throughout the National Air Space (NAS).

Learning in Unknown Reward Games: Application to Sensor Networks

This paper demonstrates a decentralised method for optimisation using game-theoretic multi-agent techniques, applied to a sensor network management problem and compares six algorithms across a variety of parameter settings to ascertain the quality of the solutions they produce, their speed of convergence, and their robustness to pre-specified parameter choices.

Coordinated work of large collections of agents using COllective INtelligence

A hierarchical model is defined which expand the conventional COIN framework to the heterogeneous multi-agent system and a concrete world utility is constructed to solve the rapid self-response of agents.

Learning System-Efficient Equilibria in Route Choice Using Tolls

A toll-based Q-learning algorithm that leads agents to behave in a socially-desirable way so that the system optimum (SO) is attainable and theoretical results show that this approach converges to a system-efficient equilibrium in the limit.


Many systems of self-interested agents have an associated performance criterion that rates the dynamic behavior of the overall system. This chapter presents an introduction to the science of such



Collective Intelligence and Braess' Paradox

A practical machine-learning-based version of this algorithm, in which costs are only imprecisely estimated substantially outperforms theISPA, despite having access to less information than does the ISPA.

Adaptivity in agent-based routing for data networks

It is demonstrated on the OPNET event-driven network simulator the perhaps surprising fact that simply changing the action space of the agents to be better suited to RL can result in very large improvements in their potential performance: at their best settings, the learning-amenable router agents achieve throughputs up to three and one half times better than that of the standard Bellman-Ford routing algorithm.

An Introduction to Collective Intelligence

This paper surveys the emerging science of how to design a “COllective INtelligence” (COIN). A COIN is a large multi-agent system where: i) There is little to no centralized communication or control.

Architecting noncooperative networks

It is shown that, for systems of parallel links, such paradoxes cannot occur and the optimal solution coincides with the solution in the single-user case, and some extensions to general network topologies are derived.

Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks

Two new distributed routing algorithms for data networks based on simple biological "ants" that explore the network and rapidly learn good routes, using a novel variation of reinforcement learning are investigated, and they scale well with increase in network size-using a realistic topology.

Achieving network optima using Stackelberg routing strategies

The existence of maximally efficient strategies for the manager, i.e., strategies that drive the system into the global network optimum, is investigated and it is shown that they are met in many cases of practical interest.

Optimal Payoff Functions for Members of Collectives

It is demonstrated experimentally that using these new utility functions can result in significantly improved performance over that of previously investigated COIN payoff utilities, over and above those previous utilities' superiority to the conventional team game utility.

Routing in Telecommunications Networks with “ Smart ” Ant-Like Agents

A simple mechanism is presented, based on ant-like agents, for routing and load balancing in telecommunications networks, supplemented with a simplified dynamic programming capability, initially experimented by Guérin (1997) with more complex agents, which is shown to significantly improve the network's relaxation and its response to perturbations.

Capacity allocation under noncooperative routing

It is shown that such paradoxes do not occur and that the capacity allocation problem has a simple and intuitive optimal solution that coincides with the solution in the single-user case.

Anytime Coalition Structure Generation with Worst Case Guarantees

This work presents an algorithm that establishes a tight bound within this minimal amount of search, and shows how to distribute the desired search across self-interested manipulative agents.