Social Network Structure Shapes Innovation: Experience-sharing in RL with SAPIENS

@article{Nisioti2022SocialNS,
  title={Social Network Structure Shapes Innovation: Experience-sharing in RL with SAPIENS},
  author={Eleni Nisioti and Mateo Mahaut and Pierre-Yves Oudeyer and Ida Momennejad and Cl{\'e}ment Moulin-Frier},
  journal={ArXiv},
  year={2022},
  volume={abs/2206.05060}
}
The human cultural repertoire relies on innovation: our ability to continuously and hierarchically explore how existing elements can be combined to create new ones. Innovation is not solitary, it relies on collective accumulation and merging of previous solutions. Machine learning approaches commonly assume that fully connected multi-agent networks are best suited for innovation. However, human laboratory and field studies have shown that hierarchical innovation is more robustly achieved by… 

References

SHOWING 1-10 OF 26 REFERENCES

The Network Structure of Exploration and Exploitation

An agent-based computer simulation model of information sharing in which the less successful emulate the more successful suggests that when agents are dealing with a complex problem, the more efficient the network at disseminating information, the better the short-run but the lower the long-run performance of the system.

Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research

The hypothesis that multi-agent systems sometimes display intrinsic dynamics arising from competition and cooperation that provide a naturally emergent curriculum, which is termed an autocurriculum, is explored.

Collective minds: social network topology shapes collective cognition

  • I. Momennejad
  • Computer Science
    Philosophical Transactions of the Royal Society B
  • 2021
Research that combines network structure with psychological and neural experiments and modelling to understand how the topology of social networks shapes collective cognition is reviewed, showing that different topologies of communication networks synchronize or integrate knowledge differently, serving diverse collective goals.

Bridge ties bind collective memories

The authors show that when early conversations involve individuals that bridge across clusters of a social network, the network reaches higher mnemonic convergence compared to whenEarly conversations occur within clusters.

Mnemonic convergence in social networks: The emergent properties of cognition at a collective level

The social-interactionist approach proposed herein points to optimal strategies for spreading information in social networks and provides a framework for measuring and forging collective memories in communities of individuals.

Collaborative learning in networks

This paper reports on a series of 256 Web-based experiments in which groups of 16 individuals collectively solved a complex problem and shared information through different communication networks, finding that collective exploration improved average success over independent exploration because good solutions could diffuse through the network.

Experience Sharing Between Cooperative Reinforcement Learning Agents

While sharing randomly selected experiences between two Deep Q-Network agents shows no improvement over a single agent baseline, it is shown that the proposed ES methods can successfully outperform the baseline and accelerates learning by a factor of 2.

Partial connectivity increases cultural accumulation within groups

An experiment demonstrating that partially connected groups produce more diverse and complex cultural traits than fully connected groups is presented, which suggests that changes in patterns of interaction between human groups may have created propitious conditions for the emergence of complex cultural repertoires in the authors' evolutionary past.

The evolutionary ecology of technological innovations

Major distinctions are illustrated by current specific examples, including the evolution of cornets and the historical dynamics of information technologies, which provide examples of planned design that have no equivalent with natural evolution.

Off-Policy Actor-Critic with Shared Experience Replay

This work analyzes the bias-variance tradeoffs in V- Trace, a form of importance sampling for actor-critic methods, and argues for mixing experience sampled from replay with on-policy experience, and proposes a new trust region scheme that scales effectively to data distributions where V-trace becomes unstable.