Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study

  title={Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study},
  author={Tianye Shu and Jialin Liu and Georgios N. Yannakakis},
  journal={2021 IEEE Conference on Games (CoG)},
We introduce a procedural content generation (PCG) framework at the intersections of experience-driven PCG and PCG via reinforcement learning, named ED(PCG)RL, EDRL in short. EDRL is able to teach RL designers to generate endless playable levels in an online manner while respecting particular experiences for the player as designed in the form of reward functions. The framework is tested initially in the Super Mario Bros game. In particular, the RL designers of Super Mario Bros generate and… 

Figures and Tables from this paper

Online Game Level Generation from Music

Results of simulation-based experiments show that the implementation of OPARL is competent to generate playable levels withulty degree matched to the “energy” dynamic of music for different artificial players in an online fashion.

Learning to Generate Levels by Imitating Evolution

This work introduces a new type of iterative level generator using machine learning that is able to modify noisy levels sequentially to create better levels without the need for a fitness function during inference.

Mutation Models: Learning to Generate Levels by Imitating Evolution

Mutation models are introduced, a new type of iterative level generator based on machine learning that is able to modify noisy levels sequentially to create better levels without the need for a fitness function during inference.

Persona-driven Dominant/Submissive Map (PDSM) Generation for Tutorials

It is shown that the generated maps can strongly encourage or discourage different persona-like behaviors and range from simple solutions to complex puzzle-levels, making them perfect candidates for a tutorial generative system.

Generative Personas That Behave and Experience Like Humans

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of

Towards Objective Metrics for Procedurally Generated Video Game Levels

Two simulation-based evaluation metrics that involve analysing the behaviour of an A* agent to measure the diversity and difficulty of generated levels in a general, gameindependent manner are introduced.

Go-Blend Behavior and Affect

The proposed framework introduces a new paradigm shift for affect modeling by viewing the affect modeling task as a reinforcement learning process and empowers believable AI-based game testing by providing agents that can blend and express a multitude of behavioral and affective patterns.

Generating Game Levels of Diverse Behaviour Engagement

This work questions whether experience metrics can adapt to agents with different personas, and designs a framework integrating various agents and evaluation metrics for evaluating game levels on platformer games.

Adapting Procedural Content Generation to Player Personas Through Evolution

This paper proposes an architecture using persona agents and experience metrics, which enables evolving procedurally generated levels tailored for particular player personas, and demonstrates that this approach successfully adapts to four rule-based persona agents over three different experience metrics.

Procedural content generation using neuroevolution and novelty search for diverse video game levels

This work introduces an evolutionary search-based approach for evolving level generators using novelty search to procedurally generate diverse levels in real time, without requiring training data or detailed domain-specific knowledge.



PCGRL: Procedural Content Generation via Reinforcement Learning

This work investigates three different ways of transforming two-dimensional level design problems into Markov decision processes and applies these to three game environments.

Online level generation in Super Mario Bros via learning constructive primitives

  • Peizhi ShiKe Chen
  • Computer Science
    2016 IEEE Conference on Computational Intelligence and Games (CIG)
  • 2016
This paper exploits the synergy between rule-based and learning-based methods to produce quality yet controllable game segments in Super Mario Bros, hereinafter named constructive primitives (CPs), and efficiently generates controLLable yet quality game levels in terms of different quality measures.

Increasing generality in machine learning through procedural content generation

Existing work on PCG, its overlap with current efforts in ML, and promising new research directions such as procedurally generated learning environments are reviewed, including how PCG may be crucial for training agents which generalise well.

Learning Constructive Primitives for Real-Time Dynamic Difficulty Adjustment in Super Mario Bros

This work proposes a novel approach to seamlessly address two issues in Super Mario Bros (SMB), to address the quality assurance issue and the synergy between rule-based and learning-based methods to produce quality game segments in SMB, named constructive primitives (CPs).

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

A novel test-bed platform for reinforcement learning research from raw visual information which employs the first-person perspective in a semi-realistic 3D world and confirms the utility of ViZDoom as an AI research platform and implies that visual reinforcement learning in 3D realistic first- person perspective environments is feasible.

Adversarial Reinforcement Learning for Procedural Content Generation

A new approach ARLPCG: Adversarial Reinforcement Learning for Procedural Content Generation is presented, which procedurally generates and tests previously unseen environments with an auxiliary input as a control variable to drive diversity and control of the environment generation.

The Arcade Learning Environment: An Evaluation Platform for General Agents (Extended Abstract)

The promise of ALE is illustrated by developing and benchmarking domain-independent agents designed using well-established AI techniques for both reinforcement learning and planning, and an evaluation methodology made possible by ALE is proposed.

Procedural Content Generation via Machine Learning (PCGML)

This paper addresses the relatively new paradigm of using machine learning (in contrast with search-based, solver- based, and constructive methods), and focuses on what is most often considered functional game content, such as platformer levels, game maps, interactive fiction stories, and cards in collectible card games.

Evolving mario levels in the latent space of a deep convolutional generative adversarial network

This paper trains a GAN to generate levels for Super Mario Bros using a level from the Video Game Level Corpus, and uses the champion A* agent from the 2009 Mario AI competition to assess whether a level is playable, and how many jumping actions are required to beat it.

Player-adaptive Spelunky level generation

This work shows how personalized, difficulty-adjusted levels can be generated for the more complex 2D platformer Spelunky using a rather generic rule set that may also be transferred to similar games.