• Corpus ID: 44076151

A Causal Model Approach to Dynamic Control

  title={A Causal Model Approach to Dynamic Control},
  author={Zachary Davis and Neil R. Bramley and Bob Rehder and Todd M. Gureckis},
  journal={Cognitive Science},
Acting effectively in the world requires learning and controlling dynamic systems, that is, systems involving feedback relations among continuous variables that vary in real time. We introduce a novel class of dynamic control environments using Ornstein-Uhlenbeck processes connected in causal Markov graphs that allow us to systematically test people’s ability to learn and control various dynamic systems. We find that performance varied across a range of test environments, roughly matching with… 

Figures from this paper

Causal Structure Learning in Continuous Systems
This work develops a new framework that represents a causal system as a network of stationary Gauss–Markov (“Ornstein–Uhlenbeck”) processes and shows how such OU networks can express complex dynamic phenomena, such as feedback loops and oscillations.
Human dynamic control under changing goals
This project examines human goal flexibility when interacting with dynamic environments, showing how despite being incredibly adaptive, people are in fact computationally frugal, minimizing the complexity of their representations and decision policies even in situations that might warrant richer ones.
1 Intervening in Time
This chapter traces how causal inference tools developed in data science have been applied to understanding human causal learning and reasoning, highlights the current shortcomings of both intervention-based and time-based approaches taken separately, and describes recent work that starts to bring the two together.


A Control Theoretic Model of Adaptive Learning in Dynamic Environments
These experiments provide preliminary evidence that adaptive learning in dynamic environments resembles PID control, and examined how participants weighted different PID terms in response to salient environmental events, finding that these control terms were modulated by reward, surprise, and outcome entropy.
Causal Structure Learning with Continuous Variables in Continuous Time
A generative model and framework for causal inference over continuous variables in continuous time based on Ornstein-Uhlenbeck processes is presented, finding that when interacting with systems acting according to this formalism people directly compare relationships between individual variable pairs rather than considering the full space of possible models.
Spontaneous Causal Learning While Controlling A Dynamic System
When dealing with a dynamic causal system people may employ a variety of different strategies. One of these strategies is causal learning, that is, learning about the causal structure and parameters
Strategic exploration in human adaptive control
This work studies a novel control task in which participants must steer a boat on a grid, assessing whether participants explore strategically in order to produce higher rewards later on.
Learning in Noise: Dynamic Decision-Making in a Variable Environment.
Human-level control through deep reinforcement learning
This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Reinforcement Learning: An Introduction
This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Controlling uncertainty: a review of human behavior in complex dynamic environments.
  • M. Osman
  • Business
    Psychological bulletin
  • 2010
The main thesis of the Monitoring and control framework is that CDC tasks are characteristically uncertain environments, and subjective judgments of uncertainty guide the way in which monitoring and control behaviors attempt to reduce it.
Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model
It is found that many human performance phenomena, measured through novel behavioral experiments, are naturally produced by the operation of the authors' ideal observer model (a Rao-Blackwelized particle filter).
Micro-worlds and the circular relation between people and their environment
Although early studies did not succeed in finding relations between micro-world performance and psychometric variables, probably because the experiments did not yield reliable data, one now knows how to obtain such data and stable correlations between standard measures of intelligence and micro- world perfor...