A survey of random processes with reinforcement

@article{Pemantle2007ASO,
  title={A survey of random processes with reinforcement},
  author={Robin Pemantle},
  journal={Probability Surveys},
  year={2007},
  volume={4},
  pages={1-79}
}
  • R. Pemantle
  • Published 2 October 2006
  • Mathematics
  • Probability Surveys
The models surveyed include generalized Polya urns, reinforced random walks, interacting urn models, and continuous reinforced processes. Emphasis is on methods and results, with sketches provided of some proofs. Applications are discussed in statistics, biology, economics and a number of other areas. 
View PDF on arXiv
460 Citations
Highly Influential Citations
28
Background Citations
194
Methods Citations
67
Results Citations
5

Figures from this paper

460 Citations

Citation Type

Citation Type

PR ] 2 O ct 2 00 6 A survey of random processes with reinforcement
The models surveyed include generalized Pólya urns, reinforced random walks, interacting urn models, and continuous reinforced processes. Emphasis is on methods and results, with sketches provided of…
On some generalized reinforced random walk on integers
We consider Reinforced Random Walks where transitions probabilities are a function of the proportions of times the walk has traversed an edge. We give conditions for recurrence or transience. A phase…
Simulated Annealing, Vertex-Reinforced Random Walks and Learning in Games
This paper studies a class of non Markovian and non homogeneous stochastic processes on a finite state space. It provides an unified approach to simulated annealing type processes, certain vertex…
The speed of a general random walk reinforced by its recent history
A class of non homogeneous self interacting random processes with applications to learning in games and vertex-reinforced random walks
Using an approximation by a set-valued dynamical system, this paper studies a class of non Markovian and non homogeneous stochastic processes on a finite state space. It provides an unified approach…
Synchronization via Interacting Reinforcement
TLDR
It is shown that the urns synchronizealmost surely, in the sense that the fraction of balls of a given color converges almost surely, as the time goes to infinity, to the same limit for all urneds.
A Class of Self-Interacting Processes with Applications to Games and Reinforced Random Walks
TLDR
It is shown that, under certain assumptions, the asymptotic behavior of occupation measures can be described in terms of a certain set-valued deterministic dynamical system, which provides a unified approach to simulated annealing type processes and permits the study of new models of vertex reinforced random walks andnew models of learning in games such as Markovian fictitious play.
Generalized Interacting Urn Models
Interacting urns with exponential reinforcement were introduced and studied in Launay (2011). As its parametertends to 1 , this reinforcement mechanism converges to the "generalized" reinforcement,…
A balanced excited random walk
Recurrence for vertex-reinforced random walks on Z with weak reinforcements.
We prove that any vertex-reinforced random walk on the integer lattice with non-decreasing reinforcement sequence $w$ satisfying $w(k) = o(k^{\alpha})$ for some $\alpha <1/2$ is recurrent. This…
...
...

References

SHOWING 1-10 OF 240 REFERENCES
Urn schemes and reinforced random walks
Polya-Type Urn Models with Multiple Drawings
We investigate the distribution, mean value, variance and some limiting properties of an urn model of white and red balls under random multiple drawing (either with or without replacement) when the…
Limit theorems for reinforced random walks on certain trees
Consider a linearly edge-reinforced random walk defined on the b-ary tree, b≥70. We prove the strong law of large numbers for the distance of this process from the root. We give a sufficient…
A time-dependent version of Pólya's urn
A process is defined that consists of drawing balls from an urn and replacing them along with extra balls. The process generalizes the well-knownPólya urn process. It is shown that the proportion of…
Attracting edge property for a class of reinforced random walks
Using martingale techniques and comparison with the generalized Urn scheme, it is shown that the edge reinforced random walk on a graph of bounded degree, with the weight function W(k)=kρ,ρ>1,…
A Once edge-reinforced random walk on a Galton–Watson tree is transient
A System of Reaction Diffusion Equations Arising in the Theory of Reinforced Random Walks
TLDR
It is shown that under some circumstances, finite-time blow-up of solutions is possible and in other circumstances, the solutions will decay to a spatially constant solution (collapse).
Continuous time vertex-reinforced jump processes
Abstract. We study the continuous time integer valued process , which jumps to each of its two nearest neighbors at the rate of one plus the total time the process has previously spent at that…
A random environment for linearly edge-reinforced random walks on infinite graphs
We consider linearly edge-reinforced random walk on an arbitrary locally finite connected graph. It is shown that the process has the same distribution as a mixture of reversible Markov chains,…
An urn model of Diaconis
SUMMARY An urn model of Diaconis and some generalizations are discussed. A convergence theorem is proved that implies for Diaconis’ model that the empirical distribution of balls in the urn converges…
...
...