Introducing Communication in Dis-POMDPs with Finite State Machines

Abstract

Distributed Partially Observable Markov Decision Problems (Dis-POMDPs) are emerging as a popular approach for modeling sequential decision making in teams operating under uncertainty. To achieve coherent behaviors of agents, it is essential to perform appropriate run-time communication. Thus, there have been many works on the run-time communication schemes in Dis-POMDPs. Also, a Finite State Machine (FSM) is a popular representation for describing a local policy that works in a very long or infinite time horizon. In this paper, we examine a run-time communication scheme when the local policy of each agent is represented as an FSM. In this scheme, the meaning of each message is not predefined; it is given implicitly by the interaction between local policies. We propose an iterative-improvement type algorithm that searches for a joint policy where run-time communication incurs some cost. Thus, agents use runtime communication only when doing so is cost-effective. Interestingly, our algorithm can find a joint policy that obtains a better expected reward than a hand-crafted joint policy, and it requires fewer nodes in the local FSM and fewer message types. Furthermore, we experimentally show that our algorithm can obtain a joint policy that consists of sufficiently complex local FSMs within a reasonable amount of time.

DOI: 10.1109/WI-IAT.2009.161

Extracted Key Phrases

4 Figures and Tables

Cite this paper

@article{Iwanari2009IntroducingCI, title={Introducing Communication in Dis-POMDPs with Finite State Machines}, author={Yuki Iwanari and Makoto Tasaki and Makoto Yokoo and Atsushi Iwasaki and Yuko Sakurai}, journal={2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology}, year={2009}, volume={2}, pages={267-270} }