Graphical models for online solutions to interactive POMDPs

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Graphical models for online solutions to interactive POMDPs

Full text

Pdf (394 KB)

Source

International Conference on Autonomous Agents archive
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems table of contents

Honolulu, Hawaii

SESSION: Multiagent planning: full papers table of contents

Article No. 217

Year of Publication: 2007

ISBN:978-81-904262-7-5

Authors

Prashant Doshi	University of Georgia, Athens, GA
Yifeng Zeng	Aalborg University, Aalborg, Denmark
Qiongyu Chen	National Univ. of Singapore, Singapore

Sponsor

: IFAAMAS

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 10, Downloads (12 Months): 52, Citation Count: 2

Additional Information:

abstract references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1329125.1329387 What is a DOI?

ABSTRACT

We develop a new graphical representation for interactive partially observable Markov decision processes (I-POMDPs) that is significantly more transparent and semantically clear than the previous representation. These graphical models called interactive dynamic influence diagrams (I-DIDs) seek to explicitly model the structure that is often present in real-world problems by decomposing the situation into chance and decision variables, and the dependencies between the variables. I-DIDs generalize DIDs, which may be viewed as graphical representations of POMDPs, to multiagent settings in the same way that I-POMDPs generalize POMDPs. I-DIDs may be used to compute the policy of an agent online as the agent acts and observes in a setting that is populated by other interacting agents. Using several examples, we show how I-DIDs may be applied and demonstrate their usefulness.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	R. J. Aumann. Interactive epistemology i: Knowledge. International Journal of Game Theory, 28:263--300, 1999.
	2	Darse Billings , Aaron Davidson , Jonathan Schaeffer , Duane Szafron, The challenge of poker, Artificial Intelligence, v.134 n.1-2, p.201-240, January 2002 [doi>10.1016/S0004-3702(01)00130-8]
	3	A. Brandenburger and E. Dekel. Hierarchies of beliefs and common knowledge. Journal of Economic Theory, 59:189--198, 1993.
	4	C. Camerer. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, 2003.
	5	E. Fehr and S. Gachter. Cooperation and punishment in public goods experiments. American Economic Review, 90(4):980--994, 2000.
	6	D. Fudenberg and D. K. Levine. The Theory of Learning in Games. MIT Press, 1998.
	7	D. Fudenberg and J. Tirole. Game Theory. MIT Press, 1991.
	8	Ya'akov Gal , Avi Pfeffer, A language for modeling agents' decision making processes in games, Proceedings of the second international joint conference on Autonomous agents and multiagent systems, July 14-18, 2003, Melbourne, Australia [doi>10.1145/860575.860618]
	9	P. Gmytrasiewicz and P. Doshi. A framework for sequential planning in multiagent settings. JAIR, 24:49--79, 2005.
	10	Piotr J. Gmytrasiewicz , Edmund H. Durfee, Rational Coordination in Multi-Agent Environments, Autonomous Agents and Multi-Agent Systems, v.3 n.4, p.319-350, December 2000 [doi>10.1023/A:1010028119149]
	11	John C. Harsanyi, Games with Incomplete Information Played by "Bayesian" Players, I-III, Management Science, v.50 n.12 Supplement, p.1804-1817, December 2004 [doi>10.1287/mnsc.1040.0270]
	12	Ronald A. Howard , James E. Matheson, Influence Diagrams, Decision Analysis, v.2 n.3, p.127-143, September 2005 [doi>10.1287/deca.1050.0020]
	13	Leslie Pack Kaelbling , Michael L. Littman , Anthony R. Cassandra, Planning and acting in partially observable stochastic domains, Artificial Intelligence, v.101 n.1-2, p.99-134, May 1998 [doi>10.1016/S0004-3702(98)00023-X]
	14	D. Koller and B. Milch. Multi-agent influence diagrams for representing and solving games. In IJCAI, pages 1027--1034, 2001.
	15	Kyle Polich , Piotr Gmytrasiewicz, Interactive dynamic influence diagrams, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, May 14-18, 2007, Honolulu, Hawaii [doi>10.1145/1329125.1329166]
	16	Bharanee Rathnasabapathy , Prashant Doshi , Piotr Gmytrasiewicz, Exact solutions of interactive POMDPs using behavioral equivalence, Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, May 08-12, 2006, Hakodate, Japan [doi>10.1145/1160633.1160816]
	17	Stuart J. Russell , Peter Norvig, Artificial Intelligence: A Modern Approach, Pearson Education, 2003
	18	R. D. Shachter, Evaluating influence diagrams, Operations Research, v.34 n.6, p.871-882, Nov.-Dec. 1986 [doi>10.1287/opre.34.6.871]
	19	Dicky Suryadi , Piotr J. Gmytrasiewicz, Learning models of other agents using influence diagrams, Proceedings of the seventh international conference on User modeling, p.223-232, June 1999, Banff, Canada

CITED BY 2

	Yifeng Zeng , Prashant Doshi , Qiongyu Chen, Approximate solutions of interactive dynamic influence diagrams using model clustering, Proceedings of the 22nd national conference on Artificial intelligence, p.782-787, July 22-26, 2007, Vancouver, British Columbia, Canada
	Prashant Doshi , Piotr J. Gmytrasiewicz, Monte Carlo sampling methods for approximating interactive POMDPs, Journal of Artificial Intelligence Research, v.34 n.1, p.297-337, January 2009