Improving adjustable autonomy strategies for time-critical domains

ABSTRACT

As agents begin to perform complex tasks alongside humans as collaborative teammates, it becomes crucial that the resulting human-multiagent teams adapt to time-critical domains. In such domains, adjustable autonomy has proven useful by allowing for a dynamic transfer of control of decision making between human and agents. However, existing adjustable autonomy algorithms commonly discretize time, which not only results in high algorithm runtimes but also translates into inaccurate transfer of control policies. In addition, existing techniques fail to address decision making inconsistencies often encountered in human multiagent decision making. To address these limitations, we present novel approach for Resolving Inconsistencies in Adjustable Autonomy in Continuous Time (RIAACT) that makes three contributions: First, we apply continuous time planning paradigm to adjustable autonomy, resulting in high-accuracy transfer of control policies. Second, our new adjustable autonomy framework both models and plans for the resolving of inconsistencies between human and agent decisions. Third, we introduce a new model, Interruptible Action Time-dependent Markov Decision Problem (IA-TMDP), which allows for actions to be interrupted at any point in continuous time. We show how to solve IA-TMDPs efficiently and leverage them to plan for the resolving of inconsistencies in RIAACT. Furthermore, these contributions have been realized and evaluated in a complex disaster response simulation system.

REFERENCES

• 1: J. Boyan and M. Littman. Exact solutions to time-dependent MDPs. In NIPS, pages 1026--1032, 2000.
• 2: CALO http://www.ai.sri.com/project/CALO, http://calo.sri.com. CALO: Cognitive Agent that Learns and Organizes, 2003.
• 3: Philip R. Cohen , Hector J. Levesque, Intention is choice with commitment, Artificial Intelligence, v.42 n.2-3, p.213-261, March 1990 [doi>10.1016/0004-3702(90)90055-5]
• 4: Michael A. Goodrich , Timothy W. McLain , Jeffrey D. Anderson , Jisang Sun , Jacob W. Crandall, Managing autonomy in robot teams: observations from four experiments, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228721]
• 5: L. Li and M. Littman. Lazy approximation for solving continuous finite-horizon MDPs. In AAAI, pages 1175--1180, 2005.
• 6: J. Marecki, S. Koenig, and M. Tambe. A fast analytical algorithm for solving markov decision processes with real-valued resources. In IJCAI, January 2007.
• 7: R. Nair and M. Tambe. Hybrid bdi-pomdp framework for multiagent teaming. Journal of Artificial Intelligence Research (JAIR), 23:367--420, 2005.
• 8: Martin L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, Inc., New York, NY, 1994
• 9: P. Scerri, D. Pynadath, and M. Tambe. Towards adjustable autonomy for the real world. Journal of Artificial Intelligence Research, 17:171--228, 2002.
• 10: Nathan Schurr , Pratik Patil , Fred Pighin , Milind Tambe, Using multiagent teams to improve the training of incident commanders, Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, May 08-12, 2006, Hakodate, Japan [doi>10.1145/1160633.1160924]
• 11: B. P. Sellner, F. Heger, L. Hiatt, R. Simmons, and S. Singh. Coordinated multi-agent teams and sliding autonomy for large-scale assembly. Proceedings of the IEEE - Special Issue on Multi-Robot Systems, July 2006.
• 12: Pradeep Varakantham , Rajiv Maheswaran , Milind Tambe, Exploiting belief bounds: practical POMDPs for personal assistant agents, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, July 25-29, 2005, The Netherlands [doi>10.1145/1082473.1082621]
• 13: Håkan L. S. Younes , Reid G. Simmons, Solving generalized semi-Markov decision processes using continuous phase-type distributions, Proceedings of the 19th national conference on Artifical intelligence, p.742-747, July 25-29, 2004, San Jose, California

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Coherence and coordination

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
          Subjects: Decision support (e.g., MIS)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Multiagent systems
  I.6 SIMULATION AND MODELING
      I.6.5 Model Development

General Terms:
Design, Management, Theory