The ThreadMill architecture for stream-oriented human communication analysis applications

ABSTRACT

This work introduces a new component software architecture - ThreadMill - whose main purpose is to facilitate the development of applications in domains where high volumes of streamed data need to be efficiently analyzed. It focuses particularly on applications that target the analysis of human communication e.g. in speech and gesture recognition. Applications in this domain usually employ costly signal processing techniques, but offer in many cases ample opportunities for concurrent execution in many different phases. ThreadMill's abstractions facilitate the development of applications that take advantage of this potential concurrency by hiding the complexity of parallel and distributed programming. As a result, ThreadMill applications can be made to run unchanged on a wide variety of execution environments, ranging from a single-processor machine to a cluster of multi-processor nodes. The architecture is illustrated by an implementation of a tracker for hands and face of American Sign Language signers that uses a parallel and concurrent version of the Joint Likelihood Filter method.

REFERENCES

• 1: Gul Agha, Actors: a model of concurrent computation in distributed systems, MIT Press, Cambridge, MA, 1986
• 2: Daisaku Arita , Rin-ichiro Taniguchi, RPV-II: A Stream-Based Real-Time Parallel Vision System and Its Application to Real-Time Volume Reconstruction, Proceedings of the Second International Workshop on Computer Vision Systems, p.174-189, July 07-08, 2001
• 3: Paulo Barthelmess , Clarence A. Ellis, Threadmill: a highly configurable architecture for human communication analysis applications, 2003
• 4: Adam Cheyer , David Martin, The Open Agent Architecture, Autonomous Agents and Multi-Agent Systems, v.4 n.1-2, p.143-148, March-June 2001 [doi>10.1023/A:1010091302035]
• 5: Greg Eisenhauer , Fabián E. Bustamante , Karsten Schwan, Native Data Representation: An Efficient Wire Format for High-Performance Distributed Computing, IEEE Transactions on Parallel and Distributed Systems, v.13 n.12, p.1234-1246, December 2002 [doi>10.1109/TPDS.2002.1158262]
• 6: G. Fink, N. Jungclaus, F. Kummert, H. Ritter, and G. Sagerer. A distributed system for integrated speech and image understanding. In International Symposium on Artificial Intelligence, pages 117--126, Cancun, Mexico, 1996.
• 7: R. J. Firby, R. E. Kahn, P. N. Prokopopowitz, and M. J. Swain. An architecture for vision and action. In C. Mellish, editor, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 72--79, San Francisco, 1995. Morgan Kaufmann.
• 8: M. P. I. Forum. MPI: A Message-Passing Interface Standard, 1994.
• 9: Alexandre R. J. François , Gérard G. Medioni, A Modular Software Architecture for Real-Time Video Processing, Proceedings of the Second International Workshop on Computer Vision Systems, p.35-49, July 07-08, 2001
• 10: Erich Gamma , Richard Helm , Ralph Johnson , John Vlissides, Design patterns: elements of reusable object-oriented software, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 11: Michael M. Gorlick , Rami R. Razouk, Using weaves for software construction and analysis, Proceedings of the 13th international conference on Software engineering, p.23-34, May 13-17, 1991, Austin, Texas, United States
• 12: Roger Ewing Kahn , Michael J. Swain, Perseus: an extensible vision system for human-machine interaction, 1996
• 13: Attila Gürsoy , Laxmikant V. Kale, Performance and modularity benefits of message-driven execution, Journal of Parallel and Distributed Computing, v.64 n.4, p.461-480, April 2004 [doi>10.1016/j.jpdc.2004.03.006]
• 14: L. V. Kale. The virtualization approach to parallel programming: Runtime optimizations and the state of the art. In Los Alamos Computer Science Institute Symposium - LACSI 2002, Albuquerque, 2002. "State of the field" paper.
• 15: Markus Kohler , Sven Schröter , Heinrich Müller, The ARGUS-Architecture for Global Computer Vision-Based Interaction and its Application in Domestic Enviroments, Proceedings of HCI International (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I, p.296-300, August 22-26, 1999
• 16: Mitre Corporation. Galaxy Communicator Documentation, 2002. Available on the web at http://communicator.sourceforge.net/sites/MITRE/distributions/GalaxyCommunicator/docs/manual/index.html.
• 17: Peyman Oreizy , Michael M. Gorlick , Richard N. Taylor , Dennis Heimbigner , Gregory Johnson , Nenad Medvidovic , Alex Quilici , David S. Rosenblum , Alexander L. Wolf, An Architecture-Based Approach to Self-Adaptive Software, IEEE Intelligent Systems, v.14 n.3, p.54-62, May 1999 [doi>10.1109/5254.769885]
• 18: U. Ramachandran, R. Nikhil, J. M. Rehg, Y. Angelov, A. Paul, S. Adhikari, K. Mackenzie, N. Harel, and K. Knobe. Stampede: A cluster programming middleware for interactive stream-oriented applications. IEEE Transactions on Parallel and Distributed Systems, pages 1140--1154, November 2003.
• 19: Christopher Eric Rasmussen , Gregory D. Hager, Integrating multiple visual cues for robust tracking, 2000

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
        D.2.11 Software Architectures
            Subjects: Domain-specific architectures

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming; Distributed programming
  D.2 SOFTWARE ENGINEERING
        D.2.11 Software Architectures
            Subjects: Languages (e.g., description, interconnection, definition)
      D.2.2 Design Tools and Techniques
          Subjects: Modules and interfaces

General Terms:
Design, Languages, Performance

Keywords:
human-communication analysis applications, software evolution