DANS: decentralized, autonomous, and networkwide service delivery and multimedia workflow processing

ABSTRACT

Fundamental challenges in designing environments with media-rich ambient services involves not only the development of appropriate sensing technologies, but as importantly, the implementation of a distributed media processing system which can process, integrate, and leverage the sensed data in real time to provide the various services. In recent years, a great deal of progress has been made in media service workflow processing systems. In most existing solutions, however, the workflow nodes, which operate on the data, are selected out of a centrally assigned candidate pool. These candidate organizations cause either extensive resource provisioning or poor-quality operator mapping between logical workflow nodes and the available physical resources nodes. Consequently, instantiating a media processing workflow to the underlying hardware before the workflow execution begins does not lends itself to adaptive and autonomous operation of the workflow, scalable to resources and demand.In this paper, we propose a novel decentralized multimedia workflow processing system, DANS, in which operators defined in workflows are mapped into (distributed) physical nodes through Distributed Hash Table (DHT)-based overlay substrate in a purely decentralized and adaptive manner. The redundancy in the system, in terms of availability of multiple nodes able to perform the same task, enables the system to scale with demand. Furthermore, physical workflow nodes (operator instances) are able to locate and select the next filter or fusion operator instance autonomously, while ensuring the correct execution of the workflow.

REFERENCES

• 1: M.S.Raisinghani, A.Benoit, J.Ding, M.Gomez, K.Gupta, V.Gusila, D.Power, and O.Schmedding. Ambient Intelligence: Changing Forms of Human-Computer Interaction and their Social Implications. Journal of Digital Information, Volume 5 Issue 4 Article No. 271, 2004.
• 2: Lina Peng , K. Selcuk Candan , Kyung D. Ryu , Karamvir S. Chatha , Hari Sundaram, ARIA: an adaptive and programmable media-flow architecture for interactive arts, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027657]
• 3: K. Selcuk Candan, Gisik Kwon, Lina Peng, and Maria Luisa Sapino. Modelling Adaptive Media Processing Workflows. IEEE International Conference on Multimedia and Expo (ICME), July, 2006.
• 4: X.Gu and K.Nahrstedt. Distributed Multimedia Service Composition with Statistical QoS Assurances. IEEE Transactions on Multimedia, 2005.
• 5: B.Raman and R.H.Katz. An Architecture for Highly Available Wide-Area Service Composition. Computer Communication, 26(15):1727--1740, September 2003.
• 6: B.Raman and R.H.Katz. Load Balancing and Stability Issues in Algorithms for Service Composition. IEEE INFOCOM, 2003.
• 7: D.Xu and K.Nahrstedt. Finding Services Paths in a Media Service Proxy Network. MMCN, 2002.
• 8: X.Fu, W.Shi, A.Akkerman, and V.Karamcheti. CANS: Composable, Adaptive Network Services Infrastructure. Proc. of USITS, March, 2001.
• 9: Andrew P. Black , Jie Huang , Rainer Koster , Jonathan Walpole , Calton Pu, Infopipes: an abstraction for multimedia streaming, Multimedia Systems, v.8 n.5, p.406-419, December 2002 [doi>10.1007/s00530-002-0062-3]
• 10: S.Chandrasekaran, O.Cooper, A.Deshpande, and M.Franklin. TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. The First Biennial Conf. on Innovative Data SystemsResearch (CIDR), Asilomar, CA, 2003.
• 11: M.Cherniack, H.Balakrishnan, M.Balazinska, D.Carney, U.Cetintemel, Y.Xing, and S.Zdonik. Scalable Distributed Stream Processing. The First Biennial Conference on Innovative Data Systems Research (CIDR), Asilomar, CA, 2003.
• 12: D.Abadi, Y.Ahmad, and H.Balakrishnan. The Design of the Borealis Stream Processing Engine. Technical Report CS-04-08, Brown University, 2004.
• 13: M.Balazinska, H.Balakrishnan, and M.Stonebraker. Contract-Based Load Management in Federated Distributed Systems. NSDI, San Francisco, CA, 2004.
• 14: Y.Ahmad and U.C.Cetintemel. Network-Aware Query Processing for Stream-based Applications. The 30th Int. Conf. on Very Large Data Bases (VLDB), Toronto, Canada, 2004.
• 15: R.Huebsch, J.M.Hellerstein, and N.Lanham. Querying the Internet with PIER. The 29th International Conference on Very Large Data Bases (VLDB), Berlin, Germany, 2003.
• 16: Michael Stonebraker , Paul M. Aoki , Witold Litwin , Avi Pfeffer , Adam Sah , Jeff Sidell , Carl Staelin , Andrew Yu, Mariposa: a wide-area distributed database system, The VLDB Journal — The International Journal on Very Large Data Bases, v.5 n.1, p.048-063, January 1996 [doi>10.1007/s007780050015]
• 17: Stratis D. Viglas , Jeffrey F. Naughton, Rate-based query optimization for streaming information sources, Proceedings of the 2002 ACM SIGMOD international conference on Management of data, June 03-06, 2002, Madison, Wisconsin [doi>10.1145/564691.564697]
• 18: M.A.Shah, J.M.Hellerstein, S.Chandrasekaran, and M.J.Franklin. Flux: An Adaptive Partitioning Operator for Continuous Query Systems. The 19th International Conference on Data Engineering (ICDE), Bangalore, India, 2003.
• 19: Yang-hua Chu , Sanjay G. Rao , Hui Zhang, A case for end system multicast (keynote address), Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.1-12, June 18-21, 2000, Santa Clara, California, United States
• 20: J.Jannotti, D.K.Gifford, and K.L.Johnson. Overcast: Reliable Multicasting with an Overlay Network. The 4th Symposium on Operating Systems Design and Implementation (OSDI), San Diego, CA, 2000.
• 21: M.Castro and M.Jones. An Evaluation of Scalable Application-Level Multicast using Peer-to-peer Overlay Networks. INFOCOM, San Francisco, CA, 2003.
• 22: P.Pietzuch, J.Shneidman, M.Roussopoulos, M.Seltzer, and M.Welsh. Path Optimization in Stream-Based Overlay Networks. Harvard University Technical Report TR-26-04, 2004.
• 23: Ion Stoica , Robert Morris , David Karger , M. Frans Kaashoek , Hari Balakrishnan, Chord: A scalable peer-to-peer lookup service for internet applications, Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, p.149-160, August 2001, San Diego, California, United States
• 24: Ben Y. Zhao , John D. Kubiatowicz , Anthony D. Joseph, Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and, University of California at Berkeley, Berkeley, CA, 2001
• 25: Antony I. T. Rowstron , Peter Druschel, Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems, Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, p.329-350, November 12-16, 2001
• 26: Sylvia Ratnasamy , Paul Francis , Mark Handley , Richard Karp , Scott Schenker, A scalable content-addressable network, Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, p.161-172, August 2001, San Diego, California, United States
• 27: D. Eastlake, 3rd , P. Jones, US Secure Hash Algorithm 1 (SHA1), RFC Editor, 2001
• 28: K.Calvert and E.Zegura. GT Internetwork Topology Models (GT-ITM). College of Computing,Georgia Institute of Technology, 1996.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Reliability, availability, and serviceability

  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Algorithms, Experimentation, Performance, Reliability

Keywords:
media processing workflows, peer-to-peer computing, stream workflow processing