Distributed image search in camera sensor networks

ABSTRACT

Recent advances in sensor networks permit the use of a large number of relatively inexpensive distributed computational nodes with camera sensors linked in a network and possibly linked to one or more central servers. We argue that the full potential of such a distributed system can be realized if it is designed as a distributed search engine where images from different sensors can be captured, stored, searched and queried. However, unlike traditional image search engines that are focused on resource-rich situations, the resource limitations of camera sensor networks in terms of energy, bandwidth, computational power, and memory capacity present significant challenges. In this paper, we describe the design and implementation of a distributed search system over a camera sensor network where each node is a search engine that senses, stores and searches information. Our work involves innovation at many levels including local storage, local search, and distributed search, all of which are designed to be efficient under the resource constraints of sensor networks. We present an implementation of the search engine on a network of iMote2 sensor nodes equipped with low-power cameras and extended flash storage. We evaluate our system for a dataset comprising book images, and demonstrate more than two orders of magnitude reduction in the amount of data communicated and up to 5x reduction in overall energy consumption over alternate techniques.

REFERENCES

• 1: Enalab imote2 camera. http://enaweb.eng.yale.edu/drupal/. 2007.
• 2: H. Aghajan, J. Augusto, C. Wu, P. McCullagh, and J. Walkden. Distributed vision-based accident management for assisted living. In ICOST, pages 196--205, 2007.
• 3: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 4: Hui Dai , Michael Neufeld , Richard Han, ELF: an efficient log-structured flash file system for micro sensor nodes, Proceedings of the 2nd international conference on Embedded networked sensor systems, November 03-05, 2004, Baltimore, MD, USA [doi>10.1145/1031495.1031516]
• 5: Amol Deshpande , Carlos Guestrin , Samuel R. Madden , Joseph M. Hellerstein , Wei Hong, Model-driven data acquisition in sensor networks, Proceedings of the Thirtieth international conference on Very large data bases, p.588-599, August 31-September 03, 2004, Toronto, Canada
• 6: S. Feng, R. Manmatha, and V. Lavrenko. Multiple Bernoulli relevance models for image and video annotation. In IEEE CVPR, pages 1002--1009, 2004.
• 7: Shravan Gaonkar , Jack Li , Romit Roy Choudhury , Landon Cox , Al Schmidt, Micro-Blog: sharing and querying content through mobile phones and social participation, Proceeding of the 6th international conference on Mobile systems, applications, and services, June 17-20, 2008, Breckenridge, CO, USA [doi>10.1145/1378600.1378620]
• 8: R. Goshorn, J. Goshorn, D. Goshorn, and H. Aghajan. Architecture for cluster-based automated surveillance network for detecting and tracking multiple persons. In ICDSC, 2007.
• 9: J. Hellerstein, W. Hong, S. Madden, and K. Stanek. Beyond average: Towards sophisticated sensing with queries. In IPSN'03, pages 63--79, 2003.
• 10: Stephan Hengstler , Daniel Prashanth , Sufen Fong , Hamid Aghajan, MeshEye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance, Proceedings of the 6th international conference on Information processing in sensor networks, April 25-27, 2007, Cambridge, Massachusetts, USA [doi>10.1145/1236360.1236406]
• 11: Jonathan W. Hui , David Culler, The dynamic behavior of a data dissemination protocol for network programming at scale, Proceedings of the 2nd international conference on Embedded networked sensor systems, November 03-05, 2004, Baltimore, MD, USA [doi>10.1145/1031495.1031506]
• 12: http://www.intel.com/research/exploratory/motes.htm. Intel imote2.
• 13: Aman Kansal , Michel Goraczko , Feng Zhao, Building a sensor network of mobile phones, Proceedings of the 6th international conference on Information processing in sensor networks, April 25-27, 2007, Cambridge, Massachusetts, USA [doi>10.1145/1236360.1236433]
• 14: T. Ko, Z. M. Charbiwala, S. Ahmadian, M. Rahimi, M. B. Srivastava, S. Soatto, and D. Estrin. Exploring tradeoffs in accuracy, energy and latency of scale invariant feature transform in wireless camera networks. In ICDSC, 2007.
• 15: http://people.csail.mit.edu/kkl/libpmk/. LIBPMK: A Pyramid Match Toolkit.
• 16: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 17: Dimitrios Lymberopoulos , Abhijit S. Ogale , Andreas Savvides , Yiannis Aloimonos, A sensory grammar for inferring behaviors in sensor networks, Proceedings of the 5th international conference on Information processing in sensor networks, April 19-21, 2006, Nashville, Tennessee, USA [doi>10.1145/1127777.1127817]
• 18: Mohammad Rahimi , Rick Baer , Obimdinachi I. Iroezi , Juan C. Garcia , Jay Warrior , Deborah Estrin , Mani Srivastava, Cyclops: in situ image sensing and interpretation in wireless sensor networks, Proceedings of the 3rd international conference on Embedded networked sensor systems, November 02-04, 2005, San Diego, California, USA [doi>10.1145/1098918.1098939]
• 19: Samuel Madden , Michael J. Franklin , Joseph M. Hellerstein , Wei Hong, TAG: a Tiny AGgregation service for Ad-Hoc sensor networks, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060303]
• 20: Gaurav Mathur , Peter Desnoyers , Deepak Ganesan , Prashant Shenoy, Capsule: an energy-optimized object storage system for memory-constrained sensor devices, Proceedings of the 4th international conference on Embedded networked sensor systems, October 31-November 03, 2006, Boulder, Colorado, USA [doi>10.1145/1182807.1182827]
• 21: Gaurav Mathur , Peter Desnoyers , Deepak Ganesan , Prashant Shenoy, Ultra-low power data storage for sensor networks, Proceedings of the 5th international conference on Information processing in sensor networks, April 19-21, 2006, Nashville, Tennessee, USA [doi>10.1145/1127777.1127833]
• 22: Krystian Mikolajczyk , Cordelia Schmid, A Performance Evaluation of Local Descriptors, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.10, p.1615-1630, October 2005 [doi>10.1109/TPAMI.2005.188]
• 23: Suman Nath , Aman Kansal, FlashDB: dynamic self-tuning database for NAND flash, Proceedings of the 6th international conference on Information processing in sensor networks, April 25-27, 2007, Cambridge, Massachusetts, USA [doi>10.1145/1236360.1236412]
• 24: David Nister , Henrik Stewenius, Scalable Recognition with a Vocabulary Tree, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2161-2168, June 17-22, 2006 [doi>10.1109/CVPR.2006.264]
• 25: J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.
• 26: Sasank Reddy , Andrew Parker , Josh Hyman , Jeff Burke , Deborah Estrin , Mark Hansen, Image browsing, processing, and clustering for participatory sensing: lessons from a DietSense prototype, Proceedings of the 4th workshop on Embedded networked sensors, June 25-26, 2007, Cork, Ireland [doi>10.1145/1278972.1278975]
• 27: http://vision.ucla.edu/~vedaldi/code/siftpp/siftpp.html. SIFT++: A lightweight C++ implementation of SIFT.
• 28: Adam Silberstein Silberstein , Rebecca Braynard , Carla Ellis , Kamesh Munagala , Jun Yang, A Sampling-Based Approach to Optimizing Top-k Queries in Sensor Networks, Proceedings of the 22nd International Conference on Data Engineering, p.68, April 03-07, 2006 [doi>10.1109/ICDE.2006.10]
• 29: Adam Silberstein , Kamesh Munagala , Jun Yang, Energy-efficient monitoring of extreme values in sensor networks, Proceedings of the 2006 ACM SIGMOD international conference on Management of data, June 27-29, 2006, Chicago, IL, USA [doi>10.1145/1142473.1142493]
• 30: Josef Sivic , Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1470, October 13-16, 2003
• 31: C. C. Tan, B. Sheng, H. Wang, and Q. Li. Microsearch: When search engines meet small devices. In Pervasive, pages 93--110, 2008.
• 32: H. Wang, C. C. Tan, and Q. Li. Snoogle: A search engine for physical world. In IEEE Infocom, 2008.
• 33: Demetrios Zeinalipour-Yazti , Song Lin , Vana Kalogeraki , Dimitrios Gunopulos , Walid A. Najjar, Microhash: an efficient index structure for fash-based sensor devices, Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies, p.3-3, December 13-16, 2005, San Francisco, CA

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Design, Management, Performance

Keywords:
camera sensor networks, distributed search, visterms, vocabulary tree