Subobject detection through spatial relationships on mobile phones

ABSTRACT

We present a novel image classification technique for detecting multiple objects (called subobjects) in a single image. In addition to image classifiers, we apply spatial relationships among the subobjects to verify and to predict the locations of detected and undetected subobjects, respectively. By continuously refining the spatial relationships throughout the detection process, even locations of completely occluded exhibits can be determined. This approach is applied in the context of PhoneGuide, an adaptive museum guidance system for camera-equipped mobile phones.

Laboratory tests as well as a field experiment reveal recognition rates and performance improvements when compared to related approaches.

REFERENCES

• 1: Shivani Agarwal , Aatif Awan , Dan Roth, Learning to Detect Objects in Images via a Sparse, Part-Based Representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.11, p.1475-1490, November 2004 [doi>10.1109/TPAMI.2004.108]
• 2: H. Bay, B. Fasel, and L. V. Gool. Interactive museum guide. In Proc. Ubiquitous Computing, Workshop on Smart Environments and Their Applications to Cultural Heritage, 2005.
• 3: H. Bay, B. Fasel, and L. V. Gool. Interactive museum guide: Fast and robust recognition of museum objects. In Workshop on Mobile Vision, 2006.
• 4: H. Bay, T. Tuytelaars, and L. V. Gool. Surf: Speeded up robust features. In Proc. Conference on Computer Vision, 2006.
• 5: Erich Bruns , Oliver Bimber, Adaptive training of video sets for image recognition on mobile phones, Personal and Ubiquitous Computing, v.13 n.2, p.165-178, February 2009 [doi>10.1007/s00779-008-0194-3]
• 6: Erich Bruns , Oliver Bimber, Phone-to-phone communication for adaptive image classification, Proceedings of the 6th International Conference on Advances in Mobile Computing and Multimedia, November 24-26, 2008, Linz, Austria [doi>10.1145/1497185.1497243]
• 7: Erich Bruns , Benjamin Brombach , Oliver Bimber, Mobile Phone-Enabled Museum Guidance with Adaptive Classification, IEEE Computer Graphics and Applications, v.28 n.4, p.98-102, July 2008 [doi>10.1109/MCG.2008.77]
• 8: Erich Bruns , Benjamnin Brombach , Thomas Zeidler , Oliver Bimber, Enabling Mobile Phones To Support Large-Scale Museum Guidance, IEEE MultiMedia, v.14 n.2, p.16-25, April 2007 [doi>10.1109/MMUL.2007.33]
• 9: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 10: Nigel Davies , Keith Cheverst , Alan Dix , Andre Hesse, Understanding the role of image recognition in mobile tour guides, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085809]
• 11: M. Egenhofer and R. Franzosa. Point-set topological spatial relations. Journal of Geographical Information Systems, 5 (2):161--174, 1991.
• 12: Essam A. El-Kwae , Mansur R. Kabuka, A robust framework for content-based retrieval by spatial similarity in image databases, ACM Transactions on Information Systems (TOIS), v.17 n.2, p.174-198, April 1999 [doi>10.1145/306686.306689]
• 13: R. Fergus, P. Perona, A. Zisserman, and O. P. U. K. Object class recognition by unsupervised scale-invariant learning. In In CVPR, pages 264--271, 2003.
• 14: Paul Föckler , Thomas Zeidler , Benjamin Brombach , Erich Bruns , Oliver Bimber, PhoneGuide: museum guidance supported by on-device object recognition on mobile phones, Proceedings of the 4th international conference on Mobile and ubiquitous multimedia, December 08-10, 2005, Christchurch, New Zealand [doi>10.1145/1149488.1149490]
• 15: W. Förstner and E. Gülch. A fast operator for detection and precise location of distinct points, corners and centers of circular features. ISPRS Intercommission Workshop on Fast Processing of Photogrammetric Data, pages 149--155, 1987..
• 16: Gerald Fritz , Christin Seifert , Lucas Paletta, A Mobile Vision System for Urban Detection with Informative Local Descriptors, Proceedings of the Fourth IEEE International Conference on Computer Vision Systems, p.30, January 04-07, 2006 [doi>10.1109/ICVS.2006.5]
• 17: J. S. Hare and P. H. Lewis. Content-based image retrieval using a mobile device as a novel interface. In Proc. Storage and Retrieval Methods and Applications for Multimedia, pages 64--75, 2004.
• 18: J. Lewis. Fast normalized cross-correlation. In Vision Interface, pages 120--123, 1995.
• 19: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 20: C. Millet, I. Bloch, P. Hede, and P.-A. Moellic. Using relative spatial relationships to improve individual region recognition. In Proc. Integration of Knowledge, Semantic and Digital Media Technologies, pages 119--126, 2005.
• 21: D. Papadias and Y. Theodoridis. Spatial relations, minimum bounding rectangles, and spatial data structures. Journal of Geographical Information Science, 11(2):111--138, 1997.
• 22: Thang V. Pham , Arnold W. M. Smeulders, Learning spatial relations in object recognition, Pattern Recognition Letters, v.27 n.14, p.1673-1684, 15 October 2006 [doi>10.1016/j.patrec.2006.03.016]
• 23: Gabriel Takacs , Vijay Chandrasekhar , Natasha Gelfand , Yingen Xiong , Wei-Chao Chen , Thanos Bismpigiannis , Radek Grzeszczuk , Kari Pulli , Bernd Girod, Outdoors augmented reality on mobile phone using loxel-based visual feature organization, Proceeding of the 1st ACM international conference on Multimedia information retrieval, October 30-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1460096.1460165]
• 24: P. Viola and M. Jones. Robust real-time object detection. Technical report, Compaq CRL, 2002.
• 25: N. Wanas and M. Kamel. Decision fusion in neural network ensembles. Proc. Joint Conference on Neural Networks, 4:2952--2957, 2001.
• 26: Laurenz Wiskott , Jean-Marc Fellous , Norbert Krüger , Christopher von der Malsburg, Face Recognition by Elastic Bunch Graph Matching, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.775-779, July 1997 [doi>10.1109/34.598235]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
image classification, mobile computing, museum guidance application, spatial relationships, subobject detection