Interactive tracking of movable objects for the blind on the basis of environment models and perception-oriented object recognition methods

ABSTRACT

In previous work we have presented a prototype of an assistant system for the blind that can be used for self-localization and interactive object identification of static objects stored within 3D environment models. In this paper we present a new method for interactive tracking of various types of movable objects. The state of fixed movable objects, like doors, can be recognized by comparing the distance between sensor data and a 3D model. For the identification and model-based tracking of free movable objects, like chairs, we have developed an algorithm that is similar to human perception, based on shape and color comparisons to trained objects. Further, using a common face detection algorithm, our assistant system informs the user of the presence of people, and enables the localization of a real person based on interactive tracking of virtual models of humans.

REFERENCES

• 1: Biber, P., Fleck, S., Strasser, W. The Wägele: A Mobile Platform for Acquisition of 3D Models of Indoor Outdoor Environments. 9th Tübingen Perception Conference (TWK 2006), Tübingen, Germany, 66, 2006. http://www.twk.tuebingen.mpg.de/twk06/twk2006.pdf
• 2: Bowen, C. L. III, Using Wireless Networks To Assist Navigation For Individuals With Disabilities. California State University, Northridge Center on Disabilities' 21st Annual International Technology and Persons with Disabilities Conference, Los Angeles, CA, USA, 2006. http://www.csun.edu/cod/conf/2006/proceedings/2931.htm
• 3: Face detection. http://www.intel.com/technology/itj/2005/volume09issue02/art03_learning_vision/p04_face_detection.htm
• 4: Gomez, L. Trekker Reviewed an Advanced GPS System for the Blind. California State University, Northridge Center on Disabilities' 21st Annual International Technology and Persons with Disabilities Conference, Los Angeles, CA, USA, 2006. http://www.csun.edu/cod/conf/2006/proceedings/2829.htm
• 5: Hang, R., Black, J. The Use of a GPS-Mapping Program in Orientation and Mobility. California State University, Northridge Center on Disabilities' 21st Annual International Technology and Persons with Disabilities Conference, Los Angeles, CA, USA, 2006. http://www.csun.edu/cod/conf/2006/proceedings/2950.htm
• 6: Hu, M. K. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory, Vol. 8, 179--187, 1962.
• 7: Hub, A., Diepstraten, J., Ertl, T. Augmented Indoor Modeling for Navigation Support for the Blind. Conference Proceedings: CPSN'05 - The International Conference on Computers for People with Special Needs, Las Vegas, NV, USA, 54--59, 2005.
• 8: Andreas Hub , Joachim Diepstraten , Thomas Ertl, Design and development of an indoor navigation and object identification system for the blind, Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility, October 18-20, 2004, Atlanta, GA, USA
• 9: Hub, A., Diepstraten, J., Ertl, T. Design of an Object Identification and Orientation Assistant for the Deafblind. Conference Proceedings: 6th DbI European Conference On Deafblindness, Presov, Slowakia, 2005.
• 10: Hub, A., Diepstraten, J., Ertl, T. Learning foreign languages by using a new type of orientation assistant for the blind. Conference Proceedings: International Council for Education of People with Visual Impairment, European Conference, Chemnitz, Germany, 339--341, 2005.
• 11: Hub, A., Hartter, T., Ertl, T. Interactive Localization and Recognition of Objects for the Blind. California State University, Northridge Center on Disabilities' 21st Annual International Technology and Persons with Disabilities Conference, Los Angeles, CA, USA, 2006. http://www.csun.edu/cod/conf/2006/proceedings/2661.htm
• 12: Hub, A., Teufel, H. Messung von wahrgenommenen Farben in realen Szenen und Vorausberechnung von Objektfarben in verschiedenfarbigen Umgebungen. In: Rahman Jamal, Hans Jaschinski (ed.): Virtuelle Instrumente in der Praxis, Begleitband zum Kongress VIP (2003), Hüthig, 11--14.
• 13: Krishna, S., Black, J. iCARE Interaction Assistant: A Wearable Face Recognition Device to Facilitate Social Interaction. California State University, Northridge Center on Disabilities' 21st Annual International Technology and Persons with Disabilities Conference, Los Angeles, CA,USA, 2006. http://www.csun.edu/cod/conf/2006/proceedings/2893.htm
• 14: Kulykukin, V., Gharpure C., DeGraw N. Human-Robot Interaction in a Robotic Guide for the Visually Impaired, AAAI Spring Symposium, 158--164, 2004.
• 15: Laser long cane. http://www.vistac.com/VISTAC_Laser_Long_Cane.htm#Detailed%20Description
• 16: Lienhart, R., Maydt, J. An Extended Set of Haar-like Features for Rapid Object Detection. IEEE International Conference on Image Processing, Rochester, NY, USA, Vol.1, 900--903, 2002.
• 17: Loomis, J., Gooledge, R. G., Klatzky, R. L. Navigation System for the Blind: Auditory Display Modes and Guidance. Presence Vol.7 No.2, 192--203, 1998.
• 18: Luo, A., Zhang X. F., Tao W., Burkhardt H. Recognition of Artificial 3-D Landmarks from Depth and Color - a first Prototype of Electronic Glasses for Blind People. Proceedings of the 11th Scandinavian Conference on Image Analysis (SCIA'99), Kangerlussuaq, Greenland, 1999.
• 19: Magatani, K., Sawa, K., Yanashima K. Development of the Navigation System for the Visually Impaired by Using Optical Beacons. Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Istanbul, Turkey, 488--1490, 2001.
• 20: Mecocci, A., Lodola, R., Salvatore U., Outdoor scenes interpretation suitable for blind people navigation. Proceedings of the 5th IEEE International Conference on Image Processing and its Applications, Edinburgh, UK, 256--260, 1995.
• 21: Miniguide. http://www.gdp-research.com.au/minig_1.htm
• 22: Naviwalk. http://www.idfone.co.kr/
• 23: Nexus. http://www.nexus.uni-stuttgart.de/index.en.html
• 24: Daniela Nicklas , Matthias Großmann , Thomas Schwarz , Steffen Volz , Bernhard Mitschang, A Model-Based, Open Architecture for Mobile, Spatially Aware Applications, Proceedings of the 7th International Symposium on Advances in Spatial and Temporal Databases, p.117-135, July 12-15, 2001
• 25: Open Computer Vision Library Documentation. http://www.cs.bham.ac.uk/resources/courses/robotics/doc/index.php
• 26: OpenSceneGraph API. http://www.openscenegraph.org/.
• 27: Lisa Ran , Sumi Helal , Steve Moore, Drishti: An Integrated Indoor/Outdoor Blind Navigation System and Service, Proceedings of the Second IEEE International Conference on Pervasive Computing and Communications (PerCom'04), p.23, March 14-17, 2004
• 28: David A. Ross , Bruce B. Blasch, Wearable interfaces for orientation and wayfinding, Proceedings of the fourth international ACM conference on Assistive technologies, p.193-200, November 13-15, 2000, Arlington, Virginia, United States [doi>10.1145/354324.354380]
• 29: Seeing with Sound. http://www.seeingwithsound.com/
• 30: Shoval, S., Borenstein, J., Koren, Y. Mobile Robot Obstacle Avoidance in a Computerized Travel for the Blind. The IEEE International Conference on Robotics and Automation, San Diego, CA, USA, 2023--2028, 1994.
• 31: Strothotte, T., Petrie H., Johnson, V., Reichert L. MoBIC: User Needs and Premlimary Design for a Mobility Aid for Blind and Elderly Travellers. Proceedings of the 2nd Tide congress, Paris, La Villette, France, 348--351, 1995.
• 32: UltraCane. http://www.soundforesight.co.uk/product.htm
• 33: Viola, P. and Michael J. Jones, Rapid Object Detection using a Boosted Cascade of Simple Features. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA, 511--518, 2001.
• 34: Zelek, J. The E. (Ben) & Mary Hochhausen Fund for Research in Adaptive Technology For Blind and Visually Impaired Persons. Research Project. Available from: http://www.soe.uoguelph.ca/webfiles/jzelek/cnib2.pdf.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Prototyping

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
          Subjects: Region growing, partitioning; Edge and feature detection
      I.4.7 Feature Measurement
          Subjects: Feature representation
      I.4.8 Scene Analysis
          Subjects: Depth cues; Object recognition; Tracking; Stereo; Shape; Color

  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.1 Computer Uses in Education
          Subjects: Distance learning
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Algorithms, Design, Experimentation, Human Factors, Measurement

Keywords:
blind users, impaired vision, indoor navigation, mobile computing