RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors

ABSTRACT

This paper describes how to instrument the physical world so that objects become self-describing, communicating their identity, geometry, and other information such as history or user annotation. The enabling technology is a wireless tag which acts as a radio frequency identity and geometry (RFIG) transponder. We show how addition of a photo-sensor to a wireless tag significantly extends its functionality to allow geometric operations - such as finding the 3D position of a tag, or detecting change in the shape of a tagged object. Tag data is presented to the user by direct projection using a handheld locale-aware mobile projector. We introduce a novel technique that we call interactive projection to allow a user to interact with projected information e.g. to navigate or update the projected information.The ideas are demonstrated using objects with active radio frequency (RF) tags. But the work was motivated by the advent of unpowered passive-RFID, a technology that promises to have significant impact in real-world applications. We discuss how our current prototypes could evolve to passive-RFID in the future.

REFERENCES

• 1: Harold Abelson , Don Allen , Daniel Coore , Chris Hanson , George Homsy , Thomas F. Knight, Jr. , Radhika Nagpal , Erik Rauch , Gerald Jay Sussman , Ron Weiss, Amorphous computing, Communications of the ACM, v.43 n.5, p.74-82, May 2000 [doi>10.1145/332833.332842]
• 2: Ronald Azuma , Yohan Baillot , Reinhold Behringer , Steven Feiner , Simon Julier , Blair MacIntyre, Recent Advances in Augmented Reality, IEEE Computer Graphics and Applications, v.21 n.6, p.34-47, November 2001 [doi>10.1109/38.963459]
• 3: BEARDSLEY, P., RASKAR, R., FORLINES, C., AND VANBAAR, J. 2004. Interactive Projection. TR 2004/042, MERL.
• 4: BEARDSLEY, P., VANBAAR, J., AND RASKAR, R. 2004. Augmenting a Projector-Camera Device with Laser Pointers. TR 2004/035, MERL.
• 5: Oliver Bimber , Bernd Fröhlich , Dieter Schmalstieg , L. Miguel Encarnação, The Virtual Showcase, IEEE Computer Graphics and Applications, v.21 n.6, p.48-55, November 2001 [doi>10.1109/38.963460]
• 6: CANESTA, 2002. Miniature Laser Projector Keyboard. http://www.canesta.com.
• 7: Martin A. Fischler , Robert C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, v.24 n.6, p.381-395, June 1981 [doi>10.1145/358669.358692]
• 8: FOXLIN, E., AND NAIMARK, M. 2002. Shadow Effects of Virtual Objects on Real Surfaces with a Handheld Projector. Unpublished.
• 9: Richard Hartley , Andrew Zisserman, Multiple view geometry in computer visiond, Cambridge University Press, New York, NY, 2001
• 10: Ken Hinckley , Jeff Pierce , Mike Sinclair , Eric Horvitz, Sensing techniques for mobile interaction, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.91-100, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354417]
• 11: Lars Erik Holmquist , Friedemann Mattern , Bernt Schiele , Petteri Alahuhta , Michael Beigl , Hans-Werner Gellersen, Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proceedings of the 3rd international conference on Ubiquitous Computing, p.116-122, September 30-October 02, 2001, Atlanta, Georgia, USA
• 12: LUMILEDS, 2003. (Bright LEDs). http://lumileds.com.
• 13: Hongshen Ma , Joseph A. Paradiso, The FindIT Flashlight: Responsive Tagging Based on Optically Triggered Microprocessor Wakeup, Proceedings of the 4th international conference on Ubiquitous Computing, p.160-167, September 29-October 01, 2002, Göteborg, Sweden
• 14: Darnell J. Moore , Roy Want , Beverly L. Harrison , Anuj Gujar , Ken Fishkin, Implementing phicons: combining computer vision with infrared technology for interactive physical icons, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.67-68, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322585]
• 15: NAYAR, S. K., PERI, H., GROSSBERG, M. D., AND BELHUMEUR, P. N. 2003. A Projection System with Radiometric Compensation for Screen Imperfections. In Proc. ICCV Workshop on Projector-Camera Systems (PROCAMS).
• 16: O. Omojola , E. Rehmi Post , M. D. Hancher , Y. Maguire , R. Pappu , B. Schoner , P. R. Russo , R. Fletcher , N. Gershenfeld, An installation of interactive furniture, IBM Systems Journal, v.39 n.3-4, p.861-879, July 2000
• 17: PATEL, S. N., AND ABOWD, G. D. 2003. A 2-Way Laser-Assisted Selection Scheme for Handhelds in a Physical Environment. In Ubicomp, 200--207.
• 18: James Patten , Hiroshi Ishii , Jim Hines , Gian Pangaro, Sensetable: a wireless object tracking platform for tangible user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.253-260, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365112]
• 19: Claudio S. Pinhanez, The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces, Proceedings of the 3rd international conference on Ubiquitous Computing, p.315-331, September 30-October 02, 2001, Atlanta, Georgia, USA
• 20: RASKAR, R., WELCH, G., AND FUCHS, H. 1998. Spatially Augmented Reality. In The First IEEE International Workshop on Augmented Reality (IWAR).
• 21: Ramesh Raskar , Greg Welch , Kok-Lim Low , Deepak Bandyopadhyay, Shader Lamps: Animating Real Objects With Image-Based Illumination, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.89-102, June 25-27, 2001
• 22: Ramesh Raskar , Jeroen van Baar , Paul Beardsley , Thomas Willwacher , Srinivas Rao , Clifton Forlines, iLamps: geometrically aware and self-configuring projectors, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 23: Jun Rekimoto , Brygg Ullmer , Haruo Oba, DataTiles: a modular platform for mixed physical and graphical interactions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.269-276, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365115]
• 24: RINGWALD, M. 2002. Spontaneous Interaction with Everyday Devices Using a PDA Workshop on Supporting Spontaneous Interaction in Ubiquitous Computing Settings. In UbiComp.
• 25: SIEMENS, 2002. Siemens Mini Beamer. http://w4.siemens.de/en2/html/press//newsdesk_archive/2002/foe02121b.html.
• 26: SYMBOL, 2002. Laser Projection Display. http://www.symbol.com/products/oem/lpd.html.
• 27: Seth Teller , Jiawen Chen , Hari Balakrishnan, Pervasive Pose-Aware Applications and Infrastructure, IEEE Computer Graphics and Applications, v.23 n.4, p.14-18, July 2003 [doi>10.1109/MCG.2003.1210859]
• 28: THE COOLTOWN PROJECT, 2001. http://www.cooltown.com/research/.
• 29: John Underkoffler , Brygg Ullmer , Hiroshi Ishii, Emancipated pixels: real-world graphics in the luminous room, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.385-392, July 1999 [doi>10.1145/311535.311593]
• 30: VERLINDEN, J. C., DE SMIT, A., PEETERS, A. W. J., AND VAN GELDEREN, M. H. 2003. Development of a Flexible Augmented Prototyping System. In The 11th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision'2003.
• 31: Roy Want , Andy Hopper , Veronica Falcão , Jonathan Gibbons, The active badge location system, ACM Transactions on Information Systems (TOIS), v.10 n.1, p.91-102, Jan. 1992 [doi>10.1145/128756.128759]
• 32: WANT, R., SCHILIT, B. N., ADAMS, N. I., GOLD, R., PETERSEN, K., GOLDBERG, D., ELLIS, J. R., AND WEISER, M. 1995. An Overview of the ParcTab Ubiquitous Computing Experiment. In IEEE Personal Communications, 28--43.
• 33: Roy Want , Kenneth P. Fishkin , Anuj Gujar , Beverly L. Harrison, Bridging physical and virtual worlds with electronic tags, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.370-377, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303111]
• 34: WANT, R. 2003. RFID, A Key to Automating Everything. In Scientific American, vol. 290(1), 56--65.
• 35: Zhengyou Zhang, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.11, p.1330-1334, November 2000 [doi>10.1109/34.888718]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
          Subjects: Imaging geometry

Keywords:
augmented reality, human-machine communication, image stabilization, projector, radio frequency identification, stucture from motion