Bokode: imperceptible visual tags for camera based interaction from a distance

ABSTRACT

We show a new camera based interaction solution where an ordinary camera can detect small optical tags from a relatively large distance. Current optical tags, such as barcodes, must be read within a short range and the codes occupy valuable physical space on products. We present a new low-cost optical design so that the tags can be shrunk to 3mm visible diameter, and unmodified ordinary cameras several meters away can be set up to decode the identity plus the relative distance and angle. The design exploits the bokeh effect of ordinary cameras lenses, which maps rays exiting from an out of focus scene point into a disk like blur on the camera sensor. This bokeh-code or Bokode is a barcode design with a simple lenslet over the pattern. We show that a code with 15μm features can be read using an off-the-shelf camera from distances of up to 2 meters. We use intelligent binary coding to estimate the relative distance and angle to the camera, and show potential for applications in augmented reality and motion capture. We analyze the constraints and performance of the optical system, and discuss several plausible application scenarios.

REFERENCES

• 1: Gregory D. Abowd , Elizabeth D. Mynatt, Charting past, present, and future research in ubiquitous computing, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.29-58, March 2000 [doi>10.1145/344949.344988]
• 2: Ronald Azuma , Yohan Baillot , Reinhold Behringer , Steven Feiner , Simon Julier , Blair MacIntyre, Recent Advances in Augmented Reality, IEEE Computer Graphics and Applications, v.21 n.6, p.34-47, November 2001 [doi>10.1109/38.963459]
• 3: Claus, D., and Fitzgibbon, A. W. 2004. Reliable fiducial detection in natural scenes. In ECCV 2004, 469--480. 2
• 4: Diego López de Ipiña , Paulo R. S. Mendonça , Andy Hopper, TRIP: A Low-Cost Vision-Based Location System for Ubiquitous Computing, Personal and Ubiquitous Computing, v.6 n.3, p.206-219, May 2002 [doi>10.1007/s007790200020]
• 5: Fernald, R. D. 2006. Casting a genetic light on the evolution of eyes. Science 313, 5795, 1914--1918. 5
• 6: Mark Fiala, ARTag, a Fiducial Marker System Using Digital Techniques, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.590-596, June 20-26, 2005 [doi>10.1109/CVPR.2005.74]
• 7: ISO, 2006. Data Matrix bar code symbology specification. ISO/IEC 16022:2006. 2, 4
• 8: ISO, 2006. QR code 2005 bar code symbology specification. ISO/IEC 18004:2006. 2
• 9: Hirokazu Kato , Mark Billinghurst, Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System, Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality, p.85, October 20-21, 1999
• 10: Hiroko Kato , Keng T. Tan, Pervasive 2D Barcodes for Camera Phone Applications, IEEE Pervasive Computing, v.6 n.4, p.76-85, October 2007 [doi>10.1109/MPRV.2007.80]
• 11: Yoshifumi Kitamura , Takashige Konishi , Sumihiko Yamamoto , Fumio Kishino, Interactive stereoscopic display for three or more users, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.231-240, August 2001 [doi>10.1145/383259.383285]
• 12: Langlotz, T., and Bimber, O. 2007. Unsynchronized 4D barcodes. In ISVC, 363--374. 2
• 13: Anat Levin , Rob Fergus , Frédo Durand , William T. Freeman, Image and depth from a conventional camera with a coded aperture, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 14: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 15: David J. C. MacKay, Information Theory, Inference & Learning Algorithms, Cambridge University Press, New York, NY, 2002
• 16: Nobuyuki Matsushita , Daisuke Hihara , Teruyuki Ushiro , Shinichi Yoshimura , Jun Rekimoto , Yoshikazu Yamamoto, ID CAM: A Smart Camera for Scene Capturing and ID Recognition, Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, p.227, October 07-10, 2003
• 17: Mohan, A., Lanman, D., Hiura, S., and Raskar, R. 2009. Image Destabilization: Programmable defocus using lens and sensor motion. In IEEE ICCP. 5
• 18: Morton, A. Q. 1994. Packaging history: The emergence of the Uniform Product Code (UPC) in the United States. History and Technology 11, 1, 101--111. 2
• 19: Jerome Swartz , Ynjiun P. Wang, Fundamentals of Bar Code Information Theory, Computer, v.23 n.4, p.74-86, April 1990 [doi>10.1109/2.55471]
• 20: A. P. Pentland, A new sense for depth of field, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.9 n.4, p.523-531, July 1987 [doi>10.1109/TPAMI.1987.4767940]
• 21: Ramesh Raskar , Paul Beardsley , Jeroen van Baar , Yao Wang , Paul Dietz , Johnny Lee , Darren Leigh , Thomas Willwacher, RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 22: Sony, EyeToy. http://www.eyetoy.com. 2
• 23: Tateno, K., Kitahara, I., and Ohta, Y. 2007. A nested marker for augmented reality. In IEEE VR, 259--262. 2
• 24: Seth Teller , Jiawen Chen , Hari Balakrishnan, Pervasive Pose-Aware Applications and Infrastructure, IEEE Computer Graphics and Applications, v.23 n.4, p.14-18, July 2003 [doi>10.1109/MCG.2003.1210859]
• 25: Tremblay, E. J., Stack, R. A., Morrison, R. L., and Ford, J. E. 2007. Ultrathin cameras using annular folded optics. Applied Optics 46, 4, 463--471. 7
• 26: Ashok Veeraraghavan , Ramesh Raskar , Amit Agrawal , Ankit Mohan , Jack Tumblin, Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 27: Want, R. 2003. RFID: A key to automating everything. Scientific American. 2
• 28: M. Weiser, Ubiquitous Computing, Computer, v.26 n.10, p.71-72, October 1993 [doi>10.1109/2.237456]
• 29: Greg Welch , Gary Bishop , Leandra Vicci , Stephen Brumback , Kurtis Keller , D'nardo Colucci, The HiBall Tracker: high-performance wide-area tracking for virtual and augmented environments, Proceedings of the ACM symposium on Virtual reality software and technology, p.1-ff., December 20-22, 1999, London, United Kingdom [doi>10.1145/323663.323664]
• 30: Zhang, L., Curless, B., and Seitz, S. M. 2002. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In IEEE 3DPVT, 24--36. 5
• 31: Xiang Zhang , Stephan Fronz , Nassir Navab, Visual Marker Detection and Decoding in AR Systems: A Comparative Study, Proceedings of the 1st International Symposium on Mixed and Augmented Reality, p.97, September 30-October 01, 2002
• 32: Zhang, L., Subramaniam, N., Lin, R., Raskar, R., and Nayar, S. 2008. Capturing images with sparse informational pixels using projected 3D tags. In IEEE Virtual Reality. 2

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

Additional Classification:
  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.2 Nonnumerical Algorithms and Problems
          Subjects: Pattern matching

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology

General Terms:
Algorithms, Design, Measurement

Keywords:
2D fiducials, augmented reality, computational probes, defocus blur, human-computer interaction, motion capture