Improving static and dynamic registration in an optical see-through HMD

ABSTRACT

In Augmented Reality, see-through HMDs superimpose virtual 3D objects on the real world. This technology has the potential to enhance a user's perception and interaction with the real world. However, many Augmented Reality applications will not be accepted until we can accurately register virtual objects with their real counterparts. In previous systems, such registration was achieved only from a limited range of viewpoints, when the user kept his head still. This paper offers improved registration in two areas. First, our system demonstrates accurate static registration across a wide variety of viewing angles and positions. An optoelectronic tracker provides the required range and accuracy. Three calibration steps determine the viewing parameters. Second, dynamic errors that occur when the user moves his head are reduced by predicting future head locations. Inertial sensors mounted on the HMD aid head-motion prediction. Accurate determination of prediction distances requires low-overhead operating systems and eliminating unpredictable sources of latency. On average, prediction with inertial sensors produces errors 2-3 times lower than prediction without inertial sensors and 5-10 times lower than using no prediction at all. Future steps that may further improve registration are outlined.

REFERENCES

• 1: Albrecht, R. E. An adaptive digital filter to predict pilot head look direction for helmet-mounted displays. MS Thesis, University of Dayton, Ohio (July 1989).
• 2: Robert O. Andres , Kenny J. Hartung, Prediction of head movement time using Fitts' Law, Human Factors, v.31 n.6, p.703-714, Dec. 1989
• 3: Ronald Azuma, Tracking requirements for augmented reality, Communications of the ACM, v.36 n.7, p.50-51, July 1993 [doi>10.1145/159544.159581]
• 4: Michael Bajura , Henry Fuchs , Ryutarou Ohbuchi, Merging virtual objects with the real world: seeing ultrasound imagery within the patient, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.203-210, July 1992
• 5: Beer, Ferdinand P. and E. Russell Johnston, Jr. Vector Mechanics for Engineers: Statics and Dynamics (5th ed). McGraw-Hill, 1988.
• 6: Caudell, Thomas P. and David W. Mizell. Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufac-turing Processes. Proceedings of Hawaii International Conference on System Sciences (Jan. 1992), 659-669.
• 7: Chou, Jack C.K. Quaternion Kinematic and Dynamic Differential Equations. IEEE Trans Robotics and Automation 8, 1 (Feb. 1992), 53-64.
• 8: Cohen, Jonathan, and Mark Olano. Low Latency Rendering on Pixel-Planes 5. UNC Chapel Hill Dept. of Computer Science technical report TR94-028 (1994).
• 9: Carolina Cruz-Neira , Daniel J. Sandin , Thomas A. DeFanti, Surround-screen projection-based virtual reality: the design and implementation of the CAVE, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.135-142, September 1993 [doi>10.1145/166117.166134]
• 10: Michael Deering, High resolution virtual reality, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.195-202, July 1992
• 11: Steven Feiner , Blair Macintyre , Dorée Seligmann, Knowledge-based augmented reality, Communications of the ACM, v.36 n.7, p.53-62, July 1993 [doi>10.1145/159544.159587]
• 12: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics: principles and practice (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 13: M. Friedmann , T. Starner , Alex Pentland, Device synchronization using an optimal linear filter, Proceedings of the 1992 symposium on Interactive 3D graphics, p.57-62, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147164]
• 14: Henry Fuchs , John Poulton , John Eyles , Trey Greer , Jack Goldfeather , David Ellsworth , Steve Molnar , Greg Turk , Brice Tebbs , Laura Israel, Pixel-planes 5: a heterogeneous multiprocessor graphics system using processor-enhanced memories, Proceedings of the 16th annual conference on Computer graphics and interactive techniques, p.79-88, July 1989
• 15: Stefan Gottschalk , John F. Hughes, Autocalibration for virtual environments tracking hardware, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.65-72, September 1993 [doi>10.1145/166117.166124]
• 16: Douglas E. Holmgren, Design and Construction of a 30-Degree See-Through Head-Mounted Display, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1992
• 17: Jagacinski, Richard J., and Donald L. Monk. Fitts' Law in Two Di-mensions with Hand and Head Movements. Journal of Motor Be-havior 17, 1 (1985), 77-95.
• 18: Janin, Adam L., David W. Mizell, and Thomas P. Caudell. Calibra-tion of Head-Mounted Displays for Augmented Reality Applications. Proceedings of IEEE VRAIS '93 (Seattle, WA, Sept. 18-22, 1993), 246-255.
• 19: Kalman, R. E., and R. S. Bucy. New Results in Linear Filtering and Prediction Theory. Trans ASME, J. Basic Eng., Series 83D (Mar. 1961), 95-108.
• 20: Krueger, Myron W. Simulation versus artificial reality. Proceedings of IMAGE VI Conference (Scottsdale, AZ, 14-17 July 1992), 147- 155.
• 21: R. K. Lenz , R. Y. Tsai, Techniques for Calibration of the Scale Factor and Image Center for High Accuracy 3-D Machine Vision Metrology, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.10 n.5, p.713-720, September 1988 [doi>10.1109/34.6781]
• 22: Lewis, Frank L. Optimal Estimation. John Wiley & Sons, 1986.
• 23: Jiandong Liang , Chris Shaw , Mark Green, On temporal-spatial realism in the virtual reality environment, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.19-25, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120784]
• 24: List, Uwe H. Nonlinear Prediction of Head Movements for Helmet-Mounted Displays. Technical report AFHRL-TP-83-45 {AD-A136590}, Williams AFB, AZ: Operations Training Division (1984).
• 25: Murray, P.M. and B. Barber. Visual Display Research Tool. AGARD Conference Proceedings No. 408 Flight Simulation (Cambridge, UK, 30 Sept. - 3 Oct. 1985).
• 26: Paley, W. Bradford. Head-Tracking Stereo Display: Experiments and Applications. SPIE Vol. 1669 Stereoscopic Displays and Applica-tions III (San Jose, CA, Feb. 12-13, 1992), 84-89.
• 27: Press, William H., et al. Numerical Recipes in C. Cambridge Uni-versity Press, 1988.
• 28: Rebo, Robert. A Helmet-Mounted Virtual Environment Display Sys-tem. MS Thesis, Air Force Institute of Technology (Dec 1988).
• 29: Warren Robinett, Synthetic experience: a proposed taxonomy, Presence: Teleoperators and Virtual Environments, v.1 n.2, p.229-247, Spring 1992
• 30: Warren Robinett , Richard Holloway, Implementation of flying, scaling and grabbing in virtual worlds, Proceedings of the 1992 symposium on Interactive 3D graphics, p.189-192, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147201]
• 31: Warren Robinett , Jannick P. Rolland, A computational model for the stereoscopic optics of a head-mounted display, Presence: Teleoperators and Virtual Environments, v.1 n.1, p.45-62, Winter 1992
• 32: Shaw, Chris and Jiandong Liang. An Experiment to Characterize Head Motion in VR and RR Using MR. Proceedings of 1992 Western Com-puter Graphics Symposium (Banff, Alberta, Canada, April 6-8, 1992), 99-101.
• 33: Sims, Dave. New Realities in Aircraft Design and Manufacture. IEEE CG&A 14, 2 (March 1994), 91.
• 34: Smith Jr., B. R. Digital head tracking and position prediction for hel-met mounted visual display systems. Proceedings of AIAA 22nd Aerospace Sciences Meeting (Reno, NV, Jan. 9-12, 1984).
• 35: So, Richard H. Y. and Michael J. Griffin. Compensating Lags in Head-Coupled Displays Using Head Position Prediction and Image Deflec-tion. Journal of Aircraft 29, 6 (Nov-Dec 1992), 1064-1068.
• 36: Mark Ward , Ronald Azuma , Robert Bennett , Stefan Gottschalk , Henry Fuchs, A demonstrated optical tracker with scalable work area for head-mounted display systems, Proceedings of the 1992 symposium on Interactive 3D graphics, p.43-52, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147162]
• 37: Welch, Brian L., Ron Kruk, et al. Flight Simulator Wide Field-of-View Helmet-Mounted Infinity Display System. Technical report AFHRL-TR-85-59, WilliamsAFB,AZ, Operations Training Division (May 1986).

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.1 Hardware architecture
          Subjects: Three-dimensional displays**

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS

General Terms:
Design, Theory

Keywords:
augmented reality, calibration, registration