Combatting rendering latency

ABSTRACT

Latency or lag in an interactive graphics system is the delay between user input and displayed output. We have found latency and the apparent bobbing and swimming of objects that it produces to be a serious problem for head-mounted display (HMD) and augmented reality applications. At UNC, we have been investigating a number of ways to reduce latency; we present two of these. Slats is an experimental rendering system for our Pixel-Planes 5 graphics machine guaranteeing a constant single NTSC field of latency. This guaranteed response is especially important for predictive tracking. Just-in-time pixels is an attempt to compensate for rendering latency by rendering the pixels in a scanned display based on their position in the scan.

REFERENCES

• 1: Ronald Azuma , Gary Bishop, Improving static and dynamic registration in an optical see-through HMD, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.197-204, July 1994 [doi>10.1145/192161.192199]
• 2: Ronald Tadao Azuma, Predictive tracking for augmented reality, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1996
• 3: Michael Bajura , Henry Fuchs , Ryutarou Ohbuchi, Merging virtual objects with the real world: seeing ultrasound imagery within the patient, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.203-210, July 1992
• 4: Gary Bishop , Henry Fuchs , Leonard McMillan , Ellen J. Scher Zagier, Frameless rendering: double buffering considered harmful, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.175-176, July 1994 [doi>10.1145/192161.192195]
• 5: Breglia, Denis, Michael Spooner and Dan Lobb. Helmet Mounted Laser Projector. Proceedings of the Image Generation/Display Conference II (Scottsdale, Arizona June: IO- 12, 1981). pp. 241-258.
• 6: Burbidge, Dick, Paul Murray. Hardware Improvements To The Helmet Mounted Projector On the Visual Display Research Tool (VDRT) At The Naval Training Systems Center. Proceedings of the SPIE conference on Head-Mounted Displays, 1989.
• 7: Cohen, Jon and Marc Olano. Low Latency Rendering on Pixel-Planes 5. UNC Chapel Hill Department of Com,puter Science technical report TR94-028, 1994.
• 8: David Ellsworth. Pixel-Planes 5 Rendering Control. UNC Chapel Hill Department of Computer Science Software Documentation, 1989.
• 9: Steven Feiner , Blair Macintyre , Dorée Seligmann, Knowledge-based augmented reality, Communications of the ACM, v.36 n.7, p.53-62, July 1993 [doi>10.1145/159544.159587]
• 10: M. Friedmann , T. Starner , Alex Pentland, Device synchronization using an optimal linear filter, Proceedings of the 1992 symposium on Interactive 3D graphics, p.57-62, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147164]
• 11: Henry Fuchs , John Poulton , John Eyles , Trey Greer , Jack Goldfeather , David Ellsworth , Steve Molnar , Greg Turk , Brice Tebbs , Laura Israel, Pixel-planes 5: a heterogeneous multiprocessor graphics system using processor-enhanced memories, Proceedings of the 16th annual conference on Computer graphics and interactive techniques, p.79-88, July 1989
• 12: List, Uwe Nonlinear Prediction of Head Movements for Helmet-Mounted Displays. Technical Paper AFHRL-TP-83-45. December 1983.
• 13: Mark R. Mine, Characterization of End-to-End Delays in Head-Mounted Display Systems, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1993
• 14: Mark Mine , Gary Bishop, Just-In-Time Pixels, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1995
• 15: Matthew Regan , Ronald Pose, Priority rendering with a virtual reality address recalculation pipeline, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.155-162, July 1994 [doi>10.1145/192161.192192]
• 16: Mark Ward , Ronald Azuma , Robert Bennett , Stefan Gottschalk , Henry Fuchs, A demonstrated optical tracker with scalable work area for head-mounted display systems, Proceedings of the 1992 symposium on Interactive 3D graphics, p.43-52, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147162]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

General Terms:
Algorithms, Experimentation, Performance