Vision-realistic rendering: simulation of the scanned foveal image from wavefront data of human subjects

ABSTRACT

We introduce the concept of vision-realistic rendering -- the computer generation of synthetic images that incorporate the characteristics of a particular individual's entire optical system. Specifically, this paper develops a method for simulating the scanned foveal image from wavefront data of actual human subjects, and demonstrates those methods on sample images.First, a subject's optical system is measured by a Shack-Hartmann wavefront aberrometry device. This device outputs a measured wavefront which is sampled to calculate an object space point spread function (OSPSF). The OSPSF is then used to blur input images. This blurring is accomplished by creating a set of depth images, convolving them with the OSPSF, and finally compositing to form a vision-realistic rendered image.Applications of vision-realistic rendering in computer graphics as well as in optometry and ophthalmology are discussed.

REFERENCES

• 1: ADELSON, E., AND BERGEN, J. 1991. Computational Models of Visual Processing. The MIT Press, Cambridge, Mass.
• 2: BARSKY, B. A., BARGTEIL, A. W., GARCIA, D. D., AND KLEIN, S. A. 2002. Introducing vision-realistic rendering. In Eurographics 2002 Poster Proceedings, Eurographics, 1--7. Held in Pisa, Italy 26--28 June 2002.
• 3: BARSKY, B. A., HORN, D. R., KLEIN, S. A., PANG, J. A., AND YU, M. 2003. Camera models and optical systems used in computer graphics: Part I, object based techniques. In Proceedings of the 2003 International Conference on Computational Science and its Applications (ICCSA'03), Montreal, Second International Workshop on Computer Graphics and Geometric Modeling (CGGM'2003), Springer-Verlag Lecture Notes in Computer Science (LNCS), Berlin/Heidelberg, 246--255.
• 4: BARSKY, B. A., HORN, D. R., KLEIN, S. A., PANG, J. A., AND YU, M. 2003. Camera models and optical systems used in computer graphics: Part II, image based techniques. In Proceedings of the 2003 International Conference on Computational Science and its Applications (ICCSA'03), Montreal, Second International Workshop on Computer Graphics and Geometric Modeling (CGGM'2003), Springer-Verlag Lecture Notes in Computer Science (LNCS), Berlin/Heidelberg, 256--265.
• 5: BARSKY, B. A., TOBIAS, M. J., HORN, D. R., AND CHU, D. P. 2003. Investigating occlusion and discretization problems in image space blurring techniques. In First International Conference on Vision, Video, and Graphics, Bath, Peter Hall and Philip Willis (editors), 97--102.
• 6: BENNETT, A. G., AND RABBETTS, R. B. 1998. Clinical Visual Optics. Butterworth-Heinemann Ltd.
• 7: Mark R. Bolin , Gary W. Meyer, A perceptually based adaptive sampling algorithm, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.299-309, July 1998 [doi>10.1145/280814.280924]
• 8: CAMP, J. J., MAGUIRE, L. J., CAMERON, B. M., AND ROBB, R. A. 1990. A computer model for the evaluation of the effect of corneal topography on optical performance. Am. J. Ophthalmol. 109, 379--386.
• 9: CAMP, J. J., MAGUIRE, L. J., AND ROBB, R. A. 1990. An efficient ray tracing algorithm for modeling visual performance from corneal topography. In First Conference on Visualization in Biomedical Computing, IEEE Computer Society Press, The Institute of Electrical and Electronics Engineers, Inc., 279--285.
• 10: CHIU, K., HERF, M., SHIRLEY, P., SWAMY, S., WANG, C., AND ZIMMERMAN, K. 1993. Spatially non-uniform scaling functions for high contrast images. In Proceedings of Graphics Interface '93, Canadian Information Processing Society, 245--254.
• 11: Robert L. Cook , Thomas Porter , Loren Carpenter, Distributed ray tracing, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.137-145, January 1984
• 12: Mark A. Z. Dippé , Erling Henry Wold, Antialiasing through stochastic sampling, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.69-78, July 1985
• 13: James A. Ferwerda , Sumanta N. Pattanaik , Peter Shirley , Donald P. Greenberg, A model of visual adaptation for realistic image synthesis, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.249-258, August 1996 [doi>10.1145/237170.237262]
• 14: FINE, E. M., AND RUBIN, G. S. 1999. Effects of cataract and scotoma on visual acuity: A simulation study. Optometry and Vision Science 76, 7 (July), 468--473.
• 15: FINE, E. M., AND RUBIN, G. S. 1999. The effects of simulated cataract on reading with normal vision and simulated central scotoma. Vision Research 39, 25, 4274--4285.
• 16: GARCIA, D. D., BARSKY, B. A., AND KLEIN, S. A. 1998. CWhatUC: A visual acuity simulator. In Proceedings of Ophthalmic Technologies VIII, SPIE International Symposium on Biomedical Optics, SPIE, 290--298.
• 17: GARCIA, D. D., BARSKY, B. A., AND KLEIN, S. A. 1998. The OPTICAL project at UC Berkeley: Simulating visual acuity. Medicine Meets Virtual Reality: 6 (Art, Science, Technology: Healthcare (r)Evolution) (January 28--31).
• 18: Daniel Dante Garcia , Brian A. Barsky, Cwhatuc: software tools for predicting, visualizing and simulating corneal visual acuity, 2000
• 19: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 20: GREIVENKAMP, J. E., SCHWEIGERLING, J., MILLER, J. M., AND MELLINGER, M. D. 1995. Visual acuity modeling using optical raytracing of schematic eyes. Am. J. Ophthalmol. 120, 227--240.
• 21: HE, J. C., BURNS, S. A., AND MARCOS, S. 2000. Monochromatic aberrations in the accommodated human eye. Vision Research 40, 41--48.
• 22: Wolfgang Heidrich , Philipp Slusallek , Hans-Peter Seidel, An image-based model for realistic lens systems in interactive computer graphics, Proceedings of the conference on Graphics interface '97, p.68-75, May 1997, Kelowna, British Columbia, Canada
• 23: Aaron Isaksen , Leonard McMillan , Steven J. Gortler, Dynamically reparameterized light fields, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.297-306, July 2000 [doi>10.1145/344779.344929]
• 24: KEATING, M. P. 2002. Geometric, Physical, and Visual Optics, second edition. Butterworth-Heinemann Ltd.
• 25: KLEIN, S. A. 1998. Optimal corneal ablation for eyes with arbitrary Hartmann-Shack aberrations. J. Opt. Soc. Am. A 15, 9, 2580--2588.
• 26: Craig Kolb , Don Mitchell , Pat Hanrahan, A realistic camera model for computer graphics, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.317-324, September 1995 [doi>10.1145/218380.218463]
• 27: Mark E. Lee , Richard A. Redner , Samuel P. Uselton, Statistically optimized sampling for distributed ray tracing, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.61-68, July 1985
• 28: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 29: LIANG, J. 1991. A New Method to Precisely Measure the Wave Aberrations of the Human Eye with a Hartmann-Shack Wavefront Sensor. PhD thesis, Department of Mathematics, Universität Heidelberg.
• 30: LOOS, J., SLUSALLEK, P., AND SEIDEL, H.-P. 1998. Using wavefront tracing for the visualization and optimization of progressive lenses. In Computer Graphics Forum (Eurographics '98 Conference Proceedings, Eurographics, 255--266. held in Lisbon, Portugal, 02--04 September 1998.
• 31: LUBIN, J. 1995. A visual discrimination model for imaging system design and evaluation. In Vision Models for Target Detection and Recognition, E. Peli, Ed., vol. 2. World Scientific Publishing Co., Inc., Washington, DC, 245--357.
• 32: MAGUIE, L. J., ZABEL, R. W., PARKER, P., AND LINDSTROM, R. L. 1991. Topography and raytracing analysis of patients with excellent visual acuity 3 months after excimer laser photorefractive keratectomy for myopia. Refract. Corneal Surg. 8 (March/April), 122--128.
• 33: MAGUIRE, L. J., CAMP, J. J., AND ROBB, R. A. 1992. Informing interested parties of changes in the optical performance of the cornea caused by keratorefractive surgery --- a ray tracing model that tailors presentation of results to fit the level of sophistication of the audience. In SPIE Vol. 1808 Visualization in Biomedical Computing, SPIE, The International Society for Optical Engineering, 601--609.
• 34: MARCOS, S., BURNS, S. A., MORENO-BARRIUSO, E., AND NAVARRO, R. 1999. A new approach to the study of ocular chromatic aberrations. Vision Research 39, 4309--4323.
• 35: Gary W. Meyer , Donald P. Greenberg, Color-Defective Vision and Computer Graphics Displays, IEEE Computer Graphics and Applications, v.8 n.5, p.28-40, September 1988 [doi>10.1109/38.7759]
• 36: Gary W. Meyer, Image synthesis and color vision, Computer graphics techniques: theory and practice, Springer-Verlag New York, Inc., New York, NY, 1990
• 37: MOON, P., AND SPENCER, D. E. 1944. On the Stiles-Crawford effect. J. Opt. Soc. Am. 34, 6, 319--329.
• 38: Sumanta N. Pattanaik , James A. Ferwerda , Mark D. Fairchild , Donald P. Greenberg, A multiscale model of adaptation and spatial vision for realistic image display, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.287-298, July 1998 [doi>10.1145/280814.280922]
• 39: PELI, E. 1996. Test of a model of foveal vision by using simulations. Journal of Optical Society of America 13, 6 (June), 1131--1138.
• 40: Fabio Pellacini , James A. Ferwerda , Donald P. Greenberg, Toward a psychophysically-based light reflection model for image synthesis, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.55-64, July 2000 [doi>10.1145/344779.344812]
• 41: PLATT, B. C., AND SHACK, R. V. 1971. Lenticular Hartmann-screen. Newsletter 5, 15, Optical Science Center, University of Arizona.
• 42: Michael Potmesil , Indranil Chakravarty, Synthetic Image Generation with a Lens and Aperture Camera Model, ACM Transactions on Graphics (TOG), v.1 n.2, p.85-108, April 1982 [doi>10.1145/357299.357300]
• 43: Erik Reinhard , Michael Stark , Peter Shirley , James Ferwerda, Photographic tone reproduction for digital images, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 44: ROKITA, P. 1993. Fast generation of depth-of-field effects in computer graphics. Computers & Graphics 17, 5 (September), 593--595.
• 45: ROKITA, P. 1996. Generating depth-of-field effects in virtual reality applications. IEEE Computer Graphics and Applications 16, 2 (March), 18--21.
• 46: SHINYA, M. 1994. Post-filtering for depth of field simulation with ray distribution buffer. In Proceedings of Graphics Interface '94, Canadian Information Processing Society, 59--66.
• 47: Greg Spencer , Peter Shirley , Kurt Zimmerman , Donald P. Greenberg, Physically-based glare effects for digital images, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.325-334, September 1995 [doi>10.1145/218380.218466]
• 48: THIBOS, L. N., APPLEGATE, R., SCHWIEGERLING, J. T., WEBB, R., AND MEMBERS, V. S. T. 2000. Standards for reporting the optical aberrations of eyes. In Vision Science and its Application, V. Lakshminarayanan, Ed., vol. 35. Optical Society of America -- Trends in Optics and Photonics, 232--244.
• 49: THIBOS, L. N. 2000. Principles of Hartmann-Shack aberrometry. In Vision Science and its Applications, vol. 15. Optical Society of America -- Trends in Optics and Photonics, 163--169.
• 50: Jack Tumblin , Holly Rushmeier, Tone Reproduction for Realistic Images, IEEE Computer Graphics and Applications, v.13 n.6, p.42-48, November 1993 [doi>10.1109/38.252554]
• 51: Steven Dale Upstill, The realistic presentation of synthetic images: image processing in computer graphics, 1985
• 52: Gregory Ward Larson , Holly Rushmeier , Christine Piatko, A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes, IEEE Transactions on Visualization and Computer Graphics, v.3 n.4, p.291-306, October 1997 [doi>10.1109/2945.646233]
• 53: ZERNIKE, F. 1934. Beugungstheorie des schneidenverfahrens und seiner verbesserten form, der phasenkontrastmethode. Physica 1, 689--704.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation

Keywords:
LASIK, Point Spread Function (PSF), blur, human visual system, image synthesis, ophthalmology, optics, optometry, pupil, ray tracing, vision-realistic rendering