A neural network scheme for transparent surface modelling

ABSTRACT

This paper presents a new neural network (NN) scheme for recovering three dimensional (3D) transparent surface. We view the transparent surface modeling, not as a separate problem, but as an extension of opaque surface modeling. The main insight of this work is we simulate transparency not only for generating visually realistic images, but for recovering the object shape. We construct a formulation of transparent surface modeling using ray tracing framework into our NN. We compared this ray tracing method, with a texture mapping method that simultaneously map the silhouette images and smooth shaded images (obtained form our NN), and textured images (obtained from the teacher image) to an initial 3D model. By minimizing the images error between the output images of our NN and the teacher images, observed in multiple views, we refine vertices position of the initial 3D model. We show that our NN can refine the initial 3D model obtained by polarization images and converge into more accurate surface.

REFERENCES

• 1: Yung-Yu Chuang , Douglas E. Zongker , Joel Hindorff , Brian Curless , David H. Salesin , Richard Szeliski, Environment matting extensions: towards higher accuracy and real-time capture, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.121-130, July 2000 [doi>10.1145/344779.344844]
• 2: R. L. Cook , T. Porter , L. Carpenter, Distributed ray tracing, Tutorial: computer graphics; image synthesis, Computer Science Press, Inc., New York, NY, 1988
• 3: Fanany, M. I., and Kumazawa, I. 2003. SA-optimized multiple view smooth polyhedron representation nn. In Discovery Science, 306--310.
• 4: Fanany, M. I., and Kumazawa, I. 2004. Multiple-view shape extraction from shading as local regression by analytic nn scheme. Mathematical and Computer Modelling 40, 9-10, 959--975.
• 5: Mohamad Ivan Fanany , Itsuo Kumazawa, A neural network for recovering 3D shape from erroneous and few depth maps of shaded images, Pattern Recognition Letters, v.25 n.4, p.377-389, March 2004 [doi>10.1016/j.patrec.2003.11.001]
• 6: Fanany, M. I., Ohno, M., and Kumazawa, I. 2002. A Scheme for Reconstructing Face from Shading using Smooth Projected Polygon Representation NN. In Proc. of the IEEEICIP (volume II), 305--308.
• 7: Fanany, M. I., Kobayashi, K., and Kumazawa, I. 2004. A combinatorial transparent surface modeling from polarization images. In IWCIA, 65--76.
• 8: Olivier D. Faugeras , Quang-Tuan Luong , Stephen J. Maybank, Camera Self-Calibration: Theory and Experiments, Proceedings of the Second European Conference on Computer Vision, p.321-334, May 19-22, 1992
• 9: Shape Extraction of Transparent Object Using Genetic Algorithm, Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume IV-Volume 7472, p.684, August 25-29, 1996
• 10: Donald Hearn , M. Pauline Baker, Computer graphics (2nd ed.): C version, Prentice-Hall, Inc., Upper Saddle River, NJ, 1996
• 11: Victoria Interrante , Henry Fuchs , Stephen Pizer, Conveying the 3D Shape of Transparent Surfaces via Texture, Institute for Computer Applications in Science and Engineering (ICASE), 1997
• 12: Wojciech Matusik , Hanspeter Pfister , Remo Ziegler , Addy Ngan , Leonard McMillan, Acquisition and rendering of transparent and refractive objects, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 13: Miyazaki, D., Saito, M., Sato, Y., and Ikeuchi, K. 2002. Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. JOSAA 19, 4 (April), 687--694.
• 14: Daisuke Miyazaki , Masataka Kagesawa , Katsushi Ikeuchi, Polarization-based Transparent Surface Modeling from Two Views, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1381, October 13-16, 2003
• 15: Hiroshi Murase, Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.10, p.1045-1052, October 1992 [doi>10.1109/34.159906]
• 16: Nakayama, K., Shimojo, S., and Ramachandran, V. 1990. Transparency: Relation to depth, subjective contours, luminance and neon color spreading. Perception 19, 497-513, 497--513.
• 17: Rahmann, S., and Canterakis, N. 2001. Reconstruction of specular surfaces using polarization imaging. In CVPR01, I:149--155.
• 18: Saito, M., Kashiwagi, H., Sato, Y., and Ikeuchi, K. 1999. Measurement of surface orientations of transparent objects using polarization in highlight. In CVPR, 1381-.
• 19: Alvy Ray Smith , James F. Blinn, Blue screen matting, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.259-268, August 1996 [doi>10.1145/237170.237263]
• 20: Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In CVPR, 1246-.
• 21: Yonatan Wexler , Andrew. W. Fitzgibbon , Andrew. Zisserman, Image-based environment matting, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 22: Lawrence B. Wolff , David J. Kurlander, Ray Tracing with Polarization Parameters, IEEE Computer Graphics and Applications, v.10 n.6, p.44-55, November 1990 [doi>10.1109/38.62695]
• 23: Wolff, L. 1987. Shape from polarization images. In CVWS87, 79--85.
• 24: Douglas E. Zongker , Dawn M. Werner , Brian Curless , David H. Salesin, Environment matting and compositing, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.205-214, July 1999 [doi>10.1145/311535.311558]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling

Keywords:
3D modeling, neural network, transparent surface