Interactive normal reconstruction from a single image

ABSTRACT

We present an interactive system for reconstructing surface normals from a single image. Our approach has two complementary contributions. First, we introduce a novel shape-from-shading algorithm (SfS) that produces faithful normal reconstruction for local image region (high-frequency component), but it fails to faithfully recover the overall global structure (low-frequency component). Our second contribution consists of an approach that corrects low-frequency error using a simple markup procedure. This approach, aptly called rotation palette, allows the user to specify large scale corrections of surface normals by drawing simple stroke correspondences between the normal map and a sphere image which represents rotation directions. Combining these two approaches, we can produce high-quality surfaces quickly from single images.

REFERENCES

• 1: Agrawal, A., Raskar, R., and Chellappa, R. 2006. What is the range of surface reconstructions from a gradient field? In ECCV06. http://www.umiacs.umd.edu/-aagrawal/eccv06/RangeofSurfaceReconstructions.html.
• 2: Peter N. Belhumeur , David J. Kriegman , Alan L. Yuille, The Bas-Relief Ambiguity, International Journal of Computer Vision, v.35 n.1, p.33-44, Nov. 1999 [doi>10.1023/A:1008154927611]
• 3: A. Criminisi , I. Reid , A. Zisserman, Single View Metrology, International Journal of Computer Vision, v.40 n.2, p.123-148, Nov. 2000 [doi>10.1023/A:1026598000963]
• 4: Hui Fang , John C. Hart, Textureshop: texture synthesis as a photograph editing tool, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 5: Robert T. Frankot , Rama Chellappa, A Method for Enforcing Integrability in Shape from Shading Algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.10 n.4, p.439-451, July 1988 [doi>10.1109/34.3909]
• 6: Yotam Gingold , Denis Zorin, Shading-based surface editing, ACM Transactions on Graphics (TOG), v.27 n.3, August 2008
• 7: Gray, A. 1997. Osculating circles to plane curves. In Modern Differential Geometry of Curves and Surfaces with Mathematica, 2nd ed., Boca Raton, FL: CRC Press, 111--115.
• 8: Derek Hoiem , Alexei A. Efros , Martial Hebert, Automatic photo pop-up, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 9: Berthold K. P. Horn , Michael J. Brooks, Shape from shading, MIT Press, Cambridge, MA, 1989
• 10: Youichi Horry , Ken-Ichi Anjyo , Kiyoshi Arai, Tour into the picture: using a spidery mesh interface to make animation from a single image, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.225-232, August 1997 [doi>10.1145/258734.258854]
• 11: Olga A. Karpenko , John F. Hughes, SmoothSketch: 3D free-form shapes from complex sketches, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 12: Erum Arif Khan , Erik Reinhard , Roland W. Fleming , Heinrich H. Bülthoff, Image-based material editing, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 13: Koenderink, J. 1998. Pictorial relief. Royal 356, 1740, 1071--1086.
• 14: Peter Kovesi, Shapelets Correlated with Surface Normals Produce Surfaces, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.994-1001, October 17-20, 2005 [doi>10.1109/ICCV.2005.224]
• 15: Potetz, B. 2007. Efficient belief propagation for vision using linear constraint nodes. In CVPR 2007: Proceedings of the 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Minneapolis, MN, USA.
• 16: E. Prados , O. Faugeras, Shape from Shading: A Well-Posed Problem?, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.870-877, June 20-26, 2005 [doi>10.1109/CVPR.2005.319]
• 17: Mukta Prasad , Andrew Fitzgibbon, Single View Reconstruction of Curved Surfaces, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1345-1354, June 17-22, 2006 [doi>10.1109/CVPR.2006.281]
• 18: O. Sorkine , D. Cohen-Or , Y. Lipman , M. Alexa , C. Rössl , H.-P. Seidel, Laplacian surface editing, Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, July 08-10, 2004, Nice, France [doi>10.1145/1057432.1057456]
• 19: Tai-Pang Wu , Chi-Keung Tang , Michael S. Brown , Heung-Yeung Shum, ShapePalettes: interactive normal transfer via sketching, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 20: Yizhou Yu , Kun Zhou , Dong Xu , Xiaohan Shi , Hujun Bao , Baining Guo , Heung-Yeung Shum, Mesh editing with poisson-based gradient field manipulation, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 21: Gang Zeng , Yasuyuki Matsushita , Long Quan , Heung-Yeung Shum, Interactive Shape from Shading, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.343-350, June 20-26, 2005 [doi>10.1109/CVPR.2005.194]
• 22: Ruo Zhang , Ping-Sing Tsai , James Edwin Cryer , Mubarak Shah, Shape from Shading: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.8, p.690-706, August 1999 [doi>10.1109/34.784284]
• 23: Zhang, L., Dugas-Phocion, G., Samson, J., and Seitz, S. 2001. Single view modeling of free-form scenes. In CVPR01, I:990--997.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Modeling and recovery of physical attributes

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Shape

General Terms:
Design, Performance