Physically guided liquid surface modeling from videos

ABSTRACT

We present an image-based reconstruction framework to model real water scenes captured by stereoscopic video. In contrast to many image-based modeling techniques that rely on user interaction to obtain high-quality 3D models, we instead apply automatically calculated physically-based constraints to refine the initial model. The combination of image-based reconstruction with physically-based simulation allows us to model complex and dynamic objects such as fluid. Using a depth map sequence as initial conditions, we use a physically based approach that automatically fills in missing regions, removes outliers, and refines the geometric shape so that the final 3D model is consistent to both the input video data and the laws of physics. Physically-guided modeling also makes interpolation or extrapolation in the space-time domain possible, and even allows the fusion of depth maps that were taken at different times or viewpoints. We demonstrated the effectiveness of our framework with a number of real scenes, all captured using only a single pair of cameras.

REFERENCES

• 1: Bradley Atcheson , Ivo Ihrke , Wolfgang Heidrich , Art Tevs , Derek Bradley , Marcus Magnor , Hans-Peter Seidel, Time-resolved 3d capture of non-stationary gas flows, ACM SIGGRAPH Asia 2008 papers, December 10-13, 2008, Singapore
• 2: Kiran S. Bhat , Steven M. Seitz , Jessica K. Hodgins , Pradeep K. Khosla, Flow-based video synthesis and editing, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 3: Derek Bradley , Tiberiu Popa , Alla Sheffer , Wolfgang Heidrich , Tamy Boubekeur, Markerless garment capture, ACM SIGGRAPH 2008 papers, August 11-15, 2008, Los Angeles, California
• 4: Mark Carlson , Peter J. Mucha , R. Brooks Van Horn, III , Greg Turk, Melting and flowing, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545289]
• 5: Edilson de Aguiar , Carsten Stoll , Christian Theobalt , Naveed Ahmed , Hans-Peter Seidel , Sebastian Thrun, Performance capture from sparse multi-view video, ACM SIGGRAPH 2008 papers, August 11-15, 2008, Los Angeles, California
• 6: Geoffrey Egnal , Richard P. Wildes, Detecting Binocular Half-Occlusions: Empirical Comparisons of Five Approaches, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1127-1133, August 2002 [doi>10.1109/TPAMI.2002.1023808]
• 7: Douglas Enright , Stephen Marschner , Ronald Fedkiw, Animation and rendering of complex water surfaces, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 8: Raanan Fattal , Dani Lischinski, Target-driven smoke animation, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 9: Nick Foster , Ronald Fedkiw, Practical animation of liquids, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.23-30, August 2001 [doi>10.1145/383259.383261]
• 10: Nick Foster , Dimitri Metaxas, Realistic animation of liquids, Graphical Models and Image Processing, v.58 n.5, p.471-483, Sept. 1996 [doi>10.1006/gmip.1996.0039]
• 11: Grant, I. 1997. Particle image velocimetry: a review. In Proc. of the Institution of Mechanical Engineers, vol. 211, 55C76.
• 12: Tim Hawkins , Per Einarsson , Paul Debevec, Acquisition of time-varying participating media, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 13: Matthias B. Hullin , Martin Fuchs , Ivo Ihrke , Hans-Peter Seidel , Hendrik P. A. Lensch, Fluorescent immersion range scanning, ACM SIGGRAPH 2008 papers, August 11-15, 2008, Los Angeles, California
• 14: Ivo Ihrke , Bastian Goldluecke , Marcus Magnor, Reconstructing the Geometry of Flowing Water, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1055-1060, October 17-20, 2005 [doi>10.1109/ICCV.2005.202]
• 15: Kanade, T., Rander, P., Vedula, S., and Saito, H. 1999. Virtualized reality: Digitizing a 3d time-varying event as is and in real time. In Mixed Reality, Merging Real and Virtual Worlds. 41--57.
• 16: Antoine McNamara , Adrien Treuille , Zoran Popović , Jos Stam, Fluid control using the adjoint method, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 17: Niloy J. Mitra , Simon Flöry , Maks Ovsjanikov , Natasha Gelfand , Leonidas Guibas , Helmut Pottmann, Dynamic geometry registration, Proceedings of the fifth Eurographics symposium on Geometry processing, July 04-06, 2007, Barcelona, Spain
• 18: Nigel J. W. Morris , Kiriakos N. Kutulakos, Dynamic Refraction Stereo, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1573-1580, October 17-20, 2005 [doi>10.1109/ICCV.2005.79]
• 19: Morris, N. J. W., and Kutulakos, K. N. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. In Proc. of 11th Int. Conf. Computer Vision.
• 20: Long Quan , Ping Tan , Gang Zeng , Lu Yuan , Jingdong Wang , Sing Bing Kang, Image-based plant modeling, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 21: Schneider, R., and Kobbelt, L. 2001. Geometric fairing of irregular meshes for freeform surface design. Computer aided geometric design 18, 359--379.
• 22: Andrei Sharf , Dan A. Alcantara , Thomas Lewiner , Chen Greif , Alla Sheffer , Nina Amenta , Daniel Cohen-Or, Space-time surface reconstruction using incompressible flow, ACM SIGGRAPH Asia 2008 papers, December 10-13, 2008, Singapore
• 23: Shi, J., and Tomasi, C. 1994. Good features to track. In Proc. of CVPR 1994, 593--600.
• 24: Simon, S. V., Baker, S., Seitz, S., and Kanade, T. 2000. Shape and motion carving in 6d. In Computer Vision and Pattern Recognition.
• 25: Sudipta N. Sinha , Drew Steedly , Richard Szeliski , Maneesh Agrawala , Marc Pollefeys, Interactive 3D architectural modeling from unordered photo collections, ACM SIGGRAPH Asia 2008 papers, December 10-13, 2008, Singapore
• 26: Jos Stam, Stable fluids, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.121-128, July 1999 [doi>10.1145/311535.311548]
• 27: Staniforth, A., and Côté, J. 1991. Semi-lagrangian integration schemes for atmospheric models. Monthly Weather Review 119, 9, 2206.
• 28: Jian Sun , Nan-Ning Zheng , Heung-Yeung Shum, Stereo Matching Using Belief Propagation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.7, p.787-800, July 2003 [doi>10.1109/TPAMI.2003.1206509]
• 29: Ping Tan , Gang Zeng , Jingdong Wang , Sing Bing Kang , Long Quan, Image-based tree modeling, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 30: Michael Wand , Philipp Jenke , Qixing Huang , Martin Bokeloh , Leonidas Guibas , Andreas Schilling, Reconstruction of deforming geometry from time-varying point clouds, Proceedings of the fifth Eurographics symposium on Geometry processing, July 04-06, 2007, Barcelona, Spain
• 31: Yichen Wei , Eyal Ofek , Long Quan , Heung-Yeung Shum, Modeling hair from multiple views, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 32: Ryan White , Keenan Crane , D. A. Forsyth, Capturing and animating occluded cloth, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 33: Jianxiong Xiao , Tian Fang , Ping Tan , Peng Zhao , Eyal Ofek , Long Quan, Image-based façade modeling, ACM SIGGRAPH Asia 2008 papers, December 10-13, 2008, Singapore
• 34: Yang, Q., Yang, R., Davis, J., and Nister, D. 2007. Spatial-depth super resolution for range images. In Proc. of CVPR 2007, vol. 0, 1--8.
• 35: C. Lawrence Zitnick , Sing Bing Kang , Matthew Uyttendaele , Simon Winder , Richard Szeliski, High-quality video view interpolation using a layered representation, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 36: C. Lawrence Zitnick , Sing Bing Kang , Matthew Uyttendaele , Simon Winder , Richard Szeliski, High-quality video view interpolation using a layered representation, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
          Subjects: Physically based modeling

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Keywords:
image-based reconstruction, physically-based fluid simulation, space-time model completion