Image-based procedural modeling of facades

ABSTRACT

This paper describes algorithms to automatically derive 3D models of high visual quality from single facade images of arbitrary resolutions. We combine the procedural modeling pipeline of shape grammars with image analysis to derive a meaningful hierarchical facade subdivision. Our system gives rise to three exciting applications: urban reconstruction based on low resolution oblique aerial imagery, reconstruction of facades based on higher resolution ground-based imagery, and the automatic derivation of shape grammar rules from facade images to build a rule base for procedural modeling technology.

REFERENCES

• 1: Alegre, F., and Dellaert, F. 2004. A probabilistic approach to the semantic interpretation of building facades. In International Workshop on Vision Techniques Applied to the Rehabilitation of City Centres.
• 2: Bekins, D., and Aliaga, D. 2005. Build-by-number: Rearranging the real world to visualize novel architectural spaces. In IEEE Visualization.
• 3: Brenner, C., and Ripperda, N. 2006. Extraction of facades using rjM-CMC and constraint equations. In Photogrammetric Computer Vision, 155--160.
• 4: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 5: Dick, A., Torr, P., Ruffle, S., and Cipolla, R. 2001. Combining single view recognition and multiple view stereo for architectural scenes. In ICCV, IEEE Computer Society, Los Alamitos, CA, 268--274.
• 6: A. R. Dick , P. H. S. Torr , R. Cipolla, Modelling and Interpretation of Architecture from Several Images, International Journal of Computer Vision, v.60 n.2, p.111-134, November 2004 [doi>10.1023/B:VISI.0000029665.07652.61]
• 7: Duarte, J. 2002. Malagueira Grammar --- towards a tool for customizing Alvaro Siza's mass houses at Malagueira. PhD thesis, MIT School of Architecture and Planning.
• 8: Flemming, U. 1987. More than the sum of its parts: the grammar of queen anne houses. Environment and Planning B 14, 323--350.
• 9: Früh, C., and Zakhor, A. 2001. 3D model generation for cities using aerial photographs and ground level laser scans. In CVPR, IEEE Computer Society, 31--38.
• 10: Christian Früh , Avideh Zakhor, Constructing 3D City Models by Merging Aerial and Ground Views, IEEE Computer Graphics and Applications, v.23 n.6, p.52-61, November 2003 [doi>10.1109/MCG.2003.1242382]
• 11: Feng Han , Song-Chun Zhu, Bottom-up/Top-Down Image Parsing by Attribute Graph Grammar, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1778-1785, October 17-20, 2005 [doi>10.1109/ICCV.2005.50]
• 12: Havemann, S. 2005. Generative Mesh Modeling. PhD thesis, TU Braunschweig.
• 13: Jinhui Hu , Suya You , Ulrich Neumann, Integrating LiDAR, Aerial Image and Ground Images for Complete Urban Building Modeling, Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), p.184-191, June 14-16, 2006 [doi>10.1109/3DPVT.2006.82]
• 14: Jepson, W., Liggett, R., and Friedman, S. 1996. Virtual modeling of urban environments. PRESENCES 5, 1, 72--86.
• 15: Nebojsa Jojic , Brendan J. Frey , Anitha Kannan, Epitomic analysis of appearance and shape, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.34, October 13-16, 2003
• 16: Karner, K., Bauer, J., Klaus, A., Leberl, F., and Grabner, M. 2001. Virtual habitat: Models of the urban outdoors. In Third International Workshop on Automatic Extraction of Man-Made Objects from Aerial and Space Imaging, 393--402.
• 17: Koning, H., and Eizenberg, J. 1981. The language of the prairie: Frank lloyd wrights prairie houses. Environment and Planning B 8, 295--323.
• 18: Kullback, S. 1959. Information theory and statistics. John Wiley and Sons., New York.
• 19: Sung Chun Lee , Ram. Nevatia, Interactive 3D Building Modeling Using a Hierarchical Representation, Proceedings of the First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, p.58, October 17-17, 2003
• 20: Lee, S. C., and Nevatia, R. 2004. Extraction and integration of window in a 3D building model from ground view image. In CVPR, IEEE Computer Society, 113--120.
• 21: Lee, S. C., Jung, S. K., and Nevatia, R. 2002. Automatic integration of facade textures into 3D building models with a projective geometry based line clustering. Computer Graphics Forum 21, 3 (Sept.), 511--519.
• 22: Justin Legakis , Julie Dorsey , Steven Gortler, Feature-based cellular texturing for architectural models, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.309-316, August 2001 [doi>10.1145/383259.383293]
• 23: D. Liebowitz , A. Zisserman, Metric Rectification for Perspective Images of Planes, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.482, June 23-25, 1998
• 24: Marvie, J.-E., Perret, J., and Bouatouch, K. 2005. The FL-system: a functional L-system for procedural geometric modeling. The Visual Computer 21, 5, 329--339.
• 25: Radomír Měch , Przemyslaw Prusinkiewicz, Visual models of plants interacting with their environment, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.397-410, August 1996 [doi>10.1145/237170.237279]
• 26: Pascal Müller , Peter Wonka , Simon Haegler , Andreas Ulmer , Luc Van Gool, Procedural modeling of buildings, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 27: Yoav I. H. Parish , Pascal Müller, Procedural modeling of cities, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.301-308, August 2001 [doi>10.1145/383259.383292]
• 28: P. Prusinkiewicz , Aristid Lindenmayer, The algorithmic beauty of plants, Springer-Verlag New York, Inc., New York, NY, 1990
• 29: Przemyslaw Prusinkiewicz , Mark James , Radomír Měch, Synthetic topiary, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.351-358, July 1994 [doi>10.1145/192161.192254]
• 30: Przemyslaw Prusinkiewicz , Lars Mündermann , Radoslaw Karwowski , Brendan Lane, The use of positional information in the modeling of plants, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.289-300, August 2001 [doi>10.1145/383259.383291]
• 31: Ravi Ramamoorthi , James Arvo, Creating generative models from range images, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.195-204, July 1999 [doi>10.1145/311535.311557]
• 32: Realviz, 2007. Realviz ImageModeler V4.0 product information. http://www.realviz.com.
• 33: William Ribarsky , Tony Wasilewski , Nickolas Faust, From Urban Terrain Models to Visible Cities, IEEE Computer Graphics and Applications, v.22 n.4, p.10-15, July 2002 [doi>10.1109/MCG.2002.1016692]
• 34: Konrad Schindler , Joachim Bauer, A Model-Based Method For Building Reconstruction, Proceedings of the First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, p.74, October 17-17, 2003
• 35: Stiny, G., and Mitchell, W. J. 1978. The palladian grammar. Environment and Planning B 5, 5--18.
• 36: Stiny, G. 1975. Pictorial and Formal Aspects of Shape and Shape Grammars. Birkhauser Verlag, Basel.
• 37: Stiny, G. 1982. Spatial relations and grammars. Environment and Planning B 9, 313--314.
• 38: Takase, Y., Sho, N., Sone, A., and Shimiya, K. 2003. Automatic generation of 3d city models and related applications. In International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 113--120.
• 39: Wang, X., Totaro, S., Taillandier, F., Hanson, A., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Proc. ISPRS Commission III Symposium on Photogrammetric Computer Vision, 381--386.
• 40: Lu Wang , Suya You , Ulrich Neumann, Large-Scale Urban Modeling by Combining Ground Level Panoramic and Aerial Imagery, Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), p.806-813, June 14-16, 2006 [doi>10.1109/3DPVT.2006.87]
• 41: Wells, W., Viola, P., Atsumi, H., Nakajima, S., and Kikinis, R., 1996. Multi-modal volume registration by maximization of mutual information.
• 42: Peter Wonka , Michael Wimmer , François Sillion , William Ribarsky, Instant architecture, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.4 MATHEMATICAL LOGIC AND FORMAL LANGUAGES
      F.4.2 Grammars and Other Rewriting Systems

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
      I.3.7 Three-Dimensional Graphics and Realism
  I.6 SIMULATION AND MODELING
      I.6.3 Applications

Keywords:
architecture, design computation, image-based modeling, procedural modeling, urban reconstruction