Image-based procedural modeling of facades

ABSTRACT

This paper describes algorithms to automatically derive 3D models of high visual quality from single facade images of arbitrary resolutions. We combine the procedural modeling pipeline of shape grammars with image analysis to derive a meaningful hierarchical facade subdivision. Our system gives rise to three exciting applications: urban reconstruction based on low resolution oblique aerial imagery, reconstruction of facades based on higher resolution ground-based imagery, and the automatic derivation of shape grammar rules from facade images to build a rule base for procedural modeling technology.

REFERENCES

• 1: Alegre, F., and Dellaert, F. 2004. A probabilistic approach to the semantic interpretation of building facades. In International Workshop on Vision Techniques Applied to the Rehabilitation of City Centres.
• 2: Bekins, D., and Aliaga, D. 2005. Build-by-number: Rearranging the real world to visualize novel architectural spaces. In IEEE Visualization.
• 3: Brenner, C., and Ripperda, N. 2006. Extraction of facades using rjM-CMC and constraint equations. In Photogrammetric Computer Vision, 155--160.
• 4: Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings of ACM SIGGRAPH 96, ACM Press, H. Rushmeier, Ed., 11--20.
• 5: Dick, A., Torr, P., Ruffle, S., and Cipolla, R. 2001. Combining single view recognition and multiple view stereo for architectural scenes. In ICCV, IEEE Computer Society, Los Alamitos, CA, 268--274.
• 6: Dick, A. R., Torr, P. H. S., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. International Journal of Computer Vision 60, 2, 111--134.
• 7: Duarte, J. 2002. Malagueira Grammar --- towards a tool for customizing Alvaro Siza's mass houses at Malagueira. PhD thesis, MIT School of Architecture and Planning.
• 8: Flemming, U. 1987. More than the sum of its parts: the grammar of queen anne houses. Environment and Planning B 14, 323--350.
• 9: Früh, C., and Zakhor, A. 2001. 3D model generation for cities using aerial photographs and ground level laser scans. In CVPR, IEEE Computer Society, 31--38.
• 10: Früh, C., and Zakhor, A. 2003. Constructing 3d city models by merging ground-based and airborne views. Computer Graphics and Applications (Nov.), 52--61.
• 11: Han, F., and Zhu, S.-C. 2005. Bottom-up/top-down image parsing by attribute graph grammar. In ICCV, IEEE Computer Society, Washington, DC, USA, 1778--1785.
• 12: Havemann, S. 2005. Generative Mesh Modeling. PhD thesis, TU Braunschweig.
• 13: Hu, J., You, S., and Neumann, U. 2006. Integrating lidar, aerial image and ground images for complete urban building modeling. In 3DPVT.
• 14: Jepson, W., Liggett, R., and Friedman, S. 1996. Virtual modeling of urban environments. PRESENCES 5, 1, 72--86.
• 15: Jojic, N., Frey, B., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In ICCV, IEEE Computer Society, 34--41.
• 16: Karner, K., Bauer, J., Klaus, A., Leberl, F., and Grabner, M. 2001. Virtual habitat: Models of the urban outdoors. In Third International Workshop on Automatic Extraction of Man-Made Objects from Aerial and Space Imaging, 393--402.
• 17: Koning, H., and Eizenberg, J. 1981. The language of the prairie: Frank lloyd wrights prairie houses. Environment and Planning B 8, 295--323.
• 18: Kullback, S. 1959. Information theory and statistics. John Wiley and Sons., New York.
• 19: Lee, S. C., and Nevatia, R. 2003. Interactive 3D building modeling using a hierarchical representation. In International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, IEEE Computer Society, 58--65.
• 20: Lee, S. C., and Nevatia, R. 2004. Extraction and integration of window in a 3D building model from ground view image. In CVPR, IEEE Computer Society, 113--120.
• 21: Lee, S. C., Jung, S. K., and Nevatia, R. 2002. Automatic integration of facade textures into 3D building models with a projective geometry based line clustering. Computer Graphics Forum 21, 3 (Sept.), 511--519.
• 22: Legakis, J., Dorsey, J., and Gortler, S. J. 2001. Feature-based cellular texturing for architectural models. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 309--316.
• 23: Liebowitz, D., and Zisserman, A. 1998. Metric rectification for perspective images of planes. In CVPR, IEEE Computer Society, 482--488.
• 24: Marvie, J.-E., Perret, J., and Bouatouch, K. 2005. The FL-system: a functional L-system for procedural geometric modeling. The Visual Computer 21, 5, 329--339.
• 25: Měch, R., and Prusinkiewicz, P. 1996. Visual models of plants interacting with their environment. In Proceedings of ACM SIGGRAPH 96, ACM Press, H. Rushmeier, Ed., 397--410.
• 26: Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. 2006. Procedural Modeling of Buildings. In Proceedings of ACM SIGGRAPH 2006 / ACM Transactions on Graphics, ACM Press, New York, NY, USA, vol. 25, 614--623.
• 27: Parish, Y. I. H., and Müller, P. 2001. Procedural modeling of cities. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 301--308.
• 28: Prusinkiewicz, P., and Lindenmayer, A. 1991. The Algorithmic Beauty of Plants. Springer Verlag.
• 29: Prusinkiewicz, P., James, M., and Měch, R. 1994. Synthetic topiary. In Proceedings of ACM SIGGRAPH 94, ACM Press, A. Glassner, Ed., 351--358.
• 30: Prusinkiewicz, P., Mündermann, P., Karwowski, R., and Lane, B. 2001. The use of positional information in the modeling of plants. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 289--300.
• 31: Ramamoorthi, R., and Arvo, J. 1999. Creating generative models from range images. In SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 195--204.
• 32: Realviz, 2007. Realviz ImageModeler V4.0 product information. http://www.realviz.com.
• 33: Ribarsky, W., Wasilewski, T., and Faust, N. 2002. From urban terrain models to visible cities. IEEE Computer Graphics & Applications 22, 4, 231--238.
• 34: Schindler, K., and Bauer, J. 2003. A model-based method for building reconstruction. In International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, IEEE Computer Society, 74--82.
• 35: Stiny, G., and Mitchell, W. J. 1978. The palladian grammar. Environment and Planning B 5, 5--18.
• 36: Stiny, G. 1975. Pictorial and Formal Aspects of Shape and Shape Grammars. Birkhauser Verlag, Basel.
• 37: Stiny, G. 1982. Spatial relations and grammars. Environment and Planning B 9, 313--314.
• 38: Takase, Y., Sho, N., Sone, A., and Shimiya, K. 2003. Automatic generation of 3d city models and related applications. In International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 113--120.
• 39: Wang, X., Totaro, S., Taillandier, F., Hanson, A., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Proc. ISPRS Commission III Symposium on Photogrammetric Computer Vision, 381--386.
• 40: Wang, L., You, S., and Neumann, U. 2006. Large-scale urban modeling by combining ground level panoramic and aerial imagery. In 3DPVT.
• 41: Wells, W., Viola, P., Atsumi, H., Nakajima, S., and Kikinis, R., 1996. Multi-modal volume registration by maximization of mutual information.
• 42: Wonka, P., Wimmer, M., Sillion, F., and Ribarsky, W. 2003. Instant architecture. ACM Transactions on Graphics 22, 3, 669--677.

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.4 MATHEMATICAL LOGIC AND FORMAL LANGUAGES
      F.4.2 Grammars and Other Rewriting Systems

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
      I.3.7 Three-Dimensional Graphics and Realism
  I.6 SIMULATION AND MODELING
      I.6.3 Applications

Keywords:
architecture, design computation, image-based modeling, procedural modeling, urban reconstruction