The ModelCraft framework: Capturing freehand annotations and edits to facilitate the 3D model design process using a digital pen

ABSTRACT

Recent advancements in rapid prototyping techniques such as 3D printing and laser cutting are changing the perception of physical 3D models in architecture and industrial design. Physical models are frequently created not only to finalize a project but also to demonstrate an idea in early design stages. For such tasks, models can easily be annotated to capture comments, edits, and other forms of feedback. Unfortunately, these annotations remain in the physical world and cannot easily be transferred back to the digital world. Our system, ModelCraft, addresses this problem by augmenting the surface of a model with a traceable pattern. Any sketch drawn on the surface of the model using a digital pen is recovered as part of a digital representation. Sketches can also be interpreted as edit marks that trigger the corresponding operations on the CAD model. ModelCraft supports a wide range of operations on complex models, from editing a model to assembling multiple models, and offers physical tools to capture free-space input. Several interviews and a formal study with the potential users of our system proved the ModelCraft system useful. Our system is inexpensive, requires no tracking infrastructure or per object calibration, and we show how it could be extended seamlessly to use current 3D printing technology.

REFERENCES

• 1: Agrawala, M., Beers, A. C., and Levoy, M. 1995. 3D painting on scanned surfaces. In Proceedings of the I3D'95. 145--150.
• 2: Anderson, D., Yedidia, J. S., Frankel, J. L., Marks, J., Agarwala, A., Beardsley, P., Leigh, J. H. D., Ryall, K., and Sullivan, E. 2000. Tangible interaction + graphical interpretation: A new approach to 3D modeling. In Proccedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 393--402.
• 3: Anoto. 2002. Development Guide for Service Enabled by Anoto Functionality. Anoto Lund.
• 4: Ching, F. D. K. 1996. Architecture: Form, Space, and Order 2nd Ed. Wiley.
• 5: Google. 2006. SketchUp. http://sketchup.google.com/
• 6: Grasset, R. L., Boissieux, J. D., Gascuel, and Schmalstieg, D. 2005. Ineractive mediated reality. In Proceedings of the 6th Australasian Conference on User Interfaces. 21--29.
• 7: Guimbretiere, F. 2003. Paper augmented digital documents. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'03). 51--60.
• 8: Hanrahan, P. and Haeberli, P. 1990. Direct WYSIWYG painting and texturing on 3D shapes. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 215--223.
• 9: Hart, S. 2005. Building a state-of-the-art home: Part II. Architectural Record Innovation, 24--29.
• 10: Hinckley, K., Baudisch, P., Ramos, G., and Guimbreteire, F. 2005. Design and analysis of delimiters for selection-action pen gesture phrases in sriboli. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'05). 451--460.
• 11: Hinckley, K., Pausch, R., Goble, J. C., and Kassell, N. F. 1994. Passive real-world interface props for neurosurgical visualization. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'94). 452--458.
• 12: Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., and Smith, M. 2004. Stitching: Pen gestures that span multiple displays. In Proceedings of the AVI'04. 23--31.
• 13: Igarashi, T. and Hughes, J. F. 2003. Smooth meshes for sketch-based freeform modeling. In Proceedings of the I3D'03. 139--142.
• 14: Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3D freeform design. In Proccedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 234--241.
• 15: Ishii, H. and Ullmer, B. 1997. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'97). 234--241.
• 16: Jansen, K. and Zhang, G. 2004. On rectangle packing: Maximizing benefits. In Proccedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms. 204--213.
• 17: Jung, T., Gross, M. D., and Do, E. Y.-L. 2002. Sketching annotations in a 3D Web environment. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'02) Extended Abstracts. 618--619.
• 18: Liao, C., Guimbretiere, F., and Hinckley, K. 2005. PapierCraft: A command system for interactive paper. In Proccedings of the ACM Symposium on User Interface Software and Technology (UIST'05). 241--244.
• 19: Liao, C., Guimbretiere, F., and Loeckenhoff, C. E. 2006. Pen-Top feedback for paper-based interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'06). 201--210.
• 20: Liu, X. 2004. Editing digital models using physical materials. Master thesis, University of Toronto.
• 21: LiveScribe. LiveScribe homepage. http://www.livescribe.com/
• 22: LogiTech. 2005. IO digital pen. http://www.logitech.com
• 23: Lynggard, S. and Pettersson, M. P. 2005. Devices method and computer program for position determination. U.S. Patent Office, Anoto AB: USA.
• 24: Markosian, L., Cohen, J. M., Crulli, T., and Hughes, J. 1999. Skin: A constructive approach to modeling free-form shapes. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 393--400.
• 25: Masry, M., Kang, D., and Lipson, H. 2005. A pen-based freehand sketching interface for progressive construction of 3D objects. Comput. Graph. 29, 563--575.
• 26: Mitani, J. and Suzuki, H. 2004. Making papercraft toys from meshes using strip-based approximate unfolding. ACM Trans. Graph. 23, 3, 259--263.
• 27: Petrie, G. W. and Hecht, D. L. 1999. Parallel propagating embedded binary sequences for characterizing objects in N-dimensional address space. U.S. Patent Office, Xerox Corporation.
• 28: Piper, B., Ratti, C., and Ishii, H. 2002. Illuminating clay: A 3-D tangible interface for landscape analysis. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'02). 355--362.
• 29: Polthier, K. 2003. Imaging maths—Unfolding polyhedra. Plus Mag. 27.
• 30: Schkolne, S., Pruett, M., and Schroder, P. 2001. Surface drawing: Creating organic 3D shapes with the hand and tangible tools. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'01). 261--268.
• 31: Sekendur, O. F. 1998. Absolute optical polistion determination. U.S. Patent Office.
• 32: Sharlin, E., Itoh, Y., Watson, B., Kitamura, Y., Sutphen, S., and Liu, L. 2002. Cognitive cubes: A tangible user interface for cognitive assessment. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'02). 347--354.
• 33: Sheng, J., Balakrishnan, R., and Singh, K. 2006. An interface for virtual 3D sculpting via physical proxy. In Proceedings of the GRAPHITE'06. 213--220.
• 34: Signer, B. and Norrie, M. C. 2007. PaperPoint: A paper-based presentation and interactive paper prototyping tool. In Proceedings of the Conference on Tangible and Embedded Interaction. 57--64.
• 35: Solid Concepts Inc. 2004. SolidView. http://www.solidview.com/
• 36: SolidWorks. 2005. SolidWorks homepage. http://www.solidworks.com/
• 37: Song, H., Guimbreteire, F., Ambrose, M., and Lostritto, C. 2007. CubeExplorer: An evaluation of interaction techniques in architectural education. In Proceedings of the INTERACT Socially Responsible Interaction Conference. 43--56.
• 38: Song, H., Guimbretiere, F., Lipson, H., and Changhu. 2006. ModelCraft: Capturing freehand annotations and edits on physical 3D models. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'06). 13--22.
• 39: Steinberg, A. 1997. A strip-packing algorithm with absolute performance bound 2. SIAM J. Comput. 26, 2, 401--409.
• 40: Suzuki, H. and Kato, H. 1995. Interaction-level support for collaborative learning: AlgoBlock—an open programming language. In Proceedings of the 1st International Conference on Computer Support for Collaborative Learning (CSCL). 349--355.
• 41: Ullmer, B. and Ishii, H. 1997. The metaDESK: Models and prototypes for tangible user interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'97). 223--232.
• 42: Underkoffler, J. and Ishii, H. 1999. Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'99). 386--393.
• 43: Underkoffler, H. and Ishii, H. 1998. Illuminating light: An optical design tool with a luminous-tangible interface. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'98). 542--549.
• 44: Watanabe, R., Itoh, Y., Asai, M., Kitamura, Y., Kishino, F., and Kikuchi, H. 2004. The soul of ActiveCube: Implementing a flexible, multimodal, three-dimensional spatial tangible interface. In Proceedings of the ACE'04. 173--180.
• 45: ZCorp. 2005. ZCorp 3D printing system. http://www.zcorp.com/en/Products/3D-Printers/Spectrum-Z510/spage.aspx
• 46: Zeleznik, R., Miller, T., Holden, L., and LaViola, J. J. 2004. Fluid inking: An inclusive approach to integrating inking and gestures. Tech. rep. CS-04-11, Department of Computer Science, Brown University.
• 47: Zeleznik, R. C. and Herndon, K. P. 1996. SKETCH: An interface for sketching 3D scenes. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 163--170.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Experimentation, Human Factors

Keywords:
Pen-based interactions, rapid prototyping, tangible interactions