Towards the next generation of 3D content creation

ABSTRACT

In this paper we present a novel integrated 3D editing environment that combines recent advantages in various fields of computer graphics, such as shape modelling, video-based Human Computer Interaction, force feedback and VR fine-manipulation techniques. This integration allows us to create a new compelling form of 3D object creation and manipulation preserving the metaphors designers, artists and painters have accustomed to during their day to day practice. Our system comprises a novel augmented reality workbench and enables users to simultaneously perform natural fine pose determination of the edited object with one hand and model or paint the object with the other hand. The hardware setup features a non-intrusive, video-based hand tracking subsystem, see-through glasses and a 3D 6-degree of freedom input device. The possibilities delivered by our AR workbench enable us to implement traditional and recent editing metaphors in an immersive and fully three-dimensional environment, as well as to develop novel approaches to 3D object interaction.

REFERENCES

• 1: Maneesh Agrawala , Andrew C. Beers , Ian McDowall , Bernd Fröhlich , Mark Bolas , Pat Hanrahan, The two-user Responsive Workbench: support for collaboration through individual views of a shared space, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.327-332, August 1997 [doi>10.1145/258734.258875]
• 2: Marc Alexa, Linear combination of transformations, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 3: V. Athitsos and S. Sclaroff. An appearance-based framework for 3D hand shape classification and camera viewpoint estimation. Technical Report 2001-022, 22 2001.
• 4: Alan H. Barr, Global and local deformations of solid primitives, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.21-30, January 1984
• 5: G. H. Bendels , R. Klein, Mesh forging: editing of 3D-meshes using implicitly defined occluders, Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing, June 23-25, 2003, Aachen, Germany
• 6: G. H. Bendels, R. Klein, and A. Schilling. Image and 3d-object editing with precisely specified editing regions. In T. Ertl, B. Girod, G. Greiner, H. Niemann, H.-P. Seidel, E. Steinbach, and R. Westermann, editors, Vision, Modeling and Visualisation 2003, pages 451--460. Akademische Verlagsgesellschaft Aka GmbH, Berlin, November 2003.
• 7: Henning Biermann , Ioana Martin , Fausto Bernardini , Denis Zorin, Cut-and-paste editing of multiresolution surfaces, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 8: O. Bimber. Gesture controlled object interaction: A virtual table case-study. In V. Skala, editor, WSCG'99 Conference Proceedings, 1999.
• 9: Sabine Coquillart, Extended free-form deformation: a sculpturing tool for 3D geometric modeling, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.187-196, September 1990, Dallas, TX, USA
• 10: William M. Hsu , John F. Hughes , Henry Kaufman, Direct manipulation of free-form deformations, ACM SIGGRAPH Computer Graphics, v.26 n.2, p.177-184, July 1992
• 11: S.-M. Hu, H. Zhang, C.-L. Tai, and J.-G. Sun. Direct manipulation of ffd: efficient explicit solutions and decomposible multiple point constraints. The Visual Computer, 17, 2001.
• 12: R. Klein. 3d mesh-editing. In Dagstuhl Seminar: Image Synthesis and Interactive 3D Graphics, volume 25. Michael Cohen, Heinrich Mueller, Claude Puech, Hans-Peter Seidel, June 2000 -- http://cg.cs.unibonn.de/docs/publications/2000/dagstuhlpresentation.pdf.
• 13: Leif Kobbelt , Swen Campagna , Jens Vorsatz , Hans-Peter Seidel, Interactive multi-resolution modeling on arbitrary meshes, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.105-114, July 1998 [doi>10.1145/280814.280831]
• 14: W. Krger, C.-A. Bohn, B. Frhlich, H. Schth, W. Strauss, and G. Wesche. The responsive workbench, 1995.
• 15: Ignacio Llamas , Byungmoon Kim , Joshua Gargus , Jarek Rossignac , Chris D. Shaw, Twister: a space-warp operator for the two-handed editing of 3D shapes, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 16: F. P. M. Koutek. The Responsive Workbench Simulator: a Tool for Application Development and Analysis. In V. Skala, editor, Journal of WSCG 2002, pages 255--262, February 2002.
• 17: Y. S. N. Shimada, K. Kimura. Real-time hand posture estimation based on 2d-appearance retreaval using monocular camera, 2001.
• 18: H. Ouhaddi and P. Horain. Hand tracking by 3d model registration, 1999.
• 19: J. M. Rehg , T. Kanade, Model-based tracking of self-occluding articulated objects, Proceedings of the Fifth International Conference on Computer Vision, p.612, June 20-23, 1995
• 20: R. Rosales, V. Athitsos, L. Sigal, and S. Sclaroff. 3d hand pose estimation using specialized mappings. Technical report, Los Alamitos, CA, July 9--12 2001.
• 21: J. Rossignac and J. J. Kim. Computing and visualizing pose-interpolating 3d motions. Computer Aided Design, 33(4):279--291, April 2001.
• 22: A. Seay, D. Krum, W. Ribarsky, and L. Hodges. Multimodal interaction techniques for the virtual workbench.
• 23: Thomas W. Sederberg , Scott R. Parry, Free-form deformation of solid geometric models, Proceedings of the 13th annual conference on Computer graphics and interactive techniques, p.151-160, August 1986
• 24: T. Starner and et al. The perceptive workbench: Computer-vision-based gesture tracking, object tracking, and 3d reconstruction for augmented desks, 2003.
• 25: B. Stenger, P. R. S. Mendonça, and R. Cipolla. Model-based hand tracking using an unscented kalman filter. In Proc. British Machine Vision Conference, volume I, pages 63--72, Manchester, UK, September 2001.
• 26: A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla. Learning a kinematic prior for tree-based filtering. In British Machine Vision Conference, 2003.
• 27: Brygg Ullmer , Hiroshi Ishii, The metaDESK: models and prototypes for tangible user interfaces, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.223-232, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263551]
• 28: John Underkoffler , Hiroshi Ishii, Illuminating light: an optical design tool with a luminous-tangible interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.542-549, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274717]
• 29: v Pol, R. William, R. Larry, and H. Frits. Interaction in semi-immersive large display environments, 1999.
• 30: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]
• 31: Y. Wu, J. Lin, and T. Huang. Capturing natural hand articulation. pages 426--432, 2001.
• 32: www.sensable.com.
• 33: Denis Zorin , Peter Schröder , Wim Sweldens, Interactive multiresolution mesh editing, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.259-268, August 1997 [doi>10.1145/258734.258863]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

Keywords:
AR, HCI, augmented reality, human computer interaction, mesh-editing