Generating photo manipulation tutorials by demonstration

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Generating photo manipulation tutorials by demonstration

Full text

Pdf (15.59 MB)

Source

ACM Transactions on Graphics (TOG) archive
Volume 28 , Issue 3 (August 2009) table of contents
Proceedings of ACM SIGGRAPH 2009

SESSION: Interacting with hands, eyes, and images table of contents

Article No. 66

Year of Publication: 2009

ISSN:0730-0301

Also published in ...

Authors

Floraine Grabler	University of California, Berkeley
Maneesh Agrawala	University of California, Berkeley
Wilmot Li	Adobe Systems
Mira Dontcheva	Adobe Systems
Takeo Igarashi	University of Tokyo and JST ERATO

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 55, Downloads (12 Months): 164, Citation Count: 0

Additional Information:

abstract references index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1531326.1531372 What is a DOI?

ABSTRACT

We present a demonstration-based system for automatically generating succinct step-by-step visual tutorials of photo manipulations. An author first demonstrates the manipulation using an instrumented version of GIMP that records all changes in interface and application state. From the example recording, our system automatically generates tutorials that illustrate the manipulation using images, text, and annotations. It leverages automated image labeling (recognition of facial features and outdoor scene structures in our implementation) to generate more precise text descriptions of many of the steps in the tutorials. A user study comparing our automatically generated tutorials to hand-designed tutorials and screen-capture video recordings finds that users are 20--44% faster and make 60--95% fewer errors using our tutorials. While our system focuses on tutorial generation, we also present some initial work on generating content-dependent macros that use image recognition to automatically transfer selection operations from the example image used in the demonstration to new target images. While our macros are limited to transferring selection operations we demonstrate automatic transfer of several common retouching techniques including eye recoloring, whitening teeth and sunset enhancement.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Maneesh Agrawala , Doantam Phan , Julie Heiser , John Haymaker , Jeff Klingner , Pat Hanrahan , Barbara Tversky, Designing effective step-by-step assembly instructions, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
	2	Soonmin Bae , Sylvain Paris , Frédo Durand, Two-scale tone management for photographic look, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
	3	Lawrence Bergman , Vittorio Castelli , Tessa Lau , Daniel Oblinger, DocWizards: a system for authoring follow-me documentation wizards, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095067]
	4	Michael Bolin , Matthew Webber , Philip Rha , Tom Wilson , Robert C. Miller, Automation and customization of rendered web pages, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095062]
	5	Booher, H. 1975. Relative comprehensibility of pictoral information and printed words in proceduralized instructions. In Human Factors, vol. 17, 266--277.
	6	Allen Cypher , Daniel C. Halbert , David Kurlander , Henry Lieberman , David Maulsby , Brad A. Myers , Alan Turransky, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
	7	Alexei A. Efros , William T. Freeman, Image quilting for texture synthesis and transfer, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.341-346, August 2001 [doi>10.1145/383259.383296]
	8	S. K. Feiner, A grid-based approach to automating display layout, Proceedings on Graphics interface '88, p.192-197, December 1989, Edmonton, Alberta, Canada
	9	Susan M. Harrison, A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.82-89, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223915]
	10	Julie Heiser , Doantam Phan , Maneesh Agrawala , Barbara Tversky , Pat Hanrahan, Identification and validation of cognitive design principles for automated generation of assembly instructions, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989917]
	11	Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 [doi>10.1145/383259.383295]
	12	Derek Hoiem , Alexei A. Efros , Martial Hebert, Geometric Context from a Single Image, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.654-661, October 17-20, 2005 [doi>10.1109/ICCV.2005.107]
	13	Huang, G., Ramesh, M., Berg, T., and Learned-Miller, E. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. UMass, Amherst Technical Report 07--49.
	14	Barry Huggins, Photoshop Retouching Cookbook for Digital Photographers (Cookbooks (O'Reilly)), O'Reilly Media, Inc., 2005
	15	Charles Jacobs , Wilmot Li , Evan Schrier , David Bargeron , David Salesin, Adaptive grid-based document layout, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
	16	Scott Kelby, Adobe Photoshop Lightroom Book for Digital Photographers,The (Voices That Matter), New Riders Publishing, Thousand Oaks, CA, 2007
	17	Caitlin Kelleher , Randy Pausch, Stencils-based tutorials: design and evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055047]
	18	Kevin Knabe, Apple guide: a case study in user-aided design of online help, Conference companion on Human factors in computing systems, p.286-287, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223677]
	19	David S. Kosbie , Brad A. Myers, A system-wide macro facility based on aggregate events: a proposal, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
	20	David Kurlander , Steven Feiner, A history-based macro by example system, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.99-106, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142633]
	21	Tessa Lau , Lawrence Bergman , Vittorio Castelli , Daniel Oblinger, Sheepdog: learning procedures for technical support, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964464]
	22	Henry Lieberman, Mondrian: a teachable graphical editor, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
	23	Lieberman, H. 2001. Your Wish is My Command: Giving Users the Power to Instruct their Software. Morgan Kaufmann.
	24	Greg Little , Tessa A. Lau , Allen Cypher , James Lin , Eben M. Haber , Eser Kandogan, Koala: capture, share, automate, personalize business processes on the web, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240767]
	25	Meng, C., Yasue, M., Imamiya, A., and Mao, X. 1998. Visualizing histories for selective undo and redo. 459--464.
	26	Francesmary Modugno , Brad A. Myers, Pursuit: graphically representing programs in a demonstrational visual shell, Conference companion on Human factors in computing systems, p.455-456, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260464]
	27	Toshio Nakamura , Takeo Igarashi, An application-independent system for visualizing user operation history, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA [doi>10.1145/1449715.1449721]
	28	Novick, L. R., and Morse, D. L. 2000. Folding a fish, making a mushroom: The role of diagrams in executing assembly procedures. Memory and Cognition 28, 7, 1242--56.
	29	Susan Palmiter , Jay Elkerton, An evaluation of animated demonstrations of learning computer-based tasks, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.257-263, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108906]
	30	Su, S. 2007. Visualizing, editing, and inferring structure in 2D graphics. In UIST 2007 Doctoral Symposium.
	31	Michael Terry , Matthew Kay , Brad Van Vugt , Brandon Slack , Terry Park, Ingimp: introducing instrumentation to an end-user open source application, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357152]
	32	Varis, L. 2006. Skin. Wiley Publishing.
	33	Zhou, Y., Gu, L., and Zhang, H. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proc. CVPR, 109--116.