Snap and share your photobooks

ABSTRACT

The sharing of photos with others, friends and family, has become even more popular with digital photography and Internet applications such as email or Web albums. At the same time, the physical touch of printed photos is still appreciated and customers use different services to print their photos on post cards, calendars or photobooks, often to give them as a present or to create a physical souvenir. Once printed, however, the sharing of photos with others becomes difficult as there is no link back from the now physical item to its digital counterpart. With Bookmarkr, we developed a system that employs a mobile camera phone to bridge the gap between the printed photo and its digital counterpart. The user takes an image of a photo in a photobook with the mobile phone's camera. The image is transmitted to a photobook server, which employs image analysis techniques to retrieve the corresponding photo. The photo is sent back to the user and establishes the digital-physical match. Bookmarkr allows the user to interact with printed photos in a similar way as interacting with digital photos by using a point-and-shoot metaphor. Our performance evaluation shows that the system is able to return the correct photo in up to 99% of all cases. A conducted user study revealed that the designed interaction is suitable for the participants and their printed photobooks.

REFERENCES

• 1: E. Auchard. Flickr to map the world's latest photo hotspots. http://www.reuters.com/article/technologyNews/idUS/HO94233920071119, November 2007.
• 2: Marko Balabanović , Lonny L. Chu , Gregory J. Wolff, Storytelling with digital photographs, Proceedings of the SIGCHI conference on Human factors in computing systems, p.564-571, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332505]
• 3: Rafael Ballagas , Jan Borchers , Michael Rohs , Jennifer G. Sheridan, The Smart Phone: A Ubiquitous Input Device, IEEE Pervasive Computing, v.5 n.1, p.70, January 2006 [doi>10.1109/MPRV.2006.18]
• 4: Rafael Ballagas , Michael Rohs , Jennifer G. Sheridan, Sweep and point and shoot: phonecam-based interactions for large public displays, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056876]
• 5: H. Bay, T. Tuytelaars, and L. V. Gool. Surf: Speeded up robust features. Proceedings of the European Conference on Computer Vision, 1:404--417, 2006.
• 6: Jeffrey S. Beis , David G. Lowe, Shape Indexing Using Approximate Nearest-Neighbour Search in High-Dimensional Spaces, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.1000, June 17-19, 1997
• 7: CeWe Color Holding AG. Cewe color holding ag factbook, November 2007.
• 8: CeWe Color Holding AG. Cewe color achieves projected earnings for 2007. Press release, April 2008.
• 9: Scott Counts , Eric Fellheimer, Supporting social presence through lightweight photo sharing on and off the desktop, Proceedings of the SIGCHI conference on Human factors in computing systems, p.599-606, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985768]
• 10: Andy Crabtree , Tom Rodden , John Mariani, Collaborating around collections: informing the continued development of photoware, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031673]
• 11: B. Erol and J. J. Hull. Linking presentation documents using image analysis. Proceedings of the Conference on Signals, Systems, and Computers, 1, 2003.
• 12: Xin Fan , Xing Xie , Zhiwei Li , Mingjing Li , Wei-Ying Ma, Photo-to-search: using multimodal queries to search the web from mobile devices, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, November 10-11, 2005, Hilton, Singapore [doi>10.1145/1101826.1101851]
• 13: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 14: David Frohlich , Allan Kuchinsky , Celine Pering , Abbe Don , Steven Ariss, Requirements for photoware, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587102]
• 15: N. Henze, R. Reiners, X. Righetti, E. Rukzio, and S. Boll. Using camera-equipped mobile phones for interacting with real-world objects. Proceedings of the Computer Graphics International Conference, 2008.
• 16: R. Hess. Sift feature detector. http://web.engr.oregonstate.edu/~hess, 2008.
• 17: O. Hilliges, M. Wagner, L. Terrenghi, and A. Butz. The living-room: browsing, organizing and presenting digital image collections in interactive environments. Proceedings of the Conference on Intelligent Environments, pages 552--559, 2007.
• 18: Nancy Van House , Marc Davis , Morgan Ames , Megan Finn , Vijay Viswanathan, The uses of personal networked digital imaging: an empirical study of cameraphone photos and sharing, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057039]
• 19: Intel Corporation. Open source computer vision library website. http://www.intel.com/technology/computing/opencv, 2008.
• 20: International Organization for Standardization: Information Technology. Automatic identification and data capture techniques - bar code symbology - qr code. ISO/IEC 18004, 2000.
• 21: Y. Ke and R. Sukthankar. PCA-SIFT: A more distinctive representation for local image descriptors. Proceedings of the Conference on Computer Vision and Pattern Recogniton, 2:506--513, 2004.
• 22: David Kirk , Abigail Sellen , Carsten Rother , Ken Wood, Understanding photowork, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124885]
• 23: K. Leichtenstern, A. D. Luca, and E. Rukzio. Analysis of Built-in Mobile Phone Sensors for Supporting Interactions with the Real World. Proceedings of the Pervasive Mobile Interaction Devices workshop, pages 31--34, 2005.
• 24: Qiong Liu , Paul McEvoy , Cheng-Jia Lai, Mobile camera supported document redirection, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180811]
• 25: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 26: Andrew D. Miller , W. Keith Edwards, Give and take: a study of consumer photo-sharing culture and practice, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240682]
• 27: Nokia Corporation. Nokia N95 8GB technical specifications. http://www.forum.nokia.com/devices/N95 8GB, 2008.
• 28: M. Nunes, S. Greenberg, and C. Neustaedter. Sharing digital photographs in the home through physical mementos, souvenirs, and keepsakes. Research Report 2007-875-27, Dept Computer Science, University of Calgary, 2007.
• 29: D. Okabe. Emergent Social Practices, Situations and Relations through Everyday Camera Phone Use. Proceedings of the Conference on Mobile Communication, 2004.
• 30: Photo Marketing Association International. Pma photo book report, 2006.
• 31: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 32: M. Rohs and B. Gfeller. Using camera-equipped mobile phones for interacting with real-world objects. Proceedings of the Conference on Pervasive Computing (PERVASIV 2004), pages 265--271, 2004.
• 33: M. Rohs, J. Schöning, A. Krüger, and B. Hecht. Towards Real-Time Markerless Tracking of Magic Lenses on Paper Maps. Adjunct Proceedings of the 5th International Conference on Pervasive Computing, pages 69--72, 2007.
• 34: E. Rukzio, G. Broll, K. Leichtenstern, and A. Schmidt. Mobile Interaction with the Real World: An Evaluation and Comparison of Physical Mobile Interaction Techniques. Proceedings of the European Conference on Ambient Intelligence, pages 1--18, 2007.
• 35: K. Tanaka. Mechanisms of visual object recognition: monkey and human studies. Current Opinion in Neurobiology, 7(4):523--529, 1997.
• 36: Roy Want , Kenneth P. Fishkin , Anuj Gujar , Beverly L. Harrison, Bridging physical and virtual worlds with electronic tags, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.370-377, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303111]
• 37: Y. Zhou, X. Fan, X. Xie, Y. Gong, and W.-Y. Ma. Inquiring of the Sights from the Web via Camera Mobiles. Proceedings of the IEEE International Conference on Multimedia and Expo, pages 661--664, 2006.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Algorithms, Design, Experimentation, Human Factors

Keywords:
contextual bookmark, image analysis, mobile interaction, photo sharing, photobook