Designing a vision-based mobile interface for in-store shopping

ABSTRACT

Due to the situated nature of mobile applications, designing them requires more emphasis on users' cognitive load and interaction style. Considering that users can only devote limited and fragmented attention to mobile interface when moving between locations, what interaction-styles and services are appropriate to a specific user scenario? To explore this issue for in-store shopping, we designed a vision-based mobile interface for supporting shopper's communicational and organizational requirements on-the-go. With this interface, the physical objects can be automatically recognized by the camera phone in real time, so that shoppers can easily access related internet services. In this paper, we present an ethnographic study from which the design rationale is generated, and a formative evaluation to understand how mobile visual interface can be used in the field. The issues uncovered and lessons learned are applicable to our design improvement. Moreover, we use this to motivate the discussion on vision-based mobile interfaces in general, including embodied interaction and alternate interfaces.

REFERENCES

• 1: SnapTell. http://www.snaptell.com/.
• 2: Gregory D. Abowd , Elizabeth D. Mynatt, Charting past, present, and future research in ubiquitous computing, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.29-58, March 2000 [doi>10.1145/344949.344988]
• 3: Joel Brandt , Noah Weiss , Scott R. Klemmer, txt 4 l8r: lowering the burden for diary studies under mobile conditions, CHI '07 extended abstracts on Human factors in computing systems, April 28-May 03, 2007, San Jose, CA, USA [doi>10.1145/1240866.1240998]
• 4: Adam B. Brody , Edward J. Gottsman, Pocket Bargain Finder: A Handheld Device for Augmented Commerce, Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing, p.44-51, September 27-29, 1999, Karlsruhe, Germany
• 5: Brush, A. J. B., Turner, T. C., Smith, M. A. and Gupta, N. 2005. Scanning Objects in the Wild: Assessing an Object Triggered Information System. in Proc. Ubicomp, Springer 2005, 305--322.
• 6: Chen, W.-C., Xiong, Y., Gao, J., Gelfand, N., Grzeszczuk, R. 2007. Efficient Extraction of Robust Image Features on Mobile Devices. Proc. ISMAR 2007.
• 7: Consolvo, S., Paulos, E. and Smith, I. 2007. Mobile Persuasion for Everyday Behavior Change. Mobile Persuasion, Stanford Captology Media.
• 8: Gregory Cuellar , Dean Eckles , Mirjana Spasojevic, Photos for information: a field study of cameraphone computer vision interactions in tourism, CHI '08 extended abstracts on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1358628.1358838]
• 9: Dunne, A. J. and Raby, F. Design Noir. 2001. The Secret Life of Electronic Objects. Birkhäuser, 2001.
• 10: Andrew E. Fano, Shopper's eye: using location-based filtering for a shopping agent in the physical world, Proceedings of the second international conference on Autonomous agents, p.416-421, May 10-13, 1998, Minneapolis, Minnesota, United States [doi>10.1145/280765.280875]
• 11: Fowler, G. A. 2005. QR codes: In Japan, Billboards Take Code-Crazy Ads to New Heights. Wall Street Journal 10.
• 12: Gao J., S. M., Jacob M., Setlur V., Reponen E., Pulkkinen M., Schloter P., Pulli K. 2007. Intelligent Visual Matching for Providing Context-Aware Information to Mobile Users. UbiComp 2007 Adjunct Proceedings, 68--71.
• 13: Victor M. González , Gloria Mark, "Constant, constant, multi-tasking craziness": managing multiple working spheres, Proceedings of the SIGCHI conference on Human factors in computing systems, p.113-120, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985707]
• 14: Kallio, T. and Kaikkonen, A. 2005. Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing. Journal of Usability Studies 1, 4--6.
• 15: Tim Kindberg , Mirjana Spasojevic , Rowanne Fleck , Abigail Sellen, The Ubiquitous Camera: An In-Depth Study of Camera Phone Use, IEEE Pervasive Computing, v.4 n.2, p.42-50, April 2005 [doi>10.1109/MPRV.2005.42]
• 16: Steinar Kristoffersen , Fredrik Ljungberg, “Making place” to make IT work: empirical explorations of HCI for mobile CSCW, Proceedings of the international ACM SIGGROUP conference on Supporting group work, p.276-285, November 14-17, 1999, Phoenix, Arizona, United States [doi>10.1145/320297.320330]
• 17: Kyoung Jun Lee , Young Hwan Seo, A pervasive comparison shopping business model for integrating offline and online marketplace, Proceedings of the 8th international conference on Electronic commerce: The new e-commerce: innovations for conquering current barriers, obstacles and limitations to conducting successful business on the internet, August 13-16, 2006, Fredericton, New Brunswick, Canada [doi>10.1145/1151454.1151503]
• 18: Young Eun Lee , Izak Benbasat, Interface design for mobile commerce, Communications of the ACM, v.46 n.12, December 2003 [doi>10.1145/953460.953487]
• 19: Paul May, Mobile commerce: opportunities, applications, and technologies of wireless business, Cambridge University Press, New York, NY, 2001
• 20: Miller, D. 1998. A Theory of Shopping. Cornell University Press.
• 21: Erica Newcomb , Toni Pashley , John Stasko, Mobile computing in the retail arena, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642670]
• 22: E. W. T. Ngai , A. Gunasekaran, A review for mobile commerce research and applications, Decision Support Systems, v.43 n.1, p.3-15, February, 2007 [doi>10.1016/j.dss.2005.05.003]
• 23: Christian Monrad Nielsen , Michael Overgaard , Michael Bach Pedersen , Jan Stage , Sigge Stenild, It's worth the hassle!: the added value of evaluating the usability of mobile systems in the field, Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles, p.272-280, October 14-18, 2006, Oslo, Norway [doi>10.1145/1182475.1182504]
• 24: Kenton O'Hara , Mark Perry, User Centred Opportunities for Supporting Consumer Behaviour Through Handheld and Ubiquitous Computing, Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 9, p.293.1, January 06-09, 2003
• 25: Antti Oulasvirta, The fragmentation of attention in mobile interaction, and what to do with it, interactions, v.12 n.6, p.16-18, November + December 2005 [doi>10.1145/1096554.1096555]
• 26: Antti Oulasvirta , Sakari Tamminen , Virpi Roto , Jaana Kuorelahti, Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055101]
• 27: Jason Pascoe , Nick Ryan , David Morse, Using while moving: HCI issues in fieldwork environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.417-437, Sept. 2000 [doi>10.1145/355324.355329]
• 28: Howard Rheingold, Smart Mobs: The Next Social Revolution, Perseus Publishing, 2002
• 29: Schneider, M. 2004. Towards a Transparent Proactive User Interface for a Shopping Assistant. Workshop on Multi-User and Ubiquitous User Interfaces 04.
• 30: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 31: Peter Tarasewich, Designing mobile commerce applications, Communications of the ACM, v.46 n.12, December 2003 [doi>10.1145/953460.953489]
• 32: Paco Underhill , Rick Adamson, Why We Buy: The Science of Shopping, RH Audio, 2007
• 33: Viswanath Venkatesh , V. Ramesh , Anne P. Massey, Understanding usability in mobile commerce, Communications of the ACM, v.46 n.12, December 2003 [doi>10.1145/953460.953488]
• 34: Sean Michael White , Dominic Marino , Steven Feiner, Designing a mobile user interface for automated species identification, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240672]
• 35: Wolfinbarger, M. and Gilly, M. C. 2001. Shopping online for freedom, control, and fun. California Management Review.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
diary study, filed study, formative evaluation, mobile computing, vision-based object recognition