Evaluating example-based search tools

ABSTRACT

A crucial element in consumer electronic commerce is a catalog tool that not only finds the product for the user, but also convinces him that he has made the best choice. To do that, it is important to show him ample choices while keeping his interaction effort below an acceptable limit. Among the various interaction models used in operational e-commerce sites, ranked lists are by far the most popular tool for product navigation and selection. However, as the number of product features and the complexity of user's criteria increase, a ranked list's efficiency becomes less satisfactory. As an alternative, research groups from the intelligent user interface community have developed various example-based search tools, including SmartClient from our laboratory. These tools not only perform personalized search, but also support tradeoff analysis. However, despite the academic interest, example-based search paradigms have not been widely adopted in practice. We have examined the usability of such tools on a variety of tasks involving selection and tradeoff. The studies clearly show that example-based search is comparable to ranked lists on simple tasks, but significantly reduces the error rate and search time when complex tradeoffs are involved. This shows that such tools are likely to be useful particularly for extending the scope of consumer e-commerce to more complex products.

REFERENCES

• 1: Stefano Bistarelli , Ugo Montanari , Francesca Rossi, Semiring-based constraint satisfaction and optimization, Journal of the ACM (JACM), v.44 n.2, p.201-236, March 1997 [doi>10.1145/256303.256306]
• 2: Robin D. Burke , Kristian J. Hammond , Benjamin C. Young, The FindMe Approach to Assisted Browsing, IEEE Expert: Intelligent Systems and Their Applications, v.12 n.4, p.32-40, July 1997 [doi>10.1109/64.608186]
• 3: Carenini G. and Poole D. Constructed Preferences and Value-focused Thinking: Implications for AI research on Preference Elicitation. AAAI-02 Workshop on Preferences in AI and CP: symbolic approaches, Edmonton, Canada, 2002.
• 4: L.F. Cranor, J. Reagle, and M.S. Ackerman. Beyond Concern: Understanding Net Users' Attitudes About Online Privacy. In Ingo Vogelsang and Benjamin M. Compaine (editors). The Internet Upheaval: Raising Questions, Seeking Answers in Communications Policy. The MIT Press, Cambridge, Massachusetts, 2000, 47--70.
• 5: Faltings, B., Torrens, M., and Pu, P. Solution Generation with Qualitative Models of Preferences, International Journal of Computational Intelligence and Applications, 2004. To appear in 2004.
• 6: Josef Fink , Alfred Kobsa, A Review and Analysis of Commercial User Modeling Servers for Personalization on the World Wide Web, User Modeling and User-Adapted Interaction, v.10 n.2-3, p.209-249, 2000 [doi>10.1023/A:1026597308943]
• 7: Keeney, R. Value-Focused Thinking: A Path to Creative Decision Making. Harvard University Press, 1992.
• 8: Linden, G., Hanks, S., and Lesh, N. Interactive assessment of user preference models: The automated travel assistant. Proceedings of User Modeling '97, 1997, 67--78.
• 9: Nguyen, H., and Haddawy, P. The Decision-Theoretic Video Advisor. In workshop notes, Recommender Systems, the Fifteenth National Conference on Artificial Intelligence (AAAI'98), 77--80.
• 10: Payne, J.W., Bettman, J.R., and Johnson, E.J., The Adaptive Decision Maker. Cambridge University Press, 1993.
• 11: Payne, J.W., Bettman, J.R., and Schkade, D.A. Measuring Constructed Preference: Towards a Building Code. Journal of Risk and Uncertainty, 19, 1-3 (1999), 243--270.
• 12: Pearl Pu , Boi Faltings, Enriching buyers' experiences: the SmartClient approach, Proceedings of the SIGCHI conference on Human factors in computing systems, p.289-296, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332446]
• 13: Pu, P., Faltings, B., and Torrens, M. User-Involved Preference Elicitation. In workshop notes, workshop on Configuration, the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI'03), 2003.
• 14: Pu, P., Kumar, P., and Faltings, B. User-Involved Tradeoff Analysis in Configuration Tasks. In workshop notes, the Third International Workshop on User-Interaction in Constraint Satisfaction, Ninth International Conference on Principles and Practice of Constraint Programming (CP2003).
• 15: Pu, P. and Faltings, B. User Preference Elicitation Via Tradeoff. Submitted to the International Journal of Constraints, December, 2003.
• 16: Sybil Shearin , Henry Lieberman, Intelligent profiling by example, Proceedings of the 6th international conference on Intelligent user interfaces, p.145-151, January 14-17, 2001, Santa Fe, New Mexico, United States [doi>10.1145/359784.360325]
• 17: Shimazu, H. ExpertClerk: Navigating Shoppers' Buying Process with the Combination of Asking and Proposing. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI'01) (Seattle, Washington, USA, August 4-10, 2001).
• 18: Sarah Spiekermann , Corina Paraschiv, Motivating Human–Agent Interaction: Transferring Insights from Behavioral Marketing to Interface Design, Electronic Commerce Research, v.2 n.3, p.255-285, July 2002 [doi>10.1023/A:1016062632182]
• 19: Stolze, M. Comparative Study of Analytical Product Selection Support Mechanisms, Proceedings of INTERACT 99, Edinborough, UK, August 30 - September 3, 1999.
• 20: M. Torrens, R. Weigel and B. Faltings. Java Constraint Library: bringing constraints technology on the Internet using the Java language. In workshop notes, Constraints and Agents, the Fourteenth National Conference on Artificial Intelligence (AAAI'97), July 1997.
• 21: M. Torrens and B. Faltings. SmartClients: Constraint satisfaction as a paradigm for scaleable intelligent information systems. In workshop notes, Artificial Intelligence for Electronic Commerce, the Sixteenth National Conference on Artificial Intelligence (AAAI'99), July 1999, 10--15.
• 22: Marc Torrens , Boi Faltings , Pearl Pu, SmartClients: Constraint Satisfaction as a Paradigm for Scaleable Intelligent Information Systems, Constraints, v.7 n.1, p.49-69, January 2002 [doi>10.1023/A:1017940426216]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

General Terms:
Experimentation, Human Factors, Performance