Argo: intelligent advertising by mining a user's interest from his photo collections

ABSTRACT

In this paper, we introduce a system named Argo which provides intelligent advertising made possible from users' photo collections. Based on the intuition that user-generated photos imply user interests which are the key for profitable targeted ads, the Argo system attempts to learn a user's profile from his shared photos and suggests relevant ads accordingly. To learn a user interest, in an offline step, a hierarchical and efficient topic space is constructed based on the ODP ontology, which is used later on for bridging the vocabulary gap between ads and photos as well as reducing the effect of noisy photo tags. In the online stage, the process of Argo contains three steps: 1) understanding the content and semantics of a user's photos and auto-tagging each photo to supplement user-submitted tags (such tags may not be available); 2) learning the user interest given a set of photos based on the learnt hierarchical topic space; and 3) representing ads in the topic space and matching their topic distributions with the target user interest; the top ranked ads are output as the suggested ads. Two key challenges are tackled during the process: 1) the semantic gap between the low-level image visual features and the high-level user semantics; and 2) the vocabulary impedance between photos and ads. We conducted a series of experiments based on real Flickr users and Amazon.com products (as candidate ads), which show the effectiveness of the proposed approach.

REFERENCES

• 1: Badi, R. Recognition and Representation of User Interest. Master Thesis, Texas A&M University, 2005.
• 2: Andrei Broder , Marcus Fontoura , Vanja Josifovski , Lance Riedel, A semantic approach to contextual advertising, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277837]
• 3: Chien Chin Chen , Meng Chang Chen , Yeali Sun, PVA: A Self-Adaptive Personal View Agent, Journal of Intelligent Information Systems, v.18 n.2-3, p.173-194, March-May 2002 [doi>10.1023/A:1013629527840]
• 4: Yifan Chen , Gui-Rong Xue , Yong Yu, Advertising keyword suggestion based on concept hierarchy, Proceedings of the international conference on Web search and web data mining, February 11-12, 2008, Palo Alto, California, USA [doi>10.1145/1341531.1341564]
• 5: David J. Crandall , Lars Backstrom , Daniel Huttenlocher , Jon Kleinberg, Mapping the world's photos, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526812]
• 6: Evans, A., Fernandez, M., et al. Adaptive Multimedia Access: From User Needs Semantic Personalization. In Proc. Of IEEE Int. Symposium on Circuits and Systems, 2006.
• 7: Google AdWords Keyword Tool. https://adwords.google.com/select/KeywordToolExternal.
• 8: Grcar, M., Mladenic, D., and Grobelnik, M. User Profiling for Interest-focused Browsing History. UserSWeb, 2005.
• 9: Gunduz, S., and Ozsu, M. A User Interest Model for Web Page Navigation. In Proc. of Int. Workshop on Data Mining for Actionable Knowledge (DMAK), Seoul, Korea, April, pages 46--57, 2003.
• 10: Hua, X.-S., Mei, T., and Li, S. P. When Multimedia Advertising Meets the New Internet Era. Intl Workshop on Multimedia Signal Processing, pp. 1--5, 2008.
• 11: Jing, F., Li, M. et al. Learning Region Weighting from Relevance Feedback in Image Retrieval. In Proc. of IEEE Int. Conf. on Acoustics Speech and Signal, 2002.
• 12: Amruta Joshi , Rajeev Motwani, Keyword Generation for Search Engine Advertising, Proceedings of the Sixth IEEE International Conference on Data Mining - Workshops, p.490-496, December 18-22, 2006 [doi>10.1109/ICDMW.2006.104]
• 13: Hyoung R. Kim , Philip K. Chan, Learning implicit user interest hierarchy for context in personalization, Proceedings of the 8th international conference on Intelligent user interfaces, January 12-15, 2003, Miami, Florida, USA [doi>10.1145/604045.604064]
• 14: Anísio Lacerda , Marco Cristo , Marcos André Gonçalves , Weiguo Fan , Nivio Ziviani , Berthier Ribeiro-Neto, Learning to advertise, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148265]
• 15: Letizia, L. H. An Agent That Assists Web Browsing. In Proc. of the International Joint Conference on Artificial Intelligence, Montreal, CA, 1995.
• 16: Xin Li , Lei Guo , Yihong Eric Zhao, Tag-based social interest discovery, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367589]
• 17: Zhiwei Li , Lei Zhang , Wei-Ying Ma, Delivering online advertisements inside images, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459572]
• 18: Zhongming Ma , Gautam Pant , Olivia R. Liu Sheng, Interest-based personalized search, ACM Transactions on Information Systems (TOIS), v.25 n.1, p.5-es, February 2007 [doi>10.1145/1198296.1198301]
• 19: Stuart E. Middleton , David C. De Roure , Nigel R. Shadbolt, Capturing knowledge of user preferences: ontologies in recommender systems, Proceedings of the 1st international conference on Knowledge capture, October 22-23, 2001, Victoria, British Columbia, Canada [doi>10.1145/500737.500755]
• 20: Vanessa Murdock , Massimiliano Ciaramita , Vassilis Plachouras, A noisy-channel approach to contextual advertising, Proceedings of the 1st international workshop on Data mining and audience intelligence for advertising, p.21-27, August 12-12, 2007, San Jose, California [doi>10.1145/1348599.1348603]
• 21: Anon Plangprasopchok , Kristina Lerman, Constructing folksonomies from user-specified relations on flickr, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526814]
• 22: Berthier Ribeiro-Neto , Marco Cristo , Paulo B. Golgher , Edleno Silva de Moura, Impedance coupling in content-targeted advertising, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076119]
• 23: Trajkova, J., and Gauch, S. Improving Ontology-Based User Profiles. In Proc. of RIAO, pp. 380--389, 2004.
• 24: Xin-Jing Wang , Lei Zhang , Xirong Li , Wei-Ying Ma, Annotating Images by Mining Image Search Results, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.11, p.1919-1932, November 2008 [doi>10.1109/TPAMI.2008.127]
• 25: WordTracker. http://www.wordtracker.com.
• 26: Xujuan Zhou , Sheng-Tang Wu , Yuefeng Li , Yue Xu , Raymond Y. K. Lau , Peter D. Bruza, Utilizing Search Intent in Topic Ontology-Based User Profile for Web Mining, Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, p.558-564, December 18-22, 2006 [doi>10.1109/WI.2006.186]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Commercial services

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

General Terms:
Experimentation, Human Factors, Performance

Keywords:
image understanding, photo monetization, user interest modeling