Ranking and classifying attractiveness of photos in folksonomies

ABSTRACT

Web 2.0 applications like Flickr, YouTube, or Del.icio.us are increasingly popular online communities for creating, editing and sharing content. The growing size of these folksonomies poses new challenges in terms of search and data mining. In this paper we introduce a novel methodology for automatically ranking and classifying photos according to their attractiveness for folksonomy members. To this end, we exploit image features known for having significant effects on the visual quality perceived by humans (e.g. sharpness and colorfulness) as well as textual meta data, in what is a multi-modal approach. Using feedback and annotations available in the Web 2.0 photo sharing system Flickr, we assign relevance values to the photos and train classification and regression models based on these relevance assignments. With the resulting machine learning models we categorize and rank photos according to their attractiveness. Applications include enhanced ranking functions for search and recommender methods for attractive content. Large scale experiments on a collection of Flickr photos demonstrate the viability of our approach.

REFERENCES

• 1: Josef Bigun, Vision with Direction: A Systematic Introduction to Image Processing and Computer Vision, Springer-Verlag New York, Inc., Secaucus, NJ, 2005
• 2: Chris Burges , Tal Shaked , Erin Renshaw , Ari Lazier , Matt Deeds , Nicole Hamilton , Greg Hullender, Learning to rank using gradient descent, Proceedings of the 22nd international conference on Machine learning, p.89-96, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102363]
• 3: Christopher J. C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, v.2 n.2, p.121-167, June 1998 [doi>10.1023/A:1009715923555]
• 4: Soumen Chakrabarti, Mining the Web: Discovering Knowledge from HyperText Data, Science & Technology Books, 2002
• 5: Micah Dubinko , Ravi Kumar , Joseph Magnani , Jasmine Novak , Prabhakar Raghavan , Andrew Tomkins, Visualizing tags over time, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135810]
• 6: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 7: Susan Dumais , Hao Chen, Hierarchical classification of Web content, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.256-263, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345593]
• 8: Susan Dumais , John Platt , David Heckerman , Mehran Sahami, Inductive learning algorithms and representations for text categorization, Proceedings of the seventh international conference on Information and knowledge management, p.148-155, November 02-07, 1998, Bethesda, Maryland, United States [doi>10.1145/288627.288651]
• 9: T. Hammond, T. Hannay, B. Lund, and J. Scott. Social Bookmarking Tools (I): A General Review. D-Lib Magazine, 11(4), April 2005.
• 10: S. Hasler and S. Susstrunk. Measuring colorfulness in real images. volume 5007, pages 87--95, 2003.
• 11: A. Hotho, R. J¨aschke, C. Schmitz, and G. Stumme. Information Retrieval in Folksonomies: Search and Ranking. In The Semantic Web: Research and Applications, volume 4011 of LNAI, pages 411--426, Heidelberg, 2006. Springer.
• 12: Kai-Qi Huang , Qiao Wang , Zhen-Yang Wu, Natural color image enhancement and evaluation algorithm based on human visual system, Computer Vision and Image Understanding, v.103 n.1, p.52-63, July 2006 [doi>10.1016/j.cviu.2006.02.007]
• 13: Yushi Jing , Shumeet Baluja, Pagerank for product image search, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367540]
• 14: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 15: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 16: D. Kalenova, P. Toivanen, and V. Bochko. Preferential spectral image quality model. pages 389--398. 2005.
• 17: W. H. Kruskal. Ordinal measures of association. Journal of the American Statistical Association, 53(284):814--861, 1958.
• 18: B. Lund, T. Hammond, M. Flack, and T. Hannay. Social Bookmarking Tools (II): A Case Study -- Connotea. D-Lib Magazine, 11(4), 2005.
• 19: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997
• 20: B. S. Manjunath , W. Y. Ma, Texture Features for Browsing and Retrieval of Image Data, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.8, p.837-842, August 1996 [doi>10.1109/34.531803]
• 21: Christopher D. Manning , Hinrich Schütze, Foundations of statistical natural language processing, MIT Press, Cambridge, MA, 1999
• 22: E. Peli. Contrast in complex images. Journal of the Optical Society of America, 7:2032--2040, 1990.
• 23: Matthew Richardson , Amit Prakash , Eric Brill, Beyond PageRank: machine learning for static ranking, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135881]
• 24: A. E. Savakis, S. P. Etz, and A. C. Loui. Evaluation of image appeal in consumer photography. In B. E. Rogowitz and T. N. Pappas, editors, SPIE Conference Series, volume 3959, pages 111--120, June 2000.
• 25: A. E. Savakis and A. C. Loui. Method For Automatically Detecting Digital Images that are Undesirable for Placing in Albums, volume US 6535636. March 2003.
• 26: A. E. Savakis and R. Mehrotra. Retrieval and browsing of database images based on image emphasis and appeal. US 6847733, 2005.
• 27: C. Schmitz, A. Hotho, R. Jaeschke, and G. Stumme. Mining Association Rules in Folksonomies. In Data Science and Classification, pages 261--270. Springer Berlin Heidelberg, 2006.
• 28: Alex J. Smola , Bernhard Schölkopf, A tutorial on support vector regression, Statistics and Computing, v.14 n.3, p.199-222, August 2004 [doi>10.1023/B:STCO.0000035301.49549.88]
• 29: Stijn Dongen, A cluster algorithm for graphs, CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands, 2000
• 30: Chong-Yaw Wee , Raveendran Paramesran, Measure of image sharpness using eigenvalues, Information Sciences: an International Journal, v.177 n.12, p.2533-2552, June, 2007 [doi>10.1016/j.ins.2006.12.023]
• 31: S. Winkler. Visual fidelity and perceived quality: Towards comprehensive metrics. In in Proc. SPIE, volume 4299, pages 114--125, 2001.
• 32: S. Winkler and C. Faller. Perceived audiovisual quality of low-bitrate multimedia content. Multimedia, IEEE Transactions on, 8(5):973--980, 2006.
• 33: S. Yao, W. Lin, S. Rahardja, X. Lin, E. P. Ong, Z. K. Lu, and X. K. Yang. Perceived visual quality metric based on error spread and contrast. In Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on, pages 3793--3796 Vol. 4, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services

General Terms:
Algorithms

Keywords:
attractiveness features, classification, folksonomy feedback, image analysis, photo appeal, ranking, web 2.0