Mining images on semantics via statistical learning

ABSTRACT

In this paper, we have proposed a novel framework to enable hierarchical image classification via statistical learning. By integrating the concept hierarchy for semantic image concept organization, a hierarchical mixture model is proposed to enable multi-level modeling of semantic image concepts and hierarchical classifier combination. Thus, learning the classifiers for the semantic image concepts at the high level of the concept hierarchy can be effectively achieved by detecting the presences of the relevant base-level atomic image concepts. To effectively learn the base-level classifiers for the atomic image concepts at the first level of the concept hierarchy, we have proposed a novel adaptive EM algorithm to achieve more effective model selection and parameter estimation. In addition, a novel penalty term is proposed to effectively eliminate the misleading effects of the outlying unlabeled images on semi-supervised classifier training. Our experimental results in a specific image domain of outdoor photos are very attractive.

REFERENCES

• 1: Osmar R. Zaïane , Jiawei Han , Ze-Nian Li , Sonny H. Chee , Jenny Y. Chiang, MultiMediaMiner: a system prototype for multimedia data mining, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.581-583, June 01-04, 1998, Seattle, Washington, United States
• 2: Simeon J. Simoff , Chabane Djeraba , Osmar R. Zaïane, MDM/KDD2002: multimedia data mining between promises and problems, ACM SIGKDD Explorations Newsletter, v.4 n.2, p.118-121, December 2002 [doi>10.1145/772862.772886]
• 3: C. Djeraba, "When image indexing meets knowledge discovery", ACM MDM/KDD, pp.73--81, 2000.
• 4: Casey Breen , Latifur Khan , Arunkumar Ponnusamy, Image Classification Using Neural Networks and Ontologies, Proceedings of the 13th International Workshop on Database and Expert Systems Applications, p.98-102, September 02-06, 2002
• 5: Jianping Fan , Yuli Gao , Hangzai Luo, Multi-level annotation of natural scenes using dominant image components and semantic concepts, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027660]
• 6: Gholamhosein Sheikholeslami , Wendy Chang , Aidong Zhang, Semantic clustering and querying on heterogeneous features for visual data, Proceedings of the sixth ACM international conference on Multimedia, p.3-12, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290749]
• 7: E. Chang, K. Goh, G. Sychay, G. Wu, "CBSA: Content-based annotation for multimodal image retrieval using Bayes point machines", IEEE Trans. CSVT, 2002.
• 8: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 9: Ruofei Zhang , Zhongfei (Mark) Zhang , Sandeep Khanzode, A data mining approach to modeling relationships among categories in image collection, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014151]
• 10: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800]
• 11: Y. Wu, A. Zhang, "Adaptive pattern discovery for interactive multimedia retrieval", IEEE CVPR, 2003.
• 12: Apostol (Paul) Natsev , Milind R. Naphade , John R. Smith, Semantic representation: search and mining of multimedia content, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014133]
• 13: Wei Wang , Yuqing Song , Aidong Zhang, Semantics-Based Image Retrieval by Region Saliency, Proceedings of the International Conference on Image and Video Retrieval, p.29-37, July 18-19, 2002
• 14: Jia-Yu Pan , Hyung-Jeong Yang , Christos Faloutsos , Pinar Duygulu, Automatic multimedia cross-modal correlation discovery, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014135]
• 15: Jing Huang , S. Ravi Kumar , Ramin Zabih, An automatic hierarchical image classification scheme, Proceedings of the sixth ACM international conference on Multimedia, p.219-228, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290774]
• 16: Y. Alp Aslandogan , Chuck Thier , Clement T. Yu , Jon Zou , Naphtali Rishe, Using semantic contents and WordNet in image retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.286-295, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 17: G. McLachlan and T. Krishnan, The EM algorithm and extensions, New York, John Wiley & Sons, 2000.
• 18: Naonori Ueda , Ryohei Nakano , Zoubin Ghahramani , Geoffrey E. Hinton, SMEM algorithm for mixture models, Proceedings of the 1998 conference on Advances in neural information processing systems II, p.599-605, July 1999
• 19: Michael I. Jordan , Robert A. Jacobs, Hierarchical mixtures of experts and the EM algorithm, Neural Computation, v.6 n.2, p.181-214, March 1994 [doi>10.1162/neco.1994.6.2.181]
• 20: Mario A. T. Figueiredo , Anil K. Jain, Unsupervised Learning of Finite Mixture Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.381-396, March 2002 [doi>10.1109/34.990138]
• 21: Andrew McCallum , Ronald Rosenfeld , Tom M. Mitchell , Andrew Y. Ng, Improving Text Classification by Shrinkage in a Hierarchy of Classes, Proceedings of the Fifteenth International Conference on Machine Learning, p.359-367, July 24-27, 1998
• 22: Thomas Hofmann, The Cluster-Abstraction Model: Unsupervised Learning of Topic Hierarchies from Text Data, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.682-687, July 31-August 06, 1999
• 23: Daphne Koller , Mehran Sahami, Hierarchically Classifying Documents Using Very Few Words, Proceedings of the Fourteenth International Conference on Machine Learning, p.170-178, July 08-12, 1997
• 24: Soumen Chakrabarti , Byron Dom , Rakesh Agrawal , Prabhakar Raghavan, Using Taxonomy, Discriminants, and Signatures for Navigating in Text Databases, Proceedings of the 23rd International Conference on Very Large Data Bases, p.446-455, August 25-29, 1997
• 25: C. Fellbaum, WordNet: An electronic lexical database, MIT Press, 1998.
• 26: George A. Miller, WordNet: a lexical database for English, Communications of the ACM, v.38 n.11, p.39-41, Nov. 1995 [doi>10.1145/219717.219748]
• 27: C.A. Bouman, M. Shapiro, G. Cook, C. Atkins, H. Cheng, "Cluster: An unsupervised algorithm for modeling Gaussian mixtures", Technical Report, Purdue University.
• 28: Chabane Djeraba, Multimedia Mining: A Highway to Intelligent Multimedia Documents (Multimedia Systems and Applications Series), Kluwer Academic Publishers, Norwell, MA, 2002
• 29: F. Cozman, I. Cohen, "Unlabeled data can degrade classification performance of generative classifier", TR-HPL-2001-234, 2001.
• 30: M.R. Naphade, X. Zhou, and T.S. Huang, "Image classification using a set of labeled and unlabeled images", Proc. SPIE, 2000.
• 31: Kamal Nigam , Andrew Kachites McCallum , Sebastian Thrun , Tom Mitchell, Text Classification from Labeled and Unlabeled Documents using EM, Machine Learning, v.39 n.2-3, p.103-134, May-June 2000 [doi>10.1023/A:1007692713085]
• 32: M. Szummer and T. Jaakkola, "Information Regularization with Partially Labeled Data", Proc. NIPS, 2002.
• 33: Kristin P. Bennett , Usama Fayyad , Dan Geiger, Density-based indexing for approximate nearest-neighbor queries, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, p.233-243, August 15-18, 1999, San Diego, California, United States [doi>10.1145/312129.312236]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
adaptive EM algorithm, hierarchical mixture model, image classification