An effective region-based image retrieval framework

ABSTRACT

We present a region-based image retrieval framework that integrates efficient region-based representation in terms of storage and retrieval and effective on-line learning capability. The framework consists of methods for image segmentation and grouping, indexing using modified inverted file, relevance feedback, and continuous learning. By exploiting a vector quantization method, a compact region-based image representation is achieved. Based on this representation, an indexing scheme similar to the inverted file technology is proposed. In addition, it supports relevance feedback based on the vector model with a weighting scheme. A continuous learning strategy is also proposed to enable the system to self improve. Experimental results on a database of 10,000 general-purposed images demonstrate the efficiency and effectiveness of the proposed framework.

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: Ilaria Bartolini , Paolo Ciaccia , Florian Waas, FeedbackBypass: A New Approach to Interactive Similarity Query Processing, Proceedings of the 27th International Conference on Very Large Data Bases, p.201-210, September 11-14, 2001
• 3: Norbert Beckmann , Hans-Peter Kriegel , Ralf Schneider , Bernhard Seeger, The R*-tree: an efficient and robust access method for points and rectangles, Proceedings of the 1990 ACM SIGMOD international conference on Management of data, p.322-331, May 23-26, 1990, Atlantic City, New Jersey, United States
• 4: Chad Carson , Megan Thomas , Serge Belongie , Joseph M. Hellerstein , Jitendra Malik, Blobworld: A System for Region-Based Image Indexing and Retrieval, Proceedings of the Third International Conference on Visual Information and Information Systems, p.509-516, June 02-04, 1999
• 5: Paolo Ciaccia , Marco Patella , Pavel Zezula, M-tree: An Efficient Access Method for Similarity Search in Metric Spaces, Proceedings of the 23rd International Conference on Very Large Data Bases, p.426-435, August 25-29, 1997
• 6: Deng, Y., Manjunath, B. S. and Shin, H., "Color Image Segmentation", in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR '99, Fort Collins, CO, vol.2, pp. 446--51, June 1999.
• 7: Allen Gersho , Robert M. Gray, Vector quantization and signal compression, Kluwer Academic Publishers, Norwell, MA, 1991
• 8: Gong, Y., Zhang, H.J., Chuan, H. C., and Sakauchi, M., "An Image Database System with Content Capturing and Fast Image Indexing Abilities". In Proceedings of IEEE International Conference on Multimedia Computing and Systems, pages 121--130, Boston, May 1994.
• 9: H. Greenspan , G. Dvir , Y. Rubner, Region Correspondence for Image Matching via EMD Flow, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'00), p.27, June 16-16, 2000
• 10: Antonin Guttman, R-trees: a dynamic index structure for spatial searching, Proceedings of the 1984 ACM SIGMOD international conference on Management of data, June 18-21, 1984, Boston, Massachusetts
• 11: Hitchcock, F. L., "The distribution of a product from several sources to numerous localities". J. Math. Phys., 20:224--230, 1941.
• 12: Jing Huang , S. Ravi Kumar , Mandar Mitra , Wei-Jing Zhu , Ramin Zabih, Image Indexing Using Color Correlograms, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.762, June 17-19, 1997
• 13: Yoshiharu Ishikawa , Ravishankar Subramanya , Christos Faloutsos, MindReader: Querying Databases Through Multiple Examples, Proceedings of the 24rd International Conference on Very Large Data Bases, p.218-227, August 24-27, 1998
• 14: Feng Jing , Bo Zhang , Fuzong Lin , Wei-Ying Ma , Hong-Jiang Zhang, A novel region-based image retrieval method using relevance feedback, Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval, October 05-05, 2001, Ottawa, Ontario, Canada [doi>10.1145/500933.500943]
• 15: Jing, F., Li, M., Zhang, H.J., Zhang, B., "Learning Region Weighting from Relevance Feedback in Image Retrieval", Proc. the 27th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2002.
• 16: Jing, F., Li, M., Zhang, H.J., Zhang, B., "Region - based relevance feedback in image retrieval", Proc. IEEE International Symposium on Circuits and Systems (ISCAS), 2002.
• 17: Norio Katayama , Shin'ichi Satoh, The SR-tree: an index structure for high-dimensional nearest neighbor queries, Proceedings of the 1997 ACM SIGMOD international conference on Management of data, p.369-380, May 11-15, 1997, Tucson, Arizona, United States
• 18: Lee, C., Ma, W.Y., and Zhang, H.J., "Information Embedding Based on user's relevance Feedback for Image Retrieval," Proc. of SPIE Photonics East, 1998.
• 19: W. Y. Ma , B. S. Manjunath, NeTra: a toolbox for navigating large image databases, Proceedings of the 1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 1, p.568, October 26-29, 1997
• 20: Minka, T.P., Picard, R.W., "Interactive Learning Using A Society of Models", Pattern Recognition, vol. 30, no. 4, pp. 565--581, April 1997.
• 21: Minka, T.P. "An image database browser that learns from user interaction". Master's thesis, MIT Media Laboratory, 20 Ames St., Cambridge, MA 02139, 1996.
• 22: Muller, H., Squire, D. M., Muller, W., and Pun, T., "Efficient access methods for content-based image retrieval with inverted files," in Panchanathan et al. 23 (SPIE Symposium on Voice, Video and Data Communications).
• 23: Apostol Natsev , Rajeev Rastogi , Kyuseok Shim, WALRUS: a similarity retrieval algorithm for image databases, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.395-406, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 24: Niblack, W. et al. "The QBIC project: querying images by content using color, texture, and shape", in Proc. SPIE, vol. 1908, pp. 173--187, San Jose, February 1993.
• 25: Pentland, A., Picard, R., and Sclaroff, S., "Photobook: Content-based Manipulation of Image Databases." In SPIE Storage and Retrieval for Image and Video Databases II, number 2185, Feb. 1994, San Jose, CA.
• 26: Rocchio, J. J., "Relevance feedback in information retrieval". In The SMART Retrieval System-- Experiments in Automatic Document Processing, pp. 313--323, Englewood Cliffs, NJ, 1971. Prentice Hall, Inc.
• 27: Yossi Rubner , Carlo Tomasi , Leonidas J. Guibas, A Metric for Distributions with Applications to Image Databases, Proceedings of the Sixth International Conference on Computer Vision, p.59, January 04-07, 1998
• 28: Rui, Y., and Huang, T.S., "Optimizing Learning in Image Retrieval", Proceeding of IEEE int. Conf. On Computer Vision and Pattern Recognition, Jun. 2000.
• 29: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 30: John R. Smith , Chung-Sheng Li, Image classification and querying using composite region templates, Computer Vision and Image Understanding, v.75 n.1-2, p.165-174, July/Aug. 1999 [doi>10.1006/cviu.1999.0771]
• 31: Stricker, M., and Orengo, M., "Similarity of Color Images", in Storage and Retrieval for Image and Video Databases, Proc. SPIE 2420, pp 381--392, 1995.
• 32: Zhong Su , Stan Li , Hongjiang Zhang, Extraction of feature subspaces for content-based retrieval using relevance feedback, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500158]
• 33: Creating a Customized Access Method for Blobworld, Proceedings of the 16th International Conference on Data Engineering, p.82, February 28-March 03, 2000
• 34: Vasconcelos, N., and Lippman, A., "Learning from user feedback in image retrieval system", in Proc. of NIPS'99, Denver, Colorado, 1999.
• 35: James Z. Wang , Yanping Du, Scalable integrated region-based image retrieval using IRM and statistical clustering, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.268-277, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379679]
• 36: James Z. Wang , Jia Li , Gio Wiederhold, SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.9, p.947-963, September 2001 [doi>10.1109/34.955109]
• 37: White, D.A., and Jain, R., "Algorithms and strategies for similarity retrieval," Storage and Retrieval in Image, and Video Databases, vol. 2,060, pp. 62--72, 1996.
• 38: Ian H. Witten , Timothy C. Bell , Alistair Moffat, Managing Gigabytes: Compressing and Indexing Documents and Images, John Wiley & Sons, Inc., New York, NY, 1994
• 39: M. E. J. Wood , B. T. Thomas , N. W. Campbell, Iterative refinement by relevance feedback in content-based digital image retrieval, Proceedings of the sixth ACM international conference on Multimedia, p.13-20, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290750]
• 40: P. Wu , B. S. Manjunath, Adaptive nearest neighbor search for relevance feedback in large image databases, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500157]
• 41: Xiang Sean Zhou , Thomas S. Huang, Comparing discriminating transformations and SVM for learning during multimedia retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500163]
• 42: Lei Zhu , Aidong Zhang , Aibing Rao , Rohini Srihari, Keyblock: an approach for content-based image retrieval, Proceedings of the eighth ACM international conference on Multimedia, p.157-166, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354455]

INDEX TERMS

Keywords:
continuous learning, inverted file, region-based image retrieval, relevance feedback