Multi-class image segmentation using conditional random fields and global classification

ABSTRACT

A key aspect of semantic image segmentation is to integrate local and global features for the prediction of local segment labels. We present an approach to multi-class segmentation which combines two methods for this integration: a Conditional Random Field (CRF) which couples to local image features and an image classification method which considers global features. The CRF follows the approach of Reynolds & Murphy (2007) and is based on an unsupervised multi scale pre-segmentation of the image into patches, where patch labels correspond to the random variables of the CRF. The output of the classifier is used to constraint this CRF. We demonstrate and compare the approach on a standard semantic segmentation data set.

REFERENCES

• 1: Altun, Y., Tsochantaridis, I., & Hofmann, T. (2003). Hidden markov support vector machines. Proceedings of the 20th International Conference on Machine Learning (ICML 2003) (pp. 3--10).
• 2: Awasthi, P., Gagrani, A., & Ravindran, B. (2007). Image modelling using tree structured conditional random fields. Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007) (pp. 2060--2065).
• 3: S. Belongie , J. Malik , J. Puzicha, Shape Matching and Object Recognition Using Shape Contexts, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.4, p.509-522, April 2002 [doi>10.1109/34.993558]
• 4: Anna Bosch , Andrew Zisserman , Xavier Munoz, Representing shape with a spatial pyramid kernel, Proceedings of the 6th ACM international conference on Image and video retrieval, p.401-408, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282340]
• 5: Csurka, G., & Perronnin, F. (2008). A simple high performance approach to semantic segmentation. Proceedings of the British Machine Vision Conference (BMVC 2008).
• 6: Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2008). The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results. http://www.pascal-network.org/challenges/VOC/voc2008/workshop/.
• 7: Pedro F. Felzenszwalb , Daniel P. Huttenlocher, Efficient Graph-Based Image Segmentation, International Journal of Computer Vision, v.59 n.2, p.167-181, September 2004 [doi>10.1023/B:VISI.0000022288.19776.77]
• 8: Hayman, E., Caputo, B., Fritz, M., & Eklundh, J. (2004). On the significance of real-world conditions for material classification. Proceedings of the European Conference on Computer Vision (ECCV 2004) (pp. 253--266).
• 9: John D. Lafferty , Andrew McCallum , Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.282-289, June 28-July 01, 2001
• 10: Christoph H. Lampert , Matthew B. Blaschko, A Multiple Kernel Learning Approach to Joint Multi-class Object Detection, Proceedings of the 30th DAGM symposium on Pattern Recognition, June 10-13, 2008, Munich, Germany [doi>10.1007/978-3-540-69321-5_4]
• 11: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 12: Nowak, E., Jurie, F., & Triggs, B. (2006). Sampling strategies for bag-of-features image classification. Proceedings of the European Conference on Computer Vision (ECCV 2006) (pp. 490--503).
• 13: Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. advances in large margin classifiers. Advances in Large Margin Classifiers, 1, 61--74.
• 14: Jordan Reynolds , Kevin Murphy, Figure-ground segmentation using a hierarchical conditional random field, Proceedings of the Fourth Canadian Conference on Computer and Robot Vision, p.175-182, May 28-30, 2007 [doi>10.1109/CRV.2007.32]
• 15: Sören Sonnenburg , Gunnar Rätsch , Christin Schäfer , Bernhard Schölkopf, Large Scale Multiple Kernel Learning, The Journal of Machine Learning Research, 7, p.1531-1565, 12/1/2006
• 16: Ioannis Tsochantaridis , Thorsten Joachims , Thomas Hofmann , Yasemin Altun, Large Margin Methods for Structured and Interdependent Output Variables, The Journal of Machine Learning Research, 6, p.1453-1484, 12/1/2005
• 17: J. Zhang , M. Marszałek , S. Lazebnik , C. Schmid, Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study, International Journal of Computer Vision, v.73 n.2, p.213-238, June 2007 [doi>10.1007/s11263-006-9794-4]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation; Feature evaluation and selection

General Terms:
Algorithms, Design, Measurement, Performance