Mining GPS traces and visual words for event classification

ABSTRACT

It is of great interest to recognize semantic events (e.g., hiking, skiing, party), in particular when given a collection of personal photos, where each photo is tagged with a timestamp and GPS (Global Positioning System) information at the capture. We address this emerging multiclass classification problem by mining informative features derived from traces of GPS coordinates and a bag of visual words, both based on the entire collection as opposed to individual photos. Considering that semantic events are best characterized by a compositional description of the visual content in terms of the co-occurrence of objects and scenes, we focus on mining compositional features (equivalent to word combinations in the "bag-of-words" method) that have better discriminative and descriptive abilities than individual features. In order to handle the combinatorial complexity in discovering such compositional features, we apply a data mining method based on frequent itemset mining (FIM). Complementary features are also derived from GPS traces and mined to characterize the underlying movement patterns of various event types. Upon compositional feature mining, we perform multiclass AdaBoost to solve the multiclass problem. Based on a dataset of eight event classes and a total of more than 3000 geotagged images from 88 events, experimental results using leave-one-out cross validation have shown the synergy of all of the components in our proposed approach to event classification.

REFERENCES

• 1: L. Cao, J. Luo, H. Kautz, and T. Huang. Annotating collections of photos using hierarchical event and scene models. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition 2008.
• 2: H. Cheng, X. Yan, J. Han, and C.-W. Hsu. Discriminative frequent pattern analysis for effective classification. In Proc. of Intl. Conf. on Data Engineering 2007.
• 3: H. Cheng, X. Yan, J. Han, and P. S. Yu. Direct discriminative pattern mining for effective classification. In Proc. of Intl. Conf. on Data Engineering 2008.
• 4: S. Ebadollahi, L. Xie, S.-F. Chang, and J. R. Smith. Visual event detection using multi-dimensional concept dynamics. In Proc. IEEE Conf. on Multimedia Expo 2006.
• 5: Gosta Grahne , Jianfei Zhu, Fast Algorithms for Frequent Itemset Mining Using FP-Trees, IEEE Transactions on Knowledge and Data Engineering, v.17 n.10, p.1347-1362, October 2005 [doi>10.1109/TKDE.2005.166]
• 6: Jiawei Han , Hong Cheng , Dong Xin , Xifeng Yan, Frequent pattern mining: current status and future directions, Data Mining and Knowledge Discovery, v.15 n.1, p.55-86, August 2007 [doi>10.1007/s10618-006-0059-1]
• 7: Lyndon Kennedy , Mor Naaman , Shane Ahern , Rahul Nair , Tye Rattenbury, How flickr helps us make sense of the world: context and content in community-contributed media collections, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291384]
• 8: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 9: L.-J. Li and L. Fei-Fei. What, where and who? classifying events by scene and object recognition. In Proc. IEEE Intl. Conf. on Computer Vision 2007.
• 10: B. Liu, W. Hsu, and Y. Ma. Integrating classification and association rule mining. In Proc. SIGKDD 1998.
• 11: D. Liu, G. Hua, P. Viola, and T. Chen. Integrated feature selection and higher-order spatial feature extraction for object categorization. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition 2008.
• 12: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 13: Jiebo Luo , Matthew Boutell, Automatic Image Orientation Detection via Confidence-Based Integration of Low-Level and Semantic Cues, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.5, p.715-726, May 2005 [doi>10.1109/TPAMI.2005.96]
• 14: Thomas M. Mitchell, Machine Learning, McGraw-Hill Higher Education, 1997
• 15: X. Yin and J. Han. Cpar: classification based on predictive association rules. In SIAM International Conference data mining (SDM)2003.
• 16: J. Yuan, J. Luo, and Y. Wu. Mining compositional features for boosting. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition 2008.
• 17: J. Yuan, Y. Wu, and M. Yang. Discovery of collocation patterns:from visual words to visual phrases. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition 2007.
• 18: Junsong Yuan , Ying Wu , Ming Yang, From frequent itemsets to semantically meaningful visual patterns, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281284]
• 19: J. Zhu, S. Rosset, H. Zou, and T. Hastie. Multi-class adaboost. Technique Report 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

General Terms:
Algorithms, Performance

Keywords:
event categorization, gps information, image data mining