Deep learning from temporal coherence in video

ABSTRACT

This work proposes a learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. This coherence is used as a supervisory signal over the unlabeled data, and is used to improve the performance on a supervised task of interest. We demonstrate the effectiveness of this method on some pose invariant object and face recognition tasks.

REFERENCES

• 1: Becker, S. (1996a). Learning Temporally Persistent Hierarchical Representations. Advances in Neural Information Processing Systems (pp. 824--830).
• 2: Becker, S. (1996b). Mutual information maximization: models of cortical self-organization. Network: Computation in Neural Systems, 7, 7--31.
• 3: Suzanna Becker, Implicit learning in 3D object recognition: the importance of temporal context, Neural Computation, v.11 n.2, p.347-374, Feb. 15, 1999 [doi>10.1162/089976699300016683]
• 4: Becker, S., & Hinton, G. (1992). Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature, 355, 161--163.
• 5: Belkin, M., Niyogi, P., & Sindhwani, V. (2005). On manifold regularization. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (AISTATS) (pp. 17--24).
• 6: Bottou, L. (1991). Stochastic gradient learning in neural networks. Proceedings of Neuro-Nîmes 91. Nimes, France: EC2.
• 7: Michael Bowling , Ali Ghodsi , Dana Wilkinson, Action respecting embedding, Proceedings of the 22nd international conference on Machine learning, p.65-72, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102360]
• 8: Bromley, J., Bentz, J., W. Bottou, L., & Guyon, I. (1993). Signature verification using a siamese time delay neural network. International Journal of Pattern Recognition and Artificial Intelligence (p. 669).
• 9: Caputo, B., Hornegger, J., Paulus, D., & Niemann, H. (2002). A spin-glass markov random field for 3-d object recognition (Technical Report LME-TR-2002-01). Institut fur Informatik, Universitat Erlangen Nurnberg.
• 10: Olivier Chapelle , Bernhard Schölkopf , Alexander Zien, Semi-Supervised Learning (Adaptive Computation and Machine Learning), The MIT Press, 2006
• 11: Chapelle, O., & Zien, A. (2003). Semi-Supervised Classification by Low Density Separation. Advances in Neural Information Processing Systems, 17, 1633--1640.
• 12: Sumit Chopra , Raia Hadsell , Yann LeCun, Learning a Similarity Metric Discriminatively, with Application to Face Verification, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.539-546, June 20-26, 2005 [doi>10.1109/CVPR.2005.202]
• 13: Geoffrey Hinton , Terrence J. Sejnowski, Unsupervised learning, Bradford Company, Scituate, MA, 1999
• 14: Rui Huang , Dimitris N. Metaxas , Vladimir Pavlovic, A Hybrid Face Recognition Method using Markov Random Fields, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 3, p.157-160, August 23-26, 2004 [doi>10.1109/ICPR.2004.43]
• 15: LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86, 2278--2324.
• 16: LeCun, Y., Huang, F. J., & Bottou, L. (2004). Learning methods for generic object recognition with invariance to pose and lighting. Proc. Computer Vision and Pattern Recognition Conference (pp. 97--104).
• 17: Shree K. Nayar , Masahiro Watanabe , Minori Noguchi, Real-Time Focus Range Sensor, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.12, p.1186-1198, December 1996 [doi>10.1109/34.546256]
• 18: Margarita Osadchy , Yann Le Cun , Matthew L. Miller, Synergistic Face Detection and Pose Estimation with Energy-Based Models, The Journal of Machine Learning Research, 8, p.1197-1215, 5/1/2007
• 19: Roobaert, D., & Hulle, M. M. V. (1999). View-based 3d object recognition with support vector machines. In IEEE International Workshop on Neural Networks for Signal Processing (pp. 77--84).
• 20: Roweis, S., & Saul, L. (2000). Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 290, 2323--2326.
• 21: Samaria, F., & Harter, A. (1994). Parameterisation of a stochastic model for human face identification. Proceedings of 2nd IEEE Workshop on Applications of Computer Vision (pp. 138--142).
• 22: Tenenbaum, J., Silva, V., & Langford, J. (2000). A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, 290, 2319--2323.
• 23: Antonio Torralba , Rob Fergus , William T. Freeman, 80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.11, p.1958-1970, November 2008 [doi>10.1109/TPAMI.2008.128]
• 24: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 25: Heiko Wersing , Edgar Körner, Learning optimized features for hierarchical models of invariant object recognition, Neural Computation, v.15 n.7, p.1559-1588, July 2003 [doi>10.1162/089976603321891800]
• 26: Jason Weston , Frédéric Ratle , Ronan Collobert, Deep learning via semi-supervised embedding, Proceedings of the 25th international conference on Machine learning, p.1168-1175, July 05-09, 2008, Helsinki, Finland [doi>10.1145/1390156.1390303]
• 27: Laurenz Wiskott , Terrence J. Sejnowski, Slow feature analysis: unsupervised learning of invariances, Neural Computation, v.14 n.4, p.715-770, April 2002 [doi>10.1162/089976602317318938]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Feature evaluation and selection; Classifier design and evaluation
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory