Learning video browsing behavior and its application in the generation of video previews

ABSTRACT

With more and more streaming media servers becoming commonplace, streaming video has now become a popular medium of instruction, advertisement, and entertainment. With such prevalence comes a new challenge to the servers: Can they track browsing behavior of users to determine what interest users? Learning this information is potentially valuable not only for improved customer tracking and context-sensitive e-commerce, but also in the generation of fast previews of videos for easy pre-downloads. In this paper, we present a formal learning mechanism to track video browsing behavior of users. This information is then used to generate fast video previews. Specifically, we model the states a user transitions while browsing through videos to be the hidden states of a Hidden Markov Model. We estimate the parameters of the HMM using maximum likelihood estimation for each sample observation sequence of user interaction with videos. Video previews are then formed from interesting segments of the video automatically inferred from an analysis of the browsing states of viewers. Audio coherence in the previews is maintained by selecting clips spanning complete clauses containing topically significant spoken phrases. The utility of learning video browsing behavior is demonstrated through user studies and experiments.

REFERENCES

• 1: T. Akiba and H. Tanaka. A bayesian approach for user modcling in dialogur systems. Technical report. Dept. of Computer Science. ISSN 091%28u2. February 1992.
• 2: G. Cohen , A. Amir , D. Ponceleon , B. Blanchard , D. Petkovic , S. Srinivasan, Using Audio Time Scale Modification for Video Browsing, Proceedings of the 33rd Hawaii International Conference on System Sciences-Volume 3, p.3046, January 04-07, 2000
• 3: K. Bharat and R1. Hcnzingrr. lmprov-ed algorithms for topic distillation in a hypedinked environment. In Proc. 2Znd Annual SIGIR Conference, pages 326-327. 1999.
• 4: G. Hauptmann, D. Lw. and P.E. Kennedy. Topic labeling of multilingual broadcast news in the informedia digital video library. In Proc. ACM Dzyztul Libmies/SIGIR MIDAS Workshop, 1999.
• 5: Liwei He , Elizabeth Sanocki , Anoop Gupta , Jonathan Grudin, Auto-summarization of audio-video presentations, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.489-498, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319691]
• 6: Steve Jones , Gordon Paynter, Topic-based browsing within a digital library using keyphrases, Proceedings of the fourth ACM conference on Digital libraries, p.114-121, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313279]
• 7: .I.S. .Tustcson and S. Katz. Technical trrminology: Some iinruistic urovaties and an algorithm for identification in text Natural Langugae Enginering 1:9-27,1995
• 8: L.Baum et al. A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains. Ann. Math Statzstzcs. 41:164-171. ,January 197*.
• 9: Rainer Lienhart , Silvia Pfeiffer , Wolfgang Effelsberg, Video abstracting, Communications of the ACM, v.40 n.12, p.54-62, Dec. 1997 [doi>10.1145/265563.265572]
• 10: E. Saud hl. Sahami; hl. Hearst. Applying multiple cause mixture model to text categorization. In Proe. Thzrteenth Intl. Conf Machme Learnmg, 1996.
• 11: J. Orwant. Heterogeneous learning in the doppelganger User-modding system. User Modellng and User-Adapted Intemctzon. 4(2):107-130, 1995.
• 12: Dulce Ponceleon , Savitha Srinivasan, Structure and content-based segmentation of speech transcripts, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.404-405, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384041]
• 13: L.R. Rabiner. A tutorial on hidden markov models and x+cted applications in spwcb recognition. Proceedings IEEE. 77(2):257-2X6; 1989.
• 14: R. Schwartz et al. A maximum likelihood model for topic classification in broadcast news. In Proc. Ewopean Conf. on Speech Communication and Technology. 1997.
• 15: Michael A. Smith, Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.775, June 17-19, 1997
• 16: Tanveer Syeda-Mahmood , S. Srinivasan, Detecting topical events in digital video, Proceedings of the eighth ACM international conference on Multimedia, p.85-94, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354433]
• 17: Yiming Yang, An Evaluation of Statistical Approaches to Text Categorization, Information Retrieval, v.1 n.1-2, p.69-90, 1999 [doi>10.1023/A:1009982220290]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.2 Document Preparation
          Subjects: Multi/mixed media

General Terms:
Measurement, Performance, Theory

Keywords:
audio, browsing behavior, interesting content, learning, topics, video previews