Operational BI platform for video analytics

ABSTRACT

Video analytics is a data-intensive and knowledge-rich computation chain from collected video frames to high-level scene and behavior descriptions. The platform separation of video storage and video analysis, as it is now, has become the major bottleneck for scalability, efficiency and effectiveness of video analysis. We solve this problem by (a) completely pushing down video analysis computation to the database engine for fast data access and reduced data transfer; (b) systematically managing domain knowledge and context information, and consistently applying them to video analysis; (c) combining multilevel, multidimensional analytics with data loading for "just-in-time" meta-data materialization; (d) supporting analytical data streaming by database engine, towards a new paradigm for Operational Business Intelligence (OpBI). An OpBI system integrates the management of data, knowledge and analytics programs, along the canonical "eco-chain" of information abstraction, derivation, induction, and feedback.

Then we focus on extending the query engine, the SQL framework and the UDF (User Defined Function) technology to support real-time, process-level and data streaming based OpBI, resulting in a highly efficient system contained entirely in a database system. Our experiment al results reveal that in-DB streaming and materializing meta-data, aggregates and other commonly interested analysis results along data loading, effectively enable near real-time analysis, and thus confirm the advantages of extending DB-engine to support OpBI.

REFERENCES

• 1: Arasu, B. Babcock, S. Babu, M. Datar, K. Ito, I. Nishizawa, J. Rosenstein, J. Widom. "STREAM: The Stanford Stream Data Manager", Proceedings of SIGMOD, 2003.
• 2: Yu Cao, Gopal C. Das, Chee-Yong Chan, Kian-Lee Tan, "Optimizing Complex Queries with Multiple Relation Instances", ACM SIGMOD 2008.
• 3: Qiming Chen, Meichun Hsu, "Data-Continuous SQL Process Model", Proc. CoopIS'08, 2008.
• 4: Qiming Chen, Meichun Hsu, Rui Liu, Weihong Wang, "Scaling-up and Speeding-up Video Analytics Inside Database Engine", Proc. DEXA09, 2009.
• 5: Qiming Chen, Meichun Hsu, Rui Liu, "Extend UDF Technology for Integrated Analy tics", Proc. 11th Int. Conf. on Data Warehousing and Knowledge Discovery (DaWaK '09). 2009.
• 6: Qiming Chen, Meichun Hsu, "Inter-Enterprise Collaborative Business Process Management", Proc. of 17th Int'l Conf on Data Engineering (ICDE-2001), 2001, Germany.
• 7: B. F. Cooper, et. al, "PNUTS: Yahoo!'s Hosted Data Serving Platform", VLDB 2008.
• 8: U. Dayal, Meichun Hsu, R. Ladin, "A Transaction Model for Long-Running Activities", VLDB 1991 (received 10 years award in 2001).
• 9: D. J. DeWitt, E. Paulson, E. Robinson, J. Naughton, J. Royalty, S. Shankar, A. Krioukov, "Clustera: An Integrated Computation And Data Management System", VLDB 2008.
• 10: M. Jaedicke, B. Mitschang, "User-Defined Table Operators: Enhancing Extensibility of ORDBMS", VLDB 1999.
• 11: Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S. Yu, Myung Cheol Doo, "SPADE: The System S Declarative Stream Processing Engine", ACM SIGMOD 2008.
• 12: Marcin Zukowski, S' andor H'eman, Niels Nes, Peter Boncz, "Cooperative Scans: Dynamic Bandwidth Sharing in a DBMS", VLDB 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.3 Languages
          Subjects: Query languages

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Relational databases; Query processing

General Terms:
Algorithms, Design, Languages, Management, Performance

Keywords:
model management, query dataflow process, relation valued function, user defined function