Viewable scene modeling for geospatial video search

ABSTRACT

Video sensors are becoming ubiquitous and the volume of captured video material is very large. Therefore, tools for searching video databases are indispensable. Current techniques that extract features purely based on the visual signals of a video are struggling to achieve good results. By considering video related meta-information, more relevant and precisely delimited search results can be obtained. In this study we propose a novel approach for querying videos based on the notion that the geographical location of the captured scene in addition to the location of a camera can provide valuable information and may be used as a search criterion in many applications. This study provides an estimation model of the viewable area of a scene for indexing and searching and reports on a prototype implementation. Among our objectives is to stimulate a discussion of these topics in the research community as information fusion of different georeferenced data sources is becoming increasingly important. Initial results illustrate the feasibility of the proposed approach.

REFERENCES

• 1: Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib doc/.
• 2: Flickr. http://www.flickr.com.
• 3: Geobloggers. http://www.geobloggers.com.
• 4: Woophy. http://www.woophy.com.
• 5: Boris Epshtein , Eyal Ofek , Yonatan Wexler , Pusheng Zhang, Hierarchical photo organization using geo-relevance, Proceedings of the 15th annual ACM international symposium on Advances in geographic information systems, November 07-09, 2007, Seattle, Washington [doi>10.1145/1341012.1341036]
• 6: Clarence H. Graham, Neil R. Bartlett, John Lott Brown, Yun Hsia, Conrad C. Mueller, and Lorrin A. Riggs. Vision and Visual Perception. John Wiley & Sons, Inc., 1965.
• 7: Antonin Guttman, R-trees: a dynamic index structure for spatial searching, Proceedings of the 1984 ACM SIGMOD international conference on Management of data, June 18-21, 1984, Boston, Massachusetts
• 8: Eugene Hecht. Optics. Addison-Wesley Publishing Company, 4th edition, August 2001.
• 9: Tae-Hyun Hwang, Kyoung-Ho Choi, In-Hak Joo, and Jong-Hun Lee. MPEG-7 Metadata for Video-Based GIS Applications. In Geoscience and Remote Sensing Symposium, pages 3641--3643, vol.6, 2003.
• 10: Rieko Kadobayashi , Katsumi Tanaka, 3D viewpoint-based photo search and information browsing, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076158]
• 11: Kyong-Ho Kim, Sung-Soo Kim, Sung-Ho Lee, Jong-Hyun Park, and Jong-Hyun Lee. The Interactive Geographic Video. In Geoscience and Remote Sensing Symposium, pages 59--61, vol.1, 2003.
• 12: Xiaotao Liu , Mark Corner , Prashant Shenoy, SEVA: sensor-enhanced video annotation, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101290]
• 13: Mor Naaman , Yee Jiun Song , Andreas Paepcke , Hector Garcia-Molina, Automatic organization for digital photographs with geographic coordinates, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996366]
• 14: A. Pigeau , M. Gelgon, Building and tracking hierarchical geographical & temporal partitions for image collection management on mobile devices, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101170]
• 15: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 16: Rainer Simon , Peter Fröhlich, A mobile application framework for the geospatial web, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242624]
• 17: Yannis Theodoridis, Michael Vazirgiannis, and Timos Sellis. Spatio-Temporal Indexing for Large Multimedia Applications. In IEEE Intl. Conf. on Multimedia Systems, 1996.
• 18: Carlo Torniai, Steve Battle, and Steve Cayzer. Sharing, Discovering and Browsing Geotagged Pictures on the Web. Springer, 2006.
• 19: Kentaro Toyama , Ron Logan , Asta Roseway, Geographic location tags on digital images, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957046]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Query processing

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques

  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Multimedia databases

General Terms:
Algorithms, Measurement, Performance

Keywords:
georeferencing, gps, meta-data, video search