Towards integrated and efficient scientific sensor data processing: a database approach

ABSTRACT

In this work, we focus on managing scientific environmental data, which are measurement readings collected from wireless sensors. In environmental science applications, raw sensor data often need to be validated, interpolated, aligned and aggregated before being used to construct meaningful result sets. Due to the lack of a system that integrates all the necessary processing steps, scientists often resort to multiple tools to manage and process the data, which can severely affect the efficiency of their work. In this paper, we propose a new data processing framework, HyperGrid, to address the problem. HyperGrid adopts a generic data model and a generic query processing and optimization framework. It offers an integrated environment to store, query, analyze and visualize scientific datasets. The experiments on real query set and data set show that the framework not only introduces little processing overhead, but also provides abundant opportunities to optimize the processing cost and thus significantly enhances the processing efficiency.

REFERENCES

• 1: Rakesh Agrawal , Ashish Gupta , Sunita Sarawagi, Modeling Multidimensional Databases, Proceedings of the Thirteenth International Conference on Data Engineering, p.232-243, April 07-11, 1997
• 2: Guillermo Barrenetxea , François Ingelrest , Gunnar Schaefer , Martin Vetterli , Olivier Couach , Marc Parlange, SensorScope: Out-of-the-Box Environmental Monitoring, Proceedings of the 7th international conference on Information processing in sensor networks, p.332-343, April 22-24, 2008 [doi>10.1109/IPSN.2008.28]
• 3: Peter Baumann, Management of multidimensional discrete data, The VLDB Journal — The International Journal on Very Large Data Bases, v.3 n.4, October 1994
• 4: Peter Baumann , Paula Furtado , Roland Ritsch , Norbert Widmann, Geo/Environmental and Medical Data Management in the RasDaMan System, Proceedings of the 23rd International Conference on Very Large Data Bases, p.548-552, August 25-29, 1997
• 5: M. Chen, R. H. Clayton, A. V. Holden, and J. V. Tucker. Visualising cardiac anatomy using constructive volume geometry. In FIMH, pages 30--38, 2003.
• 6: M. Chen, A. S. Winter, D. Rodgman, and S. Treuvett. Enriching volume modelling with scalar fields. In Data Visualization: The State of the Art, pages 345--362, 2003.
• 7: Zhimin Chen , Vivek Narasayya, Efficient computation of multiple group by queries, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066188]
• 8: Amol Deshpande , Samuel Madden, MauveDB: supporting model-based user views in database systems, Proceedings of the 2006 ACM SIGMOD international conference on Management of data, June 27-29, 2006, Chicago, IL, USA [doi>10.1145/1142473.1142483]
• 9: Bill Howe , David Maier, Algebraic manipulation of scientific datasets, Proceedings of the Thirtieth international conference on Very large data bases, p.924-935, August 31-September 03, 2004, Toronto, Canada
• 10: Leonid Libkin , Rona Machlin , Limsoon Wong, A query language for multidimensional arrays: design, implementation, and optimization techniques, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.228-239, June 04-06, 1996, Montreal, Quebec, Canada
• 11: Eric Lo , Ben Kao , Wai-Shing Ho , Sau Dan Lee , Chun Kit Chui , David W. Cheung, OLAP on sequence data, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376682]
• 12: Arunprasad P. Marathe , Kenneth Salem, Query processing techniques for arrays, The VLDB Journal — The International Journal on Very Large Data Bases, v.11 n.1, p.68-91, August 2002 [doi>10.1007/s007780200062]
• 13: The wolfram mathematica homepage. http://www.wolfram.com/, 2008.
• 14: The mathematica mathlink. http://www.wolfram.com/solutions/mathlink/mathlink.html, 2008.
• 15: Stratos Papadomanolakis , Anastassia Ailamaki , Julio C. Lopez , Tiankai Tu , David R. O'Hallaron , Gerd Heber, Efficient query processing on unstructured tetrahedral meshes, Proceedings of the 2006 ACM SIGMOD international conference on Management of data, June 27-29, 2006, Chicago, IL, USA [doi>10.1145/1142473.1142535]
• 16: Joel E. Richardson , Michael J. Carey, Programming constructs for database system implementation in EXODUS, Proceedings of the 1987 ACM SIGMOD international conference on Management of data, p.208-219, May 27-29, 1987, San Francisco, California, United States
• 17: Hanan Samet, Foundations of Multidimensional and Metric Data Structures (The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 18: Ben Shneiderman, The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations, Proceedings of the 1996 IEEE Symposium on Visual Languages, p.336, September 03-06, 1996
• 19: Ben Shneiderman, Extreme visualization: squeezing a billion records into a million pixels, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, p.3-12, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376618]
• 20: T. Wang, S. Santini, and A. Gupta. An interpolated volume model for databases. In ER, pages 335--348, 2003.