ProDA: a suite of web-services for progressive data analysis

ABSTRACT

Online Scientific Applications (OSA) require statistical analysis of large multidimensional datasets. Towards this end, we have designed and developed a data storage and retrieval system, called ProDA, which deploys wavelet transform and provides fast approximate answers with progressively increasing accuracy in support of the OSA queries. ProDA employs a standard web-service infrastructure to enable remote users to interact with their data. These web-services enable wavelet transformation of large multidimensional datasets as well as inserting, updating, and exact, approximate and progressive querying of these datasets in the wavelet domain. We demonstrate the features of ProDA on a massive atmospheric dataset provided to us by NASA/JPL.

REFERENCES

• 1: ProDA: http://infolab.usc.edu/projects/proda/.
• 2: Mehrdad Jahangiri , Dimitris Sacharidis , Cyrus Shahabi, SHIFT-SPLIT: I/O efficient maintenance of wavelet-transformed multidimensional data, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066189]
• 3: Rolfe R. Schmidt , Cyrus Shahabi, How to evaluate multiple range-sum queries progressively, Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 03-05, 2002, Madison, Wisconsin [doi>10.1145/543613.543630]
• 4: Rolfe R. Schmidt , Cyrus Shahabi, ProPolyne: A Fast Wavelet-Based Algorithm for Progressive Evaluation of Polynomial Range-Sum Queries, Proceedings of the 8th International Conference on Extending Database Technology: Advances in Database Technology, p.664-681, March 25-27, 2002
• 5: C. Shahabi, M. Jahangiri, and D. Sacharidis. Hybrid Query and Data Ordering for Fast and Progressive Range-Aggregate Query Answering. International Journal of Data Warehousing and Mining, 1(2):49--69, April-June 2005.