Simulation data as data streams

ABSTRACT

Computational or scientific simulations are increasingly being applied to solve a variety of scientific problems. Domains such as astrophysics, engineering, chemistry, biology, and environmental studies are benefiting from this important capability. Simulations, however, produce enormous amounts of data that need to be analyzed and understood. In this overview paper, we describe scientific simulation data, its characteristics, and the way scientists generate and use the data. We then compare and contrast simulation data to data streams. Finally, we describe our approach to analyzing simulation data, present the AQSim (Ad-hoc Queries for Simulation data) system, and discuss some of the challenges that result from handling this kind of data.

REFERENCES

• 1: Louis, S., The NNSA ASCI Program: Advanced Simulation and Computing, presented at the October 9, 2001 THIC Meeting WestCoast Silverdale Hotel, Silverdale WA 98383-9191
• 2: Ron Musick , Terence Critchlow, Practical lessons in supporting large-scale computational science, ACM SIGMOD Record, v.28 n.4, p.49-57, Dec.1999 [doi>10.1145/344816.344860]
• 3: Ghaleb Abdulla , Chuck Baldwin , Terence Critchlow , Roy Kamimura , Ida Lozares , Ron Musick , Nu Ai Tang , Byung S. Lee , Robert Snapp, Approximate ad-hoc query engine for simulation data, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.255-256, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379673]
• 4: Chuck Baldwin , Ghaleb Abdulla , Terence Critchlow, Multi-resolution modeling of large scale scientific simulation data, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956872]
• 5: Tina Eliassi-Rad , Terence Critchlow , Ghaleb Abdulla, Statistical modeling of large-scale simulation data, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775118]
• 6: B. Lee , T. Critchlow , G. Abdulla , C. Baldwin , R. Kamimura , R. Musick , R. Snapp , N. Tang, The framework for approximate queries on simulation data, Information Sciences—Informatics and Computer Science: An International Journal, v.157 n.1-2, p.3-20, December 2003 [doi>10.1016/S0020-0255(03)00185-3]
• 7: Brian Babcock , Shivnath Babu , Mayur Datar , Rajeev Motwani , Jennifer Widom, Models and issues in data stream systems, Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 03-05, 2002, Madison, Wisconsin [doi>10.1145/543613.543615]
• 8: Chen, Y.; Dong, G.; Han, J.; Pei, J.; Wah, B. W; Wang, J.; Online Analytical Processing Stream Data: Is It Feasible? 2002 Workshop on Research Issues in Data Mining and Knowledge Discovery (DMKD'2002).
• 9: David M. Malon , Edward N. May, Critical Database Technologies for High Energy Physics, Proceedings of the 23rd International Conference on Very Large Data Bases, p.580-584, August 25-29, 1997
• 10: Eliassi-Rad, T., Critchlow, T., Multivariate Clustering of Large-Scale Simulation Data, LLNL Technical Report, UCRL-JC-151860, 2003.
• 11: Butler, D. M., Pendley, M. H., A Visualization Model based on the Mathematics of Fiber Bundles, Computers in Physiscs, September/October 1989.
• 12: Laszewski, G., Insley, J. A., Foster, I, Bresnahan, J., Kesselman, C., Su, M., Thiebaux, M., Rivers, M. L., Wang, S., Tieman, B., McNutly, I., Real-Time Analysis, Visualization, and Steering of Microtomography Experiments at Photon Sources, In Proceedings of Ninth SIAM Conference on Parallel Processing for Scientific Computing, (San Antonio, TX, March 1999).