Analysis of a deployed software

ABSTRACT

Analyzing a deployed software provides a means to characterize and leverage the software's runtime behavior as it is employed by its intended users. Preliminary studies have shown that leveraging the information obtained from the field provides engineers an opportunity to improve their software testing activities. The analysis of a deployed software can be performed in three stages: (1) the analysis to determine, before the software is deployed, where the instrumentation probes should be inserted into the software and what information that they should capture, (2) the analysis to determine when the field data should be sent back to the company during deployment, and (3) the analysis to leverage the field information after deployment. To make the analysis activities more feasible, we need to take into consideration that there are distinct characteristic differences between the development and the deployed environment. Deployed environment allows for less overhead, provides less control for the engineers, and requires highly scalable techniques due to the high volume of information. Hence, the existing approaches for in-house analysis may become ineffective, inefficient, or even useless when they are directly applied to the deployed environment. Existing approaches for analyzing deployed software also need to be more aware that a technique in one analysis stage may affect the performance of a technique in other analysis stage. This research proposal details the challenges that arise when analyzing a deployed software and seeks to develop a set techniques to address these challenges that can be applied to each stage or across the analysis stages.

REFERENCES

• 1: James F. Bowring , James M. Rehg , Mary Jean Harrold, Active learning for automatic classification of software behavior, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 2: Madeline Diep , Myra Cohen , Sebastian Elbaum, Probe Distribution Techniques to Profile Events in Deployed Software, Proceedings of the 17th International Symposium on Software Reliability Engineering, p.331-342, November 07-10, 2006 [doi>10.1109/ISSRE.2006.36]
• 3: Sebastian Elbaum , Madeline Diep, Profiling Deployed Software: Assessing Strategies and Testing Opportunities, IEEE Transactions on Software Engineering, v.31 n.4, p.312-327, April 2005 [doi>10.1109/TSE.2005.50]
• 4: Sebastian Elbaum , Srikanth Karre , Gregg Rothermel, Improving web application testing with user session data, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 5: Murali Haran , Alan Karr , Michael Last , Alessandro Orso , Adam A. Porter , Ashish Sanil , Sandro Fouche, Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks, IEEE Transactions on Software Engineering, v.33 n.5, p.287-304, May 2007 [doi>10.1109/TSE.2007.1004]
• 6: David M. Hilbert , David F. Redmiles, An approach to large-scale collection of application usage data over the Internet, Proceedings of the 20th international conference on Software engineering, p.136-145, April 19-25, 1998, Kyoto, Japan
• 7: Ben Liblit , Mayur Naik , Alice X. Zheng , Alex Aiken , Michael I. Jordan, Scalable statistical bug isolation, Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, June 12-15, 2005, Chicago, IL, USA
• 8: A. Memon , A. Porter , C. Yilmaz , A. Nagarajan , D. Schmidt , B. Natarajan, Skoll: Distributed Continuous Quality Assurance, Proceedings of the 26th International Conference on Software Engineering, p.459-468, May 23-28, 2004
• 9: Microsoft, "Microsoft Online Crash Analysis," http://oca.microsoft.com/en/Welcome.aspx.
• 10: Netscape, "Netscape Quality Feedback System," home.netscape.com/communicator/navigator/v4.5/qfs1.html.
• 11: Alessandro Orso , James Jones , Mary Jean Harrold, Visualization of program-execution data for deployed software, Proceedings of the 2003 ACM symposium on Software visualization, June 11-13, 2003, San Diego, California [doi>10.1145/774833.774843]
• 12: Alessandro Orso , Taweesup Apiwattanapong , Mary Jean Harrold, Leveraging field data for impact analysis and regression testing, Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, September 01-05, 2003, Helsinki, Finland
• 13: Alessandro Orso , Donglin Liang , Mary Jean Harrold , Richard Lipton, Gamma system: continuous evolution of software after deployment, Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis, July 22-24, 2002, Roma, Italy
• 14: Christina Pavlopoulou , Michal Young, Residual test coverage monitoring, Proceedings of the 21st international conference on Software engineering, p.277-284, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302637]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Experimentation

Keywords:
deployed software, profiling, software analysis