MonDe: safe updating through monitored deployment of new component versions

ABSTRACT

Safely updating software at remote sites is a cautious balance of enabling new functionality and avoiding adverse effects on existing functionality. A useful first step in this process would be to evaluate the performance of a new version of a component on the current workload before enabling its functionality. This step would let the engineers assess the component's performance over more (and more realistic) data points than by simply performing regression testing in-house.In this paper we propose to evaluate the performance of a new version of a component by (1) deploying it to remote sites, (2) running it in a controlled environment with the actual workloads being generated at that site, and (3) reporting the results back to the development engineers. Running the new version can either be done on-line, alongside the current system, or offline, using capture-replay techniques. By running at the remote site and reporting concise results, issues of data security, protection, and confidentiality are diminished, yet the new version can be evaluated on real workloads.

REFERENCES

• 1: N. Abbas, J. Cook, and S. Tambe. Reliable Runtime Upgrading of Binary C++ Classes. Technical report, New Mexico State University.
• 2: V. Kharchenko, P. Popov, and A. Romanovsky. On Dependability of Composite Web Services with Components Upgraded Online. In Proc. 2004 DSN Workshop on Architecting Dependable Systems, June 2004.
• 3: C. Ko, G. Fink, and K. Levitt. Automated detection of vulnerabilities in privileged programs by execution monitoring. In Proceedings of the 10th Annual Computer Security Applications Conference, pages 134--144, Orlando, FL, 1994. IEEE Computer Society Press.
• 4: Scott Malabarba , Raju Pandey , Jeff Gragg , Earl Barr , J. Fritz Barnes, Runtime Support for Type-Safe Dynamic Java Classes, Proceedings of the 14th European Conference on Object-Oriented Programming, p.337-361, June 12-16, 2000
• 5: A. Mos and J. Murphy. COMPAS: Adaptive Performance Monitoring of Component-Based Systems. In Proc. 2nd ICSE Workshop on Remote Analysis and Measurement of Software Systems, May 2004.
• 6: Alessandro Orso , Taweesup Apiwattanapong , Mary Jean Harrold, Leveraging field data for impact analysis and regression testing, Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, September 01-05, 2003, Helsinki, Finland
• 7: Alessandro Orso , Bryan Kennedy, Selective capture and replay of program executions, Proceedings of the third international workshop on Dynamic analysis, p.1-7, May 17-17, 2005, St. Louis, Missouri
• 8: Andy Podgurski , Elaine J. Weyuker, Re-estimation of software reliability after maintenance, Proceedings of the 19th international conference on Software engineering, p.79-85, May 17-23, 1997, Boston, Massachusetts, United States [doi>10.1145/253228.253245]
• 9: S. Tambe, N. Vedagiri, N. Abbas, and J. Cook. DDL: Extending Dynamic Linking for Program Customization, Analysis, and Evolution. In Proc. International Conference on Software Maintenance, Budapest, Hungary, September 2005. to appear.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General
          Subjects: Protection mechanisms
        D.2.13 Reusable Software
            Subjects: Reusable libraries
      D.2.9 Management
          Subjects: Life cycle

General Terms:
Design, Performance, Theory