Selection of customers for operational and usage profiling

ABSTRACT

Operational and usage profiles collected from customers provide developers and testers with valuable quantitative information on usage patterns of software being developed. Unfortunately, gathering such profiles from a large set of customers can be challenging due to time and resource constraints.

In this paper we propose to use information about customer defects to narrow down a list of candidate customers to profile. We present a technique for selection and prioritization of minimal set of customers for operational and usage profiling to cover certain set of defects. The technique optimally selects a minimal set of customers for profiling and, once the set is identified, prioritizes the customers within the minimal set. We describe a validation case study confirming that this approach is scalable for a large customer base. Analysis results can then be used to close gaps in testing coverage and to improve the maintenance process.

REFERENCES

• 1: Thomas H. Cormen , Clifford Stein , Ronald L. Rivest , Charles E. Leiserson, Introduction to Algorithms, McGraw-Hill Higher Education, 2001
• 2: Sebastian Elbaum , Gregg Rothermel , Satya Kanduri , Alexey G. Malishevsky, Selecting a Cost-Effective Test Case Prioritization Technique, Software Quality Control, v.12 n.3, p.185-210, September 2004 [doi>10.1023/B:SQJO.0000034708.84524.22]
• 3: Mechelle Gittens , Hanan Lutfiyya , Michael Bauer, An Extended Operational Profile Model, Proceedings of the 15th International Symposium on Software Reliability Engineering, p.314-325, November 02-05, 2004 [doi>10.1109/ISSRE.2004.8]
• 4: Ahmed E. Hassan , Daryl J. Martin , Parminder Flora , Paul Mansfield , Dave Dietz, An Industrial Case Study of Customizing Operational Profiles Using Log Compression, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1379445]
• 5: IBM® ILOG® CPLEX®: http://www.ilog.com/
• 6: Robert Jasper , Mike Brennan , Keith Williamson , Bill Currier , David Zimmerman, Test data generation and feasible path analysis, Proceedings of the 1994 ACM SIGSOFT international symposium on Software testing and analysis, p.95-107, August 17-19, 1994, Seattle, Washington, United States [doi>10.1145/186258.187150]
• 7: John D. Musa, Operational Profiles in Software-Reliability Engineering, IEEE Software, v.10 n.2, p.14-32, March 1993 [doi>10.1109/52.199724]
• 8: Christos H. Papadimitriou , Kenneth Steiglitz, Combinatorial optimization: algorithms and complexity, Prentice-Hall, Inc., Upper Saddle River, NJ, 1982
• 9: Kim Weyns , Per Runeson, Sensitivity of software system reliability to usage profile changes, Proceedings of the 2007 ACM symposium on Applied computing, March 11-15, 2007, Seoul, Korea [doi>10.1145/1244002.1244311]
• 10: Yamany, H. EL and Capretz M. A. M., 2007. A Multi-Agent Framework for Building an Automatic Operational Profile, In Advances and Innovations in Systems, Computing Sciences and Software Engineering, Springer, 161--166

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Reliability

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management
          Subjects: Software quality assurance (SQA)

General Terms:
Algorithms, Measurement, Reliability

Keywords:
customer usage, customers, data mining, operational profile, optimization