Automated test data generation for aspect-oriented programs

ABSTRACT

Despite the upsurge of interest in the Aspect-Oriented Programming (AOP) paradigm, there remain few results on test data generation techniques for AOP. Furthermore, there is no work on search-based optimization for test data generation, an approach that has been shown to be successful in other programming paradigms.

In this paper, we introduce a search-based optimization approach to automated test data generation for structural coverage of AOP systems. We present the results of an empirical study that demonstrates the effectiveness of the approach. We also introduce a domain reduction approach for AOP testing and show that this approach not only reduces test effort, but also increases test effectiveness. This finding is significant, because similar studies for non-AOP programming paradigms show no such improvement in effectiveness, merely a reduction in effort. We also present the results of an empirical study of the reduction in test effort achieved by focusing specifically on branches inside aspects.

REFERENCES

• 1: The AOSD-Europe project. http://aosd-europe.net/.
• 2: The AspectOPTIMA aspect-oriented framework. http://www.cs.mcgill.ca/~joerg/SEL/AspectOPTIMA.html.
• 3: R. T. Alexander, J. M. Bieman, and A. A. Andrews. Towards the systematic testing of aspect-oriented programs. Technical Report CS-4-105, Department of Computer Science, Colorado State University, Fort Collins, Colorado, 2004.
• 4: Prasanth Anbalagan , Tao Xie, Automated Generation of Pointcut Mutants for Testing Pointcuts in AspectJ Programs, Proceedings of the 2008 19th International Symposium on Software Reliability Engineering, p.239-248, November 10-14, 2008 [doi>10.1109/ISSRE.2008.58]
• 5: Jon S. Baekken , Roger T. Alexander, A Candidate Fault Model for AspectJ Pointcuts, Proceedings of the 17th International Symposium on Software Reliability Engineering, p.169-178, November 07-10, 2006 [doi>10.1109/ISSRE.2006.6]
• 6: André Baresel , Harmen Sthamer , Michael Schmidt, Fitness Function Design To Improve Evolutionary Structural Testing, Proceedings of the Genetic and Evolutionary Computation Conference, p.1329-1336, July 09-13, 2002
• 7: André Baresel , Harmen Sthamer , Michael Schmidt, Fitness Function Design To Improve Evolutionary Structural Testing, Proceedings of the Genetic and Evolutionary Computation Conference, p.1329-1336, July 09-13, 2002
• 8: Robert V. Binder, Testing object-oriented systems: models, patterns, and tools, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 9: Lionel C. Briand, A Critical Analysis of Empirical Research in Software Testing, Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, p.1-8, September 20-21, 2007 [doi>10.1109/ESEM.2007.8]
• 10: British Standards Institute. BS 7925-2 software component testing, 1998.
• 11: L. A. Clarke, A System to Generate Test Data and Symbolically Execute Programs, IEEE Transactions on Software Engineering, v.2 n.3, p.215-222, May 1976 [doi>10.1109/TSE.1976.233817]
• 12: Bruno Dufour , Christopher Goard , Laurie Hendren , Oege de Moor , Ganesh Sittampalam , Clark Verbrugge, Measuring the dynamic behaviour of AspectJ programs, Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, October 24-28, 2004, Vancouver, BC, Canada
• 13: Fabiano Cutigi Ferrari , José Carlos Maldonado , Awais Rashid, Mutation Testing for Aspect-Oriented Programs, Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, p.52-61, April 09-11, 2008 [doi>10.1109/ICST.2008.37]
• 14: Patrice Godefroid , Nils Klarlund , Koushik Sen, DART: directed automated random testing, Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, June 12-15, 2005, Chicago, IL, USA
• 15: R. Hamlet. Random testing. In Encyclopedia of Software Engineering, pages 970--978. Wiley, 1994.
• 16: Jan Hannemann , Gregor Kiczales, Design pattern implementation in Java and aspectJ, Proceedings of the 17th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, November 04-08, 2002, Seattle, Washington, USA
• 17: Mark Harman , Youssef Hassoun , Kiran Lakhotia , Phil McMinn , Joachim Wegener, The impact of input domain reduction on search-based test data generation, Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, September 03-07, 2007, Dubrovnik, Croatia [doi>10.1145/1287624.1287647]
• 18: A. Baresel, Evolutionary Testing Supported by Slicing and Transformation, Proceedings of the International Conference on Software Maintenance (ICSM'02), p.285, October 03-06, 2002
• 19: Mark Harman , Phil McMinn, A theoretical & empirical znalysis of evolutionary testing and hill climbing for structural test data generation, Proceedings of the 2007 international symposium on Software testing and analysis, July 09-12, 2007, London, United Kingdom [doi>10.1145/1273463.1273475]
• 20: K. Inkumsah and T. Xie. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proc. ASE, pages 297--306, 2008.
• 21: S. Khurshid, C. S. Pasareanu, and W. Visser. Generalized symbolic execution for model checking and testing. In Proc. TACAS, pages 553--568, April 2003.
• 22: Gregor Kiczales , Erik Hilsdale , Jim Hugunin , Mik Kersten , Jeffrey Palm , William G. Griswold, An Overview of AspectJ, Proceedings of the 15th European Conference on Object-Oriented Programming, p.327-353, June 18-22, 2001
• 23: G. Kiczales, J. Lamping, A. Menhdhekar, C. Maeda, C. Lopes, J.-M. Loingtier, and J. Irwin. Aspect-oriented programming. In Proc. ECOOP, pages 220--242, 1997.
• 24: James C. King, Symbolic execution and program testing, Communications of the ACM, v.19 n.7, p.385-394, July 1976 [doi>10.1145/360248.360252]
• 25: B. Korel, Automated Software Test Data Generation, IEEE Transactions on Software Engineering, v.16 n.8, p.870-879, August 1990 [doi>10.1109/32.57624]
• 26: Ramnivas Laddad, AspectJ in Action: Practical Aspect-Oriented Programming, Manning Publications Co., Greenwich, CT, 2003
• 27: Karl Lieberherr , Doug Orleans , Johan Ovlinger, Aspect-oriented programming with adaptive methods, Communications of the ACM, v.44 n.10, p.39-41, Oct. 2001 [doi>10.1145/383845.383855]
• 28: Phil McMinn, Search-based software test data generation: a survey: Research Articles, Software Testing, Verification & Reliability, v.14 n.2, p.105-156, June 2004 [doi>10.1002/stvr.v14:2]
• 29: Phil McMinn , Mark Harman , David Binkley , Paolo Tonella, The species per path approach to SearchBased test data generation, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146241]
• 30: National Institute of Standards and Technology (NIST). The economic impacts of inadequate infrastructure for software testing, May 2002. Planning Report 02-3.
• 31: Parasoft. Jtest manuals version 4.5. Online manual, April 2003. http://www.parasoft.com/.
• 32: R. Pargas, M. Harrold, and R. Peck. Test-data generation using genetic algorithms. Software Testing, Verification and Reliability, 9(4):263--282, 1999.
• 33: Radio Technical Commission for Aeronautics. RTCA DO178-B Software considerations in airborne systems and equipment certification, 1992.
• 34: Venkatesh Prasad Ranganath , Torben Amtoft , Anindya Banerjee , John Hatcliff , Matthew B. Dwyer, A new foundation for control dependence and slicing for modern program structures, ACM Transactions on Programming Languages and Systems (TOPLAS), v.29 n.5, p.27-es, August 2007 [doi>10.1145/1275497.1275502]
• 35: Martin Rinard , Alexandru Salcianu , Suhabe Bugrara, A classification system and analysis for aspect-oriented programs, Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, October 31-November 06, 2004, Newport Beach, CA, USA
• 36: Peri Tarr , Harold Ossher , William Harrison , Stanley M. Sutton, Jr., N degrees of separation: multi-dimensional separation of concerns, Proceedings of the 21st international conference on Software engineering, p.107-119, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302457]
• 37: Paolo Tonella, Evolutionary testing of classes, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 38: Nigel Tracey , John Clark , Keith Mander, Automated program flaw finding using simulated annealing, Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, p.73-81, March 02-04, 1998, Clearwater Beach, Florida, United States
• 39: R. Vallée-Rai, L. Hendren, V. Sundaresan, P. Lam, E. Gagnon, and P. Co. Soot -- a Java optimization framework. In Proc. CASCON, pages 125--135, 1999.
• 40: S. Wappler. Automatic generation of object-oriented unit tests using genetic programming. PhD thesis, Technical University of Berlin, 2008.
• 41: J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43(14):841--854, 2001.
• 42: Mark Weiser, Program slicing, Proceedings of the 5th international conference on Software engineering, p.439-449, March 09-12, 1981, San Diego, California, United States
• 43: Tao Xie , Jianjun Zhao, A framework and tool supports for generating test inputs of AspectJ programs, Proceedings of the 5th international conference on Aspect-oriented software development, March 20-24, 2006, Bonn, Germany [doi>10.1145/1119655.1119681]
• 44: T. Xie, J. Zhao, D. Marinov, and D. Notkin. Automated test generation for AspectJ program. In Proc. WTAOP, 2005.
• 45: Tao Xie , Jianjun Zhao , Darko Marinov , David Notkin, Detecting Redundant Unit Tests for AspectJ Programs, Proceedings of the 17th International Symposium on Software Reliability Engineering, p.179-190, November 07-10, 2006 [doi>10.1109/ISSRE.2006.21]
• 46: Dianxiang Xu , Weifeng Xu, State-based incremental testing of aspect-oriented programs, Proceedings of the 5th international conference on Aspect-oriented software development, March 20-24, 2006, Bonn, Germany [doi>10.1145/1119655.1119680]
• 47: D. Xu, W. Xu, and K. Nygard. A state-based approach to testing aspect-oriented programs. In Proc. SEKE, pages 366--371, 2005.
• 48: Guoqing Xu , Atanas Rountev, Regression Test Selection for AspectJ Software, Proceedings of the 29th international conference on Software Engineering, p.65-74, May 20-26, 2007 [doi>10.1109/ICSE.2007.72]
• 49: Jianjun Zhao, Data-Flow-Based Unit Testing of Aspect-Oriented Programs, Proceedings of the 27th Annual International Conference on Computer Software and Applications, p.188, November 03-06, 2003

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Verification

Keywords:
aspect-oriented software development, evolutionary testing, search-based software engineering, test data generation