Model based testing of data constraints: testing the business logic of a Mnesia application with Quviq QuickCheck

ABSTRACT

Correct implementation of data constraints, such as referential integrity constraints and business rules is an essential precondition for data consistency. Though most modern commercial DBMSs support data constraints, the latter are often implemented in the business logic of the applications. This is especially true for non relational DBMS like Mnesia, which do not provide constraints enforcement mechanisms. This case study examines a database application which uses Mnesia as data storage in order to determine, express and test data constraints with Quviq QuickCheck, adopting a model-based testing approach. Some of the important stages of the study described in the article are: reverse engineering of the database, analysis of the obtained database structure diagrams and extraction of data constraint, validation of constraints, formulating the test specifications and finally running the generated test suits. As a result of running the test suits randomly generated by QuickCheck, we have detected several violations of the identified and validated business rules. We have found that the applied methodology is suitable for applications using non relational, unnormalized databases. It is important to note the methodology applied within the case study is not bound to a specific application or DBMS, and can be applied to other database applications.

REFERENCES

• 1: M. Andersson, "Extracting an entity relationship schema from a relational database through reverse engineering," in ER '94: Proceedings of the13th International Conference on the Entity-Relationship Approach, (London, UK), pp. 403--419, Springer--Verlag, 1994.
• 2: J. Armstrong, Programming Erlang: Software for a Concurrent World. Pragmatic Bookshelf, 2007.
• 3: J. Arlat, A. Costes, Y. Crouzet, J. C. Laprie, and D. Powell, "Fault injection and dependability evaluation of fault-tolerant systems," IEEE Trans. Comput., vol. 42, no. 8, pp. 913--923, 1993.
• 4: T. Arts and L. M. Castro, "Testing data consistency of data-intensive applications using quickcheck," Technical report ITU, to be published, 2009.
• 5: T. Arts, J. Hughes, J. Johansson, and U. Wiger, "Testing telecoms software with Quviq QuickCheck," in ERLANG '06:Proc. of the 2006 ACM SIGPLAN workshop on Erlang, pp. 2--10, ACM, 2006.
• 6: N. Bassiliades and I. P. Vlahavas, "Modelling constraints with exceptions in object-oriented databases," in ER '94: Proc. of the13th Int. Conf. on the Entity-Relationship Approach, (London, UK), pp. 189--204, Springer-Verlag, 1994.
• 7: J. Blom and B. Jonsson, "Automated test generation for industrial erlang applications," in ERLANG '03: Proceedings of the 2003 ACM SIGPLAN workshop on Erlang, (New York, NY, USA), pp. 8--14, ACM, 2003.
• 8: M.-C. Boudreau, D. Gefen, and D. W. Straub, "Validation in information systems research: A state-of-the-art assessment," MIS Quarterly, vol. 25, no. 1, pp. 1--16, 2001.
• 9: C. Calero, M. Piattini, and M. Genero, "Empirical validation of referential integrity metrics," Information and Software Technology, vol. 43, no. 15, pp. 949 -- 957, 2001.
• 10: M. Y. Chan and S. C. Cheung, "Testing database applications with sql semantics," in In Proc. of the 2nd Int. Symp. on Cooperative Database Systems for Advanced Applications, pp. 363--374, Springer, 1999.
• 11: W. K. Chan, S. C. Cheung, and T. H. Tse, "Fault--based testing of database application programs with conceptual data model," in QSIC '05: Proc. of the Fifth Int. Conf. on Quality Software, (Washington, DC, USA), pp. 187--196, IEEE Computer Society, 2005.
• 12: D. Chays, Y. Deng, P. G. Frankl, S. Dan, F. I. Vokolos, and E. J. Weyuker, "An agenda for testing relational database applications: Research articles," Softw. Test. Verif. Reliab., vol. 14, no. 1, pp. 17--44, 2004.
• 13: P. P.-S. Chen, "The entity-relationship model toward a unified view of data," ACM Trans. Database Syst., vol. 1, no. 1, pp. 9--36, 1976.
• 14: K. H. Davis and A. K. Arora, "Converting a relational database model into an entity-relationship model," in Proc. of the Sixth Int. Conf. on Entity-Relationship Approach, pp. 271--285, 1988.
• 15: Y. Deng, P. Frankl, and D. Chays, "Testing database transactions with agenda," in ICSE '05: Proceedings of the 27th int. conf. on Software engineering, (New York, NY, USA), pp. 78--87, ACM, 2005.
• 16: J. Dietrich and A. Paschke, "On the test-driven development and validation of business rules," in Information Systems Technology and its Applications, 4th Int. Conf., 23-25 May, 2005, Palmerston North, New Zealand, volume 63 of LNI, pp. 31--48, GI, 2005.
• 17: A. Eisenberg and J. Melton, "Background sql:1999, formerly known as sql3," Commun. ACM, 2008.
• 18: M. Grochtmann and D. benz Ag, "Test case design using classification trees," 1994.
• 19: J.-L. Hainaut, "Database reverse engineering: Models, techniques and strategies," in Proc. Of the 10 th Int. Conf. on Entity-Relationship Approach.
• 20: S. Helke, T. Neustupny, and T. Santen, "Automating test case generation from z specifications with isabelle," in ZUM '97: Proc. of the 10th Int. Conf. of Z Users on The Z Formal Specification Notation, (London, UK), pp. 52--71, Springer-Verlag, 1997.
• 21: H. Huang, W.-T. Tsai, S. Bhattacharya, X. Chen, Y. Wang,and J. Sun, "Business rule extraction techniques for cobol programs," vol. 10, (New York, NY, USA), pp. 3--35, John Wiley & Sons, Inc., 1998.
• 22: W. Kent, "A simple guide to five normal forms in relational database theory," Commun. ACM, vol. 26, no. 2, pp. 120--125, 1983.
• 23: F. N. Kerlinger, Foundations of Behavioral Research. Harcourt Brace Jovanovich, 1986.
• 24: M. Ketabchi, S. Mathur, T. Risch, and J. Chen, "Comparative analysis of rdbms and oodbms: a case study," in Compcon Spring '90. Intellectual Leverage. Digest of Papers. Thirty-Fifth IEEE Comp. Soc. Int. Conf., (New York, NY, USA), pp. 528--537, IEEE, 1990.
• 25: V. V. Kuliamin, "Model based testing of large-scale software: How can simple models help to test complex system," in In Proc. of 1st Int. Symp. on Leveraging Applications of Formal Methods, pp. 311--316, 2004.
• 26: H. Mattsson, H. Nilsson, and C. Wikstrom, "Mnesia - a distributed robust dbms for telecommunications applications," in PADL '99: Proc. of the First Int. Workshop on Practical Aspects of Declarative Languages, (London, UK), pp. 152--163, Springer-Verlag, 1998.
• 27: J. Nielsen and V. L. Phillips, "Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared," in CHI '93: Proc. of the INTERACT '93 and CHI '93 conf. on Human factors in computing systems, pp. 214--221, ACM, 1993.
• 28: M. Phalippou and R. Castanet, "Relations d'implantation et hypotheses de test sur des automates a entrees et sorties = implementation relations and test hypotheses on input-output automata," in Travaux Universitaires -- These nouveau doctorat, (Universite de Bordeaux 1, Talence, FRANCE), 1994.
• 29: W. J. Premerlani and M. R. Blaha, "An approach for reverse engineering of relational databases," Commun. ACM, vol. 37, no. 5, pp. 42--ff., 1994.
• 30: G. J. Tretmans and A. F. E. Belinfante, "Automatic testing with formal methods," Technical Report TR-CTIT-99-17, Enschede, December 1999.
• 31: J. Tretmans, P. Kars, and E. Brinksma, "Protocol conformance testing: A formal perspective on iso is-9646," in Proc. of the IFIP TC6/WG6.1 Fourth Int. Workshop on Protocol Test Systems IV, (Amsterdam, The Netherlands), pp. 131--142, 1992.
• 32: H. Singh, M. Conrad, and S. Sadeghipour, "Test case design based on z and the classification-tree method," in ICFEM '97: Proc. of the 1st Int. Conf. on Formal Engineering Methods, (Washington, DC, USA), p. 81, IEEE Computer Society, 1997.
• 33: S. De Souza, D. R. S. Maldonado, J. C. Fabbri, S. Camargo, P. Ferraz, D. Souza, and W. Lopes, "Mutation testing applied to estelle specifications," Software Quality Control, vol. 8, no. 4, pp. 285--301, 1999.
• 34: I. O. for Standardization, "Information technology, open systems interconnection, conformance testing methodology and framework. international standard is-9646," (Geneve, CH), p.290--294, ISO, 1991.
• 35: G. project, "http://projects.gnome.org/dia/," 2009.
• 36: W. E. Wong and A. P. Mathur, "Reducing the cost of mutation testing: an empirical study," J. Syst. Softw., vol. 31, no. 3, pp. 185--196, 1995.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification

General Terms:
Verification

Keywords:
QuickCheck, amnesia, business rules, erlang, formal specifications, model based testing