A new method to evaluate software artifacts against predefined profiles

ABSTRACT

Software artifacts are characterised by many attributes, each one in its turn can be measured by one or more measures. In several cases the software artifact has to be evaluated as a whole, thus raising the problem of aggregating measures to give an overall, single view on the artifact.This paper presents a method to aggregate measures, that works comparing the artifact with predefined, ideal artifacts, or profiles. Profiles are defined starting fromranges of values on measures of attributes. The method is based on two main phases, namely definition of the evaluation model and application of the evaluation model, and is presented in a simplified case study that deals with evaluating the level of quality of an asset to decide if accepting it in a reuse repository. The advantages of the method are that it allows using ordinal scales, while it deals explicitly with preferences expressed, implicitly or explicitly, by the evaluator.

REFERENCES

• 1: Basili V. B., Rombach H. D.(1988). The TAME Project: Towards Improvement-Oriented Software Environments, IEEE Transactions on software engineering, 14,6 (June 88 758-773.
• 2: Blin M. J., Tsoukiàs A., "Evaluation of COTS using multi-criteria methodology", in Proceedings of the 6th European Conference on Software Quality (1999) 429 - 438.
• 3: Marie-José Blin , Alexis Tsoukiàs, Multi-Criteria Methodology Contribution to the Software Quality Evaluation, Software Quality Control, v.9 n.2, p.113-132, June 2001 [doi>10.1023/A:1016626919680]
• 4: Bouyssou D., Marchant Th., Perny P., Pirlot M., Tsoukias A., Vincke Ph., Evaluation and Decision Models: a critical perspective, Kluwer Academic, Dordrecht (2000).
• 5: ISO/IEC JTC1, International Standard 9126 Information Technology - Software Product Evaluation - Quality Characteristics and Guidelines for their Use (1991) Geneva.
• 6: Karlsson, E. A. Software Reuse. John Wiley & Sons (1995).
• 7: Barbara Kitchenham, Towards a constructive quality model: part 1: software quality modelling, measurement and prediction, Software Engineering Journal, v.2 n.4, p.105-113, July 1987
• 8: Jyrki Kontio, A case study in applying a systematic method for COTS selection, Proceedings of the 18th international conference on Software engineering, p.201-209, March 25-29, 1996, Berlin, Germany
• 9: Morisio, M., Tsoukiàs, A. IusWare: A methodology for the evaluation and selection of software products. IEE Proceedings Software Engineering (June 1997) 162-174.
• 10: Paschetta E., Tsoukiàs A., "A real world MCDA application: evaluating software", Journal of Multi-Criteria Decision Analysis, 9 (2000) 205 - 226.
• 11: Roubens M., Vincke, Ph. Preference Modeling, LNEMS 250, Springer Verlag (1985).
• 12: Saaty, T. The analytic hierarchy process. Mc Graw Hill, NY (1980).
• 13: Stamelos, I., Tsoukias, A. Software Evaluation Problem Situations. Cahier du LAMSADE, No 156, Université Paris Dauphine, to appear in European Journal of Operational Research.
• 14: Stamelos I., Vlahavas I.,Refanidis I., Tsoukiàs A., "Knowledge Based Evaluation of Software Systems: a case study", Information and Software Technology, 42 (2000) 333-345.
• 15: Vincke, Ph. Multicriteria Decision Aid. John Wiley (1992).
• 16: Vlahavas I., Refanidis I., Stamelos I., Tsoukiàs A., "ESSE: an expert system for software evaluation", Journal of Knowledge Based Systems, 12 (1999) 183 - 197.
• 17: Yu, W. Aide multicritere a la decision dans le cadre de la problematique du tri: methodes et applications LAMSADE, Universite Paris Dauphine, Paris (1992).

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics

General Terms:
Management, Measurement

Keywords:
multicriteria decision aid, quality models, software evaluation