Requirements modelling and evaluation for digital preservation: a COTS selection method based on controlled experimentation

ABSTRACT

Most methods for the general problem of Commercial-off-the-shelf component selection use goal-oriented requirements modelling and multi-criteria decision making techniques and are applicable across a wide range of domains. This usually implies high levels of complexity. Recently a very specific selection problem emerged in the context of digital preservation. The selection of the most suitable tool to keep a type of digital object alive when the original technical environment ceases to exist is a highly complex domain-specific selection problem with several peculiarities: Highly homogeneous functionality across tools, complex evaluation of quality across settings, and a high need for automation, standardisation, and documentation. This paper describes an evidence-based empirical methodology for COTS component selection in digital preservation through controlled experimentation. We describe the specific selection problem, show how the process of utility analysis can be tailored to fit the problem space and describe the methodology, which is geared towards automated evaluation in an empirical setting. We outline existing tool support and discuss case studies and future directions.

REFERENCES

• 1: Christoph Becker , Hannes Kulovits , Andreas Rauber , Hans Hofman, Plato: a service oriented decision support system for preservation planning, Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries, June 16-20, 2008, Pittsburgh PA, PA, USA [doi>10.1145/1378889.1378954]
• 2: Christoph Becker , Andreas Rauber , Volker Heydegger , Jan Schnasse , Manfred Thaller, A generic XML language for characterising objects to support digital preservation, Proceedings of the 2008 ACM symposium on Applied computing, March 16-20, 2008, Fortaleza, Ceara, Brazil [doi>10.1145/1363686.1363786]
• 3: ISO. Open archival information system -- Reference model (ISO 14721: 2003), 2003.
• 4: Abdallah Mohamed , Guenther Ruhe , Armin Eberlein, COTS Selection: Past, Present, and Future, Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, p.103-114, March 26-29, 2007 [doi>10.1109/ECBS.2007.28]
• 5: Cornelius Ncube , John C. Dean, The Limitations of Current Decision-Making Techniques in the Procurement of COTS Software Components, Proceedings of the First International Conference on COTS-Based Software Systems, p.176-187, February 04-06, 2002
• 6: Stephan Strodl , Christoph Becker , Robert Neumayer , Andreas Rauber, How to choose a digital preservation strategy: evaluating a preservation planning procedure, Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, June 18-23, 2007, Vancouver, BC, Canada [doi>10.1145/1255175.1255181]
• 7: Weirich, P., Skyrms, B., Adams, E., Binmore, K., Butterfield, J., Diaconis, P., and Harper, W. Decision Space: Multidimensional Utility Analysis. Cambridge University Press, 2001.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications

General Terms:
Design, Documentation, Experimentation, Measurement

Keywords:
COTS selection, decision support tool, empirical evaluation, requirements engineering, utility analysis