Re-engineering a distributed IR system through a disciplined, task-centered approach based on evaluation and metrics

ABSTRACT

System re-engineering of the Italian Tax Documentation Service smoothly progressed through a phased set of stages:• Definition and refinement of key end-user-profiles through observational and contextual analysis• Previous version in-depth screening through focused usability-lab tests• Interface and interaction design through mock-up screen-shots and interaction graphs• Early prototype construction• Newer and older versions usability-lab pair-wise comparisonThe project served as a test-bed for the Esprit n.20857 Project MAPI (MUSiC Assisted Process Improvement) aimed at engineering usability evaluation and metrics methods and tools, as part of the MUSiC (Measuring Usability in Context) method and toolset. The industrial experience which is reported in this paper, shows that a challenge for the future resides in understanding and modeling how to re-engineer legacy systems and turn the "legacy software life cycle" into a new "user-oriented software life cycle", based on prototyping, early evaluation and metrics collection within a cost-justifying usability approach.

REFERENCES

• 1: Natale D. Quantità e Qualità dei Sistemi Software: Teoria ed Esperienze, F. Angeli/Informatica, 1995.
• 2: Narsim Ganti , William Brayman, The transition of legacy systems to a distributed architecture, Wiley-QED Publishing, Somerset, NJ, 1995
• 3: Downey L. and Laskowski S., Usability Engineering: Industry-Government Collaboration for System Effectiveness and Efficiency, NIST Special Publication 500-237, July 1997.
• 4: Andrew P. Sage , James D. Palmer, Software systems engineering, Wiley-Interscience, New York, NY, 1990
• 5: Randolph G. Bias , Deborah J. Mayhew, Cost-justifying usability, Academic Press, Inc., Orlando, FL, 1994
• 6: A. Rose, B. Shneiderman, C. Plaisant, Using Ethnographic methods in the Re-Design of User Interfaces, CS-TR-3454, University of Maryland, 1995
• 7: Bagnara S. and Citarella F. (eds), L'Usabilità nella Progettazione e Realizzazione di Sistemi Software orientati all'utente - Report finale di sintesi della consulenza SOGEI, aprile 1995.
• 8: Ben Shneiderman, Designing the user interface (2nd ed.): strategies for effective human-computer interaction, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1992
• 9: ISO 9241 "Ergonomics Requirements for office work with visual display terminal (VDT)"- Part 1-17, 1997
• 10: Giannetti A. ESPRIT n.20857 MAPI Deliverable "SOGEI Trial Application Summary Report", December 1996.
• 11: Macleod M., Bowden R., Bevan N. and Curson I. "The MUSiC performance measurement method" in Behaviour and Information Technology, 1997, vol.16, no. 4/5.
• 12: Bowden R. and Curson I., Performance Measurement Handbook, v. 03, National Physical Laboratory DITC, NPL Usability Services, Teddington, Middlesex, UK, 1995
• 13: J. Kirakowski, M. Porteous, M. Corbett - Software Usability Measurement Inventory questionnaire SUMI End-user Handbook, 1993, Human Factors Research Group, Ireland)
• 14: Armellini M, Centonze M., Fiorilli F., Fiorelli F., Giannetti A., Mancieri A., The Tax Documentation Service in Italy, in "Atti della Conferenza RIAO 94", CID-CASIS, Parigi, 1994
• 15: Armellini M., Dassovich P., Giannetti A., Mancieri A. "Il Serviczio di Documentazione Tributaria. utilizzo di tecniche avanzate in un sistema di IR: esperienze e possibili evoluzioni" in Mercatali, Soda, Tiscornia (eds.) Progetti di I.A. per la Pubblica Amministrazione, F. Angeli, 1996.
• 16: Bowden, R.; Thomas, C.; Bevan, N. (1995) Usability Context Analysis: A Practical Guide. v.4.0 National Physical Laboratory DITC, NPL Usability Services, Teddington, Middlesex, UK
• 17: Frederick P. Brooks, Jr., The mythical man-month (anniversary ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 18: James Martin, Rapid application development, Macmillan Publishing Co., Inc., Indianapolis, IN, 1991
• 19: James O. Coplien, A generative development-process pattern language, Pattern languages of program design, ACM Press/Addison-Wesley Publishing Co., New York, NY, 1995
• 20: Ivar Jacobson, Object-Oriented Software Engineering: A Use Case Driven Approach, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 2004
• 21: L. Costantine and L. Lockwood, Rapid Visual Development, tutorial handouts, Rome, 10--12 April 1996.
• 22: Michael A. Cusumano , Richard W. Selby, How Microsoft builds software, Communications of the ACM, v.40 n.6, p.53-61, June 1997 [doi>10.1145/255656.255698]
• 23: Unified Modeling Language Summary, Rational Software Corporation, Santa Clara CA, 1997.
• 24: Jon Bing, Handbook of Legal Information Retrieval, Elsevier Science Inc., New York, NY, 1984
• 25: ISO 13407 User Centered Design Processes for Interactive Systems, Draft International Standard, May 1997.
• 26: Giannetti A. and Mignosi R., Annex D: "Evaluation Module for Usability and Quality in Use", ISO 14598_6 Software Product Evaluation, Committee Draft, January 1997.

INDEX TERMS

Keywords:
MUSiC methods and tools, evaluation and metrics, information retrieval, interaction design, re-engineering of legacy systems, task analysis, usability testing