An evaluation of web accessibility metrics based on their attributes

ABSTRACT

Accessibility is a concept related to providing access to Web content to people with different abilities and people using different devices. A number of metrics have been proposed in the last years to help to obtain quantitative Web accessibility levels. Each Web development project has different aspects, and the choice of a given Web accessibility metric should be carefully considered according to the needs of the project. In this paper, we present a review on the main existing Web accessibility metrics, emphasizing the main features and comparing their correlations from experimental data obtained from large scale evaluations. Project managers and evaluators should consider specific features for each given project to effectively choose a proper metric. Needs to adjust customized coefficients or to adhere to automatic evaluation tools are important issues to be observed.

REFERENCES

• 1: C. Benavídez, J. Fuertes, E. Gutiérrez, and L. Martínez. Semi-Automatic Evaluation of Web Accessibility with HERA 2.0. In Proceedings of the 10th International Conference on Computers Helping People with Special Needs (ICCHP 2006), pages 199--206. Springer, 2006
• 2: L. C. Briand, C. M. Differding, and H. D. Rombach. Practical Guidelines for Measurement-Based Process Improvement. Software Process Improvement and Practice Journal, 2(4), 1997.
• 3: C. Buhler, H. Heck, O. Perlick, A. Nietzio, and N. Ulltveit-Moe. Interpreting results from large-scale automatic evaluation of web accessibility. In Computers Helping People with Special Needs, volume 4061/2006, pages 184--191. Springer Berlin / Heidelberg, 2006.
• 4: W. W. A. B. Cluster. Unified Web Evaluation Methodology (UWEM 1.0), 2006. Available at http://www.wabcluster.org/uwem/. Last access on January/2008.
• 5: Michael K. Daskalantonakis, A Practical View of Software Measurement and Implementation Experiences Within Motorola, IEEE Transactions on Software Engineering, v.18 n.11, p.998-1010, November 1992 [doi>10.1109/32.177369]
• 6: Alan Dix , Janet E. Finlay , Gregory D. Abowd , Russell Beale, Human-Computer Interaction (3rd Edition), Prentice-Hall, Inc., Upper Saddle River, NJ, 2003
• 7: Norman E. Fenton , Shari Lawrence Pfleeger, Software Metrics: A Rigorous and Practical Approach, PWS Publishing Co., Boston, MA, 1998
• 8: André P. Freire , Thiago J. Bittar , Renata P. M. Fortes, An approach based on metrics for monitoring web accessibility in Brazilian municipalities web sites, Proceedings of the 2008 ACM symposium on Applied computing, March 16-20, 2008, Fortaleza, Ceara, Brazil [doi>10.1145/1363686.1364259]
• 9: Andre Pimenta Freire , Rudinei Goularte , Renata Pontin de Mattos Fortes, Techniques for developing more accessible web applications: a survey towards a process classification, Proceedings of the 25th annual ACM international conference on Design of communication, October 22-24, 2007, El Paso, Texas, USA [doi>10.1145/1297144.1297177]
• 10: P. Freire, E. H. Tanaka, S. M. A. de Lara, H. V. Rocha, and R. P. M. Fortes. Comparing accessibility in web based e-learning environments: a metric based approach. In Proceedings of the XIII Brazilian Symposium on Multimedia and the Web (Webmedia), pages 49--56. ACM Press, 2007.
• 11: Alfonso Fuggetta , Luigi Lavazza , Sandro Morasca , Stefano Cinti , Giandomenico Oldano , Elena Orazi, Applying GQM in an industrial software factory, ACM Transactions on Software Engineering and Methodology (TOSEM), v.7 n.4, p.411-448, Oct. 1998 [doi>10.1145/292182.292197]
• 12: J. Gonzalez, M. Macias, R. Rodriguez, and F. Sanchez. Accessibility metrics of web pages for blind end-users. In Proceedings of the 2003 International Conference on Web Engineering (ICWE), pages 374--383, Oviedo, Spain, 2003. Springer Berlin / Heidelberg.
• 13: ISO - International Standard Organization, New York, NY. (ISO/IEC 12207) Standard for Information Technology - Software Lifecycle Processes, 1998. 85 S.
• 14: ISO - International Standard Organization. (ISO 9126-1) Software Engineering - Product Quality-Part 1: Quality Model. Geneva, 2001.
• 15: Melody Y. Ivory , Rashmi R. Sinha , Marti A. Hearst, Empirically validated web page design metrics, Proceedings of the SIGCHI conference on Human factors in computing systems, p.53-60, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365035]
• 16: Stephen H. Kan , Brian Thomas, Metrics and Models in Software Quality Engineering, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1994
• 17: Luigi Lavazza , Giancarlo Barresi, Automated support for process-aware definition and execution of measurement plans, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062507]
• 18: Emilia Mendes , Nile Mosley , Steve Counsell, Web Metrics— Estimating Design and Authoring Effort, IEEE MultiMedia, v.8 n.1, p.50-57, January 2001 [doi>10.1109/93.923953]
• 19: Luis Olsina , Gustavo Rossi, Measuring Web Application Quality with WebQEM, IEEE MultiMedia, v.9 n.4, p.20-29, October 2002 [doi>10.1109/MMUL.2002.1041945]
• 20: Bambang Parmanto , Xiaoming Zeng, Metric for web accessibility evaluation, Journal of the American Society for Information Science and Technology, v.56 n.13, p.1394-1404, November 2005 [doi>10.1002/asi.20233]
• 21: Helen Petrie , Fraser Hamilton , Neil King , Pete Pavan, Remote usability evaluations With disabled people, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124942]
• 22: Shari Lawrence Pfleeger, Software Engineering: Theory and Practice, Prentice Hall PTR, Upper Saddle River, NJ, 2001
• 23: Clay I. Spinuzzi, Learning our ABCs: accessibility, bottlenecks, and control in an organized research unit's website, Proceedings of the 25th annual ACM international conference on Design of communication, October 22-24, 2007, El Paso, Texas, USA [doi>10.1145/1297144.1297186]
• 24: Terry Sullivan , Rebecca Matson, Barriers to use: usability and content accessibility on the Web's most popular sites, Proceedings on the 2000 conference on Universal Usability, p.139-144, November 16-17, 2000, Arlington, Virginia, United States [doi>10.1145/355460.355549]
• 25: Markel Vigo , Myriam Arrue , Giorgio Brajnik , Raffaella Lomuscio , Julio Abascal, Quantitative metrics for measuring web accessibility, Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A), May 07-08, 2007, Banff, Canada [doi>10.1145/1243441.1243465]
• 26: W3C. Web Content Accessibility Guidelines 1.0, 1999. Available at http://www.w3c.org/TR/WCAG10. Last access at February/2008.
• 27: W3C. Web Content Accessibility Guidelines 2.0 - Working Draft, 2007. Available at http://www.w3c.org/TR/WCAG20. Last access at February/2008.
• 28: X. Zeng. Evaluation and Enhancement of Web Content Accessibility for Persons with Disabilities. PhD thesis, University of Pittsburgh, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.4 Hypertext/Hypermedia

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General

  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues

General Terms:
Human Factors

Keywords:
accessibility metrics, information accessibility, web accessibility, web engineering