Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ

ABSTRACT

A three-item after-scenario questionnaire was used in three related usability tests in different areas of the United States. The studies had eight scenarios in common. After participants finished a scenario, they completed the After-Scenario Questionnaire (the ASQ). A factor analysis of the responses to the ASQ items revealed that an eight-factor solution explained 94 percent of the variability of the 24 (eight scenarios by three items per scenario) items. The varimax-rotated factor pattern showed that these eight factors were clearly associated with the eight scenarios. The benefit of this research to system designers is that this three-item questionnaire has acceptable psychometric properties of reliability, sensitivity, and concurrent validity, and may be used with confidence in other, similar usability studies.

REFERENCES

• 1: John P. Chin , Virginia A. Diehl , Kent L. Norman, Development of an instrument measuring user satisfaction of the human-computer interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.213-218, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57203]
• 2: Cliff, N. (1987). <u>Analyzing multivariate data</u>. San Diego, CA: Harcourt Brace Jovanovich.
• 3: Blake Ives , Margrethe H. Olson , Jack J. Baroudi, The measurement of user information satisfaction, Communications of the ACM, v.26 n.10, p.785-793, Oct. 1983 [doi>10.1145/358413.358430]
• 4: Kirakowski, J. and Corbett, M. (1988). <u>The Computer User Satisfaction Inventory (CUSI); Manual and scoring key</u>. Ireland: University College of Cork, Human Factors Research Group.
• 5: Lewis, J. R. (1990). <u>A psychometric evaluation of a post-study system usability questionnaire: The PSSUQ</u> (IBM Tech. Report 54. 535). Boca Raton, FL: IBM Corp.
• 6: McIver, J. P. and Carmines, E. G. (1981). <u>Unidimensional scaling</u>. Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07--024. Beverly Hills, CA: Sage Publications.
• 7: Nunnally, J. C. (1978). <u>Psychometric theory</u>. New York, NY: McGraw-Hill.
• 8: SAS Institute. (1985). <u>SAS user's guide: statistics</u>, version 5 edition. Cary, NC: SAS Institute.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Ergonomics

General Terms:
Human Factors, Measurement, Performance, Theory