Locating question difficulty through explorations in question space

ABSTRACT

Three different search effectiveness measures were used to classify 50 question narratives as easy or hard. Each measure was then encoded onto a spatial representation of interquestion similarity. Discriminant analysis based on the resulting map was able to predict question difficulty with approximately 80% accuracy, robust across multiple measures. Implications for the design of digital document collections are discussed.

REFERENCES

• 1: Boyce, B., Meadow, C., and Kraft, D. (1994). Measurement in information science. San Diego, CA: Academic Press.
• 2: Rorvig, M. (1999). Retrieval performance and visual dispersion of query sets. In Voorhees, E., and Harman, D. (Eds.) NIST Special Publication 500-246: The Eighth Text REtrieval Conference (TREC-8).
• 3: Sullivan, T., Norris, C., and Pavur, R. (2000). Visual question maps as search aids. Presented at the IEEE Symposium on Information Visualization 2000.
• 4: Voorhees, E., and Harman, D. (Eds.) (1997). NIST Special Publication 500-240: The Sixth Text REtrieval Conference (TREC-6).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

General Terms:
Design, Documentation, Human Factors, Management, Measurement, Performance, Theory

Keywords:
information visualization, question classification