Practical issues in subjective video quality evaluation: human factors vs. psychophysical image quality evaluation

ABSTRACT

In this paper, we describe subjective video quality testing using naturally occurring, realistic, video content. The use of realistic content poses a number of challenges that are not encountered when using standard video test clips. The methods that we have developed are targeted to define levels of video quality that will be acceptable to customers of a video service. The methods were developed in the context of home TV viewing, but are applicable to other emerging areas such as mobile video.

Some of the key elements of testing with natural content are controlling for natural content variability, measuring content desirability, and using a testing method that encourages natural viewing behaviors.

REFERENCES

• 1: ITU -T Rec. P.500-11, "Methodology for the subjective assessment of the quality of television pictures," International Telecommunications Union, Geneva, Switzerland, 1996
• 2: Kortum, P. and Sullivan, M. (2004) Content is King: The Effect of Content on the Perception of Video Quality. Proceedings of the Human Factors and Ergonomics Society. Santa Monica, CA: Human Factors and Ergonomics Society.
• 3: ITU- T Rec. P.800, "Methods for subjective determination of transmission quality," International Telecommunications Union, Geneva, Switzerland, 1996.
• 4: Celis, R. L. and Olson, J. C. (2006) The role of Involvement in Attention and Comprehension Processes. The Journal of Consumer Research 15(2) pp
• 5: Rohaly, , A. M., Corriveau, P. J., Libert, J. M., Webster, A. A., Baroncini, V., Beerends, J., Blin, J., Contin, L., Hamada, T., Harrison, D., Hekstra, A. P., Lubin, J., Nishida, Y.,Nishihara, R., Pearson, J. C., Pessoa, A. F., Pickford, N., Schertz, A., Visca, M., Watson, A. B., and Winkler, S. (2000). Video Quality Experts Group: current results and future directions Proc. SPIE Vol. 4067, Visual Communications and Image Processing 2000. p. 742--75

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Psychology

General Terms:
Experimentation, Human Factors, Measurement

Keywords:
mos, subjective video quality