Psychophysical and metric assessment of fused images

ABSTRACT

The prevalence of image fusion - the fusing of images of different modalities, such as visible and infrared radiation - has increased the demand for accurate methods of image quality assessment. Two traditional methods of assessment that have been used are computational metrics and subjective quality assessment; we propose an alternative task-based method of image assessment, which represents a more accurate description of image 'quality' than subjective ratings. The current study used a signal detection paradigm, identifying the presence or absence of a target in briefly presented images followed by an energy mask, which was compared with computational metric results. In Experiment 1, 18 participants were presented with composites of fused infrared and visible light images, with a soldier either present or not. There were two independent variables, each with three levels: image fusion method (averaging, contrast pyramid, dual-tree complex wavelet transform), and JPEG2000 compression (no compression, low, and high compression), in a repeated measures design. Participants were presented with images and asked to state whether or not they detected the target. In addition, metric results were calculated and compared with task performance. Images were blocked by fusion type, with compression type randomised within blocks. This process was repeated in Experiment 2, but with JPEG images substituted for JPEG2000. The results showed a significant effect for fusion but not compression in JPEG2000 images, whilst JPEG images showed significant effects for both fusion and compression. The metric results for both JPEG and JPEG2000 showed similar trends with more advanced metrics matching the performance of the psychophysical tests more accurately.

REFERENCES

• 1: Breitmeyer, B., and Ogmen, H. 2000. Recent models and findings in visual backward masking: A comparison, review, and update. Perception and Psychophysics 62, 8, 1572--1595.
• 2: Canga, E. F., Dixon, T., Nikolov, S. G., Canagarajah, C. N., Bull, D. R., Troscianko, T., and Noyes, J. 2005. Characterisation of image fusion quality metrics for surveillance applications over bandlimited channels. Proceedings of the Eighth International Conference on Information Fusion (July).
• 3: Cedrus. 1999. Superlab pro version 2.0. website http://www.superlab.com/.
• 4: Ghafourian, M., and Huang, C. 1995. Comparison between several adaptive search vector quantization schemes and jpeg standard for image compression. IEEE Transactions on Communications 43, 24 (February/March/April), 1308--1312.
• 5: Guihong, Q., Dali, Z., and Pingfan, Y. 2001. Medical image fusion by wavelet transform modulus maxima. Optics Express 9, 4 (August), 184.
• 6: Kingsbury, N. G. 1998. The dual-tree complex wavelet transform: a new technique for shift invariance and directional filters. In IEEE Digital Signal Processing Workshop. 86, August.
• 7: Miyahara, M., Kotani, K., and Algazi, V. 1998. Objective picture quality scale (pqs) for image coding. IEEE Transactions on Communications 46, 9 (September), 1215--1226.
• 8: Nikolov, S., Hill, P., Bull, D., and Canagarajah, N. 2001. Wavelets for image fusion. In Wavelets in Signal and Image Analysis, A. Petrosian and F. Meyer, Eds., Computational Imaging and Vision Series. Kluwer Academic Publishers, Dordrecht, The Netherlands, 213--244.
• 9: Petrovic, V., and Xydeas, C. 2000. On the effects of sensor noise in pixel-level image fusion performance. Proceedings of the Third International Conference on Information Fusion 2 (July), 14--19.
• 10: Petrovic, V., and Xydeas, C. 2003. Sensor noise effects on signal-level image fusion performance. Information Fusion 4, 3 (September), 167--183.
• 11: Piella, G., and Heijmans, H. A. 2003. A new quality metric for image fusion. International Conference on Image Processing, ICIP, Barcelona.
• 12: Piella, G. 2003. A general framework for multiresolution image fusion: from pixels to regions. Information Fusion 4, 4 (December), 259--280.
• 13: Pohl, C., and Van Genderen, J. 1998. Multisensor image fusion in remote sensing: Concepts, methods and applications. International Journal of Remote Sensing 19, 5, 823--854.
• 14: Rigolin, V., Robiolio, P., Spero, L. A. and Harrawood, B., Morris, K., Fortin, D., Baker, W., Bashore, T., and Cusma, J. 1996. Compression of digital coronary an-giograms does not affect visual or quantitative assessment of coronary artery stenosis severity. American Journal of Cardiology 78, 2 (July), 131--135.
• 15: Santa-Cruz, D., Grosbois, R., and Ebrahimi, T. 2002. Jpeg 2000 performance evaluation and assessment. Signal Processing: Image Communication 17, 1 (January), 113--130.
• 16: A. N. Skodras , C. A. Christopoulos , T. Ebrahimi, JPEG2000: the upcoming still image compression standard, Pattern Recognition Letters, v.22 n.12, p.1337-1345, October 2001 [doi>10.1016/S0167-8655(01)00079-4]
• 17: Steingrímsson, U., and Simon, K. 2003. Perceptive quality estimations: Jpeg 2000 versus jpeg. Journal of Imaging Science and Technology 47, 6 (November/December), 572--585.
• 18: Stiles, P., and Hofmann, M. 1997. Demonstrated value of data fusion. American Helicopter Society Avionics and Crew Systems Technical Specialist's Meeting. (September), 23--25.
• 19: David S. Taubman , Michael W. Marcellin, JPEG 2000: Image Compression Fundamentals, Standards and Practice, Kluwer Academic Publishers, Norwell, MA, 2001
• 20: Toet, A., and Franken, E. M. 2003. Perceptual evaluation of different image fusion schemes. Displays 24, 1 (February), 25--37.
• 21: Toet, A., IJspeert, J., Waxman, A., and Aguilar, M. 1997. Fusion of visible and thermal imagery improves situational awareness. Displays 18, 85--95.
• 22: Toet, A. 1992. Multiscale contrast enhancement with applications to image fusion. Optical Engineering 31 (May), 1026--1031.
• 23: Tomura, N., Watanabe, O., Omachi, K., Sakuma, I., Takahashi, S., Otani, T., Kidani, H., and Watarai, J. 2004. Image fusion of thallium-201 spect and mr imaging for the assessment of recurrent head and neck tumors following flap reconstructive surgery. European Radiology 14, 7 (July), 1249--1254.
• 24: Wallace, G. 1992. The jpeg still picture compression standard. IEEE Transactions on Consumer Electronics 38, 1 (February), 18--34.
• 25: Wang, Z., and Bovik, A. 2002. A universal image quality index. Signal Processing Letters, IEEE 9, 3 (March), 81--84.
• 26: Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (April), 600--612.
• 27: Wickens, T. 2002. Elementary Signal Detection Theory. Oxford University Press.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.2 PHYSICAL SCIENCES AND ENGINEERING

Additional Classification:
  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES

Keywords:
image fusion, image quality metrics, psychophysical testing