Intrinsic colorization

ABSTRACT

In this paper, we present an example-based colorization technique robust to illumination differences between grayscale target and color reference images. To achieve this goal, our method performs color transfer in an illumination-independent domain that is relatively free of shadows and highlights. It first recovers an illumination-independent intrinsic reflectance image of the target scene from multiple color references obtained by web search. The reference images from the web search may be taken from different vantage points, under different illumination conditions, and with different cameras. Grayscale versions of these reference images are then used in decomposing the grayscale target image into its intrinsic reflectance and illumination components. We transfer color from the color reflectance image to the grayscale reflectance image, and obtain the final result by relighting with the illumination component of the target image. We demonstrate via several examples that our method generates results with excellent color consistency.

REFERENCES

• 1: Barrow, H., and Tenenbaum., J. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems, 3--26.
• 2: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 3: Martin A. Fischler , Robert C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Readings in computer vision: issues, problems, principles, and paradigms, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1987
• 4: William T. Freeman , Paul A. Viola, Bayesian model of surface perception, Proceedings of the 1997 conference on Advances in neural information processing systems 10, p.787-793, July 1998, Denver, Colorado, United States
• 5: James Hays , Alexei A. Efros, Scene completion using millions of photographs, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 6: Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. ACM Trans. Graph., 327--340.
• 7: Yi-Chin Huang , Yi-Shin Tung , Jun-Cheng Chen , Sung-Wen Wang , Ja-Ling Wu, An adaptive edge detection based colorization algorithm and its applications, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101223]
• 8: Irony, R., Cohen-Or, D., and Lischinski, D. 2005. Colorization by example. In Rendering Techniques 2005, IEEE Computer Society Press, 201--210.
• 9: Jean-François Lalonde , Derek Hoiem , Alexei A. Efros , Carsten Rother , John Winn , Antonio Criminisi, Photo clip art, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 10: Land, E. H., and McCann, J. J. 1971. Lightness and retinex theory. Journal of the Optical Society of America (1917--1983) 61 (Jan.), 1.
• 11: Anat Levin , Dani Lischinski , Yair Weiss, Colorization using optimization, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 12: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 13: Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. 2007. Natural image colorization. In Rendering Techniques 2007 (Proceedings Eurographics Symposium on Rendering), 309--320.
• 14: Portilla, J., Strela, V., Wainwright, M. J., and Simon-celli, E. P. 2002. Image denoising using gaussian scale mixtures in the wavelet domain. IEEE Transactions on Image Processing 12, 1338--1351.
• 15: Yingge Qu , Tien-Tsin Wong , Pheng-Ann Heng, Manga colorization, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 16: Erik Reinhard , Michael Ashikhmin , Bruce Gooch , Peter Shirley, Color Transfer between Images, IEEE Computer Graphics and Applications, v.21 n.5, p.34-41, September 2001
• 17: Schnitman, Y., Caspi, Y., Cohen-Or, D., and Lischinski, D. 2006. Inducing semantic segmentation from an example. In Asian Conference on Computer Vision, 373--384.
• 18: Jonathan Richard Shewchuk, Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator, Selected papers from the Workshop on Applied Computational Geormetry, Towards Geometric Engineering, p.203-222, May 27-28, 1996
• 19: Noah Snavely , Steven M. Seitz , Richard Szeliski, Photo tourism: exploring photo collections in 3D, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 20: Marshall F. Tappen , William T. Freeman , Edward H. Adelson, Recovering Intrinsic Images from a Single Image, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.9, p.1459-1472, September 2005 [doi>10.1109/TPAMI.2005.185]
• 21: Weiss, Y. 2001. Deriving intrinsic images from image sequences. In Eighth IEEE International Conference on Computer Vision, vol. 2, 68--75.
• 22: Tomihisa Welsh , Michael Ashikhmin , Klaus Mueller, Transferring color to greyscale images, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 23: Yatziv, L., and Sapiro, G. 2006. Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing 15, 5, 1120--1129.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.3 Enhancement

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Color

General Terms:
Design

Keywords:
colorization, intrinsic images