Fast image/video upsampling

ABSTRACT

We propose a simple but effective upsampling method for automatically enhancing the image/video resolution, while preserving the essential structural information. The main advantage of our method lies in a feedback-control framework which faithfully recovers the high-resolution image information from the input data, without imposing additional local structure constraints learned from other examples. This makes our method independent of the quality and number of the selected examples, which are issues typical of learning-based algorithms, while producing high-quality results without observable unsightly artifacts. Another advantage is that our method naturally extends to video upsampling, where the temporal coherence is maintained automatically. Finally, our method runs very fast. We demonstrate the effectiveness of our algorithm by experimenting with different image/video data.

REFERENCES

• 1: Shai Avidan , Ariel Shamir, Seam carving for content-aware image resizing, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 2: Baker, S., and Kanade, T. 2000. Limits on super-resolution and how to break them. In CVPR, IEEE Computer Society, 2372--2379.
• 3: Yosuke Bando , Tomoyuki Nishita, Towards Digital Refocusing from a Single Photograph, Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, p.363-372, October 29-November 02, 2007 [doi>10.1109/PG.2007.64]
• 4: Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Eurographics Symposium on Rendering, Eurographics, 327--338.
• 5: Bishop, C. M., Blake, A., and Marthi, B. 2003. Super-resolutioin enhancement of video. In In 9th Conf. on Artificial Intelligence and Statistics.
• 6: Chan, T. F., Osher, S., and Shen, J. 2001. The digital tv filter and nonlinear denoising. IEEE Trans. on Image Processing 10, 2, 231--241.
• 7: Raanan Fattal, Image upsampling via imposed edge statistics, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 8: Rob Fergus , Barun Singh , Aaron Hertzmann , Sam T. Roweis , William T. Freeman, Removing camera shake from a single photograph, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 9: William T. Freeman , Thouis R. Jones , Egon C Pasztor, Example-Based Super-Resolution, IEEE Computer Graphics and Applications, v.22 n.2, p.56-65, March 2002 [doi>10.1109/38.988747]
• 10: Frigo, M., and Johnson, S. G., 2006. FFTW Home Page. WWW page. http://www.fftw.org/.
• 11: Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 [doi>10.1145/383259.383295]
• 12: Huang, J. G., and Mumford, D. 1999. Statistics of natural images and models. In CVPR, 1541--1547.
• 13: Irani, M., and Peleg, S. 1993. Motion analysis for image enhancement: resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation 4, 324--335.
• 14: Keys, R. G. 1981. Cubic convolution interpolation for digital image processing. IEEE Trans. Acoustics, Speech, and Signal Processing 29, 1153--1160.
• 15: Kong, D., Han, M., Xu, W., Tao, H., and Gong, Y. H. 2006. Video super-resolution with scene-specific priors. In BMVC.
• 16: Johannes Kopf , Michael F. Cohen , Dani Lischinski , Matt Uyttendaele, Joint bilateral upsampling, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 17: Mon, D., 2006. Video enhancer. WWW page. http://www.thedeemon.com/VideoEnhancer/.
• 18: Osher, S., Sole, A., and Vese, L. 2003. Image decomposition and restoration using total variation minimization and the H^-1. Multiscale Modeling and Simulation 1, 3, 349--370.
• 19: Patti, A., Sezan, M., and Tekalp, A. 1997. Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. on Image Processing 6, 1064--1076.
• 20: QELabs, 2005. Qe super resolution. WWW page. http://www.qelabs.com/index.
• 21: Schultz, R. R., and Stevenson, R. L. 1996. Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 5, 996--1011.
• 22: Qi Shan , Jiaya Jia , Aseem Agarwala, High-quality motion deblurring from a single image, ACM Transactions on Graphics (TOG), v.27 n.3, August 2008
• 23: Sun, J., Sun, J., Xu, Z. B., and Shum, H. Y. 2008. Image super-rosolution using gradient profile prior. In CVPR.
• 24: Tappen, M. F., Russell, B. C., and Freeman, W. T. 2003. Exploiting the sparse derivative prior for super-resolution and image demosaicing. In Intl. Workshop on Statistical and Computational Theories of Vision.
• 25: Tappen, M. F., Russell, B. C., and Freeman, W. T. 2004. Efficient graphical models for processing images. In CVPR, IEEE Computer Society, 673--680.
• 26: Tipping, M. E., and Bishop, C. M. 2002. Bayesian image super resolution. In NIPS, MIT Press, 1279--1286.
• 27: Lu Yuan , Jian Sun , Long Quan , Heung-Yeung Shum, Progressive inter-scale and intra-scale non-blind image deconvolution, ACM Transactions on Graphics (TOG), v.27 n.3, August 2008
• 28: W. Zhao , Harpreet S. Sawhney, Is Super-Resolution with Optical Flow Feasible?, Proceedings of the 7th European Conference on Computer Vision-Part I, p.599-613, May 28-31, 2002

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.3 Enhancement

Keywords:
image deconvolution, image/video enhancement, image/video upsampling