A hybrid method of image synthesis in IBR for novel viewpoints

ABSTRACT

Due to visibility change and surface enlargement in producing a novel view from a new viewpoint, 3D re-projection from one reference image in IBMR inevitably produces holes in the destination image. Even worse, exposure errors occur when a background region occluded is visible in a desired image because of the absence of some background elements in the reference image. The general solution to this kind of problems is to use multiple images from different viewpoints as input source. By doing so however, the rendering cost would increase with the number of reference images and the composition algorithm has to rely on the z-buffer processing.In fact, plenty of redundant information exists among different reference images. Seeking for a nice way to extract the information needed in the novel view from the reference images is the key issue in solving the problem. In this paper, we propose a new method of image synthesis from multiple reference images. The method combines forward warping and backward warping to fulfil the image composition task for a novel viewpoint. The primary inspiration behind the development of our image synthesis method comes from a fact that the polygon edge geometry may indicate where an exposure and, possible an exposure error occur in the destination image if object silhouettes are prior known. The feature that intersection between scanline and polygons must be in pairs is employed to distinguish holes caused by enlargement of surfaces from those by visibility change. Different heuristic methods are used to choose one image as a primary reference image which shares the most resemblance with the destination image, and other reference images for filling different kinds of holes. Depth continuity along scanline and the depth information already present in the destination image are employed to accelerate the searching process of the corresponding pixels for filling holes.

REFERENCES

• 1: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 2: Leonard McMillan, Jr., An image-based approach to three-dimensional computer graphics, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1997
• 3: Leonard McMillan and Gary Bishop. Head-tracked Stereoscopic Display Using Image Warping. IN S.Fisher, J.Merritt, and B.Bolas, editors, Proceddings SPIE, Volume 2409, Los Angeles, CA, Auguest 1995, pp.39-46.
• 4: Jonathan Shade , Steven Gortler , Li-wei He , Richard Szeliski, Layered depth images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.231-242, July 1998 [doi>10.1145/280814.280882]
• 5: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, September 1993 [doi>10.1145/166117.166153]
• 6: Steven M. Seitz , Charles R. Dyer, View morphing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.21-30, August 1996 [doi>10.1145/237170.237196]
• 7: William R. Mark , Leonard McMillan , Gary Bishop, Post-rendering 3D warping, Proceedings of the 1997 symposium on Interactive 3D graphics, p.7-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253292]
• 8: William R. Mark , Gary Bishop, Postrendering 3d image warping: visibility, reconstruction, and performance for depth-image warping, 1999
• 9: Laveau S. and Faugeras O. 3-D Scene Representation as a Collection of Images. Stereoscopic Displays and Virtual Reality Systems II, Scott S. Fisher, John O. Merritt, Mark T. Bolas, Editors, SPIE Proceedings 2409, San Jose, CA, Feb. 5- 10, 1995, pp.21-30.
• 10: Lawrence W. Chang. Combining Multiple Reference Images in an Inverse Warper{Master D. Thesis}, MIT, 1998
• 11: Gernot Schaufler, Markus Priglinger. Efficient Displacement Mapping by Image Warping. Proceedings of the Eurographics Workshop on Rendering, 1999.

INDEX TERMS

Keywords:
Image-Based modeling and rendering, backward warping, depth-image map, epipolar geometry, forward warping