Automating joiners

ABSTRACT

Pictures taken from different view points cannot be stitched into a geometrically consistent mosaic, unless the structure of the scene is very special. However, geometrical consistency is not the only criterion for success: incorporating multiple view points into the same picture may produce compelling and informative representations. A multi viewpoint form of visual expression that has recently become highly popular is that of joiners (a term coined by artist David Hockney). Joiners are compositions where photographs are layered on a 2D canvas, with some photographs occluding others and boundaries fully visible.

Composing joiners is currently a tedious manual process, especially when a great number of photographs is involved. We are thus interested in automating their construction. Our approach is based on optimizing a cost function encouraging image-to-image consistency which is measured on point-features and along picture boundaries. The optimization looks for consistency in the 2D composition rather than 3D geometrical scene consistency and explicitly considers occlusion between pictures. We illustrate our ideas with a number of experiments on collections of images of objects, people, and outdoor scenes.

REFERENCES

• 1: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 2: Aseem Agarwala , Maneesh Agrawala , Michael Cohen , David Salesin , Richard Szeliski, Photographing long scenes with multi-viewpoint panoramas, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 3: Valmir C. Barbosa , Jayme L. Szwarcfiter, Generating all the acyclic orientations of an undirected graph, Information Processing Letters, v.72 n.1-2, p.71-74, Oct. 29, 1999 [doi>10.1016/S0020-0190(99)00120-9]
• 4: M. Brown , D. G. Lowe, Recognising Panoramas, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1218, October 13-16, 2003
• 5: Peter J. Burt , Edward H. Adelson, A multiresolution spline with application to image mosaics, ACM Transactions on Graphics (TOG), v.2 n.4, p.217-236, October 1983 [doi>10.1145/245.247]
• 6: Paul E. Debevec , Jitendra Malik, Recovering high dynamic range radiance maps from photographs, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.369-378, August 1997 [doi>10.1145/258734.258884]
• 7: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 8: Richard Hartley , Andrew Zisserman, Multiple view geometry in computer visiond, Cambridge University Press, New York, NY, 2001
• 9: Yin Li , Heung-Yeung Shum , Chi-Keung Tang , Richard Szeliski, Stereo Reconstruction from Multiperspective Panoramas, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.1, p.45-62, January 2004 [doi>10.1109/TPAMI.2004.1261078]
• 10: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 11: Pierre Moreels , Pietro Perona, Evaluation of Features Detectors and Descriptors based on 3D Objects, International Journal of Computer Vision, v.73 n.3, p.263-284, July 2007 [doi>10.1007/s11263-006-9967-1]
• 12: Shmuel Peleg , Moshe Ben-Ezra , Yael Pritch, Omnistereo: Panoramic Stereo Imaging, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.3, p.279-290, March 2001 [doi>10.1109/34.910880]
• 13: Patrick Pérez , Michel Gangnet , Andrew Blake, Poisson image editing, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
• 14: Paul Rademacher , Gary Bishop, Multiple-center-of-projection images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.199-206, July 1998 [doi>10.1145/280814.280871]
• 15: Carsten Rother , Sanjiv Kumar , Vladimir Kolmogorov , Andrew Blake, Digital Tapestry, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.589-596, June 20-26, 2005 [doi>10.1109/CVPR.2005.130]
• 16: Carsten Rother , Lucas Bordeaux , Youssef Hamadi , Andrew Blake, AutoCollage, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 17: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 18: Richard Szeliski, Image alignment and stitching: a tutorial, Foundations and Trends® in Computer Graphics and Vision, v.2 n.1, p.1-104, January 2006 [doi>10.1561/0600000009]
• 19: Shinji Umeyama, Least-Squares Estimation of Transformation Parameters Between Two Point Patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.13 n.4, p.376-380, April 1991 [doi>10.1109/34.88573]
• 20: Jingdong Wang , Long Quan , Jian Sun , Xiaoou Tang , Heung-Yeung Shum, Picture Collage, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.347-354, June 17-22, 2006 [doi>10.1109/CVPR.2006.224]
• 21: Daniel N. Wood , Adam Finkelstein , John F. Hughes , Craig E. Thayer , David H. Salesin, Multiperspective panoramas for cel animation, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.243-250, August 1997 [doi>10.1145/258734.258859]
• 22: Lihi Zelnik-Manor , Gabriele Peters , Pietro Perona, Squaring the Circles in Panoramas, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1292-1299, October 17-20, 2005 [doi>10.1109/ICCV.2005.231]
• 23: Zomet, A., Feldman, D., Peleg, S., and Weinshall, D. 2003. Mosaicing new views: The crossed-slits projection. IEEE Trans. on Pattern Analysis and Machine Intelligence (June).

INDEX TERMS

Keywords:
composition, joiner, mosaic, multiple view point, panorama