Video tooning

ABSTRACT

We describe a system for transforming an input video into a highly abstracted, spatio-temporally coherent cartoon animation with a range of styles. To achieve this, we treat video as a space-time volume of image data. We have developed an anisotropic kernel mean shift technique to segment the video data into contiguous volumes. These provide a simple cartoon style in themselves, but more importantly provide the capability to semi-automatically rotoscope semantically meaningful regions.In our system, the user simply outlines objects on keyframes. A mean shift guided interpolation algorithm is then employed to create three dimensional semantic regions by interpolation between the keyframes, while maintaining smooth trajectories along the time dimension. These regions provide the basis for creating smooth two dimensional edge sheets and stroke sheets embedded within the spatio-temporal video volume. The regions, edge sheets, and stroke sheets are rendered by slicing them at particular times. A variety of styles of rendering are shown. The temporal coherence provided by the smoothed semantic regions and sheets results in a temporally consistent non-photorealistic appearance.

REFERENCES

• 1: Aseem Agarwala, SnakeToonz: a semi-automatic approach to creating cel animation from video, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508554]
• 2: S. Belongie , J. Malik , J. Puzicha, Shape Matching and Object Recognition Using Shape Contexts, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.4, p.509-522, April 2002 [doi>10.1109/34.993558]
• 3: Christopher M. Christoudias, Synergism in Low Level Vision, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 4, p.40150, August 11-15, 2002
• 4: COLLOMOSSE, J. P., ROWNTREE, D., AND HALL, P. M. 2003. Stroke surfaces: A spatio-temporal framework for temporally coherent non-photorealistic animations. University of Bath, Technical Report CSBU 2003-01 (June 2003).
• 5: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 6: COMANICIU, D., RAMESH, V., AND MEER, P. 2000. Real-time tracking of non-rigid objects using mean shift. In Porc. of IEEE Conf. on Comp. Vis. and Pat. Rec (CVPROO), 142--151.
• 7: Doug DeCarlo , Anthony Santella, Stylization and abstraction of photographs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 8: DEMENTHON, D. 2002. Spatio-temporal segmentation of video by hierarchical mean shift analysis. In Porc. of Statistical Methods in Video Processing Workshop.
• 9: Michael S. Floater, Mean value coordinates, Computer Aided Geometric Design, v.20 n.1, p.19-27, March 2003 [doi>10.1016/S0167-8396(02)00002-5]
• 10: FUKUNAGA, K., AND HOSTETLER, L. 1975. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Information Theory 21, 32--40.
• 11: Aaron Hertzmann , Ken Perlin, Painterly rendering for video and interaction, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.7-12, June 05-07, 2000, Annecy, France [doi>10.1145/340916.340917]
• 12: Aaron Hertzmann, Paint By Relaxation, Computer Graphics International 2001, p.47-54, July 03-06, 2001
• 13: HOCH, M., AND LITWINOWICZ, P. C. 1996. A semi-automatic system for edge tracking with snakes. The Visual Computer 12, 2, 75--83.
• 14: Siu Chi Hsu , Irene H. H. Lee, Drawing and animation using skeletal strokes, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.109-118, July 1994 [doi>10.1145/192161.192186]
• 15: R. Jonker , A. Volgenant, A shortest augmenting path algorithm for dense and sparse linear assignment problems, Computing, v.38 n.4, p.325-340, March 1987 [doi>10.1007/BF02278710]
• 16: KASS, M., WITKIN, A., AND TERZOPOULOS, D. 1987. Snakes: Active contour models. International Journal of Computer Vision 1, 4, 321--331.
• 17: Allison W. Klein , Peter-Pike J. Sloan , Adam Finkelstein , Michael F. Cohen, Stylized video cubes, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545264]
• 18: Alexander Kort, Computer aided inbetweening, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508552]
• 19: LINKLATER, R. 2001. Waking Life DVD. Twentieth Century Fox Home Video.
• 20: Peter Litwinowicz, Processing images and video for an impressionist effect, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.407-414, August 1997 [doi>10.1145/258734.258893]
• 21: William E. Lorensen , Harvey E. Cline, Marching cubes: A high resolution 3D surface construction algorithm, Proceedings of the 14th annual conference on Computer graphics and interactive techniques, p.163-169, August 1987
• 22: WANG, J., THIESSON, B., XUU, Y., AND COHEN, M. F. 2004. Image and video segmentation by anisotropic kernel mean shift. In Proc. European Conference on Computer Vision, 2004.