ACM Home Page
Please provide us with feedback. Feedback
The computation of optical flow
Full text PdfPdf (3.06 MB)
Source ACM Computing Surveys (CSUR) archive
Volume 27 ,  Issue 3  (September 1995) table of contents
Pages: 433 - 466  
Year of Publication: 1995
ISSN:0360-0300
Authors
S. S. Beauchemin  Univ. of Western Ontario, London, Ont., Canada
J. L. Barron  Univ. of Western Ontario, London, Ont., Canada
Publisher
ACM  New York, NY, USA
Bibliometrics
Downloads (6 Weeks): 169,   Downloads (12 Months): 1314,   Citation Count: 44
Additional Information:

abstract   references   cited by   index terms   collaborative colleagues  

Tools and Actions: Request Permissions Request Permissions    Review this Article  
DOI Bookmark: Use this link to bookmark this Article: http://doi.acm.org/10.1145/212094.212141
What is a DOI?

ABSTRACT

Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-orderedimages allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image displacements. These are usually called the optical flow field or the image velocity field. Provided that optical flow is a reliable approximation to two-dimensional image motion, it may then be used to recover the three-dimensional motion of the visual sensor (to within a scale factor) and the three-dimensional surface structure (shape or relative depth) through assumptions concerning the structure of the optical flow field, the three-dimensional environment, and the motion of the sensor. Optical flow may also be used to perform motion detection, object segmentation, time-to-collision and focus of expansion calculations, motion compensated encoding, and stereo disparity measurement. We investigate the computation of optical flow in this survey: widely known methods for estimating optical flow are classified and examined by scrutinizing the hypothesis and assumptions they use. The survey concludes with a discussion of current research issues.


REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

 
1
ADELSON, E H. AND BERGEN, J R. 1986. The early detection of motion boundaries. In iEEE Proceedings o/' Workshop on Vtsual Motzon (Charleston, S.C., May). 151 156.
 
2
ADELSON, E H. mNo BERGEN, J. R. 1985 Spatiotemporal energy models for the perception of motion d. Opt Soc Am A2, 2,284 299
 
3
ADR5 G. 1985. Determining three-d~mensmnal motion and structure from optical flow generated by several movmg objects. IEEE PAMI 7, 4, 384 401.
 
4
AGGARWAL, J. K AND NANDHAKUMAR, N 1988 On the computation of motion from sequences of images a review. Prom IEEE 76, S, 917 935
 
5
 
6
ALOIMONOS, J. AND BROWN, C M 1984 Direct processing of curvilinear sensor motion from a sequence of perspective images. In IEEE Proceedzng~ of Workshop on Computer Vlston: Representatmn and Control (Annapolis, Md, April-May), IEEE, 72 77
 
7
ALOiMONOS, J. AND RISTIGOUS, I. 1986 Determining the 3d motion of a m~d planar patch without correspondence, under perspective projection. In IEEE Proceedings of Workshop on Motion, Representatzon and Analyszs (Annapohs, Md., April May), IEEE. 167 174
 
8
ANAND~X5 P. 1989. A computational framework and an algorithm for the measurement of visual motion. Int. J. Comput. -Vzszon 2,283-310.
 
9
 
10
BARMAN, H. 1991. Hmrarchmal curvature estimation m computer vision, Univ. of Linkopmg, Dept. of Electrical Engineering, Ph.D. Thesis.
 
11
BARMAN, H., HAGLUND, L, KNUTSSON, H., AND GRANLUND, G.H. 1991. Estimatmn of velocity, acceleratmn, and disparity m t~me sequences. In IEEE Proceedings of Workshop on Vzsual Motmn (Irvine, Calif., March), 44 51
 
12
BARNARD, S. T. mwD THOMPSON, W. B 1980 Disparity analysis of images IEEE PAMI 2, 4, 333-340.
 
13
BARREN, J. L. AND EAGLESON, R. 1995. Recurs~ve eshmatmn of time-varying motmn and structural parameters. Pattern Recogn. (m press).
 
14
 
15
 
16
BARREN, J. L. AND LIPTAY, A 1994. Optic flow to measure mmute increments in plant growth. Bmlrnagmg 2, 57-61
 
17
 
18
BEAUCHEMIN, S S AND BARRON, J. L 1995 The structure of occlusion in Foumer space In Vls/on Interface (Quebec City, Canada, May) 112-119
 
19
 
20
 
21
 
22
BLACK, M J. 1991. A robust gradient-method for determining optical flow Tech. Rep. YALEU/DCS/RR-891, Yale University, New Haven, CT.
 
23
BLACK, M. J. AND ANANDAN, P. 1993 A framework for robust estimation of optical flow. In Proceedings oflCCV (Berlin, May), 231 236
 
24
BLACK, M J. ANn ANANDAN, P. 1991 Robust dynamic motion estimation over t~me In Proceedrags IEEE CVPR (Los Alamitos, CAI, 296-302.
 
25
BLACK, M J. AND ANANDAN, P. 1990. A model for the detectmn of motmn over t~me. In Proceedrags of ICCV (Osaka, Dec ), 33 37.
 
26
BLACK, M. J. AND JEPSON, A. 1994 Estimating optical flow in segmented nnages using variable-order parametric models with local deforraations. Tech. Rep SPL-94-053, Xerox Systems and Practices Laboratory, Palo Alto, CA
 
27
BOBER, M. AND 14ATTLER, J 1994 Robust morton analyms In IEEE Proceedings of CVPR (Seattle, WA, June), 947-952
 
28
 
29
BURLINA, P. AND CHELLAPPA, R. 1994. Time-to-x: Analysis of motion through temporal parameters. In Proceedings IEEE CVPR (Seattle, WA, June), 461-468
 
30
BURT, P. J. AND ADELSON, E.H. 1983. The Laplaclan pyramid as a compact image code. iEEE Trans. Cornmun. 31,532 540.
 
31
BURT, P. J., HINGORANI, R., AND KOLCZYNSKI, R. J. 1991. Mechanisms for Isolating component patterns in the sequential analysis of multiple motion. In IEEE Proceedings of Workshop on V~sual Morton (Princeton, NJ, Oct.), 187 194.
 
32
BUXTON, B. F. AND BUXTON, H. 1984. Computation of optic flow from the motion of edge features in image sequences. Image Vision Cornpitt 2, 2, 59 75
 
33
CAMPANI, M. AND VERm, A 1990. Computing optical flow from an overconstrained system of linear equations. In Proceedings of ICCV (Osaka, Dec.), 22 26.
 
34
CARPENTIERi, B. AND STORER, J.A. 1992. A splitmerge parallel block-matching algorithm for video displacement estimation. In Data Comprcssion Conference (Snowbird, UT, March), 239-248.
 
35
CmN, T. M., KARL, W., AND WiLLSKY, A. 1994. Probabihstic and sequential computation of optical flow using temporal coherence. IEEE Trans. Image Process. 773 788.
 
36
CHU, C. H. AND DELP, E. J. 1989. Estimating displacement vectors from an image sequence. J. Opt. Soc. Am. A 6, 6, 871-878
 
37
 
38
 
39
DARRELL, T. AND PENTLAND, A. 1991. Robust estimation of a multi-layered motion representation. In IEEE Proceedings of Workshop on Vzsual Motion (Princeton, NJ, Oct.), 173-178.
 
40
DUBOIS, E. 1985. The sampling and reconstruction of time-varying imagery with application in video systems. Proc. IEEE 73, 4. 502 522.
 
41
 
42
DUTTA, R., MANMATHA, R., WILLIAMS, L., AND RISEMAN, E.M. 1989. A data set for quantitative motion analysis. In IEEE Proceedings of CVPR (San Diego, CA, June), 159 164.
 
43
 
44
ENKELMANN, W. 1986. Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences. In IEEE Proceedmgs of Workshop on Motmn: Representation and Analysis (Charleston, SC, May), 81 87.
 
45
FERMIN, I. ANn IMIYA, A. 1994. Two-dimensional motion computation by randomized method. Tech. Rep. TR ICS-4-6-1994, Dept of Information and Computer Sciences, Chiba University, Japan.
 
46
 
47
 
48
FLEET, D. J. AND LANGLEY, K. 1995a. Computational analysis of non-Fourier motion Vision Res. 34, 22, 3057 3079.
 
49
 
50
GEMAN, S. AND GEMAN, D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE PAMI 6, 6 (Nov.), 721 741.
 
51
 
52
GLAZER, F.C. 1987a. Computation of optical flow by multilevel relaxation. Tech. Rep. COINS- TR-87-64, University of Massachusetts.
 
53
GLAZER, F. C. 1987b. Hierarchical gradientbased motion detection. In DARPA Proceedings of Image Understanding Workshop (Los Angeles, CA, Feb.). 733 748.
 
54
 
55
GRZYWACZ, N M. AND YmLLE, A.L. 1990 A model for the estimate of local velocity by cells in the visual cortex. Proc. Royal Soczety London, B 239, 129 161.
 
56
GUPTA, N., RAC~HAVAN, S., AND KANAL, L. 1993. Robust recovery of image motion. In Proceedrags of Asza Conference on Computer Vzs~on (Osaka, Japan, Nov.).
 
57
HADDADI, N. AND KUO, C.-C. J. 1992. Computation of dense optical flow with a parametric smoothness model. Tech. Rep. USC-SIPI Report 223, Dept. of Electrical Engineering Systems, University of Southern Californm, 1992.
 
58
HAGLUND, L. 1992. Adaptive multidimensional filtering. University of Linkoping, Dept. of Electrical Engineering, Ph.D. Thesis
 
59
HANIISCEIACK, P. AND KLETTE, R. 1995. Evaluation of differential methods for image velocity measurement. Comput. Art~f Intell (to appear).
 
60
HANNA, K.J. 1991. Direct multi-resolution estimation of ego-motion and structure from motion In IEEE Workshop on ~%ual Motion (Princeton, NJ), 156 162.
 
61
H,~NA, K. J ~m'D OKAMOTO, N.E. 1993 Combining stereo and motion for direct estimation of scene structure In Proceedings of ICCV (Berlin, May), 357 365.
 
62
H4RALICK, R. M AND LEE, J. S 1982 The facet approach to optic flow In DARPA Proceedings o/Image Understanding Workshop, (Pale Alto, CA, Sept.). 84-93.
 
63
HAY, J.C. 1966 Optical motions and space perception: An extension of Gibson's analysis Psychological Rev 7,3, 6, 550-565.
 
64
HEEGER, D J. 1988. Optmal flow using spatiotemporal filters. Int. d Comput. Vzamn 1, 279 302.
 
65
 
66
 
67
HILDRETH, E C 1984 The computatmn of the velomty field. Prec. Royal SoczeO' of London, B 221, 189 220.
 
68
HORN, B. K P 1987 Motion fields are hardly ever ambiguous Int. J. Comput V~smn J, 259-274.
 
69
HORN, B K. P. AND SCt~UNCK, B. G 1981. Determining optmal flow Artzf. Intell. 17, 185-204
 
70
HORN, B. K. P. ANn WELDON, E.J. 1987. Computat(anally efficient methods for recovering translational motion. In Proceedings of ICCV (London, ,June), 2 11
 
71
 
72
IRANI, M., Rousso, B, AND PELEG, S. 1994. Recovery of egomotion using image stabilization. In CVPR (Seattle, WA, June), 454 460.
 
73
 
74
JAIN, R. C 1984 Segmentatmn of frame sequences obtained by a moving observer IEEE PAMI 6, 5, 624 629.
 
75
JAIN, R.C. 1983. Direct computation of the focus of expansion IEEE PAMI 5, 1, 58-63.
 
76
 
77
JEPSON, A. D. AND BLACK, M. 1993. Mixture models for optical flow computation. In IEEE Proceedmgs of CVPR (New York, June), 760 761.
 
78
 
79
 
80
 
81
Kocm C., WANG, H. T., BATTm, R, MATmT~, B., AND ZIOMKOWSKI, C. 1991. An adaptive multiscale approach for estimating optical flow Computatmnal theory and physiological implementation. In IEEE Proceedings of Workshop ov Vzsua{ Motion (Princeton, NJ, Oct.), 111 122
 
82
Koch, C, WANG. H. T., MATHER, B., HsU, A., AND SVAREZ, H. 1989. Computmg optical flow in resistive networks and m the primate wsual system. In IEEE Proceedings of Workshop on Vzsual Motion (Irwne, CA, Marcht, 62-72.
 
83
KcRIES, R. AND ZIMMERMAN, G 1986 A versatfie method for the estimation of displacement vector fields from image sequences In IEEE Proceedzngs of Workshop on Motzon Representation and Analysis (Charleston, SC, May), 101 1(16
 
84
LANGLEY, K., ATHERTON, T. J., ~VILSON, R. G., AND LACOIVmE, M. H.E. 1991. Vertical and homzontal disparities from phase. Image Vzswn Comput 9, 4, 296-302.
 
85
LIN, T. AND BARriON, J. L. 1994 Image reconstruction error for optma} flow. In Vision Inter- /ace (Banff National Park, Alberta, May), 713 80.
 
86
LIPTAY, A, BARREN, J. L, JEWETT, T., AND VAN WESENBEECK, l. 1995. Optic flow as an ultra-sensitive technique for measurmg seedling growth in long image sequences. J Am Sac. Hart. Scl. 120, 3, 379-385
 
87
LITTLE, J J. 1992. Accurate early detection of dmcontinmtms. In Vision Interface (Vancouver, B.C, Dec k 2-7
 
88
LITTLE, J J, BUTLHOFF, H H, AND PoC;GrO, T. 1988 Parallel optical flow using local voting In Proceedings of ICCV (Tampa, FL, Dec.), 454-459.
 
89
Lru, H., HERMAN, M , HANG, T.-H., AND CHELLAPPA, R. 1993. A reliable optmal flow algorithm using 3-d Hermite polynommls. Tech. Rep. NIST-IR-5333, National Institute of Standards and Technolog%
 
90
LONGUET-HIGGINS, H. C 1984 The visual ambigmty oi a mowng plane Proc Royal Soczet, y of London, B 223, 165 174.
 
91
LONGUET-HIG~;INS, H C 1981 A computer algorithm for reconstructing a scene from two projections Nature, 223, 133 135
 
92
LONC, UET-HIOGINS, H. C. AND PRAZDNY, K. 1980. The interpretation of a moving retinal nnage. Prec. Royal Socze(~ of London, B 208,385-397.
 
93
 
94
LUCAS, B. D. AND KANADE, T. 1981. An iterative image-regSstration technique with an application to stereo vision. In Proceedings of IJCAI (Vancouver, B.C., Aug ), 674-679
 
95
LUETTGEN, M. R., KARL, W. C., AND WILLSKY, A. S. 1994. Efficient multiscale reg~llarization with applications to the computation of optical flow. IEEE Trans. Image Process. 3, 1 (Jan.), 41-64.
 
96
MARKANDEY, V AND FLINCHBAUGH, B. E. 1990. Multispectral constraints for optical flow computation. In Procee&ngs of ICCV (Osaka, Japan, Dec.), 38-41.
 
97
MARR, D. AND HILDRETH, E. C. 1980. Theory of edge detection. Proc. Royal Society Londolz, B 290, 187 212.
 
98
 
99
 
100
MOUNTS, F. W. 1969. A video encoding system using conditional picture-element replenishment. Bell Syst. Tech. J. 48, 2545-2554.
 
101
MUKAWA, N. 1990. Estimation of shape, reflection coefficients and illuminant direction from image sequences. In Proceedzngs of ICCV (Osaka, Japan, Dec.), 507-512.
 
102
 
103
MURRAY, U. W. AND BUXTON, B.F. 1984. Reconstructing the optic flow from edge motion: An examinatloh of two different approaches. First IEEE Conference on AI Apphcations (Denver, CO, 1984), 382 388.
 
104
MUSMANN, H. G., PIRSCH, P., AND GRALLERT, H. J. 1985 Advances in picture coding. Proc. IEEE, 73, 4, 523-548.
 
105
 
106
 
107
 
108
NAGEL, H.-H. 1983a. Constraints for the estimation of vector fields from image sequences. In Proceedzngs IJCAI, (Karlsruhe, Germany, Aug.). 945-951.
 
109
NAaEL, H.-H. 1983b. Displacement vectors derived from second-order intensity variations in image sequences. CVGIP, 21, 85-117.
 
110
 
111
 
112
 
113
NEGAHDARIPOUR, S. AND YU, C.H. 1993. A generalized brightness change model for computing optical flow. In Proceedings ICCV (Berlin, May), 2-11.
 
114
NETRAVALI, A. N AND ROBBINS, J.D. 1979. Motion compensated television coding: Part 1. Bell Syst. Tech. J 58, 631 670.
 
115
OGATA, M. AND SATO, T. 1992. Motmn-detection model with two stages: Spatiotemporal filtering and feature matching. J. Opt. Soc. Am. A 9, 3, 377 387.
 
116
OKUTOMI, M. AND KANADE, T 1990. A locally adaptive window for signal matching'. In ProeeedD~gs of ICCV (Osaka, Japan, Dec.), 190 199
 
117
OVERiNGTON, I 1987 Gradient-based flow segmentation and location of the focus of expansion. In Alvey V~sion Conference (Cambridge Univ., Sept.), 860-870.
 
118
PERRONE, J.A. 1990. Simple techmque for optical flow estimation. J. Opt. Soe. Am. A 7 2, 264 277.
 
119
POGGIO, T., GAMBLE, E., AND LITTLE, J. 1988. Parallel integration of vision modules. Science 242, 436-440.
 
120
PRAZDN~, K. 1985. Detection of binocular disparities. Biol. Cyber,. 52, 93 99
 
121
PP~ZDNY, K. 1979. Motion and structure from optical flow. In Proceedings of IJCAI (Tokyo, Aug.), 702-704.
 
122
PRINCE, J. L. AND MCVEIGH, E.R. 1992. Motion estimation from tagged mr image sequences. IEEE Trans. Medical Images 11, 2,238 249
 
123
REGAN, D. AND BEVERLEY, K.I. 1982. How do we avoid confounding the direction we are looking and the direction we are moving. Science 215, 194 196.
 
124
REICHARDT, W., SCHLOGL, R. W., AND EGELHOAF, M. 1988. Movement detectors of the correlation type provide sufficient information for local computation of 2d velocity fields. Naturwissenschaften 75, 313-315.
 
125
 
126
 
127
 
128
 
129
SCHUNOK, B.G. 1985. Image flow Fundamentals and future research. In IEEE Proceedings of CVPR, (San Francmco, June), 560 571
 
130
 
131
SHfZAWA, M. AND MASE, K 1991 Principle of superposltion: A common computational framework for analysis of multiple motion In IEEE Proceedmgs of Workshop on Visual Motion IPrinceton, NJ., Oct.), 164 172.
 
132
SiMONCELLI, E P., ADELSON, E. H., AND HEEGER, D. J 1991 Probability distributions of optical flow. In IEEE Proceedings of CVPR (Los Alamitos, CA, June), 310 315
 
133
SINGII, A. 1992. Optm flow computation. A urnfled perspectme IEEE Computer Society Press, Los Alamitos, CA.
 
134
SIN~;H, A. 1991 Incremental estimation of image flow using a Kalman filter In IEEE Proceedrags of Workshop on Visual Motion iPrinceton, NJ, Oct.), 36 43.
 
135
SINGH, A. 1990. An estimation-theoretic fkamework Ibr image flow computation. In Proceedrags oflCCV (Osaka, Dec.), 168-177.
 
136
SOBEY, P AND SmNIVASAN, M.V. 1991. Measurement of optical flow by a generalized gradient scheme. J Opt. Soc. Am. A 8, 9, 1488-1498
 
137
 
138
SPETSAKIS, M. E. 1994. An optical flow estimation algorithm that uses Gabor filters and affine model for flow. Tech. Rep., Dept of Computer Science, York University.
 
139
 
140
 
141
SUBBARAO, M. AND WAXMAN, A.M. 1985 On the umqueness of image flow solutions for planar surfaces in motion. In Third Workshop on Computer Vision: Representation and Control (Bellaire, MI, Oct ), 129-140.
 
142
 
143
SUTTON, M. A, WALTERS, W. J., PETERS, W H., RANSON, W F., AND McNEIL, S R. 1983. Determination of disp}acement using an improved digital correlatmn method. Image Vzsmn Cornput 1, 3, 133-139.
 
144
TaOMPSON, W B, MUTCH, K M., .~ND BERZINS, V. A. 1985 Dynaniic occlusion analysis in optical flow fields IEEE PAMI 7, 4, 374 383.
 
145
 
146
 
147
TRETIAK, O. AND PASTOR, L 1984. Velocity estimatron from image sequences with second-order differential operators. In IEEE Proceedings of ICPR IMontreal, Quebec, July Oct.). 20-22.
 
148
Ts^i, R. Y. m'4a HUANG, T. S. 1984. Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces. IEEE PAMI 6, 1, 13 27.
 
149
TSAI, R. Y., HUANG, T. S., aND ZHU, W 1982. Eshmating three-dimensional motion parameters of a rigid planar patch 2. Singular value decomposition. IEEE Trans. Acoustics, Speech Signal Process 30, 4, 525-534.
 
150
ULLMAN, S. 1979. The Interpretatmn of Visual Motion MIT Press, Cambridge, MA.
 
151
URAS, S, Gmosh F, VERRh A., AND TORRE, V 1988 A computational approach to motion perception Biol Cybern 60, 79 87.
 
152
WANG, J. Y. A AND ADELSON, E H. 1994. Representing moving images with layers IEEE Trans. Image Process 3, 5, 625-638.
 
153
WATSON, A B ANn AHUMADA, A. J JR. 1985 Model of human visual motion sensing- J. Opt Soc. Am. A 2, 2, 322-341.
 
154
WAXMAN, A. M. AND WOHN, a. 1985 Contour evolution, nmghborhood deformation, and global image flow: planar surfaces in motion Int. J. Robotms Res. 4, 3, 95-108.
 
155
WAXMAN, A M., Wu, J, ANn BERGHOLM, F. 1988 Convected activation profiles and the measurement of visual motmn. In IEEE Proceedings of CVPR (Ann Arbor, MI, June), 717 723.
 
156
WEBER, J. AND MALIK, J. 1993. Robust computation of optical flow m a multi-scale differentml framework In Proceedmgs of ICCV (Berlin, May), 12-20.
 
157
WENG, J 1990 A theory of image matching. In Proceedings of ICCV (Osaka, Dec.), 200-209.
 
158
WmTTEN, G 1990. A framework for adaptive scale space tracking solutions to prob}ems in computer vision. In Proceedings of ICCV /Osaka, Dec.), 210-220
 
159
WILLICK, D. AND YANG, Y. 1989. Experimental evaluation of motion constraint equatmns. Tech. Rep. 89-4, Dept. of Computational Science, Univ. of Saskatchewan.
 
160
WoomtmxL R.J. 1990. Multiple light source optical flow. In Proceedings oflCCV (Osaka, Dec.), 42 46.
 
161
X~ON% Y. AND SHAFER, S.A. 1994. Moment and hypergeometric filters for high precision computation of focus, stereo and optical flow. Tech. Rep CMU-RI-TR-94-28. Dept. of Computer Science, Carnegie-Mellon Univ.
 
162
 
163
ZHENC~, H. AND BLOSTEIN, S.D. 1993. An errorweighted regularization algorithm for image motion-field estimation. IEEE Trans. Image Process. 2, 2, 246 252.
 
164
ZHENG, Q. AND CHELLAPPA, R. 1993. Automatm feature point extraction and tracking in image sequences for unknown image motion. In Proceedmgs oflCCV (Berlin, May), 335-339.
 
165
ZINNER, H. 1986. Determining the kinematic parameters of a moving imaging sensor by processing spatial and temporal intensity changes. J. Opt. Soc. Am. A 3, 9, 1512-1517.
 
166
ZOLTOWSKt, M.D. 1987 Signal processing applications of the method of total least squares. In IEEE 21st Annual Aszlomar Conference on Signals, Systems, and Computers (Pacific Grove, CA, Nov.), 290 296.

CITED BY  44

Collaborative Colleagues:
S. S. Beauchemin: colleagues
J. L. Barron: colleagues