Abstract
We have developed a new algorithm for computing optical flow in a differential framework. The image sequence is first convolved with a set of linear, separable spatiotemporal filter kernels similar to those that have been used in other early vision problems such as texture and stereopsis. The brightness constancy constraint can then be applied to each of the resulting images, giving us, in general, an overdetermined system of equations for the optical flow at each pixel. There are three principal sources of error: (a) stochastic error due to sensor noise (b) systematic errors in the presence of large displacements and (c) errors due to failure of the brightness constancy model. Our analysis of these errors leads us to develop an algorithm based on a robust version of total least squares. Each optical flow vector computed has an associated reliability measure which can be used in subsequent processing. The performance of the algorithm on the data set used by Barron et al. (IJCV 1994) compares favorably with other techniques. In addition to being separable, the filters used are also causal, incorporating only past time frames. The algorithm is fully parallel and has been implemented on a multiple processor machine.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barron, J., Fleet, D., and Beauchemin, S. 1993. “ Performance of optical flow techniques,” Tech. Rep. RPL-TR-9107, Queen's University, Ontario. Revised version of U. Western Ontario TR 299.
Barron, J., Fleet, D., Beauchemin, S., and Burkitt, T. 1994. “Performance of optical flow techniques,”International Journal of Computer Vision, vol. 12, pp. 43–77.
Fleet, D. and Jepson, A. 1990. “Computation of component image velocity from local phase information,”International Journal of Computer Vision, vol. 5, pp. 77–104.
Bergen, J., and Adelson, E. 1988. “Early vision and texture perception,”Nature, vol. 333, pp. 363–364.
Canny, J. 1986. “A computational approach to edge detection,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, pp. 679–698.
Jones, D. and Malik, J. 1992. “Computational framework for determining stereo correspondence from a set of linear spatial filters,”Image and Vision Computing, vol. 10, no. 10, pp. 699–708.
Jones, D. and Malik, J. 1992. “Determining three-dimensional shape from orientation and spatial frequency disparities,” inProceedings of the Second European Conference on Computer Vision, pp. 661–669.
Malik, J. and Perona, P. 1990. “Preattentive texture discrimination with early vision mechanisms,” Journal of the Optical Society of America A, vol. 7, no. 5, pp. 923–932.
Turner, M. 1986. “Texture discrimination by Gabor functions,”Biological Cybernetics, vol. 55, pp. 71–82.
Heeger, D.J. 1988. “Optical flow using spatiotemporal filters,”International Journal of Computer Vision, vol. 1, pp. 279–302.
Fennema, C. and Thompson, W. 1979. “Velocity determination in scenes containing several moving objects,”Computer Graphics and Image Processing, vol. 9, pp. 301–315.
Horn, B. and Schunck, B. 1981. “Determining optical flow,”Artificial Intelligence, no. 17, pp. 185–203.
Tretiak, O. and Pastor, L. 1984. “Velocity estimation from image sequences with second order differential operators,” inProceedings of the International Conference on Pattern Recognition, (Montreal).
Nagel, H.-H. 1987. “On the estimation of optical flow: relations between different approaches and some new results,”Artificial Intelligence, vol. 33, pp. 299–324.
Uras, S., Girosi, F., Verri, A. and Torre, V. 1988. “A computational approach to motion perception,”Biological Cybernetics, vol. 60, pp. 79–87.
Verri, A., Girosi, F. and Torre, V. 1990. “Differential techniques for optical flow,”Journal of the Optical Society of America A, vol. 5, pp. 912–922.
Srinivasan, M. 1990. “Generalized gradient schemes for the measurement of two-dimensional image motion,”Biological Cybernetics, vol. 63, pp. 421–431.
Lucas, B. and Kanade, T. 1981. “An iterative image restoration technique with an application to stereo vision,” inProceedings of the DARPA IU Workshop, pp. 121–130.
Campani, M. and Verri, A. 1990. “Computing optical flow from an overconstrained system of linear algebraic equations,” inProceedings of the 3rd International Conference on Computer Vision, (Osaka), pp. 22–26.
Wang, S., Markandey, V. and Reid, A. 1992. “Total least squares fitting spatiotemporal derivatives to smooth optical flow fields,” inProceedings of the SPIE: Signal and Data Processing of Small Targets, vol. 1698, pp. 42–55.
VanHuffel, S. and Vandewalle, J. 1991.The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics, Philadelphia: SIAM.
Duda, R. and Hart, P. 1973.Pattern Classification and Scene Analysis, New York, Chichester, Brisbane, Toronto, Singapore: John Wiley & Sons.
Pearson, K. 1901. “On lines and planes of closest fit to points in space,”Philos. Mag., vol. 2, pp. 559–572.
Madansky, A. 1959. “The fitting of straight lines when both variables are subject to error,”/. Amer. Statist. Assoc., vol. 54, pp. 173–205.
Sprent, P. 1969. Models in Regression and Related Topics. London: Methuen.
Golub, G. and VanLoan, C. 1980. “An analysis of the total least squares problem,”SIAM Journal Numer. Anal., vol. 17, pp. 883–893.
Shizawa, M. and Mase, K. 1990. “Simultaneous multiple optical flow estimation,” inProceedings of the 10th International Conference on Pattern Recognition (Atlantic City, New Jersey), pp. 274–278.
Simoncelli, E., Adelson, E. and Heeger, D. 1991. “Probability distributions of optical flow,” inProceedings of the IEEE Computer Vision and Pattern Recognition Conference, pp. 310–315.
Cleary, R. and Braddick, O. 1990. “Directional discrimination for band-pass filtered random dot kinematograms,”Vision Research, vol. 30, pp. 303–316.
Battiti, R., Amaldi, E. and Koch, C. 1991. “Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy,“International Journal of Computer Vision, vol. 6, no. 2, pp. 133–145.
Young, R. 1985. “The gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field line-weighting profiles,” Technical Report GMR-4920, General Motors Research.
Black, M.J. and Anandan, P. 1993. “A framework for the robust estimation of optical flow,” inProceedings of the Fourth ICCV, (Berlin), pp. 231–236.
Huber, P.J. 1981.Robust Statistics. Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons.
Weber, J. and Malik, J. 1992. “Robust computation of optical flow in a multi-scale differential framework,” Tech. Rep. UCB/CSD 92/709, Computer Science Division (EECS), University of California, Berkeley.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Weber, J., Malik, J. Robust computation of optical flow in a multi-scale differential framework. Int J Comput Vision 14, 67–81 (1995). https://doi.org/10.1007/BF01421489
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01421489