Top

Neural Processing Letters

Published in:

05-08-2020

Unsupervised Optical Flow Estimation Based on Improved Feature Pyramid

Authors: Bo Yang, Huan Xie, Hongbin Li, Nuohan Li, Anchang Liu, Zhigang Ren, Kuan Ye, Rong Zhu, Xuezhi Xiang

Published in: Neural Processing Letters | Issue 2/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Deep learning methods for optical flow estimation usually increase the receptive field of convolution through reducing image resolution, which results in loss of spatial detail information during feature extraction. In this paper, we introduce dilated convolution into feature pyramid network, which can extract multi-scale features containing more motion details and can further improve the accuracy of optical flow estimation. The unsupervised loss function is based on forward–backward consistency check and robust census transform that has a good constraint performance in the case of illumination changes, which can train an unsupervised learning optical flow model with higher accuracy. Our network is trained on FlyingChairs and KITTI raw datasets with an unsupervised manner and tested on MPI-Sintel, KITTI 2012 and KITTI 2015 benchmarks. The experimental results show the advantages of our method in unsupervised learning approaches.

previous article Logic Negation with Spiking Neural P Systems

next article A Dynamic Programming Framework for Large-Scale Online Clustering on Graphs

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Horn B, Schunck B (1981) Determining optical flow. Artif Intell 17:185–203CrossRef

Dakhia A, Wang T, Lu H (2019) Multi-scale pyramid pooling network for salient object detection. Neurocomputing 333:211–220CrossRef

Xu W, Liu W, Huang X, Yang J, Qiu S (2018) Multi-modal self-paced learning for image classification. Neurocomputing 309:134–144CrossRef

Wang F, Ainouz S, Lian C, Bensrhair A (2017) Multimodality semantic segmentation based on polarization and color images. Neurocomputing 253:1–9CrossRef

Dosovitskiy A, Fischer P, Ilg E, Häusser P, Hazirbas C, Golkov V, Smagt P. v. d, Cremers D, Brox T (2015) FlowNet: learning optical flow with convolutional networks. In: 2015 IEEE international conference on computer vision (ICCV), pp 2758–2766

Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: evolution of optical flow estimation with deep networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 1647–1655

Ranjan A, Black MJ (2017) Optical flow estimation using a spatial pyramid network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2720–2729

Sun D, Yang X, Liu M, Kautz J (2018) PWC-net: CNNs for optical flow using pyramid, warping, and cost volume. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 8934–8943

Ahmadi A, Patras I (2016) Unsupervised convolutional neural networks for motion estimation. In: 2016 IEEE international conference on image processing (ICIP), pp 1629–1633

10.

Yu J, Harley A, Derpanis K (2016) Back to basics: unsupervised learning of optical flow via brightness constancy and motion smoothness. In: European conference on computer visio, pp 3–10

11.

Zhu Y, Newsam S (2017) Densenet for dense flow. In: 2017 IEEE international conference on image processing (ICIP), pp 790–794

12.

Huang G, Liu Z, van der Maaten L (2017) Weinberger KQ, Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269

13.

Ren Z, Yan J, Ni B, Liu B, Yang X, Zha H (2017) Unsupervised deep learning for optical flow estimation. In: AAAI conference on artificial intelligence, San Francisco, California

14.

Meister S, Hur J, Roth S (2018) UnFlow: unsupervised learning of optical flow with a bidirectional census loss. In: AAAI conference on artificial intelligence, New Orleans, Louisiana

15.

Wang Y, Yang Y, Yang Z, Zhao L, Wang P, Xu W (2018) Occlusion aware unsupervised learning of optical flow. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 4884–4893

16.

Liu P, King I, Lyu M R, Xu J (2019) DDFlow: learning optical flow with unlabeled data distillation. In: The thirty-third AAAI conference on artificial intelligence, pp 8770–8777

17.

Hafner D, Demetz O, Weickert J (2013) Why is the census transform good for robust optic flow computation? In: Scale space and variational methods in computer vision: 4th international conference, pp 210–221

18.

Tretiak O, Pastor L (1984) Velocity estimation from image sequences with second order differential operators. In: Proceedings of IEEE international conference on pattern recognition, pp 16–19

19.

Brox T, Bruhn A, Papenberg N, Weickert J (2004) High accuracy optical flow estimation based on a theory for warping. In: Conference paper in lecture notes in computer science, pp 25–36

20.

Bruhn A, Weickert J, Schnorr C (2005) Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int J Comput Vis 61(3):211–231CrossRef

21.

Papenberg N, Bruhn A, Brox T, Didas S, Weickert J (2006) Highly accurate optic flow computation with theoretically justified warping. Int J Comput Vis 67(2):141–158CrossRef

22.

Brox T, Malik J (2011) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513CrossRef

23.

Weinzaepfel P, Revaud J, Harchaoui Z, Schmid C (2013) Deepflow: large displacement optical flow with deep matching. In: 2013 IEEE international conference on computer vision, pp 1385–1392

24.

Revaud J, Weinzaepfel P, Harchaoui Z, Schmid C (2015) EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 1164–1172

25.

Nagel H, Enkelmann W (1986) An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. In: IEEE transactions on pattern analysis and machine intelligence PAMI-8(5), pp 565–593

26.

Sun D, Roth S, Black MJ (2010) Secrets of optical flow estimation and their principles. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 2432–2439

27.

Hua M, Bie X, Zhang M, Wang W (2014) Edge-aware gradient domain optimization framework for image filtering by local propagation. In: 2014 IEEE conference on computer vision and pattern recognition, pp 2838–2845

28.

Cheng J, Tsai Y, Wang S, Yang M (2017) Segflow: joint learning for video object segmentation and optical flow. In: 2017 IEEE international conference on computer vision (ICCV), pp 686–695

29.

Gadot D, Wolf L (2016) PatchBatch: a batch augmented loss for optical flow. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 4236–4245

30.

Guney F, Geiger A, Lai SH, Lepetit V, Nishino K, Sato Y (2017) Deep discrete flow, computer vision—ACCV 2016. Springer, Berlin, pp 207–224CrossRef

31.

Dai J, Huang S, Nguyen T (2018) Pyramid structured optical flow learning with motion cues. In: 2018 25th IEEE international conference on image processing (ICIP), pp 3338–3342

32.

Hu P, Wang G, Tan Y (2018) Recurrent spatial pyramid cnn for optical flow estimation. IEEE Trans Multimedia 20(10):2814–2823CrossRef

Title: Unsupervised Optical Flow Estimation Based on Improved Feature Pyramid
Authors: Bo Yang
Huan Xie
Hongbin Li
Nuohan Li
Anchang Liu
Zhigang Ren
Kuan Ye
Rong Zhu
Xuezhi Xiang
Publication date: 05-08-2020
Publisher: Springer US
Published in: Neural Processing Letters / Issue 2/2020
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI: https://doi.org/10.1007/s11063-020-10328-2

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 2/2020

Robust Subspace Clustering via Latent Smooth Representation Clustering

Partial Pinning Control for the Synchronization of Fractional-Order Directed Complex Networks

Pairwise Generalization Network for Cross-Domain Image Recognition

Conditional Generative Adversarial Networks with Multi-scale Discriminators for Prostate MRI Segmentation

Spiking Neural Networks: Background, Recent Development and the NeuCube Architecture

Unsupervised Learning Approach for Abnormal Event Detection in Surveillance Video by Hybrid Autoencoder