Skip to main content
Erschienen in: Multimedia Systems 3/2018

25.03.2017 | Regular Paper

Restricted affine motion compensation and estimation in video coding with particle filtering and importance sampling: a multi-resolution approach

verfasst von: Mithilesh Kumar Jha, Ravi Chaudhary, Sumantra Dutta Roy, Mona Mathur, Brejesh Lall

Erschienen in: Multimedia Systems | Ausgabe 3/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose a multi-resolution affine block-based tracker for motion estimation and compensation, compatible with existing video coding standards such as H.264 and HEVC. We propose three modifications to traditional motion compensation techniques in video coding standards such as H.264 and HEVC. First, we replace traditional search methods with an efficient particle filtering-based method, which incorporates information from both spatial and temporal continuity. Second, we use a higher order linear model in place of the traditional translation motion model in these standards to efficiently represent complex motions such as rotation and zoom. Third, we propose a multi-resolution framework that enables efficient parameter estimation. Results of extensive experimentation show reduced residual energy and better Peak Signal-to-Noise Ratio (PSNR, hereafter) as compared to H.264/HEVC for instance, especially in regions of complex motion such as zooming and rotation.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat ITU-T Rec. H.262 and ISO/IEC 13818-2 (MPEG-2): Generic coding of moving pictures and associated audio information - Part 2: Video (1994) ITU-T Rec. H.262 and ISO/IEC 13818-2 (MPEG-2): Generic coding of moving pictures and associated audio information - Part 2: Video (1994)
2.
Zurück zum Zitat ITU-T Rec. H.264 and ISO/IEC 14496-10 (MPEG4-AVC): Advanced video coding for generic audiovisual services (v1, May 2003; v2, Jan. 2004; v3, Sept. 2004; v4, July 2005) ITU-T Rec. H.264 and ISO/IEC 14496-10 (MPEG4-AVC): Advanced video coding for generic audiovisual services (v1, May 2003; v2, Jan. 2004; v3, Sept. 2004; v4, July 2005)
3.
Zurück zum Zitat ITU-T Rec. H.265 and ISO/IEC 23008-2 (HEVC): High Efficiency Video Coding (v1, Apr. 2013; v2, Oct. 2014; v3, Apr. 2015) ITU-T Rec. H.265 and ISO/IEC 23008-2 (HEVC): High Efficiency Video Coding (v1, Apr. 2013; v2, Oct. 2014; v3, Apr. 2015)
4.
Zurück zum Zitat Minezawa, A., Sekiguch, S., Sugimoto, K.: Te12.2 report (m18142) on motion vector (MV) prediction AMVP/IMVP. In: Mitsubishi Electric, Technical Report JCTVC-C119 (2010) Minezawa, A., Sekiguch, S., Sugimoto, K.: Te12.2 report (m18142) on motion vector (MV) prediction AMVP/IMVP. In: Mitsubishi Electric, Technical Report JCTVC-C119 (2010)
5.
Zurück zum Zitat Kamp, S.: Video coding using decoder-side motion vector derivation. In: RWTH Aachen University, Germany, Technical Report (Online) (2008) Kamp, S.: Video coding using decoder-side motion vector derivation. In: RWTH Aachen University, Germany, Technical Report (Online) (2008)
6.
Zurück zum Zitat Zhang, K., Bober, M., Kittler, J.: Video coding using affine motion compensated prediction. In: Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1978–1981 (1996) Zhang, K., Bober, M., Kittler, J.: Video coding using affine motion compensated prediction. In: Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1978–1981 (1996)
7.
Zurück zum Zitat Wiegand, T., Steinbach, E., Girod, B.: Affine multi-picture motion-compensated prediction. IEEE Trans. Circ. Syst. Video Technol. 15(2), 197–209 (2005)CrossRef Wiegand, T., Steinbach, E., Girod, B.: Affine multi-picture motion-compensated prediction. IEEE Trans. Circ. Syst. Video Technol. 15(2), 197–209 (2005)CrossRef
8.
Zurück zum Zitat Kordasiewicz, R.C., Gallant, M.D., Shirani, S.: Affine motion prediction based on translational motion vectors. IEEE Trans. Circ. Syst. Video Technol. 17(11), 1388–1394 (2007)CrossRef Kordasiewicz, R.C., Gallant, M.D., Shirani, S.: Affine motion prediction based on translational motion vectors. IEEE Trans. Circ. Syst. Video Technol. 17(11), 1388–1394 (2007)CrossRef
9.
Zurück zum Zitat Cheung, H.K., Siu, W.C.: Local affine motion prediction for H.264 without extra overhead. In: IEEE International Symposium on Circuits and Systems, pp. 1555–1558 (2010) Cheung, H.K., Siu, W.C.: Local affine motion prediction for H.264 without extra overhead. In: IEEE International Symposium on Circuits and Systems, pp. 1555–1558 (2010)
10.
Zurück zum Zitat Yuan, H., Liu, J., Sun, J., Liu, H., Li, Y.: Affine model based motion compensation prediction for zoom. IEEE Trans Multimed 14(4), 1370–1375 (2012)CrossRef Yuan, H., Liu, J., Sun, J., Liu, H., Li, Y.: Affine model based motion compensation prediction for zoom. IEEE Trans Multimed 14(4), 1370–1375 (2012)CrossRef
11.
Zurück zum Zitat Kuo, C.M., Hsiesg, C., Jou, Y.D., Lin, H.C., Liu, P.C.: Motion estimation for video compression using Kalman filtering. IEEE Trans. Broadcast. 42(2), 111–116 (1996) Kuo, C.M., Hsiesg, C., Jou, Y.D., Lin, H.C., Liu, P.C.: Motion estimation for video compression using Kalman filtering. IEEE Trans. Broadcast. 42(2), 111–116 (1996)
12.
Zurück zum Zitat Kuo, C.M., Chung, S.C., Shih, P.Y.: Kalman filtering based rate-constrained motion estimation for very low bit rate video coding. IEEE Trans. Circ. Syst. Video Technol. 16(1), 3–18 (2006)CrossRef Kuo, C.M., Chung, S.C., Shih, P.Y.: Kalman filtering based rate-constrained motion estimation for very low bit rate video coding. IEEE Trans. Circ. Syst. Video Technol. 16(1), 3–18 (2006)CrossRef
13.
Zurück zum Zitat Luo, Y., Celenk, M.: Kalman filtering based motion estimation for video coding with adaptive block partitioning. In: IEEE Workshop on Signal Processing Systems, pp. 129–134 (2008) Luo, Y., Celenk, M.: Kalman filtering based motion estimation for video coding with adaptive block partitioning. In: IEEE Workshop on Signal Processing Systems, pp. 129–134 (2008)
14.
Zurück zum Zitat Yang, S.: Particle filtering based estimation of consistent motion and disparity with reduced search points. IEEE Trans. Circ. Syst. Video Technol. 22(1), 91–104 (2012)CrossRef Yang, S.: Particle filtering based estimation of consistent motion and disparity with reduced search points. IEEE Trans. Circ. Syst. Video Technol. 22(1), 91–104 (2012)CrossRef
15.
Zurück zum Zitat Kwolek, B.: Face tracking for H.264 encoded video sequences. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 449–452 (2005) Kwolek, B.: Face tracking for H.264 encoded video sequences. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 449–452 (2005)
16.
Zurück zum Zitat Chung, K.L., Yao, T.J.: New prediction and affine transformation—based three-step search scheme for motion estimation with applications. J. Inf. Sci. Eng. 24, 1095–1109 (2008) Chung, K.L., Yao, T.J.: New prediction and affine transformation—based three-step search scheme for motion estimation with applications. J. Inf. Sci. Eng. 24, 1095–1109 (2008)
17.
Zurück zum Zitat Jing, X., Chau, L.P.: An efficient three-step search algorithm for block motion estimation. IEEE Trans. Multimed. 6, 435–438 (2004)CrossRef Jing, X., Chau, L.P.: An efficient three-step search algorithm for block motion estimation. IEEE Trans. Multimed. 6, 435–438 (2004)CrossRef
18.
Zurück zum Zitat Muhit, A., Pickering, M.R., Frater, M.R., Arnold, J.F.: Video coding using elastic motion model and larger blocks. IEEE Trans. Circ. Syst. Video Technol. 20(5), 661–672 (2010)CrossRef Muhit, A., Pickering, M.R., Frater, M.R., Arnold, J.F.: Video coding using elastic motion model and larger blocks. IEEE Trans. Circ. Syst. Video Technol. 20(5), 661–672 (2010)CrossRef
19.
Zurück zum Zitat Matthias, N., Swoboda, R.: Extending HEVC by an affine motion model. In: Picture Coding Symposium (PCS), pp. 321–324 (2013) Matthias, N., Swoboda, R.: Extending HEVC by an affine motion model. In: Picture Coding Symposium (PCS), pp. 321–324 (2013)
20.
Zurück zum Zitat Huang, H., Woods, J. W., Zhao, Y., Bai, H.: Affine SKIP and DIRECT modes for efficient video coding. Visual Communications and Image Processing (VCIP), pp. 1–6 (2012) Huang, H., Woods, J. W., Zhao, Y., Bai, H.: Affine SKIP and DIRECT modes for efficient video coding. Visual Communications and Image Processing (VCIP), pp. 1–6 (2012)
21.
Zurück zum Zitat Huang, H., Woods, J.W., Zhao, Y., Bai, H.: Control-point representation and differential coding affine-motion compensation. IEEE Trans. Circ. Syst. Video Technol. 23(10), 1651–1660 (2013)CrossRef Huang, H., Woods, J.W., Zhao, Y., Bai, H.: Control-point representation and differential coding affine-motion compensation. IEEE Trans. Circ. Syst. Video Technol. 23(10), 1651–1660 (2013)CrossRef
22.
Zurück zum Zitat Chen, H., Liang, F., Lin, S.: Affine SKIP and MERGE modes for video coding. In: IEEE 17th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–5 (2015) Chen, H., Liang, F., Lin, S.: Affine SKIP and MERGE modes for video coding. In: IEEE 17th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–5 (2015)
23.
Zurück zum Zitat Yoo, J.H.L., Seok, H., Zhang, B.: Evolutionary particle filtering for sequential dependency learning from video data. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2012) Yoo, J.H.L., Seok, H., Zhang, B.: Evolutionary particle filtering for sequential dependency learning from video data. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2012)
24.
Zurück zum Zitat Li, L., Li, H., Lv, H., Yang, H.: An affine motion compensation framework for high efficiency video coding. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 525–528 (2015) Li, L., Li, H., Lv, H., Yang, H.: An affine motion compensation framework for high efficiency video coding. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 525–528 (2015)
25.
Zurück zum Zitat Heithausen, C., Vorwerk, J.H.: Motion compensation with higher order motion models for HEVC. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1438–1442 (2015) Heithausen, C., Vorwerk, J.H.: Motion compensation with higher order motion models for HEVC. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1438–1442 (2015)
26.
Zurück zum Zitat Lee, J.H., Lim, K.W., Song, B.C., Ra, J.B.: A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding. IEEE Trans. Circ. Syst. Video Technol. 11(12), 1289–1301 (2001)CrossRef Lee, J.H., Lim, K.W., Song, B.C., Ra, J.B.: A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding. IEEE Trans. Circ. Syst. Video Technol. 11(12), 1289–1301 (2001)CrossRef
27.
Zurück zum Zitat Gahlot, A., Arya, S., Ghosh, D.: Object-based affine motion estimation. In: Proceedings of IEEE Region 10 Conference, pp. 1343–1347 (2003) Gahlot, A., Arya, S., Ghosh, D.: Object-based affine motion estimation. In: Proceedings of IEEE Region 10 Conference, pp. 1343–1347 (2003)
28.
Zurück zum Zitat Alwani, M., Chaudhary, R., Mathur, M., Dutta Roy, S., Chaudhury, S.: Restricted affine motion compensation in video coding using particle filtering. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 479–484 (2010) Alwani, M., Chaudhary, R., Mathur, M., Dutta Roy, S., Chaudhury, S.: Restricted affine motion compensation in video coding using particle filtering. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 479–484 (2010)
29.
Zurück zum Zitat Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994)CrossRef Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994)CrossRef
30.
Zurück zum Zitat Isard, M., Blake, A.: CONDENSATION: conditional density propagation for visual tracking. Int. J. Comput. Vis. 28(1), 5–28 (1998)CrossRef Isard, M., Blake, A.: CONDENSATION: conditional density propagation for visual tracking. Int. J. Comput. Vis. 28(1), 5–28 (1998)CrossRef
31.
Zurück zum Zitat Sullivan, J., Rittscher, J.: Guiding random particles by deterministic search. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1–18 (2001) Sullivan, J., Rittscher, J.: Guiding random particles by deterministic search. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1–18 (2001)
32.
Zurück zum Zitat Dutta Roy, S., Tran, S.D., Davis, L.S., Vikram, B.S.: Multi-resolution tracking in space and time. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 352–358 (2008) Dutta Roy, S., Tran, S.D., Davis, L.S., Vikram, B.S.: Multi-resolution tracking in space and time. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 352–358 (2008)
33.
Zurück zum Zitat Lai, Y. K., Lai, Y. F. Quality enhancement for scalable view window in touchable display systems. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 539–540 (2011) Lai, Y. K., Lai, Y. F. Quality enhancement for scalable view window in touchable display systems. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 539–540 (2011)
34.
Zurück zum Zitat Richardson, I.E.G.: H.264 and MPEG-4: Video Compression. Wiley, New York (2003) Richardson, I.E.G.: H.264 and MPEG-4: Video Compression. Wiley, New York (2003)
35.
Zurück zum Zitat Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the H.264/AVC video coding standard. IEEE Trans. Circ. Syst. Video Technol. 13(7), 688–703 (2003)CrossRef Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the H.264/AVC video coding standard. IEEE Trans. Circ. Syst. Video Technol. 13(7), 688–703 (2003)CrossRef
36.
Zurück zum Zitat Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)CrossRef Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)CrossRef
Metadaten
Titel
Restricted affine motion compensation and estimation in video coding with particle filtering and importance sampling: a multi-resolution approach
verfasst von
Mithilesh Kumar Jha
Ravi Chaudhary
Sumantra Dutta Roy
Mona Mathur
Brejesh Lall
Publikationsdatum
25.03.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Multimedia Systems / Ausgabe 3/2018
Print ISSN: 0942-4962
Elektronische ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-017-0543-z

Weitere Artikel der Ausgabe 3/2018

Multimedia Systems 3/2018 Zur Ausgabe

Neuer Inhalt