Abstract
In video coding, a well-designed rate control scheme should be concerned with both the objective and subjective quality. However, the existing H.264 rate control algorithms mainly aim at improving the objective quality without considering the human visual system. In this paper, we propose a novel rate control algorithm that takes into account visual attention. In a group of pictures, bits allocated to each frame are related to the local motion attention in it, and more bits are allocated to the frames with strong local motion attention. Similarly, in each frame, more bits are assigned to visually significant macroblocks (MBs), and fewer to visually insignificant MBs. Experiment results show that the proposed algorithm improves the coding quality in frames with strong local motion, and reduces PSNR fluctuation across frames by up to 22.15%. In addition, PSNR in visually important regions is increased by up to 1.45 dB as compared to the standard H.264 rate control scheme that improves the subjective quality. Increased computation complexity of the proposed algorithm is less than 4%, which is negligible.
Similar content being viewed by others
References
Adiono T, Isshiki T, Ito K, Ohtsuka T, Li D, Honsawek C, Kunieda H (2000) Face focus coding under H.263+ video coding standard. IEEE Asia-Pacific Conf Circuits Syst 1(No. 1):461–464
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI-8:679–698
Chen ZZ, Han JW, Ngan KN (2006) Dynamic bit allocation for multiple video object coding. IEEE Trans Multimed 8(6):1117–1124
Deutsch J, Deutsch D (1963) Attention: some theoretical considerations. Psychol Rev 70:80–90
Ding W, Liu B (1996) Rate control of MPEG video coding and recording by rate-quantization modeling. IEEE Trans Circuits Syst Video Technol 6(1):12–20
Eleftheriadis A, Jacquin A (1994) Model assisted coding of video teleconferencing sequences at low bit rates. IEEE Int Symp Circuits Syst pp. 177–180
http://iphome.hhi.de/suehring/tml/ for JM 17.2 software, 2010
ISO-IEC JTC1/SC29/WG11 (1993) Coding representation of pictures and audio information, MPEG Phase 2 Test Model 5, Apr
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 10(11):1254–1259
Jo SJ, Lee YH (1996) A statistical feedforward/feedback buffer control for the transmission of digital video signals compressed by DCT-based intrafield coding. IEEE Trans Image Process 5(3):527–529
Juan M, Chi MC, Hsu CT, Chen JW (2003) ROI video coding based on H.263+ with robust skin-color detection technique. IEEE Trans Consum Electron 49(No. 3):724–730
Kim WJ, Yi JW, Kim SD (1999) A bit allocation method based on picture activity for still image coding. IEEE Trans Image Process 8(7):974–977
Koch C, Ullman S (1985) Shift in selective visual attention: toward the underlying neural circuitry. Human Neurobiol 4(4):219–227
Li ZG, Pan F, Lim KP, Feng GN (2003) Adaptive basic unit layer rate control for JVT. JVT-G012, 7Th meeting, Pattaya, Thailand, 7–14, Mar
Ma YF, Hua XS, Lu L, Zhang HJ (2005) A generic framework of user attention model and its application in video summarization. IEEE Trans Multimed 7(5):907–919
Park N, Choi K, Kim J (1995) Adaptive cosine transform coding in human visual model for data broadcasting channel. Asia Pacific Conference Communications (APCC), pp. 658–662
Ramos MG, Hemami SS (1996) Edge-adaptive JPEG image compression. SPIE Vis Commun Image Process pp. 1082–1093
Rapantzikos K, Tsapatsoulis N, Auvrithis Y, Kollias S (2007) Bottom-up spatiotemporal visual attention model for video analysis. IET Image Process 1(2):237–248
Ribas-Corbera J, Lei SM (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circuits Syst Video Technol 9(1):172–185
Shen L, Liu Z, Zhang Z, Shi X (2009) Frame-level bit allocation based on incremental PID algorithm and frame complexity estimation. J Vis Commun Image Represent 20(1):28–34
Shen L, Liu Z, Zhang Z, Yan H (2009) Motion attention based frame-level bit allocation scheme for H.264. ACM International Conference on Internet Multimedia Computing and Service, pp. 175–180, Nov
Song HJ, Jay Kuo CC (2004) A region-based H.263+ codec and its rate control for low VAR video. IEEE Trans Multimed 6(3):489–500
Su YP, Sun MT, Hsu V (2005) Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans Circuits Syst Video Technol 15(No. 2):232–242
Sun Y, Ahmad I (2004) A robust and adaptive rate control algorithm for object-based video coding. IEEE Trans Circuits Syst Video Technol 14(10):1167–1182
Sun Y, Ahmad I, Li DD, Zhang YQ (2006) Region-based rate control and bit allocation for wireless video transmission. IEEE Trans Multimed 8(1):1–10
Tang CW (2007) Spatiotemporal visual considerations for video coding. IEEE Trans Multimed 9(2):231–238
Tang CW, Chen CH, Tsai CJ (2006) Visual sensitivity guided bit allocation for video coding. IEEE Trans Multimed 8(1):11–18
Tian L, Sun Y, Sun SX (2009) Frame complexity prediction for H.264/AVC rate control. IEEE International conference on Multimedia and Expo, pp. 45–48, July
Yang X, Lin W, Lu Z, Lin X, Rahardja S, Ong EP, Yao SS (2005) Rate control for videophone using local perceptual cues. IEEE Trans Circuits Syst Video Technol 15(4):496–507
Yang L, Zhang L, Ma S, Zhao D (2009) A ROI quality adjustable rate control scheme for low bitrate video coding. Picture Coding Symposium, Chicago, USA, May.06-08
Zhu Z, Wang Y, Bai Y, Jiang G (2010) On optimizing H.264/AVC rate control by improving R-D model and incorporating HVS characteristics. EURASIP J Adv Signal Process. doi:10.1155/2010/830605
Acknowledgment
The authors are grateful to Ms Xiuya Yuan for her assistance in improving the language usage. This work is sponsored by Shanghai Rising-Star Program (11QA1402400) and Innovation Program of Shanghai Municipal Education Commission (No.10YZ09), and is supported by the National Natural Science Foundation of China under grant No. 60832003, 60902085 and 61171084.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shen, L., Liu, Z. & Zhang, Z. A novel H.264 rate control algorithm with consideration of visual attention. Multimed Tools Appl 63, 709–727 (2013). https://doi.org/10.1007/s11042-011-0893-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-011-0893-z