Abstract
High efficiency video coding (HEVC) has achieved high coding efficiency as the video coding standard. For rate control in HEVC, the conventional R-λ scheme is based on mean absolute difference in allocating bits; however, the scheme does not fully utilize the perceptual importance variation to guide rate control, thus the subjective and objective quality of coded videos has room to improve. Therefore, in this paper, we propose a rate control scheme that considers perceptual importance. We first develop a perceptual importance analysis scheme to accurately abstract the spatial and temporal perceptual importance maps of video contents. The results of the analysis are then used to guide the bit allocation. Utilizing this model, a region-level bit allocation procedure is developed to maintain video quality balance. Subsequently, a largest coding unit (LCU)-level bit allocation scheme is designed to obtain the target bit of each LCU. To achieve a more accurate bitrate, an improved R-λ model based on the Broyden-Fletcher-Goldfarb-Shanno model is utilized to update the R-λ parameter. The experimental results showed that our method not only improved subjective and objective video quality with lower bitrate errors compared to the original RC in HEVC, but also outperformed state-of-the-art methods.
Similar content being viewed by others
References
An C, Nguyen TQ (2008) Iterative rate-distortion optimization of H. 264 with constant bit rate constraint. IEEE Trans Image Process 17(9):1605–1615
Bai L, Song L, Xie R, Xie J, Chen M (Dec 2016) Saliency based rate control scheme for high efficiency video coding. In: Proc. IEEE Int. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pp. 1–6
Bazen AM, Gerez SH (2002) Systematic methods for the computation of the directional fields and singular points of fingerprints. IEEE Trans Pattern Anal Mach Intell 24(7):905–919
Bross B, (Oct. 2012) High efficiency video coding (HEVC) text specification draft 9 (SoDIS). In: 11th JCT-VC meeting
Chadha A, Andreopoulos Y, (2021) Deep Perceptual Preprocessing for Video Coding,” In: Proc. IEEE Int. Computer Vision and Pattern Recognition (CVPR), pp. 14852–14861
Chen Z, Pan X (2019) An optimized rate control for low-delay H. 265/HEVC. IEEE Trans Image Process 28(9):4541–4552
Choi H, Nam J, Yoo J, Sim D, Bajic IV, (2012) Rate control based on unified RQ model for HEVC, ITU-T SG16 contribution, JCTVC-H0213, 1–13
Choi H, Nam J, Yoo J, Sim D, Bajic IV, (April 2012) Improvement of the rate control based on pixel-based URQ model for HEVC, ITU-T/ISO/IEC JCT-VC Document In JCT-VC I0094
Dong J, Ling N (2009) A context-adaptive prediction scheme for parameter estimation in H. 264/AVC macroblock layer rate control. IEEE Trans Circuits Syst Video Technol 19(8):1108–1117
Gao Y, Zhu C, Li S, Yang T (2017) Temporally dependent rate-distortion optimization for low-delay hierarchical video coding. IEEE Trans Image Process 4457-4470:4457–4470
Gao W, Kwong S, Jia Y (2017) Joint machine learning and game theory for rate control in high efficiency video coding. IEEE Trans Image Process 26(12):6074–6089
Gao W, Kwong S, Jiang Q, Fong CK, Wong PH, Yuen WY (2018) Data-driven rate control for rate-distortion optimization in HEVC based on simplified effective initial QP learning. IEEE Trans Broadcasting 65(1):94–108
Girod B (1993) What’s wrong with mean-squared error? Digital images and human vision[J]. AB Watson ed, pp 207–220
Gong Y, Wan S, Yang K, Wu HR, Liu Y (2019) Temporal-layer-motivated lambda domain picture level rate control for random-access configuration in H.265/HEVC. IEEE Trans Circuits Syst Video Technol 29(1):156–170
Guo H, Zhu C, Xu M, Li S (2019) Inter-Block Dependency-Based CTU Level Rate Control for HEVC. IEEE Trans Broadcasting 66(1):113–126
svn_HEVCSoftware. HM Reference Software 16.19. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.19/. Accessed 2018
ITU-R, Methodology for the subjective assessment of the quality of television pictures, ITU-R Recommendation BT.500–10. 2000.
Lee JS, Ebrahimi T (2012) Perceptual video compression: A survey. IEEE J Sel Top Signal Process 6(6):684–697
Lee B, Kim M, Nguyen TQ (2013) A frame-level rate control scheme based on texture and nontexture rate models for high efficiency video coding. IEEE Trans Circuits Syst Video Technol 24(3):465–479
Li B, Li H, Li L, Zhang J, (Oct. 2012) Rate control by R-lambda model for HEVC, ITU-T/ISO/IEC JCT-VC document JCTVC-K0103, Shanghai, CN
Li B, Li H, Li L, (2013) Adaptive bit allocation for R-lambda modelrate control in HM, ITU-T/ISO/IEC JCT-VC document JCTVC-M0036, Incheon, Korea
Li B, Li H, Li L, Zhang J (2014) λ domain rate control algorithm for high efficiency video coding. IEEE Trans Image Process 23(9):3841–3854
Li Y, Liao W, Huang J, He D, Chen Z, (July 2014) Saliency based perceptual HEVC. In: Proc. IEEE Int. Multimedia and Expo Workshops (ICMEW), pp. 1–5.
Li S, Xu M, Deng X, Wang Z (2015) Weight-based R-λ rate control for perceptual HEVC coding on conversational videos. Signal Process Image Commun 38:127–140
Li L, Li B, Li H, Chen CW (2016) λ-Domain optimal bit allocation algorithm for high efficiency video coding. IEEE Trans Circuits Syst Video Technol 28(1):130–142
Li S, Xu M, Wang Z, Sun X (2016) Optimal bit allocation for CTU level rate control in HEVC. IEEE Trans Circuits Syst Video Technol 27(11):2409–2424
Liang X, Wang Q, Zhou Y, Luo B, Men, A, (Nov 2013) A novel RQ model based rate control scheme in HEVC. In: Proc. IEEE Int. Visual Communications and Image Processing (VCIP), pp. 1–6
Lim KP, Sullivan G, Wiegand T, (2005) Text description of joint model reference encoding methods and decoding concealment methods. JVT-O079, Busan, Korea
Lin H, He X, Teng QZ, Fu W, Xiong S (2016) Adaptive bit allocation scheme for extremely low-delay intraframe rate control in high efficiency video coding. Journal of Electronic Imaging 25(4):043008
Liu Y, Li ZG, Soh YC (2006) A novel rate control scheme for low delay video communication of H. 264/AVC standard. IEEE Trans Circuits Syst Video Technol 17(1):68–78
Ma YF, Zhang HJ, (Nov 2003) Contrast-based image attention analysis by using fuzzy growing. In: Proc. the eleventh ACM international conference on Multimedia, pp. 374–381
Meddeb M, Cagnazzo M, Pesquet-Popescu B, (2014) Region-of-interest-based rate control scheme for high-efficiency video coding. APSIPA Transactions on Signal and Information Processing, 3
Nami S, Pakdaman F, Hashemi MR, (2020) Juniper: A Jnd-Based Perceptual Video Coding Framework to Jointly Utilize Saliency and JND. In: Proc. IEEE Int. Multimedia & Expo Workshops (ICMEW), July 2020, pp. 1–6.
Oh H, Kim W (2012) Video processing for human perceptual visual quality-oriented video coding. IEEE Trans Image Process 22(4):1526–1535
Ohm JR, Sullivan GJ, Schwarz H, Tan TK, Wiegand T (2012) Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans Circuits Syst Video Technol 22(12):1669–1684
Ou YF, Ma Z, Liu T, Wang Y (2010) Perceptual quality assessment of video considering both frame rate and quantization artifacts. IEEE Trans Circuits Syst Video Technol 21(3):286–298
Park S, Kim M, (Nov 2006) Extracting moving/static objects of interest in video,” In: Proc. Pacific-Rim Conference on Multimedia, pp. 722–729
Ribas-Corbera J, Lei S (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circuits Syst Video Technol 9(1):172–185
Seshadrinathan K, Bovik AC (2009) Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 19(2):335–350
Takeuchi M, Saika S, Sakamoto Y, Nagashima T, Cheng Z, Kanai K, Wei X (2018) Perceptual quality driven adaptive video coding using JND estimation. In: Proc. IEEE Int. Picture Coding Symposium (PCS), pp. 179–183
Wang Z, Zeng H, Chen J, Cai C (June 2014) Key techniques of high efficiency video coding standard and its extension. In: Proc. IEEE Int. Industrial Electronics and Applications, pp. 1169–1173.
Wang H, Song L, Xie R, Luo Z, Wang X (May 2018) Masking Effects Based Rate Control Scheme for High Efficiency Video Coding. In: Proc. IEEE Int. Symposium on Circuits and Systems (ISCAS), pp. 1–5
Wei H et al (2018) A Rate Control Algorithm for HEVC Considering Visual Saliency. asia pacific signal and information processing association annual summit and conference, 36–42.
Wei H, Zhou W, Zhou X, Bai R, Duan Z (2018) Saliency-based coding tree unit-level rate control for high-efficiency video coding. J Electron Imaging 27(4):043009
Wiegand T, Schwarz H, Joch A, Kossentini F, Sullivan GJ (2003) Rate-constrained coder control and comparison of video coding standards. IEEE Trans Circuits Syst Video Techno 13(7):688–703
Wong CW, Au OC, Meng B, Lam HK (2003) Perceptual rate control for low-delay video communications. Proc IEEE Int Multimedia Expo 3:III–361
Yang Z, Xu Q, Bao S, Cao X, Huang Q (2021) Learning with Multiclass AUC: Theory and Algorithms. IEEE Transs Pattern Anal Machine Intelligence PP:1
Ye Y, He X, Teng Q, Qing L, Lin H, Xia D (2018) Adaptive gradient information and BFGS based inter frame rate control for high efficiency video coding. Multimed Tools Appl 77(12):14557–14577
Zeng H, Yang A, Ngan KN, Wang M (2016) Perceptual sensitivity-based rate control method for high efficiency video coding. Multimed Tools Appl 75(17):10383–10396
Zhang W, Martin RR, Liu H (2017) A saliency dispersion measure for improving saliency-based image quality metrics. IEEE Trans Circuits Syst Video Technol 28(6):1462–1466
Zhou M, Wei X, Wang S, Kwong S, Fong CK, Wong P, Gao W (2019) SSIM-based global optimization for CTU-level rate control in HEVC. IEEE Trans Multimedia 21(8):1921–1933
Zhu C, Huang Y, Xie R, Song L (2021) HEVC VMAF-oriented Perceptual Rate Distortion Optimization using CNN. In: Proc. IEEE Int. Picture Coding Symposium (PCS), pp. 1–5.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant No. 62041109, No. 61861038), the Fundamental Research Funds for the Central Universities (Grant No. 31920210073, 31920180115, 31920190039) and Gansu Province Natural Sciences Fund (21JR1RA206).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lin, H., Li, X., Gao, M. et al. Perceptual importance analysis-based rate control method for HEVC. Multimed Tools Appl 81, 12495–12518 (2022). https://doi.org/10.1007/s11042-022-12146-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-12146-1